2024-03-18T21:37:35.427 INFO:root:teuthology version: 0.0.1.dev262+ge691533 2024-03-18T21:37:35.427 DEBUG:teuthology.run:Teuthology command: teuthology --archive /home/teuthworker/archive/pdonnell-2024-03-18_21:05:49-fs-wip-batrick-testing-20240318.181317-distro-default-smithi/7609140 --verbose --name pdonnell-2024-03-18_21:05:49-fs-wip-batrick-testing-20240318.181317-distro-default-smithi --owner scheduled_pdonnell@teuthology --description fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/secure wsync/no} objectstore-ec/bluestore-comp omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/random export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/yes 5-workunit/suites/dbench}} -- /home/teuthworker/archive/pdonnell-2024-03-18_21:05:49-fs-wip-batrick-testing-20240318.181317-distro-default-smithi/7609140/orig.config.yaml 2024-03-18T21:37:35.480 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-18T21:37:35.575 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/pdonnell-2024-03-18_21:05:49-fs-wip-batrick-testing-20240318.181317-distro-default-smithi/7609140 branch: wip-batrick-testing-20240318.181317 description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/secure wsync/no} objectstore-ec/bluestore-comp omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/random export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/yes 5-workunit/suites/dbench}} email: pdonnell@redhat.com first_in_suite: false job_id: '7609140' kernel: client: sha1: distro kdb: true sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: pdonnell-2024-03-18_21:05:49-fs-wip-batrick-testing-20240318.181317-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-batrick-testing-20240318.181317 ceph: cephfs: max_mds: 5 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true mds_export_ephemeral_random_max: 0.1 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug cephsqlite: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - PG_AVAILABILITY - PG_DEGRADED - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding sha1: b6ae24918b03b1b8a19c9263a50655bba80bb810 subvols: create: 2 subvol_options: --namespace-isolated ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - mds.root_rsnaps - mds_server.req_mksnap_latency.avgcount - mds_server.req_rmsnap_latency.avgcount install: ceph: flavor: default sha1: b6ae24918b03b1b8a19c9263a50655bba80bb810 kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=secure - nowsync syntax: v1 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-batrick-testing-20240318.181317 sha1: e48b1f6c0608b2615e1890cc46286c3877d6e910 owner: scheduled_pdonnell@teuthology priority: 75 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 3005 sha1: b6ae24918b03b1b8a19c9263a50655bba80bb810 sleep_before_teardown: 0 subset: 433/512 suite: fs suite_branch: wip-batrick-testing-20240318.181317 suite_path: /home/teuthworker/src/github.com_batrick_ceph_e48b1f6c0608b2615e1890cc46286c3877d6e910/qa suite_relpath: qa suite_repo: https://github.com/batrick/ceph suite_sha1: e48b1f6c0608b2615e1890cc46286c3877d6e910 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi049.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJrClwSujA3DxhjycCDFz+K4daUSvkXJBl95QePSunM+HntcgrIVUb8euaF3T0w9tjReVqIi006AQ3/nOy+gofA= smithi139.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLwe1SvMi33uDIyC/VCOTEcd/IY0tArlGLqy6o9Bzy9NtvqPlBloCCaFq4bfWoTtxn8IppxcUnb0JBNfcQEiq8E= smithi175.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEHAkrTqf+Qms1LnY4oDNHCeEuoGn7ch+KOKfl2VjBe0g9M/00RWBB6IH67I7+E0oDx4dS8J2tNHdbJTkuQ7BYA= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate false - ceph fs subvolumegroup pin cephfs qa random 0.10 - check-counter: {} - exec: mon.a: - ceph mgr module enable snap_schedule - ceph config set mgr mgr/snap_schedule/allow_m_granularity true - ceph config set mgr mgr/snap_schedule/dump_on_update true - ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m - ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - date +%s > START_TIME - full_sequential_finally: - exec: mon.a: - date +%s > END_TIME - START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - workunit: clients: all: - suites/dbench.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: e691533f9cbb33d85b2187bba20d7102f098636d timestamp: 2024-03-18_21:05:49 tube: smithi user: pdonnell verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-03-18T21:37:35.576 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_batrick_ceph_e48b1f6c0608b2615e1890cc46286c3877d6e910/qa; will attempt to use it 2024-03-18T21:37:35.576 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_batrick_ceph_e48b1f6c0608b2615e1890cc46286c3877d6e910/qa/tasks 2024-03-18T21:37:35.577 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-03-18T21:37:35.578 INFO:teuthology.task.internal:Checking packages... 2024-03-18T21:37:35.601 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'b6ae24918b03b1b8a19c9263a50655bba80bb810' 2024-03-18T21:37:35.601 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-03-18T21:37:35.601 INFO:teuthology.packaging:ref: None 2024-03-18T21:37:35.601 INFO:teuthology.packaging:tag: None 2024-03-18T21:37:35.601 INFO:teuthology.packaging:branch: wip-batrick-testing-20240318.181317 2024-03-18T21:37:35.601 INFO:teuthology.packaging:sha1: b6ae24918b03b1b8a19c9263a50655bba80bb810 2024-03-18T21:37:35.601 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-batrick-testing-20240318.181317 2024-03-18T21:37:35.741 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-2206.gb6ae2491 2024-03-18T21:37:35.743 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-03-18T21:37:35.751 INFO:teuthology.task.internal:no buildpackages task found 2024-03-18T21:37:35.752 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-03-18T21:37:35.784 INFO:teuthology.task.internal:Saving configuration 2024-03-18T21:37:35.804 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-03-18T21:37:35.818 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-18T21:37:35.844 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi049.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-03-18_21:05:49-fs-wip-batrick-testing-20240318.181317-distro-default-smithi/7609140', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-18 21:31:05.526149', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJrClwSujA3DxhjycCDFz+K4daUSvkXJBl95QePSunM+HntcgrIVUb8euaF3T0w9tjReVqIi006AQ3/nOy+gofA='} 2024-03-18T21:37:35.865 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi139.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-03-18_21:05:49-fs-wip-batrick-testing-20240318.181317-distro-default-smithi/7609140', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-18 21:31:05.523470', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLwe1SvMi33uDIyC/VCOTEcd/IY0tArlGLqy6o9Bzy9NtvqPlBloCCaFq4bfWoTtxn8IppxcUnb0JBNfcQEiq8E='} 2024-03-18T21:37:35.888 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi175.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-03-18_21:05:49-fs-wip-batrick-testing-20240318.181317-distro-default-smithi/7609140', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-18 21:31:05.524876', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEHAkrTqf+Qms1LnY4oDNHCeEuoGn7ch+KOKfl2VjBe0g9M/00RWBB6IH67I7+E0oDx4dS8J2tNHdbJTkuQ7BYA='} 2024-03-18T21:37:35.888 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-03-18T21:37:35.894 INFO:teuthology.task.internal:roles: ubuntu@smithi049.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-03-18T21:37:35.894 INFO:teuthology.task.internal:roles: ubuntu@smithi139.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-03-18T21:37:35.894 INFO:teuthology.task.internal:roles: ubuntu@smithi175.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-03-18T21:37:35.894 INFO:teuthology.run_tasks:Running task console_log... 2024-03-18T21:37:36.007 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f4d4fb23160>, signals=[15]) 2024-03-18T21:37:36.007 INFO:teuthology.run_tasks:Running task internal.connect... 2024-03-18T21:37:36.014 INFO:teuthology.task.internal:Opening connections... 2024-03-18T21:37:36.014 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi049.front.sepia.ceph.com 2024-03-18T21:37:36.016 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi049.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-18T21:37:36.127 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi139.front.sepia.ceph.com 2024-03-18T21:37:36.128 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi139.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-18T21:37:36.219 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi175.front.sepia.ceph.com 2024-03-18T21:37:36.220 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi175.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-18T21:37:36.312 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-03-18T21:37:36.319 DEBUG:teuthology.orchestra.run.smithi049:> uname -m 2024-03-18T21:37:36.337 INFO:teuthology.orchestra.run.smithi049.stdout:x86_64 2024-03-18T21:37:36.338 DEBUG:teuthology.orchestra.run.smithi049:> cat /etc/os-release 2024-03-18T21:37:36.396 INFO:teuthology.orchestra.run.smithi049.stdout:NAME="CentOS Stream" 2024-03-18T21:37:36.396 INFO:teuthology.orchestra.run.smithi049.stdout:VERSION="9" 2024-03-18T21:37:36.396 INFO:teuthology.orchestra.run.smithi049.stdout:ID="centos" 2024-03-18T21:37:36.396 INFO:teuthology.orchestra.run.smithi049.stdout:ID_LIKE="rhel fedora" 2024-03-18T21:37:36.396 INFO:teuthology.orchestra.run.smithi049.stdout:VERSION_ID="9" 2024-03-18T21:37:36.396 INFO:teuthology.orchestra.run.smithi049.stdout:PLATFORM_ID="platform:el9" 2024-03-18T21:37:36.396 INFO:teuthology.orchestra.run.smithi049.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-18T21:37:36.396 INFO:teuthology.orchestra.run.smithi049.stdout:ANSI_COLOR="0;31" 2024-03-18T21:37:36.396 INFO:teuthology.orchestra.run.smithi049.stdout:LOGO="fedora-logo-icon" 2024-03-18T21:37:36.396 INFO:teuthology.orchestra.run.smithi049.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-18T21:37:36.396 INFO:teuthology.orchestra.run.smithi049.stdout:HOME_URL="https://centos.org/" 2024-03-18T21:37:36.397 INFO:teuthology.orchestra.run.smithi049.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-18T21:37:36.397 INFO:teuthology.orchestra.run.smithi049.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-18T21:37:36.397 INFO:teuthology.orchestra.run.smithi049.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-18T21:37:36.398 INFO:teuthology.lock.ops:Updating smithi049.front.sepia.ceph.com on lock server 2024-03-18T21:37:36.419 DEBUG:teuthology.orchestra.run.smithi139:> uname -m 2024-03-18T21:37:36.436 INFO:teuthology.orchestra.run.smithi139.stdout:x86_64 2024-03-18T21:37:36.437 DEBUG:teuthology.orchestra.run.smithi139:> cat /etc/os-release 2024-03-18T21:37:36.495 INFO:teuthology.orchestra.run.smithi139.stdout:NAME="CentOS Stream" 2024-03-18T21:37:36.495 INFO:teuthology.orchestra.run.smithi139.stdout:VERSION="9" 2024-03-18T21:37:36.495 INFO:teuthology.orchestra.run.smithi139.stdout:ID="centos" 2024-03-18T21:37:36.495 INFO:teuthology.orchestra.run.smithi139.stdout:ID_LIKE="rhel fedora" 2024-03-18T21:37:36.495 INFO:teuthology.orchestra.run.smithi139.stdout:VERSION_ID="9" 2024-03-18T21:37:36.495 INFO:teuthology.orchestra.run.smithi139.stdout:PLATFORM_ID="platform:el9" 2024-03-18T21:37:36.495 INFO:teuthology.orchestra.run.smithi139.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-18T21:37:36.495 INFO:teuthology.orchestra.run.smithi139.stdout:ANSI_COLOR="0;31" 2024-03-18T21:37:36.495 INFO:teuthology.orchestra.run.smithi139.stdout:LOGO="fedora-logo-icon" 2024-03-18T21:37:36.495 INFO:teuthology.orchestra.run.smithi139.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-18T21:37:36.495 INFO:teuthology.orchestra.run.smithi139.stdout:HOME_URL="https://centos.org/" 2024-03-18T21:37:36.495 INFO:teuthology.orchestra.run.smithi139.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-18T21:37:36.496 INFO:teuthology.orchestra.run.smithi139.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-18T21:37:36.496 INFO:teuthology.orchestra.run.smithi139.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-18T21:37:36.496 INFO:teuthology.lock.ops:Updating smithi139.front.sepia.ceph.com on lock server 2024-03-18T21:37:36.519 DEBUG:teuthology.orchestra.run.smithi175:> uname -m 2024-03-18T21:37:36.536 INFO:teuthology.orchestra.run.smithi175.stdout:x86_64 2024-03-18T21:37:36.537 DEBUG:teuthology.orchestra.run.smithi175:> cat /etc/os-release 2024-03-18T21:37:36.593 INFO:teuthology.orchestra.run.smithi175.stdout:NAME="CentOS Stream" 2024-03-18T21:37:36.593 INFO:teuthology.orchestra.run.smithi175.stdout:VERSION="9" 2024-03-18T21:37:36.593 INFO:teuthology.orchestra.run.smithi175.stdout:ID="centos" 2024-03-18T21:37:36.593 INFO:teuthology.orchestra.run.smithi175.stdout:ID_LIKE="rhel fedora" 2024-03-18T21:37:36.593 INFO:teuthology.orchestra.run.smithi175.stdout:VERSION_ID="9" 2024-03-18T21:37:36.593 INFO:teuthology.orchestra.run.smithi175.stdout:PLATFORM_ID="platform:el9" 2024-03-18T21:37:36.593 INFO:teuthology.orchestra.run.smithi175.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-18T21:37:36.594 INFO:teuthology.orchestra.run.smithi175.stdout:ANSI_COLOR="0;31" 2024-03-18T21:37:36.594 INFO:teuthology.orchestra.run.smithi175.stdout:LOGO="fedora-logo-icon" 2024-03-18T21:37:36.594 INFO:teuthology.orchestra.run.smithi175.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-18T21:37:36.594 INFO:teuthology.orchestra.run.smithi175.stdout:HOME_URL="https://centos.org/" 2024-03-18T21:37:36.594 INFO:teuthology.orchestra.run.smithi175.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-18T21:37:36.594 INFO:teuthology.orchestra.run.smithi175.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-18T21:37:36.594 INFO:teuthology.orchestra.run.smithi175.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-18T21:37:36.594 INFO:teuthology.lock.ops:Updating smithi175.front.sepia.ceph.com on lock server 2024-03-18T21:37:36.618 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-03-18T21:37:36.630 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-03-18T21:37:36.672 INFO:teuthology.task.internal:Checking for old test directory... 2024-03-18T21:37:36.672 DEBUG:teuthology.orchestra.run.smithi049:> test '!' -e /home/ubuntu/cephtest 2024-03-18T21:37:36.675 DEBUG:teuthology.orchestra.run.smithi139:> test '!' -e /home/ubuntu/cephtest 2024-03-18T21:37:36.677 DEBUG:teuthology.orchestra.run.smithi175:> test '!' -e /home/ubuntu/cephtest 2024-03-18T21:37:36.694 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-03-18T21:37:36.705 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-03-18T21:37:36.705 DEBUG:teuthology.orchestra.run.smithi049:> test -z $(ls -A /var/lib/ceph) 2024-03-18T21:37:36.733 DEBUG:teuthology.orchestra.run.smithi139:> test -z $(ls -A /var/lib/ceph) 2024-03-18T21:37:36.735 DEBUG:teuthology.orchestra.run.smithi175:> test -z $(ls -A /var/lib/ceph) 2024-03-18T21:37:36.852 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-03-18T21:37:36.935 INFO:teuthology.run_tasks:Running task kernel... 2024-03-18T21:37:36.955 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'kdb': True, 'sha1': 'distro'} 2024-03-18T21:37:36.955 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}} 2024-03-18T21:37:36.955 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}}, timeout 300 2024-03-18T21:37:36.955 DEBUG:teuthology.orchestra.run.smithi049:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-18T21:37:36.972 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-18T21:37:36.973 DEBUG:teuthology.orchestra.run.smithi049:> uname -r 2024-03-18T21:37:37.031 INFO:teuthology.orchestra.run.smithi049.stdout:5.14.0-361.el9.x86_64 2024-03-18T21:37:37.031 INFO:teuthology.task.kernel:Running kernel on smithi049: 5.14.0-361.el9.x86_64 2024-03-18T21:37:37.031 DEBUG:teuthology.orchestra.run.smithi049:> sudo yum install -y kernel 2024-03-18T21:37:39.143 INFO:teuthology.orchestra.run.smithi049.stdout:CentOS-9 - AppStream 14 MB/s | 6.7 MB 00:00 2024-03-18T21:37:40.577 INFO:teuthology.orchestra.run.smithi049.stdout:CentOS-9 - BaseOS 7.2 MB/s | 2.0 MB 00:00 2024-03-18T21:48:36.707 INFO:teuthology.orchestra.run.smithi049.stdout:CentOS Stream 9 - BaseOS 13 kB/s | 8.0 MB 10:55 2024-03-18T21:48:50.681 INFO:teuthology.orchestra.run.smithi049.stdout:CentOS Stream 9 - AppStream 1.5 MB/s | 19 MB 00:12 2024-03-18T21:48:55.707 INFO:teuthology.orchestra.run.smithi049.stdout:CentOS Stream 9 - CRB 4.5 MB/s | 6.1 MB 00:01 2024-03-18T21:48:57.099 INFO:teuthology.orchestra.run.smithi049.stdout:CentOS Stream 9 - Extras packages 75 kB/s | 16 kB 00:00 2024-03-18T21:53:17.986 INFO:teuthology.orchestra.run.smithi049.stdout:Extra Packages for Enterprise Linux 81 kB/s | 21 MB 04:20 2024-03-18T21:53:23.036 INFO:teuthology.orchestra.run.smithi049.stdout:lab-extras 36 kB/s | 1.7 kB 00:00 2024-03-18T21:53:25.193 INFO:teuthology.orchestra.run.smithi049.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-03-18T21:53:25.193 INFO:teuthology.orchestra.run.smithi049.stdout:Package kernel-5.14.0-361.el9.x86_64 is already installed. 2024-03-18T21:53:25.277 INFO:teuthology.orchestra.run.smithi049.stdout:Dependencies resolved. 2024-03-18T21:53:25.287 INFO:teuthology.orchestra.run.smithi049.stdout:================================================================================ 2024-03-18T21:53:25.287 INFO:teuthology.orchestra.run.smithi049.stdout: Package Architecture Version Repository Size 2024-03-18T21:53:25.287 INFO:teuthology.orchestra.run.smithi049.stdout:================================================================================ 2024-03-18T21:53:25.287 INFO:teuthology.orchestra.run.smithi049.stdout:Installing: 2024-03-18T21:53:25.287 INFO:teuthology.orchestra.run.smithi049.stdout: kernel x86_64 5.14.0-430.el9 baseos 44 k 2024-03-18T21:53:25.287 INFO:teuthology.orchestra.run.smithi049.stdout:Installing dependencies: 2024-03-18T21:53:25.288 INFO:teuthology.orchestra.run.smithi049.stdout: kernel-core x86_64 5.14.0-430.el9 baseos 15 M 2024-03-18T21:53:25.288 INFO:teuthology.orchestra.run.smithi049.stdout: kernel-modules x86_64 5.14.0-430.el9 baseos 33 M 2024-03-18T21:53:25.288 INFO:teuthology.orchestra.run.smithi049.stdout: kernel-modules-core x86_64 5.14.0-430.el9 baseos 27 M 2024-03-18T21:53:25.288 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-18T21:53:25.288 INFO:teuthology.orchestra.run.smithi049.stdout:Transaction Summary 2024-03-18T21:53:25.288 INFO:teuthology.orchestra.run.smithi049.stdout:================================================================================ 2024-03-18T21:53:25.288 INFO:teuthology.orchestra.run.smithi049.stdout:Install 4 Packages 2024-03-18T21:53:25.288 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-18T21:53:25.288 INFO:teuthology.orchestra.run.smithi049.stdout:Total download size: 76 M 2024-03-18T21:53:25.288 INFO:teuthology.orchestra.run.smithi049.stdout:Installed size: 123 M 2024-03-18T21:53:25.288 INFO:teuthology.orchestra.run.smithi049.stdout:Downloading Packages: 2024-03-18T21:53:25.696 INFO:teuthology.orchestra.run.smithi049.stdout:(1/4): kernel-5.14.0-430.el9.x86_64.rpm 129 kB/s | 44 kB 00:00 2024-03-18T21:53:40.719 INFO:teuthology.orchestra.run.smithi049.stdout:(2/4): kernel-core-5.14.0-430.el9.x86_64.rpm 993 kB/s | 15 MB 00:15 2024-03-18T21:53:56.259 INFO:teuthology.orchestra.run.smithi049.stdout:(3/4): kernel-modules-core-5.14.0-430.el9.x86_6 920 kB/s | 27 MB 00:30 2024-03-18T21:54:03.283 INFO:teuthology.orchestra.run.smithi049.stdout:(4/4): kernel-modules-5.14.0-430.el9.x86_64.rpm 903 kB/s | 33 MB 00:37 2024-03-18T21:54:03.283 INFO:teuthology.orchestra.run.smithi049.stdout:-------------------------------------------------------------------------------- 2024-03-18T21:54:03.283 INFO:teuthology.orchestra.run.smithi049.stdout:Total 2.0 MB/s | 76 MB 00:37 2024-03-18T21:54:03.975 INFO:teuthology.orchestra.run.smithi049.stdout:Running transaction check 2024-03-18T21:54:04.407 INFO:teuthology.orchestra.run.smithi049.stdout:Transaction check succeeded. 2024-03-18T21:54:04.407 INFO:teuthology.orchestra.run.smithi049.stdout:Running transaction test 2024-03-18T21:54:05.460 INFO:teuthology.orchestra.run.smithi049.stdout:Transaction test succeeded. 2024-03-18T21:54:05.460 INFO:teuthology.orchestra.run.smithi049.stdout:Running transaction 2024-03-18T21:54:08.068 INFO:teuthology.orchestra.run.smithi049.stdout: Preparing : 1/1 2024-03-18T21:54:08.705 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : kernel-modules-core-5.14.0-430.el9.x86_64 1/4 2024-03-18T21:54:08.792 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : kernel-core-5.14.0-430.el9.x86_64 2/4 2024-03-18T21:54:11.666 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: kernel-core-5.14.0-430.el9.x86_64 2/4 2024-03-18T21:54:11.803 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : kernel-modules-5.14.0-430.el9.x86_64 3/4 2024-03-18T21:54:16.875 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: kernel-modules-5.14.0-430.el9.x86_64 3/4 2024-03-18T21:54:16.920 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : kernel-5.14.0-430.el9.x86_64 4/4 2024-03-18T21:54:22.025 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: kernel-modules-core-5.14.0-430.el9.x86_64 4/4 2024-03-18T21:54:52.744 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: kernel-core-5.14.0-430.el9.x86_64 4/4 2024-03-18T21:54:52.744 INFO:teuthology.orchestra.run.smithi049.stdout:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-03-18T21:54:52.744 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-18T21:54:52.789 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: kernel-modules-5.14.0-430.el9.x86_64 4/4 2024-03-18T21:54:53.164 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: kernel-5.14.0-430.el9.x86_64 4/4 2024-03-18T21:54:53.164 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : kernel-5.14.0-430.el9.x86_64 1/4 2024-03-18T21:54:53.165 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : kernel-core-5.14.0-430.el9.x86_64 2/4 2024-03-18T21:54:53.165 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : kernel-modules-5.14.0-430.el9.x86_64 3/4 2024-03-18T21:54:53.591 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : kernel-modules-core-5.14.0-430.el9.x86_64 4/4 2024-03-18T21:54:53.591 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-18T21:54:53.591 INFO:teuthology.orchestra.run.smithi049.stdout:Installed: 2024-03-18T21:54:53.591 INFO:teuthology.orchestra.run.smithi049.stdout: kernel-5.14.0-430.el9.x86_64 2024-03-18T21:54:53.591 INFO:teuthology.orchestra.run.smithi049.stdout: kernel-core-5.14.0-430.el9.x86_64 2024-03-18T21:54:53.591 INFO:teuthology.orchestra.run.smithi049.stdout: kernel-modules-5.14.0-430.el9.x86_64 2024-03-18T21:54:53.591 INFO:teuthology.orchestra.run.smithi049.stdout: kernel-modules-core-5.14.0-430.el9.x86_64 2024-03-18T21:54:53.591 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-18T21:54:53.592 INFO:teuthology.orchestra.run.smithi049.stdout:Complete! 2024-03-18T21:54:53.943 DEBUG:teuthology.orchestra.run.smithi049:> rpm -q kernel | sort -rV | head -n 1 2024-03-18T21:54:54.040 INFO:teuthology.orchestra.run.smithi049.stdout:kernel-5.14.0-430.el9.x86_64 2024-03-18T21:54:54.041 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-430.el9.x86_64 2024-03-18T21:54:54.041 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-361.el9.x86_64 Expected: 5.14.0-430.el9.x86_64 2024-03-18T21:54:54.041 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-03-18T21:54:54.041 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-03-18T21:54:54.041 INFO:teuthology.task.kernel:Installing distro kernel on client.0... 2024-03-18T21:54:54.041 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi049.front.sepia.ceph.com, path=None, version=distro) 2024-03-18T21:54:54.042 DEBUG:teuthology.orchestra.run.smithi049:> rpm -q kernel | sort -rV | head -n 1 2024-03-18T21:54:54.125 INFO:teuthology.orchestra.run.smithi049.stdout:kernel-5.14.0-430.el9.x86_64 2024-03-18T21:54:54.126 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-430.el9.x86_64 2024-03-18T21:54:54.126 DEBUG:teuthology.orchestra.run.smithi049:> sudo rpm -qi grub2-tools 2024-03-18T21:54:54.163 INFO:teuthology.orchestra.run.smithi049.stdout:Name : grub2-tools 2024-03-18T21:54:54.163 INFO:teuthology.orchestra.run.smithi049.stdout:Epoch : 1 2024-03-18T21:54:54.163 INFO:teuthology.orchestra.run.smithi049.stdout:Version : 2.06 2024-03-18T21:54:54.163 INFO:teuthology.orchestra.run.smithi049.stdout:Release : 68.el9 2024-03-18T21:54:54.164 INFO:teuthology.orchestra.run.smithi049.stdout:Architecture: x86_64 2024-03-18T21:54:54.164 INFO:teuthology.orchestra.run.smithi049.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-03-18T21:54:54.164 INFO:teuthology.orchestra.run.smithi049.stdout:Group : Unspecified 2024-03-18T21:54:54.164 INFO:teuthology.orchestra.run.smithi049.stdout:Size : 8264254 2024-03-18T21:54:54.164 INFO:teuthology.orchestra.run.smithi049.stdout:License : GPLv3+ 2024-03-18T21:54:54.164 INFO:teuthology.orchestra.run.smithi049.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-03-18T21:54:54.164 INFO:teuthology.orchestra.run.smithi049.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-03-18T21:54:54.164 INFO:teuthology.orchestra.run.smithi049.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-03-18T21:54:54.164 INFO:teuthology.orchestra.run.smithi049.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-03-18T21:54:54.164 INFO:teuthology.orchestra.run.smithi049.stdout:Packager : builder@centos.org 2024-03-18T21:54:54.164 INFO:teuthology.orchestra.run.smithi049.stdout:Vendor : CentOS 2024-03-18T21:54:54.164 INFO:teuthology.orchestra.run.smithi049.stdout:URL : http://www.gnu.org/software/grub/ 2024-03-18T21:54:54.164 INFO:teuthology.orchestra.run.smithi049.stdout:Summary : Support tools for GRUB. 2024-03-18T21:54:54.165 INFO:teuthology.orchestra.run.smithi049.stdout:Description : 2024-03-18T21:54:54.165 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-18T21:54:54.165 INFO:teuthology.orchestra.run.smithi049.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-03-18T21:54:54.165 INFO:teuthology.orchestra.run.smithi049.stdout:customizable bootloader with modular architecture. It supports a rich 2024-03-18T21:54:54.165 INFO:teuthology.orchestra.run.smithi049.stdout:variety of kernel formats, file systems, computer architectures and 2024-03-18T21:54:54.165 INFO:teuthology.orchestra.run.smithi049.stdout:hardware devices. 2024-03-18T21:54:54.165 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-18T21:54:54.165 INFO:teuthology.orchestra.run.smithi049.stdout:This subpackage provides tools for support of all platforms. 2024-03-18T21:54:54.166 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-03-18T21:54:54.166 INFO:teuthology.task.kernel:Updating grub on smithi049 to boot 5.14.0-430.el9.x86_64 2024-03-18T21:54:54.166 DEBUG:teuthology.orchestra.run.smithi049:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-03-18T21:54:54.719 INFO:teuthology.orchestra.run.smithi049.stderr:Generating grub configuration file ... 2024-03-18T21:54:56.115 INFO:teuthology.orchestra.run.smithi049.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-03-18T21:54:56.142 INFO:teuthology.orchestra.run.smithi049.stderr:done 2024-03-18T21:54:56.144 DEBUG:teuthology.orchestra.run.smithi049:> mktemp 2024-03-18T21:54:56.159 INFO:teuthology.orchestra.run.smithi049.stdout:/tmp/tmp.evyGszT75N 2024-03-18T21:54:56.160 DEBUG:teuthology.orchestra.run.smithi049:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.evyGszT75N 2024-03-18T21:54:56.232 DEBUG:teuthology.orchestra.run.smithi049:> sudo chmod 0666 /tmp/tmp.evyGszT75N 2024-03-18T21:54:56.380 DEBUG:teuthology.orchestra.remote:smithi049:/tmp/tmp.evyGszT75N is 6KB 2024-03-18T21:54:56.428 DEBUG:teuthology.orchestra.run.smithi049:> rm -fr /tmp/tmp.evyGszT75N 2024-03-18T21:54:56.443 DEBUG:teuthology.orchestra.run.smithi049:> sudo /bin/ls /boot/loader/entries || true 2024-03-18T21:54:56.511 INFO:teuthology.orchestra.run.smithi049.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-03-18T21:54:56.512 INFO:teuthology.orchestra.run.smithi049.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-03-18T21:54:56.512 INFO:teuthology.orchestra.run.smithi049.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-03-18T21:54:56.512 INFO:teuthology.orchestra.run.smithi049.stdout:15949e3c88704300882482eddfd7baae-5.14.0-430.el9.x86_64.conf 2024-03-18T21:54:56.513 DEBUG:teuthology.orchestra.run.smithi049:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-5.14.0-430.el9.x86_64 2024-03-18T21:54:56.678 DEBUG:teuthology.orchestra.run.smithi049:> sudo shutdown -r now 2024-03-18T21:55:26.681 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-03-18T21:55:26.682 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi049.front.sepia.ceph.com' 2024-03-18T21:55:26.683 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi049.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-18T21:56:01.183 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.49 2024-03-18T21:56:10.184 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi049.front.sepia.ceph.com' 2024-03-18T21:56:10.185 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi049.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-18T21:56:13.246 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.49 2024-03-18T21:56:25.247 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi049.front.sepia.ceph.com' 2024-03-18T21:56:25.248 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi049.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-18T21:56:28.318 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.49 2024-03-18T21:56:43.319 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi049.front.sepia.ceph.com' 2024-03-18T21:56:43.320 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi049.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-18T21:57:43.323 DEBUG:teuthology.orchestra.remote:timed out 2024-03-18T21:58:01.324 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi049.front.sepia.ceph.com' 2024-03-18T21:58:01.325 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi049.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-18T21:58:01.626 DEBUG:teuthology.orchestra.run.smithi049:> true 2024-03-18T21:58:02.029 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi049.front.sepia.ceph.com' 2024-03-18T21:58:02.030 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "5.14.0-430.el9.x86_64"... 2024-03-18T21:58:02.030 DEBUG:teuthology.orchestra.run.smithi049:> uname -r 2024-03-18T21:58:02.046 INFO:teuthology.orchestra.run.smithi049.stdout:5.14.0-430.el9.x86_64 2024-03-18T21:58:02.046 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-430.el9.x86_64 vs 5.14.0-430.el9.x86_64 2024-03-18T21:58:02.046 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-03-18T21:58:02.046 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-03-18T21:58:03.047 DEBUG:teuthology.parallel:result is None 2024-03-18T21:58:03.048 INFO:teuthology.run_tasks:Running task internal.base... 2024-03-18T21:58:03.056 INFO:teuthology.task.internal:Creating test directory... 2024-03-18T21:58:03.056 DEBUG:teuthology.orchestra.run.smithi049:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-18T21:58:03.059 DEBUG:teuthology.orchestra.run.smithi139:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-18T21:58:03.062 DEBUG:teuthology.orchestra.run.smithi175:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-18T21:58:03.081 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-03-18T21:58:03.087 INFO:teuthology.run_tasks:Running task internal.archive... 2024-03-18T21:58:03.136 INFO:teuthology.task.internal:Creating archive directory... 2024-03-18T21:58:03.136 DEBUG:teuthology.orchestra.run.smithi049:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-18T21:58:03.142 DEBUG:teuthology.orchestra.run.smithi139:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-18T21:58:03.145 DEBUG:teuthology.orchestra.run.smithi175:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-18T21:58:03.231 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-03-18T21:58:03.238 INFO:teuthology.task.internal:Enabling coredump saving... 2024-03-18T21:58:03.238 DEBUG:teuthology.orchestra.run.smithi139:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-18T21:58:03.254 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-18T21:58:03.254 DEBUG:teuthology.orchestra.run.smithi175:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-18T21:58:03.270 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-18T21:58:03.271 DEBUG:teuthology.orchestra.run.smithi049:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-18T21:58:03.273 DEBUG:teuthology.orchestra.run.smithi139:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-18T21:58:03.297 DEBUG:teuthology.orchestra.run.smithi175:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-18T21:58:03.346 INFO:teuthology.orchestra.run.smithi139.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-18T21:58:03.349 INFO:teuthology.orchestra.run.smithi175.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-18T21:58:03.366 INFO:teuthology.orchestra.run.smithi139.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-18T21:58:03.369 INFO:teuthology.orchestra.run.smithi049.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-18T21:58:03.374 INFO:teuthology.orchestra.run.smithi175.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-18T21:58:03.407 INFO:teuthology.orchestra.run.smithi049.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-18T21:58:03.408 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-03-18T21:58:03.415 INFO:teuthology.task.internal:Configuring sudo... 2024-03-18T21:58:03.416 DEBUG:teuthology.orchestra.run.smithi049:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-18T21:58:03.418 DEBUG:teuthology.orchestra.run.smithi139:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-18T21:58:03.421 DEBUG:teuthology.orchestra.run.smithi175:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-18T21:58:03.451 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-03-18T21:58:03.459 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-03-18T21:58:03.459 DEBUG:teuthology.orchestra.run.smithi049:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-18T21:58:03.494 DEBUG:teuthology.orchestra.run.smithi139:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-18T21:58:03.497 DEBUG:teuthology.orchestra.run.smithi175:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-18T21:58:03.514 DEBUG:teuthology.orchestra.run.smithi049:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-18T21:58:03.594 DEBUG:teuthology.orchestra.run.smithi049:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-18T21:58:03.648 DEBUG:teuthology.orchestra.run.smithi049:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-18T21:58:03.732 DEBUG:teuthology.orchestra.run.smithi049:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-18T21:58:03.779 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-03-18T21:58:03.779 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-18T21:58:03.851 DEBUG:teuthology.orchestra.run.smithi139:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-18T21:58:03.895 DEBUG:teuthology.orchestra.run.smithi139:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-18T21:58:03.988 DEBUG:teuthology.orchestra.run.smithi139:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-18T21:58:04.033 DEBUG:teuthology.orchestra.run.smithi139:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-18T21:58:04.077 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-03-18T21:58:04.077 DEBUG:teuthology.orchestra.run.smithi139:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-18T21:58:04.146 DEBUG:teuthology.orchestra.run.smithi175:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-18T21:58:04.191 DEBUG:teuthology.orchestra.run.smithi175:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-18T21:58:04.267 DEBUG:teuthology.orchestra.run.smithi175:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-18T21:58:04.309 DEBUG:teuthology.orchestra.run.smithi175:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-18T21:58:04.355 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-03-18T21:58:04.355 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-18T21:58:04.424 DEBUG:teuthology.orchestra.run.smithi049:> sudo service rsyslog restart 2024-03-18T21:58:04.427 DEBUG:teuthology.orchestra.run.smithi139:> sudo service rsyslog restart 2024-03-18T21:58:04.430 DEBUG:teuthology.orchestra.run.smithi175:> sudo service rsyslog restart 2024-03-18T21:58:04.472 INFO:teuthology.orchestra.run.smithi139.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-18T21:58:04.476 INFO:teuthology.orchestra.run.smithi049.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-18T21:58:04.511 INFO:teuthology.orchestra.run.smithi175.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-18T21:58:04.993 INFO:teuthology.run_tasks:Running task internal.timer... 2024-03-18T21:58:05.001 INFO:teuthology.task.internal:Starting timer... 2024-03-18T21:58:05.001 INFO:teuthology.run_tasks:Running task pcp... 2024-03-18T21:58:05.048 INFO:teuthology.run_tasks:Running task selinux... 2024-03-18T21:58:05.096 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-03-18T21:58:05.096 DEBUG:teuthology.orchestra.run.smithi049:> sudo service auditd rotate 2024-03-18T21:58:05.174 INFO:teuthology.orchestra.run.smithi049.stdout:Rotating logs: 2024-03-18T21:58:05.176 DEBUG:teuthology.orchestra.run.smithi139:> sudo service auditd rotate 2024-03-18T21:58:05.253 INFO:teuthology.orchestra.run.smithi139.stdout:Rotating logs: 2024-03-18T21:58:05.255 DEBUG:teuthology.orchestra.run.smithi175:> sudo service auditd rotate 2024-03-18T21:58:05.341 INFO:teuthology.orchestra.run.smithi175.stdout:Rotating logs: 2024-03-18T21:58:05.343 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-03-18T21:58:05.343 DEBUG:teuthology.orchestra.run.smithi049:> /usr/sbin/getenforce 2024-03-18T21:58:05.366 INFO:teuthology.orchestra.run.smithi049.stdout:Permissive 2024-03-18T21:58:05.366 DEBUG:teuthology.orchestra.run.smithi139:> /usr/sbin/getenforce 2024-03-18T21:58:05.390 INFO:teuthology.orchestra.run.smithi139.stdout:Permissive 2024-03-18T21:58:05.390 DEBUG:teuthology.orchestra.run.smithi175:> /usr/sbin/getenforce 2024-03-18T21:58:05.418 INFO:teuthology.orchestra.run.smithi175.stdout:Permissive 2024-03-18T21:58:05.419 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi049.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi139.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi175.front.sepia.ceph.com': 'permissive'} 2024-03-18T21:58:05.419 DEBUG:teuthology.orchestra.run.smithi049:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-18T21:58:05.453 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-18T21:58:05.454 DEBUG:teuthology.orchestra.run.smithi139:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-18T21:58:05.482 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-18T21:58:05.483 DEBUG:teuthology.orchestra.run.smithi175:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-18T21:58:05.511 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-18T21:58:05.512 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-03-18T21:58:05.512 DEBUG:teuthology.orchestra.run.smithi049:> sudo /usr/sbin/setenforce permissive 2024-03-18T21:58:05.552 DEBUG:teuthology.orchestra.run.smithi139:> sudo /usr/sbin/setenforce permissive 2024-03-18T21:58:05.584 DEBUG:teuthology.orchestra.run.smithi175:> sudo /usr/sbin/setenforce permissive 2024-03-18T21:58:05.637 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-03-18T21:58:05.647 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-03-18T21:58:05.664 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-03-18T21:58:05.665 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi049.front.sepia.ceph.com,smithi139.front.sepia.ceph.com,smithi175.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-03-18T22:24:23.435 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi049.front.sepia.ceph.com'), Remote(name='ubuntu@smithi139.front.sepia.ceph.com'), Remote(name='ubuntu@smithi175.front.sepia.ceph.com')] 2024-03-18T22:24:23.436 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi049.front.sepia.ceph.com' 2024-03-18T22:24:23.437 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi049.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-18T22:24:23.541 DEBUG:teuthology.orchestra.run.smithi049:> true 2024-03-18T22:24:23.625 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi049.front.sepia.ceph.com' 2024-03-18T22:24:23.626 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi139.front.sepia.ceph.com' 2024-03-18T22:24:23.626 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi139.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-18T22:24:23.715 DEBUG:teuthology.orchestra.run.smithi139:> true 2024-03-18T22:24:23.780 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi139.front.sepia.ceph.com' 2024-03-18T22:24:23.781 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi175.front.sepia.ceph.com' 2024-03-18T22:24:23.782 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi175.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-18T22:24:23.872 DEBUG:teuthology.orchestra.run.smithi175:> true 2024-03-18T22:24:23.946 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi175.front.sepia.ceph.com' 2024-03-18T22:24:23.946 INFO:teuthology.run_tasks:Running task clock... 2024-03-18T22:24:23.957 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-03-18T22:24:23.957 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-18T22:24:23.957 DEBUG:teuthology.orchestra.run.smithi049:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-18T22:24:23.960 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-18T22:24:23.960 DEBUG:teuthology.orchestra.run.smithi139:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-18T22:24:23.962 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-18T22:24:23.962 DEBUG:teuthology.orchestra.run.smithi175:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-18T22:24:23.991 INFO:teuthology.orchestra.run.smithi139.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-03-18T22:24:23.992 INFO:teuthology.orchestra.run.smithi049.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-03-18T22:24:24.008 INFO:teuthology.orchestra.run.smithi139.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-03-18T22:24:24.013 INFO:teuthology.orchestra.run.smithi049.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-03-18T22:24:24.019 INFO:teuthology.orchestra.run.smithi175.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-03-18T22:24:24.036 INFO:teuthology.orchestra.run.smithi175.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-03-18T22:24:24.037 INFO:teuthology.orchestra.run.smithi139.stderr:sudo: ntpd: command not found 2024-03-18T22:24:24.049 INFO:teuthology.orchestra.run.smithi049.stderr:sudo: ntpd: command not found 2024-03-18T22:24:24.050 INFO:teuthology.orchestra.run.smithi139.stdout:506 Cannot talk to daemon 2024-03-18T22:24:24.066 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: ntpd: command not found 2024-03-18T22:24:24.066 INFO:teuthology.orchestra.run.smithi049.stdout:506 Cannot talk to daemon 2024-03-18T22:24:24.067 INFO:teuthology.orchestra.run.smithi139.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-03-18T22:24:24.080 INFO:teuthology.orchestra.run.smithi175.stdout:506 Cannot talk to daemon 2024-03-18T22:24:24.083 INFO:teuthology.orchestra.run.smithi139.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-03-18T22:24:24.086 INFO:teuthology.orchestra.run.smithi049.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-03-18T22:24:24.096 INFO:teuthology.orchestra.run.smithi175.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-03-18T22:24:24.106 INFO:teuthology.orchestra.run.smithi049.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-03-18T22:24:24.113 INFO:teuthology.orchestra.run.smithi175.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-03-18T22:24:24.126 INFO:teuthology.orchestra.run.smithi139.stderr:bash: line 1: ntpq: command not found 2024-03-18T22:24:24.129 INFO:teuthology.orchestra.run.smithi139.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-18T22:24:24.129 INFO:teuthology.orchestra.run.smithi139.stdout:=============================================================================== 2024-03-18T22:24:24.130 INFO:teuthology.orchestra.run.smithi139.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-18T22:24:24.130 INFO:teuthology.orchestra.run.smithi139.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-18T22:24:24.130 INFO:teuthology.orchestra.run.smithi139.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-18T22:24:24.130 INFO:teuthology.orchestra.run.smithi139.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-18T22:24:24.156 INFO:teuthology.orchestra.run.smithi175.stderr:bash: line 1: ntpq: command not found 2024-03-18T22:24:24.159 INFO:teuthology.orchestra.run.smithi175.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-18T22:24:24.160 INFO:teuthology.orchestra.run.smithi175.stdout:=============================================================================== 2024-03-18T22:24:24.160 INFO:teuthology.orchestra.run.smithi175.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-18T22:24:24.160 INFO:teuthology.orchestra.run.smithi175.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-18T22:24:24.160 INFO:teuthology.orchestra.run.smithi175.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-18T22:24:24.160 INFO:teuthology.orchestra.run.smithi175.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-18T22:24:24.160 INFO:teuthology.orchestra.run.smithi049.stderr:bash: line 1: ntpq: command not found 2024-03-18T22:24:24.163 INFO:teuthology.orchestra.run.smithi049.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-18T22:24:24.164 INFO:teuthology.orchestra.run.smithi049.stdout:=============================================================================== 2024-03-18T22:24:24.164 INFO:teuthology.orchestra.run.smithi049.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-18T22:24:24.164 INFO:teuthology.orchestra.run.smithi049.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-18T22:24:24.164 INFO:teuthology.orchestra.run.smithi049.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-18T22:24:24.164 INFO:teuthology.orchestra.run.smithi049.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-18T22:24:24.165 INFO:teuthology.run_tasks:Running task install... 2024-03-18T22:24:24.173 DEBUG:teuthology.task.install:project ceph 2024-03-18T22:24:24.173 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'b6ae24918b03b1b8a19c9263a50655bba80bb810'}} 2024-03-18T22:24:24.173 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'b6ae24918b03b1b8a19c9263a50655bba80bb810'} 2024-03-18T22:24:24.174 INFO:teuthology.task.install:Using flavor: default 2024-03-18T22:24:24.182 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-03-18T22:24:24.182 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-03-18T22:24:24.182 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'b6ae24918b03b1b8a19c9263a50655bba80bb810', 'tag': None, 'wait_for_package': False} 2024-03-18T22:24:24.183 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b6ae24918b03b1b8a19c9263a50655bba80bb810 2024-03-18T22:24:24.185 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'b6ae24918b03b1b8a19c9263a50655bba80bb810', 'tag': None, 'wait_for_package': False} 2024-03-18T22:24:24.185 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b6ae24918b03b1b8a19c9263a50655bba80bb810 2024-03-18T22:24:24.186 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'b6ae24918b03b1b8a19c9263a50655bba80bb810', 'tag': None, 'wait_for_package': False} 2024-03-18T22:24:24.186 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b6ae24918b03b1b8a19c9263a50655bba80bb810 2024-03-18T22:24:24.330 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240318.181317/b6ae24918b03b1b8a19c9263a50655bba80bb810/centos/9/flavors/default/ 2024-03-18T22:24:24.331 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2206.gb6ae2491 2024-03-18T22:24:24.399 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240318.181317/b6ae24918b03b1b8a19c9263a50655bba80bb810/centos/9/flavors/default/ 2024-03-18T22:24:24.399 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2206.gb6ae2491 2024-03-18T22:24:24.406 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240318.181317/b6ae24918b03b1b8a19c9263a50655bba80bb810/centos/9/flavors/default/ 2024-03-18T22:24:24.406 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2206.gb6ae2491 2024-03-18T22:24:24.508 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240318.181317/b6ae24918b03b1b8a19c9263a50655bba80bb810/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240318.181317/b6ae24918b03b1b8a19c9263a50655bba80bb810/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240318.181317/b6ae24918b03b1b8a19c9263a50655bba80bb810/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-03-18T22:24:24.508 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-03-18T22:24:24.508 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-03-18T22:24:24.541 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-03-18T22:24:24.541 DEBUG:teuthology.orchestra.run.smithi049:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/b6ae24918b03b1b8a19c9263a50655bba80bb810/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-03-18T22:24:24.574 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240318.181317/b6ae24918b03b1b8a19c9263a50655bba80bb810/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240318.181317/b6ae24918b03b1b8a19c9263a50655bba80bb810/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240318.181317/b6ae24918b03b1b8a19c9263a50655bba80bb810/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-03-18T22:24:24.575 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-03-18T22:24:24.575 DEBUG:teuthology.orchestra.run.smithi139:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-03-18T22:24:24.580 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240318.181317/b6ae24918b03b1b8a19c9263a50655bba80bb810/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240318.181317/b6ae24918b03b1b8a19c9263a50655bba80bb810/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240318.181317/b6ae24918b03b1b8a19c9263a50655bba80bb810/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-03-18T22:24:24.581 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-03-18T22:24:24.581 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-03-18T22:24:24.603 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-03-18T22:24:24.603 DEBUG:teuthology.orchestra.run.smithi139:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/b6ae24918b03b1b8a19c9263a50655bba80bb810/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-03-18T22:24:24.609 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-03-18T22:24:24.609 DEBUG:teuthology.orchestra.run.smithi175:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/b6ae24918b03b1b8a19c9263a50655bba80bb810/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-03-18T22:24:24.612 DEBUG:teuthology.orchestra.run.smithi049:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-03-18T22:24:24.671 DEBUG:teuthology.orchestra.run.smithi139:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-03-18T22:24:24.677 DEBUG:teuthology.orchestra.run.smithi175:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-03-18T22:24:24.698 DEBUG:teuthology.orchestra.run.smithi049:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-03-18T22:24:24.728 INFO:teuthology.orchestra.run.smithi049.stdout:check_obsoletes = 1 2024-03-18T22:24:24.730 DEBUG:teuthology.orchestra.run.smithi049:> sudo yum clean all 2024-03-18T22:24:24.752 DEBUG:teuthology.orchestra.run.smithi139:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-03-18T22:24:24.757 DEBUG:teuthology.orchestra.run.smithi175:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-03-18T22:24:24.818 INFO:teuthology.orchestra.run.smithi139.stdout:check_obsoletes = 1 2024-03-18T22:24:24.819 DEBUG:teuthology.orchestra.run.smithi139:> sudo yum clean all 2024-03-18T22:24:24.823 INFO:teuthology.orchestra.run.smithi175.stdout:check_obsoletes = 1 2024-03-18T22:24:24.825 DEBUG:teuthology.orchestra.run.smithi175:> sudo yum clean all 2024-03-18T22:24:24.978 INFO:teuthology.orchestra.run.smithi049.stdout:74 files removed 2024-03-18T22:24:25.005 DEBUG:teuthology.orchestra.run.smithi049:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-03-18T22:24:25.053 INFO:teuthology.orchestra.run.smithi139.stdout:74 files removed 2024-03-18T22:24:25.059 INFO:teuthology.orchestra.run.smithi175.stdout:74 files removed 2024-03-18T22:24:25.080 DEBUG:teuthology.orchestra.run.smithi139:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-03-18T22:24:25.086 DEBUG:teuthology.orchestra.run.smithi175:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-03-18T22:24:25.670 INFO:teuthology.orchestra.run.smithi049.stdout:ceph packages for x86_64 240 kB/s | 83 kB 00:00 2024-03-18T22:24:25.772 INFO:teuthology.orchestra.run.smithi175.stdout:ceph packages for x86_64 209 kB/s | 83 kB 00:00 2024-03-18T22:24:25.783 INFO:teuthology.orchestra.run.smithi139.stdout:ceph packages for x86_64 201 kB/s | 83 kB 00:00 2024-03-18T22:24:26.054 INFO:teuthology.orchestra.run.smithi049.stdout:ceph noarch packages 30 kB/s | 10 kB 00:00 2024-03-18T22:24:26.189 INFO:teuthology.orchestra.run.smithi175.stdout:ceph noarch packages 27 kB/s | 10 kB 00:00 2024-03-18T22:24:26.234 INFO:teuthology.orchestra.run.smithi139.stdout:ceph noarch packages 24 kB/s | 10 kB 00:00 2024-03-18T22:24:26.430 INFO:teuthology.orchestra.run.smithi049.stdout:ceph source packages 5.6 kB/s | 1.9 kB 00:00 2024-03-18T22:24:26.598 INFO:teuthology.orchestra.run.smithi175.stdout:ceph source packages 5.0 kB/s | 1.9 kB 00:00 2024-03-18T22:24:26.634 INFO:teuthology.orchestra.run.smithi139.stdout:ceph source packages 5.1 kB/s | 1.9 kB 00:00 2024-03-18T22:24:26.865 INFO:teuthology.orchestra.run.smithi049.stdout:CentOS-9 - AppStream 17 MB/s | 6.7 MB 00:00 2024-03-18T22:24:27.003 INFO:teuthology.orchestra.run.smithi175.stdout:CentOS-9 - AppStream 18 MB/s | 6.7 MB 00:00 2024-03-18T22:24:27.070 INFO:teuthology.orchestra.run.smithi139.stdout:CentOS-9 - AppStream 16 MB/s | 6.7 MB 00:00 2024-03-18T22:24:28.332 INFO:teuthology.orchestra.run.smithi049.stdout:CentOS-9 - BaseOS 6.6 MB/s | 2.0 MB 00:00 2024-03-18T22:24:28.387 INFO:teuthology.orchestra.run.smithi175.stdout:CentOS-9 - BaseOS 6.9 MB/s | 2.0 MB 00:00 2024-03-18T22:24:28.479 INFO:teuthology.orchestra.run.smithi139.stdout:CentOS-9 - BaseOS 6.5 MB/s | 2.0 MB 00:00 2024-03-18T22:24:29.981 INFO:teuthology.orchestra.run.smithi049.stdout:CentOS Stream 9 - BaseOS 6.1 MB/s | 8.0 MB 00:01 2024-03-18T22:24:30.153 INFO:teuthology.orchestra.run.smithi139.stdout:CentOS Stream 9 - BaseOS 6.0 MB/s | 8.0 MB 00:01 2024-03-18T22:24:32.186 INFO:teuthology.orchestra.run.smithi175.stdout:CentOS Stream 9 - BaseOS 2.3 MB/s | 8.0 MB 00:03 2024-03-18T22:24:34.824 INFO:teuthology.orchestra.run.smithi049.stdout:CentOS Stream 9 - AppStream 5.1 MB/s | 19 MB 00:03 2024-03-18T22:24:37.754 INFO:teuthology.orchestra.run.smithi139.stdout:CentOS Stream 9 - AppStream 2.9 MB/s | 19 MB 00:06 2024-03-18T22:24:40.616 INFO:teuthology.orchestra.run.smithi049.stdout:CentOS Stream 9 - CRB 2.8 MB/s | 6.1 MB 00:02 2024-03-18T22:24:43.458 INFO:teuthology.orchestra.run.smithi049.stdout:CentOS Stream 9 - Extras packages 9.7 kB/s | 16 kB 00:01 2024-03-18T22:24:46.093 INFO:teuthology.orchestra.run.smithi049.stdout:Extra Packages for Enterprise Linux 8.2 MB/s | 21 MB 00:02 2024-03-18T22:24:51.251 INFO:teuthology.orchestra.run.smithi049.stdout:lab-extras 43 kB/s | 1.7 kB 00:00 2024-03-18T22:24:53.721 INFO:teuthology.orchestra.run.smithi049.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-18T22:24:53.722 INFO:teuthology.orchestra.run.smithi049.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-18T22:24:53.734 INFO:teuthology.orchestra.run.smithi049.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-03-18T22:24:53.735 INFO:teuthology.orchestra.run.smithi049.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-03-18T22:24:53.736 INFO:teuthology.orchestra.run.smithi049.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-03-18T22:24:53.737 INFO:teuthology.orchestra.run.smithi049.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-03-18T22:24:53.739 INFO:teuthology.orchestra.run.smithi049.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-03-18T22:24:53.741 INFO:teuthology.orchestra.run.smithi049.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-03-18T22:24:53.742 INFO:teuthology.orchestra.run.smithi049.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-03-18T22:24:53.830 INFO:teuthology.orchestra.run.smithi049.stdout:Dependencies resolved. 2024-03-18T22:24:53.850 INFO:teuthology.orchestra.run.smithi049.stdout:============================================================================================ 2024-03-18T22:24:53.850 INFO:teuthology.orchestra.run.smithi049.stdout: Package Arch Version Repository Size 2024-03-18T22:24:53.850 INFO:teuthology.orchestra.run.smithi049.stdout:============================================================================================ 2024-03-18T22:24:53.850 INFO:teuthology.orchestra.run.smithi049.stdout:Installing: 2024-03-18T22:24:53.850 INFO:teuthology.orchestra.run.smithi049.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-03-18T22:24:53.850 INFO:teuthology.orchestra.run.smithi049.stdout: ceph x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 6.6 k 2024-03-18T22:24:53.850 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-base x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 5.4 M 2024-03-18T22:24:53.851 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-fuse x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 891 k 2024-03-18T22:24:53.851 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 145 k 2024-03-18T22:24:53.851 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mgr x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 1.5 M 2024-03-18T22:24:53.851 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2206.gb6ae2491.el9 ceph-noarch 139 k 2024-03-18T22:24:53.851 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2206.gb6ae2491.el9 ceph-noarch 1.8 M 2024-03-18T22:24:53.851 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2206.gb6ae2491.el9 ceph-noarch 7.4 M 2024-03-18T22:24:53.851 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mgr-rook noarch 2:19.0.0-2206.gb6ae2491.el9 ceph-noarch 49 k 2024-03-18T22:24:53.851 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-radosgw x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 9.8 M 2024-03-18T22:24:53.851 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-test x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 48 M 2024-03-18T22:24:53.851 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-volume noarch 2:19.0.0-2206.gb6ae2491.el9 ceph-noarch 264 k 2024-03-18T22:24:53.851 INFO:teuthology.orchestra.run.smithi049.stdout: cephadm noarch 2:19.0.0-2206.gb6ae2491.el9 ceph-noarch 749 k 2024-03-18T22:24:53.851 INFO:teuthology.orchestra.run.smithi049.stdout: cephfs-mirror x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 221 k 2024-03-18T22:24:53.851 INFO:teuthology.orchestra.run.smithi049.stdout: cephfs-shell x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 27 k 2024-03-18T22:24:53.852 INFO:teuthology.orchestra.run.smithi049.stdout: cephfs-top noarch 2:19.0.0-2206.gb6ae2491.el9 ceph-noarch 18 k 2024-03-18T22:24:53.852 INFO:teuthology.orchestra.run.smithi049.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-03-18T22:24:53.852 INFO:teuthology.orchestra.run.smithi049.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-03-18T22:24:53.852 INFO:teuthology.orchestra.run.smithi049.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-03-18T22:24:53.852 INFO:teuthology.orchestra.run.smithi049.stdout: libcephfs-devel x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 32 k 2024-03-18T22:24:53.852 INFO:teuthology.orchestra.run.smithi049.stdout: libcephfs2 x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 746 k 2024-03-18T22:24:53.852 INFO:teuthology.orchestra.run.smithi049.stdout: librados-devel x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 127 k 2024-03-18T22:24:53.852 INFO:teuthology.orchestra.run.smithi049.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-03-18T22:24:53.852 INFO:teuthology.orchestra.run.smithi049.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-03-18T22:24:53.852 INFO:teuthology.orchestra.run.smithi049.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-03-18T22:24:53.852 INFO:teuthology.orchestra.run.smithi049.stdout: postgresql x86_64 13.11-1.el9 CentOS-AppStream 1.5 M 2024-03-18T22:24:53.852 INFO:teuthology.orchestra.run.smithi049.stdout: postgresql-contrib x86_64 13.11-1.el9 CentOS-AppStream 811 k 2024-03-18T22:24:53.853 INFO:teuthology.orchestra.run.smithi049.stdout: postgresql-server x86_64 13.11-1.el9 CentOS-AppStream 5.7 M 2024-03-18T22:24:53.853 INFO:teuthology.orchestra.run.smithi049.stdout: python3-cephfs x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 162 k 2024-03-18T22:24:53.853 INFO:teuthology.orchestra.run.smithi049.stdout: python3-rados x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 321 k 2024-03-18T22:24:53.853 INFO:teuthology.orchestra.run.smithi049.stdout: python3-rbd x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 298 k 2024-03-18T22:24:53.853 INFO:teuthology.orchestra.run.smithi049.stdout: python3-rgw x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 99 k 2024-03-18T22:24:53.853 INFO:teuthology.orchestra.run.smithi049.stdout: rbd-fuse x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 85 k 2024-03-18T22:24:53.853 INFO:teuthology.orchestra.run.smithi049.stdout: rbd-mirror x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 3.1 M 2024-03-18T22:24:53.853 INFO:teuthology.orchestra.run.smithi049.stdout: rbd-nbd x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 170 k 2024-03-18T22:24:53.853 INFO:teuthology.orchestra.run.smithi049.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-03-18T22:24:53.853 INFO:teuthology.orchestra.run.smithi049.stdout:Upgrading: 2024-03-18T22:24:53.853 INFO:teuthology.orchestra.run.smithi049.stdout: NetworkManager x86_64 1:1.47.2-1.el9 baseos 2.3 M 2024-03-18T22:24:53.853 INFO:teuthology.orchestra.run.smithi049.stdout: NetworkManager-libnm x86_64 1:1.47.2-1.el9 baseos 1.8 M 2024-03-18T22:24:53.854 INFO:teuthology.orchestra.run.smithi049.stdout: NetworkManager-team x86_64 1:1.47.2-1.el9 baseos 42 k 2024-03-18T22:24:53.854 INFO:teuthology.orchestra.run.smithi049.stdout: NetworkManager-tui x86_64 1:1.47.2-1.el9 baseos 248 k 2024-03-18T22:24:53.854 INFO:teuthology.orchestra.run.smithi049.stdout: acl x86_64 2.3.1-4.el9 baseos 71 k 2024-03-18T22:24:53.854 INFO:teuthology.orchestra.run.smithi049.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 baseos 36 k 2024-03-18T22:24:53.854 INFO:teuthology.orchestra.run.smithi049.stdout: elfutils-libelf x86_64 0.190-2.el9 baseos 192 k 2024-03-18T22:24:53.854 INFO:teuthology.orchestra.run.smithi049.stdout: elfutils-libs x86_64 0.190-2.el9 baseos 256 k 2024-03-18T22:24:53.854 INFO:teuthology.orchestra.run.smithi049.stdout: libacl x86_64 2.3.1-4.el9 baseos 23 k 2024-03-18T22:24:53.854 INFO:teuthology.orchestra.run.smithi049.stdout: libacl-devel x86_64 2.3.1-4.el9 appstream 74 k 2024-03-18T22:24:53.854 INFO:teuthology.orchestra.run.smithi049.stdout: libblkid x86_64 2.37.4-18.el9 baseos 108 k 2024-03-18T22:24:53.854 INFO:teuthology.orchestra.run.smithi049.stdout: libfdisk x86_64 2.37.4-18.el9 baseos 155 k 2024-03-18T22:24:53.854 INFO:teuthology.orchestra.run.smithi049.stdout: libmount x86_64 2.37.4-18.el9 baseos 136 k 2024-03-18T22:24:53.854 INFO:teuthology.orchestra.run.smithi049.stdout: librados2 x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 3.4 M 2024-03-18T22:24:53.854 INFO:teuthology.orchestra.run.smithi049.stdout: librbd1 x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 3.1 M 2024-03-18T22:24:53.855 INFO:teuthology.orchestra.run.smithi049.stdout: libsmartcols x86_64 2.37.4-18.el9 baseos 64 k 2024-03-18T22:24:53.855 INFO:teuthology.orchestra.run.smithi049.stdout: libuuid x86_64 2.37.4-18.el9 baseos 29 k 2024-03-18T22:24:53.855 INFO:teuthology.orchestra.run.smithi049.stdout: openssl x86_64 1:3.0.7-27.el9 baseos 1.2 M 2024-03-18T22:24:53.855 INFO:teuthology.orchestra.run.smithi049.stdout: openssl-libs x86_64 1:3.0.7-27.el9 baseos 2.2 M 2024-03-18T22:24:53.855 INFO:teuthology.orchestra.run.smithi049.stdout: util-linux x86_64 2.37.4-18.el9 baseos 2.3 M 2024-03-18T22:24:53.855 INFO:teuthology.orchestra.run.smithi049.stdout: util-linux-core x86_64 2.37.4-18.el9 baseos 465 k 2024-03-18T22:24:53.855 INFO:teuthology.orchestra.run.smithi049.stdout: util-linux-user x86_64 2.37.4-18.el9 baseos 31 k 2024-03-18T22:24:53.855 INFO:teuthology.orchestra.run.smithi049.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-03-18T22:24:53.855 INFO:teuthology.orchestra.run.smithi049.stdout: xfsprogs x86_64 6.3.0-1.el9 baseos 1.1 M 2024-03-18T22:24:53.855 INFO:teuthology.orchestra.run.smithi049.stdout:Installing dependencies: 2024-03-18T22:24:53.855 INFO:teuthology.orchestra.run.smithi049.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-03-18T22:24:53.855 INFO:teuthology.orchestra.run.smithi049.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-03-18T22:24:53.855 INFO:teuthology.orchestra.run.smithi049.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-03-18T22:24:53.856 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-common x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 21 M 2024-03-18T22:24:53.856 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2206.gb6ae2491.el9 ceph-noarch 34 k 2024-03-18T22:24:53.856 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mds x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 2.4 M 2024-03-18T22:24:53.856 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2206.gb6ae2491.el9 ceph-noarch 248 k 2024-03-18T22:24:53.856 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mon x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 4.7 M 2024-03-18T22:24:53.856 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-osd x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 17 M 2024-03-18T22:24:53.856 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2206.gb6ae2491.el9 ceph-noarch 16 k 2024-03-18T22:24:53.856 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-selinux x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 25 k 2024-03-18T22:24:53.856 INFO:teuthology.orchestra.run.smithi049.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-03-18T22:24:53.856 INFO:teuthology.orchestra.run.smithi049.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-03-18T22:24:53.856 INFO:teuthology.orchestra.run.smithi049.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-03-18T22:24:53.856 INFO:teuthology.orchestra.run.smithi049.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-03-18T22:24:53.856 INFO:teuthology.orchestra.run.smithi049.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-03-18T22:24:53.856 INFO:teuthology.orchestra.run.smithi049.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-03-18T22:24:53.857 INFO:teuthology.orchestra.run.smithi049.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-03-18T22:24:53.857 INFO:teuthology.orchestra.run.smithi049.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-03-18T22:24:53.857 INFO:teuthology.orchestra.run.smithi049.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-03-18T22:24:53.857 INFO:teuthology.orchestra.run.smithi049.stdout: libcephsqlite x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 163 k 2024-03-18T22:24:53.857 INFO:teuthology.orchestra.run.smithi049.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-03-18T22:24:53.857 INFO:teuthology.orchestra.run.smithi049.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-03-18T22:24:53.857 INFO:teuthology.orchestra.run.smithi049.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-03-18T22:24:53.857 INFO:teuthology.orchestra.run.smithi049.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-03-18T22:24:53.857 INFO:teuthology.orchestra.run.smithi049.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-03-18T22:24:53.858 INFO:teuthology.orchestra.run.smithi049.stdout: libradosstriper1 x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 501 k 2024-03-18T22:24:53.858 INFO:teuthology.orchestra.run.smithi049.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-03-18T22:24:53.858 INFO:teuthology.orchestra.run.smithi049.stdout: librgw2 x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 5.1 M 2024-03-18T22:24:53.858 INFO:teuthology.orchestra.run.smithi049.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-03-18T22:24:53.858 INFO:teuthology.orchestra.run.smithi049.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-03-18T22:24:53.858 INFO:teuthology.orchestra.run.smithi049.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-03-18T22:24:53.858 INFO:teuthology.orchestra.run.smithi049.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-03-18T22:24:53.858 INFO:teuthology.orchestra.run.smithi049.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-03-18T22:24:53.858 INFO:teuthology.orchestra.run.smithi049.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-03-18T22:24:53.858 INFO:teuthology.orchestra.run.smithi049.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-03-18T22:24:53.858 INFO:teuthology.orchestra.run.smithi049.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-03-18T22:24:53.858 INFO:teuthology.orchestra.run.smithi049.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-03-18T22:24:53.858 INFO:teuthology.orchestra.run.smithi049.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-03-18T22:24:53.859 INFO:teuthology.orchestra.run.smithi049.stdout: postgresql-private-libs x86_64 13.11-1.el9 CentOS-AppStream 134 k 2024-03-18T22:24:53.859 INFO:teuthology.orchestra.run.smithi049.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-03-18T22:24:53.859 INFO:teuthology.orchestra.run.smithi049.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-03-18T22:24:53.859 INFO:teuthology.orchestra.run.smithi049.stdout: python3-asyncssh noarch 2.13.2-4.el9 epel 548 k 2024-03-18T22:24:53.859 INFO:teuthology.orchestra.run.smithi049.stdout: python3-attrs noarch 20.3.0-7.el9 CentOS-AppStream 78 k 2024-03-18T22:24:53.859 INFO:teuthology.orchestra.run.smithi049.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-03-18T22:24:53.859 INFO:teuthology.orchestra.run.smithi049.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-03-18T22:24:53.859 INFO:teuthology.orchestra.run.smithi049.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-03-18T22:24:53.859 INFO:teuthology.orchestra.run.smithi049.stdout: python3-ceph-argparse x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 45 k 2024-03-18T22:24:53.859 INFO:teuthology.orchestra.run.smithi049.stdout: python3-ceph-common x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 131 k 2024-03-18T22:24:53.859 INFO:teuthology.orchestra.run.smithi049.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-03-18T22:24:53.859 INFO:teuthology.orchestra.run.smithi049.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-03-18T22:24:53.859 INFO:teuthology.orchestra.run.smithi049.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-03-18T22:24:53.859 INFO:teuthology.orchestra.run.smithi049.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-03-18T22:24:53.860 INFO:teuthology.orchestra.run.smithi049.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-03-18T22:24:53.860 INFO:teuthology.orchestra.run.smithi049.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-03-18T22:24:53.860 INFO:teuthology.orchestra.run.smithi049.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-03-18T22:24:53.860 INFO:teuthology.orchestra.run.smithi049.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-03-18T22:24:53.860 INFO:teuthology.orchestra.run.smithi049.stdout: python3-google-auth noarch 1:2.28.0-1.el9 epel 203 k 2024-03-18T22:24:53.860 INFO:teuthology.orchestra.run.smithi049.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-03-18T22:24:53.860 INFO:teuthology.orchestra.run.smithi049.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-03-18T22:24:53.860 INFO:teuthology.orchestra.run.smithi049.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-03-18T22:24:53.860 INFO:teuthology.orchestra.run.smithi049.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-03-18T22:24:53.860 INFO:teuthology.orchestra.run.smithi049.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-03-18T22:24:53.860 INFO:teuthology.orchestra.run.smithi049.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-03-18T22:24:53.860 INFO:teuthology.orchestra.run.smithi049.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-03-18T22:24:53.860 INFO:teuthology.orchestra.run.smithi049.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-03-18T22:24:53.861 INFO:teuthology.orchestra.run.smithi049.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-03-18T22:24:53.861 INFO:teuthology.orchestra.run.smithi049.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-03-18T22:24:53.861 INFO:teuthology.orchestra.run.smithi049.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-03-18T22:24:53.861 INFO:teuthology.orchestra.run.smithi049.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-03-18T22:24:53.861 INFO:teuthology.orchestra.run.smithi049.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-03-18T22:24:53.861 INFO:teuthology.orchestra.run.smithi049.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-03-18T22:24:53.861 INFO:teuthology.orchestra.run.smithi049.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-03-18T22:24:53.861 INFO:teuthology.orchestra.run.smithi049.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-03-18T22:24:53.861 INFO:teuthology.orchestra.run.smithi049.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-03-18T22:24:53.861 INFO:teuthology.orchestra.run.smithi049.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-03-18T22:24:53.861 INFO:teuthology.orchestra.run.smithi049.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-03-18T22:24:53.861 INFO:teuthology.orchestra.run.smithi049.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-03-18T22:24:53.861 INFO:teuthology.orchestra.run.smithi049.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-03-18T22:24:53.861 INFO:teuthology.orchestra.run.smithi049.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-03-18T22:24:53.861 INFO:teuthology.orchestra.run.smithi049.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-03-18T22:24:53.862 INFO:teuthology.orchestra.run.smithi049.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-03-18T22:24:53.862 INFO:teuthology.orchestra.run.smithi049.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-03-18T22:24:53.862 INFO:teuthology.orchestra.run.smithi049.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-03-18T22:24:53.862 INFO:teuthology.orchestra.run.smithi049.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-03-18T22:24:53.862 INFO:teuthology.orchestra.run.smithi049.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-03-18T22:24:53.862 INFO:teuthology.orchestra.run.smithi049.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-03-18T22:24:53.862 INFO:teuthology.orchestra.run.smithi049.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-03-18T22:24:53.862 INFO:teuthology.orchestra.run.smithi049.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-03-18T22:24:53.862 INFO:teuthology.orchestra.run.smithi049.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-03-18T22:24:53.862 INFO:teuthology.orchestra.run.smithi049.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-03-18T22:24:53.862 INFO:teuthology.orchestra.run.smithi049.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-03-18T22:24:53.862 INFO:teuthology.orchestra.run.smithi049.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-03-18T22:24:53.863 INFO:teuthology.orchestra.run.smithi049.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-03-18T22:24:53.863 INFO:teuthology.orchestra.run.smithi049.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-03-18T22:24:53.863 INFO:teuthology.orchestra.run.smithi049.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-03-18T22:24:53.863 INFO:teuthology.orchestra.run.smithi049.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-03-18T22:24:53.863 INFO:teuthology.orchestra.run.smithi049.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-03-18T22:24:53.863 INFO:teuthology.orchestra.run.smithi049.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-03-18T22:24:53.863 INFO:teuthology.orchestra.run.smithi049.stdout: python3-wcwidth noarch 0.2.5-8.el9 CentOS-AppStream 41 k 2024-03-18T22:24:53.863 INFO:teuthology.orchestra.run.smithi049.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-03-18T22:24:53.863 INFO:teuthology.orchestra.run.smithi049.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-03-18T22:24:53.863 INFO:teuthology.orchestra.run.smithi049.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-03-18T22:24:53.863 INFO:teuthology.orchestra.run.smithi049.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-03-18T22:24:53.863 INFO:teuthology.orchestra.run.smithi049.stdout: qatlib x86_64 23.11.0-1.el9 appstream 216 k 2024-03-18T22:24:53.863 INFO:teuthology.orchestra.run.smithi049.stdout: qatzip-libs x86_64 1.1.2-2.el9 appstream 44 k 2024-03-18T22:24:53.863 INFO:teuthology.orchestra.run.smithi049.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-03-18T22:24:53.864 INFO:teuthology.orchestra.run.smithi049.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-03-18T22:24:53.864 INFO:teuthology.orchestra.run.smithi049.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-03-18T22:24:53.864 INFO:teuthology.orchestra.run.smithi049.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-03-18T22:24:53.864 INFO:teuthology.orchestra.run.smithi049.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-03-18T22:24:53.864 INFO:teuthology.orchestra.run.smithi049.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-03-18T22:24:53.864 INFO:teuthology.orchestra.run.smithi049.stdout:Installing weak dependencies: 2024-03-18T22:24:53.864 INFO:teuthology.orchestra.run.smithi049.stdout: qatlib-service x86_64 23.11.0-1.el9 appstream 35 k 2024-03-18T22:24:53.864 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-18T22:24:53.864 INFO:teuthology.orchestra.run.smithi049.stdout:Transaction Summary 2024-03-18T22:24:53.864 INFO:teuthology.orchestra.run.smithi049.stdout:============================================================================================ 2024-03-18T22:24:53.864 INFO:teuthology.orchestra.run.smithi049.stdout:Install 150 Packages 2024-03-18T22:24:53.864 INFO:teuthology.orchestra.run.smithi049.stdout:Upgrade 24 Packages 2024-03-18T22:24:53.864 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-18T22:24:53.865 INFO:teuthology.orchestra.run.smithi049.stdout:Total download size: 224 M 2024-03-18T22:24:53.865 INFO:teuthology.orchestra.run.smithi049.stdout:Downloading Packages: 2024-03-18T22:24:54.444 INFO:teuthology.orchestra.run.smithi049.stdout:(1/174): ceph-19.0.0-2206.gb6ae2491.el9.x86_64. 28 kB/s | 6.6 kB 00:00 2024-03-18T22:24:55.436 INFO:teuthology.orchestra.run.smithi049.stdout:(2/174): ceph-common-19.0.0-2206.gb6ae2491.el9. 17 MB/s | 21 MB 00:01 2024-03-18T22:24:55.494 INFO:teuthology.orchestra.run.smithi049.stdout:(3/174): ceph-fuse-19.0.0-2206.gb6ae2491.el9.x8 849 kB/s | 891 kB 00:01 2024-03-18T22:24:55.519 INFO:teuthology.orchestra.run.smithi049.stdout:(4/174): ceph-immutable-object-cache-19.0.0-220 1.7 MB/s | 145 kB 00:00 2024-03-18T22:24:55.628 INFO:teuthology.orchestra.run.smithi049.stdout:(5/174): ceph-mgr-19.0.0-2206.gb6ae2491.el9.x86 14 MB/s | 1.5 MB 00:00 2024-03-18T22:24:55.895 INFO:teuthology.orchestra.run.smithi049.stdout:(6/174): ceph-mon-19.0.0-2206.gb6ae2491.el9.x86 18 MB/s | 4.7 MB 00:00 2024-03-18T22:24:56.520 INFO:teuthology.orchestra.run.smithi049.stdout:(7/174): ceph-mds-19.0.0-2206.gb6ae2491.el9.x86 2.3 MB/s | 2.4 MB 00:01 2024-03-18T22:24:56.771 INFO:teuthology.orchestra.run.smithi049.stdout:(8/174): ceph-osd-19.0.0-2206.gb6ae2491.el9.x86 19 MB/s | 17 MB 00:00 2024-03-18T22:24:56.804 INFO:teuthology.orchestra.run.smithi049.stdout:(9/174): ceph-selinux-19.0.0-2206.gb6ae2491.el9 764 kB/s | 25 kB 00:00 2024-03-18T22:24:56.896 INFO:teuthology.orchestra.run.smithi049.stdout:(10/174): ceph-base-19.0.0-2206.gb6ae2491.el9.x 2.0 MB/s | 5.4 MB 00:02 2024-03-18T22:24:57.063 INFO:teuthology.orchestra.run.smithi049.stdout:(11/174): cephfs-mirror-19.0.0-2206.gb6ae2491.e 1.3 MB/s | 221 kB 00:00 2024-03-18T22:24:57.163 INFO:teuthology.orchestra.run.smithi049.stdout:(12/174): cephfs-shell-19.0.0-2206.gb6ae2491.el 271 kB/s | 27 kB 00:00 2024-03-18T22:24:57.255 INFO:teuthology.orchestra.run.smithi049.stdout:(13/174): libcephfs-devel-19.0.0-2206.gb6ae2491 346 kB/s | 32 kB 00:00 2024-03-18T22:24:57.614 INFO:teuthology.orchestra.run.smithi049.stdout:(14/174): libcephfs2-19.0.0-2206.gb6ae2491.el9. 2.0 MB/s | 746 kB 00:00 2024-03-18T22:24:57.730 INFO:teuthology.orchestra.run.smithi049.stdout:(15/174): libcephsqlite-19.0.0-2206.gb6ae2491.e 1.4 MB/s | 163 kB 00:00 2024-03-18T22:24:57.848 INFO:teuthology.orchestra.run.smithi049.stdout:(16/174): librados-devel-19.0.0-2206.gb6ae2491. 1.1 MB/s | 127 kB 00:00 2024-03-18T22:24:58.148 INFO:teuthology.orchestra.run.smithi049.stdout:(17/174): libradosstriper1-19.0.0-2206.gb6ae249 1.6 MB/s | 501 kB 00:00 2024-03-18T22:24:58.506 INFO:teuthology.orchestra.run.smithi049.stdout:(18/174): ceph-radosgw-19.0.0-2206.gb6ae2491.el 5.0 MB/s | 9.8 MB 00:01 2024-03-18T22:24:58.581 INFO:teuthology.orchestra.run.smithi049.stdout:(19/174): python3-ceph-argparse-19.0.0-2206.gb6 603 kB/s | 45 kB 00:00 2024-03-18T22:24:58.690 INFO:teuthology.orchestra.run.smithi049.stdout:(20/174): python3-ceph-common-19.0.0-2206.gb6ae 1.2 MB/s | 131 kB 00:00 2024-03-18T22:24:58.807 INFO:teuthology.orchestra.run.smithi049.stdout:(21/174): python3-cephfs-19.0.0-2206.gb6ae2491. 1.4 MB/s | 162 kB 00:00 2024-03-18T22:24:59.466 INFO:teuthology.orchestra.run.smithi049.stdout:(22/174): ceph-test-19.0.0-2206.gb6ae2491.el9.x 18 MB/s | 48 MB 00:02 2024-03-18T22:24:59.500 INFO:teuthology.orchestra.run.smithi049.stdout:(23/174): python3-rados-19.0.0-2206.gb6ae2491.e 464 kB/s | 321 kB 00:00 2024-03-18T22:24:59.533 INFO:teuthology.orchestra.run.smithi049.stdout:(24/174): python3-rbd-19.0.0-2206.gb6ae2491.el9 4.4 MB/s | 298 kB 00:00 2024-03-18T22:24:59.567 INFO:teuthology.orchestra.run.smithi049.stdout:(25/174): python3-rgw-19.0.0-2206.gb6ae2491.el9 1.5 MB/s | 99 kB 00:00 2024-03-18T22:24:59.592 INFO:teuthology.orchestra.run.smithi049.stdout:(26/174): rbd-fuse-19.0.0-2206.gb6ae2491.el9.x8 1.4 MB/s | 85 kB 00:00 2024-03-18T22:24:59.642 INFO:teuthology.orchestra.run.smithi049.stdout:(27/174): rbd-nbd-19.0.0-2206.gb6ae2491.el9.x86 3.3 MB/s | 170 kB 00:00 2024-03-18T22:24:59.693 INFO:teuthology.orchestra.run.smithi049.stdout:(28/174): ceph-grafana-dashboards-19.0.0-2206.g 680 kB/s | 34 kB 00:00 2024-03-18T22:24:59.743 INFO:teuthology.orchestra.run.smithi049.stdout:(29/174): ceph-mgr-cephadm-19.0.0-2206.gb6ae249 2.7 MB/s | 139 kB 00:00 2024-03-18T22:24:59.885 INFO:teuthology.orchestra.run.smithi049.stdout:(30/174): ceph-mgr-dashboard-19.0.0-2206.gb6ae2 13 MB/s | 1.8 MB 00:00 2024-03-18T22:24:59.968 INFO:teuthology.orchestra.run.smithi049.stdout:(31/174): librgw2-19.0.0-2206.gb6ae2491.el9.x86 2.8 MB/s | 5.1 MB 00:01 2024-03-18T22:25:00.019 INFO:teuthology.orchestra.run.smithi049.stdout:(32/174): rbd-mirror-19.0.0-2206.gb6ae2491.el9. 6.8 MB/s | 3.1 MB 00:00 2024-03-18T22:25:00.102 INFO:teuthology.orchestra.run.smithi049.stdout:(33/174): ceph-mgr-rook-19.0.0-2206.gb6ae2491.e 592 kB/s | 49 kB 00:00 2024-03-18T22:25:00.144 INFO:teuthology.orchestra.run.smithi049.stdout:(34/174): ceph-mgr-modules-core-19.0.0-2206.gb6 1.4 MB/s | 248 kB 00:00 2024-03-18T22:25:00.253 INFO:teuthology.orchestra.run.smithi049.stdout:(35/174): ceph-mgr-diskprediction-local-19.0.0- 20 MB/s | 7.4 MB 00:00 2024-03-18T22:25:00.278 INFO:teuthology.orchestra.run.smithi049.stdout:(36/174): ceph-prometheus-alerts-19.0.0-2206.gb 94 kB/s | 16 kB 00:00 2024-03-18T22:25:00.303 INFO:teuthology.orchestra.run.smithi049.stdout:(37/174): ceph-volume-19.0.0-2206.gb6ae2491.el9 1.6 MB/s | 264 kB 00:00 2024-03-18T22:25:00.337 INFO:teuthology.orchestra.run.smithi049.stdout:(38/174): cephadm-19.0.0-2206.gb6ae2491.el9.noa 8.8 MB/s | 749 kB 00:00 2024-03-18T22:25:00.387 INFO:teuthology.orchestra.run.smithi049.stdout:(39/174): cephfs-top-19.0.0-2206.gb6ae2491.el9. 168 kB/s | 18 kB 00:00 2024-03-18T22:25:00.579 INFO:teuthology.orchestra.run.smithi049.stdout:(40/174): boost-program-options-1.75.0-8.el9.x8 642 kB/s | 106 kB 00:00 2024-03-18T22:25:00.604 INFO:teuthology.orchestra.run.smithi049.stdout:(41/174): automake-1.16.2-8.el9.noarch.rpm 2.2 MB/s | 662 kB 00:00 2024-03-18T22:25:00.638 INFO:teuthology.orchestra.run.smithi049.stdout:(42/174): bison-3.7.4-5.el9.x86_64.rpm 3.3 MB/s | 920 kB 00:00 2024-03-18T22:25:00.671 INFO:teuthology.orchestra.run.smithi049.stdout:(43/174): flex-2.6.4-9.el9.x86_64.rpm 3.3 MB/s | 307 kB 00:00 2024-03-18T22:25:00.697 INFO:teuthology.orchestra.run.smithi049.stdout:(44/174): flexiblas-3.0.4-8.el9.x86_64.rpm 347 kB/s | 32 kB 00:00 2024-03-18T22:25:00.722 INFO:teuthology.orchestra.run.smithi049.stdout:(45/174): flexiblas-openblas-openmp-3.0.4-8.el9 342 kB/s | 17 kB 00:00 2024-03-18T22:25:00.822 INFO:teuthology.orchestra.run.smithi049.stdout:(46/174): flexiblas-netlib-3.0.4-8.el9.x86_64.r 20 MB/s | 3.0 MB 00:00 2024-03-18T22:25:00.847 INFO:teuthology.orchestra.run.smithi049.stdout:(47/174): libaio-devel-0.3.111-13.el9.x86_64.rp 67 kB/s | 10 kB 00:00 2024-03-18T22:25:00.872 INFO:teuthology.orchestra.run.smithi049.stdout:(48/174): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.0 MB/s | 159 kB 00:00 2024-03-18T22:25:00.898 INFO:teuthology.orchestra.run.smithi049.stdout:(49/174): librabbitmq-0.11.0-7.el9.x86_64.rpm 890 kB/s | 44 kB 00:00 2024-03-18T22:25:00.923 INFO:teuthology.orchestra.run.smithi049.stdout:(50/174): librdkafka-1.6.1-102.el9.x86_64.rpm 8.6 MB/s | 661 kB 00:00 2024-03-18T22:25:00.949 INFO:teuthology.orchestra.run.smithi049.stdout:(51/174): libtool-2.4.6-45.el9.x86_64.rpm 7.6 MB/s | 578 kB 00:00 2024-03-18T22:25:00.974 INFO:teuthology.orchestra.run.smithi049.stdout:(52/174): libzstd-devel-1.5.1-2.el9.x86_64.rpm 941 kB/s | 46 kB 00:00 2024-03-18T22:25:00.999 INFO:teuthology.orchestra.run.smithi049.stdout:(53/174): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.7 MB/s | 282 kB 00:00 2024-03-18T22:25:01.024 INFO:teuthology.orchestra.run.smithi049.stdout:(54/174): lua-5.4.4-4.el9.x86_64.rpm 2.4 MB/s | 187 kB 00:00 2024-03-18T22:25:01.049 INFO:teuthology.orchestra.run.smithi049.stdout:(55/174): openblas-0.3.21-2.el9.x86_64.rpm 692 kB/s | 34 kB 00:00 2024-03-18T22:25:01.075 INFO:teuthology.orchestra.run.smithi049.stdout:(56/174): oniguruma-6.9.6-1.el9.5.x86_64.rpm 2.8 MB/s | 217 kB 00:00 2024-03-18T22:25:01.200 INFO:teuthology.orchestra.run.smithi049.stdout:(57/174): openblas-openmp-0.3.21-2.el9.x86_64.r 27 MB/s | 4.7 MB 00:00 2024-03-18T22:25:01.259 INFO:teuthology.orchestra.run.smithi049.stdout:(58/174): postgresql-13.11-1.el9.x86_64.rpm 7.4 MB/s | 1.5 MB 00:00 2024-03-18T22:25:01.292 INFO:teuthology.orchestra.run.smithi049.stdout:(59/174): postgresql-contrib-13.11-1.el9.x86_64 3.7 MB/s | 811 kB 00:00 2024-03-18T22:25:01.317 INFO:teuthology.orchestra.run.smithi049.stdout:(60/174): postgresql-private-libs-13.11-1.el9.x 1.1 MB/s | 134 kB 00:00 2024-03-18T22:25:01.376 INFO:teuthology.orchestra.run.smithi049.stdout:(61/174): protobuf-3.14.0-13.el9.x86_64.rpm 12 MB/s | 1.0 MB 00:00 2024-03-18T22:25:01.476 INFO:teuthology.orchestra.run.smithi049.stdout:(62/174): postgresql-server-13.11-1.el9.x86_64. 31 MB/s | 5.7 MB 00:00 2024-03-18T22:25:01.502 INFO:teuthology.orchestra.run.smithi049.stdout:(63/174): python3-attrs-20.3.0-7.el9.noarch.rpm 427 kB/s | 78 kB 00:00 2024-03-18T22:25:01.593 INFO:teuthology.orchestra.run.smithi049.stdout:(64/174): python3-babel-2.9.1-2.el9.noarch.rpm 27 MB/s | 5.8 MB 00:00 2024-03-18T22:25:01.618 INFO:teuthology.orchestra.run.smithi049.stdout:(65/174): python3-jsonpatch-1.21-16.el9.noarch. 175 kB/s | 24 kB 00:00 2024-03-18T22:25:01.635 INFO:teuthology.orchestra.run.smithi049.stdout:(66/174): python3-jsonpointer-2.0-4.el9.noarch. 138 kB/s | 18 kB 00:00 2024-03-18T22:25:01.663 INFO:teuthology.orchestra.run.smithi049.stdout:(67/174): python3-mako-1.1.4-6.el9.noarch.rpm 2.0 MB/s | 144 kB 00:00 2024-03-18T22:25:01.688 INFO:teuthology.orchestra.run.smithi049.stdout:(68/174): python3-markupsafe-1.1.1-12.el9.x86_6 603 kB/s | 32 kB 00:00 2024-03-18T22:25:01.780 INFO:teuthology.orchestra.run.smithi049.stdout:(69/174): python3-numpy-1.20.1-5.el9.x86_64.rpm 33 MB/s | 4.8 MB 00:00 2024-03-18T22:25:01.806 INFO:teuthology.orchestra.run.smithi049.stdout:(70/174): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.5 MB/s | 217 kB 00:00 2024-03-18T22:25:01.831 INFO:teuthology.orchestra.run.smithi049.stdout:(71/174): python3-oauthlib-3.1.1-5.el9.noarch.r 1.2 MB/s | 174 kB 00:00 2024-03-18T22:25:01.848 INFO:teuthology.orchestra.run.smithi049.stdout:(72/174): python3-packaging-20.9-5.el9.noarch.r 1.0 MB/s | 69 kB 00:00 2024-03-18T22:25:01.873 INFO:teuthology.orchestra.run.smithi049.stdout:(73/174): python3-prettytable-0.7.2-27.el9.noar 989 kB/s | 41 kB 00:00 2024-03-18T22:25:01.898 INFO:teuthology.orchestra.run.smithi049.stdout:(74/174): python3-protobuf-3.14.0-13.el9.noarch 3.5 MB/s | 238 kB 00:00 2024-03-18T22:25:01.924 INFO:teuthology.orchestra.run.smithi049.stdout:(75/174): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.7 MB/s | 132 kB 00:00 2024-03-18T22:25:01.949 INFO:teuthology.orchestra.run.smithi049.stdout:(76/174): python3-pytz-2021.1-5.el9.noarch.rpm 954 kB/s | 47 kB 00:00 2024-03-18T22:25:01.974 INFO:teuthology.orchestra.run.smithi049.stdout:(77/174): python3-pyasn1-modules-0.4.8-6.el9.no 2.7 MB/s | 211 kB 00:00 2024-03-18T22:25:01.999 INFO:teuthology.orchestra.run.smithi049.stdout:(78/174): python3-requests-oauthlib-1.3.0-12.el 569 kB/s | 43 kB 00:00 2024-03-18T22:25:02.200 INFO:teuthology.orchestra.run.smithi049.stdout:(79/174): python3-toml-0.10.2-6.el9.noarch.rpm 196 kB/s | 39 kB 00:00 2024-03-18T22:25:02.350 INFO:teuthology.orchestra.run.smithi049.stdout:(80/174): python3-scipy-1.6.2-8.el9.x86_64.rpm 39 MB/s | 16 MB 00:00 2024-03-18T22:25:02.375 INFO:teuthology.orchestra.run.smithi049.stdout:(81/174): python3-wcwidth-0.2.5-8.el9.noarch.rp 110 kB/s | 41 kB 00:00 2024-03-18T22:25:02.400 INFO:teuthology.orchestra.run.smithi049.stdout:(82/174): socat-1.7.4.1-5.el9.x86_64.rpm 1.5 MB/s | 300 kB 00:00 2024-03-18T22:25:02.425 INFO:teuthology.orchestra.run.smithi049.stdout:(83/174): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 56 kB 00:00 2024-03-18T22:25:02.459 INFO:teuthology.orchestra.run.smithi049.stdout:(84/174): xmlstarlet-1.6.1-20.el9.x86_64.rpm 754 kB/s | 63 kB 00:00 2024-03-18T22:25:02.484 INFO:teuthology.orchestra.run.smithi049.stdout:(85/174): zlib-devel-1.2.11-40.el9.x86_64.rpm 528 kB/s | 44 kB 00:00 2024-03-18T22:25:02.509 INFO:teuthology.orchestra.run.smithi049.stdout:(86/174): fuse-2.9.9-15.el9.x86_64.rpm 1.5 MB/s | 79 kB 00:00 2024-03-18T22:25:02.534 INFO:teuthology.orchestra.run.smithi049.stdout:(87/174): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.1 MB/s | 241 kB 00:00 2024-03-18T22:25:02.585 INFO:teuthology.orchestra.run.smithi049.stdout:(88/174): python3-cryptography-36.0.1-4.el9.x86 11 MB/s | 1.1 MB 00:00 2024-03-18T22:25:02.611 INFO:teuthology.orchestra.run.smithi049.stdout:(89/174): python3-ply-3.11-14.el9.noarch.rpm 1.4 MB/s | 103 kB 00:00 2024-03-18T22:25:02.636 INFO:teuthology.orchestra.run.smithi049.stdout:(90/174): python3-pycparser-2.20-6.el9.noarch.r 1.2 MB/s | 124 kB 00:00 2024-03-18T22:25:02.661 INFO:teuthology.orchestra.run.smithi049.stdout:(91/174): python3-urllib3-1.26.5-5.el9.noarch.r 2.4 MB/s | 187 kB 00:00 2024-03-18T22:25:03.236 INFO:teuthology.orchestra.run.smithi049.stdout:(92/174): jq-1.6-16.el9.x86_64.rpm 312 kB/s | 187 kB 00:00 2024-03-18T22:25:03.261 INFO:teuthology.orchestra.run.smithi049.stdout:(93/174): libquadmath-11.4.1-3.el9.x86_64.rpm 325 kB/s | 195 kB 00:00 2024-03-18T22:25:03.362 INFO:teuthology.orchestra.run.smithi049.stdout:(94/174): python3-requests-2.25.1-8.el9.noarch. 997 kB/s | 125 kB 00:00 2024-03-18T22:25:03.387 INFO:teuthology.orchestra.run.smithi049.stdout:(95/174): elfutils-libelf-devel-0.190-2.el9.x86 172 kB/s | 21 kB 00:00 2024-03-18T22:25:03.479 INFO:teuthology.orchestra.run.smithi049.stdout:(96/174): libuuid-devel-2.37.4-18.el9.x86_64.rp 218 kB/s | 25 kB 00:00 2024-03-18T22:25:03.529 INFO:teuthology.orchestra.run.smithi049.stdout:(97/174): libgfortran-11.4.1-3.el9.x86_64.rpm 900 kB/s | 803 kB 00:00 2024-03-18T22:25:03.654 INFO:teuthology.orchestra.run.smithi049.stdout:(98/174): python3-devel-3.9.17-2.el9.x86_64.rpm 1.4 MB/s | 245 kB 00:00 2024-03-18T22:25:03.738 INFO:teuthology.orchestra.run.smithi049.stdout:(99/174): python3-jinja2-2.11.3-5.el9.noarch.rp 1.2 MB/s | 248 kB 00:00 2024-03-18T22:25:03.763 INFO:teuthology.orchestra.run.smithi049.stdout:(100/174): qatlib-23.11.0-1.el9.x86_64.rpm 1.9 MB/s | 216 kB 00:00 2024-03-18T22:25:03.780 INFO:teuthology.orchestra.run.smithi049.stdout:(101/174): qatlib-service-23.11.0-1.el9.x86_64. 832 kB/s | 35 kB 00:00 2024-03-18T22:25:03.822 INFO:teuthology.orchestra.run.smithi049.stdout:(102/174): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 2.2 MB/s | 94 kB 00:00 2024-03-18T22:25:03.839 INFO:teuthology.orchestra.run.smithi049.stdout:(103/174): qatzip-libs-1.1.2-2.el9.x86_64.rpm 590 kB/s | 44 kB 00:00 2024-03-18T22:25:03.956 INFO:teuthology.orchestra.run.smithi049.stdout:(104/174): lua-devel-5.4.4-4.el9.x86_64.rpm 167 kB/s | 22 kB 00:00 2024-03-18T22:25:04.131 INFO:teuthology.orchestra.run.smithi049.stdout:(105/174): protobuf-compiler-3.14.0-13.el9.x86_ 2.9 MB/s | 863 kB 00:00 2024-03-18T22:25:04.615 INFO:teuthology.orchestra.run.smithi049.stdout:(106/174): abseil-cpp-20211102.0-3.el9.x86_64.r 832 kB/s | 548 kB 00:00 2024-03-18T22:25:04.640 INFO:teuthology.orchestra.run.smithi049.stdout:(107/174): gperftools-libs-2.9.1-2.el9.x86_64.r 607 kB/s | 309 kB 00:00 2024-03-18T22:25:04.690 INFO:teuthology.orchestra.run.smithi049.stdout:(108/174): grpc-data-1.46.7-10.el9.noarch.rpm 260 kB/s | 19 kB 00:00 2024-03-18T22:25:04.765 INFO:teuthology.orchestra.run.smithi049.stdout:(109/174): openssl-devel-3.0.7-27.el9.x86_64.rp 3.0 MB/s | 4.1 MB 00:01 2024-03-18T22:25:04.949 INFO:teuthology.orchestra.run.smithi049.stdout:(110/174): libarrow-doc-9.0.0-10.el9.noarch.rpm 100 kB/s | 26 kB 00:00 2024-03-18T22:25:05.016 INFO:teuthology.orchestra.run.smithi049.stdout:(111/174): libunwind-1.6.2-1.el9.x86_64.rpm 1.0 MB/s | 67 kB 00:00 2024-03-18T22:25:05.108 INFO:teuthology.orchestra.run.smithi049.stdout:(112/174): luarocks-3.9.2-1.el9.noarch.rpm 1.6 MB/s | 151 kB 00:00 2024-03-18T22:25:05.166 INFO:teuthology.orchestra.run.smithi049.stdout:(113/174): liboath-2.6.7-2.el9.x86_64.rpm 121 kB/s | 49 kB 00:00 2024-03-18T22:25:05.292 INFO:teuthology.orchestra.run.smithi049.stdout:(114/174): parquet-libs-9.0.0-10.el9.x86_64.rpm 4.5 MB/s | 838 kB 00:00 2024-03-18T22:25:05.375 INFO:teuthology.orchestra.run.smithi049.stdout:(115/174): python3-bcrypt-3.2.2-1.el9.x86_64.rp 523 kB/s | 43 kB 00:00 2024-03-18T22:25:05.450 INFO:teuthology.orchestra.run.smithi049.stdout:(116/174): python3-cachetools-4.2.4-1.el9.noarc 431 kB/s | 32 kB 00:00 2024-03-18T22:25:05.584 INFO:teuthology.orchestra.run.smithi049.stdout:(117/174): python3-asyncssh-2.13.2-4.el9.noarch 1.3 MB/s | 548 kB 00:00 2024-03-18T22:25:05.626 INFO:teuthology.orchestra.run.smithi049.stdout:(118/174): python3-certifi-2021.10.8-2.el9.noar 85 kB/s | 15 kB 00:00 2024-03-18T22:25:05.676 INFO:teuthology.orchestra.run.smithi049.stdout:(119/174): python3-cheroot-8.6.0-4.el9.noarch.r 1.8 MB/s | 172 kB 00:00 2024-03-18T22:25:05.727 INFO:teuthology.orchestra.run.smithi049.stdout:(120/174): python3-cherrypy-18.6.1-2.el9.noarch 3.5 MB/s | 358 kB 00:00 2024-03-18T22:25:05.777 INFO:teuthology.orchestra.run.smithi049.stdout:(121/174): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.9 MB/s | 303 kB 00:00 2024-03-18T22:25:05.810 INFO:teuthology.orchestra.run.smithi049.stdout:(122/174): python3-colorama-0.4.4-12.el9.noarch 422 kB/s | 35 kB 00:00 2024-03-18T22:25:06.086 INFO:teuthology.orchestra.run.smithi049.stdout:(123/174): python3-google-auth-2.28.0-1.el9.noa 657 kB/s | 203 kB 00:00 2024-03-18T22:25:06.386 INFO:teuthology.orchestra.run.smithi049.stdout:(124/174): libarrow-9.0.0-10.el9.x86_64.rpm 2.5 MB/s | 4.4 MB 00:01 2024-03-18T22:25:06.437 INFO:teuthology.orchestra.run.smithi049.stdout:(125/174): python3-grpcio-1.46.7-10.el9.x86_64. 3.3 MB/s | 2.0 MB 00:00 2024-03-18T22:25:06.462 INFO:teuthology.orchestra.run.smithi049.stdout:(126/174): python3-grpcio-tools-1.46.7-10.el9.x 384 kB/s | 144 kB 00:00 2024-03-18T22:25:06.487 INFO:teuthology.orchestra.run.smithi049.stdout:(127/174): python3-jaraco-8.2.1-3.el9.noarch.rp 106 kB/s | 11 kB 00:00 2024-03-18T22:25:06.521 INFO:teuthology.orchestra.run.smithi049.stdout:(128/174): python3-jaraco-classes-3.2.1-5.el9.n 211 kB/s | 18 kB 00:00 2024-03-18T22:25:06.546 INFO:teuthology.orchestra.run.smithi049.stdout:(129/174): python3-jaraco-collections-3.0.0-8.e 276 kB/s | 23 kB 00:00 2024-03-18T22:25:06.571 INFO:teuthology.orchestra.run.smithi049.stdout:(130/174): python3-jaraco-functools-3.5.0-2.el9 232 kB/s | 19 kB 00:00 2024-03-18T22:25:06.597 INFO:teuthology.orchestra.run.smithi049.stdout:(131/174): python3-jaraco-text-3.2.0-6.el9.noar 260 kB/s | 20 kB 00:00 2024-03-18T22:25:06.655 INFO:teuthology.orchestra.run.smithi049.stdout:(132/174): python3-logutils-0.3.5-21.el9.noarch 554 kB/s | 46 kB 00:00 2024-03-18T22:25:06.681 INFO:teuthology.orchestra.run.smithi049.stdout:(133/174): python3-more-itertools-8.12.0-2.el9. 945 kB/s | 79 kB 00:00 2024-03-18T22:25:06.739 INFO:teuthology.orchestra.run.smithi049.stdout:(134/174): python3-natsort-7.1.1-5.el9.noarch.r 693 kB/s | 58 kB 00:00 2024-03-18T22:25:06.772 INFO:teuthology.orchestra.run.smithi049.stdout:(135/174): python3-pecan-1.4.2-3.el9.noarch.rpm 2.9 MB/s | 272 kB 00:00 2024-03-18T22:25:06.806 INFO:teuthology.orchestra.run.smithi049.stdout:(136/174): python3-kubernetes-26.1.0-2.el9.noar 3.9 MB/s | 1.0 MB 00:00 2024-03-18T22:25:06.831 INFO:teuthology.orchestra.run.smithi049.stdout:(137/174): python3-portend-3.1.0-2.el9.noarch.r 178 kB/s | 16 kB 00:00 2024-03-18T22:25:06.848 INFO:teuthology.orchestra.run.smithi049.stdout:(138/174): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.2 MB/s | 90 kB 00:00 2024-03-18T22:25:06.882 INFO:teuthology.orchestra.run.smithi049.stdout:(139/174): python3-pyperclip-1.8.2-2.el9.noarch 374 kB/s | 28 kB 00:00 2024-03-18T22:25:06.916 INFO:teuthology.orchestra.run.smithi049.stdout:(140/174): python3-repoze-lru-0.7-16.el9.noarch 367 kB/s | 31 kB 00:00 2024-03-18T22:25:06.941 INFO:teuthology.orchestra.run.smithi049.stdout:(141/174): python3-routes-2.5.1-5.el9.noarch.rp 2.0 MB/s | 188 kB 00:00 2024-03-18T22:25:06.966 INFO:teuthology.orchestra.run.smithi049.stdout:(142/174): python3-rsa-4.9-2.el9.noarch.rpm 705 kB/s | 59 kB 00:00 2024-03-18T22:25:07.025 INFO:teuthology.orchestra.run.smithi049.stdout:(143/174): python3-tempora-5.0.0-2.el9.noarch.r 329 kB/s | 36 kB 00:00 2024-03-18T22:25:07.050 INFO:teuthology.orchestra.run.smithi049.stdout:(144/174): python3-typing-extensions-4.4.0-2.el 468 kB/s | 51 kB 00:00 2024-03-18T22:25:07.075 INFO:teuthology.orchestra.run.smithi049.stdout:(145/174): python3-webob-1.8.7-6.el9.noarch.rpm 2.1 MB/s | 230 kB 00:00 2024-03-18T22:25:07.109 INFO:teuthology.orchestra.run.smithi049.stdout:(146/174): python3-websocket-client-1.2.3-2.el9 1.0 MB/s | 90 kB 00:00 2024-03-18T22:25:07.409 INFO:teuthology.orchestra.run.smithi049.stdout:(147/174): python3-zc-lockfile-2.0-10.el9.noarc 60 kB/s | 20 kB 00:00 2024-03-18T22:25:07.434 INFO:teuthology.orchestra.run.smithi049.stdout:(148/174): python3-werkzeug-2.0.3-3.el9.1.noarc 1.1 MB/s | 427 kB 00:00 2024-03-18T22:25:07.459 INFO:teuthology.orchestra.run.smithi049.stdout:(149/174): re2-20211101-3.el9.x86_64.rpm 547 kB/s | 192 kB 00:00 2024-03-18T22:25:07.785 INFO:teuthology.orchestra.run.smithi049.stdout:(150/174): thrift-0.15.0-2.el9.x86_64.rpm 4.2 MB/s | 1.6 MB 00:00 2024-03-18T22:25:07.843 INFO:teuthology.orchestra.run.smithi049.stdout:(151/174): librbd1-19.0.0-2206.gb6ae2491.el9.x8 8.2 MB/s | 3.1 MB 00:00 2024-03-18T22:25:07.868 INFO:teuthology.orchestra.run.smithi049.stdout:(152/174): xfsdump-3.1.12-4.el9.x86_64.rpm 3.8 MB/s | 321 kB 00:00 2024-03-18T22:25:08.169 INFO:teuthology.orchestra.run.smithi049.stdout:(153/174): librados2-19.0.0-2206.gb6ae2491.el9. 4.6 MB/s | 3.4 MB 00:00 2024-03-18T22:25:08.544 INFO:teuthology.orchestra.run.smithi049.stdout:(154/174): NetworkManager-team-1.47.2-1.el9.x86 112 kB/s | 42 kB 00:00 2024-03-18T22:25:08.678 INFO:teuthology.orchestra.run.smithi049.stdout:(155/174): NetworkManager-libnm-1.47.2-1.el9.x8 2.3 MB/s | 1.8 MB 00:00 2024-03-18T22:25:08.720 INFO:teuthology.orchestra.run.smithi049.stdout:(156/174): NetworkManager-1.47.2-1.el9.x86_64.r 2.6 MB/s | 2.3 MB 00:00 2024-03-18T22:25:08.787 INFO:teuthology.orchestra.run.smithi049.stdout:(157/174): acl-2.3.1-4.el9.x86_64.rpm 657 kB/s | 71 kB 00:00 2024-03-18T22:25:08.820 INFO:teuthology.orchestra.run.smithi049.stdout:(158/174): elfutils-debuginfod-client-0.190-2.e 364 kB/s | 36 kB 00:00 2024-03-18T22:25:08.887 INFO:teuthology.orchestra.run.smithi049.stdout:(159/174): NetworkManager-tui-1.47.2-1.el9.x86_ 724 kB/s | 248 kB 00:00 2024-03-18T22:25:08.912 INFO:teuthology.orchestra.run.smithi049.stdout:(160/174): elfutils-libelf-0.190-2.el9.x86_64.r 1.5 MB/s | 192 kB 00:00 2024-03-18T22:25:08.938 INFO:teuthology.orchestra.run.smithi049.stdout:(161/174): elfutils-libs-0.190-2.el9.x86_64.rpm 2.1 MB/s | 256 kB 00:00 2024-03-18T22:25:08.988 INFO:teuthology.orchestra.run.smithi049.stdout:(162/174): libacl-2.3.1-4.el9.x86_64.rpm 229 kB/s | 23 kB 00:00 2024-03-18T22:25:09.021 INFO:teuthology.orchestra.run.smithi049.stdout:(163/174): libblkid-2.37.4-18.el9.x86_64.rpm 998 kB/s | 108 kB 00:00 2024-03-18T22:25:09.047 INFO:teuthology.orchestra.run.smithi049.stdout:(164/174): libfdisk-2.37.4-18.el9.x86_64.rpm 1.4 MB/s | 155 kB 00:00 2024-03-18T22:25:09.097 INFO:teuthology.orchestra.run.smithi049.stdout:(165/174): libmount-2.37.4-18.el9.x86_64.rpm 1.2 MB/s | 136 kB 00:00 2024-03-18T22:25:09.131 INFO:teuthology.orchestra.run.smithi049.stdout:(166/174): libsmartcols-2.37.4-18.el9.x86_64.rp 587 kB/s | 64 kB 00:00 2024-03-18T22:25:09.157 INFO:teuthology.orchestra.run.smithi049.stdout:(167/174): libuuid-2.37.4-18.el9.x86_64.rpm 264 kB/s | 29 kB 00:00 2024-03-18T22:25:09.414 INFO:teuthology.orchestra.run.smithi049.stdout:(168/174): openssl-libs-3.0.7-27.el9.x86_64.rpm 7.6 MB/s | 2.2 MB 00:00 2024-03-18T22:25:09.473 INFO:teuthology.orchestra.run.smithi049.stdout:(169/174): util-linux-2.37.4-18.el9.x86_64.rpm 7.2 MB/s | 2.3 MB 00:00 2024-03-18T22:25:09.506 INFO:teuthology.orchestra.run.smithi049.stdout:(170/174): openssl-3.0.7-27.el9.x86_64.rpm 3.0 MB/s | 1.2 MB 00:00 2024-03-18T22:25:09.574 INFO:teuthology.orchestra.run.smithi049.stdout:(171/174): util-linux-user-2.37.4-18.el9.x86_64 311 kB/s | 31 kB 00:00 2024-03-18T22:25:09.632 INFO:teuthology.orchestra.run.smithi049.stdout:(172/174): xfsprogs-6.3.0-1.el9.x86_64.rpm 8.6 MB/s | 1.1 MB 00:00 2024-03-18T22:25:09.657 INFO:teuthology.orchestra.run.smithi049.stdout:(173/174): libacl-devel-2.3.1-4.el9.x86_64.rpm 889 kB/s | 74 kB 00:00 2024-03-18T22:25:09.933 INFO:teuthology.orchestra.run.smithi049.stdout:(174/174): util-linux-core-2.37.4-18.el9.x86_64 898 kB/s | 465 kB 00:00 2024-03-18T22:25:09.937 INFO:teuthology.orchestra.run.smithi049.stdout:-------------------------------------------------------------------------------- 2024-03-18T22:25:09.937 INFO:teuthology.orchestra.run.smithi049.stdout:Total 14 MB/s | 224 MB 00:16 2024-03-18T22:25:10.436 INFO:teuthology.orchestra.run.smithi049.stdout:Running transaction check 2024-03-18T22:25:10.739 INFO:teuthology.orchestra.run.smithi049.stdout:Transaction check succeeded. 2024-03-18T22:25:10.739 INFO:teuthology.orchestra.run.smithi049.stdout:Running transaction test 2024-03-18T22:25:12.874 INFO:teuthology.orchestra.run.smithi049.stdout:Transaction test succeeded. 2024-03-18T22:25:12.874 INFO:teuthology.orchestra.run.smithi049.stdout:Running transaction 2024-03-18T22:25:19.999 INFO:teuthology.orchestra.run.smithi049.stdout: Preparing : 1/1 2024-03-18T22:25:20.112 INFO:teuthology.orchestra.run.smithi049.stdout: Upgrading : openssl-libs-1:3.0.7-27.el9.x86_64 1/198 2024-03-18T22:25:20.146 INFO:teuthology.orchestra.run.smithi049.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/198 2024-03-18T22:25:20.184 INFO:teuthology.orchestra.run.smithi049.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-18T22:25:20.194 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-18T22:25:20.235 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/198 2024-03-18T22:25:20.289 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 5/198 2024-03-18T22:25:20.322 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/198 2024-03-18T22:25:20.347 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 7/198 2024-03-18T22:25:20.392 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 8/198 2024-03-18T22:25:20.744 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 9/198 2024-03-18T22:25:20.789 INFO:teuthology.orchestra.run.smithi049.stdout: Upgrading : librados2-2:19.0.0-2206.gb6ae2491.el9.x86_64 10/198 2024-03-18T22:25:20.914 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: librados2-2:19.0.0-2206.gb6ae2491.el9.x86_64 10/198 2024-03-18T22:25:20.940 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : libcephfs2-2:19.0.0-2206.gb6ae2491.el9.x86_64 11/198 2024-03-18T22:25:21.042 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: libcephfs2-2:19.0.0-2206.gb6ae2491.el9.x86_64 11/198 2024-03-18T22:25:21.078 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-rados-2:19.0.0-2206.gb6ae2491.el9.x86_64 12/198 2024-03-18T22:25:21.121 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : postgresql-private-libs-13.11-1.el9.x86_64 13/198 2024-03-18T22:25:21.308 INFO:teuthology.orchestra.run.smithi049.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 14/198 2024-03-18T22:25:21.368 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : postgresql-13.11-1.el9.x86_64 15/198 2024-03-18T22:25:21.395 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : libcephsqlite-2:19.0.0-2206.gb6ae2491.el9.x86_64 16/198 2024-03-18T22:25:21.501 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2206.gb6ae2491.el9.x86_64 16/198 2024-03-18T22:25:21.530 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : libradosstriper1-2:19.0.0-2206.gb6ae2491.el9.x86 17/198 2024-03-18T22:25:21.579 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2206.gb6ae2491.el9.x86 17/198 2024-03-18T22:25:21.618 INFO:teuthology.orchestra.run.smithi049.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/198 2024-03-18T22:25:21.631 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : qatlib-service-23.11.0-1.el9.x86_64 19/198 2024-03-18T22:25:21.648 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: qatlib-service-23.11.0-1.el9.x86_64 19/198 2024-03-18T22:25:21.803 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: qatlib-23.11.0-1.el9.x86_64 20/198 2024-03-18T22:25:21.830 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : qatlib-23.11.0-1.el9.x86_64 20/198 2024-03-18T22:25:21.869 INFO:teuthology.orchestra.run.smithi049.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 21/198 2024-03-18T22:25:22.030 INFO:teuthology.orchestra.run.smithi049.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 22/198 2024-03-18T22:25:22.158 INFO:teuthology.orchestra.run.smithi049.stdout: Upgrading : NetworkManager-libnm-1:1.47.2-1.el9.x86_64 23/198 2024-03-18T22:25:22.223 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.2-1.el9.x86_64 23/198 2024-03-18T22:25:22.437 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-18T22:25:22.674 INFO:teuthology.orchestra.run.smithi049.stdout: Upgrading : NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-18T22:25:22.909 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-18T22:25:23.025 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : re2-1:20211101-3.el9.x86_64 25/198 2024-03-18T22:25:23.120 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 26/198 2024-03-18T22:25:23.156 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 27/198 2024-03-18T22:25:23.192 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : liboath-2.6.7-2.el9.x86_64 28/198 2024-03-18T22:25:23.236 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/198 2024-03-18T22:25:23.312 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/198 2024-03-18T22:25:23.365 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 31/198 2024-03-18T22:25:23.402 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 32/198 2024-03-18T22:25:23.469 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 33/198 2024-03-18T22:25:23.509 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 34/198 2024-03-18T22:25:23.553 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 35/198 2024-03-18T22:25:23.586 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 36/198 2024-03-18T22:25:23.612 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : lua-5.4.4-4.el9.x86_64 37/198 2024-03-18T22:25:23.684 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 38/198 2024-03-18T22:25:23.718 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-ceph-common-2:19.0.0-2206.gb6ae2491.el9. 39/198 2024-03-18T22:25:23.753 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-ceph-argparse-2:19.0.0-2206.gb6ae2491.el 40/198 2024-03-18T22:25:23.773 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-cephfs-2:19.0.0-2206.gb6ae2491.el9.x86_6 41/198 2024-03-18T22:25:23.848 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 42/198 2024-03-18T22:25:23.896 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : luarocks-3.9.2-1.el9.noarch 43/198 2024-03-18T22:25:23.975 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 44/198 2024-03-18T22:25:24.121 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 45/198 2024-03-18T22:25:24.183 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 46/198 2024-03-18T22:25:24.747 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-rsa-4.9-2.el9.noarch 47/198 2024-03-18T22:25:24.831 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 48/198 2024-03-18T22:25:24.924 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 49/198 2024-03-18T22:25:24.972 INFO:teuthology.orchestra.run.smithi049.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-18T22:25:24.981 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-18T22:25:25.019 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : qatzip-libs-1.1.2-2.el9.x86_64 51/198 2024-03-18T22:25:25.070 INFO:teuthology.orchestra.run.smithi049.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 52/198 2024-03-18T22:25:25.107 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : librados-devel-2:19.0.0-2206.gb6ae2491.el9.x86_6 53/198 2024-03-18T22:25:25.137 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 54/198 2024-03-18T22:25:25.411 INFO:teuthology.orchestra.run.smithi049.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 55/198 2024-03-18T22:25:25.576 INFO:teuthology.orchestra.run.smithi049.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 56/198 2024-03-18T22:25:25.664 INFO:teuthology.orchestra.run.smithi049.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 57/198 2024-03-18T22:25:25.709 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 58/198 2024-03-18T22:25:26.005 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 59/198 2024-03-18T22:25:26.096 INFO:teuthology.orchestra.run.smithi049.stdout: Upgrading : openssl-1:3.0.7-27.el9.x86_64 60/198 2024-03-18T22:25:26.154 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 61/198 2024-03-18T22:25:26.244 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 62/198 2024-03-18T22:25:26.287 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 63/198 2024-03-18T22:25:26.324 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 64/198 2024-03-18T22:25:26.373 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 65/198 2024-03-18T22:25:26.406 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 66/198 2024-03-18T22:25:26.450 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 67/198 2024-03-18T22:25:26.493 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 68/198 2024-03-18T22:25:26.635 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 69/198 2024-03-18T22:25:26.697 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 70/198 2024-03-18T22:25:26.736 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 71/198 2024-03-18T22:25:26.778 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 72/198 2024-03-18T22:25:26.876 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 73/198 2024-03-18T22:25:26.910 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-google-auth-1:2.28.0-1.el9.noarch 74/198 2024-03-18T22:25:26.951 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 75/198 2024-03-18T22:25:26.976 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 76/198 2024-03-18T22:25:27.000 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 77/198 2024-03-18T22:25:27.088 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 78/198 2024-03-18T22:25:27.216 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 79/198 2024-03-18T22:25:27.324 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 80/198 2024-03-18T22:25:27.367 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 81/198 2024-03-18T22:25:27.439 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-ply-3.11-14.el9.noarch 82/198 2024-03-18T22:25:27.498 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 83/198 2024-03-18T22:25:27.695 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 84/198 2024-03-18T22:25:27.747 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 85/198 2024-03-18T22:25:27.893 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 86/198 2024-03-18T22:25:27.937 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-asyncssh-2.13.2-4.el9.noarch 87/198 2024-03-18T22:25:27.968 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 88/198 2024-03-18T22:25:27.999 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : fuse-2.9.9-15.el9.x86_64 89/198 2024-03-18T22:25:28.018 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 90/198 2024-03-18T22:25:28.053 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 91/198 2024-03-18T22:25:28.092 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : uuid-1.6.2-55.el9.x86_64 92/198 2024-03-18T22:25:28.132 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 93/198 2024-03-18T22:25:28.162 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 94/198 2024-03-18T22:25:28.243 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 95/198 2024-03-18T22:25:28.281 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 96/198 2024-03-18T22:25:28.314 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 97/198 2024-03-18T22:25:28.356 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 98/198 2024-03-18T22:25:28.390 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 99/198 2024-03-18T22:25:28.497 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 100/198 2024-03-18T22:25:28.600 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 101/198 2024-03-18T22:25:28.655 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 102/198 2024-03-18T22:25:28.686 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 103/198 2024-03-18T22:25:28.734 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 104/198 2024-03-18T22:25:28.845 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-packaging-20.9-5.el9.noarch 105/198 2024-03-18T22:25:28.897 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 106/198 2024-03-18T22:25:29.672 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 107/198 2024-03-18T22:25:29.717 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 108/198 2024-03-18T22:25:29.751 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 109/198 2024-03-18T22:25:29.801 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 110/198 2024-03-18T22:25:29.893 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 111/198 2024-03-18T22:25:29.927 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 112/198 2024-03-18T22:25:30.103 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : openblas-0.3.21-2.el9.x86_64 113/198 2024-03-18T22:25:30.124 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 114/198 2024-03-18T22:25:30.197 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 115/198 2024-03-18T22:25:30.890 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 116/198 2024-03-18T22:25:31.010 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 117/198 2024-03-18T22:25:32.310 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 118/198 2024-03-18T22:25:32.392 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 119/198 2024-03-18T22:25:32.431 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 120/198 2024-03-18T22:25:32.453 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : jq-1.6-16.el9.x86_64 121/198 2024-03-18T22:25:32.489 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 122/198 2024-03-18T22:25:32.818 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 123/198 2024-03-18T22:25:32.852 INFO:teuthology.orchestra.run.smithi049.stdout: Upgrading : librbd1-2:19.0.0-2206.gb6ae2491.el9.x86_64 124/198 2024-03-18T22:25:32.944 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: librbd1-2:19.0.0-2206.gb6ae2491.el9.x86_64 124/198 2024-03-18T22:25:32.999 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-rbd-2:19.0.0-2206.gb6ae2491.el9.x86_64 125/198 2024-03-18T22:25:33.036 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 126/198 2024-03-18T22:25:33.524 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 127/198 2024-03-18T22:25:33.547 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : librgw2-2:19.0.0-2206.gb6ae2491.el9.x86_64 128/198 2024-03-18T22:25:33.601 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: librgw2-2:19.0.0-2206.gb6ae2491.el9.x86_64 128/198 2024-03-18T22:25:33.632 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-rgw-2:19.0.0-2206.gb6ae2491.el9.x86_64 129/198 2024-03-18T22:25:36.019 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: ceph-common-2:19.0.0-2206.gb6ae2491.el9.x86_64 130/198 2024-03-18T22:25:36.053 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-common-2:19.0.0-2206.gb6ae2491.el9.x86_64 130/198 2024-03-18T22:25:36.630 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: ceph-common-2:19.0.0-2206.gb6ae2491.el9.x86_64 130/198 2024-03-18T22:25:36.665 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-base-2:19.0.0-2206.gb6ae2491.el9.x86_64 131/198 2024-03-18T22:25:36.756 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: ceph-base-2:19.0.0-2206.gb6ae2491.el9.x86_64 131/198 2024-03-18T22:25:36.756 INFO:teuthology.orchestra.run.smithi049.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-03-18T22:25:36.756 INFO:teuthology.orchestra.run.smithi049.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-03-18T22:25:36.757 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-18T22:25:36.791 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-selinux-2:19.0.0-2206.gb6ae2491.el9.x86_64 132/198 2024-03-18T22:25:48.854 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2206.gb6ae2491.el9.x86_64 132/198 2024-03-18T22:25:48.855 INFO:teuthology.orchestra.run.smithi049.stdout:skipping the directory /sys 2024-03-18T22:25:48.855 INFO:teuthology.orchestra.run.smithi049.stdout:skipping the directory /proc 2024-03-18T22:25:48.855 INFO:teuthology.orchestra.run.smithi049.stdout:skipping the directory /mnt 2024-03-18T22:25:48.855 INFO:teuthology.orchestra.run.smithi049.stdout:skipping the directory /var/tmp 2024-03-18T22:25:48.855 INFO:teuthology.orchestra.run.smithi049.stdout:skipping the directory /home 2024-03-18T22:25:48.855 INFO:teuthology.orchestra.run.smithi049.stdout:skipping the directory /root 2024-03-18T22:25:48.855 INFO:teuthology.orchestra.run.smithi049.stdout:skipping the directory /tmp 2024-03-18T22:25:48.855 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-18T22:25:49.028 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-volume-2:19.0.0-2206.gb6ae2491.el9.noarch 133/198 2024-03-18T22:25:49.044 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: ceph-volume-2:19.0.0-2206.gb6ae2491.el9.noarch 133/198 2024-03-18T22:25:49.044 INFO:teuthology.orchestra.run.smithi049.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-18T22:25:49.044 INFO:teuthology.orchestra.run.smithi049.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-03-18T22:25:49.044 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-18T22:25:50.582 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-osd-2:19.0.0-2206.gb6ae2491.el9.x86_64 134/198 2024-03-18T22:25:50.610 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: ceph-osd-2:19.0.0-2206.gb6ae2491.el9.x86_64 134/198 2024-03-18T22:25:50.610 INFO:teuthology.orchestra.run.smithi049.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-18T22:25:50.610 INFO:teuthology.orchestra.run.smithi049.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-03-18T22:25:50.610 INFO:teuthology.orchestra.run.smithi049.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-18T22:25:50.610 INFO:teuthology.orchestra.run.smithi049.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-18T22:25:50.610 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-18T22:25:50.849 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-mds-2:19.0.0-2206.gb6ae2491.el9.x86_64 135/198 2024-03-18T22:25:50.874 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: ceph-mds-2:19.0.0-2206.gb6ae2491.el9.x86_64 135/198 2024-03-18T22:25:50.874 INFO:teuthology.orchestra.run.smithi049.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-18T22:25:50.874 INFO:teuthology.orchestra.run.smithi049.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-03-18T22:25:50.874 INFO:teuthology.orchestra.run.smithi049.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-18T22:25:50.874 INFO:teuthology.orchestra.run.smithi049.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-18T22:25:50.874 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-18T22:25:51.342 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-mon-2:19.0.0-2206.gb6ae2491.el9.x86_64 136/198 2024-03-18T22:25:51.366 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: ceph-mon-2:19.0.0-2206.gb6ae2491.el9.x86_64 136/198 2024-03-18T22:25:51.366 INFO:teuthology.orchestra.run.smithi049.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-18T22:25:51.367 INFO:teuthology.orchestra.run.smithi049.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-03-18T22:25:51.367 INFO:teuthology.orchestra.run.smithi049.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-18T22:25:51.367 INFO:teuthology.orchestra.run.smithi049.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-18T22:25:51.367 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-18T22:25:51.468 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : automake-1.16.2-8.el9.noarch 137/198 2024-03-18T22:25:51.854 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: cephadm-2:19.0.0-2206.gb6ae2491.el9.noarch 138/198 2024-03-18T22:25:51.892 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : cephadm-2:19.0.0-2206.gb6ae2491.el9.noarch 138/198 2024-03-18T22:25:51.925 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2206.gb6ae2491.e 139/198 2024-03-18T22:25:51.987 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2206.gb6ae2491. 140/198 2024-03-18T22:25:52.015 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2206.gb6ae2491.el9.noa 141/198 2024-03-18T22:25:52.261 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2206.gb6ae2491.el9.noa 141/198 2024-03-18T22:25:52.291 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2206.gb6ae2491.el9.n 142/198 2024-03-18T22:25:53.308 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2206.gb6ae2491.el9.n 142/198 2024-03-18T22:25:53.341 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2206.gb6a 143/198 2024-03-18T22:25:53.448 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2206.gb6a 143/198 2024-03-18T22:25:53.627 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2206.gb6ae2491.el 144/198 2024-03-18T22:25:53.668 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-mgr-2:19.0.0-2206.gb6ae2491.el9.x86_64 145/198 2024-03-18T22:25:53.693 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2206.gb6ae2491.el9.x86_64 145/198 2024-03-18T22:25:53.693 INFO:teuthology.orchestra.run.smithi049.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-18T22:25:53.693 INFO:teuthology.orchestra.run.smithi049.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-03-18T22:25:53.693 INFO:teuthology.orchestra.run.smithi049.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-18T22:25:53.694 INFO:teuthology.orchestra.run.smithi049.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-18T22:25:53.694 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-18T22:25:53.744 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-mgr-rook-2:19.0.0-2206.gb6ae2491.el9.noarch 146/198 2024-03-18T22:25:53.755 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2206.gb6ae2491.el9.noarch 146/198 2024-03-18T22:25:53.805 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-2:19.0.0-2206.gb6ae2491.el9.x86_64 147/198 2024-03-18T22:25:54.729 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : libtool-2.4.6-45.el9.x86_64 148/198 2024-03-18T22:25:54.762 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-radosgw-2:19.0.0-2206.gb6ae2491.el9.x86_64 149/198 2024-03-18T22:25:54.787 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2206.gb6ae2491.el9.x86_64 149/198 2024-03-18T22:25:54.787 INFO:teuthology.orchestra.run.smithi049.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-18T22:25:54.787 INFO:teuthology.orchestra.run.smithi049.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-03-18T22:25:54.787 INFO:teuthology.orchestra.run.smithi049.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-18T22:25:54.787 INFO:teuthology.orchestra.run.smithi049.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-18T22:25:54.787 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-18T22:25:54.830 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2206.gb6ae2 150/198 2024-03-18T22:25:54.854 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2206.gb6ae2 150/198 2024-03-18T22:25:54.854 INFO:teuthology.orchestra.run.smithi049.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-18T22:25:54.854 INFO:teuthology.orchestra.run.smithi049.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-03-18T22:25:54.854 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-18T22:25:54.914 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : cephfs-mirror-2:19.0.0-2206.gb6ae2491.el9.x86_64 151/198 2024-03-18T22:25:54.938 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2206.gb6ae2491.el9.x86_64 151/198 2024-03-18T22:25:54.938 INFO:teuthology.orchestra.run.smithi049.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-18T22:25:54.939 INFO:teuthology.orchestra.run.smithi049.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-03-18T22:25:54.939 INFO:teuthology.orchestra.run.smithi049.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-18T22:25:54.939 INFO:teuthology.orchestra.run.smithi049.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-18T22:25:54.939 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-18T22:25:55.256 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : rbd-mirror-2:19.0.0-2206.gb6ae2491.el9.x86_64 152/198 2024-03-18T22:25:55.281 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2206.gb6ae2491.el9.x86_64 152/198 2024-03-18T22:25:55.281 INFO:teuthology.orchestra.run.smithi049.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-18T22:25:55.281 INFO:teuthology.orchestra.run.smithi049.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-03-18T22:25:55.281 INFO:teuthology.orchestra.run.smithi049.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-18T22:25:55.281 INFO:teuthology.orchestra.run.smithi049.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-18T22:25:55.281 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-18T22:25:59.994 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-test-2:19.0.0-2206.gb6ae2491.el9.x86_64 153/198 2024-03-18T22:26:00.052 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : rbd-fuse-2:19.0.0-2206.gb6ae2491.el9.x86_64 154/198 2024-03-18T22:26:00.082 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : rbd-nbd-2:19.0.0-2206.gb6ae2491.el9.x86_64 155/198 2024-03-18T22:26:00.109 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 156/198 2024-03-18T22:26:00.311 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : cephfs-shell-2:19.0.0-2206.gb6ae2491.el9.x86_64 157/198 2024-03-18T22:26:00.438 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : postgresql-contrib-13.11-1.el9.x86_64 158/198 2024-03-18T22:26:00.484 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-fuse-2:19.0.0-2206.gb6ae2491.el9.x86_64 159/198 2024-03-18T22:26:00.535 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 160/198 2024-03-18T22:26:00.591 INFO:teuthology.orchestra.run.smithi049.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 161/198 2024-03-18T22:26:00.642 INFO:teuthology.orchestra.run.smithi049.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 162/198 2024-03-18T22:26:00.675 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : libcephfs-devel-2:19.0.0-2206.gb6ae2491.el9.x86_ 163/198 2024-03-18T22:26:00.707 INFO:teuthology.orchestra.run.smithi049.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 164/198 2024-03-18T22:26:00.737 INFO:teuthology.orchestra.run.smithi049.stdout: Upgrading : NetworkManager-team-1:1.47.2-1.el9.x86_64 165/198 2024-03-18T22:26:00.798 INFO:teuthology.orchestra.run.smithi049.stdout: Upgrading : NetworkManager-tui-1:1.47.2-1.el9.x86_64 166/198 2024-03-18T22:26:00.881 INFO:teuthology.orchestra.run.smithi049.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 167/198 2024-03-18T22:26:00.935 INFO:teuthology.orchestra.run.smithi049.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 168/198 2024-03-18T22:26:01.515 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-18T22:26:01.550 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-18T22:26:01.570 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-18T22:26:02.475 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : cephfs-top-2:19.0.0-2206.gb6ae2491.el9.noarch 170/198 2024-03-18T22:26:02.555 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 171/198 2024-03-18T22:26:02.597 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 172/198 2024-03-18T22:26:02.688 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : flex-2.6.4-9.el9.x86_64 173/198 2024-03-18T22:26:02.725 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : bison-3.7.4-5.el9.x86_64 174/198 2024-03-18T22:26:02.750 INFO:teuthology.orchestra.run.smithi049.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 175/198 2024-03-18T22:26:02.776 INFO:teuthology.orchestra.run.smithi049.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 176/198 2024-03-18T22:26:02.802 INFO:teuthology.orchestra.run.smithi049.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 177/198 2024-03-18T22:26:02.802 INFO:teuthology.orchestra.run.smithi049.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-18T22:26:02.890 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-18T22:26:02.919 INFO:teuthology.orchestra.run.smithi049.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 179/198 2024-03-18T22:26:02.919 INFO:teuthology.orchestra.run.smithi049.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-18T22:26:03.004 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-18T22:26:03.030 INFO:teuthology.orchestra.run.smithi049.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 181/198 2024-03-18T22:26:03.063 INFO:teuthology.orchestra.run.smithi049.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 182/198 2024-03-18T22:26:03.089 INFO:teuthology.orchestra.run.smithi049.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 183/198 2024-03-18T22:26:03.114 INFO:teuthology.orchestra.run.smithi049.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 184/198 2024-03-18T22:26:03.148 INFO:teuthology.orchestra.run.smithi049.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 185/198 2024-03-18T22:26:03.195 INFO:teuthology.orchestra.run.smithi049.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 186/198 2024-03-18T22:26:03.235 INFO:teuthology.orchestra.run.smithi049.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 187/198 2024-03-18T22:26:03.260 INFO:teuthology.orchestra.run.smithi049.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 188/198 2024-03-18T22:26:03.286 INFO:teuthology.orchestra.run.smithi049.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 189/198 2024-03-18T22:26:03.311 INFO:teuthology.orchestra.run.smithi049.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 190/198 2024-03-18T22:26:03.336 INFO:teuthology.orchestra.run.smithi049.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 191/198 2024-03-18T22:26:03.343 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-18T22:26:03.343 INFO:teuthology.orchestra.run.smithi049.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-18T22:26:03.614 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-18T22:26:03.614 INFO:teuthology.orchestra.run.smithi049.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-18T22:26:03.697 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-18T22:26:03.730 INFO:teuthology.orchestra.run.smithi049.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 194/198 2024-03-18T22:26:03.755 INFO:teuthology.orchestra.run.smithi049.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 195/198 2024-03-18T22:26:03.789 INFO:teuthology.orchestra.run.smithi049.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 196/198 2024-03-18T22:26:03.814 INFO:teuthology.orchestra.run.smithi049.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 197/198 2024-03-18T22:26:03.883 INFO:teuthology.orchestra.run.smithi049.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-18T22:26:08.667 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-18T22:26:08.667 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-2:19.0.0-2206.gb6ae2491.el9.x86_64 1/198 2024-03-18T22:26:08.667 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-base-2:19.0.0-2206.gb6ae2491.el9.x86_64 2/198 2024-03-18T22:26:08.667 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-common-2:19.0.0-2206.gb6ae2491.el9.x86_64 3/198 2024-03-18T22:26:08.667 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-fuse-2:19.0.0-2206.gb6ae2491.el9.x86_64 4/198 2024-03-18T22:26:08.667 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2206.gb6ae2 5/198 2024-03-18T22:26:08.667 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-mds-2:19.0.0-2206.gb6ae2491.el9.x86_64 6/198 2024-03-18T22:26:08.667 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-mgr-2:19.0.0-2206.gb6ae2491.el9.x86_64 7/198 2024-03-18T22:26:08.668 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-mon-2:19.0.0-2206.gb6ae2491.el9.x86_64 8/198 2024-03-18T22:26:08.668 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-osd-2:19.0.0-2206.gb6ae2491.el9.x86_64 9/198 2024-03-18T22:26:08.668 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-radosgw-2:19.0.0-2206.gb6ae2491.el9.x86_64 10/198 2024-03-18T22:26:08.668 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-selinux-2:19.0.0-2206.gb6ae2491.el9.x86_64 11/198 2024-03-18T22:26:08.668 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-test-2:19.0.0-2206.gb6ae2491.el9.x86_64 12/198 2024-03-18T22:26:08.668 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : cephfs-mirror-2:19.0.0-2206.gb6ae2491.el9.x86_64 13/198 2024-03-18T22:26:08.668 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : cephfs-shell-2:19.0.0-2206.gb6ae2491.el9.x86_64 14/198 2024-03-18T22:26:08.668 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libcephfs-devel-2:19.0.0-2206.gb6ae2491.el9.x86_ 15/198 2024-03-18T22:26:08.668 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libcephfs2-2:19.0.0-2206.gb6ae2491.el9.x86_64 16/198 2024-03-18T22:26:08.668 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libcephsqlite-2:19.0.0-2206.gb6ae2491.el9.x86_64 17/198 2024-03-18T22:26:08.668 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : librados-devel-2:19.0.0-2206.gb6ae2491.el9.x86_6 18/198 2024-03-18T22:26:08.668 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libradosstriper1-2:19.0.0-2206.gb6ae2491.el9.x86 19/198 2024-03-18T22:26:08.668 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : librgw2-2:19.0.0-2206.gb6ae2491.el9.x86_64 20/198 2024-03-18T22:26:08.668 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2206.gb6ae2491.el 21/198 2024-03-18T22:26:08.669 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-ceph-common-2:19.0.0-2206.gb6ae2491.el9. 22/198 2024-03-18T22:26:08.669 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-cephfs-2:19.0.0-2206.gb6ae2491.el9.x86_6 23/198 2024-03-18T22:26:08.669 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-rados-2:19.0.0-2206.gb6ae2491.el9.x86_64 24/198 2024-03-18T22:26:08.669 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-rbd-2:19.0.0-2206.gb6ae2491.el9.x86_64 25/198 2024-03-18T22:26:08.669 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-rgw-2:19.0.0-2206.gb6ae2491.el9.x86_64 26/198 2024-03-18T22:26:08.669 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : rbd-fuse-2:19.0.0-2206.gb6ae2491.el9.x86_64 27/198 2024-03-18T22:26:08.669 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : rbd-mirror-2:19.0.0-2206.gb6ae2491.el9.x86_64 28/198 2024-03-18T22:26:08.669 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : rbd-nbd-2:19.0.0-2206.gb6ae2491.el9.x86_64 29/198 2024-03-18T22:26:08.669 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2206.gb6ae2491. 30/198 2024-03-18T22:26:08.669 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2206.gb6ae2491.el9.noa 31/198 2024-03-18T22:26:08.669 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2206.gb6ae2491.el9.n 32/198 2024-03-18T22:26:08.669 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2206.gb6a 33/198 2024-03-18T22:26:08.669 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2206.gb6ae2491.el 34/198 2024-03-18T22:26:08.670 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2206.gb6ae2491.el9.noarch 35/198 2024-03-18T22:26:08.670 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2206.gb6ae2491.e 36/198 2024-03-18T22:26:08.670 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-volume-2:19.0.0-2206.gb6ae2491.el9.noarch 37/198 2024-03-18T22:26:08.670 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : cephadm-2:19.0.0-2206.gb6ae2491.el9.noarch 38/198 2024-03-18T22:26:08.670 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : cephfs-top-2:19.0.0-2206.gb6ae2491.el9.noarch 39/198 2024-03-18T22:26:08.670 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : automake-1.16.2-8.el9.noarch 40/198 2024-03-18T22:26:08.670 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : bison-3.7.4-5.el9.x86_64 41/198 2024-03-18T22:26:08.670 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 42/198 2024-03-18T22:26:08.670 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : flex-2.6.4-9.el9.x86_64 43/198 2024-03-18T22:26:08.670 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 44/198 2024-03-18T22:26:08.670 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 45/198 2024-03-18T22:26:08.670 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 46/198 2024-03-18T22:26:08.671 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 47/198 2024-03-18T22:26:08.671 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 48/198 2024-03-18T22:26:08.671 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 49/198 2024-03-18T22:26:08.671 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 50/198 2024-03-18T22:26:08.671 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 51/198 2024-03-18T22:26:08.671 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 52/198 2024-03-18T22:26:08.671 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 53/198 2024-03-18T22:26:08.671 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : lua-5.4.4-4.el9.x86_64 54/198 2024-03-18T22:26:08.671 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 55/198 2024-03-18T22:26:08.671 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 56/198 2024-03-18T22:26:08.671 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 57/198 2024-03-18T22:26:08.671 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : postgresql-13.11-1.el9.x86_64 58/198 2024-03-18T22:26:08.671 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : postgresql-contrib-13.11-1.el9.x86_64 59/198 2024-03-18T22:26:08.671 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : postgresql-private-libs-13.11-1.el9.x86_64 60/198 2024-03-18T22:26:08.672 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : postgresql-server-13.11-1.el9.x86_64 61/198 2024-03-18T22:26:08.672 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 62/198 2024-03-18T22:26:08.672 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 63/198 2024-03-18T22:26:08.672 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 64/198 2024-03-18T22:26:08.672 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 65/198 2024-03-18T22:26:08.672 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 66/198 2024-03-18T22:26:08.672 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 67/198 2024-03-18T22:26:08.672 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 68/198 2024-03-18T22:26:08.672 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 69/198 2024-03-18T22:26:08.672 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 70/198 2024-03-18T22:26:08.672 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 71/198 2024-03-18T22:26:08.672 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 72/198 2024-03-18T22:26:08.672 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 73/198 2024-03-18T22:26:08.672 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 74/198 2024-03-18T22:26:08.673 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 75/198 2024-03-18T22:26:08.673 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 76/198 2024-03-18T22:26:08.673 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 77/198 2024-03-18T22:26:08.673 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 78/198 2024-03-18T22:26:08.673 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 79/198 2024-03-18T22:26:08.673 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 80/198 2024-03-18T22:26:08.673 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 81/198 2024-03-18T22:26:08.673 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 82/198 2024-03-18T22:26:08.673 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 83/198 2024-03-18T22:26:08.673 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 84/198 2024-03-18T22:26:08.673 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 85/198 2024-03-18T22:26:08.673 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 86/198 2024-03-18T22:26:08.673 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 87/198 2024-03-18T22:26:08.674 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 88/198 2024-03-18T22:26:08.674 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-ply-3.11-14.el9.noarch 89/198 2024-03-18T22:26:08.674 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 90/198 2024-03-18T22:26:08.674 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 91/198 2024-03-18T22:26:08.674 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : jq-1.6-16.el9.x86_64 92/198 2024-03-18T22:26:08.674 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 93/198 2024-03-18T22:26:08.674 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 94/198 2024-03-18T22:26:08.674 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 95/198 2024-03-18T22:26:08.674 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 96/198 2024-03-18T22:26:08.674 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 97/198 2024-03-18T22:26:08.674 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 98/198 2024-03-18T22:26:08.674 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 99/198 2024-03-18T22:26:08.674 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 100/198 2024-03-18T22:26:08.675 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : qatlib-23.11.0-1.el9.x86_64 101/198 2024-03-18T22:26:08.675 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : qatlib-service-23.11.0-1.el9.x86_64 102/198 2024-03-18T22:26:08.675 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : qatzip-libs-1.1.2-2.el9.x86_64 103/198 2024-03-18T22:26:08.675 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 104/198 2024-03-18T22:26:08.675 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/198 2024-03-18T22:26:08.675 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/198 2024-03-18T22:26:08.675 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/198 2024-03-18T22:26:08.675 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/198 2024-03-18T22:26:08.675 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/198 2024-03-18T22:26:08.675 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/198 2024-03-18T22:26:08.675 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/198 2024-03-18T22:26:08.675 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/198 2024-03-18T22:26:08.675 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/198 2024-03-18T22:26:08.675 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/198 2024-03-18T22:26:08.676 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/198 2024-03-18T22:26:08.676 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-asyncssh-2.13.2-4.el9.noarch 116/198 2024-03-18T22:26:08.676 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/198 2024-03-18T22:26:08.676 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/198 2024-03-18T22:26:08.676 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/198 2024-03-18T22:26:08.676 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/198 2024-03-18T22:26:08.676 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/198 2024-03-18T22:26:08.676 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/198 2024-03-18T22:26:08.676 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/198 2024-03-18T22:26:08.676 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-google-auth-1:2.28.0-1.el9.noarch 124/198 2024-03-18T22:26:08.676 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/198 2024-03-18T22:26:08.676 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/198 2024-03-18T22:26:08.676 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/198 2024-03-18T22:26:08.676 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/198 2024-03-18T22:26:08.677 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/198 2024-03-18T22:26:08.677 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/198 2024-03-18T22:26:08.677 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/198 2024-03-18T22:26:08.677 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/198 2024-03-18T22:26:08.677 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/198 2024-03-18T22:26:08.677 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/198 2024-03-18T22:26:08.677 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/198 2024-03-18T22:26:08.677 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/198 2024-03-18T22:26:08.677 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/198 2024-03-18T22:26:08.677 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/198 2024-03-18T22:26:08.677 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/198 2024-03-18T22:26:08.677 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/198 2024-03-18T22:26:08.677 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/198 2024-03-18T22:26:08.677 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/198 2024-03-18T22:26:08.678 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/198 2024-03-18T22:26:08.678 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/198 2024-03-18T22:26:08.678 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/198 2024-03-18T22:26:08.678 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/198 2024-03-18T22:26:08.678 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/198 2024-03-18T22:26:08.678 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/198 2024-03-18T22:26:08.678 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/198 2024-03-18T22:26:08.678 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/198 2024-03-18T22:26:08.678 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : librados2-2:19.0.0-2206.gb6ae2491.el9.x86_64 151/198 2024-03-18T22:26:08.678 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/198 2024-03-18T22:26:08.678 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : librbd1-2:19.0.0-2206.gb6ae2491.el9.x86_64 153/198 2024-03-18T22:26:08.678 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/198 2024-03-18T22:26:08.678 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 155/198 2024-03-18T22:26:08.678 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 156/198 2024-03-18T22:26:08.679 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : NetworkManager-1:1.47.2-1.el9.x86_64 157/198 2024-03-18T22:26:08.679 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 158/198 2024-03-18T22:26:08.679 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : NetworkManager-libnm-1:1.47.2-1.el9.x86_64 159/198 2024-03-18T22:26:08.679 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 160/198 2024-03-18T22:26:08.679 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : NetworkManager-team-1:1.47.2-1.el9.x86_64 161/198 2024-03-18T22:26:08.679 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 162/198 2024-03-18T22:26:08.679 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : NetworkManager-tui-1:1.47.2-1.el9.x86_64 163/198 2024-03-18T22:26:08.679 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 164/198 2024-03-18T22:26:08.679 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : acl-2.3.1-4.el9.x86_64 165/198 2024-03-18T22:26:08.679 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : acl-2.3.1-3.el9.x86_64 166/198 2024-03-18T22:26:08.679 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 167/198 2024-03-18T22:26:08.679 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 168/198 2024-03-18T22:26:08.679 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 169/198 2024-03-18T22:26:08.679 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 170/198 2024-03-18T22:26:08.680 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 171/198 2024-03-18T22:26:08.680 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 172/198 2024-03-18T22:26:08.680 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 173/198 2024-03-18T22:26:08.680 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 174/198 2024-03-18T22:26:08.680 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 175/198 2024-03-18T22:26:08.680 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 176/198 2024-03-18T22:26:08.680 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 177/198 2024-03-18T22:26:08.680 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 178/198 2024-03-18T22:26:08.680 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 179/198 2024-03-18T22:26:08.680 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 180/198 2024-03-18T22:26:08.680 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 181/198 2024-03-18T22:26:08.680 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 182/198 2024-03-18T22:26:08.680 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 183/198 2024-03-18T22:26:08.680 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 184/198 2024-03-18T22:26:08.681 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 185/198 2024-03-18T22:26:08.681 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 186/198 2024-03-18T22:26:08.681 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 187/198 2024-03-18T22:26:08.681 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 188/198 2024-03-18T22:26:08.681 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 189/198 2024-03-18T22:26:08.681 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 190/198 2024-03-18T22:26:08.681 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 191/198 2024-03-18T22:26:08.681 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 192/198 2024-03-18T22:26:08.681 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 193/198 2024-03-18T22:26:08.681 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 194/198 2024-03-18T22:26:08.681 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 195/198 2024-03-18T22:26:08.681 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 196/198 2024-03-18T22:26:08.681 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 197/198 2024-03-18T22:26:11.463 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 198/198 2024-03-18T22:26:11.463 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-18T22:26:11.464 INFO:teuthology.orchestra.run.smithi049.stdout:Upgraded: 2024-03-18T22:26:11.464 INFO:teuthology.orchestra.run.smithi049.stdout: NetworkManager-1:1.47.2-1.el9.x86_64 2024-03-18T22:26:11.464 INFO:teuthology.orchestra.run.smithi049.stdout: NetworkManager-libnm-1:1.47.2-1.el9.x86_64 2024-03-18T22:26:11.464 INFO:teuthology.orchestra.run.smithi049.stdout: NetworkManager-team-1:1.47.2-1.el9.x86_64 2024-03-18T22:26:11.464 INFO:teuthology.orchestra.run.smithi049.stdout: NetworkManager-tui-1:1.47.2-1.el9.x86_64 2024-03-18T22:26:11.464 INFO:teuthology.orchestra.run.smithi049.stdout: acl-2.3.1-4.el9.x86_64 2024-03-18T22:26:11.464 INFO:teuthology.orchestra.run.smithi049.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-03-18T22:26:11.464 INFO:teuthology.orchestra.run.smithi049.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-03-18T22:26:11.464 INFO:teuthology.orchestra.run.smithi049.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-03-18T22:26:11.464 INFO:teuthology.orchestra.run.smithi049.stdout: libacl-2.3.1-4.el9.x86_64 2024-03-18T22:26:11.464 INFO:teuthology.orchestra.run.smithi049.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-03-18T22:26:11.464 INFO:teuthology.orchestra.run.smithi049.stdout: libblkid-2.37.4-18.el9.x86_64 2024-03-18T22:26:11.464 INFO:teuthology.orchestra.run.smithi049.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-03-18T22:26:11.464 INFO:teuthology.orchestra.run.smithi049.stdout: libmount-2.37.4-18.el9.x86_64 2024-03-18T22:26:11.465 INFO:teuthology.orchestra.run.smithi049.stdout: librados2-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-18T22:26:11.465 INFO:teuthology.orchestra.run.smithi049.stdout: librbd1-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-18T22:26:11.465 INFO:teuthology.orchestra.run.smithi049.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-03-18T22:26:11.465 INFO:teuthology.orchestra.run.smithi049.stdout: libuuid-2.37.4-18.el9.x86_64 2024-03-18T22:26:11.465 INFO:teuthology.orchestra.run.smithi049.stdout: openssl-1:3.0.7-27.el9.x86_64 2024-03-18T22:26:11.465 INFO:teuthology.orchestra.run.smithi049.stdout: openssl-libs-1:3.0.7-27.el9.x86_64 2024-03-18T22:26:11.465 INFO:teuthology.orchestra.run.smithi049.stdout: util-linux-2.37.4-18.el9.x86_64 2024-03-18T22:26:11.465 INFO:teuthology.orchestra.run.smithi049.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-03-18T22:26:11.465 INFO:teuthology.orchestra.run.smithi049.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-03-18T22:26:11.465 INFO:teuthology.orchestra.run.smithi049.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-03-18T22:26:11.465 INFO:teuthology.orchestra.run.smithi049.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-03-18T22:26:11.465 INFO:teuthology.orchestra.run.smithi049.stdout:Installed: 2024-03-18T22:26:11.465 INFO:teuthology.orchestra.run.smithi049.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-03-18T22:26:11.465 INFO:teuthology.orchestra.run.smithi049.stdout: automake-1.16.2-8.el9.noarch 2024-03-18T22:26:11.466 INFO:teuthology.orchestra.run.smithi049.stdout: bison-3.7.4-5.el9.x86_64 2024-03-18T22:26:11.466 INFO:teuthology.orchestra.run.smithi049.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-03-18T22:26:11.466 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-18T22:26:11.466 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-base-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-18T22:26:11.466 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-common-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-18T22:26:11.466 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-fuse-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-18T22:26:11.466 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-grafana-dashboards-2:19.0.0-2206.gb6ae2491.el9.noarch 2024-03-18T22:26:11.466 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-immutable-object-cache-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-18T22:26:11.466 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mds-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-18T22:26:11.466 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mgr-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-18T22:26:11.466 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mgr-cephadm-2:19.0.0-2206.gb6ae2491.el9.noarch 2024-03-18T22:26:11.466 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mgr-dashboard-2:19.0.0-2206.gb6ae2491.el9.noarch 2024-03-18T22:26:11.466 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2206.gb6ae2491.el9.noarch 2024-03-18T22:26:11.467 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mgr-modules-core-2:19.0.0-2206.gb6ae2491.el9.noarch 2024-03-18T22:26:11.467 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mgr-rook-2:19.0.0-2206.gb6ae2491.el9.noarch 2024-03-18T22:26:11.467 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mon-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-18T22:26:11.467 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-osd-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-18T22:26:11.467 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-prometheus-alerts-2:19.0.0-2206.gb6ae2491.el9.noarch 2024-03-18T22:26:11.467 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-radosgw-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-18T22:26:11.467 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-selinux-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-18T22:26:11.467 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-test-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-18T22:26:11.467 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-volume-2:19.0.0-2206.gb6ae2491.el9.noarch 2024-03-18T22:26:11.467 INFO:teuthology.orchestra.run.smithi049.stdout: cephadm-2:19.0.0-2206.gb6ae2491.el9.noarch 2024-03-18T22:26:11.467 INFO:teuthology.orchestra.run.smithi049.stdout: cephfs-mirror-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-18T22:26:11.467 INFO:teuthology.orchestra.run.smithi049.stdout: cephfs-shell-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-18T22:26:11.467 INFO:teuthology.orchestra.run.smithi049.stdout: cephfs-top-2:19.0.0-2206.gb6ae2491.el9.noarch 2024-03-18T22:26:11.468 INFO:teuthology.orchestra.run.smithi049.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-03-18T22:26:11.468 INFO:teuthology.orchestra.run.smithi049.stdout: flex-2.6.4-9.el9.x86_64 2024-03-18T22:26:11.468 INFO:teuthology.orchestra.run.smithi049.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-03-18T22:26:11.468 INFO:teuthology.orchestra.run.smithi049.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-03-18T22:26:11.468 INFO:teuthology.orchestra.run.smithi049.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-03-18T22:26:11.468 INFO:teuthology.orchestra.run.smithi049.stdout: fuse-2.9.9-15.el9.x86_64 2024-03-18T22:26:11.468 INFO:teuthology.orchestra.run.smithi049.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-03-18T22:26:11.468 INFO:teuthology.orchestra.run.smithi049.stdout: grpc-data-1.46.7-10.el9.noarch 2024-03-18T22:26:11.468 INFO:teuthology.orchestra.run.smithi049.stdout: jq-1.6-16.el9.x86_64 2024-03-18T22:26:11.468 INFO:teuthology.orchestra.run.smithi049.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-03-18T22:26:11.468 INFO:teuthology.orchestra.run.smithi049.stdout: libarrow-9.0.0-10.el9.x86_64 2024-03-18T22:26:11.468 INFO:teuthology.orchestra.run.smithi049.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-03-18T22:26:11.468 INFO:teuthology.orchestra.run.smithi049.stdout: libcephfs-devel-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-18T22:26:11.468 INFO:teuthology.orchestra.run.smithi049.stdout: libcephfs2-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-18T22:26:11.469 INFO:teuthology.orchestra.run.smithi049.stdout: libcephsqlite-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-18T22:26:11.469 INFO:teuthology.orchestra.run.smithi049.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-03-18T22:26:11.469 INFO:teuthology.orchestra.run.smithi049.stdout: liboath-2.6.7-2.el9.x86_64 2024-03-18T22:26:11.469 INFO:teuthology.orchestra.run.smithi049.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-03-18T22:26:11.469 INFO:teuthology.orchestra.run.smithi049.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-03-18T22:26:11.469 INFO:teuthology.orchestra.run.smithi049.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-03-18T22:26:11.469 INFO:teuthology.orchestra.run.smithi049.stdout: librados-devel-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-18T22:26:11.469 INFO:teuthology.orchestra.run.smithi049.stdout: libradosstriper1-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-18T22:26:11.469 INFO:teuthology.orchestra.run.smithi049.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-03-18T22:26:11.469 INFO:teuthology.orchestra.run.smithi049.stdout: librgw2-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-18T22:26:11.469 INFO:teuthology.orchestra.run.smithi049.stdout: libtool-2.4.6-45.el9.x86_64 2024-03-18T22:26:11.469 INFO:teuthology.orchestra.run.smithi049.stdout: libunwind-1.6.2-1.el9.x86_64 2024-03-18T22:26:11.469 INFO:teuthology.orchestra.run.smithi049.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-03-18T22:26:11.469 INFO:teuthology.orchestra.run.smithi049.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-03-18T22:26:11.470 INFO:teuthology.orchestra.run.smithi049.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-03-18T22:26:11.470 INFO:teuthology.orchestra.run.smithi049.stdout: lua-5.4.4-4.el9.x86_64 2024-03-18T22:26:11.470 INFO:teuthology.orchestra.run.smithi049.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-03-18T22:26:11.470 INFO:teuthology.orchestra.run.smithi049.stdout: luarocks-3.9.2-1.el9.noarch 2024-03-18T22:26:11.470 INFO:teuthology.orchestra.run.smithi049.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-03-18T22:26:11.470 INFO:teuthology.orchestra.run.smithi049.stdout: openblas-0.3.21-2.el9.x86_64 2024-03-18T22:26:11.470 INFO:teuthology.orchestra.run.smithi049.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-03-18T22:26:11.470 INFO:teuthology.orchestra.run.smithi049.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-03-18T22:26:11.470 INFO:teuthology.orchestra.run.smithi049.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-03-18T22:26:11.470 INFO:teuthology.orchestra.run.smithi049.stdout: postgresql-13.11-1.el9.x86_64 2024-03-18T22:26:11.470 INFO:teuthology.orchestra.run.smithi049.stdout: postgresql-contrib-13.11-1.el9.x86_64 2024-03-18T22:26:11.470 INFO:teuthology.orchestra.run.smithi049.stdout: postgresql-private-libs-13.11-1.el9.x86_64 2024-03-18T22:26:11.470 INFO:teuthology.orchestra.run.smithi049.stdout: postgresql-server-13.11-1.el9.x86_64 2024-03-18T22:26:11.471 INFO:teuthology.orchestra.run.smithi049.stdout: protobuf-3.14.0-13.el9.x86_64 2024-03-18T22:26:11.471 INFO:teuthology.orchestra.run.smithi049.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-03-18T22:26:11.471 INFO:teuthology.orchestra.run.smithi049.stdout: python3-asyncssh-2.13.2-4.el9.noarch 2024-03-18T22:26:11.471 INFO:teuthology.orchestra.run.smithi049.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-03-18T22:26:11.471 INFO:teuthology.orchestra.run.smithi049.stdout: python3-babel-2.9.1-2.el9.noarch 2024-03-18T22:26:11.471 INFO:teuthology.orchestra.run.smithi049.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-03-18T22:26:11.471 INFO:teuthology.orchestra.run.smithi049.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-03-18T22:26:11.471 INFO:teuthology.orchestra.run.smithi049.stdout: python3-ceph-argparse-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-18T22:26:11.471 INFO:teuthology.orchestra.run.smithi049.stdout: python3-ceph-common-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-18T22:26:11.471 INFO:teuthology.orchestra.run.smithi049.stdout: python3-cephfs-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-18T22:26:11.471 INFO:teuthology.orchestra.run.smithi049.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-03-18T22:26:11.471 INFO:teuthology.orchestra.run.smithi049.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-03-18T22:26:11.471 INFO:teuthology.orchestra.run.smithi049.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-03-18T22:26:11.471 INFO:teuthology.orchestra.run.smithi049.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-03-18T22:26:11.472 INFO:teuthology.orchestra.run.smithi049.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-03-18T22:26:11.472 INFO:teuthology.orchestra.run.smithi049.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-03-18T22:26:11.472 INFO:teuthology.orchestra.run.smithi049.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-03-18T22:26:11.472 INFO:teuthology.orchestra.run.smithi049.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-03-18T22:26:11.472 INFO:teuthology.orchestra.run.smithi049.stdout: python3-google-auth-1:2.28.0-1.el9.noarch 2024-03-18T22:26:11.472 INFO:teuthology.orchestra.run.smithi049.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-03-18T22:26:11.472 INFO:teuthology.orchestra.run.smithi049.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-03-18T22:26:11.472 INFO:teuthology.orchestra.run.smithi049.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-03-18T22:26:11.472 INFO:teuthology.orchestra.run.smithi049.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-03-18T22:26:11.472 INFO:teuthology.orchestra.run.smithi049.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-03-18T22:26:11.472 INFO:teuthology.orchestra.run.smithi049.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-03-18T22:26:11.472 INFO:teuthology.orchestra.run.smithi049.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-03-18T22:26:11.472 INFO:teuthology.orchestra.run.smithi049.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-03-18T22:26:11.472 INFO:teuthology.orchestra.run.smithi049.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-03-18T22:26:11.473 INFO:teuthology.orchestra.run.smithi049.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-03-18T22:26:11.473 INFO:teuthology.orchestra.run.smithi049.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-03-18T22:26:11.473 INFO:teuthology.orchestra.run.smithi049.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-03-18T22:26:11.473 INFO:teuthology.orchestra.run.smithi049.stdout: python3-mako-1.1.4-6.el9.noarch 2024-03-18T22:26:11.473 INFO:teuthology.orchestra.run.smithi049.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-03-18T22:26:11.473 INFO:teuthology.orchestra.run.smithi049.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-03-18T22:26:11.473 INFO:teuthology.orchestra.run.smithi049.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-03-18T22:26:11.473 INFO:teuthology.orchestra.run.smithi049.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-03-18T22:26:11.473 INFO:teuthology.orchestra.run.smithi049.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-03-18T22:26:11.473 INFO:teuthology.orchestra.run.smithi049.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-03-18T22:26:11.473 INFO:teuthology.orchestra.run.smithi049.stdout: python3-packaging-20.9-5.el9.noarch 2024-03-18T22:26:11.473 INFO:teuthology.orchestra.run.smithi049.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-03-18T22:26:11.473 INFO:teuthology.orchestra.run.smithi049.stdout: python3-ply-3.11-14.el9.noarch 2024-03-18T22:26:11.474 INFO:teuthology.orchestra.run.smithi049.stdout: python3-portend-3.1.0-2.el9.noarch 2024-03-18T22:26:11.474 INFO:teuthology.orchestra.run.smithi049.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-03-18T22:26:11.474 INFO:teuthology.orchestra.run.smithi049.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-03-18T22:26:11.474 INFO:teuthology.orchestra.run.smithi049.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-03-18T22:26:11.474 INFO:teuthology.orchestra.run.smithi049.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-03-18T22:26:11.474 INFO:teuthology.orchestra.run.smithi049.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-03-18T22:26:11.474 INFO:teuthology.orchestra.run.smithi049.stdout: python3-pycparser-2.20-6.el9.noarch 2024-03-18T22:26:11.474 INFO:teuthology.orchestra.run.smithi049.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-03-18T22:26:11.474 INFO:teuthology.orchestra.run.smithi049.stdout: python3-pytz-2021.1-5.el9.noarch 2024-03-18T22:26:11.474 INFO:teuthology.orchestra.run.smithi049.stdout: python3-rados-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-18T22:26:11.474 INFO:teuthology.orchestra.run.smithi049.stdout: python3-rbd-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-18T22:26:11.474 INFO:teuthology.orchestra.run.smithi049.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-03-18T22:26:11.474 INFO:teuthology.orchestra.run.smithi049.stdout: python3-requests-2.25.1-8.el9.noarch 2024-03-18T22:26:11.474 INFO:teuthology.orchestra.run.smithi049.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-03-18T22:26:11.475 INFO:teuthology.orchestra.run.smithi049.stdout: python3-rgw-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-18T22:26:11.475 INFO:teuthology.orchestra.run.smithi049.stdout: python3-routes-2.5.1-5.el9.noarch 2024-03-18T22:26:11.475 INFO:teuthology.orchestra.run.smithi049.stdout: python3-rsa-4.9-2.el9.noarch 2024-03-18T22:26:11.475 INFO:teuthology.orchestra.run.smithi049.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-03-18T22:26:11.475 INFO:teuthology.orchestra.run.smithi049.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-03-18T22:26:11.475 INFO:teuthology.orchestra.run.smithi049.stdout: python3-toml-0.10.2-6.el9.noarch 2024-03-18T22:26:11.475 INFO:teuthology.orchestra.run.smithi049.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-03-18T22:26:11.475 INFO:teuthology.orchestra.run.smithi049.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-03-18T22:26:11.475 INFO:teuthology.orchestra.run.smithi049.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-03-18T22:26:11.475 INFO:teuthology.orchestra.run.smithi049.stdout: python3-webob-1.8.7-6.el9.noarch 2024-03-18T22:26:11.475 INFO:teuthology.orchestra.run.smithi049.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-03-18T22:26:11.475 INFO:teuthology.orchestra.run.smithi049.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-03-18T22:26:11.475 INFO:teuthology.orchestra.run.smithi049.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-03-18T22:26:11.475 INFO:teuthology.orchestra.run.smithi049.stdout: qatlib-23.11.0-1.el9.x86_64 2024-03-18T22:26:11.475 INFO:teuthology.orchestra.run.smithi049.stdout: qatlib-service-23.11.0-1.el9.x86_64 2024-03-18T22:26:11.476 INFO:teuthology.orchestra.run.smithi049.stdout: qatzip-libs-1.1.2-2.el9.x86_64 2024-03-18T22:26:11.476 INFO:teuthology.orchestra.run.smithi049.stdout: rbd-fuse-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-18T22:26:11.476 INFO:teuthology.orchestra.run.smithi049.stdout: rbd-mirror-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-18T22:26:11.476 INFO:teuthology.orchestra.run.smithi049.stdout: rbd-nbd-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-18T22:26:11.476 INFO:teuthology.orchestra.run.smithi049.stdout: re2-1:20211101-3.el9.x86_64 2024-03-18T22:26:11.476 INFO:teuthology.orchestra.run.smithi049.stdout: socat-1.7.4.1-5.el9.x86_64 2024-03-18T22:26:11.476 INFO:teuthology.orchestra.run.smithi049.stdout: thrift-0.15.0-2.el9.x86_64 2024-03-18T22:26:11.476 INFO:teuthology.orchestra.run.smithi049.stdout: uuid-1.6.2-55.el9.x86_64 2024-03-18T22:26:11.476 INFO:teuthology.orchestra.run.smithi049.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-03-18T22:26:11.476 INFO:teuthology.orchestra.run.smithi049.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-03-18T22:26:11.477 INFO:teuthology.orchestra.run.smithi049.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-03-18T22:26:11.477 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-18T22:26:11.477 INFO:teuthology.orchestra.run.smithi049.stdout:Complete! 2024-03-18T22:26:11.952 DEBUG:teuthology.parallel:result is None 2024-03-18T22:37:37.675 INFO:teuthology.orchestra.run.smithi175.stdout:CentOS Stream 9 - AppStream 25 kB/s | 19 MB 13:04 2024-03-18T22:37:48.608 INFO:teuthology.orchestra.run.smithi139.stdout:CentOS Stream 9 - CRB 8.0 kB/s | 6.1 MB 13:07 2024-03-18T22:37:50.375 INFO:teuthology.orchestra.run.smithi139.stdout:CentOS Stream 9 - Extras packages 26 kB/s | 16 kB 00:00 2024-03-18T22:38:10.676 INFO:teuthology.orchestra.run.smithi139.stdout:Extra Packages for Enterprise Linux 1.0 MB/s | 21 MB 00:20 2024-03-18T22:38:15.665 INFO:teuthology.orchestra.run.smithi139.stdout:lab-extras 13 kB/s | 1.7 kB 00:00 2024-03-18T22:38:17.995 INFO:teuthology.orchestra.run.smithi139.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-18T22:38:17.996 INFO:teuthology.orchestra.run.smithi139.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-18T22:38:18.010 INFO:teuthology.orchestra.run.smithi139.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-03-18T22:38:18.011 INFO:teuthology.orchestra.run.smithi139.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-03-18T22:38:18.012 INFO:teuthology.orchestra.run.smithi139.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-03-18T22:38:18.014 INFO:teuthology.orchestra.run.smithi139.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-03-18T22:38:18.016 INFO:teuthology.orchestra.run.smithi139.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-03-18T22:38:18.018 INFO:teuthology.orchestra.run.smithi139.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-03-18T22:38:18.019 INFO:teuthology.orchestra.run.smithi139.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-03-18T22:38:18.102 INFO:teuthology.orchestra.run.smithi139.stdout:Dependencies resolved. 2024-03-18T22:38:18.126 INFO:teuthology.orchestra.run.smithi139.stdout:============================================================================================ 2024-03-18T22:38:18.126 INFO:teuthology.orchestra.run.smithi139.stdout: Package Arch Version Repository Size 2024-03-18T22:38:18.126 INFO:teuthology.orchestra.run.smithi139.stdout:============================================================================================ 2024-03-18T22:38:18.126 INFO:teuthology.orchestra.run.smithi139.stdout:Installing: 2024-03-18T22:38:18.126 INFO:teuthology.orchestra.run.smithi139.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-03-18T22:38:18.126 INFO:teuthology.orchestra.run.smithi139.stdout: ceph x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 6.6 k 2024-03-18T22:38:18.126 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-base x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 5.4 M 2024-03-18T22:38:18.126 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-fuse x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 891 k 2024-03-18T22:38:18.126 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 145 k 2024-03-18T22:38:18.126 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-mgr x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 1.5 M 2024-03-18T22:38:18.127 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2206.gb6ae2491.el9 ceph-noarch 139 k 2024-03-18T22:38:18.127 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2206.gb6ae2491.el9 ceph-noarch 1.8 M 2024-03-18T22:38:18.127 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2206.gb6ae2491.el9 ceph-noarch 7.4 M 2024-03-18T22:38:18.127 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-mgr-rook noarch 2:19.0.0-2206.gb6ae2491.el9 ceph-noarch 49 k 2024-03-18T22:38:18.127 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-radosgw x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 9.8 M 2024-03-18T22:38:18.127 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-test x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 48 M 2024-03-18T22:38:18.127 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-volume noarch 2:19.0.0-2206.gb6ae2491.el9 ceph-noarch 264 k 2024-03-18T22:38:18.127 INFO:teuthology.orchestra.run.smithi139.stdout: cephadm noarch 2:19.0.0-2206.gb6ae2491.el9 ceph-noarch 749 k 2024-03-18T22:38:18.127 INFO:teuthology.orchestra.run.smithi139.stdout: cephfs-mirror x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 221 k 2024-03-18T22:38:18.127 INFO:teuthology.orchestra.run.smithi139.stdout: cephfs-shell x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 27 k 2024-03-18T22:38:18.128 INFO:teuthology.orchestra.run.smithi139.stdout: cephfs-top noarch 2:19.0.0-2206.gb6ae2491.el9 ceph-noarch 18 k 2024-03-18T22:38:18.128 INFO:teuthology.orchestra.run.smithi139.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-03-18T22:38:18.128 INFO:teuthology.orchestra.run.smithi139.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-03-18T22:38:18.128 INFO:teuthology.orchestra.run.smithi139.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-03-18T22:38:18.128 INFO:teuthology.orchestra.run.smithi139.stdout: libcephfs-devel x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 32 k 2024-03-18T22:38:18.128 INFO:teuthology.orchestra.run.smithi139.stdout: libcephfs2 x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 746 k 2024-03-18T22:38:18.128 INFO:teuthology.orchestra.run.smithi139.stdout: librados-devel x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 127 k 2024-03-18T22:38:18.128 INFO:teuthology.orchestra.run.smithi139.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-03-18T22:38:18.128 INFO:teuthology.orchestra.run.smithi139.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-03-18T22:38:18.128 INFO:teuthology.orchestra.run.smithi139.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-03-18T22:38:18.128 INFO:teuthology.orchestra.run.smithi139.stdout: postgresql x86_64 13.11-1.el9 CentOS-AppStream 1.5 M 2024-03-18T22:38:18.129 INFO:teuthology.orchestra.run.smithi139.stdout: postgresql-contrib x86_64 13.11-1.el9 CentOS-AppStream 811 k 2024-03-18T22:38:18.129 INFO:teuthology.orchestra.run.smithi139.stdout: postgresql-server x86_64 13.11-1.el9 CentOS-AppStream 5.7 M 2024-03-18T22:38:18.129 INFO:teuthology.orchestra.run.smithi139.stdout: python3-cephfs x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 162 k 2024-03-18T22:38:18.129 INFO:teuthology.orchestra.run.smithi139.stdout: python3-rados x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 321 k 2024-03-18T22:38:18.129 INFO:teuthology.orchestra.run.smithi139.stdout: python3-rbd x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 298 k 2024-03-18T22:38:18.129 INFO:teuthology.orchestra.run.smithi139.stdout: python3-rgw x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 99 k 2024-03-18T22:38:18.129 INFO:teuthology.orchestra.run.smithi139.stdout: rbd-fuse x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 85 k 2024-03-18T22:38:18.129 INFO:teuthology.orchestra.run.smithi139.stdout: rbd-mirror x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 3.1 M 2024-03-18T22:38:18.129 INFO:teuthology.orchestra.run.smithi139.stdout: rbd-nbd x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 170 k 2024-03-18T22:38:18.129 INFO:teuthology.orchestra.run.smithi139.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-03-18T22:38:18.129 INFO:teuthology.orchestra.run.smithi139.stdout:Upgrading: 2024-03-18T22:38:18.130 INFO:teuthology.orchestra.run.smithi139.stdout: NetworkManager x86_64 1:1.47.2-1.el9 baseos 2.3 M 2024-03-18T22:38:18.130 INFO:teuthology.orchestra.run.smithi139.stdout: NetworkManager-libnm x86_64 1:1.47.2-1.el9 baseos 1.8 M 2024-03-18T22:38:18.130 INFO:teuthology.orchestra.run.smithi139.stdout: NetworkManager-team x86_64 1:1.47.2-1.el9 baseos 42 k 2024-03-18T22:38:18.130 INFO:teuthology.orchestra.run.smithi139.stdout: NetworkManager-tui x86_64 1:1.47.2-1.el9 baseos 248 k 2024-03-18T22:38:18.130 INFO:teuthology.orchestra.run.smithi139.stdout: acl x86_64 2.3.1-4.el9 baseos 71 k 2024-03-18T22:38:18.130 INFO:teuthology.orchestra.run.smithi139.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 baseos 36 k 2024-03-18T22:38:18.130 INFO:teuthology.orchestra.run.smithi139.stdout: elfutils-libelf x86_64 0.190-2.el9 baseos 192 k 2024-03-18T22:38:18.130 INFO:teuthology.orchestra.run.smithi139.stdout: elfutils-libs x86_64 0.190-2.el9 baseos 256 k 2024-03-18T22:38:18.130 INFO:teuthology.orchestra.run.smithi139.stdout: libacl x86_64 2.3.1-4.el9 baseos 23 k 2024-03-18T22:38:18.130 INFO:teuthology.orchestra.run.smithi139.stdout: libacl-devel x86_64 2.3.1-4.el9 appstream 74 k 2024-03-18T22:38:18.130 INFO:teuthology.orchestra.run.smithi139.stdout: libblkid x86_64 2.37.4-18.el9 baseos 108 k 2024-03-18T22:38:18.130 INFO:teuthology.orchestra.run.smithi139.stdout: libfdisk x86_64 2.37.4-18.el9 baseos 155 k 2024-03-18T22:38:18.131 INFO:teuthology.orchestra.run.smithi139.stdout: libmount x86_64 2.37.4-18.el9 baseos 136 k 2024-03-18T22:38:18.131 INFO:teuthology.orchestra.run.smithi139.stdout: librados2 x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 3.4 M 2024-03-18T22:38:18.131 INFO:teuthology.orchestra.run.smithi139.stdout: librbd1 x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 3.1 M 2024-03-18T22:38:18.131 INFO:teuthology.orchestra.run.smithi139.stdout: libsmartcols x86_64 2.37.4-18.el9 baseos 64 k 2024-03-18T22:38:18.131 INFO:teuthology.orchestra.run.smithi139.stdout: libuuid x86_64 2.37.4-18.el9 baseos 29 k 2024-03-18T22:38:18.131 INFO:teuthology.orchestra.run.smithi139.stdout: openssl x86_64 1:3.0.7-27.el9 baseos 1.2 M 2024-03-18T22:38:18.131 INFO:teuthology.orchestra.run.smithi139.stdout: openssl-libs x86_64 1:3.0.7-27.el9 baseos 2.2 M 2024-03-18T22:38:18.131 INFO:teuthology.orchestra.run.smithi139.stdout: util-linux x86_64 2.37.4-18.el9 baseos 2.3 M 2024-03-18T22:38:18.131 INFO:teuthology.orchestra.run.smithi139.stdout: util-linux-core x86_64 2.37.4-18.el9 baseos 465 k 2024-03-18T22:38:18.131 INFO:teuthology.orchestra.run.smithi139.stdout: util-linux-user x86_64 2.37.4-18.el9 baseos 31 k 2024-03-18T22:38:18.131 INFO:teuthology.orchestra.run.smithi139.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-03-18T22:38:18.131 INFO:teuthology.orchestra.run.smithi139.stdout: xfsprogs x86_64 6.3.0-1.el9 baseos 1.1 M 2024-03-18T22:38:18.131 INFO:teuthology.orchestra.run.smithi139.stdout:Installing dependencies: 2024-03-18T22:38:18.131 INFO:teuthology.orchestra.run.smithi139.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-03-18T22:38:18.132 INFO:teuthology.orchestra.run.smithi139.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-03-18T22:38:18.132 INFO:teuthology.orchestra.run.smithi139.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-03-18T22:38:18.132 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-common x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 21 M 2024-03-18T22:38:18.132 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2206.gb6ae2491.el9 ceph-noarch 34 k 2024-03-18T22:38:18.132 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-mds x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 2.4 M 2024-03-18T22:38:18.132 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2206.gb6ae2491.el9 ceph-noarch 248 k 2024-03-18T22:38:18.132 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-mon x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 4.7 M 2024-03-18T22:38:18.132 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-osd x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 17 M 2024-03-18T22:38:18.132 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2206.gb6ae2491.el9 ceph-noarch 16 k 2024-03-18T22:38:18.132 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-selinux x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 25 k 2024-03-18T22:38:18.132 INFO:teuthology.orchestra.run.smithi139.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-03-18T22:38:18.132 INFO:teuthology.orchestra.run.smithi139.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-03-18T22:38:18.133 INFO:teuthology.orchestra.run.smithi139.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-03-18T22:38:18.133 INFO:teuthology.orchestra.run.smithi139.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-03-18T22:38:18.133 INFO:teuthology.orchestra.run.smithi139.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-03-18T22:38:18.133 INFO:teuthology.orchestra.run.smithi139.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-03-18T22:38:18.133 INFO:teuthology.orchestra.run.smithi139.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-03-18T22:38:18.133 INFO:teuthology.orchestra.run.smithi139.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-03-18T22:38:18.133 INFO:teuthology.orchestra.run.smithi139.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-03-18T22:38:18.133 INFO:teuthology.orchestra.run.smithi139.stdout: libcephsqlite x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 163 k 2024-03-18T22:38:18.133 INFO:teuthology.orchestra.run.smithi139.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-03-18T22:38:18.133 INFO:teuthology.orchestra.run.smithi139.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-03-18T22:38:18.133 INFO:teuthology.orchestra.run.smithi139.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-03-18T22:38:18.133 INFO:teuthology.orchestra.run.smithi139.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-03-18T22:38:18.133 INFO:teuthology.orchestra.run.smithi139.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-03-18T22:38:18.133 INFO:teuthology.orchestra.run.smithi139.stdout: libradosstriper1 x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 501 k 2024-03-18T22:38:18.133 INFO:teuthology.orchestra.run.smithi139.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-03-18T22:38:18.134 INFO:teuthology.orchestra.run.smithi139.stdout: librgw2 x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 5.1 M 2024-03-18T22:38:18.134 INFO:teuthology.orchestra.run.smithi139.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-03-18T22:38:18.134 INFO:teuthology.orchestra.run.smithi139.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-03-18T22:38:18.134 INFO:teuthology.orchestra.run.smithi139.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-03-18T22:38:18.134 INFO:teuthology.orchestra.run.smithi139.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-03-18T22:38:18.134 INFO:teuthology.orchestra.run.smithi139.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-03-18T22:38:18.134 INFO:teuthology.orchestra.run.smithi139.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-03-18T22:38:18.134 INFO:teuthology.orchestra.run.smithi139.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-03-18T22:38:18.134 INFO:teuthology.orchestra.run.smithi139.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-03-18T22:38:18.134 INFO:teuthology.orchestra.run.smithi139.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-03-18T22:38:18.134 INFO:teuthology.orchestra.run.smithi139.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-03-18T22:38:18.134 INFO:teuthology.orchestra.run.smithi139.stdout: postgresql-private-libs x86_64 13.11-1.el9 CentOS-AppStream 134 k 2024-03-18T22:38:18.134 INFO:teuthology.orchestra.run.smithi139.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-03-18T22:38:18.135 INFO:teuthology.orchestra.run.smithi139.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-03-18T22:38:18.135 INFO:teuthology.orchestra.run.smithi139.stdout: python3-asyncssh noarch 2.13.2-4.el9 epel 548 k 2024-03-18T22:38:18.135 INFO:teuthology.orchestra.run.smithi139.stdout: python3-attrs noarch 20.3.0-7.el9 CentOS-AppStream 78 k 2024-03-18T22:38:18.135 INFO:teuthology.orchestra.run.smithi139.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-03-18T22:38:18.135 INFO:teuthology.orchestra.run.smithi139.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-03-18T22:38:18.135 INFO:teuthology.orchestra.run.smithi139.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-03-18T22:38:18.135 INFO:teuthology.orchestra.run.smithi139.stdout: python3-ceph-argparse x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 45 k 2024-03-18T22:38:18.135 INFO:teuthology.orchestra.run.smithi139.stdout: python3-ceph-common x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 131 k 2024-03-18T22:38:18.135 INFO:teuthology.orchestra.run.smithi139.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-03-18T22:38:18.135 INFO:teuthology.orchestra.run.smithi139.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-03-18T22:38:18.135 INFO:teuthology.orchestra.run.smithi139.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-03-18T22:38:18.135 INFO:teuthology.orchestra.run.smithi139.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-03-18T22:38:18.135 INFO:teuthology.orchestra.run.smithi139.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-03-18T22:38:18.135 INFO:teuthology.orchestra.run.smithi139.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-03-18T22:38:18.136 INFO:teuthology.orchestra.run.smithi139.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-03-18T22:38:18.136 INFO:teuthology.orchestra.run.smithi139.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-03-18T22:38:18.136 INFO:teuthology.orchestra.run.smithi139.stdout: python3-google-auth noarch 1:2.28.0-1.el9 epel 203 k 2024-03-18T22:38:18.136 INFO:teuthology.orchestra.run.smithi139.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-03-18T22:38:18.136 INFO:teuthology.orchestra.run.smithi139.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-03-18T22:38:18.136 INFO:teuthology.orchestra.run.smithi139.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-03-18T22:38:18.136 INFO:teuthology.orchestra.run.smithi139.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-03-18T22:38:18.136 INFO:teuthology.orchestra.run.smithi139.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-03-18T22:38:18.136 INFO:teuthology.orchestra.run.smithi139.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-03-18T22:38:18.136 INFO:teuthology.orchestra.run.smithi139.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-03-18T22:38:18.136 INFO:teuthology.orchestra.run.smithi139.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-03-18T22:38:18.136 INFO:teuthology.orchestra.run.smithi139.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-03-18T22:38:18.136 INFO:teuthology.orchestra.run.smithi139.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-03-18T22:38:18.136 INFO:teuthology.orchestra.run.smithi139.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-03-18T22:38:18.137 INFO:teuthology.orchestra.run.smithi139.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-03-18T22:38:18.137 INFO:teuthology.orchestra.run.smithi139.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-03-18T22:38:18.137 INFO:teuthology.orchestra.run.smithi139.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-03-18T22:38:18.137 INFO:teuthology.orchestra.run.smithi139.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-03-18T22:38:18.137 INFO:teuthology.orchestra.run.smithi139.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-03-18T22:38:18.137 INFO:teuthology.orchestra.run.smithi139.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-03-18T22:38:18.137 INFO:teuthology.orchestra.run.smithi139.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-03-18T22:38:18.137 INFO:teuthology.orchestra.run.smithi139.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-03-18T22:38:18.137 INFO:teuthology.orchestra.run.smithi139.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-03-18T22:38:18.137 INFO:teuthology.orchestra.run.smithi139.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-03-18T22:38:18.137 INFO:teuthology.orchestra.run.smithi139.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-03-18T22:38:18.137 INFO:teuthology.orchestra.run.smithi139.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-03-18T22:38:18.137 INFO:teuthology.orchestra.run.smithi139.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-03-18T22:38:18.138 INFO:teuthology.orchestra.run.smithi139.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-03-18T22:38:18.138 INFO:teuthology.orchestra.run.smithi139.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-03-18T22:38:18.138 INFO:teuthology.orchestra.run.smithi139.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-03-18T22:38:18.138 INFO:teuthology.orchestra.run.smithi139.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-03-18T22:38:18.138 INFO:teuthology.orchestra.run.smithi139.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-03-18T22:38:18.138 INFO:teuthology.orchestra.run.smithi139.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-03-18T22:38:18.138 INFO:teuthology.orchestra.run.smithi139.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-03-18T22:38:18.138 INFO:teuthology.orchestra.run.smithi139.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-03-18T22:38:18.138 INFO:teuthology.orchestra.run.smithi139.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-03-18T22:38:18.138 INFO:teuthology.orchestra.run.smithi139.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-03-18T22:38:18.138 INFO:teuthology.orchestra.run.smithi139.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-03-18T22:38:18.138 INFO:teuthology.orchestra.run.smithi139.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-03-18T22:38:18.138 INFO:teuthology.orchestra.run.smithi139.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-03-18T22:38:18.139 INFO:teuthology.orchestra.run.smithi139.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-03-18T22:38:18.139 INFO:teuthology.orchestra.run.smithi139.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-03-18T22:38:18.139 INFO:teuthology.orchestra.run.smithi139.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-03-18T22:38:18.139 INFO:teuthology.orchestra.run.smithi139.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-03-18T22:38:18.139 INFO:teuthology.orchestra.run.smithi139.stdout: python3-wcwidth noarch 0.2.5-8.el9 CentOS-AppStream 41 k 2024-03-18T22:38:18.139 INFO:teuthology.orchestra.run.smithi139.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-03-18T22:38:18.139 INFO:teuthology.orchestra.run.smithi139.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-03-18T22:38:18.139 INFO:teuthology.orchestra.run.smithi139.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-03-18T22:38:18.139 INFO:teuthology.orchestra.run.smithi139.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-03-18T22:38:18.139 INFO:teuthology.orchestra.run.smithi139.stdout: qatlib x86_64 23.11.0-1.el9 appstream 216 k 2024-03-18T22:38:18.139 INFO:teuthology.orchestra.run.smithi139.stdout: qatzip-libs x86_64 1.1.2-2.el9 appstream 44 k 2024-03-18T22:38:18.139 INFO:teuthology.orchestra.run.smithi139.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-03-18T22:38:18.139 INFO:teuthology.orchestra.run.smithi139.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-03-18T22:38:18.139 INFO:teuthology.orchestra.run.smithi139.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-03-18T22:38:18.139 INFO:teuthology.orchestra.run.smithi139.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-03-18T22:38:18.139 INFO:teuthology.orchestra.run.smithi139.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-03-18T22:38:18.140 INFO:teuthology.orchestra.run.smithi139.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-03-18T22:38:18.140 INFO:teuthology.orchestra.run.smithi139.stdout:Installing weak dependencies: 2024-03-18T22:38:18.140 INFO:teuthology.orchestra.run.smithi139.stdout: qatlib-service x86_64 23.11.0-1.el9 appstream 35 k 2024-03-18T22:38:18.140 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-03-18T22:38:18.140 INFO:teuthology.orchestra.run.smithi139.stdout:Transaction Summary 2024-03-18T22:38:18.140 INFO:teuthology.orchestra.run.smithi139.stdout:============================================================================================ 2024-03-18T22:38:18.140 INFO:teuthology.orchestra.run.smithi139.stdout:Install 150 Packages 2024-03-18T22:38:18.140 INFO:teuthology.orchestra.run.smithi139.stdout:Upgrade 24 Packages 2024-03-18T22:38:18.140 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-03-18T22:38:18.140 INFO:teuthology.orchestra.run.smithi139.stdout:Total download size: 224 M 2024-03-18T22:38:18.141 INFO:teuthology.orchestra.run.smithi139.stdout:Downloading Packages: 2024-03-18T22:38:19.241 INFO:teuthology.orchestra.run.smithi139.stdout:(1/174): ceph-19.0.0-2206.gb6ae2491.el9.x86_64. 28 kB/s | 6.6 kB 00:00 2024-03-18T22:38:19.691 INFO:teuthology.orchestra.run.smithi139.stdout:(2/174): ceph-fuse-19.0.0-2206.gb6ae2491.el9.x8 1.9 MB/s | 891 kB 00:00 2024-03-18T22:38:19.767 INFO:teuthology.orchestra.run.smithi139.stdout:(3/174): ceph-immutable-object-cache-19.0.0-220 1.9 MB/s | 145 kB 00:00 2024-03-18T22:38:20.217 INFO:teuthology.orchestra.run.smithi139.stdout:(4/174): ceph-base-19.0.0-2206.gb6ae2491.el9.x8 4.4 MB/s | 5.4 MB 00:01 2024-03-18T22:38:20.425 INFO:teuthology.orchestra.run.smithi139.stdout:(5/174): ceph-mgr-19.0.0-2206.gb6ae2491.el9.x86 7.2 MB/s | 1.5 MB 00:00 2024-03-18T22:38:20.467 INFO:teuthology.orchestra.run.smithi139.stdout:(6/174): ceph-mds-19.0.0-2206.gb6ae2491.el9.x86 3.4 MB/s | 2.4 MB 00:00 2024-03-18T22:38:20.884 INFO:teuthology.orchestra.run.smithi139.stdout:(7/174): ceph-mon-19.0.0-2206.gb6ae2491.el9.x86 10 MB/s | 4.7 MB 00:00 2024-03-18T22:38:21.734 INFO:teuthology.orchestra.run.smithi139.stdout:(8/174): ceph-radosgw-19.0.0-2206.gb6ae2491.el9 12 MB/s | 9.8 MB 00:00 2024-03-18T22:38:21.834 INFO:teuthology.orchestra.run.smithi139.stdout:(9/174): ceph-selinux-19.0.0-2206.gb6ae2491.el9 252 kB/s | 25 kB 00:00 2024-03-18T22:38:22.526 INFO:teuthology.orchestra.run.smithi139.stdout:(10/174): ceph-common-19.0.0-2206.gb6ae2491.el9 5.9 MB/s | 21 MB 00:03 2024-03-18T22:38:22.618 INFO:teuthology.orchestra.run.smithi139.stdout:(11/174): cephfs-mirror-19.0.0-2206.gb6ae2491.e 2.4 MB/s | 221 kB 00:00 2024-03-18T22:38:22.710 INFO:teuthology.orchestra.run.smithi139.stdout:(12/174): cephfs-shell-19.0.0-2206.gb6ae2491.el 296 kB/s | 27 kB 00:00 2024-03-18T22:38:22.919 INFO:teuthology.orchestra.run.smithi139.stdout:(13/174): ceph-osd-19.0.0-2206.gb6ae2491.el9.x8 6.9 MB/s | 17 MB 00:02 2024-03-18T22:38:22.944 INFO:teuthology.orchestra.run.smithi139.stdout:(14/174): libcephfs-devel-19.0.0-2206.gb6ae2491 136 kB/s | 32 kB 00:00 2024-03-18T22:38:23.028 INFO:teuthology.orchestra.run.smithi139.stdout:(15/174): libcephsqlite-19.0.0-2206.gb6ae2491.e 1.9 MB/s | 163 kB 00:00 2024-03-18T22:38:23.061 INFO:teuthology.orchestra.run.smithi139.stdout:(16/174): libcephfs2-19.0.0-2206.gb6ae2491.el9. 5.1 MB/s | 746 kB 00:00 2024-03-18T22:38:23.114 INFO:teuthology.orchestra.run.smithi139.stdout:(17/174): librados-devel-19.0.0-2206.gb6ae2491. 1.4 MB/s | 127 kB 00:00 2024-03-18T22:38:23.147 INFO:teuthology.orchestra.run.smithi139.stdout:(18/174): libradosstriper1-19.0.0-2206.gb6ae249 5.7 MB/s | 501 kB 00:00 2024-03-18T22:38:23.222 INFO:teuthology.orchestra.run.smithi139.stdout:(19/174): python3-ceph-argparse-19.0.0-2206.gb6 602 kB/s | 45 kB 00:00 2024-03-18T22:38:23.323 INFO:teuthology.orchestra.run.smithi139.stdout:(20/174): python3-ceph-common-19.0.0-2206.gb6ae 1.3 MB/s | 131 kB 00:00 2024-03-18T22:38:23.465 INFO:teuthology.orchestra.run.smithi139.stdout:(21/174): python3-cephfs-19.0.0-2206.gb6ae2491. 1.1 MB/s | 162 kB 00:00 2024-03-18T22:38:23.573 INFO:teuthology.orchestra.run.smithi139.stdout:(22/174): python3-rados-19.0.0-2206.gb6ae2491.e 2.9 MB/s | 321 kB 00:00 2024-03-18T22:38:23.690 INFO:teuthology.orchestra.run.smithi139.stdout:(23/174): librgw2-19.0.0-2206.gb6ae2491.el9.x86 8.9 MB/s | 5.1 MB 00:00 2024-03-18T22:38:23.715 INFO:teuthology.orchestra.run.smithi139.stdout:(24/174): python3-rbd-19.0.0-2206.gb6ae2491.el9 2.1 MB/s | 298 kB 00:00 2024-03-18T22:38:23.766 INFO:teuthology.orchestra.run.smithi139.stdout:(25/174): python3-rgw-19.0.0-2206.gb6ae2491.el9 1.3 MB/s | 99 kB 00:00 2024-03-18T22:38:23.799 INFO:teuthology.orchestra.run.smithi139.stdout:(26/174): rbd-fuse-19.0.0-2206.gb6ae2491.el9.x8 1.0 MB/s | 85 kB 00:00 2024-03-18T22:38:23.908 INFO:teuthology.orchestra.run.smithi139.stdout:(27/174): rbd-nbd-19.0.0-2206.gb6ae2491.el9.x86 1.5 MB/s | 170 kB 00:00 2024-03-18T22:38:24.025 INFO:teuthology.orchestra.run.smithi139.stdout:(28/174): ceph-grafana-dashboards-19.0.0-2206.g 291 kB/s | 34 kB 00:00 2024-03-18T22:38:24.100 INFO:teuthology.orchestra.run.smithi139.stdout:(29/174): rbd-mirror-19.0.0-2206.gb6ae2491.el9. 9.3 MB/s | 3.1 MB 00:00 2024-03-18T22:38:24.125 INFO:teuthology.orchestra.run.smithi139.stdout:(30/174): ceph-mgr-cephadm-19.0.0-2206.gb6ae249 1.4 MB/s | 139 kB 00:00 2024-03-18T22:38:24.367 INFO:teuthology.orchestra.run.smithi139.stdout:(31/174): ceph-mgr-dashboard-19.0.0-2206.gb6ae2 6.7 MB/s | 1.8 MB 00:00 2024-03-18T22:38:24.476 INFO:teuthology.orchestra.run.smithi139.stdout:(32/174): ceph-mgr-modules-core-19.0.0-2206.gb6 2.2 MB/s | 248 kB 00:00 2024-03-18T22:38:24.576 INFO:teuthology.orchestra.run.smithi139.stdout:(33/174): ceph-mgr-rook-19.0.0-2206.gb6ae2491.e 494 kB/s | 49 kB 00:00 2024-03-18T22:38:24.676 INFO:teuthology.orchestra.run.smithi139.stdout:(34/174): ceph-prometheus-alerts-19.0.0-2206.gb 165 kB/s | 16 kB 00:00 2024-03-18T22:38:24.777 INFO:teuthology.orchestra.run.smithi139.stdout:(35/174): ceph-volume-19.0.0-2206.gb6ae2491.el9 2.6 MB/s | 264 kB 00:00 2024-03-18T22:38:25.227 INFO:teuthology.orchestra.run.smithi139.stdout:(36/174): ceph-test-19.0.0-2206.gb6ae2491.el9.x 14 MB/s | 48 MB 00:03 2024-03-18T22:38:25.269 INFO:teuthology.orchestra.run.smithi139.stdout:(37/174): cephadm-19.0.0-2206.gb6ae2491.el9.noa 1.5 MB/s | 749 kB 00:00 2024-03-18T22:38:25.294 INFO:teuthology.orchestra.run.smithi139.stdout:(38/174): cephfs-top-19.0.0-2206.gb6ae2491.el9. 275 kB/s | 18 kB 00:00 2024-03-18T22:38:25.386 INFO:teuthology.orchestra.run.smithi139.stdout:(39/174): ceph-mgr-diskprediction-local-19.0.0- 5.9 MB/s | 7.4 MB 00:01 2024-03-18T22:38:25.536 INFO:teuthology.orchestra.run.smithi139.stdout:(40/174): boost-program-options-1.75.0-8.el9.x8 1.1 MB/s | 106 kB 00:00 2024-03-18T22:38:25.561 INFO:teuthology.orchestra.run.smithi139.stdout:(41/174): automake-1.16.2-8.el9.noarch.rpm 2.2 MB/s | 662 kB 00:00 2024-03-18T22:38:25.587 INFO:teuthology.orchestra.run.smithi139.stdout:(42/174): bison-3.7.4-5.el9.x86_64.rpm 3.1 MB/s | 920 kB 00:00 2024-03-18T22:38:25.612 INFO:teuthology.orchestra.run.smithi139.stdout:(43/174): flex-2.6.4-9.el9.x86_64.rpm 4.1 MB/s | 307 kB 00:00 2024-03-18T22:38:25.637 INFO:teuthology.orchestra.run.smithi139.stdout:(44/174): flexiblas-3.0.4-8.el9.x86_64.rpm 640 kB/s | 32 kB 00:00 2024-03-18T22:38:25.696 INFO:teuthology.orchestra.run.smithi139.stdout:(45/174): flexiblas-netlib-3.0.4-8.el9.x86_64.r 28 MB/s | 3.0 MB 00:00 2024-03-18T22:38:25.721 INFO:teuthology.orchestra.run.smithi139.stdout:(46/174): flexiblas-openblas-openmp-3.0.4-8.el9 160 kB/s | 17 kB 00:00 2024-03-18T22:38:25.746 INFO:teuthology.orchestra.run.smithi139.stdout:(47/174): libaio-devel-0.3.111-13.el9.x86_64.rp 94 kB/s | 10 kB 00:00 2024-03-18T22:38:25.772 INFO:teuthology.orchestra.run.smithi139.stdout:(48/174): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.1 MB/s | 159 kB 00:00 2024-03-18T22:38:25.797 INFO:teuthology.orchestra.run.smithi139.stdout:(49/174): librabbitmq-0.11.0-7.el9.x86_64.rpm 584 kB/s | 44 kB 00:00 2024-03-18T22:38:25.822 INFO:teuthology.orchestra.run.smithi139.stdout:(50/174): librdkafka-1.6.1-102.el9.x86_64.rpm 8.6 MB/s | 661 kB 00:00 2024-03-18T22:38:25.847 INFO:teuthology.orchestra.run.smithi139.stdout:(51/174): libzstd-devel-1.5.1-2.el9.x86_64.rpm 949 kB/s | 46 kB 00:00 2024-03-18T22:38:25.874 INFO:teuthology.orchestra.run.smithi139.stdout:(52/174): libtool-2.4.6-45.el9.x86_64.rpm 7.6 MB/s | 578 kB 00:00 2024-03-18T22:38:25.898 INFO:teuthology.orchestra.run.smithi139.stdout:(53/174): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.7 MB/s | 282 kB 00:00 2024-03-18T22:38:25.914 INFO:teuthology.orchestra.run.smithi139.stdout:(54/174): lua-5.4.4-4.el9.x86_64.rpm 2.8 MB/s | 187 kB 00:00 2024-03-18T22:38:25.940 INFO:teuthology.orchestra.run.smithi139.stdout:(55/174): openblas-0.3.21-2.el9.x86_64.rpm 826 kB/s | 34 kB 00:00 2024-03-18T22:38:25.965 INFO:teuthology.orchestra.run.smithi139.stdout:(56/174): oniguruma-6.9.6-1.el9.5.x86_64.rpm 3.2 MB/s | 217 kB 00:00 2024-03-18T22:38:26.065 INFO:teuthology.orchestra.run.smithi139.stdout:(57/174): openblas-openmp-0.3.21-2.el9.x86_64.r 32 MB/s | 4.7 MB 00:00 2024-03-18T22:38:26.107 INFO:teuthology.orchestra.run.smithi139.stdout:(58/174): postgresql-13.11-1.el9.x86_64.rpm 9.2 MB/s | 1.5 MB 00:00 2024-03-18T22:38:26.141 INFO:teuthology.orchestra.run.smithi139.stdout:(59/174): postgresql-contrib-13.11-1.el9.x86_64 4.6 MB/s | 811 kB 00:00 2024-03-18T22:38:26.166 INFO:teuthology.orchestra.run.smithi139.stdout:(60/174): postgresql-private-libs-13.11-1.el9.x 1.3 MB/s | 134 kB 00:00 2024-03-18T22:38:26.225 INFO:teuthology.orchestra.run.smithi139.stdout:(61/174): protobuf-3.14.0-13.el9.x86_64.rpm 12 MB/s | 1.0 MB 00:00 2024-03-18T22:38:26.308 INFO:teuthology.orchestra.run.smithi139.stdout:(62/174): postgresql-server-13.11-1.el9.x86_64. 34 MB/s | 5.7 MB 00:00 2024-03-18T22:38:26.333 INFO:teuthology.orchestra.run.smithi139.stdout:(63/174): python3-attrs-20.3.0-7.el9.noarch.rpm 467 kB/s | 78 kB 00:00 2024-03-18T22:38:26.434 INFO:teuthology.orchestra.run.smithi139.stdout:(64/174): python3-babel-2.9.1-2.el9.noarch.rpm 28 MB/s | 5.8 MB 00:00 2024-03-18T22:38:26.459 INFO:teuthology.orchestra.run.smithi139.stdout:(65/174): python3-jsonpatch-1.21-16.el9.noarch. 165 kB/s | 24 kB 00:00 2024-03-18T22:38:26.484 INFO:teuthology.orchestra.run.smithi139.stdout:(66/174): python3-jsonpointer-2.0-4.el9.noarch. 122 kB/s | 18 kB 00:00 2024-03-18T22:38:26.509 INFO:teuthology.orchestra.run.smithi139.stdout:(67/174): python3-mako-1.1.4-6.el9.noarch.rpm 1.9 MB/s | 144 kB 00:00 2024-03-18T22:38:26.534 INFO:teuthology.orchestra.run.smithi139.stdout:(68/174): python3-markupsafe-1.1.1-12.el9.x86_6 633 kB/s | 32 kB 00:00 2024-03-18T22:38:26.618 INFO:teuthology.orchestra.run.smithi139.stdout:(69/174): python3-numpy-1.20.1-5.el9.x86_64.rpm 36 MB/s | 4.8 MB 00:00 2024-03-18T22:38:26.635 INFO:teuthology.orchestra.run.smithi139.stdout:(70/174): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.7 MB/s | 217 kB 00:00 2024-03-18T22:38:26.660 INFO:teuthology.orchestra.run.smithi139.stdout:(71/174): python3-oauthlib-3.1.1-5.el9.noarch.r 1.4 MB/s | 174 kB 00:00 2024-03-18T22:38:26.686 INFO:teuthology.orchestra.run.smithi139.stdout:(72/174): python3-packaging-20.9-5.el9.noarch.r 1.0 MB/s | 69 kB 00:00 2024-03-18T22:38:26.711 INFO:teuthology.orchestra.run.smithi139.stdout:(73/174): python3-prettytable-0.7.2-27.el9.noar 824 kB/s | 41 kB 00:00 2024-03-18T22:38:26.736 INFO:teuthology.orchestra.run.smithi139.stdout:(74/174): python3-protobuf-3.14.0-13.el9.noarch 3.1 MB/s | 238 kB 00:00 2024-03-18T22:38:26.761 INFO:teuthology.orchestra.run.smithi139.stdout:(75/174): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.7 MB/s | 132 kB 00:00 2024-03-18T22:38:26.786 INFO:teuthology.orchestra.run.smithi139.stdout:(76/174): python3-pytz-2021.1-5.el9.noarch.rpm 951 kB/s | 47 kB 00:00 2024-03-18T22:38:26.811 INFO:teuthology.orchestra.run.smithi139.stdout:(77/174): python3-pyasn1-modules-0.4.8-6.el9.no 2.7 MB/s | 211 kB 00:00 2024-03-18T22:38:26.837 INFO:teuthology.orchestra.run.smithi139.stdout:(78/174): python3-requests-oauthlib-1.3.0-12.el 570 kB/s | 43 kB 00:00 2024-03-18T22:38:27.145 INFO:teuthology.orchestra.run.smithi139.stdout:(79/174): python3-scipy-1.6.2-8.el9.x86_64.rpm 44 MB/s | 16 MB 00:00 2024-03-18T22:38:27.171 INFO:teuthology.orchestra.run.smithi139.stdout:(80/174): python3-wcwidth-0.2.5-8.el9.noarch.rp 124 kB/s | 41 kB 00:00 2024-03-18T22:38:27.196 INFO:teuthology.orchestra.run.smithi139.stdout:(81/174): python3-toml-0.10.2-6.el9.noarch.rpm 109 kB/s | 39 kB 00:00 2024-03-18T22:38:27.221 INFO:teuthology.orchestra.run.smithi139.stdout:(82/174): socat-1.7.4.1-5.el9.x86_64.rpm 3.9 MB/s | 300 kB 00:00 2024-03-18T22:38:27.238 INFO:teuthology.orchestra.run.smithi139.stdout:(83/174): uuid-1.6.2-55.el9.x86_64.rpm 850 kB/s | 56 kB 00:00 2024-03-18T22:38:27.272 INFO:teuthology.orchestra.run.smithi139.stdout:(84/174): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 63 kB 00:00 2024-03-18T22:38:27.288 INFO:teuthology.orchestra.run.smithi139.stdout:(85/174): zlib-devel-1.2.11-40.el9.x86_64.rpm 658 kB/s | 44 kB 00:00 2024-03-18T22:38:27.306 INFO:teuthology.orchestra.run.smithi139.stdout:(86/174): fuse-2.9.9-15.el9.x86_64.rpm 1.1 MB/s | 79 kB 00:00 2024-03-18T22:38:27.331 INFO:teuthology.orchestra.run.smithi139.stdout:(87/174): python3-cffi-1.14.5-5.el9.x86_64.rpm 5.7 MB/s | 241 kB 00:00 2024-03-18T22:38:27.364 INFO:teuthology.orchestra.run.smithi139.stdout:(88/174): python3-cryptography-36.0.1-4.el9.x86 15 MB/s | 1.1 MB 00:00 2024-03-18T22:38:27.389 INFO:teuthology.orchestra.run.smithi139.stdout:(89/174): python3-ply-3.11-14.el9.noarch.rpm 1.2 MB/s | 103 kB 00:00 2024-03-18T22:38:27.415 INFO:teuthology.orchestra.run.smithi139.stdout:(90/174): python3-pycparser-2.20-6.el9.noarch.r 1.5 MB/s | 124 kB 00:00 2024-03-18T22:38:27.440 INFO:teuthology.orchestra.run.smithi139.stdout:(91/174): python3-urllib3-1.26.5-5.el9.noarch.r 3.7 MB/s | 187 kB 00:00 2024-03-18T22:38:27.590 INFO:teuthology.orchestra.run.smithi139.stdout:(92/174): libquadmath-11.4.1-3.el9.x86_64.rpm 1.3 MB/s | 195 kB 00:00 2024-03-18T22:38:27.615 INFO:teuthology.orchestra.run.smithi139.stdout:(93/174): jq-1.6-16.el9.x86_64.rpm 831 kB/s | 187 kB 00:00 2024-03-18T22:38:27.649 INFO:teuthology.orchestra.run.smithi139.stdout:(94/174): python3-requests-2.25.1-8.el9.noarch. 2.1 MB/s | 125 kB 00:00 2024-03-18T22:38:27.674 INFO:teuthology.orchestra.run.smithi139.stdout:(95/174): libgfortran-11.4.1-3.el9.x86_64.rpm 3.0 MB/s | 803 kB 00:00 2024-03-18T22:38:27.883 INFO:teuthology.orchestra.run.smithi139.stdout:(96/174): elfutils-libelf-devel-0.190-2.el9.x86 80 kB/s | 21 kB 00:00 2024-03-18T22:38:27.908 INFO:teuthology.orchestra.run.smithi139.stdout:(97/174): libuuid-devel-2.37.4-18.el9.x86_64.rp 98 kB/s | 25 kB 00:00 2024-03-18T22:38:28.225 INFO:teuthology.orchestra.run.smithi139.stdout:(98/174): python3-devel-3.9.17-2.el9.x86_64.rpm 718 kB/s | 245 kB 00:00 2024-03-18T22:38:28.250 INFO:teuthology.orchestra.run.smithi139.stdout:(99/174): python3-jinja2-2.11.3-5.el9.noarch.rp 725 kB/s | 248 kB 00:00 2024-03-18T22:38:28.359 INFO:teuthology.orchestra.run.smithi139.stdout:(100/174): qatlib-service-23.11.0-1.el9.x86_64. 322 kB/s | 35 kB 00:00 2024-03-18T22:38:28.409 INFO:teuthology.orchestra.run.smithi139.stdout:(101/174): qatlib-23.11.0-1.el9.x86_64.rpm 1.2 MB/s | 216 kB 00:00 2024-03-18T22:38:28.459 INFO:teuthology.orchestra.run.smithi139.stdout:(102/174): qatzip-libs-1.1.2-2.el9.x86_64.rpm 444 kB/s | 44 kB 00:00 2024-03-18T22:38:28.518 INFO:teuthology.orchestra.run.smithi139.stdout:(103/174): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 864 kB/s | 94 kB 00:00 2024-03-18T22:38:28.560 INFO:teuthology.orchestra.run.smithi139.stdout:(104/174): lua-devel-5.4.4-4.el9.x86_64.rpm 222 kB/s | 22 kB 00:00 2024-03-18T22:38:28.735 INFO:teuthology.orchestra.run.smithi139.stdout:(105/174): openssl-devel-3.0.7-27.el9.x86_64.rp 3.9 MB/s | 4.1 MB 00:01 2024-03-18T22:38:28.835 INFO:teuthology.orchestra.run.smithi139.stdout:(106/174): protobuf-compiler-3.14.0-13.el9.x86_ 2.7 MB/s | 863 kB 00:00 2024-03-18T22:38:29.135 INFO:teuthology.orchestra.run.smithi139.stdout:(107/174): grpc-data-1.46.7-10.el9.noarch.rpm 65 kB/s | 19 kB 00:00 2024-03-18T22:38:29.669 INFO:teuthology.orchestra.run.smithi139.stdout:(108/174): gperftools-libs-2.9.1-2.el9.x86_64.r 331 kB/s | 309 kB 00:00 2024-03-18T22:38:29.794 INFO:teuthology.orchestra.run.smithi139.stdout:(109/174): abseil-cpp-20211102.0-3.el9.x86_64.r 444 kB/s | 548 kB 00:01 2024-03-18T22:38:29.819 INFO:teuthology.orchestra.run.smithi139.stdout:(110/174): libarrow-doc-9.0.0-10.el9.noarch.rpm 173 kB/s | 26 kB 00:00 2024-03-18T22:38:29.936 INFO:teuthology.orchestra.run.smithi139.stdout:(111/174): libunwind-1.6.2-1.el9.x86_64.rpm 578 kB/s | 67 kB 00:00 2024-03-18T22:38:29.953 INFO:teuthology.orchestra.run.smithi139.stdout:(112/174): liboath-2.6.7-2.el9.x86_64.rpm 364 kB/s | 49 kB 00:00 2024-03-18T22:38:30.145 INFO:teuthology.orchestra.run.smithi139.stdout:(113/174): luarocks-3.9.2-1.el9.noarch.rpm 725 kB/s | 151 kB 00:00 2024-03-18T22:38:30.879 INFO:teuthology.orchestra.run.smithi139.stdout:(114/174): python3-asyncssh-2.13.2-4.el9.noarch 747 kB/s | 548 kB 00:00 2024-03-18T22:38:30.987 INFO:teuthology.orchestra.run.smithi139.stdout:(115/174): python3-bcrypt-3.2.2-1.el9.x86_64.rp 401 kB/s | 43 kB 00:00 2024-03-18T22:38:31.013 INFO:teuthology.orchestra.run.smithi139.stdout:(116/174): parquet-libs-9.0.0-10.el9.x86_64.rpm 791 kB/s | 838 kB 00:01 2024-03-18T22:38:31.105 INFO:teuthology.orchestra.run.smithi139.stdout:(117/174): python3-cachetools-4.2.4-1.el9.noarc 274 kB/s | 32 kB 00:00 2024-03-18T22:38:31.121 INFO:teuthology.orchestra.run.smithi139.stdout:(118/174): python3-certifi-2021.10.8-2.el9.noar 138 kB/s | 15 kB 00:00 2024-03-18T22:38:31.397 INFO:teuthology.orchestra.run.smithi139.stdout:(119/174): python3-cheroot-8.6.0-4.el9.noarch.r 589 kB/s | 172 kB 00:00 2024-03-18T22:38:31.631 INFO:teuthology.orchestra.run.smithi139.stdout:(120/174): python3-cherrypy-18.6.1-2.el9.noarch 705 kB/s | 358 kB 00:00 2024-03-18T22:38:31.747 INFO:teuthology.orchestra.run.smithi139.stdout:(121/174): python3-colorama-0.4.4-12.el9.noarch 303 kB/s | 35 kB 00:00 2024-03-18T22:38:31.823 INFO:teuthology.orchestra.run.smithi139.stdout:(122/174): python3-cmd2-2.4.2-2.el9.noarch.rpm 712 kB/s | 303 kB 00:00 2024-03-18T22:38:32.048 INFO:teuthology.orchestra.run.smithi139.stdout:(123/174): python3-google-auth-2.28.0-1.el9.noa 675 kB/s | 203 kB 00:00 2024-03-18T22:38:32.248 INFO:teuthology.orchestra.run.smithi139.stdout:(124/174): python3-grpcio-tools-1.46.7-10.el9.x 723 kB/s | 144 kB 00:00 2024-03-18T22:38:32.365 INFO:teuthology.orchestra.run.smithi139.stdout:(125/174): python3-jaraco-8.2.1-3.el9.noarch.rp 92 kB/s | 11 kB 00:00 2024-03-18T22:38:32.515 INFO:teuthology.orchestra.run.smithi139.stdout:(126/174): python3-jaraco-classes-3.2.1-5.el9.n 118 kB/s | 18 kB 00:00 2024-03-18T22:38:32.632 INFO:teuthology.orchestra.run.smithi139.stdout:(127/174): python3-jaraco-collections-3.0.0-8.e 198 kB/s | 23 kB 00:00 2024-03-18T22:38:32.749 INFO:teuthology.orchestra.run.smithi139.stdout:(128/174): python3-jaraco-functools-3.5.0-2.el9 166 kB/s | 19 kB 00:00 2024-03-18T22:38:32.858 INFO:teuthology.orchestra.run.smithi139.stdout:(129/174): python3-jaraco-text-3.2.0-6.el9.noar 182 kB/s | 20 kB 00:00 2024-03-18T22:38:33.974 INFO:teuthology.orchestra.run.smithi139.stdout:(130/174): python3-kubernetes-26.1.0-2.el9.noar 936 kB/s | 1.0 MB 00:01 2024-03-18T22:38:34.017 INFO:teuthology.orchestra.run.smithi139.stdout:(131/174): python3-grpcio-1.46.7-10.el9.x86_64. 951 kB/s | 2.0 MB 00:02 2024-03-18T22:38:34.083 INFO:teuthology.orchestra.run.smithi139.stdout:(132/174): python3-logutils-0.3.5-21.el9.noarch 427 kB/s | 46 kB 00:00 2024-03-18T22:38:34.134 INFO:teuthology.orchestra.run.smithi139.stdout:(133/174): python3-more-itertools-8.12.0-2.el9. 675 kB/s | 79 kB 00:00 2024-03-18T22:38:34.201 INFO:teuthology.orchestra.run.smithi139.stdout:(134/174): python3-natsort-7.1.1-5.el9.noarch.r 493 kB/s | 58 kB 00:00 2024-03-18T22:38:34.267 INFO:teuthology.orchestra.run.smithi139.stdout:(135/174): libarrow-9.0.0-10.el9.x86_64.rpm 881 kB/s | 4.4 MB 00:05 2024-03-18T22:38:34.318 INFO:teuthology.orchestra.run.smithi139.stdout:(136/174): python3-portend-3.1.0-2.el9.noarch.r 140 kB/s | 16 kB 00:00 2024-03-18T22:38:34.385 INFO:teuthology.orchestra.run.smithi139.stdout:(137/174): python3-pyOpenSSL-21.0.0-1.el9.noarc 769 kB/s | 90 kB 00:00 2024-03-18T22:38:34.435 INFO:teuthology.orchestra.run.smithi139.stdout:(138/174): python3-pyperclip-1.8.2-2.el9.noarch 242 kB/s | 28 kB 00:00 2024-03-18T22:38:34.469 INFO:teuthology.orchestra.run.smithi139.stdout:(139/174): python3-pecan-1.4.2-3.el9.noarch.rpm 814 kB/s | 272 kB 00:00 2024-03-18T22:38:34.502 INFO:teuthology.orchestra.run.smithi139.stdout:(140/174): python3-repoze-lru-0.7-16.el9.noarch 262 kB/s | 31 kB 00:00 2024-03-18T22:38:34.602 INFO:teuthology.orchestra.run.smithi139.stdout:(141/174): python3-rsa-4.9-2.el9.noarch.rpm 443 kB/s | 59 kB 00:00 2024-03-18T22:38:34.627 INFO:teuthology.orchestra.run.smithi139.stdout:(142/174): python3-tempora-5.0.0-2.el9.noarch.r 286 kB/s | 36 kB 00:00 2024-03-18T22:38:34.720 INFO:teuthology.orchestra.run.smithi139.stdout:(143/174): python3-typing-extensions-4.4.0-2.el 436 kB/s | 51 kB 00:00 2024-03-18T22:38:34.745 INFO:teuthology.orchestra.run.smithi139.stdout:(144/174): python3-routes-2.5.1-5.el9.noarch.rp 609 kB/s | 188 kB 00:00 2024-03-18T22:38:34.837 INFO:teuthology.orchestra.run.smithi139.stdout:(145/174): python3-websocket-client-1.2.3-2.el9 769 kB/s | 90 kB 00:00 2024-03-18T22:38:34.929 INFO:teuthology.orchestra.run.smithi139.stdout:(146/174): python3-webob-1.8.7-6.el9.noarch.rpm 763 kB/s | 230 kB 00:00 2024-03-18T22:38:34.954 INFO:teuthology.orchestra.run.smithi139.stdout:(147/174): python3-zc-lockfile-2.0-10.el9.noarc 171 kB/s | 20 kB 00:00 2024-03-18T22:38:35.146 INFO:teuthology.orchestra.run.smithi139.stdout:(148/174): re2-20211101-3.el9.x86_64.rpm 883 kB/s | 192 kB 00:00 2024-03-18T22:38:35.246 INFO:teuthology.orchestra.run.smithi139.stdout:(149/174): python3-werkzeug-2.0.3-3.el9.1.noarc 852 kB/s | 427 kB 00:00 2024-03-18T22:38:35.538 INFO:teuthology.orchestra.run.smithi139.stdout:(150/174): librados2-19.0.0-2206.gb6ae2491.el9. 8.7 MB/s | 3.4 MB 00:00 2024-03-18T22:38:35.605 INFO:teuthology.orchestra.run.smithi139.stdout:(151/174): xfsdump-3.1.12-4.el9.x86_64.rpm 4.7 MB/s | 321 kB 00:00 2024-03-18T22:38:35.997 INFO:teuthology.orchestra.run.smithi139.stdout:(152/174): librbd1-19.0.0-2206.gb6ae2491.el9.x8 4.2 MB/s | 3.1 MB 00:00 2024-03-18T22:38:36.472 INFO:teuthology.orchestra.run.smithi139.stdout:(153/174): NetworkManager-libnm-1.47.2-1.el9.x8 3.9 MB/s | 1.8 MB 00:00 2024-03-18T22:38:36.547 INFO:teuthology.orchestra.run.smithi139.stdout:(154/174): NetworkManager-1.47.2-1.el9.x86_64.r 2.4 MB/s | 2.3 MB 00:00 2024-03-18T22:38:36.572 INFO:teuthology.orchestra.run.smithi139.stdout:(155/174): NetworkManager-team-1.47.2-1.el9.x86 421 kB/s | 42 kB 00:00 2024-03-18T22:38:36.631 INFO:teuthology.orchestra.run.smithi139.stdout:(156/174): acl-2.3.1-4.el9.x86_64.rpm 1.2 MB/s | 71 kB 00:00 2024-03-18T22:38:36.690 INFO:teuthology.orchestra.run.smithi139.stdout:(157/174): thrift-0.15.0-2.el9.x86_64.rpm 933 kB/s | 1.6 MB 00:01 2024-03-18T22:38:36.715 INFO:teuthology.orchestra.run.smithi139.stdout:(158/174): elfutils-debuginfod-client-0.190-2.e 437 kB/s | 36 kB 00:00 2024-03-18T22:38:36.740 INFO:teuthology.orchestra.run.smithi139.stdout:(159/174): NetworkManager-tui-1.47.2-1.el9.x86_ 1.5 MB/s | 248 kB 00:00 2024-03-18T22:38:36.790 INFO:teuthology.orchestra.run.smithi139.stdout:(160/174): elfutils-libs-0.190-2.el9.x86_64.rpm 3.3 MB/s | 256 kB 00:00 2024-03-18T22:38:36.815 INFO:teuthology.orchestra.run.smithi139.stdout:(161/174): libacl-2.3.1-4.el9.x86_64.rpm 306 kB/s | 23 kB 00:00 2024-03-18T22:38:36.866 INFO:teuthology.orchestra.run.smithi139.stdout:(162/174): libblkid-2.37.4-18.el9.x86_64.rpm 1.4 MB/s | 108 kB 00:00 2024-03-18T22:38:36.949 INFO:teuthology.orchestra.run.smithi139.stdout:(163/174): libmount-2.37.4-18.el9.x86_64.rpm 1.6 MB/s | 136 kB 00:00 2024-03-18T22:38:36.975 INFO:teuthology.orchestra.run.smithi139.stdout:(164/174): libfdisk-2.37.4-18.el9.x86_64.rpm 974 kB/s | 155 kB 00:00 2024-03-18T22:38:37.042 INFO:teuthology.orchestra.run.smithi139.stdout:(165/174): libsmartcols-2.37.4-18.el9.x86_64.rp 695 kB/s | 64 kB 00:00 2024-03-18T22:38:37.067 INFO:teuthology.orchestra.run.smithi139.stdout:(166/174): libuuid-2.37.4-18.el9.x86_64.rpm 313 kB/s | 29 kB 00:00 2024-03-18T22:38:37.100 INFO:teuthology.orchestra.run.smithi139.stdout:(167/174): elfutils-libelf-0.190-2.el9.x86_64.r 468 kB/s | 192 kB 00:00 2024-03-18T22:38:37.376 INFO:teuthology.orchestra.run.smithi139.stdout:(168/174): openssl-3.0.7-27.el9.x86_64.rpm 3.6 MB/s | 1.2 MB 00:00 2024-03-18T22:38:37.601 INFO:teuthology.orchestra.run.smithi139.stdout:(169/174): util-linux-core-2.37.4-18.el9.x86_64 2.0 MB/s | 465 kB 00:00 2024-03-18T22:38:37.651 INFO:teuthology.orchestra.run.smithi139.stdout:(170/174): util-linux-2.37.4-18.el9.x86_64.rpm 4.2 MB/s | 2.3 MB 00:00 2024-03-18T22:38:37.735 INFO:teuthology.orchestra.run.smithi139.stdout:(171/174): util-linux-user-2.37.4-18.el9.x86_64 233 kB/s | 31 kB 00:00 2024-03-18T22:38:37.818 INFO:teuthology.orchestra.run.smithi139.stdout:(172/174): openssl-libs-3.0.7-27.el9.x86_64.rpm 2.9 MB/s | 2.2 MB 00:00 2024-03-18T22:38:37.852 INFO:teuthology.orchestra.run.smithi139.stdout:(173/174): xfsprogs-6.3.0-1.el9.x86_64.rpm 5.3 MB/s | 1.1 MB 00:00 2024-03-18T22:38:38.160 INFO:teuthology.orchestra.run.smithi139.stdout:(174/174): libacl-devel-2.3.1-4.el9.x86_64.rpm 175 kB/s | 74 kB 00:00 2024-03-18T22:38:38.164 INFO:teuthology.orchestra.run.smithi139.stdout:-------------------------------------------------------------------------------- 2024-03-18T22:38:38.165 INFO:teuthology.orchestra.run.smithi139.stdout:Total 11 MB/s | 224 MB 00:20 2024-03-18T22:38:38.632 INFO:teuthology.orchestra.run.smithi139.stdout:Running transaction check 2024-03-18T22:38:38.906 INFO:teuthology.orchestra.run.smithi139.stdout:Transaction check succeeded. 2024-03-18T22:38:38.906 INFO:teuthology.orchestra.run.smithi139.stdout:Running transaction test 2024-03-18T22:38:40.630 INFO:teuthology.orchestra.run.smithi139.stdout:Transaction test succeeded. 2024-03-18T22:38:40.631 INFO:teuthology.orchestra.run.smithi139.stdout:Running transaction 2024-03-18T22:38:47.507 INFO:teuthology.orchestra.run.smithi139.stdout: Preparing : 1/1 2024-03-18T22:38:47.614 INFO:teuthology.orchestra.run.smithi139.stdout: Upgrading : openssl-libs-1:3.0.7-27.el9.x86_64 1/198 2024-03-18T22:38:47.639 INFO:teuthology.orchestra.run.smithi139.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/198 2024-03-18T22:38:47.670 INFO:teuthology.orchestra.run.smithi139.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-18T22:38:47.679 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-18T22:38:47.719 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/198 2024-03-18T22:38:47.773 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 5/198 2024-03-18T22:38:47.807 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/198 2024-03-18T22:38:47.833 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 7/198 2024-03-18T22:38:47.875 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 8/198 2024-03-18T22:38:48.196 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 9/198 2024-03-18T22:38:48.242 INFO:teuthology.orchestra.run.smithi139.stdout: Upgrading : librados2-2:19.0.0-2206.gb6ae2491.el9.x86_64 10/198 2024-03-18T22:38:48.362 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: librados2-2:19.0.0-2206.gb6ae2491.el9.x86_64 10/198 2024-03-18T22:38:48.401 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : libcephfs2-2:19.0.0-2206.gb6ae2491.el9.x86_64 11/198 2024-03-18T22:38:48.492 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: libcephfs2-2:19.0.0-2206.gb6ae2491.el9.x86_64 11/198 2024-03-18T22:38:48.521 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-rados-2:19.0.0-2206.gb6ae2491.el9.x86_64 12/198 2024-03-18T22:38:48.557 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : postgresql-private-libs-13.11-1.el9.x86_64 13/198 2024-03-18T22:38:48.732 INFO:teuthology.orchestra.run.smithi139.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 14/198 2024-03-18T22:38:48.777 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : postgresql-13.11-1.el9.x86_64 15/198 2024-03-18T22:38:48.806 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : libcephsqlite-2:19.0.0-2206.gb6ae2491.el9.x86_64 16/198 2024-03-18T22:38:48.908 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2206.gb6ae2491.el9.x86_64 16/198 2024-03-18T22:38:48.931 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : libradosstriper1-2:19.0.0-2206.gb6ae2491.el9.x86 17/198 2024-03-18T22:38:48.995 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2206.gb6ae2491.el9.x86 17/198 2024-03-18T22:38:49.051 INFO:teuthology.orchestra.run.smithi139.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/198 2024-03-18T22:38:49.074 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : qatlib-service-23.11.0-1.el9.x86_64 19/198 2024-03-18T22:38:49.090 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: qatlib-service-23.11.0-1.el9.x86_64 19/198 2024-03-18T22:38:49.282 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: qatlib-23.11.0-1.el9.x86_64 20/198 2024-03-18T22:38:49.320 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : qatlib-23.11.0-1.el9.x86_64 20/198 2024-03-18T22:38:49.352 INFO:teuthology.orchestra.run.smithi139.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 21/198 2024-03-18T22:38:49.450 INFO:teuthology.orchestra.run.smithi139.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 22/198 2024-03-18T22:38:49.554 INFO:teuthology.orchestra.run.smithi139.stdout: Upgrading : NetworkManager-libnm-1:1.47.2-1.el9.x86_64 23/198 2024-03-18T22:38:49.615 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.2-1.el9.x86_64 23/198 2024-03-18T22:38:49.692 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-18T22:38:49.732 INFO:teuthology.orchestra.run.smithi139.stdout: Upgrading : NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-18T22:38:49.939 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-18T22:38:50.046 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : re2-1:20211101-3.el9.x86_64 25/198 2024-03-18T22:38:50.140 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 26/198 2024-03-18T22:38:50.187 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 27/198 2024-03-18T22:38:50.214 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : liboath-2.6.7-2.el9.x86_64 28/198 2024-03-18T22:38:50.249 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/198 2024-03-18T22:38:50.323 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/198 2024-03-18T22:38:50.377 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 31/198 2024-03-18T22:38:50.415 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 32/198 2024-03-18T22:38:50.479 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 33/198 2024-03-18T22:38:50.514 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 34/198 2024-03-18T22:38:50.558 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 35/198 2024-03-18T22:38:50.590 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 36/198 2024-03-18T22:38:50.618 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : lua-5.4.4-4.el9.x86_64 37/198 2024-03-18T22:38:50.688 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 38/198 2024-03-18T22:38:50.723 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-ceph-common-2:19.0.0-2206.gb6ae2491.el9. 39/198 2024-03-18T22:38:50.767 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-ceph-argparse-2:19.0.0-2206.gb6ae2491.el 40/198 2024-03-18T22:38:50.795 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-cephfs-2:19.0.0-2206.gb6ae2491.el9.x86_6 41/198 2024-03-18T22:38:50.865 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 42/198 2024-03-18T22:38:50.911 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : luarocks-3.9.2-1.el9.noarch 43/198 2024-03-18T22:38:50.984 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 44/198 2024-03-18T22:38:51.127 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 45/198 2024-03-18T22:38:51.180 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 46/198 2024-03-18T22:38:51.707 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-rsa-4.9-2.el9.noarch 47/198 2024-03-18T22:38:51.783 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 48/198 2024-03-18T22:38:51.861 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 49/198 2024-03-18T22:38:51.902 INFO:teuthology.orchestra.run.smithi139.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-18T22:38:51.910 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-18T22:38:51.942 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : qatzip-libs-1.1.2-2.el9.x86_64 51/198 2024-03-18T22:38:51.983 INFO:teuthology.orchestra.run.smithi139.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 52/198 2024-03-18T22:38:52.021 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : librados-devel-2:19.0.0-2206.gb6ae2491.el9.x86_6 53/198 2024-03-18T22:38:52.051 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 54/198 2024-03-18T22:38:52.278 INFO:teuthology.orchestra.run.smithi139.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 55/198 2024-03-18T22:38:52.452 INFO:teuthology.orchestra.run.smithi139.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 56/198 2024-03-18T22:38:52.536 INFO:teuthology.orchestra.run.smithi139.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 57/198 2024-03-18T22:38:52.580 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 58/198 2024-03-18T22:38:52.712 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 59/198 2024-03-18T22:38:52.767 INFO:teuthology.orchestra.run.smithi139.stdout: Upgrading : openssl-1:3.0.7-27.el9.x86_64 60/198 2024-03-18T22:38:52.821 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 61/198 2024-03-18T22:38:52.894 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 62/198 2024-03-18T22:38:52.925 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 63/198 2024-03-18T22:38:52.962 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 64/198 2024-03-18T22:38:53.009 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 65/198 2024-03-18T22:38:53.044 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 66/198 2024-03-18T22:38:53.088 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 67/198 2024-03-18T22:38:53.138 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 68/198 2024-03-18T22:38:53.274 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 69/198 2024-03-18T22:38:53.319 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 70/198 2024-03-18T22:38:53.349 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 71/198 2024-03-18T22:38:53.391 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 72/198 2024-03-18T22:38:53.486 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 73/198 2024-03-18T22:38:53.524 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-google-auth-1:2.28.0-1.el9.noarch 74/198 2024-03-18T22:38:53.564 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 75/198 2024-03-18T22:38:53.589 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 76/198 2024-03-18T22:38:53.614 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 77/198 2024-03-18T22:38:53.686 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 78/198 2024-03-18T22:38:53.815 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 79/198 2024-03-18T22:38:53.917 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 80/198 2024-03-18T22:38:53.955 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 81/198 2024-03-18T22:38:54.004 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-ply-3.11-14.el9.noarch 82/198 2024-03-18T22:38:54.059 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 83/198 2024-03-18T22:38:54.245 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 84/198 2024-03-18T22:38:54.293 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 85/198 2024-03-18T22:38:54.434 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 86/198 2024-03-18T22:38:54.474 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-asyncssh-2.13.2-4.el9.noarch 87/198 2024-03-18T22:38:54.506 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 88/198 2024-03-18T22:38:54.537 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : fuse-2.9.9-15.el9.x86_64 89/198 2024-03-18T22:38:54.565 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 90/198 2024-03-18T22:38:54.599 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 91/198 2024-03-18T22:38:54.673 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : uuid-1.6.2-55.el9.x86_64 92/198 2024-03-18T22:38:54.712 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 93/198 2024-03-18T22:38:54.742 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 94/198 2024-03-18T22:38:54.819 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 95/198 2024-03-18T22:38:54.860 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 96/198 2024-03-18T22:38:54.894 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 97/198 2024-03-18T22:38:54.935 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 98/198 2024-03-18T22:38:54.970 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 99/198 2024-03-18T22:38:55.060 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 100/198 2024-03-18T22:38:55.158 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 101/198 2024-03-18T22:38:55.201 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 102/198 2024-03-18T22:38:55.232 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 103/198 2024-03-18T22:38:55.279 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 104/198 2024-03-18T22:38:55.378 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-packaging-20.9-5.el9.noarch 105/198 2024-03-18T22:38:55.426 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 106/198 2024-03-18T22:38:56.158 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 107/198 2024-03-18T22:38:56.214 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 108/198 2024-03-18T22:38:56.248 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 109/198 2024-03-18T22:38:56.296 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 110/198 2024-03-18T22:38:56.385 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 111/198 2024-03-18T22:38:56.423 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 112/198 2024-03-18T22:38:56.602 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : openblas-0.3.21-2.el9.x86_64 113/198 2024-03-18T22:38:56.629 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 114/198 2024-03-18T22:38:56.708 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 115/198 2024-03-18T22:38:57.363 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 116/198 2024-03-18T22:38:57.477 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 117/198 2024-03-18T22:38:58.710 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 118/198 2024-03-18T22:38:58.804 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 119/198 2024-03-18T22:38:58.835 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 120/198 2024-03-18T22:38:58.866 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : jq-1.6-16.el9.x86_64 121/198 2024-03-18T22:38:58.894 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 122/198 2024-03-18T22:38:59.207 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 123/198 2024-03-18T22:38:59.261 INFO:teuthology.orchestra.run.smithi139.stdout: Upgrading : librbd1-2:19.0.0-2206.gb6ae2491.el9.x86_64 124/198 2024-03-18T22:38:59.357 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: librbd1-2:19.0.0-2206.gb6ae2491.el9.x86_64 124/198 2024-03-18T22:38:59.387 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-rbd-2:19.0.0-2206.gb6ae2491.el9.x86_64 125/198 2024-03-18T22:38:59.423 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 126/198 2024-03-18T22:38:59.884 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 127/198 2024-03-18T22:38:59.910 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : librgw2-2:19.0.0-2206.gb6ae2491.el9.x86_64 128/198 2024-03-18T22:38:59.964 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: librgw2-2:19.0.0-2206.gb6ae2491.el9.x86_64 128/198 2024-03-18T22:38:59.986 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-rgw-2:19.0.0-2206.gb6ae2491.el9.x86_64 129/198 2024-03-18T22:39:02.226 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: ceph-common-2:19.0.0-2206.gb6ae2491.el9.x86_64 130/198 2024-03-18T22:39:02.258 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : ceph-common-2:19.0.0-2206.gb6ae2491.el9.x86_64 130/198 2024-03-18T22:39:02.797 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: ceph-common-2:19.0.0-2206.gb6ae2491.el9.x86_64 130/198 2024-03-18T22:39:02.844 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : ceph-base-2:19.0.0-2206.gb6ae2491.el9.x86_64 131/198 2024-03-18T22:39:02.926 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: ceph-base-2:19.0.0-2206.gb6ae2491.el9.x86_64 131/198 2024-03-18T22:39:02.926 INFO:teuthology.orchestra.run.smithi139.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-03-18T22:39:02.926 INFO:teuthology.orchestra.run.smithi139.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-03-18T22:39:02.926 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-03-18T22:39:02.954 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : ceph-selinux-2:19.0.0-2206.gb6ae2491.el9.x86_64 132/198 2024-03-18T22:39:14.781 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2206.gb6ae2491.el9.x86_64 132/198 2024-03-18T22:39:14.781 INFO:teuthology.orchestra.run.smithi139.stdout:skipping the directory /sys 2024-03-18T22:39:14.781 INFO:teuthology.orchestra.run.smithi139.stdout:skipping the directory /proc 2024-03-18T22:39:14.781 INFO:teuthology.orchestra.run.smithi139.stdout:skipping the directory /mnt 2024-03-18T22:39:14.782 INFO:teuthology.orchestra.run.smithi139.stdout:skipping the directory /var/tmp 2024-03-18T22:39:14.782 INFO:teuthology.orchestra.run.smithi139.stdout:skipping the directory /home 2024-03-18T22:39:14.782 INFO:teuthology.orchestra.run.smithi139.stdout:skipping the directory /root 2024-03-18T22:39:14.782 INFO:teuthology.orchestra.run.smithi139.stdout:skipping the directory /tmp 2024-03-18T22:39:14.782 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-03-18T22:39:14.958 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : ceph-volume-2:19.0.0-2206.gb6ae2491.el9.noarch 133/198 2024-03-18T22:39:14.973 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: ceph-volume-2:19.0.0-2206.gb6ae2491.el9.noarch 133/198 2024-03-18T22:39:14.973 INFO:teuthology.orchestra.run.smithi139.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-18T22:39:14.973 INFO:teuthology.orchestra.run.smithi139.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-03-18T22:39:14.973 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-03-18T22:39:16.470 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : ceph-osd-2:19.0.0-2206.gb6ae2491.el9.x86_64 134/198 2024-03-18T22:39:16.496 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: ceph-osd-2:19.0.0-2206.gb6ae2491.el9.x86_64 134/198 2024-03-18T22:39:16.496 INFO:teuthology.orchestra.run.smithi139.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-18T22:39:16.496 INFO:teuthology.orchestra.run.smithi139.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-03-18T22:39:16.496 INFO:teuthology.orchestra.run.smithi139.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-18T22:39:16.496 INFO:teuthology.orchestra.run.smithi139.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-18T22:39:16.496 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-03-18T22:39:16.721 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : ceph-mds-2:19.0.0-2206.gb6ae2491.el9.x86_64 135/198 2024-03-18T22:39:16.743 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: ceph-mds-2:19.0.0-2206.gb6ae2491.el9.x86_64 135/198 2024-03-18T22:39:16.743 INFO:teuthology.orchestra.run.smithi139.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-18T22:39:16.743 INFO:teuthology.orchestra.run.smithi139.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-03-18T22:39:16.743 INFO:teuthology.orchestra.run.smithi139.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-18T22:39:16.743 INFO:teuthology.orchestra.run.smithi139.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-18T22:39:16.744 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-03-18T22:39:17.188 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : ceph-mon-2:19.0.0-2206.gb6ae2491.el9.x86_64 136/198 2024-03-18T22:39:17.211 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: ceph-mon-2:19.0.0-2206.gb6ae2491.el9.x86_64 136/198 2024-03-18T22:39:17.211 INFO:teuthology.orchestra.run.smithi139.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-18T22:39:17.211 INFO:teuthology.orchestra.run.smithi139.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-03-18T22:39:17.211 INFO:teuthology.orchestra.run.smithi139.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-18T22:39:17.211 INFO:teuthology.orchestra.run.smithi139.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-18T22:39:17.211 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-03-18T22:39:17.307 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : automake-1.16.2-8.el9.noarch 137/198 2024-03-18T22:39:17.624 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: cephadm-2:19.0.0-2206.gb6ae2491.el9.noarch 138/198 2024-03-18T22:39:17.655 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : cephadm-2:19.0.0-2206.gb6ae2491.el9.noarch 138/198 2024-03-18T22:39:17.712 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2206.gb6ae2491.e 139/198 2024-03-18T22:39:17.773 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2206.gb6ae2491. 140/198 2024-03-18T22:39:17.803 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2206.gb6ae2491.el9.noa 141/198 2024-03-18T22:39:18.038 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2206.gb6ae2491.el9.noa 141/198 2024-03-18T22:39:18.070 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2206.gb6ae2491.el9.n 142/198 2024-03-18T22:39:19.042 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2206.gb6ae2491.el9.n 142/198 2024-03-18T22:39:19.062 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2206.gb6a 143/198 2024-03-18T22:39:19.165 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2206.gb6a 143/198 2024-03-18T22:39:19.354 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2206.gb6ae2491.el 144/198 2024-03-18T22:39:19.381 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : ceph-mgr-2:19.0.0-2206.gb6ae2491.el9.x86_64 145/198 2024-03-18T22:39:19.404 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2206.gb6ae2491.el9.x86_64 145/198 2024-03-18T22:39:19.405 INFO:teuthology.orchestra.run.smithi139.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-18T22:39:19.405 INFO:teuthology.orchestra.run.smithi139.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-03-18T22:39:19.405 INFO:teuthology.orchestra.run.smithi139.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-18T22:39:19.405 INFO:teuthology.orchestra.run.smithi139.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-18T22:39:19.405 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-03-18T22:39:19.448 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : ceph-mgr-rook-2:19.0.0-2206.gb6ae2491.el9.noarch 146/198 2024-03-18T22:39:19.459 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2206.gb6ae2491.el9.noarch 146/198 2024-03-18T22:39:19.506 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : ceph-2:19.0.0-2206.gb6ae2491.el9.x86_64 147/198 2024-03-18T22:39:20.382 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : libtool-2.4.6-45.el9.x86_64 148/198 2024-03-18T22:39:20.409 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : ceph-radosgw-2:19.0.0-2206.gb6ae2491.el9.x86_64 149/198 2024-03-18T22:39:20.431 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2206.gb6ae2491.el9.x86_64 149/198 2024-03-18T22:39:20.432 INFO:teuthology.orchestra.run.smithi139.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-18T22:39:20.432 INFO:teuthology.orchestra.run.smithi139.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-03-18T22:39:20.432 INFO:teuthology.orchestra.run.smithi139.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-18T22:39:20.432 INFO:teuthology.orchestra.run.smithi139.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-18T22:39:20.432 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-03-18T22:39:20.476 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2206.gb6ae2 150/198 2024-03-18T22:39:20.498 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2206.gb6ae2 150/198 2024-03-18T22:39:20.498 INFO:teuthology.orchestra.run.smithi139.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-18T22:39:20.498 INFO:teuthology.orchestra.run.smithi139.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-03-18T22:39:20.498 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-03-18T22:39:20.543 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : cephfs-mirror-2:19.0.0-2206.gb6ae2491.el9.x86_64 151/198 2024-03-18T22:39:20.565 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2206.gb6ae2491.el9.x86_64 151/198 2024-03-18T22:39:20.565 INFO:teuthology.orchestra.run.smithi139.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-18T22:39:20.566 INFO:teuthology.orchestra.run.smithi139.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-03-18T22:39:20.566 INFO:teuthology.orchestra.run.smithi139.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-18T22:39:20.566 INFO:teuthology.orchestra.run.smithi139.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-18T22:39:20.566 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-03-18T22:39:20.877 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : rbd-mirror-2:19.0.0-2206.gb6ae2491.el9.x86_64 152/198 2024-03-18T22:39:20.900 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2206.gb6ae2491.el9.x86_64 152/198 2024-03-18T22:39:20.900 INFO:teuthology.orchestra.run.smithi139.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-18T22:39:20.900 INFO:teuthology.orchestra.run.smithi139.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-03-18T22:39:20.900 INFO:teuthology.orchestra.run.smithi139.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-18T22:39:20.900 INFO:teuthology.orchestra.run.smithi139.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-18T22:39:20.900 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-03-18T22:39:25.348 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : ceph-test-2:19.0.0-2206.gb6ae2491.el9.x86_64 153/198 2024-03-18T22:39:25.389 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : rbd-fuse-2:19.0.0-2206.gb6ae2491.el9.x86_64 154/198 2024-03-18T22:39:25.428 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : rbd-nbd-2:19.0.0-2206.gb6ae2491.el9.x86_64 155/198 2024-03-18T22:39:25.455 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 156/198 2024-03-18T22:39:25.638 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : cephfs-shell-2:19.0.0-2206.gb6ae2491.el9.x86_64 157/198 2024-03-18T22:39:25.754 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : postgresql-contrib-13.11-1.el9.x86_64 158/198 2024-03-18T22:39:25.794 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : ceph-fuse-2:19.0.0-2206.gb6ae2491.el9.x86_64 159/198 2024-03-18T22:39:25.842 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 160/198 2024-03-18T22:39:25.906 INFO:teuthology.orchestra.run.smithi139.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 161/198 2024-03-18T22:39:25.946 INFO:teuthology.orchestra.run.smithi139.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 162/198 2024-03-18T22:39:25.971 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : libcephfs-devel-2:19.0.0-2206.gb6ae2491.el9.x86_ 163/198 2024-03-18T22:39:26.028 INFO:teuthology.orchestra.run.smithi139.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 164/198 2024-03-18T22:39:26.058 INFO:teuthology.orchestra.run.smithi139.stdout: Upgrading : NetworkManager-team-1:1.47.2-1.el9.x86_64 165/198 2024-03-18T22:39:26.112 INFO:teuthology.orchestra.run.smithi139.stdout: Upgrading : NetworkManager-tui-1:1.47.2-1.el9.x86_64 166/198 2024-03-18T22:39:26.181 INFO:teuthology.orchestra.run.smithi139.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 167/198 2024-03-18T22:39:26.214 INFO:teuthology.orchestra.run.smithi139.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 168/198 2024-03-18T22:39:26.731 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-18T22:39:26.770 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-18T22:39:26.789 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-18T22:39:27.629 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : cephfs-top-2:19.0.0-2206.gb6ae2491.el9.noarch 170/198 2024-03-18T22:39:27.706 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 171/198 2024-03-18T22:39:27.747 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 172/198 2024-03-18T22:39:27.828 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : flex-2.6.4-9.el9.x86_64 173/198 2024-03-18T22:39:27.868 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : bison-3.7.4-5.el9.x86_64 174/198 2024-03-18T22:39:27.893 INFO:teuthology.orchestra.run.smithi139.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 175/198 2024-03-18T22:39:27.918 INFO:teuthology.orchestra.run.smithi139.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 176/198 2024-03-18T22:39:27.945 INFO:teuthology.orchestra.run.smithi139.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 177/198 2024-03-18T22:39:27.945 INFO:teuthology.orchestra.run.smithi139.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-18T22:39:28.033 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-18T22:39:28.062 INFO:teuthology.orchestra.run.smithi139.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 179/198 2024-03-18T22:39:28.062 INFO:teuthology.orchestra.run.smithi139.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-18T22:39:28.138 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-18T22:39:28.164 INFO:teuthology.orchestra.run.smithi139.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 181/198 2024-03-18T22:39:28.198 INFO:teuthology.orchestra.run.smithi139.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 182/198 2024-03-18T22:39:28.223 INFO:teuthology.orchestra.run.smithi139.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 183/198 2024-03-18T22:39:28.248 INFO:teuthology.orchestra.run.smithi139.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 184/198 2024-03-18T22:39:28.281 INFO:teuthology.orchestra.run.smithi139.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 185/198 2024-03-18T22:39:28.321 INFO:teuthology.orchestra.run.smithi139.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 186/198 2024-03-18T22:39:28.353 INFO:teuthology.orchestra.run.smithi139.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 187/198 2024-03-18T22:39:28.378 INFO:teuthology.orchestra.run.smithi139.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 188/198 2024-03-18T22:39:28.403 INFO:teuthology.orchestra.run.smithi139.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 189/198 2024-03-18T22:39:28.428 INFO:teuthology.orchestra.run.smithi139.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 190/198 2024-03-18T22:39:28.454 INFO:teuthology.orchestra.run.smithi139.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 191/198 2024-03-18T22:39:28.460 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-18T22:39:28.460 INFO:teuthology.orchestra.run.smithi139.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-18T22:39:28.649 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-18T22:39:28.649 INFO:teuthology.orchestra.run.smithi139.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-18T22:39:28.714 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-18T22:39:28.739 INFO:teuthology.orchestra.run.smithi139.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 194/198 2024-03-18T22:39:28.764 INFO:teuthology.orchestra.run.smithi139.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 195/198 2024-03-18T22:39:28.797 INFO:teuthology.orchestra.run.smithi139.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 196/198 2024-03-18T22:39:28.824 INFO:teuthology.orchestra.run.smithi139.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 197/198 2024-03-18T22:39:28.888 INFO:teuthology.orchestra.run.smithi139.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-18T22:39:33.664 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-18T22:39:33.664 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : ceph-2:19.0.0-2206.gb6ae2491.el9.x86_64 1/198 2024-03-18T22:39:33.664 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : ceph-base-2:19.0.0-2206.gb6ae2491.el9.x86_64 2/198 2024-03-18T22:39:33.665 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : ceph-common-2:19.0.0-2206.gb6ae2491.el9.x86_64 3/198 2024-03-18T22:39:33.665 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : ceph-fuse-2:19.0.0-2206.gb6ae2491.el9.x86_64 4/198 2024-03-18T22:39:33.665 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2206.gb6ae2 5/198 2024-03-18T22:39:33.665 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : ceph-mds-2:19.0.0-2206.gb6ae2491.el9.x86_64 6/198 2024-03-18T22:39:33.665 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : ceph-mgr-2:19.0.0-2206.gb6ae2491.el9.x86_64 7/198 2024-03-18T22:39:33.665 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : ceph-mon-2:19.0.0-2206.gb6ae2491.el9.x86_64 8/198 2024-03-18T22:39:33.665 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : ceph-osd-2:19.0.0-2206.gb6ae2491.el9.x86_64 9/198 2024-03-18T22:39:33.665 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : ceph-radosgw-2:19.0.0-2206.gb6ae2491.el9.x86_64 10/198 2024-03-18T22:39:33.665 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : ceph-selinux-2:19.0.0-2206.gb6ae2491.el9.x86_64 11/198 2024-03-18T22:39:33.665 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : ceph-test-2:19.0.0-2206.gb6ae2491.el9.x86_64 12/198 2024-03-18T22:39:33.665 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : cephfs-mirror-2:19.0.0-2206.gb6ae2491.el9.x86_64 13/198 2024-03-18T22:39:33.665 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : cephfs-shell-2:19.0.0-2206.gb6ae2491.el9.x86_64 14/198 2024-03-18T22:39:33.665 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : libcephfs-devel-2:19.0.0-2206.gb6ae2491.el9.x86_ 15/198 2024-03-18T22:39:33.665 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : libcephfs2-2:19.0.0-2206.gb6ae2491.el9.x86_64 16/198 2024-03-18T22:39:33.666 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : libcephsqlite-2:19.0.0-2206.gb6ae2491.el9.x86_64 17/198 2024-03-18T22:39:33.666 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : librados-devel-2:19.0.0-2206.gb6ae2491.el9.x86_6 18/198 2024-03-18T22:39:33.666 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : libradosstriper1-2:19.0.0-2206.gb6ae2491.el9.x86 19/198 2024-03-18T22:39:33.666 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : librgw2-2:19.0.0-2206.gb6ae2491.el9.x86_64 20/198 2024-03-18T22:39:33.666 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2206.gb6ae2491.el 21/198 2024-03-18T22:39:33.666 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-ceph-common-2:19.0.0-2206.gb6ae2491.el9. 22/198 2024-03-18T22:39:33.666 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-cephfs-2:19.0.0-2206.gb6ae2491.el9.x86_6 23/198 2024-03-18T22:39:33.666 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-rados-2:19.0.0-2206.gb6ae2491.el9.x86_64 24/198 2024-03-18T22:39:33.666 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-rbd-2:19.0.0-2206.gb6ae2491.el9.x86_64 25/198 2024-03-18T22:39:33.666 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-rgw-2:19.0.0-2206.gb6ae2491.el9.x86_64 26/198 2024-03-18T22:39:33.666 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : rbd-fuse-2:19.0.0-2206.gb6ae2491.el9.x86_64 27/198 2024-03-18T22:39:33.666 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : rbd-mirror-2:19.0.0-2206.gb6ae2491.el9.x86_64 28/198 2024-03-18T22:39:33.667 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : rbd-nbd-2:19.0.0-2206.gb6ae2491.el9.x86_64 29/198 2024-03-18T22:39:33.667 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2206.gb6ae2491. 30/198 2024-03-18T22:39:33.667 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2206.gb6ae2491.el9.noa 31/198 2024-03-18T22:39:33.667 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2206.gb6ae2491.el9.n 32/198 2024-03-18T22:39:33.667 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2206.gb6a 33/198 2024-03-18T22:39:33.667 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2206.gb6ae2491.el 34/198 2024-03-18T22:39:33.667 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2206.gb6ae2491.el9.noarch 35/198 2024-03-18T22:39:33.667 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2206.gb6ae2491.e 36/198 2024-03-18T22:39:33.667 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : ceph-volume-2:19.0.0-2206.gb6ae2491.el9.noarch 37/198 2024-03-18T22:39:33.667 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : cephadm-2:19.0.0-2206.gb6ae2491.el9.noarch 38/198 2024-03-18T22:39:33.667 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : cephfs-top-2:19.0.0-2206.gb6ae2491.el9.noarch 39/198 2024-03-18T22:39:33.667 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : automake-1.16.2-8.el9.noarch 40/198 2024-03-18T22:39:33.667 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : bison-3.7.4-5.el9.x86_64 41/198 2024-03-18T22:39:33.668 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 42/198 2024-03-18T22:39:33.668 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : flex-2.6.4-9.el9.x86_64 43/198 2024-03-18T22:39:33.668 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 44/198 2024-03-18T22:39:33.668 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 45/198 2024-03-18T22:39:33.668 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 46/198 2024-03-18T22:39:33.668 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 47/198 2024-03-18T22:39:33.668 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 48/198 2024-03-18T22:39:33.668 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 49/198 2024-03-18T22:39:33.668 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 50/198 2024-03-18T22:39:33.668 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 51/198 2024-03-18T22:39:33.668 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 52/198 2024-03-18T22:39:33.668 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 53/198 2024-03-18T22:39:33.668 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : lua-5.4.4-4.el9.x86_64 54/198 2024-03-18T22:39:33.669 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 55/198 2024-03-18T22:39:33.669 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 56/198 2024-03-18T22:39:33.669 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 57/198 2024-03-18T22:39:33.669 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : postgresql-13.11-1.el9.x86_64 58/198 2024-03-18T22:39:33.669 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : postgresql-contrib-13.11-1.el9.x86_64 59/198 2024-03-18T22:39:33.669 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : postgresql-private-libs-13.11-1.el9.x86_64 60/198 2024-03-18T22:39:33.669 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : postgresql-server-13.11-1.el9.x86_64 61/198 2024-03-18T22:39:33.669 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 62/198 2024-03-18T22:39:33.669 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 63/198 2024-03-18T22:39:33.669 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 64/198 2024-03-18T22:39:33.669 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 65/198 2024-03-18T22:39:33.669 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 66/198 2024-03-18T22:39:33.669 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 67/198 2024-03-18T22:39:33.669 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 68/198 2024-03-18T22:39:33.669 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 69/198 2024-03-18T22:39:33.670 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 70/198 2024-03-18T22:39:33.670 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 71/198 2024-03-18T22:39:33.670 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 72/198 2024-03-18T22:39:33.670 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 73/198 2024-03-18T22:39:33.670 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 74/198 2024-03-18T22:39:33.670 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 75/198 2024-03-18T22:39:33.670 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 76/198 2024-03-18T22:39:33.670 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 77/198 2024-03-18T22:39:33.670 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 78/198 2024-03-18T22:39:33.670 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 79/198 2024-03-18T22:39:33.670 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 80/198 2024-03-18T22:39:33.670 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 81/198 2024-03-18T22:39:33.670 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 82/198 2024-03-18T22:39:33.671 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 83/198 2024-03-18T22:39:33.671 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 84/198 2024-03-18T22:39:33.671 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 85/198 2024-03-18T22:39:33.671 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 86/198 2024-03-18T22:39:33.671 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 87/198 2024-03-18T22:39:33.671 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 88/198 2024-03-18T22:39:33.671 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-ply-3.11-14.el9.noarch 89/198 2024-03-18T22:39:33.671 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 90/198 2024-03-18T22:39:33.671 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 91/198 2024-03-18T22:39:33.671 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : jq-1.6-16.el9.x86_64 92/198 2024-03-18T22:39:33.671 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 93/198 2024-03-18T22:39:33.671 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 94/198 2024-03-18T22:39:33.672 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 95/198 2024-03-18T22:39:33.672 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 96/198 2024-03-18T22:39:33.672 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 97/198 2024-03-18T22:39:33.672 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 98/198 2024-03-18T22:39:33.672 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 99/198 2024-03-18T22:39:33.672 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 100/198 2024-03-18T22:39:33.672 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : qatlib-23.11.0-1.el9.x86_64 101/198 2024-03-18T22:39:33.672 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : qatlib-service-23.11.0-1.el9.x86_64 102/198 2024-03-18T22:39:33.672 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : qatzip-libs-1.1.2-2.el9.x86_64 103/198 2024-03-18T22:39:33.672 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 104/198 2024-03-18T22:39:33.672 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/198 2024-03-18T22:39:33.672 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/198 2024-03-18T22:39:33.672 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/198 2024-03-18T22:39:33.673 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/198 2024-03-18T22:39:33.673 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/198 2024-03-18T22:39:33.673 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/198 2024-03-18T22:39:33.673 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/198 2024-03-18T22:39:33.673 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/198 2024-03-18T22:39:33.673 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/198 2024-03-18T22:39:33.673 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/198 2024-03-18T22:39:33.673 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/198 2024-03-18T22:39:33.673 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-asyncssh-2.13.2-4.el9.noarch 116/198 2024-03-18T22:39:33.673 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/198 2024-03-18T22:39:33.673 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/198 2024-03-18T22:39:33.673 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/198 2024-03-18T22:39:33.673 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/198 2024-03-18T22:39:33.673 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/198 2024-03-18T22:39:33.674 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/198 2024-03-18T22:39:33.674 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/198 2024-03-18T22:39:33.674 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-google-auth-1:2.28.0-1.el9.noarch 124/198 2024-03-18T22:39:33.674 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/198 2024-03-18T22:39:33.674 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/198 2024-03-18T22:39:33.674 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/198 2024-03-18T22:39:33.674 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/198 2024-03-18T22:39:33.674 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/198 2024-03-18T22:39:33.674 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/198 2024-03-18T22:39:33.674 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/198 2024-03-18T22:39:33.674 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/198 2024-03-18T22:39:33.674 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/198 2024-03-18T22:39:33.675 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/198 2024-03-18T22:39:33.675 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/198 2024-03-18T22:39:33.675 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/198 2024-03-18T22:39:33.675 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/198 2024-03-18T22:39:33.675 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/198 2024-03-18T22:39:33.675 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/198 2024-03-18T22:39:33.675 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/198 2024-03-18T22:39:33.675 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/198 2024-03-18T22:39:33.675 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/198 2024-03-18T22:39:33.675 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/198 2024-03-18T22:39:33.675 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/198 2024-03-18T22:39:33.675 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/198 2024-03-18T22:39:33.675 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/198 2024-03-18T22:39:33.675 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/198 2024-03-18T22:39:33.676 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/198 2024-03-18T22:39:33.676 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/198 2024-03-18T22:39:33.676 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/198 2024-03-18T22:39:33.676 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : librados2-2:19.0.0-2206.gb6ae2491.el9.x86_64 151/198 2024-03-18T22:39:33.676 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/198 2024-03-18T22:39:33.676 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : librbd1-2:19.0.0-2206.gb6ae2491.el9.x86_64 153/198 2024-03-18T22:39:33.676 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/198 2024-03-18T22:39:33.676 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 155/198 2024-03-18T22:39:33.676 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 156/198 2024-03-18T22:39:33.676 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : NetworkManager-1:1.47.2-1.el9.x86_64 157/198 2024-03-18T22:39:33.676 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 158/198 2024-03-18T22:39:33.676 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : NetworkManager-libnm-1:1.47.2-1.el9.x86_64 159/198 2024-03-18T22:39:33.676 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 160/198 2024-03-18T22:39:33.676 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : NetworkManager-team-1:1.47.2-1.el9.x86_64 161/198 2024-03-18T22:39:33.676 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 162/198 2024-03-18T22:39:33.677 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : NetworkManager-tui-1:1.47.2-1.el9.x86_64 163/198 2024-03-18T22:39:33.677 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 164/198 2024-03-18T22:39:33.677 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : acl-2.3.1-4.el9.x86_64 165/198 2024-03-18T22:39:33.677 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : acl-2.3.1-3.el9.x86_64 166/198 2024-03-18T22:39:33.677 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 167/198 2024-03-18T22:39:33.677 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 168/198 2024-03-18T22:39:33.677 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 169/198 2024-03-18T22:39:33.677 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 170/198 2024-03-18T22:39:33.677 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 171/198 2024-03-18T22:39:33.677 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 172/198 2024-03-18T22:39:33.677 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 173/198 2024-03-18T22:39:33.677 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 174/198 2024-03-18T22:39:33.677 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 175/198 2024-03-18T22:39:33.677 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 176/198 2024-03-18T22:39:33.678 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 177/198 2024-03-18T22:39:33.678 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 178/198 2024-03-18T22:39:33.678 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 179/198 2024-03-18T22:39:33.678 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 180/198 2024-03-18T22:39:33.678 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 181/198 2024-03-18T22:39:33.678 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 182/198 2024-03-18T22:39:33.678 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 183/198 2024-03-18T22:39:33.678 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 184/198 2024-03-18T22:39:33.678 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 185/198 2024-03-18T22:39:33.678 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 186/198 2024-03-18T22:39:33.678 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 187/198 2024-03-18T22:39:33.678 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 188/198 2024-03-18T22:39:33.678 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 189/198 2024-03-18T22:39:33.679 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 190/198 2024-03-18T22:39:33.679 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 191/198 2024-03-18T22:39:33.679 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 192/198 2024-03-18T22:39:33.679 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 193/198 2024-03-18T22:39:33.679 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 194/198 2024-03-18T22:39:33.679 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 195/198 2024-03-18T22:39:33.679 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 196/198 2024-03-18T22:39:33.679 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 197/198 2024-03-18T22:39:36.724 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 198/198 2024-03-18T22:39:36.724 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-03-18T22:39:36.724 INFO:teuthology.orchestra.run.smithi139.stdout:Upgraded: 2024-03-18T22:39:36.724 INFO:teuthology.orchestra.run.smithi139.stdout: NetworkManager-1:1.47.2-1.el9.x86_64 2024-03-18T22:39:36.724 INFO:teuthology.orchestra.run.smithi139.stdout: NetworkManager-libnm-1:1.47.2-1.el9.x86_64 2024-03-18T22:39:36.724 INFO:teuthology.orchestra.run.smithi139.stdout: NetworkManager-team-1:1.47.2-1.el9.x86_64 2024-03-18T22:39:36.724 INFO:teuthology.orchestra.run.smithi139.stdout: NetworkManager-tui-1:1.47.2-1.el9.x86_64 2024-03-18T22:39:36.724 INFO:teuthology.orchestra.run.smithi139.stdout: acl-2.3.1-4.el9.x86_64 2024-03-18T22:39:36.725 INFO:teuthology.orchestra.run.smithi139.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-03-18T22:39:36.725 INFO:teuthology.orchestra.run.smithi139.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-03-18T22:39:36.725 INFO:teuthology.orchestra.run.smithi139.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-03-18T22:39:36.725 INFO:teuthology.orchestra.run.smithi139.stdout: libacl-2.3.1-4.el9.x86_64 2024-03-18T22:39:36.725 INFO:teuthology.orchestra.run.smithi139.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-03-18T22:39:36.725 INFO:teuthology.orchestra.run.smithi139.stdout: libblkid-2.37.4-18.el9.x86_64 2024-03-18T22:39:36.725 INFO:teuthology.orchestra.run.smithi139.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-03-18T22:39:36.725 INFO:teuthology.orchestra.run.smithi139.stdout: libmount-2.37.4-18.el9.x86_64 2024-03-18T22:39:36.725 INFO:teuthology.orchestra.run.smithi139.stdout: librados2-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-18T22:39:36.725 INFO:teuthology.orchestra.run.smithi139.stdout: librbd1-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-18T22:39:36.725 INFO:teuthology.orchestra.run.smithi139.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-03-18T22:39:36.726 INFO:teuthology.orchestra.run.smithi139.stdout: libuuid-2.37.4-18.el9.x86_64 2024-03-18T22:39:36.726 INFO:teuthology.orchestra.run.smithi139.stdout: openssl-1:3.0.7-27.el9.x86_64 2024-03-18T22:39:36.726 INFO:teuthology.orchestra.run.smithi139.stdout: openssl-libs-1:3.0.7-27.el9.x86_64 2024-03-18T22:39:36.726 INFO:teuthology.orchestra.run.smithi139.stdout: util-linux-2.37.4-18.el9.x86_64 2024-03-18T22:39:36.726 INFO:teuthology.orchestra.run.smithi139.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-03-18T22:39:36.726 INFO:teuthology.orchestra.run.smithi139.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-03-18T22:39:36.726 INFO:teuthology.orchestra.run.smithi139.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-03-18T22:39:36.726 INFO:teuthology.orchestra.run.smithi139.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-03-18T22:39:36.726 INFO:teuthology.orchestra.run.smithi139.stdout:Installed: 2024-03-18T22:39:36.726 INFO:teuthology.orchestra.run.smithi139.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-03-18T22:39:36.726 INFO:teuthology.orchestra.run.smithi139.stdout: automake-1.16.2-8.el9.noarch 2024-03-18T22:39:36.726 INFO:teuthology.orchestra.run.smithi139.stdout: bison-3.7.4-5.el9.x86_64 2024-03-18T22:39:36.726 INFO:teuthology.orchestra.run.smithi139.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-03-18T22:39:36.727 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-18T22:39:36.727 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-base-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-18T22:39:36.727 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-common-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-18T22:39:36.727 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-fuse-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-18T22:39:36.727 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-grafana-dashboards-2:19.0.0-2206.gb6ae2491.el9.noarch 2024-03-18T22:39:36.727 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-immutable-object-cache-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-18T22:39:36.727 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-mds-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-18T22:39:36.727 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-mgr-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-18T22:39:36.727 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-mgr-cephadm-2:19.0.0-2206.gb6ae2491.el9.noarch 2024-03-18T22:39:36.727 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-mgr-dashboard-2:19.0.0-2206.gb6ae2491.el9.noarch 2024-03-18T22:39:36.727 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2206.gb6ae2491.el9.noarch 2024-03-18T22:39:36.727 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-mgr-modules-core-2:19.0.0-2206.gb6ae2491.el9.noarch 2024-03-18T22:39:36.728 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-mgr-rook-2:19.0.0-2206.gb6ae2491.el9.noarch 2024-03-18T22:39:36.728 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-mon-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-18T22:39:36.728 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-osd-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-18T22:39:36.728 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-prometheus-alerts-2:19.0.0-2206.gb6ae2491.el9.noarch 2024-03-18T22:39:36.728 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-radosgw-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-18T22:39:36.728 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-selinux-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-18T22:39:36.728 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-test-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-18T22:39:36.728 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-volume-2:19.0.0-2206.gb6ae2491.el9.noarch 2024-03-18T22:39:36.728 INFO:teuthology.orchestra.run.smithi139.stdout: cephadm-2:19.0.0-2206.gb6ae2491.el9.noarch 2024-03-18T22:39:36.728 INFO:teuthology.orchestra.run.smithi139.stdout: cephfs-mirror-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-18T22:39:36.728 INFO:teuthology.orchestra.run.smithi139.stdout: cephfs-shell-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-18T22:39:36.728 INFO:teuthology.orchestra.run.smithi139.stdout: cephfs-top-2:19.0.0-2206.gb6ae2491.el9.noarch 2024-03-18T22:39:36.728 INFO:teuthology.orchestra.run.smithi139.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-03-18T22:39:36.729 INFO:teuthology.orchestra.run.smithi139.stdout: flex-2.6.4-9.el9.x86_64 2024-03-18T22:39:36.729 INFO:teuthology.orchestra.run.smithi139.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-03-18T22:39:36.729 INFO:teuthology.orchestra.run.smithi139.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-03-18T22:39:36.729 INFO:teuthology.orchestra.run.smithi139.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-03-18T22:39:36.729 INFO:teuthology.orchestra.run.smithi139.stdout: fuse-2.9.9-15.el9.x86_64 2024-03-18T22:39:36.729 INFO:teuthology.orchestra.run.smithi139.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-03-18T22:39:36.729 INFO:teuthology.orchestra.run.smithi139.stdout: grpc-data-1.46.7-10.el9.noarch 2024-03-18T22:39:36.729 INFO:teuthology.orchestra.run.smithi139.stdout: jq-1.6-16.el9.x86_64 2024-03-18T22:39:36.729 INFO:teuthology.orchestra.run.smithi139.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-03-18T22:39:36.729 INFO:teuthology.orchestra.run.smithi139.stdout: libarrow-9.0.0-10.el9.x86_64 2024-03-18T22:39:36.729 INFO:teuthology.orchestra.run.smithi139.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-03-18T22:39:36.729 INFO:teuthology.orchestra.run.smithi139.stdout: libcephfs-devel-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-18T22:39:36.729 INFO:teuthology.orchestra.run.smithi139.stdout: libcephfs2-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-18T22:39:36.729 INFO:teuthology.orchestra.run.smithi139.stdout: libcephsqlite-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-18T22:39:36.730 INFO:teuthology.orchestra.run.smithi139.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-03-18T22:39:36.730 INFO:teuthology.orchestra.run.smithi139.stdout: liboath-2.6.7-2.el9.x86_64 2024-03-18T22:39:36.730 INFO:teuthology.orchestra.run.smithi139.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-03-18T22:39:36.730 INFO:teuthology.orchestra.run.smithi139.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-03-18T22:39:36.730 INFO:teuthology.orchestra.run.smithi139.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-03-18T22:39:36.730 INFO:teuthology.orchestra.run.smithi139.stdout: librados-devel-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-18T22:39:36.730 INFO:teuthology.orchestra.run.smithi139.stdout: libradosstriper1-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-18T22:39:36.730 INFO:teuthology.orchestra.run.smithi139.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-03-18T22:39:36.730 INFO:teuthology.orchestra.run.smithi139.stdout: librgw2-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-18T22:39:36.730 INFO:teuthology.orchestra.run.smithi139.stdout: libtool-2.4.6-45.el9.x86_64 2024-03-18T22:39:36.730 INFO:teuthology.orchestra.run.smithi139.stdout: libunwind-1.6.2-1.el9.x86_64 2024-03-18T22:39:36.730 INFO:teuthology.orchestra.run.smithi139.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-03-18T22:39:36.730 INFO:teuthology.orchestra.run.smithi139.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-03-18T22:39:36.730 INFO:teuthology.orchestra.run.smithi139.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-03-18T22:39:36.731 INFO:teuthology.orchestra.run.smithi139.stdout: lua-5.4.4-4.el9.x86_64 2024-03-18T22:39:36.731 INFO:teuthology.orchestra.run.smithi139.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-03-18T22:39:36.731 INFO:teuthology.orchestra.run.smithi139.stdout: luarocks-3.9.2-1.el9.noarch 2024-03-18T22:39:36.731 INFO:teuthology.orchestra.run.smithi139.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-03-18T22:39:36.731 INFO:teuthology.orchestra.run.smithi139.stdout: openblas-0.3.21-2.el9.x86_64 2024-03-18T22:39:36.731 INFO:teuthology.orchestra.run.smithi139.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-03-18T22:39:36.731 INFO:teuthology.orchestra.run.smithi139.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-03-18T22:39:36.731 INFO:teuthology.orchestra.run.smithi139.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-03-18T22:39:36.731 INFO:teuthology.orchestra.run.smithi139.stdout: postgresql-13.11-1.el9.x86_64 2024-03-18T22:39:36.731 INFO:teuthology.orchestra.run.smithi139.stdout: postgresql-contrib-13.11-1.el9.x86_64 2024-03-18T22:39:36.731 INFO:teuthology.orchestra.run.smithi139.stdout: postgresql-private-libs-13.11-1.el9.x86_64 2024-03-18T22:39:36.731 INFO:teuthology.orchestra.run.smithi139.stdout: postgresql-server-13.11-1.el9.x86_64 2024-03-18T22:39:36.731 INFO:teuthology.orchestra.run.smithi139.stdout: protobuf-3.14.0-13.el9.x86_64 2024-03-18T22:39:36.731 INFO:teuthology.orchestra.run.smithi139.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-03-18T22:39:36.732 INFO:teuthology.orchestra.run.smithi139.stdout: python3-asyncssh-2.13.2-4.el9.noarch 2024-03-18T22:39:36.732 INFO:teuthology.orchestra.run.smithi139.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-03-18T22:39:36.732 INFO:teuthology.orchestra.run.smithi139.stdout: python3-babel-2.9.1-2.el9.noarch 2024-03-18T22:39:36.732 INFO:teuthology.orchestra.run.smithi139.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-03-18T22:39:36.732 INFO:teuthology.orchestra.run.smithi139.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-03-18T22:39:36.732 INFO:teuthology.orchestra.run.smithi139.stdout: python3-ceph-argparse-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-18T22:39:36.732 INFO:teuthology.orchestra.run.smithi139.stdout: python3-ceph-common-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-18T22:39:36.732 INFO:teuthology.orchestra.run.smithi139.stdout: python3-cephfs-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-18T22:39:36.732 INFO:teuthology.orchestra.run.smithi139.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-03-18T22:39:36.732 INFO:teuthology.orchestra.run.smithi139.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-03-18T22:39:36.732 INFO:teuthology.orchestra.run.smithi139.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-03-18T22:39:36.732 INFO:teuthology.orchestra.run.smithi139.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-03-18T22:39:36.732 INFO:teuthology.orchestra.run.smithi139.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-03-18T22:39:36.732 INFO:teuthology.orchestra.run.smithi139.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-03-18T22:39:36.733 INFO:teuthology.orchestra.run.smithi139.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-03-18T22:39:36.733 INFO:teuthology.orchestra.run.smithi139.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-03-18T22:39:36.733 INFO:teuthology.orchestra.run.smithi139.stdout: python3-google-auth-1:2.28.0-1.el9.noarch 2024-03-18T22:39:36.733 INFO:teuthology.orchestra.run.smithi139.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-03-18T22:39:36.733 INFO:teuthology.orchestra.run.smithi139.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-03-18T22:39:36.733 INFO:teuthology.orchestra.run.smithi139.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-03-18T22:39:36.733 INFO:teuthology.orchestra.run.smithi139.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-03-18T22:39:36.733 INFO:teuthology.orchestra.run.smithi139.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-03-18T22:39:36.733 INFO:teuthology.orchestra.run.smithi139.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-03-18T22:39:36.733 INFO:teuthology.orchestra.run.smithi139.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-03-18T22:39:36.733 INFO:teuthology.orchestra.run.smithi139.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-03-18T22:39:36.733 INFO:teuthology.orchestra.run.smithi139.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-03-18T22:39:36.733 INFO:teuthology.orchestra.run.smithi139.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-03-18T22:39:36.734 INFO:teuthology.orchestra.run.smithi139.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-03-18T22:39:36.734 INFO:teuthology.orchestra.run.smithi139.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-03-18T22:39:36.734 INFO:teuthology.orchestra.run.smithi139.stdout: python3-mako-1.1.4-6.el9.noarch 2024-03-18T22:39:36.734 INFO:teuthology.orchestra.run.smithi139.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-03-18T22:39:36.734 INFO:teuthology.orchestra.run.smithi139.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-03-18T22:39:36.734 INFO:teuthology.orchestra.run.smithi139.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-03-18T22:39:36.734 INFO:teuthology.orchestra.run.smithi139.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-03-18T22:39:36.734 INFO:teuthology.orchestra.run.smithi139.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-03-18T22:39:36.734 INFO:teuthology.orchestra.run.smithi139.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-03-18T22:39:36.734 INFO:teuthology.orchestra.run.smithi139.stdout: python3-packaging-20.9-5.el9.noarch 2024-03-18T22:39:36.734 INFO:teuthology.orchestra.run.smithi139.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-03-18T22:39:36.734 INFO:teuthology.orchestra.run.smithi139.stdout: python3-ply-3.11-14.el9.noarch 2024-03-18T22:39:36.734 INFO:teuthology.orchestra.run.smithi139.stdout: python3-portend-3.1.0-2.el9.noarch 2024-03-18T22:39:36.735 INFO:teuthology.orchestra.run.smithi139.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-03-18T22:39:36.735 INFO:teuthology.orchestra.run.smithi139.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-03-18T22:39:36.735 INFO:teuthology.orchestra.run.smithi139.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-03-18T22:39:36.735 INFO:teuthology.orchestra.run.smithi139.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-03-18T22:39:36.735 INFO:teuthology.orchestra.run.smithi139.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-03-18T22:39:36.735 INFO:teuthology.orchestra.run.smithi139.stdout: python3-pycparser-2.20-6.el9.noarch 2024-03-18T22:39:36.735 INFO:teuthology.orchestra.run.smithi139.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-03-18T22:39:36.735 INFO:teuthology.orchestra.run.smithi139.stdout: python3-pytz-2021.1-5.el9.noarch 2024-03-18T22:39:36.735 INFO:teuthology.orchestra.run.smithi139.stdout: python3-rados-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-18T22:39:36.735 INFO:teuthology.orchestra.run.smithi139.stdout: python3-rbd-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-18T22:39:36.735 INFO:teuthology.orchestra.run.smithi139.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-03-18T22:39:36.735 INFO:teuthology.orchestra.run.smithi139.stdout: python3-requests-2.25.1-8.el9.noarch 2024-03-18T22:39:36.735 INFO:teuthology.orchestra.run.smithi139.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-03-18T22:39:36.735 INFO:teuthology.orchestra.run.smithi139.stdout: python3-rgw-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-18T22:39:36.735 INFO:teuthology.orchestra.run.smithi139.stdout: python3-routes-2.5.1-5.el9.noarch 2024-03-18T22:39:36.736 INFO:teuthology.orchestra.run.smithi139.stdout: python3-rsa-4.9-2.el9.noarch 2024-03-18T22:39:36.736 INFO:teuthology.orchestra.run.smithi139.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-03-18T22:39:36.736 INFO:teuthology.orchestra.run.smithi139.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-03-18T22:39:36.736 INFO:teuthology.orchestra.run.smithi139.stdout: python3-toml-0.10.2-6.el9.noarch 2024-03-18T22:39:36.736 INFO:teuthology.orchestra.run.smithi139.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-03-18T22:39:36.736 INFO:teuthology.orchestra.run.smithi139.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-03-18T22:39:36.736 INFO:teuthology.orchestra.run.smithi139.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-03-18T22:39:36.736 INFO:teuthology.orchestra.run.smithi139.stdout: python3-webob-1.8.7-6.el9.noarch 2024-03-18T22:39:36.736 INFO:teuthology.orchestra.run.smithi139.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-03-18T22:39:36.736 INFO:teuthology.orchestra.run.smithi139.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-03-18T22:39:36.736 INFO:teuthology.orchestra.run.smithi139.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-03-18T22:39:36.736 INFO:teuthology.orchestra.run.smithi139.stdout: qatlib-23.11.0-1.el9.x86_64 2024-03-18T22:39:36.736 INFO:teuthology.orchestra.run.smithi139.stdout: qatlib-service-23.11.0-1.el9.x86_64 2024-03-18T22:39:36.736 INFO:teuthology.orchestra.run.smithi139.stdout: qatzip-libs-1.1.2-2.el9.x86_64 2024-03-18T22:39:36.737 INFO:teuthology.orchestra.run.smithi139.stdout: rbd-fuse-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-18T22:39:36.737 INFO:teuthology.orchestra.run.smithi139.stdout: rbd-mirror-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-18T22:39:36.737 INFO:teuthology.orchestra.run.smithi139.stdout: rbd-nbd-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-18T22:39:36.737 INFO:teuthology.orchestra.run.smithi139.stdout: re2-1:20211101-3.el9.x86_64 2024-03-18T22:39:36.737 INFO:teuthology.orchestra.run.smithi139.stdout: socat-1.7.4.1-5.el9.x86_64 2024-03-18T22:39:36.737 INFO:teuthology.orchestra.run.smithi139.stdout: thrift-0.15.0-2.el9.x86_64 2024-03-18T22:39:36.737 INFO:teuthology.orchestra.run.smithi139.stdout: uuid-1.6.2-55.el9.x86_64 2024-03-18T22:39:36.737 INFO:teuthology.orchestra.run.smithi139.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-03-18T22:39:36.737 INFO:teuthology.orchestra.run.smithi139.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-03-18T22:39:36.737 INFO:teuthology.orchestra.run.smithi139.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-03-18T22:39:36.738 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-03-18T22:39:36.738 INFO:teuthology.orchestra.run.smithi139.stdout:Complete! 2024-03-18T22:39:37.177 DEBUG:teuthology.parallel:result is None 2024-03-18T22:50:48.609 INFO:teuthology.orchestra.run.smithi175.stdout:CentOS Stream 9 - CRB 8.0 kB/s | 6.1 MB 13:07 2024-03-18T22:55:10.330 INFO:teuthology.orchestra.run.smithi175.stdout:CentOS Stream 9 - Extras packages 61 B/s | 16 kB 04:20 2024-03-18T22:55:12.239 INFO:teuthology.orchestra.run.smithi175.stdout:Extra Packages for Enterprise Linux 12 MB/s | 21 MB 00:01 2024-03-18T22:55:17.230 INFO:teuthology.orchestra.run.smithi175.stdout:lab-extras 9.8 kB/s | 1.7 kB 00:00 2024-03-18T22:55:19.567 INFO:teuthology.orchestra.run.smithi175.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-18T22:55:19.567 INFO:teuthology.orchestra.run.smithi175.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-18T22:55:19.579 INFO:teuthology.orchestra.run.smithi175.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-03-18T22:55:19.580 INFO:teuthology.orchestra.run.smithi175.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-03-18T22:55:19.581 INFO:teuthology.orchestra.run.smithi175.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-03-18T22:55:19.582 INFO:teuthology.orchestra.run.smithi175.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-03-18T22:55:19.583 INFO:teuthology.orchestra.run.smithi175.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-03-18T22:55:19.586 INFO:teuthology.orchestra.run.smithi175.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-03-18T22:55:19.587 INFO:teuthology.orchestra.run.smithi175.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-03-18T22:55:19.665 INFO:teuthology.orchestra.run.smithi175.stdout:Dependencies resolved. 2024-03-18T22:55:19.689 INFO:teuthology.orchestra.run.smithi175.stdout:============================================================================================ 2024-03-18T22:55:19.690 INFO:teuthology.orchestra.run.smithi175.stdout: Package Arch Version Repository Size 2024-03-18T22:55:19.690 INFO:teuthology.orchestra.run.smithi175.stdout:============================================================================================ 2024-03-18T22:55:19.690 INFO:teuthology.orchestra.run.smithi175.stdout:Installing: 2024-03-18T22:55:19.690 INFO:teuthology.orchestra.run.smithi175.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-03-18T22:55:19.690 INFO:teuthology.orchestra.run.smithi175.stdout: ceph x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 6.6 k 2024-03-18T22:55:19.690 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-base x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 5.4 M 2024-03-18T22:55:19.690 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-fuse x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 891 k 2024-03-18T22:55:19.690 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 145 k 2024-03-18T22:55:19.690 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mgr x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 1.5 M 2024-03-18T22:55:19.690 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2206.gb6ae2491.el9 ceph-noarch 139 k 2024-03-18T22:55:19.690 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2206.gb6ae2491.el9 ceph-noarch 1.8 M 2024-03-18T22:55:19.691 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2206.gb6ae2491.el9 ceph-noarch 7.4 M 2024-03-18T22:55:19.691 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mgr-rook noarch 2:19.0.0-2206.gb6ae2491.el9 ceph-noarch 49 k 2024-03-18T22:55:19.691 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-radosgw x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 9.8 M 2024-03-18T22:55:19.691 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-test x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 48 M 2024-03-18T22:55:19.691 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-volume noarch 2:19.0.0-2206.gb6ae2491.el9 ceph-noarch 264 k 2024-03-18T22:55:19.691 INFO:teuthology.orchestra.run.smithi175.stdout: cephadm noarch 2:19.0.0-2206.gb6ae2491.el9 ceph-noarch 749 k 2024-03-18T22:55:19.691 INFO:teuthology.orchestra.run.smithi175.stdout: cephfs-mirror x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 221 k 2024-03-18T22:55:19.691 INFO:teuthology.orchestra.run.smithi175.stdout: cephfs-shell x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 27 k 2024-03-18T22:55:19.691 INFO:teuthology.orchestra.run.smithi175.stdout: cephfs-top noarch 2:19.0.0-2206.gb6ae2491.el9 ceph-noarch 18 k 2024-03-18T22:55:19.691 INFO:teuthology.orchestra.run.smithi175.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-03-18T22:55:19.691 INFO:teuthology.orchestra.run.smithi175.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-03-18T22:55:19.691 INFO:teuthology.orchestra.run.smithi175.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-03-18T22:55:19.691 INFO:teuthology.orchestra.run.smithi175.stdout: libcephfs-devel x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 32 k 2024-03-18T22:55:19.692 INFO:teuthology.orchestra.run.smithi175.stdout: libcephfs2 x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 746 k 2024-03-18T22:55:19.692 INFO:teuthology.orchestra.run.smithi175.stdout: librados-devel x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 127 k 2024-03-18T22:55:19.692 INFO:teuthology.orchestra.run.smithi175.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-03-18T22:55:19.692 INFO:teuthology.orchestra.run.smithi175.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-03-18T22:55:19.692 INFO:teuthology.orchestra.run.smithi175.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-03-18T22:55:19.692 INFO:teuthology.orchestra.run.smithi175.stdout: postgresql x86_64 13.11-1.el9 CentOS-AppStream 1.5 M 2024-03-18T22:55:19.692 INFO:teuthology.orchestra.run.smithi175.stdout: postgresql-contrib x86_64 13.11-1.el9 CentOS-AppStream 811 k 2024-03-18T22:55:19.692 INFO:teuthology.orchestra.run.smithi175.stdout: postgresql-server x86_64 13.11-1.el9 CentOS-AppStream 5.7 M 2024-03-18T22:55:19.692 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cephfs x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 162 k 2024-03-18T22:55:19.692 INFO:teuthology.orchestra.run.smithi175.stdout: python3-rados x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 321 k 2024-03-18T22:55:19.692 INFO:teuthology.orchestra.run.smithi175.stdout: python3-rbd x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 298 k 2024-03-18T22:55:19.692 INFO:teuthology.orchestra.run.smithi175.stdout: python3-rgw x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 99 k 2024-03-18T22:55:19.692 INFO:teuthology.orchestra.run.smithi175.stdout: rbd-fuse x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 85 k 2024-03-18T22:55:19.693 INFO:teuthology.orchestra.run.smithi175.stdout: rbd-mirror x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 3.1 M 2024-03-18T22:55:19.693 INFO:teuthology.orchestra.run.smithi175.stdout: rbd-nbd x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 170 k 2024-03-18T22:55:19.693 INFO:teuthology.orchestra.run.smithi175.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-03-18T22:55:19.693 INFO:teuthology.orchestra.run.smithi175.stdout:Upgrading: 2024-03-18T22:55:19.693 INFO:teuthology.orchestra.run.smithi175.stdout: NetworkManager x86_64 1:1.47.2-1.el9 baseos 2.3 M 2024-03-18T22:55:19.693 INFO:teuthology.orchestra.run.smithi175.stdout: NetworkManager-libnm x86_64 1:1.47.2-1.el9 baseos 1.8 M 2024-03-18T22:55:19.693 INFO:teuthology.orchestra.run.smithi175.stdout: NetworkManager-team x86_64 1:1.47.2-1.el9 baseos 42 k 2024-03-18T22:55:19.693 INFO:teuthology.orchestra.run.smithi175.stdout: NetworkManager-tui x86_64 1:1.47.2-1.el9 baseos 248 k 2024-03-18T22:55:19.693 INFO:teuthology.orchestra.run.smithi175.stdout: acl x86_64 2.3.1-4.el9 baseos 71 k 2024-03-18T22:55:19.693 INFO:teuthology.orchestra.run.smithi175.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 baseos 36 k 2024-03-18T22:55:19.693 INFO:teuthology.orchestra.run.smithi175.stdout: elfutils-libelf x86_64 0.190-2.el9 baseos 192 k 2024-03-18T22:55:19.693 INFO:teuthology.orchestra.run.smithi175.stdout: elfutils-libs x86_64 0.190-2.el9 baseos 256 k 2024-03-18T22:55:19.694 INFO:teuthology.orchestra.run.smithi175.stdout: libacl x86_64 2.3.1-4.el9 baseos 23 k 2024-03-18T22:55:19.694 INFO:teuthology.orchestra.run.smithi175.stdout: libacl-devel x86_64 2.3.1-4.el9 appstream 74 k 2024-03-18T22:55:19.694 INFO:teuthology.orchestra.run.smithi175.stdout: libblkid x86_64 2.37.4-18.el9 baseos 108 k 2024-03-18T22:55:19.694 INFO:teuthology.orchestra.run.smithi175.stdout: libfdisk x86_64 2.37.4-18.el9 baseos 155 k 2024-03-18T22:55:19.694 INFO:teuthology.orchestra.run.smithi175.stdout: libmount x86_64 2.37.4-18.el9 baseos 136 k 2024-03-18T22:55:19.694 INFO:teuthology.orchestra.run.smithi175.stdout: librados2 x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 3.4 M 2024-03-18T22:55:19.694 INFO:teuthology.orchestra.run.smithi175.stdout: librbd1 x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 3.1 M 2024-03-18T22:55:19.694 INFO:teuthology.orchestra.run.smithi175.stdout: libsmartcols x86_64 2.37.4-18.el9 baseos 64 k 2024-03-18T22:55:19.694 INFO:teuthology.orchestra.run.smithi175.stdout: libuuid x86_64 2.37.4-18.el9 baseos 29 k 2024-03-18T22:55:19.694 INFO:teuthology.orchestra.run.smithi175.stdout: openssl x86_64 1:3.0.7-27.el9 baseos 1.2 M 2024-03-18T22:55:19.694 INFO:teuthology.orchestra.run.smithi175.stdout: openssl-libs x86_64 1:3.0.7-27.el9 baseos 2.2 M 2024-03-18T22:55:19.694 INFO:teuthology.orchestra.run.smithi175.stdout: util-linux x86_64 2.37.4-18.el9 baseos 2.3 M 2024-03-18T22:55:19.694 INFO:teuthology.orchestra.run.smithi175.stdout: util-linux-core x86_64 2.37.4-18.el9 baseos 465 k 2024-03-18T22:55:19.695 INFO:teuthology.orchestra.run.smithi175.stdout: util-linux-user x86_64 2.37.4-18.el9 baseos 31 k 2024-03-18T22:55:19.695 INFO:teuthology.orchestra.run.smithi175.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-03-18T22:55:19.695 INFO:teuthology.orchestra.run.smithi175.stdout: xfsprogs x86_64 6.3.0-1.el9 baseos 1.1 M 2024-03-18T22:55:19.695 INFO:teuthology.orchestra.run.smithi175.stdout:Installing dependencies: 2024-03-18T22:55:19.695 INFO:teuthology.orchestra.run.smithi175.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-03-18T22:55:19.695 INFO:teuthology.orchestra.run.smithi175.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-03-18T22:55:19.695 INFO:teuthology.orchestra.run.smithi175.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-03-18T22:55:19.695 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-common x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 21 M 2024-03-18T22:55:19.695 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2206.gb6ae2491.el9 ceph-noarch 34 k 2024-03-18T22:55:19.695 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mds x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 2.4 M 2024-03-18T22:55:19.695 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2206.gb6ae2491.el9 ceph-noarch 248 k 2024-03-18T22:55:19.695 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mon x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 4.7 M 2024-03-18T22:55:19.696 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-osd x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 17 M 2024-03-18T22:55:19.696 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2206.gb6ae2491.el9 ceph-noarch 16 k 2024-03-18T22:55:19.696 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-selinux x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 25 k 2024-03-18T22:55:19.696 INFO:teuthology.orchestra.run.smithi175.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-03-18T22:55:19.696 INFO:teuthology.orchestra.run.smithi175.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-03-18T22:55:19.696 INFO:teuthology.orchestra.run.smithi175.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-03-18T22:55:19.696 INFO:teuthology.orchestra.run.smithi175.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-03-18T22:55:19.696 INFO:teuthology.orchestra.run.smithi175.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-03-18T22:55:19.696 INFO:teuthology.orchestra.run.smithi175.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-03-18T22:55:19.696 INFO:teuthology.orchestra.run.smithi175.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-03-18T22:55:19.696 INFO:teuthology.orchestra.run.smithi175.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-03-18T22:55:19.696 INFO:teuthology.orchestra.run.smithi175.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-03-18T22:55:19.696 INFO:teuthology.orchestra.run.smithi175.stdout: libcephsqlite x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 163 k 2024-03-18T22:55:19.697 INFO:teuthology.orchestra.run.smithi175.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-03-18T22:55:19.697 INFO:teuthology.orchestra.run.smithi175.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-03-18T22:55:19.697 INFO:teuthology.orchestra.run.smithi175.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-03-18T22:55:19.697 INFO:teuthology.orchestra.run.smithi175.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-03-18T22:55:19.697 INFO:teuthology.orchestra.run.smithi175.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-03-18T22:55:19.697 INFO:teuthology.orchestra.run.smithi175.stdout: libradosstriper1 x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 501 k 2024-03-18T22:55:19.697 INFO:teuthology.orchestra.run.smithi175.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-03-18T22:55:19.697 INFO:teuthology.orchestra.run.smithi175.stdout: librgw2 x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 5.1 M 2024-03-18T22:55:19.697 INFO:teuthology.orchestra.run.smithi175.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-03-18T22:55:19.697 INFO:teuthology.orchestra.run.smithi175.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-03-18T22:55:19.697 INFO:teuthology.orchestra.run.smithi175.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-03-18T22:55:19.697 INFO:teuthology.orchestra.run.smithi175.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-03-18T22:55:19.697 INFO:teuthology.orchestra.run.smithi175.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-03-18T22:55:19.697 INFO:teuthology.orchestra.run.smithi175.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-03-18T22:55:19.698 INFO:teuthology.orchestra.run.smithi175.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-03-18T22:55:19.698 INFO:teuthology.orchestra.run.smithi175.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-03-18T22:55:19.698 INFO:teuthology.orchestra.run.smithi175.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-03-18T22:55:19.698 INFO:teuthology.orchestra.run.smithi175.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-03-18T22:55:19.698 INFO:teuthology.orchestra.run.smithi175.stdout: postgresql-private-libs x86_64 13.11-1.el9 CentOS-AppStream 134 k 2024-03-18T22:55:19.698 INFO:teuthology.orchestra.run.smithi175.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-03-18T22:55:19.698 INFO:teuthology.orchestra.run.smithi175.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-03-18T22:55:19.698 INFO:teuthology.orchestra.run.smithi175.stdout: python3-asyncssh noarch 2.13.2-4.el9 epel 548 k 2024-03-18T22:55:19.698 INFO:teuthology.orchestra.run.smithi175.stdout: python3-attrs noarch 20.3.0-7.el9 CentOS-AppStream 78 k 2024-03-18T22:55:19.698 INFO:teuthology.orchestra.run.smithi175.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-03-18T22:55:19.698 INFO:teuthology.orchestra.run.smithi175.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-03-18T22:55:19.698 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-03-18T22:55:19.699 INFO:teuthology.orchestra.run.smithi175.stdout: python3-ceph-argparse x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 45 k 2024-03-18T22:55:19.699 INFO:teuthology.orchestra.run.smithi175.stdout: python3-ceph-common x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 131 k 2024-03-18T22:55:19.699 INFO:teuthology.orchestra.run.smithi175.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-03-18T22:55:19.699 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-03-18T22:55:19.699 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-03-18T22:55:19.699 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-03-18T22:55:19.699 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-03-18T22:55:19.699 INFO:teuthology.orchestra.run.smithi175.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-03-18T22:55:19.699 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-03-18T22:55:19.699 INFO:teuthology.orchestra.run.smithi175.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-03-18T22:55:19.699 INFO:teuthology.orchestra.run.smithi175.stdout: python3-google-auth noarch 1:2.28.0-1.el9 epel 203 k 2024-03-18T22:55:19.699 INFO:teuthology.orchestra.run.smithi175.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-03-18T22:55:19.699 INFO:teuthology.orchestra.run.smithi175.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-03-18T22:55:19.699 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-03-18T22:55:19.700 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-03-18T22:55:19.700 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-03-18T22:55:19.700 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-03-18T22:55:19.700 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-03-18T22:55:19.700 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-03-18T22:55:19.700 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-03-18T22:55:19.700 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-03-18T22:55:19.700 INFO:teuthology.orchestra.run.smithi175.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-03-18T22:55:19.700 INFO:teuthology.orchestra.run.smithi175.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-03-18T22:55:19.700 INFO:teuthology.orchestra.run.smithi175.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-03-18T22:55:19.700 INFO:teuthology.orchestra.run.smithi175.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-03-18T22:55:19.700 INFO:teuthology.orchestra.run.smithi175.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-03-18T22:55:19.700 INFO:teuthology.orchestra.run.smithi175.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-03-18T22:55:19.701 INFO:teuthology.orchestra.run.smithi175.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-03-18T22:55:19.701 INFO:teuthology.orchestra.run.smithi175.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-03-18T22:55:19.701 INFO:teuthology.orchestra.run.smithi175.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-03-18T22:55:19.701 INFO:teuthology.orchestra.run.smithi175.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-03-18T22:55:19.701 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-03-18T22:55:19.701 INFO:teuthology.orchestra.run.smithi175.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-03-18T22:55:19.701 INFO:teuthology.orchestra.run.smithi175.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-03-18T22:55:19.701 INFO:teuthology.orchestra.run.smithi175.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-03-18T22:55:19.701 INFO:teuthology.orchestra.run.smithi175.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-03-18T22:55:19.701 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-03-18T22:55:19.701 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-03-18T22:55:19.701 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-03-18T22:55:19.701 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-03-18T22:55:19.701 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-03-18T22:55:19.702 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-03-18T22:55:19.702 INFO:teuthology.orchestra.run.smithi175.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-03-18T22:55:19.702 INFO:teuthology.orchestra.run.smithi175.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-03-18T22:55:19.702 INFO:teuthology.orchestra.run.smithi175.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-03-18T22:55:19.702 INFO:teuthology.orchestra.run.smithi175.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-03-18T22:55:19.702 INFO:teuthology.orchestra.run.smithi175.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-03-18T22:55:19.702 INFO:teuthology.orchestra.run.smithi175.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-03-18T22:55:19.702 INFO:teuthology.orchestra.run.smithi175.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-03-18T22:55:19.702 INFO:teuthology.orchestra.run.smithi175.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-03-18T22:55:19.702 INFO:teuthology.orchestra.run.smithi175.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-03-18T22:55:19.702 INFO:teuthology.orchestra.run.smithi175.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-03-18T22:55:19.702 INFO:teuthology.orchestra.run.smithi175.stdout: python3-wcwidth noarch 0.2.5-8.el9 CentOS-AppStream 41 k 2024-03-18T22:55:19.702 INFO:teuthology.orchestra.run.smithi175.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-03-18T22:55:19.702 INFO:teuthology.orchestra.run.smithi175.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-03-18T22:55:19.703 INFO:teuthology.orchestra.run.smithi175.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-03-18T22:55:19.703 INFO:teuthology.orchestra.run.smithi175.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-03-18T22:55:19.703 INFO:teuthology.orchestra.run.smithi175.stdout: qatlib x86_64 23.11.0-1.el9 appstream 216 k 2024-03-18T22:55:19.703 INFO:teuthology.orchestra.run.smithi175.stdout: qatzip-libs x86_64 1.1.2-2.el9 appstream 44 k 2024-03-18T22:55:19.703 INFO:teuthology.orchestra.run.smithi175.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-03-18T22:55:19.703 INFO:teuthology.orchestra.run.smithi175.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-03-18T22:55:19.703 INFO:teuthology.orchestra.run.smithi175.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-03-18T22:55:19.703 INFO:teuthology.orchestra.run.smithi175.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-03-18T22:55:19.703 INFO:teuthology.orchestra.run.smithi175.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-03-18T22:55:19.703 INFO:teuthology.orchestra.run.smithi175.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-03-18T22:55:19.703 INFO:teuthology.orchestra.run.smithi175.stdout:Installing weak dependencies: 2024-03-18T22:55:19.703 INFO:teuthology.orchestra.run.smithi175.stdout: qatlib-service x86_64 23.11.0-1.el9 appstream 35 k 2024-03-18T22:55:19.703 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-03-18T22:55:19.703 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction Summary 2024-03-18T22:55:19.704 INFO:teuthology.orchestra.run.smithi175.stdout:============================================================================================ 2024-03-18T22:55:19.704 INFO:teuthology.orchestra.run.smithi175.stdout:Install 150 Packages 2024-03-18T22:55:19.704 INFO:teuthology.orchestra.run.smithi175.stdout:Upgrade 24 Packages 2024-03-18T22:55:19.704 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-03-18T22:55:19.704 INFO:teuthology.orchestra.run.smithi175.stdout:Total download size: 224 M 2024-03-18T22:55:19.704 INFO:teuthology.orchestra.run.smithi175.stdout:Downloading Packages: 2024-03-18T22:55:20.674 INFO:teuthology.orchestra.run.smithi175.stdout:(1/174): ceph-19.0.0-2206.gb6ae2491.el9.x86_64. 34 kB/s | 6.6 kB 00:00 2024-03-18T22:55:20.951 INFO:teuthology.orchestra.run.smithi175.stdout:(2/174): ceph-fuse-19.0.0-2206.gb6ae2491.el9.x8 3.1 MB/s | 891 kB 00:00 2024-03-18T22:55:21.035 INFO:teuthology.orchestra.run.smithi175.stdout:(3/174): ceph-base-19.0.0-2206.gb6ae2491.el9.x8 9.7 MB/s | 5.4 MB 00:00 2024-03-18T22:55:21.060 INFO:teuthology.orchestra.run.smithi175.stdout:(4/174): ceph-immutable-object-cache-19.0.0-220 1.3 MB/s | 145 kB 00:00 2024-03-18T22:55:21.193 INFO:teuthology.orchestra.run.smithi175.stdout:(5/174): ceph-mds-19.0.0-2206.gb6ae2491.el9.x86 15 MB/s | 2.4 MB 00:00 2024-03-18T22:55:21.528 INFO:teuthology.orchestra.run.smithi175.stdout:(6/174): ceph-mgr-19.0.0-2206.gb6ae2491.el9.x86 3.2 MB/s | 1.5 MB 00:00 2024-03-18T22:55:21.594 INFO:teuthology.orchestra.run.smithi175.stdout:(7/174): ceph-mon-19.0.0-2206.gb6ae2491.el9.x86 12 MB/s | 4.7 MB 00:00 2024-03-18T22:55:22.194 INFO:teuthology.orchestra.run.smithi175.stdout:(8/174): ceph-radosgw-19.0.0-2206.gb6ae2491.el9 16 MB/s | 9.8 MB 00:00 2024-03-18T22:55:22.235 INFO:teuthology.orchestra.run.smithi175.stdout:(9/174): ceph-selinux-19.0.0-2206.gb6ae2491.el9 608 kB/s | 25 kB 00:00 2024-03-18T22:55:24.311 INFO:teuthology.orchestra.run.smithi175.stdout:(10/174): ceph-osd-19.0.0-2206.gb6ae2491.el9.x8 6.1 MB/s | 17 MB 00:02 2024-03-18T22:55:24.477 INFO:teuthology.orchestra.run.smithi175.stdout:(11/174): cephfs-mirror-19.0.0-2206.gb6ae2491.e 1.3 MB/s | 221 kB 00:00 2024-03-18T22:55:24.569 INFO:teuthology.orchestra.run.smithi175.stdout:(12/174): cephfs-shell-19.0.0-2206.gb6ae2491.el 295 kB/s | 27 kB 00:00 2024-03-18T22:55:24.661 INFO:teuthology.orchestra.run.smithi175.stdout:(13/174): libcephfs-devel-19.0.0-2206.gb6ae2491 346 kB/s | 32 kB 00:00 2024-03-18T22:55:25.195 INFO:teuthology.orchestra.run.smithi175.stdout:(14/174): ceph-test-19.0.0-2206.gb6ae2491.el9.x 16 MB/s | 48 MB 00:02 2024-03-18T22:55:25.228 INFO:teuthology.orchestra.run.smithi175.stdout:(15/174): libcephfs2-19.0.0-2206.gb6ae2491.el9. 1.3 MB/s | 746 kB 00:00 2024-03-18T22:55:25.279 INFO:teuthology.orchestra.run.smithi175.stdout:(16/174): libcephsqlite-19.0.0-2206.gb6ae2491.e 1.9 MB/s | 163 kB 00:00 2024-03-18T22:55:25.312 INFO:teuthology.orchestra.run.smithi175.stdout:(17/174): librados-devel-19.0.0-2206.gb6ae2491. 1.5 MB/s | 127 kB 00:00 2024-03-18T22:55:25.371 INFO:teuthology.orchestra.run.smithi175.stdout:(18/174): libradosstriper1-19.0.0-2206.gb6ae249 5.3 MB/s | 501 kB 00:00 2024-03-18T22:55:25.446 INFO:teuthology.orchestra.run.smithi175.stdout:(19/174): python3-ceph-argparse-19.0.0-2206.gb6 601 kB/s | 45 kB 00:00 2024-03-18T22:55:25.546 INFO:teuthology.orchestra.run.smithi175.stdout:(20/174): python3-ceph-common-19.0.0-2206.gb6ae 1.3 MB/s | 131 kB 00:00 2024-03-18T22:55:25.630 INFO:teuthology.orchestra.run.smithi175.stdout:(21/174): python3-cephfs-19.0.0-2206.gb6ae2491. 1.9 MB/s | 162 kB 00:00 2024-03-18T22:55:25.730 INFO:teuthology.orchestra.run.smithi175.stdout:(22/174): python3-rados-19.0.0-2206.gb6ae2491.e 3.1 MB/s | 321 kB 00:00 2024-03-18T22:55:25.839 INFO:teuthology.orchestra.run.smithi175.stdout:(23/174): python3-rbd-19.0.0-2206.gb6ae2491.el9 2.7 MB/s | 298 kB 00:00 2024-03-18T22:55:25.930 INFO:teuthology.orchestra.run.smithi175.stdout:(24/174): python3-rgw-19.0.0-2206.gb6ae2491.el9 1.1 MB/s | 99 kB 00:00 2024-03-18T22:55:26.039 INFO:teuthology.orchestra.run.smithi175.stdout:(25/174): rbd-fuse-19.0.0-2206.gb6ae2491.el9.x8 788 kB/s | 85 kB 00:00 2024-03-18T22:55:26.306 INFO:teuthology.orchestra.run.smithi175.stdout:(26/174): librgw2-19.0.0-2206.gb6ae2491.el9.x86 5.2 MB/s | 5.1 MB 00:00 2024-03-18T22:55:26.365 INFO:teuthology.orchestra.run.smithi175.stdout:(27/174): rbd-mirror-19.0.0-2206.gb6ae2491.el9. 9.5 MB/s | 3.1 MB 00:00 2024-03-18T22:55:26.398 INFO:teuthology.orchestra.run.smithi175.stdout:(28/174): rbd-nbd-19.0.0-2206.gb6ae2491.el9.x86 1.8 MB/s | 170 kB 00:00 2024-03-18T22:55:26.457 INFO:teuthology.orchestra.run.smithi175.stdout:(29/174): ceph-grafana-dashboards-19.0.0-2206.g 370 kB/s | 34 kB 00:00 2024-03-18T22:55:26.482 INFO:teuthology.orchestra.run.smithi175.stdout:(30/174): ceph-mgr-cephadm-19.0.0-2206.gb6ae249 1.6 MB/s | 139 kB 00:00 2024-03-18T22:55:26.633 INFO:teuthology.orchestra.run.smithi175.stdout:(31/174): ceph-mgr-dashboard-19.0.0-2206.gb6ae2 10 MB/s | 1.8 MB 00:00 2024-03-18T22:55:26.749 INFO:teuthology.orchestra.run.smithi175.stdout:(32/174): ceph-mgr-modules-core-19.0.0-2206.gb6 2.1 MB/s | 248 kB 00:00 2024-03-18T22:55:26.842 INFO:teuthology.orchestra.run.smithi175.stdout:(33/174): ceph-mgr-rook-19.0.0-2206.gb6ae2491.e 537 kB/s | 49 kB 00:00 2024-03-18T22:55:26.942 INFO:teuthology.orchestra.run.smithi175.stdout:(34/174): ceph-prometheus-alerts-19.0.0-2206.gb 165 kB/s | 16 kB 00:00 2024-03-18T22:55:27.042 INFO:teuthology.orchestra.run.smithi175.stdout:(35/174): ceph-volume-19.0.0-2206.gb6ae2491.el9 2.6 MB/s | 264 kB 00:00 2024-03-18T22:55:27.176 INFO:teuthology.orchestra.run.smithi175.stdout:(36/174): cephadm-19.0.0-2206.gb6ae2491.el9.noa 5.5 MB/s | 749 kB 00:00 2024-03-18T22:55:27.276 INFO:teuthology.orchestra.run.smithi175.stdout:(37/174): cephfs-top-19.0.0-2206.gb6ae2491.el9. 182 kB/s | 18 kB 00:00 2024-03-18T22:55:27.409 INFO:teuthology.orchestra.run.smithi175.stdout:(38/174): automake-1.16.2-8.el9.noarch.rpm 4.9 MB/s | 662 kB 00:00 2024-03-18T22:55:27.460 INFO:teuthology.orchestra.run.smithi175.stdout:(39/174): bison-3.7.4-5.el9.x86_64.rpm 18 MB/s | 920 kB 00:00 2024-03-18T22:55:27.493 INFO:teuthology.orchestra.run.smithi175.stdout:(40/174): boost-program-options-1.75.0-8.el9.x8 3.1 MB/s | 106 kB 00:00 2024-03-18T22:55:27.535 INFO:teuthology.orchestra.run.smithi175.stdout:(41/174): flex-2.6.4-9.el9.x86_64.rpm 7.2 MB/s | 307 kB 00:00 2024-03-18T22:55:27.569 INFO:teuthology.orchestra.run.smithi175.stdout:(42/174): flexiblas-3.0.4-8.el9.x86_64.rpm 948 kB/s | 32 kB 00:00 2024-03-18T22:55:27.669 INFO:teuthology.orchestra.run.smithi175.stdout:(43/174): flexiblas-netlib-3.0.4-8.el9.x86_64.r 30 MB/s | 3.0 MB 00:00 2024-03-18T22:55:27.703 INFO:teuthology.orchestra.run.smithi175.stdout:(44/174): flexiblas-openblas-openmp-3.0.4-8.el9 515 kB/s | 17 kB 00:00 2024-03-18T22:55:27.736 INFO:teuthology.orchestra.run.smithi175.stdout:(45/174): libaio-devel-0.3.111-13.el9.x86_64.rp 302 kB/s | 10 kB 00:00 2024-03-18T22:55:27.820 INFO:teuthology.orchestra.run.smithi175.stdout:(46/174): ceph-mgr-diskprediction-local-19.0.0- 5.5 MB/s | 7.4 MB 00:01 2024-03-18T22:55:27.837 INFO:teuthology.orchestra.run.smithi175.stdout:(47/174): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.5 MB/s | 159 kB 00:00 2024-03-18T22:55:27.862 INFO:teuthology.orchestra.run.smithi175.stdout:(48/174): librabbitmq-0.11.0-7.el9.x86_64.rpm 1.0 MB/s | 44 kB 00:00 2024-03-18T22:55:27.887 INFO:teuthology.orchestra.run.smithi175.stdout:(49/174): librdkafka-1.6.1-102.el9.x86_64.rpm 13 MB/s | 661 kB 00:00 2024-03-18T22:55:27.912 INFO:teuthology.orchestra.run.smithi175.stdout:(50/174): libtool-2.4.6-45.el9.x86_64.rpm 12 MB/s | 578 kB 00:00 2024-03-18T22:55:27.938 INFO:teuthology.orchestra.run.smithi175.stdout:(51/174): libzstd-devel-1.5.1-2.el9.x86_64.rpm 941 kB/s | 46 kB 00:00 2024-03-18T22:55:27.963 INFO:teuthology.orchestra.run.smithi175.stdout:(52/174): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.5 MB/s | 282 kB 00:00 2024-03-18T22:55:27.988 INFO:teuthology.orchestra.run.smithi175.stdout:(53/174): lua-5.4.4-4.el9.x86_64.rpm 3.7 MB/s | 187 kB 00:00 2024-03-18T22:55:28.014 INFO:teuthology.orchestra.run.smithi175.stdout:(54/174): oniguruma-6.9.6-1.el9.5.x86_64.rpm 4.3 MB/s | 217 kB 00:00 2024-03-18T22:55:28.030 INFO:teuthology.orchestra.run.smithi175.stdout:(55/174): openblas-0.3.21-2.el9.x86_64.rpm 827 kB/s | 34 kB 00:00 2024-03-18T22:55:28.131 INFO:teuthology.orchestra.run.smithi175.stdout:(56/174): openblas-openmp-0.3.21-2.el9.x86_64.r 41 MB/s | 4.7 MB 00:00 2024-03-18T22:55:28.173 INFO:teuthology.orchestra.run.smithi175.stdout:(57/174): postgresql-13.11-1.el9.x86_64.rpm 11 MB/s | 1.5 MB 00:00 2024-03-18T22:55:28.245 INFO:teuthology.orchestra.run.smithi175.stdout:(58/174): postgresql-contrib-13.11-1.el9.x86_64 7.1 MB/s | 811 kB 00:00 2024-03-18T22:55:28.273 INFO:teuthology.orchestra.run.smithi175.stdout:(59/174): postgresql-private-libs-13.11-1.el9.x 1.3 MB/s | 134 kB 00:00 2024-03-18T22:55:28.382 INFO:teuthology.orchestra.run.smithi175.stdout:(60/174): postgresql-server-13.11-1.el9.x86_64. 42 MB/s | 5.7 MB 00:00 2024-03-18T22:55:28.415 INFO:teuthology.orchestra.run.smithi175.stdout:(61/174): protobuf-3.14.0-13.el9.x86_64.rpm 7.3 MB/s | 1.0 MB 00:00 2024-03-18T22:55:28.441 INFO:teuthology.orchestra.run.smithi175.stdout:(62/174): python3-attrs-20.3.0-7.el9.noarch.rpm 1.3 MB/s | 78 kB 00:00 2024-03-18T22:55:28.558 INFO:teuthology.orchestra.run.smithi175.stdout:(63/174): python3-babel-2.9.1-2.el9.noarch.rpm 41 MB/s | 5.8 MB 00:00 2024-03-18T22:55:28.583 INFO:teuthology.orchestra.run.smithi175.stdout:(64/174): python3-jsonpatch-1.21-16.el9.noarch. 175 kB/s | 24 kB 00:00 2024-03-18T22:55:28.608 INFO:teuthology.orchestra.run.smithi175.stdout:(65/174): python3-jsonpointer-2.0-4.el9.noarch. 362 kB/s | 18 kB 00:00 2024-03-18T22:55:28.634 INFO:teuthology.orchestra.run.smithi175.stdout:(66/174): python3-mako-1.1.4-6.el9.noarch.rpm 2.8 MB/s | 144 kB 00:00 2024-03-18T22:55:28.659 INFO:teuthology.orchestra.run.smithi175.stdout:(67/174): python3-markupsafe-1.1.1-12.el9.x86_6 632 kB/s | 32 kB 00:00 2024-03-18T22:55:28.754 INFO:teuthology.orchestra.run.smithi175.stdout:(68/174): python3-numpy-1.20.1-5.el9.x86_64.rpm 41 MB/s | 4.8 MB 00:00 2024-03-18T22:55:28.776 INFO:teuthology.orchestra.run.smithi175.stdout:(69/174): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.9 MB/s | 217 kB 00:00 2024-03-18T22:55:28.801 INFO:teuthology.orchestra.run.smithi175.stdout:(70/174): python3-oauthlib-3.1.1-5.el9.noarch.r 3.4 MB/s | 174 kB 00:00 2024-03-18T22:55:28.827 INFO:teuthology.orchestra.run.smithi175.stdout:(71/174): python3-packaging-20.9-5.el9.noarch.r 1.3 MB/s | 69 kB 00:00 2024-03-18T22:55:28.852 INFO:teuthology.orchestra.run.smithi175.stdout:(72/174): python3-prettytable-0.7.2-27.el9.noar 822 kB/s | 41 kB 00:00 2024-03-18T22:55:28.877 INFO:teuthology.orchestra.run.smithi175.stdout:(73/174): python3-protobuf-3.14.0-13.el9.noarch 4.6 MB/s | 238 kB 00:00 2024-03-18T22:55:28.902 INFO:teuthology.orchestra.run.smithi175.stdout:(74/174): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.6 MB/s | 132 kB 00:00 2024-03-18T22:55:28.928 INFO:teuthology.orchestra.run.smithi175.stdout:(75/174): python3-pyasn1-modules-0.4.8-6.el9.no 4.1 MB/s | 211 kB 00:00 2024-03-18T22:55:28.953 INFO:teuthology.orchestra.run.smithi175.stdout:(76/174): python3-pytz-2021.1-5.el9.noarch.rpm 946 kB/s | 47 kB 00:00 2024-03-18T22:55:28.978 INFO:teuthology.orchestra.run.smithi175.stdout:(77/174): python3-requests-oauthlib-1.3.0-12.el 852 kB/s | 43 kB 00:00 2024-03-18T22:55:29.129 INFO:teuthology.orchestra.run.smithi175.stdout:(78/174): python3-toml-0.10.2-6.el9.noarch.rpm 262 kB/s | 39 kB 00:00 2024-03-18T22:55:29.287 INFO:teuthology.orchestra.run.smithi175.stdout:(79/174): python3-scipy-1.6.2-8.el9.x86_64.rpm 47 MB/s | 16 MB 00:00 2024-03-18T22:55:29.312 INFO:teuthology.orchestra.run.smithi175.stdout:(80/174): python3-wcwidth-0.2.5-8.el9.noarch.rp 226 kB/s | 41 kB 00:00 2024-03-18T22:55:29.338 INFO:teuthology.orchestra.run.smithi175.stdout:(81/174): socat-1.7.4.1-5.el9.x86_64.rpm 5.9 MB/s | 300 kB 00:00 2024-03-18T22:55:29.363 INFO:teuthology.orchestra.run.smithi175.stdout:(82/174): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 56 kB 00:00 2024-03-18T22:55:29.388 INFO:teuthology.orchestra.run.smithi175.stdout:(83/174): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 63 kB 00:00 2024-03-18T22:55:29.413 INFO:teuthology.orchestra.run.smithi175.stdout:(84/174): zlib-devel-1.2.11-40.el9.x86_64.rpm 877 kB/s | 44 kB 00:00 2024-03-18T22:55:29.439 INFO:teuthology.orchestra.run.smithi175.stdout:(85/174): fuse-2.9.9-15.el9.x86_64.rpm 1.5 MB/s | 79 kB 00:00 2024-03-18T22:55:29.464 INFO:teuthology.orchestra.run.smithi175.stdout:(86/174): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.7 MB/s | 241 kB 00:00 2024-03-18T22:55:29.498 INFO:teuthology.orchestra.run.smithi175.stdout:(87/174): python3-cryptography-36.0.1-4.el9.x86 20 MB/s | 1.1 MB 00:00 2024-03-18T22:55:29.523 INFO:teuthology.orchestra.run.smithi175.stdout:(88/174): python3-ply-3.11-14.el9.noarch.rpm 1.8 MB/s | 103 kB 00:00 2024-03-18T22:55:29.548 INFO:teuthology.orchestra.run.smithi175.stdout:(89/174): python3-pycparser-2.20-6.el9.noarch.r 2.4 MB/s | 124 kB 00:00 2024-03-18T22:55:29.573 INFO:teuthology.orchestra.run.smithi175.stdout:(90/174): python3-urllib3-1.26.5-5.el9.noarch.r 3.7 MB/s | 187 kB 00:00 2024-03-18T22:55:29.715 INFO:teuthology.orchestra.run.smithi175.stdout:(91/174): jq-1.6-16.el9.x86_64.rpm 1.1 MB/s | 187 kB 00:00 2024-03-18T22:55:29.799 INFO:teuthology.orchestra.run.smithi175.stdout:(92/174): libquadmath-11.4.1-3.el9.x86_64.rpm 2.3 MB/s | 195 kB 00:00 2024-03-18T22:55:29.832 INFO:teuthology.orchestra.run.smithi175.stdout:(93/174): libgfortran-11.4.1-3.el9.x86_64.rpm 3.0 MB/s | 803 kB 00:00 2024-03-18T22:55:29.866 INFO:teuthology.orchestra.run.smithi175.stdout:(94/174): python3-requests-2.25.1-8.el9.noarch. 1.8 MB/s | 125 kB 00:00 2024-03-18T22:55:29.892 INFO:teuthology.orchestra.run.smithi175.stdout:(95/174): elfutils-libelf-devel-0.190-2.el9.x86 365 kB/s | 21 kB 00:00 2024-03-18T22:55:29.917 INFO:teuthology.orchestra.run.smithi175.stdout:(96/174): libuuid-devel-2.37.4-18.el9.x86_64.rp 506 kB/s | 25 kB 00:00 2024-03-18T22:55:30.001 INFO:teuthology.orchestra.run.smithi175.stdout:(97/174): python3-devel-3.9.17-2.el9.x86_64.rpm 2.9 MB/s | 245 kB 00:00 2024-03-18T22:55:30.234 INFO:teuthology.orchestra.run.smithi175.stdout:(98/174): ceph-common-19.0.0-2206.gb6ae2491.el9 2.1 MB/s | 21 MB 00:09 2024-03-18T22:55:30.260 INFO:teuthology.orchestra.run.smithi175.stdout:(99/174): python3-jinja2-2.11.3-5.el9.noarch.rp 958 kB/s | 248 kB 00:00 2024-03-18T22:55:30.310 INFO:teuthology.orchestra.run.smithi175.stdout:(100/174): qatlib-service-23.11.0-1.el9.x86_64. 699 kB/s | 35 kB 00:00 2024-03-18T22:55:30.360 INFO:teuthology.orchestra.run.smithi175.stdout:(101/174): qatzip-libs-1.1.2-2.el9.x86_64.rpm 894 kB/s | 44 kB 00:00 2024-03-18T22:55:30.402 INFO:teuthology.orchestra.run.smithi175.stdout:(102/174): qatlib-23.11.0-1.el9.x86_64.rpm 1.3 MB/s | 216 kB 00:00 2024-03-18T22:55:30.427 INFO:teuthology.orchestra.run.smithi175.stdout:(103/174): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 1.4 MB/s | 94 kB 00:00 2024-03-18T22:55:30.452 INFO:teuthology.orchestra.run.smithi175.stdout:(104/174): lua-devel-5.4.4-4.el9.x86_64.rpm 444 kB/s | 22 kB 00:00 2024-03-18T22:55:30.628 INFO:teuthology.orchestra.run.smithi175.stdout:(105/174): protobuf-compiler-3.14.0-13.el9.x86_ 4.2 MB/s | 863 kB 00:00 2024-03-18T22:55:30.878 INFO:teuthology.orchestra.run.smithi175.stdout:(106/174): gperftools-libs-2.9.1-2.el9.x86_64.r 1.2 MB/s | 309 kB 00:00 2024-03-18T22:55:30.945 INFO:teuthology.orchestra.run.smithi175.stdout:(107/174): openssl-devel-3.0.7-27.el9.x86_64.rp 3.9 MB/s | 4.1 MB 00:01 2024-03-18T22:55:31.003 INFO:teuthology.orchestra.run.smithi175.stdout:(108/174): grpc-data-1.46.7-10.el9.noarch.rpm 155 kB/s | 19 kB 00:00 2024-03-18T22:55:31.037 INFO:teuthology.orchestra.run.smithi175.stdout:(109/174): abseil-cpp-20211102.0-3.el9.x86_64.r 938 kB/s | 548 kB 00:00 2024-03-18T22:55:31.170 INFO:teuthology.orchestra.run.smithi175.stdout:(110/174): libarrow-doc-9.0.0-10.el9.noarch.rpm 156 kB/s | 26 kB 00:00 2024-03-18T22:55:31.196 INFO:teuthology.orchestra.run.smithi175.stdout:(111/174): liboath-2.6.7-2.el9.x86_64.rpm 306 kB/s | 49 kB 00:00 2024-03-18T22:55:31.254 INFO:teuthology.orchestra.run.smithi175.stdout:(112/174): libunwind-1.6.2-1.el9.x86_64.rpm 805 kB/s | 67 kB 00:00 2024-03-18T22:55:31.307 INFO:teuthology.orchestra.run.smithi175.stdout:(113/174): luarocks-3.9.2-1.el9.noarch.rpm 1.3 MB/s | 151 kB 00:00 2024-03-18T22:55:31.514 INFO:teuthology.orchestra.run.smithi175.stdout:(114/174): parquet-libs-9.0.0-10.el9.x86_64.rpm 3.2 MB/s | 838 kB 00:00 2024-03-18T22:55:31.564 INFO:teuthology.orchestra.run.smithi175.stdout:(115/174): python3-bcrypt-3.2.2-1.el9.x86_64.rp 874 kB/s | 43 kB 00:00 2024-03-18T22:55:31.597 INFO:teuthology.orchestra.run.smithi175.stdout:(116/174): python3-asyncssh-2.13.2-4.el9.noarch 1.8 MB/s | 548 kB 00:00 2024-03-18T22:55:31.647 INFO:teuthology.orchestra.run.smithi175.stdout:(117/174): python3-cachetools-4.2.4-1.el9.noarc 385 kB/s | 32 kB 00:00 2024-03-18T22:55:31.706 INFO:teuthology.orchestra.run.smithi175.stdout:(118/174): python3-cheroot-8.6.0-4.el9.noarch.r 2.9 MB/s | 172 kB 00:00 2024-03-18T22:55:31.773 INFO:teuthology.orchestra.run.smithi175.stdout:(119/174): python3-cherrypy-18.6.1-2.el9.noarch 5.3 MB/s | 358 kB 00:00 2024-03-18T22:55:31.831 INFO:teuthology.orchestra.run.smithi175.stdout:(120/174): python3-certifi-2021.10.8-2.el9.noar 64 kB/s | 15 kB 00:00 2024-03-18T22:55:31.857 INFO:teuthology.orchestra.run.smithi175.stdout:(121/174): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.5 MB/s | 303 kB 00:00 2024-03-18T22:55:31.915 INFO:teuthology.orchestra.run.smithi175.stdout:(122/174): libarrow-9.0.0-10.el9.x86_64.rpm 4.6 MB/s | 4.4 MB 00:00 2024-03-18T22:55:31.999 INFO:teuthology.orchestra.run.smithi175.stdout:(123/174): python3-google-auth-2.28.0-1.el9.noa 1.4 MB/s | 203 kB 00:00 2024-03-18T22:55:32.024 INFO:teuthology.orchestra.run.smithi175.stdout:(124/174): python3-colorama-0.4.4-12.el9.noarch 183 kB/s | 35 kB 00:00 2024-03-18T22:55:32.225 INFO:teuthology.orchestra.run.smithi175.stdout:(125/174): python3-jaraco-8.2.1-3.el9.noarch.rp 53 kB/s | 11 kB 00:00 2024-03-18T22:55:32.267 INFO:teuthology.orchestra.run.smithi175.stdout:(126/174): python3-grpcio-1.46.7-10.el9.x86_64. 5.8 MB/s | 2.0 MB 00:00 2024-03-18T22:55:32.292 INFO:teuthology.orchestra.run.smithi175.stdout:(127/174): python3-jaraco-classes-3.2.1-5.el9.n 265 kB/s | 18 kB 00:00 2024-03-18T22:55:32.317 INFO:teuthology.orchestra.run.smithi175.stdout:(128/174): python3-grpcio-tools-1.46.7-10.el9.x 454 kB/s | 144 kB 00:00 2024-03-18T22:55:32.342 INFO:teuthology.orchestra.run.smithi175.stdout:(129/174): python3-jaraco-collections-3.0.0-8.e 307 kB/s | 23 kB 00:00 2024-03-18T22:55:32.368 INFO:teuthology.orchestra.run.smithi175.stdout:(130/174): python3-jaraco-functools-3.5.0-2.el9 257 kB/s | 19 kB 00:00 2024-03-18T22:55:32.393 INFO:teuthology.orchestra.run.smithi175.stdout:(131/174): python3-jaraco-text-3.2.0-6.el9.noar 260 kB/s | 20 kB 00:00 2024-03-18T22:55:32.452 INFO:teuthology.orchestra.run.smithi175.stdout:(132/174): python3-more-itertools-8.12.0-2.el9. 1.3 MB/s | 79 kB 00:00 2024-03-18T22:55:32.701 INFO:teuthology.orchestra.run.smithi175.stdout:(133/174): python3-natsort-7.1.1-5.el9.noarch.r 231 kB/s | 58 kB 00:00 2024-03-18T22:55:32.735 INFO:teuthology.orchestra.run.smithi175.stdout:(134/174): python3-logutils-0.3.5-21.el9.noarch 126 kB/s | 46 kB 00:00 2024-03-18T22:55:32.777 INFO:teuthology.orchestra.run.smithi175.stdout:(135/174): python3-kubernetes-26.1.0-2.el9.noar 2.4 MB/s | 1.0 MB 00:00 2024-03-18T22:55:32.802 INFO:teuthology.orchestra.run.smithi175.stdout:(136/174): python3-portend-3.1.0-2.el9.noarch.r 245 kB/s | 16 kB 00:00 2024-03-18T22:55:32.827 INFO:teuthology.orchestra.run.smithi175.stdout:(137/174): python3-pecan-1.4.2-3.el9.noarch.rpm 2.1 MB/s | 272 kB 00:00 2024-03-18T22:55:32.853 INFO:teuthology.orchestra.run.smithi175.stdout:(138/174): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.2 MB/s | 90 kB 00:00 2024-03-18T22:55:32.961 INFO:teuthology.orchestra.run.smithi175.stdout:(139/174): python3-pyperclip-1.8.2-2.el9.noarch 178 kB/s | 28 kB 00:00 2024-03-18T22:55:32.986 INFO:teuthology.orchestra.run.smithi175.stdout:(140/174): python3-repoze-lru-0.7-16.el9.noarch 193 kB/s | 31 kB 00:00 2024-03-18T22:55:33.012 INFO:teuthology.orchestra.run.smithi175.stdout:(141/174): python3-routes-2.5.1-5.el9.noarch.rp 1.2 MB/s | 188 kB 00:00 2024-03-18T22:55:33.070 INFO:teuthology.orchestra.run.smithi175.stdout:(142/174): python3-tempora-5.0.0-2.el9.noarch.r 428 kB/s | 36 kB 00:00 2024-03-18T22:55:33.171 INFO:teuthology.orchestra.run.smithi175.stdout:(143/174): python3-webob-1.8.7-6.el9.noarch.rpm 2.2 MB/s | 230 kB 00:00 2024-03-18T22:55:33.596 INFO:teuthology.orchestra.run.smithi175.stdout:(144/174): python3-rsa-4.9-2.el9.noarch.rpm 93 kB/s | 59 kB 00:00 2024-03-18T22:55:33.621 INFO:teuthology.orchestra.run.smithi175.stdout:(145/174): python3-websocket-client-1.2.3-2.el9 199 kB/s | 90 kB 00:00 2024-03-18T22:55:33.680 INFO:teuthology.orchestra.run.smithi175.stdout:(146/174): python3-typing-extensions-4.4.0-2.el 76 kB/s | 51 kB 00:00 2024-03-18T22:55:33.705 INFO:teuthology.orchestra.run.smithi175.stdout:(147/174): python3-zc-lockfile-2.0-10.el9.noarc 239 kB/s | 20 kB 00:00 2024-03-18T22:55:33.730 INFO:teuthology.orchestra.run.smithi175.stdout:(148/174): python3-werkzeug-2.0.3-3.el9.1.noarc 3.1 MB/s | 427 kB 00:00 2024-03-18T22:55:33.855 INFO:teuthology.orchestra.run.smithi175.stdout:(149/174): re2-20211101-3.el9.x86_64.rpm 1.1 MB/s | 192 kB 00:00 2024-03-18T22:55:33.897 INFO:teuthology.orchestra.run.smithi175.stdout:(150/174): thrift-0.15.0-2.el9.x86_64.rpm 8.2 MB/s | 1.6 MB 00:00 2024-03-18T22:55:33.948 INFO:teuthology.orchestra.run.smithi175.stdout:(151/174): xfsdump-3.1.12-4.el9.x86_64.rpm 6.3 MB/s | 321 kB 00:00 2024-03-18T22:55:34.081 INFO:teuthology.orchestra.run.smithi175.stdout:(152/174): librados2-19.0.0-2206.gb6ae2491.el9. 9.7 MB/s | 3.4 MB 00:00 2024-03-18T22:55:34.223 INFO:teuthology.orchestra.run.smithi175.stdout:(153/174): librbd1-19.0.0-2206.gb6ae2491.el9.x8 8.6 MB/s | 3.1 MB 00:00 2024-03-18T22:55:34.290 INFO:teuthology.orchestra.run.smithi175.stdout:(154/174): NetworkManager-team-1.47.2-1.el9.x86 633 kB/s | 42 kB 00:00 2024-03-18T22:55:34.415 INFO:teuthology.orchestra.run.smithi175.stdout:(155/174): NetworkManager-tui-1.47.2-1.el9.x86_ 1.9 MB/s | 248 kB 00:00 2024-03-18T22:55:34.499 INFO:teuthology.orchestra.run.smithi175.stdout:(156/174): acl-2.3.1-4.el9.x86_64.rpm 857 kB/s | 71 kB 00:00 2024-03-18T22:55:34.574 INFO:teuthology.orchestra.run.smithi175.stdout:(157/174): elfutils-debuginfod-client-0.190-2.e 485 kB/s | 36 kB 00:00 2024-03-18T22:55:34.625 INFO:teuthology.orchestra.run.smithi175.stdout:(158/174): NetworkManager-1.47.2-1.el9.x86_64.r 3.4 MB/s | 2.3 MB 00:00 2024-03-18T22:55:34.667 INFO:teuthology.orchestra.run.smithi175.stdout:(159/174): NetworkManager-libnm-1.47.2-1.el9.x8 3.1 MB/s | 1.8 MB 00:00 2024-03-18T22:55:34.700 INFO:teuthology.orchestra.run.smithi175.stdout:(160/174): elfutils-libelf-0.190-2.el9.x86_64.r 1.5 MB/s | 192 kB 00:00 2024-03-18T22:55:34.725 INFO:teuthology.orchestra.run.smithi175.stdout:(161/174): elfutils-libs-0.190-2.el9.x86_64.rpm 2.5 MB/s | 256 kB 00:00 2024-03-18T22:55:34.751 INFO:teuthology.orchestra.run.smithi175.stdout:(162/174): libacl-2.3.1-4.el9.x86_64.rpm 275 kB/s | 23 kB 00:00 2024-03-18T22:55:34.801 INFO:teuthology.orchestra.run.smithi175.stdout:(163/174): libfdisk-2.37.4-18.el9.x86_64.rpm 2.0 MB/s | 155 kB 00:00 2024-03-18T22:55:34.826 INFO:teuthology.orchestra.run.smithi175.stdout:(164/174): libmount-2.37.4-18.el9.x86_64.rpm 1.8 MB/s | 136 kB 00:00 2024-03-18T22:55:34.876 INFO:teuthology.orchestra.run.smithi175.stdout:(165/174): libsmartcols-2.37.4-18.el9.x86_64.rp 848 kB/s | 64 kB 00:00 2024-03-18T22:55:34.901 INFO:teuthology.orchestra.run.smithi175.stdout:(166/174): libuuid-2.37.4-18.el9.x86_64.rpm 382 kB/s | 29 kB 00:00 2024-03-18T22:55:35.068 INFO:teuthology.orchestra.run.smithi175.stdout:(167/174): libblkid-2.37.4-18.el9.x86_64.rpm 294 kB/s | 108 kB 00:00 2024-03-18T22:55:35.135 INFO:teuthology.orchestra.run.smithi175.stdout:(168/174): openssl-3.0.7-27.el9.x86_64.rpm 4.7 MB/s | 1.2 MB 00:00 2024-03-18T22:55:35.202 INFO:teuthology.orchestra.run.smithi175.stdout:(169/174): util-linux-core-2.37.4-18.el9.x86_64 6.8 MB/s | 465 kB 00:00 2024-03-18T22:55:35.277 INFO:teuthology.orchestra.run.smithi175.stdout:(170/174): util-linux-user-2.37.4-18.el9.x86_64 416 kB/s | 31 kB 00:00 2024-03-18T22:55:35.320 INFO:teuthology.orchestra.run.smithi175.stdout:(171/174): openssl-libs-3.0.7-27.el9.x86_64.rpm 5.2 MB/s | 2.2 MB 00:00 2024-03-18T22:55:35.386 INFO:teuthology.orchestra.run.smithi175.stdout:(172/174): libacl-devel-2.3.1-4.el9.x86_64.rpm 1.1 MB/s | 74 kB 00:00 2024-03-18T22:55:35.478 INFO:teuthology.orchestra.run.smithi175.stdout:(173/174): xfsprogs-6.3.0-1.el9.x86_64.rpm 5.4 MB/s | 1.1 MB 00:00 2024-03-18T22:55:35.795 INFO:teuthology.orchestra.run.smithi175.stdout:(174/174): util-linux-2.37.4-18.el9.x86_64.rpm 3.1 MB/s | 2.3 MB 00:00 2024-03-18T22:55:35.800 INFO:teuthology.orchestra.run.smithi175.stdout:-------------------------------------------------------------------------------- 2024-03-18T22:55:35.800 INFO:teuthology.orchestra.run.smithi175.stdout:Total 14 MB/s | 224 MB 00:16 2024-03-18T22:55:36.251 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction check 2024-03-18T22:55:36.522 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction check succeeded. 2024-03-18T22:55:36.522 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction test 2024-03-18T22:55:38.215 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction test succeeded. 2024-03-18T22:55:38.215 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction 2024-03-18T22:55:45.025 INFO:teuthology.orchestra.run.smithi175.stdout: Preparing : 1/1 2024-03-18T22:55:45.147 INFO:teuthology.orchestra.run.smithi175.stdout: Upgrading : openssl-libs-1:3.0.7-27.el9.x86_64 1/198 2024-03-18T22:55:45.181 INFO:teuthology.orchestra.run.smithi175.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/198 2024-03-18T22:55:45.212 INFO:teuthology.orchestra.run.smithi175.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-18T22:55:45.220 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-18T22:55:45.261 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/198 2024-03-18T22:55:45.314 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 5/198 2024-03-18T22:55:45.348 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/198 2024-03-18T22:55:45.374 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 7/198 2024-03-18T22:55:45.415 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 8/198 2024-03-18T22:55:45.736 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 9/198 2024-03-18T22:55:45.783 INFO:teuthology.orchestra.run.smithi175.stdout: Upgrading : librados2-2:19.0.0-2206.gb6ae2491.el9.x86_64 10/198 2024-03-18T22:55:45.903 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: librados2-2:19.0.0-2206.gb6ae2491.el9.x86_64 10/198 2024-03-18T22:55:45.926 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : libcephfs2-2:19.0.0-2206.gb6ae2491.el9.x86_64 11/198 2024-03-18T22:55:46.016 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: libcephfs2-2:19.0.0-2206.gb6ae2491.el9.x86_64 11/198 2024-03-18T22:55:46.047 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-rados-2:19.0.0-2206.gb6ae2491.el9.x86_64 12/198 2024-03-18T22:55:46.084 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : postgresql-private-libs-13.11-1.el9.x86_64 13/198 2024-03-18T22:55:46.252 INFO:teuthology.orchestra.run.smithi175.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 14/198 2024-03-18T22:55:46.302 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : postgresql-13.11-1.el9.x86_64 15/198 2024-03-18T22:55:46.330 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : libcephsqlite-2:19.0.0-2206.gb6ae2491.el9.x86_64 16/198 2024-03-18T22:55:46.434 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2206.gb6ae2491.el9.x86_64 16/198 2024-03-18T22:55:46.457 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : libradosstriper1-2:19.0.0-2206.gb6ae2491.el9.x86 17/198 2024-03-18T22:55:46.520 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2206.gb6ae2491.el9.x86 17/198 2024-03-18T22:55:46.569 INFO:teuthology.orchestra.run.smithi175.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/198 2024-03-18T22:55:46.592 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : qatlib-service-23.11.0-1.el9.x86_64 19/198 2024-03-18T22:55:46.607 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: qatlib-service-23.11.0-1.el9.x86_64 19/198 2024-03-18T22:55:46.827 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: qatlib-23.11.0-1.el9.x86_64 20/198 2024-03-18T22:55:46.856 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : qatlib-23.11.0-1.el9.x86_64 20/198 2024-03-18T22:55:46.897 INFO:teuthology.orchestra.run.smithi175.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 21/198 2024-03-18T22:55:46.999 INFO:teuthology.orchestra.run.smithi175.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 22/198 2024-03-18T22:55:47.106 INFO:teuthology.orchestra.run.smithi175.stdout: Upgrading : NetworkManager-libnm-1:1.47.2-1.el9.x86_64 23/198 2024-03-18T22:55:47.166 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.2-1.el9.x86_64 23/198 2024-03-18T22:55:47.239 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-18T22:55:47.301 INFO:teuthology.orchestra.run.smithi175.stdout: Upgrading : NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-18T22:55:47.499 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-18T22:55:47.605 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : re2-1:20211101-3.el9.x86_64 25/198 2024-03-18T22:55:47.691 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 26/198 2024-03-18T22:55:47.730 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 27/198 2024-03-18T22:55:47.757 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : liboath-2.6.7-2.el9.x86_64 28/198 2024-03-18T22:55:47.792 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/198 2024-03-18T22:55:47.866 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/198 2024-03-18T22:55:47.920 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 31/198 2024-03-18T22:55:47.959 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 32/198 2024-03-18T22:55:48.022 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 33/198 2024-03-18T22:55:48.058 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 34/198 2024-03-18T22:55:48.101 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 35/198 2024-03-18T22:55:48.134 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 36/198 2024-03-18T22:55:48.162 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : lua-5.4.4-4.el9.x86_64 37/198 2024-03-18T22:55:48.231 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 38/198 2024-03-18T22:55:48.267 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-ceph-common-2:19.0.0-2206.gb6ae2491.el9. 39/198 2024-03-18T22:55:48.310 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-ceph-argparse-2:19.0.0-2206.gb6ae2491.el 40/198 2024-03-18T22:55:48.339 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-cephfs-2:19.0.0-2206.gb6ae2491.el9.x86_6 41/198 2024-03-18T22:55:48.407 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 42/198 2024-03-18T22:55:48.453 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : luarocks-3.9.2-1.el9.noarch 43/198 2024-03-18T22:55:48.526 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 44/198 2024-03-18T22:55:48.820 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 45/198 2024-03-18T22:55:48.872 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 46/198 2024-03-18T22:55:49.397 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-rsa-4.9-2.el9.noarch 47/198 2024-03-18T22:55:49.475 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 48/198 2024-03-18T22:55:49.554 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 49/198 2024-03-18T22:55:49.604 INFO:teuthology.orchestra.run.smithi175.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-18T22:55:49.611 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-18T22:55:49.644 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : qatzip-libs-1.1.2-2.el9.x86_64 51/198 2024-03-18T22:55:49.693 INFO:teuthology.orchestra.run.smithi175.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 52/198 2024-03-18T22:55:49.756 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : librados-devel-2:19.0.0-2206.gb6ae2491.el9.x86_6 53/198 2024-03-18T22:55:49.785 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 54/198 2024-03-18T22:55:50.000 INFO:teuthology.orchestra.run.smithi175.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 55/198 2024-03-18T22:55:50.155 INFO:teuthology.orchestra.run.smithi175.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 56/198 2024-03-18T22:55:50.229 INFO:teuthology.orchestra.run.smithi175.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 57/198 2024-03-18T22:55:50.272 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 58/198 2024-03-18T22:55:50.402 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 59/198 2024-03-18T22:55:50.460 INFO:teuthology.orchestra.run.smithi175.stdout: Upgrading : openssl-1:3.0.7-27.el9.x86_64 60/198 2024-03-18T22:55:50.514 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 61/198 2024-03-18T22:55:50.577 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 62/198 2024-03-18T22:55:50.610 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 63/198 2024-03-18T22:55:50.646 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 64/198 2024-03-18T22:55:50.693 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 65/198 2024-03-18T22:55:50.729 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 66/198 2024-03-18T22:55:50.771 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 67/198 2024-03-18T22:55:50.813 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 68/198 2024-03-18T22:55:50.946 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 69/198 2024-03-18T22:55:50.986 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 70/198 2024-03-18T22:55:51.017 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 71/198 2024-03-18T22:55:51.058 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 72/198 2024-03-18T22:55:51.151 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 73/198 2024-03-18T22:55:51.199 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-google-auth-1:2.28.0-1.el9.noarch 74/198 2024-03-18T22:55:51.240 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 75/198 2024-03-18T22:55:51.267 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 76/198 2024-03-18T22:55:51.290 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 77/198 2024-03-18T22:55:51.361 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 78/198 2024-03-18T22:55:51.482 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 79/198 2024-03-18T22:55:51.583 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 80/198 2024-03-18T22:55:51.623 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 81/198 2024-03-18T22:55:51.672 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-ply-3.11-14.el9.noarch 82/198 2024-03-18T22:55:51.726 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 83/198 2024-03-18T22:55:51.910 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 84/198 2024-03-18T22:55:51.960 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 85/198 2024-03-18T22:55:52.087 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 86/198 2024-03-18T22:55:52.133 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-asyncssh-2.13.2-4.el9.noarch 87/198 2024-03-18T22:55:52.165 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 88/198 2024-03-18T22:55:52.196 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : fuse-2.9.9-15.el9.x86_64 89/198 2024-03-18T22:55:52.224 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 90/198 2024-03-18T22:55:52.258 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 91/198 2024-03-18T22:55:52.297 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : uuid-1.6.2-55.el9.x86_64 92/198 2024-03-18T22:55:52.329 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 93/198 2024-03-18T22:55:52.359 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 94/198 2024-03-18T22:55:52.434 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 95/198 2024-03-18T22:55:52.469 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 96/198 2024-03-18T22:55:52.502 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 97/198 2024-03-18T22:55:52.543 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 98/198 2024-03-18T22:55:52.578 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 99/198 2024-03-18T22:55:52.768 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 100/198 2024-03-18T22:55:53.062 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 101/198 2024-03-18T22:55:53.159 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 102/198 2024-03-18T22:55:53.190 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 103/198 2024-03-18T22:55:53.238 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 104/198 2024-03-18T22:55:53.335 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-packaging-20.9-5.el9.noarch 105/198 2024-03-18T22:55:53.392 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 106/198 2024-03-18T22:55:54.108 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 107/198 2024-03-18T22:55:54.155 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 108/198 2024-03-18T22:55:54.189 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 109/198 2024-03-18T22:55:54.237 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 110/198 2024-03-18T22:55:54.324 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 111/198 2024-03-18T22:55:54.365 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 112/198 2024-03-18T22:55:54.534 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : openblas-0.3.21-2.el9.x86_64 113/198 2024-03-18T22:55:54.571 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 114/198 2024-03-18T22:55:54.649 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 115/198 2024-03-18T22:55:55.280 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 116/198 2024-03-18T22:55:55.392 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 117/198 2024-03-18T22:55:56.598 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 118/198 2024-03-18T22:55:56.687 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 119/198 2024-03-18T22:55:56.718 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 120/198 2024-03-18T22:55:56.758 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : jq-1.6-16.el9.x86_64 121/198 2024-03-18T22:55:56.785 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 122/198 2024-03-18T22:55:57.098 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 123/198 2024-03-18T22:55:57.123 INFO:teuthology.orchestra.run.smithi175.stdout: Upgrading : librbd1-2:19.0.0-2206.gb6ae2491.el9.x86_64 124/198 2024-03-18T22:55:57.197 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: librbd1-2:19.0.0-2206.gb6ae2491.el9.x86_64 124/198 2024-03-18T22:55:57.228 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-rbd-2:19.0.0-2206.gb6ae2491.el9.x86_64 125/198 2024-03-18T22:55:57.264 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 126/198 2024-03-18T22:55:57.723 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 127/198 2024-03-18T22:55:57.751 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : librgw2-2:19.0.0-2206.gb6ae2491.el9.x86_64 128/198 2024-03-18T22:55:57.822 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: librgw2-2:19.0.0-2206.gb6ae2491.el9.x86_64 128/198 2024-03-18T22:55:57.845 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-rgw-2:19.0.0-2206.gb6ae2491.el9.x86_64 129/198 2024-03-18T22:56:00.094 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-common-2:19.0.0-2206.gb6ae2491.el9.x86_64 130/198 2024-03-18T22:56:00.132 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-common-2:19.0.0-2206.gb6ae2491.el9.x86_64 130/198 2024-03-18T22:56:00.670 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-common-2:19.0.0-2206.gb6ae2491.el9.x86_64 130/198 2024-03-18T22:56:00.702 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-base-2:19.0.0-2206.gb6ae2491.el9.x86_64 131/198 2024-03-18T22:56:00.775 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-base-2:19.0.0-2206.gb6ae2491.el9.x86_64 131/198 2024-03-18T22:56:00.776 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-03-18T22:56:00.776 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-03-18T22:56:00.776 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-03-18T22:56:00.803 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-selinux-2:19.0.0-2206.gb6ae2491.el9.x86_64 132/198 2024-03-18T22:56:12.533 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2206.gb6ae2491.el9.x86_64 132/198 2024-03-18T22:56:12.533 INFO:teuthology.orchestra.run.smithi175.stdout:skipping the directory /sys 2024-03-18T22:56:12.533 INFO:teuthology.orchestra.run.smithi175.stdout:skipping the directory /proc 2024-03-18T22:56:12.533 INFO:teuthology.orchestra.run.smithi175.stdout:skipping the directory /mnt 2024-03-18T22:56:12.534 INFO:teuthology.orchestra.run.smithi175.stdout:skipping the directory /var/tmp 2024-03-18T22:56:12.534 INFO:teuthology.orchestra.run.smithi175.stdout:skipping the directory /home 2024-03-18T22:56:12.534 INFO:teuthology.orchestra.run.smithi175.stdout:skipping the directory /root 2024-03-18T22:56:12.534 INFO:teuthology.orchestra.run.smithi175.stdout:skipping the directory /tmp 2024-03-18T22:56:12.534 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-03-18T22:56:12.700 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-volume-2:19.0.0-2206.gb6ae2491.el9.noarch 133/198 2024-03-18T22:56:12.720 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-volume-2:19.0.0-2206.gb6ae2491.el9.noarch 133/198 2024-03-18T22:56:12.720 INFO:teuthology.orchestra.run.smithi175.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-18T22:56:12.720 INFO:teuthology.orchestra.run.smithi175.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-03-18T22:56:12.720 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-03-18T22:56:14.187 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-osd-2:19.0.0-2206.gb6ae2491.el9.x86_64 134/198 2024-03-18T22:56:14.212 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-osd-2:19.0.0-2206.gb6ae2491.el9.x86_64 134/198 2024-03-18T22:56:14.212 INFO:teuthology.orchestra.run.smithi175.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-18T22:56:14.212 INFO:teuthology.orchestra.run.smithi175.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-03-18T22:56:14.212 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-18T22:56:14.212 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-18T22:56:14.212 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-03-18T22:56:14.429 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-mds-2:19.0.0-2206.gb6ae2491.el9.x86_64 135/198 2024-03-18T22:56:14.450 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-mds-2:19.0.0-2206.gb6ae2491.el9.x86_64 135/198 2024-03-18T22:56:14.451 INFO:teuthology.orchestra.run.smithi175.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-18T22:56:14.451 INFO:teuthology.orchestra.run.smithi175.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-03-18T22:56:14.451 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-18T22:56:14.451 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-18T22:56:14.451 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-03-18T22:56:14.904 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-mon-2:19.0.0-2206.gb6ae2491.el9.x86_64 136/198 2024-03-18T22:56:14.926 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-mon-2:19.0.0-2206.gb6ae2491.el9.x86_64 136/198 2024-03-18T22:56:14.926 INFO:teuthology.orchestra.run.smithi175.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-18T22:56:14.926 INFO:teuthology.orchestra.run.smithi175.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-03-18T22:56:14.926 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-18T22:56:14.927 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-18T22:56:14.927 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-03-18T22:56:15.023 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : automake-1.16.2-8.el9.noarch 137/198 2024-03-18T22:56:15.440 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: cephadm-2:19.0.0-2206.gb6ae2491.el9.noarch 138/198 2024-03-18T22:56:15.471 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : cephadm-2:19.0.0-2206.gb6ae2491.el9.noarch 138/198 2024-03-18T22:56:15.503 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2206.gb6ae2491.e 139/198 2024-03-18T22:56:15.562 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2206.gb6ae2491. 140/198 2024-03-18T22:56:15.586 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2206.gb6ae2491.el9.noa 141/198 2024-03-18T22:56:15.819 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2206.gb6ae2491.el9.noa 141/198 2024-03-18T22:56:15.853 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2206.gb6ae2491.el9.n 142/198 2024-03-18T22:56:16.811 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2206.gb6ae2491.el9.n 142/198 2024-03-18T22:56:16.837 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2206.gb6a 143/198 2024-03-18T22:56:16.937 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2206.gb6a 143/198 2024-03-18T22:56:17.099 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2206.gb6ae2491.el 144/198 2024-03-18T22:56:17.122 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-mgr-2:19.0.0-2206.gb6ae2491.el9.x86_64 145/198 2024-03-18T22:56:17.144 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2206.gb6ae2491.el9.x86_64 145/198 2024-03-18T22:56:17.144 INFO:teuthology.orchestra.run.smithi175.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-18T22:56:17.145 INFO:teuthology.orchestra.run.smithi175.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-03-18T22:56:17.145 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-18T22:56:17.145 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-18T22:56:17.145 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-03-18T22:56:17.189 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-mgr-rook-2:19.0.0-2206.gb6ae2491.el9.noarch 146/198 2024-03-18T22:56:17.200 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2206.gb6ae2491.el9.noarch 146/198 2024-03-18T22:56:17.245 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-2:19.0.0-2206.gb6ae2491.el9.x86_64 147/198 2024-03-18T22:56:18.116 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : libtool-2.4.6-45.el9.x86_64 148/198 2024-03-18T22:56:18.149 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-radosgw-2:19.0.0-2206.gb6ae2491.el9.x86_64 149/198 2024-03-18T22:56:18.172 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2206.gb6ae2491.el9.x86_64 149/198 2024-03-18T22:56:18.172 INFO:teuthology.orchestra.run.smithi175.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-18T22:56:18.172 INFO:teuthology.orchestra.run.smithi175.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-03-18T22:56:18.172 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-18T22:56:18.172 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-18T22:56:18.173 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-03-18T22:56:18.217 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2206.gb6ae2 150/198 2024-03-18T22:56:18.238 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2206.gb6ae2 150/198 2024-03-18T22:56:18.238 INFO:teuthology.orchestra.run.smithi175.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-18T22:56:18.239 INFO:teuthology.orchestra.run.smithi175.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-03-18T22:56:18.239 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-03-18T22:56:18.284 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : cephfs-mirror-2:19.0.0-2206.gb6ae2491.el9.x86_64 151/198 2024-03-18T22:56:18.306 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2206.gb6ae2491.el9.x86_64 151/198 2024-03-18T22:56:18.306 INFO:teuthology.orchestra.run.smithi175.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-18T22:56:18.306 INFO:teuthology.orchestra.run.smithi175.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-03-18T22:56:18.306 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-18T22:56:18.306 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-18T22:56:18.306 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-03-18T22:56:18.618 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : rbd-mirror-2:19.0.0-2206.gb6ae2491.el9.x86_64 152/198 2024-03-18T22:56:18.640 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2206.gb6ae2491.el9.x86_64 152/198 2024-03-18T22:56:18.640 INFO:teuthology.orchestra.run.smithi175.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-18T22:56:18.640 INFO:teuthology.orchestra.run.smithi175.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-03-18T22:56:18.640 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-18T22:56:18.640 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-18T22:56:18.640 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-03-18T22:56:23.088 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-test-2:19.0.0-2206.gb6ae2491.el9.x86_64 153/198 2024-03-18T22:56:23.129 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : rbd-fuse-2:19.0.0-2206.gb6ae2491.el9.x86_64 154/198 2024-03-18T22:56:23.168 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : rbd-nbd-2:19.0.0-2206.gb6ae2491.el9.x86_64 155/198 2024-03-18T22:56:23.195 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 156/198 2024-03-18T22:56:23.374 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : cephfs-shell-2:19.0.0-2206.gb6ae2491.el9.x86_64 157/198 2024-03-18T22:56:23.486 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : postgresql-contrib-13.11-1.el9.x86_64 158/198 2024-03-18T22:56:23.527 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-fuse-2:19.0.0-2206.gb6ae2491.el9.x86_64 159/198 2024-03-18T22:56:23.566 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 160/198 2024-03-18T22:56:23.635 INFO:teuthology.orchestra.run.smithi175.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 161/198 2024-03-18T22:56:23.670 INFO:teuthology.orchestra.run.smithi175.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 162/198 2024-03-18T22:56:23.694 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : libcephfs-devel-2:19.0.0-2206.gb6ae2491.el9.x86_ 163/198 2024-03-18T22:56:23.727 INFO:teuthology.orchestra.run.smithi175.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 164/198 2024-03-18T22:56:23.757 INFO:teuthology.orchestra.run.smithi175.stdout: Upgrading : NetworkManager-team-1:1.47.2-1.el9.x86_64 165/198 2024-03-18T22:56:23.810 INFO:teuthology.orchestra.run.smithi175.stdout: Upgrading : NetworkManager-tui-1:1.47.2-1.el9.x86_64 166/198 2024-03-18T22:56:23.879 INFO:teuthology.orchestra.run.smithi175.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 167/198 2024-03-18T22:56:23.913 INFO:teuthology.orchestra.run.smithi175.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 168/198 2024-03-18T22:56:24.533 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-18T22:56:24.569 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-18T22:56:24.588 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-18T22:56:25.399 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : cephfs-top-2:19.0.0-2206.gb6ae2491.el9.noarch 170/198 2024-03-18T22:56:25.480 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 171/198 2024-03-18T22:56:25.522 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 172/198 2024-03-18T22:56:25.602 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : flex-2.6.4-9.el9.x86_64 173/198 2024-03-18T22:56:25.659 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : bison-3.7.4-5.el9.x86_64 174/198 2024-03-18T22:56:25.684 INFO:teuthology.orchestra.run.smithi175.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 175/198 2024-03-18T22:56:25.709 INFO:teuthology.orchestra.run.smithi175.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 176/198 2024-03-18T22:56:25.736 INFO:teuthology.orchestra.run.smithi175.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 177/198 2024-03-18T22:56:25.736 INFO:teuthology.orchestra.run.smithi175.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-18T22:56:25.815 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-18T22:56:25.845 INFO:teuthology.orchestra.run.smithi175.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 179/198 2024-03-18T22:56:25.845 INFO:teuthology.orchestra.run.smithi175.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-18T22:56:25.913 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-18T22:56:25.947 INFO:teuthology.orchestra.run.smithi175.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 181/198 2024-03-18T22:56:25.972 INFO:teuthology.orchestra.run.smithi175.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 182/198 2024-03-18T22:56:25.997 INFO:teuthology.orchestra.run.smithi175.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 183/198 2024-03-18T22:56:26.022 INFO:teuthology.orchestra.run.smithi175.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 184/198 2024-03-18T22:56:26.055 INFO:teuthology.orchestra.run.smithi175.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 185/198 2024-03-18T22:56:26.103 INFO:teuthology.orchestra.run.smithi175.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 186/198 2024-03-18T22:56:26.144 INFO:teuthology.orchestra.run.smithi175.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 187/198 2024-03-18T22:56:26.169 INFO:teuthology.orchestra.run.smithi175.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 188/198 2024-03-18T22:56:26.203 INFO:teuthology.orchestra.run.smithi175.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 189/198 2024-03-18T22:56:26.228 INFO:teuthology.orchestra.run.smithi175.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 190/198 2024-03-18T22:56:26.253 INFO:teuthology.orchestra.run.smithi175.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 191/198 2024-03-18T22:56:26.259 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-18T22:56:26.259 INFO:teuthology.orchestra.run.smithi175.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-18T22:56:26.439 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-18T22:56:26.439 INFO:teuthology.orchestra.run.smithi175.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-18T22:56:26.522 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-18T22:56:26.547 INFO:teuthology.orchestra.run.smithi175.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 194/198 2024-03-18T22:56:26.572 INFO:teuthology.orchestra.run.smithi175.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 195/198 2024-03-18T22:56:26.605 INFO:teuthology.orchestra.run.smithi175.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 196/198 2024-03-18T22:56:26.631 INFO:teuthology.orchestra.run.smithi175.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 197/198 2024-03-18T22:56:26.695 INFO:teuthology.orchestra.run.smithi175.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-18T22:56:31.509 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-18T22:56:31.510 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-2:19.0.0-2206.gb6ae2491.el9.x86_64 1/198 2024-03-18T22:56:31.510 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-base-2:19.0.0-2206.gb6ae2491.el9.x86_64 2/198 2024-03-18T22:56:31.510 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-common-2:19.0.0-2206.gb6ae2491.el9.x86_64 3/198 2024-03-18T22:56:31.510 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-fuse-2:19.0.0-2206.gb6ae2491.el9.x86_64 4/198 2024-03-18T22:56:31.510 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2206.gb6ae2 5/198 2024-03-18T22:56:31.510 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-mds-2:19.0.0-2206.gb6ae2491.el9.x86_64 6/198 2024-03-18T22:56:31.510 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-mgr-2:19.0.0-2206.gb6ae2491.el9.x86_64 7/198 2024-03-18T22:56:31.510 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-mon-2:19.0.0-2206.gb6ae2491.el9.x86_64 8/198 2024-03-18T22:56:31.510 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-osd-2:19.0.0-2206.gb6ae2491.el9.x86_64 9/198 2024-03-18T22:56:31.510 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-radosgw-2:19.0.0-2206.gb6ae2491.el9.x86_64 10/198 2024-03-18T22:56:31.510 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-selinux-2:19.0.0-2206.gb6ae2491.el9.x86_64 11/198 2024-03-18T22:56:31.510 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-test-2:19.0.0-2206.gb6ae2491.el9.x86_64 12/198 2024-03-18T22:56:31.511 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : cephfs-mirror-2:19.0.0-2206.gb6ae2491.el9.x86_64 13/198 2024-03-18T22:56:31.511 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : cephfs-shell-2:19.0.0-2206.gb6ae2491.el9.x86_64 14/198 2024-03-18T22:56:31.511 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libcephfs-devel-2:19.0.0-2206.gb6ae2491.el9.x86_ 15/198 2024-03-18T22:56:31.511 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libcephfs2-2:19.0.0-2206.gb6ae2491.el9.x86_64 16/198 2024-03-18T22:56:31.511 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libcephsqlite-2:19.0.0-2206.gb6ae2491.el9.x86_64 17/198 2024-03-18T22:56:31.511 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : librados-devel-2:19.0.0-2206.gb6ae2491.el9.x86_6 18/198 2024-03-18T22:56:31.511 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libradosstriper1-2:19.0.0-2206.gb6ae2491.el9.x86 19/198 2024-03-18T22:56:31.511 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : librgw2-2:19.0.0-2206.gb6ae2491.el9.x86_64 20/198 2024-03-18T22:56:31.511 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2206.gb6ae2491.el 21/198 2024-03-18T22:56:31.511 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-ceph-common-2:19.0.0-2206.gb6ae2491.el9. 22/198 2024-03-18T22:56:31.511 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-cephfs-2:19.0.0-2206.gb6ae2491.el9.x86_6 23/198 2024-03-18T22:56:31.511 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-rados-2:19.0.0-2206.gb6ae2491.el9.x86_64 24/198 2024-03-18T22:56:31.511 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-rbd-2:19.0.0-2206.gb6ae2491.el9.x86_64 25/198 2024-03-18T22:56:31.512 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-rgw-2:19.0.0-2206.gb6ae2491.el9.x86_64 26/198 2024-03-18T22:56:31.512 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : rbd-fuse-2:19.0.0-2206.gb6ae2491.el9.x86_64 27/198 2024-03-18T22:56:31.512 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : rbd-mirror-2:19.0.0-2206.gb6ae2491.el9.x86_64 28/198 2024-03-18T22:56:31.512 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : rbd-nbd-2:19.0.0-2206.gb6ae2491.el9.x86_64 29/198 2024-03-18T22:56:31.512 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2206.gb6ae2491. 30/198 2024-03-18T22:56:31.528 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2206.gb6ae2491.el9.noa 31/198 2024-03-18T22:56:31.528 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2206.gb6ae2491.el9.n 32/198 2024-03-18T22:56:31.528 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2206.gb6a 33/198 2024-03-18T22:56:31.528 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2206.gb6ae2491.el 34/198 2024-03-18T22:56:31.528 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2206.gb6ae2491.el9.noarch 35/198 2024-03-18T22:56:31.528 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2206.gb6ae2491.e 36/198 2024-03-18T22:56:31.528 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-volume-2:19.0.0-2206.gb6ae2491.el9.noarch 37/198 2024-03-18T22:56:31.528 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : cephadm-2:19.0.0-2206.gb6ae2491.el9.noarch 38/198 2024-03-18T22:56:31.528 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : cephfs-top-2:19.0.0-2206.gb6ae2491.el9.noarch 39/198 2024-03-18T22:56:31.528 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : automake-1.16.2-8.el9.noarch 40/198 2024-03-18T22:56:31.528 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : bison-3.7.4-5.el9.x86_64 41/198 2024-03-18T22:56:31.528 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 42/198 2024-03-18T22:56:31.529 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : flex-2.6.4-9.el9.x86_64 43/198 2024-03-18T22:56:31.529 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 44/198 2024-03-18T22:56:31.529 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 45/198 2024-03-18T22:56:31.529 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 46/198 2024-03-18T22:56:31.529 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 47/198 2024-03-18T22:56:31.529 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 48/198 2024-03-18T22:56:31.529 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 49/198 2024-03-18T22:56:31.529 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 50/198 2024-03-18T22:56:31.529 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 51/198 2024-03-18T22:56:31.529 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 52/198 2024-03-18T22:56:31.529 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 53/198 2024-03-18T22:56:31.529 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : lua-5.4.4-4.el9.x86_64 54/198 2024-03-18T22:56:31.530 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 55/198 2024-03-18T22:56:31.530 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 56/198 2024-03-18T22:56:31.530 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 57/198 2024-03-18T22:56:31.530 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : postgresql-13.11-1.el9.x86_64 58/198 2024-03-18T22:56:31.530 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : postgresql-contrib-13.11-1.el9.x86_64 59/198 2024-03-18T22:56:31.530 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : postgresql-private-libs-13.11-1.el9.x86_64 60/198 2024-03-18T22:56:31.530 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : postgresql-server-13.11-1.el9.x86_64 61/198 2024-03-18T22:56:31.530 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 62/198 2024-03-18T22:56:31.530 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 63/198 2024-03-18T22:56:31.530 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 64/198 2024-03-18T22:56:31.530 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 65/198 2024-03-18T22:56:31.530 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 66/198 2024-03-18T22:56:31.530 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 67/198 2024-03-18T22:56:31.530 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 68/198 2024-03-18T22:56:31.531 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 69/198 2024-03-18T22:56:31.531 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 70/198 2024-03-18T22:56:31.531 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 71/198 2024-03-18T22:56:31.531 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 72/198 2024-03-18T22:56:31.531 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 73/198 2024-03-18T22:56:31.531 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 74/198 2024-03-18T22:56:31.531 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 75/198 2024-03-18T22:56:31.531 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 76/198 2024-03-18T22:56:31.531 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 77/198 2024-03-18T22:56:31.531 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 78/198 2024-03-18T22:56:31.532 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 79/198 2024-03-18T22:56:31.532 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 80/198 2024-03-18T22:56:31.532 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 81/198 2024-03-18T22:56:31.532 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 82/198 2024-03-18T22:56:31.532 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 83/198 2024-03-18T22:56:31.532 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 84/198 2024-03-18T22:56:31.532 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 85/198 2024-03-18T22:56:31.532 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 86/198 2024-03-18T22:56:31.532 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 87/198 2024-03-18T22:56:31.532 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 88/198 2024-03-18T22:56:31.532 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-ply-3.11-14.el9.noarch 89/198 2024-03-18T22:56:31.532 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 90/198 2024-03-18T22:56:31.532 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 91/198 2024-03-18T22:56:31.533 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : jq-1.6-16.el9.x86_64 92/198 2024-03-18T22:56:31.533 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 93/198 2024-03-18T22:56:31.533 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 94/198 2024-03-18T22:56:31.533 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 95/198 2024-03-18T22:56:31.533 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 96/198 2024-03-18T22:56:31.533 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 97/198 2024-03-18T22:56:31.533 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 98/198 2024-03-18T22:56:31.533 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 99/198 2024-03-18T22:56:31.533 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 100/198 2024-03-18T22:56:31.533 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : qatlib-23.11.0-1.el9.x86_64 101/198 2024-03-18T22:56:31.533 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : qatlib-service-23.11.0-1.el9.x86_64 102/198 2024-03-18T22:56:31.533 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : qatzip-libs-1.1.2-2.el9.x86_64 103/198 2024-03-18T22:56:31.533 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 104/198 2024-03-18T22:56:31.533 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/198 2024-03-18T22:56:31.534 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/198 2024-03-18T22:56:31.534 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/198 2024-03-18T22:56:31.534 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/198 2024-03-18T22:56:31.534 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/198 2024-03-18T22:56:31.534 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/198 2024-03-18T22:56:31.534 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/198 2024-03-18T22:56:31.534 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/198 2024-03-18T22:56:31.534 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/198 2024-03-18T22:56:31.534 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/198 2024-03-18T22:56:31.534 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/198 2024-03-18T22:56:31.534 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-asyncssh-2.13.2-4.el9.noarch 116/198 2024-03-18T22:56:31.535 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/198 2024-03-18T22:56:31.535 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/198 2024-03-18T22:56:31.535 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/198 2024-03-18T22:56:31.535 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/198 2024-03-18T22:56:31.535 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/198 2024-03-18T22:56:31.535 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/198 2024-03-18T22:56:31.535 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/198 2024-03-18T22:56:31.535 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-google-auth-1:2.28.0-1.el9.noarch 124/198 2024-03-18T22:56:31.535 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/198 2024-03-18T22:56:31.535 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/198 2024-03-18T22:56:31.535 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/198 2024-03-18T22:56:31.535 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/198 2024-03-18T22:56:31.535 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/198 2024-03-18T22:56:31.535 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/198 2024-03-18T22:56:31.536 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/198 2024-03-18T22:56:31.536 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/198 2024-03-18T22:56:31.536 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/198 2024-03-18T22:56:31.536 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/198 2024-03-18T22:56:31.536 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/198 2024-03-18T22:56:31.536 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/198 2024-03-18T22:56:31.536 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/198 2024-03-18T22:56:31.536 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/198 2024-03-18T22:56:31.536 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/198 2024-03-18T22:56:31.536 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/198 2024-03-18T22:56:31.536 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/198 2024-03-18T22:56:31.536 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/198 2024-03-18T22:56:31.536 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/198 2024-03-18T22:56:31.536 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/198 2024-03-18T22:56:31.536 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/198 2024-03-18T22:56:31.536 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/198 2024-03-18T22:56:31.537 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/198 2024-03-18T22:56:31.537 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/198 2024-03-18T22:56:31.537 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/198 2024-03-18T22:56:31.537 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/198 2024-03-18T22:56:31.537 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : librados2-2:19.0.0-2206.gb6ae2491.el9.x86_64 151/198 2024-03-18T22:56:31.537 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/198 2024-03-18T22:56:31.537 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : librbd1-2:19.0.0-2206.gb6ae2491.el9.x86_64 153/198 2024-03-18T22:56:31.537 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/198 2024-03-18T22:56:31.537 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 155/198 2024-03-18T22:56:31.537 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 156/198 2024-03-18T22:56:31.537 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : NetworkManager-1:1.47.2-1.el9.x86_64 157/198 2024-03-18T22:56:31.537 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 158/198 2024-03-18T22:56:31.537 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : NetworkManager-libnm-1:1.47.2-1.el9.x86_64 159/198 2024-03-18T22:56:31.537 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 160/198 2024-03-18T22:56:31.537 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : NetworkManager-team-1:1.47.2-1.el9.x86_64 161/198 2024-03-18T22:56:31.537 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 162/198 2024-03-18T22:56:31.538 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : NetworkManager-tui-1:1.47.2-1.el9.x86_64 163/198 2024-03-18T22:56:31.538 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 164/198 2024-03-18T22:56:31.538 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : acl-2.3.1-4.el9.x86_64 165/198 2024-03-18T22:56:31.538 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : acl-2.3.1-3.el9.x86_64 166/198 2024-03-18T22:56:31.538 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 167/198 2024-03-18T22:56:31.538 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 168/198 2024-03-18T22:56:31.538 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 169/198 2024-03-18T22:56:31.538 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 170/198 2024-03-18T22:56:31.538 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 171/198 2024-03-18T22:56:31.538 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 172/198 2024-03-18T22:56:31.538 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 173/198 2024-03-18T22:56:31.538 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 174/198 2024-03-18T22:56:31.538 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 175/198 2024-03-18T22:56:31.539 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 176/198 2024-03-18T22:56:31.539 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 177/198 2024-03-18T22:56:31.539 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 178/198 2024-03-18T22:56:31.539 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 179/198 2024-03-18T22:56:31.539 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 180/198 2024-03-18T22:56:31.539 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 181/198 2024-03-18T22:56:31.539 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 182/198 2024-03-18T22:56:31.539 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 183/198 2024-03-18T22:56:31.539 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 184/198 2024-03-18T22:56:31.539 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 185/198 2024-03-18T22:56:31.539 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 186/198 2024-03-18T22:56:31.539 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 187/198 2024-03-18T22:56:31.539 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 188/198 2024-03-18T22:56:31.540 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 189/198 2024-03-18T22:56:31.540 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 190/198 2024-03-18T22:56:31.540 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 191/198 2024-03-18T22:56:31.540 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 192/198 2024-03-18T22:56:31.540 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 193/198 2024-03-18T22:56:31.540 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 194/198 2024-03-18T22:56:31.540 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 195/198 2024-03-18T22:56:31.540 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 196/198 2024-03-18T22:56:31.540 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 197/198 2024-03-18T22:56:34.607 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 198/198 2024-03-18T22:56:34.607 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-03-18T22:56:34.607 INFO:teuthology.orchestra.run.smithi175.stdout:Upgraded: 2024-03-18T22:56:34.607 INFO:teuthology.orchestra.run.smithi175.stdout: NetworkManager-1:1.47.2-1.el9.x86_64 2024-03-18T22:56:34.607 INFO:teuthology.orchestra.run.smithi175.stdout: NetworkManager-libnm-1:1.47.2-1.el9.x86_64 2024-03-18T22:56:34.607 INFO:teuthology.orchestra.run.smithi175.stdout: NetworkManager-team-1:1.47.2-1.el9.x86_64 2024-03-18T22:56:34.607 INFO:teuthology.orchestra.run.smithi175.stdout: NetworkManager-tui-1:1.47.2-1.el9.x86_64 2024-03-18T22:56:34.608 INFO:teuthology.orchestra.run.smithi175.stdout: acl-2.3.1-4.el9.x86_64 2024-03-18T22:56:34.608 INFO:teuthology.orchestra.run.smithi175.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-03-18T22:56:34.608 INFO:teuthology.orchestra.run.smithi175.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-03-18T22:56:34.608 INFO:teuthology.orchestra.run.smithi175.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-03-18T22:56:34.608 INFO:teuthology.orchestra.run.smithi175.stdout: libacl-2.3.1-4.el9.x86_64 2024-03-18T22:56:34.608 INFO:teuthology.orchestra.run.smithi175.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-03-18T22:56:34.608 INFO:teuthology.orchestra.run.smithi175.stdout: libblkid-2.37.4-18.el9.x86_64 2024-03-18T22:56:34.608 INFO:teuthology.orchestra.run.smithi175.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-03-18T22:56:34.608 INFO:teuthology.orchestra.run.smithi175.stdout: libmount-2.37.4-18.el9.x86_64 2024-03-18T22:56:34.608 INFO:teuthology.orchestra.run.smithi175.stdout: librados2-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-18T22:56:34.608 INFO:teuthology.orchestra.run.smithi175.stdout: librbd1-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-18T22:56:34.608 INFO:teuthology.orchestra.run.smithi175.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-03-18T22:56:34.608 INFO:teuthology.orchestra.run.smithi175.stdout: libuuid-2.37.4-18.el9.x86_64 2024-03-18T22:56:34.608 INFO:teuthology.orchestra.run.smithi175.stdout: openssl-1:3.0.7-27.el9.x86_64 2024-03-18T22:56:34.609 INFO:teuthology.orchestra.run.smithi175.stdout: openssl-libs-1:3.0.7-27.el9.x86_64 2024-03-18T22:56:34.609 INFO:teuthology.orchestra.run.smithi175.stdout: util-linux-2.37.4-18.el9.x86_64 2024-03-18T22:56:34.609 INFO:teuthology.orchestra.run.smithi175.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-03-18T22:56:34.609 INFO:teuthology.orchestra.run.smithi175.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-03-18T22:56:34.609 INFO:teuthology.orchestra.run.smithi175.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-03-18T22:56:34.609 INFO:teuthology.orchestra.run.smithi175.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-03-18T22:56:34.609 INFO:teuthology.orchestra.run.smithi175.stdout:Installed: 2024-03-18T22:56:34.609 INFO:teuthology.orchestra.run.smithi175.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-03-18T22:56:34.609 INFO:teuthology.orchestra.run.smithi175.stdout: automake-1.16.2-8.el9.noarch 2024-03-18T22:56:34.609 INFO:teuthology.orchestra.run.smithi175.stdout: bison-3.7.4-5.el9.x86_64 2024-03-18T22:56:34.609 INFO:teuthology.orchestra.run.smithi175.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-03-18T22:56:34.609 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-18T22:56:34.609 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-base-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-18T22:56:34.609 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-common-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-18T22:56:34.609 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-fuse-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-18T22:56:34.610 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-grafana-dashboards-2:19.0.0-2206.gb6ae2491.el9.noarch 2024-03-18T22:56:34.610 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-immutable-object-cache-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-18T22:56:34.610 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mds-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-18T22:56:34.610 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mgr-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-18T22:56:34.610 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mgr-cephadm-2:19.0.0-2206.gb6ae2491.el9.noarch 2024-03-18T22:56:34.610 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mgr-dashboard-2:19.0.0-2206.gb6ae2491.el9.noarch 2024-03-18T22:56:34.610 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2206.gb6ae2491.el9.noarch 2024-03-18T22:56:34.610 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mgr-modules-core-2:19.0.0-2206.gb6ae2491.el9.noarch 2024-03-18T22:56:34.610 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mgr-rook-2:19.0.0-2206.gb6ae2491.el9.noarch 2024-03-18T22:56:34.610 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mon-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-18T22:56:34.610 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-osd-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-18T22:56:34.610 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-prometheus-alerts-2:19.0.0-2206.gb6ae2491.el9.noarch 2024-03-18T22:56:34.610 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-radosgw-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-18T22:56:34.611 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-selinux-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-18T22:56:34.611 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-test-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-18T22:56:34.611 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-volume-2:19.0.0-2206.gb6ae2491.el9.noarch 2024-03-18T22:56:34.611 INFO:teuthology.orchestra.run.smithi175.stdout: cephadm-2:19.0.0-2206.gb6ae2491.el9.noarch 2024-03-18T22:56:34.611 INFO:teuthology.orchestra.run.smithi175.stdout: cephfs-mirror-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-18T22:56:34.611 INFO:teuthology.orchestra.run.smithi175.stdout: cephfs-shell-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-18T22:56:34.611 INFO:teuthology.orchestra.run.smithi175.stdout: cephfs-top-2:19.0.0-2206.gb6ae2491.el9.noarch 2024-03-18T22:56:34.611 INFO:teuthology.orchestra.run.smithi175.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-03-18T22:56:34.611 INFO:teuthology.orchestra.run.smithi175.stdout: flex-2.6.4-9.el9.x86_64 2024-03-18T22:56:34.611 INFO:teuthology.orchestra.run.smithi175.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-03-18T22:56:34.611 INFO:teuthology.orchestra.run.smithi175.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-03-18T22:56:34.611 INFO:teuthology.orchestra.run.smithi175.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-03-18T22:56:34.611 INFO:teuthology.orchestra.run.smithi175.stdout: fuse-2.9.9-15.el9.x86_64 2024-03-18T22:56:34.611 INFO:teuthology.orchestra.run.smithi175.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-03-18T22:56:34.612 INFO:teuthology.orchestra.run.smithi175.stdout: grpc-data-1.46.7-10.el9.noarch 2024-03-18T22:56:34.612 INFO:teuthology.orchestra.run.smithi175.stdout: jq-1.6-16.el9.x86_64 2024-03-18T22:56:34.612 INFO:teuthology.orchestra.run.smithi175.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-03-18T22:56:34.612 INFO:teuthology.orchestra.run.smithi175.stdout: libarrow-9.0.0-10.el9.x86_64 2024-03-18T22:56:34.612 INFO:teuthology.orchestra.run.smithi175.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-03-18T22:56:34.612 INFO:teuthology.orchestra.run.smithi175.stdout: libcephfs-devel-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-18T22:56:34.612 INFO:teuthology.orchestra.run.smithi175.stdout: libcephfs2-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-18T22:56:34.612 INFO:teuthology.orchestra.run.smithi175.stdout: libcephsqlite-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-18T22:56:34.612 INFO:teuthology.orchestra.run.smithi175.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-03-18T22:56:34.612 INFO:teuthology.orchestra.run.smithi175.stdout: liboath-2.6.7-2.el9.x86_64 2024-03-18T22:56:34.612 INFO:teuthology.orchestra.run.smithi175.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-03-18T22:56:34.612 INFO:teuthology.orchestra.run.smithi175.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-03-18T22:56:34.612 INFO:teuthology.orchestra.run.smithi175.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-03-18T22:56:34.612 INFO:teuthology.orchestra.run.smithi175.stdout: librados-devel-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-18T22:56:34.613 INFO:teuthology.orchestra.run.smithi175.stdout: libradosstriper1-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-18T22:56:34.613 INFO:teuthology.orchestra.run.smithi175.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-03-18T22:56:34.613 INFO:teuthology.orchestra.run.smithi175.stdout: librgw2-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-18T22:56:34.613 INFO:teuthology.orchestra.run.smithi175.stdout: libtool-2.4.6-45.el9.x86_64 2024-03-18T22:56:34.613 INFO:teuthology.orchestra.run.smithi175.stdout: libunwind-1.6.2-1.el9.x86_64 2024-03-18T22:56:34.613 INFO:teuthology.orchestra.run.smithi175.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-03-18T22:56:34.613 INFO:teuthology.orchestra.run.smithi175.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-03-18T22:56:34.613 INFO:teuthology.orchestra.run.smithi175.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-03-18T22:56:34.613 INFO:teuthology.orchestra.run.smithi175.stdout: lua-5.4.4-4.el9.x86_64 2024-03-18T22:56:34.613 INFO:teuthology.orchestra.run.smithi175.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-03-18T22:56:34.613 INFO:teuthology.orchestra.run.smithi175.stdout: luarocks-3.9.2-1.el9.noarch 2024-03-18T22:56:34.613 INFO:teuthology.orchestra.run.smithi175.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-03-18T22:56:34.613 INFO:teuthology.orchestra.run.smithi175.stdout: openblas-0.3.21-2.el9.x86_64 2024-03-18T22:56:34.613 INFO:teuthology.orchestra.run.smithi175.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-03-18T22:56:34.614 INFO:teuthology.orchestra.run.smithi175.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-03-18T22:56:34.614 INFO:teuthology.orchestra.run.smithi175.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-03-18T22:56:34.614 INFO:teuthology.orchestra.run.smithi175.stdout: postgresql-13.11-1.el9.x86_64 2024-03-18T22:56:34.614 INFO:teuthology.orchestra.run.smithi175.stdout: postgresql-contrib-13.11-1.el9.x86_64 2024-03-18T22:56:34.614 INFO:teuthology.orchestra.run.smithi175.stdout: postgresql-private-libs-13.11-1.el9.x86_64 2024-03-18T22:56:34.614 INFO:teuthology.orchestra.run.smithi175.stdout: postgresql-server-13.11-1.el9.x86_64 2024-03-18T22:56:34.614 INFO:teuthology.orchestra.run.smithi175.stdout: protobuf-3.14.0-13.el9.x86_64 2024-03-18T22:56:34.614 INFO:teuthology.orchestra.run.smithi175.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-03-18T22:56:34.614 INFO:teuthology.orchestra.run.smithi175.stdout: python3-asyncssh-2.13.2-4.el9.noarch 2024-03-18T22:56:34.615 INFO:teuthology.orchestra.run.smithi175.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-03-18T22:56:34.615 INFO:teuthology.orchestra.run.smithi175.stdout: python3-babel-2.9.1-2.el9.noarch 2024-03-18T22:56:34.615 INFO:teuthology.orchestra.run.smithi175.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-03-18T22:56:34.615 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-03-18T22:56:34.615 INFO:teuthology.orchestra.run.smithi175.stdout: python3-ceph-argparse-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-18T22:56:34.615 INFO:teuthology.orchestra.run.smithi175.stdout: python3-ceph-common-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-18T22:56:34.615 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cephfs-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-18T22:56:34.615 INFO:teuthology.orchestra.run.smithi175.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-03-18T22:56:34.615 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-03-18T22:56:34.615 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-03-18T22:56:34.615 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-03-18T22:56:34.615 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-03-18T22:56:34.615 INFO:teuthology.orchestra.run.smithi175.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-03-18T22:56:34.616 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-03-18T22:56:34.616 INFO:teuthology.orchestra.run.smithi175.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-03-18T22:56:34.616 INFO:teuthology.orchestra.run.smithi175.stdout: python3-google-auth-1:2.28.0-1.el9.noarch 2024-03-18T22:56:34.616 INFO:teuthology.orchestra.run.smithi175.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-03-18T22:56:34.616 INFO:teuthology.orchestra.run.smithi175.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-03-18T22:56:34.616 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-03-18T22:56:34.616 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-03-18T22:56:34.616 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-03-18T22:56:34.616 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-03-18T22:56:34.616 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-03-18T22:56:34.616 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-03-18T22:56:34.616 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-03-18T22:56:34.616 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-03-18T22:56:34.616 INFO:teuthology.orchestra.run.smithi175.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-03-18T22:56:34.616 INFO:teuthology.orchestra.run.smithi175.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-03-18T22:56:34.617 INFO:teuthology.orchestra.run.smithi175.stdout: python3-mako-1.1.4-6.el9.noarch 2024-03-18T22:56:34.617 INFO:teuthology.orchestra.run.smithi175.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-03-18T22:56:34.617 INFO:teuthology.orchestra.run.smithi175.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-03-18T22:56:34.617 INFO:teuthology.orchestra.run.smithi175.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-03-18T22:56:34.617 INFO:teuthology.orchestra.run.smithi175.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-03-18T22:56:34.617 INFO:teuthology.orchestra.run.smithi175.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-03-18T22:56:34.617 INFO:teuthology.orchestra.run.smithi175.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-03-18T22:56:34.617 INFO:teuthology.orchestra.run.smithi175.stdout: python3-packaging-20.9-5.el9.noarch 2024-03-18T22:56:34.617 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-03-18T22:56:34.617 INFO:teuthology.orchestra.run.smithi175.stdout: python3-ply-3.11-14.el9.noarch 2024-03-18T22:56:34.617 INFO:teuthology.orchestra.run.smithi175.stdout: python3-portend-3.1.0-2.el9.noarch 2024-03-18T22:56:34.617 INFO:teuthology.orchestra.run.smithi175.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-03-18T22:56:34.617 INFO:teuthology.orchestra.run.smithi175.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-03-18T22:56:34.617 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-03-18T22:56:34.618 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-03-18T22:56:34.618 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-03-18T22:56:34.618 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pycparser-2.20-6.el9.noarch 2024-03-18T22:56:34.618 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-03-18T22:56:34.618 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pytz-2021.1-5.el9.noarch 2024-03-18T22:56:34.618 INFO:teuthology.orchestra.run.smithi175.stdout: python3-rados-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-18T22:56:34.618 INFO:teuthology.orchestra.run.smithi175.stdout: python3-rbd-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-18T22:56:34.618 INFO:teuthology.orchestra.run.smithi175.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-03-18T22:56:34.618 INFO:teuthology.orchestra.run.smithi175.stdout: python3-requests-2.25.1-8.el9.noarch 2024-03-18T22:56:34.618 INFO:teuthology.orchestra.run.smithi175.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-03-18T22:56:34.618 INFO:teuthology.orchestra.run.smithi175.stdout: python3-rgw-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-18T22:56:34.618 INFO:teuthology.orchestra.run.smithi175.stdout: python3-routes-2.5.1-5.el9.noarch 2024-03-18T22:56:34.618 INFO:teuthology.orchestra.run.smithi175.stdout: python3-rsa-4.9-2.el9.noarch 2024-03-18T22:56:34.618 INFO:teuthology.orchestra.run.smithi175.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-03-18T22:56:34.619 INFO:teuthology.orchestra.run.smithi175.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-03-18T22:56:34.619 INFO:teuthology.orchestra.run.smithi175.stdout: python3-toml-0.10.2-6.el9.noarch 2024-03-18T22:56:34.619 INFO:teuthology.orchestra.run.smithi175.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-03-18T22:56:34.619 INFO:teuthology.orchestra.run.smithi175.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-03-18T22:56:34.619 INFO:teuthology.orchestra.run.smithi175.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-03-18T22:56:34.619 INFO:teuthology.orchestra.run.smithi175.stdout: python3-webob-1.8.7-6.el9.noarch 2024-03-18T22:56:34.619 INFO:teuthology.orchestra.run.smithi175.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-03-18T22:56:34.619 INFO:teuthology.orchestra.run.smithi175.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-03-18T22:56:34.619 INFO:teuthology.orchestra.run.smithi175.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-03-18T22:56:34.619 INFO:teuthology.orchestra.run.smithi175.stdout: qatlib-23.11.0-1.el9.x86_64 2024-03-18T22:56:34.619 INFO:teuthology.orchestra.run.smithi175.stdout: qatlib-service-23.11.0-1.el9.x86_64 2024-03-18T22:56:34.620 INFO:teuthology.orchestra.run.smithi175.stdout: qatzip-libs-1.1.2-2.el9.x86_64 2024-03-18T22:56:34.620 INFO:teuthology.orchestra.run.smithi175.stdout: rbd-fuse-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-18T22:56:34.620 INFO:teuthology.orchestra.run.smithi175.stdout: rbd-mirror-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-18T22:56:34.620 INFO:teuthology.orchestra.run.smithi175.stdout: rbd-nbd-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-18T22:56:34.620 INFO:teuthology.orchestra.run.smithi175.stdout: re2-1:20211101-3.el9.x86_64 2024-03-18T22:56:34.620 INFO:teuthology.orchestra.run.smithi175.stdout: socat-1.7.4.1-5.el9.x86_64 2024-03-18T22:56:34.620 INFO:teuthology.orchestra.run.smithi175.stdout: thrift-0.15.0-2.el9.x86_64 2024-03-18T22:56:34.620 INFO:teuthology.orchestra.run.smithi175.stdout: uuid-1.6.2-55.el9.x86_64 2024-03-18T22:56:34.620 INFO:teuthology.orchestra.run.smithi175.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-03-18T22:56:34.620 INFO:teuthology.orchestra.run.smithi175.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-03-18T22:56:34.621 INFO:teuthology.orchestra.run.smithi175.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-03-18T22:56:34.621 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-03-18T22:56:34.621 INFO:teuthology.orchestra.run.smithi175.stdout:Complete! 2024-03-18T22:56:35.067 DEBUG:teuthology.parallel:result is None 2024-03-18T22:56:35.068 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b6ae24918b03b1b8a19c9263a50655bba80bb810 2024-03-18T22:56:35.295 DEBUG:teuthology.orchestra.run.smithi049:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-03-18T22:56:35.323 INFO:teuthology.orchestra.run.smithi049.stdout:19.0.0-2206.gb6ae2491.el9 2024-03-18T22:56:35.324 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2206.gb6ae2491.el9 2024-03-18T22:56:35.324 INFO:teuthology.task.install:The correct ceph version 19.0.0-2206.gb6ae2491 is installed. 2024-03-18T22:56:35.326 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b6ae24918b03b1b8a19c9263a50655bba80bb810 2024-03-18T22:56:35.538 DEBUG:teuthology.orchestra.run.smithi139:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-03-18T22:56:35.567 INFO:teuthology.orchestra.run.smithi139.stdout:19.0.0-2206.gb6ae2491.el9 2024-03-18T22:56:35.567 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2206.gb6ae2491.el9 2024-03-18T22:56:35.567 INFO:teuthology.task.install:The correct ceph version 19.0.0-2206.gb6ae2491 is installed. 2024-03-18T22:56:35.569 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b6ae24918b03b1b8a19c9263a50655bba80bb810 2024-03-18T22:56:35.782 DEBUG:teuthology.orchestra.run.smithi175:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-03-18T22:56:35.804 INFO:teuthology.orchestra.run.smithi175.stdout:19.0.0-2206.gb6ae2491.el9 2024-03-18T22:56:35.804 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2206.gb6ae2491.el9 2024-03-18T22:56:35.804 INFO:teuthology.task.install:The correct ceph version 19.0.0-2206.gb6ae2491 is installed. 2024-03-18T22:56:35.808 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-03-18T22:56:35.808 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-03-18T22:56:35.808 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-18T22:56:35.848 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-03-18T22:56:35.849 DEBUG:teuthology.orchestra.run.smithi139:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-18T22:56:35.883 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-03-18T22:56:35.884 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-18T22:56:35.914 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-03-18T22:56:35.914 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-03-18T22:56:35.914 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd of=/usr/bin/daemon-helper 2024-03-18T22:56:35.949 DEBUG:teuthology.orchestra.run.smithi049:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-18T22:56:36.023 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-03-18T22:56:36.024 DEBUG:teuthology.orchestra.run.smithi139:> sudo dd of=/usr/bin/daemon-helper 2024-03-18T22:56:36.055 DEBUG:teuthology.orchestra.run.smithi139:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-18T22:56:36.125 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-03-18T22:56:36.125 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd of=/usr/bin/daemon-helper 2024-03-18T22:56:36.153 DEBUG:teuthology.orchestra.run.smithi175:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-18T22:56:36.221 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-03-18T22:56:36.221 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-03-18T22:56:36.221 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-18T22:56:36.254 DEBUG:teuthology.orchestra.run.smithi049:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-18T22:56:36.327 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-03-18T22:56:36.327 DEBUG:teuthology.orchestra.run.smithi139:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-18T22:56:36.358 DEBUG:teuthology.orchestra.run.smithi139:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-18T22:56:36.428 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-03-18T22:56:36.428 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-18T22:56:36.455 DEBUG:teuthology.orchestra.run.smithi175:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-18T22:56:36.522 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-03-18T22:56:36.522 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-03-18T22:56:36.522 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd of=/usr/bin/stdin-killer 2024-03-18T22:56:36.555 DEBUG:teuthology.orchestra.run.smithi049:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-18T22:56:36.628 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-03-18T22:56:36.628 DEBUG:teuthology.orchestra.run.smithi139:> sudo dd of=/usr/bin/stdin-killer 2024-03-18T22:56:36.657 DEBUG:teuthology.orchestra.run.smithi139:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-18T22:56:36.726 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-03-18T22:56:36.726 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd of=/usr/bin/stdin-killer 2024-03-18T22:56:36.753 DEBUG:teuthology.orchestra.run.smithi175:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-18T22:56:36.820 INFO:teuthology.run_tasks:Running task cephadm... 2024-03-18T22:56:36.922 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'max_mds': 5, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_export_ephemeral_random_max': 0.1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug cephsqlite': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'PG_AVAILABILITY', 'PG_DEGRADED', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding'], 'sha1': 'b6ae24918b03b1b8a19c9263a50655bba80bb810', 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated'}} 2024-03-18T22:56:36.922 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 2024-03-18T22:56:36.923 INFO:tasks.cephadm:Cluster fsid is c63591ea-e57a-11ee-95c9-87774f69a715 2024-03-18T22:56:36.923 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-03-18T22:56:36.923 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.49', 'mon.b': '172.21.15.139', 'mon.c': '172.21.15.175'} 2024-03-18T22:56:36.924 INFO:tasks.cephadm:First mon is mon.a on smithi049 2024-03-18T22:56:36.924 INFO:tasks.cephadm:First mgr is x 2024-03-18T22:56:36.924 INFO:tasks.cephadm:Normalizing hostnames... 2024-03-18T22:56:36.924 DEBUG:teuthology.orchestra.run.smithi049:> sudo hostname $(hostname -s) 2024-03-18T22:56:36.957 DEBUG:teuthology.orchestra.run.smithi139:> sudo hostname $(hostname -s) 2024-03-18T22:56:36.988 DEBUG:teuthology.orchestra.run.smithi175:> sudo hostname $(hostname -s) 2024-03-18T22:56:37.015 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-03-18T22:56:37.016 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b6ae24918b03b1b8a19c9263a50655bba80bb810 2024-03-18T22:56:37.154 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': 'b6ae24918b03b1b8a19c9263a50655bba80bb810', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/77736/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-2206-gb6ae2491', 'node_name': '172.21.2.14+braggi14', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-2206.gb6ae2491'}, 'url': 'https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240318.181317/b6ae24918b03b1b8a19c9263a50655bba80bb810/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-03-18 20:42:27.872142', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-batrick-testing-20240318.181317', 'chacra_url': 'https://4.chacra.ceph.com/repos/ceph/wip-batrick-testing-20240318.181317/b6ae24918b03b1b8a19c9263a50655bba80bb810/centos/9/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2024-03-18T22:56:37.288 INFO:tasks.util.chacra:got chacra host 4.chacra.ceph.com, ref wip-batrick-testing-20240318.181317, sha1 b6ae24918b03b1b8a19c9263a50655bba80bb810 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=b6ae24918b03b1b8a19c9263a50655bba80bb810 2024-03-18T22:56:37.289 INFO:tasks.cephadm:Discovered cachra url: https://4.chacra.ceph.com/binaries/ceph/wip-batrick-testing-20240318.181317/b6ae24918b03b1b8a19c9263a50655bba80bb810/centos/9/x86_64/flavors/default/cephadm 2024-03-18T22:56:37.290 DEBUG:teuthology.orchestra.run.smithi049:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/wip-batrick-testing-20240318.181317/b6ae24918b03b1b8a19c9263a50655bba80bb810/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-03-18T22:56:38.371 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-r--r--. 1 ubuntu ubuntu 767404 Mar 18 22:56 /home/ubuntu/cephtest/cephadm 2024-03-18T22:56:38.371 DEBUG:teuthology.orchestra.run.smithi139:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/wip-batrick-testing-20240318.181317/b6ae24918b03b1b8a19c9263a50655bba80bb810/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-03-18T22:56:39.514 INFO:teuthology.orchestra.run.smithi139.stdout:-rw-r--r--. 1 ubuntu ubuntu 767404 Mar 18 22:56 /home/ubuntu/cephtest/cephadm 2024-03-18T22:56:39.515 DEBUG:teuthology.orchestra.run.smithi175:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/wip-batrick-testing-20240318.181317/b6ae24918b03b1b8a19c9263a50655bba80bb810/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-03-18T22:56:40.411 INFO:teuthology.orchestra.run.smithi175.stdout:-rw-r--r--. 1 ubuntu ubuntu 767404 Mar 18 22:56 /home/ubuntu/cephtest/cephadm 2024-03-18T22:56:40.411 DEBUG:teuthology.orchestra.run.smithi049:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-03-18T22:56:40.431 DEBUG:teuthology.orchestra.run.smithi139:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-03-18T22:56:40.450 DEBUG:teuthology.orchestra.run.smithi175:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-03-18T22:56:40.504 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 on all hosts... 2024-03-18T22:56:40.505 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 pull 2024-03-18T22:56:40.507 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 pull 2024-03-18T22:56:40.511 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 pull 2024-03-18T22:56:41.328 INFO:teuthology.orchestra.run.smithi049.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810... 2024-03-18T22:56:41.350 INFO:teuthology.orchestra.run.smithi175.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810... 2024-03-18T22:56:41.351 INFO:teuthology.orchestra.run.smithi139.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810... 2024-03-18T22:57:46.455 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-03-18T22:57:46.456 INFO:teuthology.orchestra.run.smithi139.stdout: "ceph_version": "ceph version 19.0.0-2206-gb6ae2491 (b6ae24918b03b1b8a19c9263a50655bba80bb810) squid (dev)", 2024-03-18T22:57:46.456 INFO:teuthology.orchestra.run.smithi139.stdout: "image_id": "2312c8e42d0069a2dbb2364bddb00264db4b800e7f18a7e7e4eb8f3ecab2ff1c", 2024-03-18T22:57:46.456 INFO:teuthology.orchestra.run.smithi139.stdout: "repo_digests": [ 2024-03-18T22:57:46.456 INFO:teuthology.orchestra.run.smithi139.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639" 2024-03-18T22:57:46.456 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-03-18T22:57:46.456 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-03-18T22:57:49.570 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-03-18T22:57:49.571 INFO:teuthology.orchestra.run.smithi175.stdout: "ceph_version": "ceph version 19.0.0-2206-gb6ae2491 (b6ae24918b03b1b8a19c9263a50655bba80bb810) squid (dev)", 2024-03-18T22:57:49.571 INFO:teuthology.orchestra.run.smithi175.stdout: "image_id": "2312c8e42d0069a2dbb2364bddb00264db4b800e7f18a7e7e4eb8f3ecab2ff1c", 2024-03-18T22:57:49.571 INFO:teuthology.orchestra.run.smithi175.stdout: "repo_digests": [ 2024-03-18T22:57:49.571 INFO:teuthology.orchestra.run.smithi175.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639" 2024-03-18T22:57:49.571 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-03-18T22:57:49.571 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-03-18T22:57:49.685 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-03-18T22:57:49.685 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph_version": "ceph version 19.0.0-2206-gb6ae2491 (b6ae24918b03b1b8a19c9263a50655bba80bb810) squid (dev)", 2024-03-18T22:57:49.685 INFO:teuthology.orchestra.run.smithi049.stdout: "image_id": "2312c8e42d0069a2dbb2364bddb00264db4b800e7f18a7e7e4eb8f3ecab2ff1c", 2024-03-18T22:57:49.685 INFO:teuthology.orchestra.run.smithi049.stdout: "repo_digests": [ 2024-03-18T22:57:49.685 INFO:teuthology.orchestra.run.smithi049.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639" 2024-03-18T22:57:49.685 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-03-18T22:57:49.685 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-03-18T22:57:49.717 DEBUG:teuthology.orchestra.run.smithi049:> sudo mkdir -p /etc/ceph 2024-03-18T22:57:49.752 DEBUG:teuthology.orchestra.run.smithi139:> sudo mkdir -p /etc/ceph 2024-03-18T22:57:49.786 DEBUG:teuthology.orchestra.run.smithi175:> sudo mkdir -p /etc/ceph 2024-03-18T22:57:49.816 DEBUG:teuthology.orchestra.run.smithi049:> sudo chmod 777 /etc/ceph 2024-03-18T22:57:49.849 DEBUG:teuthology.orchestra.run.smithi139:> sudo chmod 777 /etc/ceph 2024-03-18T22:57:49.879 DEBUG:teuthology.orchestra.run.smithi175:> sudo chmod 777 /etc/ceph 2024-03-18T22:57:49.905 INFO:tasks.cephadm:Writing seed config... 2024-03-18T22:57:49.906 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-03-18T22:57:49.906 INFO:tasks.cephadm: override: [client] debug client = 20 2024-03-18T22:57:49.907 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-03-18T22:57:49.907 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-03-18T22:57:49.907 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-03-18T22:57:49.907 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-03-18T22:57:49.907 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-03-18T22:57:49.907 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-03-18T22:57:49.907 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-03-18T22:57:49.907 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-03-18T22:57:49.907 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-03-18T22:57:49.907 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-03-18T22:57:49.907 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-03-18T22:57:49.907 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-03-18T22:57:49.907 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-03-18T22:57:49.907 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-03-18T22:57:49.907 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-03-18T22:57:49.908 INFO:tasks.cephadm: override: [mds] mds_export_ephemeral_random_max = 0.1 2024-03-18T22:57:49.908 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-03-18T22:57:49.908 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-03-18T22:57:49.908 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-03-18T22:57:49.908 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-03-18T22:57:49.908 INFO:tasks.cephadm: override: [mgr] debug cephsqlite = 20 2024-03-18T22:57:49.908 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-03-18T22:57:49.908 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-03-18T22:57:49.908 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-03-18T22:57:49.908 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-03-18T22:57:49.909 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-03-18T22:57:49.909 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-03-18T22:57:49.909 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-03-18T22:57:49.909 INFO:tasks.cephadm: override: [osd] bluestore compression mode = aggressive 2024-03-18T22:57:49.909 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-03-18T22:57:49.909 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-03-18T22:57:49.909 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-03-18T22:57:49.909 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-03-18T22:57:49.909 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-03-18T22:57:49.909 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-03-18T22:57:49.909 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-03-18T22:57:49.909 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-03-18T22:57:49.909 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-03-18T22:57:49.909 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-03-18T22:57:49.910 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-03-18T22:57:49.910 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-03-18T22:57:49.910 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-03-18T22:57:49.910 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10 2024-03-18T22:57:49.911 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-03-18T22:57:49.911 DEBUG:teuthology.orchestra.run.smithi049:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-03-18T22:57:49.930 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = c63591ea-e57a-11ee-95c9-87774f69a715 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore compression mode = aggressive bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug cephsqlite = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True mds_export_ephemeral_random_max = 0.1 osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-03-18T22:57:49.931 DEBUG:teuthology.orchestra.run.smithi049:mon.a> sudo journalctl -f -n 0 -u ceph-c63591ea-e57a-11ee-95c9-87774f69a715@mon.a.service 2024-03-18T22:57:49.973 DEBUG:teuthology.orchestra.run.smithi049:mgr.x> sudo journalctl -f -n 0 -u ceph-c63591ea-e57a-11ee-95c9-87774f69a715@mgr.x.service 2024-03-18T22:57:50.017 INFO:tasks.cephadm:Bootstrapping... 2024-03-18T22:57:50.017 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 -v bootstrap --fsid c63591ea-e57a-11ee-95c9-87774f69a715 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.49 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-03-18T22:57:50.220 INFO:teuthology.orchestra.run.smithi049.stdout:-------------------------------------------------------------------------------- 2024-03-18T22:57:50.220 INFO:teuthology.orchestra.run.smithi049.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810', '-v', 'bootstrap', '--fsid', 'c63591ea-e57a-11ee-95c9-87774f69a715', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.49', '--skip-admin-label'] 2024-03-18T22:57:50.244 INFO:teuthology.orchestra.run.smithi049.stdout:/bin/podman: stdout 4.9.3-dev 2024-03-18T22:57:50.245 INFO:teuthology.orchestra.run.smithi049.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-03-18T22:57:50.245 INFO:teuthology.orchestra.run.smithi049.stdout:Verifying podman|docker is present... 2024-03-18T22:57:50.270 INFO:teuthology.orchestra.run.smithi049.stdout:/bin/podman: stdout 4.9.3-dev 2024-03-18T22:57:50.270 INFO:teuthology.orchestra.run.smithi049.stdout:Verifying lvm2 is present... 2024-03-18T22:57:50.270 INFO:teuthology.orchestra.run.smithi049.stdout:Verifying time synchronization is in place... 2024-03-18T22:57:50.277 INFO:teuthology.orchestra.run.smithi049.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-03-18T22:57:50.277 INFO:teuthology.orchestra.run.smithi049.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-03-18T22:57:50.283 INFO:teuthology.orchestra.run.smithi049.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-03-18T22:57:50.284 INFO:teuthology.orchestra.run.smithi049.stdout:systemctl: stdout inactive 2024-03-18T22:57:50.290 INFO:teuthology.orchestra.run.smithi049.stdout:systemctl: stdout enabled 2024-03-18T22:57:50.296 INFO:teuthology.orchestra.run.smithi049.stdout:systemctl: stdout active 2024-03-18T22:57:50.296 INFO:teuthology.orchestra.run.smithi049.stdout:Unit chronyd.service is enabled and running 2024-03-18T22:57:50.296 INFO:teuthology.orchestra.run.smithi049.stdout:Repeating the final host check... 2024-03-18T22:57:50.319 INFO:teuthology.orchestra.run.smithi049.stdout:/bin/podman: stdout 4.9.3-dev 2024-03-18T22:57:50.319 INFO:teuthology.orchestra.run.smithi049.stdout:podman (/bin/podman) version 4.9.3 is present 2024-03-18T22:57:50.319 INFO:teuthology.orchestra.run.smithi049.stdout:systemctl is present 2024-03-18T22:57:50.320 INFO:teuthology.orchestra.run.smithi049.stdout:lvcreate is present 2024-03-18T22:57:50.326 INFO:teuthology.orchestra.run.smithi049.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-03-18T22:57:50.326 INFO:teuthology.orchestra.run.smithi049.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-03-18T22:57:50.332 INFO:teuthology.orchestra.run.smithi049.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-03-18T22:57:50.332 INFO:teuthology.orchestra.run.smithi049.stdout:systemctl: stdout inactive 2024-03-18T22:57:50.338 INFO:teuthology.orchestra.run.smithi049.stdout:systemctl: stdout enabled 2024-03-18T22:57:50.345 INFO:teuthology.orchestra.run.smithi049.stdout:systemctl: stdout active 2024-03-18T22:57:50.345 INFO:teuthology.orchestra.run.smithi049.stdout:Unit chronyd.service is enabled and running 2024-03-18T22:57:50.345 INFO:teuthology.orchestra.run.smithi049.stdout:Host looks OK 2024-03-18T22:57:50.345 INFO:teuthology.orchestra.run.smithi049.stdout:Cluster fsid: c63591ea-e57a-11ee-95c9-87774f69a715 2024-03-18T22:57:50.345 INFO:teuthology.orchestra.run.smithi049.stdout:Acquiring lock 139878235062176 on /run/cephadm/c63591ea-e57a-11ee-95c9-87774f69a715.lock 2024-03-18T22:57:50.345 INFO:teuthology.orchestra.run.smithi049.stdout:Lock 139878235062176 acquired on /run/cephadm/c63591ea-e57a-11ee-95c9-87774f69a715.lock 2024-03-18T22:57:50.345 INFO:teuthology.orchestra.run.smithi049.stdout:Verifying IP 172.21.15.49 port 3300 ... 2024-03-18T22:57:50.346 INFO:teuthology.orchestra.run.smithi049.stdout:Verifying IP 172.21.15.49 port 6789 ... 2024-03-18T22:57:50.346 INFO:teuthology.orchestra.run.smithi049.stdout:Base mon IP(s) is [172.21.15.49:3300, 172.21.15.49:6789], mon addrv is [v2:172.21.15.49:3300,v1:172.21.15.49:6789] 2024-03-18T22:57:50.349 INFO:teuthology.orchestra.run.smithi049.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp src 172.21.15.49 metric 100 2024-03-18T22:57:50.349 INFO:teuthology.orchestra.run.smithi049.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.49 metric 100 2024-03-18T22:57:50.352 INFO:teuthology.orchestra.run.smithi049.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-03-18T22:57:50.352 INFO:teuthology.orchestra.run.smithi049.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 1024 pref medium 2024-03-18T22:57:50.352 INFO:teuthology.orchestra.run.smithi049.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 100 pref medium 2024-03-18T22:57:50.354 INFO:teuthology.orchestra.run.smithi049.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-03-18T22:57:50.354 INFO:teuthology.orchestra.run.smithi049.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-03-18T22:57:50.354 INFO:teuthology.orchestra.run.smithi049.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-03-18T22:57:50.354 INFO:teuthology.orchestra.run.smithi049.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2024-03-18T22:57:50.355 INFO:teuthology.orchestra.run.smithi049.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:febd:1566/64 scope link noprefixroute 2024-03-18T22:57:50.355 INFO:teuthology.orchestra.run.smithi049.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-03-18T22:57:50.355 INFO:teuthology.orchestra.run.smithi049.stdout:Mon IP `172.21.15.49` is in CIDR network `172.21.0.0/20` 2024-03-18T22:57:50.356 INFO:teuthology.orchestra.run.smithi049.stdout:Mon IP `172.21.15.49` is in CIDR network `172.21.0.0/20` 2024-03-18T22:57:50.356 INFO:teuthology.orchestra.run.smithi049.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-03-18T22:57:50.356 INFO:teuthology.orchestra.run.smithi049.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-03-18T22:57:50.356 INFO:teuthology.orchestra.run.smithi049.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810... 2024-03-18T22:57:50.844 INFO:teuthology.orchestra.run.smithi049.stdout:/bin/podman: stdout 2312c8e42d0069a2dbb2364bddb00264db4b800e7f18a7e7e4eb8f3ecab2ff1c 2024-03-18T22:57:50.845 INFO:teuthology.orchestra.run.smithi049.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810... 2024-03-18T22:57:50.845 INFO:teuthology.orchestra.run.smithi049.stdout:/bin/podman: stderr Getting image source signatures 2024-03-18T22:57:50.845 INFO:teuthology.orchestra.run.smithi049.stdout:/bin/podman: stderr Copying blob sha256:f58b4670ef4403d11d1c912aa26a4dbcf73447df5302e9dc66f6a7fc0d9d1e47 2024-03-18T22:57:50.845 INFO:teuthology.orchestra.run.smithi049.stdout:/bin/podman: stderr Copying blob sha256:93b5a8d362aeee7e8b1f0bbf18238e1bb256c50573ad5b72aff9e382b1bb3ccb 2024-03-18T22:57:50.845 INFO:teuthology.orchestra.run.smithi049.stdout:/bin/podman: stderr Copying config sha256:2312c8e42d0069a2dbb2364bddb00264db4b800e7f18a7e7e4eb8f3ecab2ff1c 2024-03-18T22:57:50.845 INFO:teuthology.orchestra.run.smithi049.stdout:/bin/podman: stderr Writing manifest to image destination 2024-03-18T22:57:51.783 INFO:teuthology.orchestra.run.smithi049.stdout:ceph: stdout ceph version 19.0.0-2206-gb6ae2491 (b6ae24918b03b1b8a19c9263a50655bba80bb810) squid (dev) 2024-03-18T22:57:51.783 INFO:teuthology.orchestra.run.smithi049.stdout:Ceph version: ceph version 19.0.0-2206-gb6ae2491 (b6ae24918b03b1b8a19c9263a50655bba80bb810) squid (dev) 2024-03-18T22:57:51.783 INFO:teuthology.orchestra.run.smithi049.stdout:Extracting ceph user uid/gid from container image... 2024-03-18T22:57:52.706 INFO:teuthology.orchestra.run.smithi049.stdout:stat: stdout 167 167 2024-03-18T22:57:52.706 INFO:teuthology.orchestra.run.smithi049.stdout:Creating initial keys... 2024-03-18T22:57:53.587 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph-authtool: stdout AQDxxvhlySGEARAApXHnhn6ugxitPvRchMfsRw== 2024-03-18T22:57:54.493 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph-authtool: stdout AQDxxvhljzMENRAAKw49cf3HJ/Ue0Z5e93rnxA== 2024-03-18T22:57:55.364 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph-authtool: stdout AQDyxvhlfBvTLhAAr7aMwPuw4YE55etL74gplQ== 2024-03-18T22:57:55.365 INFO:teuthology.orchestra.run.smithi049.stdout:Creating initial monmap... 2024-03-18T22:57:56.238 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-03-18T22:57:56.239 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-03-18T22:57:56.239 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to c63591ea-e57a-11ee-95c9-87774f69a715 2024-03-18T22:57:56.239 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-03-18T22:57:56.239 INFO:teuthology.orchestra.run.smithi049.stdout:monmaptool for a [v2:172.21.15.49:3300,v1:172.21.15.49:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-03-18T22:57:56.239 INFO:teuthology.orchestra.run.smithi049.stdout:setting min_mon_release = quincy 2024-03-18T22:57:56.239 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/monmaptool: set fsid to c63591ea-e57a-11ee-95c9-87774f69a715 2024-03-18T22:57:56.239 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-03-18T22:57:56.239 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-18T22:57:56.239 INFO:teuthology.orchestra.run.smithi049.stdout:Creating mon... 2024-03-18T22:57:57.169 INFO:teuthology.orchestra.run.smithi049.stdout:create mon.a on 2024-03-18T22:57:57.380 INFO:teuthology.orchestra.run.smithi049.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-03-18T22:57:57.543 INFO:teuthology.orchestra.run.smithi049.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-03-18T22:57:57.757 INFO:teuthology.orchestra.run.smithi049.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-c63591ea-e57a-11ee-95c9-87774f69a715.target → /etc/systemd/system/ceph-c63591ea-e57a-11ee-95c9-87774f69a715.target. 2024-03-18T22:57:57.757 INFO:teuthology.orchestra.run.smithi049.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-c63591ea-e57a-11ee-95c9-87774f69a715.target → /etc/systemd/system/ceph-c63591ea-e57a-11ee-95c9-87774f69a715.target. 2024-03-18T22:57:58.007 INFO:teuthology.orchestra.run.smithi049.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-c63591ea-e57a-11ee-95c9-87774f69a715@mon.a 2024-03-18T22:57:58.007 INFO:teuthology.orchestra.run.smithi049.stdout:systemctl: stderr Failed to reset failed state of unit ceph-c63591ea-e57a-11ee-95c9-87774f69a715@mon.a.service: Unit ceph-c63591ea-e57a-11ee-95c9-87774f69a715@mon.a.service not loaded. 2024-03-18T22:57:58.195 INFO:teuthology.orchestra.run.smithi049.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-c63591ea-e57a-11ee-95c9-87774f69a715.target.wants/ceph-c63591ea-e57a-11ee-95c9-87774f69a715@mon.a.service → /etc/systemd/system/ceph-c63591ea-e57a-11ee-95c9-87774f69a715@.service. 2024-03-18T22:57:58.261 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:57:58 smithi049 systemd[1]: Starting Ceph mon.a for c63591ea-e57a-11ee-95c9-87774f69a715... 2024-03-18T22:57:58.529 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:57:58 smithi049 podman[62600]: 2024-03-18 22:57:58.443012674 +0000 UTC m=+0.106181903 container create dd9519965f8cbb923af25d99269514a3af5d89388176f2180639b31ef9933524 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810, name=ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mon-a, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240318.181317-b6ae249) 2024-03-18T22:57:58.530 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:57:58 smithi049 podman[62600]: 2024-03-18 22:57:58.355853759 +0000 UTC m=+0.019023000 image pull 2312c8e42d0069a2dbb2364bddb00264db4b800e7f18a7e7e4eb8f3ecab2ff1c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 2024-03-18T22:57:58.754 INFO:teuthology.orchestra.run.smithi049.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-18T22:57:58.754 INFO:teuthology.orchestra.run.smithi049.stdout:systemctl: stdout disabled 2024-03-18T22:57:58.761 INFO:teuthology.orchestra.run.smithi049.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-18T22:57:58.762 INFO:teuthology.orchestra.run.smithi049.stdout:systemctl: stdout inactive 2024-03-18T22:57:58.762 INFO:teuthology.orchestra.run.smithi049.stdout:firewalld.service is not enabled 2024-03-18T22:57:58.762 INFO:teuthology.orchestra.run.smithi049.stdout:Not possible to enable service . firewalld.service is not available 2024-03-18T22:57:58.762 INFO:teuthology.orchestra.run.smithi049.stdout:Waiting for mon to start... 2024-03-18T22:57:58.762 INFO:teuthology.orchestra.run.smithi049.stdout:Waiting for mon... 2024-03-18T22:57:58.998 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:57:58 smithi049 podman[62600]: 2024-03-18 22:57:58.662567797 +0000 UTC m=+0.325737038 container init dd9519965f8cbb923af25d99269514a3af5d89388176f2180639b31ef9933524 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810, name=ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mon-a, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240318.181317-b6ae249, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-03-18T22:57:58.999 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:57:58 smithi049 podman[62600]: 2024-03-18 22:57:58.665941459 +0000 UTC m=+0.329110688 container start dd9519965f8cbb923af25d99269514a3af5d89388176f2180639b31ef9933524 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810, name=ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mon-a, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, ceph=True, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-03-18T22:57:58.999 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:57:58 smithi049 ceph-mon[62619]: mkfs c63591ea-e57a-11ee-95c9-87774f69a715 2024-03-18T22:57:58.999 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:57:58 smithi049 bash[62600]: dd9519965f8cbb923af25d99269514a3af5d89388176f2180639b31ef9933524 2024-03-18T22:57:58.999 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:57:58 smithi049 ceph-mon[62619]: mon.a is new leader, mons a in quorum (ranks 0) 2024-03-18T22:57:58.999 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:57:58 smithi049 systemd[1]: Started Ceph mon.a for c63591ea-e57a-11ee-95c9-87774f69a715. 2024-03-18T22:57:59.835 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:57:59 smithi049 ceph-mon[62619]: mon.a is new leader, mons a in quorum (ranks 0) 2024-03-18T22:57:59.835 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:57:59 smithi049 ceph-mon[62619]: monmap epoch 1 2024-03-18T22:57:59.835 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:57:59 smithi049 ceph-mon[62619]: fsid c63591ea-e57a-11ee-95c9-87774f69a715 2024-03-18T22:57:59.835 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:57:59 smithi049 ceph-mon[62619]: last_changed 2024-03-18T22:57:55.689519+0000 2024-03-18T22:57:59.835 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:57:59 smithi049 ceph-mon[62619]: created 2024-03-18T22:57:55.689519+0000 2024-03-18T22:57:59.835 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:57:59 smithi049 ceph-mon[62619]: min_mon_release 19 (squid) 2024-03-18T22:57:59.836 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:57:59 smithi049 ceph-mon[62619]: election_strategy: 1 2024-03-18T22:57:59.836 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:57:59 smithi049 ceph-mon[62619]: 0: [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] mon.a 2024-03-18T22:57:59.836 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:57:59 smithi049 ceph-mon[62619]: fsmap 2024-03-18T22:57:59.836 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:57:59 smithi049 ceph-mon[62619]: osdmap e1: 0 total, 0 up, 0 in 2024-03-18T22:57:59.836 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:57:59 smithi049 ceph-mon[62619]: mgrmap e1: no daemons active 2024-03-18T22:57:59.836 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:57:59 smithi049 ceph-mon[62619]: from='client.? 172.21.15.49:0/3385647599' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-03-18T22:57:59.844 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout cluster: 2024-03-18T22:57:59.844 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout id: c63591ea-e57a-11ee-95c9-87774f69a715 2024-03-18T22:57:59.844 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-03-18T22:57:59.844 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout 2024-03-18T22:57:59.845 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout services: 2024-03-18T22:57:59.845 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.548215s) 2024-03-18T22:57:59.845 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-03-18T22:57:59.845 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-03-18T22:57:59.845 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout 2024-03-18T22:57:59.845 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout data: 2024-03-18T22:57:59.845 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-03-18T22:57:59.845 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-03-18T22:57:59.845 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-03-18T22:57:59.845 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout pgs: 2024-03-18T22:57:59.845 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout 2024-03-18T22:57:59.845 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:57:59.199+0000 7f337ce2e700 1 Processor -- start 2024-03-18T22:57:59.845 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:57:59.199+0000 7f337ce2e700 1 -- start start 2024-03-18T22:57:59.845 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:57:59.199+0000 7f337ce2e700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f337807de20 0x7f337807c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T22:57:59.846 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:57:59.199+0000 7f337ce2e700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f337807c8a0 con 0x7f337807de20 2024-03-18T22:57:59.846 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:57:59.200+0000 7f337659c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f337807de20 0x7f337807c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T22:57:59.846 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:57:59.200+0000 7f337659c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f337807de20 0x7f337807c2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:50912/0 (socket says 172.21.15.49:50912) 2024-03-18T22:57:59.846 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:57:59.200+0000 7f337659c700 1 -- 172.21.15.49:0/1717456599 learned_addr learned my addr 172.21.15.49:0/1717456599 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T22:57:59.846 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:57:59.200+0000 7f337659c700 1 -- 172.21.15.49:0/1717456599 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f337807ca20 con 0x7f337807de20 2024-03-18T22:57:59.846 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:57:59.200+0000 7f337659c700 1 --2- 172.21.15.49:0/1717456599 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f337807de20 0x7f337807c2d0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f33680098a0 tx=0x7f3368009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2aeb11fcb25fbd9 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T22:57:59.846 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:57:59.200+0000 7f337559a700 1 -- 172.21.15.49:0/1717456599 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3368004160 con 0x7f337807de20 2024-03-18T22:57:59.846 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:57:59.200+0000 7f337559a700 1 -- 172.21.15.49:0/1717456599 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3368004300 con 0x7f337807de20 2024-03-18T22:57:59.846 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:57:59.200+0000 7f337559a700 1 -- 172.21.15.49:0/1717456599 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f33680036a0 con 0x7f337807de20 2024-03-18T22:57:59.846 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:57:59.201+0000 7f337ce2e700 1 -- 172.21.15.49:0/1717456599 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f337807de20 msgr2=0x7f337807c2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T22:57:59.846 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:57:59.201+0000 7f337ce2e700 1 --2- 172.21.15.49:0/1717456599 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f337807de20 0x7f337807c2d0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f33680098a0 tx=0x7f3368009bb0 comp rx=0 tx=0).stop 2024-03-18T22:57:59.846 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:57:59.201+0000 7f337ce2e700 1 -- 172.21.15.49:0/1717456599 shutdown_connections 2024-03-18T22:57:59.847 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:57:59.201+0000 7f337ce2e700 1 --2- 172.21.15.49:0/1717456599 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f337807de20 0x7f337807c2d0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T22:57:59.847 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:57:59.201+0000 7f337ce2e700 1 -- 172.21.15.49:0/1717456599 >> 172.21.15.49:0/1717456599 conn(0x7f33781032f0 msgr2=0x7f3378105710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T22:57:59.847 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:57:59.201+0000 7f337ce2e700 1 -- 172.21.15.49:0/1717456599 shutdown_connections 2024-03-18T22:57:59.847 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:57:59.201+0000 7f337ce2e700 1 -- 172.21.15.49:0/1717456599 wait complete. 2024-03-18T22:57:59.847 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:57:59.202+0000 7f337ce2e700 1 Processor -- start 2024-03-18T22:57:59.847 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:57:59.202+0000 7f337ce2e700 1 -- start start 2024-03-18T22:57:59.847 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:57:59.202+0000 7f337ce2e700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f337807de20 0x7f337819d500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T22:57:59.847 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:57:59.202+0000 7f337ce2e700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f337807cf50 con 0x7f337807de20 2024-03-18T22:57:59.847 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:57:59.202+0000 7f337659c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f337807de20 0x7f337819d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T22:57:59.847 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:57:59.202+0000 7f337659c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f337807de20 0x7f337819d500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:50920/0 (socket says 172.21.15.49:50920) 2024-03-18T22:57:59.847 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:57:59.202+0000 7f337659c700 1 -- 172.21.15.49:0/3385647599 learned_addr learned my addr 172.21.15.49:0/3385647599 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T22:57:59.847 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:57:59.202+0000 7f337659c700 1 -- 172.21.15.49:0/3385647599 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f337819da40 con 0x7f337807de20 2024-03-18T22:57:59.847 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:57:59.202+0000 7f337659c700 1 --2- 172.21.15.49:0/3385647599 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f337807de20 0x7f337819d500 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f3368000c00 tx=0x7f3368004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T22:57:59.847 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:57:59.203+0000 7f33677fe700 1 -- 172.21.15.49:0/3385647599 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3368003db0 con 0x7f337807de20 2024-03-18T22:57:59.848 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:57:59.203+0000 7f33677fe700 1 -- 172.21.15.49:0/3385647599 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3368011630 con 0x7f337807de20 2024-03-18T22:57:59.848 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:57:59.203+0000 7f337ce2e700 1 -- 172.21.15.49:0/3385647599 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f337819dcd0 con 0x7f337807de20 2024-03-18T22:57:59.848 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:57:59.203+0000 7f33677fe700 1 -- 172.21.15.49:0/3385647599 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3368011950 con 0x7f337807de20 2024-03-18T22:57:59.848 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:57:59.203+0000 7f337ce2e700 1 -- 172.21.15.49:0/3385647599 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33781a09c0 con 0x7f337807de20 2024-03-18T22:57:59.848 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:57:59.203+0000 7f33677fe700 1 -- 172.21.15.49:0/3385647599 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f3368011af0 con 0x7f337807de20 2024-03-18T22:57:59.848 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:57:59.203+0000 7f33677fe700 1 -- 172.21.15.49:0/3385647599 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f336801c450 con 0x7f337807de20 2024-03-18T22:57:59.848 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:57:59.204+0000 7f337ce2e700 1 -- 172.21.15.49:0/3385647599 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f33780693e0 con 0x7f337807de20 2024-03-18T22:57:59.848 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:57:59.206+0000 7f33677fe700 1 -- 172.21.15.49:0/3385647599 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f3368032020 con 0x7f337807de20 2024-03-18T22:57:59.848 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:57:59.281+0000 7f337ce2e700 1 -- 172.21.15.49:0/3385647599 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7f33781a0eb0 con 0x7f337807de20 2024-03-18T22:57:59.848 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:57:59.281+0000 7f33677fe700 1 -- 172.21.15.49:0/3385647599 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+317 (secure 0 0 0) 0x7f3368004260 con 0x7f337807de20 2024-03-18T22:57:59.848 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:57:59.282+0000 7f337ce2e700 1 -- 172.21.15.49:0/3385647599 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f337807de20 msgr2=0x7f337819d500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T22:57:59.848 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:57:59.282+0000 7f337ce2e700 1 --2- 172.21.15.49:0/3385647599 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f337807de20 0x7f337819d500 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f3368000c00 tx=0x7f3368004750 comp rx=0 tx=0).stop 2024-03-18T22:57:59.848 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:57:59.282+0000 7f337ce2e700 1 -- 172.21.15.49:0/3385647599 shutdown_connections 2024-03-18T22:57:59.849 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:57:59.282+0000 7f337ce2e700 1 --2- 172.21.15.49:0/3385647599 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f337807de20 0x7f337819d500 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T22:57:59.849 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:57:59.282+0000 7f337ce2e700 1 -- 172.21.15.49:0/3385647599 >> 172.21.15.49:0/3385647599 conn(0x7f33781032f0 msgr2=0x7f33781040e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T22:57:59.849 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:57:59.282+0000 7f337ce2e700 1 -- 172.21.15.49:0/3385647599 shutdown_connections 2024-03-18T22:57:59.849 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:57:59.282+0000 7f337ce2e700 1 -- 172.21.15.49:0/3385647599 wait complete. 2024-03-18T22:57:59.849 INFO:teuthology.orchestra.run.smithi049.stdout:mon is available 2024-03-18T22:57:59.849 INFO:teuthology.orchestra.run.smithi049.stdout:Assimilating anything we can from ceph.conf... 2024-03-18T22:58:00.941 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:00 smithi049 ceph-mon[62619]: from='client.? 172.21.15.49:0/4185564033' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-03-18T22:58:00.941 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:00 smithi049 ceph-mon[62619]: from='client.? 172.21.15.49:0/4185564033' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-03-18T22:58:00.951 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout 2024-03-18T22:58:00.951 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout [global] 2024-03-18T22:58:00.951 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout fsid = c63591ea-e57a-11ee-95c9-87774f69a715 2024-03-18T22:58:00.951 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.49:3300,v1:172.21.15.49:6789] 2024-03-18T22:58:00.951 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-03-18T22:58:00.951 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-03-18T22:58:00.951 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-03-18T22:58:00.951 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-03-18T22:58:00.951 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout 2024-03-18T22:58:00.951 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout [mgr] 2024-03-18T22:58:00.951 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-03-18T22:58:00.951 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout 2024-03-18T22:58:00.952 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout [osd] 2024-03-18T22:58:00.952 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-03-18T22:58:00.952 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-03-18T22:58:00.952 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-03-18T22:58:00.952 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-03-18T22:58:00.952 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-03-18T22:58:00.952 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:00.259+0000 7f3ab8a0c700 1 Processor -- start 2024-03-18T22:58:00.952 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:00.259+0000 7f3ab8a0c700 1 -- start start 2024-03-18T22:58:00.952 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:00.259+0000 7f3ab8a0c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3ab4106c00 0x7f3ab4107020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T22:58:00.952 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:00.259+0000 7f3ab8a0c700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f3ab41075f0 con 0x7f3ab4106c00 2024-03-18T22:58:00.952 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:00.259+0000 7f3ab259c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3ab4106c00 0x7f3ab4107020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T22:58:00.952 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:00.260+0000 7f3ab259c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3ab4106c00 0x7f3ab4107020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:50924/0 (socket says 172.21.15.49:50924) 2024-03-18T22:58:00.952 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:00.260+0000 7f3ab259c700 1 -- 172.21.15.49:0/4171615458 learned_addr learned my addr 172.21.15.49:0/4171615458 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T22:58:00.952 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:00.260+0000 7f3ab259c700 1 -- 172.21.15.49:0/4171615458 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ab4107e00 con 0x7f3ab4106c00 2024-03-18T22:58:00.952 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:00.260+0000 7f3ab259c700 1 --2- 172.21.15.49:0/4171615458 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3ab4106c00 0x7f3ab4107020 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f3aa40098a0 tx=0x7f3aa4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2542f7d18a2c8094 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T22:58:00.953 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:00.260+0000 7f3ab159a700 1 -- 172.21.15.49:0/4171615458 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3aa4004160 con 0x7f3ab4106c00 2024-03-18T22:58:00.953 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:00.260+0000 7f3ab159a700 1 -- 172.21.15.49:0/4171615458 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3aa4004300 con 0x7f3ab4106c00 2024-03-18T22:58:00.953 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:00.260+0000 7f3ab159a700 1 -- 172.21.15.49:0/4171615458 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3aa40036a0 con 0x7f3ab4106c00 2024-03-18T22:58:00.953 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:00.261+0000 7f3ab8a0c700 1 -- 172.21.15.49:0/4171615458 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3ab4106c00 msgr2=0x7f3ab4107020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T22:58:00.953 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:00.261+0000 7f3ab8a0c700 1 --2- 172.21.15.49:0/4171615458 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3ab4106c00 0x7f3ab4107020 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f3aa40098a0 tx=0x7f3aa4009bb0 comp rx=0 tx=0).stop 2024-03-18T22:58:00.953 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:00.261+0000 7f3ab8a0c700 1 -- 172.21.15.49:0/4171615458 shutdown_connections 2024-03-18T22:58:00.953 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:00.261+0000 7f3ab8a0c700 1 --2- 172.21.15.49:0/4171615458 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3ab4106c00 0x7f3ab4107020 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T22:58:00.953 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:00.261+0000 7f3ab8a0c700 1 -- 172.21.15.49:0/4171615458 >> 172.21.15.49:0/4171615458 conn(0x7f3ab41021a0 msgr2=0x7f3ab41045e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T22:58:00.953 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:00.261+0000 7f3ab8a0c700 1 -- 172.21.15.49:0/4171615458 shutdown_connections 2024-03-18T22:58:00.953 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:00.261+0000 7f3ab8a0c700 1 -- 172.21.15.49:0/4171615458 wait complete. 2024-03-18T22:58:00.953 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:00.262+0000 7f3ab8a0c700 1 Processor -- start 2024-03-18T22:58:00.953 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:00.262+0000 7f3ab8a0c700 1 -- start start 2024-03-18T22:58:00.953 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:00.262+0000 7f3ab8a0c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3ab4106c00 0x7f3ab419b420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T22:58:00.953 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:00.262+0000 7f3ab8a0c700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f3ab4108330 con 0x7f3ab4106c00 2024-03-18T22:58:00.954 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:00.262+0000 7f3ab259c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3ab4106c00 0x7f3ab419b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T22:58:00.954 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:00.262+0000 7f3ab259c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3ab4106c00 0x7f3ab419b420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:50934/0 (socket says 172.21.15.49:50934) 2024-03-18T22:58:00.954 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:00.262+0000 7f3ab259c700 1 -- 172.21.15.49:0/4185564033 learned_addr learned my addr 172.21.15.49:0/4185564033 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T22:58:00.954 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:00.262+0000 7f3ab259c700 1 -- 172.21.15.49:0/4185564033 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ab419b960 con 0x7f3ab4106c00 2024-03-18T22:58:00.954 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:00.262+0000 7f3ab259c700 1 --2- 172.21.15.49:0/4185564033 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3ab4106c00 0x7f3ab419b420 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f3aa4000c00 tx=0x7f3aa4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T22:58:00.954 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:00.262+0000 7f3aa37fe700 1 -- 172.21.15.49:0/4185564033 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3aa4003db0 con 0x7f3ab4106c00 2024-03-18T22:58:00.954 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:00.263+0000 7f3aa37fe700 1 -- 172.21.15.49:0/4185564033 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3aa4011630 con 0x7f3ab4106c00 2024-03-18T22:58:00.954 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:00.263+0000 7f3ab8a0c700 1 -- 172.21.15.49:0/4185564033 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ab419bbf0 con 0x7f3ab4106c00 2024-03-18T22:58:00.954 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:00.263+0000 7f3aa37fe700 1 -- 172.21.15.49:0/4185564033 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3aa4011950 con 0x7f3ab4106c00 2024-03-18T22:58:00.954 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:00.263+0000 7f3ab8a0c700 1 -- 172.21.15.49:0/4185564033 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ab419e8e0 con 0x7f3ab4106c00 2024-03-18T22:58:00.954 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:00.263+0000 7f3aa37fe700 1 -- 172.21.15.49:0/4185564033 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f3aa4011af0 con 0x7f3ab4106c00 2024-03-18T22:58:00.954 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:00.263+0000 7f3aa37fe700 1 -- 172.21.15.49:0/4185564033 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f3aa401c790 con 0x7f3ab4106c00 2024-03-18T22:58:00.954 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:00.264+0000 7f3ab8a0c700 1 -- 172.21.15.49:0/4185564033 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ab40693e0 con 0x7f3ab4106c00 2024-03-18T22:58:00.955 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:00.266+0000 7f3aa37fe700 1 -- 172.21.15.49:0/4185564033 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f3aa4033080 con 0x7f3ab4106c00 2024-03-18T22:58:00.955 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:00.336+0000 7f3ab8a0c700 1 -- 172.21.15.49:0/4185564033 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f3ab402dd30 con 0x7f3ab4106c00 2024-03-18T22:58:00.955 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:00.339+0000 7f3aa37fe700 1 -- 172.21.15.49:0/4185564033 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+429 (secure 0 0 0) 0x7f3aa4004260 con 0x7f3ab4106c00 2024-03-18T22:58:00.955 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:00.339+0000 7f3aa37fe700 1 -- 172.21.15.49:0/4185564033 <== mon.0 v2:172.21.15.49:3300/0 8 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f3aa401b450 con 0x7f3ab4106c00 2024-03-18T22:58:00.955 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:00.339+0000 7f3ab8a0c700 1 -- 172.21.15.49:0/4185564033 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3ab4106c00 msgr2=0x7f3ab419b420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T22:58:00.955 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:00.339+0000 7f3ab8a0c700 1 --2- 172.21.15.49:0/4185564033 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3ab4106c00 0x7f3ab419b420 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f3aa4000c00 tx=0x7f3aa4004750 comp rx=0 tx=0).stop 2024-03-18T22:58:00.955 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:00.340+0000 7f3ab8a0c700 1 -- 172.21.15.49:0/4185564033 shutdown_connections 2024-03-18T22:58:00.955 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:00.340+0000 7f3ab8a0c700 1 --2- 172.21.15.49:0/4185564033 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3ab4106c00 0x7f3ab419b420 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T22:58:00.955 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:00.340+0000 7f3ab8a0c700 1 -- 172.21.15.49:0/4185564033 >> 172.21.15.49:0/4185564033 conn(0x7f3ab41021a0 msgr2=0x7f3ab4191fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T22:58:00.955 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:00.340+0000 7f3ab8a0c700 1 -- 172.21.15.49:0/4185564033 shutdown_connections 2024-03-18T22:58:00.955 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:00.340+0000 7f3ab8a0c700 1 -- 172.21.15.49:0/4185564033 wait complete. 2024-03-18T22:58:00.955 INFO:teuthology.orchestra.run.smithi049.stdout:Generating new minimal ceph.conf... 2024-03-18T22:58:02.055 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:01 smithi049 ceph-mon[62619]: from='client.? 172.21.15.49:0/362913336' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T22:58:02.062 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:01.356+0000 7fee21733700 1 Processor -- start 2024-03-18T22:58:02.063 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:01.356+0000 7fee21733700 1 -- start start 2024-03-18T22:58:02.063 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:01.356+0000 7fee21733700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fee1c1065e0 0x7fee1c106a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T22:58:02.063 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:01.357+0000 7fee21733700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fee1c07be00 con 0x7fee1c1065e0 2024-03-18T22:58:02.063 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:01.357+0000 7fee1affd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fee1c1065e0 0x7fee1c106a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T22:58:02.063 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:01.357+0000 7fee1affd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fee1c1065e0 0x7fee1c106a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:50938/0 (socket says 172.21.15.49:50938) 2024-03-18T22:58:02.063 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:01.357+0000 7fee1affd700 1 -- 172.21.15.49:0/1195453040 learned_addr learned my addr 172.21.15.49:0/1195453040 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T22:58:02.063 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:01.357+0000 7fee1affd700 1 -- 172.21.15.49:0/1195453040 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee1c106f40 con 0x7fee1c1065e0 2024-03-18T22:58:02.063 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:01.357+0000 7fee1affd700 1 --2- 172.21.15.49:0/1195453040 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fee1c1065e0 0x7fee1c106a00 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fee14009e80 tx=0x7fee1400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3a6f3efbd18c6c9 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T22:58:02.063 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:01.357+0000 7fee19ffb700 1 -- 172.21.15.49:0/1195453040 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fee14017070 con 0x7fee1c1065e0 2024-03-18T22:58:02.063 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:01.357+0000 7fee19ffb700 1 -- 172.21.15.49:0/1195453040 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fee1400b8c0 con 0x7fee1c1065e0 2024-03-18T22:58:02.063 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:01.357+0000 7fee19ffb700 1 -- 172.21.15.49:0/1195453040 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fee140045c0 con 0x7fee1c1065e0 2024-03-18T22:58:02.063 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:01.358+0000 7fee21733700 1 -- 172.21.15.49:0/1195453040 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fee1c1065e0 msgr2=0x7fee1c106a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T22:58:02.064 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:01.358+0000 7fee21733700 1 --2- 172.21.15.49:0/1195453040 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fee1c1065e0 0x7fee1c106a00 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fee14009e80 tx=0x7fee1400b330 comp rx=0 tx=0).stop 2024-03-18T22:58:02.064 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:01.358+0000 7fee21733700 1 -- 172.21.15.49:0/1195453040 shutdown_connections 2024-03-18T22:58:02.064 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:01.358+0000 7fee21733700 1 --2- 172.21.15.49:0/1195453040 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fee1c1065e0 0x7fee1c106a00 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T22:58:02.064 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:01.358+0000 7fee21733700 1 -- 172.21.15.49:0/1195453040 >> 172.21.15.49:0/1195453040 conn(0x7fee1c1021c0 msgr2=0x7fee1c1045e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T22:58:02.064 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:01.358+0000 7fee21733700 1 -- 172.21.15.49:0/1195453040 shutdown_connections 2024-03-18T22:58:02.064 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:01.358+0000 7fee21733700 1 -- 172.21.15.49:0/1195453040 wait complete. 2024-03-18T22:58:02.064 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:01.358+0000 7fee21733700 1 Processor -- start 2024-03-18T22:58:02.064 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:01.359+0000 7fee21733700 1 -- start start 2024-03-18T22:58:02.064 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:01.359+0000 7fee21733700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fee1c1065e0 0x7fee1c19de20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T22:58:02.064 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:01.359+0000 7fee21733700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fee1c107f60 con 0x7fee1c1065e0 2024-03-18T22:58:02.064 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:01.359+0000 7fee1affd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fee1c1065e0 0x7fee1c19de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T22:58:02.064 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:01.359+0000 7fee1affd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fee1c1065e0 0x7fee1c19de20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:50954/0 (socket says 172.21.15.49:50954) 2024-03-18T22:58:02.064 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:01.359+0000 7fee1affd700 1 -- 172.21.15.49:0/362913336 learned_addr learned my addr 172.21.15.49:0/362913336 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T22:58:02.065 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:01.359+0000 7fee1affd700 1 -- 172.21.15.49:0/362913336 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee1c19e360 con 0x7fee1c1065e0 2024-03-18T22:58:02.065 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:01.359+0000 7fee1affd700 1 --2- 172.21.15.49:0/362913336 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fee1c1065e0 0x7fee1c19de20 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fee140072a0 tx=0x7fee1400bd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T22:58:02.065 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:01.360+0000 7fee03fff700 1 -- 172.21.15.49:0/362913336 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fee14020020 con 0x7fee1c1065e0 2024-03-18T22:58:02.065 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:01.360+0000 7fee03fff700 1 -- 172.21.15.49:0/362913336 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fee14003b50 con 0x7fee1c1065e0 2024-03-18T22:58:02.065 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:01.360+0000 7fee21733700 1 -- 172.21.15.49:0/362913336 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee1c19e5f0 con 0x7fee1c1065e0 2024-03-18T22:58:02.065 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:01.360+0000 7fee03fff700 1 -- 172.21.15.49:0/362913336 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fee14017040 con 0x7fee1c1065e0 2024-03-18T22:58:02.065 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:01.360+0000 7fee21733700 1 -- 172.21.15.49:0/362913336 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee1c1a12e0 con 0x7fee1c1065e0 2024-03-18T22:58:02.065 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:01.360+0000 7fee03fff700 1 -- 172.21.15.49:0/362913336 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fee14023020 con 0x7fee1c1065e0 2024-03-18T22:58:02.065 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:01.360+0000 7fee03fff700 1 -- 172.21.15.49:0/362913336 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fee1400f050 con 0x7fee1c1065e0 2024-03-18T22:58:02.065 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:01.361+0000 7fee21733700 1 -- 172.21.15.49:0/362913336 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee1c0693e0 con 0x7fee1c1065e0 2024-03-18T22:58:02.065 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:01.363+0000 7fee03fff700 1 -- 172.21.15.49:0/362913336 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fee1402a020 con 0x7fee1c1065e0 2024-03-18T22:58:02.065 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:01.434+0000 7fee21733700 1 -- 172.21.15.49:0/362913336 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7fee1c02dd80 con 0x7fee1c1065e0 2024-03-18T22:58:02.065 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:01.435+0000 7fee03fff700 1 -- 172.21.15.49:0/362913336 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+175 (secure 0 0 0) 0x7fee1400b860 con 0x7fee1c1065e0 2024-03-18T22:58:02.065 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:01.435+0000 7fee21733700 1 -- 172.21.15.49:0/362913336 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fee1c1065e0 msgr2=0x7fee1c19de20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T22:58:02.066 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:01.435+0000 7fee21733700 1 --2- 172.21.15.49:0/362913336 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fee1c1065e0 0x7fee1c19de20 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fee140072a0 tx=0x7fee1400bd50 comp rx=0 tx=0).stop 2024-03-18T22:58:02.066 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:01.435+0000 7fee21733700 1 -- 172.21.15.49:0/362913336 shutdown_connections 2024-03-18T22:58:02.066 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:01.435+0000 7fee21733700 1 --2- 172.21.15.49:0/362913336 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fee1c1065e0 0x7fee1c19de20 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T22:58:02.066 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:01.435+0000 7fee21733700 1 -- 172.21.15.49:0/362913336 >> 172.21.15.49:0/362913336 conn(0x7fee1c1021c0 msgr2=0x7fee1c194a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T22:58:02.066 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:01.435+0000 7fee21733700 1 -- 172.21.15.49:0/362913336 shutdown_connections 2024-03-18T22:58:02.066 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:01.435+0000 7fee21733700 1 -- 172.21.15.49:0/362913336 wait complete. 2024-03-18T22:58:02.066 INFO:teuthology.orchestra.run.smithi049.stdout:Restarting the monitor... 2024-03-18T22:58:02.327 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:02 smithi049 systemd[1]: Stopping Ceph mon.a for c63591ea-e57a-11ee-95c9-87774f69a715... 2024-03-18T22:58:02.630 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:02 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mon-a[62615]: 2024-03-18T22:58:02.325+0000 7ff43c6c8700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-18T22:58:02.630 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:02 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mon-a[62615]: 2024-03-18T22:58:02.325+0000 7ff43c6c8700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-03-18T22:58:02.630 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:02 smithi049 podman[62865]: 2024-03-18 22:58:02.489883991 +0000 UTC m=+0.265204499 container died dd9519965f8cbb923af25d99269514a3af5d89388176f2180639b31ef9933524 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810, name=ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mon-a, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-03-18T22:58:03.076 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:02 smithi049 podman[62865]: 2024-03-18 22:58:02.833602146 +0000 UTC m=+0.608922660 container cleanup dd9519965f8cbb923af25d99269514a3af5d89388176f2180639b31ef9933524 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810, name=ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-18T22:58:03.076 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:02 smithi049 bash[62865]: ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mon-a 2024-03-18T22:58:03.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 podman[62879]: 2024-03-18 22:58:03.07620046 +0000 UTC m=+0.587291175 container remove dd9519965f8cbb923af25d99269514a3af5d89388176f2180639b31ef9933524 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810, name=ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mon-a, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-18T22:58:03.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 systemd[1]: ceph-c63591ea-e57a-11ee-95c9-87774f69a715@mon.a.service: Deactivated successfully. 2024-03-18T22:58:03.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 systemd[1]: Stopped Ceph mon.a for c63591ea-e57a-11ee-95c9-87774f69a715. 2024-03-18T22:58:03.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 systemd[1]: Starting Ceph mon.a for c63591ea-e57a-11ee-95c9-87774f69a715... 2024-03-18T22:58:03.687 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 podman[62972]: 2024-03-18 22:58:03.435312363 +0000 UTC m=+0.113606499 container create 8521c70f9f18562cf8f0bcd4a0490e538a41d90c32424c25d0d78aa66254e35e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810, name=ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mon-a, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240318.181317-b6ae249) 2024-03-18T22:58:03.687 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 podman[62972]: 2024-03-18 22:58:03.340899291 +0000 UTC m=+0.019193422 image pull 2312c8e42d0069a2dbb2364bddb00264db4b800e7f18a7e7e4eb8f3ecab2ff1c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 2024-03-18T22:58:03.687 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 podman[62972]: 2024-03-18 22:58:03.612388312 +0000 UTC m=+0.290682462 container init 8521c70f9f18562cf8f0bcd4a0490e538a41d90c32424c25d0d78aa66254e35e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810, name=ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mon-a, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-03-18T22:58:03.688 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 podman[62972]: 2024-03-18 22:58:03.615534013 +0000 UTC m=+0.293828145 container start 8521c70f9f18562cf8f0bcd4a0490e538a41d90c32424c25d0d78aa66254e35e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810, name=ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mon-a, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-03-18T22:58:03.688 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 ceph-mon[62991]: set uid:gid to 167:167 (ceph:ceph) 2024-03-18T22:58:03.688 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 ceph-mon[62991]: ceph version 19.0.0-2206-gb6ae2491 (b6ae24918b03b1b8a19c9263a50655bba80bb810) squid (dev), process ceph-mon, pid 2 2024-03-18T22:58:03.688 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 ceph-mon[62991]: pidfile_write: ignore empty --pid-file 2024-03-18T22:58:03.688 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 ceph-mon[62991]: load: jerasure load: lrc 2024-03-18T22:58:03.688 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 ceph-mon[62991]: rocksdb: RocksDB version: 7.9.2 2024-03-18T22:58:03.688 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 ceph-mon[62991]: rocksdb: Git sha 0 2024-03-18T22:58:03.688 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 ceph-mon[62991]: rocksdb: Compile date 2024-03-18 18:57:12 2024-03-18T22:58:03.688 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 ceph-mon[62991]: rocksdb: DB SUMMARY 2024-03-18T22:58:03.688 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 ceph-mon[62991]: rocksdb: DB Session ID: 28K0JSCP4EGIACVO9D6K 2024-03-18T22:58:03.688 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 ceph-mon[62991]: rocksdb: CURRENT file: CURRENT 2024-03-18T22:58:03.688 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 ceph-mon[62991]: rocksdb: IDENTITY file: IDENTITY 2024-03-18T22:58:03.688 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 ceph-mon[62991]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-03-18T22:58:03.689 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 ceph-mon[62991]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-03-18T22:58:03.689 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 ceph-mon[62991]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 104780 ; 2024-03-18T22:58:03.689 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 ceph-mon[62991]: rocksdb: Options.error_if_exists: 0 2024-03-18T22:58:03.689 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 ceph-mon[62991]: rocksdb: Options.create_if_missing: 0 2024-03-18T22:58:03.689 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 ceph-mon[62991]: rocksdb: Options.paranoid_checks: 1 2024-03-18T22:58:03.689 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 ceph-mon[62991]: rocksdb: Options.flush_verify_memtable_count: 1 2024-03-18T22:58:03.689 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 ceph-mon[62991]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-03-18T22:58:03.689 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 ceph-mon[62991]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-03-18T22:58:03.689 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 ceph-mon[62991]: rocksdb: Options.env: 0x5613df258c60 2024-03-18T22:58:03.689 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 ceph-mon[62991]: rocksdb: Options.fs: PosixFileSystem 2024-03-18T22:58:03.689 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 ceph-mon[62991]: rocksdb: Options.info_log: 0x5613e184bae0 2024-03-18T22:58:03.689 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 ceph-mon[62991]: rocksdb: Options.max_file_opening_threads: 16 2024-03-18T22:58:03.689 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 ceph-mon[62991]: rocksdb: Options.statistics: (nil) 2024-03-18T22:58:03.689 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 ceph-mon[62991]: rocksdb: Options.use_fsync: 0 2024-03-18T22:58:03.689 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 ceph-mon[62991]: rocksdb: Options.max_log_file_size: 0 2024-03-18T22:58:03.690 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 ceph-mon[62991]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-03-18T22:58:03.690 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 ceph-mon[62991]: rocksdb: Options.log_file_time_to_roll: 0 2024-03-18T22:58:03.690 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 ceph-mon[62991]: rocksdb: Options.keep_log_file_num: 1000 2024-03-18T22:58:03.690 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 ceph-mon[62991]: rocksdb: Options.recycle_log_file_num: 0 2024-03-18T22:58:03.690 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 ceph-mon[62991]: rocksdb: Options.allow_fallocate: 1 2024-03-18T22:58:03.690 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 ceph-mon[62991]: rocksdb: Options.allow_mmap_reads: 0 2024-03-18T22:58:03.690 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 ceph-mon[62991]: rocksdb: Options.allow_mmap_writes: 0 2024-03-18T22:58:03.690 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 ceph-mon[62991]: rocksdb: Options.use_direct_reads: 0 2024-03-18T22:58:03.690 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 ceph-mon[62991]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-03-18T22:58:03.690 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 ceph-mon[62991]: rocksdb: Options.create_missing_column_families: 0 2024-03-18T22:58:03.690 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 ceph-mon[62991]: rocksdb: Options.db_log_dir: 2024-03-18T22:58:03.690 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 ceph-mon[62991]: rocksdb: Options.wal_dir: 2024-03-18T22:58:03.690 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 ceph-mon[62991]: rocksdb: Options.table_cache_numshardbits: 6 2024-03-18T22:58:03.690 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 ceph-mon[62991]: rocksdb: Options.WAL_ttl_seconds: 0 2024-03-18T22:58:03.691 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 ceph-mon[62991]: rocksdb: Options.WAL_size_limit_MB: 0 2024-03-18T22:58:03.691 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 ceph-mon[62991]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-03-18T22:58:03.691 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 ceph-mon[62991]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-03-18T22:58:03.691 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 ceph-mon[62991]: rocksdb: Options.is_fd_close_on_exec: 1 2024-03-18T22:58:03.691 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 ceph-mon[62991]: rocksdb: Options.advise_random_on_open: 1 2024-03-18T22:58:03.691 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 ceph-mon[62991]: rocksdb: Options.db_write_buffer_size: 0 2024-03-18T22:58:03.691 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 ceph-mon[62991]: rocksdb: Options.write_buffer_manager: 0x5613e181e1e0 2024-03-18T22:58:03.691 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 ceph-mon[62991]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-03-18T22:58:03.691 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 ceph-mon[62991]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-03-18T22:58:03.691 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 ceph-mon[62991]: rocksdb: Options.use_adaptive_mutex: 0 2024-03-18T22:58:03.691 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 ceph-mon[62991]: rocksdb: Options.rate_limiter: (nil) 2024-03-18T22:58:03.691 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 ceph-mon[62991]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-03-18T22:58:03.692 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 ceph-mon[62991]: rocksdb: Options.wal_recovery_mode: 2 2024-03-18T22:58:03.693 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 ceph-mon[62991]: rocksdb: Options.enable_thread_tracking: 0 2024-03-18T22:58:03.693 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 ceph-mon[62991]: rocksdb: Options.enable_pipelined_write: 0 2024-03-18T22:58:03.693 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 ceph-mon[62991]: rocksdb: Options.unordered_write: 0 2024-03-18T22:58:03.693 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 ceph-mon[62991]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-03-18T22:58:03.693 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 ceph-mon[62991]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-03-18T22:58:03.693 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 ceph-mon[62991]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-03-18T22:58:03.693 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 ceph-mon[62991]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-03-18T22:58:03.693 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 ceph-mon[62991]: rocksdb: Options.row_cache: None 2024-03-18T22:58:03.693 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 ceph-mon[62991]: rocksdb: Options.wal_filter: None 2024-03-18T22:58:03.693 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 ceph-mon[62991]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-03-18T22:58:03.693 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 ceph-mon[62991]: rocksdb: Options.allow_ingest_behind: 0 2024-03-18T22:58:03.693 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 ceph-mon[62991]: rocksdb: Options.two_write_queues: 0 2024-03-18T22:58:03.693 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 ceph-mon[62991]: rocksdb: Options.manual_wal_flush: 0 2024-03-18T22:58:03.693 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 ceph-mon[62991]: rocksdb: Options.wal_compression: 0 2024-03-18T22:58:03.694 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 ceph-mon[62991]: rocksdb: Options.atomic_flush: 0 2024-03-18T22:58:03.694 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 ceph-mon[62991]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-03-18T22:58:03.694 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 ceph-mon[62991]: rocksdb: Options.persist_stats_to_disk: 0 2024-03-18T22:58:03.694 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 ceph-mon[62991]: rocksdb: Options.write_dbid_to_manifest: 0 2024-03-18T22:58:03.694 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 ceph-mon[62991]: rocksdb: Options.log_readahead_size: 0 2024-03-18T22:58:03.694 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 ceph-mon[62991]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-03-18T22:58:03.694 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 ceph-mon[62991]: rocksdb: Options.best_efforts_recovery: 0 2024-03-18T22:58:03.694 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 ceph-mon[62991]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-03-18T22:58:03.694 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 ceph-mon[62991]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-03-18T22:58:03.694 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 ceph-mon[62991]: rocksdb: Options.allow_data_in_errors: 0 2024-03-18T22:58:03.694 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 ceph-mon[62991]: rocksdb: Options.db_host_id: __hostname__ 2024-03-18T22:58:03.694 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 ceph-mon[62991]: rocksdb: Options.enforce_single_del_contracts: true 2024-03-18T22:58:03.694 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 ceph-mon[62991]: rocksdb: Options.max_background_jobs: 2 2024-03-18T22:58:03.694 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 ceph-mon[62991]: rocksdb: Options.max_background_compactions: -1 2024-03-18T22:58:03.694 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 ceph-mon[62991]: rocksdb: Options.max_subcompactions: 1 2024-03-18T22:58:03.695 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 ceph-mon[62991]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-03-18T22:58:03.695 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 ceph-mon[62991]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-03-18T22:58:03.695 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 ceph-mon[62991]: rocksdb: Options.delayed_write_rate : 16777216 2024-03-18T22:58:03.695 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 ceph-mon[62991]: rocksdb: Options.max_total_wal_size: 0 2024-03-18T22:58:03.695 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 ceph-mon[62991]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-03-18T22:58:03.695 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 ceph-mon[62991]: rocksdb: Options.stats_dump_period_sec: 600 2024-03-18T22:58:03.695 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 ceph-mon[62991]: rocksdb: Options.stats_persist_period_sec: 600 2024-03-18T22:58:03.695 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 ceph-mon[62991]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-03-18T22:58:03.695 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 ceph-mon[62991]: rocksdb: Options.max_open_files: -1 2024-03-18T22:58:03.695 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 ceph-mon[62991]: rocksdb: Options.bytes_per_sync: 0 2024-03-18T22:58:03.695 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 ceph-mon[62991]: rocksdb: Options.wal_bytes_per_sync: 0 2024-03-18T22:58:03.695 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 ceph-mon[62991]: rocksdb: Options.strict_bytes_per_sync: 0 2024-03-18T22:58:03.695 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 ceph-mon[62991]: rocksdb: Options.compaction_readahead_size: 0 2024-03-18T22:58:03.695 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 ceph-mon[62991]: rocksdb: Options.max_background_flushes: -1 2024-03-18T22:58:03.696 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 ceph-mon[62991]: rocksdb: Compression algorithms supported: 2024-03-18T22:58:03.696 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 ceph-mon[62991]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-03-18T22:58:03.696 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 ceph-mon[62991]: rocksdb: kZSTD supported: 0 2024-03-18T22:58:03.696 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 ceph-mon[62991]: rocksdb: kXpressCompression supported: 0 2024-03-18T22:58:03.696 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 ceph-mon[62991]: rocksdb: kLZ4HCCompression supported: 1 2024-03-18T22:58:03.696 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 ceph-mon[62991]: rocksdb: kZlibCompression supported: 1 2024-03-18T22:58:03.696 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 ceph-mon[62991]: rocksdb: kSnappyCompression supported: 1 2024-03-18T22:58:03.696 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 ceph-mon[62991]: rocksdb: kLZ4Compression supported: 1 2024-03-18T22:58:03.696 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 ceph-mon[62991]: rocksdb: kBZip2Compression supported: 0 2024-03-18T22:58:03.696 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 ceph-mon[62991]: rocksdb: Fast CRC32 supported: Supported on x86 2024-03-18T22:58:03.696 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 ceph-mon[62991]: rocksdb: DMutex implementation: pthread_mutex_t 2024-03-18T22:58:03.696 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 ceph-mon[62991]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-03-18T22:58:03.696 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 ceph-mon[62991]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-03-18T22:58:03.696 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 ceph-mon[62991]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-03-18T22:58:03.696 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 ceph-mon[62991]: rocksdb: Options.merge_operator: 2024-03-18T22:58:03.697 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 ceph-mon[62991]: rocksdb: Options.compaction_filter: None 2024-03-18T22:58:03.697 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 ceph-mon[62991]: rocksdb: Options.compaction_filter_factory: None 2024-03-18T22:58:03.697 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 ceph-mon[62991]: rocksdb: Options.sst_partitioner_factory: None 2024-03-18T22:58:03.697 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 ceph-mon[62991]: rocksdb: Options.memtable_factory: SkipListFactory 2024-03-18T22:58:03.697 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 ceph-mon[62991]: rocksdb: Options.table_factory: BlockBasedTable 2024-03-18T22:58:03.697 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 ceph-mon[62991]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x5613e184bbe0) 2024-03-18T22:58:03.697 INFO:journalctl@ceph.mon.a.smithi049.stdout: cache_index_and_filter_blocks: 1 2024-03-18T22:58:03.697 INFO:journalctl@ceph.mon.a.smithi049.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-03-18T22:58:03.697 INFO:journalctl@ceph.mon.a.smithi049.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-03-18T22:58:03.697 INFO:journalctl@ceph.mon.a.smithi049.stdout: pin_top_level_index_and_filter: 1 2024-03-18T22:58:03.697 INFO:journalctl@ceph.mon.a.smithi049.stdout: index_type: 0 2024-03-18T22:58:03.697 INFO:journalctl@ceph.mon.a.smithi049.stdout: data_block_index_type: 0 2024-03-18T22:58:03.697 INFO:journalctl@ceph.mon.a.smithi049.stdout: index_shortening: 1 2024-03-18T22:58:03.697 INFO:journalctl@ceph.mon.a.smithi049.stdout: data_block_hash_table_util_ratio: 0.750000 2024-03-18T22:58:03.697 INFO:journalctl@ceph.mon.a.smithi049.stdout: checksum: 4 2024-03-18T22:58:03.698 INFO:journalctl@ceph.mon.a.smithi049.stdout: no_block_cache: 0 2024-03-18T22:58:03.698 INFO:journalctl@ceph.mon.a.smithi049.stdout: block_cache: 0x5613e1851090 2024-03-18T22:58:03.698 INFO:journalctl@ceph.mon.a.smithi049.stdout: block_cache_name: BinnedLRUCache 2024-03-18T22:58:03.698 INFO:journalctl@ceph.mon.a.smithi049.stdout: block_cache_options: 2024-03-18T22:58:03.698 INFO:journalctl@ceph.mon.a.smithi049.stdout: capacity : 536870912 2024-03-18T22:58:03.698 INFO:journalctl@ceph.mon.a.smithi049.stdout: num_shard_bits : 4 2024-03-18T22:58:03.698 INFO:journalctl@ceph.mon.a.smithi049.stdout: strict_capacity_limit : 0 2024-03-18T22:58:03.698 INFO:journalctl@ceph.mon.a.smithi049.stdout: high_pri_pool_ratio: 0.000 2024-03-18T22:58:03.698 INFO:journalctl@ceph.mon.a.smithi049.stdout: block_cache_compressed: (nil) 2024-03-18T22:58:03.698 INFO:journalctl@ceph.mon.a.smithi049.stdout: persistent_cache: (nil) 2024-03-18T22:58:03.698 INFO:journalctl@ceph.mon.a.smithi049.stdout: block_size: 4096 2024-03-18T22:58:03.698 INFO:journalctl@ceph.mon.a.smithi049.stdout: block_size_deviation: 10 2024-03-18T22:58:03.698 INFO:journalctl@ceph.mon.a.smithi049.stdout: block_restart_interval: 16 2024-03-18T22:58:03.698 INFO:journalctl@ceph.mon.a.smithi049.stdout: index_block_restart_interval: 1 2024-03-18T22:58:03.698 INFO:journalctl@ceph.mon.a.smithi049.stdout: metadata_block_size: 4096 2024-03-18T22:58:03.699 INFO:journalctl@ceph.mon.a.smithi049.stdout: partition_filters: 0 2024-03-18T22:58:03.699 INFO:journalctl@ceph.mon.a.smithi049.stdout: use_delta_encoding: 1 2024-03-18T22:58:03.699 INFO:journalctl@ceph.mon.a.smithi049.stdout: filter_policy: bloomfilter 2024-03-18T22:58:03.699 INFO:journalctl@ceph.mon.a.smithi049.stdout: whole_key_filtering: 1 2024-03-18T22:58:03.699 INFO:journalctl@ceph.mon.a.smithi049.stdout: verify_compression: 0 2024-03-18T22:58:03.699 INFO:journalctl@ceph.mon.a.smithi049.stdout: read_amp_bytes_per_bit: 0 2024-03-18T22:58:03.699 INFO:journalctl@ceph.mon.a.smithi049.stdout: format_version: 5 2024-03-18T22:58:03.699 INFO:journalctl@ceph.mon.a.smithi049.stdout: enable_index_compression: 1 2024-03-18T22:58:03.699 INFO:journalctl@ceph.mon.a.smithi049.stdout: block_align: 0 2024-03-18T22:58:03.699 INFO:journalctl@ceph.mon.a.smithi049.stdout: max_auto_readahead_size: 262144 2024-03-18T22:58:03.699 INFO:journalctl@ceph.mon.a.smithi049.stdout: prepopulate_block_cache: 0 2024-03-18T22:58:03.699 INFO:journalctl@ceph.mon.a.smithi049.stdout: initial_auto_readahead_size: 8192 2024-03-18T22:58:03.699 INFO:journalctl@ceph.mon.a.smithi049.stdout: num_file_reads_for_auto_readahead: 2 2024-03-18T22:58:03.699 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 ceph-mon[62991]: rocksdb: Options.write_buffer_size: 33554432 2024-03-18T22:58:03.699 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 ceph-mon[62991]: rocksdb: Options.max_write_buffer_number: 2 2024-03-18T22:58:03.700 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 ceph-mon[62991]: rocksdb: Options.compression: NoCompression 2024-03-18T22:58:03.700 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 ceph-mon[62991]: rocksdb: Options.bottommost_compression: Disabled 2024-03-18T22:58:03.700 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 ceph-mon[62991]: rocksdb: Options.prefix_extractor: nullptr 2024-03-18T22:58:03.700 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 ceph-mon[62991]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-03-18T22:58:03.700 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 ceph-mon[62991]: rocksdb: Options.num_levels: 7 2024-03-18T22:58:03.700 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 ceph-mon[62991]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-03-18T22:58:03.700 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 ceph-mon[62991]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-03-18T22:58:03.700 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 ceph-mon[62991]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-03-18T22:58:03.700 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 ceph-mon[62991]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-03-18T22:58:03.700 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 ceph-mon[62991]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-03-18T22:58:03.700 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 ceph-mon[62991]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-03-18T22:58:03.700 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 ceph-mon[62991]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-03-18T22:58:03.700 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 ceph-mon[62991]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-03-18T22:58:03.700 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 ceph-mon[62991]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-03-18T22:58:03.700 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 ceph-mon[62991]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-03-18T22:58:03.701 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 ceph-mon[62991]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-03-18T22:58:03.701 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 ceph-mon[62991]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-03-18T22:58:03.701 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 ceph-mon[62991]: rocksdb: Options.compression_opts.window_bits: -14 2024-03-18T22:58:03.701 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 ceph-mon[62991]: rocksdb: Options.compression_opts.level: 32767 2024-03-18T22:58:03.701 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 ceph-mon[62991]: rocksdb: Options.compression_opts.strategy: 0 2024-03-18T22:58:03.701 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 ceph-mon[62991]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-03-18T22:58:03.701 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 ceph-mon[62991]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-03-18T22:58:03.701 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 ceph-mon[62991]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-03-18T22:58:03.701 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 ceph-mon[62991]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-03-18T22:58:03.701 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 ceph-mon[62991]: rocksdb: Options.compression_opts.enabled: false 2024-03-18T22:58:03.701 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 ceph-mon[62991]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-03-18T22:58:03.701 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 ceph-mon[62991]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-03-18T22:58:03.701 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 ceph-mon[62991]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-03-18T22:58:03.701 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 ceph-mon[62991]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-03-18T22:58:03.701 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 ceph-mon[62991]: rocksdb: Options.target_file_size_base: 67108864 2024-03-18T22:58:03.702 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 ceph-mon[62991]: rocksdb: Options.target_file_size_multiplier: 1 2024-03-18T22:58:03.702 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 ceph-mon[62991]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-03-18T22:58:03.702 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 ceph-mon[62991]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-03-18T22:58:03.702 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 ceph-mon[62991]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-03-18T22:58:03.702 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 ceph-mon[62991]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-03-18T22:58:03.702 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 ceph-mon[62991]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-03-18T22:58:03.702 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 ceph-mon[62991]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-03-18T22:58:03.702 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 ceph-mon[62991]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-03-18T22:58:03.702 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 ceph-mon[62991]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-03-18T22:58:03.702 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 ceph-mon[62991]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-03-18T22:58:03.702 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 ceph-mon[62991]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-03-18T22:58:03.702 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 ceph-mon[62991]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-03-18T22:58:03.702 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 ceph-mon[62991]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-03-18T22:58:03.702 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 ceph-mon[62991]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-03-18T22:58:03.702 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 ceph-mon[62991]: rocksdb: Options.arena_block_size: 1048576 2024-03-18T22:58:03.703 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 ceph-mon[62991]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-03-18T22:58:03.703 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 ceph-mon[62991]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-03-18T22:58:03.703 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 ceph-mon[62991]: rocksdb: Options.disable_auto_compactions: 0 2024-03-18T22:58:03.703 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 ceph-mon[62991]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-03-18T22:58:03.703 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 ceph-mon[62991]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-03-18T22:58:03.703 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 ceph-mon[62991]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-03-18T22:58:03.703 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 ceph-mon[62991]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-03-18T22:58:03.703 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 ceph-mon[62991]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-03-18T22:58:03.703 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 ceph-mon[62991]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-03-18T22:58:03.703 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 ceph-mon[62991]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-03-18T22:58:03.703 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 ceph-mon[62991]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-03-18T22:58:03.703 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 ceph-mon[62991]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-03-18T22:58:03.703 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 ceph-mon[62991]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-03-18T22:58:03.703 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 ceph-mon[62991]: rocksdb: Options.table_properties_collectors: 2024-03-18T22:58:03.703 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 ceph-mon[62991]: rocksdb: Options.inplace_update_support: 0 2024-03-18T22:58:03.704 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 ceph-mon[62991]: rocksdb: Options.inplace_update_num_locks: 10000 2024-03-18T22:58:03.704 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 ceph-mon[62991]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-03-18T22:58:03.704 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 ceph-mon[62991]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-03-18T22:58:03.704 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 ceph-mon[62991]: rocksdb: Options.memtable_huge_page_size: 0 2024-03-18T22:58:03.704 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 ceph-mon[62991]: rocksdb: Options.bloom_locality: 0 2024-03-18T22:58:03.704 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 ceph-mon[62991]: rocksdb: Options.max_successive_merges: 0 2024-03-18T22:58:03.704 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 ceph-mon[62991]: rocksdb: Options.optimize_filters_for_hits: 0 2024-03-18T22:58:03.704 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 ceph-mon[62991]: rocksdb: Options.paranoid_file_checks: 0 2024-03-18T22:58:03.704 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 ceph-mon[62991]: rocksdb: Options.force_consistency_checks: 1 2024-03-18T22:58:03.704 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 ceph-mon[62991]: rocksdb: Options.report_bg_io_stats: 0 2024-03-18T22:58:03.704 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 ceph-mon[62991]: rocksdb: Options.ttl: 2592000 2024-03-18T22:58:03.704 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 ceph-mon[62991]: rocksdb: Options.periodic_compaction_seconds: 0 2024-03-18T22:58:03.704 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 ceph-mon[62991]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-03-18T22:58:03.704 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 ceph-mon[62991]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-03-18T22:58:03.704 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 ceph-mon[62991]: rocksdb: Options.enable_blob_files: false 2024-03-18T22:58:03.705 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 ceph-mon[62991]: rocksdb: Options.min_blob_size: 0 2024-03-18T22:58:03.705 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 ceph-mon[62991]: rocksdb: Options.blob_file_size: 268435456 2024-03-18T22:58:03.705 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 ceph-mon[62991]: rocksdb: Options.blob_compression_type: NoCompression 2024-03-18T22:58:03.705 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 ceph-mon[62991]: rocksdb: Options.enable_blob_garbage_collection: false 2024-03-18T22:58:03.705 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 ceph-mon[62991]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-03-18T22:58:03.705 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 ceph-mon[62991]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-03-18T22:58:03.705 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 ceph-mon[62991]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-03-18T22:58:03.705 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 ceph-mon[62991]: rocksdb: Options.blob_file_starting_level: 0 2024-03-18T22:58:03.705 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 ceph-mon[62991]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-03-18T22:58:03.705 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 ceph-mon[62991]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-03-18T22:58:03.705 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 ceph-mon[62991]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-03-18T22:58:03.705 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 ceph-mon[62991]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 33ad4dcf-0279-4bce-908e-0d6bd990a602 2024-03-18T22:58:03.705 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 ceph-mon[62991]: rocksdb: EVENT_LOG_v1 {"time_micros": 1710802683645460, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-03-18T22:58:03.705 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 ceph-mon[62991]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-03-18T22:58:03.705 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 ceph-mon[62991]: rocksdb: EVENT_LOG_v1 {"time_micros": 1710802683646451, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 99939, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 324, "table_properties": {"data_size": 97847, "index_size": 310, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 773, "raw_key_size": 14086, "raw_average_key_size": 49, "raw_value_size": 90189, "raw_average_value_size": 318, "num_data_blocks": 12, "num_entries": 283, "num_filter_entries": 283, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1710802683, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "33ad4dcf-0279-4bce-908e-0d6bd990a602", "db_session_id": "28K0JSCP4EGIACVO9D6K", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-03-18T22:58:03.706 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 ceph-mon[62991]: rocksdb: EVENT_LOG_v1 {"time_micros": 1710802683646535, "job": 1, "event": "recovery_finished"} 2024-03-18T22:58:03.706 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 ceph-mon[62991]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-03-18T22:58:03.706 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 ceph-mon[62991]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-03-18T22:58:03.706 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 ceph-mon[62991]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x5613e1944000 2024-03-18T22:58:03.706 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 ceph-mon[62991]: rocksdb: DB pointer 0x5613e192e000 2024-03-18T22:58:03.706 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 ceph-mon[62991]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-03-18T22:58:03.706 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 ceph-mon[62991]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-03-18T22:58:03.706 INFO:journalctl@ceph.mon.a.smithi049.stdout: ** DB Stats ** 2024-03-18T22:58:03.706 INFO:journalctl@ceph.mon.a.smithi049.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-18T22:58:03.706 INFO:journalctl@ceph.mon.a.smithi049.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-03-18T22:58:03.706 INFO:journalctl@ceph.mon.a.smithi049.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-18T22:58:03.706 INFO:journalctl@ceph.mon.a.smithi049.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-18T22:58:03.706 INFO:journalctl@ceph.mon.a.smithi049.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-03-18T22:58:03.707 INFO:journalctl@ceph.mon.a.smithi049.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-18T22:58:03.707 INFO:journalctl@ceph.mon.a.smithi049.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-18T22:58:03.707 INFO:journalctl@ceph.mon.a.smithi049.stdout: 2024-03-18T22:58:03.707 INFO:journalctl@ceph.mon.a.smithi049.stdout: ** Compaction Stats [default] ** 2024-03-18T22:58:03.707 INFO:journalctl@ceph.mon.a.smithi049.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-18T22:58:03.707 INFO:journalctl@ceph.mon.a.smithi049.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-03-18T22:58:03.707 INFO:journalctl@ceph.mon.a.smithi049.stdout: L0 2/0 99.43 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 158.3 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-18T22:58:03.707 INFO:journalctl@ceph.mon.a.smithi049.stdout: Sum 2/0 99.43 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 158.3 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-18T22:58:03.707 INFO:journalctl@ceph.mon.a.smithi049.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 158.3 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-18T22:58:03.707 INFO:journalctl@ceph.mon.a.smithi049.stdout: 2024-03-18T22:58:03.707 INFO:journalctl@ceph.mon.a.smithi049.stdout: ** Compaction Stats [default] ** 2024-03-18T22:58:03.707 INFO:journalctl@ceph.mon.a.smithi049.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-18T22:58:03.707 INFO:journalctl@ceph.mon.a.smithi049.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-03-18T22:58:03.707 INFO:journalctl@ceph.mon.a.smithi049.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 158.3 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-18T22:58:03.707 INFO:journalctl@ceph.mon.a.smithi049.stdout: 2024-03-18T22:58:03.708 INFO:journalctl@ceph.mon.a.smithi049.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-03-18T22:58:03.708 INFO:journalctl@ceph.mon.a.smithi049.stdout: 2024-03-18T22:58:03.708 INFO:journalctl@ceph.mon.a.smithi049.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-18T22:58:03.708 INFO:journalctl@ceph.mon.a.smithi049.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-03-18T22:58:03.708 INFO:journalctl@ceph.mon.a.smithi049.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-03-18T22:58:03.708 INFO:journalctl@ceph.mon.a.smithi049.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-03-18T22:58:03.708 INFO:journalctl@ceph.mon.a.smithi049.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-03-18T22:58:03.708 INFO:journalctl@ceph.mon.a.smithi049.stdout: AddFile(Keys): cumulative 0, interval 0 2024-03-18T22:58:03.708 INFO:journalctl@ceph.mon.a.smithi049.stdout: Cumulative compaction: 0.00 GB write, 31.57 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-18T22:58:03.708 INFO:journalctl@ceph.mon.a.smithi049.stdout: Interval compaction: 0.00 GB write, 31.57 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-18T22:58:03.708 INFO:journalctl@ceph.mon.a.smithi049.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-03-18T22:58:03.708 INFO:journalctl@ceph.mon.a.smithi049.stdout: Block cache BinnedLRUCache@0x5613e1851090#2 capacity: 512.00 MB usage: 1.52 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-03-18T22:58:03.708 INFO:journalctl@ceph.mon.a.smithi049.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,1.02 KB,0.000193715%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-03-18T22:58:03.708 INFO:journalctl@ceph.mon.a.smithi049.stdout: 2024-03-18T22:58:03.709 INFO:journalctl@ceph.mon.a.smithi049.stdout: ** File Read Latency Histogram By Level [default] ** 2024-03-18T22:58:03.709 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 ceph-mon[62991]: starting mon.a rank 0 at public addrs [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] at bind addrs [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid c63591ea-e57a-11ee-95c9-87774f69a715 2024-03-18T22:58:03.709 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 ceph-mon[62991]: mon.a@-1(???) e1 preinit fsid c63591ea-e57a-11ee-95c9-87774f69a715 2024-03-18T22:58:03.709 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 ceph-mon[62991]: mon.a@-1(???).mds e1 new map 2024-03-18T22:58:03.709 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 ceph-mon[62991]: mon.a@-1(???).mds e1 print_map 2024-03-18T22:58:03.709 INFO:journalctl@ceph.mon.a.smithi049.stdout: e1 2024-03-18T22:58:03.709 INFO:journalctl@ceph.mon.a.smithi049.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-03-18T22:58:03.709 INFO:journalctl@ceph.mon.a.smithi049.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-03-18T22:58:03.709 INFO:journalctl@ceph.mon.a.smithi049.stdout: legacy client fscid: -1 2024-03-18T22:58:03.709 INFO:journalctl@ceph.mon.a.smithi049.stdout: 2024-03-18T22:58:03.709 INFO:journalctl@ceph.mon.a.smithi049.stdout: No filesystems configured 2024-03-18T22:58:03.709 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 ceph-mon[62991]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-03-18T22:58:03.709 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 ceph-mon[62991]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-03-18T22:58:03.709 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 ceph-mon[62991]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-03-18T22:58:03.709 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 ceph-mon[62991]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-03-18T22:58:03.710 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 ceph-mon[62991]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-03-18T22:58:03.710 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 bash[62972]: 8521c70f9f18562cf8f0bcd4a0490e538a41d90c32424c25d0d78aa66254e35e 2024-03-18T22:58:03.710 INFO:teuthology.orchestra.run.smithi049.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-03-18T22:58:03.941 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 systemd[1]: Started Ceph mon.a for c63591ea-e57a-11ee-95c9-87774f69a715. 2024-03-18T22:58:03.941 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 ceph-mon[62991]: mon.a is new leader, mons a in quorum (ranks 0) 2024-03-18T22:58:03.941 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 ceph-mon[62991]: monmap epoch 1 2024-03-18T22:58:03.941 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 ceph-mon[62991]: fsid c63591ea-e57a-11ee-95c9-87774f69a715 2024-03-18T22:58:03.942 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 ceph-mon[62991]: last_changed 2024-03-18T22:57:55.689519+0000 2024-03-18T22:58:03.942 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 ceph-mon[62991]: created 2024-03-18T22:57:55.689519+0000 2024-03-18T22:58:03.942 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 ceph-mon[62991]: min_mon_release 19 (squid) 2024-03-18T22:58:03.942 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 ceph-mon[62991]: election_strategy: 1 2024-03-18T22:58:03.942 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 ceph-mon[62991]: 0: [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] mon.a 2024-03-18T22:58:03.942 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 ceph-mon[62991]: fsmap 2024-03-18T22:58:03.942 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 ceph-mon[62991]: osdmap e1: 0 total, 0 up, 0 in 2024-03-18T22:58:03.942 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:03 smithi049 ceph-mon[62991]: mgrmap e1: no daemons active 2024-03-18T22:58:04.821 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:04.131+0000 7fe3ab223700 1 Processor -- start 2024-03-18T22:58:04.825 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:04.131+0000 7fe3ab223700 1 -- start start 2024-03-18T22:58:04.825 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:04.131+0000 7fe3ab223700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe3a4107d30 0x7fe3a4108150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T22:58:04.826 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:04.131+0000 7fe3ab223700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fe3a4108720 con 0x7fe3a4107d30 2024-03-18T22:58:04.826 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:04.131+0000 7fe3a8fbf700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe3a4107d30 0x7fe3a4108150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T22:58:04.826 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:04.131+0000 7fe3a8fbf700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe3a4107d30 0x7fe3a4108150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:50968/0 (socket says 172.21.15.49:50968) 2024-03-18T22:58:04.826 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:04.131+0000 7fe3a8fbf700 1 -- 172.21.15.49:0/3295386955 learned_addr learned my addr 172.21.15.49:0/3295386955 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T22:58:04.826 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:04.132+0000 7fe3a8fbf700 1 -- 172.21.15.49:0/3295386955 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3a4108ee0 con 0x7fe3a4107d30 2024-03-18T22:58:04.826 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:04.132+0000 7fe3a8fbf700 1 --2- 172.21.15.49:0/3295386955 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe3a4107d30 0x7fe3a4108150 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fe3980098a0 tx=0x7fe398009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5325f5a075ab7dc8 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T22:58:04.826 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:04.132+0000 7fe3a37fe700 1 -- 172.21.15.49:0/3295386955 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe398004160 con 0x7fe3a4107d30 2024-03-18T22:58:04.826 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:04.132+0000 7fe3a37fe700 1 -- 172.21.15.49:0/3295386955 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fe398004300 con 0x7fe3a4107d30 2024-03-18T22:58:04.826 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:04.132+0000 7fe3a37fe700 1 -- 172.21.15.49:0/3295386955 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe398011b50 con 0x7fe3a4107d30 2024-03-18T22:58:04.826 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:04.132+0000 7fe3ab223700 1 -- 172.21.15.49:0/3295386955 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe3a4107d30 msgr2=0x7fe3a4108150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T22:58:04.826 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:04.132+0000 7fe3ab223700 1 --2- 172.21.15.49:0/3295386955 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe3a4107d30 0x7fe3a4108150 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fe3980098a0 tx=0x7fe398009bb0 comp rx=0 tx=0).stop 2024-03-18T22:58:04.826 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:04.133+0000 7fe3ab223700 1 -- 172.21.15.49:0/3295386955 shutdown_connections 2024-03-18T22:58:04.826 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:04.133+0000 7fe3ab223700 1 --2- 172.21.15.49:0/3295386955 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe3a4107d30 0x7fe3a4108150 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T22:58:04.827 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:04.133+0000 7fe3ab223700 1 -- 172.21.15.49:0/3295386955 >> 172.21.15.49:0/3295386955 conn(0x7fe3a41032b0 msgr2=0x7fe3a4105710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T22:58:04.827 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:04.133+0000 7fe3ab223700 1 -- 172.21.15.49:0/3295386955 shutdown_connections 2024-03-18T22:58:04.827 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:04.133+0000 7fe3ab223700 1 -- 172.21.15.49:0/3295386955 wait complete. 2024-03-18T22:58:04.827 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:04.133+0000 7fe3ab223700 1 Processor -- start 2024-03-18T22:58:04.827 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:04.133+0000 7fe3ab223700 1 -- start start 2024-03-18T22:58:04.827 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:04.133+0000 7fe3ab223700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe3a4107d30 0x7fe3a407db40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T22:58:04.827 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:04.133+0000 7fe3ab223700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fe3a410c290 con 0x7fe3a4107d30 2024-03-18T22:58:04.827 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:04.134+0000 7fe3a8fbf700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe3a4107d30 0x7fe3a407db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T22:58:04.827 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:04.134+0000 7fe3a8fbf700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe3a4107d30 0x7fe3a407db40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:50984/0 (socket says 172.21.15.49:50984) 2024-03-18T22:58:04.827 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:04.134+0000 7fe3a8fbf700 1 -- 172.21.15.49:0/2636985049 learned_addr learned my addr 172.21.15.49:0/2636985049 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T22:58:04.827 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:04.134+0000 7fe3a8fbf700 1 -- 172.21.15.49:0/2636985049 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3a407e080 con 0x7fe3a4107d30 2024-03-18T22:58:04.827 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:04.134+0000 7fe3a8fbf700 1 --2- 172.21.15.49:0/2636985049 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe3a4107d30 0x7fe3a407db40 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fe398009fb0 tx=0x7fe398011bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T22:58:04.827 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:04.134+0000 7fe3a1ffb700 1 -- 172.21.15.49:0/2636985049 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe3980038b0 con 0x7fe3a4107d30 2024-03-18T22:58:04.827 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:04.134+0000 7fe3a1ffb700 1 -- 172.21.15.49:0/2636985049 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fe398011e50 con 0x7fe3a4107d30 2024-03-18T22:58:04.828 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:04.134+0000 7fe3a1ffb700 1 -- 172.21.15.49:0/2636985049 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe398019550 con 0x7fe3a4107d30 2024-03-18T22:58:04.828 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:04.134+0000 7fe3ab223700 1 -- 172.21.15.49:0/2636985049 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3a407c270 con 0x7fe3a4107d30 2024-03-18T22:58:04.828 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:04.134+0000 7fe3ab223700 1 -- 172.21.15.49:0/2636985049 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3a407c7b0 con 0x7fe3a4107d30 2024-03-18T22:58:04.828 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:04.135+0000 7fe3a1ffb700 1 -- 172.21.15.49:0/2636985049 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fe398019be0 con 0x7fe3a4107d30 2024-03-18T22:58:04.828 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:04.135+0000 7fe3a1ffb700 1 -- 172.21.15.49:0/2636985049 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fe398024730 con 0x7fe3a4107d30 2024-03-18T22:58:04.828 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:04.135+0000 7fe3ab223700 1 -- 172.21.15.49:0/2636985049 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe3a4063940 con 0x7fe3a4107d30 2024-03-18T22:58:04.828 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:04.138+0000 7fe3a1ffb700 1 -- 172.21.15.49:0/2636985049 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fe398003a50 con 0x7fe3a4107d30 2024-03-18T22:58:04.828 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:04.208+0000 7fe3ab223700 1 -- 172.21.15.49:0/2636985049 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7fe3a407cce0 con 0x7fe3a4107d30 2024-03-18T22:58:04.828 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:04.210+0000 7fe3a1ffb700 1 -- 172.21.15.49:0/2636985049 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7fe398024930 con 0x7fe3a4107d30 2024-03-18T22:58:04.828 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:04.210+0000 7fe3ab223700 1 -- 172.21.15.49:0/2636985049 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe3a4107d30 msgr2=0x7fe3a407db40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T22:58:04.828 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:04.210+0000 7fe3ab223700 1 --2- 172.21.15.49:0/2636985049 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe3a4107d30 0x7fe3a407db40 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fe398009fb0 tx=0x7fe398011bf0 comp rx=0 tx=0).stop 2024-03-18T22:58:04.828 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:04.210+0000 7fe3ab223700 1 -- 172.21.15.49:0/2636985049 shutdown_connections 2024-03-18T22:58:04.828 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:04.210+0000 7fe3ab223700 1 --2- 172.21.15.49:0/2636985049 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe3a4107d30 0x7fe3a407db40 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T22:58:04.829 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:04.210+0000 7fe3ab223700 1 -- 172.21.15.49:0/2636985049 >> 172.21.15.49:0/2636985049 conn(0x7fe3a41032b0 msgr2=0x7fe3a4103e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T22:58:04.829 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:04.211+0000 7fe3ab223700 1 -- 172.21.15.49:0/2636985049 shutdown_connections 2024-03-18T22:58:04.829 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:04.211+0000 7fe3ab223700 1 -- 172.21.15.49:0/2636985049 wait complete. 2024-03-18T22:58:04.829 INFO:teuthology.orchestra.run.smithi049.stdout:Wrote config to /etc/ceph/ceph.conf 2024-03-18T22:58:04.851 INFO:teuthology.orchestra.run.smithi049.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-03-18T22:58:04.851 INFO:teuthology.orchestra.run.smithi049.stdout:Creating mgr... 2024-03-18T22:58:04.852 INFO:teuthology.orchestra.run.smithi049.stdout:Verifying port 0.0.0.0:9283 ... 2024-03-18T22:58:04.852 INFO:teuthology.orchestra.run.smithi049.stdout:Verifying port 0.0.0.0:8765 ... 2024-03-18T22:58:05.110 INFO:teuthology.orchestra.run.smithi049.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-c63591ea-e57a-11ee-95c9-87774f69a715@mgr.x 2024-03-18T22:58:05.110 INFO:teuthology.orchestra.run.smithi049.stdout:systemctl: stderr Failed to reset failed state of unit ceph-c63591ea-e57a-11ee-95c9-87774f69a715@mgr.x.service: Unit ceph-c63591ea-e57a-11ee-95c9-87774f69a715@mgr.x.service not loaded. 2024-03-18T22:58:05.262 INFO:teuthology.orchestra.run.smithi049.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-c63591ea-e57a-11ee-95c9-87774f69a715.target.wants/ceph-c63591ea-e57a-11ee-95c9-87774f69a715@mgr.x.service → /etc/systemd/system/ceph-c63591ea-e57a-11ee-95c9-87774f69a715@.service. 2024-03-18T22:58:05.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:05 smithi049 ceph-mon[62991]: from='client.? 172.21.15.49:0/2636985049' entity='client.admin' 2024-03-18T22:58:05.380 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 18 22:58:05 smithi049 systemd[1]: Starting Ceph mgr.x for c63591ea-e57a-11ee-95c9-87774f69a715... 2024-03-18T22:58:05.764 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 18 22:58:05 smithi049 podman[63208]: 2024-03-18 22:58:05.487648635 +0000 UTC m=+0.105521207 container create 75469e5b5506479dba1c6e29cc22757cfe9aa72093836210b62226b7d8c841a9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810, name=ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-x, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-03-18T22:58:05.765 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 18 22:58:05 smithi049 podman[63208]: 2024-03-18 22:58:05.401273385 +0000 UTC m=+0.019145927 image pull 2312c8e42d0069a2dbb2364bddb00264db4b800e7f18a7e7e4eb8f3ecab2ff1c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 2024-03-18T22:58:05.765 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 18 22:58:05 smithi049 podman[63208]: 2024-03-18 22:58:05.690095711 +0000 UTC m=+0.307968239 container init 75469e5b5506479dba1c6e29cc22757cfe9aa72093836210b62226b7d8c841a9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810, name=ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-x, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240307) 2024-03-18T22:58:05.765 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 18 22:58:05 smithi049 podman[63208]: 2024-03-18 22:58:05.692626842 +0000 UTC m=+0.310499369 container start 75469e5b5506479dba1c6e29cc22757cfe9aa72093836210b62226b7d8c841a9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810, name=ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-x, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux ) 2024-03-18T22:58:05.765 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 18 22:58:05 smithi049 bash[63208]: 75469e5b5506479dba1c6e29cc22757cfe9aa72093836210b62226b7d8c841a9 2024-03-18T22:58:05.779 INFO:teuthology.orchestra.run.smithi049.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-18T22:58:05.779 INFO:teuthology.orchestra.run.smithi049.stdout:systemctl: stdout disabled 2024-03-18T22:58:05.788 INFO:teuthology.orchestra.run.smithi049.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-18T22:58:05.788 INFO:teuthology.orchestra.run.smithi049.stdout:systemctl: stdout inactive 2024-03-18T22:58:05.788 INFO:teuthology.orchestra.run.smithi049.stdout:firewalld.service is not enabled 2024-03-18T22:58:05.788 INFO:teuthology.orchestra.run.smithi049.stdout:Not possible to enable service . firewalld.service is not available 2024-03-18T22:58:05.797 INFO:teuthology.orchestra.run.smithi049.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-18T22:58:05.798 INFO:teuthology.orchestra.run.smithi049.stdout:systemctl: stdout disabled 2024-03-18T22:58:05.806 INFO:teuthology.orchestra.run.smithi049.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-18T22:58:05.806 INFO:teuthology.orchestra.run.smithi049.stdout:systemctl: stdout inactive 2024-03-18T22:58:05.806 INFO:teuthology.orchestra.run.smithi049.stdout:firewalld.service is not enabled 2024-03-18T22:58:05.806 INFO:teuthology.orchestra.run.smithi049.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-03-18T22:58:05.806 INFO:teuthology.orchestra.run.smithi049.stdout:Waiting for mgr to start... 2024-03-18T22:58:05.806 INFO:teuthology.orchestra.run.smithi049.stdout:Waiting for mgr... 2024-03-18T22:58:06.037 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 18 22:58:05 smithi049 systemd[1]: Started Ceph mgr.x for c63591ea-e57a-11ee-95c9-87774f69a715. 2024-03-18T22:58:06.313 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 18 22:58:06 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-x[63224]: 2024-03-18T22:58:06.277+0000 7fbcdf4a5200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-18T22:58:06.630 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:06 smithi049 ceph-mon[62991]: from='client.? 172.21.15.49:0/3003286488' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-18T22:58:06.630 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 18 22:58:06 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-x[63224]: 2024-03-18T22:58:06.381+0000 7fbcdf4a5200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-18T22:58:06.921 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout 2024-03-18T22:58:06.924 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout { 2024-03-18T22:58:06.924 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "fsid": "c63591ea-e57a-11ee-95c9-87774f69a715", 2024-03-18T22:58:06.924 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "health": { 2024-03-18T22:58:06.924 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-18T22:58:06.924 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-18T22:58:06.924 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-18T22:58:06.924 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout }, 2024-03-18T22:58:06.924 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-18T22:58:06.924 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-18T22:58:06.924 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout 0 2024-03-18T22:58:06.925 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout ], 2024-03-18T22:58:06.925 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-18T22:58:06.925 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "a" 2024-03-18T22:58:06.925 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout ], 2024-03-18T22:58:06.925 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-03-18T22:58:06.925 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-18T22:58:06.925 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-18T22:58:06.925 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-03-18T22:58:06.925 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-18T22:58:06.925 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout }, 2024-03-18T22:58:06.925 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-18T22:58:06.925 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-18T22:58:06.925 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-18T22:58:06.925 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-18T22:58:06.925 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-18T22:58:06.933 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-18T22:58:06.933 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-18T22:58:06.933 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-18T22:58:06.933 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout }, 2024-03-18T22:58:06.933 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-18T22:58:06.933 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-18T22:58:06.934 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-18T22:58:06.934 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-18T22:58:06.934 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-18T22:58:06.934 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-18T22:58:06.934 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-18T22:58:06.934 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-18T22:58:06.934 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-18T22:58:06.934 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout }, 2024-03-18T22:58:06.934 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-18T22:58:06.934 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-18T22:58:06.934 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-18T22:58:06.934 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-18T22:58:06.934 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout }, 2024-03-18T22:58:06.935 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-18T22:58:06.935 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "available": false, 2024-03-18T22:58:06.935 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-18T22:58:06.935 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-18T22:58:06.935 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-18T22:58:06.935 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-18T22:58:06.935 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "restful" 2024-03-18T22:58:06.935 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout ], 2024-03-18T22:58:06.935 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-18T22:58:06.935 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout }, 2024-03-18T22:58:06.935 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-18T22:58:06.935 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-18T22:58:06.935 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "modified": "2024-03-18T22:57:58.734955+0000", 2024-03-18T22:58:06.935 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-18T22:58:06.936 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout }, 2024-03-18T22:58:06.936 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-18T22:58:06.936 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout } 2024-03-18T22:58:06.936 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:06.229+0000 7fdd29327700 1 Processor -- start 2024-03-18T22:58:06.936 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:06.230+0000 7fdd29327700 1 -- start start 2024-03-18T22:58:06.936 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:06.230+0000 7fdd29327700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdd24108870 0x7fdd2410ac60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T22:58:06.936 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:06.230+0000 7fdd29327700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fdd2407bda0 con 0x7fdd24108870 2024-03-18T22:58:06.936 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:06.230+0000 7fdd22ffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdd24108870 0x7fdd2410ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T22:58:06.936 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:06.230+0000 7fdd22ffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdd24108870 0x7fdd2410ac60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:57420/0 (socket says 172.21.15.49:57420) 2024-03-18T22:58:06.936 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:06.230+0000 7fdd22ffd700 1 -- 172.21.15.49:0/3889901327 learned_addr learned my addr 172.21.15.49:0/3889901327 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T22:58:06.936 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:06.230+0000 7fdd22ffd700 1 -- 172.21.15.49:0/3889901327 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd2410b1a0 con 0x7fdd24108870 2024-03-18T22:58:06.936 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:06.231+0000 7fdd22ffd700 1 --2- 172.21.15.49:0/3889901327 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdd24108870 0x7fdd2410ac60 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fdd140098a0 tx=0x7fdd14009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2b4a64f79567c29 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T22:58:06.936 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:06.231+0000 7fdd21ffb700 1 -- 172.21.15.49:0/3889901327 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdd14004160 con 0x7fdd24108870 2024-03-18T22:58:06.936 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:06.231+0000 7fdd21ffb700 1 -- 172.21.15.49:0/3889901327 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fdd14004300 con 0x7fdd24108870 2024-03-18T22:58:06.936 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:06.231+0000 7fdd21ffb700 1 -- 172.21.15.49:0/3889901327 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdd14011b50 con 0x7fdd24108870 2024-03-18T22:58:06.937 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:06.231+0000 7fdd29327700 1 -- 172.21.15.49:0/3889901327 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdd24108870 msgr2=0x7fdd2410ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T22:58:06.937 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:06.232+0000 7fdd29327700 1 --2- 172.21.15.49:0/3889901327 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdd24108870 0x7fdd2410ac60 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fdd140098a0 tx=0x7fdd14009bb0 comp rx=0 tx=0).stop 2024-03-18T22:58:06.937 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:06.232+0000 7fdd29327700 1 -- 172.21.15.49:0/3889901327 shutdown_connections 2024-03-18T22:58:06.937 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:06.232+0000 7fdd29327700 1 --2- 172.21.15.49:0/3889901327 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdd24108870 0x7fdd2410ac60 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T22:58:06.937 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:06.232+0000 7fdd29327700 1 -- 172.21.15.49:0/3889901327 >> 172.21.15.49:0/3889901327 conn(0x7fdd241021e0 msgr2=0x7fdd24104600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T22:58:06.937 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:06.232+0000 7fdd29327700 1 -- 172.21.15.49:0/3889901327 shutdown_connections 2024-03-18T22:58:06.937 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:06.232+0000 7fdd29327700 1 -- 172.21.15.49:0/3889901327 wait complete. 2024-03-18T22:58:06.937 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:06.232+0000 7fdd29327700 1 Processor -- start 2024-03-18T22:58:06.937 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:06.233+0000 7fdd29327700 1 -- start start 2024-03-18T22:58:06.937 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:06.233+0000 7fdd29327700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdd24108870 0x7fdd2419bce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T22:58:06.937 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:06.233+0000 7fdd29327700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fdd2410c090 con 0x7fdd24108870 2024-03-18T22:58:06.937 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:06.233+0000 7fdd22ffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdd24108870 0x7fdd2419bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T22:58:06.937 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:06.233+0000 7fdd22ffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdd24108870 0x7fdd2419bce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:57434/0 (socket says 172.21.15.49:57434) 2024-03-18T22:58:06.937 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:06.233+0000 7fdd22ffd700 1 -- 172.21.15.49:0/3003286488 learned_addr learned my addr 172.21.15.49:0/3003286488 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T22:58:06.938 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:06.233+0000 7fdd22ffd700 1 -- 172.21.15.49:0/3003286488 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd2419c220 con 0x7fdd24108870 2024-03-18T22:58:06.938 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:06.233+0000 7fdd22ffd700 1 --2- 172.21.15.49:0/3003286488 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdd24108870 0x7fdd2419bce0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fdd14009fb0 tx=0x7fdd14011bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T22:58:06.938 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:06.233+0000 7fdd0bfff700 1 -- 172.21.15.49:0/3003286488 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdd140038b0 con 0x7fdd24108870 2024-03-18T22:58:06.938 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:06.233+0000 7fdd29327700 1 -- 172.21.15.49:0/3003286488 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd2419c4b0 con 0x7fdd24108870 2024-03-18T22:58:06.938 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:06.233+0000 7fdd0bfff700 1 -- 172.21.15.49:0/3003286488 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fdd14011e50 con 0x7fdd24108870 2024-03-18T22:58:06.938 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:06.234+0000 7fdd0bfff700 1 -- 172.21.15.49:0/3003286488 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdd14019420 con 0x7fdd24108870 2024-03-18T22:58:06.938 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:06.234+0000 7fdd29327700 1 -- 172.21.15.49:0/3003286488 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd2419f190 con 0x7fdd24108870 2024-03-18T22:58:06.938 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:06.234+0000 7fdd0bfff700 1 -- 172.21.15.49:0/3003286488 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fdd14019ab0 con 0x7fdd24108870 2024-03-18T22:58:06.938 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:06.234+0000 7fdd0bfff700 1 -- 172.21.15.49:0/3003286488 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fdd1402d730 con 0x7fdd24108870 2024-03-18T22:58:06.938 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:06.234+0000 7fdd29327700 1 -- 172.21.15.49:0/3003286488 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd240693e0 con 0x7fdd24108870 2024-03-18T22:58:06.938 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:06.239+0000 7fdd0bfff700 1 -- 172.21.15.49:0/3003286488 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fdd14032070 con 0x7fdd24108870 2024-03-18T22:58:06.938 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:06.313+0000 7fdd29327700 1 -- 172.21.15.49:0/3003286488 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fdd2419f7b0 con 0x7fdd24108870 2024-03-18T22:58:06.938 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:06.313+0000 7fdd0bfff700 1 -- 172.21.15.49:0/3003286488 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7fdd14003a50 con 0x7fdd24108870 2024-03-18T22:58:06.938 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:06.315+0000 7fdd29327700 1 -- 172.21.15.49:0/3003286488 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdd24108870 msgr2=0x7fdd2419bce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T22:58:06.939 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:06.315+0000 7fdd29327700 1 --2- 172.21.15.49:0/3003286488 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdd24108870 0x7fdd2419bce0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fdd14009fb0 tx=0x7fdd14011bf0 comp rx=0 tx=0).stop 2024-03-18T22:58:06.939 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:06.315+0000 7fdd29327700 1 -- 172.21.15.49:0/3003286488 shutdown_connections 2024-03-18T22:58:06.939 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:06.315+0000 7fdd29327700 1 --2- 172.21.15.49:0/3003286488 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdd24108870 0x7fdd2419bce0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T22:58:06.939 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:06.315+0000 7fdd29327700 1 -- 172.21.15.49:0/3003286488 >> 172.21.15.49:0/3003286488 conn(0x7fdd241021e0 msgr2=0x7fdd24102d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T22:58:06.939 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:06.315+0000 7fdd29327700 1 -- 172.21.15.49:0/3003286488 shutdown_connections 2024-03-18T22:58:06.939 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:06.315+0000 7fdd29327700 1 -- 172.21.15.49:0/3003286488 wait complete. 2024-03-18T22:58:06.939 INFO:teuthology.orchestra.run.smithi049.stdout:mgr not available, waiting (1/15)... 2024-03-18T22:58:07.190 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 18 22:58:07 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-x[63224]: 2024-03-18T22:58:07.094+0000 7fbcdf4a5200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-18T22:58:07.190 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 18 22:58:07 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-x[63224]: 2024-03-18T22:58:07.190+0000 7fbcdf4a5200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-18T22:58:07.520 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 18 22:58:07 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-x[63224]: 2024-03-18T22:58:07.335+0000 7fbcdf4a5200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-18T22:58:07.520 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 18 22:58:07 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-x[63224]: 2024-03-18T22:58:07.434+0000 7fbcdf4a5200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-18T22:58:07.520 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 18 22:58:07 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-x[63224]: 2024-03-18T22:58:07.520+0000 7fbcdf4a5200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-18T22:58:08.096 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 18 22:58:07 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-x[63224]: 2024-03-18T22:58:07.797+0000 7fbcdf4a5200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-18T22:58:08.097 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 18 22:58:07 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-x[63224]: 2024-03-18T22:58:07.910+0000 7fbcdf4a5200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-18T22:58:08.097 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 18 22:58:07 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-x[63224]: 2024-03-18T22:58:07.998+0000 7fbcdf4a5200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-18T22:58:08.097 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 18 22:58:08 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-x[63224]: 2024-03-18T22:58:08.096+0000 7fbcdf4a5200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-18T22:58:08.380 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 18 22:58:08 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-x[63224]: 2024-03-18T22:58:08.281+0000 7fbcdf4a5200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-18T22:58:09.469 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:09 smithi049 ceph-mon[62991]: from='client.? 172.21.15.49:0/382766921' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-18T22:58:09.470 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 18 22:58:09 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-x[63224]: 2024-03-18T22:58:09.326+0000 7fbcdf4a5200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-18T22:58:09.470 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 18 22:58:09 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-x[63224]: 2024-03-18T22:58:09.468+0000 7fbcdf4a5200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-18T22:58:09.982 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout 2024-03-18T22:58:09.982 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout { 2024-03-18T22:58:09.982 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "fsid": "c63591ea-e57a-11ee-95c9-87774f69a715", 2024-03-18T22:58:09.982 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "health": { 2024-03-18T22:58:09.982 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-18T22:58:09.982 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-18T22:58:09.982 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-18T22:58:09.982 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout }, 2024-03-18T22:58:09.982 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-18T22:58:09.982 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-18T22:58:09.983 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout 0 2024-03-18T22:58:09.983 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout ], 2024-03-18T22:58:09.983 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-18T22:58:09.983 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "a" 2024-03-18T22:58:09.983 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout ], 2024-03-18T22:58:09.983 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "quorum_age": 5, 2024-03-18T22:58:09.983 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-18T22:58:09.983 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-18T22:58:09.983 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-03-18T22:58:09.983 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-18T22:58:09.983 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout }, 2024-03-18T22:58:09.983 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-18T22:58:09.983 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-18T22:58:09.983 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-18T22:58:09.983 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-18T22:58:09.984 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-18T22:58:09.991 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-18T22:58:09.991 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-18T22:58:09.991 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-18T22:58:09.992 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout }, 2024-03-18T22:58:09.992 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-18T22:58:09.992 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-18T22:58:09.992 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-18T22:58:09.992 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-18T22:58:09.992 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-18T22:58:09.992 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-18T22:58:09.992 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-18T22:58:09.992 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-18T22:58:09.992 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-18T22:58:09.992 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout }, 2024-03-18T22:58:09.992 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-18T22:58:09.992 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-18T22:58:09.992 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-18T22:58:09.992 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-18T22:58:09.993 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout }, 2024-03-18T22:58:09.993 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-18T22:58:09.993 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "available": false, 2024-03-18T22:58:09.993 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-18T22:58:09.993 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-18T22:58:09.993 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-18T22:58:09.993 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-18T22:58:09.993 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "restful" 2024-03-18T22:58:09.993 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout ], 2024-03-18T22:58:09.993 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-18T22:58:09.993 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout }, 2024-03-18T22:58:09.993 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-18T22:58:09.993 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-18T22:58:09.993 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "modified": "2024-03-18T22:57:58.734955+0000", 2024-03-18T22:58:09.993 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-18T22:58:09.993 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout }, 2024-03-18T22:58:09.994 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-18T22:58:09.994 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout } 2024-03-18T22:58:09.994 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:09.298+0000 7f8092f9f700 1 Processor -- start 2024-03-18T22:58:09.994 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:09.298+0000 7f8092f9f700 1 -- start start 2024-03-18T22:58:09.994 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:09.298+0000 7f8092f9f700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f808c107db0 0x7f808c1081d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T22:58:09.994 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:09.298+0000 7f8092f9f700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f808c1087a0 con 0x7f808c107db0 2024-03-18T22:58:09.994 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:09.298+0000 7f8090d3b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f808c107db0 0x7f808c1081d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T22:58:09.994 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:09.298+0000 7f8090d3b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f808c107db0 0x7f808c1081d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:57442/0 (socket says 172.21.15.49:57442) 2024-03-18T22:58:09.994 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:09.298+0000 7f8090d3b700 1 -- 172.21.15.49:0/2595850458 learned_addr learned my addr 172.21.15.49:0/2595850458 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T22:58:09.994 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:09.299+0000 7f8090d3b700 1 -- 172.21.15.49:0/2595850458 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f808c108f60 con 0x7f808c107db0 2024-03-18T22:58:09.994 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:09.299+0000 7f8090d3b700 1 --2- 172.21.15.49:0/2595850458 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f808c107db0 0x7f808c1081d0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f80800098a0 tx=0x7f8080009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b936726be64d2264 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T22:58:09.994 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:09.299+0000 7f808b7fe700 1 -- 172.21.15.49:0/2595850458 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8080004160 con 0x7f808c107db0 2024-03-18T22:58:09.994 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:09.299+0000 7f808b7fe700 1 -- 172.21.15.49:0/2595850458 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f8080004300 con 0x7f808c107db0 2024-03-18T22:58:09.994 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:09.299+0000 7f808b7fe700 1 -- 172.21.15.49:0/2595850458 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8080011b50 con 0x7f808c107db0 2024-03-18T22:58:09.995 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:09.300+0000 7f8092f9f700 1 -- 172.21.15.49:0/2595850458 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f808c107db0 msgr2=0x7f808c1081d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T22:58:09.995 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:09.300+0000 7f8092f9f700 1 --2- 172.21.15.49:0/2595850458 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f808c107db0 0x7f808c1081d0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f80800098a0 tx=0x7f8080009bb0 comp rx=0 tx=0).stop 2024-03-18T22:58:09.995 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:09.300+0000 7f8092f9f700 1 -- 172.21.15.49:0/2595850458 shutdown_connections 2024-03-18T22:58:09.995 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:09.300+0000 7f8092f9f700 1 --2- 172.21.15.49:0/2595850458 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f808c107db0 0x7f808c1081d0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T22:58:09.995 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:09.300+0000 7f8092f9f700 1 -- 172.21.15.49:0/2595850458 >> 172.21.15.49:0/2595850458 conn(0x7f808c103390 msgr2=0x7f808c1057b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T22:58:09.995 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:09.300+0000 7f8092f9f700 1 -- 172.21.15.49:0/2595850458 shutdown_connections 2024-03-18T22:58:09.995 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:09.300+0000 7f8092f9f700 1 -- 172.21.15.49:0/2595850458 wait complete. 2024-03-18T22:58:09.995 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:09.300+0000 7f8092f9f700 1 Processor -- start 2024-03-18T22:58:09.995 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:09.300+0000 7f8092f9f700 1 -- start start 2024-03-18T22:58:09.995 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:09.301+0000 7f8092f9f700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f808c107db0 0x7f808c07db40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T22:58:09.995 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:09.301+0000 7f8092f9f700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f808c10c190 con 0x7f808c107db0 2024-03-18T22:58:09.995 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:09.301+0000 7f8090d3b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f808c107db0 0x7f808c07db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T22:58:09.995 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:09.301+0000 7f8090d3b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f808c107db0 0x7f808c07db40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:57448/0 (socket says 172.21.15.49:57448) 2024-03-18T22:58:09.995 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:09.301+0000 7f8090d3b700 1 -- 172.21.15.49:0/382766921 learned_addr learned my addr 172.21.15.49:0/382766921 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T22:58:09.996 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:09.301+0000 7f8090d3b700 1 -- 172.21.15.49:0/382766921 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f808c07e080 con 0x7f808c107db0 2024-03-18T22:58:09.996 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:09.301+0000 7f8090d3b700 1 --2- 172.21.15.49:0/382766921 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f808c107db0 0x7f808c07db40 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f8080011850 tx=0x7f8080011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T22:58:09.996 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:09.301+0000 7f8089ffb700 1 -- 172.21.15.49:0/382766921 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8080003af0 con 0x7f808c107db0 2024-03-18T22:58:09.996 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:09.301+0000 7f8089ffb700 1 -- 172.21.15.49:0/382766921 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f8080003c90 con 0x7f808c107db0 2024-03-18T22:58:09.996 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:09.301+0000 7f8089ffb700 1 -- 172.21.15.49:0/382766921 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f80800236b0 con 0x7f808c107db0 2024-03-18T22:58:09.996 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:09.301+0000 7f8092f9f700 1 -- 172.21.15.49:0/382766921 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f808c07c270 con 0x7f808c107db0 2024-03-18T22:58:09.996 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:09.302+0000 7f8092f9f700 1 -- 172.21.15.49:0/382766921 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f808c07c7b0 con 0x7f808c107db0 2024-03-18T22:58:09.996 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:09.302+0000 7f8089ffb700 1 -- 172.21.15.49:0/382766921 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f8080022030 con 0x7f808c107db0 2024-03-18T22:58:09.996 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:09.302+0000 7f8089ffb700 1 -- 172.21.15.49:0/382766921 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f8080011e00 con 0x7f808c107db0 2024-03-18T22:58:09.996 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:09.303+0000 7f8092f9f700 1 -- 172.21.15.49:0/382766921 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f808c10c310 con 0x7f808c107db0 2024-03-18T22:58:09.996 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:09.305+0000 7f8089ffb700 1 -- 172.21.15.49:0/382766921 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f808001a980 con 0x7f808c107db0 2024-03-18T22:58:09.996 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:09.378+0000 7f8092f9f700 1 -- 172.21.15.49:0/382766921 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f808c104630 con 0x7f808c107db0 2024-03-18T22:58:09.996 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:09.379+0000 7f8089ffb700 1 -- 172.21.15.49:0/382766921 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f80800042c0 con 0x7f808c107db0 2024-03-18T22:58:09.996 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:09.380+0000 7f8092f9f700 1 -- 172.21.15.49:0/382766921 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f808c107db0 msgr2=0x7f808c07db40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T22:58:09.997 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:09.380+0000 7f8092f9f700 1 --2- 172.21.15.49:0/382766921 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f808c107db0 0x7f808c07db40 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f8080011850 tx=0x7f8080011c10 comp rx=0 tx=0).stop 2024-03-18T22:58:09.997 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:09.380+0000 7f8092f9f700 1 -- 172.21.15.49:0/382766921 shutdown_connections 2024-03-18T22:58:09.997 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:09.380+0000 7f8092f9f700 1 --2- 172.21.15.49:0/382766921 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f808c107db0 0x7f808c07db40 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T22:58:09.997 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:09.380+0000 7f8092f9f700 1 -- 172.21.15.49:0/382766921 >> 172.21.15.49:0/382766921 conn(0x7f808c103390 msgr2=0x7f808c103f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T22:58:09.997 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:09.380+0000 7f8092f9f700 1 -- 172.21.15.49:0/382766921 shutdown_connections 2024-03-18T22:58:09.997 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:09.380+0000 7f8092f9f700 1 -- 172.21.15.49:0/382766921 wait complete. 2024-03-18T22:58:09.997 INFO:teuthology.orchestra.run.smithi049.stdout:mgr not available, waiting (2/15)... 2024-03-18T22:58:10.115 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 18 22:58:09 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-x[63224]: 2024-03-18T22:58:09.817+0000 7fbcdf4a5200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-18T22:58:10.116 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 18 22:58:09 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-x[63224]: 2024-03-18T22:58:09.910+0000 7fbcdf4a5200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-18T22:58:10.116 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 18 22:58:10 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-x[63224]: 2024-03-18T22:58:10.114+0000 7fbcdf4a5200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-18T22:58:10.382 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 18 22:58:10 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-x[63224]: 2024-03-18T22:58:10.308+0000 7fbcdf4a5200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-18T22:58:10.872 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 18 22:58:10 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-x[63224]: 2024-03-18T22:58:10.603+0000 7fbcdf4a5200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-18T22:58:11.130 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 18 22:58:10 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-x[63224]: 2024-03-18T22:58:10.870+0000 7fbcdf4a5200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-18T22:58:11.130 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 18 22:58:11 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-x[63224]: 2024-03-18T22:58:11.116+0000 7fbcdf4a5200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-18T22:58:12.380 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 18 22:58:11 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-x[63224]: 2024-03-18T22:58:11.955+0000 7fbcdf4a5200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-18T22:58:12.880 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 18 22:58:12 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-x[63224]: 2024-03-18T22:58:12.556+0000 7fbcdf4a5200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-18T22:58:12.880 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 18 22:58:12 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-x[63224]: 2024-03-18T22:58:12.658+0000 7fbcdf4a5200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-18T22:58:13.195 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 18 22:58:12 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-x[63224]: 2024-03-18T22:58:12.882+0000 7fbcdf4a5200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-18T22:58:13.196 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:12 smithi049 ceph-mon[62991]: Activating manager daemon x 2024-03-18T22:58:13.196 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:12 smithi049 ceph-mon[62991]: mgrmap e2: x(active, starting, since 0.00308008s) 2024-03-18T22:58:13.196 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:12 smithi049 ceph-mon[62991]: from='mgr.14100 172.21.15.49:0/4274904587' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-18T22:58:13.196 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:12 smithi049 ceph-mon[62991]: from='mgr.14100 172.21.15.49:0/4274904587' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-18T22:58:13.196 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:12 smithi049 ceph-mon[62991]: from='mgr.14100 172.21.15.49:0/4274904587' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-18T22:58:13.196 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:12 smithi049 ceph-mon[62991]: from='mgr.14100 172.21.15.49:0/4274904587' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-18T22:58:13.196 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:12 smithi049 ceph-mon[62991]: from='mgr.14100 172.21.15.49:0/4274904587' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-18T22:58:13.196 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:12 smithi049 ceph-mon[62991]: Manager daemon x is now available 2024-03-18T22:58:13.196 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:12 smithi049 ceph-mon[62991]: from='mgr.14100 172.21.15.49:0/4274904587' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-18T22:58:13.196 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:12 smithi049 ceph-mon[62991]: from='mgr.14100 172.21.15.49:0/4274904587' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-18T22:58:13.196 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:12 smithi049 ceph-mon[62991]: from='mgr.14100 172.21.15.49:0/4274904587' entity='mgr.x' 2024-03-18T22:58:13.196 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:12 smithi049 ceph-mon[62991]: from='mgr.14100 172.21.15.49:0/4274904587' entity='mgr.x' 2024-03-18T22:58:13.196 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:12 smithi049 ceph-mon[62991]: from='mgr.14100 172.21.15.49:0/4274904587' entity='mgr.x' 2024-03-18T22:58:15.130 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:14 smithi049 ceph-mon[62991]: mgrmap e3: x(active, since 1.00728s) 2024-03-18T22:58:16.130 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:15 smithi049 ceph-mon[62991]: from='client.? 172.21.15.49:0/2110955264' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-18T22:58:17.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:16 smithi049 ceph-mon[62991]: mgrmap e4: x(active, since 3s) 2024-03-18T22:58:21.607 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout 2024-03-18T22:58:21.608 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout { 2024-03-18T22:58:21.608 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "fsid": "c63591ea-e57a-11ee-95c9-87774f69a715", 2024-03-18T22:58:21.608 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "health": { 2024-03-18T22:58:21.608 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-18T22:58:21.608 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-18T22:58:21.608 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-18T22:58:21.608 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout }, 2024-03-18T22:58:21.608 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-18T22:58:21.608 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-18T22:58:21.608 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout 0 2024-03-18T22:58:21.608 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout ], 2024-03-18T22:58:21.608 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-18T22:58:21.608 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "a" 2024-03-18T22:58:21.608 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout ], 2024-03-18T22:58:21.609 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "quorum_age": 11, 2024-03-18T22:58:21.609 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-18T22:58:21.609 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-18T22:58:21.609 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-03-18T22:58:21.609 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-18T22:58:21.609 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout }, 2024-03-18T22:58:21.609 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-18T22:58:21.609 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-18T22:58:21.609 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-18T22:58:21.609 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-18T22:58:21.609 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-18T22:58:21.609 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-18T22:58:21.609 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-18T22:58:21.609 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-18T22:58:21.609 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout }, 2024-03-18T22:58:21.609 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-18T22:58:21.610 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-18T22:58:21.610 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-18T22:58:21.610 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-18T22:58:21.610 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-18T22:58:21.610 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-18T22:58:21.610 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-18T22:58:21.610 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-18T22:58:21.610 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-18T22:58:21.610 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout }, 2024-03-18T22:58:21.610 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-18T22:58:21.610 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-18T22:58:21.610 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-18T22:58:21.610 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-18T22:58:21.610 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout }, 2024-03-18T22:58:21.611 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-18T22:58:21.611 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "available": true, 2024-03-18T22:58:21.611 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-18T22:58:21.611 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-18T22:58:21.611 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-18T22:58:21.611 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-18T22:58:21.611 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "restful" 2024-03-18T22:58:21.611 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout ], 2024-03-18T22:58:21.611 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-18T22:58:21.611 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout }, 2024-03-18T22:58:21.611 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-18T22:58:21.611 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-18T22:58:21.611 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "modified": "2024-03-18T22:57:58.734955+0000", 2024-03-18T22:58:21.611 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-18T22:58:21.611 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout }, 2024-03-18T22:58:21.612 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-18T22:58:21.612 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout } 2024-03-18T22:58:21.612 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:14.970+0000 7f03b4280700 1 Processor -- start 2024-03-18T22:58:21.612 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:14.970+0000 7f03b4280700 1 -- start start 2024-03-18T22:58:21.612 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:14.970+0000 7f03b4280700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f03ac106000 0x7f03ac106420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T22:58:21.612 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:14.970+0000 7f03b4280700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f03ac1069f0 con 0x7f03ac106000 2024-03-18T22:58:21.612 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:14.970+0000 7f03b201c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f03ac106000 0x7f03ac106420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T22:58:21.612 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:14.970+0000 7f03b201c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f03ac106000 0x7f03ac106420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53994/0 (socket says 172.21.15.49:53994) 2024-03-18T22:58:21.612 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:14.970+0000 7f03b201c700 1 -- 172.21.15.49:0/1475998078 learned_addr learned my addr 172.21.15.49:0/1475998078 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T22:58:21.612 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:14.971+0000 7f03b201c700 1 -- 172.21.15.49:0/1475998078 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03ac108f90 con 0x7f03ac106000 2024-03-18T22:58:21.612 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:14.971+0000 7f03b201c700 1 --2- 172.21.15.49:0/1475998078 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f03ac106000 0x7f03ac106420 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f03a80098a0 tx=0x7f03a8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bfe4cb998c66eab server_cookie=0 in_seq=0 out_seq=0 2024-03-18T22:58:21.612 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:14.971+0000 7f03b101a700 1 -- 172.21.15.49:0/1475998078 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f03a8004160 con 0x7f03ac106000 2024-03-18T22:58:21.612 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:14.971+0000 7f03b101a700 1 -- 172.21.15.49:0/1475998078 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f03a8004300 con 0x7f03ac106000 2024-03-18T22:58:21.612 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:14.971+0000 7f03b101a700 1 -- 172.21.15.49:0/1475998078 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f03a8011b50 con 0x7f03ac106000 2024-03-18T22:58:21.613 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:14.972+0000 7f03b4280700 1 -- 172.21.15.49:0/1475998078 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f03ac106000 msgr2=0x7f03ac106420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T22:58:21.613 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:14.972+0000 7f03b4280700 1 --2- 172.21.15.49:0/1475998078 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f03ac106000 0x7f03ac106420 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f03a80098a0 tx=0x7f03a8009bb0 comp rx=0 tx=0).stop 2024-03-18T22:58:21.613 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:14.972+0000 7f03b4280700 1 -- 172.21.15.49:0/1475998078 shutdown_connections 2024-03-18T22:58:21.613 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:14.972+0000 7f03b4280700 1 --2- 172.21.15.49:0/1475998078 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f03ac106000 0x7f03ac106420 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T22:58:21.613 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:14.972+0000 7f03b4280700 1 -- 172.21.15.49:0/1475998078 >> 172.21.15.49:0/1475998078 conn(0x7f03ac101990 msgr2=0x7f03ac103db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T22:58:21.613 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:14.972+0000 7f03b4280700 1 -- 172.21.15.49:0/1475998078 shutdown_connections 2024-03-18T22:58:21.613 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:14.972+0000 7f03b4280700 1 -- 172.21.15.49:0/1475998078 wait complete. 2024-03-18T22:58:21.613 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:14.972+0000 7f03b4280700 1 Processor -- start 2024-03-18T22:58:21.613 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:14.972+0000 7f03b4280700 1 -- start start 2024-03-18T22:58:21.613 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:14.973+0000 7f03b4280700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f03ac106000 0x7f03ac108810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T22:58:21.613 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:14.973+0000 7f03b4280700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f03ac10c340 con 0x7f03ac106000 2024-03-18T22:58:21.613 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:14.973+0000 7f03b201c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f03ac106000 0x7f03ac108810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T22:58:21.613 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:14.973+0000 7f03b201c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f03ac106000 0x7f03ac108810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:54006/0 (socket says 172.21.15.49:54006) 2024-03-18T22:58:21.613 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:14.973+0000 7f03b201c700 1 -- 172.21.15.49:0/2110955264 learned_addr learned my addr 172.21.15.49:0/2110955264 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T22:58:21.613 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:14.973+0000 7f03b201c700 1 -- 172.21.15.49:0/2110955264 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03ac108d50 con 0x7f03ac106000 2024-03-18T22:58:21.614 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:14.973+0000 7f03b201c700 1 --2- 172.21.15.49:0/2110955264 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f03ac106000 0x7f03ac108810 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f03a8009fb0 tx=0x7f03a8011bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T22:58:21.614 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:14.973+0000 7f03a37fe700 1 -- 172.21.15.49:0/2110955264 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f03a80038b0 con 0x7f03ac106000 2024-03-18T22:58:21.614 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:14.973+0000 7f03b4280700 1 -- 172.21.15.49:0/2110955264 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03ac106f40 con 0x7f03ac106000 2024-03-18T22:58:21.614 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:14.973+0000 7f03a37fe700 1 -- 172.21.15.49:0/2110955264 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f03a8011e50 con 0x7f03ac106000 2024-03-18T22:58:21.614 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:14.973+0000 7f03a37fe700 1 -- 172.21.15.49:0/2110955264 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f03a8019550 con 0x7f03ac106000 2024-03-18T22:58:21.614 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:14.973+0000 7f03b4280700 1 -- 172.21.15.49:0/2110955264 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03ac107480 con 0x7f03ac106000 2024-03-18T22:58:21.614 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:14.974+0000 7f03a37fe700 1 -- 172.21.15.49:0/2110955264 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 3) ==== 45989+0+0 (secure 0 0 0) 0x7f03a8019850 con 0x7f03ac106000 2024-03-18T22:58:21.614 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:14.974+0000 7f03b4280700 1 -- 172.21.15.49:0/2110955264 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03ac0693e0 con 0x7f03ac106000 2024-03-18T22:58:21.614 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:14.975+0000 7f03a37fe700 1 --2- 172.21.15.49:0/2110955264 >> [v2:172.21.15.49:6800/1985190312,v1:172.21.15.49:6801/1985190312] conn(0x7f0398038d70 0x7f039803b230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T22:58:21.614 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:14.975+0000 7f03a37fe700 1 -- 172.21.15.49:0/2110955264 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f03a804cfa0 con 0x7f03ac106000 2024-03-18T22:58:21.615 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:14.975+0000 7f03b181b700 1 --2- 172.21.15.49:0/2110955264 >> [v2:172.21.15.49:6800/1985190312,v1:172.21.15.49:6801/1985190312] conn(0x7f0398038d70 0x7f039803b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T22:58:21.615 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:14.975+0000 7f03b181b700 1 --2- 172.21.15.49:0/2110955264 >> [v2:172.21.15.49:6800/1985190312,v1:172.21.15.49:6801/1985190312] conn(0x7f0398038d70 0x7f039803b230 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f039c006fd0 tx=0x7f039c006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T22:58:21.615 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:14.980+0000 7f03a37fe700 1 -- 172.21.15.49:0/2110955264 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f03a8023e20 con 0x7f03ac106000 2024-03-18T22:58:21.615 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:15.272+0000 7f03b4280700 1 -- 172.21.15.49:0/2110955264 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f03ac107c70 con 0x7f03ac106000 2024-03-18T22:58:21.615 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:15.272+0000 7f03a37fe700 1 -- 172.21.15.49:0/2110955264 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f03a800bc40 con 0x7f03ac106000 2024-03-18T22:58:21.615 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:15.274+0000 7f03b4280700 1 -- 172.21.15.49:0/2110955264 >> [v2:172.21.15.49:6800/1985190312,v1:172.21.15.49:6801/1985190312] conn(0x7f0398038d70 msgr2=0x7f039803b230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T22:58:21.615 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:15.274+0000 7f03b4280700 1 --2- 172.21.15.49:0/2110955264 >> [v2:172.21.15.49:6800/1985190312,v1:172.21.15.49:6801/1985190312] conn(0x7f0398038d70 0x7f039803b230 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f039c006fd0 tx=0x7f039c006e40 comp rx=0 tx=0).stop 2024-03-18T22:58:21.615 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:15.274+0000 7f03b4280700 1 -- 172.21.15.49:0/2110955264 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f03ac106000 msgr2=0x7f03ac108810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T22:58:21.615 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:15.274+0000 7f03b4280700 1 --2- 172.21.15.49:0/2110955264 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f03ac106000 0x7f03ac108810 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f03a8009fb0 tx=0x7f03a8011bf0 comp rx=0 tx=0).stop 2024-03-18T22:58:21.615 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:15.274+0000 7f03b4280700 1 -- 172.21.15.49:0/2110955264 shutdown_connections 2024-03-18T22:58:21.615 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:15.274+0000 7f03b4280700 1 --2- 172.21.15.49:0/2110955264 >> [v2:172.21.15.49:6800/1985190312,v1:172.21.15.49:6801/1985190312] conn(0x7f0398038d70 0x7f039803b230 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T22:58:21.615 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:15.274+0000 7f03b4280700 1 --2- 172.21.15.49:0/2110955264 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f03ac106000 0x7f03ac108810 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T22:58:21.615 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:15.274+0000 7f03b4280700 1 -- 172.21.15.49:0/2110955264 >> 172.21.15.49:0/2110955264 conn(0x7f03ac101990 msgr2=0x7f03ac1024c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T22:58:21.616 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:15.274+0000 7f03b4280700 1 -- 172.21.15.49:0/2110955264 shutdown_connections 2024-03-18T22:58:21.616 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:15.274+0000 7f03b4280700 1 -- 172.21.15.49:0/2110955264 wait complete. 2024-03-18T22:58:21.616 INFO:teuthology.orchestra.run.smithi049.stdout:mgr is available 2024-03-18T22:58:23.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:23 smithi049 ceph-mon[62991]: from='client.? 172.21.15.49:0/2068597401' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-03-18T22:58:24.104 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout 2024-03-18T22:58:24.104 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout [global] 2024-03-18T22:58:24.105 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout fsid = c63591ea-e57a-11ee-95c9-87774f69a715 2024-03-18T22:58:24.105 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-03-18T22:58:24.105 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-03-18T22:58:24.105 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-03-18T22:58:24.105 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-03-18T22:58:24.105 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout 2024-03-18T22:58:24.105 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout [mgr] 2024-03-18T22:58:24.105 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-03-18T22:58:24.105 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout 2024-03-18T22:58:24.105 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout [osd] 2024-03-18T22:58:24.105 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-03-18T22:58:24.105 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-03-18T22:58:24.105 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-03-18T22:58:24.106 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-03-18T22:58:24.106 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-03-18T22:58:24.106 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:23.385+0000 7fa7afbc2700 1 Processor -- start 2024-03-18T22:58:24.106 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:23.385+0000 7fa7afbc2700 1 -- start start 2024-03-18T22:58:24.106 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:23.385+0000 7fa7afbc2700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa7a8106000 0x7fa7a8106420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T22:58:24.106 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:23.385+0000 7fa7afbc2700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fa7a81069f0 con 0x7fa7a8106000 2024-03-18T22:58:24.106 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:23.386+0000 7fa7ad95e700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa7a8106000 0x7fa7a8106420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T22:58:24.106 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:23.386+0000 7fa7ad95e700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa7a8106000 0x7fa7a8106420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:54016/0 (socket says 172.21.15.49:54016) 2024-03-18T22:58:24.106 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:23.386+0000 7fa7ad95e700 1 -- 172.21.15.49:0/415017574 learned_addr learned my addr 172.21.15.49:0/415017574 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T22:58:24.106 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:23.386+0000 7fa7ad95e700 1 -- 172.21.15.49:0/415017574 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7a8108f90 con 0x7fa7a8106000 2024-03-18T22:58:24.106 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:23.386+0000 7fa7ad95e700 1 --2- 172.21.15.49:0/415017574 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa7a8106000 0x7fa7a8106420 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fa7a40098a0 tx=0x7fa7a4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5502068d1bf1698d server_cookie=0 in_seq=0 out_seq=0 2024-03-18T22:58:24.106 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:23.386+0000 7fa7ac95c700 1 -- 172.21.15.49:0/415017574 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa7a4004160 con 0x7fa7a8106000 2024-03-18T22:58:24.106 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:23.387+0000 7fa7ac95c700 1 -- 172.21.15.49:0/415017574 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fa7a4004300 con 0x7fa7a8106000 2024-03-18T22:58:24.107 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:23.387+0000 7fa7ac95c700 1 -- 172.21.15.49:0/415017574 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa7a4011b50 con 0x7fa7a8106000 2024-03-18T22:58:24.107 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:23.387+0000 7fa7afbc2700 1 -- 172.21.15.49:0/415017574 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa7a8106000 msgr2=0x7fa7a8106420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T22:58:24.107 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:23.387+0000 7fa7afbc2700 1 --2- 172.21.15.49:0/415017574 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa7a8106000 0x7fa7a8106420 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fa7a40098a0 tx=0x7fa7a4009bb0 comp rx=0 tx=0).stop 2024-03-18T22:58:24.107 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:23.387+0000 7fa7afbc2700 1 -- 172.21.15.49:0/415017574 shutdown_connections 2024-03-18T22:58:24.107 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:23.387+0000 7fa7afbc2700 1 --2- 172.21.15.49:0/415017574 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa7a8106000 0x7fa7a8106420 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T22:58:24.107 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:23.387+0000 7fa7afbc2700 1 -- 172.21.15.49:0/415017574 >> 172.21.15.49:0/415017574 conn(0x7fa7a8101990 msgr2=0x7fa7a8103db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T22:58:24.107 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:23.387+0000 7fa7afbc2700 1 -- 172.21.15.49:0/415017574 shutdown_connections 2024-03-18T22:58:24.107 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:23.387+0000 7fa7afbc2700 1 -- 172.21.15.49:0/415017574 wait complete. 2024-03-18T22:58:24.107 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:23.388+0000 7fa7afbc2700 1 Processor -- start 2024-03-18T22:58:24.107 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:23.388+0000 7fa7afbc2700 1 -- start start 2024-03-18T22:58:24.107 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:23.388+0000 7fa7afbc2700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa7a8106000 0x7fa7a8108810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T22:58:24.107 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:23.388+0000 7fa7afbc2700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fa7a810c340 con 0x7fa7a8106000 2024-03-18T22:58:24.107 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:23.388+0000 7fa7ad95e700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa7a8106000 0x7fa7a8108810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T22:58:24.107 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:23.388+0000 7fa7ad95e700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa7a8106000 0x7fa7a8108810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:54022/0 (socket says 172.21.15.49:54022) 2024-03-18T22:58:24.107 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:23.388+0000 7fa7ad95e700 1 -- 172.21.15.49:0/2068597401 learned_addr learned my addr 172.21.15.49:0/2068597401 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T22:58:24.108 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:23.388+0000 7fa7ad95e700 1 -- 172.21.15.49:0/2068597401 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7a8108d50 con 0x7fa7a8106000 2024-03-18T22:58:24.108 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:23.389+0000 7fa7ad95e700 1 --2- 172.21.15.49:0/2068597401 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa7a8106000 0x7fa7a8108810 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fa7a4009fb0 tx=0x7fa7a4011bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T22:58:24.108 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:23.389+0000 7fa79effd700 1 -- 172.21.15.49:0/2068597401 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa7a40038b0 con 0x7fa7a8106000 2024-03-18T22:58:24.108 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:23.389+0000 7fa79effd700 1 -- 172.21.15.49:0/2068597401 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fa7a4011e50 con 0x7fa7a8106000 2024-03-18T22:58:24.108 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:23.389+0000 7fa7afbc2700 1 -- 172.21.15.49:0/2068597401 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7a8106f40 con 0x7fa7a8106000 2024-03-18T22:58:24.108 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:23.389+0000 7fa79effd700 1 -- 172.21.15.49:0/2068597401 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa7a4019550 con 0x7fa7a8106000 2024-03-18T22:58:24.108 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:23.389+0000 7fa7afbc2700 1 -- 172.21.15.49:0/2068597401 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7a8107480 con 0x7fa7a8106000 2024-03-18T22:58:24.108 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:23.390+0000 7fa79effd700 1 -- 172.21.15.49:0/2068597401 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 4) ==== 46095+0+0 (secure 0 0 0) 0x7fa7a4019850 con 0x7fa7a8106000 2024-03-18T22:58:24.108 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:23.390+0000 7fa79effd700 1 --2- 172.21.15.49:0/2068597401 >> [v2:172.21.15.49:6800/1985190312,v1:172.21.15.49:6801/1985190312] conn(0x7fa794038e80 0x7fa79403b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T22:58:24.108 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:23.390+0000 7fa79effd700 1 -- 172.21.15.49:0/2068597401 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fa7a404d120 con 0x7fa7a8106000 2024-03-18T22:58:24.108 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:23.390+0000 7fa7afbc2700 1 -- 172.21.15.49:0/2068597401 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa7a80693e0 con 0x7fa7a8106000 2024-03-18T22:58:24.108 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:23.390+0000 7fa7ad15d700 1 --2- 172.21.15.49:0/2068597401 >> [v2:172.21.15.49:6800/1985190312,v1:172.21.15.49:6801/1985190312] conn(0x7fa794038e80 0x7fa79403b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T22:58:24.108 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:23.390+0000 7fa7ad15d700 1 --2- 172.21.15.49:0/2068597401 >> [v2:172.21.15.49:6800/1985190312,v1:172.21.15.49:6801/1985190312] conn(0x7fa794038e80 0x7fa79403b340 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fa798006fd0 tx=0x7fa798006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T22:58:24.108 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:23.395+0000 7fa79effd700 1 -- 172.21.15.49:0/2068597401 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fa7a4023e20 con 0x7fa7a8106000 2024-03-18T22:58:24.108 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:23.599+0000 7fa7afbc2700 1 -- 172.21.15.49:0/2068597401 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7fa7a8107700 con 0x7fa7a8106000 2024-03-18T22:58:24.109 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:23.600+0000 7fa79effd700 1 -- 172.21.15.49:0/2068597401 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+373 (secure 0 0 0) 0x7fa7a4032020 con 0x7fa7a8106000 2024-03-18T22:58:24.109 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:23.603+0000 7fa7afbc2700 1 -- 172.21.15.49:0/2068597401 >> [v2:172.21.15.49:6800/1985190312,v1:172.21.15.49:6801/1985190312] conn(0x7fa794038e80 msgr2=0x7fa79403b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T22:58:24.109 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:23.603+0000 7fa7afbc2700 1 --2- 172.21.15.49:0/2068597401 >> [v2:172.21.15.49:6800/1985190312,v1:172.21.15.49:6801/1985190312] conn(0x7fa794038e80 0x7fa79403b340 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fa798006fd0 tx=0x7fa798006e40 comp rx=0 tx=0).stop 2024-03-18T22:58:24.109 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:23.603+0000 7fa7afbc2700 1 -- 172.21.15.49:0/2068597401 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa7a8106000 msgr2=0x7fa7a8108810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T22:58:24.109 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:23.603+0000 7fa7afbc2700 1 --2- 172.21.15.49:0/2068597401 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa7a8106000 0x7fa7a8108810 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fa7a4009fb0 tx=0x7fa7a4011bf0 comp rx=0 tx=0).stop 2024-03-18T22:58:24.109 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:23.603+0000 7fa7afbc2700 1 -- 172.21.15.49:0/2068597401 shutdown_connections 2024-03-18T22:58:24.109 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:23.603+0000 7fa7afbc2700 1 --2- 172.21.15.49:0/2068597401 >> [v2:172.21.15.49:6800/1985190312,v1:172.21.15.49:6801/1985190312] conn(0x7fa794038e80 0x7fa79403b340 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T22:58:24.109 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:23.603+0000 7fa7afbc2700 1 --2- 172.21.15.49:0/2068597401 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa7a8106000 0x7fa7a8108810 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T22:58:24.109 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:23.603+0000 7fa7afbc2700 1 -- 172.21.15.49:0/2068597401 >> 172.21.15.49:0/2068597401 conn(0x7fa7a8101990 msgr2=0x7fa7a81024c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T22:58:24.109 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:23.603+0000 7fa7afbc2700 1 -- 172.21.15.49:0/2068597401 shutdown_connections 2024-03-18T22:58:24.109 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:23.603+0000 7fa7afbc2700 1 -- 172.21.15.49:0/2068597401 wait complete. 2024-03-18T22:58:24.109 INFO:teuthology.orchestra.run.smithi049.stdout:Enabling cephadm module... 2024-03-18T22:58:25.118 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:24 smithi049 ceph-mon[62991]: from='client.? 172.21.15.49:0/2363362066' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-03-18T22:58:25.119 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 18 22:58:24 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-x[63224]: ignoring --setuser ceph since I am not root 2024-03-18T22:58:25.119 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 18 22:58:24 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-x[63224]: ignoring --setgroup ceph since I am not root 2024-03-18T22:58:25.412 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:24.486+0000 7fe1e29a1700 1 Processor -- start 2024-03-18T22:58:25.412 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:24.486+0000 7fe1e29a1700 1 -- start start 2024-03-18T22:58:25.412 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:24.486+0000 7fe1e29a1700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe1dc07de50 0x7fe1dc07c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T22:58:25.412 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:24.486+0000 7fe1e29a1700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fe1dc07c880 con 0x7fe1dc07de50 2024-03-18T22:58:25.412 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:24.486+0000 7fe1e199f700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe1dc07de50 0x7fe1dc07c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T22:58:25.412 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:24.486+0000 7fe1e199f700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe1dc07de50 0x7fe1dc07c2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:54034/0 (socket says 172.21.15.49:54034) 2024-03-18T22:58:25.413 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:24.486+0000 7fe1e199f700 1 -- 172.21.15.49:0/2251630413 learned_addr learned my addr 172.21.15.49:0/2251630413 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T22:58:25.413 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:24.486+0000 7fe1e199f700 1 -- 172.21.15.49:0/2251630413 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1dc07ca00 con 0x7fe1dc07de50 2024-03-18T22:58:25.413 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:24.487+0000 7fe1e199f700 1 --2- 172.21.15.49:0/2251630413 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe1dc07de50 0x7fe1dc07c2b0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fe1d80098a0 tx=0x7fe1d8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59215ba2d58a4824 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T22:58:25.413 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:24.487+0000 7fe1e099d700 1 -- 172.21.15.49:0/2251630413 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe1d8004160 con 0x7fe1dc07de50 2024-03-18T22:58:25.413 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:24.487+0000 7fe1e099d700 1 -- 172.21.15.49:0/2251630413 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fe1d8004300 con 0x7fe1dc07de50 2024-03-18T22:58:25.413 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:24.487+0000 7fe1e099d700 1 -- 172.21.15.49:0/2251630413 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe1d8011b50 con 0x7fe1dc07de50 2024-03-18T22:58:25.413 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:24.487+0000 7fe1e29a1700 1 -- 172.21.15.49:0/2251630413 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe1dc07de50 msgr2=0x7fe1dc07c2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T22:58:25.413 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:24.487+0000 7fe1e29a1700 1 --2- 172.21.15.49:0/2251630413 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe1dc07de50 0x7fe1dc07c2b0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fe1d80098a0 tx=0x7fe1d8009bb0 comp rx=0 tx=0).stop 2024-03-18T22:58:25.413 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:24.487+0000 7fe1e29a1700 1 -- 172.21.15.49:0/2251630413 shutdown_connections 2024-03-18T22:58:25.413 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:24.487+0000 7fe1e29a1700 1 --2- 172.21.15.49:0/2251630413 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe1dc07de50 0x7fe1dc07c2b0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T22:58:25.414 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:24.487+0000 7fe1e29a1700 1 -- 172.21.15.49:0/2251630413 >> 172.21.15.49:0/2251630413 conn(0x7fe1dc103110 msgr2=0x7fe1dc105530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T22:58:25.414 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:24.488+0000 7fe1e29a1700 1 -- 172.21.15.49:0/2251630413 shutdown_connections 2024-03-18T22:58:25.414 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:24.488+0000 7fe1e29a1700 1 -- 172.21.15.49:0/2251630413 wait complete. 2024-03-18T22:58:25.414 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:24.488+0000 7fe1e29a1700 1 Processor -- start 2024-03-18T22:58:25.414 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:24.488+0000 7fe1e29a1700 1 -- start start 2024-03-18T22:58:25.414 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:24.488+0000 7fe1e29a1700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe1dc07de50 0x7fe1dc19dc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T22:58:25.414 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:24.488+0000 7fe1e29a1700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fe1dc10e0e0 con 0x7fe1dc07de50 2024-03-18T22:58:25.414 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:24.489+0000 7fe1e199f700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe1dc07de50 0x7fe1dc19dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T22:58:25.414 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:24.489+0000 7fe1e199f700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe1dc07de50 0x7fe1dc19dc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:54046/0 (socket says 172.21.15.49:54046) 2024-03-18T22:58:25.414 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:24.489+0000 7fe1e199f700 1 -- 172.21.15.49:0/2363362066 learned_addr learned my addr 172.21.15.49:0/2363362066 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T22:58:25.414 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:24.489+0000 7fe1e199f700 1 -- 172.21.15.49:0/2363362066 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1dc19e180 con 0x7fe1dc07de50 2024-03-18T22:58:25.414 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:24.489+0000 7fe1e199f700 1 --2- 172.21.15.49:0/2363362066 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe1dc07de50 0x7fe1dc19dc40 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fe1d8011670 tx=0x7fe1d8011b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T22:58:25.415 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:24.489+0000 7fe1d2ffd700 1 -- 172.21.15.49:0/2363362066 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe1d80039c0 con 0x7fe1dc07de50 2024-03-18T22:58:25.415 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:24.489+0000 7fe1d2ffd700 1 -- 172.21.15.49:0/2363362066 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fe1d8011e50 con 0x7fe1dc07de50 2024-03-18T22:58:25.415 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:24.489+0000 7fe1e29a1700 1 -- 172.21.15.49:0/2363362066 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1dc19e410 con 0x7fe1dc07de50 2024-03-18T22:58:25.415 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:24.489+0000 7fe1d2ffd700 1 -- 172.21.15.49:0/2363362066 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe1d801a5c0 con 0x7fe1dc07de50 2024-03-18T22:58:25.415 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:24.489+0000 7fe1e29a1700 1 -- 172.21.15.49:0/2363362066 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1dc1a1100 con 0x7fe1dc07de50 2024-03-18T22:58:25.415 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:24.490+0000 7fe1d2ffd700 1 -- 172.21.15.49:0/2363362066 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 4) ==== 46095+0+0 (secure 0 0 0) 0x7fe1d801a8c0 con 0x7fe1dc07de50 2024-03-18T22:58:25.415 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:24.490+0000 7fe1d2ffd700 1 --2- 172.21.15.49:0/2363362066 >> [v2:172.21.15.49:6800/1985190312,v1:172.21.15.49:6801/1985190312] conn(0x7fe1c8038e80 0x7fe1c803b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T22:58:25.415 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:24.490+0000 7fe1e29a1700 1 -- 172.21.15.49:0/2363362066 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1dc0693e0 con 0x7fe1dc07de50 2024-03-18T22:58:25.415 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:24.490+0000 7fe1d2ffd700 1 -- 172.21.15.49:0/2363362066 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fe1d804d270 con 0x7fe1dc07de50 2024-03-18T22:58:25.415 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:24.490+0000 7fe1e119e700 1 --2- 172.21.15.49:0/2363362066 >> [v2:172.21.15.49:6800/1985190312,v1:172.21.15.49:6801/1985190312] conn(0x7fe1c8038e80 0x7fe1c803b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T22:58:25.415 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:24.491+0000 7fe1e119e700 1 --2- 172.21.15.49:0/2363362066 >> [v2:172.21.15.49:6800/1985190312,v1:172.21.15.49:6801/1985190312] conn(0x7fe1c8038e80 0x7fe1c803b340 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fe1cc006fd0 tx=0x7fe1cc006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T22:58:25.415 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:24.495+0000 7fe1d2ffd700 1 -- 172.21.15.49:0/2363362066 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fe1d802cc00 con 0x7fe1dc07de50 2024-03-18T22:58:25.416 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:24.759+0000 7fe1e29a1700 1 -- 172.21.15.49:0/2363362066 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7fe1dc1a12d0 con 0x7fe1dc07de50 2024-03-18T22:58:25.416 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:24.810+0000 7fe1d2ffd700 1 -- 172.21.15.49:0/2363362066 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7fe1dc1a12d0 con 0x7fe1dc07de50 2024-03-18T22:58:25.416 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:24.812+0000 7fe1d2ffd700 1 -- 172.21.15.49:0/2363362066 <== mon.0 v2:172.21.15.49:3300/0 8 ==== mgrmap(e 5) ==== 46106+0+0 (secure 0 0 0) 0x7fe1d803f0c0 con 0x7fe1dc07de50 2024-03-18T22:58:25.416 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:24.813+0000 7fe1e29a1700 1 -- 172.21.15.49:0/2363362066 >> [v2:172.21.15.49:6800/1985190312,v1:172.21.15.49:6801/1985190312] conn(0x7fe1c8038e80 msgr2=0x7fe1c803b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T22:58:25.416 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:24.813+0000 7fe1e29a1700 1 --2- 172.21.15.49:0/2363362066 >> [v2:172.21.15.49:6800/1985190312,v1:172.21.15.49:6801/1985190312] conn(0x7fe1c8038e80 0x7fe1c803b340 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fe1cc006fd0 tx=0x7fe1cc006e40 comp rx=0 tx=0).stop 2024-03-18T22:58:25.416 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:24.813+0000 7fe1e29a1700 1 -- 172.21.15.49:0/2363362066 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe1dc07de50 msgr2=0x7fe1dc19dc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T22:58:25.416 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:24.813+0000 7fe1e29a1700 1 --2- 172.21.15.49:0/2363362066 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe1dc07de50 0x7fe1dc19dc40 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fe1d8011670 tx=0x7fe1d8011b60 comp rx=0 tx=0).stop 2024-03-18T22:58:25.416 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:24.813+0000 7fe1e29a1700 1 -- 172.21.15.49:0/2363362066 shutdown_connections 2024-03-18T22:58:25.416 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:24.813+0000 7fe1e29a1700 1 --2- 172.21.15.49:0/2363362066 >> [v2:172.21.15.49:6800/1985190312,v1:172.21.15.49:6801/1985190312] conn(0x7fe1c8038e80 0x7fe1c803b340 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T22:58:25.416 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:24.813+0000 7fe1e29a1700 1 --2- 172.21.15.49:0/2363362066 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe1dc07de50 0x7fe1dc19dc40 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T22:58:25.416 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:24.813+0000 7fe1e29a1700 1 -- 172.21.15.49:0/2363362066 >> 172.21.15.49:0/2363362066 conn(0x7fe1dc103110 msgr2=0x7fe1dc103c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T22:58:25.417 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:24.813+0000 7fe1e29a1700 1 -- 172.21.15.49:0/2363362066 shutdown_connections 2024-03-18T22:58:25.417 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:24.813+0000 7fe1e29a1700 1 -- 172.21.15.49:0/2363362066 wait complete. 2024-03-18T22:58:25.668 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 18 22:58:25 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-x[63224]: 2024-03-18T22:58:25.391+0000 7fa31b6ba200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-18T22:58:25.668 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 18 22:58:25 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-x[63224]: 2024-03-18T22:58:25.492+0000 7fa31b6ba200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-18T22:58:26.071 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:25 smithi049 ceph-mon[62991]: from='client.? 172.21.15.49:0/2363362066' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-03-18T22:58:26.071 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:25 smithi049 ceph-mon[62991]: mgrmap e5: x(active, since 11s) 2024-03-18T22:58:26.380 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 18 22:58:26 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-x[63224]: 2024-03-18T22:58:26.179+0000 7fa31b6ba200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-18T22:58:26.380 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 18 22:58:26 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-x[63224]: 2024-03-18T22:58:26.276+0000 7fa31b6ba200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-18T22:58:26.725 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 18 22:58:26 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-x[63224]: 2024-03-18T22:58:26.424+0000 7fa31b6ba200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-18T22:58:26.725 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 18 22:58:26 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-x[63224]: 2024-03-18T22:58:26.525+0000 7fa31b6ba200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-18T22:58:26.725 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 18 22:58:26 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-x[63224]: 2024-03-18T22:58:26.612+0000 7fa31b6ba200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-18T22:58:26.734 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout { 2024-03-18T22:58:26.734 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-03-18T22:58:26.734 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "available": true, 2024-03-18T22:58:26.734 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-03-18T22:58:26.734 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-03-18T22:58:26.734 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout } 2024-03-18T22:58:26.734 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:25.864+0000 7f46fc65f700 1 Processor -- start 2024-03-18T22:58:26.734 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:25.864+0000 7f46fc65f700 1 -- start start 2024-03-18T22:58:26.734 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:25.864+0000 7f46fc65f700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f46f41065c0 0x7f46f41069e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T22:58:26.735 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:25.864+0000 7f46fc65f700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f46f407be20 con 0x7f46f41065c0 2024-03-18T22:58:26.735 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:25.865+0000 7f46fa3fb700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f46f41065c0 0x7f46f41069e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T22:58:26.735 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:25.865+0000 7f46fa3fb700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f46f41065c0 0x7f46f41069e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:46352/0 (socket says 172.21.15.49:46352) 2024-03-18T22:58:26.735 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:25.865+0000 7f46fa3fb700 1 -- 172.21.15.49:0/3368473652 learned_addr learned my addr 172.21.15.49:0/3368473652 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T22:58:26.735 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:25.865+0000 7f46fa3fb700 1 -- 172.21.15.49:0/3368473652 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46f4106f20 con 0x7f46f41065c0 2024-03-18T22:58:26.735 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:25.865+0000 7f46fa3fb700 1 --2- 172.21.15.49:0/3368473652 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f46f41065c0 0x7f46f41069e0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f46f00098a0 tx=0x7f46f0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a3fd5daa0e4fa03 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T22:58:26.735 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:25.865+0000 7f46f93f9700 1 -- 172.21.15.49:0/3368473652 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f46f0004160 con 0x7f46f41065c0 2024-03-18T22:58:26.735 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:25.865+0000 7f46f93f9700 1 -- 172.21.15.49:0/3368473652 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f46f0004300 con 0x7f46f41065c0 2024-03-18T22:58:26.735 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:25.865+0000 7f46f93f9700 1 -- 172.21.15.49:0/3368473652 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f46f0011b50 con 0x7f46f41065c0 2024-03-18T22:58:26.735 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:25.866+0000 7f46fc65f700 1 -- 172.21.15.49:0/3368473652 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f46f41065c0 msgr2=0x7f46f41069e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T22:58:26.735 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:25.866+0000 7f46fc65f700 1 --2- 172.21.15.49:0/3368473652 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f46f41065c0 0x7f46f41069e0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f46f00098a0 tx=0x7f46f0009bb0 comp rx=0 tx=0).stop 2024-03-18T22:58:26.735 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:25.866+0000 7f46fc65f700 1 -- 172.21.15.49:0/3368473652 shutdown_connections 2024-03-18T22:58:26.735 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:25.866+0000 7f46fc65f700 1 --2- 172.21.15.49:0/3368473652 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f46f41065c0 0x7f46f41069e0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T22:58:26.735 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:25.866+0000 7f46fc65f700 1 -- 172.21.15.49:0/3368473652 >> 172.21.15.49:0/3368473652 conn(0x7f46f41021a0 msgr2=0x7f46f41045e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T22:58:26.736 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:25.866+0000 7f46fc65f700 1 -- 172.21.15.49:0/3368473652 shutdown_connections 2024-03-18T22:58:26.736 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:25.866+0000 7f46fc65f700 1 -- 172.21.15.49:0/3368473652 wait complete. 2024-03-18T22:58:26.736 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:25.867+0000 7f46fc65f700 1 Processor -- start 2024-03-18T22:58:26.736 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:25.867+0000 7f46fc65f700 1 -- start start 2024-03-18T22:58:26.736 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:25.867+0000 7f46fc65f700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f46f41065c0 0x7f46f419de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T22:58:26.736 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:25.867+0000 7f46fc65f700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f46f4107f40 con 0x7f46f41065c0 2024-03-18T22:58:26.736 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:25.867+0000 7f46fa3fb700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f46f41065c0 0x7f46f419de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T22:58:26.736 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:25.867+0000 7f46fa3fb700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f46f41065c0 0x7f46f419de80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:46356/0 (socket says 172.21.15.49:46356) 2024-03-18T22:58:26.736 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:25.867+0000 7f46fa3fb700 1 -- 172.21.15.49:0/3845843426 learned_addr learned my addr 172.21.15.49:0/3845843426 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T22:58:26.736 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:25.868+0000 7f46fa3fb700 1 -- 172.21.15.49:0/3845843426 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46f419e3c0 con 0x7f46f41065c0 2024-03-18T22:58:26.736 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:25.868+0000 7f46fa3fb700 1 --2- 172.21.15.49:0/3845843426 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f46f41065c0 0x7f46f419de80 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f46f0011670 tx=0x7f46f0011b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T22:58:26.736 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:25.868+0000 7f46e77fe700 1 -- 172.21.15.49:0/3845843426 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f46f00039c0 con 0x7f46f41065c0 2024-03-18T22:58:26.736 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:25.868+0000 7f46e77fe700 1 -- 172.21.15.49:0/3845843426 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f46f0011e50 con 0x7f46f41065c0 2024-03-18T22:58:26.736 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:25.868+0000 7f46fc65f700 1 -- 172.21.15.49:0/3845843426 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46f419e650 con 0x7f46f41065c0 2024-03-18T22:58:26.737 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:25.868+0000 7f46e77fe700 1 -- 172.21.15.49:0/3845843426 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f46f001a5c0 con 0x7f46f41065c0 2024-03-18T22:58:26.737 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:25.868+0000 7f46fc65f700 1 -- 172.21.15.49:0/3845843426 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46f41a1340 con 0x7f46f41065c0 2024-03-18T22:58:26.737 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:25.869+0000 7f46e77fe700 1 -- 172.21.15.49:0/3845843426 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 46106+0+0 (secure 0 0 0) 0x7f46f001a8c0 con 0x7f46f41065c0 2024-03-18T22:58:26.737 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:25.869+0000 7f46fc65f700 1 -- 172.21.15.49:0/3845843426 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46f40693e0 con 0x7f46f41065c0 2024-03-18T22:58:26.737 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:25.869+0000 7f46e77fe700 1 --2- 172.21.15.49:0/3845843426 >> [v2:172.21.15.49:6800/1985190312,v1:172.21.15.49:6801/1985190312] conn(0x7f46e0038ed0 0x7f46e003b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T22:58:26.737 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:25.869+0000 7f46e77fe700 1 -- 172.21.15.49:0/3845843426 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f46f004d170 con 0x7f46f41065c0 2024-03-18T22:58:26.737 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:25.869+0000 7f46f9bfa700 1 -- 172.21.15.49:0/3845843426 >> [v2:172.21.15.49:6800/1985190312,v1:172.21.15.49:6801/1985190312] conn(0x7f46e0038ed0 msgr2=0x7f46e003b390 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.49:6800/1985190312 2024-03-18T22:58:26.737 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:25.869+0000 7f46f9bfa700 1 --2- 172.21.15.49:0/3845843426 >> [v2:172.21.15.49:6800/1985190312,v1:172.21.15.49:6801/1985190312] conn(0x7f46e0038ed0 0x7f46e003b390 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-18T22:58:26.737 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:25.874+0000 7f46e77fe700 1 -- 172.21.15.49:0/3845843426 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f46f0019c70 con 0x7f46f41065c0 2024-03-18T22:58:26.737 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:26.070+0000 7f46f9bfa700 1 -- 172.21.15.49:0/3845843426 >> [v2:172.21.15.49:6800/1985190312,v1:172.21.15.49:6801/1985190312] conn(0x7f46e0038ed0 msgr2=0x7f46e003b390 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.49:6800/1985190312 2024-03-18T22:58:26.737 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:26.070+0000 7f46f9bfa700 1 --2- 172.21.15.49:0/3845843426 >> [v2:172.21.15.49:6800/1985190312,v1:172.21.15.49:6801/1985190312] conn(0x7f46e0038ed0 0x7f46e003b390 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-18T22:58:26.737 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:26.141+0000 7f46fc65f700 1 -- 172.21.15.49:0/3845843426 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f46f40008d0 con 0x7f46f41065c0 2024-03-18T22:58:26.737 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:26.141+0000 7f46e77fe700 1 -- 172.21.15.49:0/3845843426 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7f46f0020240 con 0x7f46f41065c0 2024-03-18T22:58:26.738 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:26.144+0000 7f46fc65f700 1 -- 172.21.15.49:0/3845843426 >> [v2:172.21.15.49:6800/1985190312,v1:172.21.15.49:6801/1985190312] conn(0x7f46e0038ed0 msgr2=0x7f46e003b390 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-18T22:58:26.738 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:26.144+0000 7f46fc65f700 1 --2- 172.21.15.49:0/3845843426 >> [v2:172.21.15.49:6800/1985190312,v1:172.21.15.49:6801/1985190312] conn(0x7f46e0038ed0 0x7f46e003b390 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T22:58:26.738 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:26.144+0000 7f46fc65f700 1 -- 172.21.15.49:0/3845843426 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f46f41065c0 msgr2=0x7f46f419de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T22:58:26.738 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:26.144+0000 7f46fc65f700 1 --2- 172.21.15.49:0/3845843426 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f46f41065c0 0x7f46f419de80 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f46f0011670 tx=0x7f46f0011b60 comp rx=0 tx=0).stop 2024-03-18T22:58:26.738 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:26.144+0000 7f46fc65f700 1 -- 172.21.15.49:0/3845843426 shutdown_connections 2024-03-18T22:58:26.738 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:26.144+0000 7f46fc65f700 1 --2- 172.21.15.49:0/3845843426 >> [v2:172.21.15.49:6800/1985190312,v1:172.21.15.49:6801/1985190312] conn(0x7f46e0038ed0 0x7f46e003b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T22:58:26.738 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:26.144+0000 7f46fc65f700 1 --2- 172.21.15.49:0/3845843426 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f46f41065c0 0x7f46f419de80 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T22:58:26.738 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:26.144+0000 7f46fc65f700 1 -- 172.21.15.49:0/3845843426 >> 172.21.15.49:0/3845843426 conn(0x7f46f41021a0 msgr2=0x7f46f4194a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T22:58:26.738 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:26.144+0000 7f46fc65f700 1 -- 172.21.15.49:0/3845843426 shutdown_connections 2024-03-18T22:58:26.738 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:26.144+0000 7f46fc65f700 1 -- 172.21.15.49:0/3845843426 wait complete. 2024-03-18T22:58:26.738 INFO:teuthology.orchestra.run.smithi049.stdout:Waiting for the mgr to restart... 2024-03-18T22:58:26.738 INFO:teuthology.orchestra.run.smithi049.stdout:Waiting for mgr epoch 5... 2024-03-18T22:58:27.011 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:26 smithi049 ceph-mon[62991]: from='client.? 172.21.15.49:0/3845843426' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-03-18T22:58:27.011 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 18 22:58:26 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-x[63224]: 2024-03-18T22:58:26.893+0000 7fa31b6ba200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-18T22:58:27.319 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 18 22:58:27 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-x[63224]: 2024-03-18T22:58:27.010+0000 7fa31b6ba200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-18T22:58:27.319 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 18 22:58:27 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-x[63224]: 2024-03-18T22:58:27.104+0000 7fa31b6ba200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-18T22:58:27.319 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 18 22:58:27 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-x[63224]: 2024-03-18T22:58:27.204+0000 7fa31b6ba200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-18T22:58:27.630 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 18 22:58:27 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-x[63224]: 2024-03-18T22:58:27.386+0000 7fa31b6ba200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-18T22:58:28.880 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 18 22:58:28 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-x[63224]: 2024-03-18T22:58:28.409+0000 7fa31b6ba200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-18T22:58:28.880 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 18 22:58:28 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-x[63224]: 2024-03-18T22:58:28.547+0000 7fa31b6ba200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-18T22:58:29.192 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 18 22:58:28 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-x[63224]: 2024-03-18T22:58:28.895+0000 7fa31b6ba200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-18T22:58:29.192 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 18 22:58:28 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-x[63224]: 2024-03-18T22:58:28.988+0000 7fa31b6ba200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-18T22:58:29.192 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 18 22:58:29 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-x[63224]: 2024-03-18T22:58:29.190+0000 7fa31b6ba200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-18T22:58:29.616 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 18 22:58:29 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-x[63224]: 2024-03-18T22:58:29.383+0000 7fa31b6ba200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-18T22:58:29.880 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 18 22:58:29 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-x[63224]: 2024-03-18T22:58:29.681+0000 7fa31b6ba200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-18T22:58:30.210 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 18 22:58:29 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-x[63224]: 2024-03-18T22:58:29.948+0000 7fa31b6ba200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-18T22:58:30.210 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 18 22:58:30 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-x[63224]: 2024-03-18T22:58:30.196+0000 7fa31b6ba200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-18T22:58:31.380 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 18 22:58:31 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-x[63224]: 2024-03-18T22:58:31.038+0000 7fa31b6ba200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-18T22:58:31.959 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 18 22:58:31 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-x[63224]: 2024-03-18T22:58:31.632+0000 7fa31b6ba200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-18T22:58:31.959 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 18 22:58:31 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-x[63224]: 2024-03-18T22:58:31.733+0000 7fa31b6ba200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-18T22:58:31.960 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 18 22:58:31 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-x[63224]: 2024-03-18T22:58:31.959+0000 7fa31b6ba200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-18T22:58:32.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:32 smithi049 ceph-mon[62991]: Active manager daemon x restarted 2024-03-18T22:58:32.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:32 smithi049 ceph-mon[62991]: Activating manager daemon x 2024-03-18T22:58:32.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:32 smithi049 ceph-mon[62991]: osdmap e2: 0 total, 0 up, 0 in 2024-03-18T22:58:32.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:32 smithi049 ceph-mon[62991]: mgrmap e6: x(active, starting, since 0.00343371s) 2024-03-18T22:58:32.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:32 smithi049 ceph-mon[62991]: from='mgr.14118 172.21.15.49:0/2769660251' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-18T22:58:32.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:32 smithi049 ceph-mon[62991]: from='mgr.14118 172.21.15.49:0/2769660251' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-18T22:58:32.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:32 smithi049 ceph-mon[62991]: from='mgr.14118 172.21.15.49:0/2769660251' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-18T22:58:32.381 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:32 smithi049 ceph-mon[62991]: from='mgr.14118 172.21.15.49:0/2769660251' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-18T22:58:32.381 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:32 smithi049 ceph-mon[62991]: from='mgr.14118 172.21.15.49:0/2769660251' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-18T22:58:32.381 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:32 smithi049 ceph-mon[62991]: Manager daemon x is now available 2024-03-18T22:58:32.381 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:32 smithi049 ceph-mon[62991]: from='mgr.14118 172.21.15.49:0/2769660251' entity='mgr.x' 2024-03-18T22:58:32.381 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:32 smithi049 ceph-mon[62991]: from='mgr.14118 172.21.15.49:0/2769660251' entity='mgr.x' 2024-03-18T22:58:32.381 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:32 smithi049 ceph-mon[62991]: from='mgr.14118 172.21.15.49:0/2769660251' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T22:58:32.381 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:32 smithi049 ceph-mon[62991]: from='mgr.14118 172.21.15.49:0/2769660251' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T22:58:33.306 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:33 smithi049 ceph-mon[62991]: Found migration_current of "None". Setting to last migration. 2024-03-18T22:58:33.306 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:33 smithi049 ceph-mon[62991]: from='mgr.14118 172.21.15.49:0/2769660251' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T22:58:33.306 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:33 smithi049 ceph-mon[62991]: from='mgr.14118 172.21.15.49:0/2769660251' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-18T22:58:33.306 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:33 smithi049 ceph-mon[62991]: from='mgr.14118 172.21.15.49:0/2769660251' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-18T22:58:33.307 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:33 smithi049 ceph-mon[62991]: from='mgr.14118 172.21.15.49:0/2769660251' entity='mgr.x' 2024-03-18T22:58:33.307 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:33 smithi049 ceph-mon[62991]: from='mgr.14118 172.21.15.49:0/2769660251' entity='mgr.x' 2024-03-18T22:58:33.307 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:33 smithi049 ceph-mon[62991]: mgrmap e7: x(active, since 1.00522s) 2024-03-18T22:58:33.591 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout { 2024-03-18T22:58:33.591 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-03-18T22:58:33.592 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "initialized": true 2024-03-18T22:58:33.592 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout } 2024-03-18T22:58:33.592 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:27.112+0000 7f0f26d21700 1 Processor -- start 2024-03-18T22:58:33.592 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:27.112+0000 7f0f26d21700 1 -- start start 2024-03-18T22:58:33.592 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:27.112+0000 7f0f26d21700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0f20106650 0x7f0f20106a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T22:58:33.592 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:27.112+0000 7f0f26d21700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f0f2007be20 con 0x7f0f20106650 2024-03-18T22:58:33.592 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:27.113+0000 7f0f24abd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0f20106650 0x7f0f20106a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T22:58:33.592 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:27.113+0000 7f0f24abd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0f20106650 0x7f0f20106a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:46370/0 (socket says 172.21.15.49:46370) 2024-03-18T22:58:33.592 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:27.113+0000 7f0f24abd700 1 -- 172.21.15.49:0/2996411084 learned_addr learned my addr 172.21.15.49:0/2996411084 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T22:58:33.592 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:27.113+0000 7f0f24abd700 1 -- 172.21.15.49:0/2996411084 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f20106fb0 con 0x7f0f20106650 2024-03-18T22:58:33.592 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:27.113+0000 7f0f24abd700 1 --2- 172.21.15.49:0/2996411084 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0f20106650 0x7f0f20106a70 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f0f140098a0 tx=0x7f0f14009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf95ff6cc31401ee server_cookie=0 in_seq=0 out_seq=0 2024-03-18T22:58:33.592 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:27.113+0000 7f0f1f7fe700 1 -- 172.21.15.49:0/2996411084 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0f14004160 con 0x7f0f20106650 2024-03-18T22:58:33.592 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:27.113+0000 7f0f1f7fe700 1 -- 172.21.15.49:0/2996411084 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f0f14004300 con 0x7f0f20106650 2024-03-18T22:58:33.592 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:27.113+0000 7f0f1f7fe700 1 -- 172.21.15.49:0/2996411084 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0f14011b50 con 0x7f0f20106650 2024-03-18T22:58:33.593 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:27.114+0000 7f0f26d21700 1 -- 172.21.15.49:0/2996411084 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0f20106650 msgr2=0x7f0f20106a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T22:58:33.593 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:27.114+0000 7f0f26d21700 1 --2- 172.21.15.49:0/2996411084 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0f20106650 0x7f0f20106a70 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f0f140098a0 tx=0x7f0f14009bb0 comp rx=0 tx=0).stop 2024-03-18T22:58:33.593 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:27.114+0000 7f0f26d21700 1 -- 172.21.15.49:0/2996411084 shutdown_connections 2024-03-18T22:58:33.593 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:27.114+0000 7f0f26d21700 1 --2- 172.21.15.49:0/2996411084 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0f20106650 0x7f0f20106a70 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T22:58:33.593 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:27.114+0000 7f0f26d21700 1 -- 172.21.15.49:0/2996411084 >> 172.21.15.49:0/2996411084 conn(0x7f0f20102270 msgr2=0x7f0f20104690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T22:58:33.593 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:27.114+0000 7f0f26d21700 1 -- 172.21.15.49:0/2996411084 shutdown_connections 2024-03-18T22:58:33.593 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:27.114+0000 7f0f26d21700 1 -- 172.21.15.49:0/2996411084 wait complete. 2024-03-18T22:58:33.593 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:27.114+0000 7f0f26d21700 1 Processor -- start 2024-03-18T22:58:33.593 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:27.115+0000 7f0f26d21700 1 -- start start 2024-03-18T22:58:33.593 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:27.115+0000 7f0f26d21700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0f20106650 0x7f0f2019de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T22:58:33.593 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:27.115+0000 7f0f26d21700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f0f20107f60 con 0x7f0f20106650 2024-03-18T22:58:33.593 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:27.115+0000 7f0f24abd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0f20106650 0x7f0f2019de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T22:58:33.593 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:27.115+0000 7f0f24abd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0f20106650 0x7f0f2019de60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:46376/0 (socket says 172.21.15.49:46376) 2024-03-18T22:58:33.593 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:27.115+0000 7f0f24abd700 1 -- 172.21.15.49:0/2144338775 learned_addr learned my addr 172.21.15.49:0/2144338775 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T22:58:33.594 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:27.115+0000 7f0f24abd700 1 -- 172.21.15.49:0/2144338775 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f2019e3a0 con 0x7f0f20106650 2024-03-18T22:58:33.594 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:27.115+0000 7f0f24abd700 1 --2- 172.21.15.49:0/2144338775 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0f20106650 0x7f0f2019de60 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f0f14011670 tx=0x7f0f14011b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T22:58:33.594 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:27.115+0000 7f0f1dffb700 1 -- 172.21.15.49:0/2144338775 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0f140039c0 con 0x7f0f20106650 2024-03-18T22:58:33.594 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:27.115+0000 7f0f1dffb700 1 -- 172.21.15.49:0/2144338775 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f0f14011e50 con 0x7f0f20106650 2024-03-18T22:58:33.594 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:27.115+0000 7f0f1dffb700 1 -- 172.21.15.49:0/2144338775 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0f1401a5c0 con 0x7f0f20106650 2024-03-18T22:58:33.594 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:27.115+0000 7f0f26d21700 1 -- 172.21.15.49:0/2144338775 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f2019e630 con 0x7f0f20106650 2024-03-18T22:58:33.594 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:27.116+0000 7f0f26d21700 1 -- 172.21.15.49:0/2144338775 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f201a1320 con 0x7f0f20106650 2024-03-18T22:58:33.594 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:27.116+0000 7f0f1dffb700 1 -- 172.21.15.49:0/2144338775 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 46106+0+0 (secure 0 0 0) 0x7f0f14024430 con 0x7f0f20106650 2024-03-18T22:58:33.594 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:27.117+0000 7f0f1dffb700 1 --2- 172.21.15.49:0/2144338775 >> [v2:172.21.15.49:6800/1985190312,v1:172.21.15.49:6801/1985190312] conn(0x7f0f10038ed0 0x7f0f1003b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T22:58:33.594 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:27.117+0000 7f0f1dffb700 1 -- 172.21.15.49:0/2144338775 --> [v2:172.21.15.49:6800/1985190312,v1:172.21.15.49:6801/1985190312] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f0f1003baa0 con 0x7f0f10038ed0 2024-03-18T22:58:33.594 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:27.117+0000 7f0f1dffb700 1 -- 172.21.15.49:0/2144338775 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f0f1404d190 con 0x7f0f20106650 2024-03-18T22:58:33.594 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:27.117+0000 7f0f1ffff700 1 -- 172.21.15.49:0/2144338775 >> [v2:172.21.15.49:6800/1985190312,v1:172.21.15.49:6801/1985190312] conn(0x7f0f10038ed0 msgr2=0x7f0f1003b390 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.49:6800/1985190312 2024-03-18T22:58:33.594 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:27.117+0000 7f0f1ffff700 1 --2- 172.21.15.49:0/2144338775 >> [v2:172.21.15.49:6800/1985190312,v1:172.21.15.49:6801/1985190312] conn(0x7f0f10038ed0 0x7f0f1003b390 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-18T22:58:33.595 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:27.318+0000 7f0f1ffff700 1 -- 172.21.15.49:0/2144338775 >> [v2:172.21.15.49:6800/1985190312,v1:172.21.15.49:6801/1985190312] conn(0x7f0f10038ed0 msgr2=0x7f0f1003b390 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.49:6800/1985190312 2024-03-18T22:58:33.595 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:27.318+0000 7f0f1ffff700 1 --2- 172.21.15.49:0/2144338775 >> [v2:172.21.15.49:6800/1985190312,v1:172.21.15.49:6801/1985190312] conn(0x7f0f10038ed0 0x7f0f1003b390 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-18T22:58:33.595 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:27.719+0000 7f0f1ffff700 1 -- 172.21.15.49:0/2144338775 >> [v2:172.21.15.49:6800/1985190312,v1:172.21.15.49:6801/1985190312] conn(0x7f0f10038ed0 msgr2=0x7f0f1003b390 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.49:6800/1985190312 2024-03-18T22:58:33.595 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:27.719+0000 7f0f1ffff700 1 --2- 172.21.15.49:0/2144338775 >> [v2:172.21.15.49:6800/1985190312,v1:172.21.15.49:6801/1985190312] conn(0x7f0f10038ed0 0x7f0f1003b390 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-18T22:58:33.595 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:28.520+0000 7f0f1ffff700 1 -- 172.21.15.49:0/2144338775 >> [v2:172.21.15.49:6800/1985190312,v1:172.21.15.49:6801/1985190312] conn(0x7f0f10038ed0 msgr2=0x7f0f1003b390 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.49:6800/1985190312 2024-03-18T22:58:33.595 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:28.520+0000 7f0f1ffff700 1 --2- 172.21.15.49:0/2144338775 >> [v2:172.21.15.49:6800/1985190312,v1:172.21.15.49:6801/1985190312] conn(0x7f0f10038ed0 0x7f0f1003b390 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-18T22:58:33.595 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:30.122+0000 7f0f1ffff700 1 -- 172.21.15.49:0/2144338775 >> [v2:172.21.15.49:6800/1985190312,v1:172.21.15.49:6801/1985190312] conn(0x7f0f10038ed0 msgr2=0x7f0f1003b390 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.49:6800/1985190312 2024-03-18T22:58:33.595 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:30.122+0000 7f0f1ffff700 1 --2- 172.21.15.49:0/2144338775 >> [v2:172.21.15.49:6800/1985190312,v1:172.21.15.49:6801/1985190312] conn(0x7f0f10038ed0 0x7f0f1003b390 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-03-18T22:58:33.595 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:31.974+0000 7f0f1dffb700 1 -- 172.21.15.49:0/2144338775 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mgrmap(e 6) ==== 45873+0+0 (secure 0 0 0) 0x7f0f1402d3f0 con 0x7f0f20106650 2024-03-18T22:58:33.595 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:31.974+0000 7f0f1dffb700 1 -- 172.21.15.49:0/2144338775 >> [v2:172.21.15.49:6800/1985190312,v1:172.21.15.49:6801/1985190312] conn(0x7f0f10038ed0 msgr2=0x7f0f1003b390 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-18T22:58:33.595 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:31.974+0000 7f0f1dffb700 1 --2- 172.21.15.49:0/2144338775 >> [v2:172.21.15.49:6800/1985190312,v1:172.21.15.49:6801/1985190312] conn(0x7f0f10038ed0 0x7f0f1003b390 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T22:58:33.595 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:32.975+0000 7f0f1dffb700 1 -- 172.21.15.49:0/2144338775 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mgrmap(e 7) ==== 46000+0+0 (secure 0 0 0) 0x7f0f1404ec50 con 0x7f0f20106650 2024-03-18T22:58:33.595 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:32.975+0000 7f0f1dffb700 1 --2- 172.21.15.49:0/2144338775 >> [v2:172.21.15.49:6800/2609458951,v1:172.21.15.49:6801/2609458951] conn(0x7f0f1003c7c0 0x7f0f1003ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T22:58:33.595 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:32.975+0000 7f0f1dffb700 1 -- 172.21.15.49:0/2144338775 --> [v2:172.21.15.49:6800/2609458951,v1:172.21.15.49:6801/2609458951] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f0f1003baa0 con 0x7f0f1003c7c0 2024-03-18T22:58:33.595 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:32.976+0000 7f0f1ffff700 1 --2- 172.21.15.49:0/2144338775 >> [v2:172.21.15.49:6800/2609458951,v1:172.21.15.49:6801/2609458951] conn(0x7f0f1003c7c0 0x7f0f1003ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T22:58:33.596 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:32.977+0000 7f0f1ffff700 1 --2- 172.21.15.49:0/2144338775 >> [v2:172.21.15.49:6800/2609458951,v1:172.21.15.49:6801/2609458951] conn(0x7f0f1003c7c0 0x7f0f1003ebb0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f0f0c003a10 tx=0x7f0f0c009290 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T22:58:33.596 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:32.978+0000 7f0f1dffb700 1 -- 172.21.15.49:0/2144338775 <== mgr.14118 v2:172.21.15.49:6800/2609458951 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7774 (secure 0 0 0) 0x7f0f1003baa0 con 0x7f0f1003c7c0 2024-03-18T22:58:33.596 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:32.989+0000 7f0f26d21700 1 -- 172.21.15.49:0/2144338775 --> [v2:172.21.15.49:6800/2609458951,v1:172.21.15.49:6801/2609458951] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f0f200693e0 con 0x7f0f1003c7c0 2024-03-18T22:58:33.596 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:32.989+0000 7f0f1dffb700 1 -- 172.21.15.49:0/2144338775 <== mgr.14118 v2:172.21.15.49:6800/2609458951 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7f0f200693e0 con 0x7f0f1003c7c0 2024-03-18T22:58:33.596 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:32.990+0000 7f0f26d21700 1 -- 172.21.15.49:0/2144338775 >> [v2:172.21.15.49:6800/2609458951,v1:172.21.15.49:6801/2609458951] conn(0x7f0f1003c7c0 msgr2=0x7f0f1003ebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T22:58:33.596 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:32.990+0000 7f0f26d21700 1 --2- 172.21.15.49:0/2144338775 >> [v2:172.21.15.49:6800/2609458951,v1:172.21.15.49:6801/2609458951] conn(0x7f0f1003c7c0 0x7f0f1003ebb0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f0f0c003a10 tx=0x7f0f0c009290 comp rx=0 tx=0).stop 2024-03-18T22:58:33.596 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:32.990+0000 7f0f26d21700 1 -- 172.21.15.49:0/2144338775 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0f20106650 msgr2=0x7f0f2019de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T22:58:33.596 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:32.990+0000 7f0f26d21700 1 --2- 172.21.15.49:0/2144338775 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0f20106650 0x7f0f2019de60 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f0f14011670 tx=0x7f0f14011b60 comp rx=0 tx=0).stop 2024-03-18T22:58:33.596 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:32.990+0000 7f0f26d21700 1 -- 172.21.15.49:0/2144338775 shutdown_connections 2024-03-18T22:58:33.596 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:32.990+0000 7f0f26d21700 1 --2- 172.21.15.49:0/2144338775 >> [v2:172.21.15.49:6800/2609458951,v1:172.21.15.49:6801/2609458951] conn(0x7f0f1003c7c0 0x7f0f1003ebb0 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T22:58:33.596 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:32.990+0000 7f0f26d21700 1 --2- 172.21.15.49:0/2144338775 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0f20106650 0x7f0f2019de60 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T22:58:33.596 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:32.990+0000 7f0f26d21700 1 --2- 172.21.15.49:0/2144338775 >> [v2:172.21.15.49:6800/1985190312,v1:172.21.15.49:6801/1985190312] conn(0x7f0f10038ed0 0x7f0f1003b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T22:58:33.596 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:32.990+0000 7f0f26d21700 1 -- 172.21.15.49:0/2144338775 >> 172.21.15.49:0/2144338775 conn(0x7f0f20102270 msgr2=0x7f0f20194ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T22:58:33.596 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:32.990+0000 7f0f26d21700 1 -- 172.21.15.49:0/2144338775 shutdown_connections 2024-03-18T22:58:33.597 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:32.990+0000 7f0f26d21700 1 -- 172.21.15.49:0/2144338775 wait complete. 2024-03-18T22:58:33.597 INFO:teuthology.orchestra.run.smithi049.stdout:mgr epoch 5 is available 2024-03-18T22:58:33.597 INFO:teuthology.orchestra.run.smithi049.stdout:Setting orchestrator backend to cephadm... 2024-03-18T22:58:34.130 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:34 smithi049 ceph-mon[62991]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-03-18T22:58:34.130 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:34 smithi049 ceph-mon[62991]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-03-18T22:58:34.686 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:33.954+0000 7ff58a457700 1 Processor -- start 2024-03-18T22:58:34.686 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:33.954+0000 7ff58a457700 1 -- start start 2024-03-18T22:58:34.686 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:33.954+0000 7ff58a457700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff584108780 0x7ff58410ab70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T22:58:34.687 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:33.954+0000 7ff58a457700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7ff58407be80 con 0x7ff584108780 2024-03-18T22:58:34.687 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:33.954+0000 7ff583fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff584108780 0x7ff58410ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T22:58:34.687 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:33.954+0000 7ff583fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff584108780 0x7ff58410ab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:46432/0 (socket says 172.21.15.49:46432) 2024-03-18T22:58:34.687 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:33.954+0000 7ff583fff700 1 -- 172.21.15.49:0/1010404410 learned_addr learned my addr 172.21.15.49:0/1010404410 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T22:58:34.687 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:33.955+0000 7ff583fff700 1 -- 172.21.15.49:0/1010404410 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff58410b0b0 con 0x7ff584108780 2024-03-18T22:58:34.687 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:33.955+0000 7ff583fff700 1 --2- 172.21.15.49:0/1010404410 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff584108780 0x7ff58410ab70 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7ff5740098a0 tx=0x7ff574009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44b82a689af108cb server_cookie=0 in_seq=0 out_seq=0 2024-03-18T22:58:34.687 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:33.955+0000 7ff582ffd700 1 -- 172.21.15.49:0/1010404410 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff574004160 con 0x7ff584108780 2024-03-18T22:58:34.687 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:33.955+0000 7ff582ffd700 1 -- 172.21.15.49:0/1010404410 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7ff574004300 con 0x7ff584108780 2024-03-18T22:58:34.687 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:33.955+0000 7ff582ffd700 1 -- 172.21.15.49:0/1010404410 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff574011b50 con 0x7ff584108780 2024-03-18T22:58:34.687 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:33.956+0000 7ff58a457700 1 -- 172.21.15.49:0/1010404410 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff584108780 msgr2=0x7ff58410ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T22:58:34.687 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:33.956+0000 7ff58a457700 1 --2- 172.21.15.49:0/1010404410 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff584108780 0x7ff58410ab70 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7ff5740098a0 tx=0x7ff574009bb0 comp rx=0 tx=0).stop 2024-03-18T22:58:34.687 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:33.956+0000 7ff58a457700 1 -- 172.21.15.49:0/1010404410 shutdown_connections 2024-03-18T22:58:34.687 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:33.956+0000 7ff58a457700 1 --2- 172.21.15.49:0/1010404410 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff584108780 0x7ff58410ab70 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T22:58:34.688 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:33.956+0000 7ff58a457700 1 -- 172.21.15.49:0/1010404410 >> 172.21.15.49:0/1010404410 conn(0x7ff5841021f0 msgr2=0x7ff584104610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T22:58:34.688 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:33.956+0000 7ff58a457700 1 -- 172.21.15.49:0/1010404410 shutdown_connections 2024-03-18T22:58:34.688 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:33.956+0000 7ff58a457700 1 -- 172.21.15.49:0/1010404410 wait complete. 2024-03-18T22:58:34.688 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:33.956+0000 7ff58a457700 1 Processor -- start 2024-03-18T22:58:34.688 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:33.957+0000 7ff58a457700 1 -- start start 2024-03-18T22:58:34.688 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:33.957+0000 7ff58a457700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff584108780 0x7ff58419bc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T22:58:34.688 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:33.957+0000 7ff58a457700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7ff58410c070 con 0x7ff584108780 2024-03-18T22:58:34.688 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:33.957+0000 7ff583fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff584108780 0x7ff58419bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T22:58:34.688 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:33.957+0000 7ff583fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff584108780 0x7ff58419bc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:46440/0 (socket says 172.21.15.49:46440) 2024-03-18T22:58:34.688 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:33.957+0000 7ff583fff700 1 -- 172.21.15.49:0/926107951 learned_addr learned my addr 172.21.15.49:0/926107951 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T22:58:34.688 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:33.957+0000 7ff583fff700 1 -- 172.21.15.49:0/926107951 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff58419c150 con 0x7ff584108780 2024-03-18T22:58:34.688 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:33.957+0000 7ff583fff700 1 --2- 172.21.15.49:0/926107951 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff584108780 0x7ff58419bc10 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7ff574011b60 tx=0x7ff574011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T22:58:34.688 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:33.958+0000 7ff5817fa700 1 -- 172.21.15.49:0/926107951 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff574003b20 con 0x7ff584108780 2024-03-18T22:58:34.688 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:33.958+0000 7ff5817fa700 1 -- 172.21.15.49:0/926107951 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7ff574003cc0 con 0x7ff584108780 2024-03-18T22:58:34.689 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:33.958+0000 7ff58a457700 1 -- 172.21.15.49:0/926107951 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff58419c3e0 con 0x7ff584108780 2024-03-18T22:58:34.689 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:33.958+0000 7ff5817fa700 1 -- 172.21.15.49:0/926107951 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff574024840 con 0x7ff584108780 2024-03-18T22:58:34.689 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:33.958+0000 7ff58a457700 1 -- 172.21.15.49:0/926107951 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff58419f0d0 con 0x7ff584108780 2024-03-18T22:58:34.689 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:33.959+0000 7ff5817fa700 1 -- 172.21.15.49:0/926107951 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 7) ==== 46000+0+0 (secure 0 0 0) 0x7ff574022020 con 0x7ff584108780 2024-03-18T22:58:34.689 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:33.959+0000 7ff58a457700 1 -- 172.21.15.49:0/926107951 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff584063940 con 0x7ff584108780 2024-03-18T22:58:34.689 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:33.959+0000 7ff5817fa700 1 --2- 172.21.15.49:0/926107951 >> [v2:172.21.15.49:6800/2609458951,v1:172.21.15.49:6801/2609458951] conn(0x7ff570038e00 0x7ff57003b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T22:58:34.689 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:33.959+0000 7ff5817fa700 1 -- 172.21.15.49:0/926107951 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7ff574011e00 con 0x7ff584108780 2024-03-18T22:58:34.689 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:33.959+0000 7ff5837fe700 1 --2- 172.21.15.49:0/926107951 >> [v2:172.21.15.49:6800/2609458951,v1:172.21.15.49:6801/2609458951] conn(0x7ff570038e00 0x7ff57003b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T22:58:34.689 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:33.959+0000 7ff5837fe700 1 --2- 172.21.15.49:0/926107951 >> [v2:172.21.15.49:6800/2609458951,v1:172.21.15.49:6801/2609458951] conn(0x7ff570038e00 0x7ff57003b2c0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7ff56c006fd0 tx=0x7ff56c006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T22:58:34.689 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:33.964+0000 7ff5817fa700 1 -- 172.21.15.49:0/926107951 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7ff574016260 con 0x7ff584108780 2024-03-18T22:58:34.689 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:34.026+0000 7ff5817fa700 1 -- 172.21.15.49:0/926107951 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mgrmap(e 8) ==== 46106+0+0 (secure 0 0 0) 0x7ff57401a950 con 0x7ff584108780 2024-03-18T22:58:34.689 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:34.212+0000 7ff58a457700 1 -- 172.21.15.49:0/926107951 --> [v2:172.21.15.49:6800/2609458951,v1:172.21.15.49:6801/2609458951] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7ff5841033a0 con 0x7ff570038e00 2024-03-18T22:58:34.689 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:34.217+0000 7ff5817fa700 1 -- 172.21.15.49:0/926107951 <== mgr.14118 v2:172.21.15.49:6800/2609458951 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7ff5841033a0 con 0x7ff570038e00 2024-03-18T22:58:34.689 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:34.220+0000 7ff58a457700 1 -- 172.21.15.49:0/926107951 >> [v2:172.21.15.49:6800/2609458951,v1:172.21.15.49:6801/2609458951] conn(0x7ff570038e00 msgr2=0x7ff57003b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T22:58:34.689 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:34.220+0000 7ff58a457700 1 --2- 172.21.15.49:0/926107951 >> [v2:172.21.15.49:6800/2609458951,v1:172.21.15.49:6801/2609458951] conn(0x7ff570038e00 0x7ff57003b2c0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7ff56c006fd0 tx=0x7ff56c006e40 comp rx=0 tx=0).stop 2024-03-18T22:58:34.690 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:34.220+0000 7ff58a457700 1 -- 172.21.15.49:0/926107951 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff584108780 msgr2=0x7ff58419bc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T22:58:34.690 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:34.220+0000 7ff58a457700 1 --2- 172.21.15.49:0/926107951 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff584108780 0x7ff58419bc10 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7ff574011b60 tx=0x7ff574011c40 comp rx=0 tx=0).stop 2024-03-18T22:58:34.690 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:34.220+0000 7ff58a457700 1 -- 172.21.15.49:0/926107951 shutdown_connections 2024-03-18T22:58:34.690 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:34.220+0000 7ff58a457700 1 --2- 172.21.15.49:0/926107951 >> [v2:172.21.15.49:6800/2609458951,v1:172.21.15.49:6801/2609458951] conn(0x7ff570038e00 0x7ff57003b2c0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T22:58:34.690 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:34.220+0000 7ff58a457700 1 --2- 172.21.15.49:0/926107951 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff584108780 0x7ff58419bc10 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T22:58:34.690 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:34.220+0000 7ff58a457700 1 -- 172.21.15.49:0/926107951 >> 172.21.15.49:0/926107951 conn(0x7ff5841021f0 msgr2=0x7ff584102c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T22:58:34.690 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:34.220+0000 7ff58a457700 1 -- 172.21.15.49:0/926107951 shutdown_connections 2024-03-18T22:58:34.690 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:34.220+0000 7ff58a457700 1 -- 172.21.15.49:0/926107951 wait complete. 2024-03-18T22:58:35.326 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:35 smithi049 ceph-mon[62991]: mgrmap e8: x(active, since 2s) 2024-03-18T22:58:35.326 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:35 smithi049 ceph-mon[62991]: from='client.14130 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T22:58:35.326 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:35 smithi049 ceph-mon[62991]: from='mgr.14118 172.21.15.49:0/2769660251' entity='mgr.x' 2024-03-18T22:58:35.326 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:35 smithi049 ceph-mon[62991]: from='mgr.14118 172.21.15.49:0/2769660251' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T22:58:35.893 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout value unchanged 2024-03-18T22:58:35.893 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:35.096+0000 7f6e652da700 1 Processor -- start 2024-03-18T22:58:35.893 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:35.096+0000 7f6e652da700 1 -- start start 2024-03-18T22:58:35.893 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:35.096+0000 7f6e652da700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6e60106c00 0x7f6e60107020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T22:58:35.894 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:35.096+0000 7f6e652da700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f6e601075f0 con 0x7f6e60106c00 2024-03-18T22:58:35.894 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:35.097+0000 7f6e5effd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6e60106c00 0x7f6e60107020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T22:58:35.894 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:35.097+0000 7f6e5effd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6e60106c00 0x7f6e60107020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:59572/0 (socket says 172.21.15.49:59572) 2024-03-18T22:58:35.894 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:35.097+0000 7f6e5effd700 1 -- 172.21.15.49:0/12600991 learned_addr learned my addr 172.21.15.49:0/12600991 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T22:58:35.894 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:35.097+0000 7f6e5effd700 1 -- 172.21.15.49:0/12600991 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e60107e00 con 0x7f6e60106c00 2024-03-18T22:58:35.894 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:35.097+0000 7f6e5effd700 1 --2- 172.21.15.49:0/12600991 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6e60106c00 0x7f6e60107020 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f6e500098a0 tx=0x7f6e50009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f217fc724136e59 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T22:58:35.894 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:35.097+0000 7f6e5dffb700 1 -- 172.21.15.49:0/12600991 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6e50004160 con 0x7f6e60106c00 2024-03-18T22:58:35.894 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:35.097+0000 7f6e5dffb700 1 -- 172.21.15.49:0/12600991 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f6e50004300 con 0x7f6e60106c00 2024-03-18T22:58:35.894 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:35.097+0000 7f6e5dffb700 1 -- 172.21.15.49:0/12600991 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6e50011b50 con 0x7f6e60106c00 2024-03-18T22:58:35.894 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:35.098+0000 7f6e652da700 1 -- 172.21.15.49:0/12600991 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6e60106c00 msgr2=0x7f6e60107020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T22:58:35.894 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:35.098+0000 7f6e652da700 1 --2- 172.21.15.49:0/12600991 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6e60106c00 0x7f6e60107020 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f6e500098a0 tx=0x7f6e50009bb0 comp rx=0 tx=0).stop 2024-03-18T22:58:35.894 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:35.098+0000 7f6e652da700 1 -- 172.21.15.49:0/12600991 shutdown_connections 2024-03-18T22:58:35.894 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:35.098+0000 7f6e652da700 1 --2- 172.21.15.49:0/12600991 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6e60106c00 0x7f6e60107020 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T22:58:35.895 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:35.098+0000 7f6e652da700 1 -- 172.21.15.49:0/12600991 >> 172.21.15.49:0/12600991 conn(0x7f6e601021a0 msgr2=0x7f6e601045e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T22:58:35.895 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:35.098+0000 7f6e652da700 1 -- 172.21.15.49:0/12600991 shutdown_connections 2024-03-18T22:58:35.895 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:35.098+0000 7f6e652da700 1 -- 172.21.15.49:0/12600991 wait complete. 2024-03-18T22:58:35.895 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:35.098+0000 7f6e652da700 1 Processor -- start 2024-03-18T22:58:35.895 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:35.099+0000 7f6e652da700 1 -- start start 2024-03-18T22:58:35.895 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:35.099+0000 7f6e652da700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6e60106c00 0x7f6e6019bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T22:58:35.895 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:35.099+0000 7f6e652da700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f6e6010b1b0 con 0x7f6e60106c00 2024-03-18T22:58:35.895 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:35.099+0000 7f6e5effd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6e60106c00 0x7f6e6019bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T22:58:35.895 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:35.099+0000 7f6e5effd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6e60106c00 0x7f6e6019bcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:59582/0 (socket says 172.21.15.49:59582) 2024-03-18T22:58:35.895 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:35.099+0000 7f6e5effd700 1 -- 172.21.15.49:0/185768307 learned_addr learned my addr 172.21.15.49:0/185768307 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T22:58:35.895 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:35.099+0000 7f6e5effd700 1 -- 172.21.15.49:0/185768307 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e6019c200 con 0x7f6e60106c00 2024-03-18T22:58:35.895 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:35.099+0000 7f6e5effd700 1 --2- 172.21.15.49:0/185768307 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6e60106c00 0x7f6e6019bcc0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f6e50011670 tx=0x7f6e50011b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T22:58:35.895 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:35.099+0000 7f6e57fff700 1 -- 172.21.15.49:0/185768307 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6e500039c0 con 0x7f6e60106c00 2024-03-18T22:58:35.895 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:35.100+0000 7f6e652da700 1 -- 172.21.15.49:0/185768307 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e6019c490 con 0x7f6e60106c00 2024-03-18T22:58:35.896 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:35.100+0000 7f6e57fff700 1 -- 172.21.15.49:0/185768307 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f6e50011e50 con 0x7f6e60106c00 2024-03-18T22:58:35.896 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:35.100+0000 7f6e57fff700 1 -- 172.21.15.49:0/185768307 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6e5001a5c0 con 0x7f6e60106c00 2024-03-18T22:58:35.896 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:35.100+0000 7f6e652da700 1 -- 172.21.15.49:0/185768307 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e6019f180 con 0x7f6e60106c00 2024-03-18T22:58:35.896 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:35.100+0000 7f6e57fff700 1 -- 172.21.15.49:0/185768307 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 8) ==== 46106+0+0 (secure 0 0 0) 0x7f6e5001a8c0 con 0x7f6e60106c00 2024-03-18T22:58:35.896 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:35.101+0000 7f6e57fff700 1 --2- 172.21.15.49:0/185768307 >> [v2:172.21.15.49:6800/2609458951,v1:172.21.15.49:6801/2609458951] conn(0x7f6e4c038ed0 0x7f6e4c03b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T22:58:35.896 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:35.101+0000 7f6e57fff700 1 -- 172.21.15.49:0/185768307 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f6e5004d4a0 con 0x7f6e60106c00 2024-03-18T22:58:35.896 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:35.101+0000 7f6e5e7fc700 1 --2- 172.21.15.49:0/185768307 >> [v2:172.21.15.49:6800/2609458951,v1:172.21.15.49:6801/2609458951] conn(0x7f6e4c038ed0 0x7f6e4c03b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T22:58:35.896 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:35.101+0000 7f6e652da700 1 -- 172.21.15.49:0/185768307 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e600693e0 con 0x7f6e60106c00 2024-03-18T22:58:35.896 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:35.101+0000 7f6e5e7fc700 1 --2- 172.21.15.49:0/185768307 >> [v2:172.21.15.49:6800/2609458951,v1:172.21.15.49:6801/2609458951] conn(0x7f6e4c038ed0 0x7f6e4c03b390 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f6e48006fd0 tx=0x7f6e48006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T22:58:35.896 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:35.110+0000 7f6e57fff700 1 -- 172.21.15.49:0/185768307 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f6e50019c70 con 0x7f6e60106c00 2024-03-18T22:58:35.896 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:35.326+0000 7f6e652da700 1 -- 172.21.15.49:0/185768307 --> [v2:172.21.15.49:6800/2609458951,v1:172.21.15.49:6801/2609458951] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7f6e6002de90 con 0x7f6e4c038ed0 2024-03-18T22:58:35.896 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:35.326+0000 7f6e57fff700 1 -- 172.21.15.49:0/185768307 <== mgr.14118 v2:172.21.15.49:6800/2609458951 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7f6e6002de90 con 0x7f6e4c038ed0 2024-03-18T22:58:35.896 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:35.329+0000 7f6e652da700 1 -- 172.21.15.49:0/185768307 >> [v2:172.21.15.49:6800/2609458951,v1:172.21.15.49:6801/2609458951] conn(0x7f6e4c038ed0 msgr2=0x7f6e4c03b390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T22:58:35.897 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:35.329+0000 7f6e652da700 1 --2- 172.21.15.49:0/185768307 >> [v2:172.21.15.49:6800/2609458951,v1:172.21.15.49:6801/2609458951] conn(0x7f6e4c038ed0 0x7f6e4c03b390 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f6e48006fd0 tx=0x7f6e48006e40 comp rx=0 tx=0).stop 2024-03-18T22:58:35.897 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:35.329+0000 7f6e652da700 1 -- 172.21.15.49:0/185768307 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6e60106c00 msgr2=0x7f6e6019bcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T22:58:35.897 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:35.329+0000 7f6e652da700 1 --2- 172.21.15.49:0/185768307 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6e60106c00 0x7f6e6019bcc0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f6e50011670 tx=0x7f6e50011b60 comp rx=0 tx=0).stop 2024-03-18T22:58:35.897 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:35.329+0000 7f6e652da700 1 -- 172.21.15.49:0/185768307 shutdown_connections 2024-03-18T22:58:35.897 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:35.329+0000 7f6e652da700 1 --2- 172.21.15.49:0/185768307 >> [v2:172.21.15.49:6800/2609458951,v1:172.21.15.49:6801/2609458951] conn(0x7f6e4c038ed0 0x7f6e4c03b390 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T22:58:35.897 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:35.329+0000 7f6e652da700 1 --2- 172.21.15.49:0/185768307 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6e60106c00 0x7f6e6019bcc0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T22:58:35.897 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:35.329+0000 7f6e652da700 1 -- 172.21.15.49:0/185768307 >> 172.21.15.49:0/185768307 conn(0x7f6e601021a0 msgr2=0x7f6e601030b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T22:58:35.897 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:35.329+0000 7f6e652da700 1 -- 172.21.15.49:0/185768307 shutdown_connections 2024-03-18T22:58:35.897 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:35.329+0000 7f6e652da700 1 -- 172.21.15.49:0/185768307 wait complete. 2024-03-18T22:58:35.897 INFO:teuthology.orchestra.run.smithi049.stdout:Generating ssh key... 2024-03-18T22:58:36.271 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:36 smithi049 ceph-mon[62991]: [18/Mar/2024:22:58:35] ENGINE Bus STARTING 2024-03-18T22:58:36.272 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:36 smithi049 ceph-mon[62991]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T22:58:36.272 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:36 smithi049 ceph-mon[62991]: [18/Mar/2024:22:58:35] ENGINE Serving on http://172.21.15.49:8765 2024-03-18T22:58:36.272 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:36 smithi049 ceph-mon[62991]: [18/Mar/2024:22:58:35] ENGINE Serving on https://172.21.15.49:7150 2024-03-18T22:58:36.272 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:36 smithi049 ceph-mon[62991]: [18/Mar/2024:22:58:35] ENGINE Bus STARTED 2024-03-18T22:58:36.272 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:36 smithi049 ceph-mon[62991]: from='mgr.14118 172.21.15.49:0/2769660251' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T22:58:37.381 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:37 smithi049 ceph-mon[62991]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T22:58:37.381 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:37 smithi049 ceph-mon[62991]: Generating ssh key... 2024-03-18T22:58:37.381 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 18 22:58:37 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-x[63224]: Generating public/private rsa key pair. 2024-03-18T22:58:37.381 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 18 22:58:37 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-x[63224]: Your identification has been saved in /tmp/tmpakhqcng4/key. 2024-03-18T22:58:37.381 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 18 22:58:37 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-x[63224]: Your public key has been saved in /tmp/tmpakhqcng4/key.pub. 2024-03-18T22:58:37.381 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 18 22:58:37 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-x[63224]: The key fingerprint is: 2024-03-18T22:58:37.381 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 18 22:58:37 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-x[63224]: SHA256:sv90FDjPEafCaTQNJOldgh6sRDprmlviTbJRVYU4mgw ceph-c63591ea-e57a-11ee-95c9-87774f69a715 2024-03-18T22:58:37.381 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 18 22:58:37 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-x[63224]: The key's randomart image is: 2024-03-18T22:58:37.381 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 18 22:58:37 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-x[63224]: +---[RSA 3072]----+ 2024-03-18T22:58:37.382 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 18 22:58:37 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-x[63224]: | ..oo**o. . | 2024-03-18T22:58:37.382 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 18 22:58:37 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-x[63224]: | E .+.*+.+o+ | 2024-03-18T22:58:37.382 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 18 22:58:37 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-x[63224]: | oo+.= oBo+ | 2024-03-18T22:58:37.382 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 18 22:58:37 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-x[63224]: | ++. o..= o | 2024-03-18T22:58:37.382 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 18 22:58:37 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-x[63224]: | + . S + | 2024-03-18T22:58:37.382 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 18 22:58:37 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-x[63224]: | = o . | 2024-03-18T22:58:37.382 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 18 22:58:37 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-x[63224]: | * o . . . | 2024-03-18T22:58:37.382 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 18 22:58:37 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-x[63224]: | . X . . . | 2024-03-18T22:58:37.382 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 18 22:58:37 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-x[63224]: | + . ... | 2024-03-18T22:58:37.382 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 18 22:58:37 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-x[63224]: +----[SHA256]-----+ 2024-03-18T22:58:37.737 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:36.296+0000 7f787eaba700 1 Processor -- start 2024-03-18T22:58:37.737 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:36.296+0000 7f787eaba700 1 -- start start 2024-03-18T22:58:37.737 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:36.297+0000 7f787eaba700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7878108830 0x7f787810ac20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T22:58:37.737 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:36.297+0000 7f787eaba700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f787807bdc0 con 0x7f7878108830 2024-03-18T22:58:37.737 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:36.297+0000 7f787c856700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7878108830 0x7f787810ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T22:58:37.738 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:36.297+0000 7f787c856700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7878108830 0x7f787810ac20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:59584/0 (socket says 172.21.15.49:59584) 2024-03-18T22:58:37.738 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:36.297+0000 7f787c856700 1 -- 172.21.15.49:0/3219230637 learned_addr learned my addr 172.21.15.49:0/3219230637 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T22:58:37.738 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:36.297+0000 7f787c856700 1 -- 172.21.15.49:0/3219230637 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f787810b160 con 0x7f7878108830 2024-03-18T22:58:37.738 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:36.297+0000 7f787c856700 1 --2- 172.21.15.49:0/3219230637 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7878108830 0x7f787810ac20 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f786c0098a0 tx=0x7f786c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=973231b563f4a599 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T22:58:37.738 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:36.297+0000 7f78777fe700 1 -- 172.21.15.49:0/3219230637 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f786c004160 con 0x7f7878108830 2024-03-18T22:58:37.742 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:36.297+0000 7f78777fe700 1 -- 172.21.15.49:0/3219230637 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f786c004300 con 0x7f7878108830 2024-03-18T22:58:37.743 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:36.297+0000 7f78777fe700 1 -- 172.21.15.49:0/3219230637 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f786c011b50 con 0x7f7878108830 2024-03-18T22:58:37.743 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:36.298+0000 7f787eaba700 1 -- 172.21.15.49:0/3219230637 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7878108830 msgr2=0x7f787810ac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T22:58:37.743 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:36.298+0000 7f787eaba700 1 --2- 172.21.15.49:0/3219230637 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7878108830 0x7f787810ac20 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f786c0098a0 tx=0x7f786c009bb0 comp rx=0 tx=0).stop 2024-03-18T22:58:37.743 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:36.298+0000 7f787eaba700 1 -- 172.21.15.49:0/3219230637 shutdown_connections 2024-03-18T22:58:37.743 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:36.298+0000 7f787eaba700 1 --2- 172.21.15.49:0/3219230637 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7878108830 0x7f787810ac20 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T22:58:37.743 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:36.298+0000 7f787eaba700 1 -- 172.21.15.49:0/3219230637 >> 172.21.15.49:0/3219230637 conn(0x7f78781021a0 msgr2=0x7f78781045e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T22:58:37.743 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:36.298+0000 7f787eaba700 1 -- 172.21.15.49:0/3219230637 shutdown_connections 2024-03-18T22:58:37.743 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:36.298+0000 7f787eaba700 1 -- 172.21.15.49:0/3219230637 wait complete. 2024-03-18T22:58:37.743 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:36.299+0000 7f787eaba700 1 Processor -- start 2024-03-18T22:58:37.743 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:36.299+0000 7f787eaba700 1 -- start start 2024-03-18T22:58:37.743 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:36.299+0000 7f787eaba700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7878108830 0x7f787819de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T22:58:37.743 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:36.299+0000 7f787eaba700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f787810c060 con 0x7f7878108830 2024-03-18T22:58:37.743 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:36.299+0000 7f787c856700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7878108830 0x7f787819de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T22:58:37.744 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:36.299+0000 7f787c856700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7878108830 0x7f787819de80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:59590/0 (socket says 172.21.15.49:59590) 2024-03-18T22:58:37.744 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:36.299+0000 7f787c856700 1 -- 172.21.15.49:0/861720333 learned_addr learned my addr 172.21.15.49:0/861720333 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T22:58:37.744 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:36.299+0000 7f787c856700 1 -- 172.21.15.49:0/861720333 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f787819e3c0 con 0x7f7878108830 2024-03-18T22:58:37.744 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:36.300+0000 7f787c856700 1 --2- 172.21.15.49:0/861720333 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7878108830 0x7f787819de80 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f786c011b60 tx=0x7f786c011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T22:58:37.744 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:36.300+0000 7f7875ffb700 1 -- 172.21.15.49:0/861720333 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f786c003b20 con 0x7f7878108830 2024-03-18T22:58:37.744 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:36.300+0000 7f7875ffb700 1 -- 172.21.15.49:0/861720333 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f786c003cc0 con 0x7f7878108830 2024-03-18T22:58:37.744 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:36.300+0000 7f7875ffb700 1 -- 172.21.15.49:0/861720333 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f786c024840 con 0x7f7878108830 2024-03-18T22:58:37.744 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:36.300+0000 7f787eaba700 1 -- 172.21.15.49:0/861720333 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f787819e650 con 0x7f7878108830 2024-03-18T22:58:37.744 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:36.300+0000 7f787eaba700 1 -- 172.21.15.49:0/861720333 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78781a1340 con 0x7f7878108830 2024-03-18T22:58:37.744 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:36.301+0000 7f7875ffb700 1 -- 172.21.15.49:0/861720333 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 8) ==== 46106+0+0 (secure 0 0 0) 0x7f786c022020 con 0x7f7878108830 2024-03-18T22:58:37.744 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:36.301+0000 7f787eaba700 1 -- 172.21.15.49:0/861720333 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78780693e0 con 0x7f7878108830 2024-03-18T22:58:37.744 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:36.301+0000 7f7875ffb700 1 --2- 172.21.15.49:0/861720333 >> [v2:172.21.15.49:6800/2609458951,v1:172.21.15.49:6801/2609458951] conn(0x7f7868038f70 0x7f786803b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T22:58:37.744 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:36.301+0000 7f7875ffb700 1 -- 172.21.15.49:0/861720333 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f786c011e00 con 0x7f7878108830 2024-03-18T22:58:37.744 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:36.301+0000 7f7877fff700 1 --2- 172.21.15.49:0/861720333 >> [v2:172.21.15.49:6800/2609458951,v1:172.21.15.49:6801/2609458951] conn(0x7f7868038f70 0x7f786803b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T22:58:37.745 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:36.302+0000 7f7877fff700 1 --2- 172.21.15.49:0/861720333 >> [v2:172.21.15.49:6800/2609458951,v1:172.21.15.49:6801/2609458951] conn(0x7f7868038f70 0x7f786803b430 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f7864006fd0 tx=0x7f7864006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T22:58:37.745 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:36.306+0000 7f7875ffb700 1 -- 172.21.15.49:0/861720333 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f786c00b3e0 con 0x7f7878108830 2024-03-18T22:58:37.745 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:36.514+0000 7f787eaba700 1 -- 172.21.15.49:0/861720333 --> [v2:172.21.15.49:6800/2609458951,v1:172.21.15.49:6801/2609458951] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7f7878103370 con 0x7f7868038f70 2024-03-18T22:58:37.745 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:37.194+0000 7f7875ffb700 1 -- 172.21.15.49:0/861720333 <== mgr.14118 v2:172.21.15.49:6800/2609458951 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f7878103370 con 0x7f7868038f70 2024-03-18T22:58:37.745 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:37.196+0000 7f787eaba700 1 -- 172.21.15.49:0/861720333 >> [v2:172.21.15.49:6800/2609458951,v1:172.21.15.49:6801/2609458951] conn(0x7f7868038f70 msgr2=0x7f786803b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T22:58:37.745 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:37.196+0000 7f787eaba700 1 --2- 172.21.15.49:0/861720333 >> [v2:172.21.15.49:6800/2609458951,v1:172.21.15.49:6801/2609458951] conn(0x7f7868038f70 0x7f786803b430 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f7864006fd0 tx=0x7f7864006e40 comp rx=0 tx=0).stop 2024-03-18T22:58:37.745 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:37.196+0000 7f787eaba700 1 -- 172.21.15.49:0/861720333 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7878108830 msgr2=0x7f787819de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T22:58:37.745 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:37.196+0000 7f787eaba700 1 --2- 172.21.15.49:0/861720333 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7878108830 0x7f787819de80 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f786c011b60 tx=0x7f786c011c40 comp rx=0 tx=0).stop 2024-03-18T22:58:37.745 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:37.196+0000 7f787eaba700 1 -- 172.21.15.49:0/861720333 shutdown_connections 2024-03-18T22:58:37.745 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:37.196+0000 7f787eaba700 1 --2- 172.21.15.49:0/861720333 >> [v2:172.21.15.49:6800/2609458951,v1:172.21.15.49:6801/2609458951] conn(0x7f7868038f70 0x7f786803b430 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T22:58:37.745 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:37.196+0000 7f787eaba700 1 --2- 172.21.15.49:0/861720333 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7878108830 0x7f787819de80 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T22:58:37.745 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:37.196+0000 7f787eaba700 1 -- 172.21.15.49:0/861720333 >> 172.21.15.49:0/861720333 conn(0x7f78781021a0 msgr2=0x7f7878102c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T22:58:37.745 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:37.196+0000 7f787eaba700 1 -- 172.21.15.49:0/861720333 shutdown_connections 2024-03-18T22:58:37.745 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:37.196+0000 7f787eaba700 1 -- 172.21.15.49:0/861720333 wait complete. 2024-03-18T22:58:38.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:38 smithi049 ceph-mon[62991]: from='mgr.14118 172.21.15.49:0/2769660251' entity='mgr.x' 2024-03-18T22:58:38.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:38 smithi049 ceph-mon[62991]: from='mgr.14118 172.21.15.49:0/2769660251' entity='mgr.x' 2024-03-18T22:58:39.023 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDFpin9GyaS7qP32c7VN4lup5pyMnlDKhNZSF3gxOpJLtF0Vxd74+aOsJoSxTomncBbbHzqPkHBIM/nivB+XmivxjKNP1s5gj1xM5Cnky4TIPkZRKcIa07FNyeYsRwCf4pks2dkeroBX7FuBmBKSp6LisGKZoAcVag+M98ahhtkVeRiDsyZia3KbFf89x0+NF6tpiJy2ACfSaRDNlXKZ8ECKawtLfaF7bj0knvGxguSWZ2toyuNiiG3vmsxTQg7K0gfkwTYblwafNw4mDXa9botVcFwFYtSQ48fYUkEblt8sD3KRFtIvbJleO0W1Y/sf6l0O971vu8sCSNNVaI4zrCwWvTe5DuBiuNwemYXdUaT4UR0/JlAVKzGG32TFpdnVFWg8NKyCOSLlj/4GKTsnwTFnHJRxoI/OuaYbVdWuRsLrBqiHuDOBYcsiylw6+lTLtbLEQf4QPfeWvKe0ZEreFsntcUAoBeRtHoU2FoJBTMhBjbtjFopqP26T7sFSRUSw40= ceph-c63591ea-e57a-11ee-95c9-87774f69a715 2024-03-18T22:58:39.023 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:38.110+0000 7f1b5e216700 1 Processor -- start 2024-03-18T22:58:39.023 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:38.110+0000 7f1b5e216700 1 -- start start 2024-03-18T22:58:39.023 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:38.110+0000 7f1b5e216700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1b58108790 0x7f1b5810ab80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T22:58:39.023 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:38.110+0000 7f1b5e216700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f1b5807be80 con 0x7f1b58108790 2024-03-18T22:58:39.023 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:38.110+0000 7f1b577fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1b58108790 0x7f1b5810ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T22:58:39.023 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:38.110+0000 7f1b577fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1b58108790 0x7f1b5810ab80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:59594/0 (socket says 172.21.15.49:59594) 2024-03-18T22:58:39.023 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:38.110+0000 7f1b577fe700 1 -- 172.21.15.49:0/3185606989 learned_addr learned my addr 172.21.15.49:0/3185606989 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T22:58:39.023 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:38.110+0000 7f1b577fe700 1 -- 172.21.15.49:0/3185606989 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b5810b0c0 con 0x7f1b58108790 2024-03-18T22:58:39.024 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:38.111+0000 7f1b577fe700 1 --2- 172.21.15.49:0/3185606989 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1b58108790 0x7f1b5810ab80 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f1b480098a0 tx=0x7f1b48009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8731c8160dc96e94 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T22:58:39.024 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:38.111+0000 7f1b567fc700 1 -- 172.21.15.49:0/3185606989 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1b48004160 con 0x7f1b58108790 2024-03-18T22:58:39.024 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:38.111+0000 7f1b567fc700 1 -- 172.21.15.49:0/3185606989 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f1b48004300 con 0x7f1b58108790 2024-03-18T22:58:39.024 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:38.111+0000 7f1b567fc700 1 -- 172.21.15.49:0/3185606989 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1b48011b50 con 0x7f1b58108790 2024-03-18T22:58:39.024 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:38.111+0000 7f1b5e216700 1 -- 172.21.15.49:0/3185606989 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1b58108790 msgr2=0x7f1b5810ab80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T22:58:39.024 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:38.111+0000 7f1b5e216700 1 --2- 172.21.15.49:0/3185606989 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1b58108790 0x7f1b5810ab80 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f1b480098a0 tx=0x7f1b48009bb0 comp rx=0 tx=0).stop 2024-03-18T22:58:39.024 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:38.111+0000 7f1b5e216700 1 -- 172.21.15.49:0/3185606989 shutdown_connections 2024-03-18T22:58:39.024 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:38.111+0000 7f1b5e216700 1 --2- 172.21.15.49:0/3185606989 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1b58108790 0x7f1b5810ab80 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T22:58:39.024 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:38.111+0000 7f1b5e216700 1 -- 172.21.15.49:0/3185606989 >> 172.21.15.49:0/3185606989 conn(0x7f1b581021e0 msgr2=0x7f1b58104600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T22:58:39.024 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:38.112+0000 7f1b5e216700 1 -- 172.21.15.49:0/3185606989 shutdown_connections 2024-03-18T22:58:39.024 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:38.112+0000 7f1b5e216700 1 -- 172.21.15.49:0/3185606989 wait complete. 2024-03-18T22:58:39.024 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:38.112+0000 7f1b5e216700 1 Processor -- start 2024-03-18T22:58:39.025 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:38.112+0000 7f1b5e216700 1 -- start start 2024-03-18T22:58:39.025 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:38.112+0000 7f1b5e216700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1b58108790 0x7f1b5819bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T22:58:39.025 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:38.113+0000 7f1b5e216700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f1b5810c080 con 0x7f1b58108790 2024-03-18T22:58:39.025 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:38.113+0000 7f1b577fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1b58108790 0x7f1b5819bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T22:58:39.025 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:38.113+0000 7f1b577fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1b58108790 0x7f1b5819bcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:59608/0 (socket says 172.21.15.49:59608) 2024-03-18T22:58:39.025 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:38.113+0000 7f1b577fe700 1 -- 172.21.15.49:0/2029144570 learned_addr learned my addr 172.21.15.49:0/2029144570 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T22:58:39.025 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:38.113+0000 7f1b577fe700 1 -- 172.21.15.49:0/2029144570 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b5819c1f0 con 0x7f1b58108790 2024-03-18T22:58:39.025 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:38.113+0000 7f1b577fe700 1 --2- 172.21.15.49:0/2029144570 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1b58108790 0x7f1b5819bcb0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f1b48009fb0 tx=0x7f1b48011bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T22:58:39.025 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:38.113+0000 7f1b54ff9700 1 -- 172.21.15.49:0/2029144570 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1b480038b0 con 0x7f1b58108790 2024-03-18T22:58:39.025 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:38.113+0000 7f1b54ff9700 1 -- 172.21.15.49:0/2029144570 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f1b48011e50 con 0x7f1b58108790 2024-03-18T22:58:39.025 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:38.113+0000 7f1b5e216700 1 -- 172.21.15.49:0/2029144570 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b5819c480 con 0x7f1b58108790 2024-03-18T22:58:39.025 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:38.113+0000 7f1b54ff9700 1 -- 172.21.15.49:0/2029144570 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1b48019420 con 0x7f1b58108790 2024-03-18T22:58:39.025 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:38.113+0000 7f1b5e216700 1 -- 172.21.15.49:0/2029144570 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b5819f170 con 0x7f1b58108790 2024-03-18T22:58:39.025 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:38.114+0000 7f1b54ff9700 1 -- 172.21.15.49:0/2029144570 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 8) ==== 46106+0+0 (secure 0 0 0) 0x7f1b480196d0 con 0x7f1b58108790 2024-03-18T22:58:39.025 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:38.114+0000 7f1b5e216700 1 -- 172.21.15.49:0/2029144570 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b580693e0 con 0x7f1b58108790 2024-03-18T22:58:39.026 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:38.114+0000 7f1b54ff9700 1 --2- 172.21.15.49:0/2029144570 >> [v2:172.21.15.49:6800/2609458951,v1:172.21.15.49:6801/2609458951] conn(0x7f1b40038f20 0x7f1b4003b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T22:58:39.026 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:38.114+0000 7f1b54ff9700 1 -- 172.21.15.49:0/2029144570 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f1b4804d270 con 0x7f1b58108790 2024-03-18T22:58:39.026 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:38.115+0000 7f1b56ffd700 1 --2- 172.21.15.49:0/2029144570 >> [v2:172.21.15.49:6800/2609458951,v1:172.21.15.49:6801/2609458951] conn(0x7f1b40038f20 0x7f1b4003b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T22:58:39.026 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:38.115+0000 7f1b56ffd700 1 --2- 172.21.15.49:0/2029144570 >> [v2:172.21.15.49:6800/2609458951,v1:172.21.15.49:6801/2609458951] conn(0x7f1b40038f20 0x7f1b4003b3e0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f1b4c006fd0 tx=0x7f1b4c006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T22:58:39.026 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:38.119+0000 7f1b54ff9700 1 -- 172.21.15.49:0/2029144570 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f1b4800bc70 con 0x7f1b58108790 2024-03-18T22:58:39.026 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:38.327+0000 7f1b5e216700 1 -- 172.21.15.49:0/2029144570 --> [v2:172.21.15.49:6800/2609458951,v1:172.21.15.49:6801/2609458951] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7f1b581033e0 con 0x7f1b40038f20 2024-03-18T22:58:39.026 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:38.328+0000 7f1b54ff9700 1 -- 172.21.15.49:0/2029144570 <== mgr.14118 v2:172.21.15.49:6800/2609458951 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7f1b581033e0 con 0x7f1b40038f20 2024-03-18T22:58:39.026 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:38.330+0000 7f1b5e216700 1 -- 172.21.15.49:0/2029144570 >> [v2:172.21.15.49:6800/2609458951,v1:172.21.15.49:6801/2609458951] conn(0x7f1b40038f20 msgr2=0x7f1b4003b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T22:58:39.027 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:38.330+0000 7f1b5e216700 1 --2- 172.21.15.49:0/2029144570 >> [v2:172.21.15.49:6800/2609458951,v1:172.21.15.49:6801/2609458951] conn(0x7f1b40038f20 0x7f1b4003b3e0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f1b4c006fd0 tx=0x7f1b4c006e40 comp rx=0 tx=0).stop 2024-03-18T22:58:39.027 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:38.330+0000 7f1b5e216700 1 -- 172.21.15.49:0/2029144570 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1b58108790 msgr2=0x7f1b5819bcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T22:58:39.027 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:38.330+0000 7f1b5e216700 1 --2- 172.21.15.49:0/2029144570 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1b58108790 0x7f1b5819bcb0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f1b48009fb0 tx=0x7f1b48011bf0 comp rx=0 tx=0).stop 2024-03-18T22:58:39.027 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:38.330+0000 7f1b5e216700 1 -- 172.21.15.49:0/2029144570 shutdown_connections 2024-03-18T22:58:39.027 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:38.330+0000 7f1b5e216700 1 --2- 172.21.15.49:0/2029144570 >> [v2:172.21.15.49:6800/2609458951,v1:172.21.15.49:6801/2609458951] conn(0x7f1b40038f20 0x7f1b4003b3e0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T22:58:39.027 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:38.330+0000 7f1b5e216700 1 --2- 172.21.15.49:0/2029144570 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1b58108790 0x7f1b5819bcb0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T22:58:39.027 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:38.330+0000 7f1b5e216700 1 -- 172.21.15.49:0/2029144570 >> 172.21.15.49:0/2029144570 conn(0x7f1b581021e0 msgr2=0x7f1b58102cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T22:58:39.027 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:38.330+0000 7f1b5e216700 1 -- 172.21.15.49:0/2029144570 shutdown_connections 2024-03-18T22:58:39.027 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:38.331+0000 7f1b5e216700 1 -- 172.21.15.49:0/2029144570 wait complete. 2024-03-18T22:58:39.027 INFO:teuthology.orchestra.run.smithi049.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-03-18T22:58:39.027 INFO:teuthology.orchestra.run.smithi049.stdout:Adding key to root@localhost authorized_keys... 2024-03-18T22:58:39.027 INFO:teuthology.orchestra.run.smithi049.stdout:Adding host smithi049... 2024-03-18T22:58:39.434 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:39 smithi049 ceph-mon[62991]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T22:58:40.291 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:40 smithi049 ceph-mon[62991]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi049", "addr": "172.21.15.49", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T22:58:41.630 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:41 smithi049 ceph-mon[62991]: Deploying cephadm binary to smithi049 2024-03-18T22:58:44.729 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout Added host 'smithi049' with addr '172.21.15.49' 2024-03-18T22:58:44.730 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:39.433+0000 7fc63b73d700 1 Processor -- start 2024-03-18T22:58:44.730 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:39.434+0000 7fc63b73d700 1 -- start start 2024-03-18T22:58:44.730 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:39.434+0000 7fc63b73d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc63407de20 0x7fc63407c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T22:58:44.730 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:39.434+0000 7fc63b73d700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fc63407c8a0 con 0x7fc63407de20 2024-03-18T22:58:44.730 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:39.434+0000 7fc6394d9700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc63407de20 0x7fc63407c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T22:58:44.730 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:39.434+0000 7fc6394d9700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc63407de20 0x7fc63407c2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:59612/0 (socket says 172.21.15.49:59612) 2024-03-18T22:58:44.730 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:39.434+0000 7fc6394d9700 1 -- 172.21.15.49:0/422007981 learned_addr learned my addr 172.21.15.49:0/422007981 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T22:58:44.730 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:39.434+0000 7fc6394d9700 1 -- 172.21.15.49:0/422007981 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc63407ca20 con 0x7fc63407de20 2024-03-18T22:58:44.730 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:39.435+0000 7fc6394d9700 1 --2- 172.21.15.49:0/422007981 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc63407de20 0x7fc63407c2d0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fc6300098a0 tx=0x7fc630009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8813cefff8a92d20 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T22:58:44.730 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:39.435+0000 7fc62bfff700 1 -- 172.21.15.49:0/422007981 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc630004160 con 0x7fc63407de20 2024-03-18T22:58:44.730 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:39.435+0000 7fc62bfff700 1 -- 172.21.15.49:0/422007981 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fc630004300 con 0x7fc63407de20 2024-03-18T22:58:44.730 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:39.435+0000 7fc62bfff700 1 -- 172.21.15.49:0/422007981 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc630011b50 con 0x7fc63407de20 2024-03-18T22:58:44.730 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:39.435+0000 7fc63b73d700 1 -- 172.21.15.49:0/422007981 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc63407de20 msgr2=0x7fc63407c2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T22:58:44.731 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:39.435+0000 7fc63b73d700 1 --2- 172.21.15.49:0/422007981 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc63407de20 0x7fc63407c2d0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fc6300098a0 tx=0x7fc630009bb0 comp rx=0 tx=0).stop 2024-03-18T22:58:44.731 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:39.436+0000 7fc63b73d700 1 -- 172.21.15.49:0/422007981 shutdown_connections 2024-03-18T22:58:44.731 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:39.436+0000 7fc63b73d700 1 --2- 172.21.15.49:0/422007981 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc63407de20 0x7fc63407c2d0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T22:58:44.731 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:39.436+0000 7fc63b73d700 1 -- 172.21.15.49:0/422007981 >> 172.21.15.49:0/422007981 conn(0x7fc6341032b0 msgr2=0x7fc634105710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T22:58:44.731 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:39.436+0000 7fc63b73d700 1 -- 172.21.15.49:0/422007981 shutdown_connections 2024-03-18T22:58:44.731 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:39.436+0000 7fc63b73d700 1 -- 172.21.15.49:0/422007981 wait complete. 2024-03-18T22:58:44.731 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:39.436+0000 7fc63b73d700 1 Processor -- start 2024-03-18T22:58:44.731 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:39.436+0000 7fc63b73d700 1 -- start start 2024-03-18T22:58:44.731 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:39.436+0000 7fc63b73d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc63407de20 0x7fc63419de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T22:58:44.731 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:39.437+0000 7fc63b73d700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fc63410e280 con 0x7fc63407de20 2024-03-18T22:58:44.731 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:39.437+0000 7fc6394d9700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc63407de20 0x7fc63419de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T22:58:44.731 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:39.437+0000 7fc6394d9700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc63407de20 0x7fc63419de60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:59614/0 (socket says 172.21.15.49:59614) 2024-03-18T22:58:44.731 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:39.437+0000 7fc6394d9700 1 -- 172.21.15.49:0/3966339017 learned_addr learned my addr 172.21.15.49:0/3966339017 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T22:58:44.731 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:39.437+0000 7fc6394d9700 1 -- 172.21.15.49:0/3966339017 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc63419e3a0 con 0x7fc63407de20 2024-03-18T22:58:44.732 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:39.437+0000 7fc6394d9700 1 --2- 172.21.15.49:0/3966339017 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc63407de20 0x7fc63419de60 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fc630009fb0 tx=0x7fc630011bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T22:58:44.732 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:39.437+0000 7fc62a7fc700 1 -- 172.21.15.49:0/3966339017 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc6300038b0 con 0x7fc63407de20 2024-03-18T22:58:44.732 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:39.437+0000 7fc62a7fc700 1 -- 172.21.15.49:0/3966339017 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fc630011e50 con 0x7fc63407de20 2024-03-18T22:58:44.732 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:39.437+0000 7fc63b73d700 1 -- 172.21.15.49:0/3966339017 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc63419e630 con 0x7fc63407de20 2024-03-18T22:58:44.732 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:39.437+0000 7fc62a7fc700 1 -- 172.21.15.49:0/3966339017 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc630019420 con 0x7fc63407de20 2024-03-18T22:58:44.732 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:39.437+0000 7fc63b73d700 1 -- 172.21.15.49:0/3966339017 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6341a1320 con 0x7fc63407de20 2024-03-18T22:58:44.732 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:39.438+0000 7fc62a7fc700 1 -- 172.21.15.49:0/3966339017 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 8) ==== 46106+0+0 (secure 0 0 0) 0x7fc630019780 con 0x7fc63407de20 2024-03-18T22:58:44.732 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:39.438+0000 7fc62a7fc700 1 --2- 172.21.15.49:0/3966339017 >> [v2:172.21.15.49:6800/2609458951,v1:172.21.15.49:6801/2609458951] conn(0x7fc620038ed0 0x7fc62003b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T22:58:44.732 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:39.438+0000 7fc63b73d700 1 -- 172.21.15.49:0/3966339017 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc634063940 con 0x7fc63407de20 2024-03-18T22:58:44.732 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:39.438+0000 7fc62a7fc700 1 -- 172.21.15.49:0/3966339017 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fc63004d0e0 con 0x7fc63407de20 2024-03-18T22:58:44.732 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:39.438+0000 7fc638cd8700 1 --2- 172.21.15.49:0/3966339017 >> [v2:172.21.15.49:6800/2609458951,v1:172.21.15.49:6801/2609458951] conn(0x7fc620038ed0 0x7fc62003b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T22:58:44.732 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:39.439+0000 7fc638cd8700 1 --2- 172.21.15.49:0/3966339017 >> [v2:172.21.15.49:6800/2609458951,v1:172.21.15.49:6801/2609458951] conn(0x7fc620038ed0 0x7fc62003b390 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fc624006fd0 tx=0x7fc624006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T22:58:44.732 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:39.444+0000 7fc62a7fc700 1 -- 172.21.15.49:0/3966339017 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fc63002ce10 con 0x7fc63407de20 2024-03-18T22:58:44.733 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:39.653+0000 7fc63b73d700 1 -- 172.21.15.49:0/3966339017 --> [v2:172.21.15.49:6800/2609458951,v1:172.21.15.49:6801/2609458951] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi049", "addr": "172.21.15.49", "target": ["mon-mgr", ""]}) -- 0x7fc634104520 con 0x7fc620038ed0 2024-03-18T22:58:44.733 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:44.143+0000 7fc62a7fc700 1 -- 172.21.15.49:0/3966339017 <== mgr.14118 v2:172.21.15.49:6800/2609458951 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7fc634104520 con 0x7fc620038ed0 2024-03-18T22:58:44.733 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:44.145+0000 7fc63b73d700 1 -- 172.21.15.49:0/3966339017 >> [v2:172.21.15.49:6800/2609458951,v1:172.21.15.49:6801/2609458951] conn(0x7fc620038ed0 msgr2=0x7fc62003b390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T22:58:44.733 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:44.145+0000 7fc63b73d700 1 --2- 172.21.15.49:0/3966339017 >> [v2:172.21.15.49:6800/2609458951,v1:172.21.15.49:6801/2609458951] conn(0x7fc620038ed0 0x7fc62003b390 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fc624006fd0 tx=0x7fc624006e40 comp rx=0 tx=0).stop 2024-03-18T22:58:44.733 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:44.145+0000 7fc63b73d700 1 -- 172.21.15.49:0/3966339017 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc63407de20 msgr2=0x7fc63419de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T22:58:44.733 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:44.145+0000 7fc63b73d700 1 --2- 172.21.15.49:0/3966339017 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc63407de20 0x7fc63419de60 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fc630009fb0 tx=0x7fc630011bf0 comp rx=0 tx=0).stop 2024-03-18T22:58:44.733 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:44.145+0000 7fc63b73d700 1 -- 172.21.15.49:0/3966339017 shutdown_connections 2024-03-18T22:58:44.733 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:44.145+0000 7fc63b73d700 1 --2- 172.21.15.49:0/3966339017 >> [v2:172.21.15.49:6800/2609458951,v1:172.21.15.49:6801/2609458951] conn(0x7fc620038ed0 0x7fc62003b390 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T22:58:44.733 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:44.145+0000 7fc63b73d700 1 --2- 172.21.15.49:0/3966339017 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc63407de20 0x7fc63419de60 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T22:58:44.733 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:44.145+0000 7fc63b73d700 1 -- 172.21.15.49:0/3966339017 >> 172.21.15.49:0/3966339017 conn(0x7fc6341032b0 msgr2=0x7fc634103e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T22:58:44.733 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:44.146+0000 7fc63b73d700 1 -- 172.21.15.49:0/3966339017 shutdown_connections 2024-03-18T22:58:44.733 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:44.146+0000 7fc63b73d700 1 -- 172.21.15.49:0/3966339017 wait complete. 2024-03-18T22:58:44.733 INFO:teuthology.orchestra.run.smithi049.stdout:Deploying unmanaged mon service... 2024-03-18T22:58:45.347 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:45 smithi049 ceph-mon[62991]: from='mgr.14118 172.21.15.49:0/2769660251' entity='mgr.x' 2024-03-18T22:58:45.347 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:45 smithi049 ceph-mon[62991]: Added host smithi049 2024-03-18T22:58:45.347 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:45 smithi049 ceph-mon[62991]: from='mgr.14118 172.21.15.49:0/2769660251' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T22:58:46.055 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-03-18T22:58:46.055 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:45.106+0000 7fb0dc14c700 1 Processor -- start 2024-03-18T22:58:46.055 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:45.106+0000 7fb0dc14c700 1 -- start start 2024-03-18T22:58:46.055 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:45.106+0000 7fb0dc14c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb0d41087c0 0x7fb0d410abb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T22:58:46.055 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:45.106+0000 7fb0dc14c700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fb0d407be40 con 0x7fb0d41087c0 2024-03-18T22:58:46.055 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:45.107+0000 7fb0d9ee8700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb0d41087c0 0x7fb0d410abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T22:58:46.055 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:45.107+0000 7fb0d9ee8700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb0d41087c0 0x7fb0d410abb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:52066/0 (socket says 172.21.15.49:52066) 2024-03-18T22:58:46.056 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:45.107+0000 7fb0d9ee8700 1 -- 172.21.15.49:0/3013966680 learned_addr learned my addr 172.21.15.49:0/3013966680 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T22:58:46.056 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:45.107+0000 7fb0d9ee8700 1 -- 172.21.15.49:0/3013966680 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0d410b0f0 con 0x7fb0d41087c0 2024-03-18T22:58:46.056 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:45.107+0000 7fb0d9ee8700 1 --2- 172.21.15.49:0/3013966680 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb0d41087c0 0x7fb0d410abb0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fb0d00098a0 tx=0x7fb0d0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c20ccf56a5d3df8 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T22:58:46.056 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:45.107+0000 7fb0d8ee6700 1 -- 172.21.15.49:0/3013966680 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb0d0004160 con 0x7fb0d41087c0 2024-03-18T22:58:46.056 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:45.108+0000 7fb0d8ee6700 1 -- 172.21.15.49:0/3013966680 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fb0d0004300 con 0x7fb0d41087c0 2024-03-18T22:58:46.056 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:45.108+0000 7fb0d8ee6700 1 -- 172.21.15.49:0/3013966680 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb0d0011b50 con 0x7fb0d41087c0 2024-03-18T22:58:46.056 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:45.108+0000 7fb0dc14c700 1 -- 172.21.15.49:0/3013966680 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb0d41087c0 msgr2=0x7fb0d410abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T22:58:46.056 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:45.108+0000 7fb0dc14c700 1 --2- 172.21.15.49:0/3013966680 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb0d41087c0 0x7fb0d410abb0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fb0d00098a0 tx=0x7fb0d0009bb0 comp rx=0 tx=0).stop 2024-03-18T22:58:46.056 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:45.108+0000 7fb0dc14c700 1 -- 172.21.15.49:0/3013966680 shutdown_connections 2024-03-18T22:58:46.056 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:45.108+0000 7fb0dc14c700 1 --2- 172.21.15.49:0/3013966680 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb0d41087c0 0x7fb0d410abb0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T22:58:46.056 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:45.108+0000 7fb0dc14c700 1 -- 172.21.15.49:0/3013966680 >> 172.21.15.49:0/3013966680 conn(0x7fb0d41021f0 msgr2=0x7fb0d4104610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T22:58:46.056 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:45.108+0000 7fb0dc14c700 1 -- 172.21.15.49:0/3013966680 shutdown_connections 2024-03-18T22:58:46.056 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:45.108+0000 7fb0dc14c700 1 -- 172.21.15.49:0/3013966680 wait complete. 2024-03-18T22:58:46.056 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:45.109+0000 7fb0dc14c700 1 Processor -- start 2024-03-18T22:58:46.056 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:45.109+0000 7fb0dc14c700 1 -- start start 2024-03-18T22:58:46.057 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:45.109+0000 7fb0dc14c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb0d41087c0 0x7fb0d419bc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T22:58:46.057 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:45.109+0000 7fb0dc14c700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fb0d410c0a0 con 0x7fb0d41087c0 2024-03-18T22:58:46.057 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:45.110+0000 7fb0d9ee8700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb0d41087c0 0x7fb0d419bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T22:58:46.057 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:45.110+0000 7fb0d9ee8700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb0d41087c0 0x7fb0d419bc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:52074/0 (socket says 172.21.15.49:52074) 2024-03-18T22:58:46.057 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:45.110+0000 7fb0d9ee8700 1 -- 172.21.15.49:0/3441804403 learned_addr learned my addr 172.21.15.49:0/3441804403 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T22:58:46.057 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:45.110+0000 7fb0d9ee8700 1 -- 172.21.15.49:0/3441804403 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0d419c170 con 0x7fb0d41087c0 2024-03-18T22:58:46.057 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:45.110+0000 7fb0d9ee8700 1 --2- 172.21.15.49:0/3441804403 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb0d41087c0 0x7fb0d419bc30 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fb0d0011b60 tx=0x7fb0d0011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T22:58:46.057 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:45.110+0000 7fb0caffd700 1 -- 172.21.15.49:0/3441804403 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb0d0003b20 con 0x7fb0d41087c0 2024-03-18T22:58:46.057 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:45.110+0000 7fb0dc14c700 1 -- 172.21.15.49:0/3441804403 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0d419c400 con 0x7fb0d41087c0 2024-03-18T22:58:46.057 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:45.110+0000 7fb0caffd700 1 -- 172.21.15.49:0/3441804403 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fb0d0003cc0 con 0x7fb0d41087c0 2024-03-18T22:58:46.057 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:45.110+0000 7fb0caffd700 1 -- 172.21.15.49:0/3441804403 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb0d0024840 con 0x7fb0d41087c0 2024-03-18T22:58:46.057 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:45.110+0000 7fb0dc14c700 1 -- 172.21.15.49:0/3441804403 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0d419f0f0 con 0x7fb0d41087c0 2024-03-18T22:58:46.057 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:45.111+0000 7fb0dc14c700 1 -- 172.21.15.49:0/3441804403 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0d4063940 con 0x7fb0d41087c0 2024-03-18T22:58:46.057 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:45.111+0000 7fb0caffd700 1 -- 172.21.15.49:0/3441804403 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 8) ==== 46106+0+0 (secure 0 0 0) 0x7fb0d0022020 con 0x7fb0d41087c0 2024-03-18T22:58:46.058 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:45.112+0000 7fb0caffd700 1 --2- 172.21.15.49:0/3441804403 >> [v2:172.21.15.49:6800/2609458951,v1:172.21.15.49:6801/2609458951] conn(0x7fb0c0038f70 0x7fb0c003b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T22:58:46.058 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:45.112+0000 7fb0caffd700 1 -- 172.21.15.49:0/3441804403 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fb0d0011e00 con 0x7fb0d41087c0 2024-03-18T22:58:46.058 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:45.112+0000 7fb0d96e7700 1 --2- 172.21.15.49:0/3441804403 >> [v2:172.21.15.49:6800/2609458951,v1:172.21.15.49:6801/2609458951] conn(0x7fb0c0038f70 0x7fb0c003b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T22:58:46.058 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:45.112+0000 7fb0d96e7700 1 --2- 172.21.15.49:0/3441804403 >> [v2:172.21.15.49:6800/2609458951,v1:172.21.15.49:6801/2609458951] conn(0x7fb0c0038f70 0x7fb0c003b430 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fb0c4006fd0 tx=0x7fb0c4006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T22:58:46.058 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:45.117+0000 7fb0caffd700 1 -- 172.21.15.49:0/3441804403 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fb0d0050030 con 0x7fb0d41087c0 2024-03-18T22:58:46.058 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:45.346+0000 7fb0dc14c700 1 -- 172.21.15.49:0/3441804403 --> [v2:172.21.15.49:6800/2609458951,v1:172.21.15.49:6801/2609458951] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7fb0d41033f0 con 0x7fb0c0038f70 2024-03-18T22:58:46.058 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:45.351+0000 7fb0caffd700 1 -- 172.21.15.49:0/3441804403 <== mgr.14118 v2:172.21.15.49:6800/2609458951 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fb0d41033f0 con 0x7fb0c0038f70 2024-03-18T22:58:46.058 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:45.353+0000 7fb0dc14c700 1 -- 172.21.15.49:0/3441804403 >> [v2:172.21.15.49:6800/2609458951,v1:172.21.15.49:6801/2609458951] conn(0x7fb0c0038f70 msgr2=0x7fb0c003b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T22:58:46.058 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:45.353+0000 7fb0dc14c700 1 --2- 172.21.15.49:0/3441804403 >> [v2:172.21.15.49:6800/2609458951,v1:172.21.15.49:6801/2609458951] conn(0x7fb0c0038f70 0x7fb0c003b430 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fb0c4006fd0 tx=0x7fb0c4006e40 comp rx=0 tx=0).stop 2024-03-18T22:58:46.058 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:45.353+0000 7fb0dc14c700 1 -- 172.21.15.49:0/3441804403 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb0d41087c0 msgr2=0x7fb0d419bc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T22:58:46.058 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:45.353+0000 7fb0dc14c700 1 --2- 172.21.15.49:0/3441804403 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb0d41087c0 0x7fb0d419bc30 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fb0d0011b60 tx=0x7fb0d0011c40 comp rx=0 tx=0).stop 2024-03-18T22:58:46.059 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:45.353+0000 7fb0dc14c700 1 -- 172.21.15.49:0/3441804403 shutdown_connections 2024-03-18T22:58:46.059 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:45.353+0000 7fb0dc14c700 1 --2- 172.21.15.49:0/3441804403 >> [v2:172.21.15.49:6800/2609458951,v1:172.21.15.49:6801/2609458951] conn(0x7fb0c0038f70 0x7fb0c003b430 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T22:58:46.059 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:45.353+0000 7fb0dc14c700 1 --2- 172.21.15.49:0/3441804403 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb0d41087c0 0x7fb0d419bc30 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T22:58:46.059 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:45.353+0000 7fb0dc14c700 1 -- 172.21.15.49:0/3441804403 >> 172.21.15.49:0/3441804403 conn(0x7fb0d41021f0 msgr2=0x7fb0d4102ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T22:58:46.059 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:45.353+0000 7fb0dc14c700 1 -- 172.21.15.49:0/3441804403 shutdown_connections 2024-03-18T22:58:46.059 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:45.353+0000 7fb0dc14c700 1 -- 172.21.15.49:0/3441804403 wait complete. 2024-03-18T22:58:46.059 INFO:teuthology.orchestra.run.smithi049.stdout:Deploying unmanaged mgr service... 2024-03-18T22:58:46.630 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:46 smithi049 ceph-mon[62991]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T22:58:46.630 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:46 smithi049 ceph-mon[62991]: Saving service mon spec with placement count:5 2024-03-18T22:58:46.631 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:46 smithi049 ceph-mon[62991]: from='mgr.14118 172.21.15.49:0/2769660251' entity='mgr.x' 2024-03-18T22:58:48.159 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:48 smithi049 ceph-mon[62991]: from='mgr.14118 172.21.15.49:0/2769660251' entity='mgr.x' 2024-03-18T22:58:48.159 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:48 smithi049 ceph-mon[62991]: from='mgr.14118 172.21.15.49:0/2769660251' entity='mgr.x' 2024-03-18T22:58:48.159 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:48 smithi049 ceph-mon[62991]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T22:58:48.160 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:48 smithi049 ceph-mon[62991]: Saving service mgr spec with placement count:2 2024-03-18T22:58:48.160 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:48 smithi049 ceph-mon[62991]: from='mgr.14118 172.21.15.49:0/2769660251' entity='mgr.x' 2024-03-18T22:58:48.167 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-03-18T22:58:48.168 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:47.351+0000 7fee8af07700 1 Processor -- start 2024-03-18T22:58:48.168 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:47.351+0000 7fee8af07700 1 -- start start 2024-03-18T22:58:48.168 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:47.351+0000 7fee8af07700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fee84106c10 0x7fee84107030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T22:58:48.168 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:47.351+0000 7fee8af07700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fee84107600 con 0x7fee84106c10 2024-03-18T22:58:48.168 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:47.351+0000 7fee88ca3700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fee84106c10 0x7fee84107030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T22:58:48.168 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:47.351+0000 7fee88ca3700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fee84106c10 0x7fee84107030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:52086/0 (socket says 172.21.15.49:52086) 2024-03-18T22:58:48.168 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:47.351+0000 7fee88ca3700 1 -- 172.21.15.49:0/698741950 learned_addr learned my addr 172.21.15.49:0/698741950 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T22:58:48.168 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:47.351+0000 7fee88ca3700 1 -- 172.21.15.49:0/698741950 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee84107e10 con 0x7fee84106c10 2024-03-18T22:58:48.168 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:47.352+0000 7fee88ca3700 1 --2- 172.21.15.49:0/698741950 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fee84106c10 0x7fee84107030 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fee780098a0 tx=0x7fee78009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb7e7cb7ec670ff server_cookie=0 in_seq=0 out_seq=0 2024-03-18T22:58:48.168 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:47.352+0000 7fee837fe700 1 -- 172.21.15.49:0/698741950 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fee78004160 con 0x7fee84106c10 2024-03-18T22:58:48.168 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:47.352+0000 7fee837fe700 1 -- 172.21.15.49:0/698741950 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fee78004300 con 0x7fee84106c10 2024-03-18T22:58:48.168 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:47.352+0000 7fee837fe700 1 -- 172.21.15.49:0/698741950 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fee78011b20 con 0x7fee84106c10 2024-03-18T22:58:48.169 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:47.353+0000 7fee8af07700 1 -- 172.21.15.49:0/698741950 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fee84106c10 msgr2=0x7fee84107030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T22:58:48.169 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:47.353+0000 7fee8af07700 1 --2- 172.21.15.49:0/698741950 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fee84106c10 0x7fee84107030 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fee780098a0 tx=0x7fee78009bb0 comp rx=0 tx=0).stop 2024-03-18T22:58:48.169 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:47.353+0000 7fee8af07700 1 -- 172.21.15.49:0/698741950 shutdown_connections 2024-03-18T22:58:48.169 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:47.353+0000 7fee8af07700 1 --2- 172.21.15.49:0/698741950 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fee84106c10 0x7fee84107030 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T22:58:48.169 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:47.353+0000 7fee8af07700 1 -- 172.21.15.49:0/698741950 >> 172.21.15.49:0/698741950 conn(0x7fee841021f0 msgr2=0x7fee84104610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T22:58:48.169 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:47.353+0000 7fee8af07700 1 -- 172.21.15.49:0/698741950 shutdown_connections 2024-03-18T22:58:48.169 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:47.353+0000 7fee8af07700 1 -- 172.21.15.49:0/698741950 wait complete. 2024-03-18T22:58:48.169 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:47.353+0000 7fee8af07700 1 Processor -- start 2024-03-18T22:58:48.169 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:47.353+0000 7fee8af07700 1 -- start start 2024-03-18T22:58:48.169 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:47.354+0000 7fee8af07700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fee84106c10 0x7fee84191d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T22:58:48.169 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:47.354+0000 7fee8af07700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fee8410b1c0 con 0x7fee84106c10 2024-03-18T22:58:48.169 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:47.354+0000 7fee88ca3700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fee84106c10 0x7fee84191d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T22:58:48.169 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:47.354+0000 7fee88ca3700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fee84106c10 0x7fee84191d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:52102/0 (socket says 172.21.15.49:52102) 2024-03-18T22:58:48.169 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:47.354+0000 7fee88ca3700 1 -- 172.21.15.49:0/1990408033 learned_addr learned my addr 172.21.15.49:0/1990408033 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T22:58:48.169 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:47.354+0000 7fee88ca3700 1 -- 172.21.15.49:0/1990408033 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee841922c0 con 0x7fee84106c10 2024-03-18T22:58:48.170 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:47.354+0000 7fee88ca3700 1 --2- 172.21.15.49:0/1990408033 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fee84106c10 0x7fee84191d80 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fee780117e0 tx=0x7fee78011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T22:58:48.170 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:47.354+0000 7fee81ffb700 1 -- 172.21.15.49:0/1990408033 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fee78003bf0 con 0x7fee84106c10 2024-03-18T22:58:48.170 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:47.354+0000 7fee81ffb700 1 -- 172.21.15.49:0/1990408033 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fee7801a910 con 0x7fee84106c10 2024-03-18T22:58:48.170 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:47.354+0000 7fee8af07700 1 -- 172.21.15.49:0/1990408033 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee841904b0 con 0x7fee84106c10 2024-03-18T22:58:48.170 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:47.354+0000 7fee81ffb700 1 -- 172.21.15.49:0/1990408033 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fee78024970 con 0x7fee84106c10 2024-03-18T22:58:48.170 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:47.355+0000 7fee8af07700 1 -- 172.21.15.49:0/1990408033 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee84190970 con 0x7fee84106c10 2024-03-18T22:58:48.170 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:47.356+0000 7fee81ffb700 1 -- 172.21.15.49:0/1990408033 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 8) ==== 46106+0+0 (secure 0 0 0) 0x7fee78022020 con 0x7fee84106c10 2024-03-18T22:58:48.170 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:47.356+0000 7fee8af07700 1 -- 172.21.15.49:0/1990408033 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee84063940 con 0x7fee84106c10 2024-03-18T22:58:48.170 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:47.356+0000 7fee81ffb700 1 --2- 172.21.15.49:0/1990408033 >> [v2:172.21.15.49:6800/2609458951,v1:172.21.15.49:6801/2609458951] conn(0x7fee74038f20 0x7fee7403b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T22:58:48.170 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:47.356+0000 7fee81ffb700 1 -- 172.21.15.49:0/1990408033 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fee7804d820 con 0x7fee84106c10 2024-03-18T22:58:48.170 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:47.356+0000 7fee83fff700 1 --2- 172.21.15.49:0/1990408033 >> [v2:172.21.15.49:6800/2609458951,v1:172.21.15.49:6801/2609458951] conn(0x7fee74038f20 0x7fee7403b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T22:58:48.170 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:47.356+0000 7fee83fff700 1 --2- 172.21.15.49:0/1990408033 >> [v2:172.21.15.49:6800/2609458951,v1:172.21.15.49:6801/2609458951] conn(0x7fee74038f20 0x7fee7403b3e0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fee70006fd0 tx=0x7fee70006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T22:58:48.170 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:47.361+0000 7fee81ffb700 1 -- 172.21.15.49:0/1990408033 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fee7802cdf0 con 0x7fee84106c10 2024-03-18T22:58:48.171 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:47.582+0000 7fee8af07700 1 -- 172.21.15.49:0/1990408033 --> [v2:172.21.15.49:6800/2609458951,v1:172.21.15.49:6801/2609458951] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7fee84191280 con 0x7fee74038f20 2024-03-18T22:58:48.171 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:47.587+0000 7fee81ffb700 1 -- 172.21.15.49:0/1990408033 <== mgr.14118 v2:172.21.15.49:6800/2609458951 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fee84191280 con 0x7fee74038f20 2024-03-18T22:58:48.171 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:47.589+0000 7fee8af07700 1 -- 172.21.15.49:0/1990408033 >> [v2:172.21.15.49:6800/2609458951,v1:172.21.15.49:6801/2609458951] conn(0x7fee74038f20 msgr2=0x7fee7403b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T22:58:48.171 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:47.589+0000 7fee8af07700 1 --2- 172.21.15.49:0/1990408033 >> [v2:172.21.15.49:6800/2609458951,v1:172.21.15.49:6801/2609458951] conn(0x7fee74038f20 0x7fee7403b3e0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fee70006fd0 tx=0x7fee70006e40 comp rx=0 tx=0).stop 2024-03-18T22:58:48.171 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:47.589+0000 7fee8af07700 1 -- 172.21.15.49:0/1990408033 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fee84106c10 msgr2=0x7fee84191d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T22:58:48.171 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:47.589+0000 7fee8af07700 1 --2- 172.21.15.49:0/1990408033 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fee84106c10 0x7fee84191d80 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fee780117e0 tx=0x7fee78011ba0 comp rx=0 tx=0).stop 2024-03-18T22:58:48.171 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:47.589+0000 7fee8af07700 1 -- 172.21.15.49:0/1990408033 shutdown_connections 2024-03-18T22:58:48.171 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:47.589+0000 7fee8af07700 1 --2- 172.21.15.49:0/1990408033 >> [v2:172.21.15.49:6800/2609458951,v1:172.21.15.49:6801/2609458951] conn(0x7fee74038f20 0x7fee7403b3e0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T22:58:48.171 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:47.589+0000 7fee8af07700 1 --2- 172.21.15.49:0/1990408033 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fee84106c10 0x7fee84191d80 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T22:58:48.171 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:47.589+0000 7fee8af07700 1 -- 172.21.15.49:0/1990408033 >> 172.21.15.49:0/1990408033 conn(0x7fee841021f0 msgr2=0x7fee84102c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T22:58:48.171 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:47.589+0000 7fee8af07700 1 -- 172.21.15.49:0/1990408033 shutdown_connections 2024-03-18T22:58:48.171 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:47.589+0000 7fee8af07700 1 -- 172.21.15.49:0/1990408033 wait complete. 2024-03-18T22:58:49.801 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:48.575+0000 7f4263e16700 1 Processor -- start 2024-03-18T22:58:49.801 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:48.575+0000 7f4263e16700 1 -- start start 2024-03-18T22:58:49.801 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:48.575+0000 7f4263e16700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f425c07de00 0x7f425c07c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T22:58:49.801 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:48.575+0000 7f4263e16700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f425c07c7f0 con 0x7f425c07de00 2024-03-18T22:58:49.801 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:48.575+0000 7f4261bb2700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f425c07de00 0x7f425c07c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T22:58:49.801 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:48.575+0000 7f4261bb2700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f425c07de00 0x7f425c07c2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:52106/0 (socket says 172.21.15.49:52106) 2024-03-18T22:58:49.801 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:48.575+0000 7f4261bb2700 1 -- 172.21.15.49:0/2120044901 learned_addr learned my addr 172.21.15.49:0/2120044901 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T22:58:49.801 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:48.575+0000 7f4261bb2700 1 -- 172.21.15.49:0/2120044901 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f425c07c970 con 0x7f425c07de00 2024-03-18T22:58:49.801 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:48.576+0000 7f4261bb2700 1 --2- 172.21.15.49:0/2120044901 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f425c07de00 0x7f425c07c2b0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f42580098a0 tx=0x7f4258009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20f00feedd8c4b86 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T22:58:49.801 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:48.576+0000 7f4260bb0700 1 -- 172.21.15.49:0/2120044901 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4258004160 con 0x7f425c07de00 2024-03-18T22:58:49.802 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:48.576+0000 7f4260bb0700 1 -- 172.21.15.49:0/2120044901 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4258004300 con 0x7f425c07de00 2024-03-18T22:58:49.802 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:48.576+0000 7f4260bb0700 1 -- 172.21.15.49:0/2120044901 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4258011b20 con 0x7f425c07de00 2024-03-18T22:58:49.802 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:48.576+0000 7f4263e16700 1 -- 172.21.15.49:0/2120044901 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f425c07de00 msgr2=0x7f425c07c2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T22:58:49.802 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:48.576+0000 7f4263e16700 1 --2- 172.21.15.49:0/2120044901 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f425c07de00 0x7f425c07c2b0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f42580098a0 tx=0x7f4258009bb0 comp rx=0 tx=0).stop 2024-03-18T22:58:49.802 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:48.576+0000 7f4263e16700 1 -- 172.21.15.49:0/2120044901 shutdown_connections 2024-03-18T22:58:49.802 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:48.576+0000 7f4263e16700 1 --2- 172.21.15.49:0/2120044901 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f425c07de00 0x7f425c07c2b0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T22:58:49.802 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:48.576+0000 7f4263e16700 1 -- 172.21.15.49:0/2120044901 >> 172.21.15.49:0/2120044901 conn(0x7f425c1032f0 msgr2=0x7f425c105710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T22:58:49.802 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:48.577+0000 7f4263e16700 1 -- 172.21.15.49:0/2120044901 shutdown_connections 2024-03-18T22:58:49.802 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:48.577+0000 7f4263e16700 1 -- 172.21.15.49:0/2120044901 wait complete. 2024-03-18T22:58:49.802 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:48.577+0000 7f4263e16700 1 Processor -- start 2024-03-18T22:58:49.802 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:48.577+0000 7f4263e16700 1 -- start start 2024-03-18T22:58:49.802 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:48.577+0000 7f4263e16700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f425c07de00 0x7f425c19dd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T22:58:49.802 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:48.577+0000 7f4263e16700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f425c10e2d0 con 0x7f425c07de00 2024-03-18T22:58:49.802 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:48.577+0000 7f4261bb2700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f425c07de00 0x7f425c19dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T22:58:49.803 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:48.578+0000 7f4261bb2700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f425c07de00 0x7f425c19dd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:52108/0 (socket says 172.21.15.49:52108) 2024-03-18T22:58:49.803 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:48.578+0000 7f4261bb2700 1 -- 172.21.15.49:0/2622563639 learned_addr learned my addr 172.21.15.49:0/2622563639 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T22:58:49.803 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:48.578+0000 7f4261bb2700 1 -- 172.21.15.49:0/2622563639 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f425c19e290 con 0x7f425c07de00 2024-03-18T22:58:49.803 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:48.578+0000 7f4261bb2700 1 --2- 172.21.15.49:0/2622563639 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f425c07de00 0x7f425c19dd50 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f42580117e0 tx=0x7f4258011ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T22:58:49.803 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:48.578+0000 7f4252ffd700 1 -- 172.21.15.49:0/2622563639 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f425801a430 con 0x7f425c07de00 2024-03-18T22:58:49.803 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:48.578+0000 7f4252ffd700 1 -- 172.21.15.49:0/2622563639 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f425801aab0 con 0x7f425c07de00 2024-03-18T22:58:49.803 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:48.578+0000 7f4263e16700 1 -- 172.21.15.49:0/2622563639 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f425c19e520 con 0x7f425c07de00 2024-03-18T22:58:49.803 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:48.578+0000 7f4252ffd700 1 -- 172.21.15.49:0/2622563639 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4258024a60 con 0x7f425c07de00 2024-03-18T22:58:49.803 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:48.578+0000 7f4263e16700 1 -- 172.21.15.49:0/2622563639 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f425c19e980 con 0x7f425c07de00 2024-03-18T22:58:49.803 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:48.579+0000 7f4252ffd700 1 -- 172.21.15.49:0/2622563639 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 8) ==== 46106+0+0 (secure 0 0 0) 0x7f4258022020 con 0x7f425c07de00 2024-03-18T22:58:49.803 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:48.579+0000 7f4263e16700 1 -- 172.21.15.49:0/2622563639 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f425c0693e0 con 0x7f425c07de00 2024-03-18T22:58:49.803 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:48.579+0000 7f4252ffd700 1 --2- 172.21.15.49:0/2622563639 >> [v2:172.21.15.49:6800/2609458951,v1:172.21.15.49:6801/2609458951] conn(0x7f4248038f70 0x7f424803b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T22:58:49.803 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:48.579+0000 7f4252ffd700 1 -- 172.21.15.49:0/2622563639 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f425804d610 con 0x7f425c07de00 2024-03-18T22:58:49.803 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:48.579+0000 7f42613b1700 1 --2- 172.21.15.49:0/2622563639 >> [v2:172.21.15.49:6800/2609458951,v1:172.21.15.49:6801/2609458951] conn(0x7f4248038f70 0x7f424803b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T22:58:49.804 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:48.580+0000 7f42613b1700 1 --2- 172.21.15.49:0/2622563639 >> [v2:172.21.15.49:6800/2609458951,v1:172.21.15.49:6801/2609458951] conn(0x7f4248038f70 0x7f424803b430 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f424c006fd0 tx=0x7f424c006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T22:58:49.804 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:48.584+0000 7f4252ffd700 1 -- 172.21.15.49:0/2622563639 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f425802c3a0 con 0x7f425c07de00 2024-03-18T22:58:49.804 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:48.795+0000 7f4263e16700 1 -- 172.21.15.49:0/2622563639 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7f425c1a12d0 con 0x7f425c07de00 2024-03-18T22:58:49.804 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:48.797+0000 7f4252ffd700 1 -- 172.21.15.49:0/2622563639 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) ==== 142+0+0 (secure 0 0 0) 0x7f42580042a0 con 0x7f425c07de00 2024-03-18T22:58:49.804 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:48.800+0000 7f4263e16700 1 -- 172.21.15.49:0/2622563639 >> [v2:172.21.15.49:6800/2609458951,v1:172.21.15.49:6801/2609458951] conn(0x7f4248038f70 msgr2=0x7f424803b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T22:58:49.804 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:48.800+0000 7f4263e16700 1 --2- 172.21.15.49:0/2622563639 >> [v2:172.21.15.49:6800/2609458951,v1:172.21.15.49:6801/2609458951] conn(0x7f4248038f70 0x7f424803b430 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f424c006fd0 tx=0x7f424c006e40 comp rx=0 tx=0).stop 2024-03-18T22:58:49.804 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:48.800+0000 7f4263e16700 1 -- 172.21.15.49:0/2622563639 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f425c07de00 msgr2=0x7f425c19dd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T22:58:49.804 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:48.800+0000 7f4263e16700 1 --2- 172.21.15.49:0/2622563639 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f425c07de00 0x7f425c19dd50 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f42580117e0 tx=0x7f4258011ce0 comp rx=0 tx=0).stop 2024-03-18T22:58:49.804 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:48.800+0000 7f4263e16700 1 -- 172.21.15.49:0/2622563639 shutdown_connections 2024-03-18T22:58:49.804 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:48.800+0000 7f4263e16700 1 --2- 172.21.15.49:0/2622563639 >> [v2:172.21.15.49:6800/2609458951,v1:172.21.15.49:6801/2609458951] conn(0x7f4248038f70 0x7f424803b430 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T22:58:49.804 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:48.800+0000 7f4263e16700 1 --2- 172.21.15.49:0/2622563639 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f425c07de00 0x7f425c19dd50 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T22:58:49.804 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:48.800+0000 7f4263e16700 1 -- 172.21.15.49:0/2622563639 >> 172.21.15.49:0/2622563639 conn(0x7f425c1032f0 msgr2=0x7f425c103cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T22:58:49.804 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:48.800+0000 7f4263e16700 1 -- 172.21.15.49:0/2622563639 shutdown_connections 2024-03-18T22:58:49.804 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:48.800+0000 7f4263e16700 1 -- 172.21.15.49:0/2622563639 wait complete. 2024-03-18T22:58:50.046 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:49 smithi049 ceph-mon[62991]: from='client.? 172.21.15.49:0/2622563639' entity='client.admin' 2024-03-18T22:58:50.047 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:49 smithi049 ceph-mon[62991]: from='mgr.14118 172.21.15.49:0/2769660251' entity='mgr.x' 2024-03-18T22:58:51.086 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:50.272+0000 7ff1462f6700 1 Processor -- start 2024-03-18T22:58:51.086 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:50.272+0000 7ff1462f6700 1 -- start start 2024-03-18T22:58:51.086 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:50.272+0000 7ff1462f6700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff140107e10 0x7ff140108230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T22:58:51.089 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:50.272+0000 7ff1462f6700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7ff140108800 con 0x7ff140107e10 2024-03-18T22:58:51.089 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:50.273+0000 7ff13ffff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff140107e10 0x7ff140108230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T22:58:51.089 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:50.273+0000 7ff13ffff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff140107e10 0x7ff140108230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:52116/0 (socket says 172.21.15.49:52116) 2024-03-18T22:58:51.089 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:50.273+0000 7ff13ffff700 1 -- 172.21.15.49:0/3691824591 learned_addr learned my addr 172.21.15.49:0/3691824591 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T22:58:51.089 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:50.273+0000 7ff13ffff700 1 -- 172.21.15.49:0/3691824591 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff140108f70 con 0x7ff140107e10 2024-03-18T22:58:51.089 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:50.273+0000 7ff13ffff700 1 --2- 172.21.15.49:0/3691824591 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff140107e10 0x7ff140108230 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7ff1300098a0 tx=0x7ff130009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fa816cd4a437aae server_cookie=0 in_seq=0 out_seq=0 2024-03-18T22:58:51.089 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:50.273+0000 7ff13effd700 1 -- 172.21.15.49:0/3691824591 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff130004160 con 0x7ff140107e10 2024-03-18T22:58:51.093 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:50.273+0000 7ff13effd700 1 -- 172.21.15.49:0/3691824591 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff130004300 con 0x7ff140107e10 2024-03-18T22:58:51.094 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:50.273+0000 7ff13effd700 1 -- 172.21.15.49:0/3691824591 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff130011b20 con 0x7ff140107e10 2024-03-18T22:58:51.094 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:50.274+0000 7ff1462f6700 1 -- 172.21.15.49:0/3691824591 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff140107e10 msgr2=0x7ff140108230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T22:58:51.094 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:50.274+0000 7ff1462f6700 1 --2- 172.21.15.49:0/3691824591 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff140107e10 0x7ff140108230 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7ff1300098a0 tx=0x7ff130009bb0 comp rx=0 tx=0).stop 2024-03-18T22:58:51.094 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:50.274+0000 7ff1462f6700 1 -- 172.21.15.49:0/3691824591 shutdown_connections 2024-03-18T22:58:51.094 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:50.274+0000 7ff1462f6700 1 --2- 172.21.15.49:0/3691824591 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff140107e10 0x7ff140108230 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T22:58:51.094 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:50.274+0000 7ff1462f6700 1 -- 172.21.15.49:0/3691824591 >> 172.21.15.49:0/3691824591 conn(0x7ff1401033f0 msgr2=0x7ff140105810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T22:58:51.094 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:50.274+0000 7ff1462f6700 1 -- 172.21.15.49:0/3691824591 shutdown_connections 2024-03-18T22:58:51.094 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:50.274+0000 7ff1462f6700 1 -- 172.21.15.49:0/3691824591 wait complete. 2024-03-18T22:58:51.094 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:50.275+0000 7ff1462f6700 1 Processor -- start 2024-03-18T22:58:51.094 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:50.275+0000 7ff1462f6700 1 -- start start 2024-03-18T22:58:51.094 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:50.275+0000 7ff1462f6700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff140107e10 0x7ff14019de40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T22:58:51.095 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:50.275+0000 7ff1462f6700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7ff14010c320 con 0x7ff140107e10 2024-03-18T22:58:51.095 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:50.275+0000 7ff13ffff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff140107e10 0x7ff14019de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T22:58:51.095 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:50.275+0000 7ff13ffff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff140107e10 0x7ff14019de40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:52122/0 (socket says 172.21.15.49:52122) 2024-03-18T22:58:51.095 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:50.275+0000 7ff13ffff700 1 -- 172.21.15.49:0/555002600 learned_addr learned my addr 172.21.15.49:0/555002600 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T22:58:51.095 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:50.276+0000 7ff13ffff700 1 -- 172.21.15.49:0/555002600 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff14019e380 con 0x7ff140107e10 2024-03-18T22:58:51.095 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:50.276+0000 7ff13ffff700 1 --2- 172.21.15.49:0/555002600 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff140107e10 0x7ff14019de40 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7ff1300116a0 tx=0x7ff130011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T22:58:51.095 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:50.276+0000 7ff13d7fa700 1 -- 172.21.15.49:0/555002600 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff130003a40 con 0x7ff140107e10 2024-03-18T22:58:51.095 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:50.276+0000 7ff13d7fa700 1 -- 172.21.15.49:0/555002600 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff130003be0 con 0x7ff140107e10 2024-03-18T22:58:51.095 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:50.276+0000 7ff1462f6700 1 -- 172.21.15.49:0/555002600 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff14019e610 con 0x7ff140107e10 2024-03-18T22:58:51.095 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:50.277+0000 7ff13d7fa700 1 -- 172.21.15.49:0/555002600 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff13001a720 con 0x7ff140107e10 2024-03-18T22:58:51.095 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:50.277+0000 7ff1462f6700 1 -- 172.21.15.49:0/555002600 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff14019ea70 con 0x7ff140107e10 2024-03-18T22:58:51.095 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:50.277+0000 7ff13d7fa700 1 -- 172.21.15.49:0/555002600 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 8) ==== 46106+0+0 (secure 0 0 0) 0x7ff13001aa50 con 0x7ff140107e10 2024-03-18T22:58:51.095 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:50.277+0000 7ff13d7fa700 1 --2- 172.21.15.49:0/555002600 >> [v2:172.21.15.49:6800/2609458951,v1:172.21.15.49:6801/2609458951] conn(0x7ff128038f20 0x7ff12803b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T22:58:51.096 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:50.278+0000 7ff13d7fa700 1 -- 172.21.15.49:0/555002600 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7ff130024aa0 con 0x7ff140107e10 2024-03-18T22:58:51.096 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:50.278+0000 7ff13f7fe700 1 --2- 172.21.15.49:0/555002600 >> [v2:172.21.15.49:6800/2609458951,v1:172.21.15.49:6801/2609458951] conn(0x7ff128038f20 0x7ff12803b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T22:58:51.096 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:50.278+0000 7ff1462f6700 1 -- 172.21.15.49:0/555002600 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff14010c4a0 con 0x7ff140107e10 2024-03-18T22:58:51.096 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:50.278+0000 7ff13f7fe700 1 --2- 172.21.15.49:0/555002600 >> [v2:172.21.15.49:6800/2609458951,v1:172.21.15.49:6801/2609458951] conn(0x7ff128038f20 0x7ff12803b3e0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7ff134006fd0 tx=0x7ff134006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T22:58:51.096 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:50.284+0000 7ff13d7fa700 1 -- 172.21.15.49:0/555002600 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7ff130011e20 con 0x7ff140107e10 2024-03-18T22:58:51.096 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:50.489+0000 7ff1462f6700 1 -- 172.21.15.49:0/555002600 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7ff140063940 con 0x7ff140107e10 2024-03-18T22:58:51.096 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:50.491+0000 7ff13d7fa700 1 -- 172.21.15.49:0/555002600 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7ff13000bc70 con 0x7ff140107e10 2024-03-18T22:58:51.096 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:50.493+0000 7ff1462f6700 1 -- 172.21.15.49:0/555002600 >> [v2:172.21.15.49:6800/2609458951,v1:172.21.15.49:6801/2609458951] conn(0x7ff128038f20 msgr2=0x7ff12803b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T22:58:51.096 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:50.493+0000 7ff1462f6700 1 --2- 172.21.15.49:0/555002600 >> [v2:172.21.15.49:6800/2609458951,v1:172.21.15.49:6801/2609458951] conn(0x7ff128038f20 0x7ff12803b3e0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7ff134006fd0 tx=0x7ff134006e40 comp rx=0 tx=0).stop 2024-03-18T22:58:51.096 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:50.494+0000 7ff1462f6700 1 -- 172.21.15.49:0/555002600 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff140107e10 msgr2=0x7ff14019de40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T22:58:51.096 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:50.494+0000 7ff1462f6700 1 --2- 172.21.15.49:0/555002600 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff140107e10 0x7ff14019de40 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7ff1300116a0 tx=0x7ff130011ba0 comp rx=0 tx=0).stop 2024-03-18T22:58:51.096 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:50.494+0000 7ff1462f6700 1 -- 172.21.15.49:0/555002600 shutdown_connections 2024-03-18T22:58:51.096 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:50.494+0000 7ff1462f6700 1 --2- 172.21.15.49:0/555002600 >> [v2:172.21.15.49:6800/2609458951,v1:172.21.15.49:6801/2609458951] conn(0x7ff128038f20 0x7ff12803b3e0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T22:58:51.096 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:50.494+0000 7ff1462f6700 1 --2- 172.21.15.49:0/555002600 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff140107e10 0x7ff14019de40 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T22:58:51.097 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:50.494+0000 7ff1462f6700 1 -- 172.21.15.49:0/555002600 >> 172.21.15.49:0/555002600 conn(0x7ff1401033f0 msgr2=0x7ff14007c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T22:58:51.097 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:50.494+0000 7ff1462f6700 1 -- 172.21.15.49:0/555002600 shutdown_connections 2024-03-18T22:58:51.097 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:50.494+0000 7ff1462f6700 1 -- 172.21.15.49:0/555002600 wait complete. 2024-03-18T22:58:51.097 INFO:teuthology.orchestra.run.smithi049.stdout:Enabling the dashboard module... 2024-03-18T22:58:51.822 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:51 smithi049 ceph-mon[62991]: from='client.? 172.21.15.49:0/555002600' entity='client.admin' 2024-03-18T22:58:52.881 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:52 smithi049 ceph-mon[62991]: from='mgr.14118 172.21.15.49:0/2769660251' entity='mgr.x' 2024-03-18T22:58:52.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:52 smithi049 ceph-mon[62991]: from='client.? 172.21.15.49:0/223475254' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-03-18T22:58:52.882 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 18 22:58:52 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-x[63224]: ignoring --setuser ceph since I am not root 2024-03-18T22:58:52.882 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 18 22:58:52 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-x[63224]: ignoring --setgroup ceph since I am not root 2024-03-18T22:58:53.244 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 18 22:58:53 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-x[63224]: 2024-03-18T22:58:53.226+0000 7f18f103f200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-18T22:58:53.254 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:51.541+0000 7ffb3ffab700 1 Processor -- start 2024-03-18T22:58:53.255 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:51.542+0000 7ffb3ffab700 1 -- start start 2024-03-18T22:58:53.255 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:51.542+0000 7ffb3ffab700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffb3807de40 0x7ffb3807c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T22:58:53.255 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:51.542+0000 7ffb3ffab700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7ffb3807c8c0 con 0x7ffb3807de40 2024-03-18T22:58:53.255 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:51.542+0000 7ffb3dd47700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffb3807de40 0x7ffb3807c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T22:58:53.255 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:51.542+0000 7ffb3dd47700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffb3807de40 0x7ffb3807c2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:52124/0 (socket says 172.21.15.49:52124) 2024-03-18T22:58:53.255 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:51.542+0000 7ffb3dd47700 1 -- 172.21.15.49:0/2548441092 learned_addr learned my addr 172.21.15.49:0/2548441092 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T22:58:53.255 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:51.542+0000 7ffb3dd47700 1 -- 172.21.15.49:0/2548441092 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb3807ca40 con 0x7ffb3807de40 2024-03-18T22:58:53.255 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:51.542+0000 7ffb3dd47700 1 --2- 172.21.15.49:0/2548441092 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffb3807de40 0x7ffb3807c2f0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7ffb2c0098a0 tx=0x7ffb2c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e79a87f80642f564 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T22:58:53.255 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:51.543+0000 7ffb3cd45700 1 -- 172.21.15.49:0/2548441092 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffb2c004160 con 0x7ffb3807de40 2024-03-18T22:58:53.255 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:51.543+0000 7ffb3cd45700 1 -- 172.21.15.49:0/2548441092 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffb2c004300 con 0x7ffb3807de40 2024-03-18T22:58:53.256 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:51.543+0000 7ffb3cd45700 1 -- 172.21.15.49:0/2548441092 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffb2c011b20 con 0x7ffb3807de40 2024-03-18T22:58:53.256 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:51.543+0000 7ffb3ffab700 1 -- 172.21.15.49:0/2548441092 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffb3807de40 msgr2=0x7ffb3807c2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T22:58:53.256 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:51.543+0000 7ffb3ffab700 1 --2- 172.21.15.49:0/2548441092 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffb3807de40 0x7ffb3807c2f0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7ffb2c0098a0 tx=0x7ffb2c009bb0 comp rx=0 tx=0).stop 2024-03-18T22:58:53.256 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:51.543+0000 7ffb3ffab700 1 -- 172.21.15.49:0/2548441092 shutdown_connections 2024-03-18T22:58:53.256 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:51.543+0000 7ffb3ffab700 1 --2- 172.21.15.49:0/2548441092 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffb3807de40 0x7ffb3807c2f0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T22:58:53.256 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:51.543+0000 7ffb3ffab700 1 -- 172.21.15.49:0/2548441092 >> 172.21.15.49:0/2548441092 conn(0x7ffb381032b0 msgr2=0x7ffb38105710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T22:58:53.256 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:51.543+0000 7ffb3ffab700 1 -- 172.21.15.49:0/2548441092 shutdown_connections 2024-03-18T22:58:53.256 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:51.543+0000 7ffb3ffab700 1 -- 172.21.15.49:0/2548441092 wait complete. 2024-03-18T22:58:53.256 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:51.544+0000 7ffb3ffab700 1 Processor -- start 2024-03-18T22:58:53.256 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:51.544+0000 7ffb3ffab700 1 -- start start 2024-03-18T22:58:53.256 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:51.544+0000 7ffb3ffab700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffb3807de40 0x7ffb3819de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T22:58:53.256 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:51.544+0000 7ffb3ffab700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7ffb3810e310 con 0x7ffb3807de40 2024-03-18T22:58:53.257 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:51.544+0000 7ffb3dd47700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffb3807de40 0x7ffb3819de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T22:58:53.257 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:51.544+0000 7ffb3dd47700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffb3807de40 0x7ffb3819de30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:52130/0 (socket says 172.21.15.49:52130) 2024-03-18T22:58:53.257 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:51.544+0000 7ffb3dd47700 1 -- 172.21.15.49:0/223475254 learned_addr learned my addr 172.21.15.49:0/223475254 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T22:58:53.257 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:51.544+0000 7ffb3dd47700 1 -- 172.21.15.49:0/223475254 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb3819e370 con 0x7ffb3807de40 2024-03-18T22:58:53.257 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:51.545+0000 7ffb3dd47700 1 --2- 172.21.15.49:0/223475254 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffb3807de40 0x7ffb3819de30 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7ffb2c011b60 tx=0x7ffb2c011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T22:58:53.257 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:51.545+0000 7ffb2affd700 1 -- 172.21.15.49:0/223475254 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffb2c003b60 con 0x7ffb3807de40 2024-03-18T22:58:53.257 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:51.545+0000 7ffb2affd700 1 -- 172.21.15.49:0/223475254 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffb2c01a910 con 0x7ffb3807de40 2024-03-18T22:58:53.257 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:51.545+0000 7ffb3ffab700 1 -- 172.21.15.49:0/223475254 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb3819e600 con 0x7ffb3807de40 2024-03-18T22:58:53.257 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:51.545+0000 7ffb2affd700 1 -- 172.21.15.49:0/223475254 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffb2c024930 con 0x7ffb3807de40 2024-03-18T22:58:53.257 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:51.545+0000 7ffb3ffab700 1 -- 172.21.15.49:0/223475254 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb3819ea60 con 0x7ffb3807de40 2024-03-18T22:58:53.257 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:51.546+0000 7ffb3ffab700 1 -- 172.21.15.49:0/223475254 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb38063940 con 0x7ffb3807de40 2024-03-18T22:58:53.257 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:51.546+0000 7ffb2affd700 1 -- 172.21.15.49:0/223475254 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 8) ==== 46106+0+0 (secure 0 0 0) 0x7ffb2c022020 con 0x7ffb3807de40 2024-03-18T22:58:53.258 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:51.546+0000 7ffb2affd700 1 --2- 172.21.15.49:0/223475254 >> [v2:172.21.15.49:6800/2609458951,v1:172.21.15.49:6801/2609458951] conn(0x7ffb24038f70 0x7ffb2403b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T22:58:53.258 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:51.546+0000 7ffb2affd700 1 -- 172.21.15.49:0/223475254 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7ffb2c04d040 con 0x7ffb3807de40 2024-03-18T22:58:53.258 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:51.546+0000 7ffb3d546700 1 --2- 172.21.15.49:0/223475254 >> [v2:172.21.15.49:6800/2609458951,v1:172.21.15.49:6801/2609458951] conn(0x7ffb24038f70 0x7ffb2403b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T22:58:53.258 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:51.547+0000 7ffb3d546700 1 --2- 172.21.15.49:0/223475254 >> [v2:172.21.15.49:6800/2609458951,v1:172.21.15.49:6801/2609458951] conn(0x7ffb24038f70 0x7ffb2403b430 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7ffb34006fd0 tx=0x7ffb34006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T22:58:53.258 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:51.553+0000 7ffb2affd700 1 -- 172.21.15.49:0/223475254 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7ffb2c01a4e0 con 0x7ffb3807de40 2024-03-18T22:58:53.258 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:51.821+0000 7ffb3ffab700 1 -- 172.21.15.49:0/223475254 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7ffb381a1430 con 0x7ffb3807de40 2024-03-18T22:58:53.258 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:52.636+0000 7ffb2affd700 1 -- 172.21.15.49:0/223475254 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7ffb2c02d3f0 con 0x7ffb3807de40 2024-03-18T22:58:53.258 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:52.637+0000 7ffb2affd700 1 -- 172.21.15.49:0/223475254 <== mon.0 v2:172.21.15.49:3300/0 8 ==== mgrmap(e 9) ==== 46119+0+0 (secure 0 0 0) 0x7ffb2c024ad0 con 0x7ffb3807de40 2024-03-18T22:58:53.258 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:52.639+0000 7ffb3ffab700 1 -- 172.21.15.49:0/223475254 >> [v2:172.21.15.49:6800/2609458951,v1:172.21.15.49:6801/2609458951] conn(0x7ffb24038f70 msgr2=0x7ffb2403b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T22:58:53.258 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:52.639+0000 7ffb3ffab700 1 --2- 172.21.15.49:0/223475254 >> [v2:172.21.15.49:6800/2609458951,v1:172.21.15.49:6801/2609458951] conn(0x7ffb24038f70 0x7ffb2403b430 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7ffb34006fd0 tx=0x7ffb34006e40 comp rx=0 tx=0).stop 2024-03-18T22:58:53.258 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:52.639+0000 7ffb3ffab700 1 -- 172.21.15.49:0/223475254 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffb3807de40 msgr2=0x7ffb3819de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T22:58:53.258 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:52.639+0000 7ffb3ffab700 1 --2- 172.21.15.49:0/223475254 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffb3807de40 0x7ffb3819de30 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7ffb2c011b60 tx=0x7ffb2c011c40 comp rx=0 tx=0).stop 2024-03-18T22:58:53.259 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:52.639+0000 7ffb3ffab700 1 -- 172.21.15.49:0/223475254 shutdown_connections 2024-03-18T22:58:53.259 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:52.639+0000 7ffb3ffab700 1 --2- 172.21.15.49:0/223475254 >> [v2:172.21.15.49:6800/2609458951,v1:172.21.15.49:6801/2609458951] conn(0x7ffb24038f70 0x7ffb2403b430 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T22:58:53.259 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:52.639+0000 7ffb3ffab700 1 --2- 172.21.15.49:0/223475254 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffb3807de40 0x7ffb3819de30 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T22:58:53.259 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:52.639+0000 7ffb3ffab700 1 -- 172.21.15.49:0/223475254 >> 172.21.15.49:0/223475254 conn(0x7ffb381032b0 msgr2=0x7ffb38103da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T22:58:53.259 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:52.639+0000 7ffb3ffab700 1 -- 172.21.15.49:0/223475254 shutdown_connections 2024-03-18T22:58:53.259 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:52.639+0000 7ffb3ffab700 1 -- 172.21.15.49:0/223475254 wait complete. 2024-03-18T22:58:53.621 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 18 22:58:53 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-x[63224]: 2024-03-18T22:58:53.327+0000 7f18f103f200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-18T22:58:53.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:53 smithi049 ceph-mon[62991]: from='client.? 172.21.15.49:0/223475254' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-03-18T22:58:53.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:53 smithi049 ceph-mon[62991]: mgrmap e9: x(active, since 20s) 2024-03-18T22:58:54.255 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 18 22:58:54 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-x[63224]: 2024-03-18T22:58:54.005+0000 7f18f103f200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-18T22:58:54.255 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 18 22:58:54 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-x[63224]: 2024-03-18T22:58:54.106+0000 7f18f103f200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-18T22:58:54.256 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 18 22:58:54 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-x[63224]: 2024-03-18T22:58:54.254+0000 7f18f103f200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-18T22:58:54.630 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 18 22:58:54 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-x[63224]: 2024-03-18T22:58:54.355+0000 7f18f103f200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-18T22:58:54.630 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 18 22:58:54 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-x[63224]: 2024-03-18T22:58:54.445+0000 7f18f103f200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-18T22:58:54.927 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:54 smithi049 ceph-mon[62991]: from='client.? 172.21.15.49:0/574534103' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-03-18T22:58:54.927 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 18 22:58:54 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-x[63224]: 2024-03-18T22:58:54.721+0000 7f18f103f200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-18T22:58:54.927 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 18 22:58:54 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-x[63224]: 2024-03-18T22:58:54.835+0000 7f18f103f200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-18T22:58:54.927 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 18 22:58:54 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-x[63224]: 2024-03-18T22:58:54.926+0000 7f18f103f200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-18T22:58:54.970 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout { 2024-03-18T22:58:54.971 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-03-18T22:58:54.971 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "available": true, 2024-03-18T22:58:54.971 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-03-18T22:58:54.971 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-03-18T22:58:54.971 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout } 2024-03-18T22:58:54.971 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:54.067+0000 7f3f0efaf700 1 Processor -- start 2024-03-18T22:58:54.971 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:54.067+0000 7f3f0efaf700 1 -- start start 2024-03-18T22:58:54.971 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:54.067+0000 7f3f0efaf700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3f081081d0 0x7f3f081085f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T22:58:54.971 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:54.067+0000 7f3f0efaf700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f3f08108bc0 con 0x7f3f081081d0 2024-03-18T22:58:54.971 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:54.068+0000 7f3f0cd4b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3f081081d0 0x7f3f081085f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T22:58:54.971 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:54.068+0000 7f3f0cd4b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3f081081d0 0x7f3f081085f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:52160/0 (socket says 172.21.15.49:52160) 2024-03-18T22:58:54.971 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:54.068+0000 7f3f0cd4b700 1 -- 172.21.15.49:0/2938391449 learned_addr learned my addr 172.21.15.49:0/2938391449 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T22:58:54.972 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:54.068+0000 7f3f0cd4b700 1 -- 172.21.15.49:0/2938391449 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f08106de0 con 0x7f3f081081d0 2024-03-18T22:58:54.972 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:54.068+0000 7f3f0cd4b700 1 --2- 172.21.15.49:0/2938391449 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3f081081d0 0x7f3f081085f0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f3f00009e80 tx=0x7f3f0000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11fee15a20fe8620 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T22:58:54.972 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:54.069+0000 7f3f077fe700 1 -- 172.21.15.49:0/2938391449 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3f00017070 con 0x7f3f081081d0 2024-03-18T22:58:54.972 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:54.069+0000 7f3f077fe700 1 -- 172.21.15.49:0/2938391449 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3f0000b8c0 con 0x7f3f081081d0 2024-03-18T22:58:54.972 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:54.069+0000 7f3f077fe700 1 -- 172.21.15.49:0/2938391449 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3f000045c0 con 0x7f3f081081d0 2024-03-18T22:58:54.972 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:54.070+0000 7f3f0efaf700 1 -- 172.21.15.49:0/2938391449 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3f081081d0 msgr2=0x7f3f081085f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T22:58:54.972 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:54.070+0000 7f3f0efaf700 1 --2- 172.21.15.49:0/2938391449 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3f081081d0 0x7f3f081085f0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f3f00009e80 tx=0x7f3f0000b330 comp rx=0 tx=0).stop 2024-03-18T22:58:54.972 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:54.070+0000 7f3f0efaf700 1 -- 172.21.15.49:0/2938391449 shutdown_connections 2024-03-18T22:58:54.972 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:54.070+0000 7f3f0efaf700 1 --2- 172.21.15.49:0/2938391449 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3f081081d0 0x7f3f081085f0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T22:58:54.972 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:54.070+0000 7f3f0efaf700 1 -- 172.21.15.49:0/2938391449 >> 172.21.15.49:0/2938391449 conn(0x7f3f08101950 msgr2=0x7f3f08103d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T22:58:54.972 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:54.070+0000 7f3f0efaf700 1 -- 172.21.15.49:0/2938391449 shutdown_connections 2024-03-18T22:58:54.973 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:54.070+0000 7f3f0efaf700 1 -- 172.21.15.49:0/2938391449 wait complete. 2024-03-18T22:58:54.973 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:54.070+0000 7f3f0efaf700 1 Processor -- start 2024-03-18T22:58:54.973 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:54.070+0000 7f3f0efaf700 1 -- start start 2024-03-18T22:58:54.973 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:54.071+0000 7f3f0efaf700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3f081081d0 0x7f3f0819aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T22:58:54.973 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:54.071+0000 7f3f0efaf700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f3f0810b1a0 con 0x7f3f081081d0 2024-03-18T22:58:54.973 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:54.071+0000 7f3f0cd4b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3f081081d0 0x7f3f0819aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T22:58:54.973 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:54.071+0000 7f3f0cd4b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3f081081d0 0x7f3f0819aba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:52170/0 (socket says 172.21.15.49:52170) 2024-03-18T22:58:54.973 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:54.071+0000 7f3f0cd4b700 1 -- 172.21.15.49:0/574534103 learned_addr learned my addr 172.21.15.49:0/574534103 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T22:58:54.973 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:54.071+0000 7f3f0cd4b700 1 -- 172.21.15.49:0/574534103 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f0819b0e0 con 0x7f3f081081d0 2024-03-18T22:58:54.974 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:54.071+0000 7f3f0cd4b700 1 --2- 172.21.15.49:0/574534103 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3f081081d0 0x7f3f0819aba0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f3f0000bca0 tx=0x7f3f0000bd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T22:58:54.974 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:54.071+0000 7f3f05ffb700 1 -- 172.21.15.49:0/574534103 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3f00020020 con 0x7f3f081081d0 2024-03-18T22:58:54.974 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:54.071+0000 7f3f05ffb700 1 -- 172.21.15.49:0/574534103 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3f00003d90 con 0x7f3f081081d0 2024-03-18T22:58:54.974 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:54.071+0000 7f3f05ffb700 1 -- 172.21.15.49:0/574534103 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3f00017040 con 0x7f3f081081d0 2024-03-18T22:58:54.974 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:54.072+0000 7f3f0efaf700 1 -- 172.21.15.49:0/574534103 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f0819b370 con 0x7f3f081081d0 2024-03-18T22:58:54.974 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:54.072+0000 7f3f0efaf700 1 -- 172.21.15.49:0/574534103 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f08105cd0 con 0x7f3f081081d0 2024-03-18T22:58:54.974 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:54.073+0000 7f3f05ffb700 1 -- 172.21.15.49:0/574534103 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 9) ==== 46119+0+0 (secure 0 0 0) 0x7f3f00023020 con 0x7f3f081081d0 2024-03-18T22:58:54.974 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:54.073+0000 7f3f0efaf700 1 -- 172.21.15.49:0/574534103 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f080693e0 con 0x7f3f081081d0 2024-03-18T22:58:54.974 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:54.073+0000 7f3f05ffb700 1 --2- 172.21.15.49:0/574534103 >> [v2:172.21.15.49:6800/2609458951,v1:172.21.15.49:6801/2609458951] conn(0x7f3ef403b190 0x7f3ef403d650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T22:58:54.974 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:54.073+0000 7f3f05ffb700 1 -- 172.21.15.49:0/574534103 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f3f0000f050 con 0x7f3f081081d0 2024-03-18T22:58:54.974 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:54.073+0000 7f3f07fff700 1 -- 172.21.15.49:0/574534103 >> [v2:172.21.15.49:6800/2609458951,v1:172.21.15.49:6801/2609458951] conn(0x7f3ef403b190 msgr2=0x7f3ef403d650 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.49:6800/2609458951 2024-03-18T22:58:54.975 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:54.073+0000 7f3f07fff700 1 --2- 172.21.15.49:0/574534103 >> [v2:172.21.15.49:6800/2609458951,v1:172.21.15.49:6801/2609458951] conn(0x7f3ef403b190 0x7f3ef403d650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-18T22:58:54.975 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:54.078+0000 7f3f05ffb700 1 -- 172.21.15.49:0/574534103 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f3f0002a020 con 0x7f3f081081d0 2024-03-18T22:58:54.975 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:54.274+0000 7f3f07fff700 1 -- 172.21.15.49:0/574534103 >> [v2:172.21.15.49:6800/2609458951,v1:172.21.15.49:6801/2609458951] conn(0x7f3ef403b190 msgr2=0x7f3ef403d650 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.49:6800/2609458951 2024-03-18T22:58:54.975 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:54.274+0000 7f3f07fff700 1 --2- 172.21.15.49:0/574534103 >> [v2:172.21.15.49:6800/2609458951,v1:172.21.15.49:6801/2609458951] conn(0x7f3ef403b190 0x7f3ef403d650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-18T22:58:54.975 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:54.344+0000 7f3f0efaf700 1 -- 172.21.15.49:0/574534103 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f3f081062d0 con 0x7f3f081081d0 2024-03-18T22:58:54.975 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:54.344+0000 7f3f05ffb700 1 -- 172.21.15.49:0/574534103 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7f3f0002d350 con 0x7f3f081081d0 2024-03-18T22:58:54.975 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:54.347+0000 7f3f0efaf700 1 -- 172.21.15.49:0/574534103 >> [v2:172.21.15.49:6800/2609458951,v1:172.21.15.49:6801/2609458951] conn(0x7f3ef403b190 msgr2=0x7f3ef403d650 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-18T22:58:54.975 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:54.347+0000 7f3f0efaf700 1 --2- 172.21.15.49:0/574534103 >> [v2:172.21.15.49:6800/2609458951,v1:172.21.15.49:6801/2609458951] conn(0x7f3ef403b190 0x7f3ef403d650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T22:58:54.975 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:54.347+0000 7f3f0efaf700 1 -- 172.21.15.49:0/574534103 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3f081081d0 msgr2=0x7f3f0819aba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T22:58:54.975 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:54.347+0000 7f3f0efaf700 1 --2- 172.21.15.49:0/574534103 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3f081081d0 0x7f3f0819aba0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f3f0000bca0 tx=0x7f3f0000bd80 comp rx=0 tx=0).stop 2024-03-18T22:58:54.976 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:54.347+0000 7f3f0efaf700 1 -- 172.21.15.49:0/574534103 shutdown_connections 2024-03-18T22:58:54.976 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:54.347+0000 7f3f0efaf700 1 --2- 172.21.15.49:0/574534103 >> [v2:172.21.15.49:6800/2609458951,v1:172.21.15.49:6801/2609458951] conn(0x7f3ef403b190 0x7f3ef403d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T22:58:54.976 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:54.347+0000 7f3f0efaf700 1 --2- 172.21.15.49:0/574534103 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3f081081d0 0x7f3f0819aba0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T22:58:54.976 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:54.347+0000 7f3f0efaf700 1 -- 172.21.15.49:0/574534103 >> 172.21.15.49:0/574534103 conn(0x7f3f08101950 msgr2=0x7f3f08102480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T22:58:54.976 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:54.347+0000 7f3f0efaf700 1 -- 172.21.15.49:0/574534103 shutdown_connections 2024-03-18T22:58:54.976 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:54.347+0000 7f3f0efaf700 1 -- 172.21.15.49:0/574534103 wait complete. 2024-03-18T22:58:54.976 INFO:teuthology.orchestra.run.smithi049.stdout:Waiting for the mgr to restart... 2024-03-18T22:58:54.976 INFO:teuthology.orchestra.run.smithi049.stdout:Waiting for mgr epoch 9... 2024-03-18T22:58:55.213 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 18 22:58:55 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-x[63224]: 2024-03-18T22:58:55.030+0000 7f18f103f200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-18T22:58:55.630 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 18 22:58:55 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-x[63224]: 2024-03-18T22:58:55.217+0000 7f18f103f200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-18T22:58:56.375 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 18 22:58:56 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-x[63224]: 2024-03-18T22:58:56.238+0000 7f18f103f200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-18T22:58:56.375 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 18 22:58:56 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-x[63224]: 2024-03-18T22:58:56.374+0000 7f18f103f200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-18T22:58:57.021 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 18 22:58:56 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-x[63224]: 2024-03-18T22:58:56.719+0000 7f18f103f200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-18T22:58:57.021 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 18 22:58:56 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-x[63224]: 2024-03-18T22:58:56.813+0000 7f18f103f200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-18T22:58:57.021 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 18 22:58:57 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-x[63224]: 2024-03-18T22:58:57.019+0000 7f18f103f200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-18T22:58:57.380 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 18 22:58:57 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-x[63224]: 2024-03-18T22:58:57.214+0000 7f18f103f200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-18T22:58:57.783 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 18 22:58:57 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-x[63224]: 2024-03-18T22:58:57.511+0000 7f18f103f200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-18T22:58:57.783 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 18 22:58:57 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-x[63224]: 2024-03-18T22:58:57.781+0000 7f18f103f200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-18T22:58:58.130 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 18 22:58:58 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-x[63224]: 2024-03-18T22:58:58.031+0000 7f18f103f200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-18T22:58:59.130 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 18 22:58:58 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-x[63224]: 2024-03-18T22:58:58.872+0000 7f18f103f200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-18T22:58:59.793 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 18 22:58:59 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-x[63224]: 2024-03-18T22:58:59.464+0000 7f18f103f200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-18T22:58:59.793 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 18 22:58:59 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-x[63224]: 2024-03-18T22:58:59.566+0000 7f18f103f200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-18T22:58:59.793 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 18 22:58:59 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-x[63224]: 2024-03-18T22:58:59.792+0000 7f18f103f200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-18T22:59:00.130 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:59 smithi049 ceph-mon[62991]: Active manager daemon x restarted 2024-03-18T22:59:00.130 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:59 smithi049 ceph-mon[62991]: Activating manager daemon x 2024-03-18T22:59:00.130 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:59 smithi049 ceph-mon[62991]: osdmap e3: 0 total, 0 up, 0 in 2024-03-18T22:59:00.130 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:59 smithi049 ceph-mon[62991]: mgrmap e10: x(active, starting, since 0.00349558s) 2024-03-18T22:59:00.130 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:59 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-18T22:59:00.131 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:59 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-18T22:59:00.131 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:59 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-18T22:59:00.131 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:59 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-18T22:59:00.131 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:59 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-18T22:59:00.131 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:59 smithi049 ceph-mon[62991]: Manager daemon x is now available 2024-03-18T22:59:00.131 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:59 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T22:59:00.131 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:58:59 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T22:59:01.120 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:59:00 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-18T22:59:01.120 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:59:00 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-18T22:59:01.120 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:59:00 smithi049 ceph-mon[62991]: mgrmap e11: x(active, since 1.00589s) 2024-03-18T22:59:01.370 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout { 2024-03-18T22:59:01.378 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-03-18T22:59:01.378 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "initialized": true 2024-03-18T22:59:01.379 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout } 2024-03-18T22:59:01.379 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:55.425+0000 7f9e3c9ad700 1 Processor -- start 2024-03-18T22:59:01.379 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:55.425+0000 7f9e3c9ad700 1 -- start start 2024-03-18T22:59:01.379 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:55.425+0000 7f9e3c9ad700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9e38106c00 0x7f9e38107020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T22:59:01.379 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:55.425+0000 7f9e3c9ad700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f9e381075f0 con 0x7f9e38106c00 2024-03-18T22:59:01.379 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:55.425+0000 7f9e3659c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9e38106c00 0x7f9e38107020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T22:59:01.379 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:55.425+0000 7f9e3659c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9e38106c00 0x7f9e38107020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:57672/0 (socket says 172.21.15.49:57672) 2024-03-18T22:59:01.379 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:55.425+0000 7f9e3659c700 1 -- 172.21.15.49:0/570398864 learned_addr learned my addr 172.21.15.49:0/570398864 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T22:59:01.379 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:55.425+0000 7f9e3659c700 1 -- 172.21.15.49:0/570398864 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e38107e00 con 0x7f9e38106c00 2024-03-18T22:59:01.379 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:55.426+0000 7f9e3659c700 1 --2- 172.21.15.49:0/570398864 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9e38106c00 0x7f9e38107020 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f9e280098a0 tx=0x7f9e28009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aaa6dc3436ecff44 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T22:59:01.379 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:55.426+0000 7f9e3559a700 1 -- 172.21.15.49:0/570398864 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9e28004160 con 0x7f9e38106c00 2024-03-18T22:59:01.379 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:55.426+0000 7f9e3559a700 1 -- 172.21.15.49:0/570398864 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e28004300 con 0x7f9e38106c00 2024-03-18T22:59:01.379 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:55.426+0000 7f9e3559a700 1 -- 172.21.15.49:0/570398864 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9e28011b20 con 0x7f9e38106c00 2024-03-18T22:59:01.380 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:55.426+0000 7f9e3c9ad700 1 -- 172.21.15.49:0/570398864 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9e38106c00 msgr2=0x7f9e38107020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T22:59:01.380 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:55.426+0000 7f9e3c9ad700 1 --2- 172.21.15.49:0/570398864 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9e38106c00 0x7f9e38107020 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f9e280098a0 tx=0x7f9e28009bb0 comp rx=0 tx=0).stop 2024-03-18T22:59:01.380 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:55.427+0000 7f9e3c9ad700 1 -- 172.21.15.49:0/570398864 shutdown_connections 2024-03-18T22:59:01.380 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:55.427+0000 7f9e3c9ad700 1 --2- 172.21.15.49:0/570398864 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9e38106c00 0x7f9e38107020 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T22:59:01.380 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:55.427+0000 7f9e3c9ad700 1 -- 172.21.15.49:0/570398864 >> 172.21.15.49:0/570398864 conn(0x7f9e381021a0 msgr2=0x7f9e381045e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T22:59:01.380 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:55.427+0000 7f9e3c9ad700 1 -- 172.21.15.49:0/570398864 shutdown_connections 2024-03-18T22:59:01.380 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:55.427+0000 7f9e3c9ad700 1 -- 172.21.15.49:0/570398864 wait complete. 2024-03-18T22:59:01.380 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:55.427+0000 7f9e3c9ad700 1 Processor -- start 2024-03-18T22:59:01.380 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:55.427+0000 7f9e3c9ad700 1 -- start start 2024-03-18T22:59:01.380 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:55.428+0000 7f9e3c9ad700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9e38106c00 0x7f9e3819bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T22:59:01.380 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:55.428+0000 7f9e3c9ad700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f9e3810b1b0 con 0x7f9e38106c00 2024-03-18T22:59:01.380 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:55.428+0000 7f9e3659c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9e38106c00 0x7f9e3819bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T22:59:01.380 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:55.428+0000 7f9e3659c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9e38106c00 0x7f9e3819bcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:57684/0 (socket says 172.21.15.49:57684) 2024-03-18T22:59:01.380 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:55.428+0000 7f9e3659c700 1 -- 172.21.15.49:0/312641362 learned_addr learned my addr 172.21.15.49:0/312641362 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T22:59:01.381 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:55.428+0000 7f9e3659c700 1 -- 172.21.15.49:0/312641362 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e3819c1f0 con 0x7f9e38106c00 2024-03-18T22:59:01.381 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:55.428+0000 7f9e3659c700 1 --2- 172.21.15.49:0/312641362 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9e38106c00 0x7f9e3819bcb0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f9e280117e0 tx=0x7f9e28011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T22:59:01.381 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:55.428+0000 7f9e2f7fe700 1 -- 172.21.15.49:0/312641362 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9e28003bf0 con 0x7f9e38106c00 2024-03-18T22:59:01.381 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:55.428+0000 7f9e2f7fe700 1 -- 172.21.15.49:0/312641362 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e2801a910 con 0x7f9e38106c00 2024-03-18T22:59:01.381 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:55.428+0000 7f9e3c9ad700 1 -- 172.21.15.49:0/312641362 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e3819c480 con 0x7f9e38106c00 2024-03-18T22:59:01.381 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:55.428+0000 7f9e2f7fe700 1 -- 172.21.15.49:0/312641362 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9e28024970 con 0x7f9e38106c00 2024-03-18T22:59:01.381 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:55.428+0000 7f9e3c9ad700 1 -- 172.21.15.49:0/312641362 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e3819c8e0 con 0x7f9e38106c00 2024-03-18T22:59:01.381 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:55.429+0000 7f9e2f7fe700 1 -- 172.21.15.49:0/312641362 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 9) ==== 46119+0+0 (secure 0 0 0) 0x7f9e28022020 con 0x7f9e38106c00 2024-03-18T22:59:01.381 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:55.429+0000 7f9e2f7fe700 1 --2- 172.21.15.49:0/312641362 >> [v2:172.21.15.49:6800/2609458951,v1:172.21.15.49:6801/2609458951] conn(0x7f9e24038f70 0x7f9e2403b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T22:59:01.381 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:55.429+0000 7f9e2f7fe700 1 -- 172.21.15.49:0/312641362 --> [v2:172.21.15.49:6800/2609458951,v1:172.21.15.49:6801/2609458951] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f9e2403bb40 con 0x7f9e24038f70 2024-03-18T22:59:01.381 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:55.429+0000 7f9e2f7fe700 1 -- 172.21.15.49:0/312641362 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f9e2804d670 con 0x7f9e38106c00 2024-03-18T22:59:01.381 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:55.429+0000 7f9e35d9b700 1 -- 172.21.15.49:0/312641362 >> [v2:172.21.15.49:6800/2609458951,v1:172.21.15.49:6801/2609458951] conn(0x7f9e24038f70 msgr2=0x7f9e2403b430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.49:6800/2609458951 2024-03-18T22:59:01.381 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:55.430+0000 7f9e35d9b700 1 --2- 172.21.15.49:0/312641362 >> [v2:172.21.15.49:6800/2609458951,v1:172.21.15.49:6801/2609458951] conn(0x7f9e24038f70 0x7f9e2403b430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-18T22:59:01.381 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:55.631+0000 7f9e35d9b700 1 -- 172.21.15.49:0/312641362 >> [v2:172.21.15.49:6800/2609458951,v1:172.21.15.49:6801/2609458951] conn(0x7f9e24038f70 msgr2=0x7f9e2403b430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.49:6800/2609458951 2024-03-18T22:59:01.382 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:55.631+0000 7f9e35d9b700 1 --2- 172.21.15.49:0/312641362 >> [v2:172.21.15.49:6800/2609458951,v1:172.21.15.49:6801/2609458951] conn(0x7f9e24038f70 0x7f9e2403b430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-18T22:59:01.382 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:56.032+0000 7f9e35d9b700 1 -- 172.21.15.49:0/312641362 >> [v2:172.21.15.49:6800/2609458951,v1:172.21.15.49:6801/2609458951] conn(0x7f9e24038f70 msgr2=0x7f9e2403b430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.49:6800/2609458951 2024-03-18T22:59:01.382 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:56.032+0000 7f9e35d9b700 1 --2- 172.21.15.49:0/312641362 >> [v2:172.21.15.49:6800/2609458951,v1:172.21.15.49:6801/2609458951] conn(0x7f9e24038f70 0x7f9e2403b430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-18T22:59:01.382 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:56.833+0000 7f9e35d9b700 1 -- 172.21.15.49:0/312641362 >> [v2:172.21.15.49:6800/2609458951,v1:172.21.15.49:6801/2609458951] conn(0x7f9e24038f70 msgr2=0x7f9e2403b430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.49:6800/2609458951 2024-03-18T22:59:01.382 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:56.833+0000 7f9e35d9b700 1 --2- 172.21.15.49:0/312641362 >> [v2:172.21.15.49:6800/2609458951,v1:172.21.15.49:6801/2609458951] conn(0x7f9e24038f70 0x7f9e2403b430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-18T22:59:01.382 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:58.434+0000 7f9e35d9b700 1 -- 172.21.15.49:0/312641362 >> [v2:172.21.15.49:6800/2609458951,v1:172.21.15.49:6801/2609458951] conn(0x7f9e24038f70 msgr2=0x7f9e2403b430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.49:6800/2609458951 2024-03-18T22:59:01.382 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:58.434+0000 7f9e35d9b700 1 --2- 172.21.15.49:0/312641362 >> [v2:172.21.15.49:6800/2609458951,v1:172.21.15.49:6801/2609458951] conn(0x7f9e24038f70 0x7f9e2403b430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-03-18T22:59:01.382 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:59.807+0000 7f9e2f7fe700 1 -- 172.21.15.49:0/312641362 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mgrmap(e 10) ==== 45886+0+0 (secure 0 0 0) 0x7f9e2802d3f0 con 0x7f9e38106c00 2024-03-18T22:59:01.382 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:59.807+0000 7f9e2f7fe700 1 -- 172.21.15.49:0/312641362 >> [v2:172.21.15.49:6800/2609458951,v1:172.21.15.49:6801/2609458951] conn(0x7f9e24038f70 msgr2=0x7f9e2403b430 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-18T22:59:01.382 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:58:59.807+0000 7f9e2f7fe700 1 --2- 172.21.15.49:0/312641362 >> [v2:172.21.15.49:6800/2609458951,v1:172.21.15.49:6801/2609458951] conn(0x7f9e24038f70 0x7f9e2403b430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T22:59:01.382 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:00.808+0000 7f9e2f7fe700 1 -- 172.21.15.49:0/312641362 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mgrmap(e 11) ==== 46013+0+0 (secure 0 0 0) 0x7f9e28022d30 con 0x7f9e38106c00 2024-03-18T22:59:01.382 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:00.808+0000 7f9e2f7fe700 1 --2- 172.21.15.49:0/312641362 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f9e2403c9f0 0x7f9e2403ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T22:59:01.383 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:00.808+0000 7f9e2f7fe700 1 -- 172.21.15.49:0/312641362 --> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f9e28047700 con 0x7f9e2403c9f0 2024-03-18T22:59:01.383 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:00.809+0000 7f9e35d9b700 1 --2- 172.21.15.49:0/312641362 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f9e2403c9f0 0x7f9e2403ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T22:59:01.383 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:00.810+0000 7f9e35d9b700 1 --2- 172.21.15.49:0/312641362 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f9e2403c9f0 0x7f9e2403ede0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f9e20003a10 tx=0x7f9e200092b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T22:59:01.383 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:00.810+0000 7f9e2f7fe700 1 -- 172.21.15.49:0/312641362 <== mgr.14150 v2:172.21.15.49:6800/2083805211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7774 (secure 0 0 0) 0x7f9e28047700 con 0x7f9e2403c9f0 2024-03-18T22:59:01.383 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:00.817+0000 7f9e3c9ad700 1 -- 172.21.15.49:0/312641362 --> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f9e380693e0 con 0x7f9e2403c9f0 2024-03-18T22:59:01.383 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:00.817+0000 7f9e2f7fe700 1 -- 172.21.15.49:0/312641362 <== mgr.14150 v2:172.21.15.49:6800/2083805211 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7f9e380693e0 con 0x7f9e2403c9f0 2024-03-18T22:59:01.383 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:00.818+0000 7f9e3c9ad700 1 -- 172.21.15.49:0/312641362 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f9e2403c9f0 msgr2=0x7f9e2403ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T22:59:01.383 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:00.818+0000 7f9e3c9ad700 1 --2- 172.21.15.49:0/312641362 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f9e2403c9f0 0x7f9e2403ede0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f9e20003a10 tx=0x7f9e200092b0 comp rx=0 tx=0).stop 2024-03-18T22:59:01.383 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:00.818+0000 7f9e3c9ad700 1 -- 172.21.15.49:0/312641362 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9e38106c00 msgr2=0x7f9e3819bcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T22:59:01.383 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:00.818+0000 7f9e3c9ad700 1 --2- 172.21.15.49:0/312641362 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9e38106c00 0x7f9e3819bcb0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f9e280117e0 tx=0x7f9e28011ba0 comp rx=0 tx=0).stop 2024-03-18T22:59:01.383 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:00.818+0000 7f9e3c9ad700 1 -- 172.21.15.49:0/312641362 shutdown_connections 2024-03-18T22:59:01.383 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:00.818+0000 7f9e3c9ad700 1 --2- 172.21.15.49:0/312641362 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f9e2403c9f0 0x7f9e2403ede0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T22:59:01.383 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:00.818+0000 7f9e3c9ad700 1 --2- 172.21.15.49:0/312641362 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9e38106c00 0x7f9e3819bcb0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T22:59:01.384 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:00.818+0000 7f9e3c9ad700 1 --2- 172.21.15.49:0/312641362 >> [v2:172.21.15.49:6800/2609458951,v1:172.21.15.49:6801/2609458951] conn(0x7f9e24038f70 0x7f9e2403b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T22:59:01.384 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:00.818+0000 7f9e3c9ad700 1 -- 172.21.15.49:0/312641362 >> 172.21.15.49:0/312641362 conn(0x7f9e381021a0 msgr2=0x7f9e38103030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T22:59:01.384 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:00.818+0000 7f9e3c9ad700 1 -- 172.21.15.49:0/312641362 shutdown_connections 2024-03-18T22:59:01.384 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:00.818+0000 7f9e3c9ad700 1 -- 172.21.15.49:0/312641362 wait complete. 2024-03-18T22:59:01.384 INFO:teuthology.orchestra.run.smithi049.stdout:mgr epoch 9 is available 2024-03-18T22:59:01.384 INFO:teuthology.orchestra.run.smithi049.stdout:Generating a dashboard self-signed certificate... 2024-03-18T22:59:02.120 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:59:01 smithi049 ceph-mon[62991]: [18/Mar/2024:22:59:01] ENGINE Bus STARTING 2024-03-18T22:59:02.121 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:59:01 smithi049 ceph-mon[62991]: [18/Mar/2024:22:59:01] ENGINE Serving on http://172.21.15.49:8765 2024-03-18T22:59:02.121 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:59:01 smithi049 ceph-mon[62991]: [18/Mar/2024:22:59:01] ENGINE Serving on https://172.21.15.49:7150 2024-03-18T22:59:02.121 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:59:01 smithi049 ceph-mon[62991]: [18/Mar/2024:22:59:01] ENGINE Bus STARTED 2024-03-18T22:59:03.068 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:59:02 smithi049 ceph-mon[62991]: mgrmap e12: x(active, since 2s) 2024-03-18T22:59:03.068 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:59:02 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T22:59:03.068 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:59:02 smithi049 ceph-mon[62991]: from='client.14162 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T22:59:03.068 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:59:02 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T22:59:03.068 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:59:02 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T22:59:03.075 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-03-18T22:59:03.075 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:01.889+0000 7f6c1df90700 1 Processor -- start 2024-03-18T22:59:03.075 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:01.890+0000 7f6c1df90700 1 -- start start 2024-03-18T22:59:03.075 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:01.890+0000 7f6c1df90700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6c18107db0 0x7f6c181081d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T22:59:03.075 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:01.890+0000 7f6c1df90700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f6c181087a0 con 0x7f6c18107db0 2024-03-18T22:59:03.075 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:01.890+0000 7f6c177fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6c18107db0 0x7f6c181081d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T22:59:03.076 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:01.890+0000 7f6c177fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6c18107db0 0x7f6c181081d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:57766/0 (socket says 172.21.15.49:57766) 2024-03-18T22:59:03.076 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:01.890+0000 7f6c177fe700 1 -- 172.21.15.49:0/2046545964 learned_addr learned my addr 172.21.15.49:0/2046545964 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T22:59:03.076 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:01.890+0000 7f6c177fe700 1 -- 172.21.15.49:0/2046545964 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c18108f60 con 0x7f6c18107db0 2024-03-18T22:59:03.076 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:01.890+0000 7f6c177fe700 1 --2- 172.21.15.49:0/2046545964 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6c18107db0 0x7f6c181081d0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f6c080098a0 tx=0x7f6c08009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=161b96ab8213a867 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T22:59:03.076 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:01.891+0000 7f6c167fc700 1 -- 172.21.15.49:0/2046545964 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6c08004160 con 0x7f6c18107db0 2024-03-18T22:59:03.076 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:01.891+0000 7f6c167fc700 1 -- 172.21.15.49:0/2046545964 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c08004300 con 0x7f6c18107db0 2024-03-18T22:59:03.076 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:01.891+0000 7f6c167fc700 1 -- 172.21.15.49:0/2046545964 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6c08011b20 con 0x7f6c18107db0 2024-03-18T22:59:03.076 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:01.891+0000 7f6c1df90700 1 -- 172.21.15.49:0/2046545964 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6c18107db0 msgr2=0x7f6c181081d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T22:59:03.076 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:01.891+0000 7f6c1df90700 1 --2- 172.21.15.49:0/2046545964 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6c18107db0 0x7f6c181081d0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f6c080098a0 tx=0x7f6c08009bb0 comp rx=0 tx=0).stop 2024-03-18T22:59:03.076 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:01.891+0000 7f6c1df90700 1 -- 172.21.15.49:0/2046545964 shutdown_connections 2024-03-18T22:59:03.076 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:01.891+0000 7f6c1df90700 1 --2- 172.21.15.49:0/2046545964 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6c18107db0 0x7f6c181081d0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T22:59:03.076 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:01.891+0000 7f6c1df90700 1 -- 172.21.15.49:0/2046545964 >> 172.21.15.49:0/2046545964 conn(0x7f6c18103390 msgr2=0x7f6c181057b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T22:59:03.076 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:01.891+0000 7f6c1df90700 1 -- 172.21.15.49:0/2046545964 shutdown_connections 2024-03-18T22:59:03.076 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:01.891+0000 7f6c1df90700 1 -- 172.21.15.49:0/2046545964 wait complete. 2024-03-18T22:59:03.077 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:01.892+0000 7f6c1df90700 1 Processor -- start 2024-03-18T22:59:03.077 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:01.892+0000 7f6c1df90700 1 -- start start 2024-03-18T22:59:03.077 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:01.892+0000 7f6c1df90700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6c18107db0 0x7f6c1807daf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T22:59:03.077 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:01.892+0000 7f6c1df90700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f6c1810c310 con 0x7f6c18107db0 2024-03-18T22:59:03.077 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:01.892+0000 7f6c177fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6c18107db0 0x7f6c1807daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T22:59:03.077 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:01.892+0000 7f6c177fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6c18107db0 0x7f6c1807daf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:57770/0 (socket says 172.21.15.49:57770) 2024-03-18T22:59:03.077 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:01.892+0000 7f6c177fe700 1 -- 172.21.15.49:0/437026512 learned_addr learned my addr 172.21.15.49:0/437026512 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T22:59:03.077 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:01.893+0000 7f6c177fe700 1 -- 172.21.15.49:0/437026512 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c1807e030 con 0x7f6c18107db0 2024-03-18T22:59:03.077 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:01.893+0000 7f6c177fe700 1 --2- 172.21.15.49:0/437026512 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6c18107db0 0x7f6c1807daf0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f6c08000c00 tx=0x7f6c08011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T22:59:03.077 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:01.893+0000 7f6c14ff9700 1 -- 172.21.15.49:0/437026512 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6c08003b60 con 0x7f6c18107db0 2024-03-18T22:59:03.077 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:01.893+0000 7f6c14ff9700 1 -- 172.21.15.49:0/437026512 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c0801a910 con 0x7f6c18107db0 2024-03-18T22:59:03.077 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:01.893+0000 7f6c1df90700 1 -- 172.21.15.49:0/437026512 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c1807c270 con 0x7f6c18107db0 2024-03-18T22:59:03.077 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:01.893+0000 7f6c14ff9700 1 -- 172.21.15.49:0/437026512 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6c08024930 con 0x7f6c18107db0 2024-03-18T22:59:03.077 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:01.893+0000 7f6c1df90700 1 -- 172.21.15.49:0/437026512 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c1807c6d0 con 0x7f6c18107db0 2024-03-18T22:59:03.078 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:01.894+0000 7f6c14ff9700 1 -- 172.21.15.49:0/437026512 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 12) ==== 46119+0+0 (secure 0 0 0) 0x7f6c08022020 con 0x7f6c18107db0 2024-03-18T22:59:03.078 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:01.894+0000 7f6c1df90700 1 -- 172.21.15.49:0/437026512 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c1810c490 con 0x7f6c18107db0 2024-03-18T22:59:03.078 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:01.894+0000 7f6c14ff9700 1 --2- 172.21.15.49:0/437026512 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f6c00038fd0 0x7f6c0003b490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T22:59:03.078 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:01.894+0000 7f6c14ff9700 1 -- 172.21.15.49:0/437026512 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f6c0804d290 con 0x7f6c18107db0 2024-03-18T22:59:03.078 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:01.894+0000 7f6c16ffd700 1 --2- 172.21.15.49:0/437026512 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f6c00038fd0 0x7f6c0003b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T22:59:03.078 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:01.895+0000 7f6c16ffd700 1 --2- 172.21.15.49:0/437026512 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f6c00038fd0 0x7f6c0003b490 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f6c0c006fd0 tx=0x7f6c0c006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T22:59:03.078 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:01.899+0000 7f6c14ff9700 1 -- 172.21.15.49:0/437026512 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f6c0800be30 con 0x7f6c18107db0 2024-03-18T22:59:03.078 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:02.119+0000 7f6c1df90700 1 -- 172.21.15.49:0/437026512 --> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7f6c1807cea0 con 0x7f6c00038fd0 2024-03-18T22:59:03.078 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:02.364+0000 7f6c14ff9700 1 -- 172.21.15.49:0/437026512 <== mgr.14150 v2:172.21.15.49:6800/2083805211 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7f6c1807cea0 con 0x7f6c00038fd0 2024-03-18T22:59:03.078 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:02.366+0000 7f6c1df90700 1 -- 172.21.15.49:0/437026512 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f6c00038fd0 msgr2=0x7f6c0003b490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T22:59:03.078 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:02.366+0000 7f6c1df90700 1 --2- 172.21.15.49:0/437026512 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f6c00038fd0 0x7f6c0003b490 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f6c0c006fd0 tx=0x7f6c0c006e40 comp rx=0 tx=0).stop 2024-03-18T22:59:03.078 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:02.366+0000 7f6c1df90700 1 -- 172.21.15.49:0/437026512 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6c18107db0 msgr2=0x7f6c1807daf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T22:59:03.078 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:02.366+0000 7f6c1df90700 1 --2- 172.21.15.49:0/437026512 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6c18107db0 0x7f6c1807daf0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f6c08000c00 tx=0x7f6c08011c10 comp rx=0 tx=0).stop 2024-03-18T22:59:03.078 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:02.366+0000 7f6c1df90700 1 -- 172.21.15.49:0/437026512 shutdown_connections 2024-03-18T22:59:03.079 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:02.366+0000 7f6c1df90700 1 --2- 172.21.15.49:0/437026512 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f6c00038fd0 0x7f6c0003b490 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T22:59:03.079 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:02.366+0000 7f6c1df90700 1 --2- 172.21.15.49:0/437026512 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6c18107db0 0x7f6c1807daf0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T22:59:03.079 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:02.367+0000 7f6c1df90700 1 -- 172.21.15.49:0/437026512 >> 172.21.15.49:0/437026512 conn(0x7f6c18103390 msgr2=0x7f6c18103e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T22:59:03.079 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:02.367+0000 7f6c1df90700 1 -- 172.21.15.49:0/437026512 shutdown_connections 2024-03-18T22:59:03.079 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:02.367+0000 7f6c1df90700 1 -- 172.21.15.49:0/437026512 wait complete. 2024-03-18T22:59:03.079 INFO:teuthology.orchestra.run.smithi049.stdout:Creating initial admin user... 2024-03-18T22:59:04.630 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:59:04 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T22:59:04.631 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:59:04 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T22:59:04.964 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$4W3UdEzoGS5zMq5U.tYi5.04Sj.d419zSrzb0Ln.GsacSHkpKuO6m", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1710802743, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-03-18T22:59:04.964 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:03.481+0000 7f5b44ffb700 1 Processor -- start 2024-03-18T22:59:04.964 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:03.481+0000 7f5b44ffb700 1 -- start start 2024-03-18T22:59:04.964 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:03.482+0000 7f5b44ffb700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5b401088a0 0x7f5b4010ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T22:59:04.964 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:03.482+0000 7f5b44ffb700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f5b4007bde0 con 0x7f5b401088a0 2024-03-18T22:59:04.964 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:03.482+0000 7f5b3e59c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5b401088a0 0x7f5b4010ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T22:59:04.964 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:03.482+0000 7f5b3e59c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5b401088a0 0x7f5b4010ac90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:57774/0 (socket says 172.21.15.49:57774) 2024-03-18T22:59:04.964 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:03.482+0000 7f5b3e59c700 1 -- 172.21.15.49:0/4260434860 learned_addr learned my addr 172.21.15.49:0/4260434860 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T22:59:04.964 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:03.482+0000 7f5b3e59c700 1 -- 172.21.15.49:0/4260434860 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b4010b1d0 con 0x7f5b401088a0 2024-03-18T22:59:04.965 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:03.482+0000 7f5b3e59c700 1 --2- 172.21.15.49:0/4260434860 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5b401088a0 0x7f5b4010ac90 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f5b340098a0 tx=0x7f5b34009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f6341111cfe8e79 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T22:59:04.965 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:03.482+0000 7f5b3d59a700 1 -- 172.21.15.49:0/4260434860 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5b34004160 con 0x7f5b401088a0 2024-03-18T22:59:04.965 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:03.482+0000 7f5b3d59a700 1 -- 172.21.15.49:0/4260434860 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b34004300 con 0x7f5b401088a0 2024-03-18T22:59:04.965 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:03.482+0000 7f5b3d59a700 1 -- 172.21.15.49:0/4260434860 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5b34011b20 con 0x7f5b401088a0 2024-03-18T22:59:04.965 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:03.483+0000 7f5b44ffb700 1 -- 172.21.15.49:0/4260434860 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5b401088a0 msgr2=0x7f5b4010ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T22:59:04.965 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:03.483+0000 7f5b44ffb700 1 --2- 172.21.15.49:0/4260434860 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5b401088a0 0x7f5b4010ac90 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f5b340098a0 tx=0x7f5b34009bb0 comp rx=0 tx=0).stop 2024-03-18T22:59:04.965 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:03.483+0000 7f5b44ffb700 1 -- 172.21.15.49:0/4260434860 shutdown_connections 2024-03-18T22:59:04.965 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:03.483+0000 7f5b44ffb700 1 --2- 172.21.15.49:0/4260434860 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5b401088a0 0x7f5b4010ac90 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T22:59:04.965 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:03.483+0000 7f5b44ffb700 1 -- 172.21.15.49:0/4260434860 >> 172.21.15.49:0/4260434860 conn(0x7f5b40102270 msgr2=0x7f5b40104690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T22:59:04.965 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:03.483+0000 7f5b44ffb700 1 -- 172.21.15.49:0/4260434860 shutdown_connections 2024-03-18T22:59:04.965 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:03.483+0000 7f5b44ffb700 1 -- 172.21.15.49:0/4260434860 wait complete. 2024-03-18T22:59:04.965 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:03.484+0000 7f5b44ffb700 1 Processor -- start 2024-03-18T22:59:04.965 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:03.484+0000 7f5b44ffb700 1 -- start start 2024-03-18T22:59:04.965 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:03.484+0000 7f5b44ffb700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5b401088a0 0x7f5b40108120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T22:59:04.966 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:03.484+0000 7f5b44ffb700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f5b4010c1d0 con 0x7f5b401088a0 2024-03-18T22:59:04.966 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:03.484+0000 7f5b3e59c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5b401088a0 0x7f5b40108120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T22:59:04.966 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:03.484+0000 7f5b3e59c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5b401088a0 0x7f5b40108120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:57780/0 (socket says 172.21.15.49:57780) 2024-03-18T22:59:04.966 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:03.484+0000 7f5b3e59c700 1 -- 172.21.15.49:0/4276855700 learned_addr learned my addr 172.21.15.49:0/4276855700 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T22:59:04.966 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:03.485+0000 7f5b3e59c700 1 -- 172.21.15.49:0/4276855700 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b40108660 con 0x7f5b401088a0 2024-03-18T22:59:04.966 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:03.485+0000 7f5b3e59c700 1 --2- 172.21.15.49:0/4276855700 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5b401088a0 0x7f5b40108120 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f5b340117e0 tx=0x7f5b34011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T22:59:04.966 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:03.485+0000 7f5b2f7fe700 1 -- 172.21.15.49:0/4276855700 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5b34003bf0 con 0x7f5b401088a0 2024-03-18T22:59:04.966 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:03.485+0000 7f5b2f7fe700 1 -- 172.21.15.49:0/4276855700 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b3401a910 con 0x7f5b401088a0 2024-03-18T22:59:04.966 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:03.485+0000 7f5b44ffb700 1 -- 172.21.15.49:0/4276855700 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b40106850 con 0x7f5b401088a0 2024-03-18T22:59:04.966 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:03.485+0000 7f5b2f7fe700 1 -- 172.21.15.49:0/4276855700 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5b34024970 con 0x7f5b401088a0 2024-03-18T22:59:04.966 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:03.485+0000 7f5b44ffb700 1 -- 172.21.15.49:0/4276855700 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b40106d10 con 0x7f5b401088a0 2024-03-18T22:59:04.966 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:03.486+0000 7f5b2f7fe700 1 -- 172.21.15.49:0/4276855700 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 12) ==== 46119+0+0 (secure 0 0 0) 0x7f5b34022020 con 0x7f5b401088a0 2024-03-18T22:59:04.966 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:03.486+0000 7f5b44ffb700 1 -- 172.21.15.49:0/4276855700 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b400693e0 con 0x7f5b401088a0 2024-03-18T22:59:04.967 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:03.486+0000 7f5b2f7fe700 1 --2- 172.21.15.49:0/4276855700 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f5b28038f20 0x7f5b2803b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T22:59:04.967 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:03.487+0000 7f5b2f7fe700 1 -- 172.21.15.49:0/4276855700 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f5b3404d310 con 0x7f5b401088a0 2024-03-18T22:59:04.967 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:03.487+0000 7f5b3dd9b700 1 --2- 172.21.15.49:0/4276855700 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f5b28038f20 0x7f5b2803b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T22:59:04.967 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:03.487+0000 7f5b3dd9b700 1 --2- 172.21.15.49:0/4276855700 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f5b28038f20 0x7f5b2803b3e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f5b30006fd0 tx=0x7f5b30006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T22:59:04.967 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:03.491+0000 7f5b2f7fe700 1 -- 172.21.15.49:0/4276855700 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f5b34011e20 con 0x7f5b401088a0 2024-03-18T22:59:04.967 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:03.717+0000 7f5b44ffb700 1 -- 172.21.15.49:0/4276855700 --> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7f5b40106f90 con 0x7f5b28038f20 2024-03-18T22:59:04.967 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:03.964+0000 7f5b2f7fe700 1 -- 172.21.15.49:0/4276855700 <== mgr.14150 v2:172.21.15.49:6800/2083805211 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7f5b40106f90 con 0x7f5b28038f20 2024-03-18T22:59:04.967 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:03.967+0000 7f5b44ffb700 1 -- 172.21.15.49:0/4276855700 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f5b28038f20 msgr2=0x7f5b2803b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T22:59:04.967 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:03.967+0000 7f5b44ffb700 1 --2- 172.21.15.49:0/4276855700 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f5b28038f20 0x7f5b2803b3e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f5b30006fd0 tx=0x7f5b30006e40 comp rx=0 tx=0).stop 2024-03-18T22:59:04.967 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:03.967+0000 7f5b44ffb700 1 -- 172.21.15.49:0/4276855700 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5b401088a0 msgr2=0x7f5b40108120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T22:59:04.967 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:03.967+0000 7f5b44ffb700 1 --2- 172.21.15.49:0/4276855700 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5b401088a0 0x7f5b40108120 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f5b340117e0 tx=0x7f5b34011c10 comp rx=0 tx=0).stop 2024-03-18T22:59:04.967 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:03.967+0000 7f5b44ffb700 1 -- 172.21.15.49:0/4276855700 shutdown_connections 2024-03-18T22:59:04.967 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:03.967+0000 7f5b44ffb700 1 --2- 172.21.15.49:0/4276855700 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f5b28038f20 0x7f5b2803b3e0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T22:59:04.967 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:03.967+0000 7f5b44ffb700 1 --2- 172.21.15.49:0/4276855700 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5b401088a0 0x7f5b40108120 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T22:59:04.968 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:03.967+0000 7f5b44ffb700 1 -- 172.21.15.49:0/4276855700 >> 172.21.15.49:0/4276855700 conn(0x7f5b40102270 msgr2=0x7f5b40102ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T22:59:04.968 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:03.967+0000 7f5b44ffb700 1 -- 172.21.15.49:0/4276855700 shutdown_connections 2024-03-18T22:59:04.968 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:03.967+0000 7f5b44ffb700 1 -- 172.21.15.49:0/4276855700 wait complete. 2024-03-18T22:59:04.968 INFO:teuthology.orchestra.run.smithi049.stdout:Fetching dashboard port number... 2024-03-18T22:59:05.569 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:59:05 smithi049 ceph-mon[62991]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T22:59:06.607 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:59:06 smithi049 ceph-mon[62991]: from='client.? 172.21.15.49:0/4094629150' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-03-18T22:59:06.616 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout 8443 2024-03-18T22:59:06.617 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:05.813+0000 7f0da9339700 1 Processor -- start 2024-03-18T22:59:06.617 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:05.813+0000 7f0da9339700 1 -- start start 2024-03-18T22:59:06.617 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:05.813+0000 7f0da9339700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0da4107d30 0x7f0da4108150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T22:59:06.617 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:05.813+0000 7f0da9339700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f0da4108720 con 0x7f0da4107d30 2024-03-18T22:59:06.617 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:05.814+0000 7f0da2ffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0da4107d30 0x7f0da4108150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T22:59:06.617 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:05.814+0000 7f0da2ffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0da4107d30 0x7f0da4108150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:44482/0 (socket says 172.21.15.49:44482) 2024-03-18T22:59:06.617 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:05.814+0000 7f0da2ffd700 1 -- 172.21.15.49:0/2162867368 learned_addr learned my addr 172.21.15.49:0/2162867368 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T22:59:06.617 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:05.814+0000 7f0da2ffd700 1 -- 172.21.15.49:0/2162867368 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0da4108ee0 con 0x7f0da4107d30 2024-03-18T22:59:06.617 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:05.814+0000 7f0da2ffd700 1 --2- 172.21.15.49:0/2162867368 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0da4107d30 0x7f0da4108150 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f0d94009e80 tx=0x7f0d9400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83750d5ae5ae84b server_cookie=0 in_seq=0 out_seq=0 2024-03-18T22:59:06.617 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:05.814+0000 7f0da1ffb700 1 -- 172.21.15.49:0/2162867368 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0d94017070 con 0x7f0da4107d30 2024-03-18T22:59:06.617 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:05.814+0000 7f0da1ffb700 1 -- 172.21.15.49:0/2162867368 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d9400b8c0 con 0x7f0da4107d30 2024-03-18T22:59:06.617 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:05.814+0000 7f0da1ffb700 1 -- 172.21.15.49:0/2162867368 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0d940045c0 con 0x7f0da4107d30 2024-03-18T22:59:06.618 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:05.815+0000 7f0da9339700 1 -- 172.21.15.49:0/2162867368 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0da4107d30 msgr2=0x7f0da4108150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T22:59:06.618 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:05.815+0000 7f0da9339700 1 --2- 172.21.15.49:0/2162867368 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0da4107d30 0x7f0da4108150 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f0d94009e80 tx=0x7f0d9400b330 comp rx=0 tx=0).stop 2024-03-18T22:59:06.618 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:05.815+0000 7f0da9339700 1 -- 172.21.15.49:0/2162867368 shutdown_connections 2024-03-18T22:59:06.618 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:05.815+0000 7f0da9339700 1 --2- 172.21.15.49:0/2162867368 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0da4107d30 0x7f0da4108150 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T22:59:06.618 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:05.815+0000 7f0da9339700 1 -- 172.21.15.49:0/2162867368 >> 172.21.15.49:0/2162867368 conn(0x7f0da41032b0 msgr2=0x7f0da4105710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T22:59:06.618 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:05.815+0000 7f0da9339700 1 -- 172.21.15.49:0/2162867368 shutdown_connections 2024-03-18T22:59:06.618 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:05.815+0000 7f0da9339700 1 -- 172.21.15.49:0/2162867368 wait complete. 2024-03-18T22:59:06.618 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:05.815+0000 7f0da9339700 1 Processor -- start 2024-03-18T22:59:06.618 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:05.816+0000 7f0da9339700 1 -- start start 2024-03-18T22:59:06.618 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:05.816+0000 7f0da9339700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0da4107d30 0x7f0da407db40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T22:59:06.618 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:05.816+0000 7f0da9339700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f0da410c290 con 0x7f0da4107d30 2024-03-18T22:59:06.618 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:05.816+0000 7f0da2ffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0da4107d30 0x7f0da407db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T22:59:06.618 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:05.816+0000 7f0da2ffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0da4107d30 0x7f0da407db40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:44492/0 (socket says 172.21.15.49:44492) 2024-03-18T22:59:06.619 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:05.816+0000 7f0da2ffd700 1 -- 172.21.15.49:0/4094629150 learned_addr learned my addr 172.21.15.49:0/4094629150 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T22:59:06.619 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:05.816+0000 7f0da2ffd700 1 -- 172.21.15.49:0/4094629150 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0da407e080 con 0x7f0da4107d30 2024-03-18T22:59:06.619 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:05.816+0000 7f0da2ffd700 1 --2- 172.21.15.49:0/4094629150 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0da4107d30 0x7f0da407db40 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f0d94003fa0 tx=0x7f0d94004080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T22:59:06.619 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:05.817+0000 7f0d9bfff700 1 -- 172.21.15.49:0/4094629150 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0d94020020 con 0x7f0da4107d30 2024-03-18T22:59:06.619 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:05.817+0000 7f0d9bfff700 1 -- 172.21.15.49:0/4094629150 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d9400bba0 con 0x7f0da4107d30 2024-03-18T22:59:06.619 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:05.817+0000 7f0da9339700 1 -- 172.21.15.49:0/4094629150 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0da407c270 con 0x7f0da4107d30 2024-03-18T22:59:06.619 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:05.817+0000 7f0d9bfff700 1 -- 172.21.15.49:0/4094629150 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0d94017040 con 0x7f0da4107d30 2024-03-18T22:59:06.619 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:05.817+0000 7f0da9339700 1 -- 172.21.15.49:0/4094629150 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0da407c7b0 con 0x7f0da4107d30 2024-03-18T22:59:06.619 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:05.817+0000 7f0d9bfff700 1 -- 172.21.15.49:0/4094629150 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 12) ==== 46119+0+0 (secure 0 0 0) 0x7f0d94023020 con 0x7f0da4107d30 2024-03-18T22:59:06.619 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:05.818+0000 7f0da9339700 1 -- 172.21.15.49:0/4094629150 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0da4063940 con 0x7f0da4107d30 2024-03-18T22:59:06.619 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:05.818+0000 7f0d9bfff700 1 --2- 172.21.15.49:0/4094629150 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f0d90038f20 0x7f0d9003b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T22:59:06.619 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:05.818+0000 7f0d9bfff700 1 -- 172.21.15.49:0/4094629150 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f0d9400f050 con 0x7f0da4107d30 2024-03-18T22:59:06.619 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:05.818+0000 7f0da27fc700 1 --2- 172.21.15.49:0/4094629150 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f0d90038f20 0x7f0d9003b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T22:59:06.619 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:05.818+0000 7f0da27fc700 1 --2- 172.21.15.49:0/4094629150 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f0d90038f20 0x7f0d9003b3e0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f0d8c006fd0 tx=0x7f0d8c006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T22:59:06.620 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:05.823+0000 7f0d9bfff700 1 -- 172.21.15.49:0/4094629150 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f0d9402a020 con 0x7f0da4107d30 2024-03-18T22:59:06.620 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:06.025+0000 7f0da9339700 1 -- 172.21.15.49:0/4094629150 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7f0da407caf0 con 0x7f0da4107d30 2024-03-18T22:59:06.620 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:06.025+0000 7f0d9bfff700 1 -- 172.21.15.49:0/4094629150 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7f0da407caf0 con 0x7f0da4107d30 2024-03-18T22:59:06.620 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:06.027+0000 7f0da9339700 1 -- 172.21.15.49:0/4094629150 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f0d90038f20 msgr2=0x7f0d9003b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T22:59:06.620 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:06.027+0000 7f0da9339700 1 --2- 172.21.15.49:0/4094629150 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f0d90038f20 0x7f0d9003b3e0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f0d8c006fd0 tx=0x7f0d8c006e40 comp rx=0 tx=0).stop 2024-03-18T22:59:06.620 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:06.027+0000 7f0da9339700 1 -- 172.21.15.49:0/4094629150 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0da4107d30 msgr2=0x7f0da407db40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T22:59:06.620 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:06.027+0000 7f0da9339700 1 --2- 172.21.15.49:0/4094629150 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0da4107d30 0x7f0da407db40 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f0d94003fa0 tx=0x7f0d94004080 comp rx=0 tx=0).stop 2024-03-18T22:59:06.620 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:06.028+0000 7f0da9339700 1 -- 172.21.15.49:0/4094629150 shutdown_connections 2024-03-18T22:59:06.620 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:06.028+0000 7f0da9339700 1 --2- 172.21.15.49:0/4094629150 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f0d90038f20 0x7f0d9003b3e0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T22:59:06.620 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:06.028+0000 7f0da9339700 1 --2- 172.21.15.49:0/4094629150 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0da4107d30 0x7f0da407db40 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T22:59:06.620 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:06.028+0000 7f0da9339700 1 -- 172.21.15.49:0/4094629150 >> 172.21.15.49:0/4094629150 conn(0x7f0da41032b0 msgr2=0x7f0da4103dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T22:59:06.620 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:06.028+0000 7f0da9339700 1 -- 172.21.15.49:0/4094629150 shutdown_connections 2024-03-18T22:59:06.620 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:06.028+0000 7f0da9339700 1 -- 172.21.15.49:0/4094629150 wait complete. 2024-03-18T22:59:06.622 INFO:teuthology.orchestra.run.smithi049.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-18T22:59:06.622 INFO:teuthology.orchestra.run.smithi049.stdout:systemctl: stdout disabled 2024-03-18T22:59:06.629 INFO:teuthology.orchestra.run.smithi049.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-18T22:59:06.629 INFO:teuthology.orchestra.run.smithi049.stdout:systemctl: stdout inactive 2024-03-18T22:59:06.629 INFO:teuthology.orchestra.run.smithi049.stdout:firewalld.service is not enabled 2024-03-18T22:59:06.630 INFO:teuthology.orchestra.run.smithi049.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-03-18T22:59:06.631 INFO:teuthology.orchestra.run.smithi049.stdout:Ceph Dashboard is now available at: 2024-03-18T22:59:06.631 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-18T22:59:06.631 INFO:teuthology.orchestra.run.smithi049.stdout: URL: https://smithi049.front.sepia.ceph.com:8443/ 2024-03-18T22:59:06.631 INFO:teuthology.orchestra.run.smithi049.stdout: User: admin 2024-03-18T22:59:06.631 INFO:teuthology.orchestra.run.smithi049.stdout: Password: 08h8ht9s0t 2024-03-18T22:59:06.632 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-18T22:59:06.632 INFO:teuthology.orchestra.run.smithi049.stdout:Saving cluster configuration to /var/lib/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/config directory 2024-03-18T22:59:06.632 INFO:teuthology.orchestra.run.smithi049.stdout:Enabling autotune for osd_memory_target 2024-03-18T22:59:07.554 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:59:07 smithi049 ceph-mon[62991]: mgrmap e13: x(active, since 6s) 2024-03-18T22:59:07.788 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:07.036+0000 7f376b942700 1 Processor -- start 2024-03-18T22:59:07.788 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:07.036+0000 7f376b942700 1 -- start start 2024-03-18T22:59:07.788 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:07.037+0000 7f376b942700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3764108850 0x7f376410ac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T22:59:07.788 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:07.037+0000 7f376b942700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f376407bda0 con 0x7f3764108850 2024-03-18T22:59:07.788 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:07.037+0000 7f37696de700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3764108850 0x7f376410ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T22:59:07.789 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:07.037+0000 7f37696de700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3764108850 0x7f376410ac40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:44506/0 (socket says 172.21.15.49:44506) 2024-03-18T22:59:07.789 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:07.037+0000 7f37696de700 1 -- 172.21.15.49:0/2650041230 learned_addr learned my addr 172.21.15.49:0/2650041230 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T22:59:07.789 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:07.037+0000 7f37696de700 1 -- 172.21.15.49:0/2650041230 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f376410b180 con 0x7f3764108850 2024-03-18T22:59:07.789 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:07.037+0000 7f37696de700 1 --2- 172.21.15.49:0/2650041230 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3764108850 0x7f376410ac40 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f37600098a0 tx=0x7f3760009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa7812e5b863a2e3 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T22:59:07.789 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:07.037+0000 7f375bfff700 1 -- 172.21.15.49:0/2650041230 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3760004160 con 0x7f3764108850 2024-03-18T22:59:07.789 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:07.037+0000 7f375bfff700 1 -- 172.21.15.49:0/2650041230 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3760004300 con 0x7f3764108850 2024-03-18T22:59:07.789 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:07.037+0000 7f375bfff700 1 -- 172.21.15.49:0/2650041230 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3760011b20 con 0x7f3764108850 2024-03-18T22:59:07.789 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:07.038+0000 7f376b942700 1 -- 172.21.15.49:0/2650041230 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3764108850 msgr2=0x7f376410ac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T22:59:07.789 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:07.038+0000 7f376b942700 1 --2- 172.21.15.49:0/2650041230 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3764108850 0x7f376410ac40 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f37600098a0 tx=0x7f3760009bb0 comp rx=0 tx=0).stop 2024-03-18T22:59:07.789 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:07.038+0000 7f376b942700 1 -- 172.21.15.49:0/2650041230 shutdown_connections 2024-03-18T22:59:07.789 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:07.038+0000 7f376b942700 1 --2- 172.21.15.49:0/2650041230 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3764108850 0x7f376410ac40 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T22:59:07.789 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:07.038+0000 7f376b942700 1 -- 172.21.15.49:0/2650041230 >> 172.21.15.49:0/2650041230 conn(0x7f37641021a0 msgr2=0x7f37641045e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T22:59:07.789 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:07.038+0000 7f376b942700 1 -- 172.21.15.49:0/2650041230 shutdown_connections 2024-03-18T22:59:07.789 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:07.038+0000 7f376b942700 1 -- 172.21.15.49:0/2650041230 wait complete. 2024-03-18T22:59:07.790 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:07.039+0000 7f376b942700 1 Processor -- start 2024-03-18T22:59:07.790 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:07.039+0000 7f376b942700 1 -- start start 2024-03-18T22:59:07.790 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:07.039+0000 7f376b942700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3764108850 0x7f376419bc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T22:59:07.790 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:07.039+0000 7f376b942700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f376410c120 con 0x7f3764108850 2024-03-18T22:59:07.790 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:07.039+0000 7f37696de700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3764108850 0x7f376419bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T22:59:07.790 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:07.039+0000 7f37696de700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3764108850 0x7f376419bc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:44522/0 (socket says 172.21.15.49:44522) 2024-03-18T22:59:07.790 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:07.039+0000 7f37696de700 1 -- 172.21.15.49:0/2962302463 learned_addr learned my addr 172.21.15.49:0/2962302463 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T22:59:07.790 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:07.039+0000 7f37696de700 1 -- 172.21.15.49:0/2962302463 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f376419c150 con 0x7f3764108850 2024-03-18T22:59:07.790 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:07.039+0000 7f37696de700 1 --2- 172.21.15.49:0/2962302463 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3764108850 0x7f376419bc10 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f37600117e0 tx=0x7f3760011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T22:59:07.790 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:07.040+0000 7f375a7fc700 1 -- 172.21.15.49:0/2962302463 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3760003bf0 con 0x7f3764108850 2024-03-18T22:59:07.790 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:07.040+0000 7f376b942700 1 -- 172.21.15.49:0/2962302463 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f376419c3e0 con 0x7f3764108850 2024-03-18T22:59:07.790 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:07.040+0000 7f375a7fc700 1 -- 172.21.15.49:0/2962302463 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f376001a910 con 0x7f3764108850 2024-03-18T22:59:07.791 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:07.040+0000 7f375a7fc700 1 -- 172.21.15.49:0/2962302463 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3760024970 con 0x7f3764108850 2024-03-18T22:59:07.791 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:07.040+0000 7f376b942700 1 -- 172.21.15.49:0/2962302463 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f376419c840 con 0x7f3764108850 2024-03-18T22:59:07.791 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:07.041+0000 7f375a7fc700 1 -- 172.21.15.49:0/2962302463 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 13) ==== 46162+0+0 (secure 0 0 0) 0x7f3760022020 con 0x7f3764108850 2024-03-18T22:59:07.791 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:07.041+0000 7f375a7fc700 1 --2- 172.21.15.49:0/2962302463 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f3750038fc0 0x7f375003b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T22:59:07.791 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:07.041+0000 7f376b942700 1 -- 172.21.15.49:0/2962302463 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3764063940 con 0x7f3764108850 2024-03-18T22:59:07.791 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:07.041+0000 7f375a7fc700 1 -- 172.21.15.49:0/2962302463 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f376004d390 con 0x7f3764108850 2024-03-18T22:59:07.791 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:07.041+0000 7f3768edd700 1 --2- 172.21.15.49:0/2962302463 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f3750038fc0 0x7f375003b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T22:59:07.791 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:07.041+0000 7f3768edd700 1 --2- 172.21.15.49:0/2962302463 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f3750038fc0 0x7f375003b480 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f3754006fd0 tx=0x7f3754006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T22:59:07.791 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:07.046+0000 7f375a7fc700 1 -- 172.21.15.49:0/2962302463 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f3760011e20 con 0x7f3764108850 2024-03-18T22:59:07.791 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:07.249+0000 7f376b942700 1 -- 172.21.15.49:0/2962302463 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) -- 0x7f376419f210 con 0x7f3764108850 2024-03-18T22:59:07.791 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:07.249+0000 7f375a7fc700 1 -- 172.21.15.49:0/2962302463 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) ==== 127+0+0 (secure 0 0 0) 0x7f3760019c70 con 0x7f3764108850 2024-03-18T22:59:07.791 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:07.251+0000 7f376b942700 1 -- 172.21.15.49:0/2962302463 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f3750038fc0 msgr2=0x7f375003b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T22:59:07.791 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:07.251+0000 7f376b942700 1 --2- 172.21.15.49:0/2962302463 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f3750038fc0 0x7f375003b480 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f3754006fd0 tx=0x7f3754006e40 comp rx=0 tx=0).stop 2024-03-18T22:59:07.791 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:07.251+0000 7f376b942700 1 -- 172.21.15.49:0/2962302463 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3764108850 msgr2=0x7f376419bc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T22:59:07.792 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:07.251+0000 7f376b942700 1 --2- 172.21.15.49:0/2962302463 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3764108850 0x7f376419bc10 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f37600117e0 tx=0x7f3760011c10 comp rx=0 tx=0).stop 2024-03-18T22:59:07.792 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:07.251+0000 7f376b942700 1 -- 172.21.15.49:0/2962302463 shutdown_connections 2024-03-18T22:59:07.792 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:07.251+0000 7f376b942700 1 --2- 172.21.15.49:0/2962302463 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f3750038fc0 0x7f375003b480 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T22:59:07.792 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:07.251+0000 7f376b942700 1 --2- 172.21.15.49:0/2962302463 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3764108850 0x7f376419bc10 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T22:59:07.792 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:07.251+0000 7f376b942700 1 -- 172.21.15.49:0/2962302463 >> 172.21.15.49:0/2962302463 conn(0x7f37641021a0 msgr2=0x7f3764102c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T22:59:07.792 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:07.251+0000 7f376b942700 1 -- 172.21.15.49:0/2962302463 shutdown_connections 2024-03-18T22:59:07.792 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:07.252+0000 7f376b942700 1 -- 172.21.15.49:0/2962302463 wait complete. 2024-03-18T22:59:09.079 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:08.194+0000 7fbae864b700 1 Processor -- start 2024-03-18T22:59:09.079 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:08.194+0000 7fbae864b700 1 -- start start 2024-03-18T22:59:09.079 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:08.194+0000 7fbae864b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbae01087a0 0x7fbae010ab90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T22:59:09.079 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:08.194+0000 7fbae864b700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fbae007be60 con 0x7fbae01087a0 2024-03-18T22:59:09.079 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:08.194+0000 7fbae63e7700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbae01087a0 0x7fbae010ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T22:59:09.079 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:08.194+0000 7fbae63e7700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbae01087a0 0x7fbae010ab90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:44524/0 (socket says 172.21.15.49:44524) 2024-03-18T22:59:09.079 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:08.194+0000 7fbae63e7700 1 -- 172.21.15.49:0/3694715554 learned_addr learned my addr 172.21.15.49:0/3694715554 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T22:59:09.079 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:08.195+0000 7fbae63e7700 1 -- 172.21.15.49:0/3694715554 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbae010b0d0 con 0x7fbae01087a0 2024-03-18T22:59:09.079 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:08.195+0000 7fbae63e7700 1 --2- 172.21.15.49:0/3694715554 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbae01087a0 0x7fbae010ab90 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fbadc0098a0 tx=0x7fbadc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63b1c329c8425500 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T22:59:09.079 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:08.195+0000 7fbae53e5700 1 -- 172.21.15.49:0/3694715554 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbadc004160 con 0x7fbae01087a0 2024-03-18T22:59:09.080 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:08.195+0000 7fbae53e5700 1 -- 172.21.15.49:0/3694715554 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbadc004300 con 0x7fbae01087a0 2024-03-18T22:59:09.080 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:08.195+0000 7fbae53e5700 1 -- 172.21.15.49:0/3694715554 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbadc011b20 con 0x7fbae01087a0 2024-03-18T22:59:09.080 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:08.196+0000 7fbae864b700 1 -- 172.21.15.49:0/3694715554 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbae01087a0 msgr2=0x7fbae010ab90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T22:59:09.080 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:08.196+0000 7fbae864b700 1 --2- 172.21.15.49:0/3694715554 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbae01087a0 0x7fbae010ab90 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fbadc0098a0 tx=0x7fbadc009bb0 comp rx=0 tx=0).stop 2024-03-18T22:59:09.080 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:08.196+0000 7fbae864b700 1 -- 172.21.15.49:0/3694715554 shutdown_connections 2024-03-18T22:59:09.080 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:08.196+0000 7fbae864b700 1 --2- 172.21.15.49:0/3694715554 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbae01087a0 0x7fbae010ab90 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T22:59:09.080 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:08.196+0000 7fbae864b700 1 -- 172.21.15.49:0/3694715554 >> 172.21.15.49:0/3694715554 conn(0x7fbae01021f0 msgr2=0x7fbae0104610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T22:59:09.080 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:08.196+0000 7fbae864b700 1 -- 172.21.15.49:0/3694715554 shutdown_connections 2024-03-18T22:59:09.080 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:08.196+0000 7fbae864b700 1 -- 172.21.15.49:0/3694715554 wait complete. 2024-03-18T22:59:09.081 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:08.196+0000 7fbae864b700 1 Processor -- start 2024-03-18T22:59:09.081 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:08.196+0000 7fbae864b700 1 -- start start 2024-03-18T22:59:09.081 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:08.197+0000 7fbae864b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbae01087a0 0x7fbae019bb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T22:59:09.081 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:08.197+0000 7fbae864b700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fbae010c130 con 0x7fbae01087a0 2024-03-18T22:59:09.081 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:08.197+0000 7fbae63e7700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbae01087a0 0x7fbae019bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T22:59:09.081 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:08.197+0000 7fbae63e7700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbae01087a0 0x7fbae019bb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:44532/0 (socket says 172.21.15.49:44532) 2024-03-18T22:59:09.081 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:08.197+0000 7fbae63e7700 1 -- 172.21.15.49:0/3289744852 learned_addr learned my addr 172.21.15.49:0/3289744852 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T22:59:09.082 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:08.197+0000 7fbae63e7700 1 -- 172.21.15.49:0/3289744852 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbae019c090 con 0x7fbae01087a0 2024-03-18T22:59:09.082 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:08.197+0000 7fbae63e7700 1 --2- 172.21.15.49:0/3289744852 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbae01087a0 0x7fbae019bb50 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fbadc0117e0 tx=0x7fbadc011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T22:59:09.082 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:08.197+0000 7fbad77fe700 1 -- 172.21.15.49:0/3289744852 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbadc003bf0 con 0x7fbae01087a0 2024-03-18T22:59:09.082 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:08.197+0000 7fbae864b700 1 -- 172.21.15.49:0/3289744852 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbae019c380 con 0x7fbae01087a0 2024-03-18T22:59:09.082 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:08.197+0000 7fbad77fe700 1 -- 172.21.15.49:0/3289744852 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbadc01a910 con 0x7fbae01087a0 2024-03-18T22:59:09.082 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:08.197+0000 7fbad77fe700 1 -- 172.21.15.49:0/3289744852 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbadc024970 con 0x7fbae01087a0 2024-03-18T22:59:09.082 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:08.197+0000 7fbae864b700 1 -- 172.21.15.49:0/3289744852 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbae019f010 con 0x7fbae01087a0 2024-03-18T22:59:09.083 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:08.198+0000 7fbad77fe700 1 -- 172.21.15.49:0/3289744852 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 13) ==== 46162+0+0 (secure 0 0 0) 0x7fbadc022020 con 0x7fbae01087a0 2024-03-18T22:59:09.083 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:08.198+0000 7fbae864b700 1 -- 172.21.15.49:0/3289744852 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbae010c2b0 con 0x7fbae01087a0 2024-03-18T22:59:09.083 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:08.198+0000 7fbad77fe700 1 --2- 172.21.15.49:0/3289744852 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fbacc038fc0 0x7fbacc03b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T22:59:09.083 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:08.199+0000 7fbad77fe700 1 -- 172.21.15.49:0/3289744852 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fbadc04d4a0 con 0x7fbae01087a0 2024-03-18T22:59:09.083 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:08.199+0000 7fbae5be6700 1 --2- 172.21.15.49:0/3289744852 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fbacc038fc0 0x7fbacc03b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T22:59:09.083 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:08.199+0000 7fbae5be6700 1 --2- 172.21.15.49:0/3289744852 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fbacc038fc0 0x7fbacc03b480 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fbad0006fd0 tx=0x7fbad0006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T22:59:09.083 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:08.203+0000 7fbad77fe700 1 -- 172.21.15.49:0/3289744852 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fbadc011e20 con 0x7fbae01087a0 2024-03-18T22:59:09.083 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:08.521+0000 7fbae864b700 1 -- 172.21.15.49:0/3289744852 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7fbae019f1e0 con 0x7fbae01087a0 2024-03-18T22:59:09.083 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:08.524+0000 7fbad77fe700 1 -- 172.21.15.49:0/3289744852 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) ==== 153+0+0 (secure 0 0 0) 0x7fbadc024cd0 con 0x7fbae01087a0 2024-03-18T22:59:09.083 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:08.526+0000 7fbae864b700 1 -- 172.21.15.49:0/3289744852 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fbacc038fc0 msgr2=0x7fbacc03b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T22:59:09.083 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:08.526+0000 7fbae864b700 1 --2- 172.21.15.49:0/3289744852 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fbacc038fc0 0x7fbacc03b480 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fbad0006fd0 tx=0x7fbad0006e40 comp rx=0 tx=0).stop 2024-03-18T22:59:09.083 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:08.526+0000 7fbae864b700 1 -- 172.21.15.49:0/3289744852 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbae01087a0 msgr2=0x7fbae019bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T22:59:09.083 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:08.526+0000 7fbae864b700 1 --2- 172.21.15.49:0/3289744852 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbae01087a0 0x7fbae019bb50 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fbadc0117e0 tx=0x7fbadc011c10 comp rx=0 tx=0).stop 2024-03-18T22:59:09.084 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:08.526+0000 7fbae864b700 1 -- 172.21.15.49:0/3289744852 shutdown_connections 2024-03-18T22:59:09.084 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:08.526+0000 7fbae864b700 1 --2- 172.21.15.49:0/3289744852 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fbacc038fc0 0x7fbacc03b480 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T22:59:09.084 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:08.526+0000 7fbae864b700 1 --2- 172.21.15.49:0/3289744852 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbae01087a0 0x7fbae019bb50 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T22:59:09.084 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:08.526+0000 7fbae864b700 1 -- 172.21.15.49:0/3289744852 >> 172.21.15.49:0/3289744852 conn(0x7fbae01021f0 msgr2=0x7fbae0102c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T22:59:09.084 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:08.526+0000 7fbae864b700 1 -- 172.21.15.49:0/3289744852 shutdown_connections 2024-03-18T22:59:09.084 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:08.526+0000 7fbae864b700 1 -- 172.21.15.49:0/3289744852 wait complete. 2024-03-18T22:59:09.084 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-03-18T22:59:09.084 INFO:teuthology.orchestra.run.smithi049.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-03-18T22:59:09.084 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-18T22:59:09.084 INFO:teuthology.orchestra.run.smithi049.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid c63591ea-e57a-11ee-95c9-87774f69a715 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-03-18T22:59:09.084 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-18T22:59:09.085 INFO:teuthology.orchestra.run.smithi049.stdout:Or, if you are only running a single cluster on this host: 2024-03-18T22:59:09.085 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-18T22:59:09.085 INFO:teuthology.orchestra.run.smithi049.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-03-18T22:59:09.085 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-18T22:59:09.085 INFO:teuthology.orchestra.run.smithi049.stdout:Please consider enabling telemetry to help improve Ceph: 2024-03-18T22:59:09.085 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-18T22:59:09.085 INFO:teuthology.orchestra.run.smithi049.stdout: ceph telemetry on 2024-03-18T22:59:09.085 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-18T22:59:09.085 INFO:teuthology.orchestra.run.smithi049.stdout:For more information see: 2024-03-18T22:59:09.085 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-18T22:59:09.085 INFO:teuthology.orchestra.run.smithi049.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-03-18T22:59:09.085 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-18T22:59:09.085 INFO:teuthology.orchestra.run.smithi049.stdout:Bootstrap complete. 2024-03-18T22:59:09.126 INFO:tasks.cephadm:Fetching config... 2024-03-18T22:59:09.126 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-03-18T22:59:09.126 DEBUG:teuthology.orchestra.run.smithi049:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-03-18T22:59:09.143 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-03-18T22:59:09.143 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-03-18T22:59:09.143 DEBUG:teuthology.orchestra.run.smithi049:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-03-18T22:59:09.200 INFO:tasks.cephadm:Fetching mon keyring... 2024-03-18T22:59:09.200 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-03-18T22:59:09.200 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd if=/var/lib/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/mon.a/keyring of=/dev/stdout 2024-03-18T22:59:09.270 INFO:tasks.cephadm:Fetching pub ssh key... 2024-03-18T22:59:09.271 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-03-18T22:59:09.271 DEBUG:teuthology.orchestra.run.smithi049:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-03-18T22:59:09.329 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-03-18T22:59:09.329 DEBUG:teuthology.orchestra.run.smithi049:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDFpin9GyaS7qP32c7VN4lup5pyMnlDKhNZSF3gxOpJLtF0Vxd74+aOsJoSxTomncBbbHzqPkHBIM/nivB+XmivxjKNP1s5gj1xM5Cnky4TIPkZRKcIa07FNyeYsRwCf4pks2dkeroBX7FuBmBKSp6LisGKZoAcVag+M98ahhtkVeRiDsyZia3KbFf89x0+NF6tpiJy2ACfSaRDNlXKZ8ECKawtLfaF7bj0knvGxguSWZ2toyuNiiG3vmsxTQg7K0gfkwTYblwafNw4mDXa9botVcFwFYtSQ48fYUkEblt8sD3KRFtIvbJleO0W1Y/sf6l0O971vu8sCSNNVaI4zrCwWvTe5DuBiuNwemYXdUaT4UR0/JlAVKzGG32TFpdnVFWg8NKyCOSLlj/4GKTsnwTFnHJRxoI/OuaYbVdWuRsLrBqiHuDOBYcsiylw6+lTLtbLEQf4QPfeWvKe0ZEreFsntcUAoBeRtHoU2FoJBTMhBjbtjFopqP26T7sFSRUSw40= ceph-c63591ea-e57a-11ee-95c9-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-03-18T22:59:09.415 INFO:teuthology.orchestra.run.smithi049.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDFpin9GyaS7qP32c7VN4lup5pyMnlDKhNZSF3gxOpJLtF0Vxd74+aOsJoSxTomncBbbHzqPkHBIM/nivB+XmivxjKNP1s5gj1xM5Cnky4TIPkZRKcIa07FNyeYsRwCf4pks2dkeroBX7FuBmBKSp6LisGKZoAcVag+M98ahhtkVeRiDsyZia3KbFf89x0+NF6tpiJy2ACfSaRDNlXKZ8ECKawtLfaF7bj0knvGxguSWZ2toyuNiiG3vmsxTQg7K0gfkwTYblwafNw4mDXa9botVcFwFYtSQ48fYUkEblt8sD3KRFtIvbJleO0W1Y/sf6l0O971vu8sCSNNVaI4zrCwWvTe5DuBiuNwemYXdUaT4UR0/JlAVKzGG32TFpdnVFWg8NKyCOSLlj/4GKTsnwTFnHJRxoI/OuaYbVdWuRsLrBqiHuDOBYcsiylw6+lTLtbLEQf4QPfeWvKe0ZEreFsntcUAoBeRtHoU2FoJBTMhBjbtjFopqP26T7sFSRUSw40= ceph-c63591ea-e57a-11ee-95c9-87774f69a715 2024-03-18T22:59:09.433 DEBUG:teuthology.orchestra.run.smithi139:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDFpin9GyaS7qP32c7VN4lup5pyMnlDKhNZSF3gxOpJLtF0Vxd74+aOsJoSxTomncBbbHzqPkHBIM/nivB+XmivxjKNP1s5gj1xM5Cnky4TIPkZRKcIa07FNyeYsRwCf4pks2dkeroBX7FuBmBKSp6LisGKZoAcVag+M98ahhtkVeRiDsyZia3KbFf89x0+NF6tpiJy2ACfSaRDNlXKZ8ECKawtLfaF7bj0knvGxguSWZ2toyuNiiG3vmsxTQg7K0gfkwTYblwafNw4mDXa9botVcFwFYtSQ48fYUkEblt8sD3KRFtIvbJleO0W1Y/sf6l0O971vu8sCSNNVaI4zrCwWvTe5DuBiuNwemYXdUaT4UR0/JlAVKzGG32TFpdnVFWg8NKyCOSLlj/4GKTsnwTFnHJRxoI/OuaYbVdWuRsLrBqiHuDOBYcsiylw6+lTLtbLEQf4QPfeWvKe0ZEreFsntcUAoBeRtHoU2FoJBTMhBjbtjFopqP26T7sFSRUSw40= ceph-c63591ea-e57a-11ee-95c9-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-03-18T22:59:09.478 INFO:teuthology.orchestra.run.smithi139.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDFpin9GyaS7qP32c7VN4lup5pyMnlDKhNZSF3gxOpJLtF0Vxd74+aOsJoSxTomncBbbHzqPkHBIM/nivB+XmivxjKNP1s5gj1xM5Cnky4TIPkZRKcIa07FNyeYsRwCf4pks2dkeroBX7FuBmBKSp6LisGKZoAcVag+M98ahhtkVeRiDsyZia3KbFf89x0+NF6tpiJy2ACfSaRDNlXKZ8ECKawtLfaF7bj0knvGxguSWZ2toyuNiiG3vmsxTQg7K0gfkwTYblwafNw4mDXa9botVcFwFYtSQ48fYUkEblt8sD3KRFtIvbJleO0W1Y/sf6l0O971vu8sCSNNVaI4zrCwWvTe5DuBiuNwemYXdUaT4UR0/JlAVKzGG32TFpdnVFWg8NKyCOSLlj/4GKTsnwTFnHJRxoI/OuaYbVdWuRsLrBqiHuDOBYcsiylw6+lTLtbLEQf4QPfeWvKe0ZEreFsntcUAoBeRtHoU2FoJBTMhBjbtjFopqP26T7sFSRUSw40= ceph-c63591ea-e57a-11ee-95c9-87774f69a715 2024-03-18T22:59:09.493 DEBUG:teuthology.orchestra.run.smithi175:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDFpin9GyaS7qP32c7VN4lup5pyMnlDKhNZSF3gxOpJLtF0Vxd74+aOsJoSxTomncBbbHzqPkHBIM/nivB+XmivxjKNP1s5gj1xM5Cnky4TIPkZRKcIa07FNyeYsRwCf4pks2dkeroBX7FuBmBKSp6LisGKZoAcVag+M98ahhtkVeRiDsyZia3KbFf89x0+NF6tpiJy2ACfSaRDNlXKZ8ECKawtLfaF7bj0knvGxguSWZ2toyuNiiG3vmsxTQg7K0gfkwTYblwafNw4mDXa9botVcFwFYtSQ48fYUkEblt8sD3KRFtIvbJleO0W1Y/sf6l0O971vu8sCSNNVaI4zrCwWvTe5DuBiuNwemYXdUaT4UR0/JlAVKzGG32TFpdnVFWg8NKyCOSLlj/4GKTsnwTFnHJRxoI/OuaYbVdWuRsLrBqiHuDOBYcsiylw6+lTLtbLEQf4QPfeWvKe0ZEreFsntcUAoBeRtHoU2FoJBTMhBjbtjFopqP26T7sFSRUSw40= ceph-c63591ea-e57a-11ee-95c9-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-03-18T22:59:09.538 INFO:teuthology.orchestra.run.smithi175.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDFpin9GyaS7qP32c7VN4lup5pyMnlDKhNZSF3gxOpJLtF0Vxd74+aOsJoSxTomncBbbHzqPkHBIM/nivB+XmivxjKNP1s5gj1xM5Cnky4TIPkZRKcIa07FNyeYsRwCf4pks2dkeroBX7FuBmBKSp6LisGKZoAcVag+M98ahhtkVeRiDsyZia3KbFf89x0+NF6tpiJy2ACfSaRDNlXKZ8ECKawtLfaF7bj0knvGxguSWZ2toyuNiiG3vmsxTQg7K0gfkwTYblwafNw4mDXa9botVcFwFYtSQ48fYUkEblt8sD3KRFtIvbJleO0W1Y/sf6l0O971vu8sCSNNVaI4zrCwWvTe5DuBiuNwemYXdUaT4UR0/JlAVKzGG32TFpdnVFWg8NKyCOSLlj/4GKTsnwTFnHJRxoI/OuaYbVdWuRsLrBqiHuDOBYcsiylw6+lTLtbLEQf4QPfeWvKe0ZEreFsntcUAoBeRtHoU2FoJBTMhBjbtjFopqP26T7sFSRUSw40= ceph-c63591ea-e57a-11ee-95c9-87774f69a715 2024-03-18T22:59:09.553 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c63591ea-e57a-11ee-95c9-87774f69a715 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-03-18T22:59:09.690 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:59:09 smithi049 ceph-mon[62991]: from='client.? 172.21.15.49:0/3289744852' entity='client.admin' 2024-03-18T22:59:09.802 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T22:59:12.187 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:12.186+0000 7fa74644a700 1 -- 172.21.15.49:0/1503103090 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa740101d80 msgr2=0x7fa7401021a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T22:59:12.187 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:12.186+0000 7fa74644a700 1 --2- 172.21.15.49:0/1503103090 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa740101d80 0x7fa7401021a0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fa730009910 tx=0x7fa730009c20 comp rx=0 tx=0).stop 2024-03-18T22:59:12.188 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:12.186+0000 7fa74644a700 1 -- 172.21.15.49:0/1503103090 shutdown_connections 2024-03-18T22:59:12.188 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:12.186+0000 7fa74644a700 1 --2- 172.21.15.49:0/1503103090 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa740101d80 0x7fa7401021a0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T22:59:12.188 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:12.186+0000 7fa74644a700 1 -- 172.21.15.49:0/1503103090 >> 172.21.15.49:0/1503103090 conn(0x7fa7400fd300 msgr2=0x7fa7400ff760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T22:59:12.188 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:12.186+0000 7fa74644a700 1 -- 172.21.15.49:0/1503103090 shutdown_connections 2024-03-18T22:59:12.188 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:12.186+0000 7fa74644a700 1 -- 172.21.15.49:0/1503103090 wait complete. 2024-03-18T22:59:12.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:12.186+0000 7fa74644a700 1 Processor -- start 2024-03-18T22:59:12.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:12.186+0000 7fa74644a700 1 -- start start 2024-03-18T22:59:12.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:12.187+0000 7fa74644a700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa740101d80 0x7fa740199650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T22:59:12.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:12.187+0000 7fa74644a700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fa7400761b0 con 0x7fa740101d80 2024-03-18T22:59:12.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:12.187+0000 7fa73ffff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa740101d80 0x7fa740199650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T22:59:12.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:12.187+0000 7fa73ffff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa740101d80 0x7fa740199650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:44554/0 (socket says 172.21.15.49:44554) 2024-03-18T22:59:12.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:12.187+0000 7fa73ffff700 1 -- 172.21.15.49:0/644548935 learned_addr learned my addr 172.21.15.49:0/644548935 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T22:59:12.190 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:12.187+0000 7fa73ffff700 1 -- 172.21.15.49:0/644548935 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa740199b90 con 0x7fa740101d80 2024-03-18T22:59:12.190 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:12.187+0000 7fa73ffff700 1 --2- 172.21.15.49:0/644548935 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa740101d80 0x7fa740199650 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fa730006010 tx=0x7fa730005050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T22:59:12.190 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:12.187+0000 7fa73dffb700 1 -- 172.21.15.49:0/644548935 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa730027070 con 0x7fa740101d80 2024-03-18T22:59:12.190 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:12.187+0000 7fa73dffb700 1 -- 172.21.15.49:0/644548935 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa73000be40 con 0x7fa740101d80 2024-03-18T22:59:12.190 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:12.187+0000 7fa74644a700 1 -- 172.21.15.49:0/644548935 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa740199e20 con 0x7fa740101d80 2024-03-18T22:59:12.190 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:12.187+0000 7fa73dffb700 1 -- 172.21.15.49:0/644548935 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa73000f460 con 0x7fa740101d80 2024-03-18T22:59:12.190 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:12.187+0000 7fa74644a700 1 -- 172.21.15.49:0/644548935 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa74019cb10 con 0x7fa740101d80 2024-03-18T22:59:12.191 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:12.188+0000 7fa73dffb700 1 -- 172.21.15.49:0/644548935 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 13) ==== 46162+0+0 (secure 0 0 0) 0x7fa73002a070 con 0x7fa740101d80 2024-03-18T22:59:12.191 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:12.188+0000 7fa74644a700 1 -- 172.21.15.49:0/644548935 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa74010a800 con 0x7fa740101d80 2024-03-18T22:59:12.191 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:12.188+0000 7fa73dffb700 1 --2- 172.21.15.49:0/644548935 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fa72c038fc0 0x7fa72c03b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T22:59:12.191 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:12.188+0000 7fa73dffb700 1 -- 172.21.15.49:0/644548935 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fa73001e030 con 0x7fa740101d80 2024-03-18T22:59:12.192 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:12.189+0000 7fa737fff700 1 --2- 172.21.15.49:0/644548935 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fa72c038fc0 0x7fa72c03b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T22:59:12.192 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:12.189+0000 7fa737fff700 1 --2- 172.21.15.49:0/644548935 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fa72c038fc0 0x7fa72c03b480 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fa728006fd0 tx=0x7fa728006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T22:59:12.193 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:12.193+0000 7fa73dffb700 1 -- 172.21.15.49:0/644548935 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fa730013070 con 0x7fa740101d80 2024-03-18T22:59:12.411 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:12.411+0000 7fa74644a700 1 -- 172.21.15.49:0/644548935 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7fa74002dcc0 con 0x7fa740101d80 2024-03-18T22:59:12.412 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:59:12 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T22:59:12.412 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:59:12 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T22:59:12.412 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:59:12 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi049", "name": "osd_memory_target"}]: dispatch 2024-03-18T22:59:12.412 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:59:12 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T22:59:12.412 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:59:12 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T22:59:12.412 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:59:12 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T22:59:12.414 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:12.414+0000 7fa73dffb700 1 -- 172.21.15.49:0/644548935 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7fa7300310c0 con 0x7fa740101d80 2024-03-18T22:59:12.417 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:12.416+0000 7fa74644a700 1 -- 172.21.15.49:0/644548935 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fa72c038fc0 msgr2=0x7fa72c03b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T22:59:12.417 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:12.416+0000 7fa74644a700 1 --2- 172.21.15.49:0/644548935 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fa72c038fc0 0x7fa72c03b480 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fa728006fd0 tx=0x7fa728006e40 comp rx=0 tx=0).stop 2024-03-18T22:59:12.417 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:12.416+0000 7fa74644a700 1 -- 172.21.15.49:0/644548935 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa740101d80 msgr2=0x7fa740199650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T22:59:12.417 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:12.416+0000 7fa74644a700 1 --2- 172.21.15.49:0/644548935 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa740101d80 0x7fa740199650 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fa730006010 tx=0x7fa730005050 comp rx=0 tx=0).stop 2024-03-18T22:59:12.417 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:12.416+0000 7fa74644a700 1 -- 172.21.15.49:0/644548935 shutdown_connections 2024-03-18T22:59:12.417 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:12.416+0000 7fa74644a700 1 --2- 172.21.15.49:0/644548935 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fa72c038fc0 0x7fa72c03b480 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T22:59:12.417 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:12.416+0000 7fa74644a700 1 --2- 172.21.15.49:0/644548935 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa740101d80 0x7fa740199650 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T22:59:12.417 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:12.416+0000 7fa74644a700 1 -- 172.21.15.49:0/644548935 >> 172.21.15.49:0/644548935 conn(0x7fa7400fd300 msgr2=0x7fa7400fdcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T22:59:12.417 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:12.417+0000 7fa74644a700 1 -- 172.21.15.49:0/644548935 shutdown_connections 2024-03-18T22:59:12.417 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:12.417+0000 7fa74644a700 1 -- 172.21.15.49:0/644548935 wait complete. 2024-03-18T22:59:13.173 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-03-18T22:59:13.173 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c63591ea-e57a-11ee-95c9-87774f69a715 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-03-18T22:59:13.840 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:59:13 smithi049 ceph-mon[62991]: from='client.? 172.21.15.49:0/644548935' entity='client.admin' 2024-03-18T22:59:13.841 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:59:13 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T22:59:13.841 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:59:13 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T22:59:13.944 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T22:59:15.482 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:15.481+0000 7fe573098700 1 -- 172.21.15.49:0/1206261224 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe56c1028b0 msgr2=0x7fe56c104ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T22:59:15.482 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:15.481+0000 7fe573098700 1 --2- 172.21.15.49:0/1206261224 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe56c1028b0 0x7fe56c104ca0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fe55c00b0a0 tx=0x7fe55c00b3b0 comp rx=0 tx=0).stop 2024-03-18T22:59:15.482 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:15.481+0000 7fe573098700 1 -- 172.21.15.49:0/1206261224 shutdown_connections 2024-03-18T22:59:15.482 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:15.481+0000 7fe573098700 1 --2- 172.21.15.49:0/1206261224 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe56c1028b0 0x7fe56c104ca0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T22:59:15.482 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:15.481+0000 7fe573098700 1 -- 172.21.15.49:0/1206261224 >> 172.21.15.49:0/1206261224 conn(0x7fe56c0fc200 msgr2=0x7fe56c0fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T22:59:15.483 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:15.481+0000 7fe573098700 1 -- 172.21.15.49:0/1206261224 shutdown_connections 2024-03-18T22:59:15.483 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:15.481+0000 7fe573098700 1 -- 172.21.15.49:0/1206261224 wait complete. 2024-03-18T22:59:15.483 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:15.482+0000 7fe573098700 1 Processor -- start 2024-03-18T22:59:15.483 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:15.482+0000 7fe573098700 1 -- start start 2024-03-18T22:59:15.483 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:15.482+0000 7fe573098700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe56c1028b0 0x7fe56c197390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T22:59:15.483 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:15.482+0000 7fe573098700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fe56c107df0 con 0x7fe56c1028b0 2024-03-18T22:59:15.483 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:15.482+0000 7fe570e34700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe56c1028b0 0x7fe56c197390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T22:59:15.483 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:15.482+0000 7fe570e34700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe56c1028b0 0x7fe56c197390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53846/0 (socket says 172.21.15.49:53846) 2024-03-18T22:59:15.483 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:15.482+0000 7fe570e34700 1 -- 172.21.15.49:0/3269882929 learned_addr learned my addr 172.21.15.49:0/3269882929 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T22:59:15.485 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:15.483+0000 7fe570e34700 1 -- 172.21.15.49:0/3269882929 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe56c1978d0 con 0x7fe56c1028b0 2024-03-18T22:59:15.485 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:15.483+0000 7fe570e34700 1 --2- 172.21.15.49:0/3269882929 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe56c1028b0 0x7fe56c197390 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fe55c007890 tx=0x7fe55c00bd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T22:59:15.485 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:15.483+0000 7fe56a7fc700 1 -- 172.21.15.49:0/3269882929 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe55c017040 con 0x7fe56c1028b0 2024-03-18T22:59:15.485 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:15.483+0000 7fe56a7fc700 1 -- 172.21.15.49:0/3269882929 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe55c022070 con 0x7fe56c1028b0 2024-03-18T22:59:15.485 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:15.483+0000 7fe56a7fc700 1 -- 172.21.15.49:0/3269882929 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe55c01d5b0 con 0x7fe56c1028b0 2024-03-18T22:59:15.485 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:15.483+0000 7fe573098700 1 -- 172.21.15.49:0/3269882929 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe56c197b60 con 0x7fe56c1028b0 2024-03-18T22:59:15.485 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:15.483+0000 7fe573098700 1 -- 172.21.15.49:0/3269882929 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe56c19a850 con 0x7fe56c1028b0 2024-03-18T22:59:15.485 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:15.484+0000 7fe56a7fc700 1 -- 172.21.15.49:0/3269882929 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 13) ==== 46162+0+0 (secure 0 0 0) 0x7fe55c004000 con 0x7fe56c1028b0 2024-03-18T22:59:15.485 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:15.484+0000 7fe573098700 1 -- 172.21.15.49:0/3269882929 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe56c108570 con 0x7fe56c1028b0 2024-03-18T22:59:15.485 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:15.484+0000 7fe56a7fc700 1 --2- 172.21.15.49:0/3269882929 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fe554039000 0x7fe55403b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T22:59:15.485 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:15.484+0000 7fe56a7fc700 1 -- 172.21.15.49:0/3269882929 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fe55c00f050 con 0x7fe56c1028b0 2024-03-18T22:59:15.485 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:15.484+0000 7fe55bfff700 1 --2- 172.21.15.49:0/3269882929 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fe554039000 0x7fe55403b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T22:59:15.485 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:15.485+0000 7fe55bfff700 1 --2- 172.21.15.49:0/3269882929 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fe554039000 0x7fe55403b4c0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fe560006fd0 tx=0x7fe560006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T22:59:15.489 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:15.489+0000 7fe56a7fc700 1 -- 172.21.15.49:0/3269882929 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fe55c007e20 con 0x7fe56c1028b0 2024-03-18T22:59:15.697 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:15.697+0000 7fe573098700 1 -- 172.21.15.49:0/3269882929 --> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7fe56c02dcc0 con 0x7fe554039000 2024-03-18T22:59:15.700 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:15.700+0000 7fe56a7fc700 1 -- 172.21.15.49:0/3269882929 <== mgr.14150 v2:172.21.15.49:6800/2083805211 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fe56c02dcc0 con 0x7fe554039000 2024-03-18T22:59:15.702 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:15.702+0000 7fe573098700 1 -- 172.21.15.49:0/3269882929 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fe554039000 msgr2=0x7fe55403b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T22:59:15.702 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:15.702+0000 7fe573098700 1 --2- 172.21.15.49:0/3269882929 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fe554039000 0x7fe55403b4c0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fe560006fd0 tx=0x7fe560006e40 comp rx=0 tx=0).stop 2024-03-18T22:59:15.703 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:15.702+0000 7fe573098700 1 -- 172.21.15.49:0/3269882929 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe56c1028b0 msgr2=0x7fe56c197390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T22:59:15.703 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:15.702+0000 7fe573098700 1 --2- 172.21.15.49:0/3269882929 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe56c1028b0 0x7fe56c197390 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fe55c007890 tx=0x7fe55c00bd40 comp rx=0 tx=0).stop 2024-03-18T22:59:15.703 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:15.702+0000 7fe573098700 1 -- 172.21.15.49:0/3269882929 shutdown_connections 2024-03-18T22:59:15.703 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:15.702+0000 7fe573098700 1 --2- 172.21.15.49:0/3269882929 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fe554039000 0x7fe55403b4c0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T22:59:15.703 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:15.702+0000 7fe573098700 1 --2- 172.21.15.49:0/3269882929 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe56c1028b0 0x7fe56c197390 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T22:59:15.703 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:15.702+0000 7fe573098700 1 -- 172.21.15.49:0/3269882929 >> 172.21.15.49:0/3269882929 conn(0x7fe56c0fc200 msgr2=0x7fe56c0fcbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T22:59:15.703 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:15.702+0000 7fe573098700 1 -- 172.21.15.49:0/3269882929 shutdown_connections 2024-03-18T22:59:15.703 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:15.702+0000 7fe573098700 1 -- 172.21.15.49:0/3269882929 wait complete. 2024-03-18T22:59:16.577 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi139 2024-03-18T22:59:16.578 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-03-18T22:59:16.578 DEBUG:teuthology.orchestra.run.smithi139:> dd of=/etc/ceph/ceph.conf 2024-03-18T22:59:16.597 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-03-18T22:59:16.597 DEBUG:teuthology.orchestra.run.smithi139:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-18T22:59:16.655 INFO:tasks.cephadm:Adding host smithi139 to orchestrator... 2024-03-18T22:59:16.655 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c63591ea-e57a-11ee-95c9-87774f69a715 -- ceph orch host add smithi139 2024-03-18T22:59:16.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:59:16 smithi049 ceph-mon[62991]: from='client.14174 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T22:59:16.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:59:16 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T22:59:16.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:59:16 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T22:59:16.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:59:16 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T22:59:16.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:59:16 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T22:59:16.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:59:16 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T22:59:16.881 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:59:16 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T22:59:16.881 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:59:16 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T22:59:16.900 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T22:59:17.995 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:59:17 smithi049 ceph-mon[62991]: Updating smithi049:/etc/ceph/ceph.conf 2024-03-18T22:59:17.995 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:59:17 smithi049 ceph-mon[62991]: Updating smithi049:/var/lib/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-18T22:59:17.995 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:59:17 smithi049 ceph-mon[62991]: Updating smithi049:/etc/ceph/ceph.client.admin.keyring 2024-03-18T22:59:17.995 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:59:17 smithi049 ceph-mon[62991]: Updating smithi049:/var/lib/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/config/ceph.client.admin.keyring 2024-03-18T22:59:18.412 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:18.411+0000 7f73aaeb6700 1 -- 172.21.15.49:0/2619284027 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f73a4100e40 msgr2=0x7f73a4101220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T22:59:18.412 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:18.411+0000 7f73aaeb6700 1 --2- 172.21.15.49:0/2619284027 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f73a4100e40 0x7f73a4101220 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f7394009910 tx=0x7f7394009c20 comp rx=0 tx=0).stop 2024-03-18T22:59:18.412 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:18.411+0000 7f73aaeb6700 1 -- 172.21.15.49:0/2619284027 shutdown_connections 2024-03-18T22:59:18.412 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:18.411+0000 7f73aaeb6700 1 --2- 172.21.15.49:0/2619284027 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f73a4100e40 0x7f73a4101220 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T22:59:18.412 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:18.411+0000 7f73aaeb6700 1 -- 172.21.15.49:0/2619284027 >> 172.21.15.49:0/2619284027 conn(0x7f73a4077850 msgr2=0x7f73a40feb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T22:59:18.412 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:18.411+0000 7f73aaeb6700 1 -- 172.21.15.49:0/2619284027 shutdown_connections 2024-03-18T22:59:18.412 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:18.411+0000 7f73aaeb6700 1 -- 172.21.15.49:0/2619284027 wait complete. 2024-03-18T22:59:18.412 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:18.412+0000 7f73aaeb6700 1 Processor -- start 2024-03-18T22:59:18.413 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:18.412+0000 7f73aaeb6700 1 -- start start 2024-03-18T22:59:18.413 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:18.412+0000 7f73aaeb6700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f73a4100e40 0x7f73a41964b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T22:59:18.413 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:18.412+0000 7f73aaeb6700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f73a410b210 con 0x7f73a4100e40 2024-03-18T22:59:18.414 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:18.412+0000 7f73a8c52700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f73a4100e40 0x7f73a41964b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T22:59:18.414 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:18.412+0000 7f73a8c52700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f73a4100e40 0x7f73a41964b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53860/0 (socket says 172.21.15.49:53860) 2024-03-18T22:59:18.414 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:18.412+0000 7f73a8c52700 1 -- 172.21.15.49:0/3978002703 learned_addr learned my addr 172.21.15.49:0/3978002703 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T22:59:18.414 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:18.413+0000 7f73a8c52700 1 -- 172.21.15.49:0/3978002703 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73a406a560 con 0x7f73a4100e40 2024-03-18T22:59:18.414 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:18.413+0000 7f73a8c52700 1 --2- 172.21.15.49:0/3978002703 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f73a4100e40 0x7f73a41964b0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f7394005850 tx=0x7f7394005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T22:59:18.414 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:18.413+0000 7f73a1ffb700 1 -- 172.21.15.49:0/3978002703 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7394026070 con 0x7f73a4100e40 2024-03-18T22:59:18.414 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:18.413+0000 7f73a1ffb700 1 -- 172.21.15.49:0/3978002703 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f739400bc30 con 0x7f73a4100e40 2024-03-18T22:59:18.414 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:18.413+0000 7f73aaeb6700 1 -- 172.21.15.49:0/3978002703 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73a41969f0 con 0x7f73a4100e40 2024-03-18T22:59:18.415 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:18.413+0000 7f73a1ffb700 1 -- 172.21.15.49:0/3978002703 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7394003a20 con 0x7f73a4100e40 2024-03-18T22:59:18.415 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:18.413+0000 7f73aaeb6700 1 -- 172.21.15.49:0/3978002703 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73a4196eb0 con 0x7f73a4100e40 2024-03-18T22:59:18.415 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:18.414+0000 7f73a1ffb700 1 -- 172.21.15.49:0/3978002703 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 13) ==== 46162+0+0 (secure 0 0 0) 0x7f7394029070 con 0x7f73a4100e40 2024-03-18T22:59:18.415 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:18.414+0000 7f73aaeb6700 1 -- 172.21.15.49:0/3978002703 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73a4102320 con 0x7f73a4100e40 2024-03-18T22:59:18.416 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:18.414+0000 7f73a1ffb700 1 --2- 172.21.15.49:0/3978002703 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f738c039020 0x7f738c03b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T22:59:18.416 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:18.414+0000 7f73a1ffb700 1 -- 172.21.15.49:0/3978002703 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f739401e070 con 0x7f73a4100e40 2024-03-18T22:59:18.416 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:18.414+0000 7f73a3fff700 1 --2- 172.21.15.49:0/3978002703 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f738c039020 0x7f738c03b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T22:59:18.416 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:18.415+0000 7f73a3fff700 1 --2- 172.21.15.49:0/3978002703 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f738c039020 0x7f738c03b4e0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f7398006fd0 tx=0x7f7398006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T22:59:18.419 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:18.419+0000 7f73a1ffb700 1 -- 172.21.15.49:0/3978002703 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f7394013070 con 0x7f73a4100e40 2024-03-18T22:59:18.628 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:18.627+0000 7f73aaeb6700 1 -- 172.21.15.49:0/3978002703 --> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi139", "target": ["mon-mgr", ""]}) -- 0x7f73a41971a0 con 0x7f738c039020 2024-03-18T22:59:18.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:59:18 smithi049 ceph-mon[62991]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi139", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T22:59:20.130 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:59:19 smithi049 ceph-mon[62991]: Deploying cephadm binary to smithi139 2024-03-18T22:59:21.130 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:59:20 smithi049 ceph-mon[62991]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T22:59:22.957 INFO:teuthology.orchestra.run.smithi049.stdout:Added host 'smithi139' with addr '172.21.15.139' 2024-03-18T22:59:22.957 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:22.956+0000 7f73a1ffb700 1 -- 172.21.15.49:0/3978002703 <== mgr.14150 v2:172.21.15.49:6800/2083805211 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f73a41971a0 con 0x7f738c039020 2024-03-18T22:59:22.960 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:22.959+0000 7f73aaeb6700 1 -- 172.21.15.49:0/3978002703 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f738c039020 msgr2=0x7f738c03b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T22:59:22.960 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:22.959+0000 7f73aaeb6700 1 --2- 172.21.15.49:0/3978002703 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f738c039020 0x7f738c03b4e0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f7398006fd0 tx=0x7f7398006e40 comp rx=0 tx=0).stop 2024-03-18T22:59:22.960 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:22.959+0000 7f73aaeb6700 1 -- 172.21.15.49:0/3978002703 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f73a4100e40 msgr2=0x7f73a41964b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T22:59:22.960 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:22.959+0000 7f73aaeb6700 1 --2- 172.21.15.49:0/3978002703 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f73a4100e40 0x7f73a41964b0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f7394005850 tx=0x7f7394005030 comp rx=0 tx=0).stop 2024-03-18T22:59:22.960 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:22.959+0000 7f73aaeb6700 1 -- 172.21.15.49:0/3978002703 shutdown_connections 2024-03-18T22:59:22.960 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:22.959+0000 7f73aaeb6700 1 --2- 172.21.15.49:0/3978002703 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f738c039020 0x7f738c03b4e0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T22:59:22.960 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:22.959+0000 7f73aaeb6700 1 --2- 172.21.15.49:0/3978002703 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f73a4100e40 0x7f73a41964b0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T22:59:22.960 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:22.959+0000 7f73aaeb6700 1 -- 172.21.15.49:0/3978002703 >> 172.21.15.49:0/3978002703 conn(0x7f73a4077850 msgr2=0x7f73a4103c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T22:59:22.960 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:22.959+0000 7f73aaeb6700 1 -- 172.21.15.49:0/3978002703 shutdown_connections 2024-03-18T22:59:22.961 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:22.959+0000 7f73aaeb6700 1 -- 172.21.15.49:0/3978002703 wait complete. 2024-03-18T22:59:23.130 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:59:22 smithi049 ceph-mon[62991]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T22:59:23.492 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c63591ea-e57a-11ee-95c9-87774f69a715 -- ceph orch host ls --format=json 2024-03-18T22:59:23.744 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T22:59:24.130 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:59:23 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T22:59:24.130 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:59:23 smithi049 ceph-mon[62991]: Added host smithi139 2024-03-18T22:59:24.130 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:59:23 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T22:59:24.130 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:59:23 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T22:59:25.176 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:25.174+0000 7f3b84a44700 1 -- 172.21.15.49:0/2424265509 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3b80101f50 msgr2=0x7f3b80102330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T22:59:25.176 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:25.174+0000 7f3b84a44700 1 --2- 172.21.15.49:0/2424265509 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3b80101f50 0x7f3b80102330 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f3b70009910 tx=0x7f3b70009c20 comp rx=0 tx=0).stop 2024-03-18T22:59:25.176 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:25.174+0000 7f3b84a44700 1 -- 172.21.15.49:0/2424265509 shutdown_connections 2024-03-18T22:59:25.176 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:25.174+0000 7f3b84a44700 1 --2- 172.21.15.49:0/2424265509 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3b80101f50 0x7f3b80102330 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T22:59:25.176 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:25.174+0000 7f3b84a44700 1 -- 172.21.15.49:0/2424265509 >> 172.21.15.49:0/2424265509 conn(0x7f3b800fd820 msgr2=0x7f3b800ffc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T22:59:25.176 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:25.174+0000 7f3b84a44700 1 -- 172.21.15.49:0/2424265509 shutdown_connections 2024-03-18T22:59:25.177 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:25.174+0000 7f3b84a44700 1 -- 172.21.15.49:0/2424265509 wait complete. 2024-03-18T22:59:25.177 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:25.175+0000 7f3b84a44700 1 Processor -- start 2024-03-18T22:59:25.177 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:25.175+0000 7f3b84a44700 1 -- start start 2024-03-18T22:59:25.177 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:25.175+0000 7f3b84a44700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3b80101f50 0x7f3b8019d620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T22:59:25.177 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:25.175+0000 7f3b84a44700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f3b8010c320 con 0x7f3b80101f50 2024-03-18T22:59:25.177 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:25.175+0000 7f3b7e59c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3b80101f50 0x7f3b8019d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T22:59:25.177 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:25.175+0000 7f3b7e59c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3b80101f50 0x7f3b8019d620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:45010/0 (socket says 172.21.15.49:45010) 2024-03-18T22:59:25.177 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:25.175+0000 7f3b7e59c700 1 -- 172.21.15.49:0/3608140593 learned_addr learned my addr 172.21.15.49:0/3608140593 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T22:59:25.177 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:25.175+0000 7f3b7e59c700 1 -- 172.21.15.49:0/3608140593 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b8019db60 con 0x7f3b80101f50 2024-03-18T22:59:25.177 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:25.175+0000 7f3b7e59c700 1 --2- 172.21.15.49:0/3608140593 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3b80101f50 0x7f3b8019d620 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f3b70005850 tx=0x7f3b70005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T22:59:25.178 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:25.175+0000 7f3b6f7fe700 1 -- 172.21.15.49:0/3608140593 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3b70026070 con 0x7f3b80101f50 2024-03-18T22:59:25.178 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:25.176+0000 7f3b6f7fe700 1 -- 172.21.15.49:0/3608140593 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3b7000bbd0 con 0x7f3b80101f50 2024-03-18T22:59:25.179 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:25.176+0000 7f3b84a44700 1 -- 172.21.15.49:0/3608140593 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b8019de50 con 0x7f3b80101f50 2024-03-18T22:59:25.179 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:25.176+0000 7f3b6f7fe700 1 -- 172.21.15.49:0/3608140593 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3b70003970 con 0x7f3b80101f50 2024-03-18T22:59:25.179 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:25.176+0000 7f3b84a44700 1 -- 172.21.15.49:0/3608140593 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b801978c0 con 0x7f3b80101f50 2024-03-18T22:59:25.179 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:25.176+0000 7f3b6f7fe700 1 -- 172.21.15.49:0/3608140593 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 13) ==== 46162+0+0 (secure 0 0 0) 0x7f3b70029070 con 0x7f3b80101f50 2024-03-18T22:59:25.179 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:25.177+0000 7f3b6f7fe700 1 --2- 172.21.15.49:0/3608140593 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f3b68038fc0 0x7f3b6803b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T22:59:25.179 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:25.177+0000 7f3b6f7fe700 1 -- 172.21.15.49:0/3608140593 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f3b7001e070 con 0x7f3b80101f50 2024-03-18T22:59:25.179 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:25.177+0000 7f3b7dd9b700 1 --2- 172.21.15.49:0/3608140593 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f3b68038fc0 0x7f3b6803b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T22:59:25.179 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:25.177+0000 7f3b84a44700 1 -- 172.21.15.49:0/3608140593 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b80103430 con 0x7f3b80101f50 2024-03-18T22:59:25.179 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:25.177+0000 7f3b7dd9b700 1 --2- 172.21.15.49:0/3608140593 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f3b68038fc0 0x7f3b6803b480 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f3b74006fd0 tx=0x7f3b74006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T22:59:25.183 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:25.182+0000 7f3b6f7fe700 1 -- 172.21.15.49:0/3608140593 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f3b70030070 con 0x7f3b80101f50 2024-03-18T22:59:25.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:59:24 smithi049 ceph-mon[62991]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T22:59:25.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:59:24 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T22:59:25.395 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:25.394+0000 7f3b84a44700 1 -- 172.21.15.49:0/3608140593 --> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f3b80198c20 con 0x7f3b68038fc0 2024-03-18T22:59:25.396 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:25.395+0000 7f3b6f7fe700 1 -- 172.21.15.49:0/3608140593 <== mgr.14150 v2:172.21.15.49:6800/2083805211 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+160 (secure 0 0 0) 0x7f3b80198c20 con 0x7f3b68038fc0 2024-03-18T22:59:25.396 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-18T22:59:25.396 INFO:teuthology.orchestra.run.smithi049.stdout:[{"addr": "172.21.15.49", "hostname": "smithi049", "labels": [], "status": ""}, {"addr": "172.21.15.139", "hostname": "smithi139", "labels": [], "status": ""}] 2024-03-18T22:59:25.398 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:25.397+0000 7f3b84a44700 1 -- 172.21.15.49:0/3608140593 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f3b68038fc0 msgr2=0x7f3b6803b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T22:59:25.399 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:25.397+0000 7f3b84a44700 1 --2- 172.21.15.49:0/3608140593 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f3b68038fc0 0x7f3b6803b480 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f3b74006fd0 tx=0x7f3b74006e40 comp rx=0 tx=0).stop 2024-03-18T22:59:25.399 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:25.397+0000 7f3b84a44700 1 -- 172.21.15.49:0/3608140593 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3b80101f50 msgr2=0x7f3b8019d620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T22:59:25.399 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:25.397+0000 7f3b84a44700 1 --2- 172.21.15.49:0/3608140593 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3b80101f50 0x7f3b8019d620 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f3b70005850 tx=0x7f3b70005030 comp rx=0 tx=0).stop 2024-03-18T22:59:25.399 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:25.397+0000 7f3b84a44700 1 -- 172.21.15.49:0/3608140593 shutdown_connections 2024-03-18T22:59:25.399 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:25.397+0000 7f3b84a44700 1 --2- 172.21.15.49:0/3608140593 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f3b68038fc0 0x7f3b6803b480 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T22:59:25.399 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:25.397+0000 7f3b84a44700 1 --2- 172.21.15.49:0/3608140593 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3b80101f50 0x7f3b8019d620 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T22:59:25.399 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:25.397+0000 7f3b84a44700 1 -- 172.21.15.49:0/3608140593 >> 172.21.15.49:0/3608140593 conn(0x7f3b800fd820 msgr2=0x7f3b80104d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T22:59:25.399 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:25.397+0000 7f3b84a44700 1 -- 172.21.15.49:0/3608140593 shutdown_connections 2024-03-18T22:59:25.399 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:25.397+0000 7f3b84a44700 1 -- 172.21.15.49:0/3608140593 wait complete. 2024-03-18T22:59:26.002 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi175 2024-03-18T22:59:26.002 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-03-18T22:59:26.003 DEBUG:teuthology.orchestra.run.smithi175:> dd of=/etc/ceph/ceph.conf 2024-03-18T22:59:26.021 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-03-18T22:59:26.021 DEBUG:teuthology.orchestra.run.smithi175:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-18T22:59:26.078 INFO:tasks.cephadm:Adding host smithi175 to orchestrator... 2024-03-18T22:59:26.078 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c63591ea-e57a-11ee-95c9-87774f69a715 -- ceph orch host add smithi175 2024-03-18T22:59:26.322 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T22:59:26.727 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:59:26 smithi049 ceph-mon[62991]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-18T22:59:26.727 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:59:26 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T22:59:27.600 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:59:27 smithi049 ceph-mon[62991]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T22:59:27.825 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:27.821+0000 7fc6e85fe700 1 -- 172.21.15.49:0/3551004634 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc6e0100e80 msgr2=0x7fc6e0101260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T22:59:27.825 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:27.821+0000 7fc6e85fe700 1 --2- 172.21.15.49:0/3551004634 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc6e0100e80 0x7fc6e0101260 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fc6d0009910 tx=0x7fc6d0009c20 comp rx=0 tx=0).stop 2024-03-18T22:59:27.825 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:27.822+0000 7fc6e85fe700 1 -- 172.21.15.49:0/3551004634 shutdown_connections 2024-03-18T22:59:27.825 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:27.822+0000 7fc6e85fe700 1 --2- 172.21.15.49:0/3551004634 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc6e0100e80 0x7fc6e0101260 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T22:59:27.825 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:27.822+0000 7fc6e85fe700 1 -- 172.21.15.49:0/3551004634 >> 172.21.15.49:0/3551004634 conn(0x7fc6e00fc750 msgr2=0x7fc6e00feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T22:59:27.825 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:27.822+0000 7fc6e85fe700 1 -- 172.21.15.49:0/3551004634 shutdown_connections 2024-03-18T22:59:27.825 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:27.822+0000 7fc6e85fe700 1 -- 172.21.15.49:0/3551004634 wait complete. 2024-03-18T22:59:27.825 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:27.822+0000 7fc6e85fe700 1 Processor -- start 2024-03-18T22:59:27.826 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:27.822+0000 7fc6e85fe700 1 -- start start 2024-03-18T22:59:27.826 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:27.822+0000 7fc6e85fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc6e0100e80 0x7fc6e019d580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T22:59:27.826 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:27.822+0000 7fc6e85fe700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fc6e010b250 con 0x7fc6e0100e80 2024-03-18T22:59:27.826 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:27.823+0000 7fc6e639a700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc6e0100e80 0x7fc6e019d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T22:59:27.826 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:27.823+0000 7fc6e639a700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc6e0100e80 0x7fc6e019d580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:45028/0 (socket says 172.21.15.49:45028) 2024-03-18T22:59:27.826 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:27.823+0000 7fc6e639a700 1 -- 172.21.15.49:0/2792563656 learned_addr learned my addr 172.21.15.49:0/2792563656 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T22:59:27.826 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:27.823+0000 7fc6e639a700 1 -- 172.21.15.49:0/2792563656 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6e019dac0 con 0x7fc6e0100e80 2024-03-18T22:59:27.826 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:27.823+0000 7fc6e639a700 1 --2- 172.21.15.49:0/2792563656 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc6e0100e80 0x7fc6e019d580 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fc6d0005850 tx=0x7fc6d0005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T22:59:27.826 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:27.823+0000 7fc6d77fe700 1 -- 172.21.15.49:0/2792563656 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc6d0026070 con 0x7fc6e0100e80 2024-03-18T22:59:27.826 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:27.823+0000 7fc6d77fe700 1 -- 172.21.15.49:0/2792563656 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc6d000bbd0 con 0x7fc6e0100e80 2024-03-18T22:59:27.827 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:27.823+0000 7fc6e85fe700 1 -- 172.21.15.49:0/2792563656 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6e019ddb0 con 0x7fc6e0100e80 2024-03-18T22:59:27.827 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:27.823+0000 7fc6d77fe700 1 -- 172.21.15.49:0/2792563656 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc6d0003970 con 0x7fc6e0100e80 2024-03-18T22:59:27.827 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:27.823+0000 7fc6e85fe700 1 -- 172.21.15.49:0/2792563656 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6e0197820 con 0x7fc6e0100e80 2024-03-18T22:59:27.828 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:27.824+0000 7fc6e85fe700 1 -- 172.21.15.49:0/2792563656 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc6e0102360 con 0x7fc6e0100e80 2024-03-18T22:59:27.828 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:27.824+0000 7fc6d77fe700 1 -- 172.21.15.49:0/2792563656 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 13) ==== 46162+0+0 (secure 0 0 0) 0x7fc6d0029070 con 0x7fc6e0100e80 2024-03-18T22:59:27.828 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:27.825+0000 7fc6d77fe700 1 --2- 172.21.15.49:0/2792563656 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fc6cc038fc0 0x7fc6cc03b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T22:59:27.828 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:27.825+0000 7fc6d77fe700 1 -- 172.21.15.49:0/2792563656 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fc6d001e070 con 0x7fc6e0100e80 2024-03-18T22:59:27.828 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:27.825+0000 7fc6e5b99700 1 --2- 172.21.15.49:0/2792563656 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fc6cc038fc0 0x7fc6cc03b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T22:59:27.828 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:27.825+0000 7fc6e5b99700 1 --2- 172.21.15.49:0/2792563656 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fc6cc038fc0 0x7fc6cc03b480 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fc6dc006fd0 tx=0x7fc6dc006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T22:59:27.830 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:27.830+0000 7fc6d77fe700 1 -- 172.21.15.49:0/2792563656 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fc6d0013070 con 0x7fc6e0100e80 2024-03-18T22:59:28.042 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:28.041+0000 7fc6e85fe700 1 -- 172.21.15.49:0/2792563656 --> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi175", "target": ["mon-mgr", ""]}) -- 0x7fc6e0198bb0 con 0x7fc6cc038fc0 2024-03-18T22:59:29.630 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:59:29 smithi049 ceph-mon[62991]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T22:59:29.630 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:59:29 smithi049 ceph-mon[62991]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi175", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T22:59:29.630 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:59:29 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T22:59:29.630 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:59:29 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T22:59:29.630 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:59:29 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T22:59:29.630 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:59:29 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T22:59:29.630 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:59:29 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi139", "name": "osd_memory_target"}]: dispatch 2024-03-18T22:59:29.631 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:59:29 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T22:59:29.631 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:59:29 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T22:59:29.631 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:59:29 smithi049 ceph-mon[62991]: Updating smithi139:/etc/ceph/ceph.conf 2024-03-18T22:59:30.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:59:30 smithi049 ceph-mon[62991]: Deploying cephadm binary to smithi175 2024-03-18T22:59:30.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:59:30 smithi049 ceph-mon[62991]: Updating smithi139:/var/lib/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-18T22:59:30.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:59:30 smithi049 ceph-mon[62991]: Updating smithi139:/etc/ceph/ceph.client.admin.keyring 2024-03-18T22:59:31.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:59:31 smithi049 ceph-mon[62991]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T22:59:32.337 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:32.336+0000 7fc6d77fe700 1 -- 172.21.15.49:0/2792563656 <== mgr.14150 v2:172.21.15.49:6800/2083805211 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7fc6e0198bb0 con 0x7fc6cc038fc0 2024-03-18T22:59:32.338 INFO:teuthology.orchestra.run.smithi049.stdout:Added host 'smithi175' with addr '172.21.15.175' 2024-03-18T22:59:32.340 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:32.339+0000 7fc6e85fe700 1 -- 172.21.15.49:0/2792563656 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fc6cc038fc0 msgr2=0x7fc6cc03b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T22:59:32.340 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:32.339+0000 7fc6e85fe700 1 --2- 172.21.15.49:0/2792563656 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fc6cc038fc0 0x7fc6cc03b480 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fc6dc006fd0 tx=0x7fc6dc006e40 comp rx=0 tx=0).stop 2024-03-18T22:59:32.340 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:32.339+0000 7fc6e85fe700 1 -- 172.21.15.49:0/2792563656 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc6e0100e80 msgr2=0x7fc6e019d580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T22:59:32.340 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:32.339+0000 7fc6e85fe700 1 --2- 172.21.15.49:0/2792563656 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc6e0100e80 0x7fc6e019d580 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fc6d0005850 tx=0x7fc6d0005030 comp rx=0 tx=0).stop 2024-03-18T22:59:32.340 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:32.339+0000 7fc6e85fe700 1 -- 172.21.15.49:0/2792563656 shutdown_connections 2024-03-18T22:59:32.340 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:32.339+0000 7fc6e85fe700 1 --2- 172.21.15.49:0/2792563656 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fc6cc038fc0 0x7fc6cc03b480 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T22:59:32.340 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:32.339+0000 7fc6e85fe700 1 --2- 172.21.15.49:0/2792563656 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc6e0100e80 0x7fc6e019d580 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T22:59:32.340 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:32.339+0000 7fc6e85fe700 1 -- 172.21.15.49:0/2792563656 >> 172.21.15.49:0/2792563656 conn(0x7fc6e00fc750 msgr2=0x7fc6e006a150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T22:59:32.340 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:32.339+0000 7fc6e85fe700 1 -- 172.21.15.49:0/2792563656 shutdown_connections 2024-03-18T22:59:32.340 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:32.339+0000 7fc6e85fe700 1 -- 172.21.15.49:0/2792563656 wait complete. 2024-03-18T22:59:32.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:59:32 smithi049 ceph-mon[62991]: Updating smithi139:/var/lib/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/config/ceph.client.admin.keyring 2024-03-18T22:59:32.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:59:32 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T22:59:32.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:59:32 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T22:59:32.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:59:32 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T22:59:32.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:59:32 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T22:59:32.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:59:32 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T22:59:33.099 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c63591ea-e57a-11ee-95c9-87774f69a715 -- ceph orch host ls --format=json 2024-03-18T22:59:33.347 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T22:59:33.773 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:59:33 smithi049 ceph-mon[62991]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T22:59:33.773 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:59:33 smithi049 ceph-mon[62991]: Added host smithi175 2024-03-18T22:59:33.773 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:59:33 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T22:59:33.773 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:59:33 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T22:59:34.971 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:34.960+0000 7f8ac5be1700 1 -- 172.21.15.49:0/1906808740 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8ac0102ee0 msgr2=0x7f8ac01032c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T22:59:34.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:34.960+0000 7f8ac5be1700 1 --2- 172.21.15.49:0/1906808740 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8ac0102ee0 0x7f8ac01032c0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f8ab4009910 tx=0x7f8ab4009c20 comp rx=0 tx=0).stop 2024-03-18T22:59:34.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:34.960+0000 7f8ac5be1700 1 -- 172.21.15.49:0/1906808740 shutdown_connections 2024-03-18T22:59:34.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:34.960+0000 7f8ac5be1700 1 --2- 172.21.15.49:0/1906808740 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8ac0102ee0 0x7f8ac01032c0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T22:59:34.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:34.960+0000 7f8ac5be1700 1 -- 172.21.15.49:0/1906808740 >> 172.21.15.49:0/1906808740 conn(0x7f8ac00fc750 msgr2=0x7f8ac00feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T22:59:34.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:34.960+0000 7f8ac5be1700 1 -- 172.21.15.49:0/1906808740 shutdown_connections 2024-03-18T22:59:34.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:34.960+0000 7f8ac5be1700 1 -- 172.21.15.49:0/1906808740 wait complete. 2024-03-18T22:59:34.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:34.961+0000 7f8ac5be1700 1 Processor -- start 2024-03-18T22:59:34.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:34.961+0000 7f8ac5be1700 1 -- start start 2024-03-18T22:59:34.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:34.961+0000 7f8ac5be1700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8ac0102ee0 0x7f8ac019d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T22:59:34.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:34.961+0000 7f8ac5be1700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f8ac010c3b0 con 0x7f8ac0102ee0 2024-03-18T22:59:34.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:34.962+0000 7f8abf7fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8ac0102ee0 0x7f8ac019d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T22:59:34.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:34.962+0000 7f8abf7fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8ac0102ee0 0x7f8ac019d5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:36136/0 (socket says 172.21.15.49:36136) 2024-03-18T22:59:34.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:34.962+0000 7f8abf7fe700 1 -- 172.21.15.49:0/134793449 learned_addr learned my addr 172.21.15.49:0/134793449 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T22:59:34.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:34.962+0000 7f8abf7fe700 1 -- 172.21.15.49:0/134793449 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ac019db00 con 0x7f8ac0102ee0 2024-03-18T22:59:34.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:34.962+0000 7f8abf7fe700 1 --2- 172.21.15.49:0/134793449 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8ac0102ee0 0x7f8ac019d5c0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f8ab4006010 tx=0x7f8ab4005050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T22:59:34.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:34.962+0000 7f8abcff9700 1 -- 172.21.15.49:0/134793449 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8ab4026070 con 0x7f8ac0102ee0 2024-03-18T22:59:34.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:34.962+0000 7f8abcff9700 1 -- 172.21.15.49:0/134793449 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ab400bc30 con 0x7f8ac0102ee0 2024-03-18T22:59:34.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:34.962+0000 7f8ac5be1700 1 -- 172.21.15.49:0/134793449 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ac019dd90 con 0x7f8ac0102ee0 2024-03-18T22:59:34.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:34.962+0000 7f8abcff9700 1 -- 172.21.15.49:0/134793449 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8ab4003ab0 con 0x7f8ac0102ee0 2024-03-18T22:59:34.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:34.962+0000 7f8ac5be1700 1 -- 172.21.15.49:0/134793449 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ac0197690 con 0x7f8ac0102ee0 2024-03-18T22:59:34.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:34.963+0000 7f8abcff9700 1 -- 172.21.15.49:0/134793449 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 13) ==== 46162+0+0 (secure 0 0 0) 0x7f8ab4029070 con 0x7f8ac0102ee0 2024-03-18T22:59:34.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:34.963+0000 7f8ac5be1700 1 -- 172.21.15.49:0/134793449 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ac006aa30 con 0x7f8ac0102ee0 2024-03-18T22:59:34.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:34.963+0000 7f8abcff9700 1 --2- 172.21.15.49:0/134793449 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f8aa8039020 0x7f8aa803b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T22:59:34.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:34.963+0000 7f8abcff9700 1 -- 172.21.15.49:0/134793449 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f8ab401e070 con 0x7f8ac0102ee0 2024-03-18T22:59:34.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:34.963+0000 7f8abeffd700 1 --2- 172.21.15.49:0/134793449 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f8aa8039020 0x7f8aa803b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T22:59:34.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:34.964+0000 7f8abeffd700 1 --2- 172.21.15.49:0/134793449 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f8aa8039020 0x7f8aa803b4e0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f8ab0006fd0 tx=0x7f8ab0006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T22:59:34.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:34.968+0000 7f8abcff9700 1 -- 172.21.15.49:0/134793449 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f8ab4013070 con 0x7f8ac0102ee0 2024-03-18T22:59:35.180 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:35.179+0000 7f8ac5be1700 1 -- 172.21.15.49:0/134793449 --> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f8ac0198bc0 con 0x7f8aa8039020 2024-03-18T22:59:35.181 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:35.180+0000 7f8abcff9700 1 -- 172.21.15.49:0/134793449 <== mgr.14150 v2:172.21.15.49:6800/2083805211 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+240 (secure 0 0 0) 0x7f8ac0198bc0 con 0x7f8aa8039020 2024-03-18T22:59:35.181 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-18T22:59:35.181 INFO:teuthology.orchestra.run.smithi049.stdout:[{"addr": "172.21.15.49", "hostname": "smithi049", "labels": [], "status": ""}, {"addr": "172.21.15.139", "hostname": "smithi139", "labels": [], "status": ""}, {"addr": "172.21.15.175", "hostname": "smithi175", "labels": [], "status": ""}] 2024-03-18T22:59:35.183 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:35.182+0000 7f8ac5be1700 1 -- 172.21.15.49:0/134793449 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f8aa8039020 msgr2=0x7f8aa803b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T22:59:35.183 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:35.182+0000 7f8ac5be1700 1 --2- 172.21.15.49:0/134793449 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f8aa8039020 0x7f8aa803b4e0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f8ab0006fd0 tx=0x7f8ab0006e40 comp rx=0 tx=0).stop 2024-03-18T22:59:35.183 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:35.182+0000 7f8ac5be1700 1 -- 172.21.15.49:0/134793449 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8ac0102ee0 msgr2=0x7f8ac019d5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T22:59:35.183 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:35.182+0000 7f8ac5be1700 1 --2- 172.21.15.49:0/134793449 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8ac0102ee0 0x7f8ac019d5c0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f8ab4006010 tx=0x7f8ab4005050 comp rx=0 tx=0).stop 2024-03-18T22:59:35.183 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:35.182+0000 7f8ac5be1700 1 -- 172.21.15.49:0/134793449 shutdown_connections 2024-03-18T22:59:35.183 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:35.182+0000 7f8ac5be1700 1 --2- 172.21.15.49:0/134793449 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f8aa8039020 0x7f8aa803b4e0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T22:59:35.183 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:35.182+0000 7f8ac5be1700 1 --2- 172.21.15.49:0/134793449 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8ac0102ee0 0x7f8ac019d5c0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T22:59:35.184 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:35.182+0000 7f8ac5be1700 1 -- 172.21.15.49:0/134793449 >> 172.21.15.49:0/134793449 conn(0x7f8ac00fc750 msgr2=0x7f8ac0103c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T22:59:35.184 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:35.182+0000 7f8ac5be1700 1 -- 172.21.15.49:0/134793449 shutdown_connections 2024-03-18T22:59:35.184 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:35.182+0000 7f8ac5be1700 1 -- 172.21.15.49:0/134793449 wait complete. 2024-03-18T22:59:35.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:59:35 smithi049 ceph-mon[62991]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T22:59:35.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:59:35 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T22:59:35.917 INFO:tasks.cephadm:Setting crush tunables to default 2024-03-18T22:59:35.917 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c63591ea-e57a-11ee-95c9-87774f69a715 -- ceph osd crush tunables default 2024-03-18T22:59:36.170 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T22:59:36.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:59:36 smithi049 ceph-mon[62991]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-18T22:59:37.660 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:37.659+0000 7f696d90b700 1 -- 172.21.15.49:0/1926296963 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6968102ea0 msgr2=0x7f6968103280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T22:59:37.661 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:37.659+0000 7f696d90b700 1 --2- 172.21.15.49:0/1926296963 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6968102ea0 0x7f6968103280 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f6960009910 tx=0x7f6960009c20 comp rx=0 tx=0).stop 2024-03-18T22:59:37.661 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:37.659+0000 7f696d90b700 1 -- 172.21.15.49:0/1926296963 shutdown_connections 2024-03-18T22:59:37.661 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:37.659+0000 7f696d90b700 1 --2- 172.21.15.49:0/1926296963 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6968102ea0 0x7f6968103280 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T22:59:37.661 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:37.659+0000 7f696d90b700 1 -- 172.21.15.49:0/1926296963 >> 172.21.15.49:0/1926296963 conn(0x7f6968077850 msgr2=0x7f69680feb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T22:59:37.661 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:37.659+0000 7f696d90b700 1 -- 172.21.15.49:0/1926296963 shutdown_connections 2024-03-18T22:59:37.661 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:37.659+0000 7f696d90b700 1 -- 172.21.15.49:0/1926296963 wait complete. 2024-03-18T22:59:37.661 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:37.659+0000 7f696d90b700 1 Processor -- start 2024-03-18T22:59:37.661 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:37.660+0000 7f696d90b700 1 -- start start 2024-03-18T22:59:37.661 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:37.660+0000 7f696d90b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6968102ea0 0x7f696819d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T22:59:37.661 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:37.660+0000 7f696d90b700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f696810c390 con 0x7f6968102ea0 2024-03-18T22:59:37.663 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:37.660+0000 7f6966ffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6968102ea0 0x7f696819d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T22:59:37.663 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:37.660+0000 7f6966ffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6968102ea0 0x7f696819d5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:36156/0 (socket says 172.21.15.49:36156) 2024-03-18T22:59:37.663 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:37.660+0000 7f6966ffd700 1 -- 172.21.15.49:0/3993714762 learned_addr learned my addr 172.21.15.49:0/3993714762 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T22:59:37.663 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:37.660+0000 7f6966ffd700 1 -- 172.21.15.49:0/3993714762 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f696819daf0 con 0x7f6968102ea0 2024-03-18T22:59:37.663 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:37.660+0000 7f6966ffd700 1 --2- 172.21.15.49:0/3993714762 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6968102ea0 0x7f696819d5b0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f6960006010 tx=0x7f6960005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T22:59:37.664 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:37.661+0000 7f696c909700 1 -- 172.21.15.49:0/3993714762 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6960026070 con 0x7f6968102ea0 2024-03-18T22:59:37.664 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:37.661+0000 7f696c909700 1 -- 172.21.15.49:0/3993714762 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f696000bc30 con 0x7f6968102ea0 2024-03-18T22:59:37.664 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:37.661+0000 7f696d90b700 1 -- 172.21.15.49:0/3993714762 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f696819dd80 con 0x7f6968102ea0 2024-03-18T22:59:37.664 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:37.661+0000 7f696c909700 1 -- 172.21.15.49:0/3993714762 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f69600039a0 con 0x7f6968102ea0 2024-03-18T22:59:37.664 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:37.661+0000 7f696d90b700 1 -- 172.21.15.49:0/3993714762 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6968197680 con 0x7f6968102ea0 2024-03-18T22:59:37.664 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:37.662+0000 7f696c909700 1 -- 172.21.15.49:0/3993714762 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 13) ==== 46162+0+0 (secure 0 0 0) 0x7f6960029070 con 0x7f6968102ea0 2024-03-18T22:59:37.664 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:37.662+0000 7f696d90b700 1 -- 172.21.15.49:0/3993714762 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6968069fe0 con 0x7f6968102ea0 2024-03-18T22:59:37.664 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:37.662+0000 7f696c909700 1 --2- 172.21.15.49:0/3993714762 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f6954039020 0x7f695403b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T22:59:37.664 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:37.662+0000 7f696c909700 1 -- 172.21.15.49:0/3993714762 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f696001e070 con 0x7f6968102ea0 2024-03-18T22:59:37.664 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:37.662+0000 7f69667fc700 1 --2- 172.21.15.49:0/3993714762 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f6954039020 0x7f695403b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T22:59:37.664 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:37.662+0000 7f69667fc700 1 --2- 172.21.15.49:0/3993714762 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f6954039020 0x7f695403b4e0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f6950006fd0 tx=0x7f6950006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T22:59:37.668 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:37.667+0000 7f696c909700 1 -- 172.21.15.49:0/3993714762 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f6960013070 con 0x7f6968102ea0 2024-03-18T22:59:37.880 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:37.879+0000 7f696d90b700 1 -- 172.21.15.49:0/3993714762 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f6968198bb0 con 0x7f6968102ea0 2024-03-18T22:59:37.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:59:37 smithi049 ceph-mon[62991]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T22:59:38.599 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:38.598+0000 7f696c909700 1 -- 172.21.15.49:0/3993714762 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7f6960030090 con 0x7f6968102ea0 2024-03-18T22:59:38.601 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:38.600+0000 7f696d90b700 1 -- 172.21.15.49:0/3993714762 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f6954039020 msgr2=0x7f695403b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T22:59:38.601 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:38.600+0000 7f696d90b700 1 --2- 172.21.15.49:0/3993714762 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f6954039020 0x7f695403b4e0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f6950006fd0 tx=0x7f6950006e40 comp rx=0 tx=0).stop 2024-03-18T22:59:38.602 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:38.600+0000 7f696d90b700 1 -- 172.21.15.49:0/3993714762 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6968102ea0 msgr2=0x7f696819d5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T22:59:38.602 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:38.600+0000 7f696d90b700 1 --2- 172.21.15.49:0/3993714762 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6968102ea0 0x7f696819d5b0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f6960006010 tx=0x7f6960005030 comp rx=0 tx=0).stop 2024-03-18T22:59:38.602 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:38.600+0000 7f696d90b700 1 -- 172.21.15.49:0/3993714762 shutdown_connections 2024-03-18T22:59:38.602 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:38.600+0000 7f696d90b700 1 --2- 172.21.15.49:0/3993714762 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f6954039020 0x7f695403b4e0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T22:59:38.602 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:38.600+0000 7f696d90b700 1 --2- 172.21.15.49:0/3993714762 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6968102ea0 0x7f696819d5b0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T22:59:38.602 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:38.600+0000 7f696d90b700 1 -- 172.21.15.49:0/3993714762 >> 172.21.15.49:0/3993714762 conn(0x7f6968077850 msgr2=0x7f6968103c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T22:59:38.602 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:38.600+0000 7f696d90b700 1 -- 172.21.15.49:0/3993714762 shutdown_connections 2024-03-18T22:59:38.602 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T22:59:38.600+0000 7f696d90b700 1 -- 172.21.15.49:0/3993714762 wait complete. 2024-03-18T22:59:38.602 INFO:teuthology.orchestra.run.smithi049.stderr:adjusted tunables profile to default 2024-03-18T22:59:38.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:59:38 smithi049 ceph-mon[62991]: from='client.? 172.21.15.49:0/3993714762' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-03-18T22:59:38.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:59:38 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T22:59:38.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:59:38 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T22:59:38.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:59:38 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T22:59:38.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:59:38 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T22:59:38.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:59:38 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi175", "name": "osd_memory_target"}]: dispatch 2024-03-18T22:59:38.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:59:38 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T22:59:38.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:59:38 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T22:59:39.331 INFO:tasks.cephadm:Adding mon.a on smithi049 2024-03-18T22:59:39.331 INFO:tasks.cephadm:Adding mon.b on smithi139 2024-03-18T22:59:39.331 INFO:tasks.cephadm:Adding mon.c on smithi175 2024-03-18T22:59:39.331 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c63591ea-e57a-11ee-95c9-87774f69a715 -- ceph orch apply mon '3;smithi049:172.21.15.49=a;smithi139:172.21.15.139=b;smithi175:172.21.15.175=c' 2024-03-18T22:59:39.572 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-18T22:59:39.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:59:39 smithi049 ceph-mon[62991]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T22:59:39.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:59:39 smithi049 ceph-mon[62991]: Updating smithi175:/etc/ceph/ceph.conf 2024-03-18T22:59:39.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:59:39 smithi049 ceph-mon[62991]: Updating smithi175:/var/lib/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-18T22:59:39.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:59:39 smithi049 ceph-mon[62991]: Updating smithi175:/etc/ceph/ceph.client.admin.keyring 2024-03-18T22:59:39.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:59:39 smithi049 ceph-mon[62991]: from='client.? 172.21.15.49:0/3993714762' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-03-18T22:59:39.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:59:39 smithi049 ceph-mon[62991]: osdmap e4: 0 total, 0 up, 0 in 2024-03-18T22:59:39.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:59:39 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T22:59:39.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:59:39 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T22:59:39.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:59:39 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T22:59:40.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:59:40 smithi049 ceph-mon[62991]: Updating smithi175:/var/lib/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/config/ceph.client.admin.keyring 2024-03-18T22:59:41.011 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T22:59:41.010+0000 7f4f78e26700 1 -- 172.21.15.175:0/1558483179 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4f74106d40 msgr2=0x7f4f74107120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T22:59:41.012 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T22:59:41.010+0000 7f4f78e26700 1 --2- 172.21.15.175:0/1558483179 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4f74106d40 0x7f4f74107120 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f4f64009910 tx=0x7f4f64009c20 comp rx=0 tx=0).stop 2024-03-18T22:59:41.012 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T22:59:41.010+0000 7f4f78e26700 1 -- 172.21.15.175:0/1558483179 shutdown_connections 2024-03-18T22:59:41.012 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T22:59:41.010+0000 7f4f78e26700 1 --2- 172.21.15.175:0/1558483179 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4f74106d40 0x7f4f74107120 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T22:59:41.012 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T22:59:41.010+0000 7f4f78e26700 1 -- 172.21.15.175:0/1558483179 >> 172.21.15.175:0/1558483179 conn(0x7f4f740fbe60 msgr2=0x7f4f740fe280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T22:59:41.012 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T22:59:41.010+0000 7f4f78e26700 1 -- 172.21.15.175:0/1558483179 shutdown_connections 2024-03-18T22:59:41.012 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T22:59:41.010+0000 7f4f78e26700 1 -- 172.21.15.175:0/1558483179 wait complete. 2024-03-18T22:59:41.013 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T22:59:41.011+0000 7f4f78e26700 1 Processor -- start 2024-03-18T22:59:41.013 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T22:59:41.011+0000 7f4f78e26700 1 -- start start 2024-03-18T22:59:41.014 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T22:59:41.011+0000 7f4f78e26700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4f74106d40 0x7f4f7419d580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T22:59:41.014 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T22:59:41.011+0000 7f4f78e26700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f4f7410c4a0 con 0x7f4f74106d40 2024-03-18T22:59:41.014 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T22:59:41.012+0000 7f4f7259c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4f74106d40 0x7f4f7419d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T22:59:41.014 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T22:59:41.012+0000 7f4f7259c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4f74106d40 0x7f4f7419d580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.175:42374/0 (socket says 172.21.15.175:42374) 2024-03-18T22:59:41.014 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T22:59:41.012+0000 7f4f7259c700 1 -- 172.21.15.175:0/2726777703 learned_addr learned my addr 172.21.15.175:0/2726777703 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-03-18T22:59:41.014 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T22:59:41.012+0000 7f4f7259c700 1 -- 172.21.15.175:0/2726777703 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f7419dac0 con 0x7f4f74106d40 2024-03-18T22:59:41.014 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T22:59:41.012+0000 7f4f7259c700 1 --2- 172.21.15.175:0/2726777703 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4f74106d40 0x7f4f7419d580 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f4f64005850 tx=0x7f4f64005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T22:59:41.014 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T22:59:41.012+0000 7f4f637fe700 1 -- 172.21.15.175:0/2726777703 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4f64026070 con 0x7f4f74106d40 2024-03-18T22:59:41.014 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T22:59:41.013+0000 7f4f78e26700 1 -- 172.21.15.175:0/2726777703 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f7419dd50 con 0x7f4f74106d40 2024-03-18T22:59:41.014 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T22:59:41.013+0000 7f4f637fe700 1 -- 172.21.15.175:0/2726777703 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4f6400bc30 con 0x7f4f74106d40 2024-03-18T22:59:41.014 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T22:59:41.013+0000 7f4f637fe700 1 -- 172.21.15.175:0/2726777703 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4f64003a20 con 0x7f4f74106d40 2024-03-18T22:59:41.014 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T22:59:41.013+0000 7f4f78e26700 1 -- 172.21.15.175:0/2726777703 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f74197650 con 0x7f4f74106d40 2024-03-18T22:59:41.015 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T22:59:41.014+0000 7f4f78e26700 1 -- 172.21.15.175:0/2726777703 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f74103480 con 0x7f4f74106d40 2024-03-18T22:59:41.015 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T22:59:41.014+0000 7f4f637fe700 1 -- 172.21.15.175:0/2726777703 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 13) ==== 46162+0+0 (secure 0 0 0) 0x7f4f64029070 con 0x7f4f74106d40 2024-03-18T22:59:41.016 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T22:59:41.014+0000 7f4f637fe700 1 --2- 172.21.15.175:0/2726777703 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f4f5c038fc0 0x7f4f5c03b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T22:59:41.016 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T22:59:41.014+0000 7f4f637fe700 1 -- 172.21.15.175:0/2726777703 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f4f6401e070 con 0x7f4f74106d40 2024-03-18T22:59:41.016 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T22:59:41.014+0000 7f4f71d9b700 1 --2- 172.21.15.175:0/2726777703 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f4f5c038fc0 0x7f4f5c03b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T22:59:41.016 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T22:59:41.015+0000 7f4f71d9b700 1 --2- 172.21.15.175:0/2726777703 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f4f5c038fc0 0x7f4f5c03b480 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f4f68006fd0 tx=0x7f4f68006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T22:59:41.020 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T22:59:41.019+0000 7f4f637fe700 1 -- 172.21.15.175:0/2726777703 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f4f64013070 con 0x7f4f74106d40 2024-03-18T22:59:41.226 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T22:59:41.225+0000 7f4f78e26700 1 -- 172.21.15.175:0/2726777703 --> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi049:172.21.15.49=a;smithi139:172.21.15.139=b;smithi175:172.21.15.175=c", "target": ["mon-mgr", ""]}) -- 0x7f4f74198b80 con 0x7f4f5c038fc0 2024-03-18T22:59:41.233 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T22:59:41.232+0000 7f4f637fe700 1 -- 172.21.15.175:0/2726777703 <== mgr.14150 v2:172.21.15.49:6800/2083805211 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f4f74198b80 con 0x7f4f5c038fc0 2024-03-18T22:59:41.233 INFO:teuthology.orchestra.run.smithi175.stdout:Scheduled mon update... 2024-03-18T22:59:41.236 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T22:59:41.234+0000 7f4f78e26700 1 -- 172.21.15.175:0/2726777703 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f4f5c038fc0 msgr2=0x7f4f5c03b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T22:59:41.236 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T22:59:41.234+0000 7f4f78e26700 1 --2- 172.21.15.175:0/2726777703 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f4f5c038fc0 0x7f4f5c03b480 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f4f68006fd0 tx=0x7f4f68006e40 comp rx=0 tx=0).stop 2024-03-18T22:59:41.236 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T22:59:41.234+0000 7f4f78e26700 1 -- 172.21.15.175:0/2726777703 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4f74106d40 msgr2=0x7f4f7419d580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T22:59:41.236 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T22:59:41.234+0000 7f4f78e26700 1 --2- 172.21.15.175:0/2726777703 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4f74106d40 0x7f4f7419d580 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f4f64005850 tx=0x7f4f64005030 comp rx=0 tx=0).stop 2024-03-18T22:59:41.236 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T22:59:41.234+0000 7f4f78e26700 1 -- 172.21.15.175:0/2726777703 shutdown_connections 2024-03-18T22:59:41.237 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T22:59:41.234+0000 7f4f78e26700 1 --2- 172.21.15.175:0/2726777703 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f4f5c038fc0 0x7f4f5c03b480 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T22:59:41.237 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T22:59:41.234+0000 7f4f78e26700 1 --2- 172.21.15.175:0/2726777703 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4f74106d40 0x7f4f7419d580 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T22:59:41.237 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T22:59:41.234+0000 7f4f78e26700 1 -- 172.21.15.175:0/2726777703 >> 172.21.15.175:0/2726777703 conn(0x7f4f740fbe60 msgr2=0x7f4f741097c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T22:59:41.237 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T22:59:41.235+0000 7f4f78e26700 1 -- 172.21.15.175:0/2726777703 shutdown_connections 2024-03-18T22:59:41.237 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T22:59:41.235+0000 7f4f78e26700 1 -- 172.21.15.175:0/2726777703 wait complete. 2024-03-18T22:59:41.817 DEBUG:teuthology.orchestra.run.smithi139:mon.b> sudo journalctl -f -n 0 -u ceph-c63591ea-e57a-11ee-95c9-87774f69a715@mon.b.service 2024-03-18T22:59:41.820 DEBUG:teuthology.orchestra.run.smithi175:mon.c> sudo journalctl -f -n 0 -u ceph-c63591ea-e57a-11ee-95c9-87774f69a715@mon.c.service 2024-03-18T22:59:41.822 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-03-18T22:59:41.822 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c63591ea-e57a-11ee-95c9-87774f69a715 -- ceph mon dump -f json 2024-03-18T22:59:41.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:59:41 smithi049 ceph-mon[62991]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T22:59:41.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:59:41 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T22:59:41.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:59:41 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T22:59:41.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:59:41 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T22:59:41.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:59:41 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T22:59:41.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:59:41 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T22:59:41.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:59:41 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-18T22:59:41.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:59:41 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T22:59:42.530 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-18T22:59:42.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:59:42 smithi049 ceph-mon[62991]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi049:172.21.15.49=a;smithi139:172.21.15.139=b;smithi175:172.21.15.175=c", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T22:59:42.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:59:42 smithi049 ceph-mon[62991]: Saving service mon spec with placement smithi049:172.21.15.49=a;smithi139:172.21.15.139=b;smithi175:172.21.15.175=c;count:3 2024-03-18T22:59:42.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:59:42 smithi049 ceph-mon[62991]: Deploying daemon mon.c on smithi175 2024-03-18T22:59:43.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:59:43 smithi049 ceph-mon[62991]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T22:59:44.026 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T22:59:44.024+0000 7f1b4d53e700 1 -- 172.21.15.175:0/3665865845 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1b48102f30 msgr2=0x7f1b48103310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T22:59:44.026 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T22:59:44.024+0000 7f1b4d53e700 1 --2- 172.21.15.175:0/3665865845 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1b48102f30 0x7f1b48103310 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f1b38009910 tx=0x7f1b38009c20 comp rx=0 tx=0).stop 2024-03-18T22:59:44.026 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T22:59:44.024+0000 7f1b4d53e700 1 -- 172.21.15.175:0/3665865845 shutdown_connections 2024-03-18T22:59:44.026 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T22:59:44.024+0000 7f1b4d53e700 1 --2- 172.21.15.175:0/3665865845 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1b48102f30 0x7f1b48103310 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T22:59:44.026 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T22:59:44.024+0000 7f1b4d53e700 1 -- 172.21.15.175:0/3665865845 >> 172.21.15.175:0/3665865845 conn(0x7f1b480fc700 msgr2=0x7f1b480feb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T22:59:44.026 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T22:59:44.024+0000 7f1b4d53e700 1 -- 172.21.15.175:0/3665865845 shutdown_connections 2024-03-18T22:59:44.026 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T22:59:44.024+0000 7f1b4d53e700 1 -- 172.21.15.175:0/3665865845 wait complete. 2024-03-18T22:59:44.026 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T22:59:44.024+0000 7f1b4d53e700 1 Processor -- start 2024-03-18T22:59:44.027 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T22:59:44.025+0000 7f1b4d53e700 1 -- start start 2024-03-18T22:59:44.027 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T22:59:44.025+0000 7f1b4d53e700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1b48102f30 0x7f1b48100b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T22:59:44.027 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T22:59:44.025+0000 7f1b4d53e700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f1b4810c390 con 0x7f1b48102f30 2024-03-18T22:59:44.027 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T22:59:44.025+0000 7f1b46ffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1b48102f30 0x7f1b48100b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T22:59:44.027 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T22:59:44.025+0000 7f1b46ffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1b48102f30 0x7f1b48100b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.175:42388/0 (socket says 172.21.15.175:42388) 2024-03-18T22:59:44.027 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T22:59:44.025+0000 7f1b46ffd700 1 -- 172.21.15.175:0/1226364532 learned_addr learned my addr 172.21.15.175:0/1226364532 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-03-18T22:59:44.027 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T22:59:44.025+0000 7f1b46ffd700 1 -- 172.21.15.175:0/1226364532 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b48102a80 con 0x7f1b48102f30 2024-03-18T22:59:44.027 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T22:59:44.025+0000 7f1b46ffd700 1 --2- 172.21.15.175:0/1226364532 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1b48102f30 0x7f1b48100b90 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f1b38006010 tx=0x7f1b38005050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T22:59:44.028 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T22:59:44.025+0000 7f1b2ffff700 1 -- 172.21.15.175:0/1226364532 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1b38026070 con 0x7f1b48102f30 2024-03-18T22:59:44.028 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T22:59:44.026+0000 7f1b4d53e700 1 -- 172.21.15.175:0/1226364532 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b481010d0 con 0x7f1b48102f30 2024-03-18T22:59:44.028 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T22:59:44.026+0000 7f1b2ffff700 1 -- 172.21.15.175:0/1226364532 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1b3800bc30 con 0x7f1b48102f30 2024-03-18T22:59:44.029 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T22:59:44.026+0000 7f1b2ffff700 1 -- 172.21.15.175:0/1226364532 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1b38003ab0 con 0x7f1b48102f30 2024-03-18T22:59:44.029 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T22:59:44.026+0000 7f1b4d53e700 1 -- 172.21.15.175:0/1226364532 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b48101590 con 0x7f1b48102f30 2024-03-18T22:59:44.029 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T22:59:44.026+0000 7f1b4d53e700 1 -- 172.21.15.175:0/1226364532 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b4806a9d0 con 0x7f1b48102f30 2024-03-18T22:59:44.029 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T22:59:44.027+0000 7f1b2ffff700 1 -- 172.21.15.175:0/1226364532 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 13) ==== 46162+0+0 (secure 0 0 0) 0x7f1b38029070 con 0x7f1b48102f30 2024-03-18T22:59:44.029 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T22:59:44.027+0000 7f1b2ffff700 1 --2- 172.21.15.175:0/1226364532 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f1b30038fc0 0x7f1b3003b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T22:59:44.029 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T22:59:44.027+0000 7f1b2ffff700 1 -- 172.21.15.175:0/1226364532 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f1b3801e070 con 0x7f1b48102f30 2024-03-18T22:59:44.029 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T22:59:44.027+0000 7f1b467fc700 1 --2- 172.21.15.175:0/1226364532 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f1b30038fc0 0x7f1b3003b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T22:59:44.029 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T22:59:44.027+0000 7f1b467fc700 1 --2- 172.21.15.175:0/1226364532 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f1b30038fc0 0x7f1b3003b480 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f1b3c006fd0 tx=0x7f1b3c006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T22:59:44.033 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T22:59:44.032+0000 7f1b2ffff700 1 -- 172.21.15.175:0/1226364532 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f1b38050030 con 0x7f1b48102f30 2024-03-18T22:59:44.314 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T22:59:44.313+0000 7f1b4d53e700 1 -- 172.21.15.175:0/1226364532 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f1b48102160 con 0x7f1b48102f30 2024-03-18T22:59:44.314 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T22:59:44.313+0000 7f1b2ffff700 1 -- 172.21.15.175:0/1226364532 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+745 (secure 0 0 0) 0x7f1b38030020 con 0x7f1b48102f30 2024-03-18T22:59:44.314 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-03-18T22:59:44.314 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":1,"fsid":"c63591ea-e57a-11ee-95c9-87774f69a715","modified":"2024-03-18T22:57:55.689519Z","created":"2024-03-18T22:57:55.689519Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:3300","nonce":0},{"type":"v1","addr":"172.21.15.49:6789","nonce":0}]},"addr":"172.21.15.49:6789/0","public_addr":"172.21.15.49:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-18T22:59:44.316 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T22:59:44.315+0000 7f1b4d53e700 1 -- 172.21.15.175:0/1226364532 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f1b30038fc0 msgr2=0x7f1b3003b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T22:59:44.317 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T22:59:44.315+0000 7f1b4d53e700 1 --2- 172.21.15.175:0/1226364532 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f1b30038fc0 0x7f1b3003b480 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f1b3c006fd0 tx=0x7f1b3c006e40 comp rx=0 tx=0).stop 2024-03-18T22:59:44.317 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T22:59:44.315+0000 7f1b4d53e700 1 -- 172.21.15.175:0/1226364532 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1b48102f30 msgr2=0x7f1b48100b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T22:59:44.317 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T22:59:44.315+0000 7f1b4d53e700 1 --2- 172.21.15.175:0/1226364532 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1b48102f30 0x7f1b48100b90 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f1b38006010 tx=0x7f1b38005050 comp rx=0 tx=0).stop 2024-03-18T22:59:44.317 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T22:59:44.315+0000 7f1b4d53e700 1 -- 172.21.15.175:0/1226364532 shutdown_connections 2024-03-18T22:59:44.317 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T22:59:44.315+0000 7f1b4d53e700 1 --2- 172.21.15.175:0/1226364532 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f1b30038fc0 0x7f1b3003b480 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T22:59:44.317 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T22:59:44.315+0000 7f1b4d53e700 1 --2- 172.21.15.175:0/1226364532 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1b48102f30 0x7f1b48100b90 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T22:59:44.317 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T22:59:44.315+0000 7f1b4d53e700 1 -- 172.21.15.175:0/1226364532 >> 172.21.15.175:0/1226364532 conn(0x7f1b480fc700 msgr2=0x7f1b48103ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T22:59:44.317 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T22:59:44.316+0000 7f1b4d53e700 1 -- 172.21.15.175:0/1226364532 shutdown_connections 2024-03-18T22:59:44.317 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T22:59:44.316+0000 7f1b4d53e700 1 -- 172.21.15.175:0/1226364532 wait complete. 2024-03-18T22:59:44.318 INFO:teuthology.orchestra.run.smithi175.stderr:dumped monmap epoch 1 2024-03-18T22:59:44.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:59:44 smithi049 ceph-mon[62991]: from='client.? 172.21.15.175:0/1226364532' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-18T22:59:45.876 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-03-18T22:59:45.877 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c63591ea-e57a-11ee-95c9-87774f69a715 -- ceph mon dump -f json 2024-03-18T22:59:45.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:59:45 smithi049 ceph-mon[62991]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T22:59:46.170 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/mon.c/config 2024-03-18T22:59:47.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:59:47 smithi049 ceph-mon[62991]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T22:59:47.911 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T22:59:47.908+0000 7f181bc1f700 1 -- 172.21.15.175:0/4063878762 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1814100f30 msgr2=0x7f1814101310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T22:59:47.911 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T22:59:47.908+0000 7f181bc1f700 1 --2- 172.21.15.175:0/4063878762 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1814100f30 0x7f1814101310 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f1804009910 tx=0x7f1804009c20 comp rx=0 tx=0).stop 2024-03-18T22:59:47.911 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T22:59:47.908+0000 7f181bc1f700 1 -- 172.21.15.175:0/4063878762 shutdown_connections 2024-03-18T22:59:47.911 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T22:59:47.908+0000 7f181bc1f700 1 --2- 172.21.15.175:0/4063878762 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1814100f30 0x7f1814101310 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T22:59:47.911 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T22:59:47.908+0000 7f181bc1f700 1 -- 172.21.15.175:0/4063878762 >> 172.21.15.175:0/4063878762 conn(0x7f18140fc7c0 msgr2=0x7f18140febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T22:59:47.911 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T22:59:47.908+0000 7f181bc1f700 1 -- 172.21.15.175:0/4063878762 shutdown_connections 2024-03-18T22:59:47.911 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T22:59:47.909+0000 7f181bc1f700 1 -- 172.21.15.175:0/4063878762 wait complete. 2024-03-18T22:59:47.911 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T22:59:47.909+0000 7f181bc1f700 1 Processor -- start 2024-03-18T22:59:47.911 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T22:59:47.909+0000 7f181bc1f700 1 -- start start 2024-03-18T22:59:47.911 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T22:59:47.909+0000 7f181bc1f700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1814100f30 0x7f1814198780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T22:59:47.912 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T22:59:47.909+0000 7f181bc1f700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f181410b300 con 0x7f1814100f30 2024-03-18T22:59:47.912 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T22:59:47.910+0000 7f18199bb700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1814100f30 0x7f1814198780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T22:59:47.912 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T22:59:47.910+0000 7f18199bb700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1814100f30 0x7f1814198780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.175:42404/0 (socket says 172.21.15.175:42404) 2024-03-18T22:59:47.912 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T22:59:47.910+0000 7f18199bb700 1 -- 172.21.15.175:0/119715323 learned_addr learned my addr 172.21.15.175:0/119715323 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-03-18T22:59:47.912 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T22:59:47.910+0000 7f18199bb700 1 -- 172.21.15.175:0/119715323 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1814198cc0 con 0x7f1814100f30 2024-03-18T22:59:47.912 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T22:59:47.910+0000 7f18199bb700 1 --2- 172.21.15.175:0/119715323 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1814100f30 0x7f1814198780 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f1804005850 tx=0x7f1804005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T22:59:47.913 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T22:59:47.910+0000 7f180affd700 1 -- 172.21.15.175:0/119715323 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1804026070 con 0x7f1814100f30 2024-03-18T22:59:47.913 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T22:59:47.910+0000 7f181bc1f700 1 -- 172.21.15.175:0/119715323 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1814198fb0 con 0x7f1814100f30 2024-03-18T22:59:47.913 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T22:59:47.910+0000 7f180affd700 1 -- 172.21.15.175:0/119715323 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f180400bbd0 con 0x7f1814100f30 2024-03-18T22:59:47.913 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T22:59:47.910+0000 7f181bc1f700 1 -- 172.21.15.175:0/119715323 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1814199470 con 0x7f1814100f30 2024-03-18T22:59:47.913 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T22:59:47.910+0000 7f180affd700 1 -- 172.21.15.175:0/119715323 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1804003970 con 0x7f1814100f30 2024-03-18T22:59:47.913 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T22:59:47.911+0000 7f181bc1f700 1 -- 172.21.15.175:0/119715323 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1814102410 con 0x7f1814100f30 2024-03-18T22:59:47.914 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T22:59:47.911+0000 7f180affd700 1 -- 172.21.15.175:0/119715323 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 13) ==== 46162+0+0 (secure 0 0 0) 0x7f1804029070 con 0x7f1814100f30 2024-03-18T22:59:47.914 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T22:59:47.912+0000 7f180affd700 1 --2- 172.21.15.175:0/119715323 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f1800039020 0x7f180003b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T22:59:47.914 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T22:59:47.912+0000 7f180affd700 1 -- 172.21.15.175:0/119715323 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f180401e070 con 0x7f1814100f30 2024-03-18T22:59:47.914 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T22:59:47.912+0000 7f18191ba700 1 --2- 172.21.15.175:0/119715323 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f1800039020 0x7f180003b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T22:59:47.914 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T22:59:47.912+0000 7f18191ba700 1 --2- 172.21.15.175:0/119715323 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f1800039020 0x7f180003b4e0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f1810006fd0 tx=0x7f1810006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T22:59:47.917 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T22:59:47.917+0000 7f180affd700 1 -- 172.21.15.175:0/119715323 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f1804013070 con 0x7f1814100f30 2024-03-18T22:59:48.199 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T22:59:48.198+0000 7f181bc1f700 1 -- 172.21.15.175:0/119715323 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f181406a250 con 0x7f1814100f30 2024-03-18T22:59:48.199 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T22:59:48.199+0000 7f180affd700 1 -- 172.21.15.175:0/119715323 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+745 (secure 0 0 0) 0x7f1804030090 con 0x7f1814100f30 2024-03-18T22:59:48.200 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-03-18T22:59:48.200 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":1,"fsid":"c63591ea-e57a-11ee-95c9-87774f69a715","modified":"2024-03-18T22:57:55.689519Z","created":"2024-03-18T22:57:55.689519Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:3300","nonce":0},{"type":"v1","addr":"172.21.15.49:6789","nonce":0}]},"addr":"172.21.15.49:6789/0","public_addr":"172.21.15.49:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-18T22:59:48.202 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T22:59:48.201+0000 7f181bc1f700 1 -- 172.21.15.175:0/119715323 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f1800039020 msgr2=0x7f180003b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T22:59:48.202 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T22:59:48.201+0000 7f181bc1f700 1 --2- 172.21.15.175:0/119715323 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f1800039020 0x7f180003b4e0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f1810006fd0 tx=0x7f1810006e40 comp rx=0 tx=0).stop 2024-03-18T22:59:48.202 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T22:59:48.201+0000 7f181bc1f700 1 -- 172.21.15.175:0/119715323 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1814100f30 msgr2=0x7f1814198780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T22:59:48.202 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T22:59:48.201+0000 7f181bc1f700 1 --2- 172.21.15.175:0/119715323 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1814100f30 0x7f1814198780 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f1804005850 tx=0x7f1804005030 comp rx=0 tx=0).stop 2024-03-18T22:59:48.202 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T22:59:48.201+0000 7f181bc1f700 1 -- 172.21.15.175:0/119715323 shutdown_connections 2024-03-18T22:59:48.202 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T22:59:48.201+0000 7f181bc1f700 1 --2- 172.21.15.175:0/119715323 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f1800039020 0x7f180003b4e0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T22:59:48.202 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T22:59:48.201+0000 7f181bc1f700 1 --2- 172.21.15.175:0/119715323 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1814100f30 0x7f1814198780 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T22:59:48.202 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T22:59:48.201+0000 7f181bc1f700 1 -- 172.21.15.175:0/119715323 >> 172.21.15.175:0/119715323 conn(0x7f18140fc7c0 msgr2=0x7f1814102900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T22:59:48.202 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T22:59:48.201+0000 7f181bc1f700 1 -- 172.21.15.175:0/119715323 shutdown_connections 2024-03-18T22:59:48.202 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T22:59:48.201+0000 7f181bc1f700 1 -- 172.21.15.175:0/119715323 wait complete. 2024-03-18T22:59:48.203 INFO:teuthology.orchestra.run.smithi175.stderr:dumped monmap epoch 1 2024-03-18T22:59:48.316 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 podman[67319]: 2024-03-18 22:59:48.290959406 +0000 UTC m=+0.358112751 container start 6f8dd756d02262224f9346bcaf6cb8b0f9253df9e461be56e3b37fc85425d92d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mon-c, ceph=True, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-03-18T22:59:48.316 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: set uid:gid to 167:167 (ceph:ceph) 2024-03-18T22:59:48.316 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: ceph version 19.0.0-2206-gb6ae2491 (b6ae24918b03b1b8a19c9263a50655bba80bb810) squid (dev), process ceph-mon, pid 2 2024-03-18T22:59:48.316 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: pidfile_write: ignore empty --pid-file 2024-03-18T22:59:48.649 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: load: jerasure load: lrc 2024-03-18T22:59:48.697 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: rocksdb: RocksDB version: 7.9.2 2024-03-18T22:59:48.697 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: rocksdb: Git sha 0 2024-03-18T22:59:48.697 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: rocksdb: Compile date 2024-03-18 18:57:12 2024-03-18T22:59:48.697 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: rocksdb: DB SUMMARY 2024-03-18T22:59:48.697 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: rocksdb: DB Session ID: IKSJ24757J3D3JGLT4NR 2024-03-18T22:59:48.697 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: rocksdb: CURRENT file: CURRENT 2024-03-18T22:59:48.697 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: rocksdb: IDENTITY file: IDENTITY 2024-03-18T22:59:48.697 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-03-18T22:59:48.697 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2024-03-18T22:59:48.697 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 511 ; 2024-03-18T22:59:48.698 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: rocksdb: Options.error_if_exists: 0 2024-03-18T22:59:48.698 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: rocksdb: Options.create_if_missing: 0 2024-03-18T22:59:48.698 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: rocksdb: Options.paranoid_checks: 1 2024-03-18T22:59:48.698 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: rocksdb: Options.flush_verify_memtable_count: 1 2024-03-18T22:59:48.698 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-03-18T22:59:48.698 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-03-18T22:59:48.698 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: rocksdb: Options.env: 0x55cbcca49c60 2024-03-18T22:59:48.698 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: rocksdb: Options.fs: PosixFileSystem 2024-03-18T22:59:48.698 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: rocksdb: Options.info_log: 0x55cbce6ebae0 2024-03-18T22:59:48.698 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: rocksdb: Options.max_file_opening_threads: 16 2024-03-18T22:59:48.698 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: rocksdb: Options.statistics: (nil) 2024-03-18T22:59:48.698 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: rocksdb: Options.use_fsync: 0 2024-03-18T22:59:48.698 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: rocksdb: Options.max_log_file_size: 0 2024-03-18T22:59:48.698 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-03-18T22:59:48.699 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: rocksdb: Options.log_file_time_to_roll: 0 2024-03-18T22:59:48.699 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: rocksdb: Options.keep_log_file_num: 1000 2024-03-18T22:59:48.699 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: rocksdb: Options.recycle_log_file_num: 0 2024-03-18T22:59:48.699 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: rocksdb: Options.allow_fallocate: 1 2024-03-18T22:59:48.699 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: rocksdb: Options.allow_mmap_reads: 0 2024-03-18T22:59:48.699 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: rocksdb: Options.allow_mmap_writes: 0 2024-03-18T22:59:48.699 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: rocksdb: Options.use_direct_reads: 0 2024-03-18T22:59:48.699 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-03-18T22:59:48.699 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: rocksdb: Options.create_missing_column_families: 0 2024-03-18T22:59:48.699 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: rocksdb: Options.db_log_dir: 2024-03-18T22:59:48.699 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: rocksdb: Options.wal_dir: 2024-03-18T22:59:48.699 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: rocksdb: Options.table_cache_numshardbits: 6 2024-03-18T22:59:48.699 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: rocksdb: Options.WAL_ttl_seconds: 0 2024-03-18T22:59:48.699 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: rocksdb: Options.WAL_size_limit_MB: 0 2024-03-18T22:59:48.699 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-03-18T22:59:48.701 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-03-18T22:59:48.702 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: rocksdb: Options.is_fd_close_on_exec: 1 2024-03-18T22:59:48.702 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: rocksdb: Options.advise_random_on_open: 1 2024-03-18T22:59:48.702 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: rocksdb: Options.db_write_buffer_size: 0 2024-03-18T22:59:48.702 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: rocksdb: Options.write_buffer_manager: 0x55cbce6be1e0 2024-03-18T22:59:48.702 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-03-18T22:59:48.702 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-03-18T22:59:48.702 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: rocksdb: Options.use_adaptive_mutex: 0 2024-03-18T22:59:48.702 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: rocksdb: Options.rate_limiter: (nil) 2024-03-18T22:59:48.702 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-03-18T22:59:48.702 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: rocksdb: Options.wal_recovery_mode: 2 2024-03-18T22:59:48.702 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: rocksdb: Options.enable_thread_tracking: 0 2024-03-18T22:59:48.702 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: rocksdb: Options.enable_pipelined_write: 0 2024-03-18T22:59:48.702 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: rocksdb: Options.unordered_write: 0 2024-03-18T22:59:48.703 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-03-18T22:59:48.703 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-03-18T22:59:48.703 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-03-18T22:59:48.703 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-03-18T22:59:48.703 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: rocksdb: Options.row_cache: None 2024-03-18T22:59:48.703 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: rocksdb: Options.wal_filter: None 2024-03-18T22:59:48.703 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-03-18T22:59:48.703 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: rocksdb: Options.allow_ingest_behind: 0 2024-03-18T22:59:48.703 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: rocksdb: Options.two_write_queues: 0 2024-03-18T22:59:48.703 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: rocksdb: Options.manual_wal_flush: 0 2024-03-18T22:59:48.703 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: rocksdb: Options.wal_compression: 0 2024-03-18T22:59:48.703 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: rocksdb: Options.atomic_flush: 0 2024-03-18T22:59:48.703 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-03-18T22:59:48.703 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: rocksdb: Options.persist_stats_to_disk: 0 2024-03-18T22:59:48.704 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: rocksdb: Options.write_dbid_to_manifest: 0 2024-03-18T22:59:48.704 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: rocksdb: Options.log_readahead_size: 0 2024-03-18T22:59:48.704 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-03-18T22:59:48.704 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: rocksdb: Options.best_efforts_recovery: 0 2024-03-18T22:59:48.704 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-03-18T22:59:48.704 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-03-18T22:59:48.704 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: rocksdb: Options.allow_data_in_errors: 0 2024-03-18T22:59:48.704 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: rocksdb: Options.db_host_id: __hostname__ 2024-03-18T22:59:48.704 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: rocksdb: Options.enforce_single_del_contracts: true 2024-03-18T22:59:48.704 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: rocksdb: Options.max_background_jobs: 2 2024-03-18T22:59:48.704 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: rocksdb: Options.max_background_compactions: -1 2024-03-18T22:59:48.704 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: rocksdb: Options.max_subcompactions: 1 2024-03-18T22:59:48.704 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-03-18T22:59:48.704 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-03-18T22:59:48.705 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: rocksdb: Options.delayed_write_rate : 16777216 2024-03-18T22:59:48.705 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: rocksdb: Options.max_total_wal_size: 0 2024-03-18T22:59:48.705 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-03-18T22:59:48.705 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: rocksdb: Options.stats_dump_period_sec: 600 2024-03-18T22:59:48.705 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: rocksdb: Options.stats_persist_period_sec: 600 2024-03-18T22:59:48.705 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-03-18T22:59:48.705 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: rocksdb: Options.max_open_files: -1 2024-03-18T22:59:48.705 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: rocksdb: Options.bytes_per_sync: 0 2024-03-18T22:59:48.705 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: rocksdb: Options.wal_bytes_per_sync: 0 2024-03-18T22:59:48.705 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: rocksdb: Options.strict_bytes_per_sync: 0 2024-03-18T22:59:48.705 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: rocksdb: Options.compaction_readahead_size: 0 2024-03-18T22:59:48.705 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: rocksdb: Options.max_background_flushes: -1 2024-03-18T22:59:48.705 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: rocksdb: Compression algorithms supported: 2024-03-18T22:59:48.705 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-03-18T22:59:48.705 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: rocksdb: kZSTD supported: 0 2024-03-18T22:59:48.706 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: rocksdb: kXpressCompression supported: 0 2024-03-18T22:59:48.706 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: rocksdb: kLZ4HCCompression supported: 1 2024-03-18T22:59:48.706 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: rocksdb: kZlibCompression supported: 1 2024-03-18T22:59:48.706 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: rocksdb: kSnappyCompression supported: 1 2024-03-18T22:59:48.706 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: rocksdb: kLZ4Compression supported: 1 2024-03-18T22:59:48.706 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: rocksdb: kBZip2Compression supported: 0 2024-03-18T22:59:48.706 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: rocksdb: Fast CRC32 supported: Supported on x86 2024-03-18T22:59:48.706 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: rocksdb: DMutex implementation: pthread_mutex_t 2024-03-18T22:59:48.706 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2024-03-18T22:59:48.706 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-03-18T22:59:48.706 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-03-18T22:59:48.706 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: rocksdb: Options.merge_operator: 2024-03-18T22:59:48.706 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: rocksdb: Options.compaction_filter: None 2024-03-18T22:59:48.706 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: rocksdb: Options.compaction_filter_factory: None 2024-03-18T22:59:48.707 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: rocksdb: Options.sst_partitioner_factory: None 2024-03-18T22:59:48.707 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: rocksdb: Options.memtable_factory: SkipListFactory 2024-03-18T22:59:48.707 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: rocksdb: Options.table_factory: BlockBasedTable 2024-03-18T22:59:48.707 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55cbce6ebbe0) 2024-03-18T22:59:48.707 INFO:journalctl@ceph.mon.c.smithi175.stdout: cache_index_and_filter_blocks: 1 2024-03-18T22:59:48.707 INFO:journalctl@ceph.mon.c.smithi175.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-03-18T22:59:48.707 INFO:journalctl@ceph.mon.c.smithi175.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-03-18T22:59:48.707 INFO:journalctl@ceph.mon.c.smithi175.stdout: pin_top_level_index_and_filter: 1 2024-03-18T22:59:48.707 INFO:journalctl@ceph.mon.c.smithi175.stdout: index_type: 0 2024-03-18T22:59:48.707 INFO:journalctl@ceph.mon.c.smithi175.stdout: data_block_index_type: 0 2024-03-18T22:59:48.707 INFO:journalctl@ceph.mon.c.smithi175.stdout: index_shortening: 1 2024-03-18T22:59:48.707 INFO:journalctl@ceph.mon.c.smithi175.stdout: data_block_hash_table_util_ratio: 0.750000 2024-03-18T22:59:48.707 INFO:journalctl@ceph.mon.c.smithi175.stdout: checksum: 4 2024-03-18T22:59:48.707 INFO:journalctl@ceph.mon.c.smithi175.stdout: no_block_cache: 0 2024-03-18T22:59:48.707 INFO:journalctl@ceph.mon.c.smithi175.stdout: block_cache: 0x55cbce6f1090 2024-03-18T22:59:48.708 INFO:journalctl@ceph.mon.c.smithi175.stdout: block_cache_name: BinnedLRUCache 2024-03-18T22:59:48.708 INFO:journalctl@ceph.mon.c.smithi175.stdout: block_cache_options: 2024-03-18T22:59:48.708 INFO:journalctl@ceph.mon.c.smithi175.stdout: capacity : 536870912 2024-03-18T22:59:48.708 INFO:journalctl@ceph.mon.c.smithi175.stdout: num_shard_bits : 4 2024-03-18T22:59:48.708 INFO:journalctl@ceph.mon.c.smithi175.stdout: strict_capacity_limit : 0 2024-03-18T22:59:48.708 INFO:journalctl@ceph.mon.c.smithi175.stdout: high_pri_pool_ratio: 0.000 2024-03-18T22:59:48.708 INFO:journalctl@ceph.mon.c.smithi175.stdout: block_cache_compressed: (nil) 2024-03-18T22:59:48.708 INFO:journalctl@ceph.mon.c.smithi175.stdout: persistent_cache: (nil) 2024-03-18T22:59:48.708 INFO:journalctl@ceph.mon.c.smithi175.stdout: block_size: 4096 2024-03-18T22:59:48.708 INFO:journalctl@ceph.mon.c.smithi175.stdout: block_size_deviation: 10 2024-03-18T22:59:48.708 INFO:journalctl@ceph.mon.c.smithi175.stdout: block_restart_interval: 16 2024-03-18T22:59:48.708 INFO:journalctl@ceph.mon.c.smithi175.stdout: index_block_restart_interval: 1 2024-03-18T22:59:48.708 INFO:journalctl@ceph.mon.c.smithi175.stdout: metadata_block_size: 4096 2024-03-18T22:59:48.708 INFO:journalctl@ceph.mon.c.smithi175.stdout: partition_filters: 0 2024-03-18T22:59:48.708 INFO:journalctl@ceph.mon.c.smithi175.stdout: use_delta_encoding: 1 2024-03-18T22:59:48.709 INFO:journalctl@ceph.mon.c.smithi175.stdout: filter_policy: bloomfilter 2024-03-18T22:59:48.709 INFO:journalctl@ceph.mon.c.smithi175.stdout: whole_key_filtering: 1 2024-03-18T22:59:48.709 INFO:journalctl@ceph.mon.c.smithi175.stdout: verify_compression: 0 2024-03-18T22:59:48.709 INFO:journalctl@ceph.mon.c.smithi175.stdout: read_amp_bytes_per_bit: 0 2024-03-18T22:59:48.709 INFO:journalctl@ceph.mon.c.smithi175.stdout: format_version: 5 2024-03-18T22:59:48.709 INFO:journalctl@ceph.mon.c.smithi175.stdout: enable_index_compression: 1 2024-03-18T22:59:48.709 INFO:journalctl@ceph.mon.c.smithi175.stdout: block_align: 0 2024-03-18T22:59:48.709 INFO:journalctl@ceph.mon.c.smithi175.stdout: max_auto_readahead_size: 262144 2024-03-18T22:59:48.709 INFO:journalctl@ceph.mon.c.smithi175.stdout: prepopulate_block_cache: 0 2024-03-18T22:59:48.709 INFO:journalctl@ceph.mon.c.smithi175.stdout: initial_auto_readahead_size: 8192 2024-03-18T22:59:48.709 INFO:journalctl@ceph.mon.c.smithi175.stdout: num_file_reads_for_auto_readahead: 2 2024-03-18T22:59:48.709 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: rocksdb: Options.write_buffer_size: 33554432 2024-03-18T22:59:48.709 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: rocksdb: Options.max_write_buffer_number: 2 2024-03-18T22:59:48.709 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: rocksdb: Options.compression: NoCompression 2024-03-18T22:59:48.709 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: rocksdb: Options.bottommost_compression: Disabled 2024-03-18T22:59:48.710 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: rocksdb: Options.prefix_extractor: nullptr 2024-03-18T22:59:48.710 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-03-18T22:59:48.710 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: rocksdb: Options.num_levels: 7 2024-03-18T22:59:48.710 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-03-18T22:59:48.710 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-03-18T22:59:48.710 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-03-18T22:59:48.710 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-03-18T22:59:48.710 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-03-18T22:59:48.710 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-03-18T22:59:48.710 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-03-18T22:59:48.710 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-03-18T22:59:48.710 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-03-18T22:59:48.710 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-03-18T22:59:48.710 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-03-18T22:59:48.711 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-03-18T22:59:48.711 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: rocksdb: Options.compression_opts.window_bits: -14 2024-03-18T22:59:48.711 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: rocksdb: Options.compression_opts.level: 32767 2024-03-18T22:59:48.711 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: rocksdb: Options.compression_opts.strategy: 0 2024-03-18T22:59:48.711 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-03-18T22:59:48.711 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-03-18T22:59:48.711 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-03-18T22:59:48.711 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-03-18T22:59:48.711 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: rocksdb: Options.compression_opts.enabled: false 2024-03-18T22:59:48.711 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-03-18T22:59:48.711 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-03-18T22:59:48.711 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-03-18T22:59:48.711 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-03-18T22:59:48.711 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: rocksdb: Options.target_file_size_base: 67108864 2024-03-18T22:59:48.711 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: rocksdb: Options.target_file_size_multiplier: 1 2024-03-18T22:59:48.712 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-03-18T22:59:48.712 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-03-18T22:59:48.712 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-03-18T22:59:48.712 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-03-18T22:59:48.712 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-03-18T22:59:48.712 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-03-18T22:59:48.712 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-03-18T22:59:48.712 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-03-18T22:59:48.712 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-03-18T22:59:48.712 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-03-18T22:59:48.712 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-03-18T22:59:48.712 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-03-18T22:59:48.712 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-03-18T22:59:48.712 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: rocksdb: Options.arena_block_size: 1048576 2024-03-18T22:59:48.712 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-03-18T22:59:48.713 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-03-18T22:59:48.713 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: rocksdb: Options.disable_auto_compactions: 0 2024-03-18T22:59:48.713 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-03-18T22:59:48.713 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-03-18T22:59:48.713 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-03-18T22:59:48.713 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-03-18T22:59:48.713 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-03-18T22:59:48.713 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-03-18T22:59:48.713 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-03-18T22:59:48.713 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-03-18T22:59:48.713 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-03-18T22:59:48.713 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-03-18T22:59:48.713 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: rocksdb: Options.table_properties_collectors: 2024-03-18T22:59:48.713 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: rocksdb: Options.inplace_update_support: 0 2024-03-18T22:59:48.713 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: rocksdb: Options.inplace_update_num_locks: 10000 2024-03-18T22:59:48.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-03-18T22:59:48.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-03-18T22:59:48.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: rocksdb: Options.memtable_huge_page_size: 0 2024-03-18T22:59:48.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: rocksdb: Options.bloom_locality: 0 2024-03-18T22:59:48.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: rocksdb: Options.max_successive_merges: 0 2024-03-18T22:59:48.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: rocksdb: Options.optimize_filters_for_hits: 0 2024-03-18T22:59:48.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: rocksdb: Options.paranoid_file_checks: 0 2024-03-18T22:59:48.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: rocksdb: Options.force_consistency_checks: 1 2024-03-18T22:59:48.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: rocksdb: Options.report_bg_io_stats: 0 2024-03-18T22:59:48.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: rocksdb: Options.ttl: 2592000 2024-03-18T22:59:48.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: rocksdb: Options.periodic_compaction_seconds: 0 2024-03-18T22:59:48.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-03-18T22:59:48.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-03-18T22:59:48.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: rocksdb: Options.enable_blob_files: false 2024-03-18T22:59:48.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: rocksdb: Options.min_blob_size: 0 2024-03-18T22:59:48.715 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: rocksdb: Options.blob_file_size: 268435456 2024-03-18T22:59:48.715 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: rocksdb: Options.blob_compression_type: NoCompression 2024-03-18T22:59:48.715 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: rocksdb: Options.enable_blob_garbage_collection: false 2024-03-18T22:59:48.715 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-03-18T22:59:48.715 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-03-18T22:59:48.715 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-03-18T22:59:48.715 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: rocksdb: Options.blob_file_starting_level: 0 2024-03-18T22:59:48.715 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-03-18T22:59:48.715 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-03-18T22:59:48.715 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-03-18T22:59:48.715 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 237b9371-6897-42db-9997-a0f743fc6def 2024-03-18T22:59:48.715 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: rocksdb: EVENT_LOG_v1 {"time_micros": 1710802788322814, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-03-18T22:59:48.715 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-03-18T22:59:48.715 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: rocksdb: EVENT_LOG_v1 {"time_micros": 1710802788323230, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1622, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1710802788, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "237b9371-6897-42db-9997-a0f743fc6def", "db_session_id": "IKSJ24757J3D3JGLT4NR", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-03-18T22:59:48.716 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: rocksdb: EVENT_LOG_v1 {"time_micros": 1710802788323310, "job": 1, "event": "recovery_finished"} 2024-03-18T22:59:48.716 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-03-18T22:59:48.716 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-03-18T22:59:48.716 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55cbce7e4000 2024-03-18T22:59:48.716 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: rocksdb: DB pointer 0x55cbce7ce000 2024-03-18T22:59:48.716 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-03-18T22:59:48.716 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-03-18T22:59:48.716 INFO:journalctl@ceph.mon.c.smithi175.stdout: ** DB Stats ** 2024-03-18T22:59:48.716 INFO:journalctl@ceph.mon.c.smithi175.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-18T22:59:48.716 INFO:journalctl@ceph.mon.c.smithi175.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-03-18T22:59:48.716 INFO:journalctl@ceph.mon.c.smithi175.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-18T22:59:48.716 INFO:journalctl@ceph.mon.c.smithi175.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-18T22:59:48.716 INFO:journalctl@ceph.mon.c.smithi175.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-03-18T22:59:48.716 INFO:journalctl@ceph.mon.c.smithi175.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-18T22:59:48.716 INFO:journalctl@ceph.mon.c.smithi175.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-18T22:59:48.717 INFO:journalctl@ceph.mon.c.smithi175.stdout: 2024-03-18T22:59:48.717 INFO:journalctl@ceph.mon.c.smithi175.stdout: ** Compaction Stats [default] ** 2024-03-18T22:59:48.717 INFO:journalctl@ceph.mon.c.smithi175.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-18T22:59:48.717 INFO:journalctl@ceph.mon.c.smithi175.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-03-18T22:59:48.717 INFO:journalctl@ceph.mon.c.smithi175.stdout: L0 1/0 1.58 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.9 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-03-18T22:59:48.717 INFO:journalctl@ceph.mon.c.smithi175.stdout: Sum 1/0 1.58 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.9 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-03-18T22:59:48.717 INFO:journalctl@ceph.mon.c.smithi175.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.9 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-03-18T22:59:48.717 INFO:journalctl@ceph.mon.c.smithi175.stdout: 2024-03-18T22:59:48.717 INFO:journalctl@ceph.mon.c.smithi175.stdout: ** Compaction Stats [default] ** 2024-03-18T22:59:48.717 INFO:journalctl@ceph.mon.c.smithi175.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-18T22:59:48.717 INFO:journalctl@ceph.mon.c.smithi175.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-03-18T22:59:48.717 INFO:journalctl@ceph.mon.c.smithi175.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 3.9 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-03-18T22:59:48.717 INFO:journalctl@ceph.mon.c.smithi175.stdout: 2024-03-18T22:59:48.717 INFO:journalctl@ceph.mon.c.smithi175.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-03-18T22:59:48.718 INFO:journalctl@ceph.mon.c.smithi175.stdout: 2024-03-18T22:59:48.718 INFO:journalctl@ceph.mon.c.smithi175.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-18T22:59:48.718 INFO:journalctl@ceph.mon.c.smithi175.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-03-18T22:59:48.718 INFO:journalctl@ceph.mon.c.smithi175.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-03-18T22:59:48.718 INFO:journalctl@ceph.mon.c.smithi175.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-03-18T22:59:48.718 INFO:journalctl@ceph.mon.c.smithi175.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-03-18T22:59:48.718 INFO:journalctl@ceph.mon.c.smithi175.stdout: AddFile(Keys): cumulative 0, interval 0 2024-03-18T22:59:48.718 INFO:journalctl@ceph.mon.c.smithi175.stdout: Cumulative compaction: 0.00 GB write, 0.66 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-18T22:59:48.718 INFO:journalctl@ceph.mon.c.smithi175.stdout: Interval compaction: 0.00 GB write, 0.66 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-18T22:59:48.718 INFO:journalctl@ceph.mon.c.smithi175.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-03-18T22:59:48.718 INFO:journalctl@ceph.mon.c.smithi175.stdout: Block cache BinnedLRUCache@0x55cbce6f1090#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 8e-06 secs_since: 0 2024-03-18T22:59:48.718 INFO:journalctl@ceph.mon.c.smithi175.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-03-18T22:59:48.718 INFO:journalctl@ceph.mon.c.smithi175.stdout: 2024-03-18T22:59:48.718 INFO:journalctl@ceph.mon.c.smithi175.stdout: ** File Read Latency Histogram By Level [default] ** 2024-03-18T22:59:48.719 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: mon.c does not exist in monmap, will attempt to join an existing cluster 2024-03-18T22:59:48.719 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: using public_addr v2:172.21.15.175:0/0 -> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] 2024-03-18T22:59:48.719 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: starting mon.c rank -1 at public addrs [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] at bind addrs [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon_data /var/lib/ceph/mon/ceph-c fsid c63591ea-e57a-11ee-95c9-87774f69a715 2024-03-18T22:59:48.719 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: mon.c@-1(???) e0 preinit fsid c63591ea-e57a-11ee-95c9-87774f69a715 2024-03-18T22:59:48.719 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: mon.c@-1(synchronizing).mds e1 new map 2024-03-18T22:59:48.719 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: mon.c@-1(synchronizing).mds e1 print_map 2024-03-18T22:59:48.719 INFO:journalctl@ceph.mon.c.smithi175.stdout: e1 2024-03-18T22:59:48.719 INFO:journalctl@ceph.mon.c.smithi175.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-03-18T22:59:48.719 INFO:journalctl@ceph.mon.c.smithi175.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-03-18T22:59:48.719 INFO:journalctl@ceph.mon.c.smithi175.stdout: legacy client fscid: -1 2024-03-18T22:59:48.719 INFO:journalctl@ceph.mon.c.smithi175.stdout: 2024-03-18T22:59:48.719 INFO:journalctl@ceph.mon.c.smithi175.stdout: No filesystems configured 2024-03-18T22:59:48.719 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-03-18T22:59:48.719 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-03-18T22:59:48.720 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-03-18T22:59:48.720 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-03-18T22:59:48.720 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-03-18T22:59:48.720 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-03-18T22:59:48.720 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-03-18T22:59:48.720 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-03-18T22:59:48.720 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-03-18T22:59:48.720 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-03-18T22:59:48.720 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T22:59:48.720 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T22:59:48.720 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: Added host smithi139 2024-03-18T22:59:48.720 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T22:59:48.720 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T22:59:48.720 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T22:59:48.720 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T22:59:48.721 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-18T22:59:48.721 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T22:59:48.721 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T22:59:48.721 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T22:59:48.721 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi175", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T22:59:48.721 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T22:59:48.721 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T22:59:48.721 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T22:59:48.721 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T22:59:48.721 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi139", "name": "osd_memory_target"}]: dispatch 2024-03-18T22:59:48.721 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T22:59:48.721 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T22:59:48.721 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: Updating smithi139:/etc/ceph/ceph.conf 2024-03-18T22:59:48.721 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: Deploying cephadm binary to smithi175 2024-03-18T22:59:48.722 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: Updating smithi139:/var/lib/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-18T22:59:48.722 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: Updating smithi139:/etc/ceph/ceph.client.admin.keyring 2024-03-18T22:59:48.722 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T22:59:48.722 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: Updating smithi139:/var/lib/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/config/ceph.client.admin.keyring 2024-03-18T22:59:48.722 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T22:59:48.722 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T22:59:48.722 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T22:59:48.722 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T22:59:48.722 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T22:59:48.722 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T22:59:48.722 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: Added host smithi175 2024-03-18T22:59:48.722 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T22:59:48.722 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T22:59:48.722 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T22:59:48.723 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T22:59:48.723 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-18T22:59:48.723 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T22:59:48.723 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: from='client.? 172.21.15.49:0/3993714762' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-03-18T22:59:48.723 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T22:59:48.723 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T22:59:48.723 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T22:59:48.723 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T22:59:48.723 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi175", "name": "osd_memory_target"}]: dispatch 2024-03-18T22:59:48.723 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T22:59:48.723 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T22:59:48.723 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T22:59:48.723 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: Updating smithi175:/etc/ceph/ceph.conf 2024-03-18T22:59:48.723 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: Updating smithi175:/var/lib/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-18T22:59:48.723 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: Updating smithi175:/etc/ceph/ceph.client.admin.keyring 2024-03-18T22:59:48.724 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: from='client.? 172.21.15.49:0/3993714762' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-03-18T22:59:48.724 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: osdmap e4: 0 total, 0 up, 0 in 2024-03-18T22:59:48.724 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T22:59:48.724 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T22:59:48.724 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T22:59:48.724 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: Updating smithi175:/var/lib/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/config/ceph.client.admin.keyring 2024-03-18T22:59:48.724 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T22:59:48.724 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T22:59:48.724 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T22:59:48.724 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T22:59:48.724 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T22:59:48.724 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T22:59:48.724 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-18T22:59:48.724 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T22:59:48.724 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi049:172.21.15.49=a;smithi139:172.21.15.139=b;smithi175:172.21.15.175=c", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T22:59:48.725 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: Saving service mon spec with placement smithi049:172.21.15.49=a;smithi139:172.21.15.139=b;smithi175:172.21.15.175=c;count:3 2024-03-18T22:59:48.725 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: Deploying daemon mon.c on smithi175 2024-03-18T22:59:48.725 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T22:59:48.725 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: from='client.? 172.21.15.175:0/1226364532' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-18T22:59:48.725 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T22:59:48.725 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T22:59:48.725 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 ceph-mon[67352]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-03-18T22:59:48.725 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 bash[67319]: 6f8dd756d02262224f9346bcaf6cb8b0f9253df9e461be56e3b37fc85425d92d 2024-03-18T22:59:48.725 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:48 smithi175 systemd[1]: Started Ceph mon.c for c63591ea-e57a-11ee-95c9-87774f69a715. 2024-03-18T22:59:49.952 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-03-18T22:59:49.952 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c63591ea-e57a-11ee-95c9-87774f69a715 -- ceph mon dump -f json 2024-03-18T22:59:50.195 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/mon.c/config 2024-03-18T22:59:53.712 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:53 smithi175 ceph-mon[67352]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T22:59:53.712 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:53 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-18T22:59:53.712 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:53 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-18T22:59:53.712 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:53 smithi175 ceph-mon[67352]: mon.a calling monitor election 2024-03-18T22:59:53.712 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:53 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-18T22:59:53.712 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:53 smithi175 ceph-mon[67352]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T22:59:53.712 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:53 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-18T22:59:53.712 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:53 smithi175 ceph-mon[67352]: mon.c calling monitor election 2024-03-18T22:59:53.712 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:53 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-18T22:59:53.712 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:53 smithi175 ceph-mon[67352]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T22:59:53.712 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:53 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-18T22:59:53.713 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:53 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-18T22:59:53.713 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:53 smithi175 ceph-mon[67352]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-03-18T22:59:53.713 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:53 smithi175 ceph-mon[67352]: monmap epoch 2 2024-03-18T22:59:53.713 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:53 smithi175 ceph-mon[67352]: fsid c63591ea-e57a-11ee-95c9-87774f69a715 2024-03-18T22:59:53.713 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:53 smithi175 ceph-mon[67352]: last_changed 2024-03-18T22:59:48.347509+0000 2024-03-18T22:59:53.713 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:53 smithi175 ceph-mon[67352]: created 2024-03-18T22:57:55.689519+0000 2024-03-18T22:59:53.713 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:53 smithi175 ceph-mon[67352]: min_mon_release 19 (squid) 2024-03-18T22:59:53.713 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:53 smithi175 ceph-mon[67352]: election_strategy: 1 2024-03-18T22:59:53.713 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:53 smithi175 ceph-mon[67352]: 0: [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] mon.a 2024-03-18T22:59:53.713 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:53 smithi175 ceph-mon[67352]: 1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.c 2024-03-18T22:59:53.713 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:53 smithi175 ceph-mon[67352]: fsmap 2024-03-18T22:59:53.713 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:53 smithi175 ceph-mon[67352]: osdmap e4: 0 total, 0 up, 0 in 2024-03-18T22:59:53.713 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:53 smithi175 ceph-mon[67352]: mgrmap e13: x(active, since 53s) 2024-03-18T22:59:53.713 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:53 smithi175 ceph-mon[67352]: overall HEALTH_OK 2024-03-18T22:59:53.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:53 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T22:59:53.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:53 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T22:59:53.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:53 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T22:59:53.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:53 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-18T22:59:53.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:53 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T22:59:53.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:59:53 smithi049 ceph-mon[62991]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T22:59:53.881 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:59:53 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-18T22:59:53.881 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:59:53 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-18T22:59:53.881 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:59:53 smithi049 ceph-mon[62991]: mon.a calling monitor election 2024-03-18T22:59:53.881 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:59:53 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-18T22:59:53.881 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:59:53 smithi049 ceph-mon[62991]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T22:59:53.881 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:59:53 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-18T22:59:53.881 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:59:53 smithi049 ceph-mon[62991]: mon.c calling monitor election 2024-03-18T22:59:53.881 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:59:53 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-18T22:59:53.881 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:59:53 smithi049 ceph-mon[62991]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T22:59:53.881 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:59:53 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-18T22:59:53.881 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:59:53 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-18T22:59:53.881 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:59:53 smithi049 ceph-mon[62991]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-03-18T22:59:53.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:59:53 smithi049 ceph-mon[62991]: monmap epoch 2 2024-03-18T22:59:53.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:59:53 smithi049 ceph-mon[62991]: fsid c63591ea-e57a-11ee-95c9-87774f69a715 2024-03-18T22:59:53.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:59:53 smithi049 ceph-mon[62991]: last_changed 2024-03-18T22:59:48.347509+0000 2024-03-18T22:59:53.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:59:53 smithi049 ceph-mon[62991]: created 2024-03-18T22:57:55.689519+0000 2024-03-18T22:59:53.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:59:53 smithi049 ceph-mon[62991]: min_mon_release 19 (squid) 2024-03-18T22:59:53.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:59:53 smithi049 ceph-mon[62991]: election_strategy: 1 2024-03-18T22:59:53.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:59:53 smithi049 ceph-mon[62991]: 0: [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] mon.a 2024-03-18T22:59:53.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:59:53 smithi049 ceph-mon[62991]: 1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.c 2024-03-18T22:59:53.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:59:53 smithi049 ceph-mon[62991]: fsmap 2024-03-18T22:59:53.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:59:53 smithi049 ceph-mon[62991]: osdmap e4: 0 total, 0 up, 0 in 2024-03-18T22:59:53.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:59:53 smithi049 ceph-mon[62991]: mgrmap e13: x(active, since 53s) 2024-03-18T22:59:53.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:59:53 smithi049 ceph-mon[62991]: overall HEALTH_OK 2024-03-18T22:59:53.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:59:53 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T22:59:53.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:59:53 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T22:59:53.883 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:59:53 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T22:59:53.883 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:59:53 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-18T22:59:53.883 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:59:53 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T22:59:54.659 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T22:59:54.657+0000 7f3c9c797700 1 -- 172.21.15.175:0/1986834944 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3c800035c0 msgr2=0x7f3c80005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T22:59:54.659 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T22:59:54.658+0000 7f3c9c797700 1 --2- 172.21.15.175:0/1986834944 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3c800035c0 0x7f3c80005a50 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f3c94101d20 tx=0x7f3c8c009e80 comp rx=0 tx=0).stop 2024-03-18T22:59:54.660 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T22:59:54.658+0000 7f3c9c797700 1 -- 172.21.15.175:0/1986834944 shutdown_connections 2024-03-18T22:59:54.660 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T22:59:54.658+0000 7f3c9c797700 1 --2- 172.21.15.175:0/1986834944 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3c800035c0 0x7f3c80005a50 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T22:59:54.660 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T22:59:54.658+0000 7f3c9c797700 1 -- 172.21.15.175:0/1986834944 >> 172.21.15.175:0/1986834944 conn(0x7f3c940fd8e0 msgr2=0x7f3c940ffd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T22:59:54.660 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T22:59:54.658+0000 7f3c9c797700 1 -- 172.21.15.175:0/1986834944 shutdown_connections 2024-03-18T22:59:54.660 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T22:59:54.658+0000 7f3c9c797700 1 -- 172.21.15.175:0/1986834944 wait complete. 2024-03-18T22:59:54.661 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T22:59:54.659+0000 7f3c9c797700 1 Processor -- start 2024-03-18T22:59:54.661 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T22:59:54.659+0000 7f3c9c797700 1 -- start start 2024-03-18T22:59:54.661 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T22:59:54.659+0000 7f3c9c797700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3c800035c0 0x7f3c9419f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T22:59:54.661 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T22:59:54.659+0000 7f3c9c797700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3c9419fd10 0x7f3c94199990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T22:59:54.661 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T22:59:54.659+0000 7f3c9c797700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f3c94109e80 con 0x7f3c9419fd10 2024-03-18T22:59:54.661 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T22:59:54.659+0000 7f3c9c797700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f3c94109d00 con 0x7f3c800035c0 2024-03-18T22:59:54.661 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T22:59:54.659+0000 7f3c9a533700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3c800035c0 0x7f3c9419f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T22:59:54.662 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T22:59:54.659+0000 7f3c9a533700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3c800035c0 0x7f3c9419f7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:35832/0 (socket says 172.21.15.175:35832) 2024-03-18T22:59:54.662 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T22:59:54.659+0000 7f3c9a533700 1 -- 172.21.15.175:0/1070005547 learned_addr learned my addr 172.21.15.175:0/1070005547 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-03-18T22:59:54.662 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T22:59:54.659+0000 7f3c99d32700 1 --2- 172.21.15.175:0/1070005547 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3c9419fd10 0x7f3c94199990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T22:59:54.662 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T22:59:54.659+0000 7f3c9a533700 1 -- 172.21.15.175:0/1070005547 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3c800035c0 msgr2=0x7f3c9419f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-03-18T22:59:54.662 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T22:59:54.660+0000 7f3c9a533700 1 -- 172.21.15.175:0/1070005547 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3c800035c0 msgr2=0x7f3c9419f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-18T22:59:54.662 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T22:59:54.660+0000 7f3c9a533700 1 --2- 172.21.15.175:0/1070005547 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3c800035c0 0x7f3c9419f7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-18T22:59:54.662 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T22:59:54.660+0000 7f3c9a533700 1 --2- 172.21.15.175:0/1070005547 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3c800035c0 0x7f3c9419f7d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-18T22:59:54.662 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T22:59:54.660+0000 7f3c99d32700 1 -- 172.21.15.175:0/1070005547 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3c800035c0 msgr2=0x7f3c9419f7d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-18T22:59:54.662 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T22:59:54.660+0000 7f3c99d32700 1 --2- 172.21.15.175:0/1070005547 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3c800035c0 0x7f3c9419f7d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T22:59:54.662 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T22:59:54.660+0000 7f3c99d32700 1 -- 172.21.15.175:0/1070005547 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c941a0350 con 0x7f3c9419fd10 2024-03-18T22:59:54.663 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T22:59:54.661+0000 7f3c99d32700 1 --2- 172.21.15.175:0/1070005547 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3c9419fd10 0x7f3c94199990 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f3c8400b4d0 tx=0x7f3c8400b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T22:59:54.664 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T22:59:54.661+0000 7f3c8b7fe700 1 -- 172.21.15.175:0/1070005547 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f3c84011630 con 0x7f3c9419fd10 2024-03-18T22:59:54.664 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T22:59:54.661+0000 7f3c9c797700 1 -- 172.21.15.175:0/1070005547 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c94199ed0 con 0x7f3c9419fd10 2024-03-18T22:59:54.664 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T22:59:54.661+0000 7f3c8b7fe700 1 -- 172.21.15.175:0/1070005547 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c84011cb0 con 0x7f3c9419fd10 2024-03-18T22:59:54.664 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T22:59:54.661+0000 7f3c8b7fe700 1 -- 172.21.15.175:0/1070005547 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f3c84010e60 con 0x7f3c9419fd10 2024-03-18T22:59:54.664 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T22:59:54.661+0000 7f3c9c797700 1 -- 172.21.15.175:0/1070005547 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c9419a460 con 0x7f3c9419fd10 2024-03-18T22:59:54.664 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T22:59:54.662+0000 7f3c8b7fe700 1 -- 172.21.15.175:0/1070005547 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 13) ==== 46162+0+0 (secure 0 0 0) 0x7f3c84018020 con 0x7f3c9419fd10 2024-03-18T22:59:54.664 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T22:59:54.662+0000 7f3c9c797700 1 -- 172.21.15.175:0/1070005547 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c94103530 con 0x7f3c9419fd10 2024-03-18T22:59:54.664 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T22:59:54.662+0000 7f3c8b7fe700 1 --2- 172.21.15.175:0/1070005547 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f3c78038d10 0x7f3c7803b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T22:59:54.664 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T22:59:54.663+0000 7f3c8b7fe700 1 -- 172.21.15.175:0/1070005547 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f3c8404ba90 con 0x7f3c9419fd10 2024-03-18T22:59:54.664 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T22:59:54.663+0000 7f3c9a533700 1 --2- 172.21.15.175:0/1070005547 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f3c78038d10 0x7f3c7803b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T22:59:54.665 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T22:59:54.663+0000 7f3c9a533700 1 --2- 172.21.15.175:0/1070005547 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f3c78038d10 0x7f3c7803b1d0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f3c8c019000 tx=0x7f3c8c0074e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T22:59:54.669 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T22:59:54.669+0000 7f3c8b7fe700 1 -- 172.21.15.175:0/1070005547 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f3c8401f020 con 0x7f3c9419fd10 2024-03-18T22:59:54.711 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:54 smithi175 ceph-mon[67352]: Deploying daemon mon.b on smithi139 2024-03-18T22:59:54.711 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:54 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-18T22:59:54.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:59:54 smithi049 ceph-mon[62991]: Deploying daemon mon.b on smithi139 2024-03-18T22:59:54.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:59:54 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-18T22:59:54.948 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T22:59:54.947+0000 7f3c9c797700 1 -- 172.21.15.175:0/1070005547 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f3c9419aec0 con 0x7f3c9419fd10 2024-03-18T22:59:54.949 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T22:59:54.948+0000 7f3c8b7fe700 1 -- 172.21.15.175:0/1070005547 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1015 (secure 0 0 0) 0x7f3c8401a400 con 0x7f3c9419fd10 2024-03-18T22:59:54.949 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-03-18T22:59:54.949 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":2,"fsid":"c63591ea-e57a-11ee-95c9-87774f69a715","modified":"2024-03-18T22:59:48.347509Z","created":"2024-03-18T22:57:55.689519Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:3300","nonce":0},{"type":"v1","addr":"172.21.15.49:6789","nonce":0}]},"addr":"172.21.15.49:6789/0","public_addr":"172.21.15.49:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:3300","nonce":0},{"type":"v1","addr":"172.21.15.175:6789","nonce":0}]},"addr":"172.21.15.175:6789/0","public_addr":"172.21.15.175:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-03-18T22:59:54.951 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T22:59:54.950+0000 7f3c9c797700 1 -- 172.21.15.175:0/1070005547 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f3c78038d10 msgr2=0x7f3c7803b1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T22:59:54.951 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T22:59:54.950+0000 7f3c9c797700 1 --2- 172.21.15.175:0/1070005547 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f3c78038d10 0x7f3c7803b1d0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f3c8c019000 tx=0x7f3c8c0074e0 comp rx=0 tx=0).stop 2024-03-18T22:59:54.951 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T22:59:54.950+0000 7f3c9c797700 1 -- 172.21.15.175:0/1070005547 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3c9419fd10 msgr2=0x7f3c94199990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T22:59:54.951 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T22:59:54.950+0000 7f3c9c797700 1 --2- 172.21.15.175:0/1070005547 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3c9419fd10 0x7f3c94199990 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f3c8400b4d0 tx=0x7f3c8400b890 comp rx=0 tx=0).stop 2024-03-18T22:59:54.951 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T22:59:54.950+0000 7f3c9c797700 1 -- 172.21.15.175:0/1070005547 shutdown_connections 2024-03-18T22:59:54.951 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T22:59:54.950+0000 7f3c9c797700 1 --2- 172.21.15.175:0/1070005547 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f3c78038d10 0x7f3c7803b1d0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T22:59:54.951 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T22:59:54.950+0000 7f3c9c797700 1 --2- 172.21.15.175:0/1070005547 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3c800035c0 0x7f3c9419f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T22:59:54.952 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T22:59:54.950+0000 7f3c9c797700 1 --2- 172.21.15.175:0/1070005547 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3c9419fd10 0x7f3c94199990 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T22:59:54.952 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T22:59:54.950+0000 7f3c9c797700 1 -- 172.21.15.175:0/1070005547 >> 172.21.15.175:0/1070005547 conn(0x7f3c940fd8e0 msgr2=0x7f3c940fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T22:59:54.952 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T22:59:54.950+0000 7f3c9c797700 1 -- 172.21.15.175:0/1070005547 shutdown_connections 2024-03-18T22:59:54.952 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T22:59:54.950+0000 7f3c9c797700 1 -- 172.21.15.175:0/1070005547 wait complete. 2024-03-18T22:59:54.952 INFO:teuthology.orchestra.run.smithi175.stderr:dumped monmap epoch 2 2024-03-18T22:59:55.502 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:55 smithi175 ceph-mon[67352]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T22:59:55.502 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 22:59:55 smithi175 ceph-mon[67352]: from='client.? 172.21.15.175:0/1070005547' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-18T22:59:55.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:59:55 smithi049 ceph-mon[62991]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T22:59:55.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 22:59:55 smithi049 ceph-mon[62991]: from='client.? 172.21.15.175:0/1070005547' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-18T22:59:56.530 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-03-18T22:59:56.531 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c63591ea-e57a-11ee-95c9-87774f69a715 -- ceph mon dump -f json 2024-03-18T22:59:56.775 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/mon.c/config 2024-03-18T22:59:57.065 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:56 smithi139 podman[67263]: 2024-03-18 22:59:56.963096539 +0000 UTC m=+0.118955764 container create 127fbf6c9088fad00e275208d3b9999915d73439c03ce4c63e2469f97930981b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mon-b, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240318.181317-b6ae249) 2024-03-18T22:59:57.345 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 podman[67263]: 2024-03-18 22:59:57.173626663 +0000 UTC m=+0.329485907 container init 127fbf6c9088fad00e275208d3b9999915d73439c03ce4c63e2469f97930981b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mon-b, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240318.181317-b6ae249) 2024-03-18T22:59:57.345 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 podman[67263]: 2024-03-18 22:59:57.176906487 +0000 UTC m=+0.332765712 container start 127fbf6c9088fad00e275208d3b9999915d73439c03ce4c63e2469f97930981b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mon-b, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5) 2024-03-18T22:59:57.345 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: set uid:gid to 167:167 (ceph:ceph) 2024-03-18T22:59:57.345 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: ceph version 19.0.0-2206-gb6ae2491 (b6ae24918b03b1b8a19c9263a50655bba80bb810) squid (dev), process ceph-mon, pid 2 2024-03-18T22:59:57.345 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: pidfile_write: ignore empty --pid-file 2024-03-18T22:59:57.345 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: load: jerasure load: lrc 2024-03-18T22:59:57.345 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: rocksdb: RocksDB version: 7.9.2 2024-03-18T22:59:57.346 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: rocksdb: Git sha 0 2024-03-18T22:59:57.346 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: rocksdb: Compile date 2024-03-18 18:57:12 2024-03-18T22:59:57.346 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: rocksdb: DB SUMMARY 2024-03-18T22:59:57.346 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: rocksdb: DB Session ID: GQW1HG93NC4ND04J9GCF 2024-03-18T22:59:57.346 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: rocksdb: CURRENT file: CURRENT 2024-03-18T22:59:57.346 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: rocksdb: IDENTITY file: IDENTITY 2024-03-18T22:59:57.346 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-03-18T22:59:57.346 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 0, files: 2024-03-18T22:59:57.346 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000004.log size: 636 ; 2024-03-18T22:59:57.346 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: rocksdb: Options.error_if_exists: 0 2024-03-18T22:59:57.346 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: rocksdb: Options.create_if_missing: 0 2024-03-18T22:59:57.346 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: rocksdb: Options.paranoid_checks: 1 2024-03-18T22:59:57.346 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: rocksdb: Options.flush_verify_memtable_count: 1 2024-03-18T22:59:57.347 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-03-18T22:59:57.347 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-03-18T22:59:57.347 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: rocksdb: Options.env: 0x557bfd9d5c60 2024-03-18T22:59:57.347 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: rocksdb: Options.fs: PosixFileSystem 2024-03-18T22:59:57.347 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: rocksdb: Options.info_log: 0x557c00071ae0 2024-03-18T22:59:57.347 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: rocksdb: Options.max_file_opening_threads: 16 2024-03-18T22:59:57.348 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: rocksdb: Options.statistics: (nil) 2024-03-18T22:59:57.348 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: rocksdb: Options.use_fsync: 0 2024-03-18T22:59:57.348 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: rocksdb: Options.max_log_file_size: 0 2024-03-18T22:59:57.348 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-03-18T22:59:57.348 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: rocksdb: Options.log_file_time_to_roll: 0 2024-03-18T22:59:57.348 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: rocksdb: Options.keep_log_file_num: 1000 2024-03-18T22:59:57.348 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: rocksdb: Options.recycle_log_file_num: 0 2024-03-18T22:59:57.348 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: rocksdb: Options.allow_fallocate: 1 2024-03-18T22:59:57.348 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: rocksdb: Options.allow_mmap_reads: 0 2024-03-18T22:59:57.348 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: rocksdb: Options.allow_mmap_writes: 0 2024-03-18T22:59:57.348 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: rocksdb: Options.use_direct_reads: 0 2024-03-18T22:59:57.348 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-03-18T22:59:57.349 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: rocksdb: Options.create_missing_column_families: 0 2024-03-18T22:59:57.349 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: rocksdb: Options.db_log_dir: 2024-03-18T22:59:57.349 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: rocksdb: Options.wal_dir: 2024-03-18T22:59:57.349 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: rocksdb: Options.table_cache_numshardbits: 6 2024-03-18T22:59:57.349 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: rocksdb: Options.WAL_ttl_seconds: 0 2024-03-18T22:59:57.349 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: rocksdb: Options.WAL_size_limit_MB: 0 2024-03-18T22:59:57.349 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-03-18T22:59:57.349 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-03-18T22:59:57.349 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: rocksdb: Options.is_fd_close_on_exec: 1 2024-03-18T22:59:57.349 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: rocksdb: Options.advise_random_on_open: 1 2024-03-18T22:59:57.349 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: rocksdb: Options.db_write_buffer_size: 0 2024-03-18T22:59:57.349 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: rocksdb: Options.write_buffer_manager: 0x557c000441e0 2024-03-18T22:59:57.350 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-03-18T22:59:57.350 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-03-18T22:59:57.350 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: rocksdb: Options.use_adaptive_mutex: 0 2024-03-18T22:59:57.350 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: rocksdb: Options.rate_limiter: (nil) 2024-03-18T22:59:57.350 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-03-18T22:59:57.350 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: rocksdb: Options.wal_recovery_mode: 2 2024-03-18T22:59:57.350 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: rocksdb: Options.enable_thread_tracking: 0 2024-03-18T22:59:57.350 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: rocksdb: Options.enable_pipelined_write: 0 2024-03-18T22:59:57.350 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: rocksdb: Options.unordered_write: 0 2024-03-18T22:59:57.350 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-03-18T22:59:57.350 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-03-18T22:59:57.350 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-03-18T22:59:57.350 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-03-18T22:59:57.351 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: rocksdb: Options.row_cache: None 2024-03-18T22:59:57.351 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: rocksdb: Options.wal_filter: None 2024-03-18T22:59:57.351 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-03-18T22:59:57.351 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: rocksdb: Options.allow_ingest_behind: 0 2024-03-18T22:59:57.351 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: rocksdb: Options.two_write_queues: 0 2024-03-18T22:59:57.351 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: rocksdb: Options.manual_wal_flush: 0 2024-03-18T22:59:57.351 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: rocksdb: Options.wal_compression: 0 2024-03-18T22:59:57.351 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: rocksdb: Options.atomic_flush: 0 2024-03-18T22:59:57.351 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-03-18T22:59:57.351 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: rocksdb: Options.persist_stats_to_disk: 0 2024-03-18T22:59:57.351 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: rocksdb: Options.write_dbid_to_manifest: 0 2024-03-18T22:59:57.351 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: rocksdb: Options.log_readahead_size: 0 2024-03-18T22:59:57.351 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-03-18T22:59:57.352 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: rocksdb: Options.best_efforts_recovery: 0 2024-03-18T22:59:57.352 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-03-18T22:59:57.352 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-03-18T22:59:57.352 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: rocksdb: Options.allow_data_in_errors: 0 2024-03-18T22:59:57.352 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: rocksdb: Options.db_host_id: __hostname__ 2024-03-18T22:59:57.352 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: rocksdb: Options.enforce_single_del_contracts: true 2024-03-18T22:59:57.352 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: rocksdb: Options.max_background_jobs: 2 2024-03-18T22:59:57.352 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: rocksdb: Options.max_background_compactions: -1 2024-03-18T22:59:57.352 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: rocksdb: Options.max_subcompactions: 1 2024-03-18T22:59:57.352 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-03-18T22:59:57.352 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-03-18T22:59:57.352 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: rocksdb: Options.delayed_write_rate : 16777216 2024-03-18T22:59:57.352 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: rocksdb: Options.max_total_wal_size: 0 2024-03-18T22:59:57.352 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-03-18T22:59:57.352 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: rocksdb: Options.stats_dump_period_sec: 600 2024-03-18T22:59:57.353 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: rocksdb: Options.stats_persist_period_sec: 600 2024-03-18T22:59:57.353 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-03-18T22:59:57.353 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: rocksdb: Options.max_open_files: -1 2024-03-18T22:59:57.353 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: rocksdb: Options.bytes_per_sync: 0 2024-03-18T22:59:57.353 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: rocksdb: Options.wal_bytes_per_sync: 0 2024-03-18T22:59:57.353 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: rocksdb: Options.strict_bytes_per_sync: 0 2024-03-18T22:59:57.353 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: rocksdb: Options.compaction_readahead_size: 0 2024-03-18T22:59:57.353 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: rocksdb: Options.max_background_flushes: -1 2024-03-18T22:59:57.353 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: rocksdb: Compression algorithms supported: 2024-03-18T22:59:57.353 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-03-18T22:59:57.353 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: rocksdb: kZSTD supported: 0 2024-03-18T22:59:57.353 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: rocksdb: kXpressCompression supported: 0 2024-03-18T22:59:57.353 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: rocksdb: kLZ4HCCompression supported: 1 2024-03-18T22:59:57.353 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: rocksdb: kZlibCompression supported: 1 2024-03-18T22:59:57.353 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: rocksdb: kSnappyCompression supported: 1 2024-03-18T22:59:57.354 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: rocksdb: kLZ4Compression supported: 1 2024-03-18T22:59:57.354 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: rocksdb: kBZip2Compression supported: 0 2024-03-18T22:59:57.354 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: rocksdb: Fast CRC32 supported: Supported on x86 2024-03-18T22:59:57.354 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: rocksdb: DMutex implementation: pthread_mutex_t 2024-03-18T22:59:57.354 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 2024-03-18T22:59:57.354 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-03-18T22:59:57.354 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-03-18T22:59:57.354 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: rocksdb: Options.merge_operator: 2024-03-18T22:59:57.354 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: rocksdb: Options.compaction_filter: None 2024-03-18T22:59:57.354 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: rocksdb: Options.compaction_filter_factory: None 2024-03-18T22:59:57.354 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: rocksdb: Options.sst_partitioner_factory: None 2024-03-18T22:59:57.354 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: rocksdb: Options.memtable_factory: SkipListFactory 2024-03-18T22:59:57.354 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: rocksdb: Options.table_factory: BlockBasedTable 2024-03-18T22:59:57.355 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x557c00071be0) 2024-03-18T22:59:57.355 INFO:journalctl@ceph.mon.b.smithi139.stdout: cache_index_and_filter_blocks: 1 2024-03-18T22:59:57.355 INFO:journalctl@ceph.mon.b.smithi139.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-03-18T22:59:57.355 INFO:journalctl@ceph.mon.b.smithi139.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-03-18T22:59:57.355 INFO:journalctl@ceph.mon.b.smithi139.stdout: pin_top_level_index_and_filter: 1 2024-03-18T22:59:57.355 INFO:journalctl@ceph.mon.b.smithi139.stdout: index_type: 0 2024-03-18T22:59:57.355 INFO:journalctl@ceph.mon.b.smithi139.stdout: data_block_index_type: 0 2024-03-18T22:59:57.355 INFO:journalctl@ceph.mon.b.smithi139.stdout: index_shortening: 1 2024-03-18T22:59:57.355 INFO:journalctl@ceph.mon.b.smithi139.stdout: data_block_hash_table_util_ratio: 0.750000 2024-03-18T22:59:57.355 INFO:journalctl@ceph.mon.b.smithi139.stdout: checksum: 4 2024-03-18T22:59:57.355 INFO:journalctl@ceph.mon.b.smithi139.stdout: no_block_cache: 0 2024-03-18T22:59:57.355 INFO:journalctl@ceph.mon.b.smithi139.stdout: block_cache: 0x557c00077090 2024-03-18T22:59:57.355 INFO:journalctl@ceph.mon.b.smithi139.stdout: block_cache_name: BinnedLRUCache 2024-03-18T22:59:57.355 INFO:journalctl@ceph.mon.b.smithi139.stdout: block_cache_options: 2024-03-18T22:59:57.355 INFO:journalctl@ceph.mon.b.smithi139.stdout: capacity : 536870912 2024-03-18T22:59:57.356 INFO:journalctl@ceph.mon.b.smithi139.stdout: num_shard_bits : 4 2024-03-18T22:59:57.356 INFO:journalctl@ceph.mon.b.smithi139.stdout: strict_capacity_limit : 0 2024-03-18T22:59:57.356 INFO:journalctl@ceph.mon.b.smithi139.stdout: high_pri_pool_ratio: 0.000 2024-03-18T22:59:57.356 INFO:journalctl@ceph.mon.b.smithi139.stdout: block_cache_compressed: (nil) 2024-03-18T22:59:57.356 INFO:journalctl@ceph.mon.b.smithi139.stdout: persistent_cache: (nil) 2024-03-18T22:59:57.356 INFO:journalctl@ceph.mon.b.smithi139.stdout: block_size: 4096 2024-03-18T22:59:57.356 INFO:journalctl@ceph.mon.b.smithi139.stdout: block_size_deviation: 10 2024-03-18T22:59:57.356 INFO:journalctl@ceph.mon.b.smithi139.stdout: block_restart_interval: 16 2024-03-18T22:59:57.356 INFO:journalctl@ceph.mon.b.smithi139.stdout: index_block_restart_interval: 1 2024-03-18T22:59:57.356 INFO:journalctl@ceph.mon.b.smithi139.stdout: metadata_block_size: 4096 2024-03-18T22:59:57.356 INFO:journalctl@ceph.mon.b.smithi139.stdout: partition_filters: 0 2024-03-18T22:59:57.356 INFO:journalctl@ceph.mon.b.smithi139.stdout: use_delta_encoding: 1 2024-03-18T22:59:57.356 INFO:journalctl@ceph.mon.b.smithi139.stdout: filter_policy: bloomfilter 2024-03-18T22:59:57.356 INFO:journalctl@ceph.mon.b.smithi139.stdout: whole_key_filtering: 1 2024-03-18T22:59:57.356 INFO:journalctl@ceph.mon.b.smithi139.stdout: verify_compression: 0 2024-03-18T22:59:57.357 INFO:journalctl@ceph.mon.b.smithi139.stdout: read_amp_bytes_per_bit: 0 2024-03-18T22:59:57.357 INFO:journalctl@ceph.mon.b.smithi139.stdout: format_version: 5 2024-03-18T22:59:57.357 INFO:journalctl@ceph.mon.b.smithi139.stdout: enable_index_compression: 1 2024-03-18T22:59:57.357 INFO:journalctl@ceph.mon.b.smithi139.stdout: block_align: 0 2024-03-18T22:59:57.357 INFO:journalctl@ceph.mon.b.smithi139.stdout: max_auto_readahead_size: 262144 2024-03-18T22:59:57.357 INFO:journalctl@ceph.mon.b.smithi139.stdout: prepopulate_block_cache: 0 2024-03-18T22:59:57.357 INFO:journalctl@ceph.mon.b.smithi139.stdout: initial_auto_readahead_size: 8192 2024-03-18T22:59:57.357 INFO:journalctl@ceph.mon.b.smithi139.stdout: num_file_reads_for_auto_readahead: 2 2024-03-18T22:59:57.357 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: rocksdb: Options.write_buffer_size: 33554432 2024-03-18T22:59:57.357 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: rocksdb: Options.max_write_buffer_number: 2 2024-03-18T22:59:57.357 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: rocksdb: Options.compression: NoCompression 2024-03-18T22:59:57.357 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: rocksdb: Options.bottommost_compression: Disabled 2024-03-18T22:59:57.357 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: rocksdb: Options.prefix_extractor: nullptr 2024-03-18T22:59:57.357 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-03-18T22:59:57.357 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: rocksdb: Options.num_levels: 7 2024-03-18T22:59:57.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-03-18T22:59:57.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-03-18T22:59:57.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-03-18T22:59:57.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-03-18T22:59:57.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-03-18T22:59:57.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-03-18T22:59:57.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-03-18T22:59:57.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-03-18T22:59:57.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-03-18T22:59:57.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-03-18T22:59:57.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-03-18T22:59:57.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-03-18T22:59:57.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: rocksdb: Options.compression_opts.window_bits: -14 2024-03-18T22:59:57.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: rocksdb: Options.compression_opts.level: 32767 2024-03-18T22:59:57.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: rocksdb: Options.compression_opts.strategy: 0 2024-03-18T22:59:57.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-03-18T22:59:57.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-03-18T22:59:57.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-03-18T22:59:57.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-03-18T22:59:57.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: rocksdb: Options.compression_opts.enabled: false 2024-03-18T22:59:57.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-03-18T22:59:57.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-03-18T22:59:57.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-03-18T22:59:57.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-03-18T22:59:57.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: rocksdb: Options.target_file_size_base: 67108864 2024-03-18T22:59:57.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: rocksdb: Options.target_file_size_multiplier: 1 2024-03-18T22:59:57.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-03-18T22:59:57.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-03-18T22:59:57.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-03-18T22:59:57.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-03-18T22:59:57.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-03-18T22:59:57.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-03-18T22:59:57.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-03-18T22:59:57.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-03-18T22:59:57.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-03-18T22:59:57.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-03-18T22:59:57.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-03-18T22:59:57.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-03-18T22:59:57.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-03-18T22:59:57.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: rocksdb: Options.arena_block_size: 1048576 2024-03-18T22:59:57.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-03-18T22:59:57.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-03-18T22:59:57.362 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: rocksdb: Options.disable_auto_compactions: 0 2024-03-18T22:59:57.362 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-03-18T22:59:57.362 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-03-18T22:59:57.362 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-03-18T22:59:57.362 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-03-18T22:59:57.362 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-03-18T22:59:57.362 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-03-18T22:59:57.362 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-03-18T22:59:57.362 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-03-18T22:59:57.362 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-03-18T22:59:57.362 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-03-18T22:59:57.362 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: rocksdb: Options.table_properties_collectors: 2024-03-18T22:59:57.362 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: rocksdb: Options.inplace_update_support: 0 2024-03-18T22:59:57.363 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: rocksdb: Options.inplace_update_num_locks: 10000 2024-03-18T22:59:57.363 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-03-18T22:59:57.363 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-03-18T22:59:57.363 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: rocksdb: Options.memtable_huge_page_size: 0 2024-03-18T22:59:57.363 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: rocksdb: Options.bloom_locality: 0 2024-03-18T22:59:57.363 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: rocksdb: Options.max_successive_merges: 0 2024-03-18T22:59:57.363 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: rocksdb: Options.optimize_filters_for_hits: 0 2024-03-18T22:59:57.363 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: rocksdb: Options.paranoid_file_checks: 0 2024-03-18T22:59:57.363 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: rocksdb: Options.force_consistency_checks: 1 2024-03-18T22:59:57.363 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: rocksdb: Options.report_bg_io_stats: 0 2024-03-18T22:59:57.363 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: rocksdb: Options.ttl: 2592000 2024-03-18T22:59:57.363 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: rocksdb: Options.periodic_compaction_seconds: 0 2024-03-18T22:59:57.363 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-03-18T22:59:57.363 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-03-18T22:59:57.364 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: rocksdb: Options.enable_blob_files: false 2024-03-18T22:59:57.364 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: rocksdb: Options.min_blob_size: 0 2024-03-18T22:59:57.364 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: rocksdb: Options.blob_file_size: 268435456 2024-03-18T22:59:57.364 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: rocksdb: Options.blob_compression_type: NoCompression 2024-03-18T22:59:57.364 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: rocksdb: Options.enable_blob_garbage_collection: false 2024-03-18T22:59:57.364 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-03-18T22:59:57.364 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-03-18T22:59:57.364 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-03-18T22:59:57.364 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: rocksdb: Options.blob_file_starting_level: 0 2024-03-18T22:59:57.364 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-03-18T22:59:57.364 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-03-18T22:59:57.364 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-03-18T22:59:57.364 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 34ecb772-0b67-486d-bb16-af728eccf434 2024-03-18T22:59:57.364 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: rocksdb: EVENT_LOG_v1 {"time_micros": 1710802797207213, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-03-18T22:59:57.365 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-03-18T22:59:57.365 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: rocksdb: EVENT_LOG_v1 {"time_micros": 1710802797207821, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1747, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 648, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1710802797, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "34ecb772-0b67-486d-bb16-af728eccf434", "db_session_id": "GQW1HG93NC4ND04J9GCF", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-03-18T22:59:57.365 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: rocksdb: EVENT_LOG_v1 {"time_micros": 1710802797207926, "job": 1, "event": "recovery_finished"} 2024-03-18T22:59:57.365 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-03-18T22:59:57.365 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-03-18T22:59:57.365 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x557c0016a000 2024-03-18T22:59:57.365 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: rocksdb: DB pointer 0x557c00154000 2024-03-18T22:59:57.365 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-03-18T22:59:57.365 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-03-18T22:59:57.365 INFO:journalctl@ceph.mon.b.smithi139.stdout: ** DB Stats ** 2024-03-18T22:59:57.365 INFO:journalctl@ceph.mon.b.smithi139.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-18T22:59:57.365 INFO:journalctl@ceph.mon.b.smithi139.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-03-18T22:59:57.365 INFO:journalctl@ceph.mon.b.smithi139.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-18T22:59:57.365 INFO:journalctl@ceph.mon.b.smithi139.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-18T22:59:57.366 INFO:journalctl@ceph.mon.b.smithi139.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-03-18T22:59:57.366 INFO:journalctl@ceph.mon.b.smithi139.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-18T22:59:57.366 INFO:journalctl@ceph.mon.b.smithi139.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-18T22:59:57.366 INFO:journalctl@ceph.mon.b.smithi139.stdout: 2024-03-18T22:59:57.366 INFO:journalctl@ceph.mon.b.smithi139.stdout: ** Compaction Stats [default] ** 2024-03-18T22:59:57.366 INFO:journalctl@ceph.mon.b.smithi139.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-18T22:59:57.366 INFO:journalctl@ceph.mon.b.smithi139.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-03-18T22:59:57.366 INFO:journalctl@ceph.mon.b.smithi139.stdout: L0 1/0 1.71 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-18T22:59:57.366 INFO:journalctl@ceph.mon.b.smithi139.stdout: Sum 1/0 1.71 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-18T22:59:57.366 INFO:journalctl@ceph.mon.b.smithi139.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-18T22:59:57.366 INFO:journalctl@ceph.mon.b.smithi139.stdout: 2024-03-18T22:59:57.366 INFO:journalctl@ceph.mon.b.smithi139.stdout: ** Compaction Stats [default] ** 2024-03-18T22:59:57.366 INFO:journalctl@ceph.mon.b.smithi139.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-18T22:59:57.366 INFO:journalctl@ceph.mon.b.smithi139.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-03-18T22:59:57.367 INFO:journalctl@ceph.mon.b.smithi139.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 2.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-18T22:59:57.367 INFO:journalctl@ceph.mon.b.smithi139.stdout: 2024-03-18T22:59:57.367 INFO:journalctl@ceph.mon.b.smithi139.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-03-18T22:59:57.367 INFO:journalctl@ceph.mon.b.smithi139.stdout: 2024-03-18T22:59:57.367 INFO:journalctl@ceph.mon.b.smithi139.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-18T22:59:57.367 INFO:journalctl@ceph.mon.b.smithi139.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-03-18T22:59:57.367 INFO:journalctl@ceph.mon.b.smithi139.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-03-18T22:59:57.367 INFO:journalctl@ceph.mon.b.smithi139.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-03-18T22:59:57.367 INFO:journalctl@ceph.mon.b.smithi139.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-03-18T22:59:57.367 INFO:journalctl@ceph.mon.b.smithi139.stdout: AddFile(Keys): cumulative 0, interval 0 2024-03-18T22:59:57.367 INFO:journalctl@ceph.mon.b.smithi139.stdout: Cumulative compaction: 0.00 GB write, 0.50 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-18T22:59:57.367 INFO:journalctl@ceph.mon.b.smithi139.stdout: Interval compaction: 0.00 GB write, 0.50 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-18T22:59:57.367 INFO:journalctl@ceph.mon.b.smithi139.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-03-18T22:59:57.367 INFO:journalctl@ceph.mon.b.smithi139.stdout: Block cache BinnedLRUCache@0x557c00077090#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 8e-06 secs_since: 0 2024-03-18T22:59:57.368 INFO:journalctl@ceph.mon.b.smithi139.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-03-18T22:59:57.368 INFO:journalctl@ceph.mon.b.smithi139.stdout: 2024-03-18T22:59:57.368 INFO:journalctl@ceph.mon.b.smithi139.stdout: ** File Read Latency Histogram By Level [default] ** 2024-03-18T22:59:57.368 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: mon.b does not exist in monmap, will attempt to join an existing cluster 2024-03-18T22:59:57.368 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: using public_addr v2:172.21.15.139:0/0 -> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] 2024-03-18T22:59:57.368 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: starting mon.b rank -1 at public addrs [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] at bind addrs [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid c63591ea-e57a-11ee-95c9-87774f69a715 2024-03-18T22:59:57.368 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: mon.b@-1(???) e0 preinit fsid c63591ea-e57a-11ee-95c9-87774f69a715 2024-03-18T22:59:57.368 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: mon.b@-1(synchronizing).mds e1 new map 2024-03-18T22:59:57.368 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: mon.b@-1(synchronizing).mds e1 print_map 2024-03-18T22:59:57.368 INFO:journalctl@ceph.mon.b.smithi139.stdout: e1 2024-03-18T22:59:57.368 INFO:journalctl@ceph.mon.b.smithi139.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-03-18T22:59:57.368 INFO:journalctl@ceph.mon.b.smithi139.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-03-18T22:59:57.368 INFO:journalctl@ceph.mon.b.smithi139.stdout: legacy client fscid: -1 2024-03-18T22:59:57.368 INFO:journalctl@ceph.mon.b.smithi139.stdout: 2024-03-18T22:59:57.369 INFO:journalctl@ceph.mon.b.smithi139.stdout: No filesystems configured 2024-03-18T22:59:57.369 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-03-18T22:59:57.369 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-03-18T22:59:57.369 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-03-18T22:59:57.369 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-03-18T22:59:57.369 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-03-18T22:59:57.369 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-03-18T22:59:57.369 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-03-18T22:59:57.369 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-03-18T22:59:57.369 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-03-18T22:59:57.369 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-03-18T22:59:57.369 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T22:59:57.369 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T22:59:57.369 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: Added host smithi139 2024-03-18T22:59:57.369 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T22:59:57.370 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T22:59:57.370 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T22:59:57.370 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T22:59:57.370 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-18T22:59:57.370 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T22:59:57.370 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T22:59:57.370 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T22:59:57.370 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi175", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T22:59:57.370 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T22:59:57.370 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T22:59:57.370 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T22:59:57.370 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T22:59:57.370 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi139", "name": "osd_memory_target"}]: dispatch 2024-03-18T22:59:57.370 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T22:59:57.371 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T22:59:57.371 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: Updating smithi139:/etc/ceph/ceph.conf 2024-03-18T22:59:57.371 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: Deploying cephadm binary to smithi175 2024-03-18T22:59:57.371 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: Updating smithi139:/var/lib/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-18T22:59:57.371 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: Updating smithi139:/etc/ceph/ceph.client.admin.keyring 2024-03-18T22:59:57.371 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T22:59:57.371 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: Updating smithi139:/var/lib/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/config/ceph.client.admin.keyring 2024-03-18T22:59:57.371 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T22:59:57.371 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T22:59:57.371 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T22:59:57.371 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T22:59:57.371 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T22:59:57.371 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T22:59:57.371 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: Added host smithi175 2024-03-18T22:59:57.371 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T22:59:57.371 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T22:59:57.372 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T22:59:57.372 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T22:59:57.372 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-18T22:59:57.372 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T22:59:57.372 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: from='client.? 172.21.15.49:0/3993714762' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-03-18T22:59:57.372 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T22:59:57.372 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T22:59:57.372 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T22:59:57.372 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T22:59:57.372 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi175", "name": "osd_memory_target"}]: dispatch 2024-03-18T22:59:57.372 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T22:59:57.372 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T22:59:57.372 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T22:59:57.372 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: Updating smithi175:/etc/ceph/ceph.conf 2024-03-18T22:59:57.372 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: Updating smithi175:/var/lib/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-18T22:59:57.373 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: Updating smithi175:/etc/ceph/ceph.client.admin.keyring 2024-03-18T22:59:57.373 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: from='client.? 172.21.15.49:0/3993714762' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-03-18T22:59:57.373 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: osdmap e4: 0 total, 0 up, 0 in 2024-03-18T22:59:57.373 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T22:59:57.373 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T22:59:57.373 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T22:59:57.373 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: Updating smithi175:/var/lib/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/config/ceph.client.admin.keyring 2024-03-18T22:59:57.373 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T22:59:57.373 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T22:59:57.373 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T22:59:57.373 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T22:59:57.373 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T22:59:57.373 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T22:59:57.373 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-18T22:59:57.373 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T22:59:57.374 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi049:172.21.15.49=a;smithi139:172.21.15.139=b;smithi175:172.21.15.175=c", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T22:59:57.374 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: Saving service mon spec with placement smithi049:172.21.15.49=a;smithi139:172.21.15.139=b;smithi175:172.21.15.175=c;count:3 2024-03-18T22:59:57.374 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: Deploying daemon mon.c on smithi175 2024-03-18T22:59:57.374 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T22:59:57.374 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: from='client.? 172.21.15.175:0/1226364532' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-18T22:59:57.374 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T22:59:57.374 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T22:59:57.374 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T22:59:57.374 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-18T22:59:57.374 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-18T22:59:57.374 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: mon.a calling monitor election 2024-03-18T22:59:57.374 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-18T22:59:57.374 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T22:59:57.375 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-18T22:59:57.375 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: mon.c calling monitor election 2024-03-18T22:59:57.375 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-18T22:59:57.375 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T22:59:57.375 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-18T22:59:57.375 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-18T22:59:57.375 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-03-18T22:59:57.375 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: monmap epoch 2 2024-03-18T22:59:57.375 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: fsid c63591ea-e57a-11ee-95c9-87774f69a715 2024-03-18T22:59:57.375 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: last_changed 2024-03-18T22:59:48.347509+0000 2024-03-18T22:59:57.375 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: created 2024-03-18T22:57:55.689519+0000 2024-03-18T22:59:57.375 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: min_mon_release 19 (squid) 2024-03-18T22:59:57.375 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: election_strategy: 1 2024-03-18T22:59:57.375 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: 0: [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] mon.a 2024-03-18T22:59:57.375 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: 1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.c 2024-03-18T22:59:57.375 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: fsmap 2024-03-18T22:59:57.376 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: osdmap e4: 0 total, 0 up, 0 in 2024-03-18T22:59:57.376 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: mgrmap e13: x(active, since 53s) 2024-03-18T22:59:57.376 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: overall HEALTH_OK 2024-03-18T22:59:57.376 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T22:59:57.376 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T22:59:57.376 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T22:59:57.376 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-18T22:59:57.376 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T22:59:57.376 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: Deploying daemon mon.b on smithi139 2024-03-18T22:59:57.376 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-18T22:59:57.376 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T22:59:57.376 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: from='client.? 172.21.15.175:0/1070005547' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-18T22:59:57.376 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 ceph-mon[67282]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-03-18T22:59:57.376 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 bash[67263]: 127fbf6c9088fad00e275208d3b9999915d73439c03ce4c63e2469f97930981b 2024-03-18T22:59:57.377 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 22:59:57 smithi139 systemd[1]: Started Ceph mon.b for c63591ea-e57a-11ee-95c9-87774f69a715. 2024-03-18T23:00:02.607 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:00:02.606+0000 7fabaf752700 1 -- 172.21.15.175:0/3695410900 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fab940035c0 msgr2=0x7fab94005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:00:02.607 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:00:02.606+0000 7fabaf752700 1 --2- 172.21.15.175:0/3695410900 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fab940035c0 0x7fab94005a50 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7faba806a5d0 tx=0x7faba400fe60 comp rx=0 tx=0).stop 2024-03-18T23:00:02.608 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:00:02.606+0000 7fabaf752700 1 -- 172.21.15.175:0/3695410900 shutdown_connections 2024-03-18T23:00:02.608 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:00:02.606+0000 7fabaf752700 1 --2- 172.21.15.175:0/3695410900 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fab940035c0 0x7fab94005a50 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:00:02.608 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:00:02.606+0000 7fabaf752700 1 -- 172.21.15.175:0/3695410900 >> 172.21.15.175:0/3695410900 conn(0x7faba80fc810 msgr2=0x7faba80fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:00:02.608 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:00:02.606+0000 7fabaf752700 1 -- 172.21.15.175:0/3695410900 shutdown_connections 2024-03-18T23:00:02.608 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:00:02.606+0000 7fabaf752700 1 -- 172.21.15.175:0/3695410900 wait complete. 2024-03-18T23:00:02.608 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:00:02.607+0000 7fabaf752700 1 Processor -- start 2024-03-18T23:00:02.609 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:00:02.607+0000 7fabaf752700 1 -- start start 2024-03-18T23:00:02.609 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:02 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-18T23:00:02.609 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:02 smithi175 ceph-mon[67352]: mon.a calling monitor election 2024-03-18T23:00:02.609 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:02 smithi175 ceph-mon[67352]: mon.c calling monitor election 2024-03-18T23:00:02.609 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:02 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-18T23:00:02.609 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:02 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-18T23:00:02.609 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:02 smithi175 ceph-mon[67352]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T23:00:02.609 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:02 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-18T23:00:02.610 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:02 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-18T23:00:02.610 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:02 smithi175 ceph-mon[67352]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T23:00:02.610 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:02 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-18T23:00:02.610 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:02 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-18T23:00:02.610 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:02 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-18T23:00:02.610 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:02 smithi175 ceph-mon[67352]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-03-18T23:00:02.610 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:02 smithi175 ceph-mon[67352]: monmap epoch 3 2024-03-18T23:00:02.610 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:02 smithi175 ceph-mon[67352]: fsid c63591ea-e57a-11ee-95c9-87774f69a715 2024-03-18T23:00:02.610 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:02 smithi175 ceph-mon[67352]: last_changed 2024-03-18T22:59:57.233881+0000 2024-03-18T23:00:02.610 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:02 smithi175 ceph-mon[67352]: created 2024-03-18T22:57:55.689519+0000 2024-03-18T23:00:02.610 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:02 smithi175 ceph-mon[67352]: min_mon_release 19 (squid) 2024-03-18T23:00:02.610 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:02 smithi175 ceph-mon[67352]: election_strategy: 1 2024-03-18T23:00:02.610 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:02 smithi175 ceph-mon[67352]: 0: [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] mon.a 2024-03-18T23:00:02.610 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:02 smithi175 ceph-mon[67352]: 1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.c 2024-03-18T23:00:02.611 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:02 smithi175 ceph-mon[67352]: 2: [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] mon.b 2024-03-18T23:00:02.611 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:02 smithi175 ceph-mon[67352]: fsmap 2024-03-18T23:00:02.611 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:02 smithi175 ceph-mon[67352]: osdmap e4: 0 total, 0 up, 0 in 2024-03-18T23:00:02.611 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:02 smithi175 ceph-mon[67352]: mgrmap e13: x(active, since 62s) 2024-03-18T23:00:02.611 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:02 smithi175 ceph-mon[67352]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2024-03-18T23:00:02.611 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:02 smithi175 ceph-mon[67352]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2024-03-18T23:00:02.611 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:02 smithi175 ceph-mon[67352]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2024-03-18T23:00:02.611 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:02 smithi175 ceph-mon[67352]: mon.b (rank 2) addr [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] is down (out of quorum) 2024-03-18T23:00:02.611 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:02 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:00:02.611 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:02 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:00:02.613 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:00:02.607+0000 7fabaf752700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fab940035c0 0x7faba819d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:00:02.613 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:00:02.607+0000 7fabaf752700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faba819dce0 0x7faba8197960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:00:02.614 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:00:02.607+0000 7fabaf752700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faba8197ea0 0x7faba8198350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:00:02.614 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:00:02.607+0000 7fabaf752700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7faba8109e50 con 0x7fab940035c0 2024-03-18T23:00:02.614 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:00:02.607+0000 7fabaf752700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7faba8109cd0 con 0x7faba8197ea0 2024-03-18T23:00:02.614 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:00:02.607+0000 7fabaf752700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7faba8109fd0 con 0x7faba819dce0 2024-03-18T23:00:02.614 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:00:02.607+0000 7fabacced700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faba819dce0 0x7faba8197960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:00:02.614 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:00:02.607+0000 7fabacced700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faba819dce0 0x7faba8197960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:40916/0 (socket says 172.21.15.175:40916) 2024-03-18T23:00:02.614 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:00:02.607+0000 7fabacced700 1 -- 172.21.15.175:0/2537218636 learned_addr learned my addr 172.21.15.175:0/2537218636 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-03-18T23:00:02.614 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:00:02.607+0000 7fabad4ee700 1 --2- 172.21.15.175:0/2537218636 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fab940035c0 0x7faba819d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:00:02.614 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:00:02.607+0000 7fabacced700 1 -- 172.21.15.175:0/2537218636 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faba819dce0 msgr2=0x7faba8197960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-03-18T23:00:02.614 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:00:02.607+0000 7fabacced700 1 -- 172.21.15.175:0/2537218636 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faba819dce0 msgr2=0x7faba8197960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-18T23:00:02.614 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:00:02.607+0000 7fabadcef700 1 --2- 172.21.15.175:0/2537218636 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faba8197ea0 0x7faba8198350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:00:02.614 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:00:02.607+0000 7fabacced700 1 --2- 172.21.15.175:0/2537218636 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faba819dce0 0x7faba8197960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-18T23:00:02.614 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:00:02.608+0000 7fabacced700 1 --2- 172.21.15.175:0/2537218636 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faba819dce0 0x7faba8197960 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-18T23:00:02.615 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:00:02.608+0000 7fabadcef700 1 -- 172.21.15.175:0/2537218636 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faba8197ea0 msgr2=0x7faba8198350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2024-03-18T23:00:02.615 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:00:02.608+0000 7fabadcef700 1 -- 172.21.15.175:0/2537218636 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faba8197ea0 msgr2=0x7faba8198350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-18T23:00:02.615 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:00:02.608+0000 7fabadcef700 1 --2- 172.21.15.175:0/2537218636 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faba8197ea0 0x7faba8198350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-18T23:00:02.615 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:00:02.608+0000 7fabadcef700 1 --2- 172.21.15.175:0/2537218636 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faba8197ea0 0x7faba8198350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-18T23:00:02.615 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:00:02.608+0000 7fabad4ee700 1 -- 172.21.15.175:0/2537218636 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faba819dce0 msgr2=0x7faba8197960 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-18T23:00:02.615 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:00:02.608+0000 7fabad4ee700 1 --2- 172.21.15.175:0/2537218636 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faba819dce0 0x7faba8197960 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:00:02.615 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:00:02.608+0000 7fabad4ee700 1 -- 172.21.15.175:0/2537218636 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faba8197ea0 msgr2=0x7faba8198350 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-18T23:00:02.615 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:00:02.608+0000 7fabad4ee700 1 --2- 172.21.15.175:0/2537218636 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faba8197ea0 0x7faba8198350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:00:02.615 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:00:02.608+0000 7fabad4ee700 1 -- 172.21.15.175:0/2537218636 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faba81a4670 con 0x7fab940035c0 2024-03-18T23:00:02.615 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:00:02.608+0000 7fabad4ee700 1 --2- 172.21.15.175:0/2537218636 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fab940035c0 0x7faba819d7a0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7faba401f000 tx=0x7faba4005720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:00:02.615 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:00:02.608+0000 7fab9e7fc700 1 -- 172.21.15.175:0/2537218636 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faba402b020 con 0x7fab940035c0 2024-03-18T23:00:02.615 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:00:02.608+0000 7fab9e7fc700 1 -- 172.21.15.175:0/2537218636 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faba4004110 con 0x7fab940035c0 2024-03-18T23:00:02.615 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:00:02.608+0000 7fab9e7fc700 1 -- 172.21.15.175:0/2537218636 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faba4022040 con 0x7fab940035c0 2024-03-18T23:00:02.615 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:00:02.608+0000 7fabaf752700 1 -- 172.21.15.175:0/2537218636 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faba81a4840 con 0x7fab940035c0 2024-03-18T23:00:02.615 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:00:02.609+0000 7fabaf752700 1 -- 172.21.15.175:0/2537218636 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faba81a4d80 con 0x7fab940035c0 2024-03-18T23:00:02.616 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:00:02.610+0000 7fab9e7fc700 1 -- 172.21.15.175:0/2537218636 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 13) ==== 46162+0+0 (secure 0 0 0) 0x7faba402e020 con 0x7fab940035c0 2024-03-18T23:00:02.616 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:00:02.610+0000 7fabaf752700 1 -- 172.21.15.175:0/2537218636 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faba8063940 con 0x7fab940035c0 2024-03-18T23:00:02.616 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:00:02.610+0000 7fab9e7fc700 1 --2- 172.21.15.175:0/2537218636 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fab8c039450 0x7fab8c03b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:00:02.616 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:00:02.610+0000 7fab9e7fc700 1 -- 172.21.15.175:0/2537218636 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7faba404e910 con 0x7fab940035c0 2024-03-18T23:00:02.616 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:00:02.610+0000 7fabacced700 1 --2- 172.21.15.175:0/2537218636 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fab8c039450 0x7fab8c03b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:00:02.616 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:00:02.610+0000 7fabacced700 1 --2- 172.21.15.175:0/2537218636 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fab8c039450 0x7fab8c03b910 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fab98009180 tx=0x7fab98008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:00:02.616 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:00:02.615+0000 7fab9e7fc700 1 -- 172.21.15.175:0/2537218636 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7faba4035020 con 0x7fab940035c0 2024-03-18T23:00:02.630 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:02 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-18T23:00:02.630 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:02 smithi049 ceph-mon[62991]: mon.a calling monitor election 2024-03-18T23:00:02.630 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:02 smithi049 ceph-mon[62991]: mon.c calling monitor election 2024-03-18T23:00:02.630 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:02 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-18T23:00:02.630 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:02 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-18T23:00:02.630 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:02 smithi049 ceph-mon[62991]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T23:00:02.630 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:02 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-18T23:00:02.631 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:02 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-18T23:00:02.631 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:02 smithi049 ceph-mon[62991]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T23:00:02.631 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:02 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-18T23:00:02.631 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:02 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-18T23:00:02.631 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:02 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-18T23:00:02.631 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:02 smithi049 ceph-mon[62991]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-03-18T23:00:02.631 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:02 smithi049 ceph-mon[62991]: monmap epoch 3 2024-03-18T23:00:02.631 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:02 smithi049 ceph-mon[62991]: fsid c63591ea-e57a-11ee-95c9-87774f69a715 2024-03-18T23:00:02.631 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:02 smithi049 ceph-mon[62991]: last_changed 2024-03-18T22:59:57.233881+0000 2024-03-18T23:00:02.631 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:02 smithi049 ceph-mon[62991]: created 2024-03-18T22:57:55.689519+0000 2024-03-18T23:00:02.631 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:02 smithi049 ceph-mon[62991]: min_mon_release 19 (squid) 2024-03-18T23:00:02.631 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:02 smithi049 ceph-mon[62991]: election_strategy: 1 2024-03-18T23:00:02.631 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:02 smithi049 ceph-mon[62991]: 0: [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] mon.a 2024-03-18T23:00:02.631 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:02 smithi049 ceph-mon[62991]: 1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.c 2024-03-18T23:00:02.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:02 smithi049 ceph-mon[62991]: 2: [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] mon.b 2024-03-18T23:00:02.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:02 smithi049 ceph-mon[62991]: fsmap 2024-03-18T23:00:02.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:02 smithi049 ceph-mon[62991]: osdmap e4: 0 total, 0 up, 0 in 2024-03-18T23:00:02.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:02 smithi049 ceph-mon[62991]: mgrmap e13: x(active, since 62s) 2024-03-18T23:00:02.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:02 smithi049 ceph-mon[62991]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2024-03-18T23:00:02.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:02 smithi049 ceph-mon[62991]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2024-03-18T23:00:02.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:02 smithi049 ceph-mon[62991]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2024-03-18T23:00:02.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:02 smithi049 ceph-mon[62991]: mon.b (rank 2) addr [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] is down (out of quorum) 2024-03-18T23:00:02.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:02 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:00:02.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:02 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:00:02.899 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:00:02.898+0000 7fabaf752700 1 -- 172.21.15.175:0/2537218636 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7faba802dbb0 con 0x7fab940035c0 2024-03-18T23:00:02.899 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:00:02.898+0000 7fab9e7fc700 1 -- 172.21.15.175:0/2537218636 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1283 (secure 0 0 0) 0x7faba4010c90 con 0x7fab940035c0 2024-03-18T23:00:02.899 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-03-18T23:00:02.900 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":3,"fsid":"c63591ea-e57a-11ee-95c9-87774f69a715","modified":"2024-03-18T22:59:57.233881Z","created":"2024-03-18T22:57:55.689519Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:3300","nonce":0},{"type":"v1","addr":"172.21.15.49:6789","nonce":0}]},"addr":"172.21.15.49:6789/0","public_addr":"172.21.15.49:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:3300","nonce":0},{"type":"v1","addr":"172.21.15.175:6789","nonce":0}]},"addr":"172.21.15.175:6789/0","public_addr":"172.21.15.175:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:3300","nonce":0},{"type":"v1","addr":"172.21.15.139:6789","nonce":0}]},"addr":"172.21.15.139:6789/0","public_addr":"172.21.15.139:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-03-18T23:00:02.902 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:00:02.901+0000 7fabaf752700 1 -- 172.21.15.175:0/2537218636 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fab8c039450 msgr2=0x7fab8c03b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:00:02.902 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:00:02.901+0000 7fabaf752700 1 --2- 172.21.15.175:0/2537218636 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fab8c039450 0x7fab8c03b910 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fab98009180 tx=0x7fab98008040 comp rx=0 tx=0).stop 2024-03-18T23:00:02.902 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:00:02.901+0000 7fabaf752700 1 -- 172.21.15.175:0/2537218636 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fab940035c0 msgr2=0x7faba819d7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:00:02.902 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:00:02.901+0000 7fabaf752700 1 --2- 172.21.15.175:0/2537218636 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fab940035c0 0x7faba819d7a0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7faba401f000 tx=0x7faba4005720 comp rx=0 tx=0).stop 2024-03-18T23:00:02.902 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:00:02.901+0000 7fabaf752700 1 -- 172.21.15.175:0/2537218636 shutdown_connections 2024-03-18T23:00:02.902 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:00:02.901+0000 7fabaf752700 1 --2- 172.21.15.175:0/2537218636 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faba8197ea0 0x7faba8198350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:00:02.902 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:00:02.901+0000 7fabaf752700 1 --2- 172.21.15.175:0/2537218636 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fab940035c0 0x7faba819d7a0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:00:02.902 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:00:02.901+0000 7fabaf752700 1 --2- 172.21.15.175:0/2537218636 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fab8c039450 0x7fab8c03b910 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:00:02.902 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:00:02.901+0000 7fabaf752700 1 --2- 172.21.15.175:0/2537218636 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faba819dce0 0x7faba8197960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:00:02.902 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:00:02.901+0000 7fabaf752700 1 -- 172.21.15.175:0/2537218636 >> 172.21.15.175:0/2537218636 conn(0x7faba80fc810 msgr2=0x7faba8103f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:00:02.902 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:00:02.901+0000 7fabaf752700 1 -- 172.21.15.175:0/2537218636 shutdown_connections 2024-03-18T23:00:02.902 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:00:02.901+0000 7fabaf752700 1 -- 172.21.15.175:0/2537218636 wait complete. 2024-03-18T23:00:02.903 INFO:teuthology.orchestra.run.smithi175.stderr:dumped monmap epoch 3 2024-03-18T23:00:03.462 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:03 smithi175 ceph-mon[67352]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T23:00:03.462 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:03 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:00:03.462 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:03 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:00:03.462 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:03 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:00:03.462 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:03 smithi175 ceph-mon[67352]: from='client.? 172.21.15.175:0/2537218636' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-18T23:00:03.462 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:03 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-18T23:00:03.494 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-03-18T23:00:03.494 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c63591ea-e57a-11ee-95c9-87774f69a715 -- ceph config generate-minimal-conf 2024-03-18T23:00:03.630 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:03 smithi049 ceph-mon[62991]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T23:00:03.630 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:03 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:00:03.630 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:03 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:00:03.630 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:03 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:00:03.630 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:03 smithi049 ceph-mon[62991]: from='client.? 172.21.15.175:0/2537218636' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-18T23:00:03.631 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:03 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-18T23:00:03.759 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T23:00:04.566 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:04 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-18T23:00:04.566 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:04 smithi139 ceph-mon[67282]: mon.a calling monitor election 2024-03-18T23:00:04.566 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:04 smithi139 ceph-mon[67282]: mon.c calling monitor election 2024-03-18T23:00:04.567 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:04 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-18T23:00:04.567 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:04 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-18T23:00:04.567 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:04 smithi139 ceph-mon[67282]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T23:00:04.567 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:04 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-18T23:00:04.567 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:04 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-18T23:00:04.568 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:04 smithi139 ceph-mon[67282]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T23:00:04.568 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:04 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-18T23:00:04.568 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:04 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-18T23:00:04.568 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:04 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-18T23:00:04.568 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:04 smithi139 ceph-mon[67282]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-03-18T23:00:04.568 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:04 smithi139 ceph-mon[67282]: monmap epoch 3 2024-03-18T23:00:04.569 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:04 smithi139 ceph-mon[67282]: fsid c63591ea-e57a-11ee-95c9-87774f69a715 2024-03-18T23:00:04.569 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:04 smithi139 ceph-mon[67282]: last_changed 2024-03-18T22:59:57.233881+0000 2024-03-18T23:00:04.569 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:04 smithi139 ceph-mon[67282]: created 2024-03-18T22:57:55.689519+0000 2024-03-18T23:00:04.569 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:04 smithi139 ceph-mon[67282]: min_mon_release 19 (squid) 2024-03-18T23:00:04.569 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:04 smithi139 ceph-mon[67282]: election_strategy: 1 2024-03-18T23:00:04.569 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:04 smithi139 ceph-mon[67282]: 0: [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] mon.a 2024-03-18T23:00:04.570 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:04 smithi139 ceph-mon[67282]: 1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.c 2024-03-18T23:00:04.570 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:04 smithi139 ceph-mon[67282]: 2: [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] mon.b 2024-03-18T23:00:04.570 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:04 smithi139 ceph-mon[67282]: fsmap 2024-03-18T23:00:04.570 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:04 smithi139 ceph-mon[67282]: osdmap e4: 0 total, 0 up, 0 in 2024-03-18T23:00:04.570 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:04 smithi139 ceph-mon[67282]: mgrmap e13: x(active, since 62s) 2024-03-18T23:00:04.570 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:04 smithi139 ceph-mon[67282]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2024-03-18T23:00:04.570 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:04 smithi139 ceph-mon[67282]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2024-03-18T23:00:04.571 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:04 smithi139 ceph-mon[67282]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2024-03-18T23:00:04.571 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:04 smithi139 ceph-mon[67282]: mon.b (rank 2) addr [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] is down (out of quorum) 2024-03-18T23:00:04.571 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:04 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:00:04.571 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:04 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:00:04.571 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:04 smithi139 ceph-mon[67282]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T23:00:04.571 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:04 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:00:04.571 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:04 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:00:04.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:04 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:00:04.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:04 smithi139 ceph-mon[67282]: from='client.? 172.21.15.175:0/2537218636' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-18T23:00:04.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:04 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-18T23:00:05.340 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:00:05.336+0000 7f31353cc700 1 -- 172.21.15.49:0/3820124054 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3130101f90 msgr2=0x7f3130102370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:00:05.340 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:00:05.336+0000 7f31353cc700 1 --2- 172.21.15.49:0/3820124054 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3130101f90 0x7f3130102370 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f3118009910 tx=0x7f3118009c20 comp rx=0 tx=0).stop 2024-03-18T23:00:05.340 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:00:05.336+0000 7f31353cc700 1 -- 172.21.15.49:0/3820124054 shutdown_connections 2024-03-18T23:00:05.340 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:00:05.336+0000 7f31353cc700 1 --2- 172.21.15.49:0/3820124054 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3130101f90 0x7f3130102370 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:00:05.340 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:00:05.336+0000 7f31353cc700 1 -- 172.21.15.49:0/3820124054 >> 172.21.15.49:0/3820124054 conn(0x7f31300fd840 msgr2=0x7f31300ffc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:00:05.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:00:05.336+0000 7f31353cc700 1 -- 172.21.15.49:0/3820124054 shutdown_connections 2024-03-18T23:00:05.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:00:05.336+0000 7f31353cc700 1 -- 172.21.15.49:0/3820124054 wait complete. 2024-03-18T23:00:05.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:00:05.337+0000 7f31353cc700 1 Processor -- start 2024-03-18T23:00:05.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:00:05.337+0000 7f31353cc700 1 -- start start 2024-03-18T23:00:05.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:00:05.337+0000 7f31353cc700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3130101f90 0x7f3130074610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:00:05.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:00:05.337+0000 7f31353cc700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3130074b50 0x7f3130074fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:00:05.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:00:05.337+0000 7f31353cc700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f31301a3c80 0x7f31301a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:00:05.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:00:05.337+0000 7f31353cc700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f313010c320 con 0x7f3130101f90 2024-03-18T23:00:05.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:00:05.337+0000 7f31353cc700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f313010c1a0 con 0x7f31301a3c80 2024-03-18T23:00:05.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:00:05.337+0000 7f31353cc700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f313010c4a0 con 0x7f3130074b50 2024-03-18T23:00:05.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:00:05.337+0000 7f312effd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3130101f90 0x7f3130074610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:00:05.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:00:05.337+0000 7f312effd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3130101f90 0x7f3130074610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:44058/0 (socket says 172.21.15.49:44058) 2024-03-18T23:00:05.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:00:05.337+0000 7f312effd700 1 -- 172.21.15.49:0/1294554548 learned_addr learned my addr 172.21.15.49:0/1294554548 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:00:05.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:00:05.337+0000 7f312e7fc700 1 --2- 172.21.15.49:0/1294554548 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3130074b50 0x7f3130074fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:00:05.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:00:05.337+0000 7f312effd700 1 -- 172.21.15.49:0/1294554548 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3130074b50 msgr2=0x7f3130074fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:00:05.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:00:05.337+0000 7f312effd700 1 --2- 172.21.15.49:0/1294554548 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3130074b50 0x7f3130074fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:00:05.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:00:05.337+0000 7f312effd700 1 -- 172.21.15.49:0/1294554548 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f31301a3c80 msgr2=0x7f31301a6070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:00:05.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:00:05.337+0000 7f312effd700 1 --2- 172.21.15.49:0/1294554548 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f31301a3c80 0x7f31301a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:00:05.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:00:05.337+0000 7f312effd700 1 -- 172.21.15.49:0/1294554548 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31301a67a0 con 0x7f3130101f90 2024-03-18T23:00:05.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:00:05.337+0000 7f312e7fc700 1 --2- 172.21.15.49:0/1294554548 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3130074b50 0x7f3130074fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T23:00:05.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:00:05.338+0000 7f312effd700 1 --2- 172.21.15.49:0/1294554548 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3130101f90 0x7f3130074610 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f311800bdc0 tx=0x7f311800bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:00:05.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:00:05.338+0000 7f3127fff700 1 -- 172.21.15.49:0/1294554548 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3118026070 con 0x7f3130101f90 2024-03-18T23:00:05.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:00:05.338+0000 7f3127fff700 1 -- 172.21.15.49:0/1294554548 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f31180050d0 con 0x7f3130101f90 2024-03-18T23:00:05.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:00:05.338+0000 7f31353cc700 1 -- 172.21.15.49:0/1294554548 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31301a6a90 con 0x7f3130101f90 2024-03-18T23:00:05.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:00:05.338+0000 7f3127fff700 1 -- 172.21.15.49:0/1294554548 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f311800fdf0 con 0x7f3130101f90 2024-03-18T23:00:05.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:00:05.338+0000 7f31353cc700 1 -- 172.21.15.49:0/1294554548 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31301a7020 con 0x7f3130101f90 2024-03-18T23:00:05.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:00:05.339+0000 7f3127fff700 1 -- 172.21.15.49:0/1294554548 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 13) ==== 46162+0+0 (secure 0 0 0) 0x7f3118029070 con 0x7f3130101f90 2024-03-18T23:00:05.343 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:00:05.339+0000 7f31353cc700 1 -- 172.21.15.49:0/1294554548 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3130063940 con 0x7f3130101f90 2024-03-18T23:00:05.343 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:00:05.339+0000 7f3127fff700 1 --2- 172.21.15.49:0/1294554548 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f311c039450 0x7f311c03b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:00:05.343 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:00:05.339+0000 7f3127fff700 1 -- 172.21.15.49:0/1294554548 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f311804cb40 con 0x7f3130101f90 2024-03-18T23:00:05.343 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:00:05.339+0000 7f312e7fc700 1 --2- 172.21.15.49:0/1294554548 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f311c039450 0x7f311c03b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:00:05.343 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:00:05.339+0000 7f312e7fc700 1 --2- 172.21.15.49:0/1294554548 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f311c039450 0x7f311c03b910 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f31200079b0 tx=0x7f3120008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:00:05.346 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:00:05.344+0000 7f3127fff700 1 -- 172.21.15.49:0/1294554548 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f3118013070 con 0x7f3130101f90 2024-03-18T23:00:05.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:05 smithi049 ceph-mon[62991]: mon.b calling monitor election 2024-03-18T23:00:05.381 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:05 smithi049 ceph-mon[62991]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T23:00:05.381 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:05 smithi049 ceph-mon[62991]: Updating smithi049:/etc/ceph/ceph.conf 2024-03-18T23:00:05.381 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:05 smithi049 ceph-mon[62991]: Updating smithi139:/etc/ceph/ceph.conf 2024-03-18T23:00:05.381 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:05 smithi049 ceph-mon[62991]: Updating smithi175:/etc/ceph/ceph.conf 2024-03-18T23:00:05.381 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:05 smithi049 ceph-mon[62991]: mon.b calling monitor election 2024-03-18T23:00:05.381 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:05 smithi049 ceph-mon[62991]: mon.c calling monitor election 2024-03-18T23:00:05.381 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:05 smithi049 ceph-mon[62991]: mon.a calling monitor election 2024-03-18T23:00:05.381 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:05 smithi049 ceph-mon[62991]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-03-18T23:00:05.381 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:05 smithi049 ceph-mon[62991]: monmap epoch 3 2024-03-18T23:00:05.381 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:05 smithi049 ceph-mon[62991]: fsid c63591ea-e57a-11ee-95c9-87774f69a715 2024-03-18T23:00:05.381 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:05 smithi049 ceph-mon[62991]: last_changed 2024-03-18T22:59:57.233881+0000 2024-03-18T23:00:05.381 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:05 smithi049 ceph-mon[62991]: created 2024-03-18T22:57:55.689519+0000 2024-03-18T23:00:05.381 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:05 smithi049 ceph-mon[62991]: min_mon_release 19 (squid) 2024-03-18T23:00:05.381 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:05 smithi049 ceph-mon[62991]: election_strategy: 1 2024-03-18T23:00:05.382 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:05 smithi049 ceph-mon[62991]: 0: [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] mon.a 2024-03-18T23:00:05.382 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:05 smithi049 ceph-mon[62991]: 1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.c 2024-03-18T23:00:05.382 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:05 smithi049 ceph-mon[62991]: 2: [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] mon.b 2024-03-18T23:00:05.382 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:05 smithi049 ceph-mon[62991]: fsmap 2024-03-18T23:00:05.382 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:05 smithi049 ceph-mon[62991]: osdmap e4: 0 total, 0 up, 0 in 2024-03-18T23:00:05.382 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:05 smithi049 ceph-mon[62991]: mgrmap e13: x(active, since 64s) 2024-03-18T23:00:05.382 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:05 smithi049 ceph-mon[62991]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2024-03-18T23:00:05.382 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:05 smithi049 ceph-mon[62991]: Cluster is now healthy 2024-03-18T23:00:05.382 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:05 smithi049 ceph-mon[62991]: overall HEALTH_OK 2024-03-18T23:00:05.382 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:05 smithi049 ceph-mon[62991]: Updating smithi175:/var/lib/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-18T23:00:05.382 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:05 smithi049 ceph-mon[62991]: Updating smithi139:/var/lib/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-18T23:00:05.382 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:05 smithi049 ceph-mon[62991]: Updating smithi049:/var/lib/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-18T23:00:05.382 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:05 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:00:05.382 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:05 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:00:05.383 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:05 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:00:05.383 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:05 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:00:05.383 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:05 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:00:05.383 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:05 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:00:05.383 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:05 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:00:05.383 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:05 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:00:05.383 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:05 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:00:05.383 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:05 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:00:05.383 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:05 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:00:05.383 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:05 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-18T23:00:05.383 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:05 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-18T23:00:05.383 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:05 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:00:05.383 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:05 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:00:05.462 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:05 smithi175 ceph-mon[67352]: mon.b calling monitor election 2024-03-18T23:00:05.462 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:05 smithi175 ceph-mon[67352]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T23:00:05.462 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:05 smithi175 ceph-mon[67352]: Updating smithi049:/etc/ceph/ceph.conf 2024-03-18T23:00:05.462 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:05 smithi175 ceph-mon[67352]: Updating smithi139:/etc/ceph/ceph.conf 2024-03-18T23:00:05.462 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:05 smithi175 ceph-mon[67352]: Updating smithi175:/etc/ceph/ceph.conf 2024-03-18T23:00:05.462 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:05 smithi175 ceph-mon[67352]: mon.b calling monitor election 2024-03-18T23:00:05.463 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:05 smithi175 ceph-mon[67352]: mon.c calling monitor election 2024-03-18T23:00:05.463 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:05 smithi175 ceph-mon[67352]: mon.a calling monitor election 2024-03-18T23:00:05.463 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:05 smithi175 ceph-mon[67352]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-03-18T23:00:05.463 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:05 smithi175 ceph-mon[67352]: monmap epoch 3 2024-03-18T23:00:05.463 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:05 smithi175 ceph-mon[67352]: fsid c63591ea-e57a-11ee-95c9-87774f69a715 2024-03-18T23:00:05.463 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:05 smithi175 ceph-mon[67352]: last_changed 2024-03-18T22:59:57.233881+0000 2024-03-18T23:00:05.463 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:05 smithi175 ceph-mon[67352]: created 2024-03-18T22:57:55.689519+0000 2024-03-18T23:00:05.463 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:05 smithi175 ceph-mon[67352]: min_mon_release 19 (squid) 2024-03-18T23:00:05.463 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:05 smithi175 ceph-mon[67352]: election_strategy: 1 2024-03-18T23:00:05.463 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:05 smithi175 ceph-mon[67352]: 0: [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] mon.a 2024-03-18T23:00:05.463 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:05 smithi175 ceph-mon[67352]: 1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.c 2024-03-18T23:00:05.463 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:05 smithi175 ceph-mon[67352]: 2: [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] mon.b 2024-03-18T23:00:05.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:05 smithi175 ceph-mon[67352]: fsmap 2024-03-18T23:00:05.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:05 smithi175 ceph-mon[67352]: osdmap e4: 0 total, 0 up, 0 in 2024-03-18T23:00:05.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:05 smithi175 ceph-mon[67352]: mgrmap e13: x(active, since 64s) 2024-03-18T23:00:05.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:05 smithi175 ceph-mon[67352]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2024-03-18T23:00:05.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:05 smithi175 ceph-mon[67352]: Cluster is now healthy 2024-03-18T23:00:05.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:05 smithi175 ceph-mon[67352]: overall HEALTH_OK 2024-03-18T23:00:05.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:05 smithi175 ceph-mon[67352]: Updating smithi175:/var/lib/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-18T23:00:05.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:05 smithi175 ceph-mon[67352]: Updating smithi139:/var/lib/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-18T23:00:05.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:05 smithi175 ceph-mon[67352]: Updating smithi049:/var/lib/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-18T23:00:05.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:05 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:00:05.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:05 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:00:05.465 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:05 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:00:05.465 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:05 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:00:05.465 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:05 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:00:05.465 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:05 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:00:05.465 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:05 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:00:05.465 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:05 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:00:05.465 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:05 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:00:05.465 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:05 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:00:05.465 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:05 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:00:05.465 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:05 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-18T23:00:05.465 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:05 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-18T23:00:05.465 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:05 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:00:05.465 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:05 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:00:05.555 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:00:05.554+0000 7f31353cc700 1 -- 172.21.15.49:0/1294554548 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f3130075b40 con 0x7f3130101f90 2024-03-18T23:00:05.556 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:00:05.554+0000 7f3127fff700 1 -- 172.21.15.49:0/1294554548 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+275 (secure 0 0 0) 0x7f311800e7e0 con 0x7f3130101f90 2024-03-18T23:00:05.556 INFO:teuthology.orchestra.run.smithi049.stdout:# minimal ceph.conf for c63591ea-e57a-11ee-95c9-87774f69a715 2024-03-18T23:00:05.556 INFO:teuthology.orchestra.run.smithi049.stdout:[global] 2024-03-18T23:00:05.556 INFO:teuthology.orchestra.run.smithi049.stdout: fsid = c63591ea-e57a-11ee-95c9-87774f69a715 2024-03-18T23:00:05.556 INFO:teuthology.orchestra.run.smithi049.stdout: mon_host = [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] 2024-03-18T23:00:05.558 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:00:05.557+0000 7f31353cc700 1 -- 172.21.15.49:0/1294554548 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f311c039450 msgr2=0x7f311c03b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:00:05.558 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:00:05.557+0000 7f31353cc700 1 --2- 172.21.15.49:0/1294554548 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f311c039450 0x7f311c03b910 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f31200079b0 tx=0x7f3120008040 comp rx=0 tx=0).stop 2024-03-18T23:00:05.558 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:00:05.557+0000 7f31353cc700 1 -- 172.21.15.49:0/1294554548 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3130101f90 msgr2=0x7f3130074610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:00:05.558 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:00:05.557+0000 7f31353cc700 1 --2- 172.21.15.49:0/1294554548 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3130101f90 0x7f3130074610 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f311800bdc0 tx=0x7f311800bea0 comp rx=0 tx=0).stop 2024-03-18T23:00:05.559 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:00:05.557+0000 7f31353cc700 1 -- 172.21.15.49:0/1294554548 shutdown_connections 2024-03-18T23:00:05.559 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:00:05.557+0000 7f31353cc700 1 --2- 172.21.15.49:0/1294554548 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f31301a3c80 0x7f31301a6070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:00:05.559 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:00:05.557+0000 7f31353cc700 1 --2- 172.21.15.49:0/1294554548 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3130101f90 0x7f3130074610 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:00:05.559 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:00:05.557+0000 7f31353cc700 1 --2- 172.21.15.49:0/1294554548 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f311c039450 0x7f311c03b910 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:00:05.559 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:00:05.557+0000 7f31353cc700 1 --2- 172.21.15.49:0/1294554548 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3130074b50 0x7f3130074fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:00:05.559 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:00:05.557+0000 7f31353cc700 1 -- 172.21.15.49:0/1294554548 >> 172.21.15.49:0/1294554548 conn(0x7f31300fd840 msgr2=0x7f3130103960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:00:05.559 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:00:05.557+0000 7f31353cc700 1 -- 172.21.15.49:0/1294554548 shutdown_connections 2024-03-18T23:00:05.559 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:00:05.557+0000 7f31353cc700 1 -- 172.21.15.49:0/1294554548 wait complete. 2024-03-18T23:00:05.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:05 smithi139 ceph-mon[67282]: mon.b calling monitor election 2024-03-18T23:00:05.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:05 smithi139 ceph-mon[67282]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T23:00:05.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:05 smithi139 ceph-mon[67282]: Updating smithi049:/etc/ceph/ceph.conf 2024-03-18T23:00:05.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:05 smithi139 ceph-mon[67282]: Updating smithi139:/etc/ceph/ceph.conf 2024-03-18T23:00:05.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:05 smithi139 ceph-mon[67282]: Updating smithi175:/etc/ceph/ceph.conf 2024-03-18T23:00:05.566 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:05 smithi139 ceph-mon[67282]: mon.b calling monitor election 2024-03-18T23:00:05.566 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:05 smithi139 ceph-mon[67282]: mon.c calling monitor election 2024-03-18T23:00:05.566 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:05 smithi139 ceph-mon[67282]: mon.a calling monitor election 2024-03-18T23:00:05.566 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:05 smithi139 ceph-mon[67282]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-03-18T23:00:05.566 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:05 smithi139 ceph-mon[67282]: monmap epoch 3 2024-03-18T23:00:05.566 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:05 smithi139 ceph-mon[67282]: fsid c63591ea-e57a-11ee-95c9-87774f69a715 2024-03-18T23:00:05.566 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:05 smithi139 ceph-mon[67282]: last_changed 2024-03-18T22:59:57.233881+0000 2024-03-18T23:00:05.566 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:05 smithi139 ceph-mon[67282]: created 2024-03-18T22:57:55.689519+0000 2024-03-18T23:00:05.566 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:05 smithi139 ceph-mon[67282]: min_mon_release 19 (squid) 2024-03-18T23:00:05.566 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:05 smithi139 ceph-mon[67282]: election_strategy: 1 2024-03-18T23:00:05.566 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:05 smithi139 ceph-mon[67282]: 0: [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] mon.a 2024-03-18T23:00:05.566 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:05 smithi139 ceph-mon[67282]: 1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.c 2024-03-18T23:00:05.566 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:05 smithi139 ceph-mon[67282]: 2: [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] mon.b 2024-03-18T23:00:05.567 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:05 smithi139 ceph-mon[67282]: fsmap 2024-03-18T23:00:05.567 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:05 smithi139 ceph-mon[67282]: osdmap e4: 0 total, 0 up, 0 in 2024-03-18T23:00:05.567 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:05 smithi139 ceph-mon[67282]: mgrmap e13: x(active, since 64s) 2024-03-18T23:00:05.567 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:05 smithi139 ceph-mon[67282]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2024-03-18T23:00:05.567 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:05 smithi139 ceph-mon[67282]: Cluster is now healthy 2024-03-18T23:00:05.567 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:05 smithi139 ceph-mon[67282]: overall HEALTH_OK 2024-03-18T23:00:05.567 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:05 smithi139 ceph-mon[67282]: Updating smithi175:/var/lib/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-18T23:00:05.567 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:05 smithi139 ceph-mon[67282]: Updating smithi139:/var/lib/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-18T23:00:05.567 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:05 smithi139 ceph-mon[67282]: Updating smithi049:/var/lib/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-18T23:00:05.567 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:05 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:00:05.567 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:05 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:00:05.567 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:05 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:00:05.567 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:05 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:00:05.567 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:05 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:00:05.568 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:05 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:00:05.568 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:05 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:00:05.568 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:05 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:00:05.568 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:05 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:00:05.568 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:05 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:00:05.568 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:05 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:00:05.568 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:05 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-18T23:00:05.568 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:05 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-18T23:00:05.568 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:05 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:00:05.568 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:05 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:00:06.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:06 smithi175 ceph-mon[67352]: Reconfiguring mon.a (unknown last config time)... 2024-03-18T23:00:06.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:06 smithi175 ceph-mon[67352]: Reconfiguring daemon mon.a on smithi049 2024-03-18T23:00:06.462 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:06 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-18T23:00:06.462 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:06 smithi175 ceph-mon[67352]: from='client.? 172.21.15.49:0/1294554548' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:00:06.488 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:06 smithi049 ceph-mon[62991]: Reconfiguring mon.a (unknown last config time)... 2024-03-18T23:00:06.488 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:06 smithi049 ceph-mon[62991]: Reconfiguring daemon mon.a on smithi049 2024-03-18T23:00:06.488 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:06 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-18T23:00:06.488 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:06 smithi049 ceph-mon[62991]: from='client.? 172.21.15.49:0/1294554548' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:00:06.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:06 smithi139 ceph-mon[67282]: Reconfiguring mon.a (unknown last config time)... 2024-03-18T23:00:06.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:06 smithi139 ceph-mon[67282]: Reconfiguring daemon mon.a on smithi049 2024-03-18T23:00:06.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:06 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-18T23:00:06.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:06 smithi139 ceph-mon[67282]: from='client.? 172.21.15.49:0/1294554548' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:00:07.043 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-03-18T23:00:07.043 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-03-18T23:00:07.043 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd of=/etc/ceph/ceph.conf 2024-03-18T23:00:07.076 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-03-18T23:00:07.077 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-18T23:00:07.148 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-03-18T23:00:07.148 DEBUG:teuthology.orchestra.run.smithi139:> sudo dd of=/etc/ceph/ceph.conf 2024-03-18T23:00:07.177 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-03-18T23:00:07.177 DEBUG:teuthology.orchestra.run.smithi139:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-18T23:00:07.244 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-03-18T23:00:07.245 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd of=/etc/ceph/ceph.conf 2024-03-18T23:00:07.279 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-03-18T23:00:07.279 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-18T23:00:07.351 INFO:tasks.cephadm:Adding mgr.x on smithi049 2024-03-18T23:00:07.351 INFO:tasks.cephadm:Adding mgr.y on smithi139 2024-03-18T23:00:07.351 INFO:tasks.cephadm:Adding mgr.z on smithi175 2024-03-18T23:00:07.351 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c63591ea-e57a-11ee-95c9-87774f69a715 -- ceph orch apply mgr '3;smithi049=x;smithi139=y;smithi175=z' 2024-03-18T23:00:07.626 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/mon.c/config 2024-03-18T23:00:07.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:07 smithi139 ceph-mon[67282]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T23:00:07.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:07 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:00:07.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:07 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:00:07.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:07 smithi139 ceph-mon[67282]: Reconfiguring mon.b (monmap changed)... 2024-03-18T23:00:07.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:07 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-18T23:00:07.816 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:07 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-18T23:00:07.816 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:07 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:00:07.816 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:07 smithi139 ceph-mon[67282]: Reconfiguring daemon mon.b on smithi139 2024-03-18T23:00:07.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:07 smithi049 ceph-mon[62991]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T23:00:07.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:07 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:00:07.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:07 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:00:07.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:07 smithi049 ceph-mon[62991]: Reconfiguring mon.b (monmap changed)... 2024-03-18T23:00:07.881 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:07 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-18T23:00:07.881 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:07 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-18T23:00:07.881 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:07 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:00:07.881 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:07 smithi049 ceph-mon[62991]: Reconfiguring daemon mon.b on smithi139 2024-03-18T23:00:07.903 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:07 smithi175 ceph-mon[67352]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T23:00:07.904 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:07 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:00:07.904 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:07 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:00:07.904 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:07 smithi175 ceph-mon[67352]: Reconfiguring mon.b (monmap changed)... 2024-03-18T23:00:07.904 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:07 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-18T23:00:07.904 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:07 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-18T23:00:07.904 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:07 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:00:07.904 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:07 smithi175 ceph-mon[67352]: Reconfiguring daemon mon.b on smithi139 2024-03-18T23:00:09.346 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:09 smithi175 ceph-mon[67352]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T23:00:09.346 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:09 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:00:09.346 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:09 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:00:09.346 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:09 smithi175 ceph-mon[67352]: Reconfiguring mon.c (monmap changed)... 2024-03-18T23:00:09.347 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:09 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-18T23:00:09.347 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:09 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-18T23:00:09.347 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:09 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:00:09.347 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:09 smithi175 ceph-mon[67352]: Reconfiguring daemon mon.c on smithi175 2024-03-18T23:00:09.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:09 smithi049 ceph-mon[62991]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T23:00:09.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:09 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:00:09.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:09 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:00:09.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:09 smithi049 ceph-mon[62991]: Reconfiguring mon.c (monmap changed)... 2024-03-18T23:00:09.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:09 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-18T23:00:09.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:09 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-18T23:00:09.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:09 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:00:09.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:09 smithi049 ceph-mon[62991]: Reconfiguring daemon mon.c on smithi175 2024-03-18T23:00:09.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:09 smithi139 ceph-mon[67282]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T23:00:09.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:09 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:00:09.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:09 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:00:09.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:09 smithi139 ceph-mon[67282]: Reconfiguring mon.c (monmap changed)... 2024-03-18T23:00:09.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:09 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-18T23:00:09.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:09 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-18T23:00:09.566 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:09 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:00:09.566 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:09 smithi139 ceph-mon[67282]: Reconfiguring daemon mon.c on smithi175 2024-03-18T23:00:09.673 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:00:09.670+0000 7f1f506e9700 1 -- 172.21.15.175:0/2981894983 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1f48100120 msgr2=0x7f1f48100540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:00:09.673 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:00:09.670+0000 7f1f506e9700 1 --2- 172.21.15.175:0/2981894983 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1f48100120 0x7f1f48100540 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f1f44009910 tx=0x7f1f44009c20 comp rx=0 tx=0).stop 2024-03-18T23:00:09.673 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:00:09.670+0000 7f1f506e9700 1 -- 172.21.15.175:0/2981894983 shutdown_connections 2024-03-18T23:00:09.673 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:00:09.670+0000 7f1f506e9700 1 --2- 172.21.15.175:0/2981894983 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1f48100120 0x7f1f48100540 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:00:09.673 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:00:09.670+0000 7f1f506e9700 1 -- 172.21.15.175:0/2981894983 >> 172.21.15.175:0/2981894983 conn(0x7f1f480fba60 msgr2=0x7f1f480fdea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:00:09.673 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:00:09.670+0000 7f1f506e9700 1 -- 172.21.15.175:0/2981894983 shutdown_connections 2024-03-18T23:00:09.673 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:00:09.670+0000 7f1f506e9700 1 -- 172.21.15.175:0/2981894983 wait complete. 2024-03-18T23:00:09.673 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:00:09.671+0000 7f1f506e9700 1 Processor -- start 2024-03-18T23:00:09.673 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:00:09.671+0000 7f1f506e9700 1 -- start start 2024-03-18T23:00:09.673 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:00:09.671+0000 7f1f506e9700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1f48100120 0x7f1f48197640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:00:09.674 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:00:09.671+0000 7f1f506e9700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1f48197b80 0x7f1f4819cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:00:09.674 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:00:09.671+0000 7f1f506e9700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1f48198000 0x7f1f4819f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:00:09.674 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:00:09.671+0000 7f1f506e9700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f1f48107e20 con 0x7f1f48197b80 2024-03-18T23:00:09.674 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:00:09.671+0000 7f1f506e9700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f1f48107ca0 con 0x7f1f48198000 2024-03-18T23:00:09.674 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:00:09.671+0000 7f1f506e9700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f1f48107fa0 con 0x7f1f48100120 2024-03-18T23:00:09.674 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:00:09.671+0000 7f1f4e485700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1f48100120 0x7f1f48197640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:00:09.674 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:00:09.671+0000 7f1f4e485700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1f48100120 0x7f1f48197640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:34102/0 (socket says 172.21.15.175:34102) 2024-03-18T23:00:09.674 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:00:09.671+0000 7f1f4e485700 1 -- 172.21.15.175:0/2297398593 learned_addr learned my addr 172.21.15.175:0/2297398593 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-03-18T23:00:09.674 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:00:09.671+0000 7f1f4ec86700 1 --2- 172.21.15.175:0/2297398593 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1f48198000 0x7f1f4819f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:00:09.674 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:00:09.671+0000 7f1f4e485700 1 -- 172.21.15.175:0/2297398593 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1f48198000 msgr2=0x7f1f4819f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:00:09.674 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:00:09.671+0000 7f1f4e485700 1 --2- 172.21.15.175:0/2297398593 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1f48198000 0x7f1f4819f180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:00:09.674 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:00:09.671+0000 7f1f4e485700 1 -- 172.21.15.175:0/2297398593 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1f48197b80 msgr2=0x7f1f4819cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:00:09.674 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:00:09.671+0000 7f1f4e485700 1 --2- 172.21.15.175:0/2297398593 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1f48197b80 0x7f1f4819cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:00:09.674 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:00:09.671+0000 7f1f4e485700 1 -- 172.21.15.175:0/2297398593 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f4819fa50 con 0x7f1f48100120 2024-03-18T23:00:09.675 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:00:09.671+0000 7f1f4e485700 1 --2- 172.21.15.175:0/2297398593 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1f48100120 0x7f1f48197640 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f1f44000c00 tx=0x7f1f440050b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:00:09.675 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:00:09.672+0000 7f1f3f7fe700 1 -- 172.21.15.175:0/2297398593 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f44027070 con 0x7f1f48100120 2024-03-18T23:00:09.675 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:00:09.672+0000 7f1f3f7fe700 1 -- 172.21.15.175:0/2297398593 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1f44022470 con 0x7f1f48100120 2024-03-18T23:00:09.679 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:00:09.672+0000 7f1f506e9700 1 -- 172.21.15.175:0/2297398593 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f4819fd40 con 0x7f1f48100120 2024-03-18T23:00:09.679 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:00:09.672+0000 7f1f3f7fe700 1 -- 172.21.15.175:0/2297398593 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f4400f6c0 con 0x7f1f48100120 2024-03-18T23:00:09.679 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:00:09.672+0000 7f1f506e9700 1 -- 172.21.15.175:0/2297398593 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f481a02a0 con 0x7f1f48100120 2024-03-18T23:00:09.681 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:00:09.673+0000 7f1f506e9700 1 -- 172.21.15.175:0/2297398593 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f48063940 con 0x7f1f48100120 2024-03-18T23:00:09.681 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:00:09.673+0000 7f1f3f7fe700 1 -- 172.21.15.175:0/2297398593 <== mon.1 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 13) ==== 46162+0+0 (secure 0 0 0) 0x7f1f4402a070 con 0x7f1f48100120 2024-03-18T23:00:09.681 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:00:09.673+0000 7f1f3f7fe700 1 --2- 172.21.15.175:0/2297398593 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f1f34039450 0x7f1f3403b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:00:09.681 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:00:09.673+0000 7f1f3f7fe700 1 -- 172.21.15.175:0/2297398593 <== mon.1 v2:172.21.15.175:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f1f4404d1a0 con 0x7f1f48100120 2024-03-18T23:00:09.681 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:00:09.674+0000 7f1f4dc84700 1 --2- 172.21.15.175:0/2297398593 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f1f34039450 0x7f1f3403b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:00:09.681 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:00:09.674+0000 7f1f4dc84700 1 --2- 172.21.15.175:0/2297398593 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f1f34039450 0x7f1f3403b910 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f1f380098e0 tx=0x7f1f38006dd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:00:09.681 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:00:09.678+0000 7f1f3f7fe700 1 -- 172.21.15.175:0/2297398593 <== mon.1 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f1f44013070 con 0x7f1f48100120 2024-03-18T23:00:09.895 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:00:09.894+0000 7f1f506e9700 1 -- 172.21.15.175:0/2297398593 --> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi049=x;smithi139=y;smithi175=z", "target": ["mon-mgr", ""]}) -- 0x7f1f48062750 con 0x7f1f34039450 2024-03-18T23:00:09.902 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:00:09.901+0000 7f1f3f7fe700 1 -- 172.21.15.175:0/2297398593 <== mgr.14150 v2:172.21.15.49:6800/2083805211 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f1f48062750 con 0x7f1f34039450 2024-03-18T23:00:09.902 INFO:teuthology.orchestra.run.smithi175.stdout:Scheduled mgr update... 2024-03-18T23:00:09.904 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:00:09.903+0000 7f1f506e9700 1 -- 172.21.15.175:0/2297398593 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f1f34039450 msgr2=0x7f1f3403b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:00:09.904 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:00:09.903+0000 7f1f506e9700 1 --2- 172.21.15.175:0/2297398593 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f1f34039450 0x7f1f3403b910 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f1f380098e0 tx=0x7f1f38006dd0 comp rx=0 tx=0).stop 2024-03-18T23:00:09.904 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:00:09.903+0000 7f1f506e9700 1 -- 172.21.15.175:0/2297398593 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1f48100120 msgr2=0x7f1f48197640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:00:09.904 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:00:09.903+0000 7f1f506e9700 1 --2- 172.21.15.175:0/2297398593 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1f48100120 0x7f1f48197640 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f1f44000c00 tx=0x7f1f440050b0 comp rx=0 tx=0).stop 2024-03-18T23:00:09.904 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:00:09.903+0000 7f1f506e9700 1 -- 172.21.15.175:0/2297398593 shutdown_connections 2024-03-18T23:00:09.904 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:00:09.903+0000 7f1f506e9700 1 --2- 172.21.15.175:0/2297398593 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1f48198000 0x7f1f4819f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:00:09.904 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:00:09.903+0000 7f1f506e9700 1 --2- 172.21.15.175:0/2297398593 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f1f34039450 0x7f1f3403b910 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:00:09.904 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:00:09.903+0000 7f1f506e9700 1 --2- 172.21.15.175:0/2297398593 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1f48100120 0x7f1f48197640 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:00:09.904 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:00:09.903+0000 7f1f506e9700 1 --2- 172.21.15.175:0/2297398593 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1f48197b80 0x7f1f4819cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:00:09.905 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:00:09.903+0000 7f1f506e9700 1 -- 172.21.15.175:0/2297398593 >> 172.21.15.175:0/2297398593 conn(0x7f1f480fba60 msgr2=0x7f1f480fc7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:00:09.905 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:00:09.903+0000 7f1f506e9700 1 -- 172.21.15.175:0/2297398593 shutdown_connections 2024-03-18T23:00:09.905 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:00:09.903+0000 7f1f506e9700 1 -- 172.21.15.175:0/2297398593 wait complete. 2024-03-18T23:00:10.504 DEBUG:teuthology.orchestra.run.smithi139:mgr.y> sudo journalctl -f -n 0 -u ceph-c63591ea-e57a-11ee-95c9-87774f69a715@mgr.y.service 2024-03-18T23:00:10.507 DEBUG:teuthology.orchestra.run.smithi175:mgr.z> sudo journalctl -f -n 0 -u ceph-c63591ea-e57a-11ee-95c9-87774f69a715@mgr.z.service 2024-03-18T23:00:10.509 INFO:tasks.cephadm:Deploying OSDs... 2024-03-18T23:00:10.509 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-03-18T23:00:10.509 DEBUG:teuthology.orchestra.run.smithi049:> dd if=/scratch_devs of=/dev/stdout 2024-03-18T23:00:10.526 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-18T23:00:10.527 DEBUG:teuthology.orchestra.run.smithi049:> stat /dev/vg_nvme/lv_1 2024-03-18T23:00:10.583 INFO:teuthology.orchestra.run.smithi049.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-18T23:00:10.583 INFO:teuthology.orchestra.run.smithi049.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-18T23:00:10.583 INFO:teuthology.orchestra.run.smithi049.stdout:Device: 5h/5d Inode: 668 Links: 1 2024-03-18T23:00:10.583 INFO:teuthology.orchestra.run.smithi049.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-18T23:00:10.584 INFO:teuthology.orchestra.run.smithi049.stdout:Context: system_u:object_r:device_t:s0 2024-03-18T23:00:10.584 INFO:teuthology.orchestra.run.smithi049.stdout:Access: 2024-03-18 22:59:13.024175112 +0000 2024-03-18T23:00:10.584 INFO:teuthology.orchestra.run.smithi049.stdout:Modify: 2024-03-18 22:02:03.813019562 +0000 2024-03-18T23:00:10.584 INFO:teuthology.orchestra.run.smithi049.stdout:Change: 2024-03-18 22:02:03.813019562 +0000 2024-03-18T23:00:10.584 INFO:teuthology.orchestra.run.smithi049.stdout: Birth: 2024-03-18 22:02:03.813019562 +0000 2024-03-18T23:00:10.584 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-18T23:00:10.656 INFO:teuthology.orchestra.run.smithi049.stderr:1+0 records in 2024-03-18T23:00:10.656 INFO:teuthology.orchestra.run.smithi049.stderr:1+0 records out 2024-03-18T23:00:10.656 INFO:teuthology.orchestra.run.smithi049.stderr:512 bytes copied, 9.1836e-05 s, 5.6 MB/s 2024-03-18T23:00:10.657 DEBUG:teuthology.orchestra.run.smithi049:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-18T23:00:10.713 DEBUG:teuthology.orchestra.run.smithi049:> stat /dev/vg_nvme/lv_2 2024-03-18T23:00:10.768 INFO:teuthology.orchestra.run.smithi049.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-18T23:00:10.768 INFO:teuthology.orchestra.run.smithi049.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-18T23:00:10.768 INFO:teuthology.orchestra.run.smithi049.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-03-18T23:00:10.768 INFO:teuthology.orchestra.run.smithi049.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-18T23:00:10.768 INFO:teuthology.orchestra.run.smithi049.stdout:Context: system_u:object_r:device_t:s0 2024-03-18T23:00:10.768 INFO:teuthology.orchestra.run.smithi049.stdout:Access: 2024-03-18 22:59:13.060174095 +0000 2024-03-18T23:00:10.768 INFO:teuthology.orchestra.run.smithi049.stdout:Modify: 2024-03-18 22:02:04.169009616 +0000 2024-03-18T23:00:10.768 INFO:teuthology.orchestra.run.smithi049.stdout:Change: 2024-03-18 22:02:04.169009616 +0000 2024-03-18T23:00:10.768 INFO:teuthology.orchestra.run.smithi049.stdout: Birth: 2024-03-18 22:02:04.169009616 +0000 2024-03-18T23:00:10.768 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-18T23:00:10.837 INFO:teuthology.orchestra.run.smithi049.stderr:1+0 records in 2024-03-18T23:00:10.837 INFO:teuthology.orchestra.run.smithi049.stderr:1+0 records out 2024-03-18T23:00:10.837 INFO:teuthology.orchestra.run.smithi049.stderr:512 bytes copied, 0.000109542 s, 4.7 MB/s 2024-03-18T23:00:10.838 DEBUG:teuthology.orchestra.run.smithi049:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-18T23:00:10.894 DEBUG:teuthology.orchestra.run.smithi049:> stat /dev/vg_nvme/lv_3 2024-03-18T23:00:10.950 INFO:teuthology.orchestra.run.smithi049.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-18T23:00:10.951 INFO:teuthology.orchestra.run.smithi049.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-18T23:00:10.951 INFO:teuthology.orchestra.run.smithi049.stdout:Device: 5h/5d Inode: 705 Links: 1 2024-03-18T23:00:10.951 INFO:teuthology.orchestra.run.smithi049.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-18T23:00:10.951 INFO:teuthology.orchestra.run.smithi049.stdout:Context: system_u:object_r:device_t:s0 2024-03-18T23:00:10.951 INFO:teuthology.orchestra.run.smithi049.stdout:Access: 2024-03-18 22:59:13.094173134 +0000 2024-03-18T23:00:10.951 INFO:teuthology.orchestra.run.smithi049.stdout:Modify: 2024-03-18 22:02:04.505000229 +0000 2024-03-18T23:00:10.951 INFO:teuthology.orchestra.run.smithi049.stdout:Change: 2024-03-18 22:02:04.505000229 +0000 2024-03-18T23:00:10.951 INFO:teuthology.orchestra.run.smithi049.stdout: Birth: 2024-03-18 22:02:04.505000229 +0000 2024-03-18T23:00:10.951 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-18T23:00:11.018 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:10 smithi049 ceph-mon[62991]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T23:00:11.018 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:10 smithi049 ceph-mon[62991]: from='client.24100 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi049=x;smithi139=y;smithi175=z", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T23:00:11.018 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:10 smithi049 ceph-mon[62991]: Saving service mgr spec with placement smithi049=x;smithi139=y;smithi175=z;count:3 2024-03-18T23:00:11.018 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:10 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:00:11.018 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:10 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:00:11.019 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:10 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:00:11.019 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:10 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:00:11.019 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:10 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:00:11.019 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:10 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:00:11.019 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:10 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:00:11.019 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:10 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-18T23:00:11.019 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:10 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-18T23:00:11.019 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:10 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-18T23:00:11.019 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:10 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:00:11.019 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:10 smithi049 ceph-mon[62991]: Deploying daemon mgr.z on smithi175 2024-03-18T23:00:11.020 INFO:teuthology.orchestra.run.smithi049.stderr:1+0 records in 2024-03-18T23:00:11.020 INFO:teuthology.orchestra.run.smithi049.stderr:1+0 records out 2024-03-18T23:00:11.020 INFO:teuthology.orchestra.run.smithi049.stderr:512 bytes copied, 8.9205e-05 s, 5.7 MB/s 2024-03-18T23:00:11.021 DEBUG:teuthology.orchestra.run.smithi049:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-18T23:00:11.078 DEBUG:teuthology.orchestra.run.smithi049:> stat /dev/vg_nvme/lv_4 2024-03-18T23:00:11.133 INFO:teuthology.orchestra.run.smithi049.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-18T23:00:11.133 INFO:teuthology.orchestra.run.smithi049.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-18T23:00:11.133 INFO:teuthology.orchestra.run.smithi049.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-03-18T23:00:11.133 INFO:teuthology.orchestra.run.smithi049.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-18T23:00:11.133 INFO:teuthology.orchestra.run.smithi049.stdout:Context: system_u:object_r:device_t:s0 2024-03-18T23:00:11.134 INFO:teuthology.orchestra.run.smithi049.stdout:Access: 2024-03-18 22:59:13.128172174 +0000 2024-03-18T23:00:11.134 INFO:teuthology.orchestra.run.smithi049.stdout:Modify: 2024-03-18 22:02:04.840990843 +0000 2024-03-18T23:00:11.134 INFO:teuthology.orchestra.run.smithi049.stdout:Change: 2024-03-18 22:02:04.840990843 +0000 2024-03-18T23:00:11.134 INFO:teuthology.orchestra.run.smithi049.stdout: Birth: 2024-03-18 22:02:04.840990843 +0000 2024-03-18T23:00:11.134 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-18T23:00:11.203 INFO:teuthology.orchestra.run.smithi049.stderr:1+0 records in 2024-03-18T23:00:11.203 INFO:teuthology.orchestra.run.smithi049.stderr:1+0 records out 2024-03-18T23:00:11.203 INFO:teuthology.orchestra.run.smithi049.stderr:512 bytes copied, 0.000102878 s, 5.0 MB/s 2024-03-18T23:00:11.204 DEBUG:teuthology.orchestra.run.smithi049:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-18T23:00:11.211 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:10 smithi175 ceph-mon[67352]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T23:00:11.211 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:10 smithi175 ceph-mon[67352]: from='client.24100 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi049=x;smithi139=y;smithi175=z", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T23:00:11.211 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:10 smithi175 ceph-mon[67352]: Saving service mgr spec with placement smithi049=x;smithi139=y;smithi175=z;count:3 2024-03-18T23:00:11.212 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:10 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:00:11.212 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:10 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:00:11.212 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:10 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:00:11.212 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:10 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:00:11.212 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:10 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:00:11.212 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:10 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:00:11.212 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:10 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:00:11.212 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:10 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-18T23:00:11.212 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:10 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-18T23:00:11.212 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:10 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-18T23:00:11.212 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:10 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:00:11.212 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:10 smithi175 ceph-mon[67352]: Deploying daemon mgr.z on smithi175 2024-03-18T23:00:11.261 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-03-18T23:00:11.261 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/scratch_devs of=/dev/stdout 2024-03-18T23:00:11.280 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-18T23:00:11.280 DEBUG:teuthology.orchestra.run.smithi139:> stat /dev/vg_nvme/lv_1 2024-03-18T23:00:11.319 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:10 smithi139 ceph-mon[67282]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T23:00:11.319 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:10 smithi139 ceph-mon[67282]: from='client.24100 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi049=x;smithi139=y;smithi175=z", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T23:00:11.319 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:10 smithi139 ceph-mon[67282]: Saving service mgr spec with placement smithi049=x;smithi139=y;smithi175=z;count:3 2024-03-18T23:00:11.319 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:10 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:00:11.319 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:10 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:00:11.319 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:10 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:00:11.319 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:10 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:00:11.319 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:10 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:00:11.319 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:10 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:00:11.320 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:10 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:00:11.320 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:10 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-18T23:00:11.320 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:10 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-18T23:00:11.320 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:10 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-18T23:00:11.320 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:10 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:00:11.320 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:10 smithi139 ceph-mon[67282]: Deploying daemon mgr.z on smithi175 2024-03-18T23:00:11.337 INFO:teuthology.orchestra.run.smithi139.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-18T23:00:11.337 INFO:teuthology.orchestra.run.smithi139.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-18T23:00:11.337 INFO:teuthology.orchestra.run.smithi139.stdout:Device: 5h/5d Inode: 661 Links: 1 2024-03-18T23:00:11.337 INFO:teuthology.orchestra.run.smithi139.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-18T23:00:11.337 INFO:teuthology.orchestra.run.smithi139.stdout:Context: system_u:object_r:device_t:s0 2024-03-18T23:00:11.337 INFO:teuthology.orchestra.run.smithi139.stdout:Access: 2024-03-18 22:59:56.665162767 +0000 2024-03-18T23:00:11.337 INFO:teuthology.orchestra.run.smithi139.stdout:Modify: 2024-03-18 22:24:01.990037332 +0000 2024-03-18T23:00:11.337 INFO:teuthology.orchestra.run.smithi139.stdout:Change: 2024-03-18 22:24:01.990037332 +0000 2024-03-18T23:00:11.337 INFO:teuthology.orchestra.run.smithi139.stdout: Birth: 2024-03-18 22:24:01.990037332 +0000 2024-03-18T23:00:11.338 DEBUG:teuthology.orchestra.run.smithi139:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-18T23:00:11.407 INFO:teuthology.orchestra.run.smithi139.stderr:1+0 records in 2024-03-18T23:00:11.408 INFO:teuthology.orchestra.run.smithi139.stderr:1+0 records out 2024-03-18T23:00:11.408 INFO:teuthology.orchestra.run.smithi139.stderr:512 bytes copied, 0.000267675 s, 1.9 MB/s 2024-03-18T23:00:11.409 DEBUG:teuthology.orchestra.run.smithi139:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-18T23:00:11.468 DEBUG:teuthology.orchestra.run.smithi139:> stat /dev/vg_nvme/lv_2 2024-03-18T23:00:11.526 INFO:teuthology.orchestra.run.smithi139.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-18T23:00:11.526 INFO:teuthology.orchestra.run.smithi139.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-18T23:00:11.526 INFO:teuthology.orchestra.run.smithi139.stdout:Device: 5h/5d Inode: 682 Links: 1 2024-03-18T23:00:11.526 INFO:teuthology.orchestra.run.smithi139.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-18T23:00:11.526 INFO:teuthology.orchestra.run.smithi139.stdout:Context: system_u:object_r:device_t:s0 2024-03-18T23:00:11.526 INFO:teuthology.orchestra.run.smithi139.stdout:Access: 2024-03-18 22:59:56.665162767 +0000 2024-03-18T23:00:11.526 INFO:teuthology.orchestra.run.smithi139.stdout:Modify: 2024-03-18 22:24:02.322030123 +0000 2024-03-18T23:00:11.526 INFO:teuthology.orchestra.run.smithi139.stdout:Change: 2024-03-18 22:24:02.322030123 +0000 2024-03-18T23:00:11.527 INFO:teuthology.orchestra.run.smithi139.stdout: Birth: 2024-03-18 22:24:02.322030123 +0000 2024-03-18T23:00:11.527 DEBUG:teuthology.orchestra.run.smithi139:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-18T23:00:11.595 INFO:teuthology.orchestra.run.smithi139.stderr:1+0 records in 2024-03-18T23:00:11.595 INFO:teuthology.orchestra.run.smithi139.stderr:1+0 records out 2024-03-18T23:00:11.595 INFO:teuthology.orchestra.run.smithi139.stderr:512 bytes copied, 0.000227966 s, 2.2 MB/s 2024-03-18T23:00:11.596 DEBUG:teuthology.orchestra.run.smithi139:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-18T23:00:11.654 DEBUG:teuthology.orchestra.run.smithi139:> stat /dev/vg_nvme/lv_3 2024-03-18T23:00:11.709 INFO:teuthology.orchestra.run.smithi139.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-18T23:00:11.710 INFO:teuthology.orchestra.run.smithi139.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-18T23:00:11.710 INFO:teuthology.orchestra.run.smithi139.stdout:Device: 5h/5d Inode: 701 Links: 1 2024-03-18T23:00:11.710 INFO:teuthology.orchestra.run.smithi139.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-18T23:00:11.710 INFO:teuthology.orchestra.run.smithi139.stdout:Context: system_u:object_r:device_t:s0 2024-03-18T23:00:11.710 INFO:teuthology.orchestra.run.smithi139.stdout:Access: 2024-03-18 22:59:56.665162767 +0000 2024-03-18T23:00:11.710 INFO:teuthology.orchestra.run.smithi139.stdout:Modify: 2024-03-18 22:24:02.662022740 +0000 2024-03-18T23:00:11.710 INFO:teuthology.orchestra.run.smithi139.stdout:Change: 2024-03-18 22:24:02.662022740 +0000 2024-03-18T23:00:11.710 INFO:teuthology.orchestra.run.smithi139.stdout: Birth: 2024-03-18 22:24:02.662022740 +0000 2024-03-18T23:00:11.710 DEBUG:teuthology.orchestra.run.smithi139:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-18T23:00:11.776 INFO:teuthology.orchestra.run.smithi139.stderr:1+0 records in 2024-03-18T23:00:11.776 INFO:teuthology.orchestra.run.smithi139.stderr:1+0 records out 2024-03-18T23:00:11.776 INFO:teuthology.orchestra.run.smithi139.stderr:512 bytes copied, 0.000186681 s, 2.7 MB/s 2024-03-18T23:00:11.778 DEBUG:teuthology.orchestra.run.smithi139:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-18T23:00:11.833 DEBUG:teuthology.orchestra.run.smithi139:> stat /dev/vg_nvme/lv_4 2024-03-18T23:00:11.889 INFO:teuthology.orchestra.run.smithi139.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-18T23:00:11.889 INFO:teuthology.orchestra.run.smithi139.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-18T23:00:11.889 INFO:teuthology.orchestra.run.smithi139.stdout:Device: 5h/5d Inode: 719 Links: 1 2024-03-18T23:00:11.889 INFO:teuthology.orchestra.run.smithi139.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-18T23:00:11.889 INFO:teuthology.orchestra.run.smithi139.stdout:Context: system_u:object_r:device_t:s0 2024-03-18T23:00:11.889 INFO:teuthology.orchestra.run.smithi139.stdout:Access: 2024-03-18 22:59:56.666162745 +0000 2024-03-18T23:00:11.889 INFO:teuthology.orchestra.run.smithi139.stdout:Modify: 2024-03-18 22:24:02.990015618 +0000 2024-03-18T23:00:11.889 INFO:teuthology.orchestra.run.smithi139.stdout:Change: 2024-03-18 22:24:02.990015618 +0000 2024-03-18T23:00:11.889 INFO:teuthology.orchestra.run.smithi139.stdout: Birth: 2024-03-18 22:24:02.990015618 +0000 2024-03-18T23:00:11.890 DEBUG:teuthology.orchestra.run.smithi139:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-18T23:00:11.957 INFO:teuthology.orchestra.run.smithi139.stderr:1+0 records in 2024-03-18T23:00:11.957 INFO:teuthology.orchestra.run.smithi139.stderr:1+0 records out 2024-03-18T23:00:11.957 INFO:teuthology.orchestra.run.smithi139.stderr:512 bytes copied, 0.000200168 s, 2.6 MB/s 2024-03-18T23:00:11.958 DEBUG:teuthology.orchestra.run.smithi139:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-18T23:00:12.015 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-03-18T23:00:12.015 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/scratch_devs of=/dev/stdout 2024-03-18T23:00:12.031 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-18T23:00:12.032 DEBUG:teuthology.orchestra.run.smithi175:> stat /dev/vg_nvme/lv_1 2024-03-18T23:00:12.086 INFO:teuthology.orchestra.run.smithi175.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-18T23:00:12.087 INFO:teuthology.orchestra.run.smithi175.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-18T23:00:12.087 INFO:teuthology.orchestra.run.smithi175.stdout:Device: 5h/5d Inode: 661 Links: 1 2024-03-18T23:00:12.087 INFO:teuthology.orchestra.run.smithi175.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-18T23:00:12.087 INFO:teuthology.orchestra.run.smithi175.stdout:Context: system_u:object_r:device_t:s0 2024-03-18T23:00:12.087 INFO:teuthology.orchestra.run.smithi175.stdout:Access: 2024-03-18 22:59:47.013682264 +0000 2024-03-18T23:00:12.087 INFO:teuthology.orchestra.run.smithi175.stdout:Modify: 2024-03-18 22:15:28.593630707 +0000 2024-03-18T23:00:12.087 INFO:teuthology.orchestra.run.smithi175.stdout:Change: 2024-03-18 22:15:28.593630707 +0000 2024-03-18T23:00:12.087 INFO:teuthology.orchestra.run.smithi175.stdout: Birth: 2024-03-18 22:15:28.593630707 +0000 2024-03-18T23:00:12.087 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-18T23:00:12.153 INFO:teuthology.orchestra.run.smithi175.stderr:1+0 records in 2024-03-18T23:00:12.154 INFO:teuthology.orchestra.run.smithi175.stderr:1+0 records out 2024-03-18T23:00:12.154 INFO:teuthology.orchestra.run.smithi175.stderr:512 bytes copied, 0.000269182 s, 1.9 MB/s 2024-03-18T23:00:12.155 DEBUG:teuthology.orchestra.run.smithi175:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-18T23:00:12.211 DEBUG:teuthology.orchestra.run.smithi175:> stat /dev/vg_nvme/lv_2 2024-03-18T23:00:12.267 INFO:teuthology.orchestra.run.smithi175.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-18T23:00:12.267 INFO:teuthology.orchestra.run.smithi175.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-18T23:00:12.267 INFO:teuthology.orchestra.run.smithi175.stdout:Device: 5h/5d Inode: 681 Links: 1 2024-03-18T23:00:12.267 INFO:teuthology.orchestra.run.smithi175.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-18T23:00:12.267 INFO:teuthology.orchestra.run.smithi175.stdout:Context: system_u:object_r:device_t:s0 2024-03-18T23:00:12.267 INFO:teuthology.orchestra.run.smithi175.stdout:Access: 2024-03-18 22:59:47.013682264 +0000 2024-03-18T23:00:12.267 INFO:teuthology.orchestra.run.smithi175.stdout:Modify: 2024-03-18 22:15:28.951625140 +0000 2024-03-18T23:00:12.268 INFO:teuthology.orchestra.run.smithi175.stdout:Change: 2024-03-18 22:15:28.951625140 +0000 2024-03-18T23:00:12.268 INFO:teuthology.orchestra.run.smithi175.stdout: Birth: 2024-03-18 22:15:28.951625140 +0000 2024-03-18T23:00:12.268 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-18T23:00:12.429 INFO:teuthology.orchestra.run.smithi175.stderr:1+0 records in 2024-03-18T23:00:12.429 INFO:teuthology.orchestra.run.smithi175.stderr:1+0 records out 2024-03-18T23:00:12.429 INFO:teuthology.orchestra.run.smithi175.stderr:512 bytes copied, 0.000174127 s, 2.9 MB/s 2024-03-18T23:00:12.431 DEBUG:teuthology.orchestra.run.smithi175:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-18T23:00:12.447 DEBUG:teuthology.orchestra.run.smithi175:> stat /dev/vg_nvme/lv_3 2024-03-18T23:00:12.506 INFO:teuthology.orchestra.run.smithi175.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-18T23:00:12.506 INFO:teuthology.orchestra.run.smithi175.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-18T23:00:12.506 INFO:teuthology.orchestra.run.smithi175.stdout:Device: 5h/5d Inode: 700 Links: 1 2024-03-18T23:00:12.506 INFO:teuthology.orchestra.run.smithi175.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-18T23:00:12.506 INFO:teuthology.orchestra.run.smithi175.stdout:Context: system_u:object_r:device_t:s0 2024-03-18T23:00:12.506 INFO:teuthology.orchestra.run.smithi175.stdout:Access: 2024-03-18 23:00:12.368278407 +0000 2024-03-18T23:00:12.506 INFO:teuthology.orchestra.run.smithi175.stdout:Modify: 2024-03-18 22:15:29.313619511 +0000 2024-03-18T23:00:12.506 INFO:teuthology.orchestra.run.smithi175.stdout:Change: 2024-03-18 22:15:29.313619511 +0000 2024-03-18T23:00:12.506 INFO:teuthology.orchestra.run.smithi175.stdout: Birth: 2024-03-18 22:15:29.313619511 +0000 2024-03-18T23:00:12.507 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-18T23:00:12.621 INFO:teuthology.orchestra.run.smithi175.stderr:1+0 records in 2024-03-18T23:00:12.622 INFO:teuthology.orchestra.run.smithi175.stderr:1+0 records out 2024-03-18T23:00:12.622 INFO:teuthology.orchestra.run.smithi175.stderr:512 bytes copied, 0.000184007 s, 2.8 MB/s 2024-03-18T23:00:12.623 DEBUG:teuthology.orchestra.run.smithi175:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-18T23:00:12.638 DEBUG:teuthology.orchestra.run.smithi175:> stat /dev/vg_nvme/lv_4 2024-03-18T23:00:12.694 INFO:teuthology.orchestra.run.smithi175.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-18T23:00:12.695 INFO:teuthology.orchestra.run.smithi175.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-18T23:00:12.695 INFO:teuthology.orchestra.run.smithi175.stdout:Device: 5h/5d Inode: 720 Links: 1 2024-03-18T23:00:12.695 INFO:teuthology.orchestra.run.smithi175.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-18T23:00:12.695 INFO:teuthology.orchestra.run.smithi175.stdout:Context: system_u:object_r:device_t:s0 2024-03-18T23:00:12.695 INFO:teuthology.orchestra.run.smithi175.stdout:Access: 2024-03-18 23:00:12.565275269 +0000 2024-03-18T23:00:12.695 INFO:teuthology.orchestra.run.smithi175.stdout:Modify: 2024-03-18 22:15:29.653614224 +0000 2024-03-18T23:00:12.695 INFO:teuthology.orchestra.run.smithi175.stdout:Change: 2024-03-18 22:15:29.653614224 +0000 2024-03-18T23:00:12.695 INFO:teuthology.orchestra.run.smithi175.stdout: Birth: 2024-03-18 22:15:29.653614224 +0000 2024-03-18T23:00:12.695 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-18T23:00:12.711 INFO:journalctl@ceph.mgr.z.smithi175.stdout:Mar 18 23:00:12 smithi175 systemd[1]: Starting Ceph mgr.z for c63591ea-e57a-11ee-95c9-87774f69a715... 2024-03-18T23:00:12.739 INFO:teuthology.orchestra.run.smithi175.stderr:1+0 records in 2024-03-18T23:00:12.739 INFO:teuthology.orchestra.run.smithi175.stderr:1+0 records out 2024-03-18T23:00:12.739 INFO:teuthology.orchestra.run.smithi175.stderr:512 bytes copied, 0.000181508 s, 2.8 MB/s 2024-03-18T23:00:12.740 DEBUG:teuthology.orchestra.run.smithi175:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-18T23:00:12.796 INFO:tasks.cephadm:Deploying osd.0 on smithi049 with /dev/vg_nvme/lv_4... 2024-03-18T23:00:12.797 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c63591ea-e57a-11ee-95c9-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-03-18T23:00:13.019 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:12 smithi175 ceph-mon[67352]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T23:00:13.019 INFO:journalctl@ceph.mgr.z.smithi175.stdout:Mar 18 23:00:12 smithi175 podman[69179]: 2024-03-18 23:00:12.749463961 +0000 UTC m=+0.018293578 image pull 2312c8e42d0069a2dbb2364bddb00264db4b800e7f18a7e7e4eb8f3ecab2ff1c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639 2024-03-18T23:00:13.019 INFO:journalctl@ceph.mgr.z.smithi175.stdout:Mar 18 23:00:12 smithi175 podman[69179]: 2024-03-18 23:00:12.888021429 +0000 UTC m=+0.156851045 container create 8a53c65637956d8964995800b50d22027ead645be69b2bfc84eaac7264e09068 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-z, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5) 2024-03-18T23:00:13.053 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T23:00:13.130 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:12 smithi049 ceph-mon[62991]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T23:00:13.250 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:12 smithi139 ceph-mon[67282]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T23:00:13.461 INFO:journalctl@ceph.mgr.z.smithi175.stdout:Mar 18 23:00:13 smithi175 podman[69179]: 2024-03-18 23:00:13.115441414 +0000 UTC m=+0.384271029 container init 8a53c65637956d8964995800b50d22027ead645be69b2bfc84eaac7264e09068 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-z, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-18T23:00:13.462 INFO:journalctl@ceph.mgr.z.smithi175.stdout:Mar 18 23:00:13 smithi175 podman[69179]: 2024-03-18 23:00:13.118049575 +0000 UTC m=+0.386879198 container start 8a53c65637956d8964995800b50d22027ead645be69b2bfc84eaac7264e09068 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-z, GIT_CLEAN=True, org.label-schema.build-date=20240307, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-18T23:00:13.462 INFO:journalctl@ceph.mgr.z.smithi175.stdout:Mar 18 23:00:13 smithi175 ceph-mgr[69218]: -- 172.21.15.175:0/1606301844 <== mon.1 v2:172.21.15.175:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5648648ba5a0 con 0x56486488ec00 2024-03-18T23:00:13.462 INFO:journalctl@ceph.mgr.z.smithi175.stdout:Mar 18 23:00:13 smithi175 bash[69179]: 8a53c65637956d8964995800b50d22027ead645be69b2bfc84eaac7264e09068 2024-03-18T23:00:13.462 INFO:journalctl@ceph.mgr.z.smithi175.stdout:Mar 18 23:00:13 smithi175 systemd[1]: Started Ceph mgr.z for c63591ea-e57a-11ee-95c9-87774f69a715. 2024-03-18T23:00:13.961 INFO:journalctl@ceph.mgr.z.smithi175.stdout:Mar 18 23:00:13 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-z[69214]: 2024-03-18T23:00:13.661+0000 7fad9dfd4200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-18T23:00:13.961 INFO:journalctl@ceph.mgr.z.smithi175.stdout:Mar 18 23:00:13 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-z[69214]: 2024-03-18T23:00:13.751+0000 7fad9dfd4200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-18T23:00:14.359 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:14 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:00:14.359 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:14 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:00:14.359 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:14 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:00:14.359 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:14 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-18T23:00:14.359 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:14 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-18T23:00:14.359 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:14 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-18T23:00:14.360 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:14 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:00:14.360 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:14 smithi049 ceph-mon[62991]: Deploying daemon mgr.y on smithi139 2024-03-18T23:00:14.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:14 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:00:14.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:14 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:00:14.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:14 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:00:14.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:14 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-18T23:00:14.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:14 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-18T23:00:14.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:14 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-18T23:00:14.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:14 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:00:14.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:14 smithi139 ceph-mon[67282]: Deploying daemon mgr.y on smithi139 2024-03-18T23:00:14.612 INFO:journalctl@ceph.mgr.z.smithi175.stdout:Mar 18 23:00:14 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-z[69214]: 2024-03-18T23:00:14.381+0000 7fad9dfd4200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-18T23:00:14.612 INFO:journalctl@ceph.mgr.z.smithi175.stdout:Mar 18 23:00:14 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-z[69214]: 2024-03-18T23:00:14.472+0000 7fad9dfd4200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-18T23:00:14.612 INFO:journalctl@ceph.mgr.z.smithi175.stdout:Mar 18 23:00:14 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-z[69214]: 2024-03-18T23:00:14.611+0000 7fad9dfd4200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-18T23:00:14.612 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:14 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:00:14.613 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:14 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:00:14.613 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:14 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:00:14.613 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:14 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-18T23:00:14.613 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:14 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-18T23:00:14.613 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:14 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-18T23:00:14.613 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:14 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:00:14.613 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:14 smithi175 ceph-mon[67352]: Deploying daemon mgr.y on smithi139 2024-03-18T23:00:14.926 INFO:teuthology.orchestra.run.smithi049.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-03-18T23:00:14.926 INFO:teuthology.orchestra.run.smithi049.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-03-18T23:00:14.926 INFO:teuthology.orchestra.run.smithi049.stderr: stderr: 10+0 records in 2024-03-18T23:00:14.926 INFO:teuthology.orchestra.run.smithi049.stderr:10+0 records out 2024-03-18T23:00:14.926 INFO:teuthology.orchestra.run.smithi049.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00821314 s, 1.3 GB/s 2024-03-18T23:00:14.926 INFO:teuthology.orchestra.run.smithi049.stderr:--> Zapping successful for: 2024-03-18T23:00:14.961 INFO:journalctl@ceph.mgr.z.smithi175.stdout:Mar 18 23:00:14 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-z[69214]: 2024-03-18T23:00:14.704+0000 7fad9dfd4200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-18T23:00:14.961 INFO:journalctl@ceph.mgr.z.smithi175.stdout:Mar 18 23:00:14 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-z[69214]: 2024-03-18T23:00:14.784+0000 7fad9dfd4200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-18T23:00:15.316 INFO:journalctl@ceph.mgr.y.smithi139.stdout:Mar 18 23:00:15 smithi139 systemd[1]: Starting Ceph mgr.y for c63591ea-e57a-11ee-95c9-87774f69a715... 2024-03-18T23:00:15.320 INFO:journalctl@ceph.mgr.z.smithi175.stdout:Mar 18 23:00:15 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-z[69214]: 2024-03-18T23:00:15.042+0000 7fad9dfd4200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-18T23:00:15.320 INFO:journalctl@ceph.mgr.z.smithi175.stdout:Mar 18 23:00:15 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-z[69214]: 2024-03-18T23:00:15.147+0000 7fad9dfd4200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-18T23:00:15.320 INFO:journalctl@ceph.mgr.z.smithi175.stdout:Mar 18 23:00:15 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-z[69214]: 2024-03-18T23:00:15.230+0000 7fad9dfd4200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-18T23:00:15.630 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:15 smithi049 ceph-mon[62991]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T23:00:15.676 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:15 smithi139 ceph-mon[67282]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T23:00:15.677 INFO:journalctl@ceph.mgr.y.smithi139.stdout:Mar 18 23:00:15 smithi139 podman[68670]: 2024-03-18 23:00:15.31933083 +0000 UTC m=+0.017411688 image pull 2312c8e42d0069a2dbb2364bddb00264db4b800e7f18a7e7e4eb8f3ecab2ff1c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639 2024-03-18T23:00:15.677 INFO:journalctl@ceph.mgr.y.smithi139.stdout:Mar 18 23:00:15 smithi139 podman[68670]: 2024-03-18 23:00:15.424406975 +0000 UTC m=+0.122487829 container create 03a4539507343a2b75c8f0163a6e2b6e6440c56f827341e9f37ca854a769851d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-y, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-03-18T23:00:15.677 INFO:journalctl@ceph.mgr.y.smithi139.stdout:Mar 18 23:00:15 smithi139 podman[68670]: 2024-03-18 23:00:15.676884459 +0000 UTC m=+0.374965330 container init 03a4539507343a2b75c8f0163a6e2b6e6440c56f827341e9f37ca854a769851d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-y, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.build-date=20240307, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-18T23:00:15.686 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c63591ea-e57a-11ee-95c9-87774f69a715 -- ceph orch daemon add osd smithi049:vg_nvme/lv_4 2024-03-18T23:00:15.711 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:15 smithi175 ceph-mon[67352]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T23:00:15.712 INFO:journalctl@ceph.mgr.z.smithi175.stdout:Mar 18 23:00:15 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-z[69214]: 2024-03-18T23:00:15.323+0000 7fad9dfd4200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-18T23:00:15.712 INFO:journalctl@ceph.mgr.z.smithi175.stdout:Mar 18 23:00:15 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-z[69214]: 2024-03-18T23:00:15.498+0000 7fad9dfd4200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-18T23:00:15.946 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T23:00:16.065 INFO:journalctl@ceph.mgr.y.smithi139.stdout:Mar 18 23:00:15 smithi139 podman[68670]: 2024-03-18 23:00:15.679908702 +0000 UTC m=+0.377989558 container start 03a4539507343a2b75c8f0163a6e2b6e6440c56f827341e9f37ca854a769851d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-y, org.label-schema.build-date=20240307, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-03-18T23:00:16.065 INFO:journalctl@ceph.mgr.y.smithi139.stdout:Mar 18 23:00:15 smithi139 ceph-mgr[68689]: -- 172.21.15.139:0/884409198 <== mon.2 v2:172.21.15.139:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x558c174b05a0 con 0x558c17484c00 2024-03-18T23:00:16.066 INFO:journalctl@ceph.mgr.y.smithi139.stdout:Mar 18 23:00:15 smithi139 bash[68670]: 03a4539507343a2b75c8f0163a6e2b6e6440c56f827341e9f37ca854a769851d 2024-03-18T23:00:16.066 INFO:journalctl@ceph.mgr.y.smithi139.stdout:Mar 18 23:00:15 smithi139 systemd[1]: Started Ceph mgr.y for c63591ea-e57a-11ee-95c9-87774f69a715. 2024-03-18T23:00:16.565 INFO:journalctl@ceph.mgr.y.smithi139.stdout:Mar 18 23:00:16 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-y[68685]: 2024-03-18T23:00:16.225+0000 7fd1ed302200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-18T23:00:16.565 INFO:journalctl@ceph.mgr.y.smithi139.stdout:Mar 18 23:00:16 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-y[68685]: 2024-03-18T23:00:16.320+0000 7fd1ed302200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-18T23:00:16.873 INFO:journalctl@ceph.mgr.z.smithi175.stdout:Mar 18 23:00:16 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-z[69214]: 2024-03-18T23:00:16.475+0000 7fad9dfd4200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-18T23:00:16.873 INFO:journalctl@ceph.mgr.z.smithi175.stdout:Mar 18 23:00:16 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-z[69214]: 2024-03-18T23:00:16.618+0000 7fad9dfd4200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-18T23:00:16.947 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:16 smithi049 ceph-mon[62991]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T23:00:16.947 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:16 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:00:16.947 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:16 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:00:16.947 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:16 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:00:16.948 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:16 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:00:16.948 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:16 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:00:17.211 INFO:journalctl@ceph.mgr.z.smithi175.stdout:Mar 18 23:00:16 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-z[69214]: 2024-03-18T23:00:16.968+0000 7fad9dfd4200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-18T23:00:17.212 INFO:journalctl@ceph.mgr.z.smithi175.stdout:Mar 18 23:00:17 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-z[69214]: 2024-03-18T23:00:17.061+0000 7fad9dfd4200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-18T23:00:17.212 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:16 smithi175 ceph-mon[67352]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T23:00:17.212 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:16 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:00:17.212 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:16 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:00:17.212 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:16 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:00:17.212 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:16 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:00:17.212 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:16 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:00:17.225 INFO:journalctl@ceph.mgr.y.smithi139.stdout:Mar 18 23:00:16 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-y[68685]: 2024-03-18T23:00:16.992+0000 7fd1ed302200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-18T23:00:17.225 INFO:journalctl@ceph.mgr.y.smithi139.stdout:Mar 18 23:00:17 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-y[68685]: 2024-03-18T23:00:17.083+0000 7fd1ed302200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-18T23:00:17.226 INFO:journalctl@ceph.mgr.y.smithi139.stdout:Mar 18 23:00:17 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-y[68685]: 2024-03-18T23:00:17.224+0000 7fd1ed302200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-18T23:00:17.226 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:16 smithi139 ceph-mon[67282]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T23:00:17.226 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:16 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:00:17.226 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:16 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:00:17.226 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:16 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:00:17.226 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:16 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:00:17.226 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:16 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:00:17.346 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:00:17.344+0000 7f8b07107700 1 -- 172.21.15.49:0/253742868 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8b001024e0 msgr2=0x7f8b00102940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:00:17.347 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:00:17.344+0000 7f8b07107700 1 --2- 172.21.15.49:0/253742868 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8b001024e0 0x7f8b00102940 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f8aec0099c0 tx=0x7f8aec009cd0 comp rx=0 tx=0).stop 2024-03-18T23:00:17.347 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:00:17.344+0000 7f8b07107700 1 -- 172.21.15.49:0/253742868 shutdown_connections 2024-03-18T23:00:17.347 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:00:17.344+0000 7f8b07107700 1 --2- 172.21.15.49:0/253742868 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8b00102e80 0x7f8b00109730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:00:17.347 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:00:17.344+0000 7f8b07107700 1 --2- 172.21.15.49:0/253742868 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8b001012e0 0x7f8b00101700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:00:17.347 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:00:17.344+0000 7f8b07107700 1 --2- 172.21.15.49:0/253742868 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8b001024e0 0x7f8b00102940 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:00:17.347 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:00:17.344+0000 7f8b07107700 1 -- 172.21.15.49:0/253742868 >> 172.21.15.49:0/253742868 conn(0x7f8b000fc8a0 msgr2=0x7f8b000fecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:00:17.347 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:00:17.344+0000 7f8b07107700 1 -- 172.21.15.49:0/253742868 shutdown_connections 2024-03-18T23:00:17.347 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:00:17.344+0000 7f8b07107700 1 -- 172.21.15.49:0/253742868 wait complete. 2024-03-18T23:00:17.347 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:00:17.345+0000 7f8b07107700 1 Processor -- start 2024-03-18T23:00:17.347 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:00:17.345+0000 7f8b07107700 1 -- start start 2024-03-18T23:00:17.347 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:00:17.345+0000 7f8b07107700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8b001012e0 0x7f8b001985c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:00:17.348 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:00:17.345+0000 7f8b07107700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8b001024e0 0x7f8b00198b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:00:17.348 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:00:17.345+0000 7f8b07107700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8b00102e80 0x7f8b0019fb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:00:17.348 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:00:17.345+0000 7f8b07107700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f8b0010ce90 con 0x7f8b001012e0 2024-03-18T23:00:17.348 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:00:17.345+0000 7f8b07107700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f8b0010cd10 con 0x7f8b001024e0 2024-03-18T23:00:17.348 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:00:17.345+0000 7f8b07107700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f8b0010d010 con 0x7f8b00102e80 2024-03-18T23:00:17.348 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:00:17.346+0000 7f8b04ea3700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8b001012e0 0x7f8b001985c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:00:17.348 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:00:17.346+0000 7f8b04ea3700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8b001012e0 0x7f8b001985c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:41242/0 (socket says 172.21.15.49:41242) 2024-03-18T23:00:17.348 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:00:17.346+0000 7f8b04ea3700 1 -- 172.21.15.49:0/3761972979 learned_addr learned my addr 172.21.15.49:0/3761972979 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:00:17.348 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:00:17.346+0000 7f8afffff700 1 --2- 172.21.15.49:0/3761972979 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8b001024e0 0x7f8b00198b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:00:17.348 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:00:17.346+0000 7f8b056a4700 1 --2- 172.21.15.49:0/3761972979 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8b00102e80 0x7f8b0019fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:00:17.348 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:00:17.346+0000 7f8b04ea3700 1 -- 172.21.15.49:0/3761972979 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8b00102e80 msgr2=0x7f8b0019fb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:00:17.348 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:00:17.346+0000 7f8b04ea3700 1 --2- 172.21.15.49:0/3761972979 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8b00102e80 0x7f8b0019fb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:00:17.348 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:00:17.346+0000 7f8b04ea3700 1 -- 172.21.15.49:0/3761972979 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8b001024e0 msgr2=0x7f8b00198b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:00:17.349 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:00:17.346+0000 7f8b04ea3700 1 --2- 172.21.15.49:0/3761972979 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8b001024e0 0x7f8b00198b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:00:17.349 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:00:17.346+0000 7f8b04ea3700 1 -- 172.21.15.49:0/3761972979 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b001a01b0 con 0x7f8b001012e0 2024-03-18T23:00:17.349 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:00:17.346+0000 7f8b04ea3700 1 --2- 172.21.15.49:0/3761972979 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8b001012e0 0x7f8b001985c0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f8af400d9b0 tx=0x7f8af400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:00:17.349 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:00:17.346+0000 7f8afdffb700 1 -- 172.21.15.49:0/3761972979 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8af4017030 con 0x7f8b001012e0 2024-03-18T23:00:17.349 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:00:17.346+0000 7f8afdffb700 1 -- 172.21.15.49:0/3761972979 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8af4004500 con 0x7f8b001012e0 2024-03-18T23:00:17.349 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:00:17.346+0000 7f8afdffb700 1 -- 172.21.15.49:0/3761972979 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8af4003970 con 0x7f8b001012e0 2024-03-18T23:00:17.349 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:00:17.346+0000 7f8b07107700 1 -- 172.21.15.49:0/3761972979 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b00103760 con 0x7f8b001012e0 2024-03-18T23:00:17.349 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:00:17.347+0000 7f8b07107700 1 -- 172.21.15.49:0/3761972979 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b00103c50 con 0x7f8b001012e0 2024-03-18T23:00:17.350 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:00:17.347+0000 7f8afdffb700 1 -- 172.21.15.49:0/3761972979 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 13) ==== 46162+0+0 (secure 0 0 0) 0x7f8af401a070 con 0x7f8b001012e0 2024-03-18T23:00:17.350 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:00:17.347+0000 7f8afdffb700 1 --2- 172.21.15.49:0/3761972979 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f8af0039200 0x7f8af003b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:00:17.350 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:00:17.347+0000 7f8afdffb700 1 -- 172.21.15.49:0/3761972979 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f8af404b970 con 0x7f8b001012e0 2024-03-18T23:00:17.350 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:00:17.347+0000 7f8b07107700 1 -- 172.21.15.49:0/3761972979 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b00068fd0 con 0x7f8b001012e0 2024-03-18T23:00:17.350 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:00:17.348+0000 7f8afffff700 1 --2- 172.21.15.49:0/3761972979 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f8af0039200 0x7f8af003b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:00:17.350 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:00:17.348+0000 7f8afffff700 1 --2- 172.21.15.49:0/3761972979 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f8af0039200 0x7f8af003b6c0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f8aec000c00 tx=0x7f8aec005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:00:17.352 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:00:17.353+0000 7f8afdffb700 1 -- 172.21.15.49:0/3761972979 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f8af400f3c0 con 0x7f8b001012e0 2024-03-18T23:00:17.565 INFO:journalctl@ceph.mgr.y.smithi139.stdout:Mar 18 23:00:17 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-y[68685]: 2024-03-18T23:00:17.327+0000 7fd1ed302200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-18T23:00:17.565 INFO:journalctl@ceph.mgr.y.smithi139.stdout:Mar 18 23:00:17 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-y[68685]: 2024-03-18T23:00:17.410+0000 7fd1ed302200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-18T23:00:17.577 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:00:17.577+0000 7f8b07107700 1 -- 172.21.15.49:0/3761972979 --> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi049:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f8b000fea60 con 0x7f8af0039200 2024-03-18T23:00:17.711 INFO:journalctl@ceph.mgr.z.smithi175.stdout:Mar 18 23:00:17 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-z[69214]: 2024-03-18T23:00:17.256+0000 7fad9dfd4200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-18T23:00:17.711 INFO:journalctl@ceph.mgr.z.smithi175.stdout:Mar 18 23:00:17 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-z[69214]: 2024-03-18T23:00:17.448+0000 7fad9dfd4200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-18T23:00:17.953 INFO:journalctl@ceph.mgr.y.smithi139.stdout:Mar 18 23:00:17 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-y[68685]: 2024-03-18T23:00:17.673+0000 7fd1ed302200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-18T23:00:17.953 INFO:journalctl@ceph.mgr.y.smithi139.stdout:Mar 18 23:00:17 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-y[68685]: 2024-03-18T23:00:17.778+0000 7fd1ed302200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-18T23:00:17.953 INFO:journalctl@ceph.mgr.y.smithi139.stdout:Mar 18 23:00:17 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-y[68685]: 2024-03-18T23:00:17.861+0000 7fd1ed302200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-18T23:00:17.954 INFO:journalctl@ceph.mgr.y.smithi139.stdout:Mar 18 23:00:17 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-y[68685]: 2024-03-18T23:00:17.952+0000 7fd1ed302200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-18T23:00:17.991 INFO:journalctl@ceph.mgr.z.smithi175.stdout:Mar 18 23:00:17 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-z[69214]: 2024-03-18T23:00:17.731+0000 7fad9dfd4200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-18T23:00:17.991 INFO:journalctl@ceph.mgr.z.smithi175.stdout:Mar 18 23:00:17 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-z[69214]: 2024-03-18T23:00:17.991+0000 7fad9dfd4200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-18T23:00:18.315 INFO:journalctl@ceph.mgr.y.smithi139.stdout:Mar 18 23:00:18 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-y[68685]: 2024-03-18T23:00:18.125+0000 7fd1ed302200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-18T23:00:18.363 INFO:journalctl@ceph.mgr.z.smithi175.stdout:Mar 18 23:00:18 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-z[69214]: 2024-03-18T23:00:18.223+0000 7fad9dfd4200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-18T23:00:18.441 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:18 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:00:18.442 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:18 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:00:18.442 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:18 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:00:18.442 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:18 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:00:18.442 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:18 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:00:18.442 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:18 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:00:18.442 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:18 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:00:18.442 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:18 smithi049 ceph-mon[62991]: from='client.14220 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi049:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T23:00:18.442 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:18 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-18T23:00:18.442 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:18 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-18T23:00:18.442 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:18 smithi049 ceph-mon[62991]: Reconfiguring mgr.x (unknown last config time)... 2024-03-18T23:00:18.442 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:18 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-18T23:00:18.442 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:18 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:00:18.443 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:18 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-18T23:00:18.443 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:18 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:00:18.443 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:18 smithi049 ceph-mon[62991]: Reconfiguring daemon mgr.x on smithi049 2024-03-18T23:00:18.711 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:18 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:00:18.712 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:18 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:00:18.712 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:18 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:00:18.712 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:18 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:00:18.712 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:18 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:00:18.712 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:18 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:00:18.712 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:18 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:00:18.712 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:18 smithi175 ceph-mon[67352]: from='client.14220 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi049:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T23:00:18.712 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:18 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-18T23:00:18.712 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:18 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-18T23:00:18.712 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:18 smithi175 ceph-mon[67352]: Reconfiguring mgr.x (unknown last config time)... 2024-03-18T23:00:18.712 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:18 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-18T23:00:18.712 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:18 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:00:18.712 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:18 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-18T23:00:18.712 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:18 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:00:18.713 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:18 smithi175 ceph-mon[67352]: Reconfiguring daemon mgr.x on smithi049 2024-03-18T23:00:18.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:18 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:00:18.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:18 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:00:18.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:18 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:00:18.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:18 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:00:18.816 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:18 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:00:18.816 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:18 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:00:18.816 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:18 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:00:18.816 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:18 smithi139 ceph-mon[67282]: from='client.14220 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi049:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T23:00:18.816 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:18 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-18T23:00:18.816 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:18 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-18T23:00:18.816 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:18 smithi139 ceph-mon[67282]: Reconfiguring mgr.x (unknown last config time)... 2024-03-18T23:00:18.816 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:18 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-18T23:00:18.816 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:18 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:00:18.816 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:18 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-18T23:00:18.816 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:18 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:00:18.816 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:18 smithi139 ceph-mon[67282]: Reconfiguring daemon mgr.x on smithi049 2024-03-18T23:00:19.422 INFO:journalctl@ceph.mgr.z.smithi175.stdout:Mar 18 23:00:19 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-z[69214]: 2024-03-18T23:00:19.009+0000 7fad9dfd4200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-18T23:00:19.423 INFO:journalctl@ceph.mgr.y.smithi139.stdout:Mar 18 23:00:19 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-y[68685]: 2024-03-18T23:00:19.083+0000 7fd1ed302200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-18T23:00:19.423 INFO:journalctl@ceph.mgr.y.smithi139.stdout:Mar 18 23:00:19 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-y[68685]: 2024-03-18T23:00:19.213+0000 7fd1ed302200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-18T23:00:19.676 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:19 smithi175 ceph-mon[67352]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T23:00:19.676 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:19 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:00:19.676 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:19 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:00:19.676 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:19 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:00:19.676 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:19 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:00:19.677 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:19 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:00:19.677 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:19 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:00:19.677 INFO:journalctl@ceph.mgr.z.smithi175.stdout:Mar 18 23:00:19 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-z[69214]: 2024-03-18T23:00:19.580+0000 7fad9dfd4200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-18T23:00:19.677 INFO:journalctl@ceph.mgr.z.smithi175.stdout:Mar 18 23:00:19 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-z[69214]: 2024-03-18T23:00:19.676+0000 7fad9dfd4200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-18T23:00:19.708 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:19 smithi049 ceph-mon[62991]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T23:00:19.708 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:19 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:00:19.708 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:19 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:00:19.709 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:19 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:00:19.709 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:19 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:00:19.709 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:19 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:00:19.709 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:19 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:00:19.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:19 smithi139 ceph-mon[67282]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T23:00:19.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:19 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:00:19.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:19 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:00:19.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:19 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:00:19.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:19 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:00:19.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:19 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:00:19.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:19 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:00:19.815 INFO:journalctl@ceph.mgr.y.smithi139.stdout:Mar 18 23:00:19 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-y[68685]: 2024-03-18T23:00:19.558+0000 7fd1ed302200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-18T23:00:19.815 INFO:journalctl@ceph.mgr.y.smithi139.stdout:Mar 18 23:00:19 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-y[68685]: 2024-03-18T23:00:19.647+0000 7fd1ed302200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-18T23:00:19.961 INFO:journalctl@ceph.mgr.z.smithi175.stdout:Mar 18 23:00:19 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-z[69214]: 2024-03-18T23:00:19.888+0000 7fad9dfd4200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-18T23:00:20.304 INFO:journalctl@ceph.mgr.y.smithi139.stdout:Mar 18 23:00:19 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-y[68685]: 2024-03-18T23:00:19.840+0000 7fd1ed302200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-18T23:00:20.304 INFO:journalctl@ceph.mgr.y.smithi139.stdout:Mar 18 23:00:20 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-y[68685]: 2024-03-18T23:00:20.021+0000 7fd1ed302200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-18T23:00:20.566 INFO:journalctl@ceph.mgr.y.smithi139.stdout:Mar 18 23:00:20 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-y[68685]: 2024-03-18T23:00:20.303+0000 7fd1ed302200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-18T23:00:20.883 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:00:20.882+0000 7f8afdffb700 1 -- 172.21.15.49:0/3761972979 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mgrmap(e 14) ==== 91698+0+0 (secure 0 0 0) 0x7f8af4004020 con 0x7f8b001012e0 2024-03-18T23:00:20.885 INFO:journalctl@ceph.mgr.y.smithi139.stdout:Mar 18 23:00:20 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-y[68685]: 2024-03-18T23:00:20.565+0000 7fd1ed302200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-18T23:00:20.885 INFO:journalctl@ceph.mgr.y.smithi139.stdout:Mar 18 23:00:20 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-y[68685]: 2024-03-18T23:00:20.796+0000 7fd1ed302200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-18T23:00:21.211 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:20 smithi175 ceph-mon[67352]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T23:00:21.212 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:20 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:00:21.212 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:20 smithi175 ceph-mon[67352]: Standby manager daemon z started 2024-03-18T23:00:21.212 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:20 smithi175 ceph-mon[67352]: from='mgr.? 172.21.15.175:0/1813472782' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-03-18T23:00:21.212 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:20 smithi175 ceph-mon[67352]: from='mgr.? 172.21.15.175:0/1813472782' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-18T23:00:21.212 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:20 smithi175 ceph-mon[67352]: from='mgr.? 172.21.15.175:0/1813472782' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-03-18T23:00:21.212 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:20 smithi175 ceph-mon[67352]: from='mgr.? 172.21.15.175:0/1813472782' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-18T23:00:21.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:20 smithi139 ceph-mon[67282]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T23:00:21.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:20 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:00:21.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:20 smithi139 ceph-mon[67282]: Standby manager daemon z started 2024-03-18T23:00:21.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:20 smithi139 ceph-mon[67282]: from='mgr.? 172.21.15.175:0/1813472782' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-03-18T23:00:21.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:20 smithi139 ceph-mon[67282]: from='mgr.? 172.21.15.175:0/1813472782' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-18T23:00:21.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:20 smithi139 ceph-mon[67282]: from='mgr.? 172.21.15.175:0/1813472782' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-03-18T23:00:21.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:20 smithi139 ceph-mon[67282]: from='mgr.? 172.21.15.175:0/1813472782' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-18T23:00:21.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:20 smithi049 ceph-mon[62991]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T23:00:21.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:20 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:00:21.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:20 smithi049 ceph-mon[62991]: Standby manager daemon z started 2024-03-18T23:00:21.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:20 smithi049 ceph-mon[62991]: from='mgr.? 172.21.15.175:0/1813472782' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-03-18T23:00:21.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:20 smithi049 ceph-mon[62991]: from='mgr.? 172.21.15.175:0/1813472782' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-18T23:00:21.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:20 smithi049 ceph-mon[62991]: from='mgr.? 172.21.15.175:0/1813472782' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-03-18T23:00:21.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:20 smithi049 ceph-mon[62991]: from='mgr.? 172.21.15.175:0/1813472782' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-18T23:00:21.886 INFO:journalctl@ceph.mgr.y.smithi139.stdout:Mar 18 23:00:21 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-y[68685]: 2024-03-18T23:00:21.585+0000 7fd1ed302200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-18T23:00:22.150 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:21 smithi139 ceph-mon[67282]: mgrmap e14: x(active, since 81s), standbys: z 2024-03-18T23:00:22.151 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:21 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-03-18T23:00:22.151 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:21 smithi139 ceph-mon[67282]: from='client.? 172.21.15.49:0/2450832426' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e7fc9daa-4098-4ba5-8019-9aaef1aab3e3"}]: dispatch 2024-03-18T23:00:22.151 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:21 smithi139 ceph-mon[67282]: from='client.? 172.21.15.49:0/2450832426' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e7fc9daa-4098-4ba5-8019-9aaef1aab3e3"}]': finished 2024-03-18T23:00:22.151 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:21 smithi139 ceph-mon[67282]: osdmap e5: 1 total, 0 up, 1 in 2024-03-18T23:00:22.151 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:21 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-18T23:00:22.151 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:21 smithi139 ceph-mon[67282]: from='client.? 172.21.15.49:0/754240405' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-18T23:00:22.211 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:21 smithi175 ceph-mon[67352]: mgrmap e14: x(active, since 81s), standbys: z 2024-03-18T23:00:22.212 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:21 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-03-18T23:00:22.212 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:21 smithi175 ceph-mon[67352]: from='client.? 172.21.15.49:0/2450832426' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e7fc9daa-4098-4ba5-8019-9aaef1aab3e3"}]: dispatch 2024-03-18T23:00:22.212 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:21 smithi175 ceph-mon[67352]: from='client.? 172.21.15.49:0/2450832426' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e7fc9daa-4098-4ba5-8019-9aaef1aab3e3"}]': finished 2024-03-18T23:00:22.212 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:21 smithi175 ceph-mon[67352]: osdmap e5: 1 total, 0 up, 1 in 2024-03-18T23:00:22.212 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:21 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-18T23:00:22.212 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:21 smithi175 ceph-mon[67352]: from='client.? 172.21.15.49:0/754240405' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-18T23:00:22.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:21 smithi049 ceph-mon[62991]: mgrmap e14: x(active, since 81s), standbys: z 2024-03-18T23:00:22.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:21 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-03-18T23:00:22.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:21 smithi049 ceph-mon[62991]: from='client.? 172.21.15.49:0/2450832426' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e7fc9daa-4098-4ba5-8019-9aaef1aab3e3"}]: dispatch 2024-03-18T23:00:22.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:21 smithi049 ceph-mon[62991]: from='client.? 172.21.15.49:0/2450832426' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e7fc9daa-4098-4ba5-8019-9aaef1aab3e3"}]': finished 2024-03-18T23:00:22.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:21 smithi049 ceph-mon[62991]: osdmap e5: 1 total, 0 up, 1 in 2024-03-18T23:00:22.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:21 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-18T23:00:22.381 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:21 smithi049 ceph-mon[62991]: from='client.? 172.21.15.49:0/754240405' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-18T23:00:22.455 INFO:journalctl@ceph.mgr.y.smithi139.stdout:Mar 18 23:00:22 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-y[68685]: 2024-03-18T23:00:22.149+0000 7fd1ed302200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-18T23:00:22.455 INFO:journalctl@ceph.mgr.y.smithi139.stdout:Mar 18 23:00:22 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-y[68685]: 2024-03-18T23:00:22.245+0000 7fd1ed302200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-18T23:00:22.455 INFO:journalctl@ceph.mgr.y.smithi139.stdout:Mar 18 23:00:22 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-y[68685]: 2024-03-18T23:00:22.454+0000 7fd1ed302200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-18T23:00:23.202 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:00:23.202+0000 7f8afdffb700 1 -- 172.21.15.49:0/3761972979 <== mon.0 v2:172.21.15.49:3300/0 8 ==== mgrmap(e 15) ==== 137234+0+0 (secure 0 0 0) 0x7f8af4036e30 con 0x7f8b001012e0 2024-03-18T23:00:23.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:23 smithi175 ceph-mon[67352]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T23:00:23.462 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:23 smithi175 ceph-mon[67352]: Standby manager daemon y started 2024-03-18T23:00:23.462 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:23 smithi175 ceph-mon[67352]: from='mgr.? 172.21.15.139:0/3592656615' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-03-18T23:00:23.462 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:23 smithi175 ceph-mon[67352]: from='mgr.? 172.21.15.139:0/3592656615' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-18T23:00:23.462 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:23 smithi175 ceph-mon[67352]: from='mgr.? 172.21.15.139:0/3592656615' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-03-18T23:00:23.462 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:23 smithi175 ceph-mon[67352]: from='mgr.? 172.21.15.139:0/3592656615' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-18T23:00:23.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:23 smithi139 ceph-mon[67282]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T23:00:23.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:23 smithi139 ceph-mon[67282]: Standby manager daemon y started 2024-03-18T23:00:23.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:23 smithi139 ceph-mon[67282]: from='mgr.? 172.21.15.139:0/3592656615' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-03-18T23:00:23.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:23 smithi139 ceph-mon[67282]: from='mgr.? 172.21.15.139:0/3592656615' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-18T23:00:23.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:23 smithi139 ceph-mon[67282]: from='mgr.? 172.21.15.139:0/3592656615' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-03-18T23:00:23.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:23 smithi139 ceph-mon[67282]: from='mgr.? 172.21.15.139:0/3592656615' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-18T23:00:23.581 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:23 smithi049 ceph-mon[62991]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T23:00:23.581 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:23 smithi049 ceph-mon[62991]: Standby manager daemon y started 2024-03-18T23:00:23.581 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:23 smithi049 ceph-mon[62991]: from='mgr.? 172.21.15.139:0/3592656615' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-03-18T23:00:23.581 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:23 smithi049 ceph-mon[62991]: from='mgr.? 172.21.15.139:0/3592656615' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-18T23:00:23.581 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:23 smithi049 ceph-mon[62991]: from='mgr.? 172.21.15.139:0/3592656615' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-03-18T23:00:23.582 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:23 smithi049 ceph-mon[62991]: from='mgr.? 172.21.15.139:0/3592656615' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-18T23:00:24.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:24 smithi175 ceph-mon[67352]: mgrmap e15: x(active, since 83s), standbys: z, y 2024-03-18T23:00:24.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:24 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-18T23:00:24.561 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:24 smithi049 ceph-mon[62991]: mgrmap e15: x(active, since 83s), standbys: z, y 2024-03-18T23:00:24.561 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:24 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-18T23:00:24.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:24 smithi139 ceph-mon[67282]: mgrmap e15: x(active, since 83s), standbys: z, y 2024-03-18T23:00:24.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:24 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-18T23:00:25.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:25 smithi175 ceph-mon[67352]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T23:00:25.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:25 smithi139 ceph-mon[67282]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T23:00:25.630 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:25 smithi049 ceph-mon[62991]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T23:00:27.302 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:27 smithi049 ceph-mon[62991]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T23:00:27.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:27 smithi175 ceph-mon[67352]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T23:00:27.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:27 smithi139 ceph-mon[67282]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T23:00:28.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:28 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-18T23:00:28.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:28 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:00:28.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:28 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-18T23:00:28.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:28 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:00:28.711 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:28 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-18T23:00:28.711 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:28 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:00:29.512 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:29 smithi049 ceph-mon[62991]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T23:00:29.512 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:29 smithi049 ceph-mon[62991]: Deploying daemon osd.0 on smithi049 2024-03-18T23:00:29.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:29 smithi139 ceph-mon[67282]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T23:00:29.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:29 smithi139 ceph-mon[67282]: Deploying daemon osd.0 on smithi049 2024-03-18T23:00:29.711 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:29 smithi175 ceph-mon[67352]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T23:00:29.711 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:29 smithi175 ceph-mon[67352]: Deploying daemon osd.0 on smithi049 2024-03-18T23:00:31.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:31 smithi139 ceph-mon[67282]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T23:00:31.613 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:31 smithi049 ceph-mon[62991]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T23:00:31.711 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:31 smithi175 ceph-mon[67352]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T23:00:33.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:33 smithi139 ceph-mon[67282]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T23:00:33.630 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:33 smithi049 ceph-mon[62991]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T23:00:33.711 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:33 smithi175 ceph-mon[67352]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T23:00:34.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:34 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:00:34.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:34 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:00:34.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:34 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:00:34.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:34 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:00:34.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:34 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:00:34.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:34 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:00:34.961 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:34 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:00:34.962 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:34 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:00:34.962 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:34 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:00:35.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:35 smithi139 ceph-mon[67282]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T23:00:35.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:35 smithi049 ceph-mon[62991]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T23:00:35.961 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:35 smithi175 ceph-mon[67352]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T23:00:37.184 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:00:37.182+0000 7f8afdffb700 1 -- 172.21.15.49:0/3761972979 <== mgr.14150 v2:172.21.15.49:6800/2083805211 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f8b000fea60 con 0x7f8af0039200 2024-03-18T23:00:37.184 INFO:teuthology.orchestra.run.smithi049.stdout:Created osd(s) 0 on host 'smithi049' 2024-03-18T23:00:37.186 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:00:37.185+0000 7f8b07107700 1 -- 172.21.15.49:0/3761972979 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f8af0039200 msgr2=0x7f8af003b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:00:37.186 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:00:37.185+0000 7f8b07107700 1 --2- 172.21.15.49:0/3761972979 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f8af0039200 0x7f8af003b6c0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f8aec000c00 tx=0x7f8aec005fb0 comp rx=0 tx=0).stop 2024-03-18T23:00:37.186 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:00:37.185+0000 7f8b07107700 1 -- 172.21.15.49:0/3761972979 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8b001012e0 msgr2=0x7f8b001985c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:00:37.186 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:00:37.185+0000 7f8b07107700 1 --2- 172.21.15.49:0/3761972979 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8b001012e0 0x7f8b001985c0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f8af400d9b0 tx=0x7f8af400dcc0 comp rx=0 tx=0).stop 2024-03-18T23:00:37.186 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:00:37.185+0000 7f8b07107700 1 -- 172.21.15.49:0/3761972979 shutdown_connections 2024-03-18T23:00:37.186 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:00:37.185+0000 7f8b07107700 1 --2- 172.21.15.49:0/3761972979 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f8af0039200 0x7f8af003b6c0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:00:37.186 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:00:37.185+0000 7f8b07107700 1 --2- 172.21.15.49:0/3761972979 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8b00102e80 0x7f8b0019fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:00:37.186 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:00:37.185+0000 7f8b07107700 1 --2- 172.21.15.49:0/3761972979 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8b001012e0 0x7f8b001985c0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:00:37.187 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:00:37.185+0000 7f8b07107700 1 --2- 172.21.15.49:0/3761972979 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8b001024e0 0x7f8b00198b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:00:37.187 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:00:37.185+0000 7f8b07107700 1 -- 172.21.15.49:0/3761972979 >> 172.21.15.49:0/3761972979 conn(0x7f8b000fc8a0 msgr2=0x7f8b000fe350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:00:37.187 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:00:37.185+0000 7f8b07107700 1 -- 172.21.15.49:0/3761972979 shutdown_connections 2024-03-18T23:00:37.187 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:00:37.185+0000 7f8b07107700 1 -- 172.21.15.49:0/3761972979 wait complete. 2024-03-18T23:00:37.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:36 smithi139 ceph-mon[67282]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T23:00:37.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:36 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:00:37.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:36 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:00:37.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:36 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:00:37.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:36 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:00:37.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:36 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:00:37.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:36 smithi049 ceph-mon[62991]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T23:00:37.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:36 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:00:37.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:36 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:00:37.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:36 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:00:37.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:36 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:00:37.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:36 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:00:37.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:36 smithi175 ceph-mon[67352]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T23:00:37.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:36 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:00:37.462 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:36 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:00:37.462 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:36 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:00:37.462 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:36 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:00:37.462 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:36 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:00:37.797 DEBUG:teuthology.orchestra.run.smithi049:osd.0> sudo journalctl -f -n 0 -u ceph-c63591ea-e57a-11ee-95c9-87774f69a715@osd.0.service 2024-03-18T23:00:37.799 INFO:tasks.cephadm:Deploying osd.1 on smithi139 with /dev/vg_nvme/lv_4... 2024-03-18T23:00:37.799 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c63591ea-e57a-11ee-95c9-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-03-18T23:00:38.043 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/mon.b/config 2024-03-18T23:00:38.161 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:37 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:00:38.161 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:37 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:00:38.162 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:37 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:00:38.162 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:37 smithi139 ceph-mon[67282]: from='osd.0 [v2:172.21.15.49:6802/1651162501,v1:172.21.15.49:6803/1651162501]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-18T23:00:38.241 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:37 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:00:38.241 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:37 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:00:38.241 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:37 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:00:38.241 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:37 smithi049 ceph-mon[62991]: from='osd.0 [v2:172.21.15.49:6802/1651162501,v1:172.21.15.49:6803/1651162501]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-18T23:00:38.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:37 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:00:38.462 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:37 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:00:38.462 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:37 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:00:38.462 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:37 smithi175 ceph-mon[67352]: from='osd.0 [v2:172.21.15.49:6802/1651162501,v1:172.21.15.49:6803/1651162501]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-18T23:00:39.374 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:39 smithi139 ceph-mon[67282]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T23:00:39.374 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:39 smithi139 ceph-mon[67282]: from='osd.0 [v2:172.21.15.49:6802/1651162501,v1:172.21.15.49:6803/1651162501]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-03-18T23:00:39.374 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:39 smithi139 ceph-mon[67282]: osdmap e6: 1 total, 0 up, 1 in 2024-03-18T23:00:39.374 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:39 smithi139 ceph-mon[67282]: from='osd.0 [v2:172.21.15.49:6802/1651162501,v1:172.21.15.49:6803/1651162501]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi049", "root=default"]}]: dispatch 2024-03-18T23:00:39.374 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:39 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-18T23:00:39.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:39 smithi175 ceph-mon[67352]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T23:00:39.462 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:39 smithi175 ceph-mon[67352]: from='osd.0 [v2:172.21.15.49:6802/1651162501,v1:172.21.15.49:6803/1651162501]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-03-18T23:00:39.462 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:39 smithi175 ceph-mon[67352]: osdmap e6: 1 total, 0 up, 1 in 2024-03-18T23:00:39.462 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:39 smithi175 ceph-mon[67352]: from='osd.0 [v2:172.21.15.49:6802/1651162501,v1:172.21.15.49:6803/1651162501]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi049", "root=default"]}]: dispatch 2024-03-18T23:00:39.462 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:39 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-18T23:00:39.630 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:39 smithi049 ceph-mon[62991]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T23:00:39.630 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:39 smithi049 ceph-mon[62991]: from='osd.0 [v2:172.21.15.49:6802/1651162501,v1:172.21.15.49:6803/1651162501]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-03-18T23:00:39.630 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:39 smithi049 ceph-mon[62991]: osdmap e6: 1 total, 0 up, 1 in 2024-03-18T23:00:39.630 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:39 smithi049 ceph-mon[62991]: from='osd.0 [v2:172.21.15.49:6802/1651162501,v1:172.21.15.49:6803/1651162501]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi049", "root=default"]}]: dispatch 2024-03-18T23:00:39.630 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:39 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-18T23:00:39.926 INFO:teuthology.orchestra.run.smithi139.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-03-18T23:00:39.926 INFO:teuthology.orchestra.run.smithi139.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-03-18T23:00:39.926 INFO:teuthology.orchestra.run.smithi139.stderr: stderr: 10+0 records in 2024-03-18T23:00:39.926 INFO:teuthology.orchestra.run.smithi139.stderr:10+0 records out 2024-03-18T23:00:39.926 INFO:teuthology.orchestra.run.smithi139.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0126714 s, 828 MB/s 2024-03-18T23:00:39.926 INFO:teuthology.orchestra.run.smithi139.stderr:--> Zapping successful for: 2024-03-18T23:00:40.043 INFO:journalctl@ceph.osd.0.smithi049.stdout:Mar 18 23:00:39 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-0[77740]: 2024-03-18T23:00:39.654+0000 7fc4948e6700 -1 osd.0 0 waiting for initial osdmap 2024-03-18T23:00:40.043 INFO:journalctl@ceph.osd.0.smithi049.stdout:Mar 18 23:00:39 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-0[77740]: 2024-03-18T23:00:39.657+0000 7fc48c6d1700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-18T23:00:40.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:40 smithi049 ceph-mon[62991]: from='osd.0 [v2:172.21.15.49:6802/1651162501,v1:172.21.15.49:6803/1651162501]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi049", "root=default"]}]': finished 2024-03-18T23:00:40.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:40 smithi049 ceph-mon[62991]: osdmap e7: 1 total, 0 up, 1 in 2024-03-18T23:00:40.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:40 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-18T23:00:40.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:40 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-18T23:00:40.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:40 smithi049 ceph-mon[62991]: from='osd.0 [v2:172.21.15.49:6802/1651162501,v1:172.21.15.49:6803/1651162501]' entity='osd.0' 2024-03-18T23:00:40.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:40 smithi175 ceph-mon[67352]: from='osd.0 [v2:172.21.15.49:6802/1651162501,v1:172.21.15.49:6803/1651162501]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi049", "root=default"]}]': finished 2024-03-18T23:00:40.462 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:40 smithi175 ceph-mon[67352]: osdmap e7: 1 total, 0 up, 1 in 2024-03-18T23:00:40.462 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:40 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-18T23:00:40.462 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:40 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-18T23:00:40.462 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:40 smithi175 ceph-mon[67352]: from='osd.0 [v2:172.21.15.49:6802/1651162501,v1:172.21.15.49:6803/1651162501]' entity='osd.0' 2024-03-18T23:00:40.524 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:40 smithi139 ceph-mon[67282]: from='osd.0 [v2:172.21.15.49:6802/1651162501,v1:172.21.15.49:6803/1651162501]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi049", "root=default"]}]': finished 2024-03-18T23:00:40.524 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:40 smithi139 ceph-mon[67282]: osdmap e7: 1 total, 0 up, 1 in 2024-03-18T23:00:40.524 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:40 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-18T23:00:40.524 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:40 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-18T23:00:40.524 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:40 smithi139 ceph-mon[67282]: from='osd.0 [v2:172.21.15.49:6802/1651162501,v1:172.21.15.49:6803/1651162501]' entity='osd.0' 2024-03-18T23:00:40.551 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c63591ea-e57a-11ee-95c9-87774f69a715 -- ceph orch daemon add osd smithi139:vg_nvme/lv_4 2024-03-18T23:00:40.794 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/mon.b/config 2024-03-18T23:00:41.462 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:41 smithi175 ceph-mon[67352]: purged_snaps scrub starts 2024-03-18T23:00:41.462 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:41 smithi175 ceph-mon[67352]: purged_snaps scrub ok 2024-03-18T23:00:41.463 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:41 smithi175 ceph-mon[67352]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T23:00:41.463 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:41 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-18T23:00:41.463 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:41 smithi175 ceph-mon[67352]: osd.0 [v2:172.21.15.49:6802/1651162501,v1:172.21.15.49:6803/1651162501] boot 2024-03-18T23:00:41.463 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:41 smithi175 ceph-mon[67352]: osdmap e8: 1 total, 1 up, 1 in 2024-03-18T23:00:41.463 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:41 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-18T23:00:41.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:41 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:00:41.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:41 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:00:41.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:41 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:00:41.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:41 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:00:41.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:41 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:00:41.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:41 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:00:41.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:41 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:00:41.471 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:41 smithi139 ceph-mon[67282]: purged_snaps scrub starts 2024-03-18T23:00:41.472 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:41 smithi139 ceph-mon[67282]: purged_snaps scrub ok 2024-03-18T23:00:41.472 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:41 smithi139 ceph-mon[67282]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T23:00:41.472 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:41 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-18T23:00:41.472 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:41 smithi139 ceph-mon[67282]: osd.0 [v2:172.21.15.49:6802/1651162501,v1:172.21.15.49:6803/1651162501] boot 2024-03-18T23:00:41.472 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:41 smithi139 ceph-mon[67282]: osdmap e8: 1 total, 1 up, 1 in 2024-03-18T23:00:41.472 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:41 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-18T23:00:41.472 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:41 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:00:41.472 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:41 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:00:41.472 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:41 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:00:41.472 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:41 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:00:41.472 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:41 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:00:41.473 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:41 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:00:41.473 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:41 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:00:41.630 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:41 smithi049 ceph-mon[62991]: purged_snaps scrub starts 2024-03-18T23:00:41.630 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:41 smithi049 ceph-mon[62991]: purged_snaps scrub ok 2024-03-18T23:00:41.630 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:41 smithi049 ceph-mon[62991]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T23:00:41.631 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:41 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-18T23:00:41.631 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:41 smithi049 ceph-mon[62991]: osd.0 [v2:172.21.15.49:6802/1651162501,v1:172.21.15.49:6803/1651162501] boot 2024-03-18T23:00:41.631 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:41 smithi049 ceph-mon[62991]: osdmap e8: 1 total, 1 up, 1 in 2024-03-18T23:00:41.631 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:41 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-18T23:00:41.631 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:41 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:00:41.631 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:41 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:00:41.631 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:41 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:00:41.631 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:41 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:00:41.631 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:41 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:00:41.631 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:41 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:00:41.631 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:41 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:00:42.214 INFO:teuthology.orchestra.run.smithi139.stderr:2024-03-18T23:00:42.212+0000 7f5e7ff84700 1 -- 172.21.15.139:0/4033775401 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5e78102360 msgr2=0x7f5e78102780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:00:42.214 INFO:teuthology.orchestra.run.smithi139.stderr:2024-03-18T23:00:42.212+0000 7f5e7ff84700 1 --2- 172.21.15.139:0/4033775401 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5e78102360 0x7f5e78102780 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f5e740099c0 tx=0x7f5e74009cd0 comp rx=0 tx=0).stop 2024-03-18T23:00:42.214 INFO:teuthology.orchestra.run.smithi139.stderr:2024-03-18T23:00:42.212+0000 7f5e7ff84700 1 -- 172.21.15.139:0/4033775401 shutdown_connections 2024-03-18T23:00:42.214 INFO:teuthology.orchestra.run.smithi139.stderr:2024-03-18T23:00:42.212+0000 7f5e7ff84700 1 --2- 172.21.15.139:0/4033775401 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5e78103f00 0x7f5e7810a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:00:42.214 INFO:teuthology.orchestra.run.smithi139.stderr:2024-03-18T23:00:42.212+0000 7f5e7ff84700 1 --2- 172.21.15.139:0/4033775401 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5e78102360 0x7f5e78102780 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:00:42.214 INFO:teuthology.orchestra.run.smithi139.stderr:2024-03-18T23:00:42.212+0000 7f5e7ff84700 1 --2- 172.21.15.139:0/4033775401 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5e78103560 0x7f5e781039c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:00:42.214 INFO:teuthology.orchestra.run.smithi139.stderr:2024-03-18T23:00:42.212+0000 7f5e7ff84700 1 -- 172.21.15.139:0/4033775401 >> 172.21.15.139:0/4033775401 conn(0x7f5e780fd920 msgr2=0x7f5e780ffd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:00:42.214 INFO:teuthology.orchestra.run.smithi139.stderr:2024-03-18T23:00:42.212+0000 7f5e7ff84700 1 -- 172.21.15.139:0/4033775401 shutdown_connections 2024-03-18T23:00:42.215 INFO:teuthology.orchestra.run.smithi139.stderr:2024-03-18T23:00:42.212+0000 7f5e7ff84700 1 -- 172.21.15.139:0/4033775401 wait complete. 2024-03-18T23:00:42.215 INFO:teuthology.orchestra.run.smithi139.stderr:2024-03-18T23:00:42.212+0000 7f5e7ff84700 1 Processor -- start 2024-03-18T23:00:42.215 INFO:teuthology.orchestra.run.smithi139.stderr:2024-03-18T23:00:42.212+0000 7f5e7ff84700 1 -- start start 2024-03-18T23:00:42.215 INFO:teuthology.orchestra.run.smithi139.stderr:2024-03-18T23:00:42.213+0000 7f5e7ff84700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5e78102360 0x7f5e780760b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:00:42.215 INFO:teuthology.orchestra.run.smithi139.stderr:2024-03-18T23:00:42.213+0000 7f5e7ff84700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5e78103560 0x7f5e78074700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:00:42.215 INFO:teuthology.orchestra.run.smithi139.stderr:2024-03-18T23:00:42.213+0000 7f5e7ff84700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5e78103f00 0x7f5e78074c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:00:42.215 INFO:teuthology.orchestra.run.smithi139.stderr:2024-03-18T23:00:42.213+0000 7f5e7ff84700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f5e7810e020 con 0x7f5e78102360 2024-03-18T23:00:42.215 INFO:teuthology.orchestra.run.smithi139.stderr:2024-03-18T23:00:42.213+0000 7f5e7ff84700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f5e7810dea0 con 0x7f5e78103f00 2024-03-18T23:00:42.215 INFO:teuthology.orchestra.run.smithi139.stderr:2024-03-18T23:00:42.213+0000 7f5e7ff84700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f5e7810e1a0 con 0x7f5e78103560 2024-03-18T23:00:42.215 INFO:teuthology.orchestra.run.smithi139.stderr:2024-03-18T23:00:42.213+0000 7f5e7e521700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5e78103f00 0x7f5e78074c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:00:42.215 INFO:teuthology.orchestra.run.smithi139.stderr:2024-03-18T23:00:42.213+0000 7f5e7e521700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5e78103f00 0x7f5e78074c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:52118/0 (socket says 172.21.15.139:52118) 2024-03-18T23:00:42.215 INFO:teuthology.orchestra.run.smithi139.stderr:2024-03-18T23:00:42.213+0000 7f5e7e521700 1 -- 172.21.15.139:0/4019005961 learned_addr learned my addr 172.21.15.139:0/4019005961 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-03-18T23:00:42.215 INFO:teuthology.orchestra.run.smithi139.stderr:2024-03-18T23:00:42.213+0000 7f5e7d51f700 1 --2- 172.21.15.139:0/4019005961 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5e78103560 0x7f5e78074700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:00:42.216 INFO:teuthology.orchestra.run.smithi139.stderr:2024-03-18T23:00:42.213+0000 7f5e7dd20700 1 --2- 172.21.15.139:0/4019005961 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5e78102360 0x7f5e780760b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:00:42.216 INFO:teuthology.orchestra.run.smithi139.stderr:2024-03-18T23:00:42.213+0000 7f5e7e521700 1 -- 172.21.15.139:0/4019005961 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5e78103560 msgr2=0x7f5e78074700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:00:42.216 INFO:teuthology.orchestra.run.smithi139.stderr:2024-03-18T23:00:42.213+0000 7f5e7e521700 1 --2- 172.21.15.139:0/4019005961 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5e78103560 0x7f5e78074700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:00:42.216 INFO:teuthology.orchestra.run.smithi139.stderr:2024-03-18T23:00:42.213+0000 7f5e7e521700 1 -- 172.21.15.139:0/4019005961 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5e78102360 msgr2=0x7f5e780760b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:00:42.216 INFO:teuthology.orchestra.run.smithi139.stderr:2024-03-18T23:00:42.213+0000 7f5e7e521700 1 --2- 172.21.15.139:0/4019005961 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5e78102360 0x7f5e780760b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:00:42.216 INFO:teuthology.orchestra.run.smithi139.stderr:2024-03-18T23:00:42.213+0000 7f5e7e521700 1 -- 172.21.15.139:0/4019005961 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e780754f0 con 0x7f5e78103f00 2024-03-18T23:00:42.216 INFO:teuthology.orchestra.run.smithi139.stderr:2024-03-18T23:00:42.213+0000 7f5e7e521700 1 --2- 172.21.15.139:0/4019005961 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5e78103f00 0x7f5e78074c40 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f5e7000b970 tx=0x7f5e7000bc80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:00:42.216 INFO:teuthology.orchestra.run.smithi139.stderr:2024-03-18T23:00:42.213+0000 7f5e6effd700 1 -- 172.21.15.139:0/4019005961 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e7000d860 con 0x7f5e78103f00 2024-03-18T23:00:42.216 INFO:teuthology.orchestra.run.smithi139.stderr:2024-03-18T23:00:42.213+0000 7f5e6effd700 1 -- 172.21.15.139:0/4019005961 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5e7000e430 con 0x7f5e78103f00 2024-03-18T23:00:42.216 INFO:teuthology.orchestra.run.smithi139.stderr:2024-03-18T23:00:42.213+0000 7f5e7ff84700 1 -- 172.21.15.139:0/4019005961 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e781a2880 con 0x7f5e78103f00 2024-03-18T23:00:42.216 INFO:teuthology.orchestra.run.smithi139.stderr:2024-03-18T23:00:42.213+0000 7f5e6effd700 1 -- 172.21.15.139:0/4019005961 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e70017660 con 0x7f5e78103f00 2024-03-18T23:00:42.216 INFO:teuthology.orchestra.run.smithi139.stderr:2024-03-18T23:00:42.214+0000 7f5e7ff84700 1 -- 172.21.15.139:0/4019005961 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e781a2e10 con 0x7f5e78103f00 2024-03-18T23:00:42.217 INFO:teuthology.orchestra.run.smithi139.stderr:2024-03-18T23:00:42.214+0000 7f5e7ff84700 1 -- 172.21.15.139:0/4019005961 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e78068fd0 con 0x7f5e78103f00 2024-03-18T23:00:42.217 INFO:teuthology.orchestra.run.smithi139.stderr:2024-03-18T23:00:42.216+0000 7f5e6effd700 1 -- 172.21.15.139:0/4019005961 <== mon.2 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 15) ==== 137234+0+0 (secure 0 0 0) 0x7f5e7000e5e0 con 0x7f5e78103f00 2024-03-18T23:00:42.218 INFO:teuthology.orchestra.run.smithi139.stderr:2024-03-18T23:00:42.216+0000 7f5e6effd700 1 --2- 172.21.15.139:0/4019005961 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f5e640a2800 0x7f5e640a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:00:42.218 INFO:teuthology.orchestra.run.smithi139.stderr:2024-03-18T23:00:42.216+0000 7f5e6effd700 1 -- 172.21.15.139:0/4019005961 <== mon.2 v2:172.21.15.139:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1767+0+0 (secure 0 0 0) 0x7f5e700ccd60 con 0x7f5e78103f00 2024-03-18T23:00:42.218 INFO:teuthology.orchestra.run.smithi139.stderr:2024-03-18T23:00:42.217+0000 7f5e7dd20700 1 --2- 172.21.15.139:0/4019005961 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f5e640a2800 0x7f5e640a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:00:42.218 INFO:teuthology.orchestra.run.smithi139.stderr:2024-03-18T23:00:42.217+0000 7f5e7dd20700 1 --2- 172.21.15.139:0/4019005961 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f5e640a2800 0x7f5e640a4cc0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f5e7400b580 tx=0x7f5e74005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:00:42.220 INFO:teuthology.orchestra.run.smithi139.stderr:2024-03-18T23:00:42.220+0000 7f5e6effd700 1 -- 172.21.15.139:0/4019005961 <== mon.2 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f5e70013030 con 0x7f5e78103f00 2024-03-18T23:00:42.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:42 smithi139 ceph-mon[67282]: Detected new or changed devices on smithi049 2024-03-18T23:00:42.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:42 smithi139 ceph-mon[67282]: Adjusting osd_memory_target on smithi049 to 17088M 2024-03-18T23:00:42.424 INFO:teuthology.orchestra.run.smithi139.stderr:2024-03-18T23:00:42.424+0000 7f5e7ff84700 1 -- 172.21.15.139:0/4019005961 --> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi139:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f5e78062750 con 0x7f5e640a2800 2024-03-18T23:00:42.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:42 smithi175 ceph-mon[67352]: Detected new or changed devices on smithi049 2024-03-18T23:00:42.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:42 smithi175 ceph-mon[67352]: Adjusting osd_memory_target on smithi049 to 17088M 2024-03-18T23:00:42.630 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:42 smithi049 ceph-mon[62991]: Detected new or changed devices on smithi049 2024-03-18T23:00:42.630 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:42 smithi049 ceph-mon[62991]: Adjusting osd_memory_target on smithi049 to 17088M 2024-03-18T23:00:43.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:43 smithi139 ceph-mon[67282]: pgmap v49: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-03-18T23:00:43.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:43 smithi139 ceph-mon[67282]: osdmap e9: 1 total, 1 up, 1 in 2024-03-18T23:00:43.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:43 smithi139 ceph-mon[67282]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi139:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T23:00:43.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:43 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-18T23:00:43.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:43 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-18T23:00:43.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:43 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:00:43.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:43 smithi175 ceph-mon[67352]: pgmap v49: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-03-18T23:00:43.462 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:43 smithi175 ceph-mon[67352]: osdmap e9: 1 total, 1 up, 1 in 2024-03-18T23:00:43.462 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:43 smithi175 ceph-mon[67352]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi139:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T23:00:43.462 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:43 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-18T23:00:43.462 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:43 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-18T23:00:43.462 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:43 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:00:43.630 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:43 smithi049 ceph-mon[62991]: pgmap v49: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-03-18T23:00:43.630 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:43 smithi049 ceph-mon[62991]: osdmap e9: 1 total, 1 up, 1 in 2024-03-18T23:00:43.630 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:43 smithi049 ceph-mon[62991]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi139:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T23:00:43.630 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:43 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-18T23:00:43.630 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:43 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-18T23:00:43.630 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:43 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:00:45.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:45 smithi175 ceph-mon[67352]: pgmap v51: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-03-18T23:00:45.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:45 smithi175 ceph-mon[67352]: from='client.? 172.21.15.139:0/4081447961' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "40ac929c-e90f-434c-a796-9aabd686f1fa"}]: dispatch 2024-03-18T23:00:45.462 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:45 smithi175 ceph-mon[67352]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "40ac929c-e90f-434c-a796-9aabd686f1fa"}]: dispatch 2024-03-18T23:00:45.462 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:45 smithi175 ceph-mon[67352]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "40ac929c-e90f-434c-a796-9aabd686f1fa"}]': finished 2024-03-18T23:00:45.462 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:45 smithi175 ceph-mon[67352]: osdmap e10: 2 total, 1 up, 2 in 2024-03-18T23:00:45.462 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:45 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-18T23:00:45.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:45 smithi139 ceph-mon[67282]: pgmap v51: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-03-18T23:00:45.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:45 smithi139 ceph-mon[67282]: from='client.? 172.21.15.139:0/4081447961' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "40ac929c-e90f-434c-a796-9aabd686f1fa"}]: dispatch 2024-03-18T23:00:45.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:45 smithi139 ceph-mon[67282]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "40ac929c-e90f-434c-a796-9aabd686f1fa"}]: dispatch 2024-03-18T23:00:45.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:45 smithi139 ceph-mon[67282]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "40ac929c-e90f-434c-a796-9aabd686f1fa"}]': finished 2024-03-18T23:00:45.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:45 smithi139 ceph-mon[67282]: osdmap e10: 2 total, 1 up, 2 in 2024-03-18T23:00:45.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:45 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-18T23:00:45.630 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:45 smithi049 ceph-mon[62991]: pgmap v51: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-03-18T23:00:45.630 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:45 smithi049 ceph-mon[62991]: from='client.? 172.21.15.139:0/4081447961' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "40ac929c-e90f-434c-a796-9aabd686f1fa"}]: dispatch 2024-03-18T23:00:45.630 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:45 smithi049 ceph-mon[62991]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "40ac929c-e90f-434c-a796-9aabd686f1fa"}]: dispatch 2024-03-18T23:00:45.630 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:45 smithi049 ceph-mon[62991]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "40ac929c-e90f-434c-a796-9aabd686f1fa"}]': finished 2024-03-18T23:00:45.630 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:45 smithi049 ceph-mon[62991]: osdmap e10: 2 total, 1 up, 2 in 2024-03-18T23:00:45.630 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:45 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-18T23:00:46.462 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:46 smithi175 ceph-mon[67352]: from='client.? 172.21.15.139:0/2225941484' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-18T23:00:46.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:46 smithi139 ceph-mon[67282]: from='client.? 172.21.15.139:0/2225941484' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-18T23:00:46.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:46 smithi049 ceph-mon[62991]: from='client.? 172.21.15.139:0/2225941484' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-18T23:00:47.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:47 smithi175 ceph-mon[67352]: pgmap v53: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-03-18T23:00:47.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:47 smithi139 ceph-mon[67282]: pgmap v53: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-03-18T23:00:47.629 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:47 smithi049 ceph-mon[62991]: pgmap v53: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-03-18T23:00:49.238 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:49 smithi139 ceph-mon[67282]: pgmap v54: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-03-18T23:00:49.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:49 smithi175 ceph-mon[67352]: pgmap v54: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-03-18T23:00:49.630 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:49 smithi049 ceph-mon[62991]: pgmap v54: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-03-18T23:00:51.442 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:51 smithi139 ceph-mon[67282]: pgmap v55: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-03-18T23:00:51.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:51 smithi175 ceph-mon[67352]: pgmap v55: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-03-18T23:00:51.630 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:51 smithi049 ceph-mon[62991]: pgmap v55: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-03-18T23:00:52.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:52 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-18T23:00:52.462 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:52 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:00:52.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:52 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-18T23:00:52.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:52 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:00:52.630 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:52 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-18T23:00:52.630 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:52 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:00:53.305 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:53 smithi139 ceph-mon[67282]: pgmap v56: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-03-18T23:00:53.306 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:53 smithi139 ceph-mon[67282]: Deploying daemon osd.1 on smithi139 2024-03-18T23:00:53.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:53 smithi175 ceph-mon[67352]: pgmap v56: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-03-18T23:00:53.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:53 smithi175 ceph-mon[67352]: Deploying daemon osd.1 on smithi139 2024-03-18T23:00:53.531 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:53 smithi049 ceph-mon[62991]: pgmap v56: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-03-18T23:00:53.531 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:53 smithi049 ceph-mon[62991]: Deploying daemon osd.1 on smithi139 2024-03-18T23:00:55.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:55 smithi175 ceph-mon[67352]: pgmap v57: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-03-18T23:00:55.499 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:55 smithi139 ceph-mon[67282]: pgmap v57: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-03-18T23:00:55.630 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:55 smithi049 ceph-mon[62991]: pgmap v57: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-03-18T23:00:57.465 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:57 smithi175 ceph-mon[67352]: pgmap v58: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-03-18T23:00:57.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:57 smithi139 ceph-mon[67282]: pgmap v58: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-03-18T23:00:57.630 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:57 smithi049 ceph-mon[62991]: pgmap v58: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-03-18T23:00:58.630 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:58 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:00:58.630 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:58 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:00:58.630 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:58 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:00:58.692 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:58 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:00:58.692 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:58 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:00:58.692 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:58 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:00:58.711 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:58 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:00:58.711 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:58 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:00:58.711 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:58 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:00:59.406 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:00:59 smithi139 ceph-mon[67282]: pgmap v59: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-03-18T23:00:59.630 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:00:59 smithi049 ceph-mon[62991]: pgmap v59: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-03-18T23:00:59.711 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:00:59 smithi175 ceph-mon[67352]: pgmap v59: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-03-18T23:01:00.896 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:01:00 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:01:00.896 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:01:00 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:01:00.896 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:01:00 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:01:00.896 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:01:00 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:01:00.896 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:01:00 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:01:00.896 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:01:00 smithi139 ceph-mon[67282]: pgmap v60: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-03-18T23:01:00.940 INFO:teuthology.orchestra.run.smithi139.stderr:2024-03-18T23:01:00.939+0000 7f5e6effd700 1 -- 172.21.15.139:0/4019005961 <== mgr.14150 v2:172.21.15.49:6800/2083805211 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f5e78062750 con 0x7f5e640a2800 2024-03-18T23:01:00.941 INFO:teuthology.orchestra.run.smithi139.stdout:Created osd(s) 1 on host 'smithi139' 2024-03-18T23:01:00.943 INFO:teuthology.orchestra.run.smithi139.stderr:2024-03-18T23:01:00.942+0000 7f5e7ff84700 1 -- 172.21.15.139:0/4019005961 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f5e640a2800 msgr2=0x7f5e640a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:01:00.943 INFO:teuthology.orchestra.run.smithi139.stderr:2024-03-18T23:01:00.942+0000 7f5e7ff84700 1 --2- 172.21.15.139:0/4019005961 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f5e640a2800 0x7f5e640a4cc0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f5e7400b580 tx=0x7f5e74005fb0 comp rx=0 tx=0).stop 2024-03-18T23:01:00.943 INFO:teuthology.orchestra.run.smithi139.stderr:2024-03-18T23:01:00.942+0000 7f5e7ff84700 1 -- 172.21.15.139:0/4019005961 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5e78103f00 msgr2=0x7f5e78074c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:01:00.943 INFO:teuthology.orchestra.run.smithi139.stderr:2024-03-18T23:01:00.942+0000 7f5e7ff84700 1 --2- 172.21.15.139:0/4019005961 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5e78103f00 0x7f5e78074c40 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f5e7000b970 tx=0x7f5e7000bc80 comp rx=0 tx=0).stop 2024-03-18T23:01:00.943 INFO:teuthology.orchestra.run.smithi139.stderr:2024-03-18T23:01:00.942+0000 7f5e7ff84700 1 -- 172.21.15.139:0/4019005961 shutdown_connections 2024-03-18T23:01:00.944 INFO:teuthology.orchestra.run.smithi139.stderr:2024-03-18T23:01:00.942+0000 7f5e7ff84700 1 --2- 172.21.15.139:0/4019005961 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5e78103f00 0x7f5e78074c40 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:01:00.944 INFO:teuthology.orchestra.run.smithi139.stderr:2024-03-18T23:01:00.942+0000 7f5e7ff84700 1 --2- 172.21.15.139:0/4019005961 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5e78102360 0x7f5e780760b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:01:00.944 INFO:teuthology.orchestra.run.smithi139.stderr:2024-03-18T23:01:00.942+0000 7f5e7ff84700 1 --2- 172.21.15.139:0/4019005961 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f5e640a2800 0x7f5e640a4cc0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:01:00.944 INFO:teuthology.orchestra.run.smithi139.stderr:2024-03-18T23:01:00.942+0000 7f5e7ff84700 1 --2- 172.21.15.139:0/4019005961 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5e78103560 0x7f5e78074700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:01:00.944 INFO:teuthology.orchestra.run.smithi139.stderr:2024-03-18T23:01:00.942+0000 7f5e7ff84700 1 -- 172.21.15.139:0/4019005961 >> 172.21.15.139:0/4019005961 conn(0x7f5e780fd920 msgr2=0x7f5e781068d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:01:00.944 INFO:teuthology.orchestra.run.smithi139.stderr:2024-03-18T23:01:00.943+0000 7f5e7ff84700 1 -- 172.21.15.139:0/4019005961 shutdown_connections 2024-03-18T23:01:00.944 INFO:teuthology.orchestra.run.smithi139.stderr:2024-03-18T23:01:00.943+0000 7f5e7ff84700 1 -- 172.21.15.139:0/4019005961 wait complete. 2024-03-18T23:01:01.130 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:01:00 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:01:01.130 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:01:00 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:01:01.130 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:01:00 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:01:01.130 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:01:00 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:01:01.130 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:01:00 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:01:01.131 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:01:00 smithi049 ceph-mon[62991]: pgmap v60: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-03-18T23:01:01.211 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:01:00 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:01:01.212 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:01:00 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:01:01.212 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:01:00 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:01:01.212 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:01:00 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:01:01.212 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:01:00 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:01:01.212 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:01:00 smithi175 ceph-mon[67352]: pgmap v60: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-03-18T23:01:01.542 DEBUG:teuthology.orchestra.run.smithi139:osd.1> sudo journalctl -f -n 0 -u ceph-c63591ea-e57a-11ee-95c9-87774f69a715@osd.1.service 2024-03-18T23:01:01.545 INFO:tasks.cephadm:Deploying osd.2 on smithi175 with /dev/vg_nvme/lv_4... 2024-03-18T23:01:01.545 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c63591ea-e57a-11ee-95c9-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-03-18T23:01:01.787 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/mon.c/config 2024-03-18T23:01:02.065 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:01:01 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:01:02.066 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:01:01 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:01:02.066 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:01:01 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:01:02.066 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:01:01 smithi139 ceph-mon[67282]: from='osd.1 [v2:172.21.15.139:6800/1766700359,v1:172.21.15.139:6801/1766700359]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-18T23:01:02.066 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:01:01 smithi139 ceph-mon[67282]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-18T23:01:02.130 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:01:01 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:01:02.130 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:01:01 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:01:02.130 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:01:01 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:01:02.130 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:01:01 smithi049 ceph-mon[62991]: from='osd.1 [v2:172.21.15.139:6800/1766700359,v1:172.21.15.139:6801/1766700359]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-18T23:01:02.130 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:01:01 smithi049 ceph-mon[62991]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-18T23:01:02.208 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:01:01 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:01:02.209 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:01:01 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:01:02.209 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:01:01 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:01:02.209 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:01:01 smithi175 ceph-mon[67352]: from='osd.1 [v2:172.21.15.139:6800/1766700359,v1:172.21.15.139:6801/1766700359]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-18T23:01:02.209 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:01:01 smithi175 ceph-mon[67352]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-18T23:01:03.201 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:01:02 smithi139 ceph-mon[67282]: pgmap v61: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-03-18T23:01:03.201 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:01:02 smithi139 ceph-mon[67282]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-03-18T23:01:03.201 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:01:02 smithi139 ceph-mon[67282]: osdmap e11: 2 total, 1 up, 2 in 2024-03-18T23:01:03.201 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:01:02 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-18T23:01:03.201 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:01:02 smithi139 ceph-mon[67282]: from='osd.1 [v2:172.21.15.139:6800/1766700359,v1:172.21.15.139:6801/1766700359]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]: dispatch 2024-03-18T23:01:03.201 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:01:02 smithi139 ceph-mon[67282]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]: dispatch 2024-03-18T23:01:03.206 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:01:02 smithi175 ceph-mon[67352]: pgmap v61: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-03-18T23:01:03.206 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:01:02 smithi175 ceph-mon[67352]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-03-18T23:01:03.206 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:01:02 smithi175 ceph-mon[67352]: osdmap e11: 2 total, 1 up, 2 in 2024-03-18T23:01:03.206 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:01:02 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-18T23:01:03.207 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:01:02 smithi175 ceph-mon[67352]: from='osd.1 [v2:172.21.15.139:6800/1766700359,v1:172.21.15.139:6801/1766700359]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]: dispatch 2024-03-18T23:01:03.207 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:01:02 smithi175 ceph-mon[67352]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]: dispatch 2024-03-18T23:01:03.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:01:02 smithi049 ceph-mon[62991]: pgmap v61: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-03-18T23:01:03.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:01:02 smithi049 ceph-mon[62991]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-03-18T23:01:03.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:01:02 smithi049 ceph-mon[62991]: osdmap e11: 2 total, 1 up, 2 in 2024-03-18T23:01:03.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:01:02 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-18T23:01:03.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:01:02 smithi049 ceph-mon[62991]: from='osd.1 [v2:172.21.15.139:6800/1766700359,v1:172.21.15.139:6801/1766700359]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]: dispatch 2024-03-18T23:01:03.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:01:02 smithi049 ceph-mon[62991]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]: dispatch 2024-03-18T23:01:03.565 INFO:journalctl@ceph.osd.1.smithi139.stdout:Mar 18 23:01:03 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-1[71874]: 2024-03-18T23:01:03.352+0000 7fbfab0cd700 -1 osd.1 0 waiting for initial osdmap 2024-03-18T23:01:03.565 INFO:journalctl@ceph.osd.1.smithi139.stdout:Mar 18 23:01:03 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-1[71874]: 2024-03-18T23:01:03.356+0000 7fbfa26b7700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-18T23:01:03.641 INFO:teuthology.orchestra.run.smithi175.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-03-18T23:01:03.641 INFO:teuthology.orchestra.run.smithi175.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-03-18T23:01:03.641 INFO:teuthology.orchestra.run.smithi175.stderr: stderr: 10+0 records in 2024-03-18T23:01:03.641 INFO:teuthology.orchestra.run.smithi175.stderr:10+0 records out 2024-03-18T23:01:03.641 INFO:teuthology.orchestra.run.smithi175.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0116451 s, 900 MB/s 2024-03-18T23:01:03.641 INFO:teuthology.orchestra.run.smithi175.stderr:--> Zapping successful for: 2024-03-18T23:01:04.145 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:01:03 smithi139 ceph-mon[67282]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]': finished 2024-03-18T23:01:04.146 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:01:03 smithi139 ceph-mon[67282]: osdmap e12: 2 total, 1 up, 2 in 2024-03-18T23:01:04.146 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:01:03 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-18T23:01:04.146 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:01:03 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-18T23:01:04.146 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:01:03 smithi139 ceph-mon[67282]: from='osd.1 ' entity='osd.1' 2024-03-18T23:01:04.201 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:01:03 smithi175 ceph-mon[67352]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]': finished 2024-03-18T23:01:04.201 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:01:03 smithi175 ceph-mon[67352]: osdmap e12: 2 total, 1 up, 2 in 2024-03-18T23:01:04.201 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:01:03 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-18T23:01:04.201 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:01:03 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-18T23:01:04.201 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:01:03 smithi175 ceph-mon[67352]: from='osd.1 ' entity='osd.1' 2024-03-18T23:01:04.231 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c63591ea-e57a-11ee-95c9-87774f69a715 -- ceph orch daemon add osd smithi175:vg_nvme/lv_4 2024-03-18T23:01:04.278 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:01:03 smithi049 ceph-mon[62991]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]': finished 2024-03-18T23:01:04.278 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:01:03 smithi049 ceph-mon[62991]: osdmap e12: 2 total, 1 up, 2 in 2024-03-18T23:01:04.278 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:01:03 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-18T23:01:04.278 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:01:03 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-18T23:01:04.279 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:01:03 smithi049 ceph-mon[62991]: from='osd.1 ' entity='osd.1' 2024-03-18T23:01:04.472 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/mon.c/config 2024-03-18T23:01:05.130 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:01:04 smithi049 ceph-mon[62991]: purged_snaps scrub starts 2024-03-18T23:01:05.130 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:01:04 smithi049 ceph-mon[62991]: purged_snaps scrub ok 2024-03-18T23:01:05.130 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:01:04 smithi049 ceph-mon[62991]: pgmap v64: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-03-18T23:01:05.130 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:01:04 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-18T23:01:05.130 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:01:04 smithi049 ceph-mon[62991]: osd.1 [v2:172.21.15.139:6800/1766700359,v1:172.21.15.139:6801/1766700359] boot 2024-03-18T23:01:05.130 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:01:04 smithi049 ceph-mon[62991]: osdmap e13: 2 total, 2 up, 2 in 2024-03-18T23:01:05.131 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:01:04 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-18T23:01:05.131 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:01:04 smithi049 ceph-mon[62991]: Detected new or changed devices on smithi139 2024-03-18T23:01:05.131 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:01:04 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:01:05.131 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:01:04 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:01:05.131 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:01:04 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:01:05.131 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:01:04 smithi049 ceph-mon[62991]: Adjusting osd_memory_target on smithi139 to 17080M 2024-03-18T23:01:05.131 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:01:04 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:01:05.131 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:01:04 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:01:05.131 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:01:04 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:01:05.131 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:01:04 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:01:05.211 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:01:04 smithi175 ceph-mon[67352]: purged_snaps scrub starts 2024-03-18T23:01:05.212 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:01:04 smithi175 ceph-mon[67352]: purged_snaps scrub ok 2024-03-18T23:01:05.212 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:01:04 smithi175 ceph-mon[67352]: pgmap v64: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-03-18T23:01:05.212 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:01:04 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-18T23:01:05.212 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:01:04 smithi175 ceph-mon[67352]: osd.1 [v2:172.21.15.139:6800/1766700359,v1:172.21.15.139:6801/1766700359] boot 2024-03-18T23:01:05.212 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:01:04 smithi175 ceph-mon[67352]: osdmap e13: 2 total, 2 up, 2 in 2024-03-18T23:01:05.212 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:01:04 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-18T23:01:05.212 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:01:04 smithi175 ceph-mon[67352]: Detected new or changed devices on smithi139 2024-03-18T23:01:05.212 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:01:04 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:01:05.212 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:01:04 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:01:05.212 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:01:04 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:01:05.212 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:01:04 smithi175 ceph-mon[67352]: Adjusting osd_memory_target on smithi139 to 17080M 2024-03-18T23:01:05.212 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:01:04 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:01:05.212 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:01:04 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:01:05.212 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:01:04 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:01:05.213 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:01:04 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:01:05.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:01:04 smithi139 ceph-mon[67282]: purged_snaps scrub starts 2024-03-18T23:01:05.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:01:04 smithi139 ceph-mon[67282]: purged_snaps scrub ok 2024-03-18T23:01:05.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:01:04 smithi139 ceph-mon[67282]: pgmap v64: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-03-18T23:01:05.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:01:04 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-18T23:01:05.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:01:04 smithi139 ceph-mon[67282]: osd.1 [v2:172.21.15.139:6800/1766700359,v1:172.21.15.139:6801/1766700359] boot 2024-03-18T23:01:05.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:01:04 smithi139 ceph-mon[67282]: osdmap e13: 2 total, 2 up, 2 in 2024-03-18T23:01:05.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:01:04 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-18T23:01:05.316 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:01:04 smithi139 ceph-mon[67282]: Detected new or changed devices on smithi139 2024-03-18T23:01:05.316 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:01:04 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:01:05.316 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:01:04 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:01:05.316 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:01:04 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:01:05.316 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:01:04 smithi139 ceph-mon[67282]: Adjusting osd_memory_target on smithi139 to 17080M 2024-03-18T23:01:05.316 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:01:04 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:01:05.316 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:01:04 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:01:05.316 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:01:04 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:01:05.316 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:01:04 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:01:06.041 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:01:06.038+0000 7f44cbdbf700 1 -- 172.21.15.175:0/81321678 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f44c4103560 msgr2=0x7f44c41039c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:01:06.041 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:01:06.038+0000 7f44cbdbf700 1 --2- 172.21.15.175:0/81321678 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f44c4103560 0x7f44c41039c0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f44b4009990 tx=0x7f44b4009ca0 comp rx=0 tx=0).stop 2024-03-18T23:01:06.041 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:01:06.038+0000 7f44cbdbf700 1 -- 172.21.15.175:0/81321678 shutdown_connections 2024-03-18T23:01:06.042 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:01:06.038+0000 7f44cbdbf700 1 --2- 172.21.15.175:0/81321678 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f44c4103f00 0x7f44c410a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:01:06.042 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:01:06.038+0000 7f44cbdbf700 1 --2- 172.21.15.175:0/81321678 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f44c4102360 0x7f44c4102780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:01:06.042 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:01:06.038+0000 7f44cbdbf700 1 --2- 172.21.15.175:0/81321678 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f44c4103560 0x7f44c41039c0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:01:06.042 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:01:06.038+0000 7f44cbdbf700 1 -- 172.21.15.175:0/81321678 >> 172.21.15.175:0/81321678 conn(0x7f44c40fd920 msgr2=0x7f44c40ffd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:01:06.042 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:01:06.039+0000 7f44cbdbf700 1 -- 172.21.15.175:0/81321678 shutdown_connections 2024-03-18T23:01:06.042 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:01:06.039+0000 7f44cbdbf700 1 -- 172.21.15.175:0/81321678 wait complete. 2024-03-18T23:01:06.042 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:01:06.039+0000 7f44cbdbf700 1 Processor -- start 2024-03-18T23:01:06.042 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:01:06.039+0000 7f44cbdbf700 1 -- start start 2024-03-18T23:01:06.042 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:01:06.039+0000 7f44cbdbf700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f44c4102360 0x7f44c40760b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:01:06.042 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:01:06.039+0000 7f44cbdbf700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f44c4103560 0x7f44c4074700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:01:06.042 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:01:06.039+0000 7f44cbdbf700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f44c4103f00 0x7f44c4074c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:01:06.042 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:01:06.039+0000 7f44cbdbf700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f44c410e030 con 0x7f44c4103560 2024-03-18T23:01:06.042 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:01:06.039+0000 7f44cbdbf700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f44c410deb0 con 0x7f44c4103f00 2024-03-18T23:01:06.043 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:01:06.039+0000 7f44cbdbf700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f44c410e1b0 con 0x7f44c4102360 2024-03-18T23:01:06.043 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:01:06.039+0000 7f44c9b5b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f44c4102360 0x7f44c40760b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:01:06.043 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:01:06.039+0000 7f44c9b5b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f44c4102360 0x7f44c40760b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:54520/0 (socket says 172.21.15.175:54520) 2024-03-18T23:01:06.043 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:01:06.039+0000 7f44c9b5b700 1 -- 172.21.15.175:0/2567145134 learned_addr learned my addr 172.21.15.175:0/2567145134 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-03-18T23:01:06.043 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:01:06.040+0000 7f44ca35c700 1 --2- 172.21.15.175:0/2567145134 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f44c4103f00 0x7f44c4074c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:01:06.043 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:01:06.040+0000 7f44c935a700 1 --2- 172.21.15.175:0/2567145134 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f44c4103560 0x7f44c4074700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:01:06.043 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:01:06.040+0000 7f44c9b5b700 1 -- 172.21.15.175:0/2567145134 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f44c4103f00 msgr2=0x7f44c4074c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:01:06.043 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:01:06.040+0000 7f44c9b5b700 1 --2- 172.21.15.175:0/2567145134 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f44c4103f00 0x7f44c4074c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:01:06.043 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:01:06.040+0000 7f44c9b5b700 1 -- 172.21.15.175:0/2567145134 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f44c4103560 msgr2=0x7f44c4074700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:01:06.043 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:01:06.040+0000 7f44c9b5b700 1 --2- 172.21.15.175:0/2567145134 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f44c4103560 0x7f44c4074700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:01:06.043 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:01:06.040+0000 7f44c9b5b700 1 -- 172.21.15.175:0/2567145134 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44c40754f0 con 0x7f44c4102360 2024-03-18T23:01:06.043 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:01:06.040+0000 7f44c9b5b700 1 --2- 172.21.15.175:0/2567145134 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f44c4102360 0x7f44c40760b0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f44c000b500 tx=0x7f44c000b810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:01:06.043 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:01:06.040+0000 7f44baffd700 1 -- 172.21.15.175:0/2567145134 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44c0015020 con 0x7f44c4102360 2024-03-18T23:01:06.043 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:01:06.040+0000 7f44baffd700 1 -- 172.21.15.175:0/2567145134 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f44c0011d10 con 0x7f44c4102360 2024-03-18T23:01:06.044 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:01:06.040+0000 7f44cbdbf700 1 -- 172.21.15.175:0/2567145134 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44c41a27a0 con 0x7f44c4102360 2024-03-18T23:01:06.044 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:01:06.040+0000 7f44baffd700 1 -- 172.21.15.175:0/2567145134 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44c0010db0 con 0x7f44c4102360 2024-03-18T23:01:06.044 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:01:06.040+0000 7f44cbdbf700 1 -- 172.21.15.175:0/2567145134 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44c41a2d30 con 0x7f44c4102360 2024-03-18T23:01:06.045 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:01:06.041+0000 7f44cbdbf700 1 -- 172.21.15.175:0/2567145134 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44c4068fd0 con 0x7f44c4102360 2024-03-18T23:01:06.045 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:01:06.042+0000 7f44baffd700 1 -- 172.21.15.175:0/2567145134 <== mon.1 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 15) ==== 137234+0+0 (secure 0 0 0) 0x7f44c0018020 con 0x7f44c4102360 2024-03-18T23:01:06.045 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:01:06.043+0000 7f44baffd700 1 --2- 172.21.15.175:0/2567145134 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f44b00a2800 0x7f44b00a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:01:06.045 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:01:06.043+0000 7f44baffd700 1 -- 172.21.15.175:0/2567145134 <== mon.1 v2:172.21.15.175:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 2309+0+0 (secure 0 0 0) 0x7f44c00ccec0 con 0x7f44c4102360 2024-03-18T23:01:06.045 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:01:06.043+0000 7f44c935a700 1 --2- 172.21.15.175:0/2567145134 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f44b00a2800 0x7f44b00a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:01:06.045 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:01:06.043+0000 7f44c935a700 1 --2- 172.21.15.175:0/2567145134 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f44b00a2800 0x7f44b00a4cc0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f44b400b5c0 tx=0x7f44b4005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:01:06.047 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:01:06.047+0000 7f44baffd700 1 -- 172.21.15.175:0/2567145134 <== mon.1 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f44c001f020 con 0x7f44c4102360 2024-03-18T23:01:06.255 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:01:06.254+0000 7f44cbdbf700 1 -- 172.21.15.175:0/2567145134 --> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi175:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f44c4062750 con 0x7f44b00a2800 2024-03-18T23:01:07.114 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:01:06 smithi175 ceph-mon[67352]: pgmap v66: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-03-18T23:01:07.114 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:01:06 smithi175 ceph-mon[67352]: osdmap e14: 2 total, 2 up, 2 in 2024-03-18T23:01:07.114 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:01:06 smithi175 ceph-mon[67352]: from='client.24154 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi175:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T23:01:07.114 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:01:06 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-18T23:01:07.114 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:01:06 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-18T23:01:07.114 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:01:06 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:01:07.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:01:06 smithi139 ceph-mon[67282]: pgmap v66: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-03-18T23:01:07.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:01:06 smithi139 ceph-mon[67282]: osdmap e14: 2 total, 2 up, 2 in 2024-03-18T23:01:07.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:01:06 smithi139 ceph-mon[67282]: from='client.24154 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi175:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T23:01:07.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:01:06 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-18T23:01:07.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:01:06 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-18T23:01:07.316 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:01:06 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:01:07.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:01:06 smithi049 ceph-mon[62991]: pgmap v66: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-03-18T23:01:07.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:01:06 smithi049 ceph-mon[62991]: osdmap e14: 2 total, 2 up, 2 in 2024-03-18T23:01:07.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:01:06 smithi049 ceph-mon[62991]: from='client.24154 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi175:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T23:01:07.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:01:06 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-18T23:01:07.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:01:06 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-18T23:01:07.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:01:06 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:01:09.211 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:01:08 smithi175 ceph-mon[67352]: pgmap v68: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-03-18T23:01:09.212 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:01:08 smithi175 ceph-mon[67352]: from='client.? 172.21.15.175:0/1975447534' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8ae3b666-00b4-45ed-a5ee-4d0771edabb6"}]: dispatch 2024-03-18T23:01:09.212 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:01:08 smithi175 ceph-mon[67352]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8ae3b666-00b4-45ed-a5ee-4d0771edabb6"}]: dispatch 2024-03-18T23:01:09.212 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:01:08 smithi175 ceph-mon[67352]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8ae3b666-00b4-45ed-a5ee-4d0771edabb6"}]': finished 2024-03-18T23:01:09.212 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:01:08 smithi175 ceph-mon[67352]: osdmap e15: 3 total, 2 up, 3 in 2024-03-18T23:01:09.212 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:01:08 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-18T23:01:09.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:01:08 smithi139 ceph-mon[67282]: pgmap v68: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-03-18T23:01:09.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:01:08 smithi139 ceph-mon[67282]: from='client.? 172.21.15.175:0/1975447534' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8ae3b666-00b4-45ed-a5ee-4d0771edabb6"}]: dispatch 2024-03-18T23:01:09.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:01:08 smithi139 ceph-mon[67282]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8ae3b666-00b4-45ed-a5ee-4d0771edabb6"}]: dispatch 2024-03-18T23:01:09.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:01:08 smithi139 ceph-mon[67282]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8ae3b666-00b4-45ed-a5ee-4d0771edabb6"}]': finished 2024-03-18T23:01:09.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:01:08 smithi139 ceph-mon[67282]: osdmap e15: 3 total, 2 up, 3 in 2024-03-18T23:01:09.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:01:08 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-18T23:01:09.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:01:08 smithi049 ceph-mon[62991]: pgmap v68: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-03-18T23:01:09.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:01:08 smithi049 ceph-mon[62991]: from='client.? 172.21.15.175:0/1975447534' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8ae3b666-00b4-45ed-a5ee-4d0771edabb6"}]: dispatch 2024-03-18T23:01:09.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:01:08 smithi049 ceph-mon[62991]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8ae3b666-00b4-45ed-a5ee-4d0771edabb6"}]: dispatch 2024-03-18T23:01:09.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:01:08 smithi049 ceph-mon[62991]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8ae3b666-00b4-45ed-a5ee-4d0771edabb6"}]': finished 2024-03-18T23:01:09.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:01:08 smithi049 ceph-mon[62991]: osdmap e15: 3 total, 2 up, 3 in 2024-03-18T23:01:09.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:01:08 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-18T23:01:10.212 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:01:09 smithi175 ceph-mon[67352]: from='client.? 172.21.15.175:0/1058335051' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-18T23:01:10.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:01:09 smithi139 ceph-mon[67282]: from='client.? 172.21.15.175:0/1058335051' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-18T23:01:10.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:01:09 smithi049 ceph-mon[62991]: from='client.? 172.21.15.175:0/1058335051' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-18T23:01:11.211 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:01:10 smithi175 ceph-mon[67352]: pgmap v70: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-03-18T23:01:11.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:01:10 smithi139 ceph-mon[67282]: pgmap v70: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-03-18T23:01:11.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:01:10 smithi049 ceph-mon[62991]: pgmap v70: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-03-18T23:01:13.211 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:01:12 smithi175 ceph-mon[67352]: pgmap v71: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-03-18T23:01:13.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:01:12 smithi139 ceph-mon[67282]: pgmap v71: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-03-18T23:01:13.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:01:12 smithi049 ceph-mon[62991]: pgmap v71: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-03-18T23:01:15.296 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:01:14 smithi175 ceph-mon[67352]: pgmap v72: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-03-18T23:01:15.301 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:01:14 smithi139 ceph-mon[67282]: pgmap v72: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-03-18T23:01:15.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:01:14 smithi049 ceph-mon[62991]: pgmap v72: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-03-18T23:01:16.284 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:01:15 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-18T23:01:16.285 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:01:15 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:01:16.285 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:01:15 smithi175 ceph-mon[67352]: Deploying daemon osd.2 on smithi175 2024-03-18T23:01:16.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:01:15 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-18T23:01:16.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:01:15 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:01:16.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:01:15 smithi139 ceph-mon[67282]: Deploying daemon osd.2 on smithi175 2024-03-18T23:01:16.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:01:15 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-18T23:01:16.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:01:15 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:01:16.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:01:15 smithi049 ceph-mon[62991]: Deploying daemon osd.2 on smithi175 2024-03-18T23:01:17.289 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:01:16 smithi175 ceph-mon[67352]: pgmap v73: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-03-18T23:01:17.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:01:16 smithi139 ceph-mon[67282]: pgmap v73: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-03-18T23:01:17.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:01:16 smithi049 ceph-mon[62991]: pgmap v73: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-03-18T23:01:19.155 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:01:18 smithi175 ceph-mon[67352]: pgmap v74: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-03-18T23:01:19.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:01:18 smithi139 ceph-mon[67282]: pgmap v74: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-03-18T23:01:19.379 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:01:18 smithi049 ceph-mon[62991]: pgmap v74: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-03-18T23:01:21.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:01:20 smithi139 ceph-mon[67282]: pgmap v75: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-03-18T23:01:21.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:01:20 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:01:21.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:01:20 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:01:21.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:01:20 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:01:21.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:01:20 smithi049 ceph-mon[62991]: pgmap v75: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-03-18T23:01:21.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:01:20 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:01:21.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:01:20 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:01:21.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:01:20 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:01:21.388 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:01:20 smithi175 ceph-mon[67352]: pgmap v75: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-03-18T23:01:21.388 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:01:20 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:01:21.388 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:01:20 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:01:21.389 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:01:20 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:01:23.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:01:22 smithi139 ceph-mon[67282]: pgmap v76: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-03-18T23:01:23.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:01:22 smithi049 ceph-mon[62991]: pgmap v76: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-03-18T23:01:23.462 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:01:22 smithi175 ceph-mon[67352]: pgmap v76: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-03-18T23:01:24.124 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:01:24.123+0000 7f44baffd700 1 -- 172.21.15.175:0/2567145134 <== mgr.14150 v2:172.21.15.49:6800/2083805211 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f44c4062750 con 0x7f44b00a2800 2024-03-18T23:01:24.124 INFO:teuthology.orchestra.run.smithi175.stdout:Created osd(s) 2 on host 'smithi175' 2024-03-18T23:01:24.125 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:01:24.125+0000 7f44cbdbf700 1 -- 172.21.15.175:0/2567145134 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f44b00a2800 msgr2=0x7f44b00a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:01:24.125 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:01:24.125+0000 7f44cbdbf700 1 --2- 172.21.15.175:0/2567145134 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f44b00a2800 0x7f44b00a4cc0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f44b400b5c0 tx=0x7f44b4005fb0 comp rx=0 tx=0).stop 2024-03-18T23:01:24.126 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:01:24.126+0000 7f44cbdbf700 1 -- 172.21.15.175:0/2567145134 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f44c4102360 msgr2=0x7f44c40760b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:01:24.126 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:01:24.126+0000 7f44cbdbf700 1 --2- 172.21.15.175:0/2567145134 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f44c4102360 0x7f44c40760b0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f44c000b500 tx=0x7f44c000b810 comp rx=0 tx=0).stop 2024-03-18T23:01:24.126 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:01:24.126+0000 7f44cbdbf700 1 -- 172.21.15.175:0/2567145134 shutdown_connections 2024-03-18T23:01:24.126 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:01:24.126+0000 7f44cbdbf700 1 --2- 172.21.15.175:0/2567145134 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f44c4103f00 0x7f44c4074c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:01:24.126 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:01:24.126+0000 7f44cbdbf700 1 --2- 172.21.15.175:0/2567145134 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f44b00a2800 0x7f44b00a4cc0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:01:24.126 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:01:24.126+0000 7f44cbdbf700 1 --2- 172.21.15.175:0/2567145134 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f44c4102360 0x7f44c40760b0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:01:24.126 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:01:24.126+0000 7f44cbdbf700 1 --2- 172.21.15.175:0/2567145134 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f44c4103560 0x7f44c4074700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:01:24.126 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:01:24.126+0000 7f44cbdbf700 1 -- 172.21.15.175:0/2567145134 >> 172.21.15.175:0/2567145134 conn(0x7f44c40fd920 msgr2=0x7f44c4106800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:01:24.126 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:01:24.126+0000 7f44cbdbf700 1 -- 172.21.15.175:0/2567145134 shutdown_connections 2024-03-18T23:01:24.126 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:01:24.126+0000 7f44cbdbf700 1 -- 172.21.15.175:0/2567145134 wait complete. 2024-03-18T23:01:24.686 DEBUG:teuthology.orchestra.run.smithi175:osd.2> sudo journalctl -f -n 0 -u ceph-c63591ea-e57a-11ee-95c9-87774f69a715@osd.2.service 2024-03-18T23:01:24.688 INFO:tasks.cephadm:Deploying osd.3 on smithi049 with /dev/vg_nvme/lv_3... 2024-03-18T23:01:24.689 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c63591ea-e57a-11ee-95c9-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-03-18T23:01:24.949 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T23:01:25.310 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:01:25 smithi175 ceph-mon[67352]: pgmap v77: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-03-18T23:01:25.311 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:01:25 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:01:25.311 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:01:25 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:01:25.311 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:01:25 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:01:25.311 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:01:25 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:01:25.311 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:01:25 smithi175 ceph-mon[67352]: from='osd.2 [v2:172.21.15.175:6800/3752671234,v1:172.21.15.175:6801/3752671234]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-18T23:01:25.311 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:01:25 smithi175 ceph-mon[67352]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-18T23:01:25.373 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:01:25 smithi139 ceph-mon[67282]: pgmap v77: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-03-18T23:01:25.373 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:01:25 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:01:25.374 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:01:25 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:01:25.374 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:01:25 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:01:25.374 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:01:25 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:01:25.374 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:01:25 smithi139 ceph-mon[67282]: from='osd.2 [v2:172.21.15.175:6800/3752671234,v1:172.21.15.175:6801/3752671234]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-18T23:01:25.374 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:01:25 smithi139 ceph-mon[67282]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-18T23:01:25.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:01:25 smithi049 ceph-mon[62991]: pgmap v77: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-03-18T23:01:25.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:01:25 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:01:25.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:01:25 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:01:25.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:01:25 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:01:25.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:01:25 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:01:25.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:01:25 smithi049 ceph-mon[62991]: from='osd.2 [v2:172.21.15.175:6800/3752671234,v1:172.21.15.175:6801/3752671234]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-18T23:01:25.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:01:25 smithi049 ceph-mon[62991]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-18T23:01:26.630 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:01:26 smithi049 ceph-mon[62991]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-03-18T23:01:26.630 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:01:26 smithi049 ceph-mon[62991]: osdmap e16: 3 total, 2 up, 3 in 2024-03-18T23:01:26.630 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:01:26 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-18T23:01:26.630 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:01:26 smithi049 ceph-mon[62991]: from='osd.2 [v2:172.21.15.175:6800/3752671234,v1:172.21.15.175:6801/3752671234]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]: dispatch 2024-03-18T23:01:26.630 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:01:26 smithi049 ceph-mon[62991]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]: dispatch 2024-03-18T23:01:26.711 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:01:26 smithi175 ceph-mon[67352]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-03-18T23:01:26.711 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:01:26 smithi175 ceph-mon[67352]: osdmap e16: 3 total, 2 up, 3 in 2024-03-18T23:01:26.712 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:01:26 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-18T23:01:26.712 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:01:26 smithi175 ceph-mon[67352]: from='osd.2 [v2:172.21.15.175:6800/3752671234,v1:172.21.15.175:6801/3752671234]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]: dispatch 2024-03-18T23:01:26.712 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:01:26 smithi175 ceph-mon[67352]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]: dispatch 2024-03-18T23:01:26.712 INFO:journalctl@ceph.osd.2.smithi175.stdout:Mar 18 23:01:26 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-2[72488]: 2024-03-18T23:01:26.439+0000 7eff61f79700 -1 osd.2 0 waiting for initial osdmap 2024-03-18T23:01:26.712 INFO:journalctl@ceph.osd.2.smithi175.stdout:Mar 18 23:01:26 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-2[72488]: 2024-03-18T23:01:26.442+0000 7eff5dd6c700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-18T23:01:26.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:01:26 smithi139 ceph-mon[67282]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-03-18T23:01:26.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:01:26 smithi139 ceph-mon[67282]: osdmap e16: 3 total, 2 up, 3 in 2024-03-18T23:01:26.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:01:26 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-18T23:01:26.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:01:26 smithi139 ceph-mon[67282]: from='osd.2 [v2:172.21.15.175:6800/3752671234,v1:172.21.15.175:6801/3752671234]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]: dispatch 2024-03-18T23:01:26.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:01:26 smithi139 ceph-mon[67282]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]: dispatch 2024-03-18T23:01:27.139 INFO:teuthology.orchestra.run.smithi049.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-03-18T23:01:27.140 INFO:teuthology.orchestra.run.smithi049.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-03-18T23:01:27.140 INFO:teuthology.orchestra.run.smithi049.stderr: stderr: 10+0 records in 2024-03-18T23:01:27.140 INFO:teuthology.orchestra.run.smithi049.stderr:10+0 records out 2024-03-18T23:01:27.140 INFO:teuthology.orchestra.run.smithi049.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00900736 s, 1.2 GB/s 2024-03-18T23:01:27.140 INFO:teuthology.orchestra.run.smithi049.stderr:--> Zapping successful for: 2024-03-18T23:01:27.688 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:01:27 smithi175 ceph-mon[67352]: pgmap v79: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-03-18T23:01:27.688 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:01:27 smithi175 ceph-mon[67352]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]': finished 2024-03-18T23:01:27.688 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:01:27 smithi175 ceph-mon[67352]: osdmap e17: 3 total, 2 up, 3 in 2024-03-18T23:01:27.688 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:01:27 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-18T23:01:27.689 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:01:27 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-18T23:01:27.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:01:27 smithi139 ceph-mon[67282]: pgmap v79: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-03-18T23:01:27.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:01:27 smithi139 ceph-mon[67282]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]': finished 2024-03-18T23:01:27.816 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:01:27 smithi139 ceph-mon[67282]: osdmap e17: 3 total, 2 up, 3 in 2024-03-18T23:01:27.816 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:01:27 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-18T23:01:27.816 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:01:27 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-18T23:01:27.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:01:27 smithi049 ceph-mon[62991]: pgmap v79: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-03-18T23:01:27.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:01:27 smithi049 ceph-mon[62991]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]': finished 2024-03-18T23:01:27.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:01:27 smithi049 ceph-mon[62991]: osdmap e17: 3 total, 2 up, 3 in 2024-03-18T23:01:27.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:01:27 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-18T23:01:27.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:01:27 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-18T23:01:27.992 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c63591ea-e57a-11ee-95c9-87774f69a715 -- ceph orch daemon add osd smithi049:vg_nvme/lv_3 2024-03-18T23:01:28.275 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T23:01:28.667 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:01:28 smithi049 ceph-mon[62991]: purged_snaps scrub starts 2024-03-18T23:01:28.667 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:01:28 smithi049 ceph-mon[62991]: purged_snaps scrub ok 2024-03-18T23:01:28.667 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:01:28 smithi049 ceph-mon[62991]: osd.2 [v2:172.21.15.175:6800/3752671234,v1:172.21.15.175:6801/3752671234] boot 2024-03-18T23:01:28.667 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:01:28 smithi049 ceph-mon[62991]: osdmap e18: 3 total, 3 up, 3 in 2024-03-18T23:01:28.667 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:01:28 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-18T23:01:28.667 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:01:28 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:01:28.667 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:01:28 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:01:28.667 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:01:28 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:01:28.667 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:01:28 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:01:28.667 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:01:28 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:01:28.667 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:01:28 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:01:28.667 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:01:28 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:01:28.667 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:01:28 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-03-18T23:01:28.668 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:01:28 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:01:28.668 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:01:28 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:01:28.668 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:01:28 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:01:28.668 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:01:28 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:01:28.711 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:01:28 smithi175 ceph-mon[67352]: purged_snaps scrub starts 2024-03-18T23:01:28.711 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:01:28 smithi175 ceph-mon[67352]: purged_snaps scrub ok 2024-03-18T23:01:28.711 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:01:28 smithi175 ceph-mon[67352]: osd.2 [v2:172.21.15.175:6800/3752671234,v1:172.21.15.175:6801/3752671234] boot 2024-03-18T23:01:28.712 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:01:28 smithi175 ceph-mon[67352]: osdmap e18: 3 total, 3 up, 3 in 2024-03-18T23:01:28.712 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:01:28 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-18T23:01:28.712 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:01:28 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:01:28.712 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:01:28 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:01:28.712 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:01:28 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:01:28.712 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:01:28 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:01:28.712 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:01:28 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:01:28.712 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:01:28 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:01:28.712 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:01:28 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:01:28.712 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:01:28 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-03-18T23:01:28.712 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:01:28 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:01:28.712 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:01:28 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:01:28.712 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:01:28 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:01:28.712 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:01:28 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:01:28.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:01:28 smithi139 ceph-mon[67282]: purged_snaps scrub starts 2024-03-18T23:01:28.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:01:28 smithi139 ceph-mon[67282]: purged_snaps scrub ok 2024-03-18T23:01:28.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:01:28 smithi139 ceph-mon[67282]: osd.2 [v2:172.21.15.175:6800/3752671234,v1:172.21.15.175:6801/3752671234] boot 2024-03-18T23:01:28.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:01:28 smithi139 ceph-mon[67282]: osdmap e18: 3 total, 3 up, 3 in 2024-03-18T23:01:28.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:01:28 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-18T23:01:28.816 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:01:28 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:01:28.816 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:01:28 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:01:28.816 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:01:28 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:01:28.816 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:01:28 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:01:28.816 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:01:28 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:01:28.816 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:01:28 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:01:28.816 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:01:28 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:01:28.816 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:01:28 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-03-18T23:01:28.816 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:01:28 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:01:28.816 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:01:28 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:01:28.816 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:01:28 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:01:28.816 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:01:28 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:01:29.711 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:01:29 smithi175 ceph-mon[67352]: Detected new or changed devices on smithi175 2024-03-18T23:01:29.712 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:01:29 smithi175 ceph-mon[67352]: Adjusting osd_memory_target on smithi175 to 17080M 2024-03-18T23:01:29.712 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:01:29 smithi175 ceph-mon[67352]: pgmap v82: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-03-18T23:01:29.712 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:01:29 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-03-18T23:01:29.712 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:01:29 smithi175 ceph-mon[67352]: osdmap e19: 3 total, 3 up, 3 in 2024-03-18T23:01:29.712 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:01:29 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-03-18T23:01:29.757 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:01:29 smithi049 ceph-mon[62991]: Detected new or changed devices on smithi175 2024-03-18T23:01:29.757 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:01:29 smithi049 ceph-mon[62991]: Adjusting osd_memory_target on smithi175 to 17080M 2024-03-18T23:01:29.757 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:01:29 smithi049 ceph-mon[62991]: pgmap v82: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-03-18T23:01:29.757 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:01:29 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-03-18T23:01:29.757 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:01:29 smithi049 ceph-mon[62991]: osdmap e19: 3 total, 3 up, 3 in 2024-03-18T23:01:29.757 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:01:29 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-03-18T23:01:29.772 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:01:29.771+0000 7f6dcf7b7700 1 -- 172.21.15.49:0/3286536849 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6dc81012b0 msgr2=0x7f6dc81016d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:01:29.772 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:01:29.771+0000 7f6dcf7b7700 1 --2- 172.21.15.49:0/3286536849 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6dc81012b0 0x7f6dc81016d0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f6dc40099c0 tx=0x7f6dc4009cd0 comp rx=0 tx=0).stop 2024-03-18T23:01:29.772 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:01:29.772+0000 7f6dcf7b7700 1 -- 172.21.15.49:0/3286536849 shutdown_connections 2024-03-18T23:01:29.772 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:01:29.772+0000 7f6dcf7b7700 1 --2- 172.21.15.49:0/3286536849 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6dc8102e50 0x7f6dc8109700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:01:29.772 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:01:29.772+0000 7f6dcf7b7700 1 --2- 172.21.15.49:0/3286536849 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6dc81012b0 0x7f6dc81016d0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:01:29.772 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:01:29.772+0000 7f6dcf7b7700 1 --2- 172.21.15.49:0/3286536849 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6dc81024b0 0x7f6dc8102910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:01:29.772 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:01:29.772+0000 7f6dcf7b7700 1 -- 172.21.15.49:0/3286536849 >> 172.21.15.49:0/3286536849 conn(0x7f6dc80fc870 msgr2=0x7f6dc80fec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:01:29.772 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:01:29.772+0000 7f6dcf7b7700 1 -- 172.21.15.49:0/3286536849 shutdown_connections 2024-03-18T23:01:29.772 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:01:29.772+0000 7f6dcf7b7700 1 -- 172.21.15.49:0/3286536849 wait complete. 2024-03-18T23:01:29.774 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:01:29.772+0000 7f6dcf7b7700 1 Processor -- start 2024-03-18T23:01:29.774 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:01:29.772+0000 7f6dcf7b7700 1 -- start start 2024-03-18T23:01:29.774 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:01:29.773+0000 7f6dcf7b7700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6dc81012b0 0x7f6dc8105240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:01:29.774 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:01:29.773+0000 7f6dcf7b7700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6dc81024b0 0x7f6dc8103890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:01:29.774 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:01:29.773+0000 7f6dcf7b7700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6dc8102e50 0x7f6dc8103dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:01:29.774 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:01:29.773+0000 7f6dcf7b7700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f6dc810ce00 con 0x7f6dc8102e50 2024-03-18T23:01:29.774 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:01:29.773+0000 7f6dcf7b7700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f6dc810cc80 con 0x7f6dc81024b0 2024-03-18T23:01:29.775 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:01:29.773+0000 7f6dcf7b7700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f6dc810cf80 con 0x7f6dc81012b0 2024-03-18T23:01:29.775 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:01:29.773+0000 7f6dcdd54700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6dc8102e50 0x7f6dc8103dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:01:29.775 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:01:29.773+0000 7f6dcdd54700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6dc8102e50 0x7f6dc8103dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:44490/0 (socket says 172.21.15.49:44490) 2024-03-18T23:01:29.775 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:01:29.773+0000 7f6dcdd54700 1 -- 172.21.15.49:0/3065915804 learned_addr learned my addr 172.21.15.49:0/3065915804 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:01:29.775 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:01:29.773+0000 7f6dccd52700 1 --2- 172.21.15.49:0/3065915804 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6dc81024b0 0x7f6dc8103890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:01:29.775 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:01:29.773+0000 7f6dcd553700 1 --2- 172.21.15.49:0/3065915804 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6dc81012b0 0x7f6dc8105240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:01:29.775 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:01:29.773+0000 7f6dcdd54700 1 -- 172.21.15.49:0/3065915804 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6dc81012b0 msgr2=0x7f6dc8105240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:01:29.775 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:01:29.773+0000 7f6dcdd54700 1 --2- 172.21.15.49:0/3065915804 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6dc81012b0 0x7f6dc8105240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:01:29.775 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:01:29.773+0000 7f6dcdd54700 1 -- 172.21.15.49:0/3065915804 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6dc81024b0 msgr2=0x7f6dc8103890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:01:29.775 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:01:29.773+0000 7f6dcdd54700 1 --2- 172.21.15.49:0/3065915804 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6dc81024b0 0x7f6dc8103890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:01:29.775 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:01:29.773+0000 7f6dcdd54700 1 -- 172.21.15.49:0/3065915804 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6dc8104680 con 0x7f6dc8102e50 2024-03-18T23:01:29.775 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:01:29.773+0000 7f6dcdd54700 1 --2- 172.21.15.49:0/3065915804 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6dc8102e50 0x7f6dc8103dd0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f6dc000ef30 tx=0x7f6dc000c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:01:29.775 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:01:29.773+0000 7f6dbe7fc700 1 -- 172.21.15.49:0/3065915804 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dc00092e0 con 0x7f6dc8102e50 2024-03-18T23:01:29.775 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:01:29.774+0000 7f6dbe7fc700 1 -- 172.21.15.49:0/3065915804 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6dc0004bb0 con 0x7f6dc8102e50 2024-03-18T23:01:29.776 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:01:29.774+0000 7f6dcf7b7700 1 -- 172.21.15.49:0/3065915804 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6dc81a26c0 con 0x7f6dc8102e50 2024-03-18T23:01:29.776 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:01:29.774+0000 7f6dbe7fc700 1 -- 172.21.15.49:0/3065915804 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dc0004280 con 0x7f6dc8102e50 2024-03-18T23:01:29.776 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:01:29.774+0000 7f6dcf7b7700 1 -- 172.21.15.49:0/3065915804 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6dc81a2bd0 con 0x7f6dc8102e50 2024-03-18T23:01:29.776 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:01:29.775+0000 7f6dcf7b7700 1 -- 172.21.15.49:0/3065915804 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6dc8068fd0 con 0x7f6dc8102e50 2024-03-18T23:01:29.776 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:01:29.776+0000 7f6dbe7fc700 1 -- 172.21.15.49:0/3065915804 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 15) ==== 137234+0+0 (secure 0 0 0) 0x7f6dc001b030 con 0x7f6dc8102e50 2024-03-18T23:01:29.777 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:01:29.777+0000 7f6dbe7fc700 1 --2- 172.21.15.49:0/3065915804 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f6db40a2880 0x7f6db40a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:01:29.777 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:01:29.777+0000 7f6dcd553700 1 --2- 172.21.15.49:0/3065915804 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f6db40a2880 0x7f6db40a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:01:29.777 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:01:29.777+0000 7f6dbe7fc700 1 -- 172.21.15.49:0/3065915804 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 3237+0+0 (secure 0 0 0) 0x7f6dc00cdba0 con 0x7f6dc8102e50 2024-03-18T23:01:29.777 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:01:29.778+0000 7f6dcd553700 1 --2- 172.21.15.49:0/3065915804 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f6db40a2880 0x7f6db40a4d40 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f6dc400b580 tx=0x7f6dc4005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:01:29.780 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:01:29.780+0000 7f6dbe7fc700 1 -- 172.21.15.49:0/3065915804 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f6dc00cde20 con 0x7f6dc8102e50 2024-03-18T23:01:29.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:01:29 smithi139 ceph-mon[67282]: Detected new or changed devices on smithi175 2024-03-18T23:01:29.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:01:29 smithi139 ceph-mon[67282]: Adjusting osd_memory_target on smithi175 to 17080M 2024-03-18T23:01:29.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:01:29 smithi139 ceph-mon[67282]: pgmap v82: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-03-18T23:01:29.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:01:29 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-03-18T23:01:29.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:01:29 smithi139 ceph-mon[67282]: osdmap e19: 3 total, 3 up, 3 in 2024-03-18T23:01:29.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:01:29 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-03-18T23:01:29.998 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:01:29.998+0000 7f6dcf7b7700 1 -- 172.21.15.49:0/3065915804 --> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi049:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f6dc80fea00 con 0x7f6db40a2880 2024-03-18T23:01:30.380 INFO:journalctl@ceph.osd.0.smithi049.stdout:Mar 18 23:01:30 smithi049 sudo[82292]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-18T23:01:30.380 INFO:journalctl@ceph.osd.0.smithi049.stdout:Mar 18 23:01:30 smithi049 sudo[82292]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-18T23:01:30.380 INFO:journalctl@ceph.osd.0.smithi049.stdout:Mar 18 23:01:30 smithi049 sudo[82292]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-18T23:01:30.380 INFO:journalctl@ceph.osd.0.smithi049.stdout:Mar 18 23:01:30 smithi049 sudo[82292]: pam_unix(sudo:session): session closed for user root 2024-03-18T23:01:30.715 INFO:journalctl@ceph.osd.0.smithi049.stdout:Mar 18 23:01:30 smithi049 sudo[82389]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-18T23:01:30.715 INFO:journalctl@ceph.osd.0.smithi049.stdout:Mar 18 23:01:30 smithi049 sudo[82389]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-18T23:01:30.715 INFO:journalctl@ceph.osd.0.smithi049.stdout:Mar 18 23:01:30 smithi049 sudo[82389]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-18T23:01:30.715 INFO:journalctl@ceph.osd.0.smithi049.stdout:Mar 18 23:01:30 smithi049 sudo[82389]: pam_unix(sudo:session): session closed for user root 2024-03-18T23:01:30.716 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:01:30 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-03-18T23:01:30.716 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:01:30 smithi049 ceph-mon[62991]: osdmap e20: 3 total, 3 up, 3 in 2024-03-18T23:01:30.716 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:01:30 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-18T23:01:30.716 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:01:30 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-18T23:01:30.716 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:01:30 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:01:30.741 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:01:30 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-03-18T23:01:30.741 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:01:30 smithi139 ceph-mon[67282]: osdmap e20: 3 total, 3 up, 3 in 2024-03-18T23:01:30.741 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:01:30 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-18T23:01:30.741 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:01:30 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-18T23:01:30.742 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:01:30 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:01:30.961 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:01:30 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-03-18T23:01:30.962 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:01:30 smithi175 ceph-mon[67352]: osdmap e20: 3 total, 3 up, 3 in 2024-03-18T23:01:30.962 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:01:30 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-18T23:01:30.962 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:01:30 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-18T23:01:30.962 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:01:30 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:01:30.995 INFO:journalctl@ceph.osd.1.smithi139.stdout:Mar 18 23:01:30 smithi139 sudo[75882]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-18T23:01:30.996 INFO:journalctl@ceph.osd.1.smithi139.stdout:Mar 18 23:01:30 smithi139 sudo[75882]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-18T23:01:30.996 INFO:journalctl@ceph.osd.1.smithi139.stdout:Mar 18 23:01:30 smithi139 sudo[75882]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-18T23:01:30.996 INFO:journalctl@ceph.osd.1.smithi139.stdout:Mar 18 23:01:30 smithi139 sudo[75882]: pam_unix(sudo:session): session closed for user root 2024-03-18T23:01:31.315 INFO:journalctl@ceph.osd.1.smithi139.stdout:Mar 18 23:01:30 smithi139 sudo[75885]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-18T23:01:31.315 INFO:journalctl@ceph.osd.1.smithi139.stdout:Mar 18 23:01:30 smithi139 sudo[75885]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-18T23:01:31.315 INFO:journalctl@ceph.osd.1.smithi139.stdout:Mar 18 23:01:30 smithi139 sudo[75885]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-18T23:01:31.315 INFO:journalctl@ceph.osd.1.smithi139.stdout:Mar 18 23:01:30 smithi139 sudo[75885]: pam_unix(sudo:session): session closed for user root 2024-03-18T23:01:31.508 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:01:31.508+0000 7f6dbe7fc700 1 -- 172.21.15.49:0/3065915804 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7f6dc0012050 con 0x7f6dc8102e50 2024-03-18T23:01:31.514 INFO:journalctl@ceph.osd.2.smithi175.stdout:Mar 18 23:01:31 smithi175 sudo[76569]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-18T23:01:31.515 INFO:journalctl@ceph.osd.2.smithi175.stdout:Mar 18 23:01:31 smithi175 sudo[76569]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-18T23:01:31.515 INFO:journalctl@ceph.osd.2.smithi175.stdout:Mar 18 23:01:31 smithi175 sudo[76569]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-18T23:01:31.515 INFO:journalctl@ceph.osd.2.smithi175.stdout:Mar 18 23:01:31 smithi175 sudo[76569]: pam_unix(sudo:session): session closed for user root 2024-03-18T23:01:31.515 INFO:journalctl@ceph.osd.2.smithi175.stdout:Mar 18 23:01:31 smithi175 sudo[76572]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-18T23:01:31.515 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:01:31 smithi175 ceph-mon[67352]: pgmap v85: 1 pgs: 1 unknown; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-03-18T23:01:31.515 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:01:31 smithi175 ceph-mon[67352]: from='client.14283 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi049:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T23:01:31.515 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:01:31 smithi175 ceph-mon[67352]: osdmap e21: 3 total, 3 up, 3 in 2024-03-18T23:01:31.515 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:01:31 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:01:31.515 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:01:31 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:01:31.790 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:01:31 smithi049 ceph-mon[62991]: pgmap v85: 1 pgs: 1 unknown; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-03-18T23:01:31.790 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:01:31 smithi049 ceph-mon[62991]: from='client.14283 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi049:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T23:01:31.790 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:01:31 smithi049 ceph-mon[62991]: osdmap e21: 3 total, 3 up, 3 in 2024-03-18T23:01:31.790 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:01:31 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:01:31.790 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:01:31 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:01:31.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:01:31 smithi139 ceph-mon[67282]: pgmap v85: 1 pgs: 1 unknown; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-03-18T23:01:31.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:01:31 smithi139 ceph-mon[67282]: from='client.14283 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi049:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T23:01:31.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:01:31 smithi139 ceph-mon[67282]: osdmap e21: 3 total, 3 up, 3 in 2024-03-18T23:01:31.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:01:31 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:01:31.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:01:31 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:01:31.961 INFO:journalctl@ceph.osd.2.smithi175.stdout:Mar 18 23:01:31 smithi175 sudo[76572]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-18T23:01:31.962 INFO:journalctl@ceph.osd.2.smithi175.stdout:Mar 18 23:01:31 smithi175 sudo[76572]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-18T23:01:31.962 INFO:journalctl@ceph.osd.2.smithi175.stdout:Mar 18 23:01:31 smithi175 sudo[76572]: pam_unix(sudo:session): session closed for user root 2024-03-18T23:01:32.101 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:01:32 smithi049 sudo[82466]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-18T23:01:32.101 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:01:32 smithi049 sudo[82466]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-18T23:01:32.101 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:01:32 smithi049 sudo[82466]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-18T23:01:32.101 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:01:32 smithi049 sudo[82466]: pam_unix(sudo:session): session closed for user root 2024-03-18T23:01:32.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:01:32 smithi049 sudo[82494]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-18T23:01:32.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:01:32 smithi049 sudo[82494]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-18T23:01:32.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:01:32 smithi049 sudo[82494]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-18T23:01:32.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:01:32 smithi049 sudo[82494]: pam_unix(sudo:session): session closed for user root 2024-03-18T23:01:32.957 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:01:32 smithi175 sudo[76575]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-18T23:01:32.957 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:01:32 smithi175 sudo[76575]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-18T23:01:32.957 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:01:32 smithi175 sudo[76575]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-18T23:01:32.957 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:01:32 smithi175 sudo[76575]: pam_unix(sudo:session): session closed for user root 2024-03-18T23:01:32.957 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:01:32 smithi175 sudo[76578]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-18T23:01:32.957 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:01:32 smithi175 sudo[76578]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-18T23:01:32.958 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:01:32 smithi175 sudo[76578]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-18T23:01:32.958 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:01:32 smithi175 sudo[76578]: pam_unix(sudo:session): session closed for user root 2024-03-18T23:01:33.211 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:01:32 smithi175 ceph-mon[67352]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-03-18T23:01:33.212 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:01:32 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:01:33.212 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:01:32 smithi175 ceph-mon[67352]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-18T23:01:33.212 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:01:32 smithi175 ceph-mon[67352]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-18T23:01:33.212 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:01:32 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-18T23:01:33.212 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:01:32 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-18T23:01:33.212 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:01:32 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-18T23:01:33.212 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:01:32 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-18T23:01:33.212 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:01:32 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-18T23:01:33.212 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:01:32 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-18T23:01:33.214 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:01:32 smithi139 ceph-mon[67282]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-03-18T23:01:33.214 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:01:32 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:01:33.215 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:01:32 smithi139 ceph-mon[67282]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-18T23:01:33.215 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:01:32 smithi139 ceph-mon[67282]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-18T23:01:33.215 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:01:32 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-18T23:01:33.215 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:01:32 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-18T23:01:33.215 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:01:32 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-18T23:01:33.215 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:01:32 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-18T23:01:33.215 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:01:32 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-18T23:01:33.215 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:01:32 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-18T23:01:33.215 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:01:33 smithi139 sudo[75888]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-18T23:01:33.215 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:01:33 smithi139 sudo[75888]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-18T23:01:33.215 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:01:33 smithi139 sudo[75888]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-18T23:01:33.215 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:01:33 smithi139 sudo[75888]: pam_unix(sudo:session): session closed for user root 2024-03-18T23:01:33.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:01:32 smithi049 ceph-mon[62991]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-03-18T23:01:33.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:01:32 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:01:33.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:01:32 smithi049 ceph-mon[62991]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-18T23:01:33.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:01:32 smithi049 ceph-mon[62991]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-18T23:01:33.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:01:32 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-18T23:01:33.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:01:32 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-18T23:01:33.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:01:32 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-18T23:01:33.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:01:32 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-18T23:01:33.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:01:32 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-18T23:01:33.381 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:01:32 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-18T23:01:33.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:01:33 smithi139 sudo[75891]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-18T23:01:33.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:01:33 smithi139 sudo[75891]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-18T23:01:33.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:01:33 smithi139 sudo[75891]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-18T23:01:33.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:01:33 smithi139 sudo[75891]: pam_unix(sudo:session): session closed for user root 2024-03-18T23:01:34.130 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:01:33 smithi049 ceph-mon[62991]: pgmap v87: 1 pgs: 1 active+clean; 449 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-03-18T23:01:34.130 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:01:33 smithi049 ceph-mon[62991]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-18T23:01:34.130 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:01:33 smithi049 ceph-mon[62991]: from='client.? 172.21.15.49:0/639749044' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1a62e89d-45d4-4237-b813-7f96a59aca7c"}]: dispatch 2024-03-18T23:01:34.130 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:01:33 smithi049 ceph-mon[62991]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-18T23:01:34.130 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:01:33 smithi049 ceph-mon[62991]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-18T23:01:34.130 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:01:33 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-18T23:01:34.130 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:01:33 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-18T23:01:34.131 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:01:33 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-18T23:01:34.131 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:01:33 smithi049 ceph-mon[62991]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-18T23:01:34.131 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:01:33 smithi049 ceph-mon[62991]: from='client.? 172.21.15.49:0/639749044' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1a62e89d-45d4-4237-b813-7f96a59aca7c"}]': finished 2024-03-18T23:01:34.131 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:01:33 smithi049 ceph-mon[62991]: osdmap e22: 4 total, 3 up, 4 in 2024-03-18T23:01:34.131 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:01:33 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-18T23:01:34.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:01:33 smithi139 ceph-mon[67282]: pgmap v87: 1 pgs: 1 active+clean; 449 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-03-18T23:01:34.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:01:33 smithi139 ceph-mon[67282]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-18T23:01:34.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:01:33 smithi139 ceph-mon[67282]: from='client.? 172.21.15.49:0/639749044' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1a62e89d-45d4-4237-b813-7f96a59aca7c"}]: dispatch 2024-03-18T23:01:34.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:01:33 smithi139 ceph-mon[67282]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-18T23:01:34.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:01:33 smithi139 ceph-mon[67282]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-18T23:01:34.316 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:01:33 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-18T23:01:34.316 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:01:33 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-18T23:01:34.316 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:01:33 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-18T23:01:34.316 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:01:33 smithi139 ceph-mon[67282]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-18T23:01:34.316 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:01:33 smithi139 ceph-mon[67282]: from='client.? 172.21.15.49:0/639749044' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1a62e89d-45d4-4237-b813-7f96a59aca7c"}]': finished 2024-03-18T23:01:34.316 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:01:33 smithi139 ceph-mon[67282]: osdmap e22: 4 total, 3 up, 4 in 2024-03-18T23:01:34.316 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:01:33 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-18T23:01:34.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:01:33 smithi175 ceph-mon[67352]: pgmap v87: 1 pgs: 1 active+clean; 449 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-03-18T23:01:34.462 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:01:33 smithi175 ceph-mon[67352]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-18T23:01:34.462 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:01:33 smithi175 ceph-mon[67352]: from='client.? 172.21.15.49:0/639749044' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1a62e89d-45d4-4237-b813-7f96a59aca7c"}]: dispatch 2024-03-18T23:01:34.462 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:01:33 smithi175 ceph-mon[67352]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-18T23:01:34.462 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:01:33 smithi175 ceph-mon[67352]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-18T23:01:34.462 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:01:33 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-18T23:01:34.462 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:01:33 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-18T23:01:34.462 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:01:33 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-18T23:01:34.462 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:01:33 smithi175 ceph-mon[67352]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-18T23:01:34.462 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:01:33 smithi175 ceph-mon[67352]: from='client.? 172.21.15.49:0/639749044' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1a62e89d-45d4-4237-b813-7f96a59aca7c"}]': finished 2024-03-18T23:01:34.462 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:01:33 smithi175 ceph-mon[67352]: osdmap e22: 4 total, 3 up, 4 in 2024-03-18T23:01:34.462 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:01:33 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-18T23:01:35.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:01:34 smithi139 ceph-mon[67282]: pgmap v89: 1 pgs: 1 active+clean; 449 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-03-18T23:01:35.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:01:34 smithi139 ceph-mon[67282]: from='client.? 172.21.15.49:0/2266785883' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-18T23:01:35.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:01:34 smithi049 ceph-mon[62991]: pgmap v89: 1 pgs: 1 active+clean; 449 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-03-18T23:01:35.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:01:34 smithi049 ceph-mon[62991]: from='client.? 172.21.15.49:0/2266785883' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-18T23:01:35.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:01:34 smithi175 ceph-mon[67352]: pgmap v89: 1 pgs: 1 active+clean; 449 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-03-18T23:01:35.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:01:34 smithi175 ceph-mon[67352]: from='client.? 172.21.15.49:0/2266785883' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-18T23:01:37.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:01:36 smithi139 ceph-mon[67282]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 7.4 KiB/s rd, 340 KiB/s wr, 23 op/s 2024-03-18T23:01:37.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:01:36 smithi049 ceph-mon[62991]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 7.4 KiB/s rd, 340 KiB/s wr, 23 op/s 2024-03-18T23:01:37.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:01:36 smithi175 ceph-mon[67352]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 7.4 KiB/s rd, 340 KiB/s wr, 23 op/s 2024-03-18T23:01:39.130 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:01:38 smithi049 ceph-mon[62991]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 5.4 KiB/s rd, 248 KiB/s wr, 17 op/s 2024-03-18T23:01:39.264 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:01:38 smithi175 ceph-mon[67352]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 5.4 KiB/s rd, 248 KiB/s wr, 17 op/s 2024-03-18T23:01:39.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:01:38 smithi139 ceph-mon[67282]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 5.4 KiB/s rd, 248 KiB/s wr, 17 op/s 2024-03-18T23:01:41.130 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:01:40 smithi049 ceph-mon[62991]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 4.3 KiB/s rd, 195 KiB/s wr, 13 op/s 2024-03-18T23:01:41.130 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:01:40 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-18T23:01:41.130 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:01:40 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:01:41.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:01:40 smithi139 ceph-mon[67282]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 4.3 KiB/s rd, 195 KiB/s wr, 13 op/s 2024-03-18T23:01:41.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:01:40 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-18T23:01:41.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:01:40 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:01:41.458 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:01:40 smithi175 ceph-mon[67352]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 4.3 KiB/s rd, 195 KiB/s wr, 13 op/s 2024-03-18T23:01:41.458 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:01:40 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-18T23:01:41.459 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:01:40 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:01:42.130 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:01:41 smithi049 ceph-mon[62991]: Deploying daemon osd.3 on smithi049 2024-03-18T23:01:42.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:01:41 smithi139 ceph-mon[67282]: Deploying daemon osd.3 on smithi049 2024-03-18T23:01:42.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:01:41 smithi175 ceph-mon[67352]: Deploying daemon osd.3 on smithi049 2024-03-18T23:01:43.130 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:01:42 smithi049 ceph-mon[62991]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 4.0 KiB/s rd, 183 KiB/s wr, 12 op/s 2024-03-18T23:01:43.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:01:42 smithi139 ceph-mon[67282]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 4.0 KiB/s rd, 183 KiB/s wr, 12 op/s 2024-03-18T23:01:43.463 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:01:42 smithi175 ceph-mon[67352]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 4.0 KiB/s rd, 183 KiB/s wr, 12 op/s 2024-03-18T23:01:45.130 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:01:44 smithi049 ceph-mon[62991]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 3.9 KiB/s rd, 176 KiB/s wr, 12 op/s 2024-03-18T23:01:45.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:01:44 smithi139 ceph-mon[67282]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 3.9 KiB/s rd, 176 KiB/s wr, 12 op/s 2024-03-18T23:01:45.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:01:44 smithi175 ceph-mon[67352]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 3.9 KiB/s rd, 176 KiB/s wr, 12 op/s 2024-03-18T23:01:47.711 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:01:47 smithi175 ceph-mon[67352]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 3.3 KiB/s rd, 152 KiB/s wr, 10 op/s 2024-03-18T23:01:47.711 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:01:47 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:01:47.712 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:01:47 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:01:47.712 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:01:47 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:01:47.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:01:47 smithi139 ceph-mon[67282]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 3.3 KiB/s rd, 152 KiB/s wr, 10 op/s 2024-03-18T23:01:47.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:01:47 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:01:47.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:01:47 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:01:47.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:01:47 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:01:47.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:01:47 smithi049 ceph-mon[62991]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 3.3 KiB/s rd, 152 KiB/s wr, 10 op/s 2024-03-18T23:01:47.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:01:47 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:01:47.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:01:47 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:01:47.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:01:47 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:01:49.711 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:01:49 smithi175 ceph-mon[67352]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-03-18T23:01:49.712 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:01:49 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:01:49.712 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:01:49 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:01:49.712 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:01:49 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:01:49.712 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:01:49 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:01:49.712 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:01:49 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:01:49.805 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:01:49 smithi049 ceph-mon[62991]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-03-18T23:01:49.805 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:01:49 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:01:49.805 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:01:49 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:01:49.805 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:01:49 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:01:49.806 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:01:49 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:01:49.806 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:01:49 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:01:49.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:01:49 smithi139 ceph-mon[67282]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-03-18T23:01:49.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:01:49 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:01:49.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:01:49 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:01:49.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:01:49 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:01:49.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:01:49 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:01:49.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:01:49 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:01:50.184 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:01:50.183+0000 7f6dbe7fc700 1 -- 172.21.15.49:0/3065915804 <== mgr.14150 v2:172.21.15.49:6800/2083805211 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f6dc80fea00 con 0x7f6db40a2880 2024-03-18T23:01:50.185 INFO:teuthology.orchestra.run.smithi049.stdout:Created osd(s) 3 on host 'smithi049' 2024-03-18T23:01:50.187 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:01:50.185+0000 7f6dcf7b7700 1 -- 172.21.15.49:0/3065915804 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f6db40a2880 msgr2=0x7f6db40a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:01:50.187 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:01:50.185+0000 7f6dcf7b7700 1 --2- 172.21.15.49:0/3065915804 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f6db40a2880 0x7f6db40a4d40 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f6dc400b580 tx=0x7f6dc4005fb0 comp rx=0 tx=0).stop 2024-03-18T23:01:50.187 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:01:50.185+0000 7f6dcf7b7700 1 -- 172.21.15.49:0/3065915804 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6dc8102e50 msgr2=0x7f6dc8103dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:01:50.187 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:01:50.185+0000 7f6dcf7b7700 1 --2- 172.21.15.49:0/3065915804 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6dc8102e50 0x7f6dc8103dd0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f6dc000ef30 tx=0x7f6dc000c450 comp rx=0 tx=0).stop 2024-03-18T23:01:50.187 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:01:50.185+0000 7f6dcf7b7700 1 -- 172.21.15.49:0/3065915804 shutdown_connections 2024-03-18T23:01:50.187 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:01:50.185+0000 7f6dcf7b7700 1 --2- 172.21.15.49:0/3065915804 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6dc8102e50 0x7f6dc8103dd0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:01:50.187 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:01:50.185+0000 7f6dcf7b7700 1 --2- 172.21.15.49:0/3065915804 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f6db40a2880 0x7f6db40a4d40 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:01:50.188 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:01:50.185+0000 7f6dcf7b7700 1 --2- 172.21.15.49:0/3065915804 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6dc81012b0 0x7f6dc8105240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:01:50.188 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:01:50.185+0000 7f6dcf7b7700 1 --2- 172.21.15.49:0/3065915804 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6dc81024b0 0x7f6dc8103890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:01:50.188 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:01:50.185+0000 7f6dcf7b7700 1 -- 172.21.15.49:0/3065915804 >> 172.21.15.49:0/3065915804 conn(0x7f6dc80fc870 msgr2=0x7f6dc80fe2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:01:50.188 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:01:50.185+0000 7f6dcf7b7700 1 -- 172.21.15.49:0/3065915804 shutdown_connections 2024-03-18T23:01:50.188 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:01:50.185+0000 7f6dcf7b7700 1 -- 172.21.15.49:0/3065915804 wait complete. 2024-03-18T23:01:50.882 DEBUG:teuthology.orchestra.run.smithi049:osd.3> sudo journalctl -f -n 0 -u ceph-c63591ea-e57a-11ee-95c9-87774f69a715@osd.3.service 2024-03-18T23:01:50.885 INFO:tasks.cephadm:Deploying osd.4 on smithi139 with /dev/vg_nvme/lv_3... 2024-03-18T23:01:50.885 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c63591ea-e57a-11ee-95c9-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-03-18T23:01:51.130 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/mon.b/config 2024-03-18T23:01:51.252 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:01:51 smithi139 ceph-mon[67282]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-03-18T23:01:51.252 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:01:51 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:01:51.252 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:01:51 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:01:51.252 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:01:51 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:01:51.252 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:01:51 smithi139 ceph-mon[67282]: from='osd.3 [v2:172.21.15.49:6810/956160536,v1:172.21.15.49:6811/956160536]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-18T23:01:51.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:01:51 smithi175 ceph-mon[67352]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-03-18T23:01:51.462 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:01:51 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:01:51.462 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:01:51 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:01:51.462 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:01:51 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:01:51.462 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:01:51 smithi175 ceph-mon[67352]: from='osd.3 [v2:172.21.15.49:6810/956160536,v1:172.21.15.49:6811/956160536]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-18T23:01:51.630 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:01:51 smithi049 ceph-mon[62991]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-03-18T23:01:51.630 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:01:51 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:01:51.630 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:01:51 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:01:51.630 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:01:51 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:01:51.630 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:01:51 smithi049 ceph-mon[62991]: from='osd.3 [v2:172.21.15.49:6810/956160536,v1:172.21.15.49:6811/956160536]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-18T23:01:52.329 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:01:52 smithi049 ceph-mon[62991]: from='osd.3 [v2:172.21.15.49:6810/956160536,v1:172.21.15.49:6811/956160536]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-03-18T23:01:52.329 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:01:52 smithi049 ceph-mon[62991]: osdmap e23: 4 total, 3 up, 4 in 2024-03-18T23:01:52.330 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:01:52 smithi049 ceph-mon[62991]: from='osd.3 [v2:172.21.15.49:6810/956160536,v1:172.21.15.49:6811/956160536]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi049", "root=default"]}]: dispatch 2024-03-18T23:01:52.330 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:01:52 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-18T23:01:52.330 INFO:journalctl@ceph.osd.3.smithi049.stdout:Mar 18 23:01:52 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-3[84547]: 2024-03-18T23:01:52.155+0000 7f9d52a3d700 -1 osd.3 0 waiting for initial osdmap 2024-03-18T23:01:52.330 INFO:journalctl@ceph.osd.3.smithi049.stdout:Mar 18 23:01:52 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-3[84547]: 2024-03-18T23:01:52.160+0000 7f9d4e830700 -1 osd.3 24 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-18T23:01:52.460 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:01:52 smithi139 ceph-mon[67282]: from='osd.3 [v2:172.21.15.49:6810/956160536,v1:172.21.15.49:6811/956160536]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-03-18T23:01:52.460 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:01:52 smithi139 ceph-mon[67282]: osdmap e23: 4 total, 3 up, 4 in 2024-03-18T23:01:52.460 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:01:52 smithi139 ceph-mon[67282]: from='osd.3 [v2:172.21.15.49:6810/956160536,v1:172.21.15.49:6811/956160536]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi049", "root=default"]}]: dispatch 2024-03-18T23:01:52.460 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:01:52 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-18T23:01:52.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:01:52 smithi175 ceph-mon[67352]: from='osd.3 [v2:172.21.15.49:6810/956160536,v1:172.21.15.49:6811/956160536]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-03-18T23:01:52.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:01:52 smithi175 ceph-mon[67352]: osdmap e23: 4 total, 3 up, 4 in 2024-03-18T23:01:52.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:01:52 smithi175 ceph-mon[67352]: from='osd.3 [v2:172.21.15.49:6810/956160536,v1:172.21.15.49:6811/956160536]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi049", "root=default"]}]: dispatch 2024-03-18T23:01:52.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:01:52 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-18T23:01:53.020 INFO:teuthology.orchestra.run.smithi139.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-03-18T23:01:53.020 INFO:teuthology.orchestra.run.smithi139.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-03-18T23:01:53.020 INFO:teuthology.orchestra.run.smithi139.stderr: stderr: 10+0 records in 2024-03-18T23:01:53.020 INFO:teuthology.orchestra.run.smithi139.stderr:10+0 records out 2024-03-18T23:01:53.020 INFO:teuthology.orchestra.run.smithi139.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0112181 s, 935 MB/s 2024-03-18T23:01:53.020 INFO:teuthology.orchestra.run.smithi139.stderr:--> Zapping successful for: 2024-03-18T23:01:53.332 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:01:53 smithi139 ceph-mon[67282]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-03-18T23:01:53.332 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:01:53 smithi139 ceph-mon[67282]: from='osd.3 [v2:172.21.15.49:6810/956160536,v1:172.21.15.49:6811/956160536]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi049", "root=default"]}]': finished 2024-03-18T23:01:53.332 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:01:53 smithi139 ceph-mon[67282]: osdmap e24: 4 total, 3 up, 4 in 2024-03-18T23:01:53.332 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:01:53 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-18T23:01:53.332 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:01:53 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-18T23:01:53.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:01:53 smithi175 ceph-mon[67352]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-03-18T23:01:53.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:01:53 smithi175 ceph-mon[67352]: from='osd.3 [v2:172.21.15.49:6810/956160536,v1:172.21.15.49:6811/956160536]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi049", "root=default"]}]': finished 2024-03-18T23:01:53.462 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:01:53 smithi175 ceph-mon[67352]: osdmap e24: 4 total, 3 up, 4 in 2024-03-18T23:01:53.462 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:01:53 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-18T23:01:53.462 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:01:53 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-18T23:01:53.500 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:01:53 smithi049 ceph-mon[62991]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-03-18T23:01:53.500 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:01:53 smithi049 ceph-mon[62991]: from='osd.3 [v2:172.21.15.49:6810/956160536,v1:172.21.15.49:6811/956160536]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi049", "root=default"]}]': finished 2024-03-18T23:01:53.500 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:01:53 smithi049 ceph-mon[62991]: osdmap e24: 4 total, 3 up, 4 in 2024-03-18T23:01:53.500 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:01:53 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-18T23:01:53.500 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:01:53 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-18T23:01:53.675 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c63591ea-e57a-11ee-95c9-87774f69a715 -- ceph orch daemon add osd smithi139:vg_nvme/lv_3 2024-03-18T23:01:53.919 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/mon.b/config 2024-03-18T23:01:54.304 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:01:54 smithi139 ceph-mon[67282]: purged_snaps scrub starts 2024-03-18T23:01:54.305 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:01:54 smithi139 ceph-mon[67282]: purged_snaps scrub ok 2024-03-18T23:01:54.305 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:01:54 smithi139 ceph-mon[67282]: osd.3 [v2:172.21.15.49:6810/956160536,v1:172.21.15.49:6811/956160536] boot 2024-03-18T23:01:54.305 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:01:54 smithi139 ceph-mon[67282]: osdmap e25: 4 total, 4 up, 4 in 2024-03-18T23:01:54.305 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:01:54 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-18T23:01:54.305 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:01:54 smithi139 ceph-mon[67282]: osdmap e26: 4 total, 4 up, 4 in 2024-03-18T23:01:54.305 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:01:54 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:01:54.305 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:01:54 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:01:54.305 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:01:54 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:01:54.305 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:01:54 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:01:54.305 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:01:54 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:01:54.305 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:01:54 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:01:54.306 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:01:54 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:01:54.306 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:01:54 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:01:54.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:01:54 smithi175 ceph-mon[67352]: purged_snaps scrub starts 2024-03-18T23:01:54.462 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:01:54 smithi175 ceph-mon[67352]: purged_snaps scrub ok 2024-03-18T23:01:54.462 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:01:54 smithi175 ceph-mon[67352]: osd.3 [v2:172.21.15.49:6810/956160536,v1:172.21.15.49:6811/956160536] boot 2024-03-18T23:01:54.462 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:01:54 smithi175 ceph-mon[67352]: osdmap e25: 4 total, 4 up, 4 in 2024-03-18T23:01:54.462 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:01:54 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-18T23:01:54.462 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:01:54 smithi175 ceph-mon[67352]: osdmap e26: 4 total, 4 up, 4 in 2024-03-18T23:01:54.462 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:01:54 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:01:54.462 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:01:54 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:01:54.462 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:01:54 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:01:54.462 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:01:54 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:01:54.462 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:01:54 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:01:54.462 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:01:54 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:01:54.462 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:01:54 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:01:54.462 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:01:54 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:01:54.630 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:01:54 smithi049 ceph-mon[62991]: purged_snaps scrub starts 2024-03-18T23:01:54.630 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:01:54 smithi049 ceph-mon[62991]: purged_snaps scrub ok 2024-03-18T23:01:54.630 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:01:54 smithi049 ceph-mon[62991]: osd.3 [v2:172.21.15.49:6810/956160536,v1:172.21.15.49:6811/956160536] boot 2024-03-18T23:01:54.630 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:01:54 smithi049 ceph-mon[62991]: osdmap e25: 4 total, 4 up, 4 in 2024-03-18T23:01:54.630 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:01:54 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-18T23:01:54.631 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:01:54 smithi049 ceph-mon[62991]: osdmap e26: 4 total, 4 up, 4 in 2024-03-18T23:01:54.631 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:01:54 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:01:54.631 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:01:54 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:01:54.631 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:01:54 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:01:54.631 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:01:54 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:01:54.631 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:01:54 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:01:54.631 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:01:54 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:01:54.631 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:01:54 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:01:54.631 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:01:54 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:01:55.537 INFO:teuthology.orchestra.run.smithi139.stderr:2024-03-18T23:01:55.534+0000 7f97853b6700 1 -- 172.21.15.139:0/2140432268 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9780101270 msgr2=0x7f9780101690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:01:55.537 INFO:teuthology.orchestra.run.smithi139.stderr:2024-03-18T23:01:55.534+0000 7f97853b6700 1 --2- 172.21.15.139:0/2140432268 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9780101270 0x7f9780101690 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f97700099c0 tx=0x7f9770009cd0 comp rx=0 tx=0).stop 2024-03-18T23:01:55.538 INFO:teuthology.orchestra.run.smithi139.stderr:2024-03-18T23:01:55.534+0000 7f97853b6700 1 -- 172.21.15.139:0/2140432268 shutdown_connections 2024-03-18T23:01:55.538 INFO:teuthology.orchestra.run.smithi139.stderr:2024-03-18T23:01:55.534+0000 7f97853b6700 1 --2- 172.21.15.139:0/2140432268 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9780102e10 0x7f97801096c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:01:55.538 INFO:teuthology.orchestra.run.smithi139.stderr:2024-03-18T23:01:55.534+0000 7f97853b6700 1 --2- 172.21.15.139:0/2140432268 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9780101270 0x7f9780101690 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:01:55.538 INFO:teuthology.orchestra.run.smithi139.stderr:2024-03-18T23:01:55.534+0000 7f97853b6700 1 --2- 172.21.15.139:0/2140432268 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9780102470 0x7f97801028d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:01:55.538 INFO:teuthology.orchestra.run.smithi139.stderr:2024-03-18T23:01:55.534+0000 7f97853b6700 1 -- 172.21.15.139:0/2140432268 >> 172.21.15.139:0/2140432268 conn(0x7f97800fc830 msgr2=0x7f97800fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:01:55.538 INFO:teuthology.orchestra.run.smithi139.stderr:2024-03-18T23:01:55.534+0000 7f97853b6700 1 -- 172.21.15.139:0/2140432268 shutdown_connections 2024-03-18T23:01:55.538 INFO:teuthology.orchestra.run.smithi139.stderr:2024-03-18T23:01:55.534+0000 7f97853b6700 1 -- 172.21.15.139:0/2140432268 wait complete. 2024-03-18T23:01:55.538 INFO:teuthology.orchestra.run.smithi139.stderr:2024-03-18T23:01:55.535+0000 7f97853b6700 1 Processor -- start 2024-03-18T23:01:55.538 INFO:teuthology.orchestra.run.smithi139.stderr:2024-03-18T23:01:55.535+0000 7f97853b6700 1 -- start start 2024-03-18T23:01:55.538 INFO:teuthology.orchestra.run.smithi139.stderr:2024-03-18T23:01:55.535+0000 7f97853b6700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9780101270 0x7f9780197440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:01:55.539 INFO:teuthology.orchestra.run.smithi139.stderr:2024-03-18T23:01:55.535+0000 7f97853b6700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9780102470 0x7f9780197980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:01:55.539 INFO:teuthology.orchestra.run.smithi139.stderr:2024-03-18T23:01:55.535+0000 7f97853b6700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9780102e10 0x7f978019ea00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:01:55.539 INFO:teuthology.orchestra.run.smithi139.stderr:2024-03-18T23:01:55.535+0000 7f97853b6700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f978010ce30 con 0x7f9780101270 2024-03-18T23:01:55.539 INFO:teuthology.orchestra.run.smithi139.stderr:2024-03-18T23:01:55.535+0000 7f97853b6700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f978010ccb0 con 0x7f9780102e10 2024-03-18T23:01:55.539 INFO:teuthology.orchestra.run.smithi139.stderr:2024-03-18T23:01:55.535+0000 7f97853b6700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f978010cfb0 con 0x7f9780102470 2024-03-18T23:01:55.539 INFO:teuthology.orchestra.run.smithi139.stderr:2024-03-18T23:01:55.536+0000 7f977f7fe700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9780102e10 0x7f978019ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:01:55.539 INFO:teuthology.orchestra.run.smithi139.stderr:2024-03-18T23:01:55.536+0000 7f977f7fe700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9780102e10 0x7f978019ea00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:58194/0 (socket says 172.21.15.139:58194) 2024-03-18T23:01:55.539 INFO:teuthology.orchestra.run.smithi139.stderr:2024-03-18T23:01:55.536+0000 7f977f7fe700 1 -- 172.21.15.139:0/4123836902 learned_addr learned my addr 172.21.15.139:0/4123836902 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-03-18T23:01:55.539 INFO:teuthology.orchestra.run.smithi139.stderr:2024-03-18T23:01:55.536+0000 7f977effd700 1 --2- 172.21.15.139:0/4123836902 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9780101270 0x7f9780197440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:01:55.539 INFO:teuthology.orchestra.run.smithi139.stderr:2024-03-18T23:01:55.536+0000 7f977e7fc700 1 --2- 172.21.15.139:0/4123836902 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9780102470 0x7f9780197980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:01:55.539 INFO:teuthology.orchestra.run.smithi139.stderr:2024-03-18T23:01:55.536+0000 7f977f7fe700 1 -- 172.21.15.139:0/4123836902 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9780102470 msgr2=0x7f9780197980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:01:55.539 INFO:teuthology.orchestra.run.smithi139.stderr:2024-03-18T23:01:55.536+0000 7f977f7fe700 1 --2- 172.21.15.139:0/4123836902 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9780102470 0x7f9780197980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:01:55.539 INFO:teuthology.orchestra.run.smithi139.stderr:2024-03-18T23:01:55.536+0000 7f977f7fe700 1 -- 172.21.15.139:0/4123836902 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9780101270 msgr2=0x7f9780197440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:01:55.540 INFO:teuthology.orchestra.run.smithi139.stderr:2024-03-18T23:01:55.536+0000 7f977f7fe700 1 --2- 172.21.15.139:0/4123836902 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9780101270 0x7f9780197440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:01:55.540 INFO:teuthology.orchestra.run.smithi139.stderr:2024-03-18T23:01:55.536+0000 7f977f7fe700 1 -- 172.21.15.139:0/4123836902 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f978006aaf0 con 0x7f9780102e10 2024-03-18T23:01:55.540 INFO:teuthology.orchestra.run.smithi139.stderr:2024-03-18T23:01:55.536+0000 7f977effd700 1 --2- 172.21.15.139:0/4123836902 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9780101270 0x7f9780197440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T23:01:55.540 INFO:teuthology.orchestra.run.smithi139.stderr:2024-03-18T23:01:55.536+0000 7f977f7fe700 1 --2- 172.21.15.139:0/4123836902 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9780102e10 0x7f978019ea00 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f977800ada0 tx=0x7f977800c330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:01:55.540 INFO:teuthology.orchestra.run.smithi139.stderr:2024-03-18T23:01:55.536+0000 7f9767fff700 1 -- 172.21.15.139:0/4123836902 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9778017020 con 0x7f9780102e10 2024-03-18T23:01:55.540 INFO:teuthology.orchestra.run.smithi139.stderr:2024-03-18T23:01:55.536+0000 7f9767fff700 1 -- 172.21.15.139:0/4123836902 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9778004bb0 con 0x7f9780102e10 2024-03-18T23:01:55.540 INFO:teuthology.orchestra.run.smithi139.stderr:2024-03-18T23:01:55.536+0000 7f97853b6700 1 -- 172.21.15.139:0/4123836902 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f978006ade0 con 0x7f9780102e10 2024-03-18T23:01:55.540 INFO:teuthology.orchestra.run.smithi139.stderr:2024-03-18T23:01:55.536+0000 7f9767fff700 1 -- 172.21.15.139:0/4123836902 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9778005520 con 0x7f9780102e10 2024-03-18T23:01:55.540 INFO:teuthology.orchestra.run.smithi139.stderr:2024-03-18T23:01:55.536+0000 7f97853b6700 1 -- 172.21.15.139:0/4123836902 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f978006b2d0 con 0x7f9780102e10 2024-03-18T23:01:55.541 INFO:teuthology.orchestra.run.smithi139.stderr:2024-03-18T23:01:55.537+0000 7f97853b6700 1 -- 172.21.15.139:0/4123836902 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9780068fd0 con 0x7f9780102e10 2024-03-18T23:01:55.541 INFO:teuthology.orchestra.run.smithi139.stderr:2024-03-18T23:01:55.538+0000 7f9767fff700 1 -- 172.21.15.139:0/4123836902 <== mon.2 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7f97780056c0 con 0x7f9780102e10 2024-03-18T23:01:55.541 INFO:teuthology.orchestra.run.smithi139.stderr:2024-03-18T23:01:55.539+0000 7f9767fff700 1 --2- 172.21.15.139:0/4123836902 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f97680a2890 0x7f97680a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:01:55.541 INFO:teuthology.orchestra.run.smithi139.stderr:2024-03-18T23:01:55.539+0000 7f9767fff700 1 -- 172.21.15.139:0/4123836902 <== mon.2 v2:172.21.15.139:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 3698+0+0 (secure 0 0 0) 0x7f97780cd5c0 con 0x7f9780102e10 2024-03-18T23:01:55.541 INFO:teuthology.orchestra.run.smithi139.stderr:2024-03-18T23:01:55.539+0000 7f977effd700 1 --2- 172.21.15.139:0/4123836902 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f97680a2890 0x7f97680a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:01:55.542 INFO:teuthology.orchestra.run.smithi139.stderr:2024-03-18T23:01:55.540+0000 7f977effd700 1 --2- 172.21.15.139:0/4123836902 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f97680a2890 0x7f97680a4d50 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f977000b580 tx=0x7f9770005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:01:55.543 INFO:teuthology.orchestra.run.smithi139.stderr:2024-03-18T23:01:55.543+0000 7f9767fff700 1 -- 172.21.15.139:0/4123836902 <== mon.2 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f9778021020 con 0x7f9780102e10 2024-03-18T23:01:55.747 INFO:teuthology.orchestra.run.smithi139.stderr:2024-03-18T23:01:55.746+0000 7f97853b6700 1 -- 172.21.15.139:0/4123836902 --> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi139:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f9780062750 con 0x7f97680a2890 2024-03-18T23:01:55.961 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:01:55 smithi175 ceph-mon[67352]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-03-18T23:01:55.962 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:01:55 smithi175 ceph-mon[67352]: Detected new or changed devices on smithi049 2024-03-18T23:01:55.962 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:01:55 smithi175 ceph-mon[67352]: Adjusting osd_memory_target on smithi049 to 8544M 2024-03-18T23:01:55.962 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:01:55 smithi175 ceph-mon[67352]: osdmap e27: 4 total, 4 up, 4 in 2024-03-18T23:01:55.968 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:01:55 smithi049 ceph-mon[62991]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-03-18T23:01:55.968 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:01:55 smithi049 ceph-mon[62991]: Detected new or changed devices on smithi049 2024-03-18T23:01:55.968 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:01:55 smithi049 ceph-mon[62991]: Adjusting osd_memory_target on smithi049 to 8544M 2024-03-18T23:01:55.968 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:01:55 smithi049 ceph-mon[62991]: osdmap e27: 4 total, 4 up, 4 in 2024-03-18T23:01:56.065 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:01:55 smithi139 ceph-mon[67282]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-03-18T23:01:56.065 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:01:55 smithi139 ceph-mon[67282]: Detected new or changed devices on smithi049 2024-03-18T23:01:56.065 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:01:55 smithi139 ceph-mon[67282]: Adjusting osd_memory_target on smithi049 to 8544M 2024-03-18T23:01:56.065 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:01:55 smithi139 ceph-mon[67282]: osdmap e27: 4 total, 4 up, 4 in 2024-03-18T23:01:56.961 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:01:56 smithi175 ceph-mon[67352]: osdmap e28: 4 total, 4 up, 4 in 2024-03-18T23:01:56.961 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:01:56 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-18T23:01:56.962 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:01:56 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-18T23:01:56.962 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:01:56 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:01:57.065 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:01:56 smithi139 ceph-mon[67282]: osdmap e28: 4 total, 4 up, 4 in 2024-03-18T23:01:57.065 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:01:56 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-18T23:01:57.065 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:01:56 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-18T23:01:57.065 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:01:56 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:01:57.130 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:01:56 smithi049 ceph-mon[62991]: osdmap e28: 4 total, 4 up, 4 in 2024-03-18T23:01:57.130 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:01:56 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-18T23:01:57.130 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:01:56 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-18T23:01:57.130 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:01:56 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:01:57.961 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:01:57 smithi175 ceph-mon[67352]: from='client.24203 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi139:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T23:01:57.962 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:01:57 smithi175 ceph-mon[67352]: pgmap v106: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 2/6 objects degraded (33.333%) 2024-03-18T23:01:57.962 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:01:57 smithi175 ceph-mon[67352]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2024-03-18T23:01:57.962 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:01:57 smithi175 ceph-mon[67352]: osdmap e29: 4 total, 4 up, 4 in 2024-03-18T23:01:58.065 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:01:57 smithi139 ceph-mon[67282]: from='client.24203 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi139:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T23:01:58.065 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:01:57 smithi139 ceph-mon[67282]: pgmap v106: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 2/6 objects degraded (33.333%) 2024-03-18T23:01:58.065 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:01:57 smithi139 ceph-mon[67282]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2024-03-18T23:01:58.065 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:01:57 smithi139 ceph-mon[67282]: osdmap e29: 4 total, 4 up, 4 in 2024-03-18T23:01:58.130 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:01:57 smithi049 ceph-mon[62991]: from='client.24203 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi139:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T23:01:58.130 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:01:57 smithi049 ceph-mon[62991]: pgmap v106: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 2/6 objects degraded (33.333%) 2024-03-18T23:01:58.130 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:01:57 smithi049 ceph-mon[62991]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2024-03-18T23:01:58.130 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:01:57 smithi049 ceph-mon[62991]: osdmap e29: 4 total, 4 up, 4 in 2024-03-18T23:01:59.065 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:01:58 smithi139 ceph-mon[67282]: from='client.? 172.21.15.139:0/1413428419' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c715101c-0c3f-4b98-b142-d64a35e187df"}]: dispatch 2024-03-18T23:01:59.066 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:01:58 smithi139 ceph-mon[67282]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c715101c-0c3f-4b98-b142-d64a35e187df"}]: dispatch 2024-03-18T23:01:59.066 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:01:58 smithi139 ceph-mon[67282]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c715101c-0c3f-4b98-b142-d64a35e187df"}]': finished 2024-03-18T23:01:59.066 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:01:58 smithi139 ceph-mon[67282]: osdmap e30: 5 total, 4 up, 5 in 2024-03-18T23:01:59.066 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:01:58 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-18T23:01:59.130 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:01:58 smithi049 ceph-mon[62991]: from='client.? 172.21.15.139:0/1413428419' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c715101c-0c3f-4b98-b142-d64a35e187df"}]: dispatch 2024-03-18T23:01:59.130 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:01:58 smithi049 ceph-mon[62991]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c715101c-0c3f-4b98-b142-d64a35e187df"}]: dispatch 2024-03-18T23:01:59.131 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:01:58 smithi049 ceph-mon[62991]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c715101c-0c3f-4b98-b142-d64a35e187df"}]': finished 2024-03-18T23:01:59.131 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:01:58 smithi049 ceph-mon[62991]: osdmap e30: 5 total, 4 up, 5 in 2024-03-18T23:01:59.131 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:01:58 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-18T23:01:59.212 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:01:58 smithi175 ceph-mon[67352]: from='client.? 172.21.15.139:0/1413428419' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c715101c-0c3f-4b98-b142-d64a35e187df"}]: dispatch 2024-03-18T23:01:59.212 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:01:58 smithi175 ceph-mon[67352]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c715101c-0c3f-4b98-b142-d64a35e187df"}]: dispatch 2024-03-18T23:01:59.212 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:01:58 smithi175 ceph-mon[67352]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c715101c-0c3f-4b98-b142-d64a35e187df"}]': finished 2024-03-18T23:01:59.212 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:01:58 smithi175 ceph-mon[67352]: osdmap e30: 5 total, 4 up, 5 in 2024-03-18T23:01:59.212 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:01:58 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-18T23:02:00.065 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:01:59 smithi139 ceph-mon[67282]: pgmap v108: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 2/6 objects degraded (33.333%) 2024-03-18T23:02:00.065 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:01:59 smithi139 ceph-mon[67282]: from='client.? 172.21.15.139:0/3106867698' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-18T23:02:00.130 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:01:59 smithi049 ceph-mon[62991]: pgmap v108: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 2/6 objects degraded (33.333%) 2024-03-18T23:02:00.130 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:01:59 smithi049 ceph-mon[62991]: from='client.? 172.21.15.139:0/3106867698' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-18T23:02:00.211 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:01:59 smithi175 ceph-mon[67352]: pgmap v108: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 2/6 objects degraded (33.333%) 2024-03-18T23:02:00.211 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:01:59 smithi175 ceph-mon[67352]: from='client.? 172.21.15.139:0/3106867698' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-18T23:02:02.065 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:02:01 smithi139 ceph-mon[67282]: pgmap v110: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 2/6 objects degraded (33.333%) 2024-03-18T23:02:02.130 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:02:01 smithi049 ceph-mon[62991]: pgmap v110: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 2/6 objects degraded (33.333%) 2024-03-18T23:02:02.211 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:02:01 smithi175 ceph-mon[67352]: pgmap v110: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 2/6 objects degraded (33.333%) 2024-03-18T23:02:03.065 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:02:02 smithi139 ceph-mon[67282]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 94 KiB/s, 0 objects/s recovering 2024-03-18T23:02:03.065 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:02:02 smithi139 ceph-mon[67282]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2024-03-18T23:02:03.065 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:02:02 smithi139 ceph-mon[67282]: Cluster is now healthy 2024-03-18T23:02:03.130 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:02:02 smithi049 ceph-mon[62991]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 94 KiB/s, 0 objects/s recovering 2024-03-18T23:02:03.130 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:02:02 smithi049 ceph-mon[62991]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2024-03-18T23:02:03.130 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:02:02 smithi049 ceph-mon[62991]: Cluster is now healthy 2024-03-18T23:02:03.211 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:02:02 smithi175 ceph-mon[67352]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 94 KiB/s, 0 objects/s recovering 2024-03-18T23:02:03.211 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:02:02 smithi175 ceph-mon[67352]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2024-03-18T23:02:03.211 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:02:02 smithi175 ceph-mon[67352]: Cluster is now healthy 2024-03-18T23:02:05.211 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:02:04 smithi175 ceph-mon[67352]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-18T23:02:05.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:02:04 smithi139 ceph-mon[67282]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-18T23:02:05.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:02:04 smithi049 ceph-mon[62991]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-18T23:02:06.065 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:02:05 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-18T23:02:06.065 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:02:05 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:02:06.211 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:02:05 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-18T23:02:06.212 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:02:05 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:02:06.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:02:05 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-18T23:02:06.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:02:05 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:02:07.065 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:02:06 smithi139 ceph-mon[67282]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-03-18T23:02:07.065 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:02:06 smithi139 ceph-mon[67282]: Deploying daemon osd.4 on smithi139 2024-03-18T23:02:07.130 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:02:06 smithi049 ceph-mon[62991]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-03-18T23:02:07.130 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:02:06 smithi049 ceph-mon[62991]: Deploying daemon osd.4 on smithi139 2024-03-18T23:02:07.211 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:02:06 smithi175 ceph-mon[67352]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-03-18T23:02:07.211 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:02:06 smithi175 ceph-mon[67352]: Deploying daemon osd.4 on smithi139 2024-03-18T23:02:09.214 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:02:08 smithi175 ceph-mon[67352]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-18T23:02:09.306 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:02:08 smithi139 ceph-mon[67282]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-18T23:02:09.381 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:02:08 smithi049 ceph-mon[62991]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-18T23:02:11.211 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:02:10 smithi175 ceph-mon[67352]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-03-18T23:02:11.242 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:02:10 smithi139 ceph-mon[67282]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-03-18T23:02:11.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:02:10 smithi049 ceph-mon[62991]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-03-18T23:02:12.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:02:12 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:02:12.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:02:12 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:02:12.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:02:12 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:02:12.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:02:12 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:02:12.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:02:12 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:02:12.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:02:12 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:02:12.962 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:02:12 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:02:12.963 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:02:12 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:02:12.963 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:02:12 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:02:13.595 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:02:13 smithi139 ceph-mon[67282]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-03-18T23:02:13.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:02:13 smithi049 ceph-mon[62991]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-03-18T23:02:13.961 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:02:13 smithi175 ceph-mon[67352]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-03-18T23:02:15.065 INFO:teuthology.orchestra.run.smithi139.stdout:Created osd(s) 4 on host 'smithi139' 2024-03-18T23:02:15.065 INFO:teuthology.orchestra.run.smithi139.stderr:2024-03-18T23:02:15.064+0000 7f9767fff700 1 -- 172.21.15.139:0/4123836902 <== mgr.14150 v2:172.21.15.49:6800/2083805211 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f9780062750 con 0x7f97680a2890 2024-03-18T23:02:15.067 INFO:teuthology.orchestra.run.smithi139.stderr:2024-03-18T23:02:15.068+0000 7f97853b6700 1 -- 172.21.15.139:0/4123836902 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f97680a2890 msgr2=0x7f97680a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:02:15.068 INFO:teuthology.orchestra.run.smithi139.stderr:2024-03-18T23:02:15.068+0000 7f97853b6700 1 --2- 172.21.15.139:0/4123836902 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f97680a2890 0x7f97680a4d50 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f977000b580 tx=0x7f9770005fb0 comp rx=0 tx=0).stop 2024-03-18T23:02:15.068 INFO:teuthology.orchestra.run.smithi139.stderr:2024-03-18T23:02:15.068+0000 7f97853b6700 1 -- 172.21.15.139:0/4123836902 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9780102e10 msgr2=0x7f978019ea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:02:15.068 INFO:teuthology.orchestra.run.smithi139.stderr:2024-03-18T23:02:15.068+0000 7f97853b6700 1 --2- 172.21.15.139:0/4123836902 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9780102e10 0x7f978019ea00 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f977800ada0 tx=0x7f977800c330 comp rx=0 tx=0).stop 2024-03-18T23:02:15.068 INFO:teuthology.orchestra.run.smithi139.stderr:2024-03-18T23:02:15.068+0000 7f97853b6700 1 -- 172.21.15.139:0/4123836902 shutdown_connections 2024-03-18T23:02:15.068 INFO:teuthology.orchestra.run.smithi139.stderr:2024-03-18T23:02:15.068+0000 7f97853b6700 1 --2- 172.21.15.139:0/4123836902 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9780102e10 0x7f978019ea00 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:02:15.068 INFO:teuthology.orchestra.run.smithi139.stderr:2024-03-18T23:02:15.068+0000 7f97853b6700 1 --2- 172.21.15.139:0/4123836902 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9780101270 0x7f9780197440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:02:15.068 INFO:teuthology.orchestra.run.smithi139.stderr:2024-03-18T23:02:15.068+0000 7f97853b6700 1 --2- 172.21.15.139:0/4123836902 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f97680a2890 0x7f97680a4d50 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:02:15.068 INFO:teuthology.orchestra.run.smithi139.stderr:2024-03-18T23:02:15.068+0000 7f97853b6700 1 --2- 172.21.15.139:0/4123836902 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9780102470 0x7f9780197980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:02:15.068 INFO:teuthology.orchestra.run.smithi139.stderr:2024-03-18T23:02:15.068+0000 7f97853b6700 1 -- 172.21.15.139:0/4123836902 >> 172.21.15.139:0/4123836902 conn(0x7f97800fc830 msgr2=0x7f97801057f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:02:15.068 INFO:teuthology.orchestra.run.smithi139.stderr:2024-03-18T23:02:15.068+0000 7f97853b6700 1 -- 172.21.15.139:0/4123836902 shutdown_connections 2024-03-18T23:02:15.068 INFO:teuthology.orchestra.run.smithi139.stderr:2024-03-18T23:02:15.068+0000 7f97853b6700 1 -- 172.21.15.139:0/4123836902 wait complete. 2024-03-18T23:02:15.566 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:02:15 smithi139 ceph-mon[67282]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-03-18T23:02:15.566 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:02:15 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:02:15.566 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:02:15 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:02:15.566 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:02:15 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:02:15.566 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:02:15 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:02:15.566 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:02:15 smithi139 ceph-mon[67282]: from='osd.4 [v2:172.21.15.139:6808/3129712668,v1:172.21.15.139:6809/3129712668]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-18T23:02:15.566 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:02:15 smithi139 ceph-mon[67282]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-18T23:02:15.814 DEBUG:teuthology.orchestra.run.smithi139:osd.4> sudo journalctl -f -n 0 -u ceph-c63591ea-e57a-11ee-95c9-87774f69a715@osd.4.service 2024-03-18T23:02:15.817 INFO:tasks.cephadm:Deploying osd.5 on smithi175 with /dev/vg_nvme/lv_3... 2024-03-18T23:02:15.817 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c63591ea-e57a-11ee-95c9-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-03-18T23:02:15.846 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:02:15 smithi175 ceph-mon[67352]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-03-18T23:02:15.847 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:02:15 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:02:15.847 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:02:15 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:02:15.847 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:02:15 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:02:15.847 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:02:15 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:02:15.847 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:02:15 smithi175 ceph-mon[67352]: from='osd.4 [v2:172.21.15.139:6808/3129712668,v1:172.21.15.139:6809/3129712668]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-18T23:02:15.847 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:02:15 smithi175 ceph-mon[67352]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-18T23:02:15.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:02:15 smithi049 ceph-mon[62991]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-03-18T23:02:15.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:02:15 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:02:15.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:02:15 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:02:15.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:02:15 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:02:15.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:02:15 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:02:15.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:02:15 smithi049 ceph-mon[62991]: from='osd.4 [v2:172.21.15.139:6808/3129712668,v1:172.21.15.139:6809/3129712668]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-18T23:02:15.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:02:15 smithi049 ceph-mon[62991]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-18T23:02:16.060 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/mon.c/config 2024-03-18T23:02:17.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:02:17 smithi139 ceph-mon[67282]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-03-18T23:02:17.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:02:17 smithi139 ceph-mon[67282]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-03-18T23:02:17.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:02:17 smithi139 ceph-mon[67282]: from='osd.4 [v2:172.21.15.139:6808/3129712668,v1:172.21.15.139:6809/3129712668]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]: dispatch 2024-03-18T23:02:17.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:02:17 smithi139 ceph-mon[67282]: osdmap e31: 5 total, 4 up, 5 in 2024-03-18T23:02:17.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:02:17 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-18T23:02:17.316 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:02:17 smithi139 ceph-mon[67282]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]: dispatch 2024-03-18T23:02:17.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:02:17 smithi049 ceph-mon[62991]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-03-18T23:02:17.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:02:17 smithi049 ceph-mon[62991]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-03-18T23:02:17.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:02:17 smithi049 ceph-mon[62991]: from='osd.4 [v2:172.21.15.139:6808/3129712668,v1:172.21.15.139:6809/3129712668]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]: dispatch 2024-03-18T23:02:17.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:02:17 smithi049 ceph-mon[62991]: osdmap e31: 5 total, 4 up, 5 in 2024-03-18T23:02:17.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:02:17 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-18T23:02:17.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:02:17 smithi049 ceph-mon[62991]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]: dispatch 2024-03-18T23:02:17.389 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:02:17 smithi175 ceph-mon[67352]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-03-18T23:02:17.389 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:02:17 smithi175 ceph-mon[67352]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-03-18T23:02:17.389 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:02:17 smithi175 ceph-mon[67352]: from='osd.4 [v2:172.21.15.139:6808/3129712668,v1:172.21.15.139:6809/3129712668]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]: dispatch 2024-03-18T23:02:17.389 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:02:17 smithi175 ceph-mon[67352]: osdmap e31: 5 total, 4 up, 5 in 2024-03-18T23:02:17.390 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:02:17 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-18T23:02:17.390 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:02:17 smithi175 ceph-mon[67352]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]: dispatch 2024-03-18T23:02:17.815 INFO:journalctl@ceph.osd.4.smithi139.stdout:Mar 18 23:02:17 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-4[78727]: 2024-03-18T23:02:17.546+0000 7f1004577700 -1 osd.4 0 waiting for initial osdmap 2024-03-18T23:02:17.816 INFO:journalctl@ceph.osd.4.smithi139.stdout:Mar 18 23:02:17 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-4[78727]: 2024-03-18T23:02:17.552+0000 7f0ffbb61700 -1 osd.4 32 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-18T23:02:18.268 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:02:18 smithi175 ceph-mon[67352]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]': finished 2024-03-18T23:02:18.268 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:02:18 smithi175 ceph-mon[67352]: osdmap e32: 5 total, 4 up, 5 in 2024-03-18T23:02:18.268 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:02:18 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-18T23:02:18.268 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:02:18 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-18T23:02:18.268 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:02:18 smithi175 ceph-mon[67352]: from='osd.4 ' entity='osd.4' 2024-03-18T23:02:18.269 INFO:teuthology.orchestra.run.smithi175.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-03-18T23:02:18.269 INFO:teuthology.orchestra.run.smithi175.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-03-18T23:02:18.269 INFO:teuthology.orchestra.run.smithi175.stderr: stderr: 10+0 records in 2024-03-18T23:02:18.269 INFO:teuthology.orchestra.run.smithi175.stderr:10+0 records out 2024-03-18T23:02:18.269 INFO:teuthology.orchestra.run.smithi175.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0118494 s, 885 MB/s 2024-03-18T23:02:18.269 INFO:teuthology.orchestra.run.smithi175.stderr:--> Zapping successful for: 2024-03-18T23:02:18.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:02:18 smithi049 ceph-mon[62991]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]': finished 2024-03-18T23:02:18.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:02:18 smithi049 ceph-mon[62991]: osdmap e32: 5 total, 4 up, 5 in 2024-03-18T23:02:18.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:02:18 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-18T23:02:18.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:02:18 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-18T23:02:18.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:02:18 smithi049 ceph-mon[62991]: from='osd.4 ' entity='osd.4' 2024-03-18T23:02:18.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:02:18 smithi139 ceph-mon[67282]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]': finished 2024-03-18T23:02:18.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:02:18 smithi139 ceph-mon[67282]: osdmap e32: 5 total, 4 up, 5 in 2024-03-18T23:02:18.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:02:18 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-18T23:02:18.566 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:02:18 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-18T23:02:18.566 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:02:18 smithi139 ceph-mon[67282]: from='osd.4 ' entity='osd.4' 2024-03-18T23:02:18.849 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c63591ea-e57a-11ee-95c9-87774f69a715 -- ceph orch daemon add osd smithi175:vg_nvme/lv_3 2024-03-18T23:02:19.130 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/mon.c/config 2024-03-18T23:02:19.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:02:19 smithi139 ceph-mon[67282]: purged_snaps scrub starts 2024-03-18T23:02:19.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:02:19 smithi139 ceph-mon[67282]: purged_snaps scrub ok 2024-03-18T23:02:19.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:02:19 smithi139 ceph-mon[67282]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-03-18T23:02:19.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:02:19 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-18T23:02:19.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:02:19 smithi139 ceph-mon[67282]: osd.4 [v2:172.21.15.139:6808/3129712668,v1:172.21.15.139:6809/3129712668] boot 2024-03-18T23:02:19.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:02:19 smithi139 ceph-mon[67282]: osdmap e33: 5 total, 5 up, 5 in 2024-03-18T23:02:19.316 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:02:19 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-18T23:02:19.316 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:02:19 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:02:19.316 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:02:19 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:02:19.316 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:02:19 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:02:19.316 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:02:19 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:02:19.316 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:02:19 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:02:19.316 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:02:19 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:02:19.316 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:02:19 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:02:19.316 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:02:19 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:02:19.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:02:19 smithi049 ceph-mon[62991]: purged_snaps scrub starts 2024-03-18T23:02:19.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:02:19 smithi049 ceph-mon[62991]: purged_snaps scrub ok 2024-03-18T23:02:19.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:02:19 smithi049 ceph-mon[62991]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-03-18T23:02:19.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:02:19 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-18T23:02:19.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:02:19 smithi049 ceph-mon[62991]: osd.4 [v2:172.21.15.139:6808/3129712668,v1:172.21.15.139:6809/3129712668] boot 2024-03-18T23:02:19.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:02:19 smithi049 ceph-mon[62991]: osdmap e33: 5 total, 5 up, 5 in 2024-03-18T23:02:19.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:02:19 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-18T23:02:19.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:02:19 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:02:19.381 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:02:19 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:02:19.381 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:02:19 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:02:19.381 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:02:19 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:02:19.381 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:02:19 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:02:19.381 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:02:19 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:02:19.381 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:02:19 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:02:19.381 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:02:19 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:02:19.391 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:02:19 smithi175 ceph-mon[67352]: purged_snaps scrub starts 2024-03-18T23:02:19.391 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:02:19 smithi175 ceph-mon[67352]: purged_snaps scrub ok 2024-03-18T23:02:19.391 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:02:19 smithi175 ceph-mon[67352]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-03-18T23:02:19.391 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:02:19 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-18T23:02:19.391 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:02:19 smithi175 ceph-mon[67352]: osd.4 [v2:172.21.15.139:6808/3129712668,v1:172.21.15.139:6809/3129712668] boot 2024-03-18T23:02:19.391 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:02:19 smithi175 ceph-mon[67352]: osdmap e33: 5 total, 5 up, 5 in 2024-03-18T23:02:19.391 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:02:19 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-18T23:02:19.391 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:02:19 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:02:19.391 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:02:19 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:02:19.391 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:02:19 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:02:19.391 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:02:19 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:02:19.391 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:02:19 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:02:19.392 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:02:19 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:02:19.392 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:02:19 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:02:19.392 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:02:19 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:02:20.212 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:02:20 smithi175 ceph-mon[67352]: Detected new or changed devices on smithi139 2024-03-18T23:02:20.212 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:02:20 smithi175 ceph-mon[67352]: Adjusting osd_memory_target on smithi139 to 8540M 2024-03-18T23:02:20.212 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:02:20 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:02:20.212 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:02:20 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:02:20.212 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:02:20 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:02:20.212 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:02:20 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:02:20.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:02:20 smithi049 ceph-mon[62991]: Detected new or changed devices on smithi139 2024-03-18T23:02:20.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:02:20 smithi049 ceph-mon[62991]: Adjusting osd_memory_target on smithi139 to 8540M 2024-03-18T23:02:20.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:02:20 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:02:20.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:02:20 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:02:20.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:02:20 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:02:20.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:02:20 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:02:20.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:02:20 smithi139 ceph-mon[67282]: Detected new or changed devices on smithi139 2024-03-18T23:02:20.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:02:20 smithi139 ceph-mon[67282]: Adjusting osd_memory_target on smithi139 to 8540M 2024-03-18T23:02:20.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:02:20 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:02:20.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:02:20 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:02:20.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:02:20 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:02:20.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:02:20 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:02:20.776 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:02:20.773+0000 7fa38d191700 1 -- 172.21.15.175:0/1553911897 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa388103ed0 msgr2=0x7fa38810a780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:02:20.777 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:02:20.773+0000 7fa38d191700 1 --2- 172.21.15.175:0/1553911897 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa388103ed0 0x7fa38810a780 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fa38000b0a0 tx=0x7fa38000b3b0 comp rx=0 tx=0).stop 2024-03-18T23:02:20.777 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:02:20.773+0000 7fa38d191700 1 -- 172.21.15.175:0/1553911897 shutdown_connections 2024-03-18T23:02:20.777 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:02:20.773+0000 7fa38d191700 1 --2- 172.21.15.175:0/1553911897 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa388103ed0 0x7fa38810a780 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:02:20.777 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:02:20.773+0000 7fa38d191700 1 --2- 172.21.15.175:0/1553911897 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa388102330 0x7fa388102750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:02:20.777 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:02:20.773+0000 7fa38d191700 1 --2- 172.21.15.175:0/1553911897 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa388103530 0x7fa388103990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:02:20.777 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:02:20.773+0000 7fa38d191700 1 -- 172.21.15.175:0/1553911897 >> 172.21.15.175:0/1553911897 conn(0x7fa3880fd8f0 msgr2=0x7fa3880ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:02:20.777 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:02:20.773+0000 7fa38d191700 1 -- 172.21.15.175:0/1553911897 shutdown_connections 2024-03-18T23:02:20.777 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:02:20.774+0000 7fa38d191700 1 -- 172.21.15.175:0/1553911897 wait complete. 2024-03-18T23:02:20.777 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:02:20.774+0000 7fa38d191700 1 Processor -- start 2024-03-18T23:02:20.777 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:02:20.774+0000 7fa38d191700 1 -- start start 2024-03-18T23:02:20.777 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:02:20.774+0000 7fa38d191700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa388102330 0x7fa38819a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:02:20.778 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:02:20.775+0000 7fa38d191700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa388103530 0x7fa38819abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:02:20.778 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:02:20.775+0000 7fa38d191700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa388103ed0 0x7fa3881a1c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:02:20.778 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:02:20.775+0000 7fa38d191700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fa38810dfa0 con 0x7fa388102330 2024-03-18T23:02:20.778 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:02:20.775+0000 7fa38d191700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fa38810de20 con 0x7fa388103530 2024-03-18T23:02:20.778 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:02:20.775+0000 7fa38d191700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa38810e120 con 0x7fa388103ed0 2024-03-18T23:02:20.778 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:02:20.775+0000 7fa38759e700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa388103ed0 0x7fa3881a1c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:02:20.778 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:02:20.775+0000 7fa38759e700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa388103ed0 0x7fa3881a1c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:60042/0 (socket says 172.21.15.175:60042) 2024-03-18T23:02:20.778 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:02:20.775+0000 7fa38759e700 1 -- 172.21.15.175:0/2853070156 learned_addr learned my addr 172.21.15.175:0/2853070156 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-03-18T23:02:20.778 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:02:20.775+0000 7fa38659c700 1 --2- 172.21.15.175:0/2853070156 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa388103530 0x7fa38819abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:02:20.779 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:02:20.775+0000 7fa386d9d700 1 --2- 172.21.15.175:0/2853070156 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa388102330 0x7fa38819a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:02:20.779 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:02:20.775+0000 7fa38759e700 1 -- 172.21.15.175:0/2853070156 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa388103530 msgr2=0x7fa38819abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:02:20.779 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:02:20.775+0000 7fa38759e700 1 --2- 172.21.15.175:0/2853070156 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa388103530 0x7fa38819abf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:02:20.779 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:02:20.775+0000 7fa38759e700 1 -- 172.21.15.175:0/2853070156 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa388102330 msgr2=0x7fa38819a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:02:20.779 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:02:20.775+0000 7fa38759e700 1 --2- 172.21.15.175:0/2853070156 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa388102330 0x7fa38819a6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:02:20.779 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:02:20.775+0000 7fa38759e700 1 -- 172.21.15.175:0/2853070156 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3881a2310 con 0x7fa388103ed0 2024-03-18T23:02:20.779 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:02:20.775+0000 7fa386d9d700 1 --2- 172.21.15.175:0/2853070156 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa388102330 0x7fa38819a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T23:02:20.779 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:02:20.775+0000 7fa38759e700 1 --2- 172.21.15.175:0/2853070156 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa388103ed0 0x7fa3881a1c70 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fa38000be60 tx=0x7fa380007820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:02:20.779 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:02:20.775+0000 7fa36ffff700 1 -- 172.21.15.175:0/2853070156 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa380021020 con 0x7fa388103ed0 2024-03-18T23:02:20.779 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:02:20.775+0000 7fa38d191700 1 -- 172.21.15.175:0/2853070156 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3881a25a0 con 0x7fa388103ed0 2024-03-18T23:02:20.779 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:02:20.775+0000 7fa36ffff700 1 -- 172.21.15.175:0/2853070156 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa380005520 con 0x7fa388103ed0 2024-03-18T23:02:20.779 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:02:20.775+0000 7fa36ffff700 1 -- 172.21.15.175:0/2853070156 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa38001ba00 con 0x7fa388103ed0 2024-03-18T23:02:20.779 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:02:20.775+0000 7fa38d191700 1 -- 172.21.15.175:0/2853070156 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3881a2b20 con 0x7fa388103ed0 2024-03-18T23:02:20.780 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:02:20.776+0000 7fa38d191700 1 -- 172.21.15.175:0/2853070156 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa388068fd0 con 0x7fa388103ed0 2024-03-18T23:02:20.780 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:02:20.777+0000 7fa36ffff700 1 -- 172.21.15.175:0/2853070156 <== mon.1 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7fa380024020 con 0x7fa388103ed0 2024-03-18T23:02:20.781 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:02:20.778+0000 7fa36ffff700 1 --2- 172.21.15.175:0/2853070156 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fa3700a2890 0x7fa3700a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:02:20.781 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:02:20.778+0000 7fa36ffff700 1 -- 172.21.15.175:0/2853070156 <== mon.1 v2:172.21.15.175:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 4101+0+0 (secure 0 0 0) 0x7fa38000f020 con 0x7fa388103ed0 2024-03-18T23:02:20.781 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:02:20.778+0000 7fa386d9d700 1 --2- 172.21.15.175:0/2853070156 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fa3700a2890 0x7fa3700a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:02:20.781 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:02:20.779+0000 7fa386d9d700 1 --2- 172.21.15.175:0/2853070156 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fa3700a2890 0x7fa3700a4d50 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fa388103390 tx=0x7fa378008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:02:20.782 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:02:20.782+0000 7fa36ffff700 1 -- 172.21.15.175:0/2853070156 <== mon.1 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fa38009d240 con 0x7fa388103ed0 2024-03-18T23:02:20.985 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:02:20.984+0000 7fa38d191700 1 -- 172.21.15.175:0/2853070156 --> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi175:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fa388075c60 con 0x7fa3700a2890 2024-03-18T23:02:21.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:02:21 smithi175 ceph-mon[67352]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-03-18T23:02:21.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:02:21 smithi175 ceph-mon[67352]: osdmap e34: 5 total, 5 up, 5 in 2024-03-18T23:02:21.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:02:21 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-18T23:02:21.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:02:21 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-18T23:02:21.462 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:02:21 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:02:21.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:02:21 smithi139 ceph-mon[67282]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-03-18T23:02:21.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:02:21 smithi139 ceph-mon[67282]: osdmap e34: 5 total, 5 up, 5 in 2024-03-18T23:02:21.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:02:21 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-18T23:02:21.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:02:21 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-18T23:02:21.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:02:21 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:02:21.630 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:02:21 smithi049 ceph-mon[62991]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-03-18T23:02:21.630 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:02:21 smithi049 ceph-mon[62991]: osdmap e34: 5 total, 5 up, 5 in 2024-03-18T23:02:21.630 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:02:21 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-18T23:02:21.630 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:02:21 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-18T23:02:21.630 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:02:21 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:02:22.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:02:22 smithi139 ceph-mon[67282]: from='client.24214 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi175:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T23:02:22.579 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:02:22 smithi175 ceph-mon[67352]: from='client.24214 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi175:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T23:02:22.630 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:02:22 smithi049 ceph-mon[62991]: from='client.24214 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi175:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T23:02:23.533 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:02:23 smithi049 ceph-mon[62991]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-03-18T23:02:23.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:02:23 smithi139 ceph-mon[67282]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-03-18T23:02:23.712 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:02:23 smithi175 ceph-mon[67352]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-03-18T23:02:24.462 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:02:24 smithi175 ceph-mon[67352]: from='client.? 172.21.15.175:0/1694358240' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4b641f81-77eb-45fd-8464-511e7b3062b3"}]: dispatch 2024-03-18T23:02:24.638 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:02:24 smithi175 ceph-mon[67352]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4b641f81-77eb-45fd-8464-511e7b3062b3"}]: dispatch 2024-03-18T23:02:24.639 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:02:24 smithi175 ceph-mon[67352]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4b641f81-77eb-45fd-8464-511e7b3062b3"}]': finished 2024-03-18T23:02:24.639 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:02:24 smithi175 ceph-mon[67352]: osdmap e35: 6 total, 5 up, 6 in 2024-03-18T23:02:24.639 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:02:24 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-18T23:02:24.640 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:02:24 smithi139 ceph-mon[67282]: from='client.? 172.21.15.175:0/1694358240' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4b641f81-77eb-45fd-8464-511e7b3062b3"}]: dispatch 2024-03-18T23:02:24.640 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:02:24 smithi139 ceph-mon[67282]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4b641f81-77eb-45fd-8464-511e7b3062b3"}]: dispatch 2024-03-18T23:02:24.640 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:02:24 smithi139 ceph-mon[67282]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4b641f81-77eb-45fd-8464-511e7b3062b3"}]': finished 2024-03-18T23:02:24.640 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:02:24 smithi139 ceph-mon[67282]: osdmap e35: 6 total, 5 up, 6 in 2024-03-18T23:02:24.640 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:02:24 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-18T23:02:24.640 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:02:24 smithi049 ceph-mon[62991]: from='client.? 172.21.15.175:0/1694358240' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4b641f81-77eb-45fd-8464-511e7b3062b3"}]: dispatch 2024-03-18T23:02:24.640 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:02:24 smithi049 ceph-mon[62991]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4b641f81-77eb-45fd-8464-511e7b3062b3"}]: dispatch 2024-03-18T23:02:24.640 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:02:24 smithi049 ceph-mon[62991]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4b641f81-77eb-45fd-8464-511e7b3062b3"}]': finished 2024-03-18T23:02:24.641 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:02:24 smithi049 ceph-mon[62991]: osdmap e35: 6 total, 5 up, 6 in 2024-03-18T23:02:24.641 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:02:24 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-18T23:02:25.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:02:25 smithi139 ceph-mon[67282]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-03-18T23:02:25.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:02:25 smithi139 ceph-mon[67282]: from='client.? 172.21.15.175:0/3365144364' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-18T23:02:25.630 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:02:25 smithi049 ceph-mon[62991]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-03-18T23:02:25.630 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:02:25 smithi049 ceph-mon[62991]: from='client.? 172.21.15.175:0/3365144364' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-18T23:02:25.711 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:02:25 smithi175 ceph-mon[67352]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-03-18T23:02:25.711 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:02:25 smithi175 ceph-mon[67352]: from='client.? 172.21.15.175:0/3365144364' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-18T23:02:27.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:02:27 smithi139 ceph-mon[67282]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-03-18T23:02:27.630 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:02:27 smithi049 ceph-mon[62991]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-03-18T23:02:27.703 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:02:27 smithi175 ceph-mon[67352]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-03-18T23:02:29.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:02:29 smithi175 ceph-mon[67352]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-03-18T23:02:29.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:02:29 smithi139 ceph-mon[67282]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-03-18T23:02:29.630 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:02:29 smithi049 ceph-mon[62991]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-03-18T23:02:31.559 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:02:31 smithi175 ceph-mon[67352]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-03-18T23:02:31.559 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:02:31 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-18T23:02:31.559 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:02:31 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:02:31.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:02:31 smithi139 ceph-mon[67282]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-03-18T23:02:31.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:02:31 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-18T23:02:31.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:02:31 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:02:31.630 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:02:31 smithi049 ceph-mon[62991]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-03-18T23:02:31.630 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:02:31 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-18T23:02:31.630 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:02:31 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:02:32.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:02:32 smithi139 ceph-mon[67282]: Deploying daemon osd.5 on smithi175 2024-03-18T23:02:32.623 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:02:32 smithi175 ceph-mon[67352]: Deploying daemon osd.5 on smithi175 2024-03-18T23:02:32.630 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:02:32 smithi049 ceph-mon[62991]: Deploying daemon osd.5 on smithi175 2024-03-18T23:02:33.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:02:33 smithi139 ceph-mon[67282]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-03-18T23:02:33.630 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:02:33 smithi049 ceph-mon[62991]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-03-18T23:02:33.711 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:02:33 smithi175 ceph-mon[67352]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-03-18T23:02:35.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:02:35 smithi139 ceph-mon[67282]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-03-18T23:02:35.630 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:02:35 smithi049 ceph-mon[62991]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-03-18T23:02:35.711 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:02:35 smithi175 ceph-mon[67352]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-03-18T23:02:37.711 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:02:37 smithi175 ceph-mon[67352]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-03-18T23:02:37.711 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:02:37 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:02:37.712 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:02:37 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:02:37.712 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:02:37 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:02:38.065 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:02:37 smithi139 ceph-mon[67282]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-03-18T23:02:38.065 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:02:37 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:02:38.065 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:02:37 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:02:38.065 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:02:37 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:02:38.130 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:02:37 smithi049 ceph-mon[62991]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-03-18T23:02:38.130 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:02:37 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:02:38.130 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:02:37 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:02:38.130 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:02:37 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:02:39.764 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:02:39 smithi175 ceph-mon[67352]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-03-18T23:02:39.764 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:02:39 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:02:39.764 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:02:39 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:02:39.764 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:02:39 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:02:39.764 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:02:39 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:02:39.764 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:02:39 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:02:40.065 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:02:39 smithi139 ceph-mon[67282]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-03-18T23:02:40.065 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:02:39 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:02:40.065 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:02:39 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:02:40.066 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:02:39 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:02:40.066 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:02:39 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:02:40.066 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:02:39 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:02:40.130 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:02:39 smithi049 ceph-mon[62991]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-03-18T23:02:40.130 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:02:39 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:02:40.131 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:02:39 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:02:40.131 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:02:39 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:02:40.131 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:02:39 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:02:40.131 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:02:39 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:02:40.403 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:02:40.402+0000 7fa36ffff700 1 -- 172.21.15.175:0/2853070156 <== mgr.14150 v2:172.21.15.49:6800/2083805211 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fa388075c60 con 0x7fa3700a2890 2024-03-18T23:02:40.403 INFO:teuthology.orchestra.run.smithi175.stdout:Created osd(s) 5 on host 'smithi175' 2024-03-18T23:02:40.406 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:02:40.404+0000 7fa38d191700 1 -- 172.21.15.175:0/2853070156 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fa3700a2890 msgr2=0x7fa3700a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:02:40.406 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:02:40.404+0000 7fa38d191700 1 --2- 172.21.15.175:0/2853070156 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fa3700a2890 0x7fa3700a4d50 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fa388103390 tx=0x7fa378008040 comp rx=0 tx=0).stop 2024-03-18T23:02:40.407 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:02:40.404+0000 7fa38d191700 1 -- 172.21.15.175:0/2853070156 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa388103ed0 msgr2=0x7fa3881a1c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:02:40.407 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:02:40.404+0000 7fa38d191700 1 --2- 172.21.15.175:0/2853070156 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa388103ed0 0x7fa3881a1c70 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fa38000be60 tx=0x7fa380007820 comp rx=0 tx=0).stop 2024-03-18T23:02:40.407 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:02:40.404+0000 7fa38d191700 1 -- 172.21.15.175:0/2853070156 shutdown_connections 2024-03-18T23:02:40.407 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:02:40.404+0000 7fa38d191700 1 --2- 172.21.15.175:0/2853070156 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fa3700a2890 0x7fa3700a4d50 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:02:40.407 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:02:40.404+0000 7fa38d191700 1 --2- 172.21.15.175:0/2853070156 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa388103ed0 0x7fa3881a1c70 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:02:40.407 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:02:40.404+0000 7fa38d191700 1 --2- 172.21.15.175:0/2853070156 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa388102330 0x7fa38819a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:02:40.407 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:02:40.404+0000 7fa38d191700 1 --2- 172.21.15.175:0/2853070156 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa388103530 0x7fa38819abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:02:40.407 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:02:40.405+0000 7fa38d191700 1 -- 172.21.15.175:0/2853070156 >> 172.21.15.175:0/2853070156 conn(0x7fa3880fd8f0 msgr2=0x7fa388074540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:02:40.407 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:02:40.405+0000 7fa38d191700 1 -- 172.21.15.175:0/2853070156 shutdown_connections 2024-03-18T23:02:40.407 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:02:40.405+0000 7fa38d191700 1 -- 172.21.15.175:0/2853070156 wait complete. 2024-03-18T23:02:41.062 DEBUG:teuthology.orchestra.run.smithi175:osd.5> sudo journalctl -f -n 0 -u ceph-c63591ea-e57a-11ee-95c9-87774f69a715@osd.5.service 2024-03-18T23:02:41.065 INFO:tasks.cephadm:Deploying osd.6 on smithi049 with /dev/vg_nvme/lv_2... 2024-03-18T23:02:41.065 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c63591ea-e57a-11ee-95c9-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-03-18T23:02:41.327 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T23:02:41.616 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:02:41 smithi049 ceph-mon[62991]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-03-18T23:02:41.617 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:02:41 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:02:41.617 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:02:41 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:02:41.617 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:02:41 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:02:41.617 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:02:41 smithi049 ceph-mon[62991]: from='osd.5 [v2:172.21.15.175:6808/2774567215,v1:172.21.15.175:6809/2774567215]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-18T23:02:41.617 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:02:41 smithi049 ceph-mon[62991]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-18T23:02:41.639 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:02:41 smithi175 ceph-mon[67352]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-03-18T23:02:41.639 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:02:41 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:02:41.639 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:02:41 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:02:41.639 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:02:41 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:02:41.639 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:02:41 smithi175 ceph-mon[67352]: from='osd.5 [v2:172.21.15.175:6808/2774567215,v1:172.21.15.175:6809/2774567215]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-18T23:02:41.639 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:02:41 smithi175 ceph-mon[67352]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-18T23:02:41.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:02:41 smithi139 ceph-mon[67282]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-03-18T23:02:41.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:02:41 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:02:41.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:02:41 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:02:41.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:02:41 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:02:41.816 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:02:41 smithi139 ceph-mon[67282]: from='osd.5 [v2:172.21.15.175:6808/2774567215,v1:172.21.15.175:6809/2774567215]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-18T23:02:41.816 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:02:41 smithi139 ceph-mon[67282]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-18T23:02:42.607 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:02:42 smithi175 ceph-mon[67352]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-03-18T23:02:42.607 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:02:42 smithi175 ceph-mon[67352]: from='osd.5 [v2:172.21.15.175:6808/2774567215,v1:172.21.15.175:6809/2774567215]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]: dispatch 2024-03-18T23:02:42.607 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:02:42 smithi175 ceph-mon[67352]: osdmap e36: 6 total, 5 up, 6 in 2024-03-18T23:02:42.607 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:02:42 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-18T23:02:42.607 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:02:42 smithi175 ceph-mon[67352]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]: dispatch 2024-03-18T23:02:42.622 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:02:42 smithi049 ceph-mon[62991]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-03-18T23:02:42.622 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:02:42 smithi049 ceph-mon[62991]: from='osd.5 [v2:172.21.15.175:6808/2774567215,v1:172.21.15.175:6809/2774567215]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]: dispatch 2024-03-18T23:02:42.622 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:02:42 smithi049 ceph-mon[62991]: osdmap e36: 6 total, 5 up, 6 in 2024-03-18T23:02:42.622 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:02:42 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-18T23:02:42.622 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:02:42 smithi049 ceph-mon[62991]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]: dispatch 2024-03-18T23:02:42.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:02:42 smithi139 ceph-mon[67282]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-03-18T23:02:42.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:02:42 smithi139 ceph-mon[67282]: from='osd.5 [v2:172.21.15.175:6808/2774567215,v1:172.21.15.175:6809/2774567215]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]: dispatch 2024-03-18T23:02:42.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:02:42 smithi139 ceph-mon[67282]: osdmap e36: 6 total, 5 up, 6 in 2024-03-18T23:02:42.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:02:42 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-18T23:02:42.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:02:42 smithi139 ceph-mon[67282]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]: dispatch 2024-03-18T23:02:42.961 INFO:journalctl@ceph.osd.5.smithi175.stdout:Mar 18 23:02:42 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-5[79436]: 2024-03-18T23:02:42.807+0000 7fa76bd19700 -1 osd.5 0 waiting for initial osdmap 2024-03-18T23:02:42.962 INFO:journalctl@ceph.osd.5.smithi175.stdout:Mar 18 23:02:42 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-5[79436]: 2024-03-18T23:02:42.813+0000 7fa763303700 -1 osd.5 37 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-18T23:02:43.294 INFO:teuthology.orchestra.run.smithi049.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-03-18T23:02:43.294 INFO:teuthology.orchestra.run.smithi049.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-03-18T23:02:43.294 INFO:teuthology.orchestra.run.smithi049.stderr: stderr: 10+0 records in 2024-03-18T23:02:43.294 INFO:teuthology.orchestra.run.smithi049.stderr:10+0 records out 2024-03-18T23:02:43.295 INFO:teuthology.orchestra.run.smithi049.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00860817 s, 1.2 GB/s 2024-03-18T23:02:43.295 INFO:teuthology.orchestra.run.smithi049.stderr:--> Zapping successful for: 2024-03-18T23:02:43.613 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:02:43 smithi049 ceph-mon[62991]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-03-18T23:02:43.613 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:02:43 smithi049 ceph-mon[62991]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]': finished 2024-03-18T23:02:43.613 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:02:43 smithi049 ceph-mon[62991]: osdmap e37: 6 total, 5 up, 6 in 2024-03-18T23:02:43.613 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:02:43 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-18T23:02:43.614 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:02:43 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-18T23:02:43.614 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:02:43 smithi049 ceph-mon[62991]: from='osd.5 ' entity='osd.5' 2024-03-18T23:02:43.711 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:02:43 smithi175 ceph-mon[67352]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-03-18T23:02:43.712 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:02:43 smithi175 ceph-mon[67352]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]': finished 2024-03-18T23:02:43.712 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:02:43 smithi175 ceph-mon[67352]: osdmap e37: 6 total, 5 up, 6 in 2024-03-18T23:02:43.712 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:02:43 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-18T23:02:43.712 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:02:43 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-18T23:02:43.712 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:02:43 smithi175 ceph-mon[67352]: from='osd.5 ' entity='osd.5' 2024-03-18T23:02:43.716 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:02:43 smithi139 ceph-mon[67282]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-03-18T23:02:43.716 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:02:43 smithi139 ceph-mon[67282]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]': finished 2024-03-18T23:02:43.716 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:02:43 smithi139 ceph-mon[67282]: osdmap e37: 6 total, 5 up, 6 in 2024-03-18T23:02:43.716 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:02:43 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-18T23:02:43.717 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:02:43 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-18T23:02:43.717 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:02:43 smithi139 ceph-mon[67282]: from='osd.5 ' entity='osd.5' 2024-03-18T23:02:43.925 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c63591ea-e57a-11ee-95c9-87774f69a715 -- ceph orch daemon add osd smithi049:vg_nvme/lv_2 2024-03-18T23:02:44.187 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T23:02:44.458 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:02:44 smithi139 ceph-mon[67282]: purged_snaps scrub starts 2024-03-18T23:02:44.459 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:02:44 smithi139 ceph-mon[67282]: purged_snaps scrub ok 2024-03-18T23:02:44.459 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:02:44 smithi139 ceph-mon[67282]: osd.5 [v2:172.21.15.175:6808/2774567215,v1:172.21.15.175:6809/2774567215] boot 2024-03-18T23:02:44.459 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:02:44 smithi139 ceph-mon[67282]: osdmap e38: 6 total, 6 up, 6 in 2024-03-18T23:02:44.459 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:02:44 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-18T23:02:44.459 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:02:44 smithi139 ceph-mon[67282]: osdmap e39: 6 total, 6 up, 6 in 2024-03-18T23:02:44.459 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:02:44 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:02:44.459 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:02:44 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:02:44.459 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:02:44 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:02:44.459 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:02:44 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:02:44.459 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:02:44 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:02:44.459 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:02:44 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:02:44.459 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:02:44 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:02:44.459 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:02:44 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:02:44.617 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:02:44 smithi049 ceph-mon[62991]: purged_snaps scrub starts 2024-03-18T23:02:44.617 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:02:44 smithi049 ceph-mon[62991]: purged_snaps scrub ok 2024-03-18T23:02:44.618 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:02:44 smithi049 ceph-mon[62991]: osd.5 [v2:172.21.15.175:6808/2774567215,v1:172.21.15.175:6809/2774567215] boot 2024-03-18T23:02:44.618 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:02:44 smithi049 ceph-mon[62991]: osdmap e38: 6 total, 6 up, 6 in 2024-03-18T23:02:44.618 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:02:44 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-18T23:02:44.618 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:02:44 smithi049 ceph-mon[62991]: osdmap e39: 6 total, 6 up, 6 in 2024-03-18T23:02:44.618 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:02:44 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:02:44.618 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:02:44 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:02:44.618 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:02:44 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:02:44.618 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:02:44 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:02:44.618 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:02:44 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:02:44.618 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:02:44 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:02:44.618 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:02:44 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:02:44.618 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:02:44 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:02:44.711 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:02:44 smithi175 ceph-mon[67352]: purged_snaps scrub starts 2024-03-18T23:02:44.712 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:02:44 smithi175 ceph-mon[67352]: purged_snaps scrub ok 2024-03-18T23:02:44.712 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:02:44 smithi175 ceph-mon[67352]: osd.5 [v2:172.21.15.175:6808/2774567215,v1:172.21.15.175:6809/2774567215] boot 2024-03-18T23:02:44.712 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:02:44 smithi175 ceph-mon[67352]: osdmap e38: 6 total, 6 up, 6 in 2024-03-18T23:02:44.712 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:02:44 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-18T23:02:44.712 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:02:44 smithi175 ceph-mon[67352]: osdmap e39: 6 total, 6 up, 6 in 2024-03-18T23:02:44.712 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:02:44 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:02:44.712 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:02:44 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:02:44.712 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:02:44 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:02:44.712 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:02:44 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:02:44.712 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:02:44 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:02:44.712 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:02:44 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:02:44.712 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:02:44 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:02:44.712 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:02:44 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:02:45.754 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:02:45.751+0000 7f4d37796700 1 -- 172.21.15.49:0/4103795563 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4d30102e80 msgr2=0x7f4d30109730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:02:45.754 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:02:45.751+0000 7f4d37796700 1 --2- 172.21.15.49:0/4103795563 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4d30102e80 0x7f4d30109730 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f4d2800b0d0 tx=0x7f4d2800b3e0 comp rx=0 tx=0).stop 2024-03-18T23:02:45.754 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:02:45.751+0000 7f4d37796700 1 -- 172.21.15.49:0/4103795563 shutdown_connections 2024-03-18T23:02:45.754 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:02:45.751+0000 7f4d37796700 1 --2- 172.21.15.49:0/4103795563 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4d30102e80 0x7f4d30109730 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:02:45.754 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:02:45.751+0000 7f4d37796700 1 --2- 172.21.15.49:0/4103795563 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4d301012e0 0x7f4d30101700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:02:45.754 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:02:45.751+0000 7f4d37796700 1 --2- 172.21.15.49:0/4103795563 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4d301024e0 0x7f4d30102940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:02:45.754 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:02:45.751+0000 7f4d37796700 1 -- 172.21.15.49:0/4103795563 >> 172.21.15.49:0/4103795563 conn(0x7f4d300fc8a0 msgr2=0x7f4d300fecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:02:45.754 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:02:45.752+0000 7f4d37796700 1 -- 172.21.15.49:0/4103795563 shutdown_connections 2024-03-18T23:02:45.755 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:02:45.752+0000 7f4d37796700 1 -- 172.21.15.49:0/4103795563 wait complete. 2024-03-18T23:02:45.755 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:02:45.752+0000 7f4d37796700 1 Processor -- start 2024-03-18T23:02:45.755 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:02:45.752+0000 7f4d37796700 1 -- start start 2024-03-18T23:02:45.755 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:02:45.752+0000 7f4d37796700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4d301012e0 0x7f4d30105270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:02:45.755 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:02:45.752+0000 7f4d37796700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4d301024e0 0x7f4d301038c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:02:45.755 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:02:45.752+0000 7f4d37796700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4d30102e80 0x7f4d30103e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:02:45.755 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:02:45.752+0000 7f4d37796700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f4d3010cf50 con 0x7f4d301012e0 2024-03-18T23:02:45.755 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:02:45.752+0000 7f4d37796700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f4d3010cdd0 con 0x7f4d301024e0 2024-03-18T23:02:45.755 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:02:45.752+0000 7f4d37796700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f4d3010d0d0 con 0x7f4d30102e80 2024-03-18T23:02:45.755 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:02:45.753+0000 7f4d35532700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4d301012e0 0x7f4d30105270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:02:45.755 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:02:45.753+0000 7f4d35532700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4d301012e0 0x7f4d30105270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:54660/0 (socket says 172.21.15.49:54660) 2024-03-18T23:02:45.755 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:02:45.753+0000 7f4d35532700 1 -- 172.21.15.49:0/626000582 learned_addr learned my addr 172.21.15.49:0/626000582 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:02:45.755 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:02:45.753+0000 7f4d34d31700 1 --2- 172.21.15.49:0/626000582 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4d301024e0 0x7f4d301038c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:02:45.756 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:02:45.753+0000 7f4d35d33700 1 --2- 172.21.15.49:0/626000582 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4d30102e80 0x7f4d30103e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:02:45.756 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:02:45.753+0000 7f4d35532700 1 -- 172.21.15.49:0/626000582 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4d30102e80 msgr2=0x7f4d30103e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:02:45.756 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:02:45.753+0000 7f4d35532700 1 --2- 172.21.15.49:0/626000582 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4d30102e80 0x7f4d30103e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:02:45.756 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:02:45.753+0000 7f4d35532700 1 -- 172.21.15.49:0/626000582 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4d301024e0 msgr2=0x7f4d301038c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:02:45.756 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:02:45.753+0000 7f4d35532700 1 --2- 172.21.15.49:0/626000582 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4d301024e0 0x7f4d301038c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:02:45.756 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:02:45.753+0000 7f4d35532700 1 -- 172.21.15.49:0/626000582 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d301046b0 con 0x7f4d301012e0 2024-03-18T23:02:45.756 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:02:45.753+0000 7f4d35532700 1 --2- 172.21.15.49:0/626000582 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4d301012e0 0x7f4d30105270 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f4d2400b500 tx=0x7f4d2400b810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:02:45.756 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:02:45.753+0000 7f4d227fc700 1 -- 172.21.15.49:0/626000582 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d24015020 con 0x7f4d301012e0 2024-03-18T23:02:45.756 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:02:45.753+0000 7f4d227fc700 1 -- 172.21.15.49:0/626000582 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d24011d10 con 0x7f4d301012e0 2024-03-18T23:02:45.756 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:02:45.753+0000 7f4d37796700 1 -- 172.21.15.49:0/626000582 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d301a27f0 con 0x7f4d301012e0 2024-03-18T23:02:45.756 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:02:45.753+0000 7f4d227fc700 1 -- 172.21.15.49:0/626000582 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d24010db0 con 0x7f4d301012e0 2024-03-18T23:02:45.756 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:02:45.753+0000 7f4d37796700 1 -- 172.21.15.49:0/626000582 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d301a2d80 con 0x7f4d301012e0 2024-03-18T23:02:45.757 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:02:45.754+0000 7f4d37796700 1 -- 172.21.15.49:0/626000582 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d30068fd0 con 0x7f4d301012e0 2024-03-18T23:02:45.757 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:02:45.756+0000 7f4d227fc700 1 -- 172.21.15.49:0/626000582 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7f4d24018020 con 0x7f4d301012e0 2024-03-18T23:02:45.757 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:02:45.756+0000 7f4d227fc700 1 --2- 172.21.15.49:0/626000582 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f4d1c0a2890 0x7f4d1c0a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:02:45.757 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:02:45.756+0000 7f4d34d31700 1 --2- 172.21.15.49:0/626000582 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f4d1c0a2890 0x7f4d1c0a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:02:45.758 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:02:45.756+0000 7f4d227fc700 1 -- 172.21.15.49:0/626000582 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 4533+0+0 (secure 0 0 0) 0x7f4d240cd7c0 con 0x7f4d301012e0 2024-03-18T23:02:45.758 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:02:45.757+0000 7f4d34d31700 1 --2- 172.21.15.49:0/626000582 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f4d1c0a2890 0x7f4d1c0a4d50 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f4d2c009a10 tx=0x7f4d2c009190 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:02:45.759 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:02:45.760+0000 7f4d227fc700 1 -- 172.21.15.49:0/626000582 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f4d2401f020 con 0x7f4d301012e0 2024-03-18T23:02:45.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:02:45 smithi049 ceph-mon[62991]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 536 GiB / 536 GiB avail 2024-03-18T23:02:45.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:02:45 smithi049 ceph-mon[62991]: Detected new or changed devices on smithi175 2024-03-18T23:02:45.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:02:45 smithi049 ceph-mon[62991]: Adjusting osd_memory_target on smithi175 to 8540M 2024-03-18T23:02:45.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:02:45 smithi049 ceph-mon[62991]: osdmap e40: 6 total, 6 up, 6 in 2024-03-18T23:02:45.961 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:02:45 smithi175 ceph-mon[67352]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 536 GiB / 536 GiB avail 2024-03-18T23:02:45.962 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:02:45 smithi175 ceph-mon[67352]: Detected new or changed devices on smithi175 2024-03-18T23:02:45.962 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:02:45 smithi175 ceph-mon[67352]: Adjusting osd_memory_target on smithi175 to 8540M 2024-03-18T23:02:45.962 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:02:45 smithi175 ceph-mon[67352]: osdmap e40: 6 total, 6 up, 6 in 2024-03-18T23:02:45.975 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:02:45.975+0000 7f4d37796700 1 -- 172.21.15.49:0/626000582 --> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi049:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f4d301a2fe0 con 0x7f4d1c0a2890 2024-03-18T23:02:46.065 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:02:45 smithi139 ceph-mon[67282]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 536 GiB / 536 GiB avail 2024-03-18T23:02:46.065 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:02:45 smithi139 ceph-mon[67282]: Detected new or changed devices on smithi175 2024-03-18T23:02:46.065 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:02:45 smithi139 ceph-mon[67282]: Adjusting osd_memory_target on smithi175 to 8540M 2024-03-18T23:02:46.065 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:02:45 smithi139 ceph-mon[67282]: osdmap e40: 6 total, 6 up, 6 in 2024-03-18T23:02:46.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:02:46 smithi049 ceph-mon[62991]: osdmap e41: 6 total, 6 up, 6 in 2024-03-18T23:02:46.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:02:46 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-18T23:02:46.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:02:46 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-18T23:02:46.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:02:46 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:02:47.065 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:02:46 smithi139 ceph-mon[67282]: osdmap e41: 6 total, 6 up, 6 in 2024-03-18T23:02:47.065 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:02:46 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-18T23:02:47.065 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:02:46 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-18T23:02:47.065 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:02:46 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:02:47.212 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:02:46 smithi175 ceph-mon[67352]: osdmap e41: 6 total, 6 up, 6 in 2024-03-18T23:02:47.212 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:02:46 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-18T23:02:47.212 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:02:46 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-18T23:02:47.212 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:02:46 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:02:47.841 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:02:47 smithi049 ceph-mon[62991]: pgmap v144: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 206 MiB used, 536 GiB / 536 GiB avail; 2/6 objects degraded (33.333%) 2024-03-18T23:02:47.841 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:02:47 smithi049 ceph-mon[62991]: from='client.14352 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi049:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T23:02:47.841 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:02:47 smithi049 ceph-mon[62991]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2024-03-18T23:02:47.841 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:02:47 smithi049 ceph-mon[62991]: osdmap e42: 6 total, 6 up, 6 in 2024-03-18T23:02:48.065 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:02:47 smithi139 ceph-mon[67282]: pgmap v144: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 206 MiB used, 536 GiB / 536 GiB avail; 2/6 objects degraded (33.333%) 2024-03-18T23:02:48.065 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:02:47 smithi139 ceph-mon[67282]: from='client.14352 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi049:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T23:02:48.065 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:02:47 smithi139 ceph-mon[67282]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2024-03-18T23:02:48.065 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:02:47 smithi139 ceph-mon[67282]: osdmap e42: 6 total, 6 up, 6 in 2024-03-18T23:02:48.211 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:02:47 smithi175 ceph-mon[67352]: pgmap v144: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 206 MiB used, 536 GiB / 536 GiB avail; 2/6 objects degraded (33.333%) 2024-03-18T23:02:48.211 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:02:47 smithi175 ceph-mon[67352]: from='client.14352 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi049:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T23:02:48.212 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:02:47 smithi175 ceph-mon[67352]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2024-03-18T23:02:48.212 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:02:47 smithi175 ceph-mon[67352]: osdmap e42: 6 total, 6 up, 6 in 2024-03-18T23:02:50.065 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:02:49 smithi139 ceph-mon[67282]: pgmap v146: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 206 MiB used, 536 GiB / 536 GiB avail; 2/6 objects degraded (33.333%) 2024-03-18T23:02:50.065 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:02:49 smithi139 ceph-mon[67282]: from='client.? 172.21.15.49:0/100835227' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "653da8e0-1197-4b48-abed-470f2a9c4dce"}]: dispatch 2024-03-18T23:02:50.065 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:02:49 smithi139 ceph-mon[67282]: from='client.? 172.21.15.49:0/100835227' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "653da8e0-1197-4b48-abed-470f2a9c4dce"}]': finished 2024-03-18T23:02:50.065 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:02:49 smithi139 ceph-mon[67282]: osdmap e43: 7 total, 6 up, 7 in 2024-03-18T23:02:50.065 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:02:49 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-18T23:02:50.065 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:02:49 smithi139 ceph-mon[67282]: from='client.? 172.21.15.49:0/712501099' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-18T23:02:50.130 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:02:49 smithi049 ceph-mon[62991]: pgmap v146: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 206 MiB used, 536 GiB / 536 GiB avail; 2/6 objects degraded (33.333%) 2024-03-18T23:02:50.130 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:02:49 smithi049 ceph-mon[62991]: from='client.? 172.21.15.49:0/100835227' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "653da8e0-1197-4b48-abed-470f2a9c4dce"}]: dispatch 2024-03-18T23:02:50.130 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:02:49 smithi049 ceph-mon[62991]: from='client.? 172.21.15.49:0/100835227' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "653da8e0-1197-4b48-abed-470f2a9c4dce"}]': finished 2024-03-18T23:02:50.130 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:02:49 smithi049 ceph-mon[62991]: osdmap e43: 7 total, 6 up, 7 in 2024-03-18T23:02:50.130 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:02:49 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-18T23:02:50.130 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:02:49 smithi049 ceph-mon[62991]: from='client.? 172.21.15.49:0/712501099' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-18T23:02:50.211 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:02:49 smithi175 ceph-mon[67352]: pgmap v146: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 206 MiB used, 536 GiB / 536 GiB avail; 2/6 objects degraded (33.333%) 2024-03-18T23:02:50.211 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:02:49 smithi175 ceph-mon[67352]: from='client.? 172.21.15.49:0/100835227' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "653da8e0-1197-4b48-abed-470f2a9c4dce"}]: dispatch 2024-03-18T23:02:50.211 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:02:49 smithi175 ceph-mon[67352]: from='client.? 172.21.15.49:0/100835227' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "653da8e0-1197-4b48-abed-470f2a9c4dce"}]': finished 2024-03-18T23:02:50.211 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:02:49 smithi175 ceph-mon[67352]: osdmap e43: 7 total, 6 up, 7 in 2024-03-18T23:02:50.211 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:02:49 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-18T23:02:50.212 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:02:49 smithi175 ceph-mon[67352]: from='client.? 172.21.15.49:0/712501099' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-18T23:02:52.065 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:02:51 smithi139 ceph-mon[67282]: pgmap v148: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 2/6 objects degraded (33.333%) 2024-03-18T23:02:52.130 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:02:51 smithi049 ceph-mon[62991]: pgmap v148: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 2/6 objects degraded (33.333%) 2024-03-18T23:02:52.211 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:02:51 smithi175 ceph-mon[67352]: pgmap v148: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 2/6 objects degraded (33.333%) 2024-03-18T23:02:53.065 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:02:52 smithi139 ceph-mon[67282]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 94 KiB/s, 0 objects/s recovering 2024-03-18T23:02:53.065 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:02:52 smithi139 ceph-mon[67282]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2024-03-18T23:02:53.065 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:02:52 smithi139 ceph-mon[67282]: Cluster is now healthy 2024-03-18T23:02:53.130 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:02:52 smithi049 ceph-mon[62991]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 94 KiB/s, 0 objects/s recovering 2024-03-18T23:02:53.130 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:02:52 smithi049 ceph-mon[62991]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2024-03-18T23:02:53.130 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:02:52 smithi049 ceph-mon[62991]: Cluster is now healthy 2024-03-18T23:02:53.211 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:02:52 smithi175 ceph-mon[67352]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 94 KiB/s, 0 objects/s recovering 2024-03-18T23:02:53.211 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:02:52 smithi175 ceph-mon[67352]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2024-03-18T23:02:53.212 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:02:52 smithi175 ceph-mon[67352]: Cluster is now healthy 2024-03-18T23:02:55.130 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:02:54 smithi049 ceph-mon[62991]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-18T23:02:55.211 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:02:54 smithi175 ceph-mon[67352]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-18T23:02:55.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:02:54 smithi139 ceph-mon[67282]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-18T23:02:57.185 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:02:56 smithi049 ceph-mon[62991]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-03-18T23:02:57.185 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:02:56 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-03-18T23:02:57.185 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:02:56 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:02:57.185 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:02:56 smithi049 ceph-mon[62991]: Deploying daemon osd.6 on smithi049 2024-03-18T23:02:57.211 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:02:56 smithi175 ceph-mon[67352]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-03-18T23:02:57.211 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:02:56 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-03-18T23:02:57.212 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:02:56 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:02:57.212 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:02:56 smithi175 ceph-mon[67352]: Deploying daemon osd.6 on smithi049 2024-03-18T23:02:57.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:02:56 smithi139 ceph-mon[67282]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-03-18T23:02:57.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:02:56 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-03-18T23:02:57.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:02:56 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:02:57.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:02:56 smithi139 ceph-mon[67282]: Deploying daemon osd.6 on smithi049 2024-03-18T23:02:59.211 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:02:58 smithi175 ceph-mon[67352]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-18T23:02:59.297 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:02:58 smithi049 ceph-mon[62991]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-18T23:02:59.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:02:58 smithi139 ceph-mon[67282]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-18T23:03:01.130 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:03:00 smithi049 ceph-mon[62991]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-03-18T23:03:01.211 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:03:00 smithi175 ceph-mon[67352]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-03-18T23:03:01.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:03:00 smithi139 ceph-mon[67282]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-03-18T23:03:03.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:03:03 smithi049 ceph-mon[62991]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-03-18T23:03:03.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:03:03 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:03:03.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:03:03 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:03:03.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:03:03 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:03:03.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:03:03 smithi175 ceph-mon[67352]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-03-18T23:03:03.462 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:03:03 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:03:03.462 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:03:03 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:03:03.462 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:03:03 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:03:03.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:03:03 smithi139 ceph-mon[67282]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-03-18T23:03:03.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:03:03 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:03:03.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:03:03 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:03:03.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:03:03 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:03:05.325 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:03:05 smithi049 ceph-mon[62991]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-03-18T23:03:05.325 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:03:05 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:03:05.326 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:03:05 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:03:05.326 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:03:05 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:03:05.326 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:03:05 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:03:05.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:03:05 smithi175 ceph-mon[67352]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-03-18T23:03:05.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:03:05 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:03:05.462 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:03:05 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:03:05.462 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:03:05 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:03:05.462 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:03:05 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:03:05.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:03:05 smithi139 ceph-mon[67282]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-03-18T23:03:05.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:03:05 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:03:05.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:03:05 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:03:05.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:03:05 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:03:05.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:03:05 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:03:06.196 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:03:06.194+0000 7f4d227fc700 1 -- 172.21.15.49:0/626000582 <== mgr.14150 v2:172.21.15.49:6800/2083805211 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f4d301a2fe0 con 0x7f4d1c0a2890 2024-03-18T23:03:06.196 INFO:teuthology.orchestra.run.smithi049.stdout:Created osd(s) 6 on host 'smithi049' 2024-03-18T23:03:06.198 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:03:06.197+0000 7f4d37796700 1 -- 172.21.15.49:0/626000582 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f4d1c0a2890 msgr2=0x7f4d1c0a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:03:06.198 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:03:06.197+0000 7f4d37796700 1 --2- 172.21.15.49:0/626000582 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f4d1c0a2890 0x7f4d1c0a4d50 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f4d2c009a10 tx=0x7f4d2c009190 comp rx=0 tx=0).stop 2024-03-18T23:03:06.198 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:03:06.197+0000 7f4d37796700 1 -- 172.21.15.49:0/626000582 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4d301012e0 msgr2=0x7f4d30105270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:03:06.199 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:03:06.197+0000 7f4d37796700 1 --2- 172.21.15.49:0/626000582 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4d301012e0 0x7f4d30105270 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f4d2400b500 tx=0x7f4d2400b810 comp rx=0 tx=0).stop 2024-03-18T23:03:06.199 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:03:06.197+0000 7f4d37796700 1 -- 172.21.15.49:0/626000582 shutdown_connections 2024-03-18T23:03:06.199 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:03:06.197+0000 7f4d37796700 1 --2- 172.21.15.49:0/626000582 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f4d1c0a2890 0x7f4d1c0a4d50 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:03:06.199 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:03:06.197+0000 7f4d37796700 1 --2- 172.21.15.49:0/626000582 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4d30102e80 0x7f4d30103e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:03:06.199 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:03:06.197+0000 7f4d37796700 1 --2- 172.21.15.49:0/626000582 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4d301012e0 0x7f4d30105270 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:03:06.199 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:03:06.197+0000 7f4d37796700 1 --2- 172.21.15.49:0/626000582 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4d301024e0 0x7f4d301038c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:03:06.199 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:03:06.197+0000 7f4d37796700 1 -- 172.21.15.49:0/626000582 >> 172.21.15.49:0/626000582 conn(0x7f4d300fc8a0 msgr2=0x7f4d300fe340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:03:06.199 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:03:06.197+0000 7f4d37796700 1 -- 172.21.15.49:0/626000582 shutdown_connections 2024-03-18T23:03:06.199 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:03:06.197+0000 7f4d37796700 1 -- 172.21.15.49:0/626000582 wait complete. 2024-03-18T23:03:06.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:03:06 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:03:06.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:03:06 smithi049 ceph-mon[62991]: from='osd.6 [v2:172.21.15.49:6818/1472215461,v1:172.21.15.49:6819/1472215461]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-03-18T23:03:06.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:03:06 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:03:06.462 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:03:06 smithi175 ceph-mon[67352]: from='osd.6 [v2:172.21.15.49:6818/1472215461,v1:172.21.15.49:6819/1472215461]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-03-18T23:03:06.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:03:06 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:03:06.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:03:06 smithi139 ceph-mon[67282]: from='osd.6 [v2:172.21.15.49:6818/1472215461,v1:172.21.15.49:6819/1472215461]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-03-18T23:03:06.999 DEBUG:teuthology.orchestra.run.smithi049:osd.6> sudo journalctl -f -n 0 -u ceph-c63591ea-e57a-11ee-95c9-87774f69a715@osd.6.service 2024-03-18T23:03:07.002 INFO:tasks.cephadm:Deploying osd.7 on smithi139 with /dev/vg_nvme/lv_2... 2024-03-18T23:03:07.002 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c63591ea-e57a-11ee-95c9-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-03-18T23:03:07.227 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:03:07 smithi049 ceph-mon[62991]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-03-18T23:03:07.227 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:03:07 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:03:07.227 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:03:07 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:03:07.227 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:03:07 smithi049 ceph-mon[62991]: from='osd.6 [v2:172.21.15.49:6818/1472215461,v1:172.21.15.49:6819/1472215461]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-03-18T23:03:07.227 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:03:07 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:03:07.227 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:03:07 smithi049 ceph-mon[62991]: osdmap e44: 7 total, 6 up, 7 in 2024-03-18T23:03:07.227 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:03:07 smithi049 ceph-mon[62991]: from='osd.6 [v2:172.21.15.49:6818/1472215461,v1:172.21.15.49:6819/1472215461]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi049", "root=default"]}]: dispatch 2024-03-18T23:03:07.227 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:03:07 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-18T23:03:07.247 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/mon.b/config 2024-03-18T23:03:07.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:03:07 smithi139 ceph-mon[67282]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-03-18T23:03:07.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:03:07 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:03:07.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:03:07 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:03:07.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:03:07 smithi139 ceph-mon[67282]: from='osd.6 [v2:172.21.15.49:6818/1472215461,v1:172.21.15.49:6819/1472215461]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-03-18T23:03:07.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:03:07 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:03:07.316 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:03:07 smithi139 ceph-mon[67282]: osdmap e44: 7 total, 6 up, 7 in 2024-03-18T23:03:07.316 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:03:07 smithi139 ceph-mon[67282]: from='osd.6 [v2:172.21.15.49:6818/1472215461,v1:172.21.15.49:6819/1472215461]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi049", "root=default"]}]: dispatch 2024-03-18T23:03:07.316 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:03:07 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-18T23:03:07.462 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:03:07 smithi175 ceph-mon[67352]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-03-18T23:03:07.462 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:03:07 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:03:07.462 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:03:07 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:03:07.462 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:03:07 smithi175 ceph-mon[67352]: from='osd.6 [v2:172.21.15.49:6818/1472215461,v1:172.21.15.49:6819/1472215461]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-03-18T23:03:07.462 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:03:07 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:03:07.462 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:03:07 smithi175 ceph-mon[67352]: osdmap e44: 7 total, 6 up, 7 in 2024-03-18T23:03:07.462 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:03:07 smithi175 ceph-mon[67352]: from='osd.6 [v2:172.21.15.49:6818/1472215461,v1:172.21.15.49:6819/1472215461]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi049", "root=default"]}]: dispatch 2024-03-18T23:03:07.462 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:03:07 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-18T23:03:07.880 INFO:journalctl@ceph.osd.6.smithi049.stdout:Mar 18 23:03:07 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-6[91523]: 2024-03-18T23:03:07.612+0000 7f92e24a8700 -1 osd.6 0 waiting for initial osdmap 2024-03-18T23:03:07.880 INFO:journalctl@ceph.osd.6.smithi049.stdout:Mar 18 23:03:07 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-6[91523]: 2024-03-18T23:03:07.621+0000 7f92da293700 -1 osd.6 45 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-18T23:03:08.358 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:03:08 smithi049 ceph-mon[62991]: from='osd.6 [v2:172.21.15.49:6818/1472215461,v1:172.21.15.49:6819/1472215461]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi049", "root=default"]}]': finished 2024-03-18T23:03:08.359 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:03:08 smithi049 ceph-mon[62991]: osdmap e45: 7 total, 6 up, 7 in 2024-03-18T23:03:08.359 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:03:08 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-18T23:03:08.359 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:03:08 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-18T23:03:08.359 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:03:08 smithi049 ceph-mon[62991]: from='osd.6 [v2:172.21.15.49:6818/1472215461,v1:172.21.15.49:6819/1472215461]' entity='osd.6' 2024-03-18T23:03:08.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:03:08 smithi175 ceph-mon[67352]: from='osd.6 [v2:172.21.15.49:6818/1472215461,v1:172.21.15.49:6819/1472215461]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi049", "root=default"]}]': finished 2024-03-18T23:03:08.462 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:03:08 smithi175 ceph-mon[67352]: osdmap e45: 7 total, 6 up, 7 in 2024-03-18T23:03:08.462 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:03:08 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-18T23:03:08.462 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:03:08 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-18T23:03:08.462 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:03:08 smithi175 ceph-mon[67352]: from='osd.6 [v2:172.21.15.49:6818/1472215461,v1:172.21.15.49:6819/1472215461]' entity='osd.6' 2024-03-18T23:03:08.489 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:03:08 smithi139 ceph-mon[67282]: from='osd.6 [v2:172.21.15.49:6818/1472215461,v1:172.21.15.49:6819/1472215461]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi049", "root=default"]}]': finished 2024-03-18T23:03:08.489 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:03:08 smithi139 ceph-mon[67282]: osdmap e45: 7 total, 6 up, 7 in 2024-03-18T23:03:08.489 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:03:08 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-18T23:03:08.489 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:03:08 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-18T23:03:08.490 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:03:08 smithi139 ceph-mon[67282]: from='osd.6 [v2:172.21.15.49:6818/1472215461,v1:172.21.15.49:6819/1472215461]' entity='osd.6' 2024-03-18T23:03:09.338 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:03:09 smithi139 ceph-mon[67282]: purged_snaps scrub starts 2024-03-18T23:03:09.338 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:03:09 smithi139 ceph-mon[67282]: purged_snaps scrub ok 2024-03-18T23:03:09.339 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:03:09 smithi139 ceph-mon[67282]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-03-18T23:03:09.339 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:03:09 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-18T23:03:09.339 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:03:09 smithi139 ceph-mon[67282]: osd.6 [v2:172.21.15.49:6818/1472215461,v1:172.21.15.49:6819/1472215461] boot 2024-03-18T23:03:09.339 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:03:09 smithi139 ceph-mon[67282]: osdmap e46: 7 total, 7 up, 7 in 2024-03-18T23:03:09.339 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:03:09 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-18T23:03:09.340 INFO:teuthology.orchestra.run.smithi139.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-03-18T23:03:09.340 INFO:teuthology.orchestra.run.smithi139.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-03-18T23:03:09.340 INFO:teuthology.orchestra.run.smithi139.stderr: stderr: 10+0 records in 2024-03-18T23:03:09.340 INFO:teuthology.orchestra.run.smithi139.stderr:10+0 records out 2024-03-18T23:03:09.340 INFO:teuthology.orchestra.run.smithi139.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0113745 s, 922 MB/s 2024-03-18T23:03:09.340 INFO:teuthology.orchestra.run.smithi139.stderr:--> Zapping successful for: 2024-03-18T23:03:09.462 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:03:09 smithi175 ceph-mon[67352]: purged_snaps scrub starts 2024-03-18T23:03:09.462 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:03:09 smithi175 ceph-mon[67352]: purged_snaps scrub ok 2024-03-18T23:03:09.462 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:03:09 smithi175 ceph-mon[67352]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-03-18T23:03:09.462 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:03:09 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-18T23:03:09.462 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:03:09 smithi175 ceph-mon[67352]: osd.6 [v2:172.21.15.49:6818/1472215461,v1:172.21.15.49:6819/1472215461] boot 2024-03-18T23:03:09.462 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:03:09 smithi175 ceph-mon[67352]: osdmap e46: 7 total, 7 up, 7 in 2024-03-18T23:03:09.462 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:03:09 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-18T23:03:09.630 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:03:09 smithi049 ceph-mon[62991]: purged_snaps scrub starts 2024-03-18T23:03:09.630 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:03:09 smithi049 ceph-mon[62991]: purged_snaps scrub ok 2024-03-18T23:03:09.630 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:03:09 smithi049 ceph-mon[62991]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-03-18T23:03:09.631 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:03:09 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-18T23:03:09.631 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:03:09 smithi049 ceph-mon[62991]: osd.6 [v2:172.21.15.49:6818/1472215461,v1:172.21.15.49:6819/1472215461] boot 2024-03-18T23:03:09.631 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:03:09 smithi049 ceph-mon[62991]: osdmap e46: 7 total, 7 up, 7 in 2024-03-18T23:03:09.631 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:03:09 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-18T23:03:10.057 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c63591ea-e57a-11ee-95c9-87774f69a715 -- ceph orch daemon add osd smithi139:vg_nvme/lv_2 2024-03-18T23:03:10.307 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/mon.b/config 2024-03-18T23:03:10.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:03:10 smithi139 ceph-mon[67282]: osdmap e47: 7 total, 7 up, 7 in 2024-03-18T23:03:10.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:03:10 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:03:10.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:03:10 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:03:10.816 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:03:10 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:03:10.816 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:03:10 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:03:10.816 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:03:10 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:03:10.816 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:03:10 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:03:10.816 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:03:10 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:03:10.816 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:03:10 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:03:10.816 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:03:10 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:03:10.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:03:10 smithi049 ceph-mon[62991]: osdmap e47: 7 total, 7 up, 7 in 2024-03-18T23:03:10.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:03:10 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:03:10.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:03:10 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:03:10.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:03:10 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:03:10.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:03:10 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:03:10.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:03:10 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:03:10.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:03:10 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:03:10.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:03:10 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:03:10.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:03:10 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:03:10.881 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:03:10 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:03:10.961 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:03:10 smithi175 ceph-mon[67352]: osdmap e47: 7 total, 7 up, 7 in 2024-03-18T23:03:10.961 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:03:10 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:03:10.962 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:03:10 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:03:10.962 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:03:10 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:03:10.962 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:03:10 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:03:10.962 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:03:10 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:03:10.962 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:03:10 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:03:10.962 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:03:10 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:03:10.962 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:03:10 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:03:10.962 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:03:10 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:03:11.705 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:03:11 smithi175 ceph-mon[67352]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 626 GiB / 626 GiB avail 2024-03-18T23:03:11.705 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:03:11 smithi175 ceph-mon[67352]: Detected new or changed devices on smithi049 2024-03-18T23:03:11.705 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:03:11 smithi175 ceph-mon[67352]: Adjusting osd_memory_target on smithi049 to 5696M 2024-03-18T23:03:11.753 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:03:11 smithi139 ceph-mon[67282]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 626 GiB / 626 GiB avail 2024-03-18T23:03:11.754 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:03:11 smithi139 ceph-mon[67282]: Detected new or changed devices on smithi049 2024-03-18T23:03:11.754 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:03:11 smithi139 ceph-mon[67282]: Adjusting osd_memory_target on smithi049 to 5696M 2024-03-18T23:03:11.984 INFO:teuthology.orchestra.run.smithi139.stderr:2024-03-18T23:03:11.980+0000 7fe1f817c700 1 -- 172.21.15.139:0/3599856907 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe1f0102470 msgr2=0x7fe1f01028d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:03:11.984 INFO:teuthology.orchestra.run.smithi139.stderr:2024-03-18T23:03:11.980+0000 7fe1f817c700 1 --2- 172.21.15.139:0/3599856907 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe1f0102470 0x7fe1f01028d0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fe1e00099c0 tx=0x7fe1e0009cd0 comp rx=0 tx=0).stop 2024-03-18T23:03:11.984 INFO:teuthology.orchestra.run.smithi139.stderr:2024-03-18T23:03:11.980+0000 7fe1f817c700 1 -- 172.21.15.139:0/3599856907 shutdown_connections 2024-03-18T23:03:11.984 INFO:teuthology.orchestra.run.smithi139.stderr:2024-03-18T23:03:11.980+0000 7fe1f817c700 1 --2- 172.21.15.139:0/3599856907 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe1f0102e10 0x7fe1f01096c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:03:11.984 INFO:teuthology.orchestra.run.smithi139.stderr:2024-03-18T23:03:11.980+0000 7fe1f817c700 1 --2- 172.21.15.139:0/3599856907 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe1f0101270 0x7fe1f0101690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:03:11.984 INFO:teuthology.orchestra.run.smithi139.stderr:2024-03-18T23:03:11.980+0000 7fe1f817c700 1 --2- 172.21.15.139:0/3599856907 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe1f0102470 0x7fe1f01028d0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:03:11.984 INFO:teuthology.orchestra.run.smithi139.stderr:2024-03-18T23:03:11.980+0000 7fe1f817c700 1 -- 172.21.15.139:0/3599856907 >> 172.21.15.139:0/3599856907 conn(0x7fe1f00fc830 msgr2=0x7fe1f00fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:03:11.984 INFO:teuthology.orchestra.run.smithi139.stderr:2024-03-18T23:03:11.981+0000 7fe1f817c700 1 -- 172.21.15.139:0/3599856907 shutdown_connections 2024-03-18T23:03:11.984 INFO:teuthology.orchestra.run.smithi139.stderr:2024-03-18T23:03:11.981+0000 7fe1f817c700 1 -- 172.21.15.139:0/3599856907 wait complete. 2024-03-18T23:03:11.984 INFO:teuthology.orchestra.run.smithi139.stderr:2024-03-18T23:03:11.981+0000 7fe1f817c700 1 Processor -- start 2024-03-18T23:03:11.984 INFO:teuthology.orchestra.run.smithi139.stderr:2024-03-18T23:03:11.981+0000 7fe1f817c700 1 -- start start 2024-03-18T23:03:11.985 INFO:teuthology.orchestra.run.smithi139.stderr:2024-03-18T23:03:11.981+0000 7fe1f817c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe1f0101270 0x7fe1f0198570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:03:11.985 INFO:teuthology.orchestra.run.smithi139.stderr:2024-03-18T23:03:11.981+0000 7fe1f817c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe1f0102470 0x7fe1f0198ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:03:11.985 INFO:teuthology.orchestra.run.smithi139.stderr:2024-03-18T23:03:11.981+0000 7fe1f817c700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe1f0102e10 0x7fe1f019fb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:03:11.985 INFO:teuthology.orchestra.run.smithi139.stderr:2024-03-18T23:03:11.981+0000 7fe1f817c700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fe1f010ce60 con 0x7fe1f0102470 2024-03-18T23:03:11.985 INFO:teuthology.orchestra.run.smithi139.stderr:2024-03-18T23:03:11.981+0000 7fe1f817c700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fe1f010cce0 con 0x7fe1f0102e10 2024-03-18T23:03:11.985 INFO:teuthology.orchestra.run.smithi139.stderr:2024-03-18T23:03:11.981+0000 7fe1f817c700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe1f010cfe0 con 0x7fe1f0101270 2024-03-18T23:03:11.985 INFO:teuthology.orchestra.run.smithi139.stderr:2024-03-18T23:03:11.981+0000 7fe1f6719700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe1f0102e10 0x7fe1f019fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:03:11.985 INFO:teuthology.orchestra.run.smithi139.stderr:2024-03-18T23:03:11.982+0000 7fe1f6719700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe1f0102e10 0x7fe1f019fb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:33626/0 (socket says 172.21.15.139:33626) 2024-03-18T23:03:11.985 INFO:teuthology.orchestra.run.smithi139.stderr:2024-03-18T23:03:11.982+0000 7fe1f6719700 1 -- 172.21.15.139:0/1103157082 learned_addr learned my addr 172.21.15.139:0/1103157082 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-03-18T23:03:11.985 INFO:teuthology.orchestra.run.smithi139.stderr:2024-03-18T23:03:11.982+0000 7fe1f5f18700 1 --2- 172.21.15.139:0/1103157082 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe1f0101270 0x7fe1f0198570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:03:11.985 INFO:teuthology.orchestra.run.smithi139.stderr:2024-03-18T23:03:11.982+0000 7fe1f5717700 1 --2- 172.21.15.139:0/1103157082 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe1f0102470 0x7fe1f0198ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:03:11.985 INFO:teuthology.orchestra.run.smithi139.stderr:2024-03-18T23:03:11.982+0000 7fe1f6719700 1 -- 172.21.15.139:0/1103157082 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe1f0101270 msgr2=0x7fe1f0198570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:03:11.986 INFO:teuthology.orchestra.run.smithi139.stderr:2024-03-18T23:03:11.982+0000 7fe1f6719700 1 --2- 172.21.15.139:0/1103157082 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe1f0101270 0x7fe1f0198570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:03:11.986 INFO:teuthology.orchestra.run.smithi139.stderr:2024-03-18T23:03:11.982+0000 7fe1f6719700 1 -- 172.21.15.139:0/1103157082 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe1f0102470 msgr2=0x7fe1f0198ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:03:11.986 INFO:teuthology.orchestra.run.smithi139.stderr:2024-03-18T23:03:11.982+0000 7fe1f6719700 1 --2- 172.21.15.139:0/1103157082 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe1f0102470 0x7fe1f0198ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:03:11.986 INFO:teuthology.orchestra.run.smithi139.stderr:2024-03-18T23:03:11.982+0000 7fe1f6719700 1 -- 172.21.15.139:0/1103157082 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1f01a01d0 con 0x7fe1f0102e10 2024-03-18T23:03:11.986 INFO:teuthology.orchestra.run.smithi139.stderr:2024-03-18T23:03:11.982+0000 7fe1f5717700 1 --2- 172.21.15.139:0/1103157082 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe1f0102470 0x7fe1f0198ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T23:03:11.986 INFO:teuthology.orchestra.run.smithi139.stderr:2024-03-18T23:03:11.982+0000 7fe1f6719700 1 --2- 172.21.15.139:0/1103157082 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe1f0102e10 0x7fe1f019fb30 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fe1e8007ee0 tx=0x7fe1e800d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:03:11.986 INFO:teuthology.orchestra.run.smithi139.stderr:2024-03-18T23:03:11.982+0000 7fe1e6ffd700 1 -- 172.21.15.139:0/1103157082 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1e800dbd0 con 0x7fe1f0102e10 2024-03-18T23:03:11.986 INFO:teuthology.orchestra.run.smithi139.stderr:2024-03-18T23:03:11.982+0000 7fe1e6ffd700 1 -- 172.21.15.139:0/1103157082 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe1e800dd70 con 0x7fe1f0102e10 2024-03-18T23:03:11.986 INFO:teuthology.orchestra.run.smithi139.stderr:2024-03-18T23:03:11.982+0000 7fe1e6ffd700 1 -- 172.21.15.139:0/1103157082 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1e80133f0 con 0x7fe1f0102e10 2024-03-18T23:03:11.986 INFO:teuthology.orchestra.run.smithi139.stderr:2024-03-18T23:03:11.982+0000 7fe1f817c700 1 -- 172.21.15.139:0/1103157082 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1f01a04c0 con 0x7fe1f0102e10 2024-03-18T23:03:11.986 INFO:teuthology.orchestra.run.smithi139.stderr:2024-03-18T23:03:11.982+0000 7fe1f817c700 1 -- 172.21.15.139:0/1103157082 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1f01a09b0 con 0x7fe1f0102e10 2024-03-18T23:03:11.986 INFO:teuthology.orchestra.run.smithi139.stderr:2024-03-18T23:03:11.983+0000 7fe1f817c700 1 -- 172.21.15.139:0/1103157082 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1f0068fd0 con 0x7fe1f0102e10 2024-03-18T23:03:11.987 INFO:teuthology.orchestra.run.smithi139.stderr:2024-03-18T23:03:11.984+0000 7fe1e6ffd700 1 -- 172.21.15.139:0/1103157082 <== mon.2 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7fe1e801a020 con 0x7fe1f0102e10 2024-03-18T23:03:11.987 INFO:teuthology.orchestra.run.smithi139.stderr:2024-03-18T23:03:11.985+0000 7fe1e6ffd700 1 --2- 172.21.15.139:0/1103157082 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fe1dc0a2890 0x7fe1dc0a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:03:11.987 INFO:teuthology.orchestra.run.smithi139.stderr:2024-03-18T23:03:11.985+0000 7fe1e6ffd700 1 -- 172.21.15.139:0/1103157082 <== mon.2 v2:172.21.15.139:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 4965+0+0 (secure 0 0 0) 0x7fe1e80cdb20 con 0x7fe1f0102e10 2024-03-18T23:03:11.987 INFO:teuthology.orchestra.run.smithi139.stderr:2024-03-18T23:03:11.985+0000 7fe1f5f18700 1 --2- 172.21.15.139:0/1103157082 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fe1dc0a2890 0x7fe1dc0a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:03:11.988 INFO:teuthology.orchestra.run.smithi139.stderr:2024-03-18T23:03:11.986+0000 7fe1f5f18700 1 --2- 172.21.15.139:0/1103157082 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fe1dc0a2890 0x7fe1dc0a4d50 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fe1ec005950 tx=0x7fe1ec0058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:03:11.989 INFO:teuthology.orchestra.run.smithi139.stderr:2024-03-18T23:03:11.988+0000 7fe1e6ffd700 1 -- 172.21.15.139:0/1103157082 <== mon.2 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fe1e80cde20 con 0x7fe1f0102e10 2024-03-18T23:03:12.130 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:03:11 smithi049 ceph-mon[62991]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 626 GiB / 626 GiB avail 2024-03-18T23:03:12.130 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:03:11 smithi049 ceph-mon[62991]: Detected new or changed devices on smithi049 2024-03-18T23:03:12.130 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:03:11 smithi049 ceph-mon[62991]: Adjusting osd_memory_target on smithi049 to 5696M 2024-03-18T23:03:12.192 INFO:teuthology.orchestra.run.smithi139.stderr:2024-03-18T23:03:12.191+0000 7fe1f817c700 1 -- 172.21.15.139:0/1103157082 --> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi139:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fe1f00fe9c0 con 0x7fe1dc0a2890 2024-03-18T23:03:12.894 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:03:12 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-18T23:03:12.895 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:03:12 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-18T23:03:12.895 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:03:12 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:03:12.961 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:03:12 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-18T23:03:12.961 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:03:12 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-18T23:03:12.961 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:03:12 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:03:13.130 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:03:12 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-18T23:03:13.130 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:03:12 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-18T23:03:13.130 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:03:12 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:03:13.948 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:03:13 smithi139 ceph-mon[67282]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 626 GiB / 626 GiB avail 2024-03-18T23:03:13.948 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:03:13 smithi139 ceph-mon[67282]: from='client.24257 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi139:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T23:03:13.961 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:03:13 smithi175 ceph-mon[67352]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 626 GiB / 626 GiB avail 2024-03-18T23:03:13.961 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:03:13 smithi175 ceph-mon[67352]: from='client.24257 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi139:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T23:03:14.130 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:03:13 smithi049 ceph-mon[62991]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 626 GiB / 626 GiB avail 2024-03-18T23:03:14.130 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:03:13 smithi049 ceph-mon[62991]: from='client.24257 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi139:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T23:03:15.961 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:03:15 smithi175 ceph-mon[67352]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 626 GiB / 626 GiB avail 2024-03-18T23:03:15.962 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:03:15 smithi175 ceph-mon[67352]: from='client.? 172.21.15.139:0/3378216644' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ad157c9b-4b75-451c-8740-630f0975b7fc"}]: dispatch 2024-03-18T23:03:15.962 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:03:15 smithi175 ceph-mon[67352]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ad157c9b-4b75-451c-8740-630f0975b7fc"}]: dispatch 2024-03-18T23:03:15.962 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:03:15 smithi175 ceph-mon[67352]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ad157c9b-4b75-451c-8740-630f0975b7fc"}]': finished 2024-03-18T23:03:15.962 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:03:15 smithi175 ceph-mon[67352]: osdmap e48: 8 total, 7 up, 8 in 2024-03-18T23:03:15.962 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:03:15 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-18T23:03:15.962 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:03:15 smithi175 ceph-mon[67352]: from='client.? 172.21.15.139:0/324164757' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-18T23:03:16.065 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:03:15 smithi139 ceph-mon[67282]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 626 GiB / 626 GiB avail 2024-03-18T23:03:16.065 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:03:15 smithi139 ceph-mon[67282]: from='client.? 172.21.15.139:0/3378216644' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ad157c9b-4b75-451c-8740-630f0975b7fc"}]: dispatch 2024-03-18T23:03:16.065 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:03:15 smithi139 ceph-mon[67282]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ad157c9b-4b75-451c-8740-630f0975b7fc"}]: dispatch 2024-03-18T23:03:16.065 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:03:15 smithi139 ceph-mon[67282]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ad157c9b-4b75-451c-8740-630f0975b7fc"}]': finished 2024-03-18T23:03:16.065 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:03:15 smithi139 ceph-mon[67282]: osdmap e48: 8 total, 7 up, 8 in 2024-03-18T23:03:16.065 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:03:15 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-18T23:03:16.065 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:03:15 smithi139 ceph-mon[67282]: from='client.? 172.21.15.139:0/324164757' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-18T23:03:16.130 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:03:15 smithi049 ceph-mon[62991]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 626 GiB / 626 GiB avail 2024-03-18T23:03:16.130 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:03:15 smithi049 ceph-mon[62991]: from='client.? 172.21.15.139:0/3378216644' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ad157c9b-4b75-451c-8740-630f0975b7fc"}]: dispatch 2024-03-18T23:03:16.130 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:03:15 smithi049 ceph-mon[62991]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ad157c9b-4b75-451c-8740-630f0975b7fc"}]: dispatch 2024-03-18T23:03:16.130 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:03:15 smithi049 ceph-mon[62991]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ad157c9b-4b75-451c-8740-630f0975b7fc"}]': finished 2024-03-18T23:03:16.130 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:03:15 smithi049 ceph-mon[62991]: osdmap e48: 8 total, 7 up, 8 in 2024-03-18T23:03:16.130 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:03:15 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-18T23:03:16.130 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:03:15 smithi049 ceph-mon[62991]: from='client.? 172.21.15.139:0/324164757' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-18T23:03:17.961 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:03:17 smithi175 ceph-mon[67352]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 626 GiB / 626 GiB avail 2024-03-18T23:03:18.065 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:03:17 smithi139 ceph-mon[67282]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 626 GiB / 626 GiB avail 2024-03-18T23:03:18.130 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:03:17 smithi049 ceph-mon[62991]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 626 GiB / 626 GiB avail 2024-03-18T23:03:19.898 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:03:19 smithi139 ceph-mon[67282]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 626 GiB / 626 GiB avail 2024-03-18T23:03:19.961 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:03:19 smithi175 ceph-mon[67352]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 626 GiB / 626 GiB avail 2024-03-18T23:03:20.130 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:03:19 smithi049 ceph-mon[62991]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 626 GiB / 626 GiB avail 2024-03-18T23:03:21.962 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:03:21 smithi175 ceph-mon[67352]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 626 GiB / 626 GiB avail 2024-03-18T23:03:21.985 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:03:21 smithi139 ceph-mon[67282]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 626 GiB / 626 GiB avail 2024-03-18T23:03:22.130 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:03:21 smithi049 ceph-mon[62991]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 626 GiB / 626 GiB avail 2024-03-18T23:03:22.961 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:03:22 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-03-18T23:03:22.961 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:03:22 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:03:22.971 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:03:22 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-03-18T23:03:22.971 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:03:22 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:03:23.031 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:03:22 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-03-18T23:03:23.031 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:03:22 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:03:23.961 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:03:23 smithi175 ceph-mon[67352]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 626 GiB / 626 GiB avail 2024-03-18T23:03:23.961 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:03:23 smithi175 ceph-mon[67352]: Deploying daemon osd.7 on smithi139 2024-03-18T23:03:23.967 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:03:23 smithi139 ceph-mon[67282]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 626 GiB / 626 GiB avail 2024-03-18T23:03:23.967 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:03:23 smithi139 ceph-mon[67282]: Deploying daemon osd.7 on smithi139 2024-03-18T23:03:24.043 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:03:23 smithi049 ceph-mon[62991]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 626 GiB / 626 GiB avail 2024-03-18T23:03:24.043 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:03:23 smithi049 ceph-mon[62991]: Deploying daemon osd.7 on smithi139 2024-03-18T23:03:25.948 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:03:25 smithi139 ceph-mon[67282]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 626 GiB / 626 GiB avail 2024-03-18T23:03:25.961 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:03:25 smithi175 ceph-mon[67352]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 626 GiB / 626 GiB avail 2024-03-18T23:03:26.130 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:03:25 smithi049 ceph-mon[62991]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 626 GiB / 626 GiB avail 2024-03-18T23:03:27.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:03:27 smithi139 ceph-mon[67282]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 626 GiB / 626 GiB avail 2024-03-18T23:03:27.961 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:03:27 smithi175 ceph-mon[67352]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 626 GiB / 626 GiB avail 2024-03-18T23:03:28.130 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:03:27 smithi049 ceph-mon[62991]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 626 GiB / 626 GiB avail 2024-03-18T23:03:29.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:03:29 smithi139 ceph-mon[67282]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 626 GiB / 626 GiB avail 2024-03-18T23:03:29.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:03:29 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:03:29.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:03:29 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:03:29.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:03:29 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:03:29.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:03:29 smithi049 ceph-mon[62991]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 626 GiB / 626 GiB avail 2024-03-18T23:03:29.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:03:29 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:03:29.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:03:29 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:03:29.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:03:29 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:03:29.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:03:29 smithi175 ceph-mon[67352]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 626 GiB / 626 GiB avail 2024-03-18T23:03:29.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:03:29 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:03:29.462 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:03:29 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:03:29.462 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:03:29 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:03:31.261 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:03:31 smithi139 ceph-mon[67282]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 626 GiB / 626 GiB avail 2024-03-18T23:03:31.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:03:31 smithi049 ceph-mon[62991]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 626 GiB / 626 GiB avail 2024-03-18T23:03:31.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:03:31 smithi175 ceph-mon[67352]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 626 GiB / 626 GiB avail 2024-03-18T23:03:31.912 INFO:teuthology.orchestra.run.smithi139.stdout:Created osd(s) 7 on host 'smithi139' 2024-03-18T23:03:31.912 INFO:teuthology.orchestra.run.smithi139.stderr:2024-03-18T23:03:31.910+0000 7fe1e6ffd700 1 -- 172.21.15.139:0/1103157082 <== mgr.14150 v2:172.21.15.49:6800/2083805211 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fe1f00fe9c0 con 0x7fe1dc0a2890 2024-03-18T23:03:31.914 INFO:teuthology.orchestra.run.smithi139.stderr:2024-03-18T23:03:31.912+0000 7fe1f817c700 1 -- 172.21.15.139:0/1103157082 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fe1dc0a2890 msgr2=0x7fe1dc0a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:03:31.914 INFO:teuthology.orchestra.run.smithi139.stderr:2024-03-18T23:03:31.912+0000 7fe1f817c700 1 --2- 172.21.15.139:0/1103157082 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fe1dc0a2890 0x7fe1dc0a4d50 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fe1ec005950 tx=0x7fe1ec0058e0 comp rx=0 tx=0).stop 2024-03-18T23:03:31.915 INFO:teuthology.orchestra.run.smithi139.stderr:2024-03-18T23:03:31.912+0000 7fe1f817c700 1 -- 172.21.15.139:0/1103157082 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe1f0102e10 msgr2=0x7fe1f019fb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:03:31.915 INFO:teuthology.orchestra.run.smithi139.stderr:2024-03-18T23:03:31.912+0000 7fe1f817c700 1 --2- 172.21.15.139:0/1103157082 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe1f0102e10 0x7fe1f019fb30 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fe1e8007ee0 tx=0x7fe1e800d3b0 comp rx=0 tx=0).stop 2024-03-18T23:03:31.915 INFO:teuthology.orchestra.run.smithi139.stderr:2024-03-18T23:03:31.912+0000 7fe1f817c700 1 -- 172.21.15.139:0/1103157082 shutdown_connections 2024-03-18T23:03:31.915 INFO:teuthology.orchestra.run.smithi139.stderr:2024-03-18T23:03:31.912+0000 7fe1f817c700 1 --2- 172.21.15.139:0/1103157082 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe1f0102e10 0x7fe1f019fb30 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:03:31.915 INFO:teuthology.orchestra.run.smithi139.stderr:2024-03-18T23:03:31.912+0000 7fe1f817c700 1 --2- 172.21.15.139:0/1103157082 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fe1dc0a2890 0x7fe1dc0a4d50 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:03:31.915 INFO:teuthology.orchestra.run.smithi139.stderr:2024-03-18T23:03:31.912+0000 7fe1f817c700 1 --2- 172.21.15.139:0/1103157082 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe1f0101270 0x7fe1f0198570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:03:31.915 INFO:teuthology.orchestra.run.smithi139.stderr:2024-03-18T23:03:31.912+0000 7fe1f817c700 1 --2- 172.21.15.139:0/1103157082 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe1f0102470 0x7fe1f0198ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:03:31.915 INFO:teuthology.orchestra.run.smithi139.stderr:2024-03-18T23:03:31.912+0000 7fe1f817c700 1 -- 172.21.15.139:0/1103157082 >> 172.21.15.139:0/1103157082 conn(0x7fe1f00fc830 msgr2=0x7fe1f00fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:03:31.915 INFO:teuthology.orchestra.run.smithi139.stderr:2024-03-18T23:03:31.913+0000 7fe1f817c700 1 -- 172.21.15.139:0/1103157082 shutdown_connections 2024-03-18T23:03:31.915 INFO:teuthology.orchestra.run.smithi139.stderr:2024-03-18T23:03:31.913+0000 7fe1f817c700 1 -- 172.21.15.139:0/1103157082 wait complete. 2024-03-18T23:03:32.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:03:32 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:03:32.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:03:32 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:03:32.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:03:32 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:03:32.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:03:32 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:03:32.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:03:32 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:03:32.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:03:32 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:03:32.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:03:32 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:03:32.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:03:32 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:03:32.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:03:32 smithi049 ceph-mon[62991]: from='osd.7 [v2:172.21.15.139:6816/2163986787,v1:172.21.15.139:6817/2163986787]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-18T23:03:32.381 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:03:32 smithi049 ceph-mon[62991]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-18T23:03:32.462 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:03:32 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:03:32.462 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:03:32 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:03:32.462 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:03:32 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:03:32.462 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:03:32 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:03:32.463 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:03:32 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:03:32.463 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:03:32 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:03:32.463 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:03:32 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:03:32.463 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:03:32 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:03:32.463 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:03:32 smithi175 ceph-mon[67352]: from='osd.7 [v2:172.21.15.139:6816/2163986787,v1:172.21.15.139:6817/2163986787]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-18T23:03:32.463 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:03:32 smithi175 ceph-mon[67352]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-18T23:03:32.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:03:32 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:03:32.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:03:32 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:03:32.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:03:32 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:03:32.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:03:32 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:03:32.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:03:32 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:03:32.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:03:32 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:03:32.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:03:32 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:03:32.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:03:32 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:03:32.566 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:03:32 smithi139 ceph-mon[67282]: from='osd.7 [v2:172.21.15.139:6816/2163986787,v1:172.21.15.139:6817/2163986787]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-18T23:03:32.566 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:03:32 smithi139 ceph-mon[67282]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-18T23:03:32.599 DEBUG:teuthology.orchestra.run.smithi139:osd.7> sudo journalctl -f -n 0 -u ceph-c63591ea-e57a-11ee-95c9-87774f69a715@osd.7.service 2024-03-18T23:03:32.602 INFO:tasks.cephadm:Deploying osd.8 on smithi175 with /dev/vg_nvme/lv_2... 2024-03-18T23:03:32.602 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c63591ea-e57a-11ee-95c9-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-03-18T23:03:32.850 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/mon.c/config 2024-03-18T23:03:33.265 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:03:33 smithi175 ceph-mon[67352]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 626 GiB / 626 GiB avail 2024-03-18T23:03:33.265 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:03:33 smithi175 ceph-mon[67352]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-03-18T23:03:33.265 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:03:33 smithi175 ceph-mon[67352]: osdmap e49: 8 total, 7 up, 8 in 2024-03-18T23:03:33.265 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:03:33 smithi175 ceph-mon[67352]: from='osd.7 [v2:172.21.15.139:6816/2163986787,v1:172.21.15.139:6817/2163986787]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]: dispatch 2024-03-18T23:03:33.265 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:03:33 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-18T23:03:33.265 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:03:33 smithi175 ceph-mon[67352]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]: dispatch 2024-03-18T23:03:33.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:03:33 smithi049 ceph-mon[62991]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 626 GiB / 626 GiB avail 2024-03-18T23:03:33.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:03:33 smithi049 ceph-mon[62991]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-03-18T23:03:33.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:03:33 smithi049 ceph-mon[62991]: osdmap e49: 8 total, 7 up, 8 in 2024-03-18T23:03:33.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:03:33 smithi049 ceph-mon[62991]: from='osd.7 [v2:172.21.15.139:6816/2163986787,v1:172.21.15.139:6817/2163986787]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]: dispatch 2024-03-18T23:03:33.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:03:33 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-18T23:03:33.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:03:33 smithi049 ceph-mon[62991]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]: dispatch 2024-03-18T23:03:33.497 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:03:33 smithi139 ceph-mon[67282]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 626 GiB / 626 GiB avail 2024-03-18T23:03:33.498 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:03:33 smithi139 ceph-mon[67282]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-03-18T23:03:33.498 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:03:33 smithi139 ceph-mon[67282]: osdmap e49: 8 total, 7 up, 8 in 2024-03-18T23:03:33.498 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:03:33 smithi139 ceph-mon[67282]: from='osd.7 [v2:172.21.15.139:6816/2163986787,v1:172.21.15.139:6817/2163986787]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]: dispatch 2024-03-18T23:03:33.498 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:03:33 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-18T23:03:33.498 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:03:33 smithi139 ceph-mon[67282]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]: dispatch 2024-03-18T23:03:34.065 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 18 23:03:33 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-7[85590]: 2024-03-18T23:03:33.882+0000 7f5ddbb21700 -1 osd.7 0 waiting for initial osdmap 2024-03-18T23:03:34.066 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 18 23:03:33 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-7[85590]: 2024-03-18T23:03:33.889+0000 7f5dd7914700 -1 osd.7 50 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-18T23:03:34.755 INFO:teuthology.orchestra.run.smithi175.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-03-18T23:03:34.756 INFO:teuthology.orchestra.run.smithi175.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-03-18T23:03:34.756 INFO:teuthology.orchestra.run.smithi175.stderr: stderr: 10+0 records in 2024-03-18T23:03:34.756 INFO:teuthology.orchestra.run.smithi175.stderr:10+0 records out 2024-03-18T23:03:34.756 INFO:teuthology.orchestra.run.smithi175.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0132528 s, 791 MB/s 2024-03-18T23:03:34.756 INFO:teuthology.orchestra.run.smithi175.stderr:--> Zapping successful for: 2024-03-18T23:03:35.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:03:34 smithi175 ceph-mon[67352]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 626 GiB / 626 GiB avail 2024-03-18T23:03:35.126 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:03:34 smithi175 ceph-mon[67352]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]': finished 2024-03-18T23:03:35.126 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:03:34 smithi175 ceph-mon[67352]: osdmap e50: 8 total, 7 up, 8 in 2024-03-18T23:03:35.126 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:03:34 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-18T23:03:35.126 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:03:34 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-18T23:03:35.130 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:03:34 smithi049 ceph-mon[62991]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 626 GiB / 626 GiB avail 2024-03-18T23:03:35.130 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:03:34 smithi049 ceph-mon[62991]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]': finished 2024-03-18T23:03:35.130 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:03:34 smithi049 ceph-mon[62991]: osdmap e50: 8 total, 7 up, 8 in 2024-03-18T23:03:35.130 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:03:34 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-18T23:03:35.130 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:03:34 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-18T23:03:35.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:03:34 smithi139 ceph-mon[67282]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 626 GiB / 626 GiB avail 2024-03-18T23:03:35.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:03:34 smithi139 ceph-mon[67282]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]': finished 2024-03-18T23:03:35.316 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:03:34 smithi139 ceph-mon[67282]: osdmap e50: 8 total, 7 up, 8 in 2024-03-18T23:03:35.316 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:03:34 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-18T23:03:35.316 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:03:34 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-18T23:03:35.420 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c63591ea-e57a-11ee-95c9-87774f69a715 -- ceph orch daemon add osd smithi175:vg_nvme/lv_2 2024-03-18T23:03:35.666 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/mon.c/config 2024-03-18T23:03:35.961 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:03:35 smithi175 ceph-mon[67352]: purged_snaps scrub starts 2024-03-18T23:03:35.962 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:03:35 smithi175 ceph-mon[67352]: purged_snaps scrub ok 2024-03-18T23:03:35.962 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:03:35 smithi175 ceph-mon[67352]: osd.7 [v2:172.21.15.139:6816/2163986787,v1:172.21.15.139:6817/2163986787] boot 2024-03-18T23:03:35.962 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:03:35 smithi175 ceph-mon[67352]: osdmap e51: 8 total, 8 up, 8 in 2024-03-18T23:03:35.962 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:03:35 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-18T23:03:35.962 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:03:35 smithi175 ceph-mon[67352]: Detected new or changed devices on smithi139 2024-03-18T23:03:35.962 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:03:35 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:03:35.962 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:03:35 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:03:35.962 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:03:35 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:03:35.962 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:03:35 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:03:35.962 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:03:35 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:03:35.963 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:03:35 smithi175 ceph-mon[67352]: Adjusting osd_memory_target on smithi139 to 5693M 2024-03-18T23:03:35.963 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:03:35 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:03:35.963 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:03:35 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:03:35.963 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:03:35 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:03:35.963 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:03:35 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:03:36.065 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:03:35 smithi139 ceph-mon[67282]: purged_snaps scrub starts 2024-03-18T23:03:36.065 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:03:35 smithi139 ceph-mon[67282]: purged_snaps scrub ok 2024-03-18T23:03:36.065 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:03:35 smithi139 ceph-mon[67282]: osd.7 [v2:172.21.15.139:6816/2163986787,v1:172.21.15.139:6817/2163986787] boot 2024-03-18T23:03:36.065 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:03:35 smithi139 ceph-mon[67282]: osdmap e51: 8 total, 8 up, 8 in 2024-03-18T23:03:36.066 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:03:35 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-18T23:03:36.066 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:03:35 smithi139 ceph-mon[67282]: Detected new or changed devices on smithi139 2024-03-18T23:03:36.066 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:03:35 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:03:36.066 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:03:35 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:03:36.066 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:03:35 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:03:36.066 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:03:35 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:03:36.066 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:03:35 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:03:36.066 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:03:35 smithi139 ceph-mon[67282]: Adjusting osd_memory_target on smithi139 to 5693M 2024-03-18T23:03:36.066 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:03:35 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:03:36.066 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:03:35 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:03:36.066 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:03:35 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:03:36.066 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:03:35 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:03:36.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:03:35 smithi049 ceph-mon[62991]: purged_snaps scrub starts 2024-03-18T23:03:36.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:03:35 smithi049 ceph-mon[62991]: purged_snaps scrub ok 2024-03-18T23:03:36.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:03:35 smithi049 ceph-mon[62991]: osd.7 [v2:172.21.15.139:6816/2163986787,v1:172.21.15.139:6817/2163986787] boot 2024-03-18T23:03:36.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:03:35 smithi049 ceph-mon[62991]: osdmap e51: 8 total, 8 up, 8 in 2024-03-18T23:03:36.381 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:03:35 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-18T23:03:36.381 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:03:35 smithi049 ceph-mon[62991]: Detected new or changed devices on smithi139 2024-03-18T23:03:36.381 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:03:35 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:03:36.381 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:03:35 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:03:36.381 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:03:35 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:03:36.381 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:03:35 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:03:36.381 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:03:35 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:03:36.381 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:03:35 smithi049 ceph-mon[62991]: Adjusting osd_memory_target on smithi139 to 5693M 2024-03-18T23:03:36.381 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:03:35 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:03:36.381 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:03:35 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:03:36.381 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:03:35 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:03:36.381 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:03:35 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:03:37.067 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:03:36 smithi175 ceph-mon[67352]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 715 GiB / 715 GiB avail 2024-03-18T23:03:37.067 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:03:36 smithi175 ceph-mon[67352]: osdmap e52: 8 total, 8 up, 8 in 2024-03-18T23:03:37.314 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:03:37.297+0000 7f55fb89a700 1 -- 172.21.15.175:0/4164217882 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f55f41012a0 msgr2=0x7f55f41016c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:03:37.314 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:03:37.297+0000 7f55fb89a700 1 --2- 172.21.15.175:0/4164217882 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f55f41012a0 0x7f55f41016c0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f55f0009990 tx=0x7f55f0009ca0 comp rx=0 tx=0).stop 2024-03-18T23:03:37.314 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:03:37.297+0000 7f55fb89a700 1 -- 172.21.15.175:0/4164217882 shutdown_connections 2024-03-18T23:03:37.314 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:03:37.297+0000 7f55fb89a700 1 --2- 172.21.15.175:0/4164217882 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f55f4102e40 0x7f55f41096f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:03:37.315 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:03:37.297+0000 7f55fb89a700 1 --2- 172.21.15.175:0/4164217882 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f55f41012a0 0x7f55f41016c0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:03:37.315 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:03:37.297+0000 7f55fb89a700 1 --2- 172.21.15.175:0/4164217882 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f55f41024a0 0x7f55f4102900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:03:37.315 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:03:37.297+0000 7f55fb89a700 1 -- 172.21.15.175:0/4164217882 >> 172.21.15.175:0/4164217882 conn(0x7f55f40fc860 msgr2=0x7f55f40fec80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:03:37.315 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:03:37.297+0000 7f55fb89a700 1 -- 172.21.15.175:0/4164217882 shutdown_connections 2024-03-18T23:03:37.315 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:03:37.297+0000 7f55fb89a700 1 -- 172.21.15.175:0/4164217882 wait complete. 2024-03-18T23:03:37.315 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:03:37.297+0000 7f55fb89a700 1 Processor -- start 2024-03-18T23:03:37.315 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:03:37.297+0000 7f55fb89a700 1 -- start start 2024-03-18T23:03:37.315 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:03:37.297+0000 7f55fb89a700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f55f41012a0 0x7f55f41985a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:03:37.315 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:03:37.297+0000 7f55fb89a700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f55f41024a0 0x7f55f4198ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:03:37.315 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:03:37.297+0000 7f55fb89a700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f55f4102e40 0x7f55f419fb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:03:37.315 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:03:37.297+0000 7f55fb89a700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f55f410ce70 con 0x7f55f41024a0 2024-03-18T23:03:37.315 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:03:37.297+0000 7f55fb89a700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f55f410ccf0 con 0x7f55f4102e40 2024-03-18T23:03:37.315 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:03:37.297+0000 7f55fb89a700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f55f410cff0 con 0x7f55f41012a0 2024-03-18T23:03:37.316 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:03:37.297+0000 7f55f9636700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f55f41012a0 0x7f55f41985a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:03:37.316 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:03:37.297+0000 7f55f9636700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f55f41012a0 0x7f55f41985a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:39186/0 (socket says 172.21.15.175:39186) 2024-03-18T23:03:37.316 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:03:37.298+0000 7f55f9636700 1 -- 172.21.15.175:0/1638267953 learned_addr learned my addr 172.21.15.175:0/1638267953 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-03-18T23:03:37.316 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:03:37.298+0000 7f55f8e35700 1 --2- 172.21.15.175:0/1638267953 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f55f41024a0 0x7f55f4198ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:03:37.316 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:03:37.298+0000 7f55f9e37700 1 --2- 172.21.15.175:0/1638267953 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f55f4102e40 0x7f55f419fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:03:37.316 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:03:37.298+0000 7f55f9636700 1 -- 172.21.15.175:0/1638267953 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f55f4102e40 msgr2=0x7f55f419fb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:03:37.316 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:03:37.298+0000 7f55f9636700 1 --2- 172.21.15.175:0/1638267953 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f55f4102e40 0x7f55f419fb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:03:37.316 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:03:37.298+0000 7f55f9636700 1 -- 172.21.15.175:0/1638267953 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f55f41024a0 msgr2=0x7f55f4198ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:03:37.316 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:03:37.298+0000 7f55f9636700 1 --2- 172.21.15.175:0/1638267953 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f55f41024a0 0x7f55f4198ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:03:37.316 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:03:37.298+0000 7f55f9636700 1 -- 172.21.15.175:0/1638267953 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55f41a0200 con 0x7f55f41012a0 2024-03-18T23:03:37.316 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:03:37.298+0000 7f55f9636700 1 --2- 172.21.15.175:0/1638267953 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f55f41012a0 0x7f55f41985a0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f55f0000c00 tx=0x7f55f0009ee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:03:37.316 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:03:37.298+0000 7f55ea7fc700 1 -- 172.21.15.175:0/1638267953 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55f0027070 con 0x7f55f41012a0 2024-03-18T23:03:37.316 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:03:37.298+0000 7f55fb89a700 1 -- 172.21.15.175:0/1638267953 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55f41a0490 con 0x7f55f41012a0 2024-03-18T23:03:37.316 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:03:37.298+0000 7f55ea7fc700 1 -- 172.21.15.175:0/1638267953 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f55f0004150 con 0x7f55f41012a0 2024-03-18T23:03:37.317 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:03:37.298+0000 7f55ea7fc700 1 -- 172.21.15.175:0/1638267953 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55f000f6c0 con 0x7f55f41012a0 2024-03-18T23:03:37.317 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:03:37.298+0000 7f55fb89a700 1 -- 172.21.15.175:0/1638267953 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55f41a09c0 con 0x7f55f41012a0 2024-03-18T23:03:37.317 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:03:37.299+0000 7f55fb89a700 1 -- 172.21.15.175:0/1638267953 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55f4068fd0 con 0x7f55f41012a0 2024-03-18T23:03:37.317 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:03:37.300+0000 7f55ea7fc700 1 -- 172.21.15.175:0/1638267953 <== mon.1 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7f55f002a070 con 0x7f55f41012a0 2024-03-18T23:03:37.317 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:03:37.301+0000 7f55ea7fc700 1 --2- 172.21.15.175:0/1638267953 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f55e00a2890 0x7f55e00a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:03:37.317 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:03:37.301+0000 7f55ea7fc700 1 -- 172.21.15.175:0/1638267953 <== mon.1 v2:172.21.15.175:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 5426+0+0 (secure 0 0 0) 0x7f55f00cf7e0 con 0x7f55f41012a0 2024-03-18T23:03:37.317 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:03:37.301+0000 7f55f8e35700 1 --2- 172.21.15.175:0/1638267953 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f55e00a2890 0x7f55e00a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:03:37.317 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:03:37.301+0000 7f55f8e35700 1 --2- 172.21.15.175:0/1638267953 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f55e00a2890 0x7f55e00a4d50 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f55e4006fd0 tx=0x7f55e4008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:03:37.317 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:03:37.304+0000 7f55ea7fc700 1 -- 172.21.15.175:0/1638267953 <== mon.1 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f55f0031020 con 0x7f55f41012a0 2024-03-18T23:03:37.317 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:03:36 smithi139 ceph-mon[67282]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 715 GiB / 715 GiB avail 2024-03-18T23:03:37.318 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:03:36 smithi139 ceph-mon[67282]: osdmap e52: 8 total, 8 up, 8 in 2024-03-18T23:03:37.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:03:36 smithi049 ceph-mon[62991]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 715 GiB / 715 GiB avail 2024-03-18T23:03:37.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:03:36 smithi049 ceph-mon[62991]: osdmap e52: 8 total, 8 up, 8 in 2024-03-18T23:03:37.511 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:03:37.511+0000 7f55fb89a700 1 -- 172.21.15.175:0/1638267953 --> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi175:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f55f40fe9c0 con 0x7f55e00a2890 2024-03-18T23:03:38.211 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:03:37 smithi175 ceph-mon[67352]: osdmap e53: 8 total, 8 up, 8 in 2024-03-18T23:03:38.212 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:03:37 smithi175 ceph-mon[67352]: from='client.24265 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi175:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T23:03:38.212 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:03:37 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-18T23:03:38.212 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:03:37 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-18T23:03:38.212 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:03:37 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:03:38.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:03:37 smithi139 ceph-mon[67282]: osdmap e53: 8 total, 8 up, 8 in 2024-03-18T23:03:38.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:03:37 smithi139 ceph-mon[67282]: from='client.24265 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi175:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T23:03:38.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:03:37 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-18T23:03:38.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:03:37 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-18T23:03:38.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:03:37 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:03:38.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:03:37 smithi049 ceph-mon[62991]: osdmap e53: 8 total, 8 up, 8 in 2024-03-18T23:03:38.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:03:37 smithi049 ceph-mon[62991]: from='client.24265 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi175:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T23:03:38.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:03:37 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-18T23:03:38.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:03:37 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-18T23:03:38.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:03:37 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:03:39.187 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:03:38 smithi175 ceph-mon[67352]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 715 GiB / 715 GiB avail 2024-03-18T23:03:39.187 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:03:38 smithi175 ceph-mon[67352]: osdmap e54: 8 total, 8 up, 8 in 2024-03-18T23:03:39.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:03:38 smithi139 ceph-mon[67282]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 715 GiB / 715 GiB avail 2024-03-18T23:03:39.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:03:38 smithi139 ceph-mon[67282]: osdmap e54: 8 total, 8 up, 8 in 2024-03-18T23:03:39.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:03:38 smithi049 ceph-mon[62991]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 715 GiB / 715 GiB avail 2024-03-18T23:03:39.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:03:38 smithi049 ceph-mon[62991]: osdmap e54: 8 total, 8 up, 8 in 2024-03-18T23:03:40.211 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:03:39 smithi175 ceph-mon[67352]: osdmap e55: 8 total, 8 up, 8 in 2024-03-18T23:03:40.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:03:39 smithi139 ceph-mon[67282]: osdmap e55: 8 total, 8 up, 8 in 2024-03-18T23:03:40.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:03:39 smithi049 ceph-mon[62991]: osdmap e55: 8 total, 8 up, 8 in 2024-03-18T23:03:41.211 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:03:40 smithi175 ceph-mon[67352]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-03-18T23:03:41.212 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:03:40 smithi175 ceph-mon[67352]: from='client.? 172.21.15.175:0/4163961178' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c8e75f15-df75-4d5d-9b92-a064bffb6c01"}]: dispatch 2024-03-18T23:03:41.212 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:03:40 smithi175 ceph-mon[67352]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c8e75f15-df75-4d5d-9b92-a064bffb6c01"}]: dispatch 2024-03-18T23:03:41.212 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:03:40 smithi175 ceph-mon[67352]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c8e75f15-df75-4d5d-9b92-a064bffb6c01"}]': finished 2024-03-18T23:03:41.212 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:03:40 smithi175 ceph-mon[67352]: osdmap e56: 9 total, 8 up, 9 in 2024-03-18T23:03:41.212 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:03:40 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-18T23:03:41.212 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:03:40 smithi175 ceph-mon[67352]: from='client.? 172.21.15.175:0/3338144603' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-18T23:03:41.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:03:40 smithi139 ceph-mon[67282]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-03-18T23:03:41.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:03:40 smithi139 ceph-mon[67282]: from='client.? 172.21.15.175:0/4163961178' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c8e75f15-df75-4d5d-9b92-a064bffb6c01"}]: dispatch 2024-03-18T23:03:41.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:03:40 smithi139 ceph-mon[67282]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c8e75f15-df75-4d5d-9b92-a064bffb6c01"}]: dispatch 2024-03-18T23:03:41.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:03:40 smithi139 ceph-mon[67282]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c8e75f15-df75-4d5d-9b92-a064bffb6c01"}]': finished 2024-03-18T23:03:41.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:03:40 smithi139 ceph-mon[67282]: osdmap e56: 9 total, 8 up, 9 in 2024-03-18T23:03:41.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:03:40 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-18T23:03:41.316 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:03:40 smithi139 ceph-mon[67282]: from='client.? 172.21.15.175:0/3338144603' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-18T23:03:41.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:03:40 smithi049 ceph-mon[62991]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-03-18T23:03:41.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:03:40 smithi049 ceph-mon[62991]: from='client.? 172.21.15.175:0/4163961178' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c8e75f15-df75-4d5d-9b92-a064bffb6c01"}]: dispatch 2024-03-18T23:03:41.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:03:40 smithi049 ceph-mon[62991]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c8e75f15-df75-4d5d-9b92-a064bffb6c01"}]: dispatch 2024-03-18T23:03:41.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:03:40 smithi049 ceph-mon[62991]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c8e75f15-df75-4d5d-9b92-a064bffb6c01"}]': finished 2024-03-18T23:03:41.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:03:40 smithi049 ceph-mon[62991]: osdmap e56: 9 total, 8 up, 9 in 2024-03-18T23:03:41.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:03:40 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-18T23:03:41.381 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:03:40 smithi049 ceph-mon[62991]: from='client.? 172.21.15.175:0/3338144603' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-18T23:03:43.211 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:03:42 smithi175 ceph-mon[67352]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-03-18T23:03:43.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:03:42 smithi139 ceph-mon[67282]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-03-18T23:03:43.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:03:42 smithi049 ceph-mon[62991]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-03-18T23:03:45.182 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:03:44 smithi175 ceph-mon[67352]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-03-18T23:03:45.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:03:44 smithi139 ceph-mon[67282]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-03-18T23:03:45.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:03:44 smithi049 ceph-mon[62991]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-03-18T23:03:46.961 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:03:46 smithi175 ceph-mon[67352]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-03-18T23:03:47.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:03:46 smithi139 ceph-mon[67282]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-03-18T23:03:47.381 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:03:46 smithi049 ceph-mon[62991]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-03-18T23:03:48.211 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:03:47 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-03-18T23:03:48.211 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:03:47 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:03:48.211 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:03:47 smithi175 ceph-mon[67352]: Deploying daemon osd.8 on smithi175 2024-03-18T23:03:48.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:03:47 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-03-18T23:03:48.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:03:47 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:03:48.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:03:47 smithi139 ceph-mon[67282]: Deploying daemon osd.8 on smithi175 2024-03-18T23:03:48.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:03:47 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-03-18T23:03:48.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:03:47 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:03:48.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:03:47 smithi049 ceph-mon[62991]: Deploying daemon osd.8 on smithi175 2024-03-18T23:03:49.163 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:03:48 smithi175 ceph-mon[67352]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-03-18T23:03:49.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:03:48 smithi139 ceph-mon[67282]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-03-18T23:03:49.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:03:48 smithi049 ceph-mon[62991]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-03-18T23:03:51.127 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:03:50 smithi175 ceph-mon[67352]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-03-18T23:03:51.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:03:50 smithi139 ceph-mon[67282]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-03-18T23:03:51.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:03:50 smithi049 ceph-mon[62991]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-03-18T23:03:53.182 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:03:52 smithi049 ceph-mon[62991]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-03-18T23:03:53.182 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:03:52 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:03:53.183 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:03:52 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:03:53.183 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:03:52 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:03:53.211 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:03:52 smithi175 ceph-mon[67352]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-03-18T23:03:53.211 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:03:52 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:03:53.211 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:03:52 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:03:53.211 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:03:52 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:03:53.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:03:52 smithi139 ceph-mon[67282]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-03-18T23:03:53.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:03:52 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:03:53.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:03:52 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:03:53.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:03:52 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:03:55.066 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:03:54 smithi175 ceph-mon[67352]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-03-18T23:03:55.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:03:54 smithi139 ceph-mon[67282]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-03-18T23:03:55.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:03:54 smithi049 ceph-mon[62991]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-03-18T23:03:56.331 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:03:56.330+0000 7f55ea7fc700 1 -- 172.21.15.175:0/1638267953 <== mgr.14150 v2:172.21.15.49:6800/2083805211 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f55f40fe9c0 con 0x7f55e00a2890 2024-03-18T23:03:56.331 INFO:teuthology.orchestra.run.smithi175.stdout:Created osd(s) 8 on host 'smithi175' 2024-03-18T23:03:56.333 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:03:56.332+0000 7f55fb89a700 1 -- 172.21.15.175:0/1638267953 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f55e00a2890 msgr2=0x7f55e00a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:03:56.333 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:03:56.332+0000 7f55fb89a700 1 --2- 172.21.15.175:0/1638267953 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f55e00a2890 0x7f55e00a4d50 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f55e4006fd0 tx=0x7f55e4008040 comp rx=0 tx=0).stop 2024-03-18T23:03:56.333 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:03:56.332+0000 7f55fb89a700 1 -- 172.21.15.175:0/1638267953 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f55f41012a0 msgr2=0x7f55f41985a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:03:56.333 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:03:56.332+0000 7f55fb89a700 1 --2- 172.21.15.175:0/1638267953 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f55f41012a0 0x7f55f41985a0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f55f0000c00 tx=0x7f55f0009ee0 comp rx=0 tx=0).stop 2024-03-18T23:03:56.333 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:03:56.332+0000 7f55fb89a700 1 -- 172.21.15.175:0/1638267953 shutdown_connections 2024-03-18T23:03:56.333 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:03:56.332+0000 7f55fb89a700 1 --2- 172.21.15.175:0/1638267953 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f55f4102e40 0x7f55f419fb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:03:56.334 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:03:56.332+0000 7f55fb89a700 1 --2- 172.21.15.175:0/1638267953 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f55e00a2890 0x7f55e00a4d50 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:03:56.334 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:03:56.332+0000 7f55fb89a700 1 --2- 172.21.15.175:0/1638267953 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f55f41012a0 0x7f55f41985a0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:03:56.334 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:03:56.332+0000 7f55fb89a700 1 --2- 172.21.15.175:0/1638267953 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f55f41024a0 0x7f55f4198ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:03:56.334 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:03:56.332+0000 7f55fb89a700 1 -- 172.21.15.175:0/1638267953 >> 172.21.15.175:0/1638267953 conn(0x7f55f40fc860 msgr2=0x7f55f40fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:03:56.334 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:03:56.332+0000 7f55fb89a700 1 -- 172.21.15.175:0/1638267953 shutdown_connections 2024-03-18T23:03:56.334 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:03:56.332+0000 7f55fb89a700 1 -- 172.21.15.175:0/1638267953 wait complete. 2024-03-18T23:03:56.569 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:03:56 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:03:56.569 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:03:56 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:03:56.569 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:03:56 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:03:56.569 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:03:56 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:03:56.569 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:03:56 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:03:56.569 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:03:56 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:03:56.569 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:03:56 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:03:56.569 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:03:56 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:03:56.816 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:03:56 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:03:56.816 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:03:56 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:03:56.816 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:03:56 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:03:56.816 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:03:56 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:03:56.816 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:03:56 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:03:56.816 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:03:56 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:03:56.816 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:03:56 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:03:56.816 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:03:56 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:03:56.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:03:56 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:03:56.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:03:56 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:03:56.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:03:56 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:03:56.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:03:56 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:03:56.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:03:56 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:03:56.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:03:56 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:03:56.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:03:56 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:03:56.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:03:56 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:03:56.911 DEBUG:teuthology.orchestra.run.smithi175:osd.8> sudo journalctl -f -n 0 -u ceph-c63591ea-e57a-11ee-95c9-87774f69a715@osd.8.service 2024-03-18T23:03:56.913 INFO:tasks.cephadm:Deploying osd.9 on smithi049 with /dev/vg_nvme/lv_1... 2024-03-18T23:03:56.914 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c63591ea-e57a-11ee-95c9-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-03-18T23:03:57.175 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T23:03:57.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:03:57 smithi175 ceph-mon[67352]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-03-18T23:03:57.462 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:03:57 smithi175 ceph-mon[67352]: from='osd.8 [v2:172.21.15.175:6816/1194026992,v1:172.21.15.175:6817/1194026992]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-18T23:03:57.462 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:03:57 smithi175 ceph-mon[67352]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-18T23:03:57.597 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:03:57 smithi049 ceph-mon[62991]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-03-18T23:03:57.598 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:03:57 smithi049 ceph-mon[62991]: from='osd.8 [v2:172.21.15.175:6816/1194026992,v1:172.21.15.175:6817/1194026992]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-18T23:03:57.598 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:03:57 smithi049 ceph-mon[62991]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-18T23:03:57.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:03:57 smithi139 ceph-mon[67282]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-03-18T23:03:57.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:03:57 smithi139 ceph-mon[67282]: from='osd.8 [v2:172.21.15.175:6816/1194026992,v1:172.21.15.175:6817/1194026992]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-18T23:03:57.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:03:57 smithi139 ceph-mon[67282]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-18T23:03:58.630 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:03:58 smithi049 ceph-mon[62991]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-03-18T23:03:58.630 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:03:58 smithi049 ceph-mon[62991]: osdmap e57: 9 total, 8 up, 9 in 2024-03-18T23:03:58.630 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:03:58 smithi049 ceph-mon[62991]: from='osd.8 [v2:172.21.15.175:6816/1194026992,v1:172.21.15.175:6817/1194026992]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]: dispatch 2024-03-18T23:03:58.630 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:03:58 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-18T23:03:58.630 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:03:58 smithi049 ceph-mon[62991]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]: dispatch 2024-03-18T23:03:58.711 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:03:58 smithi175 ceph-mon[67352]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-03-18T23:03:58.711 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:03:58 smithi175 ceph-mon[67352]: osdmap e57: 9 total, 8 up, 9 in 2024-03-18T23:03:58.712 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:03:58 smithi175 ceph-mon[67352]: from='osd.8 [v2:172.21.15.175:6816/1194026992,v1:172.21.15.175:6817/1194026992]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]: dispatch 2024-03-18T23:03:58.712 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:03:58 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-18T23:03:58.712 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:03:58 smithi175 ceph-mon[67352]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]: dispatch 2024-03-18T23:03:58.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:03:58 smithi139 ceph-mon[67282]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-03-18T23:03:58.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:03:58 smithi139 ceph-mon[67282]: osdmap e57: 9 total, 8 up, 9 in 2024-03-18T23:03:58.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:03:58 smithi139 ceph-mon[67282]: from='osd.8 [v2:172.21.15.175:6816/1194026992,v1:172.21.15.175:6817/1194026992]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]: dispatch 2024-03-18T23:03:58.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:03:58 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-18T23:03:58.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:03:58 smithi139 ceph-mon[67282]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]: dispatch 2024-03-18T23:03:59.212 INFO:journalctl@ceph.osd.8.smithi175.stdout:Mar 18 23:03:58 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-8[86041]: 2024-03-18T23:03:58.871+0000 7fa0631aa700 -1 osd.8 0 waiting for initial osdmap 2024-03-18T23:03:59.212 INFO:journalctl@ceph.osd.8.smithi175.stdout:Mar 18 23:03:58 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-8[86041]: 2024-03-18T23:03:58.881+0000 7fa05af95700 -1 osd.8 59 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-18T23:03:59.546 INFO:teuthology.orchestra.run.smithi049.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-03-18T23:03:59.546 INFO:teuthology.orchestra.run.smithi049.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-03-18T23:03:59.546 INFO:teuthology.orchestra.run.smithi049.stderr: stderr: 10+0 records in 2024-03-18T23:03:59.546 INFO:teuthology.orchestra.run.smithi049.stderr:10+0 records out 2024-03-18T23:03:59.546 INFO:teuthology.orchestra.run.smithi049.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00834701 s, 1.3 GB/s 2024-03-18T23:03:59.546 INFO:teuthology.orchestra.run.smithi049.stderr:--> Zapping successful for: 2024-03-18T23:03:59.600 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:03:59 smithi175 ceph-mon[67352]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-03-18T23:03:59.600 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:03:59 smithi175 ceph-mon[67352]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]': finished 2024-03-18T23:03:59.600 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:03:59 smithi175 ceph-mon[67352]: osdmap e58: 9 total, 8 up, 9 in 2024-03-18T23:03:59.600 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:03:59 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-18T23:03:59.601 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:03:59 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-18T23:03:59.601 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:03:59 smithi175 ceph-mon[67352]: osdmap e59: 9 total, 8 up, 9 in 2024-03-18T23:03:59.601 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:03:59 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-18T23:03:59.601 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:03:59 smithi175 ceph-mon[67352]: from='osd.8 ' entity='osd.8' 2024-03-18T23:03:59.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:03:59 smithi139 ceph-mon[67282]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-03-18T23:03:59.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:03:59 smithi139 ceph-mon[67282]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]': finished 2024-03-18T23:03:59.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:03:59 smithi139 ceph-mon[67282]: osdmap e58: 9 total, 8 up, 9 in 2024-03-18T23:03:59.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:03:59 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-18T23:03:59.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:03:59 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-18T23:03:59.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:03:59 smithi139 ceph-mon[67282]: osdmap e59: 9 total, 8 up, 9 in 2024-03-18T23:03:59.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:03:59 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-18T23:03:59.816 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:03:59 smithi139 ceph-mon[67282]: from='osd.8 ' entity='osd.8' 2024-03-18T23:03:59.881 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:03:59 smithi049 ceph-mon[62991]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-03-18T23:03:59.881 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:03:59 smithi049 ceph-mon[62991]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]': finished 2024-03-18T23:03:59.881 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:03:59 smithi049 ceph-mon[62991]: osdmap e58: 9 total, 8 up, 9 in 2024-03-18T23:03:59.881 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:03:59 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-18T23:03:59.881 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:03:59 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-18T23:03:59.881 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:03:59 smithi049 ceph-mon[62991]: osdmap e59: 9 total, 8 up, 9 in 2024-03-18T23:03:59.881 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:03:59 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-18T23:03:59.881 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:03:59 smithi049 ceph-mon[62991]: from='osd.8 ' entity='osd.8' 2024-03-18T23:04:00.329 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c63591ea-e57a-11ee-95c9-87774f69a715 -- ceph orch daemon add osd smithi049:vg_nvme/lv_1 2024-03-18T23:04:00.581 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T23:04:00.630 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:04:00 smithi049 ceph-mon[62991]: purged_snaps scrub starts 2024-03-18T23:04:00.630 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:04:00 smithi049 ceph-mon[62991]: purged_snaps scrub ok 2024-03-18T23:04:00.630 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:04:00 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-18T23:04:00.630 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:04:00 smithi049 ceph-mon[62991]: osd.8 [v2:172.21.15.175:6816/1194026992,v1:172.21.15.175:6817/1194026992] boot 2024-03-18T23:04:00.630 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:04:00 smithi049 ceph-mon[62991]: osdmap e60: 9 total, 9 up, 9 in 2024-03-18T23:04:00.630 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:04:00 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-18T23:04:00.630 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:04:00 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:04:00.631 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:04:00 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:04:00.631 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:04:00 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:04:00.631 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:04:00 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:04:00.631 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:04:00 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:04:00.631 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:04:00 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:04:00.631 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:04:00 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:04:00.631 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:04:00 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:04:00.631 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:04:00 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:04:00.712 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:04:00 smithi175 ceph-mon[67352]: purged_snaps scrub starts 2024-03-18T23:04:00.712 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:04:00 smithi175 ceph-mon[67352]: purged_snaps scrub ok 2024-03-18T23:04:00.712 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:04:00 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-18T23:04:00.712 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:04:00 smithi175 ceph-mon[67352]: osd.8 [v2:172.21.15.175:6816/1194026992,v1:172.21.15.175:6817/1194026992] boot 2024-03-18T23:04:00.712 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:04:00 smithi175 ceph-mon[67352]: osdmap e60: 9 total, 9 up, 9 in 2024-03-18T23:04:00.712 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:04:00 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-18T23:04:00.712 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:04:00 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:04:00.713 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:04:00 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:04:00.713 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:04:00 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:04:00.713 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:04:00 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:04:00.713 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:04:00 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:04:00.713 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:04:00 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:04:00.713 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:04:00 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:04:00.713 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:04:00 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:04:00.713 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:04:00 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:04:00.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:04:00 smithi139 ceph-mon[67282]: purged_snaps scrub starts 2024-03-18T23:04:00.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:04:00 smithi139 ceph-mon[67282]: purged_snaps scrub ok 2024-03-18T23:04:00.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:04:00 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-18T23:04:00.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:04:00 smithi139 ceph-mon[67282]: osd.8 [v2:172.21.15.175:6816/1194026992,v1:172.21.15.175:6817/1194026992] boot 2024-03-18T23:04:00.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:04:00 smithi139 ceph-mon[67282]: osdmap e60: 9 total, 9 up, 9 in 2024-03-18T23:04:00.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:04:00 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-18T23:04:00.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:04:00 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:04:00.816 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:04:00 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:04:00.816 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:04:00 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:04:00.816 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:04:00 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:04:00.816 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:04:00 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:04:00.816 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:04:00 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:04:00.816 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:04:00 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:04:00.816 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:04:00 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:04:00.816 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:04:00 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:04:02.039 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:04:01 smithi049 ceph-mon[62991]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-03-18T23:04:02.039 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:04:01 smithi049 ceph-mon[62991]: Detected new or changed devices on smithi175 2024-03-18T23:04:02.039 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:04:01 smithi049 ceph-mon[62991]: Adjusting osd_memory_target on smithi175 to 5693M 2024-03-18T23:04:02.039 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:04:01 smithi049 ceph-mon[62991]: osdmap e61: 9 total, 9 up, 9 in 2024-03-18T23:04:02.065 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:04:01 smithi139 ceph-mon[67282]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-03-18T23:04:02.065 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:04:01 smithi139 ceph-mon[67282]: Detected new or changed devices on smithi175 2024-03-18T23:04:02.065 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:04:01 smithi139 ceph-mon[67282]: Adjusting osd_memory_target on smithi175 to 5693M 2024-03-18T23:04:02.065 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:04:01 smithi139 ceph-mon[67282]: osdmap e61: 9 total, 9 up, 9 in 2024-03-18T23:04:02.211 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:04:01 smithi175 ceph-mon[67352]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-03-18T23:04:02.211 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:04:01 smithi175 ceph-mon[67352]: Detected new or changed devices on smithi175 2024-03-18T23:04:02.211 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:04:01 smithi175 ceph-mon[67352]: Adjusting osd_memory_target on smithi175 to 5693M 2024-03-18T23:04:02.212 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:04:01 smithi175 ceph-mon[67352]: osdmap e61: 9 total, 9 up, 9 in 2024-03-18T23:04:02.293 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:04:02.293+0000 7fa3443a0700 1 -- 172.21.15.49:0/3673429468 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa33c102dd0 msgr2=0x7fa33c109680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:04:02.293 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:04:02.293+0000 7fa3443a0700 1 --2- 172.21.15.49:0/3673429468 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa33c102dd0 0x7fa33c109680 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fa33400b0d0 tx=0x7fa33400b3e0 comp rx=0 tx=0).stop 2024-03-18T23:04:02.293 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:04:02.293+0000 7fa3443a0700 1 -- 172.21.15.49:0/3673429468 shutdown_connections 2024-03-18T23:04:02.294 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:04:02.293+0000 7fa3443a0700 1 --2- 172.21.15.49:0/3673429468 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa33c102dd0 0x7fa33c109680 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:04:02.294 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:04:02.293+0000 7fa3443a0700 1 --2- 172.21.15.49:0/3673429468 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa33c101230 0x7fa33c101650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:04:02.294 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:04:02.293+0000 7fa3443a0700 1 --2- 172.21.15.49:0/3673429468 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa33c102430 0x7fa33c102890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:04:02.294 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:04:02.293+0000 7fa3443a0700 1 -- 172.21.15.49:0/3673429468 >> 172.21.15.49:0/3673429468 conn(0x7fa33c0fc7d0 msgr2=0x7fa33c0fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:04:02.294 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:04:02.293+0000 7fa3443a0700 1 -- 172.21.15.49:0/3673429468 shutdown_connections 2024-03-18T23:04:02.294 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:04:02.293+0000 7fa3443a0700 1 -- 172.21.15.49:0/3673429468 wait complete. 2024-03-18T23:04:02.296 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:04:02.294+0000 7fa3443a0700 1 Processor -- start 2024-03-18T23:04:02.296 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:04:02.294+0000 7fa3443a0700 1 -- start start 2024-03-18T23:04:02.296 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:04:02.294+0000 7fa3443a0700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa33c101230 0x7fa33c1051c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:04:02.296 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:04:02.294+0000 7fa3443a0700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa33c102430 0x7fa33c103810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:04:02.296 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:04:02.294+0000 7fa3443a0700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa33c102dd0 0x7fa33c103d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:04:02.296 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:04:02.294+0000 7fa3443a0700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fa33c10ce10 con 0x7fa33c102430 2024-03-18T23:04:02.296 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:04:02.294+0000 7fa3443a0700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fa33c10cc90 con 0x7fa33c101230 2024-03-18T23:04:02.296 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:04:02.294+0000 7fa3443a0700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa33c10cf90 con 0x7fa33c102dd0 2024-03-18T23:04:02.296 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:04:02.294+0000 7fa34193b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa33c102430 0x7fa33c103810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:04:02.296 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:04:02.294+0000 7fa34213c700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa33c101230 0x7fa33c1051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:04:02.296 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:04:02.294+0000 7fa34193b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa33c102430 0x7fa33c103810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:37798/0 (socket says 172.21.15.49:37798) 2024-03-18T23:04:02.297 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:04:02.294+0000 7fa34213c700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa33c101230 0x7fa33c1051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.49:52404/0 (socket says 172.21.15.49:52404) 2024-03-18T23:04:02.297 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:04:02.294+0000 7fa34193b700 1 -- 172.21.15.49:0/371120141 learned_addr learned my addr 172.21.15.49:0/371120141 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:04:02.297 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:04:02.294+0000 7fa34293d700 1 --2- 172.21.15.49:0/371120141 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa33c102dd0 0x7fa33c103d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:04:02.297 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:04:02.295+0000 7fa34193b700 1 -- 172.21.15.49:0/371120141 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa33c102dd0 msgr2=0x7fa33c103d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:04:02.297 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:04:02.295+0000 7fa34193b700 1 --2- 172.21.15.49:0/371120141 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa33c102dd0 0x7fa33c103d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:04:02.297 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:04:02.295+0000 7fa34193b700 1 -- 172.21.15.49:0/371120141 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa33c101230 msgr2=0x7fa33c1051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:04:02.297 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:04:02.295+0000 7fa34193b700 1 --2- 172.21.15.49:0/371120141 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa33c101230 0x7fa33c1051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:04:02.297 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:04:02.295+0000 7fa34193b700 1 -- 172.21.15.49:0/371120141 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa33c104600 con 0x7fa33c102430 2024-03-18T23:04:02.297 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:04:02.295+0000 7fa34213c700 1 --2- 172.21.15.49:0/371120141 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa33c101230 0x7fa33c1051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:04:02.297 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:04:02.295+0000 7fa34193b700 1 --2- 172.21.15.49:0/371120141 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa33c102430 0x7fa33c103810 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fa32c00e9a0 tx=0x7fa32c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:04:02.297 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:04:02.295+0000 7fa3337fe700 1 -- 172.21.15.49:0/371120141 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa32c00ca10 con 0x7fa33c102430 2024-03-18T23:04:02.297 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:04:02.295+0000 7fa3337fe700 1 -- 172.21.15.49:0/371120141 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa32c00cbb0 con 0x7fa33c102430 2024-03-18T23:04:02.297 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:04:02.295+0000 7fa3337fe700 1 -- 172.21.15.49:0/371120141 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa32c018680 con 0x7fa33c102430 2024-03-18T23:04:02.297 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:04:02.295+0000 7fa3443a0700 1 -- 172.21.15.49:0/371120141 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa33c1a2650 con 0x7fa33c102430 2024-03-18T23:04:02.298 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:04:02.295+0000 7fa3443a0700 1 -- 172.21.15.49:0/371120141 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa33c1a2b90 con 0x7fa33c102430 2024-03-18T23:04:02.298 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:04:02.296+0000 7fa3443a0700 1 -- 172.21.15.49:0/371120141 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa33c068fd0 con 0x7fa33c102430 2024-03-18T23:04:02.298 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:04:02.298+0000 7fa3337fe700 1 -- 172.21.15.49:0/371120141 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7fa32c010430 con 0x7fa33c102430 2024-03-18T23:04:02.299 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:04:02.298+0000 7fa3337fe700 1 --2- 172.21.15.49:0/371120141 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fa3280a2910 0x7fa3280a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:04:02.299 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:04:02.298+0000 7fa34213c700 1 --2- 172.21.15.49:0/371120141 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fa3280a2910 0x7fa3280a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:04:02.299 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:04:02.298+0000 7fa3337fe700 1 -- 172.21.15.49:0/371120141 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5858+0+0 (secure 0 0 0) 0x7fa32c0ce720 con 0x7fa33c102430 2024-03-18T23:04:02.299 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:04:02.299+0000 7fa34213c700 1 --2- 172.21.15.49:0/371120141 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fa3280a2910 0x7fa3280a4dd0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fa33c102290 tx=0x7fa338009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:04:02.302 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:04:02.302+0000 7fa3337fe700 1 -- 172.21.15.49:0/371120141 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fa32c09c150 con 0x7fa33c102430 2024-03-18T23:04:02.518 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:04:02.516+0000 7fa3443a0700 1 -- 172.21.15.49:0/371120141 --> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi049:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fa33c0fe920 con 0x7fa3280a2910 2024-03-18T23:04:03.065 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:04:02 smithi139 ceph-mon[67282]: osdmap e62: 9 total, 9 up, 9 in 2024-03-18T23:04:03.065 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:04:02 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-18T23:04:03.065 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:04:02 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-18T23:04:03.066 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:04:02 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:04:03.130 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:04:02 smithi049 ceph-mon[62991]: osdmap e62: 9 total, 9 up, 9 in 2024-03-18T23:04:03.130 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:04:02 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-18T23:04:03.131 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:04:02 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-18T23:04:03.131 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:04:02 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:04:03.212 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:04:02 smithi175 ceph-mon[67352]: osdmap e62: 9 total, 9 up, 9 in 2024-03-18T23:04:03.212 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:04:02 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-18T23:04:03.212 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:04:02 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-18T23:04:03.212 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:04:02 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:04:04.065 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:04:03 smithi139 ceph-mon[67282]: pgmap v203: 1 pgs: 1 remapped+peering; 577 KiB data, 275 MiB used, 804 GiB / 805 GiB avail 2024-03-18T23:04:04.065 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:04:03 smithi139 ceph-mon[67282]: from='client.14421 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi049:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T23:04:04.065 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:04:03 smithi139 ceph-mon[67282]: osdmap e63: 9 total, 9 up, 9 in 2024-03-18T23:04:04.130 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:04:03 smithi049 ceph-mon[62991]: pgmap v203: 1 pgs: 1 remapped+peering; 577 KiB data, 275 MiB used, 804 GiB / 805 GiB avail 2024-03-18T23:04:04.130 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:04:03 smithi049 ceph-mon[62991]: from='client.14421 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi049:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T23:04:04.130 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:04:03 smithi049 ceph-mon[62991]: osdmap e63: 9 total, 9 up, 9 in 2024-03-18T23:04:04.211 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:04:03 smithi175 ceph-mon[67352]: pgmap v203: 1 pgs: 1 remapped+peering; 577 KiB data, 275 MiB used, 804 GiB / 805 GiB avail 2024-03-18T23:04:04.211 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:04:03 smithi175 ceph-mon[67352]: from='client.14421 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi049:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T23:04:04.212 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:04:03 smithi175 ceph-mon[67352]: osdmap e63: 9 total, 9 up, 9 in 2024-03-18T23:04:04.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:04:04 smithi049 ceph-mon[62991]: osdmap e64: 9 total, 9 up, 9 in 2024-03-18T23:04:05.065 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:04:04 smithi139 ceph-mon[67282]: osdmap e64: 9 total, 9 up, 9 in 2024-03-18T23:04:05.211 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:04:04 smithi175 ceph-mon[67352]: osdmap e64: 9 total, 9 up, 9 in 2024-03-18T23:04:06.065 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:04:05 smithi139 ceph-mon[67282]: pgmap v206: 1 pgs: 1 remapped+peering; 577 KiB data, 275 MiB used, 804 GiB / 805 GiB avail 2024-03-18T23:04:06.065 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:04:05 smithi139 ceph-mon[67282]: from='client.? 172.21.15.49:0/528287869' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "edb5ffef-b6a2-4941-b5ab-40f982ad30d5"}]: dispatch 2024-03-18T23:04:06.065 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:04:05 smithi139 ceph-mon[67282]: from='client.? 172.21.15.49:0/528287869' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "edb5ffef-b6a2-4941-b5ab-40f982ad30d5"}]': finished 2024-03-18T23:04:06.065 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:04:05 smithi139 ceph-mon[67282]: osdmap e65: 10 total, 9 up, 10 in 2024-03-18T23:04:06.065 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:04:05 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-18T23:04:06.130 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:04:05 smithi049 ceph-mon[62991]: pgmap v206: 1 pgs: 1 remapped+peering; 577 KiB data, 275 MiB used, 804 GiB / 805 GiB avail 2024-03-18T23:04:06.130 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:04:05 smithi049 ceph-mon[62991]: from='client.? 172.21.15.49:0/528287869' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "edb5ffef-b6a2-4941-b5ab-40f982ad30d5"}]: dispatch 2024-03-18T23:04:06.130 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:04:05 smithi049 ceph-mon[62991]: from='client.? 172.21.15.49:0/528287869' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "edb5ffef-b6a2-4941-b5ab-40f982ad30d5"}]': finished 2024-03-18T23:04:06.130 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:04:05 smithi049 ceph-mon[62991]: osdmap e65: 10 total, 9 up, 10 in 2024-03-18T23:04:06.130 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:04:05 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-18T23:04:06.211 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:04:05 smithi175 ceph-mon[67352]: pgmap v206: 1 pgs: 1 remapped+peering; 577 KiB data, 275 MiB used, 804 GiB / 805 GiB avail 2024-03-18T23:04:06.212 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:04:05 smithi175 ceph-mon[67352]: from='client.? 172.21.15.49:0/528287869' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "edb5ffef-b6a2-4941-b5ab-40f982ad30d5"}]: dispatch 2024-03-18T23:04:06.212 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:04:05 smithi175 ceph-mon[67352]: from='client.? 172.21.15.49:0/528287869' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "edb5ffef-b6a2-4941-b5ab-40f982ad30d5"}]': finished 2024-03-18T23:04:06.212 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:04:05 smithi175 ceph-mon[67352]: osdmap e65: 10 total, 9 up, 10 in 2024-03-18T23:04:06.212 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:04:05 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-18T23:04:07.065 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:04:06 smithi139 ceph-mon[67282]: from='client.? 172.21.15.49:0/3133676359' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-18T23:04:07.129 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:04:06 smithi049 ceph-mon[62991]: from='client.? 172.21.15.49:0/3133676359' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-18T23:04:07.211 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:04:06 smithi175 ceph-mon[67352]: from='client.? 172.21.15.49:0/3133676359' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-18T23:04:08.065 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:04:07 smithi139 ceph-mon[67282]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail; 139 KiB/s, 0 objects/s recovering 2024-03-18T23:04:08.130 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:04:07 smithi049 ceph-mon[62991]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail; 139 KiB/s, 0 objects/s recovering 2024-03-18T23:04:08.211 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:04:07 smithi175 ceph-mon[67352]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail; 139 KiB/s, 0 objects/s recovering 2024-03-18T23:04:09.065 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:04:08 smithi139 ceph-mon[67282]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-18T23:04:09.070 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:04:08 smithi049 ceph-mon[62991]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-18T23:04:09.113 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:04:08 smithi175 ceph-mon[67352]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-18T23:04:11.130 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:04:10 smithi049 ceph-mon[62991]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail; 81 KiB/s, 0 objects/s recovering 2024-03-18T23:04:11.211 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:04:10 smithi175 ceph-mon[67352]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail; 81 KiB/s, 0 objects/s recovering 2024-03-18T23:04:11.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:04:10 smithi139 ceph-mon[67282]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail; 81 KiB/s, 0 objects/s recovering 2024-03-18T23:04:13.083 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:04:12 smithi049 ceph-mon[62991]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail; 71 KiB/s, 0 objects/s recovering 2024-03-18T23:04:13.211 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:04:12 smithi175 ceph-mon[67352]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail; 71 KiB/s, 0 objects/s recovering 2024-03-18T23:04:13.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:04:12 smithi139 ceph-mon[67282]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail; 71 KiB/s, 0 objects/s recovering 2024-03-18T23:04:14.008 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:04:13 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-03-18T23:04:14.008 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:04:13 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:04:14.008 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:04:13 smithi049 ceph-mon[62991]: Deploying daemon osd.9 on smithi049 2024-03-18T23:04:14.211 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:04:13 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-03-18T23:04:14.211 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:04:13 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:04:14.212 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:04:13 smithi175 ceph-mon[67352]: Deploying daemon osd.9 on smithi049 2024-03-18T23:04:14.307 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:04:13 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-03-18T23:04:14.307 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:04:13 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:04:14.307 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:04:13 smithi139 ceph-mon[67282]: Deploying daemon osd.9 on smithi049 2024-03-18T23:04:15.043 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:04:14 smithi049 ceph-mon[62991]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-18T23:04:15.211 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:04:14 smithi175 ceph-mon[67352]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-18T23:04:15.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:04:14 smithi139 ceph-mon[67282]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-18T23:04:17.211 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:04:16 smithi175 ceph-mon[67352]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail 2024-03-18T23:04:17.216 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:04:16 smithi049 ceph-mon[62991]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail 2024-03-18T23:04:17.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:04:16 smithi139 ceph-mon[67282]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail 2024-03-18T23:04:19.130 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:04:18 smithi049 ceph-mon[62991]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail 2024-03-18T23:04:19.211 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:04:18 smithi175 ceph-mon[67352]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail 2024-03-18T23:04:19.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:04:18 smithi139 ceph-mon[67282]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail 2024-03-18T23:04:20.211 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:04:20 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:04:20.212 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:04:20 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:04:20.212 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:04:20 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:04:20.318 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:04:20 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:04:20.318 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:04:20 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:04:20.318 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:04:20 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:04:20.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:04:20 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:04:20.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:04:20 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:04:20.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:04:20 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:04:21.257 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:04:21 smithi049 ceph-mon[62991]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail 2024-03-18T23:04:21.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:04:21 smithi139 ceph-mon[67282]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail 2024-03-18T23:04:21.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:04:21 smithi175 ceph-mon[67352]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail 2024-03-18T23:04:22.524 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:04:22.523+0000 7fa3337fe700 1 -- 172.21.15.49:0/371120141 <== mgr.14150 v2:172.21.15.49:6800/2083805211 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fa33c0fe920 con 0x7fa3280a2910 2024-03-18T23:04:22.525 INFO:teuthology.orchestra.run.smithi049.stdout:Created osd(s) 9 on host 'smithi049' 2024-03-18T23:04:22.527 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:04:22.526+0000 7fa3443a0700 1 -- 172.21.15.49:0/371120141 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fa3280a2910 msgr2=0x7fa3280a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:04:22.527 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:04:22.526+0000 7fa3443a0700 1 --2- 172.21.15.49:0/371120141 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fa3280a2910 0x7fa3280a4dd0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fa33c102290 tx=0x7fa338009260 comp rx=0 tx=0).stop 2024-03-18T23:04:22.528 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:04:22.526+0000 7fa3443a0700 1 -- 172.21.15.49:0/371120141 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa33c102430 msgr2=0x7fa33c103810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:04:22.528 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:04:22.526+0000 7fa3443a0700 1 --2- 172.21.15.49:0/371120141 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa33c102430 0x7fa33c103810 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fa32c00e9a0 tx=0x7fa32c00ecb0 comp rx=0 tx=0).stop 2024-03-18T23:04:22.528 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:04:22.526+0000 7fa3443a0700 1 -- 172.21.15.49:0/371120141 shutdown_connections 2024-03-18T23:04:22.528 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:04:22.526+0000 7fa3443a0700 1 --2- 172.21.15.49:0/371120141 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fa3280a2910 0x7fa3280a4dd0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:04:22.528 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:04:22.526+0000 7fa3443a0700 1 --2- 172.21.15.49:0/371120141 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa33c102dd0 0x7fa33c103d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:04:22.528 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:04:22.526+0000 7fa3443a0700 1 --2- 172.21.15.49:0/371120141 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa33c101230 0x7fa33c1051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:04:22.528 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:04:22.526+0000 7fa3443a0700 1 --2- 172.21.15.49:0/371120141 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa33c102430 0x7fa33c103810 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:04:22.528 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:04:22.526+0000 7fa3443a0700 1 -- 172.21.15.49:0/371120141 >> 172.21.15.49:0/371120141 conn(0x7fa33c0fc7d0 msgr2=0x7fa33c0fe210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:04:22.528 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:04:22.526+0000 7fa3443a0700 1 -- 172.21.15.49:0/371120141 shutdown_connections 2024-03-18T23:04:22.528 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:04:22.526+0000 7fa3443a0700 1 -- 172.21.15.49:0/371120141 wait complete. 2024-03-18T23:04:23.172 DEBUG:teuthology.orchestra.run.smithi049:osd.9> sudo journalctl -f -n 0 -u ceph-c63591ea-e57a-11ee-95c9-87774f69a715@osd.9.service 2024-03-18T23:04:23.174 INFO:tasks.cephadm:Deploying osd.10 on smithi139 with /dev/vg_nvme/lv_1... 2024-03-18T23:04:23.174 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c63591ea-e57a-11ee-95c9-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-03-18T23:04:23.419 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/mon.b/config 2024-03-18T23:04:23.759 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:04:23 smithi139 ceph-mon[67282]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail 2024-03-18T23:04:23.760 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:04:23 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:04:23.760 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:04:23 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:04:23.760 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:04:23 smithi139 ceph-mon[67282]: from='osd.9 [v2:172.21.15.49:6826/2742084884,v1:172.21.15.49:6827/2742084884]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-03-18T23:04:23.760 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:04:23 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:04:23.760 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:04:23 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:04:23.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:04:23 smithi049 ceph-mon[62991]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail 2024-03-18T23:04:23.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:04:23 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:04:23.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:04:23 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:04:23.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:04:23 smithi049 ceph-mon[62991]: from='osd.9 [v2:172.21.15.49:6826/2742084884,v1:172.21.15.49:6827/2742084884]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-03-18T23:04:23.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:04:23 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:04:23.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:04:23 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:04:23.961 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:04:23 smithi175 ceph-mon[67352]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail 2024-03-18T23:04:23.961 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:04:23 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:04:23.961 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:04:23 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:04:23.961 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:04:23 smithi175 ceph-mon[67352]: from='osd.9 [v2:172.21.15.49:6826/2742084884,v1:172.21.15.49:6827/2742084884]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-03-18T23:04:23.962 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:04:23 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:04:23.962 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:04:23 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:04:24.639 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:04:24 smithi139 ceph-mon[67282]: from='osd.9 [v2:172.21.15.49:6826/2742084884,v1:172.21.15.49:6827/2742084884]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-03-18T23:04:24.640 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:04:24 smithi139 ceph-mon[67282]: osdmap e66: 10 total, 9 up, 10 in 2024-03-18T23:04:24.640 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:04:24 smithi139 ceph-mon[67282]: from='osd.9 [v2:172.21.15.49:6826/2742084884,v1:172.21.15.49:6827/2742084884]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi049", "root=default"]}]: dispatch 2024-03-18T23:04:24.640 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:04:24 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-18T23:04:24.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:04:24 smithi049 ceph-mon[62991]: from='osd.9 [v2:172.21.15.49:6826/2742084884,v1:172.21.15.49:6827/2742084884]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-03-18T23:04:24.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:04:24 smithi049 ceph-mon[62991]: osdmap e66: 10 total, 9 up, 10 in 2024-03-18T23:04:24.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:04:24 smithi049 ceph-mon[62991]: from='osd.9 [v2:172.21.15.49:6826/2742084884,v1:172.21.15.49:6827/2742084884]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi049", "root=default"]}]: dispatch 2024-03-18T23:04:24.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:04:24 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-18T23:04:24.881 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 18 23:04:24 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-9[98163]: 2024-03-18T23:04:24.518+0000 7ff64dc78700 -1 osd.9 0 waiting for initial osdmap 2024-03-18T23:04:24.881 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 18 23:04:24 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-9[98163]: 2024-03-18T23:04:24.532+0000 7ff649a6b700 -1 osd.9 67 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-18T23:04:24.961 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:04:24 smithi175 ceph-mon[67352]: from='osd.9 [v2:172.21.15.49:6826/2742084884,v1:172.21.15.49:6827/2742084884]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-03-18T23:04:24.961 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:04:24 smithi175 ceph-mon[67352]: osdmap e66: 10 total, 9 up, 10 in 2024-03-18T23:04:24.961 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:04:24 smithi175 ceph-mon[67352]: from='osd.9 [v2:172.21.15.49:6826/2742084884,v1:172.21.15.49:6827/2742084884]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi049", "root=default"]}]: dispatch 2024-03-18T23:04:24.962 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:04:24 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-18T23:04:25.630 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:04:25 smithi049 ceph-mon[62991]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail 2024-03-18T23:04:25.630 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:04:25 smithi049 ceph-mon[62991]: from='osd.9 [v2:172.21.15.49:6826/2742084884,v1:172.21.15.49:6827/2742084884]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi049", "root=default"]}]': finished 2024-03-18T23:04:25.630 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:04:25 smithi049 ceph-mon[62991]: osdmap e67: 10 total, 9 up, 10 in 2024-03-18T23:04:25.631 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:04:25 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-18T23:04:25.631 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:04:25 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-18T23:04:25.675 INFO:teuthology.orchestra.run.smithi139.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-03-18T23:04:25.675 INFO:teuthology.orchestra.run.smithi139.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-03-18T23:04:25.675 INFO:teuthology.orchestra.run.smithi139.stderr: stderr: 10+0 records in 2024-03-18T23:04:25.675 INFO:teuthology.orchestra.run.smithi139.stderr:10+0 records out 2024-03-18T23:04:25.675 INFO:teuthology.orchestra.run.smithi139.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0198547 s, 528 MB/s 2024-03-18T23:04:25.675 INFO:teuthology.orchestra.run.smithi139.stderr:--> Zapping successful for: 2024-03-18T23:04:25.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:04:25 smithi139 ceph-mon[67282]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail 2024-03-18T23:04:25.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:04:25 smithi139 ceph-mon[67282]: from='osd.9 [v2:172.21.15.49:6826/2742084884,v1:172.21.15.49:6827/2742084884]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi049", "root=default"]}]': finished 2024-03-18T23:04:25.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:04:25 smithi139 ceph-mon[67282]: osdmap e67: 10 total, 9 up, 10 in 2024-03-18T23:04:25.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:04:25 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-18T23:04:25.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:04:25 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-18T23:04:25.961 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:04:25 smithi175 ceph-mon[67352]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail 2024-03-18T23:04:25.962 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:04:25 smithi175 ceph-mon[67352]: from='osd.9 [v2:172.21.15.49:6826/2742084884,v1:172.21.15.49:6827/2742084884]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi049", "root=default"]}]': finished 2024-03-18T23:04:25.962 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:04:25 smithi175 ceph-mon[67352]: osdmap e67: 10 total, 9 up, 10 in 2024-03-18T23:04:25.962 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:04:25 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-18T23:04:25.962 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:04:25 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-18T23:04:26.479 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c63591ea-e57a-11ee-95c9-87774f69a715 -- ceph orch daemon add osd smithi139:vg_nvme/lv_1 2024-03-18T23:04:26.732 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/mon.b/config 2024-03-18T23:04:26.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:04:26 smithi139 ceph-mon[67282]: purged_snaps scrub starts 2024-03-18T23:04:26.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:04:26 smithi139 ceph-mon[67282]: purged_snaps scrub ok 2024-03-18T23:04:26.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:04:26 smithi139 ceph-mon[67282]: osd.9 [v2:172.21.15.49:6826/2742084884,v1:172.21.15.49:6827/2742084884] boot 2024-03-18T23:04:26.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:04:26 smithi139 ceph-mon[67282]: osdmap e68: 10 total, 10 up, 10 in 2024-03-18T23:04:26.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:04:26 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-18T23:04:26.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:04:26 smithi049 ceph-mon[62991]: purged_snaps scrub starts 2024-03-18T23:04:26.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:04:26 smithi049 ceph-mon[62991]: purged_snaps scrub ok 2024-03-18T23:04:26.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:04:26 smithi049 ceph-mon[62991]: osd.9 [v2:172.21.15.49:6826/2742084884,v1:172.21.15.49:6827/2742084884] boot 2024-03-18T23:04:26.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:04:26 smithi049 ceph-mon[62991]: osdmap e68: 10 total, 10 up, 10 in 2024-03-18T23:04:26.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:04:26 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-18T23:04:26.961 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:04:26 smithi175 ceph-mon[67352]: purged_snaps scrub starts 2024-03-18T23:04:26.962 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:04:26 smithi175 ceph-mon[67352]: purged_snaps scrub ok 2024-03-18T23:04:26.962 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:04:26 smithi175 ceph-mon[67352]: osd.9 [v2:172.21.15.49:6826/2742084884,v1:172.21.15.49:6827/2742084884] boot 2024-03-18T23:04:26.962 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:04:26 smithi175 ceph-mon[67352]: osdmap e68: 10 total, 10 up, 10 in 2024-03-18T23:04:26.962 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:04:26 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-18T23:04:27.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:04:27 smithi139 ceph-mon[67282]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-03-18T23:04:27.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:04:27 smithi139 ceph-mon[67282]: osdmap e69: 10 total, 10 up, 10 in 2024-03-18T23:04:27.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:04:27 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:04:27.816 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:04:27 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:04:27.816 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:04:27 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:04:27.816 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:04:27 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:04:27.816 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:04:27 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:04:27.816 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:04:27 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:04:27.816 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:04:27 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:04:27.816 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:04:27 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:04:27.816 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:04:27 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:04:27.816 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:04:27 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:04:27.816 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:04:27 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:04:27.816 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:04:27 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:04:27.816 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:04:27 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:04:27.816 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:04:27 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:04:27.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:04:27 smithi049 ceph-mon[62991]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-03-18T23:04:27.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:04:27 smithi049 ceph-mon[62991]: osdmap e69: 10 total, 10 up, 10 in 2024-03-18T23:04:27.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:04:27 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:04:27.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:04:27 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:04:27.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:04:27 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:04:27.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:04:27 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:04:27.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:04:27 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:04:27.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:04:27 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:04:27.881 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:04:27 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:04:27.881 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:04:27 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:04:27.881 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:04:27 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:04:27.881 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:04:27 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:04:27.881 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:04:27 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:04:27.881 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:04:27 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:04:27.881 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:04:27 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:04:27.881 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:04:27 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:04:27.962 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:04:27 smithi175 ceph-mon[67352]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-03-18T23:04:27.962 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:04:27 smithi175 ceph-mon[67352]: osdmap e69: 10 total, 10 up, 10 in 2024-03-18T23:04:27.962 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:04:27 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:04:27.962 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:04:27 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:04:27.962 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:04:27 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:04:27.962 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:04:27 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:04:27.962 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:04:27 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:04:27.962 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:04:27 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:04:27.962 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:04:27 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:04:27.963 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:04:27 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:04:27.963 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:04:27 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:04:27.963 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:04:27 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:04:27.963 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:04:27 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:04:27.963 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:04:27 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:04:27.963 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:04:27 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:04:27.963 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:04:27 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:04:28.484 INFO:teuthology.orchestra.run.smithi139.stderr:2024-03-18T23:04:28.482+0000 7f0a98603700 1 -- 172.21.15.139:0/842985130 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0a90102380 msgr2=0x7f0a901027a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:04:28.484 INFO:teuthology.orchestra.run.smithi139.stderr:2024-03-18T23:04:28.482+0000 7f0a98603700 1 --2- 172.21.15.139:0/842985130 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0a90102380 0x7f0a901027a0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f0a8c0099c0 tx=0x7f0a8c009cd0 comp rx=0 tx=0).stop 2024-03-18T23:04:28.484 INFO:teuthology.orchestra.run.smithi139.stderr:2024-03-18T23:04:28.482+0000 7f0a98603700 1 -- 172.21.15.139:0/842985130 shutdown_connections 2024-03-18T23:04:28.484 INFO:teuthology.orchestra.run.smithi139.stderr:2024-03-18T23:04:28.482+0000 7f0a98603700 1 --2- 172.21.15.139:0/842985130 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0a90103f20 0x7f0a9010a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:04:28.484 INFO:teuthology.orchestra.run.smithi139.stderr:2024-03-18T23:04:28.482+0000 7f0a98603700 1 --2- 172.21.15.139:0/842985130 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0a90102380 0x7f0a901027a0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:04:28.484 INFO:teuthology.orchestra.run.smithi139.stderr:2024-03-18T23:04:28.482+0000 7f0a98603700 1 --2- 172.21.15.139:0/842985130 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0a90103580 0x7f0a901039e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:04:28.484 INFO:teuthology.orchestra.run.smithi139.stderr:2024-03-18T23:04:28.482+0000 7f0a98603700 1 -- 172.21.15.139:0/842985130 >> 172.21.15.139:0/842985130 conn(0x7f0a900fd960 msgr2=0x7f0a900ffd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:04:28.485 INFO:teuthology.orchestra.run.smithi139.stderr:2024-03-18T23:04:28.482+0000 7f0a98603700 1 -- 172.21.15.139:0/842985130 shutdown_connections 2024-03-18T23:04:28.485 INFO:teuthology.orchestra.run.smithi139.stderr:2024-03-18T23:04:28.482+0000 7f0a98603700 1 -- 172.21.15.139:0/842985130 wait complete. 2024-03-18T23:04:28.485 INFO:teuthology.orchestra.run.smithi139.stderr:2024-03-18T23:04:28.483+0000 7f0a98603700 1 Processor -- start 2024-03-18T23:04:28.485 INFO:teuthology.orchestra.run.smithi139.stderr:2024-03-18T23:04:28.483+0000 7f0a98603700 1 -- start start 2024-03-18T23:04:28.485 INFO:teuthology.orchestra.run.smithi139.stderr:2024-03-18T23:04:28.483+0000 7f0a98603700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0a90102380 0x7f0a900760b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:04:28.485 INFO:teuthology.orchestra.run.smithi139.stderr:2024-03-18T23:04:28.483+0000 7f0a98603700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0a90103580 0x7f0a90074700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:04:28.485 INFO:teuthology.orchestra.run.smithi139.stderr:2024-03-18T23:04:28.483+0000 7f0a98603700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0a90103f20 0x7f0a90074c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:04:28.485 INFO:teuthology.orchestra.run.smithi139.stderr:2024-03-18T23:04:28.483+0000 7f0a98603700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f0a9010df60 con 0x7f0a90103f20 2024-03-18T23:04:28.485 INFO:teuthology.orchestra.run.smithi139.stderr:2024-03-18T23:04:28.483+0000 7f0a98603700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f0a9010dde0 con 0x7f0a90103580 2024-03-18T23:04:28.485 INFO:teuthology.orchestra.run.smithi139.stderr:2024-03-18T23:04:28.483+0000 7f0a98603700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f0a9010e0e0 con 0x7f0a90102380 2024-03-18T23:04:28.485 INFO:teuthology.orchestra.run.smithi139.stderr:2024-03-18T23:04:28.483+0000 7f0a95b9e700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0a90103580 0x7f0a90074700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:04:28.485 INFO:teuthology.orchestra.run.smithi139.stderr:2024-03-18T23:04:28.483+0000 7f0a95b9e700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0a90103580 0x7f0a90074700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:35054/0 (socket says 172.21.15.139:35054) 2024-03-18T23:04:28.486 INFO:teuthology.orchestra.run.smithi139.stderr:2024-03-18T23:04:28.483+0000 7f0a95b9e700 1 -- 172.21.15.139:0/532707655 learned_addr learned my addr 172.21.15.139:0/532707655 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-03-18T23:04:28.486 INFO:teuthology.orchestra.run.smithi139.stderr:2024-03-18T23:04:28.483+0000 7f0a9639f700 1 --2- 172.21.15.139:0/532707655 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0a90102380 0x7f0a900760b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:04:28.486 INFO:teuthology.orchestra.run.smithi139.stderr:2024-03-18T23:04:28.483+0000 7f0a96ba0700 1 --2- 172.21.15.139:0/532707655 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0a90103f20 0x7f0a90074c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:04:28.486 INFO:teuthology.orchestra.run.smithi139.stderr:2024-03-18T23:04:28.483+0000 7f0a95b9e700 1 -- 172.21.15.139:0/532707655 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0a90102380 msgr2=0x7f0a900760b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:04:28.486 INFO:teuthology.orchestra.run.smithi139.stderr:2024-03-18T23:04:28.483+0000 7f0a95b9e700 1 --2- 172.21.15.139:0/532707655 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0a90102380 0x7f0a900760b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:04:28.486 INFO:teuthology.orchestra.run.smithi139.stderr:2024-03-18T23:04:28.483+0000 7f0a95b9e700 1 -- 172.21.15.139:0/532707655 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0a90103f20 msgr2=0x7f0a90074c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:04:28.486 INFO:teuthology.orchestra.run.smithi139.stderr:2024-03-18T23:04:28.483+0000 7f0a95b9e700 1 --2- 172.21.15.139:0/532707655 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0a90103f20 0x7f0a90074c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:04:28.486 INFO:teuthology.orchestra.run.smithi139.stderr:2024-03-18T23:04:28.483+0000 7f0a95b9e700 1 -- 172.21.15.139:0/532707655 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a900754f0 con 0x7f0a90103580 2024-03-18T23:04:28.486 INFO:teuthology.orchestra.run.smithi139.stderr:2024-03-18T23:04:28.483+0000 7f0a96ba0700 1 --2- 172.21.15.139:0/532707655 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0a90103f20 0x7f0a90074c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T23:04:28.486 INFO:teuthology.orchestra.run.smithi139.stderr:2024-03-18T23:04:28.484+0000 7f0a95b9e700 1 --2- 172.21.15.139:0/532707655 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0a90103580 0x7f0a90074700 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f0a8000d980 tx=0x7f0a8000dc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:04:28.486 INFO:teuthology.orchestra.run.smithi139.stderr:2024-03-18T23:04:28.484+0000 7f0a877fe700 1 -- 172.21.15.139:0/532707655 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a80017070 con 0x7f0a90103580 2024-03-18T23:04:28.486 INFO:teuthology.orchestra.run.smithi139.stderr:2024-03-18T23:04:28.484+0000 7f0a98603700 1 -- 172.21.15.139:0/532707655 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a901a26a0 con 0x7f0a90103580 2024-03-18T23:04:28.487 INFO:teuthology.orchestra.run.smithi139.stderr:2024-03-18T23:04:28.484+0000 7f0a877fe700 1 -- 172.21.15.139:0/532707655 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0a80005020 con 0x7f0a90103580 2024-03-18T23:04:28.487 INFO:teuthology.orchestra.run.smithi139.stderr:2024-03-18T23:04:28.484+0000 7f0a877fe700 1 -- 172.21.15.139:0/532707655 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a80009bc0 con 0x7f0a90103580 2024-03-18T23:04:28.487 INFO:teuthology.orchestra.run.smithi139.stderr:2024-03-18T23:04:28.484+0000 7f0a98603700 1 -- 172.21.15.139:0/532707655 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a901a2b60 con 0x7f0a90103580 2024-03-18T23:04:28.488 INFO:teuthology.orchestra.run.smithi139.stderr:2024-03-18T23:04:28.485+0000 7f0a98603700 1 -- 172.21.15.139:0/532707655 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a90068fd0 con 0x7f0a90103580 2024-03-18T23:04:28.488 INFO:teuthology.orchestra.run.smithi139.stderr:2024-03-18T23:04:28.486+0000 7f0a877fe700 1 -- 172.21.15.139:0/532707655 <== mon.2 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7f0a800051d0 con 0x7f0a90103580 2024-03-18T23:04:28.488 INFO:teuthology.orchestra.run.smithi139.stderr:2024-03-18T23:04:28.486+0000 7f0a877fe700 1 --2- 172.21.15.139:0/532707655 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f0a7c0a2890 0x7f0a7c0a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:04:28.488 INFO:teuthology.orchestra.run.smithi139.stderr:2024-03-18T23:04:28.487+0000 7f0a877fe700 1 -- 172.21.15.139:0/532707655 <== mon.2 v2:172.21.15.139:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 6261+0+0 (secure 0 0 0) 0x7f0a800ce5b0 con 0x7f0a90103580 2024-03-18T23:04:28.488 INFO:teuthology.orchestra.run.smithi139.stderr:2024-03-18T23:04:28.487+0000 7f0a9639f700 1 --2- 172.21.15.139:0/532707655 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f0a7c0a2890 0x7f0a7c0a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:04:28.488 INFO:teuthology.orchestra.run.smithi139.stderr:2024-03-18T23:04:28.487+0000 7f0a9639f700 1 --2- 172.21.15.139:0/532707655 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f0a7c0a2890 0x7f0a7c0a4d50 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f0a8c00b580 tx=0x7f0a8c005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:04:28.490 INFO:teuthology.orchestra.run.smithi139.stderr:2024-03-18T23:04:28.490+0000 7f0a877fe700 1 -- 172.21.15.139:0/532707655 <== mon.2 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f0a80020020 con 0x7f0a90103580 2024-03-18T23:04:28.695 INFO:teuthology.orchestra.run.smithi139.stderr:2024-03-18T23:04:28.695+0000 7f0a98603700 1 -- 172.21.15.139:0/532707655 --> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi139:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f0a90062750 con 0x7f0a7c0a2890 2024-03-18T23:04:28.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:04:28 smithi139 ceph-mon[67282]: Detected new or changed devices on smithi049 2024-03-18T23:04:28.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:04:28 smithi139 ceph-mon[67282]: Adjusting osd_memory_target on smithi049 to 4272M 2024-03-18T23:04:28.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:04:28 smithi049 ceph-mon[62991]: Detected new or changed devices on smithi049 2024-03-18T23:04:28.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:04:28 smithi049 ceph-mon[62991]: Adjusting osd_memory_target on smithi049 to 4272M 2024-03-18T23:04:28.961 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:04:28 smithi175 ceph-mon[67352]: Detected new or changed devices on smithi049 2024-03-18T23:04:28.962 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:04:28 smithi175 ceph-mon[67352]: Adjusting osd_memory_target on smithi049 to 4272M 2024-03-18T23:04:29.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:04:29 smithi139 ceph-mon[67282]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-03-18T23:04:29.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:04:29 smithi139 ceph-mon[67282]: from='client.24317 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi139:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T23:04:29.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:04:29 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-18T23:04:29.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:04:29 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-18T23:04:29.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:04:29 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:04:29.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:04:29 smithi049 ceph-mon[62991]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-03-18T23:04:29.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:04:29 smithi049 ceph-mon[62991]: from='client.24317 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi139:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T23:04:29.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:04:29 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-18T23:04:29.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:04:29 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-18T23:04:29.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:04:29 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:04:29.961 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:04:29 smithi175 ceph-mon[67352]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-03-18T23:04:29.962 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:04:29 smithi175 ceph-mon[67352]: from='client.24317 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi139:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T23:04:29.962 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:04:29 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-18T23:04:29.962 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:04:29 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-18T23:04:29.962 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:04:29 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:04:31.795 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:04:31 smithi139 ceph-mon[67282]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-03-18T23:04:31.795 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:04:31 smithi139 ceph-mon[67282]: from='client.? 172.21.15.139:0/1124939735' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3945e46d-766a-4c4a-8821-d43648dd1578"}]: dispatch 2024-03-18T23:04:31.795 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:04:31 smithi139 ceph-mon[67282]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3945e46d-766a-4c4a-8821-d43648dd1578"}]: dispatch 2024-03-18T23:04:31.795 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:04:31 smithi139 ceph-mon[67282]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3945e46d-766a-4c4a-8821-d43648dd1578"}]': finished 2024-03-18T23:04:31.795 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:04:31 smithi139 ceph-mon[67282]: osdmap e70: 11 total, 10 up, 11 in 2024-03-18T23:04:31.795 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:04:31 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-18T23:04:31.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:04:31 smithi049 ceph-mon[62991]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-03-18T23:04:31.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:04:31 smithi049 ceph-mon[62991]: from='client.? 172.21.15.139:0/1124939735' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3945e46d-766a-4c4a-8821-d43648dd1578"}]: dispatch 2024-03-18T23:04:31.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:04:31 smithi049 ceph-mon[62991]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3945e46d-766a-4c4a-8821-d43648dd1578"}]: dispatch 2024-03-18T23:04:31.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:04:31 smithi049 ceph-mon[62991]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3945e46d-766a-4c4a-8821-d43648dd1578"}]': finished 2024-03-18T23:04:31.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:04:31 smithi049 ceph-mon[62991]: osdmap e70: 11 total, 10 up, 11 in 2024-03-18T23:04:31.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:04:31 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-18T23:04:31.961 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:04:31 smithi175 ceph-mon[67352]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-03-18T23:04:31.962 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:04:31 smithi175 ceph-mon[67352]: from='client.? 172.21.15.139:0/1124939735' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3945e46d-766a-4c4a-8821-d43648dd1578"}]: dispatch 2024-03-18T23:04:31.962 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:04:31 smithi175 ceph-mon[67352]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3945e46d-766a-4c4a-8821-d43648dd1578"}]: dispatch 2024-03-18T23:04:31.962 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:04:31 smithi175 ceph-mon[67352]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3945e46d-766a-4c4a-8821-d43648dd1578"}]': finished 2024-03-18T23:04:31.962 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:04:31 smithi175 ceph-mon[67352]: osdmap e70: 11 total, 10 up, 11 in 2024-03-18T23:04:31.962 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:04:31 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-18T23:04:32.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:04:32 smithi139 ceph-mon[67282]: from='client.? 172.21.15.139:0/1002399115' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-18T23:04:32.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:04:32 smithi049 ceph-mon[62991]: from='client.? 172.21.15.139:0/1002399115' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-18T23:04:32.961 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:04:32 smithi175 ceph-mon[67352]: from='client.? 172.21.15.139:0/1002399115' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-18T23:04:33.816 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:04:33 smithi139 ceph-mon[67282]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-03-18T23:04:33.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:04:33 smithi049 ceph-mon[62991]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-03-18T23:04:33.961 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:04:33 smithi175 ceph-mon[67352]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-03-18T23:04:35.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:04:35 smithi139 ceph-mon[67282]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-03-18T23:04:35.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:04:35 smithi049 ceph-mon[62991]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-03-18T23:04:35.961 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:04:35 smithi175 ceph-mon[67352]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-03-18T23:04:37.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:04:37 smithi139 ceph-mon[67282]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-03-18T23:04:37.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:04:37 smithi049 ceph-mon[62991]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-03-18T23:04:37.961 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:04:37 smithi175 ceph-mon[67352]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-03-18T23:04:39.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:04:39 smithi139 ceph-mon[67282]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-03-18T23:04:39.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:04:39 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-03-18T23:04:39.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:04:39 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:04:39.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:04:39 smithi049 ceph-mon[62991]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-03-18T23:04:39.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:04:39 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-03-18T23:04:39.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:04:39 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:04:39.961 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:04:39 smithi175 ceph-mon[67352]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-03-18T23:04:39.962 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:04:39 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-03-18T23:04:39.962 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:04:39 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:04:40.813 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:04:40 smithi139 ceph-mon[67282]: Deploying daemon osd.10 on smithi139 2024-03-18T23:04:40.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:04:40 smithi049 ceph-mon[62991]: Deploying daemon osd.10 on smithi139 2024-03-18T23:04:40.961 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:04:40 smithi175 ceph-mon[67352]: Deploying daemon osd.10 on smithi139 2024-03-18T23:04:41.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:04:41 smithi139 ceph-mon[67282]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-03-18T23:04:41.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:04:41 smithi049 ceph-mon[62991]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-03-18T23:04:41.953 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:04:41 smithi175 ceph-mon[67352]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-03-18T23:04:43.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:04:43 smithi139 ceph-mon[67282]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-03-18T23:04:43.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:04:43 smithi049 ceph-mon[62991]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-03-18T23:04:43.961 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:04:43 smithi175 ceph-mon[67352]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-03-18T23:04:45.739 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:04:45 smithi139 ceph-mon[67282]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-03-18T23:04:45.739 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:04:45 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:04:45.739 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:04:45 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:04:45.739 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:04:45 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:04:45.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:04:45 smithi049 ceph-mon[62991]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-03-18T23:04:45.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:04:45 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:04:45.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:04:45 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:04:45.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:04:45 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:04:45.961 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:04:45 smithi175 ceph-mon[67352]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-03-18T23:04:45.962 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:04:45 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:04:45.962 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:04:45 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:04:45.962 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:04:45 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:04:47.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:04:47 smithi139 ceph-mon[67282]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-03-18T23:04:47.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:04:47 smithi049 ceph-mon[62991]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-03-18T23:04:47.961 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:04:47 smithi175 ceph-mon[67352]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-03-18T23:04:48.612 INFO:teuthology.orchestra.run.smithi139.stderr:2024-03-18T23:04:48.611+0000 7f0a877fe700 1 -- 172.21.15.139:0/532707655 <== mgr.14150 v2:172.21.15.49:6800/2083805211 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f0a90062750 con 0x7f0a7c0a2890 2024-03-18T23:04:48.613 INFO:teuthology.orchestra.run.smithi139.stdout:Created osd(s) 10 on host 'smithi139' 2024-03-18T23:04:48.615 INFO:teuthology.orchestra.run.smithi139.stderr:2024-03-18T23:04:48.614+0000 7f0a98603700 1 -- 172.21.15.139:0/532707655 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f0a7c0a2890 msgr2=0x7f0a7c0a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:04:48.615 INFO:teuthology.orchestra.run.smithi139.stderr:2024-03-18T23:04:48.614+0000 7f0a98603700 1 --2- 172.21.15.139:0/532707655 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f0a7c0a2890 0x7f0a7c0a4d50 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f0a8c00b580 tx=0x7f0a8c005fb0 comp rx=0 tx=0).stop 2024-03-18T23:04:48.615 INFO:teuthology.orchestra.run.smithi139.stderr:2024-03-18T23:04:48.614+0000 7f0a98603700 1 -- 172.21.15.139:0/532707655 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0a90103580 msgr2=0x7f0a90074700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:04:48.615 INFO:teuthology.orchestra.run.smithi139.stderr:2024-03-18T23:04:48.614+0000 7f0a98603700 1 --2- 172.21.15.139:0/532707655 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0a90103580 0x7f0a90074700 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f0a8000d980 tx=0x7f0a8000dc90 comp rx=0 tx=0).stop 2024-03-18T23:04:48.615 INFO:teuthology.orchestra.run.smithi139.stderr:2024-03-18T23:04:48.614+0000 7f0a98603700 1 -- 172.21.15.139:0/532707655 shutdown_connections 2024-03-18T23:04:48.615 INFO:teuthology.orchestra.run.smithi139.stderr:2024-03-18T23:04:48.614+0000 7f0a98603700 1 --2- 172.21.15.139:0/532707655 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0a90103f20 0x7f0a90074c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:04:48.615 INFO:teuthology.orchestra.run.smithi139.stderr:2024-03-18T23:04:48.614+0000 7f0a98603700 1 --2- 172.21.15.139:0/532707655 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f0a7c0a2890 0x7f0a7c0a4d50 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:04:48.615 INFO:teuthology.orchestra.run.smithi139.stderr:2024-03-18T23:04:48.614+0000 7f0a98603700 1 --2- 172.21.15.139:0/532707655 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0a90102380 0x7f0a900760b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:04:48.615 INFO:teuthology.orchestra.run.smithi139.stderr:2024-03-18T23:04:48.614+0000 7f0a98603700 1 --2- 172.21.15.139:0/532707655 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0a90103580 0x7f0a90074700 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:04:48.616 INFO:teuthology.orchestra.run.smithi139.stderr:2024-03-18T23:04:48.614+0000 7f0a98603700 1 -- 172.21.15.139:0/532707655 >> 172.21.15.139:0/532707655 conn(0x7f0a900fd960 msgr2=0x7f0a90106880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:04:48.616 INFO:teuthology.orchestra.run.smithi139.stderr:2024-03-18T23:04:48.614+0000 7f0a98603700 1 -- 172.21.15.139:0/532707655 shutdown_connections 2024-03-18T23:04:48.616 INFO:teuthology.orchestra.run.smithi139.stderr:2024-03-18T23:04:48.614+0000 7f0a98603700 1 -- 172.21.15.139:0/532707655 wait complete. 2024-03-18T23:04:48.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:04:48 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:04:48.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:04:48 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:04:48.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:04:48 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:04:48.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:04:48 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:04:48.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:04:48 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:04:48.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:04:48 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:04:48.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:04:48 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:04:48.816 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:04:48 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:04:48.816 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:04:48 smithi139 ceph-mon[67282]: from='osd.10 [v2:172.21.15.139:6824/2404740201,v1:172.21.15.139:6825/2404740201]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-18T23:04:48.816 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:04:48 smithi139 ceph-mon[67282]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-18T23:04:49.130 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:04:48 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:04:49.130 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:04:48 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:04:49.130 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:04:48 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:04:49.130 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:04:48 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:04:49.130 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:04:48 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:04:49.130 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:04:48 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:04:49.131 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:04:48 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:04:49.131 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:04:48 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:04:49.131 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:04:48 smithi049 ceph-mon[62991]: from='osd.10 [v2:172.21.15.139:6824/2404740201,v1:172.21.15.139:6825/2404740201]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-18T23:04:49.131 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:04:48 smithi049 ceph-mon[62991]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-18T23:04:49.213 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:04:48 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:04:49.213 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:04:48 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:04:49.215 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:04:48 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:04:49.215 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:04:48 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:04:49.215 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:04:48 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:04:49.215 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:04:48 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:04:49.215 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:04:48 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:04:49.216 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:04:48 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:04:49.216 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:04:48 smithi175 ceph-mon[67352]: from='osd.10 [v2:172.21.15.139:6824/2404740201,v1:172.21.15.139:6825/2404740201]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-18T23:04:49.216 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:04:48 smithi175 ceph-mon[67352]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-18T23:04:49.293 DEBUG:teuthology.orchestra.run.smithi139:osd.10> sudo journalctl -f -n 0 -u ceph-c63591ea-e57a-11ee-95c9-87774f69a715@osd.10.service 2024-03-18T23:04:49.296 INFO:tasks.cephadm:Deploying osd.11 on smithi175 with /dev/vg_nvme/lv_1... 2024-03-18T23:04:49.297 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c63591ea-e57a-11ee-95c9-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-03-18T23:04:49.547 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/mon.c/config 2024-03-18T23:04:49.942 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:04:49 smithi175 ceph-mon[67352]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-03-18T23:04:49.942 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:04:49 smithi175 ceph-mon[67352]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-03-18T23:04:49.942 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:04:49 smithi175 ceph-mon[67352]: from='osd.10 [v2:172.21.15.139:6824/2404740201,v1:172.21.15.139:6825/2404740201]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]: dispatch 2024-03-18T23:04:49.942 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:04:49 smithi175 ceph-mon[67352]: osdmap e71: 11 total, 10 up, 11 in 2024-03-18T23:04:49.942 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:04:49 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-18T23:04:49.942 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:04:49 smithi175 ceph-mon[67352]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]: dispatch 2024-03-18T23:04:50.065 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:04:49 smithi139 ceph-mon[67282]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-03-18T23:04:50.065 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:04:49 smithi139 ceph-mon[67282]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-03-18T23:04:50.065 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:04:49 smithi139 ceph-mon[67282]: from='osd.10 [v2:172.21.15.139:6824/2404740201,v1:172.21.15.139:6825/2404740201]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]: dispatch 2024-03-18T23:04:50.065 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:04:49 smithi139 ceph-mon[67282]: osdmap e71: 11 total, 10 up, 11 in 2024-03-18T23:04:50.065 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:04:49 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-18T23:04:50.066 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:04:49 smithi139 ceph-mon[67282]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]: dispatch 2024-03-18T23:04:50.130 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:04:49 smithi049 ceph-mon[62991]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-03-18T23:04:50.130 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:04:49 smithi049 ceph-mon[62991]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-03-18T23:04:50.130 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:04:49 smithi049 ceph-mon[62991]: from='osd.10 [v2:172.21.15.139:6824/2404740201,v1:172.21.15.139:6825/2404740201]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]: dispatch 2024-03-18T23:04:50.130 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:04:49 smithi049 ceph-mon[62991]: osdmap e71: 11 total, 10 up, 11 in 2024-03-18T23:04:50.130 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:04:49 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-18T23:04:50.130 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:04:49 smithi049 ceph-mon[62991]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]: dispatch 2024-03-18T23:04:50.766 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:04:50 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:04:50.584+0000 7f83cfc4a700 -1 osd.10 0 waiting for initial osdmap 2024-03-18T23:04:50.766 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:04:50 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:04:50.595+0000 7f83cba3d700 -1 osd.10 72 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-18T23:04:51.619 INFO:teuthology.orchestra.run.smithi175.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-03-18T23:04:51.620 INFO:teuthology.orchestra.run.smithi175.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-03-18T23:04:51.620 INFO:teuthology.orchestra.run.smithi175.stderr: stderr: 10+0 records in 2024-03-18T23:04:51.620 INFO:teuthology.orchestra.run.smithi175.stderr:10+0 records out 2024-03-18T23:04:51.620 INFO:teuthology.orchestra.run.smithi175.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0112203 s, 935 MB/s 2024-03-18T23:04:51.620 INFO:teuthology.orchestra.run.smithi175.stderr:--> Zapping successful for: 2024-03-18T23:04:51.711 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:04:51 smithi175 ceph-mon[67352]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-03-18T23:04:51.712 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:04:51 smithi175 ceph-mon[67352]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]': finished 2024-03-18T23:04:51.712 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:04:51 smithi175 ceph-mon[67352]: osdmap e72: 11 total, 10 up, 11 in 2024-03-18T23:04:51.712 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:04:51 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-18T23:04:51.712 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:04:51 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-18T23:04:51.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:04:51 smithi049 ceph-mon[62991]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-03-18T23:04:51.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:04:51 smithi049 ceph-mon[62991]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]': finished 2024-03-18T23:04:51.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:04:51 smithi049 ceph-mon[62991]: osdmap e72: 11 total, 10 up, 11 in 2024-03-18T23:04:51.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:04:51 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-18T23:04:51.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:04:51 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-18T23:04:51.922 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:04:51 smithi139 ceph-mon[67282]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-03-18T23:04:51.922 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:04:51 smithi139 ceph-mon[67282]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]': finished 2024-03-18T23:04:51.922 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:04:51 smithi139 ceph-mon[67282]: osdmap e72: 11 total, 10 up, 11 in 2024-03-18T23:04:51.922 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:04:51 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-18T23:04:51.922 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:04:51 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-18T23:04:52.280 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c63591ea-e57a-11ee-95c9-87774f69a715 -- ceph orch daemon add osd smithi175:vg_nvme/lv_1 2024-03-18T23:04:52.517 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/mon.c/config 2024-03-18T23:04:52.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:04:52 smithi049 ceph-mon[62991]: purged_snaps scrub starts 2024-03-18T23:04:52.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:04:52 smithi049 ceph-mon[62991]: purged_snaps scrub ok 2024-03-18T23:04:52.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:04:52 smithi049 ceph-mon[62991]: osd.10 [v2:172.21.15.139:6824/2404740201,v1:172.21.15.139:6825/2404740201] boot 2024-03-18T23:04:52.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:04:52 smithi049 ceph-mon[62991]: osdmap e73: 11 total, 11 up, 11 in 2024-03-18T23:04:52.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:04:52 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-18T23:04:52.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:04:52 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:04:52.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:04:52 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:04:52.881 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:04:52 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:04:52.881 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:04:52 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:04:52.881 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:04:52 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:04:52.881 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:04:52 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:04:52.881 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:04:52 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:04:52.881 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:04:52 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:04:52.881 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:04:52 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:04:52.881 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:04:52 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:04:52.915 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:04:52 smithi175 ceph-mon[67352]: purged_snaps scrub starts 2024-03-18T23:04:52.916 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:04:52 smithi175 ceph-mon[67352]: purged_snaps scrub ok 2024-03-18T23:04:52.916 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:04:52 smithi175 ceph-mon[67352]: osd.10 [v2:172.21.15.139:6824/2404740201,v1:172.21.15.139:6825/2404740201] boot 2024-03-18T23:04:52.916 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:04:52 smithi175 ceph-mon[67352]: osdmap e73: 11 total, 11 up, 11 in 2024-03-18T23:04:52.916 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:04:52 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-18T23:04:52.916 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:04:52 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:04:52.916 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:04:52 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:04:52.916 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:04:52 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:04:52.916 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:04:52 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:04:52.916 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:04:52 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:04:52.916 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:04:52 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:04:52.916 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:04:52 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:04:52.916 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:04:52 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:04:52.916 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:04:52 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:04:52.916 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:04:52 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:04:53.065 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:04:52 smithi139 ceph-mon[67282]: purged_snaps scrub starts 2024-03-18T23:04:53.065 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:04:52 smithi139 ceph-mon[67282]: purged_snaps scrub ok 2024-03-18T23:04:53.065 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:04:52 smithi139 ceph-mon[67282]: osd.10 [v2:172.21.15.139:6824/2404740201,v1:172.21.15.139:6825/2404740201] boot 2024-03-18T23:04:53.065 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:04:52 smithi139 ceph-mon[67282]: osdmap e73: 11 total, 11 up, 11 in 2024-03-18T23:04:53.065 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:04:52 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-18T23:04:53.066 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:04:52 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:04:53.066 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:04:52 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:04:53.066 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:04:52 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:04:53.066 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:04:52 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:04:53.066 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:04:52 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:04:53.066 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:04:52 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:04:53.066 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:04:52 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:04:53.066 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:04:52 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:04:53.066 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:04:52 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:04:53.066 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:04:52 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:04:53.712 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:04:53 smithi175 ceph-mon[67352]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-03-18T23:04:53.712 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:04:53 smithi175 ceph-mon[67352]: Detected new or changed devices on smithi139 2024-03-18T23:04:53.712 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:04:53 smithi175 ceph-mon[67352]: Adjusting osd_memory_target on smithi139 to 4270M 2024-03-18T23:04:53.712 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:04:53 smithi175 ceph-mon[67352]: osdmap e74: 11 total, 11 up, 11 in 2024-03-18T23:04:53.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:04:53 smithi049 ceph-mon[62991]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-03-18T23:04:53.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:04:53 smithi049 ceph-mon[62991]: Detected new or changed devices on smithi139 2024-03-18T23:04:53.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:04:53 smithi049 ceph-mon[62991]: Adjusting osd_memory_target on smithi139 to 4270M 2024-03-18T23:04:53.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:04:53 smithi049 ceph-mon[62991]: osdmap e74: 11 total, 11 up, 11 in 2024-03-18T23:04:53.967 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:04:53.966+0000 7fcae3a11700 1 -- 172.21.15.175:0/1886374748 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcadc101240 msgr2=0x7fcadc101660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:04:53.967 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:04:53.966+0000 7fcae3a11700 1 --2- 172.21.15.175:0/1886374748 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcadc101240 0x7fcadc101660 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fcad8009990 tx=0x7fcad8009ca0 comp rx=0 tx=0).stop 2024-03-18T23:04:53.968 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:04:53.966+0000 7fcae3a11700 1 -- 172.21.15.175:0/1886374748 shutdown_connections 2024-03-18T23:04:53.968 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:04:53.966+0000 7fcae3a11700 1 --2- 172.21.15.175:0/1886374748 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcadc102de0 0x7fcadc109690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:04:53.968 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:04:53.966+0000 7fcae3a11700 1 --2- 172.21.15.175:0/1886374748 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcadc101240 0x7fcadc101660 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:04:53.968 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:04:53.966+0000 7fcae3a11700 1 --2- 172.21.15.175:0/1886374748 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcadc102440 0x7fcadc1028a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:04:53.968 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:04:53.966+0000 7fcae3a11700 1 -- 172.21.15.175:0/1886374748 >> 172.21.15.175:0/1886374748 conn(0x7fcadc0fc7e0 msgr2=0x7fcadc0fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:04:53.968 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:04:53.966+0000 7fcae3a11700 1 -- 172.21.15.175:0/1886374748 shutdown_connections 2024-03-18T23:04:53.968 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:04:53.966+0000 7fcae3a11700 1 -- 172.21.15.175:0/1886374748 wait complete. 2024-03-18T23:04:53.970 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:04:53.967+0000 7fcae3a11700 1 Processor -- start 2024-03-18T23:04:53.970 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:04:53.967+0000 7fcae3a11700 1 -- start start 2024-03-18T23:04:53.970 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:04:53.967+0000 7fcae3a11700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcadc101240 0x7fcadc197450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:04:53.970 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:04:53.967+0000 7fcae3a11700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcadc102440 0x7fcadc197990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:04:53.970 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:04:53.967+0000 7fcae3a11700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcadc102de0 0x7fcadc19ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:04:53.970 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:04:53.967+0000 7fcae3a11700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fcadc10ce00 con 0x7fcadc102de0 2024-03-18T23:04:53.970 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:04:53.967+0000 7fcae3a11700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fcadc10cc80 con 0x7fcadc101240 2024-03-18T23:04:53.970 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:04:53.967+0000 7fcae3a11700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fcadc10cf80 con 0x7fcadc102440 2024-03-18T23:04:53.971 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:04:53.967+0000 7fcae0fac700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcadc102440 0x7fcadc197990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:04:53.971 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:04:53.967+0000 7fcae0fac700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcadc102440 0x7fcadc197990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:51114/0 (socket says 172.21.15.175:51114) 2024-03-18T23:04:53.971 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:04:53.967+0000 7fcae0fac700 1 -- 172.21.15.175:0/2000286674 learned_addr learned my addr 172.21.15.175:0/2000286674 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-03-18T23:04:53.971 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:04:53.967+0000 7fcae1fae700 1 --2- 172.21.15.175:0/2000286674 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcadc102de0 0x7fcadc19ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:04:53.971 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:04:53.967+0000 7fcae17ad700 1 --2- 172.21.15.175:0/2000286674 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcadc101240 0x7fcadc197450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:04:53.971 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:04:53.968+0000 7fcae0fac700 1 -- 172.21.15.175:0/2000286674 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcadc101240 msgr2=0x7fcadc197450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:04:53.971 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:04:53.968+0000 7fcae0fac700 1 --2- 172.21.15.175:0/2000286674 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcadc101240 0x7fcadc197450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:04:53.971 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:04:53.968+0000 7fcae0fac700 1 -- 172.21.15.175:0/2000286674 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcadc102de0 msgr2=0x7fcadc19ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:04:53.971 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:04:53.968+0000 7fcae0fac700 1 --2- 172.21.15.175:0/2000286674 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcadc102de0 0x7fcadc19ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:04:53.971 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:04:53.968+0000 7fcae0fac700 1 -- 172.21.15.175:0/2000286674 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcadc19f0b0 con 0x7fcadc102440 2024-03-18T23:04:53.971 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:04:53.968+0000 7fcae17ad700 1 --2- 172.21.15.175:0/2000286674 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcadc101240 0x7fcadc197450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T23:04:53.971 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:04:53.968+0000 7fcae0fac700 1 --2- 172.21.15.175:0/2000286674 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcadc102440 0x7fcadc197990 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fcacc00c9e0 tx=0x7fcacc00ccf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:04:53.971 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:04:53.968+0000 7fcad27fc700 1 -- 172.21.15.175:0/2000286674 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcacc007ac0 con 0x7fcadc102440 2024-03-18T23:04:53.972 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:04:53.968+0000 7fcad27fc700 1 -- 172.21.15.175:0/2000286674 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcacc007c60 con 0x7fcadc102440 2024-03-18T23:04:53.972 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:04:53.968+0000 7fcad27fc700 1 -- 172.21.15.175:0/2000286674 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcacc018730 con 0x7fcadc102440 2024-03-18T23:04:53.972 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:04:53.968+0000 7fcae3a11700 1 -- 172.21.15.175:0/2000286674 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcadc19f3a0 con 0x7fcadc102440 2024-03-18T23:04:53.972 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:04:53.968+0000 7fcae3a11700 1 -- 172.21.15.175:0/2000286674 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcadc19f8e0 con 0x7fcadc102440 2024-03-18T23:04:53.972 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:04:53.969+0000 7fcae3a11700 1 -- 172.21.15.175:0/2000286674 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcadc068fd0 con 0x7fcadc102440 2024-03-18T23:04:53.973 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:04:53.971+0000 7fcad27fc700 1 -- 172.21.15.175:0/2000286674 <== mon.1 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7fcacc0188d0 con 0x7fcadc102440 2024-03-18T23:04:53.973 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:04:53.971+0000 7fcad27fc700 1 --2- 172.21.15.175:0/2000286674 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fcac80a2890 0x7fcac80a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:04:53.973 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:04:53.971+0000 7fcad27fc700 1 -- 172.21.15.175:0/2000286674 <== mon.1 v2:172.21.15.175:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 6694+0+0 (secure 0 0 0) 0x7fcacc0ce680 con 0x7fcadc102440 2024-03-18T23:04:53.973 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:04:53.971+0000 7fcae17ad700 1 --2- 172.21.15.175:0/2000286674 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fcac80a2890 0x7fcac80a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:04:53.973 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:04:53.972+0000 7fcae17ad700 1 --2- 172.21.15.175:0/2000286674 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fcac80a2890 0x7fcac80a4d50 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fcad800b5c0 tx=0x7fcad8005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:04:53.976 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:04:53.975+0000 7fcad27fc700 1 -- 172.21.15.175:0/2000286674 <== mon.1 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fcacc014030 con 0x7fcadc102440 2024-03-18T23:04:54.065 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:04:53 smithi139 ceph-mon[67282]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-03-18T23:04:54.065 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:04:53 smithi139 ceph-mon[67282]: Detected new or changed devices on smithi139 2024-03-18T23:04:54.065 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:04:53 smithi139 ceph-mon[67282]: Adjusting osd_memory_target on smithi139 to 4270M 2024-03-18T23:04:54.065 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:04:53 smithi139 ceph-mon[67282]: osdmap e74: 11 total, 11 up, 11 in 2024-03-18T23:04:54.178 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:04:54.177+0000 7fcae3a11700 1 -- 172.21.15.175:0/2000286674 --> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi175:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fcadc062750 con 0x7fcac80a2890 2024-03-18T23:04:54.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:04:54 smithi049 ceph-mon[62991]: osdmap e75: 11 total, 11 up, 11 in 2024-03-18T23:04:54.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:04:54 smithi049 ceph-mon[62991]: osdmap e76: 11 total, 11 up, 11 in 2024-03-18T23:04:54.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:04:54 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-18T23:04:54.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:04:54 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-18T23:04:54.881 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:04:54 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:04:54.906 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:04:54 smithi175 ceph-mon[67352]: osdmap e75: 11 total, 11 up, 11 in 2024-03-18T23:04:54.907 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:04:54 smithi175 ceph-mon[67352]: osdmap e76: 11 total, 11 up, 11 in 2024-03-18T23:04:54.907 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:04:54 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-18T23:04:54.907 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:04:54 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-18T23:04:54.907 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:04:54 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:04:55.065 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:04:54 smithi139 ceph-mon[67282]: osdmap e75: 11 total, 11 up, 11 in 2024-03-18T23:04:55.065 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:04:54 smithi139 ceph-mon[67282]: osdmap e76: 11 total, 11 up, 11 in 2024-03-18T23:04:55.065 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:04:54 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-18T23:04:55.065 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:04:54 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-18T23:04:55.065 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:04:54 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:04:56.045 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:04:55 smithi175 ceph-mon[67352]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-03-18T23:04:56.045 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:04:55 smithi175 ceph-mon[67352]: from='client.24334 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi175:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T23:04:56.045 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:04:55 smithi175 ceph-mon[67352]: osdmap e77: 11 total, 11 up, 11 in 2024-03-18T23:04:56.065 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:04:55 smithi139 ceph-mon[67282]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-03-18T23:04:56.065 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:04:55 smithi139 ceph-mon[67282]: from='client.24334 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi175:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T23:04:56.065 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:04:55 smithi139 ceph-mon[67282]: osdmap e77: 11 total, 11 up, 11 in 2024-03-18T23:04:56.130 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:04:55 smithi049 ceph-mon[62991]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-03-18T23:04:56.130 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:04:55 smithi049 ceph-mon[62991]: from='client.24334 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi175:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T23:04:56.130 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:04:55 smithi049 ceph-mon[62991]: osdmap e77: 11 total, 11 up, 11 in 2024-03-18T23:04:58.065 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:04:57 smithi139 ceph-mon[67282]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-03-18T23:04:58.065 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:04:57 smithi139 ceph-mon[67282]: from='client.? 172.21.15.175:0/4200449891' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9954ba15-d877-46cc-b24a-893f9168deb8"}]: dispatch 2024-03-18T23:04:58.065 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:04:57 smithi139 ceph-mon[67282]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9954ba15-d877-46cc-b24a-893f9168deb8"}]: dispatch 2024-03-18T23:04:58.065 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:04:57 smithi139 ceph-mon[67282]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9954ba15-d877-46cc-b24a-893f9168deb8"}]': finished 2024-03-18T23:04:58.065 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:04:57 smithi139 ceph-mon[67282]: osdmap e78: 12 total, 11 up, 12 in 2024-03-18T23:04:58.065 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:04:57 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-18T23:04:58.065 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:04:57 smithi139 ceph-mon[67282]: from='client.? 172.21.15.175:0/424329662' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-18T23:04:58.130 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:04:57 smithi049 ceph-mon[62991]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-03-18T23:04:58.130 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:04:57 smithi049 ceph-mon[62991]: from='client.? 172.21.15.175:0/4200449891' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9954ba15-d877-46cc-b24a-893f9168deb8"}]: dispatch 2024-03-18T23:04:58.130 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:04:57 smithi049 ceph-mon[62991]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9954ba15-d877-46cc-b24a-893f9168deb8"}]: dispatch 2024-03-18T23:04:58.130 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:04:57 smithi049 ceph-mon[62991]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9954ba15-d877-46cc-b24a-893f9168deb8"}]': finished 2024-03-18T23:04:58.130 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:04:57 smithi049 ceph-mon[62991]: osdmap e78: 12 total, 11 up, 12 in 2024-03-18T23:04:58.130 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:04:57 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-18T23:04:58.130 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:04:57 smithi049 ceph-mon[62991]: from='client.? 172.21.15.175:0/424329662' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-18T23:04:58.212 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:04:57 smithi175 ceph-mon[67352]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-03-18T23:04:58.212 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:04:57 smithi175 ceph-mon[67352]: from='client.? 172.21.15.175:0/4200449891' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9954ba15-d877-46cc-b24a-893f9168deb8"}]: dispatch 2024-03-18T23:04:58.212 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:04:57 smithi175 ceph-mon[67352]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9954ba15-d877-46cc-b24a-893f9168deb8"}]: dispatch 2024-03-18T23:04:58.212 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:04:57 smithi175 ceph-mon[67352]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9954ba15-d877-46cc-b24a-893f9168deb8"}]': finished 2024-03-18T23:04:58.212 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:04:57 smithi175 ceph-mon[67352]: osdmap e78: 12 total, 11 up, 12 in 2024-03-18T23:04:58.212 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:04:57 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-18T23:04:58.212 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:04:57 smithi175 ceph-mon[67352]: from='client.? 172.21.15.175:0/424329662' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-18T23:05:00.065 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:04:59 smithi139 ceph-mon[67282]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail; 134 KiB/s, 0 objects/s recovering 2024-03-18T23:05:00.130 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:04:59 smithi049 ceph-mon[62991]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail; 134 KiB/s, 0 objects/s recovering 2024-03-18T23:05:00.211 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:04:59 smithi175 ceph-mon[67352]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail; 134 KiB/s, 0 objects/s recovering 2024-03-18T23:05:01.996 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:05:01 smithi175 ceph-mon[67352]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail; 94 KiB/s, 0 objects/s recovering 2024-03-18T23:05:02.065 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:05:01 smithi139 ceph-mon[67282]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail; 94 KiB/s, 0 objects/s recovering 2024-03-18T23:05:02.130 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:05:01 smithi049 ceph-mon[62991]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail; 94 KiB/s, 0 objects/s recovering 2024-03-18T23:05:04.065 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:05:03 smithi139 ceph-mon[67282]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-18T23:05:04.130 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:05:03 smithi049 ceph-mon[62991]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-18T23:05:04.186 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:05:03 smithi175 ceph-mon[67352]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-18T23:05:04.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:05:04 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-03-18T23:05:04.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:05:04 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:05:05.065 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:05:04 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-03-18T23:05:05.065 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:05:04 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:05:05.107 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:05:04 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-03-18T23:05:05.107 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:05:04 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:05:06.065 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:05:05 smithi139 ceph-mon[67282]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-03-18T23:05:06.065 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:05:05 smithi139 ceph-mon[67282]: Deploying daemon osd.11 on smithi175 2024-03-18T23:05:06.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:05:05 smithi175 ceph-mon[67352]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-03-18T23:05:06.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:05:05 smithi175 ceph-mon[67352]: Deploying daemon osd.11 on smithi175 2024-03-18T23:05:06.130 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:05:05 smithi049 ceph-mon[62991]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-03-18T23:05:06.130 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:05:05 smithi049 ceph-mon[62991]: Deploying daemon osd.11 on smithi175 2024-03-18T23:05:07.950 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:05:07 smithi175 ceph-mon[67352]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-03-18T23:05:08.065 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:05:07 smithi139 ceph-mon[67282]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-03-18T23:05:08.130 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:05:07 smithi049 ceph-mon[62991]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-03-18T23:05:10.004 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:05:09 smithi175 ceph-mon[67352]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-03-18T23:05:10.065 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:05:09 smithi139 ceph-mon[67282]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-03-18T23:05:10.130 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:05:09 smithi049 ceph-mon[62991]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-03-18T23:05:11.065 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:05:10 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:05:11.065 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:05:10 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:05:11.065 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:05:10 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:05:11.087 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:05:10 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:05:11.087 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:05:10 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:05:11.088 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:05:10 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:05:11.130 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:05:10 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:05:11.130 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:05:10 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:05:11.130 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:05:10 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:05:11.961 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:05:11 smithi175 ceph-mon[67352]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-03-18T23:05:12.065 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:05:11 smithi139 ceph-mon[67282]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-03-18T23:05:12.130 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:05:11 smithi049 ceph-mon[62991]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-03-18T23:05:13.065 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:05:12 smithi139 ceph-mon[67282]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-03-18T23:05:13.084 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:05:12 smithi175 ceph-mon[67352]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-03-18T23:05:13.130 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:05:12 smithi049 ceph-mon[62991]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-03-18T23:05:14.450 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:05:14.449+0000 7fcad27fc700 1 -- 172.21.15.175:0/2000286674 <== mgr.14150 v2:172.21.15.49:6800/2083805211 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7fcadc062750 con 0x7fcac80a2890 2024-03-18T23:05:14.450 INFO:teuthology.orchestra.run.smithi175.stdout:Created osd(s) 11 on host 'smithi175' 2024-03-18T23:05:14.452 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:05:14.451+0000 7fcae3a11700 1 -- 172.21.15.175:0/2000286674 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fcac80a2890 msgr2=0x7fcac80a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:05:14.452 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:05:14.451+0000 7fcae3a11700 1 --2- 172.21.15.175:0/2000286674 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fcac80a2890 0x7fcac80a4d50 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fcad800b5c0 tx=0x7fcad8005fb0 comp rx=0 tx=0).stop 2024-03-18T23:05:14.453 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:05:14.451+0000 7fcae3a11700 1 -- 172.21.15.175:0/2000286674 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcadc102440 msgr2=0x7fcadc197990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:05:14.453 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:05:14.451+0000 7fcae3a11700 1 --2- 172.21.15.175:0/2000286674 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcadc102440 0x7fcadc197990 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fcacc00c9e0 tx=0x7fcacc00ccf0 comp rx=0 tx=0).stop 2024-03-18T23:05:14.453 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:05:14.451+0000 7fcae3a11700 1 -- 172.21.15.175:0/2000286674 shutdown_connections 2024-03-18T23:05:14.453 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:05:14.451+0000 7fcae3a11700 1 --2- 172.21.15.175:0/2000286674 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcadc102de0 0x7fcadc19ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:05:14.453 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:05:14.451+0000 7fcae3a11700 1 --2- 172.21.15.175:0/2000286674 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcadc101240 0x7fcadc197450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:05:14.453 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:05:14.451+0000 7fcae3a11700 1 --2- 172.21.15.175:0/2000286674 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fcac80a2890 0x7fcac80a4d50 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:05:14.453 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:05:14.451+0000 7fcae3a11700 1 --2- 172.21.15.175:0/2000286674 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcadc102440 0x7fcadc197990 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:05:14.453 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:05:14.451+0000 7fcae3a11700 1 -- 172.21.15.175:0/2000286674 >> 172.21.15.175:0/2000286674 conn(0x7fcadc0fc7e0 msgr2=0x7fcadc1057c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:05:14.453 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:05:14.452+0000 7fcae3a11700 1 -- 172.21.15.175:0/2000286674 shutdown_connections 2024-03-18T23:05:14.453 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:05:14.452+0000 7fcae3a11700 1 -- 172.21.15.175:0/2000286674 wait complete. 2024-03-18T23:05:14.712 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:05:14 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:05:14.712 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:05:14 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:05:14.712 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:05:14 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:05:14.712 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:05:14 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:05:14.712 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:05:14 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:05:14.712 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:05:14 smithi175 ceph-mon[67352]: from='osd.11 [v2:172.21.15.175:6824/3519503868,v1:172.21.15.175:6825/3519503868]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-18T23:05:14.712 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:05:14 smithi175 ceph-mon[67352]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-18T23:05:14.712 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:05:14 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:05:14.713 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:05:14 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:05:14.713 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:05:14 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:05:14.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:05:14 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:05:14.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:05:14 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:05:14.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:05:14 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:05:14.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:05:14 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:05:14.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:05:14 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:05:14.816 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:05:14 smithi139 ceph-mon[67282]: from='osd.11 [v2:172.21.15.175:6824/3519503868,v1:172.21.15.175:6825/3519503868]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-18T23:05:14.816 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:05:14 smithi139 ceph-mon[67282]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-18T23:05:14.816 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:05:14 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:05:14.816 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:05:14 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:05:14.816 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:05:14 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:05:14.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:05:14 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:05:14.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:05:14 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:05:14.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:05:14 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:05:14.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:05:14 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:05:14.881 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:05:14 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:05:14.881 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:05:14 smithi049 ceph-mon[62991]: from='osd.11 [v2:172.21.15.175:6824/3519503868,v1:172.21.15.175:6825/3519503868]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-18T23:05:14.881 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:05:14 smithi049 ceph-mon[62991]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-18T23:05:14.881 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:05:14 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:05:14.881 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:05:14 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:05:14.881 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:05:14 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:05:15.101 DEBUG:teuthology.orchestra.run.smithi175:osd.11> sudo journalctl -f -n 0 -u ceph-c63591ea-e57a-11ee-95c9-87774f69a715@osd.11.service 2024-03-18T23:05:15.104 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-03-18T23:05:15.104 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c63591ea-e57a-11ee-95c9-87774f69a715 -- ceph osd stat -f json 2024-03-18T23:05:15.367 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T23:05:15.711 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:05:15 smithi175 ceph-mon[67352]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-03-18T23:05:15.712 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:05:15 smithi175 ceph-mon[67352]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-03-18T23:05:15.712 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:05:15 smithi175 ceph-mon[67352]: from='osd.11 [v2:172.21.15.175:6824/3519503868,v1:172.21.15.175:6825/3519503868]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]: dispatch 2024-03-18T23:05:15.712 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:05:15 smithi175 ceph-mon[67352]: osdmap e79: 12 total, 11 up, 12 in 2024-03-18T23:05:15.712 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:05:15 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-18T23:05:15.712 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:05:15 smithi175 ceph-mon[67352]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]: dispatch 2024-03-18T23:05:15.712 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:05:15 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:05:15.578+0000 7fd318dfd700 -1 osd.11 0 waiting for initial osdmap 2024-03-18T23:05:15.712 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:05:15 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:05:15.595+0000 7fd314bf0700 -1 osd.11 80 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-18T23:05:15.804 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:05:15 smithi049 ceph-mon[62991]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-03-18T23:05:15.804 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:05:15 smithi049 ceph-mon[62991]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-03-18T23:05:15.804 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:05:15 smithi049 ceph-mon[62991]: from='osd.11 [v2:172.21.15.175:6824/3519503868,v1:172.21.15.175:6825/3519503868]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]: dispatch 2024-03-18T23:05:15.804 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:05:15 smithi049 ceph-mon[62991]: osdmap e79: 12 total, 11 up, 12 in 2024-03-18T23:05:15.805 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:05:15 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-18T23:05:15.805 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:05:15 smithi049 ceph-mon[62991]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]: dispatch 2024-03-18T23:05:16.013 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:05:15 smithi139 ceph-mon[67282]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-03-18T23:05:16.013 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:05:15 smithi139 ceph-mon[67282]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-03-18T23:05:16.013 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:05:15 smithi139 ceph-mon[67282]: from='osd.11 [v2:172.21.15.175:6824/3519503868,v1:172.21.15.175:6825/3519503868]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]: dispatch 2024-03-18T23:05:16.013 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:05:15 smithi139 ceph-mon[67282]: osdmap e79: 12 total, 11 up, 12 in 2024-03-18T23:05:16.013 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:05:15 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-18T23:05:16.013 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:05:15 smithi139 ceph-mon[67282]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]: dispatch 2024-03-18T23:05:16.747 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:05:16 smithi049 ceph-mon[62991]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]': finished 2024-03-18T23:05:16.747 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:05:16 smithi049 ceph-mon[62991]: osdmap e80: 12 total, 11 up, 12 in 2024-03-18T23:05:16.747 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:05:16 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-18T23:05:16.747 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:05:16 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-18T23:05:16.962 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:05:16 smithi175 ceph-mon[67352]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]': finished 2024-03-18T23:05:16.962 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:05:16 smithi175 ceph-mon[67352]: osdmap e80: 12 total, 11 up, 12 in 2024-03-18T23:05:16.962 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:05:16 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-18T23:05:16.962 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:05:16 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-18T23:05:17.065 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:05:16 smithi139 ceph-mon[67282]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]': finished 2024-03-18T23:05:17.065 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:05:16 smithi139 ceph-mon[67282]: osdmap e80: 12 total, 11 up, 12 in 2024-03-18T23:05:17.065 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:05:16 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-18T23:05:17.065 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:05:16 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-18T23:05:17.111 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:17.108+0000 7f3359b09700 1 -- 172.21.15.49:0/146239270 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3354103540 msgr2=0x7f33541039a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:05:17.111 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:17.108+0000 7f3359b09700 1 --2- 172.21.15.49:0/146239270 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3354103540 0x7f33541039a0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f33480099c0 tx=0x7f3348009cd0 comp rx=0 tx=0).stop 2024-03-18T23:05:17.111 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:17.108+0000 7f3359b09700 1 -- 172.21.15.49:0/146239270 shutdown_connections 2024-03-18T23:05:17.111 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:17.108+0000 7f3359b09700 1 --2- 172.21.15.49:0/146239270 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3354103ee0 0x7f335410a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:05:17.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:17.108+0000 7f3359b09700 1 --2- 172.21.15.49:0/146239270 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3354102340 0x7f3354102760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:05:17.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:17.108+0000 7f3359b09700 1 --2- 172.21.15.49:0/146239270 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3354103540 0x7f33541039a0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:05:17.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:17.108+0000 7f3359b09700 1 -- 172.21.15.49:0/146239270 >> 172.21.15.49:0/146239270 conn(0x7f33540fd920 msgr2=0x7f33540ffd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:05:17.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:17.108+0000 7f3359b09700 1 -- 172.21.15.49:0/146239270 shutdown_connections 2024-03-18T23:05:17.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:17.108+0000 7f3359b09700 1 -- 172.21.15.49:0/146239270 wait complete. 2024-03-18T23:05:17.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:17.108+0000 7f3359b09700 1 Processor -- start 2024-03-18T23:05:17.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:17.109+0000 7f3359b09700 1 -- start start 2024-03-18T23:05:17.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:17.109+0000 7f3359b09700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3354102340 0x7f33540760b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:05:17.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:17.109+0000 7f3359b09700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3354103540 0x7f3354074700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:05:17.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:17.109+0000 7f3359b09700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3354103ee0 0x7f3354074c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:05:17.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:17.109+0000 7f3359b09700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f335410dfa0 con 0x7f3354103ee0 2024-03-18T23:05:17.113 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:17.109+0000 7f3359b09700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f335410de20 con 0x7f3354102340 2024-03-18T23:05:17.113 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:17.109+0000 7f3359b09700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f335410e120 con 0x7f3354103540 2024-03-18T23:05:17.113 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:17.109+0000 7f3353fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3354103ee0 0x7f3354074c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:05:17.113 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:17.109+0000 7f3353fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3354103ee0 0x7f3354074c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:45378/0 (socket says 172.21.15.49:45378) 2024-03-18T23:05:17.113 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:17.109+0000 7f3353fff700 1 -- 172.21.15.49:0/4231210944 learned_addr learned my addr 172.21.15.49:0/4231210944 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:05:17.113 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:17.109+0000 7f33537fe700 1 --2- 172.21.15.49:0/4231210944 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3354102340 0x7f33540760b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:05:17.113 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:17.109+0000 7f3352ffd700 1 --2- 172.21.15.49:0/4231210944 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3354103540 0x7f3354074700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:05:17.113 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:17.109+0000 7f3353fff700 1 -- 172.21.15.49:0/4231210944 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3354103540 msgr2=0x7f3354074700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:05:17.113 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:17.109+0000 7f3353fff700 1 --2- 172.21.15.49:0/4231210944 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3354103540 0x7f3354074700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:05:17.113 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:17.109+0000 7f3353fff700 1 -- 172.21.15.49:0/4231210944 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3354102340 msgr2=0x7f33540760b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:05:17.113 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:17.109+0000 7f3353fff700 1 --2- 172.21.15.49:0/4231210944 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3354102340 0x7f33540760b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:05:17.113 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:17.109+0000 7f3353fff700 1 -- 172.21.15.49:0/4231210944 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33540754f0 con 0x7f3354103ee0 2024-03-18T23:05:17.113 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:17.109+0000 7f3353fff700 1 --2- 172.21.15.49:0/4231210944 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3354103ee0 0x7f3354074c40 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f334c007ee0 tx=0x7f334c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:05:17.114 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:17.110+0000 7f3350ff9700 1 -- 172.21.15.49:0/4231210944 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f334c00dbd0 con 0x7f3354103ee0 2024-03-18T23:05:17.114 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:17.110+0000 7f3350ff9700 1 -- 172.21.15.49:0/4231210944 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f334c00dd70 con 0x7f3354103ee0 2024-03-18T23:05:17.114 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:17.110+0000 7f3350ff9700 1 -- 172.21.15.49:0/4231210944 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f334c005a60 con 0x7f3354103ee0 2024-03-18T23:05:17.114 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:17.110+0000 7f3359b09700 1 -- 172.21.15.49:0/4231210944 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33541a2740 con 0x7f3354103ee0 2024-03-18T23:05:17.114 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:17.110+0000 7f3359b09700 1 -- 172.21.15.49:0/4231210944 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33541a2cd0 con 0x7f3354103ee0 2024-03-18T23:05:17.115 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:17.111+0000 7f3359b09700 1 -- 172.21.15.49:0/4231210944 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3354068fd0 con 0x7f3354103ee0 2024-03-18T23:05:17.115 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:17.112+0000 7f3350ff9700 1 -- 172.21.15.49:0/4231210944 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7f334c01a020 con 0x7f3354103ee0 2024-03-18T23:05:17.115 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:17.112+0000 7f3350ff9700 1 --2- 172.21.15.49:0/4231210944 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f333c0a2910 0x7f333c0a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:05:17.115 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:17.113+0000 7f33537fe700 1 --2- 172.21.15.49:0/4231210944 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f333c0a2910 0x7f333c0a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:05:17.115 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:17.113+0000 7f3350ff9700 1 -- 172.21.15.49:0/4231210944 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f334c0cdc90 con 0x7f3354103ee0 2024-03-18T23:05:17.115 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:17.113+0000 7f33537fe700 1 --2- 172.21.15.49:0/4231210944 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f333c0a2910 0x7f333c0a4dd0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f33541033a0 tx=0x7f3344009190 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:05:17.118 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:17.117+0000 7f3350ff9700 1 -- 172.21.15.49:0/4231210944 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f334c010070 con 0x7f3354103ee0 2024-03-18T23:05:17.322 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:17.321+0000 7f3359b09700 1 -- 172.21.15.49:0/4231210944 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f3354062f20 con 0x7f3354103ee0 2024-03-18T23:05:17.323 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:17.322+0000 7f3350ff9700 1 -- 172.21.15.49:0/4231210944 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v81) ==== 74+0+133 (secure 0 0 0) 0x7f334c09b1d0 con 0x7f3354103ee0 2024-03-18T23:05:17.323 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-18T23:05:17.325 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:17.324+0000 7f3359b09700 1 -- 172.21.15.49:0/4231210944 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f333c0a2910 msgr2=0x7f333c0a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:05:17.326 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:17.324+0000 7f3359b09700 1 --2- 172.21.15.49:0/4231210944 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f333c0a2910 0x7f333c0a4dd0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f33541033a0 tx=0x7f3344009190 comp rx=0 tx=0).stop 2024-03-18T23:05:17.326 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:17.324+0000 7f3359b09700 1 -- 172.21.15.49:0/4231210944 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3354103ee0 msgr2=0x7f3354074c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:05:17.326 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:17.324+0000 7f3359b09700 1 --2- 172.21.15.49:0/4231210944 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3354103ee0 0x7f3354074c40 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f334c007ee0 tx=0x7f334c00d3b0 comp rx=0 tx=0).stop 2024-03-18T23:05:17.326 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:17.324+0000 7f3359b09700 1 -- 172.21.15.49:0/4231210944 shutdown_connections 2024-03-18T23:05:17.326 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:17.324+0000 7f3359b09700 1 --2- 172.21.15.49:0/4231210944 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3354103ee0 0x7f3354074c40 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:05:17.326 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:17.324+0000 7f3359b09700 1 --2- 172.21.15.49:0/4231210944 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3354102340 0x7f33540760b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:05:17.326 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:17.324+0000 7f3359b09700 1 --2- 172.21.15.49:0/4231210944 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f333c0a2910 0x7f333c0a4dd0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:05:17.326 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:17.324+0000 7f3359b09700 1 --2- 172.21.15.49:0/4231210944 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3354103540 0x7f3354074700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:05:17.326 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:17.324+0000 7f3359b09700 1 -- 172.21.15.49:0/4231210944 >> 172.21.15.49:0/4231210944 conn(0x7f33540fd920 msgr2=0x7f3354106890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:05:17.326 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:17.325+0000 7f3359b09700 1 -- 172.21.15.49:0/4231210944 shutdown_connections 2024-03-18T23:05:17.326 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:17.325+0000 7f3359b09700 1 -- 172.21.15.49:0/4231210944 wait complete. 2024-03-18T23:05:17.711 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:05:17 smithi175 ceph-mon[67352]: purged_snaps scrub starts 2024-03-18T23:05:17.712 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:05:17 smithi175 ceph-mon[67352]: purged_snaps scrub ok 2024-03-18T23:05:17.712 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:05:17 smithi175 ceph-mon[67352]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-03-18T23:05:17.712 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:05:17 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-18T23:05:17.712 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:05:17 smithi175 ceph-mon[67352]: osd.11 [v2:172.21.15.175:6824/3519503868,v1:172.21.15.175:6825/3519503868] boot 2024-03-18T23:05:17.712 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:05:17 smithi175 ceph-mon[67352]: osdmap e81: 12 total, 12 up, 12 in 2024-03-18T23:05:17.712 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:05:17 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-18T23:05:17.712 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:05:17 smithi175 ceph-mon[67352]: from='client.? 172.21.15.49:0/4231210944' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-18T23:05:17.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:05:17 smithi049 ceph-mon[62991]: purged_snaps scrub starts 2024-03-18T23:05:17.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:05:17 smithi049 ceph-mon[62991]: purged_snaps scrub ok 2024-03-18T23:05:17.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:05:17 smithi049 ceph-mon[62991]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-03-18T23:05:17.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:05:17 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-18T23:05:17.881 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:05:17 smithi049 ceph-mon[62991]: osd.11 [v2:172.21.15.175:6824/3519503868,v1:172.21.15.175:6825/3519503868] boot 2024-03-18T23:05:17.881 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:05:17 smithi049 ceph-mon[62991]: osdmap e81: 12 total, 12 up, 12 in 2024-03-18T23:05:17.881 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:05:17 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-18T23:05:17.881 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:05:17 smithi049 ceph-mon[62991]: from='client.? 172.21.15.49:0/4231210944' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-18T23:05:18.056 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":81,"num_osds":12,"num_up_osds":12,"osd_up_since":1710803116,"num_in_osds":12,"osd_in_since":1710803097,"num_remapped_pgs":0} 2024-03-18T23:05:18.056 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell --fsid c63591ea-e57a-11ee-95c9-87774f69a715 -- ceph osd dump --format=json 2024-03-18T23:05:18.065 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:05:17 smithi139 ceph-mon[67282]: purged_snaps scrub starts 2024-03-18T23:05:18.065 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:05:17 smithi139 ceph-mon[67282]: purged_snaps scrub ok 2024-03-18T23:05:18.065 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:05:17 smithi139 ceph-mon[67282]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-03-18T23:05:18.065 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:05:17 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-18T23:05:18.065 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:05:17 smithi139 ceph-mon[67282]: osd.11 [v2:172.21.15.175:6824/3519503868,v1:172.21.15.175:6825/3519503868] boot 2024-03-18T23:05:18.065 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:05:17 smithi139 ceph-mon[67282]: osdmap e81: 12 total, 12 up, 12 in 2024-03-18T23:05:18.065 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:05:17 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-18T23:05:18.065 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:05:17 smithi139 ceph-mon[67282]: from='client.? 172.21.15.49:0/4231210944' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-18T23:05:18.317 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T23:05:18.773 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:05:18 smithi049 ceph-mon[62991]: osdmap e82: 12 total, 12 up, 12 in 2024-03-18T23:05:18.773 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:05:18 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:05:18.773 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:05:18 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:05:18.773 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:05:18 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:05:18.773 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:05:18 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:05:18.773 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:05:18 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:05:18.773 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:05:18 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:05:18.773 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:05:18 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:05:18.773 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:05:18 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:05:18.773 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:05:18 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:05:18.773 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:05:18 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:05:18.961 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:05:18 smithi175 ceph-mon[67352]: osdmap e82: 12 total, 12 up, 12 in 2024-03-18T23:05:18.962 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:05:18 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:05:18.962 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:05:18 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:05:18.962 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:05:18 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:05:18.962 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:05:18 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:05:18.962 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:05:18 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:05:18.962 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:05:18 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:05:18.962 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:05:18 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:05:18.962 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:05:18 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:05:18.962 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:05:18 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:05:18.962 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:05:18 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:05:19.065 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:05:18 smithi139 ceph-mon[67282]: osdmap e82: 12 total, 12 up, 12 in 2024-03-18T23:05:19.065 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:05:18 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:05:19.065 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:05:18 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:05:19.065 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:05:18 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:05:19.065 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:05:18 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:05:19.065 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:05:18 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:05:19.066 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:05:18 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:05:19.066 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:05:18 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:05:19.066 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:05:18 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:05:19.066 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:05:18 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:05:19.066 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:05:18 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:05:19.871 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:05:19 smithi049 ceph-mon[62991]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T23:05:19.871 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:05:19 smithi049 ceph-mon[62991]: Detected new or changed devices on smithi175 2024-03-18T23:05:19.872 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:05:19 smithi049 ceph-mon[62991]: Adjusting osd_memory_target on smithi175 to 4270M 2024-03-18T23:05:19.961 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:05:19 smithi175 ceph-mon[67352]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T23:05:19.962 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:05:19 smithi175 ceph-mon[67352]: Detected new or changed devices on smithi175 2024-03-18T23:05:19.962 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:05:19 smithi175 ceph-mon[67352]: Adjusting osd_memory_target on smithi175 to 4270M 2024-03-18T23:05:20.065 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:05:19 smithi139 ceph-mon[67282]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T23:05:20.065 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:05:19 smithi139 ceph-mon[67282]: Detected new or changed devices on smithi175 2024-03-18T23:05:20.065 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:05:19 smithi139 ceph-mon[67282]: Adjusting osd_memory_target on smithi175 to 4270M 2024-03-18T23:05:20.145 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:20.144+0000 7f1a6be7f700 1 -- 172.21.15.49:0/529464766 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1a6406ac10 msgr2=0x7f1a6406aff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:05:20.146 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:20.144+0000 7f1a6be7f700 1 --2- 172.21.15.49:0/529464766 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1a6406ac10 0x7f1a6406aff0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f1a600099c0 tx=0x7f1a60009cd0 comp rx=0 tx=0).stop 2024-03-18T23:05:20.146 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:20.144+0000 7f1a6be7f700 1 -- 172.21.15.49:0/529464766 shutdown_connections 2024-03-18T23:05:20.146 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:20.144+0000 7f1a6be7f700 1 --2- 172.21.15.49:0/529464766 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1a64107680 0x7f1a6410f200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:05:20.146 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:20.144+0000 7f1a6be7f700 1 --2- 172.21.15.49:0/529464766 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1a6406ac10 0x7f1a6406aff0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:05:20.146 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:20.144+0000 7f1a6be7f700 1 --2- 172.21.15.49:0/529464766 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1a6406b5c0 0x7f1a64106f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:05:20.146 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:20.144+0000 7f1a6be7f700 1 -- 172.21.15.49:0/529464766 >> 172.21.15.49:0/529464766 conn(0x7f1a64078180 msgr2=0x7f1a64078590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:05:20.146 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:20.144+0000 7f1a6be7f700 1 -- 172.21.15.49:0/529464766 shutdown_connections 2024-03-18T23:05:20.146 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:20.144+0000 7f1a6be7f700 1 -- 172.21.15.49:0/529464766 wait complete. 2024-03-18T23:05:20.148 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:20.145+0000 7f1a6be7f700 1 Processor -- start 2024-03-18T23:05:20.148 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:20.145+0000 7f1a6be7f700 1 -- start start 2024-03-18T23:05:20.148 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:20.145+0000 7f1a6be7f700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1a6406ac10 0x7f1a6419e880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:05:20.148 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:20.145+0000 7f1a6be7f700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1a6406b5c0 0x7f1a6419edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:05:20.148 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:20.145+0000 7f1a6be7f700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1a64107680 0x7f1a64198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:05:20.149 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:20.145+0000 7f1a6be7f700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f1a64112740 con 0x7f1a6406ac10 2024-03-18T23:05:20.149 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:20.145+0000 7f1a6be7f700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f1a641125c0 con 0x7f1a64107680 2024-03-18T23:05:20.149 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:20.145+0000 7f1a6be7f700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f1a641128c0 con 0x7f1a6406b5c0 2024-03-18T23:05:20.149 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:20.145+0000 7f1a69c1b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1a6406ac10 0x7f1a6419e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:05:20.149 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:20.145+0000 7f1a69c1b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1a6406ac10 0x7f1a6419e880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:45400/0 (socket says 172.21.15.49:45400) 2024-03-18T23:05:20.149 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:20.145+0000 7f1a69c1b700 1 -- 172.21.15.49:0/3801621672 learned_addr learned my addr 172.21.15.49:0/3801621672 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:05:20.149 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:20.145+0000 7f1a6a41c700 1 --2- 172.21.15.49:0/3801621672 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1a64107680 0x7f1a64198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:05:20.149 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:20.145+0000 7f1a6941a700 1 --2- 172.21.15.49:0/3801621672 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1a6406b5c0 0x7f1a6419edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:05:20.149 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:20.145+0000 7f1a69c1b700 1 -- 172.21.15.49:0/3801621672 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1a6406b5c0 msgr2=0x7f1a6419edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:05:20.149 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:20.145+0000 7f1a69c1b700 1 --2- 172.21.15.49:0/3801621672 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1a6406b5c0 0x7f1a6419edc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:05:20.149 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:20.145+0000 7f1a69c1b700 1 -- 172.21.15.49:0/3801621672 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1a64107680 msgr2=0x7f1a64198a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:05:20.150 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:20.145+0000 7f1a69c1b700 1 --2- 172.21.15.49:0/3801621672 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1a64107680 0x7f1a64198a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:05:20.150 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:20.145+0000 7f1a69c1b700 1 -- 172.21.15.49:0/3801621672 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a641991a0 con 0x7f1a6406ac10 2024-03-18T23:05:20.150 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:20.145+0000 7f1a6941a700 1 --2- 172.21.15.49:0/3801621672 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1a6406b5c0 0x7f1a6419edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T23:05:20.150 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:20.146+0000 7f1a69c1b700 1 --2- 172.21.15.49:0/3801621672 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1a6406ac10 0x7f1a6419e880 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f1a60005b40 tx=0x7f1a600057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:05:20.150 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:20.146+0000 7f1a5affd700 1 -- 172.21.15.49:0/3801621672 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a60027070 con 0x7f1a6406ac10 2024-03-18T23:05:20.150 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:20.146+0000 7f1a6be7f700 1 -- 172.21.15.49:0/3801621672 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a64199430 con 0x7f1a6406ac10 2024-03-18T23:05:20.150 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:20.146+0000 7f1a5affd700 1 -- 172.21.15.49:0/3801621672 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1a600040b0 con 0x7f1a6406ac10 2024-03-18T23:05:20.150 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:20.146+0000 7f1a5affd700 1 -- 172.21.15.49:0/3801621672 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a6000f460 con 0x7f1a6406ac10 2024-03-18T23:05:20.150 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:20.146+0000 7f1a6be7f700 1 -- 172.21.15.49:0/3801621672 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a641a5750 con 0x7f1a6406ac10 2024-03-18T23:05:20.151 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:20.147+0000 7f1a6be7f700 1 -- 172.21.15.49:0/3801621672 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a6410a850 con 0x7f1a6406ac10 2024-03-18T23:05:20.151 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:20.148+0000 7f1a5affd700 1 -- 172.21.15.49:0/3801621672 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7f1a6002a070 con 0x7f1a6406ac10 2024-03-18T23:05:20.151 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:20.149+0000 7f1a5affd700 1 --2- 172.21.15.49:0/3801621672 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f1a500a2890 0x7f1a500a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:05:20.152 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:20.149+0000 7f1a6941a700 1 --2- 172.21.15.49:0/3801621672 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f1a500a2890 0x7f1a500a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:05:20.152 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:20.149+0000 7f1a5affd700 1 -- 172.21.15.49:0/3801621672 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f1a600cfc30 con 0x7f1a6406ac10 2024-03-18T23:05:20.152 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:20.149+0000 7f1a6941a700 1 --2- 172.21.15.49:0/3801621672 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f1a500a2890 0x7f1a500a4d50 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f1a64100e60 tx=0x7f1a54008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:05:20.153 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:20.152+0000 7f1a5affd700 1 -- 172.21.15.49:0/3801621672 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f1a60031020 con 0x7f1a6406ac10 2024-03-18T23:05:20.358 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:20.356+0000 7f1a6be7f700 1 -- 172.21.15.49:0/3801621672 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1a64069000 con 0x7f1a6406ac10 2024-03-18T23:05:20.359 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:20.357+0000 7f1a5affd700 1 -- 172.21.15.49:0/3801621672 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+18479 (secure 0 0 0) 0x7f1a64069000 con 0x7f1a6406ac10 2024-03-18T23:05:20.360 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-18T23:05:20.360 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":82,"fsid":"c63591ea-e57a-11ee-95c9-87774f69a715","created":"2024-03-18T22:57:58.734467+0000","modified":"2024-03-18T23:05:17.581588+0000","last_up_change":"2024-03-18T23:05:16.576310+0000","last_in_change":"2024-03-18T23:04:57.024067+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-18T23:01:28.363432+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e7fc9daa-4098-4ba5-8019-9aaef1aab3e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":1651162501},{"type":"v1","addr":"172.21.15.49:6803","nonce":1651162501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":1651162501},{"type":"v1","addr":"172.21.15.49:6805","nonce":1651162501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":1651162501},{"type":"v1","addr":"172.21.15.49:6809","nonce":1651162501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":1651162501},{"type":"v1","addr":"172.21.15.49:6807","nonce":1651162501}]},"public_addr":"172.21.15.49:6803/1651162501","cluster_addr":"172.21.15.49:6805/1651162501","heartbeat_back_addr":"172.21.15.49:6809/1651162501","heartbeat_front_addr":"172.21.15.49:6807/1651162501","state":["exists","up"]},{"osd":1,"uuid":"40ac929c-e90f-434c-a796-9aabd686f1fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":1766700359},{"type":"v1","addr":"172.21.15.139:6801","nonce":1766700359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":1766700359},{"type":"v1","addr":"172.21.15.139:6803","nonce":1766700359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":1766700359},{"type":"v1","addr":"172.21.15.139:6807","nonce":1766700359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":1766700359},{"type":"v1","addr":"172.21.15.139:6805","nonce":1766700359}]},"public_addr":"172.21.15.139:6801/1766700359","cluster_addr":"172.21.15.139:6803/1766700359","heartbeat_back_addr":"172.21.15.139:6807/1766700359","heartbeat_front_addr":"172.21.15.139:6805/1766700359","state":["exists","up"]},{"osd":2,"uuid":"8ae3b666-00b4-45ed-a5ee-4d0771edabb6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":3752671234},{"type":"v1","addr":"172.21.15.175:6801","nonce":3752671234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":3752671234},{"type":"v1","addr":"172.21.15.175:6803","nonce":3752671234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":3752671234},{"type":"v1","addr":"172.21.15.175:6807","nonce":3752671234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":3752671234},{"type":"v1","addr":"172.21.15.175:6805","nonce":3752671234}]},"public_addr":"172.21.15.175:6801/3752671234","cluster_addr":"172.21.15.175:6803/3752671234","heartbeat_back_addr":"172.21.15.175:6807/3752671234","heartbeat_front_addr":"172.21.15.175:6805/3752671234","state":["exists","up"]},{"osd":3,"uuid":"1a62e89d-45d4-4237-b813-7f96a59aca7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":956160536},{"type":"v1","addr":"172.21.15.49:6811","nonce":956160536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":956160536},{"type":"v1","addr":"172.21.15.49:6813","nonce":956160536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":956160536},{"type":"v1","addr":"172.21.15.49:6817","nonce":956160536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":956160536},{"type":"v1","addr":"172.21.15.49:6815","nonce":956160536}]},"public_addr":"172.21.15.49:6811/956160536","cluster_addr":"172.21.15.49:6813/956160536","heartbeat_back_addr":"172.21.15.49:6817/956160536","heartbeat_front_addr":"172.21.15.49:6815/956160536","state":["exists","up"]},{"osd":4,"uuid":"c715101c-0c3f-4b98-b142-d64a35e187df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":3129712668},{"type":"v1","addr":"172.21.15.139:6809","nonce":3129712668}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":3129712668},{"type":"v1","addr":"172.21.15.139:6811","nonce":3129712668}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":3129712668},{"type":"v1","addr":"172.21.15.139:6815","nonce":3129712668}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":3129712668},{"type":"v1","addr":"172.21.15.139:6813","nonce":3129712668}]},"public_addr":"172.21.15.139:6809/3129712668","cluster_addr":"172.21.15.139:6811/3129712668","heartbeat_back_addr":"172.21.15.139:6815/3129712668","heartbeat_front_addr":"172.21.15.139:6813/3129712668","state":["exists","up"]},{"osd":5,"uuid":"4b641f81-77eb-45fd-8464-511e7b3062b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":2774567215},{"type":"v1","addr":"172.21.15.175:6809","nonce":2774567215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":2774567215},{"type":"v1","addr":"172.21.15.175:6811","nonce":2774567215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":2774567215},{"type":"v1","addr":"172.21.15.175:6815","nonce":2774567215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":2774567215},{"type":"v1","addr":"172.21.15.175:6813","nonce":2774567215}]},"public_addr":"172.21.15.175:6809/2774567215","cluster_addr":"172.21.15.175:6811/2774567215","heartbeat_back_addr":"172.21.15.175:6815/2774567215","heartbeat_front_addr":"172.21.15.175:6813/2774567215","state":["exists","up"]},{"osd":6,"uuid":"653da8e0-1197-4b48-abed-470f2a9c4dce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":1472215461},{"type":"v1","addr":"172.21.15.49:6819","nonce":1472215461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":1472215461},{"type":"v1","addr":"172.21.15.49:6821","nonce":1472215461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":1472215461},{"type":"v1","addr":"172.21.15.49:6825","nonce":1472215461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":1472215461},{"type":"v1","addr":"172.21.15.49:6823","nonce":1472215461}]},"public_addr":"172.21.15.49:6819/1472215461","cluster_addr":"172.21.15.49:6821/1472215461","heartbeat_back_addr":"172.21.15.49:6825/1472215461","heartbeat_front_addr":"172.21.15.49:6823/1472215461","state":["exists","up"]},{"osd":7,"uuid":"ad157c9b-4b75-451c-8740-630f0975b7fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":2163986787},{"type":"v1","addr":"172.21.15.139:6817","nonce":2163986787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":2163986787},{"type":"v1","addr":"172.21.15.139:6819","nonce":2163986787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":2163986787},{"type":"v1","addr":"172.21.15.139:6823","nonce":2163986787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":2163986787},{"type":"v1","addr":"172.21.15.139:6821","nonce":2163986787}]},"public_addr":"172.21.15.139:6817/2163986787","cluster_addr":"172.21.15.139:6819/2163986787","heartbeat_back_addr":"172.21.15.139:6823/2163986787","heartbeat_front_addr":"172.21.15.139:6821/2163986787","state":["exists","up"]},{"osd":8,"uuid":"c8e75f15-df75-4d5d-9b92-a064bffb6c01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":1194026992},{"type":"v1","addr":"172.21.15.175:6817","nonce":1194026992}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1194026992},{"type":"v1","addr":"172.21.15.175:6819","nonce":1194026992}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":1194026992},{"type":"v1","addr":"172.21.15.175:6823","nonce":1194026992}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1194026992},{"type":"v1","addr":"172.21.15.175:6821","nonce":1194026992}]},"public_addr":"172.21.15.175:6817/1194026992","cluster_addr":"172.21.15.175:6819/1194026992","heartbeat_back_addr":"172.21.15.175:6823/1194026992","heartbeat_front_addr":"172.21.15.175:6821/1194026992","state":["exists","up"]},{"osd":9,"uuid":"edb5ffef-b6a2-4941-b5ab-40f982ad30d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":2742084884},{"type":"v1","addr":"172.21.15.49:6827","nonce":2742084884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":2742084884},{"type":"v1","addr":"172.21.15.49:6829","nonce":2742084884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":2742084884},{"type":"v1","addr":"172.21.15.49:6833","nonce":2742084884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":2742084884},{"type":"v1","addr":"172.21.15.49:6831","nonce":2742084884}]},"public_addr":"172.21.15.49:6827/2742084884","cluster_addr":"172.21.15.49:6829/2742084884","heartbeat_back_addr":"172.21.15.49:6833/2742084884","heartbeat_front_addr":"172.21.15.49:6831/2742084884","state":["exists","up"]},{"osd":10,"uuid":"3945e46d-766a-4c4a-8821-d43648dd1578","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":2404740201},{"type":"v1","addr":"172.21.15.139:6825","nonce":2404740201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":2404740201},{"type":"v1","addr":"172.21.15.139:6827","nonce":2404740201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":2404740201},{"type":"v1","addr":"172.21.15.139:6831","nonce":2404740201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":2404740201},{"type":"v1","addr":"172.21.15.139:6829","nonce":2404740201}]},"public_addr":"172.21.15.139:6825/2404740201","cluster_addr":"172.21.15.139:6827/2404740201","heartbeat_back_addr":"172.21.15.139:6831/2404740201","heartbeat_front_addr":"172.21.15.139:6829/2404740201","state":["exists","up"]},{"osd":11,"uuid":"9954ba15-d877-46cc-b24a-893f9168deb8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":3519503868},{"type":"v1","addr":"172.21.15.175:6825","nonce":3519503868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":3519503868},{"type":"v1","addr":"172.21.15.175:6827","nonce":3519503868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":3519503868},{"type":"v1","addr":"172.21.15.175:6831","nonce":3519503868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":3519503868},{"type":"v1","addr":"172.21.15.175:6829","nonce":3519503868}]},"public_addr":"172.21.15.175:6825/3519503868","cluster_addr":"172.21.15.175:6827/3519503868","heartbeat_back_addr":"172.21.15.175:6831/3519503868","heartbeat_front_addr":"172.21.15.175:6829/3519503868","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:00:38.367865+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:01:02.237687+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:01:25.609751+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:01:51.268572+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:02:16.514605+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:02:41.586523+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:03:07.029964+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:03:32.956250+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:03:57.547984+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:04:23.948972+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:04:49.667210+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:05:15.284693+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.49:0/563527416":"2024-03-19T22:58:59.806253+0000","172.21.15.49:0/1036383534":"2024-03-19T22:58:59.806253+0000","172.21.15.49:6801/2609458951":"2024-03-19T22:58:59.806253+0000","172.21.15.49:0/21283102":"2024-03-19T22:58:31.972474+0000","172.21.15.49:0/1292058922":"2024-03-19T22:58:31.972474+0000","172.21.15.49:6800/1985190312":"2024-03-19T22:58:31.972474+0000","172.21.15.49:0/1912446700":"2024-03-19T22:58:59.806253+0000","172.21.15.49:0/2176210039":"2024-03-19T22:58:31.972474+0000","172.21.15.49:6801/1985190312":"2024-03-19T22:58:31.972474+0000","172.21.15.49:6800/2609458951":"2024-03-19T22:58:59.806253+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-18T23:05:20.362 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:20.360+0000 7f1a6be7f700 1 -- 172.21.15.49:0/3801621672 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f1a500a2890 msgr2=0x7f1a500a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:05:20.362 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:20.360+0000 7f1a6be7f700 1 --2- 172.21.15.49:0/3801621672 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f1a500a2890 0x7f1a500a4d50 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f1a64100e60 tx=0x7f1a54008040 comp rx=0 tx=0).stop 2024-03-18T23:05:20.362 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:20.360+0000 7f1a6be7f700 1 -- 172.21.15.49:0/3801621672 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1a6406ac10 msgr2=0x7f1a6419e880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:05:20.362 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:20.360+0000 7f1a6be7f700 1 --2- 172.21.15.49:0/3801621672 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1a6406ac10 0x7f1a6419e880 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f1a60005b40 tx=0x7f1a600057c0 comp rx=0 tx=0).stop 2024-03-18T23:05:20.362 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:20.360+0000 7f1a6be7f700 1 -- 172.21.15.49:0/3801621672 shutdown_connections 2024-03-18T23:05:20.363 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:20.360+0000 7f1a6be7f700 1 --2- 172.21.15.49:0/3801621672 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1a64107680 0x7f1a64198a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:05:20.363 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:20.360+0000 7f1a6be7f700 1 --2- 172.21.15.49:0/3801621672 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1a6406ac10 0x7f1a6419e880 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:05:20.363 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:20.360+0000 7f1a6be7f700 1 --2- 172.21.15.49:0/3801621672 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f1a500a2890 0x7f1a500a4d50 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:05:20.363 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:20.360+0000 7f1a6be7f700 1 --2- 172.21.15.49:0/3801621672 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1a6406b5c0 0x7f1a6419edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:05:20.363 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:20.360+0000 7f1a6be7f700 1 -- 172.21.15.49:0/3801621672 >> 172.21.15.49:0/3801621672 conn(0x7f1a64078180 msgr2=0x7f1a640ff510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:05:20.363 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:20.360+0000 7f1a6be7f700 1 -- 172.21.15.49:0/3801621672 shutdown_connections 2024-03-18T23:05:20.364 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:20.360+0000 7f1a6be7f700 1 -- 172.21.15.49:0/3801621672 wait complete. 2024-03-18T23:05:20.750 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:05:20 smithi049 ceph-mon[62991]: from='client.? 172.21.15.49:0/3801621672' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T23:05:20.961 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:05:20 smithi175 ceph-mon[67352]: from='client.? 172.21.15.49:0/3801621672' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T23:05:21.031 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-18T23:01:28.363432+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-18T23:05:21.032 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell --fsid c63591ea-e57a-11ee-95c9-87774f69a715 -- ceph osd pool get .mgr pg_num 2024-03-18T23:05:21.065 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:05:20 smithi139 ceph-mon[67282]: from='client.? 172.21.15.49:0/3801621672' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T23:05:21.284 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T23:05:21.740 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:05:21 smithi049 ceph-mon[62991]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T23:05:21.953 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:05:21 smithi175 ceph-mon[67352]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T23:05:22.065 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:05:21 smithi139 ceph-mon[67282]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T23:05:22.932 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:22.929+0000 7fe7410bd700 1 -- 172.21.15.49:0/2208697279 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe73c1025f0 msgr2=0x7fe73c1029d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:05:22.932 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:22.929+0000 7fe7410bd700 1 --2- 172.21.15.49:0/2208697279 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe73c1025f0 0x7fe73c1029d0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fe72c0099c0 tx=0x7fe72c009cd0 comp rx=0 tx=0).stop 2024-03-18T23:05:22.932 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:22.929+0000 7fe7410bd700 1 -- 172.21.15.49:0/2208697279 shutdown_connections 2024-03-18T23:05:22.932 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:22.929+0000 7fe7410bd700 1 --2- 172.21.15.49:0/2208697279 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe73c1076a0 0x7fe73c10f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:05:22.932 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:22.929+0000 7fe7410bd700 1 --2- 172.21.15.49:0/2208697279 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe73c1025f0 0x7fe73c1029d0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:05:22.933 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:22.929+0000 7fe7410bd700 1 --2- 172.21.15.49:0/2208697279 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe73c102fa0 0x7fe73c106f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:05:22.933 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:22.929+0000 7fe7410bd700 1 -- 172.21.15.49:0/2208697279 >> 172.21.15.49:0/2208697279 conn(0x7fe73c079430 msgr2=0x7fe73c079840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:05:22.933 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:22.929+0000 7fe7410bd700 1 -- 172.21.15.49:0/2208697279 shutdown_connections 2024-03-18T23:05:22.933 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:22.929+0000 7fe7410bd700 1 -- 172.21.15.49:0/2208697279 wait complete. 2024-03-18T23:05:22.933 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:22.929+0000 7fe7410bd700 1 Processor -- start 2024-03-18T23:05:22.933 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:22.930+0000 7fe7410bd700 1 -- start start 2024-03-18T23:05:22.933 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:22.930+0000 7fe7410bd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe73c1025f0 0x7fe73c074610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:05:22.933 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:22.930+0000 7fe7410bd700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe73c102fa0 0x7fe73c074b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:05:22.933 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:22.930+0000 7fe7410bd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe73c1076a0 0x7fe73c075090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:05:22.933 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:22.930+0000 7fe7410bd700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fe73c112710 con 0x7fe73c1025f0 2024-03-18T23:05:22.933 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:22.930+0000 7fe7410bd700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fe73c112590 con 0x7fe73c102fa0 2024-03-18T23:05:22.933 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:22.930+0000 7fe7410bd700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe73c112890 con 0x7fe73c1076a0 2024-03-18T23:05:22.934 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:22.930+0000 7fe73ad9d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe73c1025f0 0x7fe73c074610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:05:22.934 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:22.930+0000 7fe73ad9d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe73c1025f0 0x7fe73c074610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:45422/0 (socket says 172.21.15.49:45422) 2024-03-18T23:05:22.934 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:22.930+0000 7fe73ad9d700 1 -- 172.21.15.49:0/1164509710 learned_addr learned my addr 172.21.15.49:0/1164509710 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:05:22.934 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:22.930+0000 7fe73a59c700 1 --2- 172.21.15.49:0/1164509710 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe73c102fa0 0x7fe73c074b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:05:22.934 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:22.930+0000 7fe73b59e700 1 --2- 172.21.15.49:0/1164509710 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe73c1076a0 0x7fe73c075090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:05:22.934 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:22.930+0000 7fe73ad9d700 1 -- 172.21.15.49:0/1164509710 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe73c1076a0 msgr2=0x7fe73c075090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:05:22.934 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:22.930+0000 7fe73ad9d700 1 --2- 172.21.15.49:0/1164509710 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe73c1076a0 0x7fe73c075090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:05:22.934 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:22.930+0000 7fe73ad9d700 1 -- 172.21.15.49:0/1164509710 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe73c102fa0 msgr2=0x7fe73c074b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:05:22.934 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:22.930+0000 7fe73ad9d700 1 --2- 172.21.15.49:0/1164509710 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe73c102fa0 0x7fe73c074b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:05:22.934 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:22.930+0000 7fe73ad9d700 1 -- 172.21.15.49:0/1164509710 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe73c1a7170 con 0x7fe73c1025f0 2024-03-18T23:05:22.934 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:22.930+0000 7fe73b59e700 1 --2- 172.21.15.49:0/1164509710 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe73c1076a0 0x7fe73c075090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T23:05:22.934 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:22.931+0000 7fe73ad9d700 1 --2- 172.21.15.49:0/1164509710 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe73c1025f0 0x7fe73c074610 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fe72c00c010 tx=0x7fe72c005e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:05:22.934 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:22.931+0000 7fe723fff700 1 -- 172.21.15.49:0/1164509710 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe72c027070 con 0x7fe73c1025f0 2024-03-18T23:05:22.935 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:22.931+0000 7fe7410bd700 1 -- 172.21.15.49:0/1164509710 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe73c1a7400 con 0x7fe73c1025f0 2024-03-18T23:05:22.935 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:22.931+0000 7fe723fff700 1 -- 172.21.15.49:0/1164509710 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe72c004020 con 0x7fe73c1025f0 2024-03-18T23:05:22.935 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:22.931+0000 7fe723fff700 1 -- 172.21.15.49:0/1164509710 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe72c022e00 con 0x7fe73c1025f0 2024-03-18T23:05:22.935 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:22.931+0000 7fe7410bd700 1 -- 172.21.15.49:0/1164509710 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe73c1a78b0 con 0x7fe73c1025f0 2024-03-18T23:05:22.936 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:22.932+0000 7fe7410bd700 1 -- 172.21.15.49:0/1164509710 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe73c10a850 con 0x7fe73c1025f0 2024-03-18T23:05:22.936 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:22.933+0000 7fe723fff700 1 -- 172.21.15.49:0/1164509710 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7fe72c02a070 con 0x7fe73c1025f0 2024-03-18T23:05:22.936 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:22.934+0000 7fe723fff700 1 --2- 172.21.15.49:0/1164509710 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fe7240a2890 0x7fe7240a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:05:22.936 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:22.934+0000 7fe73a59c700 1 --2- 172.21.15.49:0/1164509710 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fe7240a2890 0x7fe7240a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:05:22.936 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:22.934+0000 7fe723fff700 1 -- 172.21.15.49:0/1164509710 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fe72c0cf3e0 con 0x7fe73c1025f0 2024-03-18T23:05:22.936 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:22.934+0000 7fe73a59c700 1 --2- 172.21.15.49:0/1164509710 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fe7240a2890 0x7fe7240a4d50 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fe73c101ff0 tx=0x7fe730009190 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:05:22.939 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:22.938+0000 7fe723fff700 1 -- 172.21.15.49:0/1164509710 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fe72c05e740 con 0x7fe73c1025f0 2024-03-18T23:05:23.145 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:23.144+0000 7fe7410bd700 1 -- 172.21.15.49:0/1164509710 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fe73c075b40 con 0x7fe73c1025f0 2024-03-18T23:05:23.145 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:23.144+0000 7fe723fff700 1 -- 172.21.15.49:0/1164509710 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) ==== 93+0+10 (secure 0 0 0) 0x7fe72c013070 con 0x7fe73c1025f0 2024-03-18T23:05:23.145 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 1 2024-03-18T23:05:23.147 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:23.146+0000 7fe7410bd700 1 -- 172.21.15.49:0/1164509710 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fe7240a2890 msgr2=0x7fe7240a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:05:23.148 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:23.146+0000 7fe7410bd700 1 --2- 172.21.15.49:0/1164509710 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fe7240a2890 0x7fe7240a4d50 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fe73c101ff0 tx=0x7fe730009190 comp rx=0 tx=0).stop 2024-03-18T23:05:23.148 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:23.146+0000 7fe7410bd700 1 -- 172.21.15.49:0/1164509710 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe73c1025f0 msgr2=0x7fe73c074610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:05:23.148 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:23.146+0000 7fe7410bd700 1 --2- 172.21.15.49:0/1164509710 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe73c1025f0 0x7fe73c074610 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fe72c00c010 tx=0x7fe72c005e70 comp rx=0 tx=0).stop 2024-03-18T23:05:23.148 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:23.146+0000 7fe7410bd700 1 -- 172.21.15.49:0/1164509710 shutdown_connections 2024-03-18T23:05:23.148 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:23.146+0000 7fe7410bd700 1 --2- 172.21.15.49:0/1164509710 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fe7240a2890 0x7fe7240a4d50 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:05:23.148 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:23.146+0000 7fe7410bd700 1 --2- 172.21.15.49:0/1164509710 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe73c1076a0 0x7fe73c075090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:05:23.148 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:23.146+0000 7fe7410bd700 1 --2- 172.21.15.49:0/1164509710 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe73c1025f0 0x7fe73c074610 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:05:23.148 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:23.146+0000 7fe7410bd700 1 --2- 172.21.15.49:0/1164509710 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe73c102fa0 0x7fe73c074b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:05:23.148 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:23.146+0000 7fe7410bd700 1 -- 172.21.15.49:0/1164509710 >> 172.21.15.49:0/1164509710 conn(0x7fe73c079430 msgr2=0x7fe73c100790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:05:23.148 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:23.147+0000 7fe7410bd700 1 -- 172.21.15.49:0/1164509710 shutdown_connections 2024-03-18T23:05:23.148 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:23.147+0000 7fe7410bd700 1 -- 172.21.15.49:0/1164509710 wait complete. 2024-03-18T23:05:23.762 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:05:23 smithi049 ceph-mon[62991]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T23:05:23.762 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:05:23 smithi049 ceph-mon[62991]: from='client.? 172.21.15.49:0/1164509710' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-18T23:05:23.794 INFO:tasks.cephadm:Adding mds.a on smithi049 2024-03-18T23:05:23.794 INFO:tasks.cephadm:Adding mds.d on smithi049 2024-03-18T23:05:23.794 INFO:tasks.cephadm:Adding mds.g on smithi049 2024-03-18T23:05:23.794 INFO:tasks.cephadm:Adding mds.j on smithi049 2024-03-18T23:05:23.794 INFO:tasks.cephadm:Adding mds.b on smithi139 2024-03-18T23:05:23.794 INFO:tasks.cephadm:Adding mds.e on smithi139 2024-03-18T23:05:23.795 INFO:tasks.cephadm:Adding mds.h on smithi139 2024-03-18T23:05:23.795 INFO:tasks.cephadm:Adding mds.k on smithi139 2024-03-18T23:05:23.795 INFO:tasks.cephadm:Adding mds.c on smithi175 2024-03-18T23:05:23.795 INFO:tasks.cephadm:Adding mds.f on smithi175 2024-03-18T23:05:23.795 INFO:tasks.cephadm:Adding mds.i on smithi175 2024-03-18T23:05:23.795 INFO:tasks.cephadm:Adding mds.l on smithi175 2024-03-18T23:05:23.795 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c63591ea-e57a-11ee-95c9-87774f69a715 -- ceph orch apply mds all '12;smithi049=a;smithi049=d;smithi049=g;smithi049=j;smithi139=b;smithi139=e;smithi139=h;smithi139=k;smithi175=c;smithi175=f;smithi175=i;smithi175=l' 2024-03-18T23:05:23.961 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:05:23 smithi175 ceph-mon[67352]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T23:05:23.962 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:05:23 smithi175 ceph-mon[67352]: from='client.? 172.21.15.49:0/1164509710' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-18T23:05:24.045 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/mon.c/config 2024-03-18T23:05:24.065 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:05:23 smithi139 ceph-mon[67282]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T23:05:24.065 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:05:23 smithi139 ceph-mon[67282]: from='client.? 172.21.15.49:0/1164509710' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-18T23:05:25.736 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:05:25.735+0000 7f8e6a0e8700 1 -- 172.21.15.175:0/124351703 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8e64103f80 msgr2=0x7f8e6410a830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:05:25.736 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:05:25.735+0000 7f8e6a0e8700 1 --2- 172.21.15.175:0/124351703 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8e64103f80 0x7f8e6410a830 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f8e54009990 tx=0x7f8e54009ca0 comp rx=0 tx=0).stop 2024-03-18T23:05:25.736 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:05:25.735+0000 7f8e6a0e8700 1 -- 172.21.15.175:0/124351703 shutdown_connections 2024-03-18T23:05:25.736 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:05:25.735+0000 7f8e6a0e8700 1 --2- 172.21.15.175:0/124351703 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8e64103f80 0x7f8e6410a830 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:05:25.737 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:05:25.735+0000 7f8e6a0e8700 1 --2- 172.21.15.175:0/124351703 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8e64100600 0x7f8e64100a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:05:25.737 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:05:25.735+0000 7f8e6a0e8700 1 --2- 172.21.15.175:0/124351703 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8e641035e0 0x7f8e64103a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:05:25.737 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:05:25.735+0000 7f8e6a0e8700 1 -- 172.21.15.175:0/124351703 >> 172.21.15.175:0/124351703 conn(0x7f8e640fbf20 msgr2=0x7f8e640fe3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:05:25.737 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:05:25.735+0000 7f8e6a0e8700 1 -- 172.21.15.175:0/124351703 shutdown_connections 2024-03-18T23:05:25.737 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:05:25.735+0000 7f8e6a0e8700 1 -- 172.21.15.175:0/124351703 wait complete. 2024-03-18T23:05:25.737 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:05:25.735+0000 7f8e6a0e8700 1 Processor -- start 2024-03-18T23:05:25.739 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:05:25.736+0000 7f8e6a0e8700 1 -- start start 2024-03-18T23:05:25.739 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:05:25.736+0000 7f8e6a0e8700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8e64100600 0x7f8e64198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:05:25.739 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:05:25.736+0000 7f8e6a0e8700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8e641035e0 0x7f8e641989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:05:25.739 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:05:25.736+0000 7f8e6a0e8700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8e64103f80 0x7f8e6419fa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:05:25.739 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:05:25.736+0000 7f8e6a0e8700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f8e6410df00 con 0x7f8e64100600 2024-03-18T23:05:25.739 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:05:25.736+0000 7f8e6a0e8700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f8e6410dd80 con 0x7f8e641035e0 2024-03-18T23:05:25.739 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:05:25.736+0000 7f8e6a0e8700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f8e6410e080 con 0x7f8e64103f80 2024-03-18T23:05:25.740 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:05:25.736+0000 7f8e63fff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8e64103f80 0x7f8e6419fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:05:25.740 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:05:25.736+0000 7f8e63fff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8e64103f80 0x7f8e6419fa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:43496/0 (socket says 172.21.15.175:43496) 2024-03-18T23:05:25.740 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:05:25.736+0000 7f8e63fff700 1 -- 172.21.15.175:0/2265085991 learned_addr learned my addr 172.21.15.175:0/2265085991 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-03-18T23:05:25.740 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:05:25.736+0000 7f8e637fe700 1 --2- 172.21.15.175:0/2265085991 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8e64100600 0x7f8e64198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:05:25.740 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:05:25.736+0000 7f8e62ffd700 1 --2- 172.21.15.175:0/2265085991 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8e641035e0 0x7f8e641989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:05:25.740 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:05:25.736+0000 7f8e63fff700 1 -- 172.21.15.175:0/2265085991 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8e641035e0 msgr2=0x7f8e641989a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:05:25.740 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:05:25.736+0000 7f8e63fff700 1 --2- 172.21.15.175:0/2265085991 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8e641035e0 0x7f8e641989a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:05:25.740 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:05:25.736+0000 7f8e63fff700 1 -- 172.21.15.175:0/2265085991 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8e64100600 msgr2=0x7f8e64198460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:05:25.740 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:05:25.736+0000 7f8e63fff700 1 --2- 172.21.15.175:0/2265085991 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8e64100600 0x7f8e64198460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:05:25.740 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:05:25.736+0000 7f8e63fff700 1 -- 172.21.15.175:0/2265085991 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e641a00c0 con 0x7f8e64103f80 2024-03-18T23:05:25.740 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:05:25.736+0000 7f8e63fff700 1 --2- 172.21.15.175:0/2265085991 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8e64103f80 0x7f8e6419fa20 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f8e54005910 tx=0x7f8e540057c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:05:25.740 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:05:25.737+0000 7f8e60ff9700 1 -- 172.21.15.175:0/2265085991 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e54027070 con 0x7f8e64103f80 2024-03-18T23:05:25.741 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:05:25.737+0000 7f8e60ff9700 1 -- 172.21.15.175:0/2265085991 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8e54004150 con 0x7f8e64103f80 2024-03-18T23:05:25.741 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:05:25.737+0000 7f8e6a0e8700 1 -- 172.21.15.175:0/2265085991 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e641a0350 con 0x7f8e64103f80 2024-03-18T23:05:25.741 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:05:25.737+0000 7f8e60ff9700 1 -- 172.21.15.175:0/2265085991 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e54022e30 con 0x7f8e64103f80 2024-03-18T23:05:25.741 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:05:25.737+0000 7f8e6a0e8700 1 -- 172.21.15.175:0/2265085991 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e641a0750 con 0x7f8e64103f80 2024-03-18T23:05:25.741 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:05:25.738+0000 7f8e6a0e8700 1 -- 172.21.15.175:0/2265085991 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e64068fd0 con 0x7f8e64103f80 2024-03-18T23:05:25.741 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:05:25.739+0000 7f8e60ff9700 1 -- 172.21.15.175:0/2265085991 <== mon.1 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7f8e5402a070 con 0x7f8e64103f80 2024-03-18T23:05:25.742 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:05:25.739+0000 7f8e60ff9700 1 --2- 172.21.15.175:0/2265085991 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f8e500a2890 0x7f8e500a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:05:25.742 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:05:25.739+0000 7f8e60ff9700 1 -- 172.21.15.175:0/2265085991 <== mon.1 v2:172.21.15.175:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f8e540cf260 con 0x7f8e64103f80 2024-03-18T23:05:25.742 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:05:25.740+0000 7f8e637fe700 1 --2- 172.21.15.175:0/2265085991 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f8e500a2890 0x7f8e500a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:05:25.742 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:05:25.740+0000 7f8e637fe700 1 --2- 172.21.15.175:0/2265085991 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f8e500a2890 0x7f8e500a4d50 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f8e6406a8b0 tx=0x7f8e5c007a30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:05:25.746 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:05:25.744+0000 7f8e60ff9700 1 -- 172.21.15.175:0/2265085991 <== mon.1 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f8e54013070 con 0x7f8e64103f80 2024-03-18T23:05:25.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:05:25 smithi139 ceph-mon[67282]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T23:05:25.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:05:25 smithi049 ceph-mon[62991]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T23:05:25.951 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:05:25.949+0000 7f8e6a0e8700 1 -- 172.21.15.175:0/2265085991 --> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi049=a;smithi049=d;smithi049=g;smithi049=j;smithi139=b;smithi139=e;smithi139=h;smithi139=k;smithi175=c;smithi175=f;smithi175=i;smithi175=l", "target": ["mon-mgr", ""]}) -- 0x7f8e64062850 con 0x7f8e500a2890 2024-03-18T23:05:25.951 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:05:25 smithi175 ceph-mon[67352]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T23:05:25.959 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:05:25.958+0000 7f8e60ff9700 1 -- 172.21.15.175:0/2265085991 <== mgr.14150 v2:172.21.15.49:6800/2083805211 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7f8e64062850 con 0x7f8e500a2890 2024-03-18T23:05:25.959 INFO:teuthology.orchestra.run.smithi175.stdout:Scheduled mds.all update... 2024-03-18T23:05:25.961 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:05:25.960+0000 7f8e6a0e8700 1 -- 172.21.15.175:0/2265085991 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f8e500a2890 msgr2=0x7f8e500a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:05:25.961 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:05:25.960+0000 7f8e6a0e8700 1 --2- 172.21.15.175:0/2265085991 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f8e500a2890 0x7f8e500a4d50 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f8e6406a8b0 tx=0x7f8e5c007a30 comp rx=0 tx=0).stop 2024-03-18T23:05:25.961 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:05:25.960+0000 7f8e6a0e8700 1 -- 172.21.15.175:0/2265085991 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8e64103f80 msgr2=0x7f8e6419fa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:05:25.961 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:05:25.960+0000 7f8e6a0e8700 1 --2- 172.21.15.175:0/2265085991 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8e64103f80 0x7f8e6419fa20 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f8e54005910 tx=0x7f8e540057c0 comp rx=0 tx=0).stop 2024-03-18T23:05:25.961 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:05:25.960+0000 7f8e6a0e8700 1 -- 172.21.15.175:0/2265085991 shutdown_connections 2024-03-18T23:05:25.961 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:05:25.960+0000 7f8e6a0e8700 1 --2- 172.21.15.175:0/2265085991 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f8e500a2890 0x7f8e500a4d50 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:05:25.961 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:05:25.960+0000 7f8e6a0e8700 1 --2- 172.21.15.175:0/2265085991 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8e64103f80 0x7f8e6419fa20 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:05:25.962 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:05:25.960+0000 7f8e6a0e8700 1 --2- 172.21.15.175:0/2265085991 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8e64100600 0x7f8e64198460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:05:25.962 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:05:25.960+0000 7f8e6a0e8700 1 --2- 172.21.15.175:0/2265085991 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8e641035e0 0x7f8e641989a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:05:25.962 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:05:25.960+0000 7f8e6a0e8700 1 -- 172.21.15.175:0/2265085991 >> 172.21.15.175:0/2265085991 conn(0x7f8e640fbf20 msgr2=0x7f8e64106880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:05:25.962 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:05:25.960+0000 7f8e6a0e8700 1 -- 172.21.15.175:0/2265085991 shutdown_connections 2024-03-18T23:05:25.962 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-18T23:05:25.960+0000 7f8e6a0e8700 1 -- 172.21.15.175:0/2265085991 wait complete. 2024-03-18T23:05:26.612 DEBUG:teuthology.orchestra.run.smithi049:mds.a> sudo journalctl -f -n 0 -u ceph-c63591ea-e57a-11ee-95c9-87774f69a715@mds.a.service 2024-03-18T23:05:26.618 DEBUG:teuthology.orchestra.run.smithi049:mds.d> sudo journalctl -f -n 0 -u ceph-c63591ea-e57a-11ee-95c9-87774f69a715@mds.d.service 2024-03-18T23:05:26.665 DEBUG:teuthology.orchestra.run.smithi049:mds.g> sudo journalctl -f -n 0 -u ceph-c63591ea-e57a-11ee-95c9-87774f69a715@mds.g.service 2024-03-18T23:05:26.711 DEBUG:teuthology.orchestra.run.smithi049:mds.j> sudo journalctl -f -n 0 -u ceph-c63591ea-e57a-11ee-95c9-87774f69a715@mds.j.service 2024-03-18T23:05:26.758 DEBUG:teuthology.orchestra.run.smithi139:mds.b> sudo journalctl -f -n 0 -u ceph-c63591ea-e57a-11ee-95c9-87774f69a715@mds.b.service 2024-03-18T23:05:26.766 DEBUG:teuthology.orchestra.run.smithi139:mds.e> sudo journalctl -f -n 0 -u ceph-c63591ea-e57a-11ee-95c9-87774f69a715@mds.e.service 2024-03-18T23:05:26.814 DEBUG:teuthology.orchestra.run.smithi139:mds.h> sudo journalctl -f -n 0 -u ceph-c63591ea-e57a-11ee-95c9-87774f69a715@mds.h.service 2024-03-18T23:05:26.859 DEBUG:teuthology.orchestra.run.smithi139:mds.k> sudo journalctl -f -n 0 -u ceph-c63591ea-e57a-11ee-95c9-87774f69a715@mds.k.service 2024-03-18T23:05:26.903 DEBUG:teuthology.orchestra.run.smithi175:mds.c> sudo journalctl -f -n 0 -u ceph-c63591ea-e57a-11ee-95c9-87774f69a715@mds.c.service 2024-03-18T23:05:26.907 DEBUG:teuthology.orchestra.run.smithi175:mds.f> sudo journalctl -f -n 0 -u ceph-c63591ea-e57a-11ee-95c9-87774f69a715@mds.f.service 2024-03-18T23:05:26.953 DEBUG:teuthology.orchestra.run.smithi175:mds.i> sudo journalctl -f -n 0 -u ceph-c63591ea-e57a-11ee-95c9-87774f69a715@mds.i.service 2024-03-18T23:05:26.998 DEBUG:teuthology.orchestra.run.smithi175:mds.l> sudo journalctl -f -n 0 -u ceph-c63591ea-e57a-11ee-95c9-87774f69a715@mds.l.service 2024-03-18T23:05:27.043 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-03-18T23:05:27.044 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-18T23:05:27.065 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:05:26 smithi139 ceph-mon[67282]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T23:05:27.065 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:05:26 smithi139 ceph-mon[67282]: from='client.24370 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi049=a;smithi049=d;smithi049=g;smithi049=j;smithi139=b;smithi139=e;smithi139=h;smithi139=k;smithi175=c;smithi175=f;smithi175=i;smithi175=l", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T23:05:27.065 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:05:26 smithi139 ceph-mon[67282]: Saving service mds.all spec with placement smithi049=a;smithi049=d;smithi049=g;smithi049=j;smithi139=b;smithi139=e;smithi139=h;smithi139=k;smithi175=c;smithi175=f;smithi175=i;smithi175=l;count:12 2024-03-18T23:05:27.066 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:05:26 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:05:27.066 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:05:26 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:05:27.066 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:05:26 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:05:27.066 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:05:26 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:05:27.066 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:05:26 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:05:27.066 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:05:26 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-18T23:05:27.066 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:05:26 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-18T23:05:27.066 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:05:26 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:05:27.066 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:05:26 smithi139 ceph-mon[67282]: Deploying daemon mds.l on smithi175 2024-03-18T23:05:27.168 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:27.164+0000 7f1083b3b640 1 -- 172.21.15.49:0/491771399 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f107c102a30 msgr2=0x7f107c102eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:05:27.168 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:27.164+0000 7f1083b3b640 1 --2- 172.21.15.49:0/491771399 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f107c102a30 0x7f107c102eb0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f106c009900 tx=0x7f106c02bc30 comp rx=0 tx=0).stop 2024-03-18T23:05:27.168 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:27.164+0000 7f1083b3b640 1 -- 172.21.15.49:0/491771399 shutdown_connections 2024-03-18T23:05:27.168 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:27.164+0000 7f1083b3b640 1 --2- 172.21.15.49:0/491771399 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f107c1033f0 0x7f107c109c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:05:27.168 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:27.164+0000 7f1083b3b640 1 --2- 172.21.15.49:0/491771399 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f107c102a30 0x7f107c102eb0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:05:27.168 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:27.164+0000 7f1083b3b640 1 --2- 172.21.15.49:0/491771399 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f107c101840 0x7f107c101c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:05:27.168 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:27.164+0000 7f1083b3b640 1 -- 172.21.15.49:0/491771399 >> 172.21.15.49:0/491771399 conn(0x7f107c0fcdd0 msgr2=0x7f107c0ff1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:05:27.168 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:27.164+0000 7f1083b3b640 1 -- 172.21.15.49:0/491771399 shutdown_connections 2024-03-18T23:05:27.168 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:27.164+0000 7f1083b3b640 1 -- 172.21.15.49:0/491771399 wait complete. 2024-03-18T23:05:27.169 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:27.164+0000 7f1083b3b640 1 Processor -- start 2024-03-18T23:05:27.169 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:27.164+0000 7f1083b3b640 1 -- start start 2024-03-18T23:05:27.169 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:27.164+0000 7f1083b3b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f107c101840 0x7f107c197b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:05:27.169 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:27.164+0000 7f1083b3b640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f107c102a30 0x7f107c198080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:05:27.169 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:27.165+0000 7f1083b3b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f107c1033f0 0x7f107c19f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:05:27.169 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:27.165+0000 7f1083b3b640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f107c10d4c0 con 0x7f107c101840 2024-03-18T23:05:27.169 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:27.165+0000 7f1083b3b640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f107c10d340 con 0x7f107c1033f0 2024-03-18T23:05:27.170 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:27.165+0000 7f1083b3b640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f107c10d640 con 0x7f107c102a30 2024-03-18T23:05:27.170 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:27.165+0000 7f10818d7640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f107c101840 0x7f107c197b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:05:27.170 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:27.165+0000 7f10818d7640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f107c101840 0x7f107c197b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:55480/0 (socket says 172.21.15.49:55480) 2024-03-18T23:05:27.170 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:27.165+0000 7f10818d7640 1 -- 172.21.15.49:0/3933425593 learned_addr learned my addr 172.21.15.49:0/3933425593 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:05:27.170 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:27.165+0000 7f10810d6640 1 --2- 172.21.15.49:0/3933425593 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f107c102a30 0x7f107c198080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:05:27.170 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:27.165+0000 7f10820d8640 1 --2- 172.21.15.49:0/3933425593 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f107c1033f0 0x7f107c19f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:05:27.170 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:27.165+0000 7f10818d7640 1 -- 172.21.15.49:0/3933425593 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f107c102a30 msgr2=0x7f107c198080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:05:27.170 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:27.165+0000 7f10818d7640 1 --2- 172.21.15.49:0/3933425593 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f107c102a30 0x7f107c198080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:05:27.170 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:27.165+0000 7f10818d7640 1 -- 172.21.15.49:0/3933425593 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f107c1033f0 msgr2=0x7f107c19f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:05:27.170 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:27.165+0000 7f10818d7640 1 --2- 172.21.15.49:0/3933425593 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f107c1033f0 0x7f107c19f0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:05:27.170 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:27.165+0000 7f10818d7640 1 -- 172.21.15.49:0/3933425593 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f107c19f760 con 0x7f107c101840 2024-03-18T23:05:27.170 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:27.165+0000 7f10818d7640 1 --2- 172.21.15.49:0/3933425593 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f107c101840 0x7f107c197b40 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f107800e9c0 tx=0x7f107800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:05:27.171 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:27.165+0000 7f1072ffd640 1 -- 172.21.15.49:0/3933425593 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f107800cd80 con 0x7f107c101840 2024-03-18T23:05:27.171 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:27.165+0000 7f1072ffd640 1 -- 172.21.15.49:0/3933425593 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f10780035f0 con 0x7f107c101840 2024-03-18T23:05:27.171 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:27.165+0000 7f1083b3b640 1 -- 172.21.15.49:0/3933425593 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f107c19fa50 con 0x7f107c101840 2024-03-18T23:05:27.171 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:27.165+0000 7f1072ffd640 1 -- 172.21.15.49:0/3933425593 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1078010690 con 0x7f107c101840 2024-03-18T23:05:27.171 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:27.165+0000 7f1083b3b640 1 -- 172.21.15.49:0/3933425593 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f107c19ff70 con 0x7f107c101840 2024-03-18T23:05:27.171 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:27.166+0000 7f1083b3b640 1 -- 172.21.15.49:0/3933425593 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f107c068a60 con 0x7f107c101840 2024-03-18T23:05:27.175 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:27.167+0000 7f1072ffd640 1 -- 172.21.15.49:0/3933425593 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7f10780037a0 con 0x7f107c101840 2024-03-18T23:05:27.175 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:27.168+0000 7f1072ffd640 1 --2- 172.21.15.49:0/3933425593 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f10680a2800 0x7f10680a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:05:27.175 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:27.168+0000 7f10810d6640 1 --2- 172.21.15.49:0/3933425593 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f10680a2800 0x7f10680a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:05:27.176 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:27.168+0000 7f1072ffd640 1 -- 172.21.15.49:0/3933425593 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f10780ce970 con 0x7f107c101840 2024-03-18T23:05:27.176 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:27.169+0000 7f10810d6640 1 --2- 172.21.15.49:0/3933425593 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f10680a2800 0x7f10680a4cc0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f106c02c7b0 tx=0x7f106c005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:05:27.176 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:27.173+0000 7f1072ffd640 1 -- 172.21.15.49:0/3933425593 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f107809beb0 con 0x7f107c101840 2024-03-18T23:05:27.212 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:05:26 smithi175 ceph-mon[67352]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T23:05:27.212 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:05:26 smithi175 ceph-mon[67352]: from='client.24370 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi049=a;smithi049=d;smithi049=g;smithi049=j;smithi139=b;smithi139=e;smithi139=h;smithi139=k;smithi175=c;smithi175=f;smithi175=i;smithi175=l", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T23:05:27.212 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:05:26 smithi175 ceph-mon[67352]: Saving service mds.all spec with placement smithi049=a;smithi049=d;smithi049=g;smithi049=j;smithi139=b;smithi139=e;smithi139=h;smithi139=k;smithi175=c;smithi175=f;smithi175=i;smithi175=l;count:12 2024-03-18T23:05:27.212 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:05:26 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:05:27.212 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:05:26 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:05:27.213 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:05:26 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:05:27.213 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:05:26 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:05:27.213 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:05:26 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:05:27.213 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:05:26 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-18T23:05:27.213 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:05:26 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-18T23:05:27.213 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:05:26 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:05:27.213 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:05:26 smithi175 ceph-mon[67352]: Deploying daemon mds.l on smithi175 2024-03-18T23:05:27.363 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:27.362+0000 7f1083b3b640 1 -- 172.21.15.49:0/3933425593 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f107c108b60 con 0x7f107c101840 2024-03-18T23:05:27.364 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:27.363+0000 7f1072ffd640 1 -- 172.21.15.49:0/3933425593 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+18479 (secure 0 0 0) 0x7f107809b7b0 con 0x7f107c101840 2024-03-18T23:05:27.364 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-18T23:05:27.364 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":82,"fsid":"c63591ea-e57a-11ee-95c9-87774f69a715","created":"2024-03-18T22:57:58.734467+0000","modified":"2024-03-18T23:05:17.581588+0000","last_up_change":"2024-03-18T23:05:16.576310+0000","last_in_change":"2024-03-18T23:04:57.024067+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-18T23:01:28.363432+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e7fc9daa-4098-4ba5-8019-9aaef1aab3e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":1651162501},{"type":"v1","addr":"172.21.15.49:6803","nonce":1651162501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":1651162501},{"type":"v1","addr":"172.21.15.49:6805","nonce":1651162501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":1651162501},{"type":"v1","addr":"172.21.15.49:6809","nonce":1651162501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":1651162501},{"type":"v1","addr":"172.21.15.49:6807","nonce":1651162501}]},"public_addr":"172.21.15.49:6803/1651162501","cluster_addr":"172.21.15.49:6805/1651162501","heartbeat_back_addr":"172.21.15.49:6809/1651162501","heartbeat_front_addr":"172.21.15.49:6807/1651162501","state":["exists","up"]},{"osd":1,"uuid":"40ac929c-e90f-434c-a796-9aabd686f1fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":1766700359},{"type":"v1","addr":"172.21.15.139:6801","nonce":1766700359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":1766700359},{"type":"v1","addr":"172.21.15.139:6803","nonce":1766700359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":1766700359},{"type":"v1","addr":"172.21.15.139:6807","nonce":1766700359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":1766700359},{"type":"v1","addr":"172.21.15.139:6805","nonce":1766700359}]},"public_addr":"172.21.15.139:6801/1766700359","cluster_addr":"172.21.15.139:6803/1766700359","heartbeat_back_addr":"172.21.15.139:6807/1766700359","heartbeat_front_addr":"172.21.15.139:6805/1766700359","state":["exists","up"]},{"osd":2,"uuid":"8ae3b666-00b4-45ed-a5ee-4d0771edabb6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":3752671234},{"type":"v1","addr":"172.21.15.175:6801","nonce":3752671234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":3752671234},{"type":"v1","addr":"172.21.15.175:6803","nonce":3752671234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":3752671234},{"type":"v1","addr":"172.21.15.175:6807","nonce":3752671234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":3752671234},{"type":"v1","addr":"172.21.15.175:6805","nonce":3752671234}]},"public_addr":"172.21.15.175:6801/3752671234","cluster_addr":"172.21.15.175:6803/3752671234","heartbeat_back_addr":"172.21.15.175:6807/3752671234","heartbeat_front_addr":"172.21.15.175:6805/3752671234","state":["exists","up"]},{"osd":3,"uuid":"1a62e89d-45d4-4237-b813-7f96a59aca7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":956160536},{"type":"v1","addr":"172.21.15.49:6811","nonce":956160536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":956160536},{"type":"v1","addr":"172.21.15.49:6813","nonce":956160536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":956160536},{"type":"v1","addr":"172.21.15.49:6817","nonce":956160536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":956160536},{"type":"v1","addr":"172.21.15.49:6815","nonce":956160536}]},"public_addr":"172.21.15.49:6811/956160536","cluster_addr":"172.21.15.49:6813/956160536","heartbeat_back_addr":"172.21.15.49:6817/956160536","heartbeat_front_addr":"172.21.15.49:6815/956160536","state":["exists","up"]},{"osd":4,"uuid":"c715101c-0c3f-4b98-b142-d64a35e187df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":3129712668},{"type":"v1","addr":"172.21.15.139:6809","nonce":3129712668}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":3129712668},{"type":"v1","addr":"172.21.15.139:6811","nonce":3129712668}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":3129712668},{"type":"v1","addr":"172.21.15.139:6815","nonce":3129712668}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":3129712668},{"type":"v1","addr":"172.21.15.139:6813","nonce":3129712668}]},"public_addr":"172.21.15.139:6809/3129712668","cluster_addr":"172.21.15.139:6811/3129712668","heartbeat_back_addr":"172.21.15.139:6815/3129712668","heartbeat_front_addr":"172.21.15.139:6813/3129712668","state":["exists","up"]},{"osd":5,"uuid":"4b641f81-77eb-45fd-8464-511e7b3062b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":2774567215},{"type":"v1","addr":"172.21.15.175:6809","nonce":2774567215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":2774567215},{"type":"v1","addr":"172.21.15.175:6811","nonce":2774567215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":2774567215},{"type":"v1","addr":"172.21.15.175:6815","nonce":2774567215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":2774567215},{"type":"v1","addr":"172.21.15.175:6813","nonce":2774567215}]},"public_addr":"172.21.15.175:6809/2774567215","cluster_addr":"172.21.15.175:6811/2774567215","heartbeat_back_addr":"172.21.15.175:6815/2774567215","heartbeat_front_addr":"172.21.15.175:6813/2774567215","state":["exists","up"]},{"osd":6,"uuid":"653da8e0-1197-4b48-abed-470f2a9c4dce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":1472215461},{"type":"v1","addr":"172.21.15.49:6819","nonce":1472215461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":1472215461},{"type":"v1","addr":"172.21.15.49:6821","nonce":1472215461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":1472215461},{"type":"v1","addr":"172.21.15.49:6825","nonce":1472215461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":1472215461},{"type":"v1","addr":"172.21.15.49:6823","nonce":1472215461}]},"public_addr":"172.21.15.49:6819/1472215461","cluster_addr":"172.21.15.49:6821/1472215461","heartbeat_back_addr":"172.21.15.49:6825/1472215461","heartbeat_front_addr":"172.21.15.49:6823/1472215461","state":["exists","up"]},{"osd":7,"uuid":"ad157c9b-4b75-451c-8740-630f0975b7fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":2163986787},{"type":"v1","addr":"172.21.15.139:6817","nonce":2163986787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":2163986787},{"type":"v1","addr":"172.21.15.139:6819","nonce":2163986787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":2163986787},{"type":"v1","addr":"172.21.15.139:6823","nonce":2163986787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":2163986787},{"type":"v1","addr":"172.21.15.139:6821","nonce":2163986787}]},"public_addr":"172.21.15.139:6817/2163986787","cluster_addr":"172.21.15.139:6819/2163986787","heartbeat_back_addr":"172.21.15.139:6823/2163986787","heartbeat_front_addr":"172.21.15.139:6821/2163986787","state":["exists","up"]},{"osd":8,"uuid":"c8e75f15-df75-4d5d-9b92-a064bffb6c01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":1194026992},{"type":"v1","addr":"172.21.15.175:6817","nonce":1194026992}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1194026992},{"type":"v1","addr":"172.21.15.175:6819","nonce":1194026992}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":1194026992},{"type":"v1","addr":"172.21.15.175:6823","nonce":1194026992}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1194026992},{"type":"v1","addr":"172.21.15.175:6821","nonce":1194026992}]},"public_addr":"172.21.15.175:6817/1194026992","cluster_addr":"172.21.15.175:6819/1194026992","heartbeat_back_addr":"172.21.15.175:6823/1194026992","heartbeat_front_addr":"172.21.15.175:6821/1194026992","state":["exists","up"]},{"osd":9,"uuid":"edb5ffef-b6a2-4941-b5ab-40f982ad30d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":2742084884},{"type":"v1","addr":"172.21.15.49:6827","nonce":2742084884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":2742084884},{"type":"v1","addr":"172.21.15.49:6829","nonce":2742084884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":2742084884},{"type":"v1","addr":"172.21.15.49:6833","nonce":2742084884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":2742084884},{"type":"v1","addr":"172.21.15.49:6831","nonce":2742084884}]},"public_addr":"172.21.15.49:6827/2742084884","cluster_addr":"172.21.15.49:6829/2742084884","heartbeat_back_addr":"172.21.15.49:6833/2742084884","heartbeat_front_addr":"172.21.15.49:6831/2742084884","state":["exists","up"]},{"osd":10,"uuid":"3945e46d-766a-4c4a-8821-d43648dd1578","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":2404740201},{"type":"v1","addr":"172.21.15.139:6825","nonce":2404740201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":2404740201},{"type":"v1","addr":"172.21.15.139:6827","nonce":2404740201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":2404740201},{"type":"v1","addr":"172.21.15.139:6831","nonce":2404740201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":2404740201},{"type":"v1","addr":"172.21.15.139:6829","nonce":2404740201}]},"public_addr":"172.21.15.139:6825/2404740201","cluster_addr":"172.21.15.139:6827/2404740201","heartbeat_back_addr":"172.21.15.139:6831/2404740201","heartbeat_front_addr":"172.21.15.139:6829/2404740201","state":["exists","up"]},{"osd":11,"uuid":"9954ba15-d877-46cc-b24a-893f9168deb8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":3519503868},{"type":"v1","addr":"172.21.15.175:6825","nonce":3519503868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":3519503868},{"type":"v1","addr":"172.21.15.175:6827","nonce":3519503868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":3519503868},{"type":"v1","addr":"172.21.15.175:6831","nonce":3519503868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":3519503868},{"type":"v1","addr":"172.21.15.175:6829","nonce":3519503868}]},"public_addr":"172.21.15.175:6825/3519503868","cluster_addr":"172.21.15.175:6827/3519503868","heartbeat_back_addr":"172.21.15.175:6831/3519503868","heartbeat_front_addr":"172.21.15.175:6829/3519503868","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:00:38.367865+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:01:02.237687+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:01:25.609751+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:01:51.268572+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:02:16.514605+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:02:41.586523+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:03:07.029964+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:03:32.956250+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:03:57.547984+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:04:23.948972+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:04:49.667210+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:05:15.284693+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.49:0/563527416":"2024-03-19T22:58:59.806253+0000","172.21.15.49:0/1036383534":"2024-03-19T22:58:59.806253+0000","172.21.15.49:6801/2609458951":"2024-03-19T22:58:59.806253+0000","172.21.15.49:0/21283102":"2024-03-19T22:58:31.972474+0000","172.21.15.49:0/1292058922":"2024-03-19T22:58:31.972474+0000","172.21.15.49:6800/1985190312":"2024-03-19T22:58:31.972474+0000","172.21.15.49:0/1912446700":"2024-03-19T22:58:59.806253+0000","172.21.15.49:0/2176210039":"2024-03-19T22:58:31.972474+0000","172.21.15.49:6801/1985190312":"2024-03-19T22:58:31.972474+0000","172.21.15.49:6800/2609458951":"2024-03-19T22:58:59.806253+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-18T23:05:27.366 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:27.365+0000 7f1083b3b640 1 -- 172.21.15.49:0/3933425593 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f10680a2800 msgr2=0x7f10680a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:05:27.366 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:27.365+0000 7f1083b3b640 1 --2- 172.21.15.49:0/3933425593 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f10680a2800 0x7f10680a4cc0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f106c02c7b0 tx=0x7f106c005fd0 comp rx=0 tx=0).stop 2024-03-18T23:05:27.366 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:27.365+0000 7f1083b3b640 1 -- 172.21.15.49:0/3933425593 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f107c101840 msgr2=0x7f107c197b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:05:27.366 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:27.365+0000 7f1083b3b640 1 --2- 172.21.15.49:0/3933425593 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f107c101840 0x7f107c197b40 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f107800e9c0 tx=0x7f107800ee80 comp rx=0 tx=0).stop 2024-03-18T23:05:27.367 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:27.365+0000 7f1083b3b640 1 -- 172.21.15.49:0/3933425593 shutdown_connections 2024-03-18T23:05:27.367 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:27.365+0000 7f1083b3b640 1 --2- 172.21.15.49:0/3933425593 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f10680a2800 0x7f10680a4cc0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:05:27.367 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:27.365+0000 7f1083b3b640 1 --2- 172.21.15.49:0/3933425593 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f107c1033f0 0x7f107c19f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:05:27.367 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:27.365+0000 7f1083b3b640 1 --2- 172.21.15.49:0/3933425593 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f107c102a30 0x7f107c198080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:05:27.367 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:27.365+0000 7f1083b3b640 1 --2- 172.21.15.49:0/3933425593 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f107c101840 0x7f107c197b40 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:05:27.367 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:27.365+0000 7f1083b3b640 1 -- 172.21.15.49:0/3933425593 >> 172.21.15.49:0/3933425593 conn(0x7f107c0fcdd0 msgr2=0x7f107c105d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:05:27.367 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:27.365+0000 7f1083b3b640 1 -- 172.21.15.49:0/3933425593 shutdown_connections 2024-03-18T23:05:27.367 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:27.365+0000 7f1083b3b640 1 -- 172.21.15.49:0/3933425593 wait complete. 2024-03-18T23:05:27.381 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:05:26 smithi049 ceph-mon[62991]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T23:05:27.381 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:05:26 smithi049 ceph-mon[62991]: from='client.24370 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi049=a;smithi049=d;smithi049=g;smithi049=j;smithi139=b;smithi139=e;smithi139=h;smithi139=k;smithi175=c;smithi175=f;smithi175=i;smithi175=l", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T23:05:27.381 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:05:26 smithi049 ceph-mon[62991]: Saving service mds.all spec with placement smithi049=a;smithi049=d;smithi049=g;smithi049=j;smithi139=b;smithi139=e;smithi139=h;smithi139=k;smithi175=c;smithi175=f;smithi175=i;smithi175=l;count:12 2024-03-18T23:05:27.381 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:05:26 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:05:27.381 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:05:26 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:05:27.381 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:05:26 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:05:27.381 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:05:26 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:05:27.381 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:05:26 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:05:27.381 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:05:26 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-18T23:05:27.382 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:05:26 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-18T23:05:27.382 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:05:26 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:05:27.382 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:05:26 smithi049 ceph-mon[62991]: Deploying daemon mds.l on smithi175 2024-03-18T23:05:27.383 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-18T23:01:28.363432+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-18T23:05:27.383 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-18T23:05:27.497 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:27.494+0000 7f98a9997640 1 -- 172.21.15.49:0/3116657015 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f98a41022d0 msgr2=0x7f98a4108b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:05:27.497 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:27.494+0000 7f98a9997640 1 --2- 172.21.15.49:0/3116657015 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f98a41022d0 0x7f98a4108b60 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f989c009ca0 tx=0x7f989c02c0f0 comp rx=0 tx=0).stop 2024-03-18T23:05:27.497 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:27.494+0000 7f98a9997640 1 -- 172.21.15.49:0/3116657015 shutdown_connections 2024-03-18T23:05:27.497 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:27.494+0000 7f98a9997640 1 --2- 172.21.15.49:0/3116657015 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f98a41022d0 0x7f98a4108b60 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:05:27.497 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:27.494+0000 7f98a9997640 1 --2- 172.21.15.49:0/3116657015 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f98a4101910 0x7f98a4101d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:05:27.497 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:27.494+0000 7f98a9997640 1 --2- 172.21.15.49:0/3116657015 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f98a4100720 0x7f98a4100b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:05:27.497 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:27.494+0000 7f98a9997640 1 -- 172.21.15.49:0/3116657015 >> 172.21.15.49:0/3116657015 conn(0x7f98a40fbcb0 msgr2=0x7f98a40fe0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:05:27.497 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:27.494+0000 7f98a9997640 1 -- 172.21.15.49:0/3116657015 shutdown_connections 2024-03-18T23:05:27.497 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:27.494+0000 7f98a9997640 1 -- 172.21.15.49:0/3116657015 wait complete. 2024-03-18T23:05:27.497 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:27.494+0000 7f98a9997640 1 Processor -- start 2024-03-18T23:05:27.497 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:27.494+0000 7f98a9997640 1 -- start start 2024-03-18T23:05:27.497 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:27.495+0000 7f98a9997640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f98a4100720 0x7f98a41046b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:05:27.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:27.495+0000 7f98a9997640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f98a4101910 0x7f98a4102cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:05:27.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:27.495+0000 7f98a9997640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f98a41022d0 0x7f98a4103230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:05:27.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:27.495+0000 7f98a9997640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f98a410c380 con 0x7f98a4101910 2024-03-18T23:05:27.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:27.495+0000 7f98a9997640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f98a410c200 con 0x7f98a4100720 2024-03-18T23:05:27.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:27.495+0000 7f98a9997640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f98a410c500 con 0x7f98a41022d0 2024-03-18T23:05:27.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:27.495+0000 7f98a27fc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f98a4101910 0x7f98a4102cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:05:27.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:27.495+0000 7f98a27fc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f98a4101910 0x7f98a4102cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:55494/0 (socket says 172.21.15.49:55494) 2024-03-18T23:05:27.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:27.495+0000 7f98a27fc640 1 -- 172.21.15.49:0/824776155 learned_addr learned my addr 172.21.15.49:0/824776155 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:05:27.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:27.495+0000 7f98a37fe640 1 --2- 172.21.15.49:0/824776155 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f98a41022d0 0x7f98a4103230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:05:27.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:27.495+0000 7f98a2ffd640 1 --2- 172.21.15.49:0/824776155 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f98a4100720 0x7f98a41046b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:05:27.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:27.495+0000 7f98a27fc640 1 -- 172.21.15.49:0/824776155 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f98a41022d0 msgr2=0x7f98a4103230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:05:27.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:27.495+0000 7f98a27fc640 1 --2- 172.21.15.49:0/824776155 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f98a41022d0 0x7f98a4103230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:05:27.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:27.495+0000 7f98a27fc640 1 -- 172.21.15.49:0/824776155 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f98a4100720 msgr2=0x7f98a41046b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:05:27.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:27.495+0000 7f98a27fc640 1 --2- 172.21.15.49:0/824776155 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f98a4100720 0x7f98a41046b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:05:27.499 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:27.495+0000 7f98a27fc640 1 -- 172.21.15.49:0/824776155 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98a4103ac0 con 0x7f98a4101910 2024-03-18T23:05:27.499 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:27.495+0000 7f98a27fc640 1 --2- 172.21.15.49:0/824776155 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f98a4101910 0x7f98a4102cf0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f989400e9f0 tx=0x7f989400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:05:27.499 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:27.495+0000 7f98a8995640 1 -- 172.21.15.49:0/824776155 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f989400cd80 con 0x7f98a4101910 2024-03-18T23:05:27.499 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:27.495+0000 7f98a9997640 1 -- 172.21.15.49:0/824776155 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98a41a1c30 con 0x7f98a4101910 2024-03-18T23:05:27.499 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:27.495+0000 7f98a8995640 1 -- 172.21.15.49:0/824776155 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9894010910 con 0x7f98a4101910 2024-03-18T23:05:27.499 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:27.495+0000 7f98a8995640 1 -- 172.21.15.49:0/824776155 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9894018950 con 0x7f98a4101910 2024-03-18T23:05:27.499 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:27.495+0000 7f98a9997640 1 -- 172.21.15.49:0/824776155 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98a41a2180 con 0x7f98a4101910 2024-03-18T23:05:27.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:27.496+0000 7f98a9997640 1 -- 172.21.15.49:0/824776155 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98a41a2350 con 0x7f98a4101910 2024-03-18T23:05:27.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:27.497+0000 7f98a8995640 1 -- 172.21.15.49:0/824776155 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7f9894010ac0 con 0x7f98a4101910 2024-03-18T23:05:27.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:27.498+0000 7f98a8995640 1 --2- 172.21.15.49:0/824776155 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f98900a2800 0x7f98900a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:05:27.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:27.498+0000 7f98a8995640 1 -- 172.21.15.49:0/824776155 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f98940cea10 con 0x7f98a4101910 2024-03-18T23:05:27.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:27.498+0000 7f98a2ffd640 1 --2- 172.21.15.49:0/824776155 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f98900a2800 0x7f98900a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:05:27.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:27.499+0000 7f98a2ffd640 1 --2- 172.21.15.49:0/824776155 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f98900a2800 0x7f98900a4cc0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f988c005fd0 tx=0x7f988c005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:05:27.503 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:27.503+0000 7f98a8995640 1 -- 172.21.15.49:0/824776155 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f989409bf50 con 0x7f98a4101910 2024-03-18T23:05:27.687 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:27.686+0000 7f98a9997640 1 -- 172.21.15.49:0/824776155 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f98a41a26e0 con 0x7f98a4101910 2024-03-18T23:05:27.687 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:27.686+0000 7f98a8995640 1 -- 172.21.15.49:0/824776155 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) ==== 93+0+10 (secure 0 0 0) 0x7f989409b850 con 0x7f98a4101910 2024-03-18T23:05:27.687 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 1 2024-03-18T23:05:27.690 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:27.689+0000 7f98a9997640 1 -- 172.21.15.49:0/824776155 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f98900a2800 msgr2=0x7f98900a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:05:27.690 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:27.689+0000 7f98a9997640 1 --2- 172.21.15.49:0/824776155 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f98900a2800 0x7f98900a4cc0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f988c005fd0 tx=0x7f988c005850 comp rx=0 tx=0).stop 2024-03-18T23:05:27.690 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:27.689+0000 7f98a9997640 1 -- 172.21.15.49:0/824776155 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f98a4101910 msgr2=0x7f98a4102cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:05:27.690 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:27.689+0000 7f98a9997640 1 --2- 172.21.15.49:0/824776155 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f98a4101910 0x7f98a4102cf0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f989400e9f0 tx=0x7f989400eeb0 comp rx=0 tx=0).stop 2024-03-18T23:05:27.690 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:27.689+0000 7f98a9997640 1 -- 172.21.15.49:0/824776155 shutdown_connections 2024-03-18T23:05:27.690 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:27.689+0000 7f98a9997640 1 --2- 172.21.15.49:0/824776155 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f98900a2800 0x7f98900a4cc0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:05:27.690 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:27.689+0000 7f98a9997640 1 --2- 172.21.15.49:0/824776155 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f98a41022d0 0x7f98a4103230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:05:27.690 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:27.689+0000 7f98a9997640 1 --2- 172.21.15.49:0/824776155 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f98a4101910 0x7f98a4102cf0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:05:27.690 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:27.689+0000 7f98a9997640 1 --2- 172.21.15.49:0/824776155 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f98a4100720 0x7f98a41046b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:05:27.690 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:27.689+0000 7f98a9997640 1 -- 172.21.15.49:0/824776155 >> 172.21.15.49:0/824776155 conn(0x7f98a40fbcb0 msgr2=0x7f98a40fd720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:05:27.690 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:27.689+0000 7f98a9997640 1 -- 172.21.15.49:0/824776155 shutdown_connections 2024-03-18T23:05:27.691 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:27.689+0000 7f98a9997640 1 -- 172.21.15.49:0/824776155 wait complete. 2024-03-18T23:05:27.700 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-18T23:05:27.809 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:27.808+0000 7fefa0be6640 1 -- 172.21.15.49:0/1047647141 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fef9c101430 msgr2=0x7fef9c1018b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:05:27.809 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:27.808+0000 7fefa0be6640 1 --2- 172.21.15.49:0/1047647141 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fef9c101430 0x7fef9c1018b0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fef94009ca0 tx=0x7fef9402c0f0 comp rx=0 tx=0).stop 2024-03-18T23:05:27.810 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:27.808+0000 7fefa0be6640 1 -- 172.21.15.49:0/1047647141 shutdown_connections 2024-03-18T23:05:27.810 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:27.808+0000 7fefa0be6640 1 --2- 172.21.15.49:0/1047647141 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fef9c101430 0x7fef9c1018b0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:05:27.810 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:27.808+0000 7fefa0be6640 1 --2- 172.21.15.49:0/1047647141 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fef9c100a90 0x7fef9c100ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:05:27.810 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:27.808+0000 7fefa0be6640 1 --2- 172.21.15.49:0/1047647141 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fef9c100130 0x7fef9c100550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:05:27.810 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:27.808+0000 7fefa0be6640 1 -- 172.21.15.49:0/1047647141 >> 172.21.15.49:0/1047647141 conn(0x7fef9c0fbd50 msgr2=0x7fef9c0fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:05:27.810 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:27.808+0000 7fefa0be6640 1 -- 172.21.15.49:0/1047647141 shutdown_connections 2024-03-18T23:05:27.810 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:27.808+0000 7fefa0be6640 1 -- 172.21.15.49:0/1047647141 wait complete. 2024-03-18T23:05:27.810 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:27.809+0000 7fefa0be6640 1 Processor -- start 2024-03-18T23:05:27.810 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:27.809+0000 7fefa0be6640 1 -- start start 2024-03-18T23:05:27.812 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:27.809+0000 7fefa0be6640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fef9c100130 0x7fef9c199ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:05:27.812 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:27.809+0000 7fefa0be6640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fef9c100a90 0x7fef9c19a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:05:27.812 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:27.809+0000 7fefa0be6640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fef9c101430 0x7fef9c1a11c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:05:27.812 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:27.809+0000 7fefa0be6640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fef9c10f6a0 con 0x7fef9c100a90 2024-03-18T23:05:27.812 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:27.809+0000 7fefa0be6640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fef9c10f520 con 0x7fef9c100130 2024-03-18T23:05:27.813 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:27.809+0000 7fefa0be6640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fef9c10f820 con 0x7fef9c101430 2024-03-18T23:05:27.813 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:27.809+0000 7fef91d9b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fef9c100a90 0x7fef9c19a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:05:27.813 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:27.809+0000 7fef91d9b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fef9c100a90 0x7fef9c19a1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:55526/0 (socket says 172.21.15.49:55526) 2024-03-18T23:05:27.813 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:27.809+0000 7fef91d9b640 1 -- 172.21.15.49:0/548025897 learned_addr learned my addr 172.21.15.49:0/548025897 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:05:27.813 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:27.809+0000 7fef9ad9d640 1 --2- 172.21.15.49:0/548025897 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fef9c101430 0x7fef9c1a11c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:05:27.813 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:27.809+0000 7fef9a59c640 1 --2- 172.21.15.49:0/548025897 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fef9c100130 0x7fef9c199ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:05:27.813 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:27.809+0000 7fef91d9b640 1 -- 172.21.15.49:0/548025897 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fef9c101430 msgr2=0x7fef9c1a11c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:05:27.813 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:27.809+0000 7fef91d9b640 1 --2- 172.21.15.49:0/548025897 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fef9c101430 0x7fef9c1a11c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:05:27.813 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:27.809+0000 7fef91d9b640 1 -- 172.21.15.49:0/548025897 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fef9c100130 msgr2=0x7fef9c199ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:05:27.813 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:27.809+0000 7fef91d9b640 1 --2- 172.21.15.49:0/548025897 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fef9c100130 0x7fef9c199ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:05:27.813 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:27.809+0000 7fef91d9b640 1 -- 172.21.15.49:0/548025897 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef9c1a1860 con 0x7fef9c100a90 2024-03-18T23:05:27.813 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:27.809+0000 7fef9ad9d640 1 --2- 172.21.15.49:0/548025897 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fef9c101430 0x7fef9c1a11c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T23:05:27.813 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:27.810+0000 7fef91d9b640 1 --2- 172.21.15.49:0/548025897 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fef9c100a90 0x7fef9c19a1e0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fef8400d950 tx=0x7fef8400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:05:27.813 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:27.810+0000 7fef93fff640 1 -- 172.21.15.49:0/548025897 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef84017070 con 0x7fef9c100a90 2024-03-18T23:05:27.814 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:27.810+0000 7fef93fff640 1 -- 172.21.15.49:0/548025897 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fef84009c30 con 0x7fef9c100a90 2024-03-18T23:05:27.814 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:27.810+0000 7fefa0be6640 1 -- 172.21.15.49:0/548025897 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef9c1a1b50 con 0x7fef9c100a90 2024-03-18T23:05:27.814 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:27.810+0000 7fef93fff640 1 -- 172.21.15.49:0/548025897 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef8400f6b0 con 0x7fef9c100a90 2024-03-18T23:05:27.814 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:27.810+0000 7fefa0be6640 1 -- 172.21.15.49:0/548025897 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef9c1a20a0 con 0x7fef9c100a90 2024-03-18T23:05:27.814 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:27.811+0000 7fefa0be6640 1 -- 172.21.15.49:0/548025897 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef9c10ae80 con 0x7fef9c100a90 2024-03-18T23:05:27.815 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:27.812+0000 7fef93fff640 1 -- 172.21.15.49:0/548025897 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7fef840109d0 con 0x7fef9c100a90 2024-03-18T23:05:27.815 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:27.812+0000 7fef93fff640 1 --2- 172.21.15.49:0/548025897 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fef880a2800 0x7fef880a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:05:27.815 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:27.812+0000 7fef9a59c640 1 --2- 172.21.15.49:0/548025897 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fef880a2800 0x7fef880a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:05:27.815 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:27.813+0000 7fef93fff640 1 -- 172.21.15.49:0/548025897 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fef840ce3b0 con 0x7fef9c100a90 2024-03-18T23:05:27.815 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:27.813+0000 7fef9a59c640 1 --2- 172.21.15.49:0/548025897 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fef880a2800 0x7fef880a4cc0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fef8c005fd0 tx=0x7fef8c005ce0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:05:27.817 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:27.817+0000 7fef93fff640 1 -- 172.21.15.49:0/548025897 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fef84009de0 con 0x7fef9c100a90 2024-03-18T23:05:28.044 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:28.043+0000 7fefa0be6640 1 -- 172.21.15.49:0/548025897 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fef9c02d430 con 0x7fef9c100a90 2024-03-18T23:05:28.045 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:28.044+0000 7fef93fff640 1 -- 172.21.15.49:0/548025897 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) ==== 93+0+569 (secure 0 0 0) 0x7fef8409b8f0 con 0x7fef9c100a90 2024-03-18T23:05:28.045 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-18T23:05:28.045 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-03-18T23:05:28.045 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 1 2024-03-18T23:05:28.047 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:28.046+0000 7fefa0be6640 1 -- 172.21.15.49:0/548025897 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fef880a2800 msgr2=0x7fef880a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:05:28.047 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:28.046+0000 7fefa0be6640 1 --2- 172.21.15.49:0/548025897 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fef880a2800 0x7fef880a4cc0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fef8c005fd0 tx=0x7fef8c005ce0 comp rx=0 tx=0).stop 2024-03-18T23:05:28.047 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:28.046+0000 7fefa0be6640 1 -- 172.21.15.49:0/548025897 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fef9c100a90 msgr2=0x7fef9c19a1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:05:28.047 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:28.046+0000 7fefa0be6640 1 --2- 172.21.15.49:0/548025897 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fef9c100a90 0x7fef9c19a1e0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fef8400d950 tx=0x7fef8400de10 comp rx=0 tx=0).stop 2024-03-18T23:05:28.047 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:28.046+0000 7fefa0be6640 1 -- 172.21.15.49:0/548025897 shutdown_connections 2024-03-18T23:05:28.047 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:28.046+0000 7fefa0be6640 1 --2- 172.21.15.49:0/548025897 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fef880a2800 0x7fef880a4cc0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:05:28.047 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:28.046+0000 7fefa0be6640 1 --2- 172.21.15.49:0/548025897 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fef9c101430 0x7fef9c1a11c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:05:28.047 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:28.046+0000 7fefa0be6640 1 --2- 172.21.15.49:0/548025897 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fef9c100a90 0x7fef9c19a1e0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:05:28.048 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:28.046+0000 7fefa0be6640 1 --2- 172.21.15.49:0/548025897 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fef9c100130 0x7fef9c199ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:05:28.048 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:28.046+0000 7fefa0be6640 1 -- 172.21.15.49:0/548025897 >> 172.21.15.49:0/548025897 conn(0x7fef9c0fbd50 msgr2=0x7fef9c0fd850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:05:28.048 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:28.046+0000 7fefa0be6640 1 -- 172.21.15.49:0/548025897 shutdown_connections 2024-03-18T23:05:28.048 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:28.046+0000 7fefa0be6640 1 -- 172.21.15.49:0/548025897 wait complete. 2024-03-18T23:05:28.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:05:27 smithi139 ceph-mon[67282]: from='client.? 172.21.15.49:0/3933425593' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T23:05:28.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:05:27 smithi139 ceph-mon[67282]: from='client.? 172.21.15.49:0/824776155' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-18T23:05:28.358 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:05:27 smithi175 ceph-mon[67352]: from='client.? 172.21.15.49:0/3933425593' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T23:05:28.358 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:05:27 smithi175 ceph-mon[67352]: from='client.? 172.21.15.49:0/824776155' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-18T23:05:28.379 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:05:27 smithi049 ceph-mon[62991]: from='client.? 172.21.15.49:0/3933425593' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T23:05:28.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:05:27 smithi049 ceph-mon[62991]: from='client.? 172.21.15.49:0/824776155' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-18T23:05:29.167 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:05:28 smithi175 ceph-mon[67352]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T23:05:29.168 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:05:28 smithi175 ceph-mon[67352]: from='client.? 172.21.15.49:0/548025897' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T23:05:29.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:05:28 smithi139 ceph-mon[67282]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T23:05:29.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:05:28 smithi139 ceph-mon[67282]: from='client.? 172.21.15.49:0/548025897' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T23:05:29.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:05:28 smithi049 ceph-mon[62991]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T23:05:29.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:05:28 smithi049 ceph-mon[62991]: from='client.? 172.21.15.49:0/548025897' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T23:05:29.711 INFO:journalctl@ceph.mds.l.smithi175.stdout:Mar 18 23:05:29 smithi175 systemd[1]: Started Ceph mds.l for c63591ea-e57a-11ee-95c9-87774f69a715. 2024-03-18T23:05:30.058 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-18T23:05:30.182 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:30.178+0000 7f6ef7ef1640 1 -- 172.21.15.49:0/569194381 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6ef0100130 msgr2=0x7f6ef0102560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:05:30.182 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:30.178+0000 7f6ef7ef1640 1 --2- 172.21.15.49:0/569194381 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6ef0100130 0x7f6ef0102560 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f6eec009900 tx=0x7f6eec02bc30 comp rx=0 tx=0).stop 2024-03-18T23:05:30.182 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:30.178+0000 7f6ef7ef1640 1 -- 172.21.15.49:0/569194381 shutdown_connections 2024-03-18T23:05:30.182 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:30.178+0000 7f6ef7ef1640 1 --2- 172.21.15.49:0/569194381 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6ef0105450 0x7f6ef01058d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:05:30.182 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:30.178+0000 7f6ef7ef1640 1 --2- 172.21.15.49:0/569194381 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6ef0102aa0 0x7f6ef0104f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:05:30.182 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:30.178+0000 7f6ef7ef1640 1 --2- 172.21.15.49:0/569194381 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6ef0100130 0x7f6ef0102560 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:05:30.182 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:30.178+0000 7f6ef7ef1640 1 -- 172.21.15.49:0/569194381 >> 172.21.15.49:0/569194381 conn(0x7f6ef00fbd50 msgr2=0x7f6ef00fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:05:30.183 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:30.179+0000 7f6ef7ef1640 1 -- 172.21.15.49:0/569194381 shutdown_connections 2024-03-18T23:05:30.183 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:30.179+0000 7f6ef7ef1640 1 -- 172.21.15.49:0/569194381 wait complete. 2024-03-18T23:05:30.183 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:30.179+0000 7f6ef7ef1640 1 Processor -- start 2024-03-18T23:05:30.183 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:30.179+0000 7f6ef7ef1640 1 -- start start 2024-03-18T23:05:30.183 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:30.179+0000 7f6ef7ef1640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6ef0100130 0x7f6ef019e160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:05:30.183 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:30.179+0000 7f6ef7ef1640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6ef0102aa0 0x7f6ef019e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:05:30.183 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:30.179+0000 7f6ef7ef1640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6ef0105450 0x7f6ef01a5710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:05:30.183 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:30.179+0000 7f6ef7ef1640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f6ef0113ac0 con 0x7f6ef0100130 2024-03-18T23:05:30.183 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:30.179+0000 7f6ef7ef1640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f6ef0113940 con 0x7f6ef0105450 2024-03-18T23:05:30.183 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:30.179+0000 7f6ef7ef1640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f6ef0113c40 con 0x7f6ef0102aa0 2024-03-18T23:05:30.183 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:30.180+0000 7f6ef5c8d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6ef0100130 0x7f6ef019e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:05:30.183 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:30.180+0000 7f6ef5c8d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6ef0100130 0x7f6ef019e160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:55556/0 (socket says 172.21.15.49:55556) 2024-03-18T23:05:30.183 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:30.180+0000 7f6ef5c8d640 1 -- 172.21.15.49:0/2391170571 learned_addr learned my addr 172.21.15.49:0/2391170571 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:05:30.184 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:30.180+0000 7f6ef648e640 1 --2- 172.21.15.49:0/2391170571 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6ef0105450 0x7f6ef01a5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:05:30.184 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:30.180+0000 7f6ef548c640 1 --2- 172.21.15.49:0/2391170571 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6ef0102aa0 0x7f6ef019e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:05:30.184 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:30.180+0000 7f6ef5c8d640 1 -- 172.21.15.49:0/2391170571 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6ef0102aa0 msgr2=0x7f6ef019e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:05:30.184 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:30.180+0000 7f6ef5c8d640 1 --2- 172.21.15.49:0/2391170571 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6ef0102aa0 0x7f6ef019e6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:05:30.184 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:30.180+0000 7f6ef5c8d640 1 -- 172.21.15.49:0/2391170571 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6ef0105450 msgr2=0x7f6ef01a5710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:05:30.184 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:30.180+0000 7f6ef5c8d640 1 --2- 172.21.15.49:0/2391170571 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6ef0105450 0x7f6ef01a5710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:05:30.184 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:30.180+0000 7f6ef5c8d640 1 -- 172.21.15.49:0/2391170571 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ef01a5d80 con 0x7f6ef0100130 2024-03-18T23:05:30.184 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:30.180+0000 7f6ef5c8d640 1 --2- 172.21.15.49:0/2391170571 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6ef0100130 0x7f6ef019e160 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f6eec009230 tx=0x7f6eec005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:05:30.184 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:30.180+0000 7f6ee6ffd640 1 -- 172.21.15.49:0/2391170571 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6eec041070 con 0x7f6ef0100130 2024-03-18T23:05:30.184 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:30.180+0000 7f6ee6ffd640 1 -- 172.21.15.49:0/2391170571 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6eec0044d0 con 0x7f6ef0100130 2024-03-18T23:05:30.184 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:30.180+0000 7f6ef7ef1640 1 -- 172.21.15.49:0/2391170571 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ef01a6010 con 0x7f6ef0100130 2024-03-18T23:05:30.184 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:30.180+0000 7f6ee6ffd640 1 -- 172.21.15.49:0/2391170571 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6eec034440 con 0x7f6ef0100130 2024-03-18T23:05:30.184 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:30.180+0000 7f6ef7ef1640 1 -- 172.21.15.49:0/2391170571 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ef01a6550 con 0x7f6ef0100130 2024-03-18T23:05:30.184 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:30.181+0000 7f6ef7ef1640 1 -- 172.21.15.49:0/2391170571 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ef00fd840 con 0x7f6ef0100130 2024-03-18T23:05:30.185 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:30.182+0000 7f6ee6ffd640 1 -- 172.21.15.49:0/2391170571 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7f6eec044070 con 0x7f6ef0100130 2024-03-18T23:05:30.186 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:30.183+0000 7f6ee6ffd640 1 --2- 172.21.15.49:0/2391170571 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f6edc0a2880 0x7f6edc0a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:05:30.186 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:30.183+0000 7f6ef548c640 1 --2- 172.21.15.49:0/2391170571 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f6edc0a2880 0x7f6edc0a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:05:30.186 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:30.183+0000 7f6ee6ffd640 1 -- 172.21.15.49:0/2391170571 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f6eec0f0500 con 0x7f6ef0100130 2024-03-18T23:05:30.186 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:30.184+0000 7f6ef548c640 1 --2- 172.21.15.49:0/2391170571 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f6edc0a2880 0x7f6edc0a4d40 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f6ee0005fd0 tx=0x7f6ee0005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:05:30.187 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:30.187+0000 7f6ee6ffd640 1 -- 172.21.15.49:0/2391170571 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f6eec0bda40 con 0x7f6ef0100130 2024-03-18T23:05:30.411 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:30.410+0000 7f6ef7ef1640 1 -- 172.21.15.49:0/2391170571 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6ef002d1f0 con 0x7f6ef0100130 2024-03-18T23:05:30.411 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:30.410+0000 7f6ee6ffd640 1 -- 172.21.15.49:0/2391170571 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1396 (secure 0 0 0) 0x7f6eec0bd340 con 0x7f6ef0100130 2024-03-18T23:05:30.412 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-18T23:05:30.412 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/1702997173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":1702997173},{"type":"v1","addr":"172.21.15.175:6833","nonce":1702997173}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-03-18T23:05:30.412 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 2 2024-03-18T23:05:30.414 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:30.412+0000 7f6ef7ef1640 1 -- 172.21.15.49:0/2391170571 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f6edc0a2880 msgr2=0x7f6edc0a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:05:30.414 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:30.412+0000 7f6ef7ef1640 1 --2- 172.21.15.49:0/2391170571 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f6edc0a2880 0x7f6edc0a4d40 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f6ee0005fd0 tx=0x7f6ee0005850 comp rx=0 tx=0).stop 2024-03-18T23:05:30.414 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:30.412+0000 7f6ef7ef1640 1 -- 172.21.15.49:0/2391170571 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6ef0100130 msgr2=0x7f6ef019e160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:05:30.414 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:30.412+0000 7f6ef7ef1640 1 --2- 172.21.15.49:0/2391170571 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6ef0100130 0x7f6ef019e160 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f6eec009230 tx=0x7f6eec005d50 comp rx=0 tx=0).stop 2024-03-18T23:05:30.414 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:30.412+0000 7f6ef7ef1640 1 -- 172.21.15.49:0/2391170571 shutdown_connections 2024-03-18T23:05:30.414 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:30.412+0000 7f6ef7ef1640 1 --2- 172.21.15.49:0/2391170571 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f6edc0a2880 0x7f6edc0a4d40 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:05:30.414 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:30.412+0000 7f6ef7ef1640 1 --2- 172.21.15.49:0/2391170571 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6ef0105450 0x7f6ef01a5710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:05:30.414 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:30.412+0000 7f6ef7ef1640 1 --2- 172.21.15.49:0/2391170571 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6ef0102aa0 0x7f6ef019e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:05:30.414 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:30.412+0000 7f6ef7ef1640 1 --2- 172.21.15.49:0/2391170571 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6ef0100130 0x7f6ef019e160 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:05:30.414 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:30.412+0000 7f6ef7ef1640 1 -- 172.21.15.49:0/2391170571 >> 172.21.15.49:0/2391170571 conn(0x7f6ef00fbd50 msgr2=0x7f6ef01033f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:05:30.414 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:30.413+0000 7f6ef7ef1640 1 -- 172.21.15.49:0/2391170571 shutdown_connections 2024-03-18T23:05:30.414 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:30.413+0000 7f6ef7ef1640 1 -- 172.21.15.49:0/2391170571 wait complete. 2024-03-18T23:05:30.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:05:30 smithi139 ceph-mon[67282]: mds.? [v2:172.21.15.175:6832/1702997173,v1:172.21.15.175:6833/1702997173] up:boot 2024-03-18T23:05:30.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:05:30 smithi139 ceph-mon[67282]: fsmap 1 up:standby 2024-03-18T23:05:30.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:05:30 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-18T23:05:30.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:05:30 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:05:30.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:05:30 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:05:30.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:05:30 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:05:30.816 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:05:30 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-18T23:05:30.816 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:05:30 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-18T23:05:30.816 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:05:30 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:05:30.816 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:05:30 smithi139 ceph-mon[67282]: Deploying daemon mds.k on smithi139 2024-03-18T23:05:30.816 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:05:30 smithi139 ceph-mon[67282]: from='client.? 172.21.15.49:0/2391170571' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T23:05:30.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:05:30 smithi049 ceph-mon[62991]: mds.? [v2:172.21.15.175:6832/1702997173,v1:172.21.15.175:6833/1702997173] up:boot 2024-03-18T23:05:30.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:05:30 smithi049 ceph-mon[62991]: fsmap 1 up:standby 2024-03-18T23:05:30.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:05:30 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-18T23:05:30.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:05:30 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:05:30.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:05:30 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:05:30.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:05:30 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:05:30.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:05:30 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-18T23:05:30.881 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:05:30 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-18T23:05:30.881 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:05:30 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:05:30.881 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:05:30 smithi049 ceph-mon[62991]: Deploying daemon mds.k on smithi139 2024-03-18T23:05:30.881 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:05:30 smithi049 ceph-mon[62991]: from='client.? 172.21.15.49:0/2391170571' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T23:05:30.962 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:05:30 smithi175 ceph-mon[67352]: mds.? [v2:172.21.15.175:6832/1702997173,v1:172.21.15.175:6833/1702997173] up:boot 2024-03-18T23:05:30.962 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:05:30 smithi175 ceph-mon[67352]: fsmap 1 up:standby 2024-03-18T23:05:30.962 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:05:30 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-18T23:05:30.962 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:05:30 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:05:30.962 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:05:30 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:05:30.962 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:05:30 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:05:30.962 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:05:30 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-18T23:05:30.962 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:05:30 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-18T23:05:30.962 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:05:30 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:05:30.962 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:05:30 smithi175 ceph-mon[67352]: Deploying daemon mds.k on smithi139 2024-03-18T23:05:30.962 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:05:30 smithi175 ceph-mon[67352]: from='client.? 172.21.15.49:0/2391170571' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T23:05:31.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:05:31 smithi049 ceph-mon[62991]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T23:05:31.961 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:05:31 smithi175 ceph-mon[67352]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T23:05:32.027 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:05:31 smithi139 ceph-mon[67282]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T23:05:32.027 INFO:journalctl@ceph.mds.k.smithi139.stdout:Mar 18 23:05:31 smithi139 systemd[1]: Starting Ceph mds.k for c63591ea-e57a-11ee-95c9-87774f69a715... 2024-03-18T23:05:32.284 INFO:journalctl@ceph.mds.k.smithi139.stdout:Mar 18 23:05:32 smithi139 podman[97158]: 2024-03-18 23:05:31.930046125 +0000 UTC m=+0.017545470 image pull 2312c8e42d0069a2dbb2364bddb00264db4b800e7f18a7e7e4eb8f3ecab2ff1c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639 2024-03-18T23:05:32.284 INFO:journalctl@ceph.mds.k.smithi139.stdout:Mar 18 23:05:32 smithi139 podman[97158]: 2024-03-18 23:05:32.031780927 +0000 UTC m=+0.119280281 container create c4db76d479ee9b57345187e99a3d1ddeae05157f9deb754bd0e0a1c56dfff4d7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mds-k, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240307, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-18T23:05:32.284 INFO:journalctl@ceph.mds.k.smithi139.stdout:Mar 18 23:05:32 smithi139 podman[97158]: 2024-03-18 23:05:32.284253134 +0000 UTC m=+0.371752473 container init c4db76d479ee9b57345187e99a3d1ddeae05157f9deb754bd0e0a1c56dfff4d7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mds-k, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240318.181317-b6ae249) 2024-03-18T23:05:32.425 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-18T23:05:32.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:32.549+0000 7efcb6505640 1 -- 172.21.15.49:0/417977038 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efcb01022f0 msgr2=0x7efcb0108b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:05:32.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:32.549+0000 7efcb6505640 1 --2- 172.21.15.49:0/417977038 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efcb01022f0 0x7efcb0108b80 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7efca8009ca0 tx=0x7efca802c0f0 comp rx=0 tx=0).stop 2024-03-18T23:05:32.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:32.549+0000 7efcb6505640 1 -- 172.21.15.49:0/417977038 shutdown_connections 2024-03-18T23:05:32.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:32.549+0000 7efcb6505640 1 --2- 172.21.15.49:0/417977038 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efcb01022f0 0x7efcb0108b80 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:05:32.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:32.549+0000 7efcb6505640 1 --2- 172.21.15.49:0/417977038 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efcb0101930 0x7efcb0101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:05:32.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:32.549+0000 7efcb6505640 1 --2- 172.21.15.49:0/417977038 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efcb0100740 0x7efcb0100b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:05:32.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:32.549+0000 7efcb6505640 1 -- 172.21.15.49:0/417977038 >> 172.21.15.49:0/417977038 conn(0x7efcb00fbcb0 msgr2=0x7efcb00fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:05:32.554 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:32.550+0000 7efcb6505640 1 -- 172.21.15.49:0/417977038 shutdown_connections 2024-03-18T23:05:32.554 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:32.550+0000 7efcb6505640 1 -- 172.21.15.49:0/417977038 wait complete. 2024-03-18T23:05:32.554 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:32.550+0000 7efcb6505640 1 Processor -- start 2024-03-18T23:05:32.554 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:32.550+0000 7efcb6505640 1 -- start start 2024-03-18T23:05:32.554 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:32.551+0000 7efcb6505640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efcb0100740 0x7efcb0197b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:05:32.554 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:32.551+0000 7efcb6505640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efcb0101930 0x7efcb0198070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:05:32.554 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:32.551+0000 7efcb6505640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efcb01022f0 0x7efcb019f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:05:32.554 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:32.551+0000 7efcb6505640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7efcb010c3a0 con 0x7efcb0101930 2024-03-18T23:05:32.554 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:32.551+0000 7efcb6505640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7efcb010c220 con 0x7efcb01022f0 2024-03-18T23:05:32.554 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:32.551+0000 7efcb6505640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7efcb010c520 con 0x7efcb0100740 2024-03-18T23:05:32.554 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:32.551+0000 7efcaf7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efcb0101930 0x7efcb0198070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:05:32.554 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:32.551+0000 7efcaf7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efcb0101930 0x7efcb0198070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:55572/0 (socket says 172.21.15.49:55572) 2024-03-18T23:05:32.555 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:32.551+0000 7efcaf7fe640 1 -- 172.21.15.49:0/812264122 learned_addr learned my addr 172.21.15.49:0/812264122 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:05:32.555 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:32.551+0000 7efcb4aa2640 1 --2- 172.21.15.49:0/812264122 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efcb01022f0 0x7efcb019f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:05:32.555 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:32.551+0000 7efcaffff640 1 --2- 172.21.15.49:0/812264122 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efcb0100740 0x7efcb0197b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:05:32.555 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:32.551+0000 7efcaf7fe640 1 -- 172.21.15.49:0/812264122 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efcb0100740 msgr2=0x7efcb0197b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:05:32.555 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:32.551+0000 7efcaf7fe640 1 --2- 172.21.15.49:0/812264122 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efcb0100740 0x7efcb0197b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:05:32.555 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:32.551+0000 7efcaf7fe640 1 -- 172.21.15.49:0/812264122 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efcb01022f0 msgr2=0x7efcb019f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:05:32.555 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:32.551+0000 7efcaf7fe640 1 --2- 172.21.15.49:0/812264122 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efcb01022f0 0x7efcb019f0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:05:32.555 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:32.551+0000 7efcaf7fe640 1 -- 172.21.15.49:0/812264122 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efcb019f750 con 0x7efcb0101930 2024-03-18T23:05:32.555 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:32.551+0000 7efcaf7fe640 1 --2- 172.21.15.49:0/812264122 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efcb0101930 0x7efcb0198070 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7efca400e9f0 tx=0x7efca400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:05:32.555 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:32.551+0000 7efcad7fa640 1 -- 172.21.15.49:0/812264122 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efca400cd80 con 0x7efcb0101930 2024-03-18T23:05:32.556 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:32.552+0000 7efcb6505640 1 -- 172.21.15.49:0/812264122 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efcb019fa40 con 0x7efcb0101930 2024-03-18T23:05:32.556 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:32.552+0000 7efcad7fa640 1 -- 172.21.15.49:0/812264122 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efca4010910 con 0x7efcb0101930 2024-03-18T23:05:32.556 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:32.552+0000 7efcad7fa640 1 -- 172.21.15.49:0/812264122 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efca40188f0 con 0x7efcb0101930 2024-03-18T23:05:32.556 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:32.552+0000 7efcb6505640 1 -- 172.21.15.49:0/812264122 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efcb019ffe0 con 0x7efcb0101930 2024-03-18T23:05:32.557 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:32.553+0000 7efcb6505640 1 -- 172.21.15.49:0/812264122 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efcb01a0270 con 0x7efcb0101930 2024-03-18T23:05:32.557 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:32.554+0000 7efcad7fa640 1 -- 172.21.15.49:0/812264122 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7efca4010ac0 con 0x7efcb0101930 2024-03-18T23:05:32.557 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:32.554+0000 7efcad7fa640 1 --2- 172.21.15.49:0/812264122 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7efc980a2800 0x7efc980a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:05:32.557 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:32.555+0000 7efcaffff640 1 --2- 172.21.15.49:0/812264122 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7efc980a2800 0x7efc980a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:05:32.557 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:32.555+0000 7efcad7fa640 1 -- 172.21.15.49:0/812264122 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7efca40ce9a0 con 0x7efcb0101930 2024-03-18T23:05:32.557 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:32.555+0000 7efcaffff640 1 --2- 172.21.15.49:0/812264122 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7efc980a2800 0x7efc980a4cc0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7efca0005d00 tx=0x7efca0005c50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:05:32.560 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:32.559+0000 7efcad7fa640 1 -- 172.21.15.49:0/812264122 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7efca409bee0 con 0x7efcb0101930 2024-03-18T23:05:32.565 INFO:journalctl@ceph.mds.k.smithi139.stdout:Mar 18 23:05:32 smithi139 podman[97158]: 2024-03-18 23:05:32.286956972 +0000 UTC m=+0.374456314 container start c4db76d479ee9b57345187e99a3d1ddeae05157f9deb754bd0e0a1c56dfff4d7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mds-k, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True) 2024-03-18T23:05:32.565 INFO:journalctl@ceph.mds.k.smithi139.stdout:Mar 18 23:05:32 smithi139 ceph-mds[97177]: -- 172.21.15.139:0/4041700323 <== mon.2 v2:172.21.15.139:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5652205b3a40 con 0x565220586c00 2024-03-18T23:05:32.565 INFO:journalctl@ceph.mds.k.smithi139.stdout:Mar 18 23:05:32 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mds-k[97173]: starting mds.k at 2024-03-18T23:05:32.565 INFO:journalctl@ceph.mds.k.smithi139.stdout:Mar 18 23:05:32 smithi139 bash[97158]: c4db76d479ee9b57345187e99a3d1ddeae05157f9deb754bd0e0a1c56dfff4d7 2024-03-18T23:05:32.565 INFO:journalctl@ceph.mds.k.smithi139.stdout:Mar 18 23:05:32 smithi139 systemd[1]: Started Ceph mds.k for c63591ea-e57a-11ee-95c9-87774f69a715. 2024-03-18T23:05:32.786 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:32.785+0000 7efcb6505640 1 -- 172.21.15.49:0/812264122 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efcb01a0440 con 0x7efcb0101930 2024-03-18T23:05:32.787 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:32.786+0000 7efcad7fa640 1 -- 172.21.15.49:0/812264122 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2224 (secure 0 0 0) 0x7efca409b7e0 con 0x7efcb0101930 2024-03-18T23:05:32.787 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-18T23:05:32.787 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24389,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3766178603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3766178603},{"type":"v1","addr":"172.21.15.139:6833","nonce":3766178603}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/1702997173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":1702997173},{"type":"v1","addr":"172.21.15.175:6833","nonce":1702997173}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-03-18T23:05:32.787 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 3 2024-03-18T23:05:32.789 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:32.788+0000 7efcb6505640 1 -- 172.21.15.49:0/812264122 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7efc980a2800 msgr2=0x7efc980a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:05:32.789 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:32.788+0000 7efcb6505640 1 --2- 172.21.15.49:0/812264122 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7efc980a2800 0x7efc980a4cc0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7efca0005d00 tx=0x7efca0005c50 comp rx=0 tx=0).stop 2024-03-18T23:05:32.789 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:32.788+0000 7efcb6505640 1 -- 172.21.15.49:0/812264122 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efcb0101930 msgr2=0x7efcb0198070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:05:32.789 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:32.788+0000 7efcb6505640 1 --2- 172.21.15.49:0/812264122 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efcb0101930 0x7efcb0198070 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7efca400e9f0 tx=0x7efca400eeb0 comp rx=0 tx=0).stop 2024-03-18T23:05:32.789 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:32.788+0000 7efcb6505640 1 -- 172.21.15.49:0/812264122 shutdown_connections 2024-03-18T23:05:32.789 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:32.788+0000 7efcb6505640 1 --2- 172.21.15.49:0/812264122 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7efc980a2800 0x7efc980a4cc0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:05:32.789 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:32.788+0000 7efcb6505640 1 --2- 172.21.15.49:0/812264122 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efcb01022f0 0x7efcb019f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:05:32.789 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:32.788+0000 7efcb6505640 1 --2- 172.21.15.49:0/812264122 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efcb0101930 0x7efcb0198070 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:05:32.790 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:32.788+0000 7efcb6505640 1 --2- 172.21.15.49:0/812264122 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efcb0100740 0x7efcb0197b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:05:32.790 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:32.788+0000 7efcb6505640 1 -- 172.21.15.49:0/812264122 >> 172.21.15.49:0/812264122 conn(0x7efcb00fbcb0 msgr2=0x7efcb00fd720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:05:32.790 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:32.788+0000 7efcb6505640 1 -- 172.21.15.49:0/812264122 shutdown_connections 2024-03-18T23:05:32.790 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:32.788+0000 7efcb6505640 1 -- 172.21.15.49:0/812264122 wait complete. 2024-03-18T23:05:33.708 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:05:33 smithi139 ceph-mon[67282]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T23:05:33.708 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:05:33 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:05:33.708 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:05:33 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:05:33.708 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:05:33 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:05:33.708 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:05:33 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-18T23:05:33.708 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:05:33 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-18T23:05:33.708 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:05:33 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:05:33.708 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:05:33 smithi139 ceph-mon[67282]: Deploying daemon mds.j on smithi049 2024-03-18T23:05:33.708 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:05:33 smithi139 ceph-mon[67282]: mds.? [v2:172.21.15.139:6832/3766178603,v1:172.21.15.139:6833/3766178603] up:boot 2024-03-18T23:05:33.708 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:05:33 smithi139 ceph-mon[67282]: fsmap 2 up:standby 2024-03-18T23:05:33.709 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:05:33 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-18T23:05:33.709 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:05:33 smithi139 ceph-mon[67282]: from='client.? 172.21.15.49:0/812264122' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T23:05:33.711 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:05:33 smithi175 ceph-mon[67352]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T23:05:33.711 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:05:33 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:05:33.712 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:05:33 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:05:33.712 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:05:33 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:05:33.712 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:05:33 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-18T23:05:33.712 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:05:33 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-18T23:05:33.712 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:05:33 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:05:33.712 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:05:33 smithi175 ceph-mon[67352]: Deploying daemon mds.j on smithi049 2024-03-18T23:05:33.712 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:05:33 smithi175 ceph-mon[67352]: mds.? [v2:172.21.15.139:6832/3766178603,v1:172.21.15.139:6833/3766178603] up:boot 2024-03-18T23:05:33.712 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:05:33 smithi175 ceph-mon[67352]: fsmap 2 up:standby 2024-03-18T23:05:33.712 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:05:33 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-18T23:05:33.712 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:05:33 smithi175 ceph-mon[67352]: from='client.? 172.21.15.49:0/812264122' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T23:05:33.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:05:33 smithi049 ceph-mon[62991]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T23:05:33.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:05:33 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:05:33.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:05:33 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:05:33.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:05:33 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:05:33.881 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:05:33 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-18T23:05:33.881 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:05:33 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-18T23:05:33.881 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:05:33 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:05:33.881 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:05:33 smithi049 ceph-mon[62991]: Deploying daemon mds.j on smithi049 2024-03-18T23:05:33.881 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:05:33 smithi049 ceph-mon[62991]: mds.? [v2:172.21.15.139:6832/3766178603,v1:172.21.15.139:6833/3766178603] up:boot 2024-03-18T23:05:33.881 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:05:33 smithi049 ceph-mon[62991]: fsmap 2 up:standby 2024-03-18T23:05:33.881 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:05:33 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-18T23:05:33.881 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:05:33 smithi049 ceph-mon[62991]: from='client.? 172.21.15.49:0/812264122' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T23:05:34.801 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-18T23:05:34.834 INFO:journalctl@ceph.mds.j.smithi049.stdout:Mar 18 23:05:34 smithi049 systemd[1]: Starting Ceph mds.j for c63591ea-e57a-11ee-95c9-87774f69a715... 2024-03-18T23:05:34.921 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:34.919+0000 7fe4fa7d3640 1 -- 172.21.15.49:0/1506315093 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe4f4100a90 msgr2=0x7fe4f4100ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:05:34.922 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:34.919+0000 7fe4fa7d3640 1 --2- 172.21.15.49:0/1506315093 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe4f4100a90 0x7fe4f4100ef0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fe4e8009900 tx=0x7fe4e802bc30 comp rx=0 tx=0).stop 2024-03-18T23:05:34.922 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:34.919+0000 7fe4fa7d3640 1 -- 172.21.15.49:0/1506315093 shutdown_connections 2024-03-18T23:05:34.922 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:34.919+0000 7fe4fa7d3640 1 --2- 172.21.15.49:0/1506315093 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe4f4101430 0x7fe4f41018b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:05:34.922 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:34.919+0000 7fe4fa7d3640 1 --2- 172.21.15.49:0/1506315093 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe4f4100a90 0x7fe4f4100ef0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:05:34.922 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:34.919+0000 7fe4fa7d3640 1 --2- 172.21.15.49:0/1506315093 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe4f4100130 0x7fe4f4100550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:05:34.922 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:34.919+0000 7fe4fa7d3640 1 -- 172.21.15.49:0/1506315093 >> 172.21.15.49:0/1506315093 conn(0x7fe4f40fbd50 msgr2=0x7fe4f40fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:05:34.922 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:34.919+0000 7fe4fa7d3640 1 -- 172.21.15.49:0/1506315093 shutdown_connections 2024-03-18T23:05:34.922 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:34.919+0000 7fe4fa7d3640 1 -- 172.21.15.49:0/1506315093 wait complete. 2024-03-18T23:05:34.922 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:34.920+0000 7fe4fa7d3640 1 Processor -- start 2024-03-18T23:05:34.922 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:34.920+0000 7fe4fa7d3640 1 -- start start 2024-03-18T23:05:34.922 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:34.920+0000 7fe4fa7d3640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe4f4100130 0x7fe4f4199d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:05:34.922 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:34.920+0000 7fe4fa7d3640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe4f4100a90 0x7fe4f419a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:05:34.923 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:34.920+0000 7fe4fa7d3640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe4f4101430 0x7fe4f41a1300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:05:34.923 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:34.920+0000 7fe4fa7d3640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fe4f410f6b0 con 0x7fe4f4100a90 2024-03-18T23:05:34.923 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:34.920+0000 7fe4fa7d3640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fe4f410f530 con 0x7fe4f4101430 2024-03-18T23:05:34.923 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:34.920+0000 7fe4fa7d3640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe4f410f830 con 0x7fe4f4100130 2024-03-18T23:05:34.923 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:34.920+0000 7fe4f37fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe4f4100a90 0x7fe4f419a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:05:34.923 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:34.920+0000 7fe4f37fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe4f4100a90 0x7fe4f419a290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:34756/0 (socket says 172.21.15.49:34756) 2024-03-18T23:05:34.923 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:34.920+0000 7fe4f37fe640 1 -- 172.21.15.49:0/3473454338 learned_addr learned my addr 172.21.15.49:0/3473454338 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:05:34.923 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:34.920+0000 7fe4f3fff640 1 --2- 172.21.15.49:0/3473454338 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe4f4100130 0x7fe4f4199d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:05:34.923 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:34.921+0000 7fe4f8d70640 1 --2- 172.21.15.49:0/3473454338 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe4f4101430 0x7fe4f41a1300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:05:34.923 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:34.921+0000 7fe4f37fe640 1 -- 172.21.15.49:0/3473454338 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe4f4100130 msgr2=0x7fe4f4199d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:05:34.923 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:34.921+0000 7fe4f37fe640 1 --2- 172.21.15.49:0/3473454338 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe4f4100130 0x7fe4f4199d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:05:34.923 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:34.921+0000 7fe4f37fe640 1 -- 172.21.15.49:0/3473454338 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe4f4101430 msgr2=0x7fe4f41a1300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:05:34.923 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:34.921+0000 7fe4f37fe640 1 --2- 172.21.15.49:0/3473454338 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe4f4101430 0x7fe4f41a1300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:05:34.924 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:34.921+0000 7fe4f37fe640 1 -- 172.21.15.49:0/3473454338 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4f41a1970 con 0x7fe4f4100a90 2024-03-18T23:05:34.924 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:34.921+0000 7fe4f8d70640 1 --2- 172.21.15.49:0/3473454338 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe4f4101430 0x7fe4f41a1300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T23:05:34.924 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:34.921+0000 7fe4f37fe640 1 --2- 172.21.15.49:0/3473454338 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe4f4100a90 0x7fe4f419a290 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fe4e8009230 tx=0x7fe4e8005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:05:34.924 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:34.921+0000 7fe4f17fa640 1 -- 172.21.15.49:0/3473454338 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4e8041070 con 0x7fe4f4100a90 2024-03-18T23:05:34.924 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:34.921+0000 7fe4f17fa640 1 -- 172.21.15.49:0/3473454338 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe4e80044d0 con 0x7fe4f4100a90 2024-03-18T23:05:34.924 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:34.921+0000 7fe4fa7d3640 1 -- 172.21.15.49:0/3473454338 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4f41a1c00 con 0x7fe4f4100a90 2024-03-18T23:05:34.924 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:34.921+0000 7fe4f17fa640 1 -- 172.21.15.49:0/3473454338 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4e8034440 con 0x7fe4f4100a90 2024-03-18T23:05:34.924 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:34.921+0000 7fe4fa7d3640 1 -- 172.21.15.49:0/3473454338 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4f41a2140 con 0x7fe4f4100a90 2024-03-18T23:05:34.925 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:34.922+0000 7fe4fa7d3640 1 -- 172.21.15.49:0/3473454338 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4f4068a60 con 0x7fe4f4100a90 2024-03-18T23:05:34.925 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:34.923+0000 7fe4f17fa640 1 -- 172.21.15.49:0/3473454338 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7fe4e8044070 con 0x7fe4f4100a90 2024-03-18T23:05:34.925 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:34.924+0000 7fe4f17fa640 1 --2- 172.21.15.49:0/3473454338 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fe4dc0a2880 0x7fe4dc0a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:05:34.925 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:34.924+0000 7fe4f3fff640 1 --2- 172.21.15.49:0/3473454338 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fe4dc0a2880 0x7fe4dc0a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:05:34.925 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:34.924+0000 7fe4f17fa640 1 -- 172.21.15.49:0/3473454338 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fe4e80f01f0 con 0x7fe4f4100a90 2024-03-18T23:05:34.925 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:34.924+0000 7fe4f3fff640 1 --2- 172.21.15.49:0/3473454338 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fe4dc0a2880 0x7fe4dc0a4d40 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fe4e4009bd0 tx=0x7fe4e4005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:05:34.928 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:34.928+0000 7fe4f17fa640 1 -- 172.21.15.49:0/3473454338 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fe4e80bd730 con 0x7fe4f4100a90 2024-03-18T23:05:35.095 INFO:journalctl@ceph.mds.j.smithi049.stdout:Mar 18 23:05:34 smithi049 podman[103660]: 2024-03-18 23:05:34.86538153 +0000 UTC m=+0.019012989 image pull 2312c8e42d0069a2dbb2364bddb00264db4b800e7f18a7e7e4eb8f3ecab2ff1c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639 2024-03-18T23:05:35.095 INFO:journalctl@ceph.mds.j.smithi049.stdout:Mar 18 23:05:34 smithi049 podman[103660]: 2024-03-18 23:05:34.973604777 +0000 UTC m=+0.127236221 container create 3a80eb2dbc25fb605480d6a449fcd8920f7e591b8a886b0574b6414e817131f4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mds-j, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-03-18T23:05:35.154 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:35.154+0000 7fe4fa7d3640 1 -- 172.21.15.49:0/3473454338 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe4f402d270 con 0x7fe4f4100a90 2024-03-18T23:05:35.154 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:35.154+0000 7fe4f17fa640 1 -- 172.21.15.49:0/3473454338 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2224 (secure 0 0 0) 0x7fe4e80bd030 con 0x7fe4f4100a90 2024-03-18T23:05:35.155 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-18T23:05:35.155 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24389,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3766178603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3766178603},{"type":"v1","addr":"172.21.15.139:6833","nonce":3766178603}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/1702997173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":1702997173},{"type":"v1","addr":"172.21.15.175:6833","nonce":1702997173}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-03-18T23:05:35.155 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 3 2024-03-18T23:05:35.156 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:35.156+0000 7fe4fa7d3640 1 -- 172.21.15.49:0/3473454338 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fe4dc0a2880 msgr2=0x7fe4dc0a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:05:35.157 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:35.156+0000 7fe4fa7d3640 1 --2- 172.21.15.49:0/3473454338 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fe4dc0a2880 0x7fe4dc0a4d40 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fe4e4009bd0 tx=0x7fe4e4005f90 comp rx=0 tx=0).stop 2024-03-18T23:05:35.157 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:35.156+0000 7fe4fa7d3640 1 -- 172.21.15.49:0/3473454338 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe4f4100a90 msgr2=0x7fe4f419a290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:05:35.157 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:35.156+0000 7fe4fa7d3640 1 --2- 172.21.15.49:0/3473454338 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe4f4100a90 0x7fe4f419a290 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fe4e8009230 tx=0x7fe4e8005d50 comp rx=0 tx=0).stop 2024-03-18T23:05:35.157 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:35.156+0000 7fe4fa7d3640 1 -- 172.21.15.49:0/3473454338 shutdown_connections 2024-03-18T23:05:35.157 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:35.156+0000 7fe4fa7d3640 1 --2- 172.21.15.49:0/3473454338 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fe4dc0a2880 0x7fe4dc0a4d40 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:05:35.157 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:35.156+0000 7fe4fa7d3640 1 --2- 172.21.15.49:0/3473454338 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe4f4101430 0x7fe4f41a1300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:05:35.157 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:35.156+0000 7fe4fa7d3640 1 --2- 172.21.15.49:0/3473454338 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe4f4100a90 0x7fe4f419a290 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:05:35.157 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:35.156+0000 7fe4fa7d3640 1 --2- 172.21.15.49:0/3473454338 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe4f4100130 0x7fe4f4199d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:05:35.157 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:35.156+0000 7fe4fa7d3640 1 -- 172.21.15.49:0/3473454338 >> 172.21.15.49:0/3473454338 conn(0x7fe4f40fbd50 msgr2=0x7fe4f40fd850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:05:35.157 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:35.157+0000 7fe4fa7d3640 1 -- 172.21.15.49:0/3473454338 shutdown_connections 2024-03-18T23:05:35.157 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:35.157+0000 7fe4fa7d3640 1 -- 172.21.15.49:0/3473454338 wait complete. 2024-03-18T23:05:35.439 INFO:journalctl@ceph.mds.j.smithi049.stdout:Mar 18 23:05:35 smithi049 podman[103660]: 2024-03-18 23:05:35.220527665 +0000 UTC m=+0.374159110 container init 3a80eb2dbc25fb605480d6a449fcd8920f7e591b8a886b0574b6414e817131f4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mds-j, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-03-18T23:05:35.439 INFO:journalctl@ceph.mds.j.smithi049.stdout:Mar 18 23:05:35 smithi049 podman[103660]: 2024-03-18 23:05:35.223788952 +0000 UTC m=+0.377420398 container start 3a80eb2dbc25fb605480d6a449fcd8920f7e591b8a886b0574b6414e817131f4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mds-j, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux ) 2024-03-18T23:05:35.440 INFO:journalctl@ceph.mds.j.smithi049.stdout:Mar 18 23:05:35 smithi049 ceph-mds[103700]: -- 172.21.15.49:0/905016434 <== mon.0 v2:172.21.15.49:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x556b2594da40 con 0x556b25920c00 2024-03-18T23:05:35.440 INFO:journalctl@ceph.mds.j.smithi049.stdout:Mar 18 23:05:35 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mds-j[103694]: starting mds.j at 2024-03-18T23:05:35.440 INFO:journalctl@ceph.mds.j.smithi049.stdout:Mar 18 23:05:35 smithi049 bash[103660]: 3a80eb2dbc25fb605480d6a449fcd8920f7e591b8a886b0574b6414e817131f4 2024-03-18T23:05:35.440 INFO:journalctl@ceph.mds.j.smithi049.stdout:Mar 18 23:05:35 smithi049 systemd[1]: Started Ceph mds.j for c63591ea-e57a-11ee-95c9-87774f69a715. 2024-03-18T23:05:35.711 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:05:35 smithi175 ceph-mon[67352]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T23:05:35.712 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:05:35 smithi175 ceph-mon[67352]: from='client.? 172.21.15.49:0/3473454338' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T23:05:35.712 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:05:35 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:05:35.712 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:05:35 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:05:35.712 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:05:35 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:05:35.712 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:05:35 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-18T23:05:35.712 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:05:35 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-18T23:05:35.712 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:05:35 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:05:35.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:05:35 smithi139 ceph-mon[67282]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T23:05:35.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:05:35 smithi139 ceph-mon[67282]: from='client.? 172.21.15.49:0/3473454338' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T23:05:35.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:05:35 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:05:35.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:05:35 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:05:35.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:05:35 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:05:35.816 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:05:35 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-18T23:05:35.816 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:05:35 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-18T23:05:35.816 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:05:35 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:05:35.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:05:35 smithi049 ceph-mon[62991]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T23:05:35.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:05:35 smithi049 ceph-mon[62991]: from='client.? 172.21.15.49:0/3473454338' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T23:05:35.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:05:35 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:05:35.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:05:35 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:05:35.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:05:35 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:05:35.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:05:35 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-18T23:05:35.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:05:35 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-18T23:05:35.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:05:35 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:05:36.630 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:05:36 smithi049 ceph-mon[62991]: Deploying daemon mds.d on smithi049 2024-03-18T23:05:36.630 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:05:36 smithi049 ceph-mon[62991]: mds.? [v2:172.21.15.49:6834/3631301630,v1:172.21.15.49:6835/3631301630] up:boot 2024-03-18T23:05:36.630 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:05:36 smithi049 ceph-mon[62991]: fsmap 3 up:standby 2024-03-18T23:05:36.630 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:05:36 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-18T23:05:36.711 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:05:36 smithi175 ceph-mon[67352]: Deploying daemon mds.d on smithi049 2024-03-18T23:05:36.711 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:05:36 smithi175 ceph-mon[67352]: mds.? [v2:172.21.15.49:6834/3631301630,v1:172.21.15.49:6835/3631301630] up:boot 2024-03-18T23:05:36.712 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:05:36 smithi175 ceph-mon[67352]: fsmap 3 up:standby 2024-03-18T23:05:36.712 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:05:36 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-18T23:05:36.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:05:36 smithi139 ceph-mon[67282]: Deploying daemon mds.d on smithi049 2024-03-18T23:05:36.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:05:36 smithi139 ceph-mon[67282]: mds.? [v2:172.21.15.49:6834/3631301630,v1:172.21.15.49:6835/3631301630] up:boot 2024-03-18T23:05:36.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:05:36 smithi139 ceph-mon[67282]: fsmap 3 up:standby 2024-03-18T23:05:36.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:05:36 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-18T23:05:37.168 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-18T23:05:37.466 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:37.466+0000 7f2a4d91d640 1 -- 172.21.15.49:0/4094400924 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2a48103400 msgr2=0x7f2a48109c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:05:37.467 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:37.466+0000 7f2a4d91d640 1 --2- 172.21.15.49:0/4094400924 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2a48103400 0x7f2a48109c90 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f2a38009ca0 tx=0x7f2a3802c0f0 comp rx=0 tx=0).stop 2024-03-18T23:05:37.467 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:37.466+0000 7f2a4d91d640 1 -- 172.21.15.49:0/4094400924 shutdown_connections 2024-03-18T23:05:37.467 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:37.466+0000 7f2a4d91d640 1 --2- 172.21.15.49:0/4094400924 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2a48103400 0x7f2a48109c90 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:05:37.467 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:37.466+0000 7f2a4d91d640 1 --2- 172.21.15.49:0/4094400924 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2a48102a40 0x7f2a48102ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:05:37.467 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:37.466+0000 7f2a4d91d640 1 --2- 172.21.15.49:0/4094400924 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2a48101850 0x7f2a48101c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:05:37.467 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:37.466+0000 7f2a4d91d640 1 -- 172.21.15.49:0/4094400924 >> 172.21.15.49:0/4094400924 conn(0x7f2a480fce00 msgr2=0x7f2a480ff220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:05:37.467 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:37.466+0000 7f2a4d91d640 1 -- 172.21.15.49:0/4094400924 shutdown_connections 2024-03-18T23:05:37.467 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:37.466+0000 7f2a4d91d640 1 -- 172.21.15.49:0/4094400924 wait complete. 2024-03-18T23:05:37.467 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:37.467+0000 7f2a4d91d640 1 Processor -- start 2024-03-18T23:05:37.467 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:37.467+0000 7f2a4d91d640 1 -- start start 2024-03-18T23:05:37.469 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:37.467+0000 7f2a4d91d640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2a48101850 0x7f2a48197ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:05:37.469 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:37.467+0000 7f2a4d91d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2a48102a40 0x7f2a48198020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:05:37.469 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:37.467+0000 7f2a4d91d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2a48103400 0x7f2a4819f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:05:37.470 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:37.467+0000 7f2a4d91d640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f2a4810d4d0 con 0x7f2a48102a40 2024-03-18T23:05:37.470 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:37.467+0000 7f2a4d91d640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f2a4810d350 con 0x7f2a48103400 2024-03-18T23:05:37.470 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:37.467+0000 7f2a4d91d640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f2a4810d650 con 0x7f2a48101850 2024-03-18T23:05:37.470 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:37.467+0000 7f2a467fc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2a48102a40 0x7f2a48198020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:05:37.470 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:37.467+0000 7f2a467fc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2a48102a40 0x7f2a48198020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:34792/0 (socket says 172.21.15.49:34792) 2024-03-18T23:05:37.470 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:37.467+0000 7f2a467fc640 1 -- 172.21.15.49:0/2903914292 learned_addr learned my addr 172.21.15.49:0/2903914292 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:05:37.470 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:37.467+0000 7f2a477fe640 1 --2- 172.21.15.49:0/2903914292 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2a48103400 0x7f2a4819f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:05:37.470 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:37.467+0000 7f2a46ffd640 1 --2- 172.21.15.49:0/2903914292 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2a48101850 0x7f2a48197ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:05:37.470 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:37.467+0000 7f2a467fc640 1 -- 172.21.15.49:0/2903914292 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2a48101850 msgr2=0x7f2a48197ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:05:37.470 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:37.467+0000 7f2a467fc640 1 --2- 172.21.15.49:0/2903914292 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2a48101850 0x7f2a48197ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:05:37.470 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:37.467+0000 7f2a467fc640 1 -- 172.21.15.49:0/2903914292 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2a48103400 msgr2=0x7f2a4819f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:05:37.470 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:37.467+0000 7f2a467fc640 1 --2- 172.21.15.49:0/2903914292 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2a48103400 0x7f2a4819f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:05:37.470 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:37.467+0000 7f2a467fc640 1 -- 172.21.15.49:0/2903914292 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a4819f700 con 0x7f2a48102a40 2024-03-18T23:05:37.471 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:37.467+0000 7f2a46ffd640 1 --2- 172.21.15.49:0/2903914292 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2a48101850 0x7f2a48197ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T23:05:37.471 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:37.467+0000 7f2a477fe640 1 --2- 172.21.15.49:0/2903914292 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2a48103400 0x7f2a4819f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T23:05:37.471 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:37.468+0000 7f2a467fc640 1 --2- 172.21.15.49:0/2903914292 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2a48102a40 0x7f2a48198020 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f2a3000c970 tx=0x7f2a3000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:05:37.471 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:37.468+0000 7f2a4c91b640 1 -- 172.21.15.49:0/2903914292 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a30007bb0 con 0x7f2a48102a40 2024-03-18T23:05:37.471 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:37.468+0000 7f2a4c91b640 1 -- 172.21.15.49:0/2903914292 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2a30007d50 con 0x7f2a48102a40 2024-03-18T23:05:37.471 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:37.468+0000 7f2a4c91b640 1 -- 172.21.15.49:0/2903914292 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a3000f450 con 0x7f2a48102a40 2024-03-18T23:05:37.471 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:37.468+0000 7f2a4d91d640 1 -- 172.21.15.49:0/2903914292 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a4819f9f0 con 0x7f2a48102a40 2024-03-18T23:05:37.471 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:37.468+0000 7f2a4d91d640 1 -- 172.21.15.49:0/2903914292 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a4819fef0 con 0x7f2a48102a40 2024-03-18T23:05:37.471 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:37.469+0000 7f2a4d91d640 1 -- 172.21.15.49:0/2903914292 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a48108b10 con 0x7f2a48102a40 2024-03-18T23:05:37.472 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:37.470+0000 7f2a4c91b640 1 -- 172.21.15.49:0/2903914292 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7f2a30005020 con 0x7f2a48102a40 2024-03-18T23:05:37.472 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:37.471+0000 7f2a4c91b640 1 --2- 172.21.15.49:0/2903914292 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f2a340a2800 0x7f2a340a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:05:37.472 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:37.471+0000 7f2a46ffd640 1 --2- 172.21.15.49:0/2903914292 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f2a340a2800 0x7f2a340a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:05:37.472 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:37.471+0000 7f2a4c91b640 1 -- 172.21.15.49:0/2903914292 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f2a300ce430 con 0x7f2a48102a40 2024-03-18T23:05:37.472 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:37.471+0000 7f2a46ffd640 1 --2- 172.21.15.49:0/2903914292 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f2a340a2800 0x7f2a340a4cc0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f2a481028a0 tx=0x7f2a40005e70 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:05:37.475 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:37.475+0000 7f2a4c91b640 1 -- 172.21.15.49:0/2903914292 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f2a30014030 con 0x7f2a48102a40 2024-03-18T23:05:37.709 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:37.709+0000 7f2a4d91d640 1 -- 172.21.15.49:0/2903914292 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2a4802d1f0 con 0x7f2a48102a40 2024-03-18T23:05:37.709 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:37.709+0000 7f2a4c91b640 1 -- 172.21.15.49:0/2903914292 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3049 (secure 0 0 0) 0x7f2a3009b970 con 0x7f2a48102a40 2024-03-18T23:05:37.709 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-18T23:05:37.710 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6835/3631301630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":3631301630},{"type":"v1","addr":"172.21.15.49:6835","nonce":3631301630}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24389,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3766178603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3766178603},{"type":"v1","addr":"172.21.15.139:6833","nonce":3766178603}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/1702997173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":1702997173},{"type":"v1","addr":"172.21.15.175:6833","nonce":1702997173}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-03-18T23:05:37.710 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 4 2024-03-18T23:05:37.711 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:05:37 smithi175 ceph-mon[67352]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T23:05:37.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:37.711+0000 7f2a4d91d640 1 -- 172.21.15.49:0/2903914292 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f2a340a2800 msgr2=0x7f2a340a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:05:37.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:37.711+0000 7f2a4d91d640 1 --2- 172.21.15.49:0/2903914292 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f2a340a2800 0x7f2a340a4cc0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f2a481028a0 tx=0x7f2a40005e70 comp rx=0 tx=0).stop 2024-03-18T23:05:37.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:37.711+0000 7f2a4d91d640 1 -- 172.21.15.49:0/2903914292 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2a48102a40 msgr2=0x7f2a48198020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:05:37.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:37.711+0000 7f2a4d91d640 1 --2- 172.21.15.49:0/2903914292 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2a48102a40 0x7f2a48198020 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f2a3000c970 tx=0x7f2a3000ce30 comp rx=0 tx=0).stop 2024-03-18T23:05:37.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:37.712+0000 7f2a4d91d640 1 -- 172.21.15.49:0/2903914292 shutdown_connections 2024-03-18T23:05:37.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:37.712+0000 7f2a4d91d640 1 --2- 172.21.15.49:0/2903914292 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f2a340a2800 0x7f2a340a4cc0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:05:37.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:37.712+0000 7f2a4d91d640 1 --2- 172.21.15.49:0/2903914292 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2a48103400 0x7f2a4819f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:05:37.713 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:37.712+0000 7f2a4d91d640 1 --2- 172.21.15.49:0/2903914292 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2a48102a40 0x7f2a48198020 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:05:37.713 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:37.712+0000 7f2a4d91d640 1 --2- 172.21.15.49:0/2903914292 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2a48101850 0x7f2a48197ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:05:37.713 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:37.712+0000 7f2a4d91d640 1 -- 172.21.15.49:0/2903914292 >> 172.21.15.49:0/2903914292 conn(0x7f2a480fce00 msgr2=0x7f2a48105ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:05:37.713 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:37.712+0000 7f2a4d91d640 1 -- 172.21.15.49:0/2903914292 shutdown_connections 2024-03-18T23:05:37.713 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:37.712+0000 7f2a4d91d640 1 -- 172.21.15.49:0/2903914292 wait complete. 2024-03-18T23:05:37.723 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:05:37 smithi049 ceph-mon[62991]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T23:05:37.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:05:37 smithi139 ceph-mon[67282]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T23:05:38.380 INFO:journalctl@ceph.mds.d.smithi049.stdout:Mar 18 23:05:38 smithi049 systemd[1]: Started Ceph mds.d for c63591ea-e57a-11ee-95c9-87774f69a715. 2024-03-18T23:05:38.711 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:05:38 smithi175 ceph-mon[67352]: from='client.? 172.21.15.49:0/2903914292' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T23:05:38.712 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:05:38 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:05:38.712 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:05:38 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:05:38.712 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:05:38 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:05:38.712 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:05:38 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-18T23:05:38.712 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:05:38 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-18T23:05:38.712 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:05:38 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:05:38.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:05:38 smithi139 ceph-mon[67282]: from='client.? 172.21.15.49:0/2903914292' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T23:05:38.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:05:38 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:05:38.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:05:38 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:05:38.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:05:38 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:05:38.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:05:38 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-18T23:05:38.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:05:38 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-18T23:05:38.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:05:38 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:05:38.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:05:38 smithi049 ceph-mon[62991]: from='client.? 172.21.15.49:0/2903914292' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T23:05:38.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:05:38 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:05:38.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:05:38 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:05:38.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:05:38 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:05:38.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:05:38 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-18T23:05:38.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:05:38 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-18T23:05:38.881 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:05:38 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:05:39.711 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:05:39 smithi175 ceph-mon[67352]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T23:05:39.712 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:05:39 smithi175 ceph-mon[67352]: Deploying daemon mds.e on smithi139 2024-03-18T23:05:39.712 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:05:39 smithi175 ceph-mon[67352]: mds.? [v2:172.21.15.49:6836/444305233,v1:172.21.15.49:6837/444305233] up:boot 2024-03-18T23:05:39.712 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:05:39 smithi175 ceph-mon[67352]: fsmap 4 up:standby 2024-03-18T23:05:39.712 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:05:39 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-18T23:05:39.726 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-18T23:05:39.760 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:05:39 smithi049 ceph-mon[62991]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T23:05:39.760 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:05:39 smithi049 ceph-mon[62991]: Deploying daemon mds.e on smithi139 2024-03-18T23:05:39.760 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:05:39 smithi049 ceph-mon[62991]: mds.? [v2:172.21.15.49:6836/444305233,v1:172.21.15.49:6837/444305233] up:boot 2024-03-18T23:05:39.761 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:05:39 smithi049 ceph-mon[62991]: fsmap 4 up:standby 2024-03-18T23:05:39.761 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:05:39 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-18T23:05:39.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:05:39 smithi139 ceph-mon[67282]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T23:05:39.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:05:39 smithi139 ceph-mon[67282]: Deploying daemon mds.e on smithi139 2024-03-18T23:05:39.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:05:39 smithi139 ceph-mon[67282]: mds.? [v2:172.21.15.49:6836/444305233,v1:172.21.15.49:6837/444305233] up:boot 2024-03-18T23:05:39.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:05:39 smithi139 ceph-mon[67282]: fsmap 4 up:standby 2024-03-18T23:05:39.816 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:05:39 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-18T23:05:39.852 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:39.850+0000 7f1cc5ea1640 1 -- 172.21.15.49:0/2878649348 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1cc0102360 msgr2=0x7f1cc0104750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:05:39.852 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:39.850+0000 7f1cc5ea1640 1 --2- 172.21.15.49:0/2878649348 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1cc0102360 0x7f1cc0104750 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f1cb0009900 tx=0x7f1cb002bc30 comp rx=0 tx=0).stop 2024-03-18T23:05:39.852 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:39.850+0000 7f1cc5ea1640 1 -- 172.21.15.49:0/2878649348 shutdown_connections 2024-03-18T23:05:39.853 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:39.850+0000 7f1cc5ea1640 1 --2- 172.21.15.49:0/2878649348 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1cc01075c0 0x7f1cc01099b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:05:39.853 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:39.850+0000 7f1cc5ea1640 1 --2- 172.21.15.49:0/2878649348 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1cc0104c90 0x7f1cc0107080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:05:39.853 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:39.850+0000 7f1cc5ea1640 1 --2- 172.21.15.49:0/2878649348 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1cc0102360 0x7f1cc0104750 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:05:39.853 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:39.850+0000 7f1cc5ea1640 1 -- 172.21.15.49:0/2878649348 >> 172.21.15.49:0/2878649348 conn(0x7f1cc00fbcb0 msgr2=0x7f1cc00fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:05:39.853 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:39.850+0000 7f1cc5ea1640 1 -- 172.21.15.49:0/2878649348 shutdown_connections 2024-03-18T23:05:39.853 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:39.850+0000 7f1cc5ea1640 1 -- 172.21.15.49:0/2878649348 wait complete. 2024-03-18T23:05:39.853 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:39.851+0000 7f1cc5ea1640 1 Processor -- start 2024-03-18T23:05:39.853 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:39.851+0000 7f1cc5ea1640 1 -- start start 2024-03-18T23:05:39.853 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:39.851+0000 7f1cc5ea1640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1cc0102360 0x7f1cc0197b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:05:39.853 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:39.851+0000 7f1cc5ea1640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1cc0104c90 0x7f1cc01980c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:05:39.853 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:39.851+0000 7f1cc5ea1640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1cc01075c0 0x7f1cc019f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:05:39.853 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:39.851+0000 7f1cc5ea1640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f1cc010d4b0 con 0x7f1cc01075c0 2024-03-18T23:05:39.854 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:39.851+0000 7f1cc5ea1640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f1cc010d330 con 0x7f1cc0102360 2024-03-18T23:05:39.854 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:39.851+0000 7f1cc5ea1640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f1cc010d630 con 0x7f1cc0104c90 2024-03-18T23:05:39.854 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:39.851+0000 7f1cbffff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1cc01075c0 0x7f1cc019f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:05:39.854 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:39.851+0000 7f1cbffff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1cc01075c0 0x7f1cc019f130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:34814/0 (socket says 172.21.15.49:34814) 2024-03-18T23:05:39.854 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:39.851+0000 7f1cbffff640 1 -- 172.21.15.49:0/1177745312 learned_addr learned my addr 172.21.15.49:0/1177745312 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:05:39.854 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:39.851+0000 7f1cbeffd640 1 --2- 172.21.15.49:0/1177745312 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1cc0104c90 0x7f1cc01980c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:05:39.854 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:39.851+0000 7f1cbf7fe640 1 --2- 172.21.15.49:0/1177745312 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1cc0102360 0x7f1cc0197b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:05:39.854 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:39.852+0000 7f1cbffff640 1 -- 172.21.15.49:0/1177745312 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1cc0104c90 msgr2=0x7f1cc01980c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:05:39.854 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:39.852+0000 7f1cbffff640 1 --2- 172.21.15.49:0/1177745312 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1cc0104c90 0x7f1cc01980c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:05:39.854 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:39.852+0000 7f1cbffff640 1 -- 172.21.15.49:0/1177745312 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1cc0102360 msgr2=0x7f1cc0197b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:05:39.854 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:39.852+0000 7f1cbffff640 1 --2- 172.21.15.49:0/1177745312 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1cc0102360 0x7f1cc0197b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:05:39.854 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:39.852+0000 7f1cbffff640 1 -- 172.21.15.49:0/1177745312 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cc019f7a0 con 0x7f1cc01075c0 2024-03-18T23:05:39.854 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:39.852+0000 7f1cbffff640 1 --2- 172.21.15.49:0/1177745312 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1cc01075c0 0x7f1cc019f130 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f1cb800efd0 tx=0x7f1cb800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:05:39.855 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:39.852+0000 7f1cbcff9640 1 -- 172.21.15.49:0/1177745312 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cb8003b70 con 0x7f1cc01075c0 2024-03-18T23:05:39.855 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:39.852+0000 7f1cc5ea1640 1 -- 172.21.15.49:0/1177745312 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1cc019fa90 con 0x7f1cc01075c0 2024-03-18T23:05:39.855 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:39.852+0000 7f1cbcff9640 1 -- 172.21.15.49:0/1177745312 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1cb8003d10 con 0x7f1cc01075c0 2024-03-18T23:05:39.855 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:39.852+0000 7f1cbcff9640 1 -- 172.21.15.49:0/1177745312 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cb80109f0 con 0x7f1cc01075c0 2024-03-18T23:05:39.855 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:39.852+0000 7f1cc5ea1640 1 -- 172.21.15.49:0/1177745312 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1cc019ff90 con 0x7f1cc01075c0 2024-03-18T23:05:39.856 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:39.853+0000 7f1cc5ea1640 1 -- 172.21.15.49:0/1177745312 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1cc00fd7a0 con 0x7f1cc01075c0 2024-03-18T23:05:39.856 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:39.854+0000 7f1cbcff9640 1 -- 172.21.15.49:0/1177745312 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7f1cb8010b90 con 0x7f1cc01075c0 2024-03-18T23:05:39.856 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:39.855+0000 7f1cbcff9640 1 --2- 172.21.15.49:0/1177745312 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f1ca80a2800 0x7f1ca80a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:05:39.856 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:39.855+0000 7f1cbf7fe640 1 --2- 172.21.15.49:0/1177745312 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f1ca80a2800 0x7f1ca80a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:05:39.856 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:39.855+0000 7f1cbcff9640 1 -- 172.21.15.49:0/1177745312 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f1cb80cedd0 con 0x7f1cc01075c0 2024-03-18T23:05:39.856 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:39.855+0000 7f1cbf7fe640 1 --2- 172.21.15.49:0/1177745312 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f1ca80a2800 0x7f1ca80a4cc0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f1cb0006010 tx=0x7f1cb0000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:05:39.858 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:39.858+0000 7f1cbcff9640 1 -- 172.21.15.49:0/1177745312 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f1cb809c310 con 0x7f1cc01075c0 2024-03-18T23:05:40.081 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:40.081+0000 7f1cc5ea1640 1 -- 172.21.15.49:0/1177745312 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1cc002d440 con 0x7f1cc01075c0 2024-03-18T23:05:40.082 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:40.082+0000 7f1cbcff9640 1 -- 172.21.15.49:0/1177745312 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3871 (secure 0 0 0) 0x7f1cb809bc10 con 0x7f1cc01075c0 2024-03-18T23:05:40.082 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-18T23:05:40.082 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6835/3631301630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":3631301630},{"type":"v1","addr":"172.21.15.49:6835","nonce":3631301630}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6837/444305233","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":444305233},{"type":"v1","addr":"172.21.15.49:6837","nonce":444305233}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24389,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3766178603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3766178603},{"type":"v1","addr":"172.21.15.139:6833","nonce":3766178603}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/1702997173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":1702997173},{"type":"v1","addr":"172.21.15.175:6833","nonce":1702997173}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-03-18T23:05:40.082 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 5 2024-03-18T23:05:40.084 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:40.083+0000 7f1cc5ea1640 1 -- 172.21.15.49:0/1177745312 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f1ca80a2800 msgr2=0x7f1ca80a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:05:40.084 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:40.083+0000 7f1cc5ea1640 1 --2- 172.21.15.49:0/1177745312 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f1ca80a2800 0x7f1ca80a4cc0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f1cb0006010 tx=0x7f1cb0000bc0 comp rx=0 tx=0).stop 2024-03-18T23:05:40.084 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:40.083+0000 7f1cc5ea1640 1 -- 172.21.15.49:0/1177745312 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1cc01075c0 msgr2=0x7f1cc019f130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:05:40.084 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:40.083+0000 7f1cc5ea1640 1 --2- 172.21.15.49:0/1177745312 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1cc01075c0 0x7f1cc019f130 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f1cb800efd0 tx=0x7f1cb800c6a0 comp rx=0 tx=0).stop 2024-03-18T23:05:40.084 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:40.084+0000 7f1cc5ea1640 1 -- 172.21.15.49:0/1177745312 shutdown_connections 2024-03-18T23:05:40.084 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:40.084+0000 7f1cc5ea1640 1 --2- 172.21.15.49:0/1177745312 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f1ca80a2800 0x7f1ca80a4cc0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:05:40.084 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:40.084+0000 7f1cc5ea1640 1 --2- 172.21.15.49:0/1177745312 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1cc01075c0 0x7f1cc019f130 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:05:40.084 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:40.084+0000 7f1cc5ea1640 1 --2- 172.21.15.49:0/1177745312 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1cc0104c90 0x7f1cc01980c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:05:40.084 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:40.084+0000 7f1cc5ea1640 1 --2- 172.21.15.49:0/1177745312 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1cc0102360 0x7f1cc0197b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:05:40.084 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:40.084+0000 7f1cc5ea1640 1 -- 172.21.15.49:0/1177745312 >> 172.21.15.49:0/1177745312 conn(0x7f1cc00fbcb0 msgr2=0x7f1cc0102da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:05:40.084 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:40.084+0000 7f1cc5ea1640 1 -- 172.21.15.49:0/1177745312 shutdown_connections 2024-03-18T23:05:40.084 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:40.084+0000 7f1cc5ea1640 1 -- 172.21.15.49:0/1177745312 wait complete. 2024-03-18T23:05:40.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:05:40 smithi139 ceph-mon[67282]: from='client.? 172.21.15.49:0/1177745312' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T23:05:40.566 INFO:journalctl@ceph.mds.e.smithi139.stdout:Mar 18 23:05:40 smithi139 systemd[1]: Starting Ceph mds.e for c63591ea-e57a-11ee-95c9-87774f69a715... 2024-03-18T23:05:40.711 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:05:40 smithi175 ceph-mon[67352]: from='client.? 172.21.15.49:0/1177745312' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T23:05:40.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:05:40 smithi049 ceph-mon[62991]: from='client.? 172.21.15.49:0/1177745312' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T23:05:40.964 INFO:journalctl@ceph.mds.e.smithi139.stdout:Mar 18 23:05:40 smithi139 podman[97478]: 2024-03-18 23:05:40.608469235 +0000 UTC m=+0.017499157 image pull 2312c8e42d0069a2dbb2364bddb00264db4b800e7f18a7e7e4eb8f3ecab2ff1c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639 2024-03-18T23:05:40.965 INFO:journalctl@ceph.mds.e.smithi139.stdout:Mar 18 23:05:40 smithi139 podman[97478]: 2024-03-18 23:05:40.728945157 +0000 UTC m=+0.137975060 container create 0431e396f28373930b194f794f07e2a5e665813485416e809818c2560e941b03 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mds-e, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-18T23:05:40.965 INFO:journalctl@ceph.mds.e.smithi139.stdout:Mar 18 23:05:40 smithi139 podman[97478]: 2024-03-18 23:05:40.964875628 +0000 UTC m=+0.373905543 container init 0431e396f28373930b194f794f07e2a5e665813485416e809818c2560e941b03 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mds-e, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, io.buildah.version=1.33.5) 2024-03-18T23:05:41.315 INFO:journalctl@ceph.mds.e.smithi139.stdout:Mar 18 23:05:40 smithi139 podman[97478]: 2024-03-18 23:05:40.967593459 +0000 UTC m=+0.376623373 container start 0431e396f28373930b194f794f07e2a5e665813485416e809818c2560e941b03 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mds-e, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, ceph=True) 2024-03-18T23:05:41.315 INFO:journalctl@ceph.mds.e.smithi139.stdout:Mar 18 23:05:40 smithi139 ceph-mds[97498]: -- 172.21.15.139:0/1754409214 <== mon.2 v2:172.21.15.139:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55e350a8fa40 con 0x55e350a62c00 2024-03-18T23:05:41.315 INFO:journalctl@ceph.mds.e.smithi139.stdout:Mar 18 23:05:40 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mds-e[97494]: starting mds.e at 2024-03-18T23:05:41.315 INFO:journalctl@ceph.mds.e.smithi139.stdout:Mar 18 23:05:41 smithi139 bash[97478]: 0431e396f28373930b194f794f07e2a5e665813485416e809818c2560e941b03 2024-03-18T23:05:41.315 INFO:journalctl@ceph.mds.e.smithi139.stdout:Mar 18 23:05:41 smithi139 systemd[1]: Started Ceph mds.e for c63591ea-e57a-11ee-95c9-87774f69a715. 2024-03-18T23:05:41.712 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:05:41 smithi175 ceph-mon[67352]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T23:05:41.713 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:05:41 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:05:41.713 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:05:41 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:05:41.713 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:05:41 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:05:41.713 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:05:41 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-18T23:05:41.713 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:05:41 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-18T23:05:41.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:05:41 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:05:41.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:05:41 smithi139 ceph-mon[67282]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T23:05:41.816 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:05:41 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:05:41.816 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:05:41 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:05:41.816 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:05:41 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:05:41.816 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:05:41 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-18T23:05:41.816 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:05:41 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-18T23:05:41.816 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:05:41 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:05:41.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:05:41 smithi049 ceph-mon[62991]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T23:05:41.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:05:41 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:05:41.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:05:41 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:05:41.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:05:41 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:05:41.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:05:41 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-18T23:05:41.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:05:41 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-18T23:05:41.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:05:41 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:05:42.095 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-18T23:05:42.224 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:42.220+0000 7f838f7f4640 1 -- 172.21.15.49:0/129107045 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8388101930 msgr2=0x7f8388101db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:05:42.225 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:42.220+0000 7f838f7f4640 1 --2- 172.21.15.49:0/129107045 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8388101930 0x7f8388101db0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f8378009900 tx=0x7f837802bc30 comp rx=0 tx=0).stop 2024-03-18T23:05:42.225 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:42.220+0000 7f838f7f4640 1 -- 172.21.15.49:0/129107045 shutdown_connections 2024-03-18T23:05:42.225 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:42.220+0000 7f838f7f4640 1 --2- 172.21.15.49:0/129107045 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f83881022f0 0x7f8388108b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:05:42.225 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:42.220+0000 7f838f7f4640 1 --2- 172.21.15.49:0/129107045 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8388101930 0x7f8388101db0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:05:42.225 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:42.220+0000 7f838f7f4640 1 --2- 172.21.15.49:0/129107045 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8388100740 0x7f8388100b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:05:42.225 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:42.220+0000 7f838f7f4640 1 -- 172.21.15.49:0/129107045 >> 172.21.15.49:0/129107045 conn(0x7f83880fbcb0 msgr2=0x7f83880fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:05:42.225 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:42.220+0000 7f838f7f4640 1 -- 172.21.15.49:0/129107045 shutdown_connections 2024-03-18T23:05:42.225 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:42.220+0000 7f838f7f4640 1 -- 172.21.15.49:0/129107045 wait complete. 2024-03-18T23:05:42.225 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:42.220+0000 7f838f7f4640 1 Processor -- start 2024-03-18T23:05:42.225 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:42.220+0000 7f838f7f4640 1 -- start start 2024-03-18T23:05:42.225 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:42.221+0000 7f838f7f4640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8388100740 0x7f8388197b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:05:42.226 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:42.221+0000 7f838f7f4640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8388101930 0x7f83881980a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:05:42.226 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:42.221+0000 7f838f7f4640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f83881022f0 0x7f838819f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:05:42.226 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:42.221+0000 7f838f7f4640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f838810c3c0 con 0x7f83881022f0 2024-03-18T23:05:42.226 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:42.221+0000 7f838f7f4640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f838810c240 con 0x7f8388100740 2024-03-18T23:05:42.226 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:42.221+0000 7f838f7f4640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f838810c540 con 0x7f8388101930 2024-03-18T23:05:42.226 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:42.221+0000 7f838dd91640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f83881022f0 0x7f838819f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:05:42.226 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:42.221+0000 7f838dd91640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f83881022f0 0x7f838819f110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:34842/0 (socket says 172.21.15.49:34842) 2024-03-18T23:05:42.226 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:42.221+0000 7f838dd91640 1 -- 172.21.15.49:0/3868137142 learned_addr learned my addr 172.21.15.49:0/3868137142 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:05:42.226 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:42.221+0000 7f838d590640 1 --2- 172.21.15.49:0/3868137142 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8388100740 0x7f8388197b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:05:42.226 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:42.221+0000 7f838cd8f640 1 --2- 172.21.15.49:0/3868137142 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8388101930 0x7f83881980a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:05:42.226 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:42.221+0000 7f838dd91640 1 -- 172.21.15.49:0/3868137142 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8388101930 msgr2=0x7f83881980a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:05:42.226 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:42.221+0000 7f838dd91640 1 --2- 172.21.15.49:0/3868137142 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8388101930 0x7f83881980a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:05:42.226 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:42.221+0000 7f838dd91640 1 -- 172.21.15.49:0/3868137142 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8388100740 msgr2=0x7f8388197b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:05:42.226 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:42.221+0000 7f838dd91640 1 --2- 172.21.15.49:0/3868137142 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8388100740 0x7f8388197b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:05:42.227 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:42.221+0000 7f838dd91640 1 -- 172.21.15.49:0/3868137142 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f838819f780 con 0x7f83881022f0 2024-03-18T23:05:42.227 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:42.221+0000 7f838d590640 1 --2- 172.21.15.49:0/3868137142 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8388100740 0x7f8388197b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T23:05:42.227 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:42.221+0000 7f838cd8f640 1 --2- 172.21.15.49:0/3868137142 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8388101930 0x7f83881980a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T23:05:42.227 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:42.221+0000 7f838dd91640 1 --2- 172.21.15.49:0/3868137142 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f83881022f0 0x7f838819f110 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f838000cce0 tx=0x7f838000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:05:42.227 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:42.221+0000 7f837e7fc640 1 -- 172.21.15.49:0/3868137142 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f838000f450 con 0x7f83881022f0 2024-03-18T23:05:42.227 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:42.221+0000 7f837e7fc640 1 -- 172.21.15.49:0/3868137142 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f838000fad0 con 0x7f83881022f0 2024-03-18T23:05:42.227 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:42.221+0000 7f838f7f4640 1 -- 172.21.15.49:0/3868137142 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f838819fa70 con 0x7f83881022f0 2024-03-18T23:05:42.227 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:42.221+0000 7f837e7fc640 1 -- 172.21.15.49:0/3868137142 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8380018c10 con 0x7f83881022f0 2024-03-18T23:05:42.227 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:42.221+0000 7f838f7f4640 1 -- 172.21.15.49:0/3868137142 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f838819ff90 con 0x7f83881022f0 2024-03-18T23:05:42.227 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:42.222+0000 7f838f7f4640 1 -- 172.21.15.49:0/3868137142 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83881a0220 con 0x7f83881022f0 2024-03-18T23:05:42.227 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:42.224+0000 7f837e7fc640 1 -- 172.21.15.49:0/3868137142 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7f838000f5f0 con 0x7f83881022f0 2024-03-18T23:05:42.227 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:42.224+0000 7f837e7fc640 1 --2- 172.21.15.49:0/3868137142 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f83740a2880 0x7f83740a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:05:42.227 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:42.224+0000 7f838d590640 1 --2- 172.21.15.49:0/3868137142 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f83740a2880 0x7f83740a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:05:42.228 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:42.224+0000 7f837e7fc640 1 -- 172.21.15.49:0/3868137142 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f83800ced00 con 0x7f83881022f0 2024-03-18T23:05:42.228 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:42.225+0000 7f838d590640 1 --2- 172.21.15.49:0/3868137142 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f83740a2880 0x7f83740a4d40 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f83840058c0 tx=0x7f8384005ec0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:05:42.228 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:42.229+0000 7f837e7fc640 1 -- 172.21.15.49:0/3868137142 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f8380014030 con 0x7f83881022f0 2024-03-18T23:05:42.451 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:42.451+0000 7f838f7f4640 1 -- 172.21.15.49:0/3868137142 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f83881a04d0 con 0x7f83881022f0 2024-03-18T23:05:42.452 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:42.451+0000 7f837e7fc640 1 -- 172.21.15.49:0/3868137142 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+4699 (secure 0 0 0) 0x7f838009c240 con 0x7f83881022f0 2024-03-18T23:05:42.452 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-18T23:05:42.452 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6835/3631301630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":3631301630},{"type":"v1","addr":"172.21.15.49:6835","nonce":3631301630}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6837/444305233","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":444305233},{"type":"v1","addr":"172.21.15.49:6837","nonce":444305233}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24389,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3766178603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3766178603},{"type":"v1","addr":"172.21.15.139:6833","nonce":3766178603}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/1702997173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":1702997173},{"type":"v1","addr":"172.21.15.175:6833","nonce":1702997173}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/1663212937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":1663212937},{"type":"v1","addr":"172.21.15.139:6835","nonce":1663212937}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-03-18T23:05:42.452 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 6 2024-03-18T23:05:42.454 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:42.453+0000 7f838f7f4640 1 -- 172.21.15.49:0/3868137142 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f83740a2880 msgr2=0x7f83740a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:05:42.454 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:42.453+0000 7f838f7f4640 1 --2- 172.21.15.49:0/3868137142 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f83740a2880 0x7f83740a4d40 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f83840058c0 tx=0x7f8384005ec0 comp rx=0 tx=0).stop 2024-03-18T23:05:42.454 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:42.453+0000 7f838f7f4640 1 -- 172.21.15.49:0/3868137142 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f83881022f0 msgr2=0x7f838819f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:05:42.454 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:42.453+0000 7f838f7f4640 1 --2- 172.21.15.49:0/3868137142 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f83881022f0 0x7f838819f110 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f838000cce0 tx=0x7f838000a620 comp rx=0 tx=0).stop 2024-03-18T23:05:42.454 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:42.453+0000 7f838f7f4640 1 -- 172.21.15.49:0/3868137142 shutdown_connections 2024-03-18T23:05:42.454 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:42.453+0000 7f838f7f4640 1 --2- 172.21.15.49:0/3868137142 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f83740a2880 0x7f83740a4d40 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:05:42.454 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:42.453+0000 7f838f7f4640 1 --2- 172.21.15.49:0/3868137142 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f83881022f0 0x7f838819f110 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:05:42.454 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:42.453+0000 7f838f7f4640 1 --2- 172.21.15.49:0/3868137142 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8388101930 0x7f83881980a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:05:42.454 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:42.453+0000 7f838f7f4640 1 --2- 172.21.15.49:0/3868137142 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8388100740 0x7f8388197b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:05:42.454 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:42.453+0000 7f838f7f4640 1 -- 172.21.15.49:0/3868137142 >> 172.21.15.49:0/3868137142 conn(0x7f83880fbcb0 msgr2=0x7f83880fd720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:05:42.454 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:42.454+0000 7f838f7f4640 1 -- 172.21.15.49:0/3868137142 shutdown_connections 2024-03-18T23:05:42.454 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:42.454+0000 7f838f7f4640 1 -- 172.21.15.49:0/3868137142 wait complete. 2024-03-18T23:05:42.782 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:05:42 smithi175 ceph-mon[67352]: Deploying daemon mds.f on smithi175 2024-03-18T23:05:42.783 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:05:42 smithi175 ceph-mon[67352]: mds.? [v2:172.21.15.139:6834/1663212937,v1:172.21.15.139:6835/1663212937] up:boot 2024-03-18T23:05:42.783 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:05:42 smithi175 ceph-mon[67352]: fsmap 5 up:standby 2024-03-18T23:05:42.783 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:05:42 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-18T23:05:42.783 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:05:42 smithi175 ceph-mon[67352]: from='client.? 172.21.15.49:0/3868137142' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T23:05:42.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:05:42 smithi139 ceph-mon[67282]: Deploying daemon mds.f on smithi175 2024-03-18T23:05:42.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:05:42 smithi139 ceph-mon[67282]: mds.? [v2:172.21.15.139:6834/1663212937,v1:172.21.15.139:6835/1663212937] up:boot 2024-03-18T23:05:42.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:05:42 smithi139 ceph-mon[67282]: fsmap 5 up:standby 2024-03-18T23:05:42.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:05:42 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-18T23:05:42.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:05:42 smithi139 ceph-mon[67282]: from='client.? 172.21.15.49:0/3868137142' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T23:05:42.881 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:05:42 smithi049 ceph-mon[62991]: Deploying daemon mds.f on smithi175 2024-03-18T23:05:42.881 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:05:42 smithi049 ceph-mon[62991]: mds.? [v2:172.21.15.139:6834/1663212937,v1:172.21.15.139:6835/1663212937] up:boot 2024-03-18T23:05:42.881 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:05:42 smithi049 ceph-mon[62991]: fsmap 5 up:standby 2024-03-18T23:05:42.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:05:42 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-18T23:05:42.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:05:42 smithi049 ceph-mon[62991]: from='client.? 172.21.15.49:0/3868137142' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T23:05:43.462 INFO:journalctl@ceph.mds.f.smithi175.stdout:Mar 18 23:05:43 smithi175 systemd[1]: Starting Ceph mds.f for c63591ea-e57a-11ee-95c9-87774f69a715... 2024-03-18T23:05:43.779 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:05:43 smithi175 ceph-mon[67352]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T23:05:43.780 INFO:journalctl@ceph.mds.f.smithi175.stdout:Mar 18 23:05:43 smithi175 podman[98189]: 2024-03-18 23:05:43.38363182 +0000 UTC m=+0.017706527 image pull 2312c8e42d0069a2dbb2364bddb00264db4b800e7f18a7e7e4eb8f3ecab2ff1c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639 2024-03-18T23:05:43.780 INFO:journalctl@ceph.mds.f.smithi175.stdout:Mar 18 23:05:43 smithi175 podman[98189]: 2024-03-18 23:05:43.510088496 +0000 UTC m=+0.144163166 container create 66fdf9cee264975aed14cbe4703a2f040899e4bab87044a00970c629e19fa602 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mds-f, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True) 2024-03-18T23:05:43.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:05:43 smithi139 ceph-mon[67282]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T23:05:43.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:05:43 smithi049 ceph-mon[62991]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T23:05:44.212 INFO:journalctl@ceph.mds.f.smithi175.stdout:Mar 18 23:05:43 smithi175 podman[98189]: 2024-03-18 23:05:43.779709343 +0000 UTC m=+0.413784037 container init 66fdf9cee264975aed14cbe4703a2f040899e4bab87044a00970c629e19fa602 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mds-f, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS) 2024-03-18T23:05:44.212 INFO:journalctl@ceph.mds.f.smithi175.stdout:Mar 18 23:05:43 smithi175 podman[98189]: 2024-03-18 23:05:43.782312207 +0000 UTC m=+0.416386885 container start 66fdf9cee264975aed14cbe4703a2f040899e4bab87044a00970c629e19fa602 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mds-f, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.build-date=20240307, GIT_CLEAN=True) 2024-03-18T23:05:44.212 INFO:journalctl@ceph.mds.f.smithi175.stdout:Mar 18 23:05:43 smithi175 ceph-mds[98208]: -- 172.21.15.175:0/2275125081 <== mon.1 v2:172.21.15.175:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x555d16cefa40 con 0x555d16cc3000 2024-03-18T23:05:44.212 INFO:journalctl@ceph.mds.f.smithi175.stdout:Mar 18 23:05:43 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mds-f[98204]: starting mds.f at 2024-03-18T23:05:44.212 INFO:journalctl@ceph.mds.f.smithi175.stdout:Mar 18 23:05:43 smithi175 bash[98189]: 66fdf9cee264975aed14cbe4703a2f040899e4bab87044a00970c629e19fa602 2024-03-18T23:05:44.212 INFO:journalctl@ceph.mds.f.smithi175.stdout:Mar 18 23:05:43 smithi175 systemd[1]: Started Ceph mds.f for c63591ea-e57a-11ee-95c9-87774f69a715. 2024-03-18T23:05:44.465 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-18T23:05:44.587 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:44.586+0000 7efc39087640 1 -- 172.21.15.49:0/2811725046 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efc341019b0 msgr2=0x7efc34101e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:05:44.587 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:44.586+0000 7efc39087640 1 --2- 172.21.15.49:0/2811725046 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efc341019b0 0x7efc34101e30 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7efc24009900 tx=0x7efc2402bc30 comp rx=0 tx=0).stop 2024-03-18T23:05:44.587 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:44.587+0000 7efc39087640 1 -- 172.21.15.49:0/2811725046 shutdown_connections 2024-03-18T23:05:44.587 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:44.587+0000 7efc39087640 1 --2- 172.21.15.49:0/2811725046 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efc34102370 0x7efc34108c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:05:44.588 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:44.587+0000 7efc39087640 1 --2- 172.21.15.49:0/2811725046 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efc341019b0 0x7efc34101e30 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:05:44.588 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:44.587+0000 7efc39087640 1 --2- 172.21.15.49:0/2811725046 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efc341007c0 0x7efc34100be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:05:44.588 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:44.587+0000 7efc39087640 1 -- 172.21.15.49:0/2811725046 >> 172.21.15.49:0/2811725046 conn(0x7efc340fbd50 msgr2=0x7efc340fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:05:44.588 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:44.587+0000 7efc39087640 1 -- 172.21.15.49:0/2811725046 shutdown_connections 2024-03-18T23:05:44.588 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:44.587+0000 7efc39087640 1 -- 172.21.15.49:0/2811725046 wait complete. 2024-03-18T23:05:44.588 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:44.587+0000 7efc39087640 1 Processor -- start 2024-03-18T23:05:44.588 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:44.588+0000 7efc39087640 1 -- start start 2024-03-18T23:05:44.590 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:44.588+0000 7efc39087640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efc341007c0 0x7efc3406a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:05:44.590 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:44.588+0000 7efc39087640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efc341019b0 0x7efc3419ec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:05:44.590 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:44.588+0000 7efc39087640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efc34102370 0x7efc341a11c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:05:44.590 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:44.588+0000 7efc39087640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7efc3410c440 con 0x7efc341007c0 2024-03-18T23:05:44.591 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:44.588+0000 7efc39087640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7efc3410c2c0 con 0x7efc34102370 2024-03-18T23:05:44.591 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:44.588+0000 7efc39087640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7efc3410c5c0 con 0x7efc341019b0 2024-03-18T23:05:44.591 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:44.588+0000 7efc32d9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efc341007c0 0x7efc3406a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:05:44.591 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:44.588+0000 7efc32d9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efc341007c0 0x7efc3406a9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:34870/0 (socket says 172.21.15.49:34870) 2024-03-18T23:05:44.591 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:44.588+0000 7efc32d9d640 1 -- 172.21.15.49:0/2986406225 learned_addr learned my addr 172.21.15.49:0/2986406225 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:05:44.591 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:44.588+0000 7efc3259c640 1 --2- 172.21.15.49:0/2986406225 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efc341019b0 0x7efc3419ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:05:44.591 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:44.588+0000 7efc3359e640 1 --2- 172.21.15.49:0/2986406225 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efc34102370 0x7efc341a11c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:05:44.591 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:44.588+0000 7efc32d9d640 1 -- 172.21.15.49:0/2986406225 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efc341019b0 msgr2=0x7efc3419ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:05:44.591 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:44.588+0000 7efc32d9d640 1 --2- 172.21.15.49:0/2986406225 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efc341019b0 0x7efc3419ec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:05:44.591 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:44.588+0000 7efc32d9d640 1 -- 172.21.15.49:0/2986406225 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efc34102370 msgr2=0x7efc341a11c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:05:44.591 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:44.588+0000 7efc32d9d640 1 --2- 172.21.15.49:0/2986406225 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efc34102370 0x7efc341a11c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:05:44.591 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:44.588+0000 7efc32d9d640 1 -- 172.21.15.49:0/2986406225 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc341a19b0 con 0x7efc341007c0 2024-03-18T23:05:44.591 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:44.588+0000 7efc3259c640 1 --2- 172.21.15.49:0/2986406225 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efc341019b0 0x7efc3419ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T23:05:44.591 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:44.588+0000 7efc3359e640 1 --2- 172.21.15.49:0/2986406225 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efc34102370 0x7efc341a11c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T23:05:44.591 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:44.588+0000 7efc32d9d640 1 --2- 172.21.15.49:0/2986406225 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efc341007c0 0x7efc3406a9c0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7efc2800b450 tx=0x7efc2800b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:05:44.592 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:44.589+0000 7efc1bfff640 1 -- 172.21.15.49:0/2986406225 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc28015020 con 0x7efc341007c0 2024-03-18T23:05:44.592 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:44.589+0000 7efc1bfff640 1 -- 172.21.15.49:0/2986406225 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efc28011d70 con 0x7efc341007c0 2024-03-18T23:05:44.592 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:44.589+0000 7efc39087640 1 -- 172.21.15.49:0/2986406225 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc341a1c40 con 0x7efc341007c0 2024-03-18T23:05:44.592 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:44.589+0000 7efc1bfff640 1 -- 172.21.15.49:0/2986406225 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc2800f340 con 0x7efc341007c0 2024-03-18T23:05:44.592 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:44.589+0000 7efc39087640 1 -- 172.21.15.49:0/2986406225 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc341a2160 con 0x7efc341007c0 2024-03-18T23:05:44.592 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:44.590+0000 7efc39087640 1 -- 172.21.15.49:0/2986406225 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc34107a80 con 0x7efc341007c0 2024-03-18T23:05:44.593 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:44.591+0000 7efc1bfff640 1 -- 172.21.15.49:0/2986406225 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7efc28018020 con 0x7efc341007c0 2024-03-18T23:05:44.593 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:44.591+0000 7efc1bfff640 1 --2- 172.21.15.49:0/2986406225 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7efc1c0a2830 0x7efc1c0a4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:05:44.593 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:44.592+0000 7efc3259c640 1 --2- 172.21.15.49:0/2986406225 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7efc1c0a2830 0x7efc1c0a4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:05:44.593 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:44.592+0000 7efc1bfff640 1 -- 172.21.15.49:0/2986406225 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7efc280ce0c0 con 0x7efc341007c0 2024-03-18T23:05:44.593 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:44.592+0000 7efc3259c640 1 --2- 172.21.15.49:0/2986406225 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7efc1c0a2830 0x7efc1c0a4cf0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7efc24006010 tx=0x7efc24000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:05:44.596 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:44.596+0000 7efc1bfff640 1 -- 172.21.15.49:0/2986406225 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7efc2801f020 con 0x7efc341007c0 2024-03-18T23:05:44.818 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:44.818+0000 7efc39087640 1 -- 172.21.15.49:0/2986406225 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efc3402d440 con 0x7efc341007c0 2024-03-18T23:05:44.819 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:44.819+0000 7efc1bfff640 1 -- 172.21.15.49:0/2986406225 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5524 (secure 0 0 0) 0x7efc2800d040 con 0x7efc341007c0 2024-03-18T23:05:44.819 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-18T23:05:44.819 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6835/3631301630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":3631301630},{"type":"v1","addr":"172.21.15.49:6835","nonce":3631301630}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6837/444305233","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":444305233},{"type":"v1","addr":"172.21.15.49:6837","nonce":444305233}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24389,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3766178603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3766178603},{"type":"v1","addr":"172.21.15.139:6833","nonce":3766178603}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/1702997173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":1702997173},{"type":"v1","addr":"172.21.15.175:6833","nonce":1702997173}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/1663212937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":1663212937},{"type":"v1","addr":"172.21.15.139:6835","nonce":1663212937}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24430,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/755474931","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":755474931},{"type":"v1","addr":"172.21.15.175:6835","nonce":755474931}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[]} 2024-03-18T23:05:44.820 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 7 2024-03-18T23:05:44.821 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:44.821+0000 7efc39087640 1 -- 172.21.15.49:0/2986406225 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7efc1c0a2830 msgr2=0x7efc1c0a4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:05:44.822 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:44.821+0000 7efc39087640 1 --2- 172.21.15.49:0/2986406225 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7efc1c0a2830 0x7efc1c0a4cf0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7efc24006010 tx=0x7efc24000bc0 comp rx=0 tx=0).stop 2024-03-18T23:05:44.822 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:44.821+0000 7efc39087640 1 -- 172.21.15.49:0/2986406225 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efc341007c0 msgr2=0x7efc3406a9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:05:44.822 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:44.821+0000 7efc39087640 1 --2- 172.21.15.49:0/2986406225 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efc341007c0 0x7efc3406a9c0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7efc2800b450 tx=0x7efc2800b910 comp rx=0 tx=0).stop 2024-03-18T23:05:44.822 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:44.821+0000 7efc39087640 1 -- 172.21.15.49:0/2986406225 shutdown_connections 2024-03-18T23:05:44.822 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:44.821+0000 7efc39087640 1 --2- 172.21.15.49:0/2986406225 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7efc1c0a2830 0x7efc1c0a4cf0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:05:44.822 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:44.821+0000 7efc39087640 1 --2- 172.21.15.49:0/2986406225 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efc34102370 0x7efc341a11c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:05:44.822 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:44.821+0000 7efc39087640 1 --2- 172.21.15.49:0/2986406225 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efc341019b0 0x7efc3419ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:05:44.822 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:44.821+0000 7efc39087640 1 --2- 172.21.15.49:0/2986406225 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efc341007c0 0x7efc3406a9c0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:05:44.822 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:44.821+0000 7efc39087640 1 -- 172.21.15.49:0/2986406225 >> 172.21.15.49:0/2986406225 conn(0x7efc340fbd50 msgr2=0x7efc34104c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:05:44.822 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:44.822+0000 7efc39087640 1 -- 172.21.15.49:0/2986406225 shutdown_connections 2024-03-18T23:05:44.822 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:44.822+0000 7efc39087640 1 -- 172.21.15.49:0/2986406225 wait complete. 2024-03-18T23:05:45.130 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:05:44 smithi049 ceph-mon[62991]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T23:05:45.130 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:05:44 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:05:45.130 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:05:44 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:05:45.130 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:05:44 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:05:45.131 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:05:44 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-18T23:05:45.131 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:05:44 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-18T23:05:45.131 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:05:44 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:05:45.131 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:05:44 smithi049 ceph-mon[62991]: Deploying daemon mds.c on smithi175 2024-03-18T23:05:45.131 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:05:44 smithi049 ceph-mon[62991]: mds.? [v2:172.21.15.175:6834/755474931,v1:172.21.15.175:6835/755474931] up:boot 2024-03-18T23:05:45.131 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:05:44 smithi049 ceph-mon[62991]: fsmap 6 up:standby 2024-03-18T23:05:45.131 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:05:44 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-18T23:05:45.131 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:05:44 smithi049 ceph-mon[62991]: from='client.? 172.21.15.49:0/2986406225' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T23:05:45.212 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:05:44 smithi175 ceph-mon[67352]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T23:05:45.212 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:05:44 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:05:45.212 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:05:44 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:05:45.212 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:05:44 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:05:45.212 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:05:44 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-18T23:05:45.212 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:05:44 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-18T23:05:45.212 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:05:44 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:05:45.212 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:05:44 smithi175 ceph-mon[67352]: Deploying daemon mds.c on smithi175 2024-03-18T23:05:45.212 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:05:44 smithi175 ceph-mon[67352]: mds.? [v2:172.21.15.175:6834/755474931,v1:172.21.15.175:6835/755474931] up:boot 2024-03-18T23:05:45.212 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:05:44 smithi175 ceph-mon[67352]: fsmap 6 up:standby 2024-03-18T23:05:45.212 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:05:44 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-18T23:05:45.213 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:05:44 smithi175 ceph-mon[67352]: from='client.? 172.21.15.49:0/2986406225' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T23:05:45.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:05:44 smithi139 ceph-mon[67282]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T23:05:45.316 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:05:44 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:05:45.316 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:05:44 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:05:45.316 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:05:44 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:05:45.316 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:05:44 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-18T23:05:45.316 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:05:44 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-18T23:05:45.316 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:05:44 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:05:45.316 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:05:44 smithi139 ceph-mon[67282]: Deploying daemon mds.c on smithi175 2024-03-18T23:05:45.316 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:05:44 smithi139 ceph-mon[67282]: mds.? [v2:172.21.15.175:6834/755474931,v1:172.21.15.175:6835/755474931] up:boot 2024-03-18T23:05:45.316 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:05:44 smithi139 ceph-mon[67282]: fsmap 6 up:standby 2024-03-18T23:05:45.316 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:05:44 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-18T23:05:45.316 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:05:44 smithi139 ceph-mon[67282]: from='client.? 172.21.15.49:0/2986406225' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T23:05:46.212 INFO:journalctl@ceph.mds.c.smithi175.stdout:Mar 18 23:05:46 smithi175 systemd[1]: Starting Ceph mds.c for c63591ea-e57a-11ee-95c9-87774f69a715... 2024-03-18T23:05:46.585 INFO:journalctl@ceph.mds.c.smithi175.stdout:Mar 18 23:05:46 smithi175 podman[98497]: 2024-03-18 23:05:46.172905538 +0000 UTC m=+0.017807573 image pull 2312c8e42d0069a2dbb2364bddb00264db4b800e7f18a7e7e4eb8f3ecab2ff1c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639 2024-03-18T23:05:46.586 INFO:journalctl@ceph.mds.c.smithi175.stdout:Mar 18 23:05:46 smithi175 podman[98497]: 2024-03-18 23:05:46.28218659 +0000 UTC m=+0.127088604 container create 1bbba6b309808fbcfb1d3c2ab36f5c9cbe70bc6bcb7509b430858c581c14fd6c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mds-c, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-03-18T23:05:46.586 INFO:journalctl@ceph.mds.c.smithi175.stdout:Mar 18 23:05:46 smithi175 podman[98497]: 2024-03-18 23:05:46.585754413 +0000 UTC m=+0.430656437 container init 1bbba6b309808fbcfb1d3c2ab36f5c9cbe70bc6bcb7509b430858c581c14fd6c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mds-c, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-03-18T23:05:46.833 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-18T23:05:46.954 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:46.952+0000 7f5438fb3640 1 -- 172.21.15.49:0/1629985239 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5434100280 msgr2=0x7f5434109e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:05:46.954 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:46.952+0000 7f5438fb3640 1 --2- 172.21.15.49:0/1629985239 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5434100280 0x7f5434109e30 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f542c009ca0 tx=0x7f542c02c0f0 comp rx=0 tx=0).stop 2024-03-18T23:05:46.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:46.952+0000 7f5438fb3640 1 -- 172.21.15.49:0/1629985239 shutdown_connections 2024-03-18T23:05:46.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:46.952+0000 7f5438fb3640 1 --2- 172.21.15.49:0/1629985239 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5434100280 0x7f5434109e30 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:05:46.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:46.952+0000 7f5438fb3640 1 --2- 172.21.15.49:0/1629985239 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f54340ff8c0 0x7f54340ffd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:05:46.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:46.952+0000 7f5438fb3640 1 --2- 172.21.15.49:0/1629985239 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5434102e50 0x7f5434103270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:05:46.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:46.952+0000 7f5438fb3640 1 -- 172.21.15.49:0/1629985239 >> 172.21.15.49:0/1629985239 conn(0x7f54340fb440 msgr2=0x7f54340fd880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:05:46.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:46.952+0000 7f5438fb3640 1 -- 172.21.15.49:0/1629985239 shutdown_connections 2024-03-18T23:05:46.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:46.952+0000 7f5438fb3640 1 -- 172.21.15.49:0/1629985239 wait complete. 2024-03-18T23:05:46.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:46.953+0000 7f5438fb3640 1 Processor -- start 2024-03-18T23:05:46.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:46.953+0000 7f5438fb3640 1 -- start start 2024-03-18T23:05:46.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:46.953+0000 7f5438fb3640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f54340ff8c0 0x7f5434197bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:05:46.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:46.953+0000 7f5438fb3640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5434100280 0x7f5434198100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:05:46.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:46.953+0000 7f5438fb3640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5434102e50 0x7f543419f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:05:46.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:46.953+0000 7f5438fb3640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f543410d4b0 con 0x7f5434100280 2024-03-18T23:05:46.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:46.953+0000 7f5438fb3640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f543410d330 con 0x7f54340ff8c0 2024-03-18T23:05:46.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:46.953+0000 7f5438fb3640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f543410d630 con 0x7f5434102e50 2024-03-18T23:05:46.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:46.953+0000 7f5431d9b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5434100280 0x7f5434198100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:05:46.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:46.953+0000 7f5431d9b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5434100280 0x7f5434198100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:34592/0 (socket says 172.21.15.49:34592) 2024-03-18T23:05:46.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:46.953+0000 7f5431d9b640 1 -- 172.21.15.49:0/330951524 learned_addr learned my addr 172.21.15.49:0/330951524 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:05:46.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:46.953+0000 7f543259c640 1 --2- 172.21.15.49:0/330951524 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f54340ff8c0 0x7f5434197bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:05:46.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:46.953+0000 7f5432d9d640 1 --2- 172.21.15.49:0/330951524 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5434102e50 0x7f543419f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:05:46.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:46.953+0000 7f5431d9b640 1 -- 172.21.15.49:0/330951524 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5434102e50 msgr2=0x7f543419f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:05:46.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:46.953+0000 7f5431d9b640 1 --2- 172.21.15.49:0/330951524 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5434102e50 0x7f543419f0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:05:46.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:46.953+0000 7f5431d9b640 1 -- 172.21.15.49:0/330951524 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f54340ff8c0 msgr2=0x7f5434197bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:05:46.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:46.953+0000 7f5431d9b640 1 --2- 172.21.15.49:0/330951524 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f54340ff8c0 0x7f5434197bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:05:46.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:46.953+0000 7f5431d9b640 1 -- 172.21.15.49:0/330951524 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f543419f740 con 0x7f5434100280 2024-03-18T23:05:46.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:46.953+0000 7f5431d9b640 1 --2- 172.21.15.49:0/330951524 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5434100280 0x7f5434198100 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f542400d950 tx=0x7f542400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:05:46.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:46.954+0000 7f542b7fe640 1 -- 172.21.15.49:0/330951524 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5424017030 con 0x7f5434100280 2024-03-18T23:05:46.957 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:46.954+0000 7f542b7fe640 1 -- 172.21.15.49:0/330951524 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5424004020 con 0x7f5434100280 2024-03-18T23:05:46.957 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:46.954+0000 7f5438fb3640 1 -- 172.21.15.49:0/330951524 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f543419fa30 con 0x7f5434100280 2024-03-18T23:05:46.957 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:46.954+0000 7f542b7fe640 1 -- 172.21.15.49:0/330951524 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54240047a0 con 0x7f5434100280 2024-03-18T23:05:46.957 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:46.954+0000 7f5438fb3640 1 -- 172.21.15.49:0/330951524 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f543419ff80 con 0x7f5434100280 2024-03-18T23:05:46.957 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:46.955+0000 7f5438fb3640 1 -- 172.21.15.49:0/330951524 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5434068a60 con 0x7f5434100280 2024-03-18T23:05:46.958 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:46.956+0000 7f542b7fe640 1 -- 172.21.15.49:0/330951524 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7f54240041d0 con 0x7f5434100280 2024-03-18T23:05:46.958 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:46.957+0000 7f542b7fe640 1 --2- 172.21.15.49:0/330951524 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f54200a2800 0x7f54200a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:05:46.958 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:46.957+0000 7f543259c640 1 --2- 172.21.15.49:0/330951524 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f54200a2800 0x7f54200a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:05:46.958 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:46.957+0000 7f542b7fe640 1 -- 172.21.15.49:0/330951524 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f54240cdf90 con 0x7f5434100280 2024-03-18T23:05:46.958 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:46.957+0000 7f543259c640 1 --2- 172.21.15.49:0/330951524 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f54200a2800 0x7f54200a4cc0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f5434069bd0 tx=0x7f541c00a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:05:46.961 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:46.961+0000 7f542b7fe640 1 -- 172.21.15.49:0/330951524 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f542409b4d0 con 0x7f5434100280 2024-03-18T23:05:46.961 INFO:journalctl@ceph.mds.c.smithi175.stdout:Mar 18 23:05:46 smithi175 podman[98497]: 2024-03-18 23:05:46.588314452 +0000 UTC m=+0.433216475 container start 1bbba6b309808fbcfb1d3c2ab36f5c9cbe70bc6bcb7509b430858c581c14fd6c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mds-c, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-03-18T23:05:46.962 INFO:journalctl@ceph.mds.c.smithi175.stdout:Mar 18 23:05:46 smithi175 ceph-mds[98516]: -- 172.21.15.175:0/3224559469 <== mon.1 v2:172.21.15.175:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55c044ff3a40 con 0x55c044fc7000 2024-03-18T23:05:46.962 INFO:journalctl@ceph.mds.c.smithi175.stdout:Mar 18 23:05:46 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mds-c[98512]: starting mds.c at 2024-03-18T23:05:46.962 INFO:journalctl@ceph.mds.c.smithi175.stdout:Mar 18 23:05:46 smithi175 bash[98497]: 1bbba6b309808fbcfb1d3c2ab36f5c9cbe70bc6bcb7509b430858c581c14fd6c 2024-03-18T23:05:46.962 INFO:journalctl@ceph.mds.c.smithi175.stdout:Mar 18 23:05:46 smithi175 systemd[1]: Started Ceph mds.c for c63591ea-e57a-11ee-95c9-87774f69a715. 2024-03-18T23:05:47.185 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:47.185+0000 7f5438fb3640 1 -- 172.21.15.49:0/330951524 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f543402d220 con 0x7f5434100280 2024-03-18T23:05:47.186 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:47.186+0000 7f542b7fe640 1 -- 172.21.15.49:0/330951524 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6349 (secure 0 0 0) 0x7f542409add0 con 0x7f5434100280 2024-03-18T23:05:47.186 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-18T23:05:47.186 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6835/3631301630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":3631301630},{"type":"v1","addr":"172.21.15.49:6835","nonce":3631301630}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6837/444305233","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":444305233},{"type":"v1","addr":"172.21.15.49:6837","nonce":444305233}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24389,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3766178603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3766178603},{"type":"v1","addr":"172.21.15.139:6833","nonce":3766178603}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/1702997173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":1702997173},{"type":"v1","addr":"172.21.15.175:6833","nonce":1702997173}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/1663212937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":1663212937},{"type":"v1","addr":"172.21.15.139:6835","nonce":1663212937}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24430,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/755474931","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":755474931},{"type":"v1","addr":"172.21.15.175:6835","nonce":755474931}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24439,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/379482756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":379482756},{"type":"v1","addr":"172.21.15.175:6837","nonce":379482756}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-03-18T23:05:47.187 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 8 2024-03-18T23:05:47.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:47.188+0000 7f5438fb3640 1 -- 172.21.15.49:0/330951524 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f54200a2800 msgr2=0x7f54200a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:05:47.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:47.188+0000 7f5438fb3640 1 --2- 172.21.15.49:0/330951524 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f54200a2800 0x7f54200a4cc0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f5434069bd0 tx=0x7f541c00a2d0 comp rx=0 tx=0).stop 2024-03-18T23:05:47.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:47.188+0000 7f5438fb3640 1 -- 172.21.15.49:0/330951524 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5434100280 msgr2=0x7f5434198100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:05:47.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:47.188+0000 7f5438fb3640 1 --2- 172.21.15.49:0/330951524 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5434100280 0x7f5434198100 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f542400d950 tx=0x7f542400de10 comp rx=0 tx=0).stop 2024-03-18T23:05:47.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:47.188+0000 7f5438fb3640 1 -- 172.21.15.49:0/330951524 shutdown_connections 2024-03-18T23:05:47.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:47.188+0000 7f5438fb3640 1 --2- 172.21.15.49:0/330951524 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f54200a2800 0x7f54200a4cc0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:05:47.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:47.188+0000 7f5438fb3640 1 --2- 172.21.15.49:0/330951524 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5434102e50 0x7f543419f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:05:47.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:47.188+0000 7f5438fb3640 1 --2- 172.21.15.49:0/330951524 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5434100280 0x7f5434198100 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:05:47.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:47.188+0000 7f5438fb3640 1 --2- 172.21.15.49:0/330951524 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f54340ff8c0 0x7f5434197bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:05:47.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:47.188+0000 7f5438fb3640 1 -- 172.21.15.49:0/330951524 >> 172.21.15.49:0/330951524 conn(0x7f54340fb440 msgr2=0x7f5434105e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:05:47.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:47.188+0000 7f5438fb3640 1 -- 172.21.15.49:0/330951524 shutdown_connections 2024-03-18T23:05:47.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:47.188+0000 7f5438fb3640 1 -- 172.21.15.49:0/330951524 wait complete. 2024-03-18T23:05:48.065 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:05:47 smithi139 ceph-mon[67282]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T23:05:48.065 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:05:47 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:05:48.065 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:05:47 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:05:48.065 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:05:47 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:05:48.065 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:05:47 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-18T23:05:48.065 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:05:47 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-18T23:05:48.066 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:05:47 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:05:48.066 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:05:47 smithi139 ceph-mon[67282]: mds.? [v2:172.21.15.175:6836/379482756,v1:172.21.15.175:6837/379482756] up:boot 2024-03-18T23:05:48.066 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:05:47 smithi139 ceph-mon[67282]: fsmap 7 up:standby 2024-03-18T23:05:48.066 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:05:47 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-18T23:05:48.066 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:05:47 smithi139 ceph-mon[67282]: from='client.? 172.21.15.49:0/330951524' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T23:05:48.130 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:05:47 smithi049 ceph-mon[62991]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T23:05:48.130 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:05:47 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:05:48.130 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:05:47 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:05:48.130 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:05:47 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:05:48.131 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:05:47 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-18T23:05:48.131 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:05:47 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-18T23:05:48.131 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:05:47 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:05:48.131 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:05:47 smithi049 ceph-mon[62991]: mds.? [v2:172.21.15.175:6836/379482756,v1:172.21.15.175:6837/379482756] up:boot 2024-03-18T23:05:48.131 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:05:47 smithi049 ceph-mon[62991]: fsmap 7 up:standby 2024-03-18T23:05:48.131 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:05:47 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-18T23:05:48.131 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:05:47 smithi049 ceph-mon[62991]: from='client.? 172.21.15.49:0/330951524' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T23:05:48.211 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:05:47 smithi175 ceph-mon[67352]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T23:05:48.211 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:05:47 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:05:48.212 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:05:47 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:05:48.212 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:05:47 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:05:48.212 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:05:47 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-18T23:05:48.212 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:05:47 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-18T23:05:48.212 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:05:47 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:05:48.212 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:05:47 smithi175 ceph-mon[67352]: mds.? [v2:172.21.15.175:6836/379482756,v1:172.21.15.175:6837/379482756] up:boot 2024-03-18T23:05:48.212 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:05:47 smithi175 ceph-mon[67352]: fsmap 7 up:standby 2024-03-18T23:05:48.212 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:05:47 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-18T23:05:48.212 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:05:47 smithi175 ceph-mon[67352]: from='client.? 172.21.15.49:0/330951524' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T23:05:49.046 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:05:48 smithi139 ceph-mon[67282]: Deploying daemon mds.h on smithi139 2024-03-18T23:05:49.130 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:05:48 smithi049 ceph-mon[62991]: Deploying daemon mds.h on smithi139 2024-03-18T23:05:49.201 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-18T23:05:49.211 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:05:48 smithi175 ceph-mon[67352]: Deploying daemon mds.h on smithi139 2024-03-18T23:05:49.315 INFO:journalctl@ceph.mds.h.smithi139.stdout:Mar 18 23:05:49 smithi139 podman[97794]: 2024-03-18 23:05:49.283516849 +0000 UTC m=+0.137096629 container create d7e908d6f0324b5c47ae347bcd61f9fba75f2019756c6d99ae067ba7602df7f3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mds-h, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-18T23:05:49.323 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:49.320+0000 7fdf5d590640 1 -- 172.21.15.49:0/1237368853 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdf58102370 msgr2=0x7fdf58108c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:05:49.323 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:49.320+0000 7fdf5d590640 1 --2- 172.21.15.49:0/1237368853 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdf58102370 0x7fdf58108c00 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fdf50009ca0 tx=0x7fdf5002c0f0 comp rx=0 tx=0).stop 2024-03-18T23:05:49.323 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:49.320+0000 7fdf5d590640 1 -- 172.21.15.49:0/1237368853 shutdown_connections 2024-03-18T23:05:49.324 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:49.320+0000 7fdf5d590640 1 --2- 172.21.15.49:0/1237368853 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdf58102370 0x7fdf58108c00 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:05:49.324 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:49.320+0000 7fdf5d590640 1 --2- 172.21.15.49:0/1237368853 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdf581019b0 0x7fdf58101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:05:49.324 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:49.320+0000 7fdf5d590640 1 --2- 172.21.15.49:0/1237368853 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdf581007c0 0x7fdf58100be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:05:49.324 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:49.320+0000 7fdf5d590640 1 -- 172.21.15.49:0/1237368853 >> 172.21.15.49:0/1237368853 conn(0x7fdf580fbd50 msgr2=0x7fdf580fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:05:49.324 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:49.320+0000 7fdf5d590640 1 -- 172.21.15.49:0/1237368853 shutdown_connections 2024-03-18T23:05:49.324 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:49.320+0000 7fdf5d590640 1 -- 172.21.15.49:0/1237368853 wait complete. 2024-03-18T23:05:49.324 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:49.321+0000 7fdf5d590640 1 Processor -- start 2024-03-18T23:05:49.324 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:49.321+0000 7fdf5d590640 1 -- start start 2024-03-18T23:05:49.324 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:49.321+0000 7fdf5d590640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdf581007c0 0x7fdf5806a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:05:49.324 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:49.321+0000 7fdf5d590640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdf581019b0 0x7fdf5819c830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:05:49.324 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:49.321+0000 7fdf5d590640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdf58102370 0x7fdf5819ee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:05:49.324 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:49.321+0000 7fdf5d590640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fdf5810c3d0 con 0x7fdf581019b0 2024-03-18T23:05:49.325 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:49.321+0000 7fdf5d590640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fdf5810c250 con 0x7fdf58102370 2024-03-18T23:05:49.325 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:49.321+0000 7fdf5d590640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fdf5810c550 con 0x7fdf581007c0 2024-03-18T23:05:49.325 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:49.321+0000 7fdf4ffff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdf581019b0 0x7fdf5819c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:05:49.325 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:49.321+0000 7fdf4ffff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdf581019b0 0x7fdf5819c830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:34608/0 (socket says 172.21.15.49:34608) 2024-03-18T23:05:49.325 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:49.321+0000 7fdf4ffff640 1 -- 172.21.15.49:0/592858167 learned_addr learned my addr 172.21.15.49:0/592858167 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:05:49.325 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:49.321+0000 7fdf56ffd640 1 --2- 172.21.15.49:0/592858167 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdf581007c0 0x7fdf5806a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:05:49.325 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:49.321+0000 7fdf577fe640 1 --2- 172.21.15.49:0/592858167 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdf58102370 0x7fdf5819ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:05:49.325 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:49.322+0000 7fdf4ffff640 1 -- 172.21.15.49:0/592858167 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdf581007c0 msgr2=0x7fdf5806a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:05:49.325 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:49.322+0000 7fdf4ffff640 1 --2- 172.21.15.49:0/592858167 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdf581007c0 0x7fdf5806a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:05:49.325 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:49.322+0000 7fdf4ffff640 1 -- 172.21.15.49:0/592858167 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdf58102370 msgr2=0x7fdf5819ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:05:49.325 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:49.322+0000 7fdf4ffff640 1 --2- 172.21.15.49:0/592858167 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdf58102370 0x7fdf5819ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:05:49.325 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:49.322+0000 7fdf4ffff640 1 -- 172.21.15.49:0/592858167 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf5819f760 con 0x7fdf581019b0 2024-03-18T23:05:49.325 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:49.322+0000 7fdf56ffd640 1 --2- 172.21.15.49:0/592858167 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdf581007c0 0x7fdf5806a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T23:05:49.325 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:49.322+0000 7fdf4ffff640 1 --2- 172.21.15.49:0/592858167 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdf581019b0 0x7fdf5819c830 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fdf4000d770 tx=0x7fdf4000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:05:49.326 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:49.322+0000 7fdf54ff9640 1 -- 172.21.15.49:0/592858167 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf40017070 con 0x7fdf581019b0 2024-03-18T23:05:49.326 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:49.322+0000 7fdf54ff9640 1 -- 172.21.15.49:0/592858167 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf40005500 con 0x7fdf581019b0 2024-03-18T23:05:49.326 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:49.322+0000 7fdf5d590640 1 -- 172.21.15.49:0/592858167 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf5819f9f0 con 0x7fdf581019b0 2024-03-18T23:05:49.326 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:49.322+0000 7fdf54ff9640 1 -- 172.21.15.49:0/592858167 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf40004e00 con 0x7fdf581019b0 2024-03-18T23:05:49.326 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:49.322+0000 7fdf5d590640 1 -- 172.21.15.49:0/592858167 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf5819fef0 con 0x7fdf581019b0 2024-03-18T23:05:49.327 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:49.323+0000 7fdf5d590640 1 -- 172.21.15.49:0/592858167 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf58107a80 con 0x7fdf581019b0 2024-03-18T23:05:49.327 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:49.324+0000 7fdf54ff9640 1 -- 172.21.15.49:0/592858167 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7fdf40004fa0 con 0x7fdf581019b0 2024-03-18T23:05:49.327 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:49.325+0000 7fdf54ff9640 1 --2- 172.21.15.49:0/592858167 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fdf440a2800 0x7fdf440a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:05:49.327 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:49.325+0000 7fdf56ffd640 1 --2- 172.21.15.49:0/592858167 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fdf440a2800 0x7fdf440a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:05:49.327 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:49.325+0000 7fdf54ff9640 1 -- 172.21.15.49:0/592858167 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fdf400ce0b0 con 0x7fdf581019b0 2024-03-18T23:05:49.327 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:49.325+0000 7fdf56ffd640 1 --2- 172.21.15.49:0/592858167 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fdf440a2800 0x7fdf440a4cc0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fdf58101810 tx=0x7fdf48005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:05:49.330 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:49.329+0000 7fdf54ff9640 1 -- 172.21.15.49:0/592858167 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fdf4009b5f0 con 0x7fdf581019b0 2024-03-18T23:05:49.554 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:49.553+0000 7fdf5d590640 1 -- 172.21.15.49:0/592858167 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdf5802d3f0 con 0x7fdf581019b0 2024-03-18T23:05:49.555 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:49.554+0000 7fdf54ff9640 1 -- 172.21.15.49:0/592858167 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6349 (secure 0 0 0) 0x7fdf4009aef0 con 0x7fdf581019b0 2024-03-18T23:05:49.555 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-18T23:05:49.555 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6835/3631301630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":3631301630},{"type":"v1","addr":"172.21.15.49:6835","nonce":3631301630}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6837/444305233","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":444305233},{"type":"v1","addr":"172.21.15.49:6837","nonce":444305233}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24389,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3766178603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3766178603},{"type":"v1","addr":"172.21.15.139:6833","nonce":3766178603}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/1702997173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":1702997173},{"type":"v1","addr":"172.21.15.175:6833","nonce":1702997173}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/1663212937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":1663212937},{"type":"v1","addr":"172.21.15.139:6835","nonce":1663212937}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24430,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/755474931","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":755474931},{"type":"v1","addr":"172.21.15.175:6835","nonce":755474931}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24439,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/379482756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":379482756},{"type":"v1","addr":"172.21.15.175:6837","nonce":379482756}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-03-18T23:05:49.555 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 8 2024-03-18T23:05:49.557 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:49.556+0000 7fdf5d590640 1 -- 172.21.15.49:0/592858167 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fdf440a2800 msgr2=0x7fdf440a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:05:49.557 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:49.556+0000 7fdf5d590640 1 --2- 172.21.15.49:0/592858167 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fdf440a2800 0x7fdf440a4cc0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fdf58101810 tx=0x7fdf48005f90 comp rx=0 tx=0).stop 2024-03-18T23:05:49.557 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:49.556+0000 7fdf5d590640 1 -- 172.21.15.49:0/592858167 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdf581019b0 msgr2=0x7fdf5819c830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:05:49.557 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:49.556+0000 7fdf5d590640 1 --2- 172.21.15.49:0/592858167 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdf581019b0 0x7fdf5819c830 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fdf4000d770 tx=0x7fdf4000dc30 comp rx=0 tx=0).stop 2024-03-18T23:05:49.558 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:49.556+0000 7fdf5d590640 1 -- 172.21.15.49:0/592858167 shutdown_connections 2024-03-18T23:05:49.558 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:49.556+0000 7fdf5d590640 1 --2- 172.21.15.49:0/592858167 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fdf440a2800 0x7fdf440a4cc0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:05:49.558 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:49.556+0000 7fdf5d590640 1 --2- 172.21.15.49:0/592858167 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdf58102370 0x7fdf5819ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:05:49.558 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:49.556+0000 7fdf5d590640 1 --2- 172.21.15.49:0/592858167 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdf581019b0 0x7fdf5819c830 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:05:49.558 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:49.556+0000 7fdf5d590640 1 --2- 172.21.15.49:0/592858167 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdf581007c0 0x7fdf5806a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:05:49.558 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:49.556+0000 7fdf5d590640 1 -- 172.21.15.49:0/592858167 >> 172.21.15.49:0/592858167 conn(0x7fdf580fbd50 msgr2=0x7fdf58104c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:05:49.558 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:49.556+0000 7fdf5d590640 1 -- 172.21.15.49:0/592858167 shutdown_connections 2024-03-18T23:05:49.558 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:49.556+0000 7fdf5d590640 1 -- 172.21.15.49:0/592858167 wait complete. 2024-03-18T23:05:49.740 INFO:journalctl@ceph.mds.h.smithi139.stdout:Mar 18 23:05:49 smithi139 podman[97794]: 2024-03-18 23:05:49.545011212 +0000 UTC m=+0.398591000 container init d7e908d6f0324b5c47ae347bcd61f9fba75f2019756c6d99ae067ba7602df7f3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mds-h, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-03-18T23:05:49.740 INFO:journalctl@ceph.mds.h.smithi139.stdout:Mar 18 23:05:49 smithi139 podman[97794]: 2024-03-18 23:05:49.547900928 +0000 UTC m=+0.401480705 container start d7e908d6f0324b5c47ae347bcd61f9fba75f2019756c6d99ae067ba7602df7f3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mds-h, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-03-18T23:05:49.740 INFO:journalctl@ceph.mds.h.smithi139.stdout:Mar 18 23:05:49 smithi139 ceph-mds[97813]: -- 172.21.15.139:0/2438988282 <== mon.2 v2:172.21.15.139:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55bbdecd7a40 con 0x55bbdecab000 2024-03-18T23:05:49.740 INFO:journalctl@ceph.mds.h.smithi139.stdout:Mar 18 23:05:49 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mds-h[97809]: starting mds.h at 2024-03-18T23:05:49.740 INFO:journalctl@ceph.mds.h.smithi139.stdout:Mar 18 23:05:49 smithi139 bash[97794]: d7e908d6f0324b5c47ae347bcd61f9fba75f2019756c6d99ae067ba7602df7f3 2024-03-18T23:05:49.740 INFO:journalctl@ceph.mds.h.smithi139.stdout:Mar 18 23:05:49 smithi139 systemd[1]: Started Ceph mds.h for c63591ea-e57a-11ee-95c9-87774f69a715. 2024-03-18T23:05:49.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:05:49 smithi049 ceph-mon[62991]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T23:05:49.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:05:49 smithi049 ceph-mon[62991]: from='client.? 172.21.15.49:0/592858167' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T23:05:49.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:05:49 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:05:49.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:05:49 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:05:49.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:05:49 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:05:49.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:05:49 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-18T23:05:49.881 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:05:49 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-18T23:05:49.881 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:05:49 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:05:50.065 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:05:49 smithi139 ceph-mon[67282]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T23:05:50.065 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:05:49 smithi139 ceph-mon[67282]: from='client.? 172.21.15.49:0/592858167' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T23:05:50.065 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:05:49 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:05:50.065 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:05:49 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:05:50.065 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:05:49 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:05:50.065 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:05:49 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-18T23:05:50.065 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:05:49 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-18T23:05:50.065 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:05:49 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:05:50.211 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:05:49 smithi175 ceph-mon[67352]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T23:05:50.211 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:05:49 smithi175 ceph-mon[67352]: from='client.? 172.21.15.49:0/592858167' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T23:05:50.211 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:05:49 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:05:50.211 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:05:49 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:05:50.211 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:05:49 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:05:50.211 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:05:49 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-18T23:05:50.212 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:05:49 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-18T23:05:50.212 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:05:49 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:05:50.878 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:05:50 smithi175 ceph-mon[67352]: Deploying daemon mds.i on smithi175 2024-03-18T23:05:50.878 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:05:50 smithi175 ceph-mon[67352]: mds.? [v2:172.21.15.139:6836/2330317540,v1:172.21.15.139:6837/2330317540] up:boot 2024-03-18T23:05:50.878 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:05:50 smithi175 ceph-mon[67352]: fsmap 8 up:standby 2024-03-18T23:05:50.878 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:05:50 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-18T23:05:51.065 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:05:50 smithi139 ceph-mon[67282]: Deploying daemon mds.i on smithi175 2024-03-18T23:05:51.065 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:05:50 smithi139 ceph-mon[67282]: mds.? [v2:172.21.15.139:6836/2330317540,v1:172.21.15.139:6837/2330317540] up:boot 2024-03-18T23:05:51.065 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:05:50 smithi139 ceph-mon[67282]: fsmap 8 up:standby 2024-03-18T23:05:51.065 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:05:50 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-18T23:05:51.130 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:05:50 smithi049 ceph-mon[62991]: Deploying daemon mds.i on smithi175 2024-03-18T23:05:51.130 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:05:50 smithi049 ceph-mon[62991]: mds.? [v2:172.21.15.139:6836/2330317540,v1:172.21.15.139:6837/2330317540] up:boot 2024-03-18T23:05:51.130 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:05:50 smithi049 ceph-mon[62991]: fsmap 8 up:standby 2024-03-18T23:05:51.130 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:05:50 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-18T23:05:51.568 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-18T23:05:51.691 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:51.688+0000 7ff16af9e640 1 -- 172.21.15.49:0/846907903 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff1641007c0 msgr2=0x7ff164100be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:05:51.692 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:51.688+0000 7ff16af9e640 1 --2- 172.21.15.49:0/846907903 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff1641007c0 0x7ff164100be0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7ff158009900 tx=0x7ff15802bc30 comp rx=0 tx=0).stop 2024-03-18T23:05:51.692 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:51.688+0000 7ff16af9e640 1 -- 172.21.15.49:0/846907903 shutdown_connections 2024-03-18T23:05:51.692 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:51.688+0000 7ff16af9e640 1 --2- 172.21.15.49:0/846907903 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff164102370 0x7ff164108c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:05:51.692 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:51.688+0000 7ff16af9e640 1 --2- 172.21.15.49:0/846907903 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff1641019b0 0x7ff164101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:05:51.692 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:51.688+0000 7ff16af9e640 1 --2- 172.21.15.49:0/846907903 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff1641007c0 0x7ff164100be0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:05:51.692 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:51.688+0000 7ff16af9e640 1 -- 172.21.15.49:0/846907903 >> 172.21.15.49:0/846907903 conn(0x7ff1640fbd50 msgr2=0x7ff1640fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:05:51.692 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:51.688+0000 7ff16af9e640 1 -- 172.21.15.49:0/846907903 shutdown_connections 2024-03-18T23:05:51.692 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:51.688+0000 7ff16af9e640 1 -- 172.21.15.49:0/846907903 wait complete. 2024-03-18T23:05:51.692 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:51.689+0000 7ff16af9e640 1 Processor -- start 2024-03-18T23:05:51.692 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:51.689+0000 7ff16af9e640 1 -- start start 2024-03-18T23:05:51.692 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:51.689+0000 7ff16af9e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff1641007c0 0x7ff16406a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:05:51.692 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:51.689+0000 7ff16af9e640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff1641019b0 0x7ff16419ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:05:51.692 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:51.689+0000 7ff16af9e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff164102370 0x7ff1641a1230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:05:51.692 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:51.689+0000 7ff16af9e640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7ff16410c3f0 con 0x7ff164102370 2024-03-18T23:05:51.693 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:51.689+0000 7ff16af9e640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7ff16410c270 con 0x7ff1641007c0 2024-03-18T23:05:51.693 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:51.689+0000 7ff16af9e640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff16410c570 con 0x7ff1641019b0 2024-03-18T23:05:51.693 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:51.689+0000 7ff16953b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff164102370 0x7ff1641a1230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:05:51.693 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:51.689+0000 7ff16953b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff164102370 0x7ff1641a1230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:34628/0 (socket says 172.21.15.49:34628) 2024-03-18T23:05:51.693 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:51.689+0000 7ff16953b640 1 -- 172.21.15.49:0/820774725 learned_addr learned my addr 172.21.15.49:0/820774725 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:05:51.693 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:51.689+0000 7ff163fff640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff1641019b0 0x7ff16419ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:05:51.693 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:51.689+0000 7ff168d3a640 1 --2- 172.21.15.49:0/820774725 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff1641007c0 0x7ff16406a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:05:51.693 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:51.689+0000 7ff16953b640 1 -- 172.21.15.49:0/820774725 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff1641019b0 msgr2=0x7ff16419ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:05:51.693 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:51.689+0000 7ff16953b640 1 --2- 172.21.15.49:0/820774725 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff1641019b0 0x7ff16419ece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:05:51.693 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:51.689+0000 7ff16953b640 1 -- 172.21.15.49:0/820774725 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff1641007c0 msgr2=0x7ff16406a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:05:51.693 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:51.689+0000 7ff16953b640 1 --2- 172.21.15.49:0/820774725 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff1641007c0 0x7ff16406a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:05:51.693 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:51.689+0000 7ff16953b640 1 -- 172.21.15.49:0/820774725 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1641a1a20 con 0x7ff164102370 2024-03-18T23:05:51.693 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:51.690+0000 7ff16953b640 1 --2- 172.21.15.49:0/820774725 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff164102370 0x7ff1641a1230 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7ff15c009fd0 tx=0x7ff15c00b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:05:51.693 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:51.690+0000 7ff161ffb640 1 -- 172.21.15.49:0/820774725 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff15c017070 con 0x7ff164102370 2024-03-18T23:05:51.694 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:51.690+0000 7ff16af9e640 1 -- 172.21.15.49:0/820774725 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1641a1cb0 con 0x7ff164102370 2024-03-18T23:05:51.694 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:51.690+0000 7ff161ffb640 1 -- 172.21.15.49:0/820774725 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff15c003e50 con 0x7ff164102370 2024-03-18T23:05:51.694 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:51.690+0000 7ff161ffb640 1 -- 172.21.15.49:0/820774725 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff15c0106c0 con 0x7ff164102370 2024-03-18T23:05:51.694 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:51.690+0000 7ff16af9e640 1 -- 172.21.15.49:0/820774725 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1641a21b0 con 0x7ff164102370 2024-03-18T23:05:51.695 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:51.691+0000 7ff16af9e640 1 -- 172.21.15.49:0/820774725 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff164107ae0 con 0x7ff164102370 2024-03-18T23:05:51.695 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:51.692+0000 7ff161ffb640 1 -- 172.21.15.49:0/820774725 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7ff15c003970 con 0x7ff164102370 2024-03-18T23:05:51.695 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:51.692+0000 7ff161ffb640 1 --2- 172.21.15.49:0/820774725 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7ff1540a2830 0x7ff1540a4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:05:51.695 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:51.693+0000 7ff168d3a640 1 --2- 172.21.15.49:0/820774725 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7ff1540a2830 0x7ff1540a4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:05:51.695 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:51.693+0000 7ff161ffb640 1 -- 172.21.15.49:0/820774725 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7ff15c0ce290 con 0x7ff164102370 2024-03-18T23:05:51.695 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:51.693+0000 7ff168d3a640 1 --2- 172.21.15.49:0/820774725 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7ff1540a2830 0x7ff1540a4cf0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7ff15802c170 tx=0x7ff15802df90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:05:51.698 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:51.697+0000 7ff161ffb640 1 -- 172.21.15.49:0/820774725 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7ff15c09b180 con 0x7ff164102370 2024-03-18T23:05:51.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:05:51 smithi049 ceph-mon[62991]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T23:05:51.924 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:51.923+0000 7ff16af9e640 1 -- 172.21.15.49:0/820774725 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff164068a60 con 0x7ff164102370 2024-03-18T23:05:51.925 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:51.924+0000 7ff161ffb640 1 -- 172.21.15.49:0/820774725 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7177 (secure 0 0 0) 0x7ff15c09f220 con 0x7ff164102370 2024-03-18T23:05:51.925 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-18T23:05:51.925 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6835/3631301630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":3631301630},{"type":"v1","addr":"172.21.15.49:6835","nonce":3631301630}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6837/444305233","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":444305233},{"type":"v1","addr":"172.21.15.49:6837","nonce":444305233}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24389,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3766178603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3766178603},{"type":"v1","addr":"172.21.15.139:6833","nonce":3766178603}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/1702997173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":1702997173},{"type":"v1","addr":"172.21.15.175:6833","nonce":1702997173}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/1663212937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":1663212937},{"type":"v1","addr":"172.21.15.139:6835","nonce":1663212937}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24430,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/755474931","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":755474931},{"type":"v1","addr":"172.21.15.175:6835","nonce":755474931}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24439,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/379482756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":379482756},{"type":"v1","addr":"172.21.15.175:6837","nonce":379482756}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24452,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6837/2330317540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6836","nonce":2330317540},{"type":"v1","addr":"172.21.15.139:6837","nonce":2330317540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-03-18T23:05:51.925 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 9 2024-03-18T23:05:51.927 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:51.926+0000 7ff16af9e640 1 -- 172.21.15.49:0/820774725 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7ff1540a2830 msgr2=0x7ff1540a4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:05:51.927 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:51.926+0000 7ff16af9e640 1 --2- 172.21.15.49:0/820774725 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7ff1540a2830 0x7ff1540a4cf0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7ff15802c170 tx=0x7ff15802df90 comp rx=0 tx=0).stop 2024-03-18T23:05:51.927 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:51.926+0000 7ff16af9e640 1 -- 172.21.15.49:0/820774725 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff164102370 msgr2=0x7ff1641a1230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:05:51.927 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:51.926+0000 7ff16af9e640 1 --2- 172.21.15.49:0/820774725 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff164102370 0x7ff1641a1230 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7ff15c009fd0 tx=0x7ff15c00b260 comp rx=0 tx=0).stop 2024-03-18T23:05:51.927 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:51.926+0000 7ff16af9e640 1 -- 172.21.15.49:0/820774725 shutdown_connections 2024-03-18T23:05:51.927 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:51.926+0000 7ff16af9e640 1 --2- 172.21.15.49:0/820774725 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7ff1540a2830 0x7ff1540a4cf0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:05:51.927 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:51.926+0000 7ff16af9e640 1 --2- 172.21.15.49:0/820774725 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff164102370 0x7ff1641a1230 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:05:51.927 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:51.926+0000 7ff16af9e640 1 --2- 172.21.15.49:0/820774725 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff1641019b0 0x7ff16419ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:05:51.928 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:51.926+0000 7ff16af9e640 1 --2- 172.21.15.49:0/820774725 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff1641007c0 0x7ff16406a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:05:51.928 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:51.926+0000 7ff16af9e640 1 -- 172.21.15.49:0/820774725 >> 172.21.15.49:0/820774725 conn(0x7ff1640fbd50 msgr2=0x7ff164104cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:05:51.928 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:51.926+0000 7ff16af9e640 1 -- 172.21.15.49:0/820774725 shutdown_connections 2024-03-18T23:05:51.928 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:51.926+0000 7ff16af9e640 1 -- 172.21.15.49:0/820774725 wait complete. 2024-03-18T23:05:51.961 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:05:51 smithi175 ceph-mon[67352]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T23:05:51.962 INFO:journalctl@ceph.mds.i.smithi175.stdout:Mar 18 23:05:51 smithi175 systemd[1]: Starting Ceph mds.i for c63591ea-e57a-11ee-95c9-87774f69a715... 2024-03-18T23:05:52.065 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:05:51 smithi139 ceph-mon[67282]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T23:05:52.307 INFO:journalctl@ceph.mds.i.smithi175.stdout:Mar 18 23:05:52 smithi175 podman[98809]: 2024-03-18 23:05:51.941074116 +0000 UTC m=+0.017795374 image pull 2312c8e42d0069a2dbb2364bddb00264db4b800e7f18a7e7e4eb8f3ecab2ff1c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639 2024-03-18T23:05:52.307 INFO:journalctl@ceph.mds.i.smithi175.stdout:Mar 18 23:05:52 smithi175 podman[98809]: 2024-03-18 23:05:52.045847067 +0000 UTC m=+0.122568318 container create c9b962415920d774ae499323e32d191311f2ef10d7277ba5f771fd78b52a58ae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mds-i, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.build-date=20240307, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-18T23:05:52.307 INFO:journalctl@ceph.mds.i.smithi175.stdout:Mar 18 23:05:52 smithi175 podman[98809]: 2024-03-18 23:05:52.281906808 +0000 UTC m=+0.358628052 container init c9b962415920d774ae499323e32d191311f2ef10d7277ba5f771fd78b52a58ae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mds-i, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240307, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-03-18T23:05:52.308 INFO:journalctl@ceph.mds.i.smithi175.stdout:Mar 18 23:05:52 smithi175 podman[98809]: 2024-03-18 23:05:52.284556436 +0000 UTC m=+0.361277683 container start c9b962415920d774ae499323e32d191311f2ef10d7277ba5f771fd78b52a58ae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mds-i, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-03-18T23:05:52.711 INFO:journalctl@ceph.mds.i.smithi175.stdout:Mar 18 23:05:52 smithi175 ceph-mds[98828]: -- 172.21.15.175:0/3425337125 <== mon.1 v2:172.21.15.175:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5608f73a9a40 con 0x5608f737d000 2024-03-18T23:05:52.711 INFO:journalctl@ceph.mds.i.smithi175.stdout:Mar 18 23:05:52 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mds-i[98824]: starting mds.i at 2024-03-18T23:05:52.711 INFO:journalctl@ceph.mds.i.smithi175.stdout:Mar 18 23:05:52 smithi175 bash[98809]: c9b962415920d774ae499323e32d191311f2ef10d7277ba5f771fd78b52a58ae 2024-03-18T23:05:52.712 INFO:journalctl@ceph.mds.i.smithi175.stdout:Mar 18 23:05:52 smithi175 systemd[1]: Started Ceph mds.i for c63591ea-e57a-11ee-95c9-87774f69a715. 2024-03-18T23:05:53.065 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:05:52 smithi139 ceph-mon[67282]: from='client.? 172.21.15.49:0/820774725' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T23:05:53.065 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:05:52 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:05:53.065 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:05:52 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:05:53.065 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:05:52 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:05:53.065 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:05:52 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-18T23:05:53.065 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:05:52 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-18T23:05:53.066 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:05:52 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:05:53.083 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:05:52 smithi049 ceph-mon[62991]: from='client.? 172.21.15.49:0/820774725' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T23:05:53.083 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:05:52 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:05:53.083 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:05:52 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:05:53.083 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:05:52 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:05:53.084 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:05:52 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-18T23:05:53.084 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:05:52 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-18T23:05:53.084 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:05:52 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:05:53.211 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:05:52 smithi175 ceph-mon[67352]: from='client.? 172.21.15.49:0/820774725' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T23:05:53.212 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:05:52 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:05:53.212 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:05:52 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:05:53.212 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:05:52 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:05:53.212 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:05:52 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-18T23:05:53.212 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:05:52 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-18T23:05:53.212 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:05:52 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:05:53.939 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-18T23:05:53.969 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:05:53 smithi049 ceph-mon[62991]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T23:05:53.969 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:05:53 smithi049 ceph-mon[62991]: Deploying daemon mds.g on smithi049 2024-03-18T23:05:53.969 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:05:53 smithi049 ceph-mon[62991]: mds.? [v2:172.21.15.175:6838/3640882084,v1:172.21.15.175:6839/3640882084] up:boot 2024-03-18T23:05:53.969 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:05:53 smithi049 ceph-mon[62991]: fsmap 9 up:standby 2024-03-18T23:05:53.970 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:05:53 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-18T23:05:54.053 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:54.050+0000 7f1ef9285640 1 -- 172.21.15.49:0/431310022 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1ef41034a0 msgr2=0x7f1ef4109d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:05:54.053 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:54.050+0000 7f1ef9285640 1 --2- 172.21.15.49:0/431310022 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1ef41034a0 0x7f1ef4109d30 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f1eec009cd0 tx=0x7f1eec02c0f0 comp rx=0 tx=0).stop 2024-03-18T23:05:54.053 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:54.050+0000 7f1ef9285640 1 -- 172.21.15.49:0/431310022 shutdown_connections 2024-03-18T23:05:54.053 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:54.050+0000 7f1ef9285640 1 --2- 172.21.15.49:0/431310022 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1ef41034a0 0x7f1ef4109d30 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:05:54.054 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:54.050+0000 7f1ef9285640 1 --2- 172.21.15.49:0/431310022 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1ef4102ae0 0x7f1ef4102f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:05:54.054 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:54.050+0000 7f1ef9285640 1 --2- 172.21.15.49:0/431310022 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1ef40ffb10 0x7f1ef40fff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:05:54.054 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:54.050+0000 7f1ef9285640 1 -- 172.21.15.49:0/431310022 >> 172.21.15.49:0/431310022 conn(0x7f1ef40fb400 msgr2=0x7f1ef40fd880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:05:54.054 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:54.050+0000 7f1ef9285640 1 -- 172.21.15.49:0/431310022 shutdown_connections 2024-03-18T23:05:54.054 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:54.050+0000 7f1ef9285640 1 -- 172.21.15.49:0/431310022 wait complete. 2024-03-18T23:05:54.054 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:54.051+0000 7f1ef9285640 1 Processor -- start 2024-03-18T23:05:54.054 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:54.051+0000 7f1ef9285640 1 -- start start 2024-03-18T23:05:54.054 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:54.051+0000 7f1ef9285640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1ef40ffb10 0x7f1ef4197ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:05:54.054 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:54.051+0000 7f1ef9285640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1ef4102ae0 0x7f1ef4198020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:05:54.054 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:54.051+0000 7f1ef9285640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1ef41034a0 0x7f1ef419f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:05:54.054 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:54.051+0000 7f1ef9285640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f1ef410d4c0 con 0x7f1ef4102ae0 2024-03-18T23:05:54.054 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:54.051+0000 7f1ef9285640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f1ef410d340 con 0x7f1ef41034a0 2024-03-18T23:05:54.054 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:54.051+0000 7f1ef9285640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f1ef410d640 con 0x7f1ef40ffb10 2024-03-18T23:05:54.054 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:54.051+0000 7f1ef27fc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1ef4102ae0 0x7f1ef4198020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:05:54.055 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:54.051+0000 7f1ef27fc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1ef4102ae0 0x7f1ef4198020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:34632/0 (socket says 172.21.15.49:34632) 2024-03-18T23:05:54.055 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:54.051+0000 7f1ef27fc640 1 -- 172.21.15.49:0/3351754600 learned_addr learned my addr 172.21.15.49:0/3351754600 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:05:54.055 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:54.051+0000 7f1ef37fe640 1 --2- 172.21.15.49:0/3351754600 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1ef41034a0 0x7f1ef419f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:05:54.055 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:54.051+0000 7f1ef2ffd640 1 --2- 172.21.15.49:0/3351754600 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1ef40ffb10 0x7f1ef4197ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:05:54.055 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:54.051+0000 7f1ef27fc640 1 -- 172.21.15.49:0/3351754600 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1ef40ffb10 msgr2=0x7f1ef4197ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:05:54.055 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:54.051+0000 7f1ef27fc640 1 --2- 172.21.15.49:0/3351754600 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1ef40ffb10 0x7f1ef4197ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:05:54.055 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:54.051+0000 7f1ef27fc640 1 -- 172.21.15.49:0/3351754600 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1ef41034a0 msgr2=0x7f1ef419f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:05:54.055 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:54.051+0000 7f1ef27fc640 1 --2- 172.21.15.49:0/3351754600 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1ef41034a0 0x7f1ef419f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:05:54.055 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:54.052+0000 7f1ef27fc640 1 -- 172.21.15.49:0/3351754600 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ef419f700 con 0x7f1ef4102ae0 2024-03-18T23:05:54.055 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:54.052+0000 7f1ef27fc640 1 --2- 172.21.15.49:0/3351754600 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1ef4102ae0 0x7f1ef4198020 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f1ee800d950 tx=0x7f1ee800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:05:54.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:54.052+0000 7f1edbfff640 1 -- 172.21.15.49:0/3351754600 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ee8017030 con 0x7f1ef4102ae0 2024-03-18T23:05:54.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:54.052+0000 7f1ef9285640 1 -- 172.21.15.49:0/3351754600 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ef419f9f0 con 0x7f1ef4102ae0 2024-03-18T23:05:54.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:54.052+0000 7f1ef9285640 1 -- 172.21.15.49:0/3351754600 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ef419ff90 con 0x7f1ef4102ae0 2024-03-18T23:05:54.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:54.052+0000 7f1edbfff640 1 -- 172.21.15.49:0/3351754600 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ee8004020 con 0x7f1ef4102ae0 2024-03-18T23:05:54.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:54.052+0000 7f1edbfff640 1 -- 172.21.15.49:0/3351754600 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ee80047a0 con 0x7f1ef4102ae0 2024-03-18T23:05:54.057 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:54.053+0000 7f1ef9285640 1 -- 172.21.15.49:0/3351754600 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ee00052c0 con 0x7f1ef4102ae0 2024-03-18T23:05:54.057 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:54.054+0000 7f1edbfff640 1 -- 172.21.15.49:0/3351754600 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7f1ee80041d0 con 0x7f1ef4102ae0 2024-03-18T23:05:54.057 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:54.055+0000 7f1edbfff640 1 --2- 172.21.15.49:0/3351754600 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f1edc0a2800 0x7f1edc0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:05:54.057 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:54.055+0000 7f1ef2ffd640 1 --2- 172.21.15.49:0/3351754600 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f1edc0a2800 0x7f1edc0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:05:54.057 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:54.055+0000 7f1edbfff640 1 -- 172.21.15.49:0/3351754600 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f1ee80cdd10 con 0x7f1ef4102ae0 2024-03-18T23:05:54.057 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:54.055+0000 7f1ef2ffd640 1 --2- 172.21.15.49:0/3351754600 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f1edc0a2800 0x7f1edc0a4cc0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f1ee4005fd0 tx=0x7f1ee4005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:05:54.060 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:54.059+0000 7f1edbfff640 1 -- 172.21.15.49:0/3351754600 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f1ee8022020 con 0x7f1ef4102ae0 2024-03-18T23:05:54.065 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:05:53 smithi139 ceph-mon[67282]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T23:05:54.065 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:05:53 smithi139 ceph-mon[67282]: Deploying daemon mds.g on smithi049 2024-03-18T23:05:54.065 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:05:53 smithi139 ceph-mon[67282]: mds.? [v2:172.21.15.175:6838/3640882084,v1:172.21.15.175:6839/3640882084] up:boot 2024-03-18T23:05:54.065 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:05:53 smithi139 ceph-mon[67282]: fsmap 9 up:standby 2024-03-18T23:05:54.065 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:05:53 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-18T23:05:54.211 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:05:53 smithi175 ceph-mon[67352]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T23:05:54.212 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:05:53 smithi175 ceph-mon[67352]: Deploying daemon mds.g on smithi049 2024-03-18T23:05:54.212 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:05:53 smithi175 ceph-mon[67352]: mds.? [v2:172.21.15.175:6838/3640882084,v1:172.21.15.175:6839/3640882084] up:boot 2024-03-18T23:05:54.212 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:05:53 smithi175 ceph-mon[67352]: fsmap 9 up:standby 2024-03-18T23:05:54.212 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:05:53 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-18T23:05:54.285 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:54.284+0000 7f1ef9285640 1 -- 172.21.15.49:0/3351754600 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1ee0005880 con 0x7f1ef4102ae0 2024-03-18T23:05:54.286 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:54.285+0000 7f1edbfff640 1 -- 172.21.15.49:0/3351754600 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+8007 (secure 0 0 0) 0x7f1ee809b2e0 con 0x7f1ef4102ae0 2024-03-18T23:05:54.286 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-18T23:05:54.286 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6835/3631301630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":3631301630},{"type":"v1","addr":"172.21.15.49:6835","nonce":3631301630}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6837/444305233","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":444305233},{"type":"v1","addr":"172.21.15.49:6837","nonce":444305233}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24389,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3766178603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3766178603},{"type":"v1","addr":"172.21.15.139:6833","nonce":3766178603}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/1702997173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":1702997173},{"type":"v1","addr":"172.21.15.175:6833","nonce":1702997173}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/1663212937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":1663212937},{"type":"v1","addr":"172.21.15.139:6835","nonce":1663212937}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24430,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/755474931","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":755474931},{"type":"v1","addr":"172.21.15.175:6835","nonce":755474931}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24439,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/379482756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":379482756},{"type":"v1","addr":"172.21.15.175:6837","nonce":379482756}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24452,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6837/2330317540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6836","nonce":2330317540},{"type":"v1","addr":"172.21.15.139:6837","nonce":2330317540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24463,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6839/3640882084","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":3640882084},{"type":"v1","addr":"172.21.15.175:6839","nonce":3640882084}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-03-18T23:05:54.286 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 10 2024-03-18T23:05:54.288 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:54.287+0000 7f1ef9285640 1 -- 172.21.15.49:0/3351754600 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f1edc0a2800 msgr2=0x7f1edc0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:05:54.288 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:54.287+0000 7f1ef9285640 1 --2- 172.21.15.49:0/3351754600 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f1edc0a2800 0x7f1edc0a4cc0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f1ee4005fd0 tx=0x7f1ee4005850 comp rx=0 tx=0).stop 2024-03-18T23:05:54.288 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:54.287+0000 7f1ef9285640 1 -- 172.21.15.49:0/3351754600 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1ef4102ae0 msgr2=0x7f1ef4198020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:05:54.288 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:54.287+0000 7f1ef9285640 1 --2- 172.21.15.49:0/3351754600 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1ef4102ae0 0x7f1ef4198020 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f1ee800d950 tx=0x7f1ee800de10 comp rx=0 tx=0).stop 2024-03-18T23:05:54.288 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:54.287+0000 7f1ef9285640 1 -- 172.21.15.49:0/3351754600 shutdown_connections 2024-03-18T23:05:54.288 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:54.287+0000 7f1ef9285640 1 --2- 172.21.15.49:0/3351754600 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f1edc0a2800 0x7f1edc0a4cc0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:05:54.288 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:54.287+0000 7f1ef9285640 1 --2- 172.21.15.49:0/3351754600 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1ef41034a0 0x7f1ef419f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:05:54.288 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:54.287+0000 7f1ef9285640 1 --2- 172.21.15.49:0/3351754600 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1ef4102ae0 0x7f1ef4198020 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:05:54.288 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:54.287+0000 7f1ef9285640 1 --2- 172.21.15.49:0/3351754600 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1ef40ffb10 0x7f1ef4197ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:05:54.288 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:54.287+0000 7f1ef9285640 1 -- 172.21.15.49:0/3351754600 >> 172.21.15.49:0/3351754600 conn(0x7f1ef40fb400 msgr2=0x7f1ef4105de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:05:54.288 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:54.287+0000 7f1ef9285640 1 -- 172.21.15.49:0/3351754600 shutdown_connections 2024-03-18T23:05:54.289 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:54.287+0000 7f1ef9285640 1 -- 172.21.15.49:0/3351754600 wait complete. 2024-03-18T23:05:54.761 INFO:journalctl@ceph.mds.g.smithi049.stdout:Mar 18 23:05:54 smithi049 systemd[1]: Starting Ceph mds.g for c63591ea-e57a-11ee-95c9-87774f69a715... 2024-03-18T23:05:55.065 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:05:54 smithi139 ceph-mon[67282]: from='client.? 172.21.15.49:0/3351754600' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T23:05:55.081 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:05:54 smithi049 ceph-mon[62991]: from='client.? 172.21.15.49:0/3351754600' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T23:05:55.081 INFO:journalctl@ceph.mds.g.smithi049.stdout:Mar 18 23:05:54 smithi049 podman[104638]: 2024-03-18 23:05:54.83833446 +0000 UTC m=+0.018355994 image pull 2312c8e42d0069a2dbb2364bddb00264db4b800e7f18a7e7e4eb8f3ecab2ff1c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639 2024-03-18T23:05:55.081 INFO:journalctl@ceph.mds.g.smithi049.stdout:Mar 18 23:05:54 smithi049 podman[104638]: 2024-03-18 23:05:54.971785846 +0000 UTC m=+0.151807362 container create 6536069ee40744fa44cd76208b02e8b44cd89ef1f98d9c5da84fd868c70ee358 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mds-g, org.label-schema.build-date=20240307, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0) 2024-03-18T23:05:55.211 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:05:54 smithi175 ceph-mon[67352]: from='client.? 172.21.15.49:0/3351754600' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T23:05:55.380 INFO:journalctl@ceph.mds.g.smithi049.stdout:Mar 18 23:05:55 smithi049 podman[104638]: 2024-03-18 23:05:55.182776704 +0000 UTC m=+0.362798241 container init 6536069ee40744fa44cd76208b02e8b44cd89ef1f98d9c5da84fd868c70ee358 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mds-g, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-18T23:05:55.380 INFO:journalctl@ceph.mds.g.smithi049.stdout:Mar 18 23:05:55 smithi049 podman[104638]: 2024-03-18 23:05:55.185426842 +0000 UTC m=+0.365448372 container start 6536069ee40744fa44cd76208b02e8b44cd89ef1f98d9c5da84fd868c70ee358 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mds-g, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-03-18T23:05:55.380 INFO:journalctl@ceph.mds.g.smithi049.stdout:Mar 18 23:05:55 smithi049 ceph-mds[104657]: -- 172.21.15.49:0/215555174 <== mon.0 v2:172.21.15.49:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b6e7687a40 con 0x55b6e765b000 2024-03-18T23:05:55.380 INFO:journalctl@ceph.mds.g.smithi049.stdout:Mar 18 23:05:55 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mds-g[104653]: starting mds.g at 2024-03-18T23:05:55.380 INFO:journalctl@ceph.mds.g.smithi049.stdout:Mar 18 23:05:55 smithi049 bash[104638]: 6536069ee40744fa44cd76208b02e8b44cd89ef1f98d9c5da84fd868c70ee358 2024-03-18T23:05:55.380 INFO:journalctl@ceph.mds.g.smithi049.stdout:Mar 18 23:05:55 smithi049 systemd[1]: Started Ceph mds.g for c63591ea-e57a-11ee-95c9-87774f69a715. 2024-03-18T23:05:56.026 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:05:55 smithi049 ceph-mon[62991]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T23:05:56.026 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:05:55 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:05:56.026 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:05:55 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:05:56.026 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:05:55 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:05:56.026 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:05:55 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-18T23:05:56.026 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:05:55 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-18T23:05:56.026 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:05:55 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:05:56.065 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:05:55 smithi139 ceph-mon[67282]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T23:05:56.065 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:05:55 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:05:56.065 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:05:55 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:05:56.065 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:05:55 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:05:56.065 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:05:55 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-18T23:05:56.065 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:05:55 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-18T23:05:56.065 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:05:55 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:05:56.211 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:05:55 smithi175 ceph-mon[67352]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T23:05:56.211 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:05:55 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:05:56.212 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:05:55 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:05:56.212 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:05:55 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:05:56.212 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:05:55 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-18T23:05:56.212 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:05:55 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-18T23:05:56.212 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:05:55 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:05:56.300 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-18T23:05:56.415 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:56.411+0000 7fd8426ac640 1 -- 172.21.15.49:0/2947078913 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd83c1022e0 msgr2=0x7fd83c1046d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:05:56.415 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:56.411+0000 7fd8426ac640 1 --2- 172.21.15.49:0/2947078913 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd83c1022e0 0x7fd83c1046d0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fd82c009900 tx=0x7fd82c02bc30 comp rx=0 tx=0).stop 2024-03-18T23:05:56.415 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:56.411+0000 7fd8426ac640 1 -- 172.21.15.49:0/2947078913 shutdown_connections 2024-03-18T23:05:56.415 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:56.411+0000 7fd8426ac640 1 --2- 172.21.15.49:0/2947078913 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd83c107540 0x7fd83c109970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:05:56.415 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:56.411+0000 7fd8426ac640 1 --2- 172.21.15.49:0/2947078913 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd83c104c10 0x7fd83c107000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:05:56.415 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:56.411+0000 7fd8426ac640 1 --2- 172.21.15.49:0/2947078913 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd83c1022e0 0x7fd83c1046d0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:05:56.415 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:56.411+0000 7fd8426ac640 1 -- 172.21.15.49:0/2947078913 >> 172.21.15.49:0/2947078913 conn(0x7fd83c0fbcb0 msgr2=0x7fd83c0fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:05:56.415 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:56.411+0000 7fd8426ac640 1 -- 172.21.15.49:0/2947078913 shutdown_connections 2024-03-18T23:05:56.416 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:56.411+0000 7fd8426ac640 1 -- 172.21.15.49:0/2947078913 wait complete. 2024-03-18T23:05:56.416 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:56.412+0000 7fd8426ac640 1 Processor -- start 2024-03-18T23:05:56.416 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:56.412+0000 7fd8426ac640 1 -- start start 2024-03-18T23:05:56.416 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:56.412+0000 7fd8426ac640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd83c1022e0 0x7fd83c197b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:05:56.416 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:56.412+0000 7fd8426ac640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd83c104c10 0x7fd83c1980d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:05:56.416 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:56.412+0000 7fd8426ac640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd83c107540 0x7fd83c19f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:05:56.416 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:56.412+0000 7fd8426ac640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fd83c10d4c0 con 0x7fd83c1022e0 2024-03-18T23:05:56.416 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:56.412+0000 7fd8426ac640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fd83c10d340 con 0x7fd83c104c10 2024-03-18T23:05:56.416 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:56.412+0000 7fd8426ac640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd83c10d640 con 0x7fd83c107540 2024-03-18T23:05:56.416 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:56.412+0000 7fd83bfff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd83c1022e0 0x7fd83c197b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:05:56.416 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:56.412+0000 7fd83bfff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd83c1022e0 0x7fd83c197b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:50084/0 (socket says 172.21.15.49:50084) 2024-03-18T23:05:56.416 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:56.412+0000 7fd83bfff640 1 -- 172.21.15.49:0/552920060 learned_addr learned my addr 172.21.15.49:0/552920060 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:05:56.417 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:56.412+0000 7fd83b7fe640 1 --2- 172.21.15.49:0/552920060 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd83c104c10 0x7fd83c1980d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:05:56.417 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:56.412+0000 7fd840c49640 1 --2- 172.21.15.49:0/552920060 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd83c107540 0x7fd83c19f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:05:56.417 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:56.412+0000 7fd83bfff640 1 -- 172.21.15.49:0/552920060 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd83c107540 msgr2=0x7fd83c19f140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:05:56.417 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:56.412+0000 7fd83bfff640 1 --2- 172.21.15.49:0/552920060 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd83c107540 0x7fd83c19f140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:05:56.417 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:56.412+0000 7fd83bfff640 1 -- 172.21.15.49:0/552920060 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd83c104c10 msgr2=0x7fd83c1980d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:05:56.417 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:56.412+0000 7fd83bfff640 1 --2- 172.21.15.49:0/552920060 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd83c104c10 0x7fd83c1980d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:05:56.417 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:56.412+0000 7fd83bfff640 1 -- 172.21.15.49:0/552920060 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd83c19f7b0 con 0x7fd83c1022e0 2024-03-18T23:05:56.417 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:56.412+0000 7fd83b7fe640 1 --2- 172.21.15.49:0/552920060 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd83c104c10 0x7fd83c1980d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T23:05:56.417 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:56.413+0000 7fd840c49640 1 --2- 172.21.15.49:0/552920060 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd83c107540 0x7fd83c19f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T23:05:56.417 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:56.413+0000 7fd83bfff640 1 --2- 172.21.15.49:0/552920060 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd83c1022e0 0x7fd83c197b90 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fd82c009230 tx=0x7fd82c005dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:05:56.417 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:56.413+0000 7fd8397fa640 1 -- 172.21.15.49:0/552920060 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd82c041070 con 0x7fd83c1022e0 2024-03-18T23:05:56.417 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:56.413+0000 7fd8397fa640 1 -- 172.21.15.49:0/552920060 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd82c0054e0 con 0x7fd83c1022e0 2024-03-18T23:05:56.418 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:56.413+0000 7fd8426ac640 1 -- 172.21.15.49:0/552920060 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd83c19fa40 con 0x7fd83c1022e0 2024-03-18T23:05:56.418 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:56.413+0000 7fd8397fa640 1 -- 172.21.15.49:0/552920060 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd82c034440 con 0x7fd83c1022e0 2024-03-18T23:05:56.418 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:56.413+0000 7fd8426ac640 1 -- 172.21.15.49:0/552920060 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd83c19ff80 con 0x7fd83c1022e0 2024-03-18T23:05:56.418 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:56.414+0000 7fd8426ac640 1 -- 172.21.15.49:0/552920060 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd83c0fd7a0 con 0x7fd83c1022e0 2024-03-18T23:05:56.419 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:56.415+0000 7fd8397fa640 1 -- 172.21.15.49:0/552920060 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7fd82c044070 con 0x7fd83c1022e0 2024-03-18T23:05:56.419 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:56.415+0000 7fd8397fa640 1 --2- 172.21.15.49:0/552920060 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fd8280a2880 0x7fd8280a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:05:56.419 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:56.416+0000 7fd83b7fe640 1 --2- 172.21.15.49:0/552920060 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fd8280a2880 0x7fd8280a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:05:56.419 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:56.416+0000 7fd8397fa640 1 -- 172.21.15.49:0/552920060 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fd82c0f0300 con 0x7fd83c1022e0 2024-03-18T23:05:56.419 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:56.416+0000 7fd83b7fe640 1 --2- 172.21.15.49:0/552920060 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fd8280a2880 0x7fd8280a4d40 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fd8240058c0 tx=0x7fd824005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:05:56.421 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:56.420+0000 7fd8397fa640 1 -- 172.21.15.49:0/552920060 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fd82c0bd840 con 0x7fd83c1022e0 2024-03-18T23:05:56.645 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:56.644+0000 7fd8426ac640 1 -- 172.21.15.49:0/552920060 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd83c02d1f0 con 0x7fd83c1022e0 2024-03-18T23:05:56.646 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:56.645+0000 7fd8397fa640 1 -- 172.21.15.49:0/552920060 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8833 (secure 0 0 0) 0x7fd82c0bd140 con 0x7fd83c1022e0 2024-03-18T23:05:56.646 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-18T23:05:56.646 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6835/3631301630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":3631301630},{"type":"v1","addr":"172.21.15.49:6835","nonce":3631301630}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6837/444305233","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":444305233},{"type":"v1","addr":"172.21.15.49:6837","nonce":444305233}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14622,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6839/1934085093","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6838","nonce":1934085093},{"type":"v1","addr":"172.21.15.49:6839","nonce":1934085093}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24389,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3766178603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3766178603},{"type":"v1","addr":"172.21.15.139:6833","nonce":3766178603}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/1702997173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":1702997173},{"type":"v1","addr":"172.21.15.175:6833","nonce":1702997173}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/1663212937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":1663212937},{"type":"v1","addr":"172.21.15.139:6835","nonce":1663212937}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24430,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/755474931","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":755474931},{"type":"v1","addr":"172.21.15.175:6835","nonce":755474931}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24439,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/379482756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":379482756},{"type":"v1","addr":"172.21.15.175:6837","nonce":379482756}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24452,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6837/2330317540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6836","nonce":2330317540},{"type":"v1","addr":"172.21.15.139:6837","nonce":2330317540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24463,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6839/3640882084","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":3640882084},{"type":"v1","addr":"172.21.15.175:6839","nonce":3640882084}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-03-18T23:05:56.647 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 11 2024-03-18T23:05:56.648 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:56.647+0000 7fd8426ac640 1 -- 172.21.15.49:0/552920060 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fd8280a2880 msgr2=0x7fd8280a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:05:56.649 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:56.647+0000 7fd8426ac640 1 --2- 172.21.15.49:0/552920060 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fd8280a2880 0x7fd8280a4d40 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fd8240058c0 tx=0x7fd824005850 comp rx=0 tx=0).stop 2024-03-18T23:05:56.649 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:56.647+0000 7fd8426ac640 1 -- 172.21.15.49:0/552920060 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd83c1022e0 msgr2=0x7fd83c197b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:05:56.649 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:56.647+0000 7fd8426ac640 1 --2- 172.21.15.49:0/552920060 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd83c1022e0 0x7fd83c197b90 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fd82c009230 tx=0x7fd82c005dd0 comp rx=0 tx=0).stop 2024-03-18T23:05:56.649 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:56.648+0000 7fd8426ac640 1 -- 172.21.15.49:0/552920060 shutdown_connections 2024-03-18T23:05:56.649 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:56.648+0000 7fd8426ac640 1 --2- 172.21.15.49:0/552920060 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fd8280a2880 0x7fd8280a4d40 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:05:56.649 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:56.648+0000 7fd8426ac640 1 --2- 172.21.15.49:0/552920060 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd83c107540 0x7fd83c19f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:05:56.649 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:56.648+0000 7fd8426ac640 1 --2- 172.21.15.49:0/552920060 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd83c104c10 0x7fd83c1980d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:05:56.649 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:56.648+0000 7fd8426ac640 1 --2- 172.21.15.49:0/552920060 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd83c1022e0 0x7fd83c197b90 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:05:56.649 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:56.648+0000 7fd8426ac640 1 -- 172.21.15.49:0/552920060 >> 172.21.15.49:0/552920060 conn(0x7fd83c0fbcb0 msgr2=0x7fd83c102de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:05:56.649 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:56.648+0000 7fd8426ac640 1 -- 172.21.15.49:0/552920060 shutdown_connections 2024-03-18T23:05:56.649 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:56.648+0000 7fd8426ac640 1 -- 172.21.15.49:0/552920060 wait complete. 2024-03-18T23:05:56.999 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:05:56 smithi049 ceph-mon[62991]: Deploying daemon mds.a on smithi049 2024-03-18T23:05:56.999 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:05:56 smithi049 ceph-mon[62991]: mds.? [v2:172.21.15.49:6838/1934085093,v1:172.21.15.49:6839/1934085093] up:boot 2024-03-18T23:05:57.000 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:05:56 smithi049 ceph-mon[62991]: fsmap 10 up:standby 2024-03-18T23:05:57.000 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:05:56 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-18T23:05:57.000 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:05:56 smithi049 ceph-mon[62991]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T23:05:57.000 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:05:56 smithi049 ceph-mon[62991]: from='client.? 172.21.15.49:0/552920060' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T23:05:57.065 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:05:56 smithi139 ceph-mon[67282]: Deploying daemon mds.a on smithi049 2024-03-18T23:05:57.065 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:05:56 smithi139 ceph-mon[67282]: mds.? [v2:172.21.15.49:6838/1934085093,v1:172.21.15.49:6839/1934085093] up:boot 2024-03-18T23:05:57.065 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:05:56 smithi139 ceph-mon[67282]: fsmap 10 up:standby 2024-03-18T23:05:57.065 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:05:56 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-18T23:05:57.066 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:05:56 smithi139 ceph-mon[67282]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T23:05:57.066 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:05:56 smithi139 ceph-mon[67282]: from='client.? 172.21.15.49:0/552920060' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T23:05:57.211 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:05:56 smithi175 ceph-mon[67352]: Deploying daemon mds.a on smithi049 2024-03-18T23:05:57.211 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:05:56 smithi175 ceph-mon[67352]: mds.? [v2:172.21.15.49:6838/1934085093,v1:172.21.15.49:6839/1934085093] up:boot 2024-03-18T23:05:57.212 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:05:56 smithi175 ceph-mon[67352]: fsmap 10 up:standby 2024-03-18T23:05:57.212 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:05:56 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-18T23:05:57.212 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:05:56 smithi175 ceph-mon[67352]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T23:05:57.212 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:05:56 smithi175 ceph-mon[67352]: from='client.? 172.21.15.49:0/552920060' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T23:05:57.630 INFO:journalctl@ceph.mds.a.smithi049.stdout:Mar 18 23:05:57 smithi049 systemd[1]: Starting Ceph mds.a for c63591ea-e57a-11ee-95c9-87774f69a715... 2024-03-18T23:05:57.980 INFO:journalctl@ceph.mds.a.smithi049.stdout:Mar 18 23:05:57 smithi049 podman[104987]: 2024-03-18 23:05:57.62097719 +0000 UTC m=+0.018474427 image pull 2312c8e42d0069a2dbb2364bddb00264db4b800e7f18a7e7e4eb8f3ecab2ff1c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639 2024-03-18T23:05:57.980 INFO:journalctl@ceph.mds.a.smithi049.stdout:Mar 18 23:05:57 smithi049 podman[104987]: 2024-03-18 23:05:57.735652329 +0000 UTC m=+0.133149563 container create e54358a09cccd8f9a09ecafeb892957e681358e1d4493b0fc14fcf9a9e321982 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mds-a, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-03-18T23:05:57.980 INFO:journalctl@ceph.mds.a.smithi049.stdout:Mar 18 23:05:57 smithi049 podman[104987]: 2024-03-18 23:05:57.980385448 +0000 UTC m=+0.377882686 container init e54358a09cccd8f9a09ecafeb892957e681358e1d4493b0fc14fcf9a9e321982 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mds-a, org.label-schema.build-date=20240307, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-03-18T23:05:58.380 INFO:journalctl@ceph.mds.a.smithi049.stdout:Mar 18 23:05:57 smithi049 podman[104987]: 2024-03-18 23:05:57.983142576 +0000 UTC m=+0.380639811 container start e54358a09cccd8f9a09ecafeb892957e681358e1d4493b0fc14fcf9a9e321982 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mds-a, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-18T23:05:58.380 INFO:journalctl@ceph.mds.a.smithi049.stdout:Mar 18 23:05:58 smithi049 ceph-mds[105006]: -- 172.21.15.49:0/1288052562 <== mon.0 v2:172.21.15.49:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5567c0765a40 con 0x5567c0738800 2024-03-18T23:05:58.380 INFO:journalctl@ceph.mds.a.smithi049.stdout:Mar 18 23:05:58 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mds-a[105002]: starting mds.a at 2024-03-18T23:05:58.380 INFO:journalctl@ceph.mds.a.smithi049.stdout:Mar 18 23:05:58 smithi049 bash[104987]: e54358a09cccd8f9a09ecafeb892957e681358e1d4493b0fc14fcf9a9e321982 2024-03-18T23:05:58.380 INFO:journalctl@ceph.mds.a.smithi049.stdout:Mar 18 23:05:58 smithi049 systemd[1]: Started Ceph mds.a for c63591ea-e57a-11ee-95c9-87774f69a715. 2024-03-18T23:05:58.660 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-18T23:05:58.779 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:58.776+0000 7ff854275640 1 -- 172.21.15.49:0/2094974191 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff84c1022f0 msgr2=0x7ff84c108b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:05:58.779 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:58.776+0000 7ff854275640 1 --2- 172.21.15.49:0/2094974191 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff84c1022f0 0x7ff84c108b80 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7ff844009ca0 tx=0x7ff84402c0f0 comp rx=0 tx=0).stop 2024-03-18T23:05:58.779 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:58.776+0000 7ff854275640 1 -- 172.21.15.49:0/2094974191 shutdown_connections 2024-03-18T23:05:58.779 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:58.776+0000 7ff854275640 1 --2- 172.21.15.49:0/2094974191 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff84c1022f0 0x7ff84c108b80 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:05:58.780 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:58.776+0000 7ff854275640 1 --2- 172.21.15.49:0/2094974191 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff84c101930 0x7ff84c101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:05:58.780 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:58.776+0000 7ff854275640 1 --2- 172.21.15.49:0/2094974191 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff84c100740 0x7ff84c100b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:05:58.780 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:58.776+0000 7ff854275640 1 -- 172.21.15.49:0/2094974191 >> 172.21.15.49:0/2094974191 conn(0x7ff84c0fbcb0 msgr2=0x7ff84c0fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:05:58.780 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:58.776+0000 7ff854275640 1 -- 172.21.15.49:0/2094974191 shutdown_connections 2024-03-18T23:05:58.780 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:58.776+0000 7ff854275640 1 -- 172.21.15.49:0/2094974191 wait complete. 2024-03-18T23:05:58.780 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:58.777+0000 7ff854275640 1 Processor -- start 2024-03-18T23:05:58.780 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:58.777+0000 7ff854275640 1 -- start start 2024-03-18T23:05:58.780 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:58.777+0000 7ff854275640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff84c100740 0x7ff84c197ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:05:58.780 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:58.777+0000 7ff854275640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff84c101930 0x7ff84c198010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:05:58.780 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:58.777+0000 7ff854275640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff84c1022f0 0x7ff84c19f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:05:58.780 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:58.777+0000 7ff854275640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7ff84c10d4e0 con 0x7ff84c101930 2024-03-18T23:05:58.780 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:58.777+0000 7ff854275640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7ff84c10d360 con 0x7ff84c1022f0 2024-03-18T23:05:58.780 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:58.777+0000 7ff854275640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff84c10d660 con 0x7ff84c100740 2024-03-18T23:05:58.780 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:58.777+0000 7ff851810640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff84c101930 0x7ff84c198010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:05:58.781 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:58.777+0000 7ff851810640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff84c101930 0x7ff84c198010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:50104/0 (socket says 172.21.15.49:50104) 2024-03-18T23:05:58.781 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:58.777+0000 7ff851810640 1 -- 172.21.15.49:0/2821832423 learned_addr learned my addr 172.21.15.49:0/2821832423 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:05:58.781 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:58.777+0000 7ff852812640 1 --2- 172.21.15.49:0/2821832423 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff84c1022f0 0x7ff84c19f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:05:58.781 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:58.777+0000 7ff851810640 1 -- 172.21.15.49:0/2821832423 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff84c100740 msgr2=0x7ff84c197ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:05:58.781 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:58.777+0000 7ff852011640 1 --2- 172.21.15.49:0/2821832423 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff84c100740 0x7ff84c197ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:05:58.781 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:58.777+0000 7ff851810640 1 --2- 172.21.15.49:0/2821832423 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff84c100740 0x7ff84c197ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:05:58.781 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:58.777+0000 7ff851810640 1 -- 172.21.15.49:0/2821832423 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff84c1022f0 msgr2=0x7ff84c19f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:05:58.781 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:58.777+0000 7ff851810640 1 --2- 172.21.15.49:0/2821832423 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff84c1022f0 0x7ff84c19f080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:05:58.781 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:58.777+0000 7ff851810640 1 -- 172.21.15.49:0/2821832423 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff84c19f6f0 con 0x7ff84c101930 2024-03-18T23:05:58.781 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:58.777+0000 7ff852812640 1 --2- 172.21.15.49:0/2821832423 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff84c1022f0 0x7ff84c19f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T23:05:58.781 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:58.777+0000 7ff851810640 1 --2- 172.21.15.49:0/2821832423 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff84c101930 0x7ff84c198010 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7ff84000d950 tx=0x7ff84000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:05:58.781 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:58.778+0000 7ff83f7fe640 1 -- 172.21.15.49:0/2821832423 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff840017030 con 0x7ff84c101930 2024-03-18T23:05:58.781 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:58.778+0000 7ff854275640 1 -- 172.21.15.49:0/2821832423 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff84c19f9e0 con 0x7ff84c101930 2024-03-18T23:05:58.781 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:58.778+0000 7ff83f7fe640 1 -- 172.21.15.49:0/2821832423 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff840004020 con 0x7ff84c101930 2024-03-18T23:05:58.782 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:58.778+0000 7ff83f7fe640 1 -- 172.21.15.49:0/2821832423 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8400047a0 con 0x7ff84c101930 2024-03-18T23:05:58.782 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:58.778+0000 7ff854275640 1 -- 172.21.15.49:0/2821832423 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff84c19ff30 con 0x7ff84c101930 2024-03-18T23:05:58.782 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:58.778+0000 7ff854275640 1 -- 172.21.15.49:0/2821832423 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff84c107a60 con 0x7ff84c101930 2024-03-18T23:05:58.782 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:58.780+0000 7ff83f7fe640 1 -- 172.21.15.49:0/2821832423 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7ff8400041d0 con 0x7ff84c101930 2024-03-18T23:05:58.783 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:58.780+0000 7ff83f7fe640 1 --2- 172.21.15.49:0/2821832423 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7ff8380a2800 0x7ff8380a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:05:58.783 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:58.780+0000 7ff83f7fe640 1 -- 172.21.15.49:0/2821832423 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7ff8400ce6f0 con 0x7ff84c101930 2024-03-18T23:05:58.783 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:58.780+0000 7ff852011640 1 --2- 172.21.15.49:0/2821832423 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7ff8380a2800 0x7ff8380a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:05:58.783 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:58.781+0000 7ff852011640 1 --2- 172.21.15.49:0/2821832423 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7ff8380a2800 0x7ff8380a4cc0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7ff8480058c0 tx=0x7ff848005c80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:05:58.784 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:58.784+0000 7ff83f7fe640 1 -- 172.21.15.49:0/2821832423 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7ff84009bc30 con 0x7ff84c101930 2024-03-18T23:05:59.014 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:59.013+0000 7ff854275640 1 -- 172.21.15.49:0/2821832423 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff84c02d1f0 con 0x7ff84c101930 2024-03-18T23:05:59.015 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:59.014+0000 7ff83f7fe640 1 -- 172.21.15.49:0/2821832423 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9659 (secure 0 0 0) 0x7ff84009b530 con 0x7ff84c101930 2024-03-18T23:05:59.015 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-18T23:05:59.015 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6835/3631301630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":3631301630},{"type":"v1","addr":"172.21.15.49:6835","nonce":3631301630}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6837/444305233","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":444305233},{"type":"v1","addr":"172.21.15.49:6837","nonce":444305233}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14622,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6839/1934085093","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6838","nonce":1934085093},{"type":"v1","addr":"172.21.15.49:6839","nonce":1934085093}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14634,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6841/2869298862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6840","nonce":2869298862},{"type":"v1","addr":"172.21.15.49:6841","nonce":2869298862}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24389,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3766178603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3766178603},{"type":"v1","addr":"172.21.15.139:6833","nonce":3766178603}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/1702997173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":1702997173},{"type":"v1","addr":"172.21.15.175:6833","nonce":1702997173}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/1663212937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":1663212937},{"type":"v1","addr":"172.21.15.139:6835","nonce":1663212937}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24430,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/755474931","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":755474931},{"type":"v1","addr":"172.21.15.175:6835","nonce":755474931}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24439,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/379482756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":379482756},{"type":"v1","addr":"172.21.15.175:6837","nonce":379482756}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24452,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6837/2330317540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6836","nonce":2330317540},{"type":"v1","addr":"172.21.15.139:6837","nonce":2330317540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24463,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6839/3640882084","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":3640882084},{"type":"v1","addr":"172.21.15.175:6839","nonce":3640882084}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-03-18T23:05:59.015 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 12 2024-03-18T23:05:59.017 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:59.016+0000 7ff854275640 1 -- 172.21.15.49:0/2821832423 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7ff8380a2800 msgr2=0x7ff8380a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:05:59.017 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:59.016+0000 7ff854275640 1 --2- 172.21.15.49:0/2821832423 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7ff8380a2800 0x7ff8380a4cc0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7ff8480058c0 tx=0x7ff848005c80 comp rx=0 tx=0).stop 2024-03-18T23:05:59.017 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:59.016+0000 7ff854275640 1 -- 172.21.15.49:0/2821832423 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff84c101930 msgr2=0x7ff84c198010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:05:59.017 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:59.016+0000 7ff854275640 1 --2- 172.21.15.49:0/2821832423 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff84c101930 0x7ff84c198010 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7ff84000d950 tx=0x7ff84000de10 comp rx=0 tx=0).stop 2024-03-18T23:05:59.017 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:59.016+0000 7ff854275640 1 -- 172.21.15.49:0/2821832423 shutdown_connections 2024-03-18T23:05:59.017 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:59.016+0000 7ff854275640 1 --2- 172.21.15.49:0/2821832423 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7ff8380a2800 0x7ff8380a4cc0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:05:59.017 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:59.016+0000 7ff854275640 1 --2- 172.21.15.49:0/2821832423 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff84c1022f0 0x7ff84c19f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:05:59.018 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:59.016+0000 7ff854275640 1 --2- 172.21.15.49:0/2821832423 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff84c101930 0x7ff84c198010 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:05:59.018 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:59.016+0000 7ff854275640 1 --2- 172.21.15.49:0/2821832423 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff84c100740 0x7ff84c197ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:05:59.018 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:59.016+0000 7ff854275640 1 -- 172.21.15.49:0/2821832423 >> 172.21.15.49:0/2821832423 conn(0x7ff84c0fbcb0 msgr2=0x7ff84c104c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:05:59.018 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:59.016+0000 7ff854275640 1 -- 172.21.15.49:0/2821832423 shutdown_connections 2024-03-18T23:05:59.018 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:05:59.016+0000 7ff854275640 1 -- 172.21.15.49:0/2821832423 wait complete. 2024-03-18T23:05:59.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:05:59 smithi049 ceph-mon[62991]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T23:05:59.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:05:59 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:05:59.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:05:59 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:05:59.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:05:59 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:05:59.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:05:59 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-18T23:05:59.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:05:59 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-18T23:05:59.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:05:59 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:05:59.381 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:05:59 smithi049 ceph-mon[62991]: Deploying daemon mds.b on smithi139 2024-03-18T23:05:59.381 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:05:59 smithi049 ceph-mon[62991]: mds.? [v2:172.21.15.49:6840/2869298862,v1:172.21.15.49:6841/2869298862] up:boot 2024-03-18T23:05:59.381 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:05:59 smithi049 ceph-mon[62991]: fsmap 11 up:standby 2024-03-18T23:05:59.381 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:05:59 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-18T23:05:59.381 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:05:59 smithi049 ceph-mon[62991]: from='client.? 172.21.15.49:0/2821832423' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T23:05:59.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:05:59 smithi175 ceph-mon[67352]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T23:05:59.462 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:05:59 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:05:59.462 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:05:59 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:05:59.462 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:05:59 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:05:59.462 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:05:59 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-18T23:05:59.462 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:05:59 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-18T23:05:59.462 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:05:59 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:05:59.462 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:05:59 smithi175 ceph-mon[67352]: Deploying daemon mds.b on smithi139 2024-03-18T23:05:59.462 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:05:59 smithi175 ceph-mon[67352]: mds.? [v2:172.21.15.49:6840/2869298862,v1:172.21.15.49:6841/2869298862] up:boot 2024-03-18T23:05:59.462 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:05:59 smithi175 ceph-mon[67352]: fsmap 11 up:standby 2024-03-18T23:05:59.462 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:05:59 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-18T23:05:59.462 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:05:59 smithi175 ceph-mon[67352]: from='client.? 172.21.15.49:0/2821832423' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T23:05:59.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:05:59 smithi139 ceph-mon[67282]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T23:05:59.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:05:59 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:05:59.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:05:59 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:05:59.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:05:59 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:05:59.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:05:59 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-18T23:05:59.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:05:59 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-18T23:05:59.566 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:05:59 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:05:59.566 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:05:59 smithi139 ceph-mon[67282]: Deploying daemon mds.b on smithi139 2024-03-18T23:05:59.566 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:05:59 smithi139 ceph-mon[67282]: mds.? [v2:172.21.15.49:6840/2869298862,v1:172.21.15.49:6841/2869298862] up:boot 2024-03-18T23:05:59.566 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:05:59 smithi139 ceph-mon[67282]: fsmap 11 up:standby 2024-03-18T23:05:59.566 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:05:59 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-18T23:05:59.566 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:05:59 smithi139 ceph-mon[67282]: from='client.? 172.21.15.49:0/2821832423' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T23:06:00.565 INFO:journalctl@ceph.mds.b.smithi139.stdout:Mar 18 23:06:00 smithi139 systemd[1]: Starting Ceph mds.b for c63591ea-e57a-11ee-95c9-87774f69a715... 2024-03-18T23:06:00.958 INFO:journalctl@ceph.mds.b.smithi139.stdout:Mar 18 23:06:00 smithi139 podman[98103]: 2024-03-18 23:06:00.680509192 +0000 UTC m=+0.103507144 container create 4049f303e6e80d5528fa9e74ea442226870ef3024fd6b1e47a51f8eb50475726 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mds-b, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-18T23:06:00.958 INFO:journalctl@ceph.mds.b.smithi139.stdout:Mar 18 23:06:00 smithi139 podman[98103]: 2024-03-18 23:06:00.594370563 +0000 UTC m=+0.017368530 image pull 2312c8e42d0069a2dbb2364bddb00264db4b800e7f18a7e7e4eb8f3ecab2ff1c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639 2024-03-18T23:06:00.958 INFO:journalctl@ceph.mds.b.smithi139.stdout:Mar 18 23:06:00 smithi139 podman[98103]: 2024-03-18 23:06:00.958434519 +0000 UTC m=+0.381432466 container init 4049f303e6e80d5528fa9e74ea442226870ef3024fd6b1e47a51f8eb50475726 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mds-b, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-18T23:06:01.029 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-18T23:06:01.158 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:01.156+0000 7fe7da29a640 1 -- 172.21.15.49:0/2054153105 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe7d4102e50 msgr2=0x7fe7d4103270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:01.158 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:01.156+0000 7fe7da29a640 1 --2- 172.21.15.49:0/2054153105 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe7d4102e50 0x7fe7d4103270 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fe7c4009900 tx=0x7fe7c402bc30 comp rx=0 tx=0).stop 2024-03-18T23:06:01.158 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:01.156+0000 7fe7da29a640 1 -- 172.21.15.49:0/2054153105 shutdown_connections 2024-03-18T23:06:01.158 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:01.156+0000 7fe7da29a640 1 --2- 172.21.15.49:0/2054153105 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe7d4100280 0x7fe7d4109e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:01.158 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:01.156+0000 7fe7da29a640 1 --2- 172.21.15.49:0/2054153105 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe7d40ff8c0 0x7fe7d40ffd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:01.158 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:01.156+0000 7fe7da29a640 1 --2- 172.21.15.49:0/2054153105 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe7d4102e50 0x7fe7d4103270 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:01.158 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:01.156+0000 7fe7da29a640 1 -- 172.21.15.49:0/2054153105 >> 172.21.15.49:0/2054153105 conn(0x7fe7d40fb440 msgr2=0x7fe7d40fd880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:06:01.159 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:01.156+0000 7fe7da29a640 1 -- 172.21.15.49:0/2054153105 shutdown_connections 2024-03-18T23:06:01.159 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:01.156+0000 7fe7da29a640 1 -- 172.21.15.49:0/2054153105 wait complete. 2024-03-18T23:06:01.159 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:01.157+0000 7fe7da29a640 1 Processor -- start 2024-03-18T23:06:01.159 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:01.157+0000 7fe7da29a640 1 -- start start 2024-03-18T23:06:01.161 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:01.157+0000 7fe7da29a640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe7d40ff8c0 0x7fe7d4197ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:06:01.161 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:01.157+0000 7fe7da29a640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe7d4100280 0x7fe7d41980e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:06:01.161 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:01.157+0000 7fe7da29a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe7d4102e50 0x7fe7d419f150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:06:01.161 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:01.157+0000 7fe7da29a640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fe7d410d530 con 0x7fe7d40ff8c0 2024-03-18T23:06:01.161 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:01.157+0000 7fe7da29a640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fe7d410d3b0 con 0x7fe7d4102e50 2024-03-18T23:06:01.161 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:01.157+0000 7fe7da29a640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe7d410d6b0 con 0x7fe7d4100280 2024-03-18T23:06:01.162 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:01.158+0000 7fe7d3fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe7d40ff8c0 0x7fe7d4197ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:06:01.162 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:01.158+0000 7fe7d3fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe7d40ff8c0 0x7fe7d4197ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:50130/0 (socket says 172.21.15.49:50130) 2024-03-18T23:06:01.162 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:01.158+0000 7fe7d3fff640 1 -- 172.21.15.49:0/3070460050 learned_addr learned my addr 172.21.15.49:0/3070460050 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:06:01.162 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:01.158+0000 7fe7d8837640 1 --2- 172.21.15.49:0/3070460050 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe7d4102e50 0x7fe7d419f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:06:01.162 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:01.158+0000 7fe7d37fe640 1 --2- 172.21.15.49:0/3070460050 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe7d4100280 0x7fe7d41980e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:06:01.162 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:01.158+0000 7fe7d3fff640 1 -- 172.21.15.49:0/3070460050 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe7d4100280 msgr2=0x7fe7d41980e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:01.162 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:01.158+0000 7fe7d3fff640 1 --2- 172.21.15.49:0/3070460050 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe7d4100280 0x7fe7d41980e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:01.163 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:01.158+0000 7fe7d3fff640 1 -- 172.21.15.49:0/3070460050 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe7d4102e50 msgr2=0x7fe7d419f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:01.163 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:01.158+0000 7fe7d3fff640 1 --2- 172.21.15.49:0/3070460050 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe7d4102e50 0x7fe7d419f150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:01.163 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:01.158+0000 7fe7d3fff640 1 -- 172.21.15.49:0/3070460050 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7d419f7c0 con 0x7fe7d40ff8c0 2024-03-18T23:06:01.163 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:01.158+0000 7fe7d8837640 1 --2- 172.21.15.49:0/3070460050 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe7d4102e50 0x7fe7d419f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T23:06:01.163 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:01.158+0000 7fe7d3fff640 1 --2- 172.21.15.49:0/3070460050 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe7d40ff8c0 0x7fe7d4197ba0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fe7c4000c00 tx=0x7fe7c4004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:06:01.163 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:01.158+0000 7fe7d17fa640 1 -- 172.21.15.49:0/3070460050 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7c4041070 con 0x7fe7d40ff8c0 2024-03-18T23:06:01.163 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:01.158+0000 7fe7d17fa640 1 -- 172.21.15.49:0/3070460050 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe7c40054e0 con 0x7fe7d40ff8c0 2024-03-18T23:06:01.163 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:01.158+0000 7fe7da29a640 1 -- 172.21.15.49:0/3070460050 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7d419fa50 con 0x7fe7d40ff8c0 2024-03-18T23:06:01.163 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:01.158+0000 7fe7d17fa640 1 -- 172.21.15.49:0/3070460050 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7c40346a0 con 0x7fe7d40ff8c0 2024-03-18T23:06:01.163 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:01.158+0000 7fe7da29a640 1 -- 172.21.15.49:0/3070460050 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7d419ff90 con 0x7fe7d40ff8c0 2024-03-18T23:06:01.163 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:01.159+0000 7fe7da29a640 1 -- 172.21.15.49:0/3070460050 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7d4108d60 con 0x7fe7d40ff8c0 2024-03-18T23:06:01.163 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:01.160+0000 7fe7d17fa640 1 -- 172.21.15.49:0/3070460050 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7fe7c4044070 con 0x7fe7d40ff8c0 2024-03-18T23:06:01.164 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:01.161+0000 7fe7d17fa640 1 --2- 172.21.15.49:0/3070460050 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fe7bc0a2880 0x7fe7bc0a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:06:01.164 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:01.161+0000 7fe7d17fa640 1 -- 172.21.15.49:0/3070460050 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fe7c40f0530 con 0x7fe7d40ff8c0 2024-03-18T23:06:01.164 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:01.161+0000 7fe7d37fe640 1 --2- 172.21.15.49:0/3070460050 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fe7bc0a2880 0x7fe7bc0a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:06:01.164 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:01.161+0000 7fe7d37fe640 1 --2- 172.21.15.49:0/3070460050 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fe7bc0a2880 0x7fe7bc0a4d40 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fe7d41990b0 tx=0x7fe7c800a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:06:01.165 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:01.165+0000 7fe7d17fa640 1 -- 172.21.15.49:0/3070460050 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fe7c40bda70 con 0x7fe7d40ff8c0 2024-03-18T23:06:01.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:06:01 smithi139 ceph-mon[67282]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T23:06:01.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:06:01 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:06:01.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:06:01 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:06:01.315 INFO:journalctl@ceph.mds.b.smithi139.stdout:Mar 18 23:06:00 smithi139 podman[98103]: 2024-03-18 23:06:00.961159864 +0000 UTC m=+0.384157826 container start 4049f303e6e80d5528fa9e74ea442226870ef3024fd6b1e47a51f8eb50475726 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mds-b, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-18T23:06:01.315 INFO:journalctl@ceph.mds.b.smithi139.stdout:Mar 18 23:06:00 smithi139 ceph-mds[98122]: -- 172.21.15.139:0/268814035 <== mon.2 v2:172.21.15.139:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x556fb9271a40 con 0x556fb9244c00 2024-03-18T23:06:01.316 INFO:journalctl@ceph.mds.b.smithi139.stdout:Mar 18 23:06:00 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mds-b[98118]: starting mds.b at 2024-03-18T23:06:01.316 INFO:journalctl@ceph.mds.b.smithi139.stdout:Mar 18 23:06:01 smithi139 bash[98103]: 4049f303e6e80d5528fa9e74ea442226870ef3024fd6b1e47a51f8eb50475726 2024-03-18T23:06:01.316 INFO:journalctl@ceph.mds.b.smithi139.stdout:Mar 18 23:06:01 smithi139 systemd[1]: Started Ceph mds.b for c63591ea-e57a-11ee-95c9-87774f69a715. 2024-03-18T23:06:01.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:06:01 smithi049 ceph-mon[62991]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T23:06:01.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:06:01 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:06:01.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:06:01 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:06:01.415 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:01.414+0000 7fe7da29a640 1 -- 172.21.15.49:0/3070460050 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe7d402d440 con 0x7fe7d40ff8c0 2024-03-18T23:06:01.415 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:01.414+0000 7fe7d17fa640 1 -- 172.21.15.49:0/3070460050 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10488 (secure 0 0 0) 0x7fe7c4003e20 con 0x7fe7d40ff8c0 2024-03-18T23:06:01.416 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-18T23:06:01.416 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6835/3631301630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":3631301630},{"type":"v1","addr":"172.21.15.49:6835","nonce":3631301630}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6837/444305233","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":444305233},{"type":"v1","addr":"172.21.15.49:6837","nonce":444305233}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14622,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6839/1934085093","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6838","nonce":1934085093},{"type":"v1","addr":"172.21.15.49:6839","nonce":1934085093}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14634,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6841/2869298862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6840","nonce":2869298862},{"type":"v1","addr":"172.21.15.49:6841","nonce":2869298862}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24389,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3766178603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3766178603},{"type":"v1","addr":"172.21.15.139:6833","nonce":3766178603}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/1702997173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":1702997173},{"type":"v1","addr":"172.21.15.175:6833","nonce":1702997173}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/1663212937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":1663212937},{"type":"v1","addr":"172.21.15.139:6835","nonce":1663212937}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24430,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/755474931","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":755474931},{"type":"v1","addr":"172.21.15.175:6835","nonce":755474931}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24439,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/379482756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":379482756},{"type":"v1","addr":"172.21.15.175:6837","nonce":379482756}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24452,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6837/2330317540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6836","nonce":2330317540},{"type":"v1","addr":"172.21.15.139:6837","nonce":2330317540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24463,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6839/3640882084","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":3640882084},{"type":"v1","addr":"172.21.15.175:6839","nonce":3640882084}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24488,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6839/4280128209","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6838","nonce":4280128209},{"type":"v1","addr":"172.21.15.139:6839","nonce":4280128209}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[]} 2024-03-18T23:06:01.416 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 13 2024-03-18T23:06:01.417 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:01.416+0000 7fe7da29a640 1 -- 172.21.15.49:0/3070460050 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fe7bc0a2880 msgr2=0x7fe7bc0a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:01.418 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:01.416+0000 7fe7da29a640 1 --2- 172.21.15.49:0/3070460050 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fe7bc0a2880 0x7fe7bc0a4d40 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fe7d41990b0 tx=0x7fe7c800a2d0 comp rx=0 tx=0).stop 2024-03-18T23:06:01.418 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:01.416+0000 7fe7da29a640 1 -- 172.21.15.49:0/3070460050 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe7d40ff8c0 msgr2=0x7fe7d4197ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:01.418 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:01.416+0000 7fe7da29a640 1 --2- 172.21.15.49:0/3070460050 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe7d40ff8c0 0x7fe7d4197ba0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fe7c4000c00 tx=0x7fe7c4004040 comp rx=0 tx=0).stop 2024-03-18T23:06:01.418 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:01.417+0000 7fe7da29a640 1 -- 172.21.15.49:0/3070460050 shutdown_connections 2024-03-18T23:06:01.418 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:01.417+0000 7fe7da29a640 1 --2- 172.21.15.49:0/3070460050 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fe7bc0a2880 0x7fe7bc0a4d40 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:01.418 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:01.417+0000 7fe7da29a640 1 --2- 172.21.15.49:0/3070460050 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe7d4102e50 0x7fe7d419f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:01.418 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:01.417+0000 7fe7da29a640 1 --2- 172.21.15.49:0/3070460050 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe7d4100280 0x7fe7d41980e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:01.418 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:01.417+0000 7fe7da29a640 1 --2- 172.21.15.49:0/3070460050 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe7d40ff8c0 0x7fe7d4197ba0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:01.418 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:01.417+0000 7fe7da29a640 1 -- 172.21.15.49:0/3070460050 >> 172.21.15.49:0/3070460050 conn(0x7fe7d40fb440 msgr2=0x7fe7d4105f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:06:01.418 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:01.417+0000 7fe7da29a640 1 -- 172.21.15.49:0/3070460050 shutdown_connections 2024-03-18T23:06:01.418 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:01.417+0000 7fe7da29a640 1 -- 172.21.15.49:0/3070460050 wait complete. 2024-03-18T23:06:01.429 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-18T23:06:01.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:06:01 smithi175 ceph-mon[67352]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T23:06:01.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:06:01 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:06:01.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:06:01 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:06:01.539 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:01.538+0000 7fe975f52640 1 -- 172.21.15.49:0/2178520722 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe970101930 msgr2=0x7fe970101db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:01.539 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:01.538+0000 7fe975f52640 1 --2- 172.21.15.49:0/2178520722 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe970101930 0x7fe970101db0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fe958009900 tx=0x7fe95802bc30 comp rx=0 tx=0).stop 2024-03-18T23:06:01.540 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:01.538+0000 7fe975f52640 1 -- 172.21.15.49:0/2178520722 shutdown_connections 2024-03-18T23:06:01.540 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:01.538+0000 7fe975f52640 1 --2- 172.21.15.49:0/2178520722 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe9701022f0 0x7fe970108b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:01.540 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:01.538+0000 7fe975f52640 1 --2- 172.21.15.49:0/2178520722 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe970101930 0x7fe970101db0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:01.540 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:01.538+0000 7fe975f52640 1 --2- 172.21.15.49:0/2178520722 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe970100740 0x7fe970100b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:01.540 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:01.538+0000 7fe975f52640 1 -- 172.21.15.49:0/2178520722 >> 172.21.15.49:0/2178520722 conn(0x7fe9700fbcb0 msgr2=0x7fe9700fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:06:01.540 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:01.538+0000 7fe975f52640 1 -- 172.21.15.49:0/2178520722 shutdown_connections 2024-03-18T23:06:01.540 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:01.538+0000 7fe975f52640 1 -- 172.21.15.49:0/2178520722 wait complete. 2024-03-18T23:06:01.540 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:01.539+0000 7fe975f52640 1 Processor -- start 2024-03-18T23:06:01.542 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:01.539+0000 7fe975f52640 1 -- start start 2024-03-18T23:06:01.542 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:01.539+0000 7fe975f52640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe970100740 0x7fe970197b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:06:01.543 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:01.539+0000 7fe975f52640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe970101930 0x7fe970198080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:06:01.543 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:01.539+0000 7fe975f52640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe9701022f0 0x7fe97019f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:06:01.543 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:01.539+0000 7fe975f52640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fe97010d4e0 con 0x7fe970101930 2024-03-18T23:06:01.543 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:01.539+0000 7fe975f52640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fe97010d360 con 0x7fe9701022f0 2024-03-18T23:06:01.543 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:01.539+0000 7fe975f52640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe97010d660 con 0x7fe970100740 2024-03-18T23:06:01.543 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:01.539+0000 7fe96effd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe970101930 0x7fe970198080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:06:01.543 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:01.539+0000 7fe96effd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe970101930 0x7fe970198080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:50140/0 (socket says 172.21.15.49:50140) 2024-03-18T23:06:01.543 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:01.539+0000 7fe96effd640 1 -- 172.21.15.49:0/3792733483 learned_addr learned my addr 172.21.15.49:0/3792733483 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:06:01.543 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:01.539+0000 7fe96f7fe640 1 --2- 172.21.15.49:0/3792733483 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe970100740 0x7fe970197b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:06:01.543 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:01.539+0000 7fe96ffff640 1 --2- 172.21.15.49:0/3792733483 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe9701022f0 0x7fe97019f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:06:01.543 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:01.540+0000 7fe96effd640 1 -- 172.21.15.49:0/3792733483 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe970100740 msgr2=0x7fe970197b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:01.543 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:01.540+0000 7fe96effd640 1 --2- 172.21.15.49:0/3792733483 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe970100740 0x7fe970197b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:01.543 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:01.540+0000 7fe96effd640 1 -- 172.21.15.49:0/3792733483 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe9701022f0 msgr2=0x7fe97019f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:01.544 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:01.540+0000 7fe96effd640 1 --2- 172.21.15.49:0/3792733483 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe9701022f0 0x7fe97019f0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:01.544 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:01.540+0000 7fe96effd640 1 -- 172.21.15.49:0/3792733483 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe97019f760 con 0x7fe970101930 2024-03-18T23:06:01.544 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:01.540+0000 7fe96effd640 1 --2- 172.21.15.49:0/3792733483 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe970101930 0x7fe970198080 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fe958009230 tx=0x7fe95802c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:06:01.544 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:01.540+0000 7fe96cff9640 1 -- 172.21.15.49:0/3792733483 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe958041070 con 0x7fe970101930 2024-03-18T23:06:01.544 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:01.540+0000 7fe96cff9640 1 -- 172.21.15.49:0/3792733483 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe958004450 con 0x7fe970101930 2024-03-18T23:06:01.544 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:01.540+0000 7fe975f52640 1 -- 172.21.15.49:0/3792733483 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe97019f9f0 con 0x7fe970101930 2024-03-18T23:06:01.544 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:01.540+0000 7fe96cff9640 1 -- 172.21.15.49:0/3792733483 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe958003e20 con 0x7fe970101930 2024-03-18T23:06:01.544 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:01.540+0000 7fe975f52640 1 -- 172.21.15.49:0/3792733483 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe97019fee0 con 0x7fe970101930 2024-03-18T23:06:01.545 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:01.541+0000 7fe975f52640 1 -- 172.21.15.49:0/3792733483 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe970107b30 con 0x7fe970101930 2024-03-18T23:06:01.545 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:01.542+0000 7fe96cff9640 1 -- 172.21.15.49:0/3792733483 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7fe958044070 con 0x7fe970101930 2024-03-18T23:06:01.545 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:01.543+0000 7fe96cff9640 1 --2- 172.21.15.49:0/3792733483 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fe95c0a2800 0x7fe95c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:06:01.545 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:01.543+0000 7fe96f7fe640 1 --2- 172.21.15.49:0/3792733483 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fe95c0a2800 0x7fe95c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:06:01.546 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:01.543+0000 7fe96cff9640 1 -- 172.21.15.49:0/3792733483 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fe9580f0570 con 0x7fe970101930 2024-03-18T23:06:01.546 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:01.543+0000 7fe96f7fe640 1 --2- 172.21.15.49:0/3792733483 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fe95c0a2800 0x7fe95c0a4cc0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fe970101790 tx=0x7fe960005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:06:01.547 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:01.546+0000 7fe96cff9640 1 -- 172.21.15.49:0/3792733483 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fe9580bd3c0 con 0x7fe970101930 2024-03-18T23:06:01.734 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:01.733+0000 7fe975f52640 1 -- 172.21.15.49:0/3792733483 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe97002d440 con 0x7fe970101930 2024-03-18T23:06:01.735 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:01.734+0000 7fe96cff9640 1 -- 172.21.15.49:0/3792733483 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+18479 (secure 0 0 0) 0x7fe9580c1460 con 0x7fe970101930 2024-03-18T23:06:01.736 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-18T23:06:01.736 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":82,"fsid":"c63591ea-e57a-11ee-95c9-87774f69a715","created":"2024-03-18T22:57:58.734467+0000","modified":"2024-03-18T23:05:17.581588+0000","last_up_change":"2024-03-18T23:05:16.576310+0000","last_in_change":"2024-03-18T23:04:57.024067+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-18T23:01:28.363432+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e7fc9daa-4098-4ba5-8019-9aaef1aab3e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":1651162501},{"type":"v1","addr":"172.21.15.49:6803","nonce":1651162501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":1651162501},{"type":"v1","addr":"172.21.15.49:6805","nonce":1651162501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":1651162501},{"type":"v1","addr":"172.21.15.49:6809","nonce":1651162501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":1651162501},{"type":"v1","addr":"172.21.15.49:6807","nonce":1651162501}]},"public_addr":"172.21.15.49:6803/1651162501","cluster_addr":"172.21.15.49:6805/1651162501","heartbeat_back_addr":"172.21.15.49:6809/1651162501","heartbeat_front_addr":"172.21.15.49:6807/1651162501","state":["exists","up"]},{"osd":1,"uuid":"40ac929c-e90f-434c-a796-9aabd686f1fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":1766700359},{"type":"v1","addr":"172.21.15.139:6801","nonce":1766700359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":1766700359},{"type":"v1","addr":"172.21.15.139:6803","nonce":1766700359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":1766700359},{"type":"v1","addr":"172.21.15.139:6807","nonce":1766700359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":1766700359},{"type":"v1","addr":"172.21.15.139:6805","nonce":1766700359}]},"public_addr":"172.21.15.139:6801/1766700359","cluster_addr":"172.21.15.139:6803/1766700359","heartbeat_back_addr":"172.21.15.139:6807/1766700359","heartbeat_front_addr":"172.21.15.139:6805/1766700359","state":["exists","up"]},{"osd":2,"uuid":"8ae3b666-00b4-45ed-a5ee-4d0771edabb6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":3752671234},{"type":"v1","addr":"172.21.15.175:6801","nonce":3752671234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":3752671234},{"type":"v1","addr":"172.21.15.175:6803","nonce":3752671234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":3752671234},{"type":"v1","addr":"172.21.15.175:6807","nonce":3752671234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":3752671234},{"type":"v1","addr":"172.21.15.175:6805","nonce":3752671234}]},"public_addr":"172.21.15.175:6801/3752671234","cluster_addr":"172.21.15.175:6803/3752671234","heartbeat_back_addr":"172.21.15.175:6807/3752671234","heartbeat_front_addr":"172.21.15.175:6805/3752671234","state":["exists","up"]},{"osd":3,"uuid":"1a62e89d-45d4-4237-b813-7f96a59aca7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":956160536},{"type":"v1","addr":"172.21.15.49:6811","nonce":956160536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":956160536},{"type":"v1","addr":"172.21.15.49:6813","nonce":956160536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":956160536},{"type":"v1","addr":"172.21.15.49:6817","nonce":956160536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":956160536},{"type":"v1","addr":"172.21.15.49:6815","nonce":956160536}]},"public_addr":"172.21.15.49:6811/956160536","cluster_addr":"172.21.15.49:6813/956160536","heartbeat_back_addr":"172.21.15.49:6817/956160536","heartbeat_front_addr":"172.21.15.49:6815/956160536","state":["exists","up"]},{"osd":4,"uuid":"c715101c-0c3f-4b98-b142-d64a35e187df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":3129712668},{"type":"v1","addr":"172.21.15.139:6809","nonce":3129712668}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":3129712668},{"type":"v1","addr":"172.21.15.139:6811","nonce":3129712668}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":3129712668},{"type":"v1","addr":"172.21.15.139:6815","nonce":3129712668}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":3129712668},{"type":"v1","addr":"172.21.15.139:6813","nonce":3129712668}]},"public_addr":"172.21.15.139:6809/3129712668","cluster_addr":"172.21.15.139:6811/3129712668","heartbeat_back_addr":"172.21.15.139:6815/3129712668","heartbeat_front_addr":"172.21.15.139:6813/3129712668","state":["exists","up"]},{"osd":5,"uuid":"4b641f81-77eb-45fd-8464-511e7b3062b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":2774567215},{"type":"v1","addr":"172.21.15.175:6809","nonce":2774567215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":2774567215},{"type":"v1","addr":"172.21.15.175:6811","nonce":2774567215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":2774567215},{"type":"v1","addr":"172.21.15.175:6815","nonce":2774567215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":2774567215},{"type":"v1","addr":"172.21.15.175:6813","nonce":2774567215}]},"public_addr":"172.21.15.175:6809/2774567215","cluster_addr":"172.21.15.175:6811/2774567215","heartbeat_back_addr":"172.21.15.175:6815/2774567215","heartbeat_front_addr":"172.21.15.175:6813/2774567215","state":["exists","up"]},{"osd":6,"uuid":"653da8e0-1197-4b48-abed-470f2a9c4dce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":1472215461},{"type":"v1","addr":"172.21.15.49:6819","nonce":1472215461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":1472215461},{"type":"v1","addr":"172.21.15.49:6821","nonce":1472215461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":1472215461},{"type":"v1","addr":"172.21.15.49:6825","nonce":1472215461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":1472215461},{"type":"v1","addr":"172.21.15.49:6823","nonce":1472215461}]},"public_addr":"172.21.15.49:6819/1472215461","cluster_addr":"172.21.15.49:6821/1472215461","heartbeat_back_addr":"172.21.15.49:6825/1472215461","heartbeat_front_addr":"172.21.15.49:6823/1472215461","state":["exists","up"]},{"osd":7,"uuid":"ad157c9b-4b75-451c-8740-630f0975b7fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":2163986787},{"type":"v1","addr":"172.21.15.139:6817","nonce":2163986787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":2163986787},{"type":"v1","addr":"172.21.15.139:6819","nonce":2163986787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":2163986787},{"type":"v1","addr":"172.21.15.139:6823","nonce":2163986787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":2163986787},{"type":"v1","addr":"172.21.15.139:6821","nonce":2163986787}]},"public_addr":"172.21.15.139:6817/2163986787","cluster_addr":"172.21.15.139:6819/2163986787","heartbeat_back_addr":"172.21.15.139:6823/2163986787","heartbeat_front_addr":"172.21.15.139:6821/2163986787","state":["exists","up"]},{"osd":8,"uuid":"c8e75f15-df75-4d5d-9b92-a064bffb6c01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":1194026992},{"type":"v1","addr":"172.21.15.175:6817","nonce":1194026992}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1194026992},{"type":"v1","addr":"172.21.15.175:6819","nonce":1194026992}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":1194026992},{"type":"v1","addr":"172.21.15.175:6823","nonce":1194026992}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1194026992},{"type":"v1","addr":"172.21.15.175:6821","nonce":1194026992}]},"public_addr":"172.21.15.175:6817/1194026992","cluster_addr":"172.21.15.175:6819/1194026992","heartbeat_back_addr":"172.21.15.175:6823/1194026992","heartbeat_front_addr":"172.21.15.175:6821/1194026992","state":["exists","up"]},{"osd":9,"uuid":"edb5ffef-b6a2-4941-b5ab-40f982ad30d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":2742084884},{"type":"v1","addr":"172.21.15.49:6827","nonce":2742084884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":2742084884},{"type":"v1","addr":"172.21.15.49:6829","nonce":2742084884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":2742084884},{"type":"v1","addr":"172.21.15.49:6833","nonce":2742084884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":2742084884},{"type":"v1","addr":"172.21.15.49:6831","nonce":2742084884}]},"public_addr":"172.21.15.49:6827/2742084884","cluster_addr":"172.21.15.49:6829/2742084884","heartbeat_back_addr":"172.21.15.49:6833/2742084884","heartbeat_front_addr":"172.21.15.49:6831/2742084884","state":["exists","up"]},{"osd":10,"uuid":"3945e46d-766a-4c4a-8821-d43648dd1578","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":2404740201},{"type":"v1","addr":"172.21.15.139:6825","nonce":2404740201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":2404740201},{"type":"v1","addr":"172.21.15.139:6827","nonce":2404740201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":2404740201},{"type":"v1","addr":"172.21.15.139:6831","nonce":2404740201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":2404740201},{"type":"v1","addr":"172.21.15.139:6829","nonce":2404740201}]},"public_addr":"172.21.15.139:6825/2404740201","cluster_addr":"172.21.15.139:6827/2404740201","heartbeat_back_addr":"172.21.15.139:6831/2404740201","heartbeat_front_addr":"172.21.15.139:6829/2404740201","state":["exists","up"]},{"osd":11,"uuid":"9954ba15-d877-46cc-b24a-893f9168deb8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":3519503868},{"type":"v1","addr":"172.21.15.175:6825","nonce":3519503868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":3519503868},{"type":"v1","addr":"172.21.15.175:6827","nonce":3519503868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":3519503868},{"type":"v1","addr":"172.21.15.175:6831","nonce":3519503868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":3519503868},{"type":"v1","addr":"172.21.15.175:6829","nonce":3519503868}]},"public_addr":"172.21.15.175:6825/3519503868","cluster_addr":"172.21.15.175:6827/3519503868","heartbeat_back_addr":"172.21.15.175:6831/3519503868","heartbeat_front_addr":"172.21.15.175:6829/3519503868","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:00:38.367865+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:01:02.237687+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:01:25.609751+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:01:51.268572+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:02:16.514605+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:02:41.586523+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:03:07.029964+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:03:32.956250+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:03:57.547984+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:04:23.948972+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:04:49.667210+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:05:15.284693+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.49:0/563527416":"2024-03-19T22:58:59.806253+0000","172.21.15.49:0/1036383534":"2024-03-19T22:58:59.806253+0000","172.21.15.49:6801/2609458951":"2024-03-19T22:58:59.806253+0000","172.21.15.49:0/21283102":"2024-03-19T22:58:31.972474+0000","172.21.15.49:0/1292058922":"2024-03-19T22:58:31.972474+0000","172.21.15.49:6800/1985190312":"2024-03-19T22:58:31.972474+0000","172.21.15.49:0/1912446700":"2024-03-19T22:58:59.806253+0000","172.21.15.49:0/2176210039":"2024-03-19T22:58:31.972474+0000","172.21.15.49:6801/1985190312":"2024-03-19T22:58:31.972474+0000","172.21.15.49:6800/2609458951":"2024-03-19T22:58:59.806253+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-18T23:06:01.737 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:01.736+0000 7fe975f52640 1 -- 172.21.15.49:0/3792733483 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fe95c0a2800 msgr2=0x7fe95c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:01.738 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:01.736+0000 7fe975f52640 1 --2- 172.21.15.49:0/3792733483 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fe95c0a2800 0x7fe95c0a4cc0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fe970101790 tx=0x7fe960005f90 comp rx=0 tx=0).stop 2024-03-18T23:06:01.738 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:01.736+0000 7fe975f52640 1 -- 172.21.15.49:0/3792733483 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe970101930 msgr2=0x7fe970198080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:01.738 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:01.736+0000 7fe975f52640 1 --2- 172.21.15.49:0/3792733483 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe970101930 0x7fe970198080 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fe958009230 tx=0x7fe95802c650 comp rx=0 tx=0).stop 2024-03-18T23:06:01.738 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:01.736+0000 7fe975f52640 1 -- 172.21.15.49:0/3792733483 shutdown_connections 2024-03-18T23:06:01.738 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:01.736+0000 7fe975f52640 1 --2- 172.21.15.49:0/3792733483 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fe95c0a2800 0x7fe95c0a4cc0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:01.738 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:01.736+0000 7fe975f52640 1 --2- 172.21.15.49:0/3792733483 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe9701022f0 0x7fe97019f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:01.738 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:01.736+0000 7fe975f52640 1 --2- 172.21.15.49:0/3792733483 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe970101930 0x7fe970198080 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:01.738 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:01.736+0000 7fe975f52640 1 --2- 172.21.15.49:0/3792733483 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe970100740 0x7fe970197b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:01.738 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:01.736+0000 7fe975f52640 1 -- 172.21.15.49:0/3792733483 >> 172.21.15.49:0/3792733483 conn(0x7fe9700fbcb0 msgr2=0x7fe970104d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:06:01.738 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:01.737+0000 7fe975f52640 1 -- 172.21.15.49:0/3792733483 shutdown_connections 2024-03-18T23:06:01.738 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:01.737+0000 7fe975f52640 1 -- 172.21.15.49:0/3792733483 wait complete. 2024-03-18T23:06:01.749 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-18T23:01:28.363432+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-18T23:06:01.749 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-18T23:06:01.867 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:01.863+0000 7f7ddebf3640 1 -- 172.21.15.49:0/1904824284 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7dd8101830 msgr2=0x7f7dd8101c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:01.867 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:01.863+0000 7f7ddebf3640 1 --2- 172.21.15.49:0/1904824284 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7dd8101830 0x7f7dd8101c50 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f7dd4009900 tx=0x7f7dd402bc30 comp rx=0 tx=0).stop 2024-03-18T23:06:01.867 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:01.864+0000 7f7ddebf3640 1 -- 172.21.15.49:0/1904824284 shutdown_connections 2024-03-18T23:06:01.867 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:01.864+0000 7f7ddebf3640 1 --2- 172.21.15.49:0/1904824284 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7dd81033e0 0x7f7dd8109c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:01.867 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:01.864+0000 7f7ddebf3640 1 --2- 172.21.15.49:0/1904824284 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7dd8102a20 0x7f7dd8102ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:01.867 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:01.864+0000 7f7ddebf3640 1 --2- 172.21.15.49:0/1904824284 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7dd8101830 0x7f7dd8101c50 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:01.867 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:01.864+0000 7f7ddebf3640 1 -- 172.21.15.49:0/1904824284 >> 172.21.15.49:0/1904824284 conn(0x7f7dd80fcdc0 msgr2=0x7f7dd80ff1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:06:01.867 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:01.864+0000 7f7ddebf3640 1 -- 172.21.15.49:0/1904824284 shutdown_connections 2024-03-18T23:06:01.868 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:01.864+0000 7f7ddebf3640 1 -- 172.21.15.49:0/1904824284 wait complete. 2024-03-18T23:06:01.868 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:01.864+0000 7f7ddebf3640 1 Processor -- start 2024-03-18T23:06:01.868 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:01.864+0000 7f7ddebf3640 1 -- start start 2024-03-18T23:06:01.868 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:01.865+0000 7f7ddebf3640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7dd8101830 0x7f7dd8197bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:06:01.868 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:01.865+0000 7f7ddebf3640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7dd8102a20 0x7f7dd8198100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:06:01.868 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:01.865+0000 7f7ddebf3640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7dd81033e0 0x7f7dd819f170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:06:01.868 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:01.865+0000 7f7ddebf3640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f7dd810d480 con 0x7f7dd81033e0 2024-03-18T23:06:01.868 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:01.865+0000 7f7ddebf3640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f7dd810d300 con 0x7f7dd8101830 2024-03-18T23:06:01.868 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:01.865+0000 7f7ddebf3640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f7dd810d600 con 0x7f7dd8102a20 2024-03-18T23:06:01.868 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:01.865+0000 7f7ddd190640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7dd81033e0 0x7f7dd819f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:06:01.868 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:01.865+0000 7f7ddd190640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7dd81033e0 0x7f7dd819f170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:50160/0 (socket says 172.21.15.49:50160) 2024-03-18T23:06:01.868 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:01.865+0000 7f7ddd190640 1 -- 172.21.15.49:0/1223910015 learned_addr learned my addr 172.21.15.49:0/1223910015 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:06:01.868 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:01.865+0000 7f7dcffff640 1 --2- 172.21.15.49:0/1223910015 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7dd8102a20 0x7f7dd8198100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:06:01.868 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:01.865+0000 7f7ddc98f640 1 --2- 172.21.15.49:0/1223910015 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7dd8101830 0x7f7dd8197bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:06:01.869 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:01.865+0000 7f7ddd190640 1 -- 172.21.15.49:0/1223910015 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7dd8102a20 msgr2=0x7f7dd8198100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:01.869 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:01.865+0000 7f7ddd190640 1 --2- 172.21.15.49:0/1223910015 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7dd8102a20 0x7f7dd8198100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:01.869 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:01.865+0000 7f7ddd190640 1 -- 172.21.15.49:0/1223910015 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7dd8101830 msgr2=0x7f7dd8197bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:01.869 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:01.865+0000 7f7ddd190640 1 --2- 172.21.15.49:0/1223910015 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7dd8101830 0x7f7dd8197bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:01.869 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:01.865+0000 7f7ddd190640 1 -- 172.21.15.49:0/1223910015 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7dd819f7e0 con 0x7f7dd81033e0 2024-03-18T23:06:01.869 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:01.865+0000 7f7ddd190640 1 --2- 172.21.15.49:0/1223910015 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7dd81033e0 0x7f7dd819f170 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f7dd0009fd0 tx=0x7f7dd000b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:06:01.869 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:01.865+0000 7f7dcdffb640 1 -- 172.21.15.49:0/1223910015 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dd0017070 con 0x7f7dd81033e0 2024-03-18T23:06:01.869 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:01.866+0000 7f7ddebf3640 1 -- 172.21.15.49:0/1223910015 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7dd819fad0 con 0x7f7dd81033e0 2024-03-18T23:06:01.869 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:01.866+0000 7f7dcdffb640 1 -- 172.21.15.49:0/1223910015 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7dd0003e50 con 0x7f7dd81033e0 2024-03-18T23:06:01.869 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:01.866+0000 7f7dcdffb640 1 -- 172.21.15.49:0/1223910015 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dd00106c0 con 0x7f7dd81033e0 2024-03-18T23:06:01.869 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:01.866+0000 7f7ddebf3640 1 -- 172.21.15.49:0/1223910015 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7dd819ffd0 con 0x7f7dd81033e0 2024-03-18T23:06:01.870 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:01.867+0000 7f7ddebf3640 1 -- 172.21.15.49:0/1223910015 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7dd8108c70 con 0x7f7dd81033e0 2024-03-18T23:06:01.870 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:01.868+0000 7f7dcdffb640 1 -- 172.21.15.49:0/1223910015 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7f7dd0003970 con 0x7f7dd81033e0 2024-03-18T23:06:01.870 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:01.869+0000 7f7dcdffb640 1 --2- 172.21.15.49:0/1223910015 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f7dc00a2800 0x7f7dc00a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:06:01.870 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:01.869+0000 7f7ddc98f640 1 --2- 172.21.15.49:0/1223910015 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f7dc00a2800 0x7f7dc00a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:06:01.871 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:01.869+0000 7f7dcdffb640 1 -- 172.21.15.49:0/1223910015 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f7dd00ce2c0 con 0x7f7dd81033e0 2024-03-18T23:06:01.871 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:01.870+0000 7f7ddc98f640 1 --2- 172.21.15.49:0/1223910015 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f7dc00a2800 0x7f7dc00a4cc0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f7dd4000c00 tx=0x7f7dd4005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:06:01.875 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:01.874+0000 7f7dcdffb640 1 -- 172.21.15.49:0/1223910015 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f7dd009b1b0 con 0x7f7dd81033e0 2024-03-18T23:06:02.072 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:02.072+0000 7f7ddebf3640 1 -- 172.21.15.49:0/1223910015 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f7dd802d3f0 con 0x7f7dd81033e0 2024-03-18T23:06:02.073 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:02.072+0000 7f7dcdffb640 1 -- 172.21.15.49:0/1223910015 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) ==== 93+0+10 (secure 0 0 0) 0x7f7dd009f250 con 0x7f7dd81033e0 2024-03-18T23:06:02.073 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 1 2024-03-18T23:06:02.075 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:02.074+0000 7f7ddebf3640 1 -- 172.21.15.49:0/1223910015 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f7dc00a2800 msgr2=0x7f7dc00a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:02.076 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:02.074+0000 7f7ddebf3640 1 --2- 172.21.15.49:0/1223910015 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f7dc00a2800 0x7f7dc00a4cc0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f7dd4000c00 tx=0x7f7dd4005fd0 comp rx=0 tx=0).stop 2024-03-18T23:06:02.076 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:02.074+0000 7f7ddebf3640 1 -- 172.21.15.49:0/1223910015 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7dd81033e0 msgr2=0x7f7dd819f170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:02.076 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:02.074+0000 7f7ddebf3640 1 --2- 172.21.15.49:0/1223910015 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7dd81033e0 0x7f7dd819f170 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f7dd0009fd0 tx=0x7f7dd000b260 comp rx=0 tx=0).stop 2024-03-18T23:06:02.076 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:02.074+0000 7f7ddebf3640 1 -- 172.21.15.49:0/1223910015 shutdown_connections 2024-03-18T23:06:02.076 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:02.074+0000 7f7ddebf3640 1 --2- 172.21.15.49:0/1223910015 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f7dc00a2800 0x7f7dc00a4cc0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:02.076 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:02.074+0000 7f7ddebf3640 1 --2- 172.21.15.49:0/1223910015 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7dd81033e0 0x7f7dd819f170 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:02.076 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:02.074+0000 7f7ddebf3640 1 --2- 172.21.15.49:0/1223910015 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7dd8102a20 0x7f7dd8198100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:02.076 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:02.074+0000 7f7ddebf3640 1 --2- 172.21.15.49:0/1223910015 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7dd8101830 0x7f7dd8197bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:02.076 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:02.074+0000 7f7ddebf3640 1 -- 172.21.15.49:0/1223910015 >> 172.21.15.49:0/1223910015 conn(0x7f7dd80fcdc0 msgr2=0x7f7dd8105e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:06:02.076 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:02.074+0000 7f7ddebf3640 1 -- 172.21.15.49:0/1223910015 shutdown_connections 2024-03-18T23:06:02.076 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:02.074+0000 7f7ddebf3640 1 -- 172.21.15.49:0/1223910015 wait complete. 2024-03-18T23:06:02.086 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-03-18T23:06:02.199 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:02.196+0000 7f6307a81640 1 -- 172.21.15.49:0/2183668753 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6300101930 msgr2=0x7f6300101db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:02.199 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:02.196+0000 7f6307a81640 1 --2- 172.21.15.49:0/2183668753 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6300101930 0x7f6300101db0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f62f0009900 tx=0x7f62f002bc30 comp rx=0 tx=0).stop 2024-03-18T23:06:02.199 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:02.196+0000 7f6307a81640 1 -- 172.21.15.49:0/2183668753 shutdown_connections 2024-03-18T23:06:02.200 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:02.196+0000 7f6307a81640 1 --2- 172.21.15.49:0/2183668753 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f63001022f0 0x7f6300108b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:02.200 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:02.196+0000 7f6307a81640 1 --2- 172.21.15.49:0/2183668753 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6300101930 0x7f6300101db0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:02.200 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:02.196+0000 7f6307a81640 1 --2- 172.21.15.49:0/2183668753 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6300100740 0x7f6300100b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:02.200 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:02.196+0000 7f6307a81640 1 -- 172.21.15.49:0/2183668753 >> 172.21.15.49:0/2183668753 conn(0x7f63000fbcb0 msgr2=0x7f63000fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:06:02.200 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:02.196+0000 7f6307a81640 1 -- 172.21.15.49:0/2183668753 shutdown_connections 2024-03-18T23:06:02.200 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:02.196+0000 7f6307a81640 1 -- 172.21.15.49:0/2183668753 wait complete. 2024-03-18T23:06:02.200 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:02.197+0000 7f6307a81640 1 Processor -- start 2024-03-18T23:06:02.200 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:02.197+0000 7f6307a81640 1 -- start start 2024-03-18T23:06:02.200 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:02.197+0000 7f6307a81640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6300100740 0x7f6300197b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:06:02.200 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:02.197+0000 7f6307a81640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6300101930 0x7f63001980c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:06:02.200 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:02.197+0000 7f6307a81640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f63001022f0 0x7f630019f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:06:02.200 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:02.197+0000 7f6307a81640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f630010c3c0 con 0x7f6300101930 2024-03-18T23:06:02.200 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:02.197+0000 7f6307a81640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f630010c240 con 0x7f6300100740 2024-03-18T23:06:02.200 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:02.197+0000 7f6307a81640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f630010c540 con 0x7f63001022f0 2024-03-18T23:06:02.201 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:02.197+0000 7f630501c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6300101930 0x7f63001980c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:06:02.201 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:02.197+0000 7f630501c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6300101930 0x7f63001980c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:50178/0 (socket says 172.21.15.49:50178) 2024-03-18T23:06:02.201 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:02.197+0000 7f630501c640 1 -- 172.21.15.49:0/2981897393 learned_addr learned my addr 172.21.15.49:0/2981897393 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:06:02.201 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:02.197+0000 7f630581d640 1 --2- 172.21.15.49:0/2981897393 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6300100740 0x7f6300197b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:06:02.201 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:02.197+0000 7f630601e640 1 --2- 172.21.15.49:0/2981897393 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f63001022f0 0x7f630019f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:06:02.201 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:02.197+0000 7f630501c640 1 -- 172.21.15.49:0/2981897393 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f63001022f0 msgr2=0x7f630019f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:02.201 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:02.197+0000 7f630501c640 1 --2- 172.21.15.49:0/2981897393 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f63001022f0 0x7f630019f130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:02.201 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:02.197+0000 7f630501c640 1 -- 172.21.15.49:0/2981897393 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6300100740 msgr2=0x7f6300197b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:02.201 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:02.197+0000 7f630501c640 1 --2- 172.21.15.49:0/2981897393 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6300100740 0x7f6300197b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:02.201 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:02.197+0000 7f630501c640 1 -- 172.21.15.49:0/2981897393 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f630019f7a0 con 0x7f6300101930 2024-03-18T23:06:02.201 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:02.197+0000 7f630501c640 1 --2- 172.21.15.49:0/2981897393 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6300101930 0x7f63001980c0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f62f0009230 tx=0x7f62f0005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:06:02.201 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:02.198+0000 7f62f6ffd640 1 -- 172.21.15.49:0/2981897393 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62f0041070 con 0x7f6300101930 2024-03-18T23:06:02.201 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:02.198+0000 7f6307a81640 1 -- 172.21.15.49:0/2981897393 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f630019fa30 con 0x7f6300101930 2024-03-18T23:06:02.201 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:02.198+0000 7f62f6ffd640 1 -- 172.21.15.49:0/2981897393 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f62f00044d0 con 0x7f6300101930 2024-03-18T23:06:02.202 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:02.198+0000 7f62f6ffd640 1 -- 172.21.15.49:0/2981897393 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62f0034440 con 0x7f6300101930 2024-03-18T23:06:02.202 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:02.198+0000 7f6307a81640 1 -- 172.21.15.49:0/2981897393 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f630019ff20 con 0x7f6300101930 2024-03-18T23:06:02.202 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:02.199+0000 7f6307a81640 1 -- 172.21.15.49:0/2981897393 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f63001a00f0 con 0x7f6300101930 2024-03-18T23:06:02.203 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:02.200+0000 7f62f6ffd640 1 -- 172.21.15.49:0/2981897393 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7f62f0044070 con 0x7f6300101930 2024-03-18T23:06:02.203 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:02.200+0000 7f62f6ffd640 1 --2- 172.21.15.49:0/2981897393 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f62ec0a2800 0x7f62ec0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:06:02.203 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:02.200+0000 7f62f6ffd640 1 -- 172.21.15.49:0/2981897393 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f62f00f01c0 con 0x7f6300101930 2024-03-18T23:06:02.203 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:02.200+0000 7f630581d640 1 --2- 172.21.15.49:0/2981897393 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f62ec0a2800 0x7f62ec0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:06:02.203 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:02.201+0000 7f630581d640 1 --2- 172.21.15.49:0/2981897393 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f62ec0a2800 0x7f62ec0a4cc0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f6300101790 tx=0x7f62fc005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:06:02.206 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:02.205+0000 7f62f6ffd640 1 -- 172.21.15.49:0/2981897393 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f62f00bd0b0 con 0x7f6300101930 2024-03-18T23:06:02.364 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:06:02 smithi049 ceph-mon[62991]: mds.? [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] up:boot 2024-03-18T23:06:02.364 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:06:02 smithi049 ceph-mon[62991]: fsmap 12 up:standby 2024-03-18T23:06:02.365 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:06:02 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-18T23:06:02.365 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:06:02 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:06:02.365 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:06:02 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:06:02.365 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:06:02 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:06:02.365 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:06:02 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:06:02.365 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:06:02 smithi049 ceph-mon[62991]: from='client.? 172.21.15.49:0/3070460050' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T23:06:02.365 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:06:02 smithi049 ceph-mon[62991]: from='client.? 172.21.15.49:0/3792733483' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T23:06:02.365 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:06:02 smithi049 ceph-mon[62991]: from='client.? 172.21.15.49:0/1223910015' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-18T23:06:02.398 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:02.397+0000 7f6307a81640 1 -- 172.21.15.49:0/2981897393 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f63001a0480 con 0x7f6300101930 2024-03-18T23:06:02.399 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:02.398+0000 7f62f6ffd640 1 -- 172.21.15.49:0/2981897393 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v82) ==== 84+0+65 (secure 0 0 0) 0x7f62f00c1150 con 0x7f6300101930 2024-03-18T23:06:02.399 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-18T23:06:02.399 INFO:teuthology.orchestra.run.smithi049.stdout:[ 2024-03-18T23:06:02.399 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-18T23:06:02.399 INFO:teuthology.orchestra.run.smithi049.stdout: "poolnum": 1, 2024-03-18T23:06:02.399 INFO:teuthology.orchestra.run.smithi049.stdout: "poolname": ".mgr" 2024-03-18T23:06:02.399 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:06:02.399 INFO:teuthology.orchestra.run.smithi049.stdout:] 2024-03-18T23:06:02.401 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:02.400+0000 7f6307a81640 1 -- 172.21.15.49:0/2981897393 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f62ec0a2800 msgr2=0x7f62ec0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:02.401 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:02.400+0000 7f6307a81640 1 --2- 172.21.15.49:0/2981897393 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f62ec0a2800 0x7f62ec0a4cc0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f6300101790 tx=0x7f62fc005f90 comp rx=0 tx=0).stop 2024-03-18T23:06:02.401 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:02.400+0000 7f6307a81640 1 -- 172.21.15.49:0/2981897393 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6300101930 msgr2=0x7f63001980c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:02.401 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:02.400+0000 7f6307a81640 1 --2- 172.21.15.49:0/2981897393 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6300101930 0x7f63001980c0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f62f0009230 tx=0x7f62f0005d50 comp rx=0 tx=0).stop 2024-03-18T23:06:02.401 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:02.400+0000 7f6307a81640 1 -- 172.21.15.49:0/2981897393 shutdown_connections 2024-03-18T23:06:02.401 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:02.400+0000 7f6307a81640 1 --2- 172.21.15.49:0/2981897393 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f62ec0a2800 0x7f62ec0a4cc0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:02.401 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:02.400+0000 7f6307a81640 1 --2- 172.21.15.49:0/2981897393 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f63001022f0 0x7f630019f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:02.401 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:02.400+0000 7f6307a81640 1 --2- 172.21.15.49:0/2981897393 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6300101930 0x7f63001980c0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:02.401 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:02.400+0000 7f6307a81640 1 --2- 172.21.15.49:0/2981897393 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6300100740 0x7f6300197b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:02.401 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:02.400+0000 7f6307a81640 1 -- 172.21.15.49:0/2981897393 >> 172.21.15.49:0/2981897393 conn(0x7f63000fbcb0 msgr2=0x7f6300068980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:06:02.402 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:02.400+0000 7f6307a81640 1 -- 172.21.15.49:0/2981897393 shutdown_connections 2024-03-18T23:06:02.402 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:02.400+0000 7f6307a81640 1 -- 172.21.15.49:0/2981897393 wait complete. 2024-03-18T23:06:02.412 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-03-18T23:06:02.412 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-03-18T23:06:02.413 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:06:02 smithi139 ceph-mon[67282]: mds.? [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] up:boot 2024-03-18T23:06:02.413 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:06:02 smithi139 ceph-mon[67282]: fsmap 12 up:standby 2024-03-18T23:06:02.413 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:06:02 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-18T23:06:02.413 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:06:02 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:06:02.413 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:06:02 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:06:02.413 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:06:02 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:06:02.413 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:06:02 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:06:02.413 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:06:02 smithi139 ceph-mon[67282]: from='client.? 172.21.15.49:0/3070460050' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T23:06:02.413 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:06:02 smithi139 ceph-mon[67282]: from='client.? 172.21.15.49:0/3792733483' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T23:06:02.413 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:06:02 smithi139 ceph-mon[67282]: from='client.? 172.21.15.49:0/1223910015' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-18T23:06:02.462 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:06:02 smithi175 ceph-mon[67352]: mds.? [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] up:boot 2024-03-18T23:06:02.462 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:06:02 smithi175 ceph-mon[67352]: fsmap 12 up:standby 2024-03-18T23:06:02.462 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:06:02 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-18T23:06:02.462 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:06:02 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:06:02.462 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:06:02 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:06:02.462 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:06:02 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:06:02.463 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:06:02 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:06:02.463 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:06:02 smithi175 ceph-mon[67352]: from='client.? 172.21.15.49:0/3070460050' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T23:06:02.463 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:06:02 smithi175 ceph-mon[67352]: from='client.? 172.21.15.49:0/3792733483' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T23:06:02.463 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:06:02 smithi175 ceph-mon[67352]: from='client.? 172.21.15.49:0/1223910015' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-18T23:06:02.530 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:02.527+0000 7f762de4b640 1 -- 172.21.15.49:0/271038271 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7628102a30 msgr2=0x7f7628102eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:02.530 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:02.527+0000 7f762de4b640 1 --2- 172.21.15.49:0/271038271 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7628102a30 0x7f7628102eb0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f761c009900 tx=0x7f761c02bc30 comp rx=0 tx=0).stop 2024-03-18T23:06:02.531 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:02.527+0000 7f762de4b640 1 -- 172.21.15.49:0/271038271 shutdown_connections 2024-03-18T23:06:02.531 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:02.527+0000 7f762de4b640 1 --2- 172.21.15.49:0/271038271 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f76281033f0 0x7f7628109c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:02.531 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:02.527+0000 7f762de4b640 1 --2- 172.21.15.49:0/271038271 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7628102a30 0x7f7628102eb0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:02.531 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:02.527+0000 7f762de4b640 1 --2- 172.21.15.49:0/271038271 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7628101840 0x7f7628101c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:02.531 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:02.527+0000 7f762de4b640 1 -- 172.21.15.49:0/271038271 >> 172.21.15.49:0/271038271 conn(0x7f76280fcdb0 msgr2=0x7f76280ff1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:06:02.531 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:02.527+0000 7f762de4b640 1 -- 172.21.15.49:0/271038271 shutdown_connections 2024-03-18T23:06:02.531 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:02.527+0000 7f762de4b640 1 -- 172.21.15.49:0/271038271 wait complete. 2024-03-18T23:06:02.531 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:02.527+0000 7f762de4b640 1 Processor -- start 2024-03-18T23:06:02.531 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:02.527+0000 7f762de4b640 1 -- start start 2024-03-18T23:06:02.531 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:02.528+0000 7f762de4b640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7628101840 0x7f76280755d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:06:02.531 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:02.528+0000 7f762de4b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7628102a30 0x7f7628073c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:06:02.531 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:02.528+0000 7f762de4b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f76281033f0 0x7f7628074150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:06:02.531 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:02.528+0000 7f762de4b640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f762810d510 con 0x7f76281033f0 2024-03-18T23:06:02.532 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:02.528+0000 7f762de4b640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f762810d390 con 0x7f7628102a30 2024-03-18T23:06:02.532 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:02.528+0000 7f762de4b640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f762810d690 con 0x7f7628101840 2024-03-18T23:06:02.532 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:02.528+0000 7f7627fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f76281033f0 0x7f7628074150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:06:02.532 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:02.528+0000 7f7627fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f76281033f0 0x7f7628074150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:50200/0 (socket says 172.21.15.49:50200) 2024-03-18T23:06:02.532 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:02.528+0000 7f7627fff640 1 -- 172.21.15.49:0/613518549 learned_addr learned my addr 172.21.15.49:0/613518549 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:06:02.532 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:02.528+0000 7f7626ffd640 1 --2- 172.21.15.49:0/613518549 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7628102a30 0x7f7628073c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:06:02.532 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:02.528+0000 7f76277fe640 1 --2- 172.21.15.49:0/613518549 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7628101840 0x7f76280755d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:06:02.532 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:02.528+0000 7f7627fff640 1 -- 172.21.15.49:0/613518549 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7628101840 msgr2=0x7f76280755d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:02.532 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:02.528+0000 7f7627fff640 1 --2- 172.21.15.49:0/613518549 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7628101840 0x7f76280755d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:02.532 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:02.528+0000 7f7627fff640 1 -- 172.21.15.49:0/613518549 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7628102a30 msgr2=0x7f7628073c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:02.532 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:02.528+0000 7f7627fff640 1 --2- 172.21.15.49:0/613518549 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7628102a30 0x7f7628073c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:02.532 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:02.528+0000 7f7627fff640 1 -- 172.21.15.49:0/613518549 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76280749e0 con 0x7f76281033f0 2024-03-18T23:06:02.533 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:02.528+0000 7f76277fe640 1 --2- 172.21.15.49:0/613518549 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7628101840 0x7f76280755d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T23:06:02.533 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:02.528+0000 7f7627fff640 1 --2- 172.21.15.49:0/613518549 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f76281033f0 0x7f7628074150 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f7620009fd0 tx=0x7f762000b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:06:02.533 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:02.528+0000 7f7624ff9640 1 -- 172.21.15.49:0/613518549 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7620017070 con 0x7f76281033f0 2024-03-18T23:06:02.533 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:02.528+0000 7f7624ff9640 1 -- 172.21.15.49:0/613518549 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7620003e50 con 0x7f76281033f0 2024-03-18T23:06:02.533 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:02.528+0000 7f762de4b640 1 -- 172.21.15.49:0/613518549 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76281a1ca0 con 0x7f76281033f0 2024-03-18T23:06:02.533 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:02.528+0000 7f7624ff9640 1 -- 172.21.15.49:0/613518549 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76200106c0 con 0x7f76281033f0 2024-03-18T23:06:02.533 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:02.528+0000 7f762de4b640 1 -- 172.21.15.49:0/613518549 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76281a2170 con 0x7f76281033f0 2024-03-18T23:06:02.533 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:02.529+0000 7f762de4b640 1 -- 172.21.15.49:0/613518549 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7628108b60 con 0x7f76281033f0 2024-03-18T23:06:02.534 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:02.530+0000 7f7624ff9640 1 -- 172.21.15.49:0/613518549 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7f7620003970 con 0x7f76281033f0 2024-03-18T23:06:02.534 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:02.531+0000 7f7624ff9640 1 --2- 172.21.15.49:0/613518549 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f76100a2800 0x7f76100a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:06:02.534 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:02.531+0000 7f76277fe640 1 --2- 172.21.15.49:0/613518549 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f76100a2800 0x7f76100a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:06:02.534 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:02.531+0000 7f7624ff9640 1 -- 172.21.15.49:0/613518549 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f76200ce2e0 con 0x7f76281033f0 2024-03-18T23:06:02.534 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:02.531+0000 7f76277fe640 1 --2- 172.21.15.49:0/613518549 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f76100a2800 0x7f76100a4cc0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f7628102890 tx=0x7f7618005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:06:02.536 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:02.535+0000 7f7624ff9640 1 -- 172.21.15.49:0/613518549 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f762009b820 con 0x7f76281033f0 2024-03-18T23:06:02.724 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:02.724+0000 7f762de4b640 1 -- 172.21.15.49:0/613518549 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f762802d410 con 0x7f76281033f0 2024-03-18T23:06:03.188 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:03.187+0000 7f7624ff9640 1 -- 172.21.15.49:0/613518549 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v83) ==== 138+0+0 (secure 0 0 0) 0x7f762009b120 con 0x7f76281033f0 2024-03-18T23:06:03.188 INFO:teuthology.orchestra.run.smithi049.stderr:pool 'cephfs_metadata' created 2024-03-18T23:06:03.191 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:03.190+0000 7f762de4b640 1 -- 172.21.15.49:0/613518549 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f76100a2800 msgr2=0x7f76100a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:03.191 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:03.190+0000 7f762de4b640 1 --2- 172.21.15.49:0/613518549 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f76100a2800 0x7f76100a4cc0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f7628102890 tx=0x7f7618005f90 comp rx=0 tx=0).stop 2024-03-18T23:06:03.191 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:03.190+0000 7f762de4b640 1 -- 172.21.15.49:0/613518549 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f76281033f0 msgr2=0x7f7628074150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:03.191 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:03.190+0000 7f762de4b640 1 --2- 172.21.15.49:0/613518549 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f76281033f0 0x7f7628074150 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f7620009fd0 tx=0x7f762000b260 comp rx=0 tx=0).stop 2024-03-18T23:06:03.191 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:03.190+0000 7f762de4b640 1 -- 172.21.15.49:0/613518549 shutdown_connections 2024-03-18T23:06:03.192 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:03.190+0000 7f762de4b640 1 --2- 172.21.15.49:0/613518549 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f76100a2800 0x7f76100a4cc0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:03.192 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:03.190+0000 7f762de4b640 1 --2- 172.21.15.49:0/613518549 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f76281033f0 0x7f7628074150 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:03.192 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:03.190+0000 7f762de4b640 1 --2- 172.21.15.49:0/613518549 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7628102a30 0x7f7628073c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:03.192 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:03.190+0000 7f762de4b640 1 --2- 172.21.15.49:0/613518549 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7628101840 0x7f76280755d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:03.192 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:03.190+0000 7f762de4b640 1 -- 172.21.15.49:0/613518549 >> 172.21.15.49:0/613518549 conn(0x7f76280fcdb0 msgr2=0x7f7628105d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:06:03.192 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:03.190+0000 7f762de4b640 1 -- 172.21.15.49:0/613518549 shutdown_connections 2024-03-18T23:06:03.192 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:03.190+0000 7f762de4b640 1 -- 172.21.15.49:0/613518549 wait complete. 2024-03-18T23:06:03.206 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-03-18T23:06:03.324 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:03.323+0000 7f7440ede640 1 -- 172.21.15.49:0/3331557572 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f743c101910 msgr2=0x7f743c101d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:03.324 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:03.323+0000 7f7440ede640 1 --2- 172.21.15.49:0/3331557572 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f743c101910 0x7f743c101d90 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f7424009900 tx=0x7f742402bc30 comp rx=0 tx=0).stop 2024-03-18T23:06:03.324 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:03.323+0000 7f7440ede640 1 -- 172.21.15.49:0/3331557572 shutdown_connections 2024-03-18T23:06:03.325 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:03.323+0000 7f7440ede640 1 --2- 172.21.15.49:0/3331557572 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f743c1022d0 0x7f743c108b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:03.325 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:03.323+0000 7f7440ede640 1 --2- 172.21.15.49:0/3331557572 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f743c101910 0x7f743c101d90 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:03.325 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:03.323+0000 7f7440ede640 1 --2- 172.21.15.49:0/3331557572 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f743c100720 0x7f743c100b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:03.325 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:03.323+0000 7f7440ede640 1 -- 172.21.15.49:0/3331557572 >> 172.21.15.49:0/3331557572 conn(0x7f743c0fbcb0 msgr2=0x7f743c0fe0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:06:03.325 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:03.323+0000 7f7440ede640 1 -- 172.21.15.49:0/3331557572 shutdown_connections 2024-03-18T23:06:03.325 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:03.323+0000 7f7440ede640 1 -- 172.21.15.49:0/3331557572 wait complete. 2024-03-18T23:06:03.325 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:03.323+0000 7f7440ede640 1 Processor -- start 2024-03-18T23:06:03.325 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:03.324+0000 7f7440ede640 1 -- start start 2024-03-18T23:06:03.327 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:03.324+0000 7f7440ede640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f743c100720 0x7f743c104660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:06:03.327 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:03.324+0000 7f7440ede640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f743c101910 0x7f743c102cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:06:03.327 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:03.324+0000 7f7440ede640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f743c1022d0 0x7f743c103230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:06:03.327 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:03.324+0000 7f7440ede640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f743c10c3a0 con 0x7f743c1022d0 2024-03-18T23:06:03.327 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:03.324+0000 7f7440ede640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f743c10c220 con 0x7f743c100720 2024-03-18T23:06:03.327 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:03.324+0000 7f7440ede640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f743c10c520 con 0x7f743c101910 2024-03-18T23:06:03.327 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:03.324+0000 7f743ad9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f743c1022d0 0x7f743c103230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:06:03.328 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:03.324+0000 7f743ad9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f743c1022d0 0x7f743c103230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:50222/0 (socket says 172.21.15.49:50222) 2024-03-18T23:06:03.328 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:03.324+0000 7f743ad9d640 1 -- 172.21.15.49:0/1100966461 learned_addr learned my addr 172.21.15.49:0/1100966461 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:06:03.328 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:03.324+0000 7f7431d9b640 1 --2- 172.21.15.49:0/1100966461 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f743c101910 0x7f743c102cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:06:03.328 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:03.324+0000 7f743a59c640 1 --2- 172.21.15.49:0/1100966461 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f743c100720 0x7f743c104660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:06:03.328 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:03.324+0000 7f743ad9d640 1 -- 172.21.15.49:0/1100966461 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f743c101910 msgr2=0x7f743c102cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:03.328 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:03.324+0000 7f743ad9d640 1 --2- 172.21.15.49:0/1100966461 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f743c101910 0x7f743c102cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:03.328 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:03.324+0000 7f743ad9d640 1 -- 172.21.15.49:0/1100966461 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f743c100720 msgr2=0x7f743c104660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:03.328 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:03.324+0000 7f743ad9d640 1 --2- 172.21.15.49:0/1100966461 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f743c100720 0x7f743c104660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:03.328 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:03.324+0000 7f743ad9d640 1 -- 172.21.15.49:0/1100966461 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f743c103af0 con 0x7f743c1022d0 2024-03-18T23:06:03.328 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:03.324+0000 7f743a59c640 1 --2- 172.21.15.49:0/1100966461 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f743c100720 0x7f743c104660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T23:06:03.328 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:03.325+0000 7f743ad9d640 1 --2- 172.21.15.49:0/1100966461 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f743c1022d0 0x7f743c103230 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f743400cad0 tx=0x7f743400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:06:03.328 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:03.325+0000 7f7433fff640 1 -- 172.21.15.49:0/1100966461 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f743400ae20 con 0x7f743c1022d0 2024-03-18T23:06:03.328 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:03.325+0000 7f7440ede640 1 -- 172.21.15.49:0/1100966461 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f743c1a1ce0 con 0x7f743c1022d0 2024-03-18T23:06:03.328 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:03.325+0000 7f7433fff640 1 -- 172.21.15.49:0/1100966461 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7434005020 con 0x7f743c1022d0 2024-03-18T23:06:03.329 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:03.325+0000 7f7433fff640 1 -- 172.21.15.49:0/1100966461 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f743400f6f0 con 0x7f743c1022d0 2024-03-18T23:06:03.329 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:03.325+0000 7f7440ede640 1 -- 172.21.15.49:0/1100966461 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f743c1a21b0 con 0x7f743c1022d0 2024-03-18T23:06:03.329 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:03.326+0000 7f7440ede640 1 -- 172.21.15.49:0/1100966461 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f743c1a23e0 con 0x7f743c1022d0 2024-03-18T23:06:03.329 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:03.327+0000 7f7433fff640 1 -- 172.21.15.49:0/1100966461 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7f74340051d0 con 0x7f743c1022d0 2024-03-18T23:06:03.329 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:03.328+0000 7f7433fff640 1 --2- 172.21.15.49:0/1100966461 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f74280a2800 0x7f74280a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:06:03.329 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:03.328+0000 7f743a59c640 1 --2- 172.21.15.49:0/1100966461 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f74280a2800 0x7f74280a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:06:03.330 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:03.328+0000 7f7433fff640 1 -- 172.21.15.49:0/1100966461 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7497+0+0 (secure 0 0 0) 0x7f74340ceb70 con 0x7f743c1022d0 2024-03-18T23:06:03.330 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:03.328+0000 7f743a59c640 1 --2- 172.21.15.49:0/1100966461 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f74280a2800 0x7f74280a4cc0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f743c101770 tx=0x7f742c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:06:03.334 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:03.332+0000 7f7433fff640 1 -- 172.21.15.49:0/1100966461 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f7434014030 con 0x7f743c1022d0 2024-03-18T23:06:03.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:06:03 smithi175 ceph-mon[67352]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T23:06:03.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:06:03 smithi175 ceph-mon[67352]: from='client.? 172.21.15.49:0/2981897393' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-03-18T23:06:03.462 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:06:03 smithi175 ceph-mon[67352]: from='client.? 172.21.15.49:0/613518549' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-03-18T23:06:03.462 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:06:03 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:06:03.462 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:06:03 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:06:03.525 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:03.524+0000 7f7440ede640 1 -- 172.21.15.49:0/1100966461 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f743c1a25b0 con 0x7f743c1022d0 2024-03-18T23:06:03.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:06:03 smithi139 ceph-mon[67282]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T23:06:03.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:06:03 smithi139 ceph-mon[67282]: from='client.? 172.21.15.49:0/2981897393' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-03-18T23:06:03.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:06:03 smithi139 ceph-mon[67282]: from='client.? 172.21.15.49:0/613518549' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-03-18T23:06:03.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:06:03 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:06:03.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:06:03 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:06:03.630 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:06:03 smithi049 ceph-mon[62991]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T23:06:03.630 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:06:03 smithi049 ceph-mon[62991]: from='client.? 172.21.15.49:0/2981897393' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-03-18T23:06:03.630 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:06:03 smithi049 ceph-mon[62991]: from='client.? 172.21.15.49:0/613518549' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-03-18T23:06:03.630 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:06:03 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:06:03.630 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:06:03 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:06:04.196 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:04.195+0000 7f7433fff640 1 -- 172.21.15.49:0/1100966461 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v84) ==== 170+0+0 (secure 0 0 0) 0x7f743409bf40 con 0x7f743c1022d0 2024-03-18T23:06:04.196 INFO:teuthology.orchestra.run.smithi049.stderr:pool 'cephfs_data' created 2024-03-18T23:06:04.201 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:04.200+0000 7f7440ede640 1 -- 172.21.15.49:0/1100966461 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f74280a2800 msgr2=0x7f74280a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:04.201 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:04.200+0000 7f7440ede640 1 --2- 172.21.15.49:0/1100966461 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f74280a2800 0x7f74280a4cc0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f743c101770 tx=0x7f742c005f90 comp rx=0 tx=0).stop 2024-03-18T23:06:04.201 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:04.200+0000 7f7440ede640 1 -- 172.21.15.49:0/1100966461 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f743c1022d0 msgr2=0x7f743c103230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:04.201 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:04.200+0000 7f7440ede640 1 --2- 172.21.15.49:0/1100966461 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f743c1022d0 0x7f743c103230 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f743400cad0 tx=0x7f743400cf90 comp rx=0 tx=0).stop 2024-03-18T23:06:04.201 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:04.200+0000 7f7440ede640 1 -- 172.21.15.49:0/1100966461 shutdown_connections 2024-03-18T23:06:04.201 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:04.200+0000 7f7440ede640 1 --2- 172.21.15.49:0/1100966461 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f74280a2800 0x7f74280a4cc0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:04.201 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:04.200+0000 7f7440ede640 1 --2- 172.21.15.49:0/1100966461 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f743c1022d0 0x7f743c103230 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:04.201 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:04.200+0000 7f7440ede640 1 --2- 172.21.15.49:0/1100966461 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f743c101910 0x7f743c102cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:04.201 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:04.200+0000 7f7440ede640 1 --2- 172.21.15.49:0/1100966461 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f743c100720 0x7f743c104660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:04.201 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:04.200+0000 7f7440ede640 1 -- 172.21.15.49:0/1100966461 >> 172.21.15.49:0/1100966461 conn(0x7f743c0fbcb0 msgr2=0x7f743c0fd7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:06:04.202 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:04.200+0000 7f7440ede640 1 -- 172.21.15.49:0/1100966461 shutdown_connections 2024-03-18T23:06:04.202 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:04.200+0000 7f7440ede640 1 -- 172.21.15.49:0/1100966461 wait complete. 2024-03-18T23:06:04.221 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-03-18T23:06:04.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:04.339+0000 7f42e4310640 1 -- 172.21.15.49:0/3178014912 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f42dc100a70 msgr2=0x7f42dc100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:04.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:04.339+0000 7f42e4310640 1 --2- 172.21.15.49:0/3178014912 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f42dc100a70 0x7f42dc100ed0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f42d8009900 tx=0x7f42d802bc80 comp rx=0 tx=0).stop 2024-03-18T23:06:04.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:04.339+0000 7f42e4310640 1 -- 172.21.15.49:0/3178014912 shutdown_connections 2024-03-18T23:06:04.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:04.339+0000 7f42e4310640 1 --2- 172.21.15.49:0/3178014912 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f42dc101410 0x7f42dc101890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:04.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:04.339+0000 7f42e4310640 1 --2- 172.21.15.49:0/3178014912 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f42dc100a70 0x7f42dc100ed0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:04.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:04.339+0000 7f42e4310640 1 --2- 172.21.15.49:0/3178014912 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f42dc100110 0x7f42dc100530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:04.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:04.339+0000 7f42e4310640 1 -- 172.21.15.49:0/3178014912 >> 172.21.15.49:0/3178014912 conn(0x7f42dc0fbcd0 msgr2=0x7f42dc0fe110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:06:04.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:04.339+0000 7f42e4310640 1 -- 172.21.15.49:0/3178014912 shutdown_connections 2024-03-18T23:06:04.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:04.339+0000 7f42e4310640 1 -- 172.21.15.49:0/3178014912 wait complete. 2024-03-18T23:06:04.343 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:04.339+0000 7f42e4310640 1 Processor -- start 2024-03-18T23:06:04.343 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:04.339+0000 7f42e4310640 1 -- start start 2024-03-18T23:06:04.343 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:04.340+0000 7f42e4310640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f42dc100110 0x7f42dc199dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:06:04.343 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:04.340+0000 7f42e4310640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f42dc100a70 0x7f42dc19a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:06:04.343 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:04.340+0000 7f42e4310640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f42dc101410 0x7f42dc1a1370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:06:04.343 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:04.340+0000 7f42e4310640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f42dc10f6e0 con 0x7f42dc100a70 2024-03-18T23:06:04.343 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:04.340+0000 7f42e4310640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f42dc10f560 con 0x7f42dc100110 2024-03-18T23:06:04.343 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:04.340+0000 7f42e4310640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f42dc10f860 con 0x7f42dc101410 2024-03-18T23:06:04.343 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:04.340+0000 7f42e18ab640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f42dc100a70 0x7f42dc19a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:06:04.343 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:04.340+0000 7f42e18ab640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f42dc100a70 0x7f42dc19a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:50236/0 (socket says 172.21.15.49:50236) 2024-03-18T23:06:04.343 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:04.340+0000 7f42e18ab640 1 -- 172.21.15.49:0/2988325313 learned_addr learned my addr 172.21.15.49:0/2988325313 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:06:04.343 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:04.340+0000 7f42e28ad640 1 --2- 172.21.15.49:0/2988325313 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f42dc101410 0x7f42dc1a1370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:06:04.343 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:04.340+0000 7f42e20ac640 1 --2- 172.21.15.49:0/2988325313 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f42dc100110 0x7f42dc199dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:06:04.344 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:04.340+0000 7f42e18ab640 1 -- 172.21.15.49:0/2988325313 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f42dc101410 msgr2=0x7f42dc1a1370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:04.344 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:04.340+0000 7f42e18ab640 1 --2- 172.21.15.49:0/2988325313 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f42dc101410 0x7f42dc1a1370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:04.344 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:04.340+0000 7f42e18ab640 1 -- 172.21.15.49:0/2988325313 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f42dc100110 msgr2=0x7f42dc199dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:04.344 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:04.340+0000 7f42e18ab640 1 --2- 172.21.15.49:0/2988325313 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f42dc100110 0x7f42dc199dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:04.344 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:04.340+0000 7f42e18ab640 1 -- 172.21.15.49:0/2988325313 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42dc1a19e0 con 0x7f42dc100a70 2024-03-18T23:06:04.344 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:04.340+0000 7f42e18ab640 1 --2- 172.21.15.49:0/2988325313 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f42dc100a70 0x7f42dc19a300 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f42d8006010 tx=0x7f42d8004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:06:04.344 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:04.340+0000 7f42d37fe640 1 -- 172.21.15.49:0/2988325313 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42d8041070 con 0x7f42dc100a70 2024-03-18T23:06:04.344 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:04.340+0000 7f42e4310640 1 -- 172.21.15.49:0/2988325313 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42dc1a1c70 con 0x7f42dc100a70 2024-03-18T23:06:04.344 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:04.340+0000 7f42d37fe640 1 -- 172.21.15.49:0/2988325313 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f42d8004450 con 0x7f42dc100a70 2024-03-18T23:06:04.344 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:04.340+0000 7f42d37fe640 1 -- 172.21.15.49:0/2988325313 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42d8034440 con 0x7f42dc100a70 2024-03-18T23:06:04.344 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:04.340+0000 7f42e4310640 1 -- 172.21.15.49:0/2988325313 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42dc1a2160 con 0x7f42dc100a70 2024-03-18T23:06:04.345 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:04.341+0000 7f42e4310640 1 -- 172.21.15.49:0/2988325313 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42dc10b0a0 con 0x7f42dc100a70 2024-03-18T23:06:04.345 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:04.342+0000 7f42d37fe640 1 -- 172.21.15.49:0/2988325313 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7f42d8044070 con 0x7f42dc100a70 2024-03-18T23:06:04.345 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:04.343+0000 7f42d37fe640 1 --2- 172.21.15.49:0/2988325313 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f42c80a2800 0x7f42c80a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:06:04.345 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:04.343+0000 7f42e20ac640 1 --2- 172.21.15.49:0/2988325313 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f42c80a2800 0x7f42c80a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:06:04.345 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:04.343+0000 7f42d37fe640 1 -- 172.21.15.49:0/2988325313 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7879+0+0 (secure 0 0 0) 0x7f42d80f0470 con 0x7f42dc100a70 2024-03-18T23:06:04.345 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:04.344+0000 7f42e20ac640 1 --2- 172.21.15.49:0/2988325313 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f42c80a2800 0x7f42c80a4cc0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f42cc00bbe0 tx=0x7f42cc00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:06:04.348 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:04.348+0000 7f42d37fe640 1 -- 172.21.15.49:0/2988325313 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f42d80bd070 con 0x7f42dc100a70 2024-03-18T23:06:04.448 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:06:04 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:06:04.448 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:06:04 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:06:04.448 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:06:04 smithi049 ceph-mon[62991]: from='client.? 172.21.15.49:0/613518549' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-03-18T23:06:04.448 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:06:04 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:06:04.448 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:06:04 smithi049 ceph-mon[62991]: osdmap e83: 12 total, 12 up, 12 in 2024-03-18T23:06:04.448 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:06:04 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:06:04.448 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:06:04 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:06:04.448 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:06:04 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:06:04.448 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:06:04 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:06:04.448 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:06:04 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:06:04.448 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:06:04 smithi049 ceph-mon[62991]: from='client.? 172.21.15.49:0/1100966461' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-03-18T23:06:04.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:06:04 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:06:04.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:06:04 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:06:04.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:06:04 smithi175 ceph-mon[67352]: from='client.? 172.21.15.49:0/613518549' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-03-18T23:06:04.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:06:04 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:06:04.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:06:04 smithi175 ceph-mon[67352]: osdmap e83: 12 total, 12 up, 12 in 2024-03-18T23:06:04.462 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:06:04 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:06:04.462 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:06:04 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:06:04.462 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:06:04 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:06:04.462 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:06:04 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:06:04.462 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:06:04 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:06:04.462 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:06:04 smithi175 ceph-mon[67352]: from='client.? 172.21.15.49:0/1100966461' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-03-18T23:06:04.566 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:06:04 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:06:04.567 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:06:04 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:06:04.567 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:06:04 smithi139 ceph-mon[67282]: from='client.? 172.21.15.49:0/613518549' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-03-18T23:06:04.567 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:06:04 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:06:04.567 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:06:04 smithi139 ceph-mon[67282]: osdmap e83: 12 total, 12 up, 12 in 2024-03-18T23:06:04.567 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:06:04 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:06:04.567 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:06:04 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:06:04.567 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:06:04 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:06:04.567 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:06:04 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:06:04.567 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:06:04 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:06:04.567 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:06:04 smithi139 ceph-mon[67282]: from='client.? 172.21.15.49:0/1100966461' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-03-18T23:06:04.574 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:04.573+0000 7f42e4310640 1 -- 172.21.15.49:0/2988325313 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f42dc062490 con 0x7f42dc100a70 2024-03-18T23:06:05.214 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:05.213+0000 7f42d37fe640 1 -- 172.21.15.49:0/2988325313 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7f42d80c1110 con 0x7f42dc100a70 2024-03-18T23:06:05.214 INFO:teuthology.orchestra.run.smithi049.stderr:new fs with metadata pool 2 and data pool 3 2024-03-18T23:06:05.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:05.215+0000 7f42e4310640 1 -- 172.21.15.49:0/2988325313 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f42c80a2800 msgr2=0x7f42c80a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:05.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:05.215+0000 7f42e4310640 1 --2- 172.21.15.49:0/2988325313 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f42c80a2800 0x7f42c80a4cc0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f42cc00bbe0 tx=0x7f42cc00b390 comp rx=0 tx=0).stop 2024-03-18T23:06:05.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:05.215+0000 7f42e4310640 1 -- 172.21.15.49:0/2988325313 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f42dc100a70 msgr2=0x7f42dc19a300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:05.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:05.215+0000 7f42e4310640 1 --2- 172.21.15.49:0/2988325313 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f42dc100a70 0x7f42dc19a300 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f42d8006010 tx=0x7f42d8004ab0 comp rx=0 tx=0).stop 2024-03-18T23:06:05.217 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:05.215+0000 7f42e4310640 1 -- 172.21.15.49:0/2988325313 shutdown_connections 2024-03-18T23:06:05.217 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:05.215+0000 7f42e4310640 1 --2- 172.21.15.49:0/2988325313 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f42c80a2800 0x7f42c80a4cc0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:05.217 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:05.215+0000 7f42e4310640 1 --2- 172.21.15.49:0/2988325313 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f42dc101410 0x7f42dc1a1370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:05.217 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:05.215+0000 7f42e4310640 1 --2- 172.21.15.49:0/2988325313 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f42dc100a70 0x7f42dc19a300 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:05.217 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:05.215+0000 7f42e4310640 1 --2- 172.21.15.49:0/2988325313 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f42dc100110 0x7f42dc199dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:05.217 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:05.215+0000 7f42e4310640 1 -- 172.21.15.49:0/2988325313 >> 172.21.15.49:0/2988325313 conn(0x7f42dc0fbcd0 msgr2=0x7f42dc0fd4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:06:05.217 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:05.215+0000 7f42e4310640 1 -- 172.21.15.49:0/2988325313 shutdown_connections 2024-03-18T23:06:05.217 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:05.215+0000 7f42e4310640 1 -- 172.21.15.49:0/2988325313 wait complete. 2024-03-18T23:06:05.229 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-18T23:06:05.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:05.338+0000 7fe42b9df640 1 -- 172.21.15.49:0/3101518028 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe424101850 msgr2=0x7fe424101c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:05.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:05.338+0000 7fe42b9df640 1 --2- 172.21.15.49:0/3101518028 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe424101850 0x7fe424101c70 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fe414009900 tx=0x7fe41402bc30 comp rx=0 tx=0).stop 2024-03-18T23:06:05.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:05.338+0000 7fe42b9df640 1 -- 172.21.15.49:0/3101518028 shutdown_connections 2024-03-18T23:06:05.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:05.338+0000 7fe42b9df640 1 --2- 172.21.15.49:0/3101518028 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe424103400 0x7fe424109c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:05.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:05.338+0000 7fe42b9df640 1 --2- 172.21.15.49:0/3101518028 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe424102a40 0x7fe424102ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:05.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:05.338+0000 7fe42b9df640 1 --2- 172.21.15.49:0/3101518028 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe424101850 0x7fe424101c70 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:05.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:05.338+0000 7fe42b9df640 1 -- 172.21.15.49:0/3101518028 >> 172.21.15.49:0/3101518028 conn(0x7fe4240fce00 msgr2=0x7fe4240ff220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:06:05.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:05.338+0000 7fe42b9df640 1 -- 172.21.15.49:0/3101518028 shutdown_connections 2024-03-18T23:06:05.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:05.338+0000 7fe42b9df640 1 -- 172.21.15.49:0/3101518028 wait complete. 2024-03-18T23:06:05.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:05.339+0000 7fe42b9df640 1 Processor -- start 2024-03-18T23:06:05.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:05.339+0000 7fe42b9df640 1 -- start start 2024-03-18T23:06:05.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:05.339+0000 7fe42b9df640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe424101850 0x7fe424197ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:06:05.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:05.339+0000 7fe42b9df640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe424102a40 0x7fe424198020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:06:05.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:05.339+0000 7fe42b9df640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe424103400 0x7fe42419f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:06:05.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:05.339+0000 7fe42b9df640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fe42410d4a0 con 0x7fe424101850 2024-03-18T23:06:05.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:05.339+0000 7fe42b9df640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fe42410d320 con 0x7fe424103400 2024-03-18T23:06:05.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:05.339+0000 7fe42b9df640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe42410d620 con 0x7fe424102a40 2024-03-18T23:06:05.343 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:05.339+0000 7fe42977b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe424101850 0x7fe424197ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:06:05.343 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:05.339+0000 7fe42977b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe424101850 0x7fe424197ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:54082/0 (socket says 172.21.15.49:54082) 2024-03-18T23:06:05.343 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:05.339+0000 7fe42977b640 1 -- 172.21.15.49:0/1184137159 learned_addr learned my addr 172.21.15.49:0/1184137159 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:06:05.343 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:05.339+0000 7fe428f7a640 1 --2- 172.21.15.49:0/1184137159 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe424102a40 0x7fe424198020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:06:05.343 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:05.339+0000 7fe42977b640 1 -- 172.21.15.49:0/1184137159 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe424102a40 msgr2=0x7fe424198020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:05.343 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:05.339+0000 7fe42977b640 1 --2- 172.21.15.49:0/1184137159 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe424102a40 0x7fe424198020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:05.343 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:05.339+0000 7fe42977b640 1 -- 172.21.15.49:0/1184137159 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe424103400 msgr2=0x7fe42419f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:05.343 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:05.339+0000 7fe42977b640 1 --2- 172.21.15.49:0/1184137159 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe424103400 0x7fe42419f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:05.343 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:05.339+0000 7fe42977b640 1 -- 172.21.15.49:0/1184137159 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe42419f700 con 0x7fe424101850 2024-03-18T23:06:05.343 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:05.339+0000 7fe42977b640 1 --2- 172.21.15.49:0/1184137159 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe424101850 0x7fe424197ae0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fe414000c00 tx=0x7fe414004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:06:05.343 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:05.340+0000 7fe41a7fc640 1 -- 172.21.15.49:0/1184137159 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe414041070 con 0x7fe424101850 2024-03-18T23:06:05.343 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:05.340+0000 7fe42b9df640 1 -- 172.21.15.49:0/1184137159 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe42419f990 con 0x7fe424101850 2024-03-18T23:06:05.343 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:05.340+0000 7fe41a7fc640 1 -- 172.21.15.49:0/1184137159 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe4140054e0 con 0x7fe424101850 2024-03-18T23:06:05.344 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:05.340+0000 7fe41a7fc640 1 -- 172.21.15.49:0/1184137159 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4140346a0 con 0x7fe424101850 2024-03-18T23:06:05.344 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:05.340+0000 7fe42b9df640 1 -- 172.21.15.49:0/1184137159 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe42419fe80 con 0x7fe424101850 2024-03-18T23:06:05.344 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:05.341+0000 7fe42b9df640 1 -- 172.21.15.49:0/1184137159 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe424108b10 con 0x7fe424101850 2024-03-18T23:06:05.345 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:05.342+0000 7fe41a7fc640 1 -- 172.21.15.49:0/1184137159 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7fe414044070 con 0x7fe424101850 2024-03-18T23:06:05.345 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:05.342+0000 7fe41a7fc640 1 --2- 172.21.15.49:0/1184137159 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fe4100a2800 0x7fe4100a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:06:05.345 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:05.342+0000 7fe428f7a640 1 --2- 172.21.15.49:0/1184137159 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fe4100a2800 0x7fe4100a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:06:05.345 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:05.342+0000 7fe41a7fc640 1 -- 172.21.15.49:0/1184137159 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7fe4140f0720 con 0x7fe424101850 2024-03-18T23:06:05.345 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:05.343+0000 7fe428f7a640 1 --2- 172.21.15.49:0/1184137159 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fe4100a2800 0x7fe4100a4cc0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fe420005e10 tx=0x7fe420005da0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:06:05.347 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:05.347+0000 7fe41a7fc640 1 -- 172.21.15.49:0/1184137159 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fe4140bd910 con 0x7fe424101850 2024-03-18T23:06:05.462 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:06:05 smithi175 ceph-mon[67352]: pgmap v285: 33 pgs: 8 creating+peering, 1 active+clean, 24 unknown; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T23:06:05.462 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:06:05 smithi175 ceph-mon[67352]: from='client.? 172.21.15.49:0/1100966461' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-03-18T23:06:05.462 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:06:05 smithi175 ceph-mon[67352]: osdmap e84: 12 total, 12 up, 12 in 2024-03-18T23:06:05.462 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:06:05 smithi175 ceph-mon[67352]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-18T23:06:05.462 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:06:05 smithi175 ceph-mon[67352]: from='client.? 172.21.15.49:0/2988325313' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-03-18T23:06:05.462 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:06:05 smithi175 ceph-mon[67352]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-03-18T23:06:05.462 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:06:05 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:06:05.462 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:06:05 smithi175 ceph-mon[67352]: osdmap e85: 12 total, 12 up, 12 in 2024-03-18T23:06:05.462 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:06:05 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:06:05.462 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:06:05 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:06:05.462 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:06:05 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:06:05.463 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:06:05 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:06:05.463 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:06:05 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:06:05.463 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:06:05 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:06:05.463 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:06:05 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:06:05.463 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:06:05 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:06:05.463 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:06:05 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:06:05.537 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:05.536+0000 7fe42b9df640 1 -- 172.21.15.49:0/1184137159 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe42402d480 con 0x7fe424101850 2024-03-18T23:06:05.538 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:05.537+0000 7fe41a7fc640 1 -- 172.21.15.49:0/1184137159 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) ==== 74+0+22255 (secure 0 0 0) 0x7fe414003e20 con 0x7fe424101850 2024-03-18T23:06:05.539 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-18T23:06:05.539 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":85,"fsid":"c63591ea-e57a-11ee-95c9-87774f69a715","created":"2024-03-18T22:57:58.734467+0000","modified":"2024-03-18T23:06:04.575518+0000","last_up_change":"2024-03-18T23:05:16.576310+0000","last_in_change":"2024-03-18T23:04:57.024067+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-18T23:01:28.363432+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-18T23:06:03.172963+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-18T23:06:03.973303+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e7fc9daa-4098-4ba5-8019-9aaef1aab3e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":1651162501},{"type":"v1","addr":"172.21.15.49:6803","nonce":1651162501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":1651162501},{"type":"v1","addr":"172.21.15.49:6805","nonce":1651162501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":1651162501},{"type":"v1","addr":"172.21.15.49:6809","nonce":1651162501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":1651162501},{"type":"v1","addr":"172.21.15.49:6807","nonce":1651162501}]},"public_addr":"172.21.15.49:6803/1651162501","cluster_addr":"172.21.15.49:6805/1651162501","heartbeat_back_addr":"172.21.15.49:6809/1651162501","heartbeat_front_addr":"172.21.15.49:6807/1651162501","state":["exists","up"]},{"osd":1,"uuid":"40ac929c-e90f-434c-a796-9aabd686f1fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":1766700359},{"type":"v1","addr":"172.21.15.139:6801","nonce":1766700359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":1766700359},{"type":"v1","addr":"172.21.15.139:6803","nonce":1766700359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":1766700359},{"type":"v1","addr":"172.21.15.139:6807","nonce":1766700359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":1766700359},{"type":"v1","addr":"172.21.15.139:6805","nonce":1766700359}]},"public_addr":"172.21.15.139:6801/1766700359","cluster_addr":"172.21.15.139:6803/1766700359","heartbeat_back_addr":"172.21.15.139:6807/1766700359","heartbeat_front_addr":"172.21.15.139:6805/1766700359","state":["exists","up"]},{"osd":2,"uuid":"8ae3b666-00b4-45ed-a5ee-4d0771edabb6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":3752671234},{"type":"v1","addr":"172.21.15.175:6801","nonce":3752671234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":3752671234},{"type":"v1","addr":"172.21.15.175:6803","nonce":3752671234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":3752671234},{"type":"v1","addr":"172.21.15.175:6807","nonce":3752671234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":3752671234},{"type":"v1","addr":"172.21.15.175:6805","nonce":3752671234}]},"public_addr":"172.21.15.175:6801/3752671234","cluster_addr":"172.21.15.175:6803/3752671234","heartbeat_back_addr":"172.21.15.175:6807/3752671234","heartbeat_front_addr":"172.21.15.175:6805/3752671234","state":["exists","up"]},{"osd":3,"uuid":"1a62e89d-45d4-4237-b813-7f96a59aca7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":956160536},{"type":"v1","addr":"172.21.15.49:6811","nonce":956160536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":956160536},{"type":"v1","addr":"172.21.15.49:6813","nonce":956160536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":956160536},{"type":"v1","addr":"172.21.15.49:6817","nonce":956160536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":956160536},{"type":"v1","addr":"172.21.15.49:6815","nonce":956160536}]},"public_addr":"172.21.15.49:6811/956160536","cluster_addr":"172.21.15.49:6813/956160536","heartbeat_back_addr":"172.21.15.49:6817/956160536","heartbeat_front_addr":"172.21.15.49:6815/956160536","state":["exists","up"]},{"osd":4,"uuid":"c715101c-0c3f-4b98-b142-d64a35e187df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":3129712668},{"type":"v1","addr":"172.21.15.139:6809","nonce":3129712668}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":3129712668},{"type":"v1","addr":"172.21.15.139:6811","nonce":3129712668}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":3129712668},{"type":"v1","addr":"172.21.15.139:6815","nonce":3129712668}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":3129712668},{"type":"v1","addr":"172.21.15.139:6813","nonce":3129712668}]},"public_addr":"172.21.15.139:6809/3129712668","cluster_addr":"172.21.15.139:6811/3129712668","heartbeat_back_addr":"172.21.15.139:6815/3129712668","heartbeat_front_addr":"172.21.15.139:6813/3129712668","state":["exists","up"]},{"osd":5,"uuid":"4b641f81-77eb-45fd-8464-511e7b3062b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":2774567215},{"type":"v1","addr":"172.21.15.175:6809","nonce":2774567215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":2774567215},{"type":"v1","addr":"172.21.15.175:6811","nonce":2774567215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":2774567215},{"type":"v1","addr":"172.21.15.175:6815","nonce":2774567215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":2774567215},{"type":"v1","addr":"172.21.15.175:6813","nonce":2774567215}]},"public_addr":"172.21.15.175:6809/2774567215","cluster_addr":"172.21.15.175:6811/2774567215","heartbeat_back_addr":"172.21.15.175:6815/2774567215","heartbeat_front_addr":"172.21.15.175:6813/2774567215","state":["exists","up"]},{"osd":6,"uuid":"653da8e0-1197-4b48-abed-470f2a9c4dce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":1472215461},{"type":"v1","addr":"172.21.15.49:6819","nonce":1472215461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":1472215461},{"type":"v1","addr":"172.21.15.49:6821","nonce":1472215461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":1472215461},{"type":"v1","addr":"172.21.15.49:6825","nonce":1472215461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":1472215461},{"type":"v1","addr":"172.21.15.49:6823","nonce":1472215461}]},"public_addr":"172.21.15.49:6819/1472215461","cluster_addr":"172.21.15.49:6821/1472215461","heartbeat_back_addr":"172.21.15.49:6825/1472215461","heartbeat_front_addr":"172.21.15.49:6823/1472215461","state":["exists","up"]},{"osd":7,"uuid":"ad157c9b-4b75-451c-8740-630f0975b7fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":2163986787},{"type":"v1","addr":"172.21.15.139:6817","nonce":2163986787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":2163986787},{"type":"v1","addr":"172.21.15.139:6819","nonce":2163986787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":2163986787},{"type":"v1","addr":"172.21.15.139:6823","nonce":2163986787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":2163986787},{"type":"v1","addr":"172.21.15.139:6821","nonce":2163986787}]},"public_addr":"172.21.15.139:6817/2163986787","cluster_addr":"172.21.15.139:6819/2163986787","heartbeat_back_addr":"172.21.15.139:6823/2163986787","heartbeat_front_addr":"172.21.15.139:6821/2163986787","state":["exists","up"]},{"osd":8,"uuid":"c8e75f15-df75-4d5d-9b92-a064bffb6c01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":1194026992},{"type":"v1","addr":"172.21.15.175:6817","nonce":1194026992}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1194026992},{"type":"v1","addr":"172.21.15.175:6819","nonce":1194026992}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":1194026992},{"type":"v1","addr":"172.21.15.175:6823","nonce":1194026992}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1194026992},{"type":"v1","addr":"172.21.15.175:6821","nonce":1194026992}]},"public_addr":"172.21.15.175:6817/1194026992","cluster_addr":"172.21.15.175:6819/1194026992","heartbeat_back_addr":"172.21.15.175:6823/1194026992","heartbeat_front_addr":"172.21.15.175:6821/1194026992","state":["exists","up"]},{"osd":9,"uuid":"edb5ffef-b6a2-4941-b5ab-40f982ad30d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":2742084884},{"type":"v1","addr":"172.21.15.49:6827","nonce":2742084884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":2742084884},{"type":"v1","addr":"172.21.15.49:6829","nonce":2742084884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":2742084884},{"type":"v1","addr":"172.21.15.49:6833","nonce":2742084884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":2742084884},{"type":"v1","addr":"172.21.15.49:6831","nonce":2742084884}]},"public_addr":"172.21.15.49:6827/2742084884","cluster_addr":"172.21.15.49:6829/2742084884","heartbeat_back_addr":"172.21.15.49:6833/2742084884","heartbeat_front_addr":"172.21.15.49:6831/2742084884","state":["exists","up"]},{"osd":10,"uuid":"3945e46d-766a-4c4a-8821-d43648dd1578","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":2404740201},{"type":"v1","addr":"172.21.15.139:6825","nonce":2404740201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":2404740201},{"type":"v1","addr":"172.21.15.139:6827","nonce":2404740201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":2404740201},{"type":"v1","addr":"172.21.15.139:6831","nonce":2404740201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":2404740201},{"type":"v1","addr":"172.21.15.139:6829","nonce":2404740201}]},"public_addr":"172.21.15.139:6825/2404740201","cluster_addr":"172.21.15.139:6827/2404740201","heartbeat_back_addr":"172.21.15.139:6831/2404740201","heartbeat_front_addr":"172.21.15.139:6829/2404740201","state":["exists","up"]},{"osd":11,"uuid":"9954ba15-d877-46cc-b24a-893f9168deb8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":3519503868},{"type":"v1","addr":"172.21.15.175:6825","nonce":3519503868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":3519503868},{"type":"v1","addr":"172.21.15.175:6827","nonce":3519503868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":3519503868},{"type":"v1","addr":"172.21.15.175:6831","nonce":3519503868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":3519503868},{"type":"v1","addr":"172.21.15.175:6829","nonce":3519503868}]},"public_addr":"172.21.15.175:6825/3519503868","cluster_addr":"172.21.15.175:6827/3519503868","heartbeat_back_addr":"172.21.15.175:6831/3519503868","heartbeat_front_addr":"172.21.15.175:6829/3519503868","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:00:38.367865+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:01:02.237687+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:01:25.609751+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:01:51.268572+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:02:16.514605+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:02:41.586523+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:03:07.029964+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:03:32.956250+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:03:57.547984+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:04:23.948972+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:04:49.667210+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:05:15.284693+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.49:0/563527416":"2024-03-19T22:58:59.806253+0000","172.21.15.49:0/1036383534":"2024-03-19T22:58:59.806253+0000","172.21.15.49:6801/2609458951":"2024-03-19T22:58:59.806253+0000","172.21.15.49:0/21283102":"2024-03-19T22:58:31.972474+0000","172.21.15.49:0/1292058922":"2024-03-19T22:58:31.972474+0000","172.21.15.49:6800/1985190312":"2024-03-19T22:58:31.972474+0000","172.21.15.49:0/1912446700":"2024-03-19T22:58:59.806253+0000","172.21.15.49:0/2176210039":"2024-03-19T22:58:31.972474+0000","172.21.15.49:6801/1985190312":"2024-03-19T22:58:31.972474+0000","172.21.15.49:6800/2609458951":"2024-03-19T22:58:59.806253+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-18T23:06:05.541 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:05.540+0000 7fe42b9df640 1 -- 172.21.15.49:0/1184137159 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fe4100a2800 msgr2=0x7fe4100a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:05.541 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:05.540+0000 7fe42b9df640 1 --2- 172.21.15.49:0/1184137159 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fe4100a2800 0x7fe4100a4cc0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fe420005e10 tx=0x7fe420005da0 comp rx=0 tx=0).stop 2024-03-18T23:06:05.541 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:05.540+0000 7fe42b9df640 1 -- 172.21.15.49:0/1184137159 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe424101850 msgr2=0x7fe424197ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:05.541 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:05.540+0000 7fe42b9df640 1 --2- 172.21.15.49:0/1184137159 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe424101850 0x7fe424197ae0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fe414000c00 tx=0x7fe414004040 comp rx=0 tx=0).stop 2024-03-18T23:06:05.541 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:05.540+0000 7fe42b9df640 1 -- 172.21.15.49:0/1184137159 shutdown_connections 2024-03-18T23:06:05.541 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:05.540+0000 7fe42b9df640 1 --2- 172.21.15.49:0/1184137159 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fe4100a2800 0x7fe4100a4cc0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:05.541 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:05.540+0000 7fe42b9df640 1 --2- 172.21.15.49:0/1184137159 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe424103400 0x7fe42419f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:05.541 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:05.540+0000 7fe42b9df640 1 --2- 172.21.15.49:0/1184137159 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe424102a40 0x7fe424198020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:05.542 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:05.540+0000 7fe42b9df640 1 --2- 172.21.15.49:0/1184137159 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe424101850 0x7fe424197ae0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:05.542 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:05.540+0000 7fe42b9df640 1 -- 172.21.15.49:0/1184137159 >> 172.21.15.49:0/1184137159 conn(0x7fe4240fce00 msgr2=0x7fe424105ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:06:05.542 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:05.540+0000 7fe42b9df640 1 -- 172.21.15.49:0/1184137159 shutdown_connections 2024-03-18T23:06:05.542 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:05.540+0000 7fe42b9df640 1 -- 172.21.15.49:0/1184137159 wait complete. 2024-03-18T23:06:05.551 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:06:05 smithi049 ceph-mon[62991]: pgmap v285: 33 pgs: 8 creating+peering, 1 active+clean, 24 unknown; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T23:06:05.551 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:06:05 smithi049 ceph-mon[62991]: from='client.? 172.21.15.49:0/1100966461' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-03-18T23:06:05.551 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:06:05 smithi049 ceph-mon[62991]: osdmap e84: 12 total, 12 up, 12 in 2024-03-18T23:06:05.551 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:06:05 smithi049 ceph-mon[62991]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-18T23:06:05.551 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:06:05 smithi049 ceph-mon[62991]: from='client.? 172.21.15.49:0/2988325313' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-03-18T23:06:05.551 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:06:05 smithi049 ceph-mon[62991]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-03-18T23:06:05.551 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:06:05 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:06:05.551 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:06:05 smithi049 ceph-mon[62991]: osdmap e85: 12 total, 12 up, 12 in 2024-03-18T23:06:05.551 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:06:05 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:06:05.551 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:06:05 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:06:05.551 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:06:05 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:06:05.551 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:06:05 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:06:05.552 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:06:05 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:06:05.552 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:06:05 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:06:05.552 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:06:05 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:06:05.552 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:06:05 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:06:05.552 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:06:05 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:06:05.553 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-18T23:06:05.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:06:05 smithi139 ceph-mon[67282]: pgmap v285: 33 pgs: 8 creating+peering, 1 active+clean, 24 unknown; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T23:06:05.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:06:05 smithi139 ceph-mon[67282]: from='client.? 172.21.15.49:0/1100966461' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-03-18T23:06:05.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:06:05 smithi139 ceph-mon[67282]: osdmap e84: 12 total, 12 up, 12 in 2024-03-18T23:06:05.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:06:05 smithi139 ceph-mon[67282]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-18T23:06:05.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:06:05 smithi139 ceph-mon[67282]: from='client.? 172.21.15.49:0/2988325313' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-03-18T23:06:05.566 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:06:05 smithi139 ceph-mon[67282]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-03-18T23:06:05.566 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:06:05 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:06:05.566 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:06:05 smithi139 ceph-mon[67282]: osdmap e85: 12 total, 12 up, 12 in 2024-03-18T23:06:05.566 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:06:05 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:06:05.566 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:06:05 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:06:05.566 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:06:05 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:06:05.566 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:06:05 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:06:05.566 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:06:05 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:06:05.566 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:06:05 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:06:05.566 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:06:05 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:06:05.566 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:06:05 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:06:05.566 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:06:05 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:06:05.663 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:05.660+0000 7faba6612640 1 -- 172.21.15.49:0/3041529843 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faba01023a0 msgr2=0x7faba0104790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:05.663 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:05.660+0000 7faba6612640 1 --2- 172.21.15.49:0/3041529843 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faba01023a0 0x7faba0104790 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fab90009900 tx=0x7fab9002bc30 comp rx=0 tx=0).stop 2024-03-18T23:06:05.664 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:05.660+0000 7faba6612640 1 -- 172.21.15.49:0/3041529843 shutdown_connections 2024-03-18T23:06:05.664 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:05.660+0000 7faba6612640 1 --2- 172.21.15.49:0/3041529843 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faba0107600 0x7faba01099f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:05.664 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:05.660+0000 7faba6612640 1 --2- 172.21.15.49:0/3041529843 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faba0104cd0 0x7faba01070c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:05.664 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:05.660+0000 7faba6612640 1 --2- 172.21.15.49:0/3041529843 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faba01023a0 0x7faba0104790 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:05.664 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:05.660+0000 7faba6612640 1 -- 172.21.15.49:0/3041529843 >> 172.21.15.49:0/3041529843 conn(0x7faba00fbd50 msgr2=0x7faba00fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:06:05.664 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:05.660+0000 7faba6612640 1 -- 172.21.15.49:0/3041529843 shutdown_connections 2024-03-18T23:06:05.664 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:05.660+0000 7faba6612640 1 -- 172.21.15.49:0/3041529843 wait complete. 2024-03-18T23:06:05.664 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:05.661+0000 7faba6612640 1 Processor -- start 2024-03-18T23:06:05.664 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:05.661+0000 7faba6612640 1 -- start start 2024-03-18T23:06:05.664 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:05.661+0000 7faba6612640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faba01023a0 0x7faba0197b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:06:05.664 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:05.661+0000 7faba6612640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faba0104cd0 0x7faba0198080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:06:05.664 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:05.661+0000 7faba6612640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faba0107600 0x7faba019f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:06:05.665 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:05.661+0000 7faba6612640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7faba010d4f0 con 0x7faba01023a0 2024-03-18T23:06:05.665 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:05.661+0000 7faba6612640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7faba010d370 con 0x7faba0107600 2024-03-18T23:06:05.665 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:05.661+0000 7faba6612640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7faba010d670 con 0x7faba0104cd0 2024-03-18T23:06:05.665 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:05.661+0000 7fab9ffff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faba01023a0 0x7faba0197b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:06:05.665 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:05.661+0000 7fab9ffff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faba01023a0 0x7faba0197b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:54088/0 (socket says 172.21.15.49:54088) 2024-03-18T23:06:05.665 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:05.661+0000 7fab9ffff640 1 -- 172.21.15.49:0/3691541271 learned_addr learned my addr 172.21.15.49:0/3691541271 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:06:05.665 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:05.661+0000 7faba4baf640 1 --2- 172.21.15.49:0/3691541271 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faba0107600 0x7faba019f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:06:05.665 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:05.661+0000 7fab9f7fe640 1 --2- 172.21.15.49:0/3691541271 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faba0104cd0 0x7faba0198080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:06:05.665 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:05.661+0000 7fab9ffff640 1 -- 172.21.15.49:0/3691541271 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faba0104cd0 msgr2=0x7faba0198080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:05.665 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:05.661+0000 7fab9ffff640 1 --2- 172.21.15.49:0/3691541271 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faba0104cd0 0x7faba0198080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:05.665 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:05.661+0000 7fab9ffff640 1 -- 172.21.15.49:0/3691541271 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faba0107600 msgr2=0x7faba019f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:05.665 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:05.661+0000 7fab9ffff640 1 --2- 172.21.15.49:0/3691541271 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faba0107600 0x7faba019f0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:05.665 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:05.661+0000 7fab9ffff640 1 -- 172.21.15.49:0/3691541271 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faba019f760 con 0x7faba01023a0 2024-03-18T23:06:05.666 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:05.661+0000 7fab9f7fe640 1 --2- 172.21.15.49:0/3691541271 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faba0104cd0 0x7faba0198080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T23:06:05.666 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:05.662+0000 7fab9ffff640 1 --2- 172.21.15.49:0/3691541271 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faba01023a0 0x7faba0197b40 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fab90009230 tx=0x7fab90005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:06:05.666 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:05.662+0000 7fab9d7fa640 1 -- 172.21.15.49:0/3691541271 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab90041070 con 0x7faba01023a0 2024-03-18T23:06:05.666 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:05.662+0000 7faba6612640 1 -- 172.21.15.49:0/3691541271 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faba019f9f0 con 0x7faba01023a0 2024-03-18T23:06:05.666 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:05.662+0000 7fab9d7fa640 1 -- 172.21.15.49:0/3691541271 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fab900044d0 con 0x7faba01023a0 2024-03-18T23:06:05.666 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:05.662+0000 7fab9d7fa640 1 -- 172.21.15.49:0/3691541271 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab90034440 con 0x7faba01023a0 2024-03-18T23:06:05.666 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:05.662+0000 7faba6612640 1 -- 172.21.15.49:0/3691541271 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faba019fee0 con 0x7faba01023a0 2024-03-18T23:06:05.667 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:05.663+0000 7faba6612640 1 -- 172.21.15.49:0/3691541271 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faba00fd840 con 0x7faba01023a0 2024-03-18T23:06:05.667 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:05.664+0000 7fab9d7fa640 1 -- 172.21.15.49:0/3691541271 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7fab90044070 con 0x7faba01023a0 2024-03-18T23:06:05.668 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:05.664+0000 7fab9d7fa640 1 --2- 172.21.15.49:0/3691541271 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fab880a2800 0x7fab880a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:06:05.668 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:05.664+0000 7fab9f7fe640 1 --2- 172.21.15.49:0/3691541271 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fab880a2800 0x7fab880a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:06:05.668 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:05.665+0000 7fab9d7fa640 1 -- 172.21.15.49:0/3691541271 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7fab900f0660 con 0x7faba01023a0 2024-03-18T23:06:05.668 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:05.665+0000 7fab9f7fe640 1 --2- 172.21.15.49:0/3691541271 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fab880a2800 0x7fab880a4cc0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fab94005d00 tx=0x7fab94005ed0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:06:05.669 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:05.669+0000 7fab9d7fa640 1 -- 172.21.15.49:0/3691541271 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fab900bd7b0 con 0x7faba01023a0 2024-03-18T23:06:05.854 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:05.853+0000 7faba6612640 1 -- 172.21.15.49:0/3691541271 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7faba0062420 con 0x7faba01023a0 2024-03-18T23:06:05.856 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:05.855+0000 7fab9d7fa640 1 -- 172.21.15.49:0/3691541271 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) ==== 74+0+22255 (secure 0 0 0) 0x7fab900bd0b0 con 0x7faba01023a0 2024-03-18T23:06:05.856 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-18T23:06:05.856 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":85,"fsid":"c63591ea-e57a-11ee-95c9-87774f69a715","created":"2024-03-18T22:57:58.734467+0000","modified":"2024-03-18T23:06:04.575518+0000","last_up_change":"2024-03-18T23:05:16.576310+0000","last_in_change":"2024-03-18T23:04:57.024067+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-18T23:01:28.363432+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-18T23:06:03.172963+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-18T23:06:03.973303+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e7fc9daa-4098-4ba5-8019-9aaef1aab3e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":1651162501},{"type":"v1","addr":"172.21.15.49:6803","nonce":1651162501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":1651162501},{"type":"v1","addr":"172.21.15.49:6805","nonce":1651162501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":1651162501},{"type":"v1","addr":"172.21.15.49:6809","nonce":1651162501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":1651162501},{"type":"v1","addr":"172.21.15.49:6807","nonce":1651162501}]},"public_addr":"172.21.15.49:6803/1651162501","cluster_addr":"172.21.15.49:6805/1651162501","heartbeat_back_addr":"172.21.15.49:6809/1651162501","heartbeat_front_addr":"172.21.15.49:6807/1651162501","state":["exists","up"]},{"osd":1,"uuid":"40ac929c-e90f-434c-a796-9aabd686f1fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":1766700359},{"type":"v1","addr":"172.21.15.139:6801","nonce":1766700359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":1766700359},{"type":"v1","addr":"172.21.15.139:6803","nonce":1766700359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":1766700359},{"type":"v1","addr":"172.21.15.139:6807","nonce":1766700359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":1766700359},{"type":"v1","addr":"172.21.15.139:6805","nonce":1766700359}]},"public_addr":"172.21.15.139:6801/1766700359","cluster_addr":"172.21.15.139:6803/1766700359","heartbeat_back_addr":"172.21.15.139:6807/1766700359","heartbeat_front_addr":"172.21.15.139:6805/1766700359","state":["exists","up"]},{"osd":2,"uuid":"8ae3b666-00b4-45ed-a5ee-4d0771edabb6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":3752671234},{"type":"v1","addr":"172.21.15.175:6801","nonce":3752671234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":3752671234},{"type":"v1","addr":"172.21.15.175:6803","nonce":3752671234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":3752671234},{"type":"v1","addr":"172.21.15.175:6807","nonce":3752671234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":3752671234},{"type":"v1","addr":"172.21.15.175:6805","nonce":3752671234}]},"public_addr":"172.21.15.175:6801/3752671234","cluster_addr":"172.21.15.175:6803/3752671234","heartbeat_back_addr":"172.21.15.175:6807/3752671234","heartbeat_front_addr":"172.21.15.175:6805/3752671234","state":["exists","up"]},{"osd":3,"uuid":"1a62e89d-45d4-4237-b813-7f96a59aca7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":956160536},{"type":"v1","addr":"172.21.15.49:6811","nonce":956160536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":956160536},{"type":"v1","addr":"172.21.15.49:6813","nonce":956160536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":956160536},{"type":"v1","addr":"172.21.15.49:6817","nonce":956160536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":956160536},{"type":"v1","addr":"172.21.15.49:6815","nonce":956160536}]},"public_addr":"172.21.15.49:6811/956160536","cluster_addr":"172.21.15.49:6813/956160536","heartbeat_back_addr":"172.21.15.49:6817/956160536","heartbeat_front_addr":"172.21.15.49:6815/956160536","state":["exists","up"]},{"osd":4,"uuid":"c715101c-0c3f-4b98-b142-d64a35e187df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":3129712668},{"type":"v1","addr":"172.21.15.139:6809","nonce":3129712668}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":3129712668},{"type":"v1","addr":"172.21.15.139:6811","nonce":3129712668}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":3129712668},{"type":"v1","addr":"172.21.15.139:6815","nonce":3129712668}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":3129712668},{"type":"v1","addr":"172.21.15.139:6813","nonce":3129712668}]},"public_addr":"172.21.15.139:6809/3129712668","cluster_addr":"172.21.15.139:6811/3129712668","heartbeat_back_addr":"172.21.15.139:6815/3129712668","heartbeat_front_addr":"172.21.15.139:6813/3129712668","state":["exists","up"]},{"osd":5,"uuid":"4b641f81-77eb-45fd-8464-511e7b3062b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":2774567215},{"type":"v1","addr":"172.21.15.175:6809","nonce":2774567215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":2774567215},{"type":"v1","addr":"172.21.15.175:6811","nonce":2774567215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":2774567215},{"type":"v1","addr":"172.21.15.175:6815","nonce":2774567215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":2774567215},{"type":"v1","addr":"172.21.15.175:6813","nonce":2774567215}]},"public_addr":"172.21.15.175:6809/2774567215","cluster_addr":"172.21.15.175:6811/2774567215","heartbeat_back_addr":"172.21.15.175:6815/2774567215","heartbeat_front_addr":"172.21.15.175:6813/2774567215","state":["exists","up"]},{"osd":6,"uuid":"653da8e0-1197-4b48-abed-470f2a9c4dce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":1472215461},{"type":"v1","addr":"172.21.15.49:6819","nonce":1472215461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":1472215461},{"type":"v1","addr":"172.21.15.49:6821","nonce":1472215461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":1472215461},{"type":"v1","addr":"172.21.15.49:6825","nonce":1472215461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":1472215461},{"type":"v1","addr":"172.21.15.49:6823","nonce":1472215461}]},"public_addr":"172.21.15.49:6819/1472215461","cluster_addr":"172.21.15.49:6821/1472215461","heartbeat_back_addr":"172.21.15.49:6825/1472215461","heartbeat_front_addr":"172.21.15.49:6823/1472215461","state":["exists","up"]},{"osd":7,"uuid":"ad157c9b-4b75-451c-8740-630f0975b7fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":2163986787},{"type":"v1","addr":"172.21.15.139:6817","nonce":2163986787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":2163986787},{"type":"v1","addr":"172.21.15.139:6819","nonce":2163986787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":2163986787},{"type":"v1","addr":"172.21.15.139:6823","nonce":2163986787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":2163986787},{"type":"v1","addr":"172.21.15.139:6821","nonce":2163986787}]},"public_addr":"172.21.15.139:6817/2163986787","cluster_addr":"172.21.15.139:6819/2163986787","heartbeat_back_addr":"172.21.15.139:6823/2163986787","heartbeat_front_addr":"172.21.15.139:6821/2163986787","state":["exists","up"]},{"osd":8,"uuid":"c8e75f15-df75-4d5d-9b92-a064bffb6c01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":1194026992},{"type":"v1","addr":"172.21.15.175:6817","nonce":1194026992}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1194026992},{"type":"v1","addr":"172.21.15.175:6819","nonce":1194026992}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":1194026992},{"type":"v1","addr":"172.21.15.175:6823","nonce":1194026992}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1194026992},{"type":"v1","addr":"172.21.15.175:6821","nonce":1194026992}]},"public_addr":"172.21.15.175:6817/1194026992","cluster_addr":"172.21.15.175:6819/1194026992","heartbeat_back_addr":"172.21.15.175:6823/1194026992","heartbeat_front_addr":"172.21.15.175:6821/1194026992","state":["exists","up"]},{"osd":9,"uuid":"edb5ffef-b6a2-4941-b5ab-40f982ad30d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":2742084884},{"type":"v1","addr":"172.21.15.49:6827","nonce":2742084884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":2742084884},{"type":"v1","addr":"172.21.15.49:6829","nonce":2742084884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":2742084884},{"type":"v1","addr":"172.21.15.49:6833","nonce":2742084884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":2742084884},{"type":"v1","addr":"172.21.15.49:6831","nonce":2742084884}]},"public_addr":"172.21.15.49:6827/2742084884","cluster_addr":"172.21.15.49:6829/2742084884","heartbeat_back_addr":"172.21.15.49:6833/2742084884","heartbeat_front_addr":"172.21.15.49:6831/2742084884","state":["exists","up"]},{"osd":10,"uuid":"3945e46d-766a-4c4a-8821-d43648dd1578","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":2404740201},{"type":"v1","addr":"172.21.15.139:6825","nonce":2404740201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":2404740201},{"type":"v1","addr":"172.21.15.139:6827","nonce":2404740201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":2404740201},{"type":"v1","addr":"172.21.15.139:6831","nonce":2404740201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":2404740201},{"type":"v1","addr":"172.21.15.139:6829","nonce":2404740201}]},"public_addr":"172.21.15.139:6825/2404740201","cluster_addr":"172.21.15.139:6827/2404740201","heartbeat_back_addr":"172.21.15.139:6831/2404740201","heartbeat_front_addr":"172.21.15.139:6829/2404740201","state":["exists","up"]},{"osd":11,"uuid":"9954ba15-d877-46cc-b24a-893f9168deb8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":3519503868},{"type":"v1","addr":"172.21.15.175:6825","nonce":3519503868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":3519503868},{"type":"v1","addr":"172.21.15.175:6827","nonce":3519503868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":3519503868},{"type":"v1","addr":"172.21.15.175:6831","nonce":3519503868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":3519503868},{"type":"v1","addr":"172.21.15.175:6829","nonce":3519503868}]},"public_addr":"172.21.15.175:6825/3519503868","cluster_addr":"172.21.15.175:6827/3519503868","heartbeat_back_addr":"172.21.15.175:6831/3519503868","heartbeat_front_addr":"172.21.15.175:6829/3519503868","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:00:38.367865+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:01:02.237687+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:01:25.609751+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:01:51.268572+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:02:16.514605+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:02:41.586523+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:03:07.029964+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:03:32.956250+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:03:57.547984+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:04:23.948972+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:04:49.667210+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:05:15.284693+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.49:0/563527416":"2024-03-19T22:58:59.806253+0000","172.21.15.49:0/1036383534":"2024-03-19T22:58:59.806253+0000","172.21.15.49:6801/2609458951":"2024-03-19T22:58:59.806253+0000","172.21.15.49:0/21283102":"2024-03-19T22:58:31.972474+0000","172.21.15.49:0/1292058922":"2024-03-19T22:58:31.972474+0000","172.21.15.49:6800/1985190312":"2024-03-19T22:58:31.972474+0000","172.21.15.49:0/1912446700":"2024-03-19T22:58:59.806253+0000","172.21.15.49:0/2176210039":"2024-03-19T22:58:31.972474+0000","172.21.15.49:6801/1985190312":"2024-03-19T22:58:31.972474+0000","172.21.15.49:6800/2609458951":"2024-03-19T22:58:59.806253+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-18T23:06:05.858 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:05.857+0000 7faba6612640 1 -- 172.21.15.49:0/3691541271 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fab880a2800 msgr2=0x7fab880a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:05.858 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:05.857+0000 7faba6612640 1 --2- 172.21.15.49:0/3691541271 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fab880a2800 0x7fab880a4cc0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fab94005d00 tx=0x7fab94005ed0 comp rx=0 tx=0).stop 2024-03-18T23:06:05.858 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:05.857+0000 7faba6612640 1 -- 172.21.15.49:0/3691541271 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faba01023a0 msgr2=0x7faba0197b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:05.858 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:05.857+0000 7faba6612640 1 --2- 172.21.15.49:0/3691541271 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faba01023a0 0x7faba0197b40 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fab90009230 tx=0x7fab90005d50 comp rx=0 tx=0).stop 2024-03-18T23:06:05.858 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:05.857+0000 7faba6612640 1 -- 172.21.15.49:0/3691541271 shutdown_connections 2024-03-18T23:06:05.858 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:05.857+0000 7faba6612640 1 --2- 172.21.15.49:0/3691541271 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fab880a2800 0x7fab880a4cc0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:05.858 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:05.857+0000 7faba6612640 1 --2- 172.21.15.49:0/3691541271 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faba0107600 0x7faba019f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:05.859 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:05.857+0000 7faba6612640 1 --2- 172.21.15.49:0/3691541271 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faba0104cd0 0x7faba0198080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:05.859 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:05.857+0000 7faba6612640 1 --2- 172.21.15.49:0/3691541271 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faba01023a0 0x7faba0197b40 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:05.859 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:05.857+0000 7faba6612640 1 -- 172.21.15.49:0/3691541271 >> 172.21.15.49:0/3691541271 conn(0x7faba00fbd50 msgr2=0x7faba0102c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:06:05.859 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:05.857+0000 7faba6612640 1 -- 172.21.15.49:0/3691541271 shutdown_connections 2024-03-18T23:06:05.859 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:05.857+0000 7faba6612640 1 -- 172.21.15.49:0/3691541271 wait complete. 2024-03-18T23:06:05.869 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-03-18T23:06:05.984 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:05.981+0000 7f7d8b960640 1 -- 172.21.15.49:0/3550546973 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7d84101930 msgr2=0x7f7d84101db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:05.984 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:05.981+0000 7f7d8b960640 1 --2- 172.21.15.49:0/3550546973 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7d84101930 0x7f7d84101db0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f7d80009900 tx=0x7f7d8002bc30 comp rx=0 tx=0).stop 2024-03-18T23:06:05.984 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:05.981+0000 7f7d8b960640 1 -- 172.21.15.49:0/3550546973 shutdown_connections 2024-03-18T23:06:05.984 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:05.981+0000 7f7d8b960640 1 --2- 172.21.15.49:0/3550546973 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7d841022f0 0x7f7d84108b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:05.984 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:05.981+0000 7f7d8b960640 1 --2- 172.21.15.49:0/3550546973 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7d84101930 0x7f7d84101db0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:05.985 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:05.981+0000 7f7d8b960640 1 --2- 172.21.15.49:0/3550546973 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7d84100740 0x7f7d84100b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:05.985 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:05.981+0000 7f7d8b960640 1 -- 172.21.15.49:0/3550546973 >> 172.21.15.49:0/3550546973 conn(0x7f7d840fbcd0 msgr2=0x7f7d840fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:06:05.985 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:05.981+0000 7f7d8b960640 1 -- 172.21.15.49:0/3550546973 shutdown_connections 2024-03-18T23:06:05.985 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:05.981+0000 7f7d8b960640 1 -- 172.21.15.49:0/3550546973 wait complete. 2024-03-18T23:06:05.985 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:05.981+0000 7f7d8b960640 1 Processor -- start 2024-03-18T23:06:05.985 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:05.981+0000 7f7d8b960640 1 -- start start 2024-03-18T23:06:05.985 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:05.982+0000 7f7d8b960640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7d84100740 0x7f7d84197aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:06:05.985 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:05.982+0000 7f7d8b960640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7d84101930 0x7f7d84197fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:06:05.985 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:05.982+0000 7f7d8b960640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7d841022f0 0x7f7d8419f050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:06:05.985 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:05.982+0000 7f7d8b960640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f7d8410d4e0 con 0x7f7d84100740 2024-03-18T23:06:05.985 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:05.982+0000 7f7d8b960640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f7d8410d360 con 0x7f7d841022f0 2024-03-18T23:06:05.985 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:05.982+0000 7f7d8b960640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f7d8410d660 con 0x7f7d84101930 2024-03-18T23:06:05.985 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:05.982+0000 7f7d896fc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7d84100740 0x7f7d84197aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:06:05.985 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:05.982+0000 7f7d896fc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7d84100740 0x7f7d84197aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:54104/0 (socket says 172.21.15.49:54104) 2024-03-18T23:06:05.986 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:05.982+0000 7f7d896fc640 1 -- 172.21.15.49:0/886008344 learned_addr learned my addr 172.21.15.49:0/886008344 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:06:05.986 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:05.982+0000 7f7d88efb640 1 --2- 172.21.15.49:0/886008344 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7d84101930 0x7f7d84197fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:06:05.986 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:05.982+0000 7f7d89efd640 1 --2- 172.21.15.49:0/886008344 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7d841022f0 0x7f7d8419f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:06:05.986 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:05.982+0000 7f7d896fc640 1 -- 172.21.15.49:0/886008344 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7d84101930 msgr2=0x7f7d84197fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:05.986 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:05.982+0000 7f7d896fc640 1 --2- 172.21.15.49:0/886008344 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7d84101930 0x7f7d84197fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:05.986 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:05.982+0000 7f7d896fc640 1 -- 172.21.15.49:0/886008344 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7d841022f0 msgr2=0x7f7d8419f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:05.986 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:05.982+0000 7f7d896fc640 1 --2- 172.21.15.49:0/886008344 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7d841022f0 0x7f7d8419f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:05.986 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:05.982+0000 7f7d896fc640 1 -- 172.21.15.49:0/886008344 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d8419f6c0 con 0x7f7d84100740 2024-03-18T23:06:05.986 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:05.982+0000 7f7d89efd640 1 --2- 172.21.15.49:0/886008344 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7d841022f0 0x7f7d8419f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T23:06:05.987 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:05.982+0000 7f7d88efb640 1 --2- 172.21.15.49:0/886008344 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7d84101930 0x7f7d84197fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T23:06:05.987 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:05.982+0000 7f7d896fc640 1 --2- 172.21.15.49:0/886008344 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7d84100740 0x7f7d84197aa0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f7d7400c970 tx=0x7f7d7400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:06:05.987 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:05.982+0000 7f7d7a7fc640 1 -- 172.21.15.49:0/886008344 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d74007c10 con 0x7f7d84100740 2024-03-18T23:06:05.987 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:05.982+0000 7f7d7a7fc640 1 -- 172.21.15.49:0/886008344 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d74007db0 con 0x7f7d84100740 2024-03-18T23:06:05.987 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:05.982+0000 7f7d8b960640 1 -- 172.21.15.49:0/886008344 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d8419f9b0 con 0x7f7d84100740 2024-03-18T23:06:05.987 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:05.982+0000 7f7d7a7fc640 1 -- 172.21.15.49:0/886008344 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d7400f940 con 0x7f7d84100740 2024-03-18T23:06:05.987 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:05.982+0000 7f7d8b960640 1 -- 172.21.15.49:0/886008344 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d8419ff50 con 0x7f7d84100740 2024-03-18T23:06:05.987 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:05.983+0000 7f7d8b960640 1 -- 172.21.15.49:0/886008344 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d84107990 con 0x7f7d84100740 2024-03-18T23:06:05.989 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:05.984+0000 7f7d7a7fc640 1 -- 172.21.15.49:0/886008344 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7f7d7400fae0 con 0x7f7d84100740 2024-03-18T23:06:05.989 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:05.985+0000 7f7d7a7fc640 1 --2- 172.21.15.49:0/886008344 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f7d700a2880 0x7f7d700a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:06:05.989 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:05.985+0000 7f7d88efb640 1 --2- 172.21.15.49:0/886008344 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f7d700a2880 0x7f7d700a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:06:05.989 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:05.985+0000 7f7d7a7fc640 1 -- 172.21.15.49:0/886008344 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f7d740ced00 con 0x7f7d84100740 2024-03-18T23:06:05.989 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:05.986+0000 7f7d88efb640 1 --2- 172.21.15.49:0/886008344 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f7d700a2880 0x7f7d700a4d40 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f7d80036040 tx=0x7f7d80000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:06:05.990 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:05.990+0000 7f7d7a7fc640 1 -- 172.21.15.49:0/886008344 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f7d7409bf80 con 0x7f7d84100740 2024-03-18T23:06:06.237 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:06.236+0000 7f7d8b960640 1 -- 172.21.15.49:0/886008344 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f7d8402d1f0 con 0x7f7d84100740 2024-03-18T23:06:06.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:06:06 smithi139 ceph-mon[67282]: from='client.? 172.21.15.49:0/2988325313' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-03-18T23:06:06.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:06:06 smithi139 ceph-mon[67282]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-03-18T23:06:06.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:06:06 smithi139 ceph-mon[67282]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-03-18T23:06:06.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:06:06 smithi139 ceph-mon[67282]: from='client.? 172.21.15.49:0/1184137159' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T23:06:06.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:06:06 smithi139 ceph-mon[67282]: from='client.? 172.21.15.49:0/3691541271' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T23:06:06.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:06:06 smithi139 ceph-mon[67282]: osdmap e86: 12 total, 12 up, 12 in 2024-03-18T23:06:06.630 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:06:06 smithi049 ceph-mon[62991]: from='client.? 172.21.15.49:0/2988325313' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-03-18T23:06:06.630 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:06:06 smithi049 ceph-mon[62991]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-03-18T23:06:06.630 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:06:06 smithi049 ceph-mon[62991]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-03-18T23:06:06.630 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:06:06 smithi049 ceph-mon[62991]: from='client.? 172.21.15.49:0/1184137159' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T23:06:06.630 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:06:06 smithi049 ceph-mon[62991]: from='client.? 172.21.15.49:0/3691541271' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T23:06:06.631 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:06:06 smithi049 ceph-mon[62991]: osdmap e86: 12 total, 12 up, 12 in 2024-03-18T23:06:06.712 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:06:06 smithi175 ceph-mon[67352]: from='client.? 172.21.15.49:0/2988325313' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-03-18T23:06:06.712 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:06:06 smithi175 ceph-mon[67352]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-03-18T23:06:06.712 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:06:06 smithi175 ceph-mon[67352]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-03-18T23:06:06.712 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:06:06 smithi175 ceph-mon[67352]: from='client.? 172.21.15.49:0/1184137159' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T23:06:06.712 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:06:06 smithi175 ceph-mon[67352]: from='client.? 172.21.15.49:0/3691541271' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T23:06:06.712 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:06:06 smithi175 ceph-mon[67352]: osdmap e86: 12 total, 12 up, 12 in 2024-03-18T23:06:07.222 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:07.222+0000 7f7d7a7fc640 1 -- 172.21.15.49:0/886008344 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v17) ==== 118+0+0 (secure 0 0 0) 0x7f7d7409b880 con 0x7f7d84100740 2024-03-18T23:06:07.227 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:07.225+0000 7f7d8b960640 1 -- 172.21.15.49:0/886008344 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f7d700a2880 msgr2=0x7f7d700a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:07.227 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:07.225+0000 7f7d8b960640 1 --2- 172.21.15.49:0/886008344 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f7d700a2880 0x7f7d700a4d40 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f7d80036040 tx=0x7f7d80000bc0 comp rx=0 tx=0).stop 2024-03-18T23:06:07.227 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:07.225+0000 7f7d8b960640 1 -- 172.21.15.49:0/886008344 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7d84100740 msgr2=0x7f7d84197aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:07.227 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:07.225+0000 7f7d8b960640 1 --2- 172.21.15.49:0/886008344 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7d84100740 0x7f7d84197aa0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f7d7400c970 tx=0x7f7d7400ce30 comp rx=0 tx=0).stop 2024-03-18T23:06:07.227 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:07.226+0000 7f7d8b960640 1 -- 172.21.15.49:0/886008344 shutdown_connections 2024-03-18T23:06:07.227 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:07.226+0000 7f7d8b960640 1 --2- 172.21.15.49:0/886008344 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f7d700a2880 0x7f7d700a4d40 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:07.227 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:07.226+0000 7f7d8b960640 1 --2- 172.21.15.49:0/886008344 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7d841022f0 0x7f7d8419f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:07.227 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:07.226+0000 7f7d8b960640 1 --2- 172.21.15.49:0/886008344 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7d84101930 0x7f7d84197fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:07.227 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:07.226+0000 7f7d8b960640 1 --2- 172.21.15.49:0/886008344 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7d84100740 0x7f7d84197aa0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:07.227 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:07.226+0000 7f7d8b960640 1 -- 172.21.15.49:0/886008344 >> 172.21.15.49:0/886008344 conn(0x7f7d840fbcd0 msgr2=0x7f7d84104b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:06:07.227 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:07.226+0000 7f7d8b960640 1 -- 172.21.15.49:0/886008344 shutdown_connections 2024-03-18T23:06:07.227 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:07.226+0000 7f7d8b960640 1 -- 172.21.15.49:0/886008344 wait complete. 2024-03-18T23:06:07.242 DEBUG:tasks.cephfs.filesystem:fs_config: {'max_mds': 5, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated'}} 2024-03-18T23:06:07.242 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 5 2024-03-18T23:06:07.363 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:07.359+0000 7fcae8a12640 1 -- 172.21.15.49:0/1383591282 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcae4100720 msgr2=0x7fcae4100b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:07.363 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:07.359+0000 7fcae8a12640 1 --2- 172.21.15.49:0/1383591282 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcae4100720 0x7fcae4100b40 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fcadc009900 tx=0x7fcadc02bc30 comp rx=0 tx=0).stop 2024-03-18T23:06:07.363 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:07.360+0000 7fcae8a12640 1 -- 172.21.15.49:0/1383591282 shutdown_connections 2024-03-18T23:06:07.363 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:07.360+0000 7fcae8a12640 1 --2- 172.21.15.49:0/1383591282 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcae41022d0 0x7fcae4108b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:07.363 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:07.360+0000 7fcae8a12640 1 --2- 172.21.15.49:0/1383591282 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcae4101910 0x7fcae4101d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:07.363 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:07.360+0000 7fcae8a12640 1 --2- 172.21.15.49:0/1383591282 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcae4100720 0x7fcae4100b40 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:07.364 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:07.360+0000 7fcae8a12640 1 -- 172.21.15.49:0/1383591282 >> 172.21.15.49:0/1383591282 conn(0x7fcae40fbcb0 msgr2=0x7fcae40fe0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:06:07.364 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:07.360+0000 7fcae8a12640 1 -- 172.21.15.49:0/1383591282 shutdown_connections 2024-03-18T23:06:07.364 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:07.360+0000 7fcae8a12640 1 -- 172.21.15.49:0/1383591282 wait complete. 2024-03-18T23:06:07.364 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:07.360+0000 7fcae8a12640 1 Processor -- start 2024-03-18T23:06:07.364 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:07.360+0000 7fcae8a12640 1 -- start start 2024-03-18T23:06:07.364 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:07.361+0000 7fcae8a12640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcae4100720 0x7fcae4069510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:06:07.364 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:07.361+0000 7fcae8a12640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcae4101910 0x7fcae419c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:06:07.364 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:07.361+0000 7fcae8a12640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcae41022d0 0x7fcae419ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:06:07.364 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:07.361+0000 7fcae8a12640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fcae410c390 con 0x7fcae4100720 2024-03-18T23:06:07.364 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:07.361+0000 7fcae8a12640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fcae410c210 con 0x7fcae4101910 2024-03-18T23:06:07.364 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:07.361+0000 7fcae8a12640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fcae410c510 con 0x7fcae41022d0 2024-03-18T23:06:07.364 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:07.361+0000 7fcae259c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcae4100720 0x7fcae4069510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:06:07.364 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:07.361+0000 7fcae259c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcae4100720 0x7fcae4069510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:54120/0 (socket says 172.21.15.49:54120) 2024-03-18T23:06:07.364 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:07.361+0000 7fcae259c640 1 -- 172.21.15.49:0/1859282450 learned_addr learned my addr 172.21.15.49:0/1859282450 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:06:07.365 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:07.361+0000 7fcae1d9b640 1 --2- 172.21.15.49:0/1859282450 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcae4101910 0x7fcae419c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:06:07.365 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:07.361+0000 7fcae2d9d640 1 --2- 172.21.15.49:0/1859282450 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcae41022d0 0x7fcae419ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:06:07.365 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:07.361+0000 7fcae259c640 1 -- 172.21.15.49:0/1859282450 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcae41022d0 msgr2=0x7fcae419ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:07.365 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:07.361+0000 7fcae259c640 1 --2- 172.21.15.49:0/1859282450 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcae41022d0 0x7fcae419ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:07.365 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:07.361+0000 7fcae259c640 1 -- 172.21.15.49:0/1859282450 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcae4101910 msgr2=0x7fcae419c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:07.365 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:07.361+0000 7fcae259c640 1 --2- 172.21.15.49:0/1859282450 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcae4101910 0x7fcae419c8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:07.365 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:07.361+0000 7fcae259c640 1 -- 172.21.15.49:0/1859282450 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcae419f7c0 con 0x7fcae4100720 2024-03-18T23:06:07.365 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:07.361+0000 7fcae2d9d640 1 --2- 172.21.15.49:0/1859282450 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcae41022d0 0x7fcae419ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T23:06:07.365 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:07.361+0000 7fcae259c640 1 --2- 172.21.15.49:0/1859282450 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcae4100720 0x7fcae4069510 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fcadc000c00 tx=0x7fcadc005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:06:07.365 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:07.361+0000 7fcad37fe640 1 -- 172.21.15.49:0/1859282450 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcadc041070 con 0x7fcae4100720 2024-03-18T23:06:07.365 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:07.361+0000 7fcad37fe640 1 -- 172.21.15.49:0/1859282450 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcadc0044d0 con 0x7fcae4100720 2024-03-18T23:06:07.365 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:07.361+0000 7fcae8a12640 1 -- 172.21.15.49:0/1859282450 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcae419fa50 con 0x7fcae4100720 2024-03-18T23:06:07.365 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:07.361+0000 7fcad37fe640 1 -- 172.21.15.49:0/1859282450 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcadc034440 con 0x7fcae4100720 2024-03-18T23:06:07.365 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:07.361+0000 7fcae8a12640 1 -- 172.21.15.49:0/1859282450 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcae419ff40 con 0x7fcae4100720 2024-03-18T23:06:07.366 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:07.363+0000 7fcae8a12640 1 -- 172.21.15.49:0/1859282450 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcae4107b10 con 0x7fcae4100720 2024-03-18T23:06:07.366 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:07.364+0000 7fcad37fe640 1 -- 172.21.15.49:0/1859282450 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7fcadc044070 con 0x7fcae4100720 2024-03-18T23:06:07.367 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:07.364+0000 7fcad37fe640 1 --2- 172.21.15.49:0/1859282450 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fcacc0a2800 0x7fcacc0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:06:07.367 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:07.365+0000 7fcae1d9b640 1 --2- 172.21.15.49:0/1859282450 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fcacc0a2800 0x7fcacc0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:06:07.367 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:07.365+0000 7fcad37fe640 1 -- 172.21.15.49:0/1859282450 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7fcadc0f04e0 con 0x7fcae4100720 2024-03-18T23:06:07.367 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:07.365+0000 7fcae1d9b640 1 --2- 172.21.15.49:0/1859282450 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fcacc0a2800 0x7fcacc0a4cc0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fcae4069280 tx=0x7fcad800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:06:07.370 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:07.369+0000 7fcad37fe640 1 -- 172.21.15.49:0/1859282450 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fcadc0bd6d0 con 0x7fcae4100720 2024-03-18T23:06:07.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:06:07 smithi139 ceph-mon[67282]: pgmap v289: 97 pgs: 31 active+clean, 2 creating+activating, 1 active, 14 creating+peering, 49 unknown; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 5 op/s 2024-03-18T23:06:07.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:06:07 smithi139 ceph-mon[67282]: mds.? [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] up:active 2024-03-18T23:06:07.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:06:07 smithi139 ceph-mon[67282]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-18T23:06:07.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:06:07 smithi139 ceph-mon[67282]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-18T23:06:07.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:06:07 smithi139 ceph-mon[67282]: from='client.? 172.21.15.49:0/886008344' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-03-18T23:06:07.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:06:07 smithi139 ceph-mon[67282]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-03-18T23:06:07.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:06:07 smithi139 ceph-mon[67282]: Cluster is now healthy 2024-03-18T23:06:07.594 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:07.593+0000 7fcae8a12640 1 -- 172.21.15.49:0/1859282450 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} v 0) -- 0x7fcae402d410 con 0x7fcae4100720 2024-03-18T23:06:07.630 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:06:07 smithi049 ceph-mon[62991]: pgmap v289: 97 pgs: 31 active+clean, 2 creating+activating, 1 active, 14 creating+peering, 49 unknown; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 5 op/s 2024-03-18T23:06:07.630 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:06:07 smithi049 ceph-mon[62991]: mds.? [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] up:active 2024-03-18T23:06:07.630 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:06:07 smithi049 ceph-mon[62991]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-18T23:06:07.630 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:06:07 smithi049 ceph-mon[62991]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-18T23:06:07.630 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:06:07 smithi049 ceph-mon[62991]: from='client.? 172.21.15.49:0/886008344' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-03-18T23:06:07.630 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:06:07 smithi049 ceph-mon[62991]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-03-18T23:06:07.630 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:06:07 smithi049 ceph-mon[62991]: Cluster is now healthy 2024-03-18T23:06:07.711 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:06:07 smithi175 ceph-mon[67352]: pgmap v289: 97 pgs: 31 active+clean, 2 creating+activating, 1 active, 14 creating+peering, 49 unknown; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 5 op/s 2024-03-18T23:06:07.711 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:06:07 smithi175 ceph-mon[67352]: mds.? [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] up:active 2024-03-18T23:06:07.712 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:06:07 smithi175 ceph-mon[67352]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-18T23:06:07.712 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:06:07 smithi175 ceph-mon[67352]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-18T23:06:07.712 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:06:07 smithi175 ceph-mon[67352]: from='client.? 172.21.15.49:0/886008344' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-03-18T23:06:07.712 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:06:07 smithi175 ceph-mon[67352]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-03-18T23:06:07.712 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:06:07 smithi175 ceph-mon[67352]: Cluster is now healthy 2024-03-18T23:06:08.233 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:08.232+0000 7fcad37fe640 1 -- 172.21.15.49:0/1859282450 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]=0 v18) ==== 105+0+0 (secure 0 0 0) 0x7fcadc0bcfd0 con 0x7fcae4100720 2024-03-18T23:06:08.236 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:08.235+0000 7fcae8a12640 1 -- 172.21.15.49:0/1859282450 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fcacc0a2800 msgr2=0x7fcacc0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:08.236 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:08.235+0000 7fcae8a12640 1 --2- 172.21.15.49:0/1859282450 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fcacc0a2800 0x7fcacc0a4cc0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fcae4069280 tx=0x7fcad800b390 comp rx=0 tx=0).stop 2024-03-18T23:06:08.236 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:08.235+0000 7fcae8a12640 1 -- 172.21.15.49:0/1859282450 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcae4100720 msgr2=0x7fcae4069510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:08.236 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:08.235+0000 7fcae8a12640 1 --2- 172.21.15.49:0/1859282450 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcae4100720 0x7fcae4069510 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fcadc000c00 tx=0x7fcadc005d50 comp rx=0 tx=0).stop 2024-03-18T23:06:08.236 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:08.235+0000 7fcae8a12640 1 -- 172.21.15.49:0/1859282450 shutdown_connections 2024-03-18T23:06:08.236 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:08.235+0000 7fcae8a12640 1 --2- 172.21.15.49:0/1859282450 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fcacc0a2800 0x7fcacc0a4cc0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:08.236 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:08.235+0000 7fcae8a12640 1 --2- 172.21.15.49:0/1859282450 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcae41022d0 0x7fcae419ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:08.236 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:08.235+0000 7fcae8a12640 1 --2- 172.21.15.49:0/1859282450 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcae4101910 0x7fcae419c8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:08.236 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:08.235+0000 7fcae8a12640 1 --2- 172.21.15.49:0/1859282450 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcae4100720 0x7fcae4069510 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:08.236 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:08.235+0000 7fcae8a12640 1 -- 172.21.15.49:0/1859282450 >> 172.21.15.49:0/1859282450 conn(0x7fcae40fbcb0 msgr2=0x7fcae4104ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:06:08.236 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:08.235+0000 7fcae8a12640 1 -- 172.21.15.49:0/1859282450 shutdown_connections 2024-03-18T23:06:08.236 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:08.235+0000 7fcae8a12640 1 -- 172.21.15.49:0/1859282450 wait complete. 2024-03-18T23:06:08.251 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay true 2024-03-18T23:06:08.368 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:08.365+0000 7f458a790640 1 -- 172.21.15.49:0/272282721 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4584101930 msgr2=0x7f4584101db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:08.368 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:08.365+0000 7f458a790640 1 --2- 172.21.15.49:0/272282721 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4584101930 0x7f4584101db0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f4578009900 tx=0x7f457802bc30 comp rx=0 tx=0).stop 2024-03-18T23:06:08.368 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:08.365+0000 7f458a790640 1 -- 172.21.15.49:0/272282721 shutdown_connections 2024-03-18T23:06:08.368 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:08.365+0000 7f458a790640 1 --2- 172.21.15.49:0/272282721 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f45841022f0 0x7f4584108b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:08.368 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:08.365+0000 7f458a790640 1 --2- 172.21.15.49:0/272282721 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4584101930 0x7f4584101db0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:08.368 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:08.365+0000 7f458a790640 1 --2- 172.21.15.49:0/272282721 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4584100740 0x7f4584100b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:08.368 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:08.365+0000 7f458a790640 1 -- 172.21.15.49:0/272282721 >> 172.21.15.49:0/272282721 conn(0x7f45840fbcd0 msgr2=0x7f45840fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:06:08.368 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:08.365+0000 7f458a790640 1 -- 172.21.15.49:0/272282721 shutdown_connections 2024-03-18T23:06:08.369 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:08.365+0000 7f458a790640 1 -- 172.21.15.49:0/272282721 wait complete. 2024-03-18T23:06:08.369 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:08.365+0000 7f458a790640 1 Processor -- start 2024-03-18T23:06:08.369 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:08.365+0000 7f458a790640 1 -- start start 2024-03-18T23:06:08.369 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:08.366+0000 7f458a790640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4584100740 0x7f458406a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:06:08.369 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:08.366+0000 7f458a790640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4584101930 0x7f458419c880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:06:08.369 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:08.366+0000 7f458a790640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f45841022f0 0x7f458419eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:06:08.369 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:08.366+0000 7f458a790640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f458410c3c0 con 0x7f4584101930 2024-03-18T23:06:08.369 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:08.366+0000 7f458a790640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f458410c240 con 0x7f45841022f0 2024-03-18T23:06:08.369 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:08.366+0000 7f458a790640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f458410c540 con 0x7f4584100740 2024-03-18T23:06:08.369 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:08.366+0000 7f45837fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4584101930 0x7f458419c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:06:08.369 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:08.366+0000 7f45837fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4584101930 0x7f458419c880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:54140/0 (socket says 172.21.15.49:54140) 2024-03-18T23:06:08.369 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:08.366+0000 7f45837fe640 1 -- 172.21.15.49:0/855562320 learned_addr learned my addr 172.21.15.49:0/855562320 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:06:08.369 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:08.366+0000 7f4588d2d640 1 --2- 172.21.15.49:0/855562320 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f45841022f0 0x7f458419eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:06:08.369 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:08.366+0000 7f4583fff640 1 --2- 172.21.15.49:0/855562320 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4584100740 0x7f458406a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:06:08.370 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:08.366+0000 7f4588d2d640 1 -- 172.21.15.49:0/855562320 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4584100740 msgr2=0x7f458406a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:08.370 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:08.366+0000 7f4588d2d640 1 --2- 172.21.15.49:0/855562320 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4584100740 0x7f458406a960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:08.370 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:08.366+0000 7f4588d2d640 1 -- 172.21.15.49:0/855562320 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4584101930 msgr2=0x7f458419c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:08.370 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:08.366+0000 7f4588d2d640 1 --2- 172.21.15.49:0/855562320 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4584101930 0x7f458419c880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:08.370 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:08.366+0000 7f4588d2d640 1 -- 172.21.15.49:0/855562320 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f458419f7b0 con 0x7f45841022f0 2024-03-18T23:06:08.370 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:08.366+0000 7f45837fe640 1 --2- 172.21.15.49:0/855562320 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4584101930 0x7f458419c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:06:08.370 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:08.367+0000 7f4588d2d640 1 --2- 172.21.15.49:0/855562320 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f45841022f0 0x7f458419eea0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f457c00e940 tx=0x7f457c00ee00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:06:08.370 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:08.367+0000 7f45817fa640 1 -- 172.21.15.49:0/855562320 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f457c004d20 con 0x7f45841022f0 2024-03-18T23:06:08.370 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:08.367+0000 7f458a790640 1 -- 172.21.15.49:0/855562320 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f458419fa40 con 0x7f45841022f0 2024-03-18T23:06:08.370 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:08.367+0000 7f45817fa640 1 -- 172.21.15.49:0/855562320 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f457c005020 con 0x7f45841022f0 2024-03-18T23:06:08.370 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:08.367+0000 7f45817fa640 1 -- 172.21.15.49:0/855562320 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f457c0116d0 con 0x7f45841022f0 2024-03-18T23:06:08.370 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:08.367+0000 7f458a790640 1 -- 172.21.15.49:0/855562320 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f458419ff60 con 0x7f45841022f0 2024-03-18T23:06:08.371 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:08.368+0000 7f458a790640 1 -- 172.21.15.49:0/855562320 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4584107a60 con 0x7f45841022f0 2024-03-18T23:06:08.371 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:08.369+0000 7f45817fa640 1 -- 172.21.15.49:0/855562320 <== mon.2 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7f457c0054d0 con 0x7f45841022f0 2024-03-18T23:06:08.371 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:08.369+0000 7f45817fa640 1 --2- 172.21.15.49:0/855562320 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f456c0a2800 0x7f456c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:06:08.372 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:08.370+0000 7f4583fff640 1 --2- 172.21.15.49:0/855562320 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f456c0a2800 0x7f456c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:06:08.372 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:08.370+0000 7f45817fa640 1 -- 172.21.15.49:0/855562320 <== mon.2 v2:172.21.15.139:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f457c0cf020 con 0x7f45841022f0 2024-03-18T23:06:08.372 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:08.370+0000 7f4583fff640 1 --2- 172.21.15.49:0/855562320 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f456c0a2800 0x7f456c0a4cc0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f4584101790 tx=0x7f4574005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:06:08.374 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:08.373+0000 7f45817fa640 1 -- 172.21.15.49:0/855562320 <== mon.2 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f457c016030 con 0x7f45841022f0 2024-03-18T23:06:08.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:06:08 smithi139 ceph-mon[67282]: from='client.? 172.21.15.49:0/886008344' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-03-18T23:06:08.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:06:08 smithi139 ceph-mon[67282]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-18T23:06:08.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:06:08 smithi139 ceph-mon[67282]: from='client.? 172.21.15.49:0/1859282450' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-03-18T23:06:08.595 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:08.594+0000 7f458a790640 1 -- 172.21.15.49:0/855562320 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7f458402d1f0 con 0x7f45841022f0 2024-03-18T23:06:08.629 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:06:08 smithi049 ceph-mon[62991]: from='client.? 172.21.15.49:0/886008344' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-03-18T23:06:08.630 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:06:08 smithi049 ceph-mon[62991]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-18T23:06:08.630 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:06:08 smithi049 ceph-mon[62991]: from='client.? 172.21.15.49:0/1859282450' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-03-18T23:06:08.711 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:06:08 smithi175 ceph-mon[67352]: from='client.? 172.21.15.49:0/886008344' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-03-18T23:06:08.712 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:06:08 smithi175 ceph-mon[67352]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-18T23:06:08.712 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:06:08 smithi175 ceph-mon[67352]: from='client.? 172.21.15.49:0/1859282450' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-03-18T23:06:08.980 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:08.979+0000 7f45817fa640 1 -- 172.21.15.49:0/855562320 <== mon.2 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v20) ==== 121+0+0 (secure 0 0 0) 0x7f457c09bbc0 con 0x7f45841022f0 2024-03-18T23:06:08.983 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:08.982+0000 7f458a790640 1 -- 172.21.15.49:0/855562320 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f456c0a2800 msgr2=0x7f456c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:08.983 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:08.982+0000 7f458a790640 1 --2- 172.21.15.49:0/855562320 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f456c0a2800 0x7f456c0a4cc0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f4584101790 tx=0x7f4574005f90 comp rx=0 tx=0).stop 2024-03-18T23:06:08.983 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:08.982+0000 7f458a790640 1 -- 172.21.15.49:0/855562320 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f45841022f0 msgr2=0x7f458419eea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:08.983 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:08.982+0000 7f458a790640 1 --2- 172.21.15.49:0/855562320 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f45841022f0 0x7f458419eea0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f457c00e940 tx=0x7f457c00ee00 comp rx=0 tx=0).stop 2024-03-18T23:06:08.983 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:08.982+0000 7f458a790640 1 -- 172.21.15.49:0/855562320 shutdown_connections 2024-03-18T23:06:08.983 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:08.982+0000 7f458a790640 1 --2- 172.21.15.49:0/855562320 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f456c0a2800 0x7f456c0a4cc0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:08.983 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:08.982+0000 7f458a790640 1 --2- 172.21.15.49:0/855562320 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f45841022f0 0x7f458419eea0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:08.984 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:08.982+0000 7f458a790640 1 --2- 172.21.15.49:0/855562320 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4584101930 0x7f458419c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:08.984 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:08.982+0000 7f458a790640 1 --2- 172.21.15.49:0/855562320 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4584100740 0x7f458406a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:08.984 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:08.982+0000 7f458a790640 1 -- 172.21.15.49:0/855562320 >> 172.21.15.49:0/855562320 conn(0x7f45840fbcd0 msgr2=0x7f4584104c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:06:08.984 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:08.982+0000 7f458a790640 1 -- 172.21.15.49:0/855562320 shutdown_connections 2024-03-18T23:06:08.984 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:08.982+0000 7f458a790640 1 -- 172.21.15.49:0/855562320 wait complete. 2024-03-18T23:06:08.999 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2024-03-18T23:06:09.114 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:09.110+0000 7fc1c3752640 1 -- 172.21.15.49:0/2850586257 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc1bc100720 msgr2=0x7fc1bc100b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:09.114 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:09.110+0000 7fc1c3752640 1 --2- 172.21.15.49:0/2850586257 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc1bc100720 0x7fc1bc100b40 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fc1b4009900 tx=0x7fc1b402bc30 comp rx=0 tx=0).stop 2024-03-18T23:06:09.114 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:09.111+0000 7fc1c3752640 1 -- 172.21.15.49:0/2850586257 shutdown_connections 2024-03-18T23:06:09.114 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:09.111+0000 7fc1c3752640 1 --2- 172.21.15.49:0/2850586257 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc1bc1022d0 0x7fc1bc108b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:09.114 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:09.111+0000 7fc1c3752640 1 --2- 172.21.15.49:0/2850586257 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc1bc101910 0x7fc1bc101d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:09.114 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:09.111+0000 7fc1c3752640 1 --2- 172.21.15.49:0/2850586257 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc1bc100720 0x7fc1bc100b40 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:09.114 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:09.111+0000 7fc1c3752640 1 -- 172.21.15.49:0/2850586257 >> 172.21.15.49:0/2850586257 conn(0x7fc1bc0fbcb0 msgr2=0x7fc1bc0fe0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:06:09.114 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:09.111+0000 7fc1c3752640 1 -- 172.21.15.49:0/2850586257 shutdown_connections 2024-03-18T23:06:09.114 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:09.111+0000 7fc1c3752640 1 -- 172.21.15.49:0/2850586257 wait complete. 2024-03-18T23:06:09.115 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:09.111+0000 7fc1c3752640 1 Processor -- start 2024-03-18T23:06:09.115 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:09.111+0000 7fc1c3752640 1 -- start start 2024-03-18T23:06:09.115 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:09.111+0000 7fc1c3752640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc1bc100720 0x7fc1bc197bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:06:09.115 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:09.111+0000 7fc1c3752640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc1bc101910 0x7fc1bc1980f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:06:09.115 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:09.111+0000 7fc1c3752640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc1bc1022d0 0x7fc1bc19f160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:06:09.115 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:09.111+0000 7fc1c3752640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fc1bc10c390 con 0x7fc1bc1022d0 2024-03-18T23:06:09.115 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:09.111+0000 7fc1c3752640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fc1bc10c210 con 0x7fc1bc100720 2024-03-18T23:06:09.115 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:09.111+0000 7fc1c3752640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc1bc10c510 con 0x7fc1bc101910 2024-03-18T23:06:09.115 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:09.112+0000 7fc1c1cef640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc1bc1022d0 0x7fc1bc19f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:06:09.115 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:09.112+0000 7fc1c14ee640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc1bc100720 0x7fc1bc197bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:06:09.115 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:09.112+0000 7fc1c1cef640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc1bc1022d0 0x7fc1bc19f160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:54162/0 (socket says 172.21.15.49:54162) 2024-03-18T23:06:09.115 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:09.112+0000 7fc1c1cef640 1 -- 172.21.15.49:0/2357840250 learned_addr learned my addr 172.21.15.49:0/2357840250 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:06:09.115 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:09.112+0000 7fc1c0ced640 1 --2- 172.21.15.49:0/2357840250 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc1bc101910 0x7fc1bc1980f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:06:09.115 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:09.112+0000 7fc1c1cef640 1 -- 172.21.15.49:0/2357840250 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc1bc101910 msgr2=0x7fc1bc1980f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:09.116 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:09.112+0000 7fc1c1cef640 1 --2- 172.21.15.49:0/2357840250 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc1bc101910 0x7fc1bc1980f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:09.116 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:09.112+0000 7fc1c1cef640 1 -- 172.21.15.49:0/2357840250 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc1bc100720 msgr2=0x7fc1bc197bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:09.116 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:09.112+0000 7fc1c1cef640 1 --2- 172.21.15.49:0/2357840250 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc1bc100720 0x7fc1bc197bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:09.116 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:09.112+0000 7fc1c1cef640 1 -- 172.21.15.49:0/2357840250 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1bc19f7d0 con 0x7fc1bc1022d0 2024-03-18T23:06:09.116 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:09.112+0000 7fc1c1cef640 1 --2- 172.21.15.49:0/2357840250 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc1bc1022d0 0x7fc1bc19f160 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fc1b800eb10 tx=0x7fc1b800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:06:09.116 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:09.112+0000 7fc1b27fc640 1 -- 172.21.15.49:0/2357840250 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1b8004b20 con 0x7fc1bc1022d0 2024-03-18T23:06:09.116 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:09.112+0000 7fc1b27fc640 1 -- 172.21.15.49:0/2357840250 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc1b8004cc0 con 0x7fc1bc1022d0 2024-03-18T23:06:09.116 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:09.112+0000 7fc1c3752640 1 -- 172.21.15.49:0/2357840250 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1bc19fac0 con 0x7fc1bc1022d0 2024-03-18T23:06:09.116 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:09.112+0000 7fc1b27fc640 1 -- 172.21.15.49:0/2357840250 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1b8010760 con 0x7fc1bc1022d0 2024-03-18T23:06:09.116 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:09.112+0000 7fc1c3752640 1 -- 172.21.15.49:0/2357840250 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1bc1a0010 con 0x7fc1bc1022d0 2024-03-18T23:06:09.117 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:09.113+0000 7fc1c3752640 1 -- 172.21.15.49:0/2357840250 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1bc1a0240 con 0x7fc1bc1022d0 2024-03-18T23:06:09.117 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:09.114+0000 7fc1b27fc640 1 -- 172.21.15.49:0/2357840250 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7fc1b8005020 con 0x7fc1bc1022d0 2024-03-18T23:06:09.117 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:09.115+0000 7fc1b27fc640 1 --2- 172.21.15.49:0/2357840250 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fc1a80a2880 0x7fc1a80a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:06:09.117 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:09.115+0000 7fc1c14ee640 1 --2- 172.21.15.49:0/2357840250 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fc1a80a2880 0x7fc1a80a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:06:09.118 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:09.115+0000 7fc1b27fc640 1 -- 172.21.15.49:0/2357840250 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7fc1b80cedf0 con 0x7fc1bc1022d0 2024-03-18T23:06:09.118 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:09.115+0000 7fc1c14ee640 1 --2- 172.21.15.49:0/2357840250 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fc1a80a2880 0x7fc1a80a4d40 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fc1b4000c00 tx=0x7fc1b4005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:06:09.120 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:09.119+0000 7fc1b27fc640 1 -- 172.21.15.49:0/2357840250 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fc1b809bfe0 con 0x7fc1bc1022d0 2024-03-18T23:06:09.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:09.341+0000 7fc1c3752640 1 -- 172.21.15.49:0/2357840250 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7fc1bc1a0410 con 0x7fc1bc1022d0 2024-03-18T23:06:09.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:06:09 smithi049 ceph-mon[62991]: pgmap v290: 97 pgs: 53 active+clean, 2 creating+activating, 1 active, 14 creating+peering, 27 unknown; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s wr, 6 op/s 2024-03-18T23:06:09.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:06:09 smithi049 ceph-mon[62991]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-03-18T23:06:09.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:06:09 smithi049 ceph-mon[62991]: from='client.? 172.21.15.49:0/1859282450' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-03-18T23:06:09.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:06:09 smithi049 ceph-mon[62991]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-03-18T23:06:09.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:06:09 smithi049 ceph-mon[62991]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-18T23:06:09.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:06:09 smithi049 ceph-mon[62991]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-03-18T23:06:09.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:06:09 smithi049 ceph-mon[62991]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-03-18T23:06:09.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:06:09 smithi049 ceph-mon[62991]: from='client.? 172.21.15.49:0/855562320' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-03-18T23:06:09.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:06:09 smithi049 ceph-mon[62991]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-03-18T23:06:09.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:06:09 smithi049 ceph-mon[62991]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-03-18T23:06:09.381 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:06:09 smithi049 ceph-mon[62991]: mds.? [v2:172.21.15.175:6838/3640882084,v1:172.21.15.175:6839/3640882084] up:active 2024-03-18T23:06:09.381 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:06:09 smithi049 ceph-mon[62991]: daemon mds.f assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-03-18T23:06:09.381 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:06:09 smithi049 ceph-mon[62991]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 2 up:standby-replay 8 up:standby 2024-03-18T23:06:09.381 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:06:09 smithi049 ceph-mon[62991]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:creating} 2 up:standby-replay 7 up:standby 2024-03-18T23:06:09.381 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:06:09 smithi049 ceph-mon[62991]: daemon mds.f is now active in filesystem cephfs as rank 2 2024-03-18T23:06:09.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:06:09 smithi139 ceph-mon[67282]: pgmap v290: 97 pgs: 53 active+clean, 2 creating+activating, 1 active, 14 creating+peering, 27 unknown; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s wr, 6 op/s 2024-03-18T23:06:09.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:06:09 smithi139 ceph-mon[67282]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-03-18T23:06:09.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:06:09 smithi139 ceph-mon[67282]: from='client.? 172.21.15.49:0/1859282450' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-03-18T23:06:09.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:06:09 smithi139 ceph-mon[67282]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-03-18T23:06:09.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:06:09 smithi139 ceph-mon[67282]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-18T23:06:09.566 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:06:09 smithi139 ceph-mon[67282]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-03-18T23:06:09.566 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:06:09 smithi139 ceph-mon[67282]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-03-18T23:06:09.566 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:06:09 smithi139 ceph-mon[67282]: from='client.? 172.21.15.49:0/855562320' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-03-18T23:06:09.566 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:06:09 smithi139 ceph-mon[67282]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-03-18T23:06:09.566 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:06:09 smithi139 ceph-mon[67282]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-03-18T23:06:09.566 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:06:09 smithi139 ceph-mon[67282]: mds.? [v2:172.21.15.175:6838/3640882084,v1:172.21.15.175:6839/3640882084] up:active 2024-03-18T23:06:09.566 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:06:09 smithi139 ceph-mon[67282]: daemon mds.f assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-03-18T23:06:09.566 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:06:09 smithi139 ceph-mon[67282]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 2 up:standby-replay 8 up:standby 2024-03-18T23:06:09.566 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:06:09 smithi139 ceph-mon[67282]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:creating} 2 up:standby-replay 7 up:standby 2024-03-18T23:06:09.566 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:06:09 smithi139 ceph-mon[67282]: daemon mds.f is now active in filesystem cephfs as rank 2 2024-03-18T23:06:09.613 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:06:09 smithi175 ceph-mon[67352]: pgmap v290: 97 pgs: 53 active+clean, 2 creating+activating, 1 active, 14 creating+peering, 27 unknown; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s wr, 6 op/s 2024-03-18T23:06:09.613 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:06:09 smithi175 ceph-mon[67352]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-03-18T23:06:09.613 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:06:09 smithi175 ceph-mon[67352]: from='client.? 172.21.15.49:0/1859282450' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-03-18T23:06:09.613 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:06:09 smithi175 ceph-mon[67352]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-03-18T23:06:09.613 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:06:09 smithi175 ceph-mon[67352]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-18T23:06:09.613 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:06:09 smithi175 ceph-mon[67352]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-03-18T23:06:09.614 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:06:09 smithi175 ceph-mon[67352]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-03-18T23:06:09.614 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:06:09 smithi175 ceph-mon[67352]: from='client.? 172.21.15.49:0/855562320' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-03-18T23:06:09.614 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:06:09 smithi175 ceph-mon[67352]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-03-18T23:06:09.614 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:06:09 smithi175 ceph-mon[67352]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-03-18T23:06:09.614 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:06:09 smithi175 ceph-mon[67352]: mds.? [v2:172.21.15.175:6838/3640882084,v1:172.21.15.175:6839/3640882084] up:active 2024-03-18T23:06:09.614 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:06:09 smithi175 ceph-mon[67352]: daemon mds.f assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-03-18T23:06:09.614 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:06:09 smithi175 ceph-mon[67352]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 2 up:standby-replay 8 up:standby 2024-03-18T23:06:09.614 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:06:09 smithi175 ceph-mon[67352]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:creating} 2 up:standby-replay 7 up:standby 2024-03-18T23:06:09.614 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:06:09 smithi175 ceph-mon[67352]: daemon mds.f is now active in filesystem cephfs as rank 2 2024-03-18T23:06:09.984 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:09.984+0000 7fc1b27fc640 1 -- 172.21.15.49:0/2357840250 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v22) ==== 115+0+0 (secure 0 0 0) 0x7fc1b809b8e0 con 0x7fc1bc1022d0 2024-03-18T23:06:09.987 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:09.986+0000 7fc1c3752640 1 -- 172.21.15.49:0/2357840250 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fc1a80a2880 msgr2=0x7fc1a80a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:09.987 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:09.986+0000 7fc1c3752640 1 --2- 172.21.15.49:0/2357840250 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fc1a80a2880 0x7fc1a80a4d40 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fc1b4000c00 tx=0x7fc1b4005fd0 comp rx=0 tx=0).stop 2024-03-18T23:06:09.987 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:09.986+0000 7fc1c3752640 1 -- 172.21.15.49:0/2357840250 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc1bc1022d0 msgr2=0x7fc1bc19f160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:09.987 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:09.986+0000 7fc1c3752640 1 --2- 172.21.15.49:0/2357840250 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc1bc1022d0 0x7fc1bc19f160 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fc1b800eb10 tx=0x7fc1b800c2d0 comp rx=0 tx=0).stop 2024-03-18T23:06:09.987 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:09.986+0000 7fc1c3752640 1 -- 172.21.15.49:0/2357840250 shutdown_connections 2024-03-18T23:06:09.987 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:09.986+0000 7fc1c3752640 1 --2- 172.21.15.49:0/2357840250 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fc1a80a2880 0x7fc1a80a4d40 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:09.987 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:09.986+0000 7fc1c3752640 1 --2- 172.21.15.49:0/2357840250 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc1bc1022d0 0x7fc1bc19f160 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:09.988 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:09.986+0000 7fc1c3752640 1 --2- 172.21.15.49:0/2357840250 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc1bc101910 0x7fc1bc1980f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:09.988 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:09.986+0000 7fc1c3752640 1 --2- 172.21.15.49:0/2357840250 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc1bc100720 0x7fc1bc197bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:09.988 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:09.986+0000 7fc1c3752640 1 -- 172.21.15.49:0/2357840250 >> 172.21.15.49:0/2357840250 conn(0x7fc1bc0fbcb0 msgr2=0x7fc1bc0fd7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:06:09.988 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:09.986+0000 7fc1c3752640 1 -- 172.21.15.49:0/2357840250 shutdown_connections 2024-03-18T23:06:09.988 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:09.986+0000 7fc1c3752640 1 -- 172.21.15.49:0/2357840250 wait complete. 2024-03-18T23:06:10.004 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--namespace-isolated'} subvols for filesystem 'cephfs' 2024-03-18T23:06:10.004 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create cephfs qa 2024-03-18T23:06:10.122 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:10.120+0000 7fbe12c44640 1 -- 172.21.15.49:0/3166256133 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbe0c1033f0 msgr2=0x7fbe0c109c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:10.122 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:10.120+0000 7fbe12c44640 1 --2- 172.21.15.49:0/3166256133 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbe0c1033f0 0x7fbe0c109c80 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fbe04009ca0 tx=0x7fbe0402c0f0 comp rx=0 tx=0).stop 2024-03-18T23:06:10.122 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:10.120+0000 7fbe12c44640 1 -- 172.21.15.49:0/3166256133 shutdown_connections 2024-03-18T23:06:10.122 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:10.120+0000 7fbe12c44640 1 --2- 172.21.15.49:0/3166256133 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbe0c1033f0 0x7fbe0c109c80 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:10.122 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:10.120+0000 7fbe12c44640 1 --2- 172.21.15.49:0/3166256133 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbe0c102a30 0x7fbe0c102eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:10.123 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:10.120+0000 7fbe12c44640 1 --2- 172.21.15.49:0/3166256133 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbe0c101840 0x7fbe0c101c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:10.123 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:10.120+0000 7fbe12c44640 1 -- 172.21.15.49:0/3166256133 >> 172.21.15.49:0/3166256133 conn(0x7fbe0c0fcdb0 msgr2=0x7fbe0c0ff1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:06:10.123 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:10.120+0000 7fbe12c44640 1 -- 172.21.15.49:0/3166256133 shutdown_connections 2024-03-18T23:06:10.123 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:10.120+0000 7fbe12c44640 1 -- 172.21.15.49:0/3166256133 wait complete. 2024-03-18T23:06:10.123 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:10.121+0000 7fbe12c44640 1 Processor -- start 2024-03-18T23:06:10.123 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:10.121+0000 7fbe12c44640 1 -- start start 2024-03-18T23:06:10.123 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:10.121+0000 7fbe12c44640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbe0c101840 0x7fbe0c197ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:06:10.123 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:10.121+0000 7fbe12c44640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbe0c102a30 0x7fbe0c197ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:06:10.123 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:10.121+0000 7fbe12c44640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbe0c1033f0 0x7fbe0c19f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:06:10.123 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:10.121+0000 7fbe12c44640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fbe0c10d4a0 con 0x7fbe0c101840 2024-03-18T23:06:10.123 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:10.121+0000 7fbe12c44640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fbe0c10d320 con 0x7fbe0c102a30 2024-03-18T23:06:10.123 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:10.121+0000 7fbe12c44640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fbe0c10d620 con 0x7fbe0c1033f0 2024-03-18T23:06:10.123 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:10.121+0000 7fbe109e0640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbe0c101840 0x7fbe0c197ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:06:10.123 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:10.121+0000 7fbe109e0640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbe0c101840 0x7fbe0c197ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:54184/0 (socket says 172.21.15.49:54184) 2024-03-18T23:06:10.124 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:10.121+0000 7fbe109e0640 1 -- 172.21.15.49:0/4281723865 learned_addr learned my addr 172.21.15.49:0/4281723865 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:06:10.124 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:10.121+0000 7fbe111e1640 1 --2- 172.21.15.49:0/4281723865 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbe0c1033f0 0x7fbe0c19f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:06:10.124 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:10.121+0000 7fbe0bfff640 1 --2- 172.21.15.49:0/4281723865 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbe0c102a30 0x7fbe0c197ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:06:10.124 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:10.121+0000 7fbe109e0640 1 -- 172.21.15.49:0/4281723865 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbe0c1033f0 msgr2=0x7fbe0c19f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:10.124 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:10.121+0000 7fbe109e0640 1 --2- 172.21.15.49:0/4281723865 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbe0c1033f0 0x7fbe0c19f060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:10.124 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:10.121+0000 7fbe109e0640 1 -- 172.21.15.49:0/4281723865 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbe0c102a30 msgr2=0x7fbe0c197ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:10.124 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:10.121+0000 7fbe109e0640 1 --2- 172.21.15.49:0/4281723865 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbe0c102a30 0x7fbe0c197ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:10.124 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:10.121+0000 7fbe109e0640 1 -- 172.21.15.49:0/4281723865 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe0c19f6d0 con 0x7fbe0c101840 2024-03-18T23:06:10.124 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:10.121+0000 7fbe109e0640 1 --2- 172.21.15.49:0/4281723865 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbe0c101840 0x7fbe0c197ab0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fbe0000e9c0 tx=0x7fbe0000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:06:10.124 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:10.122+0000 7fbe09ffb640 1 -- 172.21.15.49:0/4281723865 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe0000cd80 con 0x7fbe0c101840 2024-03-18T23:06:10.124 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:10.122+0000 7fbe09ffb640 1 -- 172.21.15.49:0/4281723865 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbe000035f0 con 0x7fbe0c101840 2024-03-18T23:06:10.124 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:10.122+0000 7fbe12c44640 1 -- 172.21.15.49:0/4281723865 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe0c19f9c0 con 0x7fbe0c101840 2024-03-18T23:06:10.124 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:10.122+0000 7fbe09ffb640 1 -- 172.21.15.49:0/4281723865 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe00010690 con 0x7fbe0c101840 2024-03-18T23:06:10.124 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:10.122+0000 7fbe12c44640 1 -- 172.21.15.49:0/4281723865 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe0c19ff10 con 0x7fbe0c101840 2024-03-18T23:06:10.125 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:10.123+0000 7fbe12c44640 1 -- 172.21.15.49:0/4281723865 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe0c068a60 con 0x7fbe0c101840 2024-03-18T23:06:10.125 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:10.124+0000 7fbe09ffb640 1 -- 172.21.15.49:0/4281723865 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7fbe000037a0 con 0x7fbe0c101840 2024-03-18T23:06:10.125 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:10.125+0000 7fbe09ffb640 1 --2- 172.21.15.49:0/4281723865 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fbdfc0a2800 0x7fbdfc0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:06:10.126 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:10.125+0000 7fbe0bfff640 1 --2- 172.21.15.49:0/4281723865 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fbdfc0a2800 0x7fbdfc0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:06:10.126 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:10.125+0000 7fbe09ffb640 1 -- 172.21.15.49:0/4281723865 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7fbe000cecc0 con 0x7fbe0c101840 2024-03-18T23:06:10.126 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:10.125+0000 7fbe0bfff640 1 --2- 172.21.15.49:0/4281723865 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fbdfc0a2800 0x7fbdfc0a4cc0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fbe0c198fc0 tx=0x7fbdf800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:06:10.129 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:10.129+0000 7fbe09ffb640 1 -- 172.21.15.49:0/4281723865 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fbe0009beb0 con 0x7fbe0c101840 2024-03-18T23:06:10.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:10.341+0000 7fbe12c44640 1 -- 172.21.15.49:0/4281723865 --> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fbe0c061c60 con 0x7fbdfc0a2800 2024-03-18T23:06:10.347 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:06:10 smithi049 ceph-mon[62991]: from='client.? 172.21.15.49:0/2357840250' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-03-18T23:06:10.347 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:06:10 smithi049 ceph-mon[62991]: from='client.? 172.21.15.49:0/2357840250' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-03-18T23:06:10.347 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:06:10 smithi049 ceph-mon[62991]: mds.? [v2:172.21.15.175:6834/755474931,v1:172.21.15.175:6835/755474931] up:active 2024-03-18T23:06:10.347 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:06:10 smithi049 ceph-mon[62991]: daemon mds.e assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-03-18T23:06:10.347 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:06:10 smithi049 ceph-mon[62991]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 2 up:standby-replay 7 up:standby 2024-03-18T23:06:10.347 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:06:10 smithi049 ceph-mon[62991]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=f=up:active,3=e=up:creating} 2 up:standby-replay 6 up:standby 2024-03-18T23:06:10.347 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:06:10 smithi049 ceph-mon[62991]: daemon mds.e is now active in filesystem cephfs as rank 3 2024-03-18T23:06:10.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:06:10 smithi139 ceph-mon[67282]: from='client.? 172.21.15.49:0/2357840250' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-03-18T23:06:10.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:06:10 smithi139 ceph-mon[67282]: from='client.? 172.21.15.49:0/2357840250' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-03-18T23:06:10.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:06:10 smithi139 ceph-mon[67282]: mds.? [v2:172.21.15.175:6834/755474931,v1:172.21.15.175:6835/755474931] up:active 2024-03-18T23:06:10.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:06:10 smithi139 ceph-mon[67282]: daemon mds.e assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-03-18T23:06:10.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:06:10 smithi139 ceph-mon[67282]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 2 up:standby-replay 7 up:standby 2024-03-18T23:06:10.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:06:10 smithi139 ceph-mon[67282]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=f=up:active,3=e=up:creating} 2 up:standby-replay 6 up:standby 2024-03-18T23:06:10.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:06:10 smithi139 ceph-mon[67282]: daemon mds.e is now active in filesystem cephfs as rank 3 2024-03-18T23:06:10.630 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 18 23:06:10 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-x[63224]: 2024-03-18T23:06:10.346+0000 7f1853516700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-18T23:06:10.630 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 18 23:06:10 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-x[63224]: 2024-03-18T23:06:10.346+0000 7f1853516700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-18T23:06:10.630 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 18 23:06:10 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-x[63224]: 2024-03-18T23:06:10.346+0000 7f1853516700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-18T23:06:10.630 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 18 23:06:10 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-x[63224]: 2024-03-18T23:06:10.346+0000 7f1853516700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-18T23:06:10.630 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 18 23:06:10 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-x[63224]: 2024-03-18T23:06:10.346+0000 7f1853516700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-18T23:06:10.711 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:06:10 smithi175 ceph-mon[67352]: from='client.? 172.21.15.49:0/2357840250' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-03-18T23:06:10.712 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:06:10 smithi175 ceph-mon[67352]: from='client.? 172.21.15.49:0/2357840250' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-03-18T23:06:10.712 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:06:10 smithi175 ceph-mon[67352]: mds.? [v2:172.21.15.175:6834/755474931,v1:172.21.15.175:6835/755474931] up:active 2024-03-18T23:06:10.712 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:06:10 smithi175 ceph-mon[67352]: daemon mds.e assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-03-18T23:06:10.712 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:06:10 smithi175 ceph-mon[67352]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 2 up:standby-replay 7 up:standby 2024-03-18T23:06:10.712 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:06:10 smithi175 ceph-mon[67352]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=f=up:active,3=e=up:creating} 2 up:standby-replay 6 up:standby 2024-03-18T23:06:10.712 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:06:10 smithi175 ceph-mon[67352]: daemon mds.e is now active in filesystem cephfs as rank 3 2024-03-18T23:06:11.012 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:11.012+0000 7fbe09ffb640 1 -- 172.21.15.49:0/4281723865 <== mgr.14150 v2:172.21.15.49:6800/2083805211 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fbe0c061c60 con 0x7fbdfc0a2800 2024-03-18T23:06:11.015 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:11.015+0000 7fbe12c44640 1 -- 172.21.15.49:0/4281723865 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fbdfc0a2800 msgr2=0x7fbdfc0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:11.015 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:11.015+0000 7fbe12c44640 1 --2- 172.21.15.49:0/4281723865 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fbdfc0a2800 0x7fbdfc0a4cc0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fbe0c198fc0 tx=0x7fbdf800b390 comp rx=0 tx=0).stop 2024-03-18T23:06:11.015 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:11.015+0000 7fbe12c44640 1 -- 172.21.15.49:0/4281723865 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbe0c101840 msgr2=0x7fbe0c197ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:11.015 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:11.015+0000 7fbe12c44640 1 --2- 172.21.15.49:0/4281723865 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbe0c101840 0x7fbe0c197ab0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fbe0000e9c0 tx=0x7fbe0000ee80 comp rx=0 tx=0).stop 2024-03-18T23:06:11.015 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:11.015+0000 7fbe12c44640 1 -- 172.21.15.49:0/4281723865 shutdown_connections 2024-03-18T23:06:11.015 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:11.015+0000 7fbe12c44640 1 --2- 172.21.15.49:0/4281723865 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fbdfc0a2800 0x7fbdfc0a4cc0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:11.015 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:11.015+0000 7fbe12c44640 1 --2- 172.21.15.49:0/4281723865 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbe0c1033f0 0x7fbe0c19f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:11.015 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:11.015+0000 7fbe12c44640 1 --2- 172.21.15.49:0/4281723865 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbe0c102a30 0x7fbe0c197ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:11.016 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:11.015+0000 7fbe12c44640 1 --2- 172.21.15.49:0/4281723865 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbe0c101840 0x7fbe0c197ab0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:11.016 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:11.015+0000 7fbe12c44640 1 -- 172.21.15.49:0/4281723865 >> 172.21.15.49:0/4281723865 conn(0x7fbe0c0fcdb0 msgr2=0x7fbe0c105d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:06:11.016 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:11.015+0000 7fbe12c44640 1 -- 172.21.15.49:0/4281723865 shutdown_connections 2024-03-18T23:06:11.016 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:11.015+0000 7fbe12c44640 1 -- 172.21.15.49:0/4281723865 wait complete. 2024-03-18T23:06:11.027 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_0 --group_name qa 2024-03-18T23:06:11.144 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:11.142+0000 7f899e928640 1 -- 172.21.15.49:0/4017749240 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8998101930 msgr2=0x7f8998101db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:11.144 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:11.142+0000 7f899e928640 1 --2- 172.21.15.49:0/4017749240 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8998101930 0x7f8998101db0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f898c009900 tx=0x7f898c02bc30 comp rx=0 tx=0).stop 2024-03-18T23:06:11.144 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:11.142+0000 7f899e928640 1 -- 172.21.15.49:0/4017749240 shutdown_connections 2024-03-18T23:06:11.144 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:11.142+0000 7f899e928640 1 --2- 172.21.15.49:0/4017749240 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f89981022f0 0x7f8998108b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:11.144 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:11.142+0000 7f899e928640 1 --2- 172.21.15.49:0/4017749240 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8998101930 0x7f8998101db0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:11.145 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:11.142+0000 7f899e928640 1 --2- 172.21.15.49:0/4017749240 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8998100740 0x7f8998100b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:11.145 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:11.142+0000 7f899e928640 1 -- 172.21.15.49:0/4017749240 >> 172.21.15.49:0/4017749240 conn(0x7f89980fbcd0 msgr2=0x7f89980fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:06:11.145 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:11.142+0000 7f899e928640 1 -- 172.21.15.49:0/4017749240 shutdown_connections 2024-03-18T23:06:11.145 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:11.142+0000 7f899e928640 1 -- 172.21.15.49:0/4017749240 wait complete. 2024-03-18T23:06:11.145 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:11.143+0000 7f899e928640 1 Processor -- start 2024-03-18T23:06:11.145 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:11.143+0000 7f899e928640 1 -- start start 2024-03-18T23:06:11.145 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:11.143+0000 7f899e928640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8998100740 0x7f8998196a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:06:11.145 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:11.143+0000 7f899e928640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8998101930 0x7f8998196fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:06:11.145 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:11.143+0000 7f899e928640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f89981022f0 0x7f899819e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:06:11.145 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:11.143+0000 7f899e928640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f899810c3c0 con 0x7f8998101930 2024-03-18T23:06:11.145 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:11.143+0000 7f899e928640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f899810c240 con 0x7f89981022f0 2024-03-18T23:06:11.145 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:11.143+0000 7f899e928640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f899810c540 con 0x7f8998100740 2024-03-18T23:06:11.145 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:11.143+0000 7f89977fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8998101930 0x7f8998196fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:06:11.146 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:11.143+0000 7f8997fff640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8998100740 0x7f8998196a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:06:11.146 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:11.143+0000 7f89977fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8998101930 0x7f8998196fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:54232/0 (socket says 172.21.15.49:54232) 2024-03-18T23:06:11.146 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:11.143+0000 7f89977fe640 1 -- 172.21.15.49:0/881703897 learned_addr learned my addr 172.21.15.49:0/881703897 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:06:11.146 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:11.143+0000 7f899cec5640 1 --2- 172.21.15.49:0/881703897 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f89981022f0 0x7f899819e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:06:11.146 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:11.143+0000 7f89977fe640 1 -- 172.21.15.49:0/881703897 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8998100740 msgr2=0x7f8998196a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:11.146 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:11.143+0000 7f89977fe640 1 --2- 172.21.15.49:0/881703897 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8998100740 0x7f8998196a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:11.146 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:11.143+0000 7f89977fe640 1 -- 172.21.15.49:0/881703897 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f89981022f0 msgr2=0x7f899819e010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:11.146 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:11.143+0000 7f89977fe640 1 --2- 172.21.15.49:0/881703897 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f89981022f0 0x7f899819e010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:11.146 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:11.143+0000 7f89977fe640 1 -- 172.21.15.49:0/881703897 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f899819e680 con 0x7f8998101930 2024-03-18T23:06:11.146 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:11.143+0000 7f8997fff640 1 --2- 172.21.15.49:0/881703897 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8998100740 0x7f8998196a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:06:11.146 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:11.144+0000 7f89977fe640 1 --2- 172.21.15.49:0/881703897 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8998101930 0x7f8998196fa0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f898c009230 tx=0x7f898c02c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:06:11.146 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:11.144+0000 7f89957fa640 1 -- 172.21.15.49:0/881703897 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f898c041070 con 0x7f8998101930 2024-03-18T23:06:11.146 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:11.144+0000 7f89957fa640 1 -- 172.21.15.49:0/881703897 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f898c004450 con 0x7f8998101930 2024-03-18T23:06:11.146 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:11.144+0000 7f899e928640 1 -- 172.21.15.49:0/881703897 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f899819e910 con 0x7f8998101930 2024-03-18T23:06:11.147 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:11.144+0000 7f89957fa640 1 -- 172.21.15.49:0/881703897 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f898c003e20 con 0x7f8998101930 2024-03-18T23:06:11.147 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:11.144+0000 7f899e928640 1 -- 172.21.15.49:0/881703897 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f899819ee00 con 0x7f8998101930 2024-03-18T23:06:11.147 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:11.145+0000 7f899e928640 1 -- 172.21.15.49:0/881703897 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8998107b30 con 0x7f8998101930 2024-03-18T23:06:11.148 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:11.146+0000 7f89957fa640 1 -- 172.21.15.49:0/881703897 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7f898c044070 con 0x7f8998101930 2024-03-18T23:06:11.148 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:11.147+0000 7f89957fa640 1 --2- 172.21.15.49:0/881703897 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f89800a2800 0x7f89800a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:06:11.148 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:11.147+0000 7f8997fff640 1 --2- 172.21.15.49:0/881703897 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f89800a2800 0x7f89800a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:06:11.148 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:11.147+0000 7f89957fa640 1 -- 172.21.15.49:0/881703897 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f898c0f04a0 con 0x7f8998101930 2024-03-18T23:06:11.148 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:11.147+0000 7f8997fff640 1 --2- 172.21.15.49:0/881703897 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f89800a2800 0x7f89800a4cc0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f8998101790 tx=0x7f8988005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:06:11.151 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:11.151+0000 7f89957fa640 1 -- 172.21.15.49:0/881703897 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f898c0bd040 con 0x7f8998101930 2024-03-18T23:06:11.366 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:11.366+0000 7f899e928640 1 -- 172.21.15.49:0/881703897 --> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f8998061cf0 con 0x7f89800a2800 2024-03-18T23:06:11.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:06:11 smithi049 ceph-mon[62991]: pgmap v291: 97 pgs: 94 active+clean, 2 creating+activating, 1 active; 581 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s rd, 4.3 KiB/s wr, 11 op/s 2024-03-18T23:06:11.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:06:11 smithi049 ceph-mon[62991]: from='client.14727 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T23:06:11.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:06:11 smithi049 ceph-mon[62991]: mds.? [v2:172.21.15.139:6834/1663212937,v1:172.21.15.139:6835/1663212937] up:active 2024-03-18T23:06:11.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:06:11 smithi049 ceph-mon[62991]: daemon mds.l assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-03-18T23:06:11.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:06:11 smithi049 ceph-mon[62991]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-03-18T23:06:11.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:06:11 smithi049 ceph-mon[62991]: Cluster is now healthy 2024-03-18T23:06:11.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:06:11 smithi049 ceph-mon[62991]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=f=up:active,3=e=up:active} 2 up:standby-replay 6 up:standby 2024-03-18T23:06:11.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:06:11 smithi049 ceph-mon[62991]: fsmap cephfs:5 2 up:standby-replay 1 up:creating 5 up:standby 4 up:active 2024-03-18T23:06:11.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:06:11 smithi049 ceph-mon[62991]: daemon mds.l is now active in filesystem cephfs as rank 4 2024-03-18T23:06:11.387 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:11.387+0000 7f89957fa640 1 -- 172.21.15.49:0/881703897 <== mgr.14150 v2:172.21.15.49:6800/2083805211 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f8998061cf0 con 0x7f89800a2800 2024-03-18T23:06:11.390 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:11.389+0000 7f899e928640 1 -- 172.21.15.49:0/881703897 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f89800a2800 msgr2=0x7f89800a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:11.390 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:11.389+0000 7f899e928640 1 --2- 172.21.15.49:0/881703897 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f89800a2800 0x7f89800a4cc0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f8998101790 tx=0x7f8988005f90 comp rx=0 tx=0).stop 2024-03-18T23:06:11.390 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:11.389+0000 7f899e928640 1 -- 172.21.15.49:0/881703897 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8998101930 msgr2=0x7f8998196fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:11.390 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:11.389+0000 7f899e928640 1 --2- 172.21.15.49:0/881703897 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8998101930 0x7f8998196fa0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f898c009230 tx=0x7f898c02c650 comp rx=0 tx=0).stop 2024-03-18T23:06:11.390 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:11.389+0000 7f899e928640 1 -- 172.21.15.49:0/881703897 shutdown_connections 2024-03-18T23:06:11.390 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:11.389+0000 7f899e928640 1 --2- 172.21.15.49:0/881703897 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f89800a2800 0x7f89800a4cc0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:11.390 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:11.389+0000 7f899e928640 1 --2- 172.21.15.49:0/881703897 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f89981022f0 0x7f899819e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:11.390 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:11.389+0000 7f899e928640 1 --2- 172.21.15.49:0/881703897 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8998101930 0x7f8998196fa0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:11.390 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:11.389+0000 7f899e928640 1 --2- 172.21.15.49:0/881703897 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8998100740 0x7f8998196a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:11.390 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:11.389+0000 7f899e928640 1 -- 172.21.15.49:0/881703897 >> 172.21.15.49:0/881703897 conn(0x7f89980fbcd0 msgr2=0x7f8998104d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:06:11.390 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:11.389+0000 7f899e928640 1 -- 172.21.15.49:0/881703897 shutdown_connections 2024-03-18T23:06:11.390 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:11.389+0000 7f899e928640 1 -- 172.21.15.49:0/881703897 wait complete. 2024-03-18T23:06:11.400 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-03-18T23:06:11.515 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:11.513+0000 7f0985d09640 1 -- 172.21.15.49:0/30852087 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f09801022d0 msgr2=0x7f0980108b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:11.516 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:11.513+0000 7f0985d09640 1 --2- 172.21.15.49:0/30852087 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f09801022d0 0x7f0980108b60 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f0970009ca0 tx=0x7f097002c0f0 comp rx=0 tx=0).stop 2024-03-18T23:06:11.516 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:11.513+0000 7f0985d09640 1 -- 172.21.15.49:0/30852087 shutdown_connections 2024-03-18T23:06:11.516 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:11.513+0000 7f0985d09640 1 --2- 172.21.15.49:0/30852087 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f09801022d0 0x7f0980108b60 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:11.516 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:11.513+0000 7f0985d09640 1 --2- 172.21.15.49:0/30852087 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0980101910 0x7f0980101d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:11.516 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:11.513+0000 7f0985d09640 1 --2- 172.21.15.49:0/30852087 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0980100720 0x7f0980100b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:11.516 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:11.513+0000 7f0985d09640 1 -- 172.21.15.49:0/30852087 >> 172.21.15.49:0/30852087 conn(0x7f09800fbcb0 msgr2=0x7f09800fe0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:06:11.516 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:11.513+0000 7f0985d09640 1 -- 172.21.15.49:0/30852087 shutdown_connections 2024-03-18T23:06:11.516 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:11.513+0000 7f0985d09640 1 -- 172.21.15.49:0/30852087 wait complete. 2024-03-18T23:06:11.516 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:11.514+0000 7f0985d09640 1 Processor -- start 2024-03-18T23:06:11.516 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:11.514+0000 7f0985d09640 1 -- start start 2024-03-18T23:06:11.516 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:11.514+0000 7f0985d09640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0980100720 0x7f0980197b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:06:11.517 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:11.514+0000 7f0985d09640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0980101910 0x7f0980198050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:06:11.517 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:11.514+0000 7f0985d09640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f09801022d0 0x7f098019f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:06:11.517 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:11.514+0000 7f0985d09640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f098010d4c0 con 0x7f09801022d0 2024-03-18T23:06:11.517 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:11.514+0000 7f0985d09640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f098010d340 con 0x7f0980101910 2024-03-18T23:06:11.517 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:11.514+0000 7f0985d09640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f098010d640 con 0x7f0980100720 2024-03-18T23:06:11.517 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:11.514+0000 7f097ffff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f09801022d0 0x7f098019f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:06:11.517 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:11.514+0000 7f097effd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0980101910 0x7f0980198050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:06:11.517 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:11.514+0000 7f097ffff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f09801022d0 0x7f098019f0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:54244/0 (socket says 172.21.15.49:54244) 2024-03-18T23:06:11.517 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:11.514+0000 7f097effd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0980101910 0x7f0980198050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.49:44724/0 (socket says 172.21.15.49:44724) 2024-03-18T23:06:11.517 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:11.514+0000 7f097ffff640 1 -- 172.21.15.49:0/1297922511 learned_addr learned my addr 172.21.15.49:0/1297922511 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:06:11.517 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:11.514+0000 7f097f7fe640 1 --2- 172.21.15.49:0/1297922511 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0980100720 0x7f0980197b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:06:11.517 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:11.515+0000 7f097ffff640 1 -- 172.21.15.49:0/1297922511 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0980100720 msgr2=0x7f0980197b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:11.518 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:11.515+0000 7f097ffff640 1 --2- 172.21.15.49:0/1297922511 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0980100720 0x7f0980197b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:11.518 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:11.515+0000 7f097ffff640 1 -- 172.21.15.49:0/1297922511 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0980101910 msgr2=0x7f0980198050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:11.518 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:11.515+0000 7f097ffff640 1 --2- 172.21.15.49:0/1297922511 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0980101910 0x7f0980198050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:11.518 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:11.515+0000 7f097ffff640 1 -- 172.21.15.49:0/1297922511 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f098019f730 con 0x7f09801022d0 2024-03-18T23:06:11.518 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:11.515+0000 7f097effd640 1 --2- 172.21.15.49:0/1297922511 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0980101910 0x7f0980198050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:06:11.518 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:11.515+0000 7f097ffff640 1 --2- 172.21.15.49:0/1297922511 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f09801022d0 0x7f098019f0c0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f0970037040 tx=0x7f0970032f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:06:11.518 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:11.515+0000 7f097cff9640 1 -- 172.21.15.49:0/1297922511 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0970041020 con 0x7f09801022d0 2024-03-18T23:06:11.518 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:11.515+0000 7f097cff9640 1 -- 172.21.15.49:0/1297922511 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0970034cd0 con 0x7f09801022d0 2024-03-18T23:06:11.518 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:11.515+0000 7f0985d09640 1 -- 172.21.15.49:0/1297922511 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f098019f9c0 con 0x7f09801022d0 2024-03-18T23:06:11.518 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:11.515+0000 7f097cff9640 1 -- 172.21.15.49:0/1297922511 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0970039040 con 0x7f09801022d0 2024-03-18T23:06:11.518 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:11.515+0000 7f0985d09640 1 -- 172.21.15.49:0/1297922511 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f098019feb0 con 0x7f09801022d0 2024-03-18T23:06:11.519 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:11.516+0000 7f0985d09640 1 -- 172.21.15.49:0/1297922511 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0980107a40 con 0x7f09801022d0 2024-03-18T23:06:11.519 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:11.517+0000 7f097cff9640 1 -- 172.21.15.49:0/1297922511 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7f0970044020 con 0x7f09801022d0 2024-03-18T23:06:11.519 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:11.518+0000 7f097cff9640 1 --2- 172.21.15.49:0/1297922511 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f096c0a2800 0x7f096c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:06:11.520 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:11.518+0000 7f097f7fe640 1 --2- 172.21.15.49:0/1297922511 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f096c0a2800 0x7f096c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:06:11.520 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:11.518+0000 7f097cff9640 1 -- 172.21.15.49:0/1297922511 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f09700f07d0 con 0x7f09801022d0 2024-03-18T23:06:11.520 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:11.518+0000 7f097f7fe640 1 --2- 172.21.15.49:0/1297922511 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f096c0a2800 0x7f096c0a4cc0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f0980101770 tx=0x7f0978005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:06:11.522 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:11.522+0000 7f097cff9640 1 -- 172.21.15.49:0/1297922511 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f097004b020 con 0x7f09801022d0 2024-03-18T23:06:11.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:06:11 smithi139 ceph-mon[67282]: pgmap v291: 97 pgs: 94 active+clean, 2 creating+activating, 1 active; 581 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s rd, 4.3 KiB/s wr, 11 op/s 2024-03-18T23:06:11.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:06:11 smithi139 ceph-mon[67282]: from='client.14727 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T23:06:11.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:06:11 smithi139 ceph-mon[67282]: mds.? [v2:172.21.15.139:6834/1663212937,v1:172.21.15.139:6835/1663212937] up:active 2024-03-18T23:06:11.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:06:11 smithi139 ceph-mon[67282]: daemon mds.l assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-03-18T23:06:11.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:06:11 smithi139 ceph-mon[67282]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-03-18T23:06:11.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:06:11 smithi139 ceph-mon[67282]: Cluster is now healthy 2024-03-18T23:06:11.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:06:11 smithi139 ceph-mon[67282]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=f=up:active,3=e=up:active} 2 up:standby-replay 6 up:standby 2024-03-18T23:06:11.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:06:11 smithi139 ceph-mon[67282]: fsmap cephfs:5 2 up:standby-replay 1 up:creating 5 up:standby 4 up:active 2024-03-18T23:06:11.566 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:06:11 smithi139 ceph-mon[67282]: daemon mds.l is now active in filesystem cephfs as rank 4 2024-03-18T23:06:11.566 INFO:journalctl@ceph.mds.b.smithi139.stdout:Mar 18 23:06:11 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mds-b[98118]: 2024-03-18T23:06:11.149+0000 7f80011e6700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-18T23:06:11.566 INFO:journalctl@ceph.mds.b.smithi139.stdout:Mar 18 23:06:11 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mds-b[98118]: 2024-03-18T23:06:11.149+0000 7f80011e6700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-03-18T23:06:11.566 INFO:journalctl@ceph.mds.b.smithi139.stdout:Mar 18 23:06:11 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mds-b[98118]: 2024-03-18T23:06:11.149+0000 7f80011e6700 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-03-18T23:06:11.566 INFO:journalctl@ceph.mds.b.smithi139.stdout:Mar 18 23:06:11 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mds-b[98118]: 2024-03-18T23:06:11.149+0000 7f80011e6700 -1 mds.pinger is_rank_lagging: rank=3 was never sent ping request. 2024-03-18T23:06:11.712 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:06:11 smithi175 ceph-mon[67352]: pgmap v291: 97 pgs: 94 active+clean, 2 creating+activating, 1 active; 581 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s rd, 4.3 KiB/s wr, 11 op/s 2024-03-18T23:06:11.712 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:06:11 smithi175 ceph-mon[67352]: from='client.14727 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T23:06:11.712 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:06:11 smithi175 ceph-mon[67352]: mds.? [v2:172.21.15.139:6834/1663212937,v1:172.21.15.139:6835/1663212937] up:active 2024-03-18T23:06:11.712 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:06:11 smithi175 ceph-mon[67352]: daemon mds.l assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-03-18T23:06:11.712 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:06:11 smithi175 ceph-mon[67352]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-03-18T23:06:11.712 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:06:11 smithi175 ceph-mon[67352]: Cluster is now healthy 2024-03-18T23:06:11.712 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:06:11 smithi175 ceph-mon[67352]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=f=up:active,3=e=up:active} 2 up:standby-replay 6 up:standby 2024-03-18T23:06:11.712 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:06:11 smithi175 ceph-mon[67352]: fsmap cephfs:5 2 up:standby-replay 1 up:creating 5 up:standby 4 up:active 2024-03-18T23:06:11.712 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:06:11 smithi175 ceph-mon[67352]: daemon mds.l is now active in filesystem cephfs as rank 4 2024-03-18T23:06:11.738 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:11.738+0000 7f0985d09640 1 -- 172.21.15.49:0/1297922511 --> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7f0980061cf0 con 0x7f096c0a2800 2024-03-18T23:06:11.742 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:11.742+0000 7f097cff9640 1 -- 172.21.15.49:0/1297922511 <== mgr.14150 v2:172.21.15.49:6800/2083805211 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f0980061cf0 con 0x7f096c0a2800 2024-03-18T23:06:11.742 INFO:teuthology.orchestra.run.smithi049.stdout:/volumes/qa/sv_0/5ace2fe2-9a0b-4456-8b39-c5d3d58e533c 2024-03-18T23:06:11.744 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:11.744+0000 7f0985d09640 1 -- 172.21.15.49:0/1297922511 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f096c0a2800 msgr2=0x7f096c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:11.744 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:11.744+0000 7f0985d09640 1 --2- 172.21.15.49:0/1297922511 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f096c0a2800 0x7f096c0a4cc0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f0980101770 tx=0x7f0978005f90 comp rx=0 tx=0).stop 2024-03-18T23:06:11.744 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:11.744+0000 7f0985d09640 1 -- 172.21.15.49:0/1297922511 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f09801022d0 msgr2=0x7f098019f0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:11.745 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:11.744+0000 7f0985d09640 1 --2- 172.21.15.49:0/1297922511 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f09801022d0 0x7f098019f0c0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f0970037040 tx=0x7f0970032f40 comp rx=0 tx=0).stop 2024-03-18T23:06:11.745 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:11.744+0000 7f0985d09640 1 -- 172.21.15.49:0/1297922511 shutdown_connections 2024-03-18T23:06:11.745 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:11.744+0000 7f0985d09640 1 --2- 172.21.15.49:0/1297922511 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f096c0a2800 0x7f096c0a4cc0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:11.745 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:11.744+0000 7f0985d09640 1 --2- 172.21.15.49:0/1297922511 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f09801022d0 0x7f098019f0c0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:11.745 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:11.744+0000 7f0985d09640 1 --2- 172.21.15.49:0/1297922511 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0980101910 0x7f0980198050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:11.745 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:11.744+0000 7f0985d09640 1 --2- 172.21.15.49:0/1297922511 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0980100720 0x7f0980197b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:11.745 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:11.744+0000 7f0985d09640 1 -- 172.21.15.49:0/1297922511 >> 172.21.15.49:0/1297922511 conn(0x7f09800fbcb0 msgr2=0x7f0980104c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:06:11.745 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:11.744+0000 7f0985d09640 1 -- 172.21.15.49:0/1297922511 shutdown_connections 2024-03-18T23:06:11.745 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:11.744+0000 7f0985d09640 1 -- 172.21.15.49:0/1297922511 wait complete. 2024-03-18T23:06:11.755 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_1 --group_name qa 2024-03-18T23:06:11.870 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:11.868+0000 7f05c3574640 1 -- 172.21.15.49:0/1860140633 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f05bc1022f0 msgr2=0x7f05bc108b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:11.870 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:11.868+0000 7f05c3574640 1 --2- 172.21.15.49:0/1860140633 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f05bc1022f0 0x7f05bc108b80 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f05b8009ca0 tx=0x7f05b802c0f0 comp rx=0 tx=0).stop 2024-03-18T23:06:11.870 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:11.868+0000 7f05c3574640 1 -- 172.21.15.49:0/1860140633 shutdown_connections 2024-03-18T23:06:11.870 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:11.868+0000 7f05c3574640 1 --2- 172.21.15.49:0/1860140633 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f05bc1022f0 0x7f05bc108b80 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:11.870 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:11.868+0000 7f05c3574640 1 --2- 172.21.15.49:0/1860140633 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f05bc101930 0x7f05bc101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:11.870 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:11.868+0000 7f05c3574640 1 --2- 172.21.15.49:0/1860140633 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f05bc100740 0x7f05bc100b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:11.870 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:11.868+0000 7f05c3574640 1 -- 172.21.15.49:0/1860140633 >> 172.21.15.49:0/1860140633 conn(0x7f05bc0fbcd0 msgr2=0x7f05bc0fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:06:11.870 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:11.868+0000 7f05c3574640 1 -- 172.21.15.49:0/1860140633 shutdown_connections 2024-03-18T23:06:11.871 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:11.868+0000 7f05c3574640 1 -- 172.21.15.49:0/1860140633 wait complete. 2024-03-18T23:06:11.871 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:11.869+0000 7f05c3574640 1 Processor -- start 2024-03-18T23:06:11.871 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:11.869+0000 7f05c3574640 1 -- start start 2024-03-18T23:06:11.871 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:11.869+0000 7f05c3574640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f05bc100740 0x7f05bc06a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:06:11.871 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:11.869+0000 7f05c3574640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f05bc101930 0x7f05bc19c890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:06:11.871 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:11.869+0000 7f05c3574640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f05bc1022f0 0x7f05bc19eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:06:11.871 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:11.869+0000 7f05c3574640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f05bc10c3a0 con 0x7f05bc100740 2024-03-18T23:06:11.871 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:11.869+0000 7f05c3574640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f05bc10c220 con 0x7f05bc101930 2024-03-18T23:06:11.871 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:11.869+0000 7f05c3574640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f05bc10c520 con 0x7f05bc1022f0 2024-03-18T23:06:11.871 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:11.869+0000 7f05c1310640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f05bc100740 0x7f05bc06a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:06:11.871 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:11.869+0000 7f05c1310640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f05bc100740 0x7f05bc06a960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:54268/0 (socket says 172.21.15.49:54268) 2024-03-18T23:06:11.871 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:11.869+0000 7f05c1310640 1 -- 172.21.15.49:0/1176845621 learned_addr learned my addr 172.21.15.49:0/1176845621 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:06:11.871 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:11.869+0000 7f05c1b11640 1 --2- 172.21.15.49:0/1176845621 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f05bc1022f0 0x7f05bc19eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:06:11.872 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:11.869+0000 7f05c0b0f640 1 --2- 172.21.15.49:0/1176845621 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f05bc101930 0x7f05bc19c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:06:11.872 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:11.869+0000 7f05c1310640 1 -- 172.21.15.49:0/1176845621 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f05bc1022f0 msgr2=0x7f05bc19eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:11.872 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:11.869+0000 7f05c1310640 1 --2- 172.21.15.49:0/1176845621 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f05bc1022f0 0x7f05bc19eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:11.872 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:11.869+0000 7f05c1310640 1 -- 172.21.15.49:0/1176845621 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f05bc101930 msgr2=0x7f05bc19c890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:11.872 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:11.869+0000 7f05c1310640 1 --2- 172.21.15.49:0/1176845621 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f05bc101930 0x7f05bc19c890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:11.872 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:11.869+0000 7f05c1310640 1 -- 172.21.15.49:0/1176845621 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05bc19f7c0 con 0x7f05bc100740 2024-03-18T23:06:11.872 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:11.869+0000 7f05c0b0f640 1 --2- 172.21.15.49:0/1176845621 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f05bc101930 0x7f05bc19c890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T23:06:11.872 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:11.869+0000 7f05c1310640 1 --2- 172.21.15.49:0/1176845621 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f05bc100740 0x7f05bc06a960 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f05b400e9c0 tx=0x7f05b400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:06:11.872 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:11.869+0000 7f05b27fc640 1 -- 172.21.15.49:0/1176845621 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05b400cd80 con 0x7f05bc100740 2024-03-18T23:06:11.872 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:11.869+0000 7f05c3574640 1 -- 172.21.15.49:0/1176845621 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05bc19fa50 con 0x7f05bc100740 2024-03-18T23:06:11.872 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:11.869+0000 7f05b27fc640 1 -- 172.21.15.49:0/1176845621 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f05b40035f0 con 0x7f05bc100740 2024-03-18T23:06:11.872 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:11.869+0000 7f05b27fc640 1 -- 172.21.15.49:0/1176845621 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05b4010690 con 0x7f05bc100740 2024-03-18T23:06:11.872 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:11.870+0000 7f05c3574640 1 -- 172.21.15.49:0/1176845621 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05bc19ffa0 con 0x7f05bc100740 2024-03-18T23:06:11.873 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:11.871+0000 7f05c3574640 1 -- 172.21.15.49:0/1176845621 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05bc068a60 con 0x7f05bc100740 2024-03-18T23:06:11.874 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:11.872+0000 7f05b27fc640 1 -- 172.21.15.49:0/1176845621 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7f05b40037a0 con 0x7f05bc100740 2024-03-18T23:06:11.874 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:11.873+0000 7f05b27fc640 1 --2- 172.21.15.49:0/1176845621 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f05a80a2800 0x7f05a80a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:06:11.874 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:11.873+0000 7f05c0b0f640 1 --2- 172.21.15.49:0/1176845621 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f05a80a2800 0x7f05a80a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:06:11.874 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:11.873+0000 7f05b27fc640 1 -- 172.21.15.49:0/1176845621 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f05b40cee50 con 0x7f05bc100740 2024-03-18T23:06:11.874 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:11.873+0000 7f05c0b0f640 1 --2- 172.21.15.49:0/1176845621 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f05a80a2800 0x7f05a80a4cc0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f05bc06a6d0 tx=0x7f05ac005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:06:11.876 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:11.877+0000 7f05b27fc640 1 -- 172.21.15.49:0/1176845621 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f05b409c0d0 con 0x7f05bc100740 2024-03-18T23:06:12.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:12.093+0000 7f05c3574640 1 -- 172.21.15.49:0/1176845621 --> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f05bc061cf0 con 0x7f05a80a2800 2024-03-18T23:06:12.115 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:12.115+0000 7f05b27fc640 1 -- 172.21.15.49:0/1176845621 <== mgr.14150 v2:172.21.15.49:6800/2083805211 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f05bc061cf0 con 0x7f05a80a2800 2024-03-18T23:06:12.117 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:12.117+0000 7f05c3574640 1 -- 172.21.15.49:0/1176845621 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f05a80a2800 msgr2=0x7f05a80a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:12.117 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:12.117+0000 7f05c3574640 1 --2- 172.21.15.49:0/1176845621 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f05a80a2800 0x7f05a80a4cc0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f05bc06a6d0 tx=0x7f05ac005f90 comp rx=0 tx=0).stop 2024-03-18T23:06:12.117 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:12.117+0000 7f05c3574640 1 -- 172.21.15.49:0/1176845621 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f05bc100740 msgr2=0x7f05bc06a960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:12.117 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:12.117+0000 7f05c3574640 1 --2- 172.21.15.49:0/1176845621 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f05bc100740 0x7f05bc06a960 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f05b400e9c0 tx=0x7f05b400ee80 comp rx=0 tx=0).stop 2024-03-18T23:06:12.117 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:12.117+0000 7f05c3574640 1 -- 172.21.15.49:0/1176845621 shutdown_connections 2024-03-18T23:06:12.117 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:12.117+0000 7f05c3574640 1 --2- 172.21.15.49:0/1176845621 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f05a80a2800 0x7f05a80a4cc0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:12.118 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:12.117+0000 7f05c3574640 1 --2- 172.21.15.49:0/1176845621 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f05bc1022f0 0x7f05bc19eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:12.118 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:12.117+0000 7f05c3574640 1 --2- 172.21.15.49:0/1176845621 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f05bc101930 0x7f05bc19c890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:12.118 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:12.117+0000 7f05c3574640 1 --2- 172.21.15.49:0/1176845621 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f05bc100740 0x7f05bc06a960 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:12.118 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:12.117+0000 7f05c3574640 1 -- 172.21.15.49:0/1176845621 >> 172.21.15.49:0/1176845621 conn(0x7f05bc0fbcd0 msgr2=0x7f05bc104bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:06:12.118 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:12.117+0000 7f05c3574640 1 -- 172.21.15.49:0/1176845621 shutdown_connections 2024-03-18T23:06:12.118 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:12.117+0000 7f05c3574640 1 -- 172.21.15.49:0/1176845621 wait complete. 2024-03-18T23:06:12.128 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-03-18T23:06:12.245 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:12.244+0000 7fc9a1cf8640 1 -- 172.21.15.49:0/1855388787 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc99c101dc0 msgr2=0x7fc99c1021e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:12.245 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:12.244+0000 7fc9a1cf8640 1 --2- 172.21.15.49:0/1855388787 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc99c101dc0 0x7fc99c1021e0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fc98c009900 tx=0x7fc98c02bc30 comp rx=0 tx=0).stop 2024-03-18T23:06:12.245 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:12.244+0000 7fc9a1cf8640 1 -- 172.21.15.49:0/1855388787 shutdown_connections 2024-03-18T23:06:12.245 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:12.244+0000 7fc9a1cf8640 1 --2- 172.21.15.49:0/1855388787 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc99c100270 0x7fc99c100720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:12.245 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:12.244+0000 7fc9a1cf8640 1 --2- 172.21.15.49:0/1855388787 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc99c0ff8b0 0x7fc99c0ffd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:12.245 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:12.244+0000 7fc9a1cf8640 1 --2- 172.21.15.49:0/1855388787 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc99c101dc0 0x7fc99c1021e0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:12.245 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:12.244+0000 7fc9a1cf8640 1 -- 172.21.15.49:0/1855388787 >> 172.21.15.49:0/1855388787 conn(0x7fc99c0fb430 msgr2=0x7fc99c0fd870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:06:12.245 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:12.245+0000 7fc9a1cf8640 1 -- 172.21.15.49:0/1855388787 shutdown_connections 2024-03-18T23:06:12.245 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:12.245+0000 7fc9a1cf8640 1 -- 172.21.15.49:0/1855388787 wait complete. 2024-03-18T23:06:12.246 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:12.245+0000 7fc9a1cf8640 1 Processor -- start 2024-03-18T23:06:12.247 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:12.245+0000 7fc9a1cf8640 1 -- start start 2024-03-18T23:06:12.247 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:12.246+0000 7fc9a1cf8640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc99c0ff8b0 0x7fc99c199d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:06:12.247 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:12.246+0000 7fc9a1cf8640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc99c100270 0x7fc99c19a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:06:12.247 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:12.246+0000 7fc9a1cf8640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc99c101dc0 0x7fc99c1a1300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:06:12.247 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:12.246+0000 7fc9a1cf8640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fc99c10f710 con 0x7fc99c0ff8b0 2024-03-18T23:06:12.248 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:12.246+0000 7fc9a1cf8640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fc99c10f590 con 0x7fc99c100270 2024-03-18T23:06:12.248 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:12.246+0000 7fc9a1cf8640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc99c10f890 con 0x7fc99c101dc0 2024-03-18T23:06:12.248 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:12.246+0000 7fc99b7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc99c0ff8b0 0x7fc99c199d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:06:12.248 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:12.246+0000 7fc99bfff640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc99c101dc0 0x7fc99c1a1300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:06:12.248 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:12.246+0000 7fc99b7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc99c0ff8b0 0x7fc99c199d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:54288/0 (socket says 172.21.15.49:54288) 2024-03-18T23:06:12.248 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:12.246+0000 7fc99b7fe640 1 -- 172.21.15.49:0/3622753884 learned_addr learned my addr 172.21.15.49:0/3622753884 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:06:12.248 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:12.246+0000 7fc99affd640 1 --2- 172.21.15.49:0/3622753884 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc99c100270 0x7fc99c19a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:06:12.248 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:12.246+0000 7fc99b7fe640 1 -- 172.21.15.49:0/3622753884 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc99c101dc0 msgr2=0x7fc99c1a1300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:12.248 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:12.246+0000 7fc99b7fe640 1 --2- 172.21.15.49:0/3622753884 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc99c101dc0 0x7fc99c1a1300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:12.248 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:12.246+0000 7fc99b7fe640 1 -- 172.21.15.49:0/3622753884 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc99c100270 msgr2=0x7fc99c19a290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:12.248 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:12.246+0000 7fc99b7fe640 1 --2- 172.21.15.49:0/3622753884 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc99c100270 0x7fc99c19a290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:12.248 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:12.246+0000 7fc99b7fe640 1 -- 172.21.15.49:0/3622753884 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc99c1a1970 con 0x7fc99c0ff8b0 2024-03-18T23:06:12.248 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:12.246+0000 7fc99bfff640 1 --2- 172.21.15.49:0/3622753884 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc99c101dc0 0x7fc99c1a1300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:06:12.249 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:12.246+0000 7fc99affd640 1 --2- 172.21.15.49:0/3622753884 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc99c100270 0x7fc99c19a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:06:12.249 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:12.246+0000 7fc99b7fe640 1 --2- 172.21.15.49:0/3622753884 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc99c0ff8b0 0x7fc99c199d50 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fc98c000c00 tx=0x7fc98c02c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:06:12.249 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:12.248+0000 7fc998ff9640 1 -- 172.21.15.49:0/3622753884 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc98c043020 con 0x7fc99c0ff8b0 2024-03-18T23:06:12.249 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:12.248+0000 7fc998ff9640 1 -- 172.21.15.49:0/3622753884 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc98c0054e0 con 0x7fc99c0ff8b0 2024-03-18T23:06:12.250 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:12.248+0000 7fc9a1cf8640 1 -- 172.21.15.49:0/3622753884 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc99c1a1c00 con 0x7fc99c0ff8b0 2024-03-18T23:06:12.250 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:12.248+0000 7fc998ff9640 1 -- 172.21.15.49:0/3622753884 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc98c003c40 con 0x7fc99c0ff8b0 2024-03-18T23:06:12.250 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:12.248+0000 7fc9a1cf8640 1 -- 172.21.15.49:0/3622753884 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc99c1a21a0 con 0x7fc99c0ff8b0 2024-03-18T23:06:12.250 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:12.249+0000 7fc9a1cf8640 1 -- 172.21.15.49:0/3622753884 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc99c10b830 con 0x7fc99c0ff8b0 2024-03-18T23:06:12.250 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:12.250+0000 7fc998ff9640 1 -- 172.21.15.49:0/3622753884 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7fc98c046020 con 0x7fc99c0ff8b0 2024-03-18T23:06:12.250 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:12.250+0000 7fc998ff9640 1 --2- 172.21.15.49:0/3622753884 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fc9840a2880 0x7fc9840a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:06:12.251 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:12.250+0000 7fc99affd640 1 --2- 172.21.15.49:0/3622753884 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fc9840a2880 0x7fc9840a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:06:12.251 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:12.250+0000 7fc998ff9640 1 -- 172.21.15.49:0/3622753884 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7fc98c0f0900 con 0x7fc99c0ff8b0 2024-03-18T23:06:12.251 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:12.251+0000 7fc99affd640 1 --2- 172.21.15.49:0/3622753884 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fc9840a2880 0x7fc9840a4d40 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fc99c19b260 tx=0x7fc990005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:06:12.254 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:12.255+0000 7fc998ff9640 1 -- 172.21.15.49:0/3622753884 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fc98c04d020 con 0x7fc99c0ff8b0 2024-03-18T23:06:12.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:06:12 smithi049 ceph-mon[62991]: from='client.14739 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T23:06:12.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:06:12 smithi049 ceph-mon[62991]: from='client.14745 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T23:06:12.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:06:12 smithi049 ceph-mon[62991]: mds.? [v2:172.21.15.175:6832/1702997173,v1:172.21.15.175:6833/1702997173] up:active 2024-03-18T23:06:12.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:06:12 smithi049 ceph-mon[62991]: fsmap cephfs:5 2 up:standby-replay 5 up:standby 5 up:active 2024-03-18T23:06:12.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:06:12 smithi049 ceph-mon[62991]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T23:06:12.466 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:12.466+0000 7fc9a1cf8640 1 -- 172.21.15.49:0/3622753884 --> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7fc99c061cf0 con 0x7fc9840a2880 2024-03-18T23:06:12.469 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:12.470+0000 7fc998ff9640 1 -- 172.21.15.49:0/3622753884 <== mgr.14150 v2:172.21.15.49:6800/2083805211 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7fc99c061cf0 con 0x7fc9840a2880 2024-03-18T23:06:12.470 INFO:teuthology.orchestra.run.smithi049.stdout:/volumes/qa/sv_1/b5d9a7f7-1d19-4c17-b495-a118d7fb5c7e 2024-03-18T23:06:12.472 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:12.472+0000 7fc9a1cf8640 1 -- 172.21.15.49:0/3622753884 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fc9840a2880 msgr2=0x7fc9840a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:12.472 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:12.472+0000 7fc9a1cf8640 1 --2- 172.21.15.49:0/3622753884 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fc9840a2880 0x7fc9840a4d40 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fc99c19b260 tx=0x7fc990005f90 comp rx=0 tx=0).stop 2024-03-18T23:06:12.472 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:12.472+0000 7fc9a1cf8640 1 -- 172.21.15.49:0/3622753884 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc99c0ff8b0 msgr2=0x7fc99c199d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:12.472 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:12.472+0000 7fc9a1cf8640 1 --2- 172.21.15.49:0/3622753884 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc99c0ff8b0 0x7fc99c199d50 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fc98c000c00 tx=0x7fc98c02c650 comp rx=0 tx=0).stop 2024-03-18T23:06:12.472 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:12.472+0000 7fc9a1cf8640 1 -- 172.21.15.49:0/3622753884 shutdown_connections 2024-03-18T23:06:12.472 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:12.472+0000 7fc9a1cf8640 1 --2- 172.21.15.49:0/3622753884 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fc9840a2880 0x7fc9840a4d40 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:12.472 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:12.472+0000 7fc9a1cf8640 1 --2- 172.21.15.49:0/3622753884 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc99c101dc0 0x7fc99c1a1300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:12.472 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:12.472+0000 7fc9a1cf8640 1 --2- 172.21.15.49:0/3622753884 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc99c100270 0x7fc99c19a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:12.472 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:12.472+0000 7fc9a1cf8640 1 --2- 172.21.15.49:0/3622753884 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc99c0ff8b0 0x7fc99c199d50 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:12.472 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:12.472+0000 7fc9a1cf8640 1 -- 172.21.15.49:0/3622753884 >> 172.21.15.49:0/3622753884 conn(0x7fc99c0fb430 msgr2=0x7fc99c0fcb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:06:12.473 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:12.472+0000 7fc9a1cf8640 1 -- 172.21.15.49:0/3622753884 shutdown_connections 2024-03-18T23:06:12.473 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:12.472+0000 7fc9a1cf8640 1 -- 172.21.15.49:0/3622753884 wait complete. 2024-03-18T23:06:12.483 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-18T23:06:12.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:06:12 smithi139 ceph-mon[67282]: from='client.14739 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T23:06:12.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:06:12 smithi139 ceph-mon[67282]: from='client.14745 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T23:06:12.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:06:12 smithi139 ceph-mon[67282]: mds.? [v2:172.21.15.175:6832/1702997173,v1:172.21.15.175:6833/1702997173] up:active 2024-03-18T23:06:12.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:06:12 smithi139 ceph-mon[67282]: fsmap cephfs:5 2 up:standby-replay 5 up:standby 5 up:active 2024-03-18T23:06:12.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:06:12 smithi139 ceph-mon[67282]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T23:06:12.593 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:12.591+0000 7fd55466c640 1 -- 172.21.15.49:0/1689319097 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd54c102340 msgr2=0x7fd54c104730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:12.593 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:12.591+0000 7fd55466c640 1 --2- 172.21.15.49:0/1689319097 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd54c102340 0x7fd54c104730 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fd548009ca0 tx=0x7fd54802c0f0 comp rx=0 tx=0).stop 2024-03-18T23:06:12.593 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:12.591+0000 7fd55466c640 1 -- 172.21.15.49:0/1689319097 shutdown_connections 2024-03-18T23:06:12.594 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:12.591+0000 7fd55466c640 1 --2- 172.21.15.49:0/1689319097 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd54c1075a0 0x7fd54c1099f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:12.594 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:12.591+0000 7fd55466c640 1 --2- 172.21.15.49:0/1689319097 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd54c104c70 0x7fd54c107060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:12.594 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:12.591+0000 7fd55466c640 1 --2- 172.21.15.49:0/1689319097 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd54c102340 0x7fd54c104730 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:12.594 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:12.591+0000 7fd55466c640 1 -- 172.21.15.49:0/1689319097 >> 172.21.15.49:0/1689319097 conn(0x7fd54c0fbd50 msgr2=0x7fd54c0fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:06:12.594 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:12.591+0000 7fd55466c640 1 -- 172.21.15.49:0/1689319097 shutdown_connections 2024-03-18T23:06:12.594 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:12.591+0000 7fd55466c640 1 -- 172.21.15.49:0/1689319097 wait complete. 2024-03-18T23:06:12.594 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:12.592+0000 7fd55466c640 1 Processor -- start 2024-03-18T23:06:12.594 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:12.592+0000 7fd55466c640 1 -- start start 2024-03-18T23:06:12.594 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:12.592+0000 7fd55466c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd54c102340 0x7fd54c197bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:06:12.594 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:12.592+0000 7fd55466c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd54c104c70 0x7fd54c1980f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:06:12.594 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:12.592+0000 7fd55466c640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd54c1075a0 0x7fd54c19f160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:06:12.594 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:12.592+0000 7fd55466c640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fd54c10d500 con 0x7fd54c102340 2024-03-18T23:06:12.594 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:12.592+0000 7fd55466c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fd54c10d380 con 0x7fd54c104c70 2024-03-18T23:06:12.595 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:12.592+0000 7fd55466c640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd54c10d680 con 0x7fd54c1075a0 2024-03-18T23:06:12.595 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:12.592+0000 7fd552408640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd54c102340 0x7fd54c197bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:06:12.595 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:12.592+0000 7fd552408640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd54c102340 0x7fd54c197bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:54298/0 (socket says 172.21.15.49:54298) 2024-03-18T23:06:12.595 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:12.592+0000 7fd552408640 1 -- 172.21.15.49:0/2244712381 learned_addr learned my addr 172.21.15.49:0/2244712381 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:06:12.595 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:12.592+0000 7fd552c09640 1 --2- 172.21.15.49:0/2244712381 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd54c1075a0 0x7fd54c19f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:06:12.595 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:12.592+0000 7fd551c07640 1 --2- 172.21.15.49:0/2244712381 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd54c104c70 0x7fd54c1980f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:06:12.595 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:12.592+0000 7fd552408640 1 -- 172.21.15.49:0/2244712381 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd54c1075a0 msgr2=0x7fd54c19f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:12.595 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:12.592+0000 7fd552408640 1 --2- 172.21.15.49:0/2244712381 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd54c1075a0 0x7fd54c19f160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:12.595 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:12.592+0000 7fd552408640 1 -- 172.21.15.49:0/2244712381 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd54c104c70 msgr2=0x7fd54c1980f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:12.595 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:12.592+0000 7fd552408640 1 --2- 172.21.15.49:0/2244712381 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd54c104c70 0x7fd54c1980f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:12.595 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:12.592+0000 7fd552408640 1 -- 172.21.15.49:0/2244712381 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd54c19f7d0 con 0x7fd54c102340 2024-03-18T23:06:12.595 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:12.593+0000 7fd552408640 1 --2- 172.21.15.49:0/2244712381 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd54c102340 0x7fd54c197bb0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fd548009dc0 tx=0x7fd548032f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:06:12.595 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:12.593+0000 7fd5437fe640 1 -- 172.21.15.49:0/2244712381 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd548041020 con 0x7fd54c102340 2024-03-18T23:06:12.595 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:12.593+0000 7fd5437fe640 1 -- 172.21.15.49:0/2244712381 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd548034cd0 con 0x7fd54c102340 2024-03-18T23:06:12.595 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:12.593+0000 7fd55466c640 1 -- 172.21.15.49:0/2244712381 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd54c19fa60 con 0x7fd54c102340 2024-03-18T23:06:12.596 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:12.593+0000 7fd5437fe640 1 -- 172.21.15.49:0/2244712381 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd548039040 con 0x7fd54c102340 2024-03-18T23:06:12.596 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:12.593+0000 7fd55466c640 1 -- 172.21.15.49:0/2244712381 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd54c19ff50 con 0x7fd54c102340 2024-03-18T23:06:12.596 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:12.594+0000 7fd55466c640 1 -- 172.21.15.49:0/2244712381 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd54c0fd840 con 0x7fd54c102340 2024-03-18T23:06:12.597 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:12.595+0000 7fd5437fe640 1 -- 172.21.15.49:0/2244712381 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7fd548044020 con 0x7fd54c102340 2024-03-18T23:06:12.597 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:12.595+0000 7fd5437fe640 1 --2- 172.21.15.49:0/2244712381 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fd5380a2800 0x7fd5380a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:06:12.597 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:12.596+0000 7fd551c07640 1 --2- 172.21.15.49:0/2244712381 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fd5380a2800 0x7fd5380a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:06:12.597 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:12.596+0000 7fd5437fe640 1 -- 172.21.15.49:0/2244712381 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7fd5480f0890 con 0x7fd54c102340 2024-03-18T23:06:12.597 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:12.596+0000 7fd551c07640 1 --2- 172.21.15.49:0/2244712381 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fd5380a2800 0x7fd5380a4cc0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fd54c1990c0 tx=0x7fd53c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:06:12.599 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:12.600+0000 7fd5437fe640 1 -- 172.21.15.49:0/2244712381 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fd54804b020 con 0x7fd54c102340 2024-03-18T23:06:12.711 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:06:12 smithi175 ceph-mon[67352]: from='client.14739 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T23:06:12.711 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:06:12 smithi175 ceph-mon[67352]: from='client.14745 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T23:06:12.711 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:06:12 smithi175 ceph-mon[67352]: mds.? [v2:172.21.15.175:6832/1702997173,v1:172.21.15.175:6833/1702997173] up:active 2024-03-18T23:06:12.711 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:06:12 smithi175 ceph-mon[67352]: fsmap cephfs:5 2 up:standby-replay 5 up:standby 5 up:active 2024-03-18T23:06:12.711 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:06:12 smithi175 ceph-mon[67352]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T23:06:12.823 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:12.823+0000 7fd55466c640 1 -- 172.21.15.49:0/2244712381 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd54c02d1f0 con 0x7fd54c102340 2024-03-18T23:06:12.824 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:12.824+0000 7fd5437fe640 1 -- 172.21.15.49:0/2244712381 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11936 (secure 0 0 0) 0x7fd5480bda80 con 0x7fd54c102340 2024-03-18T23:06:12.824 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-18T23:06:12.824 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6835/3631301630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":3631301630},{"type":"v1","addr":"172.21.15.49:6835","nonce":3631301630}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6837/444305233","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":444305233},{"type":"v1","addr":"172.21.15.49:6837","nonce":444305233}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-18T23:06:04.575999+0000","modified":"2024-03-18T23:06:11.998736+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24488,"mds_1":24463,"mds_2":24430,"mds_3":24419,"mds_4":24391},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14622":{"gid":14622,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.49:6839/1934085093","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6838","nonce":1934085093},{"type":"v1","addr":"172.21.15.49:6839","nonce":1934085093}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14634":{"gid":14634,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.49:6841/2869298862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6840","nonce":2869298862},{"type":"v1","addr":"172.21.15.49:6841","nonce":2869298862}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24389":{"gid":24389,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6833/3766178603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3766178603},{"type":"v1","addr":"172.21.15.139:6833","nonce":3766178603}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24391":{"gid":24391,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.175:6833/1702997173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":1702997173},{"type":"v1","addr":"172.21.15.175:6833","nonce":1702997173}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":9,"addr":"172.21.15.139:6835/1663212937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":1663212937},{"type":"v1","addr":"172.21.15.139:6835","nonce":1663212937}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.175:6835/755474931","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":755474931},{"type":"v1","addr":"172.21.15.175:6835","nonce":755474931}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6837/379482756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":379482756},{"type":"v1","addr":"172.21.15.175:6837","nonce":379482756}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6837/2330317540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6836","nonce":2330317540},{"type":"v1","addr":"172.21.15.139:6837","nonce":2330317540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"172.21.15.175:6839/3640882084","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":3640882084},{"type":"v1","addr":"172.21.15.175:6839","nonce":3640882084}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.139:6839/4280128209","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6838","nonce":4280128209},{"type":"v1","addr":"172.21.15.139:6839","nonce":4280128209}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24463,24488,24430,24419,24391]},"id":1}]} 2024-03-18T23:06:12.824 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 27 2024-03-18T23:06:12.826 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:12.826+0000 7fd55466c640 1 -- 172.21.15.49:0/2244712381 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fd5380a2800 msgr2=0x7fd5380a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:12.826 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:12.826+0000 7fd55466c640 1 --2- 172.21.15.49:0/2244712381 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fd5380a2800 0x7fd5380a4cc0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fd54c1990c0 tx=0x7fd53c005f90 comp rx=0 tx=0).stop 2024-03-18T23:06:12.826 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:12.826+0000 7fd55466c640 1 -- 172.21.15.49:0/2244712381 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd54c102340 msgr2=0x7fd54c197bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:12.826 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:12.826+0000 7fd55466c640 1 --2- 172.21.15.49:0/2244712381 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd54c102340 0x7fd54c197bb0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fd548009dc0 tx=0x7fd548032f20 comp rx=0 tx=0).stop 2024-03-18T23:06:12.826 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:12.826+0000 7fd55466c640 1 -- 172.21.15.49:0/2244712381 shutdown_connections 2024-03-18T23:06:12.826 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:12.826+0000 7fd55466c640 1 --2- 172.21.15.49:0/2244712381 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fd5380a2800 0x7fd5380a4cc0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:12.826 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:12.826+0000 7fd55466c640 1 --2- 172.21.15.49:0/2244712381 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd54c1075a0 0x7fd54c19f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:12.826 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:12.826+0000 7fd55466c640 1 --2- 172.21.15.49:0/2244712381 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd54c104c70 0x7fd54c1980f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:12.827 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:12.826+0000 7fd55466c640 1 --2- 172.21.15.49:0/2244712381 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd54c102340 0x7fd54c197bb0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:12.827 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:12.826+0000 7fd55466c640 1 -- 172.21.15.49:0/2244712381 >> 172.21.15.49:0/2244712381 conn(0x7fd54c0fbd50 msgr2=0x7fd54c102dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:06:12.827 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:12.826+0000 7fd55466c640 1 -- 172.21.15.49:0/2244712381 shutdown_connections 2024-03-18T23:06:12.827 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:12.826+0000 7fd55466c640 1 -- 172.21.15.49:0/2244712381 wait complete. 2024-03-18T23:06:12.837 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-18T23:06:12.950 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:12.947+0000 7f8df0b9d640 1 -- 172.21.15.49:0/131170094 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8dec101760 msgr2=0x7f8dec101be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:12.950 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:12.947+0000 7f8df0b9d640 1 --2- 172.21.15.49:0/131170094 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8dec101760 0x7f8dec101be0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f8de0009900 tx=0x7f8de002bc30 comp rx=0 tx=0).stop 2024-03-18T23:06:12.950 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:12.947+0000 7f8df0b9d640 1 -- 172.21.15.49:0/131170094 shutdown_connections 2024-03-18T23:06:12.950 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:12.947+0000 7f8df0b9d640 1 --2- 172.21.15.49:0/131170094 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8dec102120 0x7f8dec1089b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:12.951 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:12.947+0000 7f8df0b9d640 1 --2- 172.21.15.49:0/131170094 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8dec101760 0x7f8dec101be0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:12.951 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:12.947+0000 7f8df0b9d640 1 --2- 172.21.15.49:0/131170094 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8dec100570 0x7f8dec100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:12.951 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:12.947+0000 7f8df0b9d640 1 -- 172.21.15.49:0/131170094 >> 172.21.15.49:0/131170094 conn(0x7f8dec0fbb00 msgr2=0x7f8dec0fdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:06:12.951 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:12.947+0000 7f8df0b9d640 1 -- 172.21.15.49:0/131170094 shutdown_connections 2024-03-18T23:06:12.951 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:12.947+0000 7f8df0b9d640 1 -- 172.21.15.49:0/131170094 wait complete. 2024-03-18T23:06:12.951 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:12.948+0000 7f8df0b9d640 1 Processor -- start 2024-03-18T23:06:12.952 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:12.948+0000 7f8df0b9d640 1 -- start start 2024-03-18T23:06:12.952 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:12.948+0000 7f8df0b9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8dec100570 0x7f8dec196880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:06:12.952 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:12.948+0000 7f8df0b9d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8dec101760 0x7f8dec196dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:06:12.952 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:12.948+0000 7f8df0b9d640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8dec102120 0x7f8dec19de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:06:12.952 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:12.948+0000 7f8df0b9d640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f8dec10c200 con 0x7f8dec100570 2024-03-18T23:06:12.952 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:12.948+0000 7f8df0b9d640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f8dec10c080 con 0x7f8dec101760 2024-03-18T23:06:12.952 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:12.948+0000 7f8df0b9d640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f8dec10c380 con 0x7f8dec102120 2024-03-18T23:06:12.952 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:12.948+0000 7f8deb7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8dec100570 0x7f8dec196880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:06:12.952 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:12.948+0000 7f8deb7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8dec100570 0x7f8dec196880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:54306/0 (socket says 172.21.15.49:54306) 2024-03-18T23:06:12.952 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:12.948+0000 7f8deb7fe640 1 -- 172.21.15.49:0/727513553 learned_addr learned my addr 172.21.15.49:0/727513553 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:06:12.952 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:12.948+0000 7f8debfff640 1 --2- 172.21.15.49:0/727513553 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8dec102120 0x7f8dec19de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:06:12.952 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:12.948+0000 7f8deaffd640 1 --2- 172.21.15.49:0/727513553 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8dec101760 0x7f8dec196dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:06:12.952 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:12.949+0000 7f8deb7fe640 1 -- 172.21.15.49:0/727513553 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8dec102120 msgr2=0x7f8dec19de30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:12.953 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:12.949+0000 7f8deb7fe640 1 --2- 172.21.15.49:0/727513553 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8dec102120 0x7f8dec19de30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:12.953 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:12.949+0000 7f8deb7fe640 1 -- 172.21.15.49:0/727513553 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8dec101760 msgr2=0x7f8dec196dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:12.953 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:12.949+0000 7f8deb7fe640 1 --2- 172.21.15.49:0/727513553 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8dec101760 0x7f8dec196dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:12.953 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:12.949+0000 7f8deb7fe640 1 -- 172.21.15.49:0/727513553 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8dec19e4a0 con 0x7f8dec100570 2024-03-18T23:06:12.953 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:12.949+0000 7f8debfff640 1 --2- 172.21.15.49:0/727513553 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8dec102120 0x7f8dec19de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:06:12.953 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:12.949+0000 7f8deb7fe640 1 --2- 172.21.15.49:0/727513553 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8dec100570 0x7f8dec196880 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f8ddc00e9c0 tx=0x7f8ddc00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:06:12.953 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:12.949+0000 7f8de8ff9640 1 -- 172.21.15.49:0/727513553 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ddc00cd80 con 0x7f8dec100570 2024-03-18T23:06:12.953 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:12.949+0000 7f8de8ff9640 1 -- 172.21.15.49:0/727513553 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ddc0035f0 con 0x7f8dec100570 2024-03-18T23:06:12.953 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:12.949+0000 7f8df0b9d640 1 -- 172.21.15.49:0/727513553 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8dec19e790 con 0x7f8dec100570 2024-03-18T23:06:12.953 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:12.949+0000 7f8de8ff9640 1 -- 172.21.15.49:0/727513553 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ddc010690 con 0x7f8dec100570 2024-03-18T23:06:12.953 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:12.949+0000 7f8df0b9d640 1 -- 172.21.15.49:0/727513553 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8dec19ec60 con 0x7f8dec100570 2024-03-18T23:06:12.953 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:12.950+0000 7f8df0b9d640 1 -- 172.21.15.49:0/727513553 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8dec107830 con 0x7f8dec100570 2024-03-18T23:06:12.954 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:12.951+0000 7f8de8ff9640 1 -- 172.21.15.49:0/727513553 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7f8ddc0037a0 con 0x7f8dec100570 2024-03-18T23:06:12.954 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:12.951+0000 7f8de8ff9640 1 --2- 172.21.15.49:0/727513553 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f8dd40a2800 0x7f8dd40a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:06:12.954 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:12.952+0000 7f8deaffd640 1 --2- 172.21.15.49:0/727513553 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f8dd40a2800 0x7f8dd40a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:06:12.954 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:12.952+0000 7f8de8ff9640 1 -- 172.21.15.49:0/727513553 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f8ddc0cec70 con 0x7f8dec100570 2024-03-18T23:06:12.954 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:12.952+0000 7f8deaffd640 1 --2- 172.21.15.49:0/727513553 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f8dd40a2800 0x7f8dd40a4cc0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f8de002c7b0 tx=0x7f8de0005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:06:12.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:12.956+0000 7f8de8ff9640 1 -- 172.21.15.49:0/727513553 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f8ddc09be60 con 0x7f8dec100570 2024-03-18T23:06:13.142 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:13.142+0000 7f8df0b9d640 1 -- 172.21.15.49:0/727513553 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8dec02d1f0 con 0x7f8dec100570 2024-03-18T23:06:13.143 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:13.144+0000 7f8de8ff9640 1 -- 172.21.15.49:0/727513553 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+22242 (secure 0 0 0) 0x7f8ddc09b760 con 0x7f8dec100570 2024-03-18T23:06:13.144 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-18T23:06:13.144 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":86,"fsid":"c63591ea-e57a-11ee-95c9-87774f69a715","created":"2024-03-18T22:57:58.734467+0000","modified":"2024-03-18T23:06:05.892182+0000","last_up_change":"2024-03-18T23:05:16.576310+0000","last_in_change":"2024-03-18T23:04:57.024067+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-18T23:01:28.363432+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-18T23:06:03.172963+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-18T23:06:03.973303+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e7fc9daa-4098-4ba5-8019-9aaef1aab3e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":1651162501},{"type":"v1","addr":"172.21.15.49:6803","nonce":1651162501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":1651162501},{"type":"v1","addr":"172.21.15.49:6805","nonce":1651162501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":1651162501},{"type":"v1","addr":"172.21.15.49:6809","nonce":1651162501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":1651162501},{"type":"v1","addr":"172.21.15.49:6807","nonce":1651162501}]},"public_addr":"172.21.15.49:6803/1651162501","cluster_addr":"172.21.15.49:6805/1651162501","heartbeat_back_addr":"172.21.15.49:6809/1651162501","heartbeat_front_addr":"172.21.15.49:6807/1651162501","state":["exists","up"]},{"osd":1,"uuid":"40ac929c-e90f-434c-a796-9aabd686f1fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":1766700359},{"type":"v1","addr":"172.21.15.139:6801","nonce":1766700359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":1766700359},{"type":"v1","addr":"172.21.15.139:6803","nonce":1766700359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":1766700359},{"type":"v1","addr":"172.21.15.139:6807","nonce":1766700359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":1766700359},{"type":"v1","addr":"172.21.15.139:6805","nonce":1766700359}]},"public_addr":"172.21.15.139:6801/1766700359","cluster_addr":"172.21.15.139:6803/1766700359","heartbeat_back_addr":"172.21.15.139:6807/1766700359","heartbeat_front_addr":"172.21.15.139:6805/1766700359","state":["exists","up"]},{"osd":2,"uuid":"8ae3b666-00b4-45ed-a5ee-4d0771edabb6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":3752671234},{"type":"v1","addr":"172.21.15.175:6801","nonce":3752671234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":3752671234},{"type":"v1","addr":"172.21.15.175:6803","nonce":3752671234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":3752671234},{"type":"v1","addr":"172.21.15.175:6807","nonce":3752671234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":3752671234},{"type":"v1","addr":"172.21.15.175:6805","nonce":3752671234}]},"public_addr":"172.21.15.175:6801/3752671234","cluster_addr":"172.21.15.175:6803/3752671234","heartbeat_back_addr":"172.21.15.175:6807/3752671234","heartbeat_front_addr":"172.21.15.175:6805/3752671234","state":["exists","up"]},{"osd":3,"uuid":"1a62e89d-45d4-4237-b813-7f96a59aca7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":956160536},{"type":"v1","addr":"172.21.15.49:6811","nonce":956160536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":956160536},{"type":"v1","addr":"172.21.15.49:6813","nonce":956160536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":956160536},{"type":"v1","addr":"172.21.15.49:6817","nonce":956160536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":956160536},{"type":"v1","addr":"172.21.15.49:6815","nonce":956160536}]},"public_addr":"172.21.15.49:6811/956160536","cluster_addr":"172.21.15.49:6813/956160536","heartbeat_back_addr":"172.21.15.49:6817/956160536","heartbeat_front_addr":"172.21.15.49:6815/956160536","state":["exists","up"]},{"osd":4,"uuid":"c715101c-0c3f-4b98-b142-d64a35e187df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":3129712668},{"type":"v1","addr":"172.21.15.139:6809","nonce":3129712668}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":3129712668},{"type":"v1","addr":"172.21.15.139:6811","nonce":3129712668}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":3129712668},{"type":"v1","addr":"172.21.15.139:6815","nonce":3129712668}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":3129712668},{"type":"v1","addr":"172.21.15.139:6813","nonce":3129712668}]},"public_addr":"172.21.15.139:6809/3129712668","cluster_addr":"172.21.15.139:6811/3129712668","heartbeat_back_addr":"172.21.15.139:6815/3129712668","heartbeat_front_addr":"172.21.15.139:6813/3129712668","state":["exists","up"]},{"osd":5,"uuid":"4b641f81-77eb-45fd-8464-511e7b3062b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":2774567215},{"type":"v1","addr":"172.21.15.175:6809","nonce":2774567215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":2774567215},{"type":"v1","addr":"172.21.15.175:6811","nonce":2774567215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":2774567215},{"type":"v1","addr":"172.21.15.175:6815","nonce":2774567215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":2774567215},{"type":"v1","addr":"172.21.15.175:6813","nonce":2774567215}]},"public_addr":"172.21.15.175:6809/2774567215","cluster_addr":"172.21.15.175:6811/2774567215","heartbeat_back_addr":"172.21.15.175:6815/2774567215","heartbeat_front_addr":"172.21.15.175:6813/2774567215","state":["exists","up"]},{"osd":6,"uuid":"653da8e0-1197-4b48-abed-470f2a9c4dce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":1472215461},{"type":"v1","addr":"172.21.15.49:6819","nonce":1472215461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":1472215461},{"type":"v1","addr":"172.21.15.49:6821","nonce":1472215461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":1472215461},{"type":"v1","addr":"172.21.15.49:6825","nonce":1472215461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":1472215461},{"type":"v1","addr":"172.21.15.49:6823","nonce":1472215461}]},"public_addr":"172.21.15.49:6819/1472215461","cluster_addr":"172.21.15.49:6821/1472215461","heartbeat_back_addr":"172.21.15.49:6825/1472215461","heartbeat_front_addr":"172.21.15.49:6823/1472215461","state":["exists","up"]},{"osd":7,"uuid":"ad157c9b-4b75-451c-8740-630f0975b7fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":2163986787},{"type":"v1","addr":"172.21.15.139:6817","nonce":2163986787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":2163986787},{"type":"v1","addr":"172.21.15.139:6819","nonce":2163986787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":2163986787},{"type":"v1","addr":"172.21.15.139:6823","nonce":2163986787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":2163986787},{"type":"v1","addr":"172.21.15.139:6821","nonce":2163986787}]},"public_addr":"172.21.15.139:6817/2163986787","cluster_addr":"172.21.15.139:6819/2163986787","heartbeat_back_addr":"172.21.15.139:6823/2163986787","heartbeat_front_addr":"172.21.15.139:6821/2163986787","state":["exists","up"]},{"osd":8,"uuid":"c8e75f15-df75-4d5d-9b92-a064bffb6c01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":1194026992},{"type":"v1","addr":"172.21.15.175:6817","nonce":1194026992}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1194026992},{"type":"v1","addr":"172.21.15.175:6819","nonce":1194026992}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":1194026992},{"type":"v1","addr":"172.21.15.175:6823","nonce":1194026992}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1194026992},{"type":"v1","addr":"172.21.15.175:6821","nonce":1194026992}]},"public_addr":"172.21.15.175:6817/1194026992","cluster_addr":"172.21.15.175:6819/1194026992","heartbeat_back_addr":"172.21.15.175:6823/1194026992","heartbeat_front_addr":"172.21.15.175:6821/1194026992","state":["exists","up"]},{"osd":9,"uuid":"edb5ffef-b6a2-4941-b5ab-40f982ad30d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":2742084884},{"type":"v1","addr":"172.21.15.49:6827","nonce":2742084884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":2742084884},{"type":"v1","addr":"172.21.15.49:6829","nonce":2742084884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":2742084884},{"type":"v1","addr":"172.21.15.49:6833","nonce":2742084884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":2742084884},{"type":"v1","addr":"172.21.15.49:6831","nonce":2742084884}]},"public_addr":"172.21.15.49:6827/2742084884","cluster_addr":"172.21.15.49:6829/2742084884","heartbeat_back_addr":"172.21.15.49:6833/2742084884","heartbeat_front_addr":"172.21.15.49:6831/2742084884","state":["exists","up"]},{"osd":10,"uuid":"3945e46d-766a-4c4a-8821-d43648dd1578","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":2404740201},{"type":"v1","addr":"172.21.15.139:6825","nonce":2404740201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":2404740201},{"type":"v1","addr":"172.21.15.139:6827","nonce":2404740201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":2404740201},{"type":"v1","addr":"172.21.15.139:6831","nonce":2404740201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":2404740201},{"type":"v1","addr":"172.21.15.139:6829","nonce":2404740201}]},"public_addr":"172.21.15.139:6825/2404740201","cluster_addr":"172.21.15.139:6827/2404740201","heartbeat_back_addr":"172.21.15.139:6831/2404740201","heartbeat_front_addr":"172.21.15.139:6829/2404740201","state":["exists","up"]},{"osd":11,"uuid":"9954ba15-d877-46cc-b24a-893f9168deb8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":3519503868},{"type":"v1","addr":"172.21.15.175:6825","nonce":3519503868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":3519503868},{"type":"v1","addr":"172.21.15.175:6827","nonce":3519503868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":3519503868},{"type":"v1","addr":"172.21.15.175:6831","nonce":3519503868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":3519503868},{"type":"v1","addr":"172.21.15.175:6829","nonce":3519503868}]},"public_addr":"172.21.15.175:6825/3519503868","cluster_addr":"172.21.15.175:6827/3519503868","heartbeat_back_addr":"172.21.15.175:6831/3519503868","heartbeat_front_addr":"172.21.15.175:6829/3519503868","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:00:38.367865+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:01:02.237687+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:01:25.609751+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:01:51.268572+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:02:16.514605+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:02:41.586523+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:03:07.029964+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:03:32.956250+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:03:57.547984+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:04:23.948972+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:04:49.667210+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:05:15.284693+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.49:0/563527416":"2024-03-19T22:58:59.806253+0000","172.21.15.49:0/1036383534":"2024-03-19T22:58:59.806253+0000","172.21.15.49:6801/2609458951":"2024-03-19T22:58:59.806253+0000","172.21.15.49:0/21283102":"2024-03-19T22:58:31.972474+0000","172.21.15.49:0/1292058922":"2024-03-19T22:58:31.972474+0000","172.21.15.49:6800/1985190312":"2024-03-19T22:58:31.972474+0000","172.21.15.49:0/1912446700":"2024-03-19T22:58:59.806253+0000","172.21.15.49:0/2176210039":"2024-03-19T22:58:31.972474+0000","172.21.15.49:6801/1985190312":"2024-03-19T22:58:31.972474+0000","172.21.15.49:6800/2609458951":"2024-03-19T22:58:59.806253+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-18T23:06:13.146 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:13.146+0000 7f8df0b9d640 1 -- 172.21.15.49:0/727513553 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f8dd40a2800 msgr2=0x7f8dd40a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:13.146 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:13.146+0000 7f8df0b9d640 1 --2- 172.21.15.49:0/727513553 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f8dd40a2800 0x7f8dd40a4cc0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f8de002c7b0 tx=0x7f8de0005fd0 comp rx=0 tx=0).stop 2024-03-18T23:06:13.146 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:13.146+0000 7f8df0b9d640 1 -- 172.21.15.49:0/727513553 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8dec100570 msgr2=0x7f8dec196880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:13.146 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:13.146+0000 7f8df0b9d640 1 --2- 172.21.15.49:0/727513553 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8dec100570 0x7f8dec196880 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f8ddc00e9c0 tx=0x7f8ddc00ee80 comp rx=0 tx=0).stop 2024-03-18T23:06:13.146 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:13.146+0000 7f8df0b9d640 1 -- 172.21.15.49:0/727513553 shutdown_connections 2024-03-18T23:06:13.146 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:13.146+0000 7f8df0b9d640 1 --2- 172.21.15.49:0/727513553 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f8dd40a2800 0x7f8dd40a4cc0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:13.146 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:13.146+0000 7f8df0b9d640 1 --2- 172.21.15.49:0/727513553 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8dec102120 0x7f8dec19de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:13.146 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:13.146+0000 7f8df0b9d640 1 --2- 172.21.15.49:0/727513553 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8dec101760 0x7f8dec196dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:13.146 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:13.146+0000 7f8df0b9d640 1 --2- 172.21.15.49:0/727513553 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8dec100570 0x7f8dec196880 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:13.147 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:13.146+0000 7f8df0b9d640 1 -- 172.21.15.49:0/727513553 >> 172.21.15.49:0/727513553 conn(0x7f8dec0fbb00 msgr2=0x7f8dec104a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:06:13.147 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:13.146+0000 7f8df0b9d640 1 -- 172.21.15.49:0/727513553 shutdown_connections 2024-03-18T23:06:13.147 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:13.146+0000 7f8df0b9d640 1 -- 172.21.15.49:0/727513553 wait complete. 2024-03-18T23:06:13.157 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-03-18T23:06:13.157 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-18T23:06:13.278 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:13.275+0000 7f9bc7fc4640 1 -- 172.21.15.49:0/3645675034 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9bc01007c0 msgr2=0x7f9bc0100be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:13.278 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:13.275+0000 7f9bc7fc4640 1 --2- 172.21.15.49:0/3645675034 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9bc01007c0 0x7f9bc0100be0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f9bbc009900 tx=0x7f9bbc02bc30 comp rx=0 tx=0).stop 2024-03-18T23:06:13.278 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:13.275+0000 7f9bc7fc4640 1 -- 172.21.15.49:0/3645675034 shutdown_connections 2024-03-18T23:06:13.279 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:13.275+0000 7f9bc7fc4640 1 --2- 172.21.15.49:0/3645675034 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9bc0102370 0x7f9bc0108c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:13.279 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:13.275+0000 7f9bc7fc4640 1 --2- 172.21.15.49:0/3645675034 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9bc01019b0 0x7f9bc0101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:13.279 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:13.275+0000 7f9bc7fc4640 1 --2- 172.21.15.49:0/3645675034 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9bc01007c0 0x7f9bc0100be0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:13.279 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:13.275+0000 7f9bc7fc4640 1 -- 172.21.15.49:0/3645675034 >> 172.21.15.49:0/3645675034 conn(0x7f9bc00fbd50 msgr2=0x7f9bc00fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:06:13.279 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:13.275+0000 7f9bc7fc4640 1 -- 172.21.15.49:0/3645675034 shutdown_connections 2024-03-18T23:06:13.279 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:13.275+0000 7f9bc7fc4640 1 -- 172.21.15.49:0/3645675034 wait complete. 2024-03-18T23:06:13.279 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:13.275+0000 7f9bc7fc4640 1 Processor -- start 2024-03-18T23:06:13.279 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:13.275+0000 7f9bc7fc4640 1 -- start start 2024-03-18T23:06:13.279 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:13.276+0000 7f9bc7fc4640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9bc01007c0 0x7f9bc006a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:06:13.279 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:13.276+0000 7f9bc7fc4640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9bc01019b0 0x7f9bc019c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:06:13.279 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:13.276+0000 7f9bc7fc4640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9bc0102370 0x7f9bc019ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:06:13.280 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:13.276+0000 7f9bc7fc4640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f9bc010c3c0 con 0x7f9bc01019b0 2024-03-18T23:06:13.280 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:13.276+0000 7f9bc7fc4640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f9bc010c240 con 0x7f9bc0102370 2024-03-18T23:06:13.280 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:13.276+0000 7f9bc7fc4640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f9bc010c540 con 0x7f9bc01007c0 2024-03-18T23:06:13.280 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:13.276+0000 7f9bc555f640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9bc01019b0 0x7f9bc019c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:06:13.280 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:13.276+0000 7f9bc555f640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9bc01019b0 0x7f9bc019c8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:54326/0 (socket says 172.21.15.49:54326) 2024-03-18T23:06:13.280 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:13.276+0000 7f9bc555f640 1 -- 172.21.15.49:0/1087177525 learned_addr learned my addr 172.21.15.49:0/1087177525 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:06:13.280 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:13.276+0000 7f9bc5d60640 1 --2- 172.21.15.49:0/1087177525 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9bc01007c0 0x7f9bc006a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:06:13.280 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:13.276+0000 7f9bc6561640 1 --2- 172.21.15.49:0/1087177525 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9bc0102370 0x7f9bc019ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:06:13.280 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:13.276+0000 7f9bc555f640 1 -- 172.21.15.49:0/1087177525 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9bc01007c0 msgr2=0x7f9bc006a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:13.280 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:13.276+0000 7f9bc555f640 1 --2- 172.21.15.49:0/1087177525 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9bc01007c0 0x7f9bc006a9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:13.280 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:13.276+0000 7f9bc555f640 1 -- 172.21.15.49:0/1087177525 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9bc0102370 msgr2=0x7f9bc019ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:13.281 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:13.276+0000 7f9bc555f640 1 --2- 172.21.15.49:0/1087177525 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9bc0102370 0x7f9bc019ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:13.281 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:13.276+0000 7f9bc555f640 1 -- 172.21.15.49:0/1087177525 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9bc019f810 con 0x7f9bc01019b0 2024-03-18T23:06:13.281 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:13.276+0000 7f9bc555f640 1 --2- 172.21.15.49:0/1087177525 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9bc01019b0 0x7f9bc019c8e0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f9bb000e9f0 tx=0x7f9bb000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:06:13.281 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:13.276+0000 7f9bb6ffd640 1 -- 172.21.15.49:0/1087177525 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bb000cd80 con 0x7f9bc01019b0 2024-03-18T23:06:13.281 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:13.276+0000 7f9bb6ffd640 1 -- 172.21.15.49:0/1087177525 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9bb0005500 con 0x7f9bc01019b0 2024-03-18T23:06:13.281 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:13.276+0000 7f9bc7fc4640 1 -- 172.21.15.49:0/1087177525 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9bc019faa0 con 0x7f9bc01019b0 2024-03-18T23:06:13.281 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:13.276+0000 7f9bb6ffd640 1 -- 172.21.15.49:0/1087177525 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bb0010690 con 0x7f9bc01019b0 2024-03-18T23:06:13.281 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:13.276+0000 7f9bc7fc4640 1 -- 172.21.15.49:0/1087177525 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9bc019ffa0 con 0x7f9bc01019b0 2024-03-18T23:06:13.281 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:13.278+0000 7f9bc7fc4640 1 -- 172.21.15.49:0/1087177525 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9bc0107b60 con 0x7f9bc01019b0 2024-03-18T23:06:13.282 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:13.279+0000 7f9bb6ffd640 1 -- 172.21.15.49:0/1087177525 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 17) ==== 137341+0+0 (secure 0 0 0) 0x7f9bb00046c0 con 0x7f9bc01019b0 2024-03-18T23:06:13.282 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:13.279+0000 7f9bb6ffd640 1 --2- 172.21.15.49:0/1087177525 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f9bac0a2890 0x7f9bac0a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:06:13.282 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:13.279+0000 7f9bc5d60640 1 --2- 172.21.15.49:0/1087177525 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f9bac0a2890 0x7f9bac0a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:06:13.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:13.279+0000 7f9bb6ffd640 1 -- 172.21.15.49:0/1087177525 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f9bb00cee20 con 0x7f9bc01019b0 2024-03-18T23:06:13.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:13.280+0000 7f9bc5d60640 1 --2- 172.21.15.49:0/1087177525 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f9bac0a2890 0x7f9bac0a4d50 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f9bbc000c00 tx=0x7f9bbc005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:06:13.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:13.284+0000 7f9bb6ffd640 1 -- 172.21.15.49:0/1087177525 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f9bb009bf10 con 0x7f9bc01019b0 2024-03-18T23:06:13.466 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:13.466+0000 7f9bc7fc4640 1 -- 172.21.15.49:0/1087177525 --> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f9bc0061c60 con 0x7f9bac0a2890 2024-03-18T23:06:13.473 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:13.473+0000 7f9bb6ffd640 1 -- 172.21.15.49:0/1087177525 <== mgr.14150 v2:172.21.15.49:6800/2083805211 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258907 (secure 0 0 0) 0x7f9bc0061c60 con 0x7f9bac0a2890 2024-03-18T23:06:13.479 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-18T23:06:13.482 INFO:teuthology.orchestra.run.smithi049.stderr:dumped all 2024-03-18T23:06:13.485 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:13.476+0000 7f9bc7fc4640 1 -- 172.21.15.49:0/1087177525 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f9bac0a2890 msgr2=0x7f9bac0a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:13.485 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:13.476+0000 7f9bc7fc4640 1 --2- 172.21.15.49:0/1087177525 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f9bac0a2890 0x7f9bac0a4d50 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f9bbc000c00 tx=0x7f9bbc005fd0 comp rx=0 tx=0).stop 2024-03-18T23:06:13.485 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:13.476+0000 7f9bc7fc4640 1 -- 172.21.15.49:0/1087177525 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9bc01019b0 msgr2=0x7f9bc019c8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:13.485 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:13.476+0000 7f9bc7fc4640 1 --2- 172.21.15.49:0/1087177525 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9bc01019b0 0x7f9bc019c8e0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f9bb000e9f0 tx=0x7f9bb000edf0 comp rx=0 tx=0).stop 2024-03-18T23:06:13.485 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:13.476+0000 7f9bc7fc4640 1 -- 172.21.15.49:0/1087177525 shutdown_connections 2024-03-18T23:06:13.485 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:13.476+0000 7f9bc7fc4640 1 --2- 172.21.15.49:0/1087177525 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f9bac0a2890 0x7f9bac0a4d50 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:13.486 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:13.476+0000 7f9bc7fc4640 1 --2- 172.21.15.49:0/1087177525 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9bc0102370 0x7f9bc019ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:13.486 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:13.476+0000 7f9bc7fc4640 1 --2- 172.21.15.49:0/1087177525 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9bc01019b0 0x7f9bc019c8e0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:13.486 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:13.476+0000 7f9bc7fc4640 1 --2- 172.21.15.49:0/1087177525 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9bc01007c0 0x7f9bc006a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:13.486 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:13.477+0000 7f9bc7fc4640 1 -- 172.21.15.49:0/1087177525 >> 172.21.15.49:0/1087177525 conn(0x7f9bc00fbd50 msgr2=0x7f9bc0104d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:06:13.486 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:13.477+0000 7f9bc7fc4640 1 -- 172.21.15.49:0/1087177525 shutdown_connections 2024-03-18T23:06:13.486 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:13.477+0000 7f9bc7fc4640 1 -- 172.21.15.49:0/1087177525 wait complete. 2024-03-18T23:06:13.487 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:06:13 smithi049 ceph-mon[62991]: pgmap v292: 97 pgs: 97 active+clean; 600 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 7.7 KiB/s wr, 21 op/s 2024-03-18T23:06:13.487 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:06:13 smithi049 ceph-mon[62991]: from='client.14751 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T23:06:13.487 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:06:13 smithi049 ceph-mon[62991]: from='client.14757 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T23:06:13.487 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:06:13 smithi049 ceph-mon[62991]: from='client.? 172.21.15.49:0/2244712381' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T23:06:13.487 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:06:13 smithi049 ceph-mon[62991]: from='client.? 172.21.15.49:0/727513553' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T23:06:13.488 INFO:teuthology.orchestra.run.smithi049.stdout:{"pg_ready":true,"pg_map":{"version":292,"stamp":"2024-03-18T23:06:11.898416+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":614408,"num_objects":67,"num_object_clones":0,"num_object_copies":201,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67,"num_whiteouts":0,"num_read":154,"num_read_kb":134,"num_write":328,"num_write_kb":2997,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":45,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":264,"ondisk_log_size":264,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":368712,"kb_used_data":11200,"kb_used_omap":18,"kb_used_meta":357357,"kb_avail":1124573112,"statfs":{"total":1151940427776,"available":1151562866688,"internally_reserved":0,"allocated":11468800,"data_stored":7121888,"data_compressed":108345,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":19142,"internal_metadata":365933882},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":24040,"num_objects":65,"num_object_clones":0,"num_object_copies":195,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":65,"num_whiteouts":0,"num_read":18,"num_read_kb":15,"num_write":147,"num_write_kb":59,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":45,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.701179"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T23:06:06.426899+0000","last_change":"2024-03-18T23:06:04.913788+0000","last_active":"2024-03-18T23:06:06.426899+0000","last_peered":"2024-03-18T23:06:06.426899+0000","last_clean":"2024-03-18T23:06:06.426899+0000","last_became_active":"2024-03-18T23:06:04.913103+0000","last_became_peered":"2024-03-18T23:06:04.913103+0000","last_unstale":"2024-03-18T23:06:06.426899+0000","last_undegraded":"2024-03-18T23:06:06.426899+0000","last_fullsized":"2024-03-18T23:06:06.426899+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:48:10.100291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T23:06:06.398424+0000","last_change":"2024-03-18T23:06:04.921126+0000","last_active":"2024-03-18T23:06:06.398424+0000","last_peered":"2024-03-18T23:06:06.398424+0000","last_clean":"2024-03-18T23:06:06.398424+0000","last_became_active":"2024-03-18T23:06:04.919934+0000","last_became_peered":"2024-03-18T23:06:04.919934+0000","last_unstale":"2024-03-18T23:06:06.398424+0000","last_undegraded":"2024-03-18T23:06:06.398424+0000","last_fullsized":"2024-03-18T23:06:06.398424+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:24:25.351428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T23:06:06.398267+0000","last_change":"2024-03-18T23:06:04.920933+0000","last_active":"2024-03-18T23:06:06.398267+0000","last_peered":"2024-03-18T23:06:06.398267+0000","last_clean":"2024-03-18T23:06:06.398267+0000","last_became_active":"2024-03-18T23:06:04.920548+0000","last_became_peered":"2024-03-18T23:06:04.920548+0000","last_unstale":"2024-03-18T23:06:06.398267+0000","last_undegraded":"2024-03-18T23:06:06.398267+0000","last_fullsized":"2024-03-18T23:06:06.398267+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:06:01.746208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T23:06:05.896767+0000","last_change":"2024-03-18T23:06:04.921219+0000","last_active":"2024-03-18T23:06:05.896767+0000","last_peered":"2024-03-18T23:06:05.896767+0000","last_clean":"2024-03-18T23:06:05.896767+0000","last_became_active":"2024-03-18T23:06:04.920069+0000","last_became_peered":"2024-03-18T23:06:04.920069+0000","last_unstale":"2024-03-18T23:06:05.896767+0000","last_undegraded":"2024-03-18T23:06:05.896767+0000","last_fullsized":"2024-03-18T23:06:05.896767+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:22:44.463061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T23:06:06.223923+0000","last_change":"2024-03-18T23:06:04.916741+0000","last_active":"2024-03-18T23:06:06.223923+0000","last_peered":"2024-03-18T23:06:06.223923+0000","last_clean":"2024-03-18T23:06:06.223923+0000","last_became_active":"2024-03-18T23:06:04.916434+0000","last_became_peered":"2024-03-18T23:06:04.916434+0000","last_unstale":"2024-03-18T23:06:06.223923+0000","last_undegraded":"2024-03-18T23:06:06.223923+0000","last_fullsized":"2024-03-18T23:06:06.223923+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:45:10.553306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T23:06:06.398243+0000","last_change":"2024-03-18T23:06:04.917198+0000","last_active":"2024-03-18T23:06:06.398243+0000","last_peered":"2024-03-18T23:06:06.398243+0000","last_clean":"2024-03-18T23:06:06.398243+0000","last_became_active":"2024-03-18T23:06:04.916897+0000","last_became_peered":"2024-03-18T23:06:04.916897+0000","last_unstale":"2024-03-18T23:06:06.398243+0000","last_undegraded":"2024-03-18T23:06:06.398243+0000","last_fullsized":"2024-03-18T23:06:06.398243+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T23:23:26.530668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T23:06:06.397867+0000","last_change":"2024-03-18T23:06:04.908446+0000","last_active":"2024-03-18T23:06:06.397867+0000","last_peered":"2024-03-18T23:06:06.397867+0000","last_clean":"2024-03-18T23:06:06.397867+0000","last_became_active":"2024-03-18T23:06:04.908093+0000","last_became_peered":"2024-03-18T23:06:04.908093+0000","last_unstale":"2024-03-18T23:06:06.397867+0000","last_undegraded":"2024-03-18T23:06:06.397867+0000","last_fullsized":"2024-03-18T23:06:06.397867+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T09:21:30.253229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T23:06:05.895448+0000","last_change":"2024-03-18T23:06:04.913080+0000","last_active":"2024-03-18T23:06:05.895448+0000","last_peered":"2024-03-18T23:06:05.895448+0000","last_clean":"2024-03-18T23:06:05.895448+0000","last_became_active":"2024-03-18T23:06:04.912255+0000","last_became_peered":"2024-03-18T23:06:04.912255+0000","last_unstale":"2024-03-18T23:06:05.895448+0000","last_undegraded":"2024-03-18T23:06:05.895448+0000","last_fullsized":"2024-03-18T23:06:05.895448+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:16:36.069545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T23:06:06.398207+0000","last_change":"2024-03-18T23:06:04.905284+0000","last_active":"2024-03-18T23:06:06.398207+0000","last_peered":"2024-03-18T23:06:06.398207+0000","last_clean":"2024-03-18T23:06:06.398207+0000","last_became_active":"2024-03-18T23:06:04.904388+0000","last_became_peered":"2024-03-18T23:06:04.904388+0000","last_unstale":"2024-03-18T23:06:06.398207+0000","last_undegraded":"2024-03-18T23:06:06.398207+0000","last_fullsized":"2024-03-18T23:06:06.398207+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:57:56.628599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T23:06:06.398697+0000","last_change":"2024-03-18T23:06:04.914602+0000","last_active":"2024-03-18T23:06:06.398697+0000","last_peered":"2024-03-18T23:06:06.398697+0000","last_clean":"2024-03-18T23:06:06.398697+0000","last_became_active":"2024-03-18T23:06:04.914288+0000","last_became_peered":"2024-03-18T23:06:04.914288+0000","last_unstale":"2024-03-18T23:06:06.398697+0000","last_undegraded":"2024-03-18T23:06:06.398697+0000","last_fullsized":"2024-03-18T23:06:06.398697+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:44:52.182950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T23:06:06.398239+0000","last_change":"2024-03-18T23:06:04.919821+0000","last_active":"2024-03-18T23:06:06.398239+0000","last_peered":"2024-03-18T23:06:06.398239+0000","last_clean":"2024-03-18T23:06:06.398239+0000","last_became_active":"2024-03-18T23:06:04.919525+0000","last_became_peered":"2024-03-18T23:06:04.919525+0000","last_unstale":"2024-03-18T23:06:06.398239+0000","last_undegraded":"2024-03-18T23:06:06.398239+0000","last_fullsized":"2024-03-18T23:06:06.398239+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:54:50.720229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T23:06:05.896703+0000","last_change":"2024-03-18T23:06:04.921904+0000","last_active":"2024-03-18T23:06:05.896703+0000","last_peered":"2024-03-18T23:06:05.896703+0000","last_clean":"2024-03-18T23:06:05.896703+0000","last_became_active":"2024-03-18T23:06:04.921581+0000","last_became_peered":"2024-03-18T23:06:04.921581+0000","last_unstale":"2024-03-18T23:06:05.896703+0000","last_undegraded":"2024-03-18T23:06:05.896703+0000","last_fullsized":"2024-03-18T23:06:05.896703+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:43:55.580983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T23:06:06.398062+0000","last_change":"2024-03-18T23:06:04.905399+0000","last_active":"2024-03-18T23:06:06.398062+0000","last_peered":"2024-03-18T23:06:06.398062+0000","last_clean":"2024-03-18T23:06:06.398062+0000","last_became_active":"2024-03-18T23:06:04.905117+0000","last_became_peered":"2024-03-18T23:06:04.905117+0000","last_unstale":"2024-03-18T23:06:06.398062+0000","last_undegraded":"2024-03-18T23:06:06.398062+0000","last_fullsized":"2024-03-18T23:06:06.398062+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:41:56.382244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T23:06:06.398151+0000","last_change":"2024-03-18T23:06:04.909993+0000","last_active":"2024-03-18T23:06:06.398151+0000","last_peered":"2024-03-18T23:06:06.398151+0000","last_clean":"2024-03-18T23:06:06.398151+0000","last_became_active":"2024-03-18T23:06:04.909483+0000","last_became_peered":"2024-03-18T23:06:04.909483+0000","last_unstale":"2024-03-18T23:06:06.398151+0000","last_undegraded":"2024-03-18T23:06:06.398151+0000","last_fullsized":"2024-03-18T23:06:06.398151+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:36:52.826582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T23:06:06.397933+0000","last_change":"2024-03-18T23:06:04.915273+0000","last_active":"2024-03-18T23:06:06.397933+0000","last_peered":"2024-03-18T23:06:06.397933+0000","last_clean":"2024-03-18T23:06:06.397933+0000","last_became_active":"2024-03-18T23:06:04.913510+0000","last_became_peered":"2024-03-18T23:06:04.913510+0000","last_unstale":"2024-03-18T23:06:06.397933+0000","last_undegraded":"2024-03-18T23:06:06.397933+0000","last_fullsized":"2024-03-18T23:06:06.397933+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:46:01.622963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T23:06:06.398462+0000","last_change":"2024-03-18T23:06:04.917599+0000","last_active":"2024-03-18T23:06:06.398462+0000","last_peered":"2024-03-18T23:06:06.398462+0000","last_clean":"2024-03-18T23:06:06.398462+0000","last_became_active":"2024-03-18T23:06:04.914961+0000","last_became_peered":"2024-03-18T23:06:04.914961+0000","last_unstale":"2024-03-18T23:06:06.398462+0000","last_undegraded":"2024-03-18T23:06:06.398462+0000","last_fullsized":"2024-03-18T23:06:06.398462+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T09:38:50.076075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T23:06:06.426992+0000","last_change":"2024-03-18T23:06:04.920814+0000","last_active":"2024-03-18T23:06:06.426992+0000","last_peered":"2024-03-18T23:06:06.426992+0000","last_clean":"2024-03-18T23:06:06.426992+0000","last_became_active":"2024-03-18T23:06:04.920484+0000","last_became_peered":"2024-03-18T23:06:04.920484+0000","last_unstale":"2024-03-18T23:06:06.426992+0000","last_undegraded":"2024-03-18T23:06:06.426992+0000","last_fullsized":"2024-03-18T23:06:06.426992+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:45:01.545059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T23:06:06.222283+0000","last_change":"2024-03-18T23:06:04.904684+0000","last_active":"2024-03-18T23:06:06.222283+0000","last_peered":"2024-03-18T23:06:06.222283+0000","last_clean":"2024-03-18T23:06:06.222283+0000","last_became_active":"2024-03-18T23:06:04.904296+0000","last_became_peered":"2024-03-18T23:06:04.904296+0000","last_unstale":"2024-03-18T23:06:06.222283+0000","last_undegraded":"2024-03-18T23:06:06.222283+0000","last_fullsized":"2024-03-18T23:06:06.222283+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:26:28.098286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T23:06:06.397802+0000","last_change":"2024-03-18T23:06:04.912803+0000","last_active":"2024-03-18T23:06:06.397802+0000","last_peered":"2024-03-18T23:06:06.397802+0000","last_clean":"2024-03-18T23:06:06.397802+0000","last_became_active":"2024-03-18T23:06:04.912495+0000","last_became_peered":"2024-03-18T23:06:04.912495+0000","last_unstale":"2024-03-18T23:06:06.397802+0000","last_undegraded":"2024-03-18T23:06:06.397802+0000","last_fullsized":"2024-03-18T23:06:06.397802+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:47:38.111260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"86'11","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T23:06:09.998428+0000","last_change":"2024-03-18T23:06:04.221492+0000","last_active":"2024-03-18T23:06:09.998428+0000","last_peered":"2024-03-18T23:06:09.998428+0000","last_clean":"2024-03-18T23:06:09.998428+0000","last_became_active":"2024-03-18T23:06:04.221190+0000","last_became_peered":"2024-03-18T23:06:04.221190+0000","last_unstale":"2024-03-18T23:06:09.998428+0000","last_undegraded":"2024-03-18T23:06:09.998428+0000","last_fullsized":"2024-03-18T23:06:09.998428+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_clean_scrub_stamp":"2024-03-18T23:06:03.184668+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:45:08.747471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T23:06:05.895828+0000","last_change":"2024-03-18T23:06:04.911680+0000","last_active":"2024-03-18T23:06:05.895828+0000","last_peered":"2024-03-18T23:06:05.895828+0000","last_clean":"2024-03-18T23:06:05.895828+0000","last_became_active":"2024-03-18T23:06:04.911356+0000","last_became_peered":"2024-03-18T23:06:04.911356+0000","last_unstale":"2024-03-18T23:06:05.895828+0000","last_undegraded":"2024-03-18T23:06:05.895828+0000","last_fullsized":"2024-03-18T23:06:05.895828+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:23:21.943304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"85'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T23:06:05.895726+0000","last_change":"2024-03-18T23:06:04.225792+0000","last_active":"2024-03-18T23:06:05.895726+0000","last_peered":"2024-03-18T23:06:05.895726+0000","last_clean":"2024-03-18T23:06:05.895726+0000","last_became_active":"2024-03-18T23:06:04.224248+0000","last_became_peered":"2024-03-18T23:06:04.224248+0000","last_unstale":"2024-03-18T23:06:05.895726+0000","last_undegraded":"2024-03-18T23:06:05.895726+0000","last_fullsized":"2024-03-18T23:06:05.895726+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_clean_scrub_stamp":"2024-03-18T23:06:03.184668+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:48:27.052831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T23:06:06.398518+0000","last_change":"2024-03-18T23:06:04.921139+0000","last_active":"2024-03-18T23:06:06.398518+0000","last_peered":"2024-03-18T23:06:06.398518+0000","last_clean":"2024-03-18T23:06:06.398518+0000","last_became_active":"2024-03-18T23:06:04.920700+0000","last_became_peered":"2024-03-18T23:06:04.920700+0000","last_unstale":"2024-03-18T23:06:06.398518+0000","last_undegraded":"2024-03-18T23:06:06.398518+0000","last_fullsized":"2024-03-18T23:06:06.398518+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T08:47:34.465176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"86'7","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T23:06:09.999475+0000","last_change":"2024-03-18T23:06:04.214134+0000","last_active":"2024-03-18T23:06:09.999475+0000","last_peered":"2024-03-18T23:06:09.999475+0000","last_clean":"2024-03-18T23:06:09.999475+0000","last_became_active":"2024-03-18T23:06:04.212256+0000","last_became_peered":"2024-03-18T23:06:04.212256+0000","last_unstale":"2024-03-18T23:06:09.999475+0000","last_undegraded":"2024-03-18T23:06:09.999475+0000","last_fullsized":"2024-03-18T23:06:09.999475+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_clean_scrub_stamp":"2024-03-18T23:06:03.184668+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:39:46.851964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T23:06:06.397748+0000","last_change":"2024-03-18T23:06:04.907465+0000","last_active":"2024-03-18T23:06:06.397748+0000","last_peered":"2024-03-18T23:06:06.397748+0000","last_clean":"2024-03-18T23:06:06.397748+0000","last_became_active":"2024-03-18T23:06:04.907142+0000","last_became_peered":"2024-03-18T23:06:04.907142+0000","last_unstale":"2024-03-18T23:06:06.397748+0000","last_undegraded":"2024-03-18T23:06:06.397748+0000","last_fullsized":"2024-03-18T23:06:06.397748+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:15:29.066708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T23:06:06.224063+0000","last_change":"2024-03-18T23:06:04.213694+0000","last_active":"2024-03-18T23:06:06.224063+0000","last_peered":"2024-03-18T23:06:06.224063+0000","last_clean":"2024-03-18T23:06:06.224063+0000","last_became_active":"2024-03-18T23:06:04.213381+0000","last_became_peered":"2024-03-18T23:06:04.213381+0000","last_unstale":"2024-03-18T23:06:06.224063+0000","last_undegraded":"2024-03-18T23:06:06.224063+0000","last_fullsized":"2024-03-18T23:06:06.224063+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_clean_scrub_stamp":"2024-03-18T23:06:03.184668+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:08:44.502915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T23:06:05.896172+0000","last_change":"2024-03-18T23:06:04.918596+0000","last_active":"2024-03-18T23:06:05.896172+0000","last_peered":"2024-03-18T23:06:05.896172+0000","last_clean":"2024-03-18T23:06:05.896172+0000","last_became_active":"2024-03-18T23:06:04.918273+0000","last_became_peered":"2024-03-18T23:06:04.918273+0000","last_unstale":"2024-03-18T23:06:05.896172+0000","last_undegraded":"2024-03-18T23:06:05.896172+0000","last_fullsized":"2024-03-18T23:06:05.896172+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T08:09:04.860241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"86'5","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T23:06:08.250233+0000","last_change":"2024-03-18T23:06:04.214573+0000","last_active":"2024-03-18T23:06:08.250233+0000","last_peered":"2024-03-18T23:06:08.250233+0000","last_clean":"2024-03-18T23:06:08.250233+0000","last_became_active":"2024-03-18T23:06:04.213891+0000","last_became_peered":"2024-03-18T23:06:04.213891+0000","last_unstale":"2024-03-18T23:06:08.250233+0000","last_undegraded":"2024-03-18T23:06:08.250233+0000","last_fullsized":"2024-03-18T23:06:08.250233+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_clean_scrub_stamp":"2024-03-18T23:06:03.184668+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:17:22.895024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T23:06:06.223967+0000","last_change":"2024-03-18T23:06:04.908336+0000","last_active":"2024-03-18T23:06:06.223967+0000","last_peered":"2024-03-18T23:06:06.223967+0000","last_clean":"2024-03-18T23:06:06.223967+0000","last_became_active":"2024-03-18T23:06:04.908023+0000","last_became_peered":"2024-03-18T23:06:04.908023+0000","last_unstale":"2024-03-18T23:06:06.223967+0000","last_undegraded":"2024-03-18T23:06:06.223967+0000","last_fullsized":"2024-03-18T23:06:06.223967+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:34:50.993715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"86'7","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T23:06:10.000008+0000","last_change":"2024-03-18T23:06:04.210036+0000","last_active":"2024-03-18T23:06:10.000008+0000","last_peered":"2024-03-18T23:06:10.000008+0000","last_clean":"2024-03-18T23:06:10.000008+0000","last_became_active":"2024-03-18T23:06:04.209658+0000","last_became_peered":"2024-03-18T23:06:04.209658+0000","last_unstale":"2024-03-18T23:06:10.000008+0000","last_undegraded":"2024-03-18T23:06:10.000008+0000","last_fullsized":"2024-03-18T23:06:10.000008+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_clean_scrub_stamp":"2024-03-18T23:06:03.184668+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T23:55:37.286542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T23:06:05.896078+0000","last_change":"2024-03-18T23:06:04.917909+0000","last_active":"2024-03-18T23:06:05.896078+0000","last_peered":"2024-03-18T23:06:05.896078+0000","last_clean":"2024-03-18T23:06:05.896078+0000","last_became_active":"2024-03-18T23:06:04.917305+0000","last_became_peered":"2024-03-18T23:06:04.917305+0000","last_unstale":"2024-03-18T23:06:05.896078+0000","last_undegraded":"2024-03-18T23:06:05.896078+0000","last_fullsized":"2024-03-18T23:06:05.896078+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:39:48.953530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"86'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T23:06:08.996518+0000","last_change":"2024-03-18T23:06:04.216030+0000","last_active":"2024-03-18T23:06:08.996518+0000","last_peered":"2024-03-18T23:06:08.996518+0000","last_clean":"2024-03-18T23:06:08.996518+0000","last_became_active":"2024-03-18T23:06:04.215695+0000","last_became_peered":"2024-03-18T23:06:04.215695+0000","last_unstale":"2024-03-18T23:06:08.996518+0000","last_undegraded":"2024-03-18T23:06:08.996518+0000","last_fullsized":"2024-03-18T23:06:08.996518+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_clean_scrub_stamp":"2024-03-18T23:06:03.184668+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:48:37.742916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T23:06:05.895518+0000","last_change":"2024-03-18T23:06:04.910749+0000","last_active":"2024-03-18T23:06:05.895518+0000","last_peered":"2024-03-18T23:06:05.895518+0000","last_clean":"2024-03-18T23:06:05.895518+0000","last_became_active":"2024-03-18T23:06:04.910313+0000","last_became_peered":"2024-03-18T23:06:04.910313+0000","last_unstale":"2024-03-18T23:06:05.895518+0000","last_undegraded":"2024-03-18T23:06:05.895518+0000","last_fullsized":"2024-03-18T23:06:05.895518+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:18:19.726537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"86'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T23:06:05.895679+0000","last_change":"2024-03-18T23:06:04.218263+0000","last_active":"2024-03-18T23:06:05.895679+0000","last_peered":"2024-03-18T23:06:05.895679+0000","last_clean":"2024-03-18T23:06:05.895679+0000","last_became_active":"2024-03-18T23:06:04.217943+0000","last_became_peered":"2024-03-18T23:06:04.217943+0000","last_unstale":"2024-03-18T23:06:05.895679+0000","last_undegraded":"2024-03-18T23:06:05.895679+0000","last_fullsized":"2024-03-18T23:06:05.895679+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_clean_scrub_stamp":"2024-03-18T23:06:03.184668+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:25:11.632767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T23:06:06.222012+0000","last_change":"2024-03-18T23:06:04.915905+0000","last_active":"2024-03-18T23:06:06.222012+0000","last_peered":"2024-03-18T23:06:06.222012+0000","last_clean":"2024-03-18T23:06:06.222012+0000","last_became_active":"2024-03-18T23:06:04.915584+0000","last_became_peered":"2024-03-18T23:06:04.915584+0000","last_unstale":"2024-03-18T23:06:06.222012+0000","last_undegraded":"2024-03-18T23:06:06.222012+0000","last_fullsized":"2024-03-18T23:06:06.222012+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:29:05.503861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"86'4","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T23:06:09.998963+0000","last_change":"2024-03-18T23:06:04.217077+0000","last_active":"2024-03-18T23:06:09.998963+0000","last_peered":"2024-03-18T23:06:09.998963+0000","last_clean":"2024-03-18T23:06:09.998963+0000","last_became_active":"2024-03-18T23:06:04.216308+0000","last_became_peered":"2024-03-18T23:06:04.216308+0000","last_unstale":"2024-03-18T23:06:09.998963+0000","last_undegraded":"2024-03-18T23:06:09.998963+0000","last_fullsized":"2024-03-18T23:06:09.998963+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_clean_scrub_stamp":"2024-03-18T23:06:03.184668+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T23:39:58.537740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T23:06:06.398686+0000","last_change":"2024-03-18T23:06:04.906669+0000","last_active":"2024-03-18T23:06:06.398686+0000","last_peered":"2024-03-18T23:06:06.398686+0000","last_clean":"2024-03-18T23:06:06.398686+0000","last_became_active":"2024-03-18T23:06:04.906338+0000","last_became_peered":"2024-03-18T23:06:04.906338+0000","last_unstale":"2024-03-18T23:06:06.398686+0000","last_undegraded":"2024-03-18T23:06:06.398686+0000","last_fullsized":"2024-03-18T23:06:06.398686+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:29:39.997671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"86'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T23:06:08.249231+0000","last_change":"2024-03-18T23:06:04.216177+0000","last_active":"2024-03-18T23:06:08.249231+0000","last_peered":"2024-03-18T23:06:08.249231+0000","last_clean":"2024-03-18T23:06:08.249231+0000","last_became_active":"2024-03-18T23:06:04.215890+0000","last_became_peered":"2024-03-18T23:06:04.215890+0000","last_unstale":"2024-03-18T23:06:08.249231+0000","last_undegraded":"2024-03-18T23:06:08.249231+0000","last_fullsized":"2024-03-18T23:06:08.249231+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_clean_scrub_stamp":"2024-03-18T23:06:03.184668+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T09:33:52.578996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T23:06:06.398307+0000","last_change":"2024-03-18T23:06:04.911928+0000","last_active":"2024-03-18T23:06:06.398307+0000","last_peered":"2024-03-18T23:06:06.398307+0000","last_clean":"2024-03-18T23:06:06.398307+0000","last_became_active":"2024-03-18T23:06:04.911501+0000","last_became_peered":"2024-03-18T23:06:04.911501+0000","last_unstale":"2024-03-18T23:06:06.398307+0000","last_undegraded":"2024-03-18T23:06:06.398307+0000","last_fullsized":"2024-03-18T23:06:06.398307+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:05:17.653574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"86'4","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T23:06:05.896036+0000","last_change":"2024-03-18T23:06:04.211035+0000","last_active":"2024-03-18T23:06:05.896036+0000","last_peered":"2024-03-18T23:06:05.896036+0000","last_clean":"2024-03-18T23:06:05.896036+0000","last_became_active":"2024-03-18T23:06:04.210665+0000","last_became_peered":"2024-03-18T23:06:04.210665+0000","last_unstale":"2024-03-18T23:06:05.896036+0000","last_undegraded":"2024-03-18T23:06:05.896036+0000","last_fullsized":"2024-03-18T23:06:05.896036+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_clean_scrub_stamp":"2024-03-18T23:06:03.184668+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:49:44.072275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T23:06:06.398050+0000","last_change":"2024-03-18T23:06:04.911856+0000","last_active":"2024-03-18T23:06:06.398050+0000","last_peered":"2024-03-18T23:06:06.398050+0000","last_clean":"2024-03-18T23:06:06.398050+0000","last_became_active":"2024-03-18T23:06:04.908034+0000","last_became_peered":"2024-03-18T23:06:04.908034+0000","last_unstale":"2024-03-18T23:06:06.398050+0000","last_undegraded":"2024-03-18T23:06:06.398050+0000","last_fullsized":"2024-03-18T23:06:06.398050+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:38:09.712683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"86'17","reported_seq":40,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T23:06:11.379605+0000","last_change":"2024-03-18T23:06:04.211152+0000","last_active":"2024-03-18T23:06:11.379605+0000","last_peered":"2024-03-18T23:06:11.379605+0000","last_clean":"2024-03-18T23:06:11.379605+0000","last_became_active":"2024-03-18T23:06:04.210841+0000","last_became_peered":"2024-03-18T23:06:04.210841+0000","last_unstale":"2024-03-18T23:06:11.379605+0000","last_undegraded":"2024-03-18T23:06:11.379605+0000","last_fullsized":"2024-03-18T23:06:11.379605+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_clean_scrub_stamp":"2024-03-18T23:06:03.184668+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:48:07.935314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20895,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":8,"num_read_kb":5,"num_write":19,"num_write_kb":24,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T23:06:06.224187+0000","last_change":"2024-03-18T23:06:04.910303+0000","last_active":"2024-03-18T23:06:06.224187+0000","last_peered":"2024-03-18T23:06:06.224187+0000","last_clean":"2024-03-18T23:06:06.224187+0000","last_became_active":"2024-03-18T23:06:04.910004+0000","last_became_peered":"2024-03-18T23:06:04.910004+0000","last_unstale":"2024-03-18T23:06:06.224187+0000","last_undegraded":"2024-03-18T23:06:06.224187+0000","last_fullsized":"2024-03-18T23:06:06.224187+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:45:25.750741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"86'4","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T23:06:08.248808+0000","last_change":"2024-03-18T23:06:04.220318+0000","last_active":"2024-03-18T23:06:08.248808+0000","last_peered":"2024-03-18T23:06:08.248808+0000","last_clean":"2024-03-18T23:06:08.248808+0000","last_became_active":"2024-03-18T23:06:04.219985+0000","last_became_peered":"2024-03-18T23:06:04.219985+0000","last_unstale":"2024-03-18T23:06:08.248808+0000","last_undegraded":"2024-03-18T23:06:08.248808+0000","last_fullsized":"2024-03-18T23:06:08.248808+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_clean_scrub_stamp":"2024-03-18T23:06:03.184668+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:49:00.090356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T23:06:06.398008+0000","last_change":"2024-03-18T23:06:04.914456+0000","last_active":"2024-03-18T23:06:06.398008+0000","last_peered":"2024-03-18T23:06:06.398008+0000","last_clean":"2024-03-18T23:06:06.398008+0000","last_became_active":"2024-03-18T23:06:04.914083+0000","last_became_peered":"2024-03-18T23:06:04.914083+0000","last_unstale":"2024-03-18T23:06:06.398008+0000","last_undegraded":"2024-03-18T23:06:06.398008+0000","last_fullsized":"2024-03-18T23:06:06.398008+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:17:00.443254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"86'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T23:06:08.249795+0000","last_change":"2024-03-18T23:06:04.224808+0000","last_active":"2024-03-18T23:06:08.249795+0000","last_peered":"2024-03-18T23:06:08.249795+0000","last_clean":"2024-03-18T23:06:08.249795+0000","last_became_active":"2024-03-18T23:06:04.224112+0000","last_became_peered":"2024-03-18T23:06:04.224112+0000","last_unstale":"2024-03-18T23:06:08.249795+0000","last_undegraded":"2024-03-18T23:06:08.249795+0000","last_fullsized":"2024-03-18T23:06:08.249795+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_clean_scrub_stamp":"2024-03-18T23:06:03.184668+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:19:20.399875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T23:06:06.398094+0000","last_change":"2024-03-18T23:06:04.912271+0000","last_active":"2024-03-18T23:06:06.398094+0000","last_peered":"2024-03-18T23:06:06.398094+0000","last_clean":"2024-03-18T23:06:06.398094+0000","last_became_active":"2024-03-18T23:06:04.911985+0000","last_became_peered":"2024-03-18T23:06:04.911985+0000","last_unstale":"2024-03-18T23:06:06.398094+0000","last_undegraded":"2024-03-18T23:06:06.398094+0000","last_fullsized":"2024-03-18T23:06:06.398094+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T09:18:27.436136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"86'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T23:06:08.995641+0000","last_change":"2024-03-18T23:06:04.209434+0000","last_active":"2024-03-18T23:06:08.995641+0000","last_peered":"2024-03-18T23:06:08.995641+0000","last_clean":"2024-03-18T23:06:08.995641+0000","last_became_active":"2024-03-18T23:06:04.209086+0000","last_became_peered":"2024-03-18T23:06:04.209086+0000","last_unstale":"2024-03-18T23:06:08.995641+0000","last_undegraded":"2024-03-18T23:06:08.995641+0000","last_fullsized":"2024-03-18T23:06:08.995641+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_clean_scrub_stamp":"2024-03-18T23:06:03.184668+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:50:20.823467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T23:06:06.398913+0000","last_change":"2024-03-18T23:06:04.911069+0000","last_active":"2024-03-18T23:06:06.398913+0000","last_peered":"2024-03-18T23:06:06.398913+0000","last_clean":"2024-03-18T23:06:06.398913+0000","last_became_active":"2024-03-18T23:06:04.910760+0000","last_became_peered":"2024-03-18T23:06:04.910760+0000","last_unstale":"2024-03-18T23:06:06.398913+0000","last_undegraded":"2024-03-18T23:06:06.398913+0000","last_fullsized":"2024-03-18T23:06:06.398913+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:16:46.264972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"86'16","reported_seq":31,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T23:06:09.000423+0000","last_change":"2024-03-18T23:06:04.216653+0000","last_active":"2024-03-18T23:06:09.000423+0000","last_peered":"2024-03-18T23:06:09.000423+0000","last_clean":"2024-03-18T23:06:09.000423+0000","last_became_active":"2024-03-18T23:06:04.216358+0000","last_became_peered":"2024-03-18T23:06:04.216358+0000","last_unstale":"2024-03-18T23:06:09.000423+0000","last_undegraded":"2024-03-18T23:06:09.000423+0000","last_fullsized":"2024-03-18T23:06:09.000423+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_clean_scrub_stamp":"2024-03-18T23:06:03.184668+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:16:56.500368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T23:06:05.895929+0000","last_change":"2024-03-18T23:06:04.917377+0000","last_active":"2024-03-18T23:06:05.895929+0000","last_peered":"2024-03-18T23:06:05.895929+0000","last_clean":"2024-03-18T23:06:05.895929+0000","last_became_active":"2024-03-18T23:06:04.917064+0000","last_became_peered":"2024-03-18T23:06:04.917064+0000","last_unstale":"2024-03-18T23:06:05.895929+0000","last_undegraded":"2024-03-18T23:06:05.895929+0000","last_fullsized":"2024-03-18T23:06:05.895929+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:03:41.518177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"86'2","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T23:06:08.982597+0000","last_change":"2024-03-18T23:06:04.210191+0000","last_active":"2024-03-18T23:06:08.982597+0000","last_peered":"2024-03-18T23:06:08.982597+0000","last_clean":"2024-03-18T23:06:08.982597+0000","last_became_active":"2024-03-18T23:06:04.208084+0000","last_became_peered":"2024-03-18T23:06:04.208084+0000","last_unstale":"2024-03-18T23:06:08.982597+0000","last_undegraded":"2024-03-18T23:06:08.982597+0000","last_fullsized":"2024-03-18T23:06:08.982597+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_clean_scrub_stamp":"2024-03-18T23:06:03.184668+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:09:07.220171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T23:06:06.426844+0000","last_change":"2024-03-18T23:06:04.910631+0000","last_active":"2024-03-18T23:06:06.426844+0000","last_peered":"2024-03-18T23:06:06.426844+0000","last_clean":"2024-03-18T23:06:06.426844+0000","last_became_active":"2024-03-18T23:06:04.910322+0000","last_became_peered":"2024-03-18T23:06:04.910322+0000","last_unstale":"2024-03-18T23:06:06.426844+0000","last_undegraded":"2024-03-18T23:06:06.426844+0000","last_fullsized":"2024-03-18T23:06:06.426844+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:39:06.389101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"86'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T23:06:06.222213+0000","last_change":"2024-03-18T23:06:04.220530+0000","last_active":"2024-03-18T23:06:06.222213+0000","last_peered":"2024-03-18T23:06:06.222213+0000","last_clean":"2024-03-18T23:06:06.222213+0000","last_became_active":"2024-03-18T23:06:04.220136+0000","last_became_peered":"2024-03-18T23:06:04.220136+0000","last_unstale":"2024-03-18T23:06:06.222213+0000","last_undegraded":"2024-03-18T23:06:06.222213+0000","last_fullsized":"2024-03-18T23:06:06.222213+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_clean_scrub_stamp":"2024-03-18T23:06:03.184668+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:58:37.037982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"86'1","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T23:06:06.595838+0000","last_change":"2024-03-18T23:06:04.912789+0000","last_active":"2024-03-18T23:06:06.595838+0000","last_peered":"2024-03-18T23:06:06.595838+0000","last_clean":"2024-03-18T23:06:06.595838+0000","last_became_active":"2024-03-18T23:06:04.910451+0000","last_became_peered":"2024-03-18T23:06:04.910451+0000","last_unstale":"2024-03-18T23:06:06.595838+0000","last_undegraded":"2024-03-18T23:06:06.595838+0000","last_fullsized":"2024-03-18T23:06:06.595838+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:06:36.539369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"86'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T23:06:06.398356+0000","last_change":"2024-03-18T23:06:04.217992+0000","last_active":"2024-03-18T23:06:06.398356+0000","last_peered":"2024-03-18T23:06:06.398356+0000","last_clean":"2024-03-18T23:06:06.398356+0000","last_became_active":"2024-03-18T23:06:04.217651+0000","last_became_peered":"2024-03-18T23:06:04.217651+0000","last_unstale":"2024-03-18T23:06:06.398356+0000","last_undegraded":"2024-03-18T23:06:06.398356+0000","last_fullsized":"2024-03-18T23:06:06.398356+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_clean_scrub_stamp":"2024-03-18T23:06:03.184668+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:25:46.901251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T23:06:06.398714+0000","last_change":"2024-03-18T23:06:04.913349+0000","last_active":"2024-03-18T23:06:06.398714+0000","last_peered":"2024-03-18T23:06:06.398714+0000","last_clean":"2024-03-18T23:06:06.398714+0000","last_became_active":"2024-03-18T23:06:04.913026+0000","last_became_peered":"2024-03-18T23:06:04.913026+0000","last_unstale":"2024-03-18T23:06:06.398714+0000","last_undegraded":"2024-03-18T23:06:06.398714+0000","last_fullsized":"2024-03-18T23:06:06.398714+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T23:24:28.884524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T23:06:05.896023+0000","last_change":"2024-03-18T23:06:04.916541+0000","last_active":"2024-03-18T23:06:05.896023+0000","last_peered":"2024-03-18T23:06:05.896023+0000","last_clean":"2024-03-18T23:06:05.896023+0000","last_became_active":"2024-03-18T23:06:04.916179+0000","last_became_peered":"2024-03-18T23:06:04.916179+0000","last_unstale":"2024-03-18T23:06:05.896023+0000","last_undegraded":"2024-03-18T23:06:05.896023+0000","last_fullsized":"2024-03-18T23:06:05.896023+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:31:59.515767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"86'4","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T23:06:08.999385+0000","last_change":"2024-03-18T23:06:04.217581+0000","last_active":"2024-03-18T23:06:08.999385+0000","last_peered":"2024-03-18T23:06:08.999385+0000","last_clean":"2024-03-18T23:06:08.999385+0000","last_became_active":"2024-03-18T23:06:04.217191+0000","last_became_peered":"2024-03-18T23:06:04.217191+0000","last_unstale":"2024-03-18T23:06:08.999385+0000","last_undegraded":"2024-03-18T23:06:08.999385+0000","last_fullsized":"2024-03-18T23:06:08.999385+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_clean_scrub_stamp":"2024-03-18T23:06:03.184668+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:47:24.728136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T23:06:05.895759+0000","last_change":"2024-03-18T23:06:04.919784+0000","last_active":"2024-03-18T23:06:05.895759+0000","last_peered":"2024-03-18T23:06:05.895759+0000","last_clean":"2024-03-18T23:06:05.895759+0000","last_became_active":"2024-03-18T23:06:04.919483+0000","last_became_peered":"2024-03-18T23:06:04.919483+0000","last_unstale":"2024-03-18T23:06:05.895759+0000","last_undegraded":"2024-03-18T23:06:05.895759+0000","last_fullsized":"2024-03-18T23:06:05.895759+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:37:29.635750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'3","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T23:06:08.984490+0000","last_change":"2024-03-18T23:06:04.215152+0000","last_active":"2024-03-18T23:06:08.984490+0000","last_peered":"2024-03-18T23:06:08.984490+0000","last_clean":"2024-03-18T23:06:08.984490+0000","last_became_active":"2024-03-18T23:06:04.214789+0000","last_became_peered":"2024-03-18T23:06:04.214789+0000","last_unstale":"2024-03-18T23:06:08.984490+0000","last_undegraded":"2024-03-18T23:06:08.984490+0000","last_fullsized":"2024-03-18T23:06:08.984490+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_clean_scrub_stamp":"2024-03-18T23:06:03.184668+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:58:11.630840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":349,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T23:06:06.398951+0000","last_change":"2024-03-18T23:04:54.734000+0000","last_active":"2024-03-18T23:06:06.398951+0000","last_peered":"2024-03-18T23:06:06.398951+0000","last_clean":"2024-03-18T23:06:06.398951+0000","last_became_active":"2024-03-18T23:04:54.733516+0000","last_became_peered":"2024-03-18T23:04:54.733516+0000","last_unstale":"2024-03-18T23:06:06.398951+0000","last_undegraded":"2024-03-18T23:06:06.398951+0000","last_fullsized":"2024-03-18T23:06:06.398951+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:01:28.441034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:01:28.441034+0000","last_clean_scrub_stamp":"2024-03-18T23:01:28.441034+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T23:48:14.029479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T23:06:06.397785+0000","last_change":"2024-03-18T23:06:04.908549+0000","last_active":"2024-03-18T23:06:06.397785+0000","last_peered":"2024-03-18T23:06:06.397785+0000","last_clean":"2024-03-18T23:06:06.397785+0000","last_became_active":"2024-03-18T23:06:04.908202+0000","last_became_peered":"2024-03-18T23:06:04.908202+0000","last_unstale":"2024-03-18T23:06:06.397785+0000","last_undegraded":"2024-03-18T23:06:06.397785+0000","last_fullsized":"2024-03-18T23:06:06.397785+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:57:09.486103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T23:06:08.994816+0000","last_change":"2024-03-18T23:06:04.213819+0000","last_active":"2024-03-18T23:06:08.994816+0000","last_peered":"2024-03-18T23:06:08.994816+0000","last_clean":"2024-03-18T23:06:08.994816+0000","last_became_active":"2024-03-18T23:06:04.213418+0000","last_became_peered":"2024-03-18T23:06:04.213418+0000","last_unstale":"2024-03-18T23:06:08.994816+0000","last_undegraded":"2024-03-18T23:06:08.994816+0000","last_fullsized":"2024-03-18T23:06:08.994816+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_clean_scrub_stamp":"2024-03-18T23:06:03.184668+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:50:03.175779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"86'3","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T23:06:09.999688+0000","last_change":"2024-03-18T23:06:04.218660+0000","last_active":"2024-03-18T23:06:09.999688+0000","last_peered":"2024-03-18T23:06:09.999688+0000","last_clean":"2024-03-18T23:06:09.999688+0000","last_became_active":"2024-03-18T23:06:04.218183+0000","last_became_peered":"2024-03-18T23:06:04.218183+0000","last_unstale":"2024-03-18T23:06:09.999688+0000","last_undegraded":"2024-03-18T23:06:09.999688+0000","last_fullsized":"2024-03-18T23:06:09.999688+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_clean_scrub_stamp":"2024-03-18T23:06:03.184668+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T08:17:42.208572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T23:06:06.398458+0000","last_change":"2024-03-18T23:06:04.901124+0000","last_active":"2024-03-18T23:06:06.398458+0000","last_peered":"2024-03-18T23:06:06.398458+0000","last_clean":"2024-03-18T23:06:06.398458+0000","last_became_active":"2024-03-18T23:06:04.899992+0000","last_became_peered":"2024-03-18T23:06:04.899992+0000","last_unstale":"2024-03-18T23:06:06.398458+0000","last_undegraded":"2024-03-18T23:06:06.398458+0000","last_fullsized":"2024-03-18T23:06:06.398458+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:27:00.965945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"86'7","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T23:06:09.999467+0000","last_change":"2024-03-18T23:06:04.227413+0000","last_active":"2024-03-18T23:06:09.999467+0000","last_peered":"2024-03-18T23:06:09.999467+0000","last_clean":"2024-03-18T23:06:09.999467+0000","last_became_active":"2024-03-18T23:06:04.227071+0000","last_became_peered":"2024-03-18T23:06:04.227071+0000","last_unstale":"2024-03-18T23:06:09.999467+0000","last_undegraded":"2024-03-18T23:06:09.999467+0000","last_fullsized":"2024-03-18T23:06:09.999467+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_clean_scrub_stamp":"2024-03-18T23:06:03.184668+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T08:15:31.531168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T23:06:05.895866+0000","last_change":"2024-03-18T23:06:04.910220+0000","last_active":"2024-03-18T23:06:05.895866+0000","last_peered":"2024-03-18T23:06:05.895866+0000","last_clean":"2024-03-18T23:06:05.895866+0000","last_became_active":"2024-03-18T23:06:04.909855+0000","last_became_peered":"2024-03-18T23:06:04.909855+0000","last_unstale":"2024-03-18T23:06:05.895866+0000","last_undegraded":"2024-03-18T23:06:05.895866+0000","last_fullsized":"2024-03-18T23:06:05.895866+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:19:09.315343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"86'4","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T23:06:08.247933+0000","last_change":"2024-03-18T23:06:04.225473+0000","last_active":"2024-03-18T23:06:08.247933+0000","last_peered":"2024-03-18T23:06:08.247933+0000","last_clean":"2024-03-18T23:06:08.247933+0000","last_became_active":"2024-03-18T23:06:04.225126+0000","last_became_peered":"2024-03-18T23:06:04.225126+0000","last_unstale":"2024-03-18T23:06:08.247933+0000","last_undegraded":"2024-03-18T23:06:08.247933+0000","last_fullsized":"2024-03-18T23:06:08.247933+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_clean_scrub_stamp":"2024-03-18T23:06:03.184668+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T09:29:26.438597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T23:06:06.398430+0000","last_change":"2024-03-18T23:06:04.911110+0000","last_active":"2024-03-18T23:06:06.398430+0000","last_peered":"2024-03-18T23:06:06.398430+0000","last_clean":"2024-03-18T23:06:06.398430+0000","last_became_active":"2024-03-18T23:06:04.910752+0000","last_became_peered":"2024-03-18T23:06:04.910752+0000","last_unstale":"2024-03-18T23:06:06.398430+0000","last_undegraded":"2024-03-18T23:06:06.398430+0000","last_fullsized":"2024-03-18T23:06:06.398430+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:40:44.736472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T23:06:08.249352+0000","last_change":"2024-03-18T23:06:04.216750+0000","last_active":"2024-03-18T23:06:08.249352+0000","last_peered":"2024-03-18T23:06:08.249352+0000","last_clean":"2024-03-18T23:06:08.249352+0000","last_became_active":"2024-03-18T23:06:04.216393+0000","last_became_peered":"2024-03-18T23:06:04.216393+0000","last_unstale":"2024-03-18T23:06:08.249352+0000","last_undegraded":"2024-03-18T23:06:08.249352+0000","last_fullsized":"2024-03-18T23:06:08.249352+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_clean_scrub_stamp":"2024-03-18T23:06:03.184668+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:40:33.486235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T23:06:06.223867+0000","last_change":"2024-03-18T23:06:04.915156+0000","last_active":"2024-03-18T23:06:06.223867+0000","last_peered":"2024-03-18T23:06:06.223867+0000","last_clean":"2024-03-18T23:06:06.223867+0000","last_became_active":"2024-03-18T23:06:04.914821+0000","last_became_peered":"2024-03-18T23:06:04.914821+0000","last_unstale":"2024-03-18T23:06:06.223867+0000","last_undegraded":"2024-03-18T23:06:06.223867+0000","last_fullsized":"2024-03-18T23:06:06.223867+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:41:22.344360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T23:06:05.895320+0000","last_change":"2024-03-18T23:06:04.224533+0000","last_active":"2024-03-18T23:06:05.895320+0000","last_peered":"2024-03-18T23:06:05.895320+0000","last_clean":"2024-03-18T23:06:05.895320+0000","last_became_active":"2024-03-18T23:06:04.224122+0000","last_became_peered":"2024-03-18T23:06:04.224122+0000","last_unstale":"2024-03-18T23:06:05.895320+0000","last_undegraded":"2024-03-18T23:06:05.895320+0000","last_fullsized":"2024-03-18T23:06:05.895320+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_clean_scrub_stamp":"2024-03-18T23:06:03.184668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T09:03:40.689085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T23:06:06.426650+0000","last_change":"2024-03-18T23:06:04.912574+0000","last_active":"2024-03-18T23:06:06.426650+0000","last_peered":"2024-03-18T23:06:06.426650+0000","last_clean":"2024-03-18T23:06:06.426650+0000","last_became_active":"2024-03-18T23:06:04.912284+0000","last_became_peered":"2024-03-18T23:06:04.912284+0000","last_unstale":"2024-03-18T23:06:06.426650+0000","last_undegraded":"2024-03-18T23:06:06.426650+0000","last_fullsized":"2024-03-18T23:06:06.426650+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:18:11.003763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T23:06:06.397470+0000","last_change":"2024-03-18T23:06:04.220364+0000","last_active":"2024-03-18T23:06:06.397470+0000","last_peered":"2024-03-18T23:06:06.397470+0000","last_clean":"2024-03-18T23:06:06.397470+0000","last_became_active":"2024-03-18T23:06:04.220074+0000","last_became_peered":"2024-03-18T23:06:04.220074+0000","last_unstale":"2024-03-18T23:06:06.397470+0000","last_undegraded":"2024-03-18T23:06:06.397470+0000","last_fullsized":"2024-03-18T23:06:06.397470+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_clean_scrub_stamp":"2024-03-18T23:06:03.184668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:10:15.385724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T23:06:05.895832+0000","last_change":"2024-03-18T23:06:04.917448+0000","last_active":"2024-03-18T23:06:05.895832+0000","last_peered":"2024-03-18T23:06:05.895832+0000","last_clean":"2024-03-18T23:06:05.895832+0000","last_became_active":"2024-03-18T23:06:04.916921+0000","last_became_peered":"2024-03-18T23:06:04.916921+0000","last_unstale":"2024-03-18T23:06:05.895832+0000","last_undegraded":"2024-03-18T23:06:05.895832+0000","last_fullsized":"2024-03-18T23:06:05.895832+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:23:06.715597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T23:06:08.249738+0000","last_change":"2024-03-18T23:06:04.211777+0000","last_active":"2024-03-18T23:06:08.249738+0000","last_peered":"2024-03-18T23:06:08.249738+0000","last_clean":"2024-03-18T23:06:08.249738+0000","last_became_active":"2024-03-18T23:06:04.211454+0000","last_became_peered":"2024-03-18T23:06:04.211454+0000","last_unstale":"2024-03-18T23:06:08.249738+0000","last_undegraded":"2024-03-18T23:06:08.249738+0000","last_fullsized":"2024-03-18T23:06:08.249738+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_clean_scrub_stamp":"2024-03-18T23:06:03.184668+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:52:14.010341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T23:06:06.398369+0000","last_change":"2024-03-18T23:06:04.911471+0000","last_active":"2024-03-18T23:06:06.398369+0000","last_peered":"2024-03-18T23:06:06.398369+0000","last_clean":"2024-03-18T23:06:06.398369+0000","last_became_active":"2024-03-18T23:06:04.910550+0000","last_became_peered":"2024-03-18T23:06:04.910550+0000","last_unstale":"2024-03-18T23:06:06.398369+0000","last_undegraded":"2024-03-18T23:06:06.398369+0000","last_fullsized":"2024-03-18T23:06:06.398369+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:34:55.084581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T23:06:06.397672+0000","last_change":"2024-03-18T23:06:04.215989+0000","last_active":"2024-03-18T23:06:06.397672+0000","last_peered":"2024-03-18T23:06:06.397672+0000","last_clean":"2024-03-18T23:06:06.397672+0000","last_became_active":"2024-03-18T23:06:04.215578+0000","last_became_peered":"2024-03-18T23:06:04.215578+0000","last_unstale":"2024-03-18T23:06:06.397672+0000","last_undegraded":"2024-03-18T23:06:06.397672+0000","last_fullsized":"2024-03-18T23:06:06.397672+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_clean_scrub_stamp":"2024-03-18T23:06:03.184668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:10:34.048011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T23:06:06.398256+0000","last_change":"2024-03-18T23:06:04.918955+0000","last_active":"2024-03-18T23:06:06.398256+0000","last_peered":"2024-03-18T23:06:06.398256+0000","last_clean":"2024-03-18T23:06:06.398256+0000","last_became_active":"2024-03-18T23:06:04.915475+0000","last_became_peered":"2024-03-18T23:06:04.915475+0000","last_unstale":"2024-03-18T23:06:06.398256+0000","last_undegraded":"2024-03-18T23:06:06.398256+0000","last_fullsized":"2024-03-18T23:06:06.398256+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:07:05.679235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"85'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T23:06:05.895852+0000","last_change":"2024-03-18T23:06:04.224237+0000","last_active":"2024-03-18T23:06:05.895852+0000","last_peered":"2024-03-18T23:06:05.895852+0000","last_clean":"2024-03-18T23:06:05.895852+0000","last_became_active":"2024-03-18T23:06:04.223820+0000","last_became_peered":"2024-03-18T23:06:04.223820+0000","last_unstale":"2024-03-18T23:06:05.895852+0000","last_undegraded":"2024-03-18T23:06:05.895852+0000","last_fullsized":"2024-03-18T23:06:05.895852+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_clean_scrub_stamp":"2024-03-18T23:06:03.184668+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:17:48.157499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T23:06:06.398205+0000","last_change":"2024-03-18T23:06:04.914846+0000","last_active":"2024-03-18T23:06:06.398205+0000","last_peered":"2024-03-18T23:06:06.398205+0000","last_clean":"2024-03-18T23:06:06.398205+0000","last_became_active":"2024-03-18T23:06:04.911629+0000","last_became_peered":"2024-03-18T23:06:04.911629+0000","last_unstale":"2024-03-18T23:06:06.398205+0000","last_undegraded":"2024-03-18T23:06:06.398205+0000","last_fullsized":"2024-03-18T23:06:06.398205+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:17:32.235568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T23:06:06.398483+0000","last_change":"2024-03-18T23:06:04.219594+0000","last_active":"2024-03-18T23:06:06.398483+0000","last_peered":"2024-03-18T23:06:06.398483+0000","last_clean":"2024-03-18T23:06:06.398483+0000","last_became_active":"2024-03-18T23:06:04.219241+0000","last_became_peered":"2024-03-18T23:06:04.219241+0000","last_unstale":"2024-03-18T23:06:06.398483+0000","last_undegraded":"2024-03-18T23:06:06.398483+0000","last_fullsized":"2024-03-18T23:06:06.398483+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_clean_scrub_stamp":"2024-03-18T23:06:03.184668+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:25:50.012659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T23:06:06.398537+0000","last_change":"2024-03-18T23:06:04.914463+0000","last_active":"2024-03-18T23:06:06.398537+0000","last_peered":"2024-03-18T23:06:06.398537+0000","last_clean":"2024-03-18T23:06:06.398537+0000","last_became_active":"2024-03-18T23:06:04.914130+0000","last_became_peered":"2024-03-18T23:06:04.914130+0000","last_unstale":"2024-03-18T23:06:06.398537+0000","last_undegraded":"2024-03-18T23:06:06.398537+0000","last_fullsized":"2024-03-18T23:06:06.398537+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:57:56.065100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T23:06:06.426705+0000","last_change":"2024-03-18T23:06:04.905951+0000","last_active":"2024-03-18T23:06:06.426705+0000","last_peered":"2024-03-18T23:06:06.426705+0000","last_clean":"2024-03-18T23:06:06.426705+0000","last_became_active":"2024-03-18T23:06:04.905614+0000","last_became_peered":"2024-03-18T23:06:04.905614+0000","last_unstale":"2024-03-18T23:06:06.426705+0000","last_undegraded":"2024-03-18T23:06:06.426705+0000","last_fullsized":"2024-03-18T23:06:06.426705+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:11:44.884671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T23:06:06.397633+0000","last_change":"2024-03-18T23:06:04.910738+0000","last_active":"2024-03-18T23:06:06.397633+0000","last_peered":"2024-03-18T23:06:06.397633+0000","last_clean":"2024-03-18T23:06:06.397633+0000","last_became_active":"2024-03-18T23:06:04.910369+0000","last_became_peered":"2024-03-18T23:06:04.910369+0000","last_unstale":"2024-03-18T23:06:06.397633+0000","last_undegraded":"2024-03-18T23:06:06.397633+0000","last_fullsized":"2024-03-18T23:06:06.397633+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:22:08.306825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T23:06:06.397551+0000","last_change":"2024-03-18T23:06:04.912818+0000","last_active":"2024-03-18T23:06:06.397551+0000","last_peered":"2024-03-18T23:06:06.397551+0000","last_clean":"2024-03-18T23:06:06.397551+0000","last_became_active":"2024-03-18T23:06:04.912511+0000","last_became_peered":"2024-03-18T23:06:04.912511+0000","last_unstale":"2024-03-18T23:06:06.397551+0000","last_undegraded":"2024-03-18T23:06:06.397551+0000","last_fullsized":"2024-03-18T23:06:06.397551+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T09:24:44.743041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T23:06:06.398123+0000","last_change":"2024-03-18T23:06:04.912177+0000","last_active":"2024-03-18T23:06:06.398123+0000","last_peered":"2024-03-18T23:06:06.398123+0000","last_clean":"2024-03-18T23:06:06.398123+0000","last_became_active":"2024-03-18T23:06:04.911638+0000","last_became_peered":"2024-03-18T23:06:04.911638+0000","last_unstale":"2024-03-18T23:06:06.398123+0000","last_undegraded":"2024-03-18T23:06:06.398123+0000","last_fullsized":"2024-03-18T23:06:06.398123+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:21:30.648246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T23:06:06.398565+0000","last_change":"2024-03-18T23:06:04.912670+0000","last_active":"2024-03-18T23:06:06.398565+0000","last_peered":"2024-03-18T23:06:06.398565+0000","last_clean":"2024-03-18T23:06:06.398565+0000","last_became_active":"2024-03-18T23:06:04.912241+0000","last_became_peered":"2024-03-18T23:06:04.912241+0000","last_unstale":"2024-03-18T23:06:06.398565+0000","last_undegraded":"2024-03-18T23:06:06.398565+0000","last_fullsized":"2024-03-18T23:06:06.398565+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:31:02.666360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T23:06:06.426849+0000","last_change":"2024-03-18T23:06:04.909618+0000","last_active":"2024-03-18T23:06:06.426849+0000","last_peered":"2024-03-18T23:06:06.426849+0000","last_clean":"2024-03-18T23:06:06.426849+0000","last_became_active":"2024-03-18T23:06:04.909301+0000","last_became_peered":"2024-03-18T23:06:04.909301+0000","last_unstale":"2024-03-18T23:06:06.426849+0000","last_undegraded":"2024-03-18T23:06:06.426849+0000","last_fullsized":"2024-03-18T23:06:06.426849+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:14:28.101360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T23:06:06.397902+0000","last_change":"2024-03-18T23:06:04.904607+0000","last_active":"2024-03-18T23:06:06.397902+0000","last_peered":"2024-03-18T23:06:06.397902+0000","last_clean":"2024-03-18T23:06:06.397902+0000","last_became_active":"2024-03-18T23:06:04.904280+0000","last_became_peered":"2024-03-18T23:06:04.904280+0000","last_unstale":"2024-03-18T23:06:06.397902+0000","last_undegraded":"2024-03-18T23:06:06.397902+0000","last_fullsized":"2024-03-18T23:06:06.397902+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:59:11.445200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T23:06:05.895775+0000","last_change":"2024-03-18T23:06:04.918342+0000","last_active":"2024-03-18T23:06:05.895775+0000","last_peered":"2024-03-18T23:06:05.895775+0000","last_clean":"2024-03-18T23:06:05.895775+0000","last_became_active":"2024-03-18T23:06:04.917947+0000","last_became_peered":"2024-03-18T23:06:04.917947+0000","last_unstale":"2024-03-18T23:06:05.895775+0000","last_undegraded":"2024-03-18T23:06:05.895775+0000","last_fullsized":"2024-03-18T23:06:05.895775+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:05:49.347374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T23:06:05.895787+0000","last_change":"2024-03-18T23:06:04.912457+0000","last_active":"2024-03-18T23:06:05.895787+0000","last_peered":"2024-03-18T23:06:05.895787+0000","last_clean":"2024-03-18T23:06:05.895787+0000","last_became_active":"2024-03-18T23:06:04.912141+0000","last_became_peered":"2024-03-18T23:06:04.912141+0000","last_unstale":"2024-03-18T23:06:05.895787+0000","last_undegraded":"2024-03-18T23:06:05.895787+0000","last_fullsized":"2024-03-18T23:06:05.895787+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:04:45.847314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T23:06:05.895715+0000","last_change":"2024-03-18T23:06:04.918723+0000","last_active":"2024-03-18T23:06:05.895715+0000","last_peered":"2024-03-18T23:06:05.895715+0000","last_clean":"2024-03-18T23:06:05.895715+0000","last_became_active":"2024-03-18T23:06:04.918375+0000","last_became_peered":"2024-03-18T23:06:04.918375+0000","last_unstale":"2024-03-18T23:06:05.895715+0000","last_undegraded":"2024-03-18T23:06:05.895715+0000","last_fullsized":"2024-03-18T23:06:05.895715+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:27:12.868420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T23:06:06.398427+0000","last_change":"2024-03-18T23:06:04.913935+0000","last_active":"2024-03-18T23:06:06.398427+0000","last_peered":"2024-03-18T23:06:06.398427+0000","last_clean":"2024-03-18T23:06:06.398427+0000","last_became_active":"2024-03-18T23:06:04.913617+0000","last_became_peered":"2024-03-18T23:06:04.913617+0000","last_unstale":"2024-03-18T23:06:06.398427+0000","last_undegraded":"2024-03-18T23:06:06.398427+0000","last_fullsized":"2024-03-18T23:06:06.398427+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:53:15.585608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T23:06:06.398230+0000","last_change":"2024-03-18T23:06:04.911951+0000","last_active":"2024-03-18T23:06:06.398230+0000","last_peered":"2024-03-18T23:06:06.398230+0000","last_clean":"2024-03-18T23:06:06.398230+0000","last_became_active":"2024-03-18T23:06:04.911621+0000","last_became_peered":"2024-03-18T23:06:04.911621+0000","last_unstale":"2024-03-18T23:06:06.398230+0000","last_undegraded":"2024-03-18T23:06:06.398230+0000","last_fullsized":"2024-03-18T23:06:06.398230+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:20:46.600106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T23:06:06.426650+0000","last_change":"2024-03-18T23:06:04.910377+0000","last_active":"2024-03-18T23:06:06.426650+0000","last_peered":"2024-03-18T23:06:06.426650+0000","last_clean":"2024-03-18T23:06:06.426650+0000","last_became_active":"2024-03-18T23:06:04.910053+0000","last_became_peered":"2024-03-18T23:06:04.910053+0000","last_unstale":"2024-03-18T23:06:06.426650+0000","last_undegraded":"2024-03-18T23:06:06.426650+0000","last_fullsized":"2024-03-18T23:06:06.426650+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T23:29:51.750365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":284006,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":23923,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":18,"num_read_kb":15,"num_write":146,"num_write_kb":58,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":45,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":266240,"data_stored":34832,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":126,"ondisk_log_size":126,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892350989,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":925696,"data_stored":444311,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612625,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28028,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717124,"statfs":{"total":95995035648,"available":95966334976,"internally_reserved":0,"allocated":1036288,"data_stored":1044627,"data_compressed":36115,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776151,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":950272,"data_stored":445219,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037789,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32100,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713052,"statfs":{"total":95995035648,"available":95962165248,"internally_reserved":0,"allocated":946176,"data_stored":445049,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332128,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":987136,"data_stored":1035883,"data_compressed":36115,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495655,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36244,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708908,"statfs":{"total":95995035648,"available":95957921792,"internally_reserved":0,"allocated":925696,"data_stored":444311,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574912,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32092,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713060,"statfs":{"total":95995035648,"available":95962173440,"internally_reserved":0,"allocated":937984,"data_stored":445923,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738437,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36272,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708880,"statfs":{"total":95995035648,"available":95957893120,"internally_reserved":0,"allocated":958464,"data_stored":446377,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411387,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":950272,"data_stored":446579,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182453,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":933888,"data_stored":444415,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920817,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32084,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713068,"statfs":{"total":95995035648,"available":95962181632,"internally_reserved":0,"allocated":929792,"data_stored":444333,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757291,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32152,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713000,"statfs":{"total":95995035648,"available":95962112000,"internally_reserved":0,"allocated":987136,"data_stored":1034861,"data_compressed":36115,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34477,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34477,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34477,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36115,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36115,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36115,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36115,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36115,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2268,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":22631,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-18T23:06:13.493 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-18T23:06:13.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:06:13 smithi139 ceph-mon[67282]: pgmap v292: 97 pgs: 97 active+clean; 600 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 7.7 KiB/s wr, 21 op/s 2024-03-18T23:06:13.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:06:13 smithi139 ceph-mon[67282]: from='client.14751 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T23:06:13.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:06:13 smithi139 ceph-mon[67282]: from='client.14757 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T23:06:13.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:06:13 smithi139 ceph-mon[67282]: from='client.? 172.21.15.49:0/2244712381' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T23:06:13.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:06:13 smithi139 ceph-mon[67282]: from='client.? 172.21.15.49:0/727513553' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T23:06:13.639 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:13.637+0000 7ff6ab9c6640 1 -- 172.21.15.49:0/3599689435 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff6a41019b0 msgr2=0x7ff6a4101e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:13.640 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:13.637+0000 7ff6ab9c6640 1 --2- 172.21.15.49:0/3599689435 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff6a41019b0 0x7ff6a4101e30 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7ff6a0009900 tx=0x7ff6a002bc30 comp rx=0 tx=0).stop 2024-03-18T23:06:13.640 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:13.637+0000 7ff6ab9c6640 1 -- 172.21.15.49:0/3599689435 shutdown_connections 2024-03-18T23:06:13.640 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:13.637+0000 7ff6ab9c6640 1 --2- 172.21.15.49:0/3599689435 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff6a4102370 0x7ff6a4108c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:13.640 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:13.637+0000 7ff6ab9c6640 1 --2- 172.21.15.49:0/3599689435 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff6a41019b0 0x7ff6a4101e30 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:13.640 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:13.637+0000 7ff6ab9c6640 1 --2- 172.21.15.49:0/3599689435 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff6a41007c0 0x7ff6a4100be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:13.640 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:13.637+0000 7ff6ab9c6640 1 -- 172.21.15.49:0/3599689435 >> 172.21.15.49:0/3599689435 conn(0x7ff6a40fbd50 msgr2=0x7ff6a40fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:06:13.640 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:13.637+0000 7ff6ab9c6640 1 -- 172.21.15.49:0/3599689435 shutdown_connections 2024-03-18T23:06:13.640 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:13.637+0000 7ff6ab9c6640 1 -- 172.21.15.49:0/3599689435 wait complete. 2024-03-18T23:06:13.640 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:13.637+0000 7ff6ab9c6640 1 Processor -- start 2024-03-18T23:06:13.640 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:13.637+0000 7ff6ab9c6640 1 -- start start 2024-03-18T23:06:13.641 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:13.637+0000 7ff6ab9c6640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff6a41007c0 0x7ff6a406aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:06:13.641 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:13.638+0000 7ff6ab9c6640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff6a41019b0 0x7ff6a419c860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:06:13.641 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:13.638+0000 7ff6ab9c6640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff6a4102370 0x7ff6a419eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:06:13.641 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:13.638+0000 7ff6ab9c6640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7ff6a410c3d0 con 0x7ff6a4102370 2024-03-18T23:06:13.641 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:13.638+0000 7ff6ab9c6640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7ff6a410c250 con 0x7ff6a41019b0 2024-03-18T23:06:13.641 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:13.638+0000 7ff6ab9c6640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff6a410c550 con 0x7ff6a41007c0 2024-03-18T23:06:13.642 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:13.638+0000 7ff6a9f63640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff6a4102370 0x7ff6a419eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:06:13.642 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:13.638+0000 7ff6a9762640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff6a41007c0 0x7ff6a406aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:06:13.642 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:13.638+0000 7ff6a9f63640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff6a4102370 0x7ff6a419eed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:54354/0 (socket says 172.21.15.49:54354) 2024-03-18T23:06:13.642 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:13.638+0000 7ff6a9f63640 1 -- 172.21.15.49:0/1625124588 learned_addr learned my addr 172.21.15.49:0/1625124588 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:06:13.642 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:13.638+0000 7ff6a8f61640 1 --2- 172.21.15.49:0/1625124588 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff6a41019b0 0x7ff6a419c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:06:13.642 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:13.638+0000 7ff6a9f63640 1 -- 172.21.15.49:0/1625124588 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff6a41007c0 msgr2=0x7ff6a406aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:13.642 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:13.638+0000 7ff6a9f63640 1 --2- 172.21.15.49:0/1625124588 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff6a41007c0 0x7ff6a406aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:13.642 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:13.638+0000 7ff6a9f63640 1 -- 172.21.15.49:0/1625124588 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff6a41019b0 msgr2=0x7ff6a419c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:13.642 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:13.638+0000 7ff6a9f63640 1 --2- 172.21.15.49:0/1625124588 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff6a41019b0 0x7ff6a419c860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:13.642 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:13.638+0000 7ff6a9f63640 1 -- 172.21.15.49:0/1625124588 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6a419f790 con 0x7ff6a4102370 2024-03-18T23:06:13.642 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:13.638+0000 7ff6a9762640 1 --2- 172.21.15.49:0/1625124588 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff6a41007c0 0x7ff6a406aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:06:13.642 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:13.638+0000 7ff6a9f63640 1 --2- 172.21.15.49:0/1625124588 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff6a4102370 0x7ff6a419eed0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7ff69c00eb10 tx=0x7ff69c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:06:13.642 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:13.638+0000 7ff6967fc640 1 -- 172.21.15.49:0/1625124588 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff69c004b20 con 0x7ff6a4102370 2024-03-18T23:06:13.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:13.638+0000 7ff6ab9c6640 1 -- 172.21.15.49:0/1625124588 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6a419fa80 con 0x7ff6a4102370 2024-03-18T23:06:13.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:13.638+0000 7ff6967fc640 1 -- 172.21.15.49:0/1625124588 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff69c004cc0 con 0x7ff6a4102370 2024-03-18T23:06:13.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:13.638+0000 7ff6967fc640 1 -- 172.21.15.49:0/1625124588 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff69c010760 con 0x7ff6a4102370 2024-03-18T23:06:13.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:13.638+0000 7ff6ab9c6640 1 -- 172.21.15.49:0/1625124588 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6a419ff50 con 0x7ff6a4102370 2024-03-18T23:06:13.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:13.639+0000 7ff6ab9c6640 1 -- 172.21.15.49:0/1625124588 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff6a4068a60 con 0x7ff6a4102370 2024-03-18T23:06:13.644 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:13.640+0000 7ff6967fc640 1 -- 172.21.15.49:0/1625124588 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 17) ==== 137341+0+0 (secure 0 0 0) 0x7ff69c005020 con 0x7ff6a4102370 2024-03-18T23:06:13.644 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:13.641+0000 7ff6967fc640 1 --2- 172.21.15.49:0/1625124588 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7ff6900a2890 0x7ff6900a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:06:13.644 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:13.641+0000 7ff6a9762640 1 --2- 172.21.15.49:0/1625124588 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7ff6900a2890 0x7ff6900a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:06:13.644 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:13.641+0000 7ff6967fc640 1 -- 172.21.15.49:0/1625124588 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7ff69c0cefa0 con 0x7ff6a4102370 2024-03-18T23:06:13.644 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:13.641+0000 7ff6a9762640 1 --2- 172.21.15.49:0/1625124588 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7ff6900a2890 0x7ff6900a4d50 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7ff6a4101810 tx=0x7ff698005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:06:13.645 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:13.645+0000 7ff6967fc640 1 -- 172.21.15.49:0/1625124588 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7ff69c09c090 con 0x7ff6a4102370 2024-03-18T23:06:13.711 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:06:13 smithi175 ceph-mon[67352]: pgmap v292: 97 pgs: 97 active+clean; 600 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 7.7 KiB/s wr, 21 op/s 2024-03-18T23:06:13.711 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:06:13 smithi175 ceph-mon[67352]: from='client.14751 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T23:06:13.711 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:06:13 smithi175 ceph-mon[67352]: from='client.14757 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T23:06:13.711 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:06:13 smithi175 ceph-mon[67352]: from='client.? 172.21.15.49:0/2244712381' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T23:06:13.711 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:06:13 smithi175 ceph-mon[67352]: from='client.? 172.21.15.49:0/727513553' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T23:06:13.826 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:13.826+0000 7ff6ab9c6640 1 -- 172.21.15.49:0/1625124588 --> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff6a4061c60 con 0x7ff6900a2890 2024-03-18T23:06:13.833 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:13.834+0000 7ff6967fc640 1 -- 172.21.15.49:0/1625124588 <== mgr.14150 v2:172.21.15.49:6800/2083805211 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258907 (secure 0 0 0) 0x7ff6a4061c60 con 0x7ff6900a2890 2024-03-18T23:06:13.836 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-18T23:06:13.838 INFO:teuthology.orchestra.run.smithi049.stderr:dumped all 2024-03-18T23:06:13.839 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:13.837+0000 7ff6ab9c6640 1 -- 172.21.15.49:0/1625124588 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7ff6900a2890 msgr2=0x7ff6900a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:13.839 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:13.837+0000 7ff6ab9c6640 1 --2- 172.21.15.49:0/1625124588 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7ff6900a2890 0x7ff6900a4d50 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7ff6a4101810 tx=0x7ff698005f90 comp rx=0 tx=0).stop 2024-03-18T23:06:13.839 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:13.837+0000 7ff6ab9c6640 1 -- 172.21.15.49:0/1625124588 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff6a4102370 msgr2=0x7ff6a419eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:13.840 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:13.837+0000 7ff6ab9c6640 1 --2- 172.21.15.49:0/1625124588 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff6a4102370 0x7ff6a419eed0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7ff69c00eb10 tx=0x7ff69c00c2d0 comp rx=0 tx=0).stop 2024-03-18T23:06:13.840 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:13.837+0000 7ff6ab9c6640 1 -- 172.21.15.49:0/1625124588 shutdown_connections 2024-03-18T23:06:13.840 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:13.837+0000 7ff6ab9c6640 1 --2- 172.21.15.49:0/1625124588 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7ff6900a2890 0x7ff6900a4d50 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:13.840 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:13.837+0000 7ff6ab9c6640 1 --2- 172.21.15.49:0/1625124588 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff6a4102370 0x7ff6a419eed0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:13.840 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:13.837+0000 7ff6ab9c6640 1 --2- 172.21.15.49:0/1625124588 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff6a41019b0 0x7ff6a419c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:13.840 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:13.837+0000 7ff6ab9c6640 1 --2- 172.21.15.49:0/1625124588 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff6a41007c0 0x7ff6a406aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:13.840 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:13.837+0000 7ff6ab9c6640 1 -- 172.21.15.49:0/1625124588 >> 172.21.15.49:0/1625124588 conn(0x7ff6a40fbd50 msgr2=0x7ff6a4104cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:06:13.840 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:13.837+0000 7ff6ab9c6640 1 -- 172.21.15.49:0/1625124588 shutdown_connections 2024-03-18T23:06:13.840 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:13.837+0000 7ff6ab9c6640 1 -- 172.21.15.49:0/1625124588 wait complete. 2024-03-18T23:06:13.848 INFO:teuthology.orchestra.run.smithi049.stdout:{"pg_ready":true,"pg_map":{"version":292,"stamp":"2024-03-18T23:06:11.898416+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":614408,"num_objects":67,"num_object_clones":0,"num_object_copies":201,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67,"num_whiteouts":0,"num_read":154,"num_read_kb":134,"num_write":328,"num_write_kb":2997,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":45,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":264,"ondisk_log_size":264,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":368712,"kb_used_data":11200,"kb_used_omap":18,"kb_used_meta":357357,"kb_avail":1124573112,"statfs":{"total":1151940427776,"available":1151562866688,"internally_reserved":0,"allocated":11468800,"data_stored":7121888,"data_compressed":108345,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":19142,"internal_metadata":365933882},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":24040,"num_objects":65,"num_object_clones":0,"num_object_copies":195,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":65,"num_whiteouts":0,"num_read":18,"num_read_kb":15,"num_write":147,"num_write_kb":59,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":45,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.701179"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T23:06:06.426899+0000","last_change":"2024-03-18T23:06:04.913788+0000","last_active":"2024-03-18T23:06:06.426899+0000","last_peered":"2024-03-18T23:06:06.426899+0000","last_clean":"2024-03-18T23:06:06.426899+0000","last_became_active":"2024-03-18T23:06:04.913103+0000","last_became_peered":"2024-03-18T23:06:04.913103+0000","last_unstale":"2024-03-18T23:06:06.426899+0000","last_undegraded":"2024-03-18T23:06:06.426899+0000","last_fullsized":"2024-03-18T23:06:06.426899+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:48:10.100291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T23:06:06.398424+0000","last_change":"2024-03-18T23:06:04.921126+0000","last_active":"2024-03-18T23:06:06.398424+0000","last_peered":"2024-03-18T23:06:06.398424+0000","last_clean":"2024-03-18T23:06:06.398424+0000","last_became_active":"2024-03-18T23:06:04.919934+0000","last_became_peered":"2024-03-18T23:06:04.919934+0000","last_unstale":"2024-03-18T23:06:06.398424+0000","last_undegraded":"2024-03-18T23:06:06.398424+0000","last_fullsized":"2024-03-18T23:06:06.398424+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:24:25.351428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T23:06:06.398267+0000","last_change":"2024-03-18T23:06:04.920933+0000","last_active":"2024-03-18T23:06:06.398267+0000","last_peered":"2024-03-18T23:06:06.398267+0000","last_clean":"2024-03-18T23:06:06.398267+0000","last_became_active":"2024-03-18T23:06:04.920548+0000","last_became_peered":"2024-03-18T23:06:04.920548+0000","last_unstale":"2024-03-18T23:06:06.398267+0000","last_undegraded":"2024-03-18T23:06:06.398267+0000","last_fullsized":"2024-03-18T23:06:06.398267+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:06:01.746208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T23:06:05.896767+0000","last_change":"2024-03-18T23:06:04.921219+0000","last_active":"2024-03-18T23:06:05.896767+0000","last_peered":"2024-03-18T23:06:05.896767+0000","last_clean":"2024-03-18T23:06:05.896767+0000","last_became_active":"2024-03-18T23:06:04.920069+0000","last_became_peered":"2024-03-18T23:06:04.920069+0000","last_unstale":"2024-03-18T23:06:05.896767+0000","last_undegraded":"2024-03-18T23:06:05.896767+0000","last_fullsized":"2024-03-18T23:06:05.896767+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:22:44.463061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T23:06:06.223923+0000","last_change":"2024-03-18T23:06:04.916741+0000","last_active":"2024-03-18T23:06:06.223923+0000","last_peered":"2024-03-18T23:06:06.223923+0000","last_clean":"2024-03-18T23:06:06.223923+0000","last_became_active":"2024-03-18T23:06:04.916434+0000","last_became_peered":"2024-03-18T23:06:04.916434+0000","last_unstale":"2024-03-18T23:06:06.223923+0000","last_undegraded":"2024-03-18T23:06:06.223923+0000","last_fullsized":"2024-03-18T23:06:06.223923+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:45:10.553306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T23:06:06.398243+0000","last_change":"2024-03-18T23:06:04.917198+0000","last_active":"2024-03-18T23:06:06.398243+0000","last_peered":"2024-03-18T23:06:06.398243+0000","last_clean":"2024-03-18T23:06:06.398243+0000","last_became_active":"2024-03-18T23:06:04.916897+0000","last_became_peered":"2024-03-18T23:06:04.916897+0000","last_unstale":"2024-03-18T23:06:06.398243+0000","last_undegraded":"2024-03-18T23:06:06.398243+0000","last_fullsized":"2024-03-18T23:06:06.398243+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T23:23:26.530668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T23:06:06.397867+0000","last_change":"2024-03-18T23:06:04.908446+0000","last_active":"2024-03-18T23:06:06.397867+0000","last_peered":"2024-03-18T23:06:06.397867+0000","last_clean":"2024-03-18T23:06:06.397867+0000","last_became_active":"2024-03-18T23:06:04.908093+0000","last_became_peered":"2024-03-18T23:06:04.908093+0000","last_unstale":"2024-03-18T23:06:06.397867+0000","last_undegraded":"2024-03-18T23:06:06.397867+0000","last_fullsized":"2024-03-18T23:06:06.397867+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T09:21:30.253229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T23:06:05.895448+0000","last_change":"2024-03-18T23:06:04.913080+0000","last_active":"2024-03-18T23:06:05.895448+0000","last_peered":"2024-03-18T23:06:05.895448+0000","last_clean":"2024-03-18T23:06:05.895448+0000","last_became_active":"2024-03-18T23:06:04.912255+0000","last_became_peered":"2024-03-18T23:06:04.912255+0000","last_unstale":"2024-03-18T23:06:05.895448+0000","last_undegraded":"2024-03-18T23:06:05.895448+0000","last_fullsized":"2024-03-18T23:06:05.895448+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:16:36.069545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T23:06:06.398207+0000","last_change":"2024-03-18T23:06:04.905284+0000","last_active":"2024-03-18T23:06:06.398207+0000","last_peered":"2024-03-18T23:06:06.398207+0000","last_clean":"2024-03-18T23:06:06.398207+0000","last_became_active":"2024-03-18T23:06:04.904388+0000","last_became_peered":"2024-03-18T23:06:04.904388+0000","last_unstale":"2024-03-18T23:06:06.398207+0000","last_undegraded":"2024-03-18T23:06:06.398207+0000","last_fullsized":"2024-03-18T23:06:06.398207+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:57:56.628599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T23:06:06.398697+0000","last_change":"2024-03-18T23:06:04.914602+0000","last_active":"2024-03-18T23:06:06.398697+0000","last_peered":"2024-03-18T23:06:06.398697+0000","last_clean":"2024-03-18T23:06:06.398697+0000","last_became_active":"2024-03-18T23:06:04.914288+0000","last_became_peered":"2024-03-18T23:06:04.914288+0000","last_unstale":"2024-03-18T23:06:06.398697+0000","last_undegraded":"2024-03-18T23:06:06.398697+0000","last_fullsized":"2024-03-18T23:06:06.398697+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:44:52.182950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T23:06:06.398239+0000","last_change":"2024-03-18T23:06:04.919821+0000","last_active":"2024-03-18T23:06:06.398239+0000","last_peered":"2024-03-18T23:06:06.398239+0000","last_clean":"2024-03-18T23:06:06.398239+0000","last_became_active":"2024-03-18T23:06:04.919525+0000","last_became_peered":"2024-03-18T23:06:04.919525+0000","last_unstale":"2024-03-18T23:06:06.398239+0000","last_undegraded":"2024-03-18T23:06:06.398239+0000","last_fullsized":"2024-03-18T23:06:06.398239+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:54:50.720229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T23:06:05.896703+0000","last_change":"2024-03-18T23:06:04.921904+0000","last_active":"2024-03-18T23:06:05.896703+0000","last_peered":"2024-03-18T23:06:05.896703+0000","last_clean":"2024-03-18T23:06:05.896703+0000","last_became_active":"2024-03-18T23:06:04.921581+0000","last_became_peered":"2024-03-18T23:06:04.921581+0000","last_unstale":"2024-03-18T23:06:05.896703+0000","last_undegraded":"2024-03-18T23:06:05.896703+0000","last_fullsized":"2024-03-18T23:06:05.896703+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:43:55.580983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T23:06:06.398062+0000","last_change":"2024-03-18T23:06:04.905399+0000","last_active":"2024-03-18T23:06:06.398062+0000","last_peered":"2024-03-18T23:06:06.398062+0000","last_clean":"2024-03-18T23:06:06.398062+0000","last_became_active":"2024-03-18T23:06:04.905117+0000","last_became_peered":"2024-03-18T23:06:04.905117+0000","last_unstale":"2024-03-18T23:06:06.398062+0000","last_undegraded":"2024-03-18T23:06:06.398062+0000","last_fullsized":"2024-03-18T23:06:06.398062+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:41:56.382244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T23:06:06.398151+0000","last_change":"2024-03-18T23:06:04.909993+0000","last_active":"2024-03-18T23:06:06.398151+0000","last_peered":"2024-03-18T23:06:06.398151+0000","last_clean":"2024-03-18T23:06:06.398151+0000","last_became_active":"2024-03-18T23:06:04.909483+0000","last_became_peered":"2024-03-18T23:06:04.909483+0000","last_unstale":"2024-03-18T23:06:06.398151+0000","last_undegraded":"2024-03-18T23:06:06.398151+0000","last_fullsized":"2024-03-18T23:06:06.398151+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:36:52.826582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T23:06:06.397933+0000","last_change":"2024-03-18T23:06:04.915273+0000","last_active":"2024-03-18T23:06:06.397933+0000","last_peered":"2024-03-18T23:06:06.397933+0000","last_clean":"2024-03-18T23:06:06.397933+0000","last_became_active":"2024-03-18T23:06:04.913510+0000","last_became_peered":"2024-03-18T23:06:04.913510+0000","last_unstale":"2024-03-18T23:06:06.397933+0000","last_undegraded":"2024-03-18T23:06:06.397933+0000","last_fullsized":"2024-03-18T23:06:06.397933+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:46:01.622963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T23:06:06.398462+0000","last_change":"2024-03-18T23:06:04.917599+0000","last_active":"2024-03-18T23:06:06.398462+0000","last_peered":"2024-03-18T23:06:06.398462+0000","last_clean":"2024-03-18T23:06:06.398462+0000","last_became_active":"2024-03-18T23:06:04.914961+0000","last_became_peered":"2024-03-18T23:06:04.914961+0000","last_unstale":"2024-03-18T23:06:06.398462+0000","last_undegraded":"2024-03-18T23:06:06.398462+0000","last_fullsized":"2024-03-18T23:06:06.398462+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T09:38:50.076075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T23:06:06.426992+0000","last_change":"2024-03-18T23:06:04.920814+0000","last_active":"2024-03-18T23:06:06.426992+0000","last_peered":"2024-03-18T23:06:06.426992+0000","last_clean":"2024-03-18T23:06:06.426992+0000","last_became_active":"2024-03-18T23:06:04.920484+0000","last_became_peered":"2024-03-18T23:06:04.920484+0000","last_unstale":"2024-03-18T23:06:06.426992+0000","last_undegraded":"2024-03-18T23:06:06.426992+0000","last_fullsized":"2024-03-18T23:06:06.426992+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:45:01.545059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T23:06:06.222283+0000","last_change":"2024-03-18T23:06:04.904684+0000","last_active":"2024-03-18T23:06:06.222283+0000","last_peered":"2024-03-18T23:06:06.222283+0000","last_clean":"2024-03-18T23:06:06.222283+0000","last_became_active":"2024-03-18T23:06:04.904296+0000","last_became_peered":"2024-03-18T23:06:04.904296+0000","last_unstale":"2024-03-18T23:06:06.222283+0000","last_undegraded":"2024-03-18T23:06:06.222283+0000","last_fullsized":"2024-03-18T23:06:06.222283+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:26:28.098286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T23:06:06.397802+0000","last_change":"2024-03-18T23:06:04.912803+0000","last_active":"2024-03-18T23:06:06.397802+0000","last_peered":"2024-03-18T23:06:06.397802+0000","last_clean":"2024-03-18T23:06:06.397802+0000","last_became_active":"2024-03-18T23:06:04.912495+0000","last_became_peered":"2024-03-18T23:06:04.912495+0000","last_unstale":"2024-03-18T23:06:06.397802+0000","last_undegraded":"2024-03-18T23:06:06.397802+0000","last_fullsized":"2024-03-18T23:06:06.397802+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:47:38.111260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"86'11","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T23:06:09.998428+0000","last_change":"2024-03-18T23:06:04.221492+0000","last_active":"2024-03-18T23:06:09.998428+0000","last_peered":"2024-03-18T23:06:09.998428+0000","last_clean":"2024-03-18T23:06:09.998428+0000","last_became_active":"2024-03-18T23:06:04.221190+0000","last_became_peered":"2024-03-18T23:06:04.221190+0000","last_unstale":"2024-03-18T23:06:09.998428+0000","last_undegraded":"2024-03-18T23:06:09.998428+0000","last_fullsized":"2024-03-18T23:06:09.998428+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_clean_scrub_stamp":"2024-03-18T23:06:03.184668+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:45:08.747471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T23:06:05.895828+0000","last_change":"2024-03-18T23:06:04.911680+0000","last_active":"2024-03-18T23:06:05.895828+0000","last_peered":"2024-03-18T23:06:05.895828+0000","last_clean":"2024-03-18T23:06:05.895828+0000","last_became_active":"2024-03-18T23:06:04.911356+0000","last_became_peered":"2024-03-18T23:06:04.911356+0000","last_unstale":"2024-03-18T23:06:05.895828+0000","last_undegraded":"2024-03-18T23:06:05.895828+0000","last_fullsized":"2024-03-18T23:06:05.895828+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:23:21.943304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"85'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T23:06:05.895726+0000","last_change":"2024-03-18T23:06:04.225792+0000","last_active":"2024-03-18T23:06:05.895726+0000","last_peered":"2024-03-18T23:06:05.895726+0000","last_clean":"2024-03-18T23:06:05.895726+0000","last_became_active":"2024-03-18T23:06:04.224248+0000","last_became_peered":"2024-03-18T23:06:04.224248+0000","last_unstale":"2024-03-18T23:06:05.895726+0000","last_undegraded":"2024-03-18T23:06:05.895726+0000","last_fullsized":"2024-03-18T23:06:05.895726+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_clean_scrub_stamp":"2024-03-18T23:06:03.184668+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:48:27.052831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T23:06:06.398518+0000","last_change":"2024-03-18T23:06:04.921139+0000","last_active":"2024-03-18T23:06:06.398518+0000","last_peered":"2024-03-18T23:06:06.398518+0000","last_clean":"2024-03-18T23:06:06.398518+0000","last_became_active":"2024-03-18T23:06:04.920700+0000","last_became_peered":"2024-03-18T23:06:04.920700+0000","last_unstale":"2024-03-18T23:06:06.398518+0000","last_undegraded":"2024-03-18T23:06:06.398518+0000","last_fullsized":"2024-03-18T23:06:06.398518+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T08:47:34.465176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"86'7","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T23:06:09.999475+0000","last_change":"2024-03-18T23:06:04.214134+0000","last_active":"2024-03-18T23:06:09.999475+0000","last_peered":"2024-03-18T23:06:09.999475+0000","last_clean":"2024-03-18T23:06:09.999475+0000","last_became_active":"2024-03-18T23:06:04.212256+0000","last_became_peered":"2024-03-18T23:06:04.212256+0000","last_unstale":"2024-03-18T23:06:09.999475+0000","last_undegraded":"2024-03-18T23:06:09.999475+0000","last_fullsized":"2024-03-18T23:06:09.999475+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_clean_scrub_stamp":"2024-03-18T23:06:03.184668+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:39:46.851964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T23:06:06.397748+0000","last_change":"2024-03-18T23:06:04.907465+0000","last_active":"2024-03-18T23:06:06.397748+0000","last_peered":"2024-03-18T23:06:06.397748+0000","last_clean":"2024-03-18T23:06:06.397748+0000","last_became_active":"2024-03-18T23:06:04.907142+0000","last_became_peered":"2024-03-18T23:06:04.907142+0000","last_unstale":"2024-03-18T23:06:06.397748+0000","last_undegraded":"2024-03-18T23:06:06.397748+0000","last_fullsized":"2024-03-18T23:06:06.397748+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:15:29.066708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T23:06:06.224063+0000","last_change":"2024-03-18T23:06:04.213694+0000","last_active":"2024-03-18T23:06:06.224063+0000","last_peered":"2024-03-18T23:06:06.224063+0000","last_clean":"2024-03-18T23:06:06.224063+0000","last_became_active":"2024-03-18T23:06:04.213381+0000","last_became_peered":"2024-03-18T23:06:04.213381+0000","last_unstale":"2024-03-18T23:06:06.224063+0000","last_undegraded":"2024-03-18T23:06:06.224063+0000","last_fullsized":"2024-03-18T23:06:06.224063+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_clean_scrub_stamp":"2024-03-18T23:06:03.184668+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:08:44.502915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T23:06:05.896172+0000","last_change":"2024-03-18T23:06:04.918596+0000","last_active":"2024-03-18T23:06:05.896172+0000","last_peered":"2024-03-18T23:06:05.896172+0000","last_clean":"2024-03-18T23:06:05.896172+0000","last_became_active":"2024-03-18T23:06:04.918273+0000","last_became_peered":"2024-03-18T23:06:04.918273+0000","last_unstale":"2024-03-18T23:06:05.896172+0000","last_undegraded":"2024-03-18T23:06:05.896172+0000","last_fullsized":"2024-03-18T23:06:05.896172+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T08:09:04.860241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"86'5","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T23:06:08.250233+0000","last_change":"2024-03-18T23:06:04.214573+0000","last_active":"2024-03-18T23:06:08.250233+0000","last_peered":"2024-03-18T23:06:08.250233+0000","last_clean":"2024-03-18T23:06:08.250233+0000","last_became_active":"2024-03-18T23:06:04.213891+0000","last_became_peered":"2024-03-18T23:06:04.213891+0000","last_unstale":"2024-03-18T23:06:08.250233+0000","last_undegraded":"2024-03-18T23:06:08.250233+0000","last_fullsized":"2024-03-18T23:06:08.250233+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_clean_scrub_stamp":"2024-03-18T23:06:03.184668+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:17:22.895024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T23:06:06.223967+0000","last_change":"2024-03-18T23:06:04.908336+0000","last_active":"2024-03-18T23:06:06.223967+0000","last_peered":"2024-03-18T23:06:06.223967+0000","last_clean":"2024-03-18T23:06:06.223967+0000","last_became_active":"2024-03-18T23:06:04.908023+0000","last_became_peered":"2024-03-18T23:06:04.908023+0000","last_unstale":"2024-03-18T23:06:06.223967+0000","last_undegraded":"2024-03-18T23:06:06.223967+0000","last_fullsized":"2024-03-18T23:06:06.223967+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:34:50.993715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"86'7","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T23:06:10.000008+0000","last_change":"2024-03-18T23:06:04.210036+0000","last_active":"2024-03-18T23:06:10.000008+0000","last_peered":"2024-03-18T23:06:10.000008+0000","last_clean":"2024-03-18T23:06:10.000008+0000","last_became_active":"2024-03-18T23:06:04.209658+0000","last_became_peered":"2024-03-18T23:06:04.209658+0000","last_unstale":"2024-03-18T23:06:10.000008+0000","last_undegraded":"2024-03-18T23:06:10.000008+0000","last_fullsized":"2024-03-18T23:06:10.000008+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_clean_scrub_stamp":"2024-03-18T23:06:03.184668+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T23:55:37.286542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T23:06:05.896078+0000","last_change":"2024-03-18T23:06:04.917909+0000","last_active":"2024-03-18T23:06:05.896078+0000","last_peered":"2024-03-18T23:06:05.896078+0000","last_clean":"2024-03-18T23:06:05.896078+0000","last_became_active":"2024-03-18T23:06:04.917305+0000","last_became_peered":"2024-03-18T23:06:04.917305+0000","last_unstale":"2024-03-18T23:06:05.896078+0000","last_undegraded":"2024-03-18T23:06:05.896078+0000","last_fullsized":"2024-03-18T23:06:05.896078+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:39:48.953530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"86'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T23:06:08.996518+0000","last_change":"2024-03-18T23:06:04.216030+0000","last_active":"2024-03-18T23:06:08.996518+0000","last_peered":"2024-03-18T23:06:08.996518+0000","last_clean":"2024-03-18T23:06:08.996518+0000","last_became_active":"2024-03-18T23:06:04.215695+0000","last_became_peered":"2024-03-18T23:06:04.215695+0000","last_unstale":"2024-03-18T23:06:08.996518+0000","last_undegraded":"2024-03-18T23:06:08.996518+0000","last_fullsized":"2024-03-18T23:06:08.996518+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_clean_scrub_stamp":"2024-03-18T23:06:03.184668+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:48:37.742916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T23:06:05.895518+0000","last_change":"2024-03-18T23:06:04.910749+0000","last_active":"2024-03-18T23:06:05.895518+0000","last_peered":"2024-03-18T23:06:05.895518+0000","last_clean":"2024-03-18T23:06:05.895518+0000","last_became_active":"2024-03-18T23:06:04.910313+0000","last_became_peered":"2024-03-18T23:06:04.910313+0000","last_unstale":"2024-03-18T23:06:05.895518+0000","last_undegraded":"2024-03-18T23:06:05.895518+0000","last_fullsized":"2024-03-18T23:06:05.895518+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:18:19.726537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"86'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T23:06:05.895679+0000","last_change":"2024-03-18T23:06:04.218263+0000","last_active":"2024-03-18T23:06:05.895679+0000","last_peered":"2024-03-18T23:06:05.895679+0000","last_clean":"2024-03-18T23:06:05.895679+0000","last_became_active":"2024-03-18T23:06:04.217943+0000","last_became_peered":"2024-03-18T23:06:04.217943+0000","last_unstale":"2024-03-18T23:06:05.895679+0000","last_undegraded":"2024-03-18T23:06:05.895679+0000","last_fullsized":"2024-03-18T23:06:05.895679+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_clean_scrub_stamp":"2024-03-18T23:06:03.184668+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:25:11.632767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T23:06:06.222012+0000","last_change":"2024-03-18T23:06:04.915905+0000","last_active":"2024-03-18T23:06:06.222012+0000","last_peered":"2024-03-18T23:06:06.222012+0000","last_clean":"2024-03-18T23:06:06.222012+0000","last_became_active":"2024-03-18T23:06:04.915584+0000","last_became_peered":"2024-03-18T23:06:04.915584+0000","last_unstale":"2024-03-18T23:06:06.222012+0000","last_undegraded":"2024-03-18T23:06:06.222012+0000","last_fullsized":"2024-03-18T23:06:06.222012+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:29:05.503861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"86'4","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T23:06:09.998963+0000","last_change":"2024-03-18T23:06:04.217077+0000","last_active":"2024-03-18T23:06:09.998963+0000","last_peered":"2024-03-18T23:06:09.998963+0000","last_clean":"2024-03-18T23:06:09.998963+0000","last_became_active":"2024-03-18T23:06:04.216308+0000","last_became_peered":"2024-03-18T23:06:04.216308+0000","last_unstale":"2024-03-18T23:06:09.998963+0000","last_undegraded":"2024-03-18T23:06:09.998963+0000","last_fullsized":"2024-03-18T23:06:09.998963+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_clean_scrub_stamp":"2024-03-18T23:06:03.184668+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T23:39:58.537740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T23:06:06.398686+0000","last_change":"2024-03-18T23:06:04.906669+0000","last_active":"2024-03-18T23:06:06.398686+0000","last_peered":"2024-03-18T23:06:06.398686+0000","last_clean":"2024-03-18T23:06:06.398686+0000","last_became_active":"2024-03-18T23:06:04.906338+0000","last_became_peered":"2024-03-18T23:06:04.906338+0000","last_unstale":"2024-03-18T23:06:06.398686+0000","last_undegraded":"2024-03-18T23:06:06.398686+0000","last_fullsized":"2024-03-18T23:06:06.398686+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:29:39.997671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"86'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T23:06:08.249231+0000","last_change":"2024-03-18T23:06:04.216177+0000","last_active":"2024-03-18T23:06:08.249231+0000","last_peered":"2024-03-18T23:06:08.249231+0000","last_clean":"2024-03-18T23:06:08.249231+0000","last_became_active":"2024-03-18T23:06:04.215890+0000","last_became_peered":"2024-03-18T23:06:04.215890+0000","last_unstale":"2024-03-18T23:06:08.249231+0000","last_undegraded":"2024-03-18T23:06:08.249231+0000","last_fullsized":"2024-03-18T23:06:08.249231+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_clean_scrub_stamp":"2024-03-18T23:06:03.184668+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T09:33:52.578996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T23:06:06.398307+0000","last_change":"2024-03-18T23:06:04.911928+0000","last_active":"2024-03-18T23:06:06.398307+0000","last_peered":"2024-03-18T23:06:06.398307+0000","last_clean":"2024-03-18T23:06:06.398307+0000","last_became_active":"2024-03-18T23:06:04.911501+0000","last_became_peered":"2024-03-18T23:06:04.911501+0000","last_unstale":"2024-03-18T23:06:06.398307+0000","last_undegraded":"2024-03-18T23:06:06.398307+0000","last_fullsized":"2024-03-18T23:06:06.398307+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:05:17.653574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"86'4","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T23:06:05.896036+0000","last_change":"2024-03-18T23:06:04.211035+0000","last_active":"2024-03-18T23:06:05.896036+0000","last_peered":"2024-03-18T23:06:05.896036+0000","last_clean":"2024-03-18T23:06:05.896036+0000","last_became_active":"2024-03-18T23:06:04.210665+0000","last_became_peered":"2024-03-18T23:06:04.210665+0000","last_unstale":"2024-03-18T23:06:05.896036+0000","last_undegraded":"2024-03-18T23:06:05.896036+0000","last_fullsized":"2024-03-18T23:06:05.896036+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_clean_scrub_stamp":"2024-03-18T23:06:03.184668+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:49:44.072275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T23:06:06.398050+0000","last_change":"2024-03-18T23:06:04.911856+0000","last_active":"2024-03-18T23:06:06.398050+0000","last_peered":"2024-03-18T23:06:06.398050+0000","last_clean":"2024-03-18T23:06:06.398050+0000","last_became_active":"2024-03-18T23:06:04.908034+0000","last_became_peered":"2024-03-18T23:06:04.908034+0000","last_unstale":"2024-03-18T23:06:06.398050+0000","last_undegraded":"2024-03-18T23:06:06.398050+0000","last_fullsized":"2024-03-18T23:06:06.398050+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:38:09.712683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"86'17","reported_seq":40,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T23:06:11.379605+0000","last_change":"2024-03-18T23:06:04.211152+0000","last_active":"2024-03-18T23:06:11.379605+0000","last_peered":"2024-03-18T23:06:11.379605+0000","last_clean":"2024-03-18T23:06:11.379605+0000","last_became_active":"2024-03-18T23:06:04.210841+0000","last_became_peered":"2024-03-18T23:06:04.210841+0000","last_unstale":"2024-03-18T23:06:11.379605+0000","last_undegraded":"2024-03-18T23:06:11.379605+0000","last_fullsized":"2024-03-18T23:06:11.379605+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_clean_scrub_stamp":"2024-03-18T23:06:03.184668+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:48:07.935314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20895,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":8,"num_read_kb":5,"num_write":19,"num_write_kb":24,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T23:06:06.224187+0000","last_change":"2024-03-18T23:06:04.910303+0000","last_active":"2024-03-18T23:06:06.224187+0000","last_peered":"2024-03-18T23:06:06.224187+0000","last_clean":"2024-03-18T23:06:06.224187+0000","last_became_active":"2024-03-18T23:06:04.910004+0000","last_became_peered":"2024-03-18T23:06:04.910004+0000","last_unstale":"2024-03-18T23:06:06.224187+0000","last_undegraded":"2024-03-18T23:06:06.224187+0000","last_fullsized":"2024-03-18T23:06:06.224187+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:45:25.750741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"86'4","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T23:06:08.248808+0000","last_change":"2024-03-18T23:06:04.220318+0000","last_active":"2024-03-18T23:06:08.248808+0000","last_peered":"2024-03-18T23:06:08.248808+0000","last_clean":"2024-03-18T23:06:08.248808+0000","last_became_active":"2024-03-18T23:06:04.219985+0000","last_became_peered":"2024-03-18T23:06:04.219985+0000","last_unstale":"2024-03-18T23:06:08.248808+0000","last_undegraded":"2024-03-18T23:06:08.248808+0000","last_fullsized":"2024-03-18T23:06:08.248808+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_clean_scrub_stamp":"2024-03-18T23:06:03.184668+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:49:00.090356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T23:06:06.398008+0000","last_change":"2024-03-18T23:06:04.914456+0000","last_active":"2024-03-18T23:06:06.398008+0000","last_peered":"2024-03-18T23:06:06.398008+0000","last_clean":"2024-03-18T23:06:06.398008+0000","last_became_active":"2024-03-18T23:06:04.914083+0000","last_became_peered":"2024-03-18T23:06:04.914083+0000","last_unstale":"2024-03-18T23:06:06.398008+0000","last_undegraded":"2024-03-18T23:06:06.398008+0000","last_fullsized":"2024-03-18T23:06:06.398008+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:17:00.443254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"86'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T23:06:08.249795+0000","last_change":"2024-03-18T23:06:04.224808+0000","last_active":"2024-03-18T23:06:08.249795+0000","last_peered":"2024-03-18T23:06:08.249795+0000","last_clean":"2024-03-18T23:06:08.249795+0000","last_became_active":"2024-03-18T23:06:04.224112+0000","last_became_peered":"2024-03-18T23:06:04.224112+0000","last_unstale":"2024-03-18T23:06:08.249795+0000","last_undegraded":"2024-03-18T23:06:08.249795+0000","last_fullsized":"2024-03-18T23:06:08.249795+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_clean_scrub_stamp":"2024-03-18T23:06:03.184668+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:19:20.399875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T23:06:06.398094+0000","last_change":"2024-03-18T23:06:04.912271+0000","last_active":"2024-03-18T23:06:06.398094+0000","last_peered":"2024-03-18T23:06:06.398094+0000","last_clean":"2024-03-18T23:06:06.398094+0000","last_became_active":"2024-03-18T23:06:04.911985+0000","last_became_peered":"2024-03-18T23:06:04.911985+0000","last_unstale":"2024-03-18T23:06:06.398094+0000","last_undegraded":"2024-03-18T23:06:06.398094+0000","last_fullsized":"2024-03-18T23:06:06.398094+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T09:18:27.436136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"86'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T23:06:08.995641+0000","last_change":"2024-03-18T23:06:04.209434+0000","last_active":"2024-03-18T23:06:08.995641+0000","last_peered":"2024-03-18T23:06:08.995641+0000","last_clean":"2024-03-18T23:06:08.995641+0000","last_became_active":"2024-03-18T23:06:04.209086+0000","last_became_peered":"2024-03-18T23:06:04.209086+0000","last_unstale":"2024-03-18T23:06:08.995641+0000","last_undegraded":"2024-03-18T23:06:08.995641+0000","last_fullsized":"2024-03-18T23:06:08.995641+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_clean_scrub_stamp":"2024-03-18T23:06:03.184668+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:50:20.823467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T23:06:06.398913+0000","last_change":"2024-03-18T23:06:04.911069+0000","last_active":"2024-03-18T23:06:06.398913+0000","last_peered":"2024-03-18T23:06:06.398913+0000","last_clean":"2024-03-18T23:06:06.398913+0000","last_became_active":"2024-03-18T23:06:04.910760+0000","last_became_peered":"2024-03-18T23:06:04.910760+0000","last_unstale":"2024-03-18T23:06:06.398913+0000","last_undegraded":"2024-03-18T23:06:06.398913+0000","last_fullsized":"2024-03-18T23:06:06.398913+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:16:46.264972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"86'16","reported_seq":31,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T23:06:09.000423+0000","last_change":"2024-03-18T23:06:04.216653+0000","last_active":"2024-03-18T23:06:09.000423+0000","last_peered":"2024-03-18T23:06:09.000423+0000","last_clean":"2024-03-18T23:06:09.000423+0000","last_became_active":"2024-03-18T23:06:04.216358+0000","last_became_peered":"2024-03-18T23:06:04.216358+0000","last_unstale":"2024-03-18T23:06:09.000423+0000","last_undegraded":"2024-03-18T23:06:09.000423+0000","last_fullsized":"2024-03-18T23:06:09.000423+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_clean_scrub_stamp":"2024-03-18T23:06:03.184668+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:16:56.500368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T23:06:05.895929+0000","last_change":"2024-03-18T23:06:04.917377+0000","last_active":"2024-03-18T23:06:05.895929+0000","last_peered":"2024-03-18T23:06:05.895929+0000","last_clean":"2024-03-18T23:06:05.895929+0000","last_became_active":"2024-03-18T23:06:04.917064+0000","last_became_peered":"2024-03-18T23:06:04.917064+0000","last_unstale":"2024-03-18T23:06:05.895929+0000","last_undegraded":"2024-03-18T23:06:05.895929+0000","last_fullsized":"2024-03-18T23:06:05.895929+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:03:41.518177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"86'2","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T23:06:08.982597+0000","last_change":"2024-03-18T23:06:04.210191+0000","last_active":"2024-03-18T23:06:08.982597+0000","last_peered":"2024-03-18T23:06:08.982597+0000","last_clean":"2024-03-18T23:06:08.982597+0000","last_became_active":"2024-03-18T23:06:04.208084+0000","last_became_peered":"2024-03-18T23:06:04.208084+0000","last_unstale":"2024-03-18T23:06:08.982597+0000","last_undegraded":"2024-03-18T23:06:08.982597+0000","last_fullsized":"2024-03-18T23:06:08.982597+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_clean_scrub_stamp":"2024-03-18T23:06:03.184668+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:09:07.220171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T23:06:06.426844+0000","last_change":"2024-03-18T23:06:04.910631+0000","last_active":"2024-03-18T23:06:06.426844+0000","last_peered":"2024-03-18T23:06:06.426844+0000","last_clean":"2024-03-18T23:06:06.426844+0000","last_became_active":"2024-03-18T23:06:04.910322+0000","last_became_peered":"2024-03-18T23:06:04.910322+0000","last_unstale":"2024-03-18T23:06:06.426844+0000","last_undegraded":"2024-03-18T23:06:06.426844+0000","last_fullsized":"2024-03-18T23:06:06.426844+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:39:06.389101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"86'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T23:06:06.222213+0000","last_change":"2024-03-18T23:06:04.220530+0000","last_active":"2024-03-18T23:06:06.222213+0000","last_peered":"2024-03-18T23:06:06.222213+0000","last_clean":"2024-03-18T23:06:06.222213+0000","last_became_active":"2024-03-18T23:06:04.220136+0000","last_became_peered":"2024-03-18T23:06:04.220136+0000","last_unstale":"2024-03-18T23:06:06.222213+0000","last_undegraded":"2024-03-18T23:06:06.222213+0000","last_fullsized":"2024-03-18T23:06:06.222213+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_clean_scrub_stamp":"2024-03-18T23:06:03.184668+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:58:37.037982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"86'1","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T23:06:06.595838+0000","last_change":"2024-03-18T23:06:04.912789+0000","last_active":"2024-03-18T23:06:06.595838+0000","last_peered":"2024-03-18T23:06:06.595838+0000","last_clean":"2024-03-18T23:06:06.595838+0000","last_became_active":"2024-03-18T23:06:04.910451+0000","last_became_peered":"2024-03-18T23:06:04.910451+0000","last_unstale":"2024-03-18T23:06:06.595838+0000","last_undegraded":"2024-03-18T23:06:06.595838+0000","last_fullsized":"2024-03-18T23:06:06.595838+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:06:36.539369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"86'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T23:06:06.398356+0000","last_change":"2024-03-18T23:06:04.217992+0000","last_active":"2024-03-18T23:06:06.398356+0000","last_peered":"2024-03-18T23:06:06.398356+0000","last_clean":"2024-03-18T23:06:06.398356+0000","last_became_active":"2024-03-18T23:06:04.217651+0000","last_became_peered":"2024-03-18T23:06:04.217651+0000","last_unstale":"2024-03-18T23:06:06.398356+0000","last_undegraded":"2024-03-18T23:06:06.398356+0000","last_fullsized":"2024-03-18T23:06:06.398356+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_clean_scrub_stamp":"2024-03-18T23:06:03.184668+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:25:46.901251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T23:06:06.398714+0000","last_change":"2024-03-18T23:06:04.913349+0000","last_active":"2024-03-18T23:06:06.398714+0000","last_peered":"2024-03-18T23:06:06.398714+0000","last_clean":"2024-03-18T23:06:06.398714+0000","last_became_active":"2024-03-18T23:06:04.913026+0000","last_became_peered":"2024-03-18T23:06:04.913026+0000","last_unstale":"2024-03-18T23:06:06.398714+0000","last_undegraded":"2024-03-18T23:06:06.398714+0000","last_fullsized":"2024-03-18T23:06:06.398714+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T23:24:28.884524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T23:06:05.896023+0000","last_change":"2024-03-18T23:06:04.916541+0000","last_active":"2024-03-18T23:06:05.896023+0000","last_peered":"2024-03-18T23:06:05.896023+0000","last_clean":"2024-03-18T23:06:05.896023+0000","last_became_active":"2024-03-18T23:06:04.916179+0000","last_became_peered":"2024-03-18T23:06:04.916179+0000","last_unstale":"2024-03-18T23:06:05.896023+0000","last_undegraded":"2024-03-18T23:06:05.896023+0000","last_fullsized":"2024-03-18T23:06:05.896023+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:31:59.515767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"86'4","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T23:06:08.999385+0000","last_change":"2024-03-18T23:06:04.217581+0000","last_active":"2024-03-18T23:06:08.999385+0000","last_peered":"2024-03-18T23:06:08.999385+0000","last_clean":"2024-03-18T23:06:08.999385+0000","last_became_active":"2024-03-18T23:06:04.217191+0000","last_became_peered":"2024-03-18T23:06:04.217191+0000","last_unstale":"2024-03-18T23:06:08.999385+0000","last_undegraded":"2024-03-18T23:06:08.999385+0000","last_fullsized":"2024-03-18T23:06:08.999385+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_clean_scrub_stamp":"2024-03-18T23:06:03.184668+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:47:24.728136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T23:06:05.895759+0000","last_change":"2024-03-18T23:06:04.919784+0000","last_active":"2024-03-18T23:06:05.895759+0000","last_peered":"2024-03-18T23:06:05.895759+0000","last_clean":"2024-03-18T23:06:05.895759+0000","last_became_active":"2024-03-18T23:06:04.919483+0000","last_became_peered":"2024-03-18T23:06:04.919483+0000","last_unstale":"2024-03-18T23:06:05.895759+0000","last_undegraded":"2024-03-18T23:06:05.895759+0000","last_fullsized":"2024-03-18T23:06:05.895759+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:37:29.635750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'3","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T23:06:08.984490+0000","last_change":"2024-03-18T23:06:04.215152+0000","last_active":"2024-03-18T23:06:08.984490+0000","last_peered":"2024-03-18T23:06:08.984490+0000","last_clean":"2024-03-18T23:06:08.984490+0000","last_became_active":"2024-03-18T23:06:04.214789+0000","last_became_peered":"2024-03-18T23:06:04.214789+0000","last_unstale":"2024-03-18T23:06:08.984490+0000","last_undegraded":"2024-03-18T23:06:08.984490+0000","last_fullsized":"2024-03-18T23:06:08.984490+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_clean_scrub_stamp":"2024-03-18T23:06:03.184668+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:58:11.630840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":349,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T23:06:06.398951+0000","last_change":"2024-03-18T23:04:54.734000+0000","last_active":"2024-03-18T23:06:06.398951+0000","last_peered":"2024-03-18T23:06:06.398951+0000","last_clean":"2024-03-18T23:06:06.398951+0000","last_became_active":"2024-03-18T23:04:54.733516+0000","last_became_peered":"2024-03-18T23:04:54.733516+0000","last_unstale":"2024-03-18T23:06:06.398951+0000","last_undegraded":"2024-03-18T23:06:06.398951+0000","last_fullsized":"2024-03-18T23:06:06.398951+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:01:28.441034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:01:28.441034+0000","last_clean_scrub_stamp":"2024-03-18T23:01:28.441034+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T23:48:14.029479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T23:06:06.397785+0000","last_change":"2024-03-18T23:06:04.908549+0000","last_active":"2024-03-18T23:06:06.397785+0000","last_peered":"2024-03-18T23:06:06.397785+0000","last_clean":"2024-03-18T23:06:06.397785+0000","last_became_active":"2024-03-18T23:06:04.908202+0000","last_became_peered":"2024-03-18T23:06:04.908202+0000","last_unstale":"2024-03-18T23:06:06.397785+0000","last_undegraded":"2024-03-18T23:06:06.397785+0000","last_fullsized":"2024-03-18T23:06:06.397785+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:57:09.486103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T23:06:08.994816+0000","last_change":"2024-03-18T23:06:04.213819+0000","last_active":"2024-03-18T23:06:08.994816+0000","last_peered":"2024-03-18T23:06:08.994816+0000","last_clean":"2024-03-18T23:06:08.994816+0000","last_became_active":"2024-03-18T23:06:04.213418+0000","last_became_peered":"2024-03-18T23:06:04.213418+0000","last_unstale":"2024-03-18T23:06:08.994816+0000","last_undegraded":"2024-03-18T23:06:08.994816+0000","last_fullsized":"2024-03-18T23:06:08.994816+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_clean_scrub_stamp":"2024-03-18T23:06:03.184668+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:50:03.175779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"86'3","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T23:06:09.999688+0000","last_change":"2024-03-18T23:06:04.218660+0000","last_active":"2024-03-18T23:06:09.999688+0000","last_peered":"2024-03-18T23:06:09.999688+0000","last_clean":"2024-03-18T23:06:09.999688+0000","last_became_active":"2024-03-18T23:06:04.218183+0000","last_became_peered":"2024-03-18T23:06:04.218183+0000","last_unstale":"2024-03-18T23:06:09.999688+0000","last_undegraded":"2024-03-18T23:06:09.999688+0000","last_fullsized":"2024-03-18T23:06:09.999688+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_clean_scrub_stamp":"2024-03-18T23:06:03.184668+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T08:17:42.208572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T23:06:06.398458+0000","last_change":"2024-03-18T23:06:04.901124+0000","last_active":"2024-03-18T23:06:06.398458+0000","last_peered":"2024-03-18T23:06:06.398458+0000","last_clean":"2024-03-18T23:06:06.398458+0000","last_became_active":"2024-03-18T23:06:04.899992+0000","last_became_peered":"2024-03-18T23:06:04.899992+0000","last_unstale":"2024-03-18T23:06:06.398458+0000","last_undegraded":"2024-03-18T23:06:06.398458+0000","last_fullsized":"2024-03-18T23:06:06.398458+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:27:00.965945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"86'7","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T23:06:09.999467+0000","last_change":"2024-03-18T23:06:04.227413+0000","last_active":"2024-03-18T23:06:09.999467+0000","last_peered":"2024-03-18T23:06:09.999467+0000","last_clean":"2024-03-18T23:06:09.999467+0000","last_became_active":"2024-03-18T23:06:04.227071+0000","last_became_peered":"2024-03-18T23:06:04.227071+0000","last_unstale":"2024-03-18T23:06:09.999467+0000","last_undegraded":"2024-03-18T23:06:09.999467+0000","last_fullsized":"2024-03-18T23:06:09.999467+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_clean_scrub_stamp":"2024-03-18T23:06:03.184668+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T08:15:31.531168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T23:06:05.895866+0000","last_change":"2024-03-18T23:06:04.910220+0000","last_active":"2024-03-18T23:06:05.895866+0000","last_peered":"2024-03-18T23:06:05.895866+0000","last_clean":"2024-03-18T23:06:05.895866+0000","last_became_active":"2024-03-18T23:06:04.909855+0000","last_became_peered":"2024-03-18T23:06:04.909855+0000","last_unstale":"2024-03-18T23:06:05.895866+0000","last_undegraded":"2024-03-18T23:06:05.895866+0000","last_fullsized":"2024-03-18T23:06:05.895866+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:19:09.315343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"86'4","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T23:06:08.247933+0000","last_change":"2024-03-18T23:06:04.225473+0000","last_active":"2024-03-18T23:06:08.247933+0000","last_peered":"2024-03-18T23:06:08.247933+0000","last_clean":"2024-03-18T23:06:08.247933+0000","last_became_active":"2024-03-18T23:06:04.225126+0000","last_became_peered":"2024-03-18T23:06:04.225126+0000","last_unstale":"2024-03-18T23:06:08.247933+0000","last_undegraded":"2024-03-18T23:06:08.247933+0000","last_fullsized":"2024-03-18T23:06:08.247933+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_clean_scrub_stamp":"2024-03-18T23:06:03.184668+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T09:29:26.438597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T23:06:06.398430+0000","last_change":"2024-03-18T23:06:04.911110+0000","last_active":"2024-03-18T23:06:06.398430+0000","last_peered":"2024-03-18T23:06:06.398430+0000","last_clean":"2024-03-18T23:06:06.398430+0000","last_became_active":"2024-03-18T23:06:04.910752+0000","last_became_peered":"2024-03-18T23:06:04.910752+0000","last_unstale":"2024-03-18T23:06:06.398430+0000","last_undegraded":"2024-03-18T23:06:06.398430+0000","last_fullsized":"2024-03-18T23:06:06.398430+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:40:44.736472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T23:06:08.249352+0000","last_change":"2024-03-18T23:06:04.216750+0000","last_active":"2024-03-18T23:06:08.249352+0000","last_peered":"2024-03-18T23:06:08.249352+0000","last_clean":"2024-03-18T23:06:08.249352+0000","last_became_active":"2024-03-18T23:06:04.216393+0000","last_became_peered":"2024-03-18T23:06:04.216393+0000","last_unstale":"2024-03-18T23:06:08.249352+0000","last_undegraded":"2024-03-18T23:06:08.249352+0000","last_fullsized":"2024-03-18T23:06:08.249352+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_clean_scrub_stamp":"2024-03-18T23:06:03.184668+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:40:33.486235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T23:06:06.223867+0000","last_change":"2024-03-18T23:06:04.915156+0000","last_active":"2024-03-18T23:06:06.223867+0000","last_peered":"2024-03-18T23:06:06.223867+0000","last_clean":"2024-03-18T23:06:06.223867+0000","last_became_active":"2024-03-18T23:06:04.914821+0000","last_became_peered":"2024-03-18T23:06:04.914821+0000","last_unstale":"2024-03-18T23:06:06.223867+0000","last_undegraded":"2024-03-18T23:06:06.223867+0000","last_fullsized":"2024-03-18T23:06:06.223867+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:41:22.344360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T23:06:05.895320+0000","last_change":"2024-03-18T23:06:04.224533+0000","last_active":"2024-03-18T23:06:05.895320+0000","last_peered":"2024-03-18T23:06:05.895320+0000","last_clean":"2024-03-18T23:06:05.895320+0000","last_became_active":"2024-03-18T23:06:04.224122+0000","last_became_peered":"2024-03-18T23:06:04.224122+0000","last_unstale":"2024-03-18T23:06:05.895320+0000","last_undegraded":"2024-03-18T23:06:05.895320+0000","last_fullsized":"2024-03-18T23:06:05.895320+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_clean_scrub_stamp":"2024-03-18T23:06:03.184668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T09:03:40.689085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T23:06:06.426650+0000","last_change":"2024-03-18T23:06:04.912574+0000","last_active":"2024-03-18T23:06:06.426650+0000","last_peered":"2024-03-18T23:06:06.426650+0000","last_clean":"2024-03-18T23:06:06.426650+0000","last_became_active":"2024-03-18T23:06:04.912284+0000","last_became_peered":"2024-03-18T23:06:04.912284+0000","last_unstale":"2024-03-18T23:06:06.426650+0000","last_undegraded":"2024-03-18T23:06:06.426650+0000","last_fullsized":"2024-03-18T23:06:06.426650+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:18:11.003763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T23:06:06.397470+0000","last_change":"2024-03-18T23:06:04.220364+0000","last_active":"2024-03-18T23:06:06.397470+0000","last_peered":"2024-03-18T23:06:06.397470+0000","last_clean":"2024-03-18T23:06:06.397470+0000","last_became_active":"2024-03-18T23:06:04.220074+0000","last_became_peered":"2024-03-18T23:06:04.220074+0000","last_unstale":"2024-03-18T23:06:06.397470+0000","last_undegraded":"2024-03-18T23:06:06.397470+0000","last_fullsized":"2024-03-18T23:06:06.397470+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_clean_scrub_stamp":"2024-03-18T23:06:03.184668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:10:15.385724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T23:06:05.895832+0000","last_change":"2024-03-18T23:06:04.917448+0000","last_active":"2024-03-18T23:06:05.895832+0000","last_peered":"2024-03-18T23:06:05.895832+0000","last_clean":"2024-03-18T23:06:05.895832+0000","last_became_active":"2024-03-18T23:06:04.916921+0000","last_became_peered":"2024-03-18T23:06:04.916921+0000","last_unstale":"2024-03-18T23:06:05.895832+0000","last_undegraded":"2024-03-18T23:06:05.895832+0000","last_fullsized":"2024-03-18T23:06:05.895832+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:23:06.715597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T23:06:08.249738+0000","last_change":"2024-03-18T23:06:04.211777+0000","last_active":"2024-03-18T23:06:08.249738+0000","last_peered":"2024-03-18T23:06:08.249738+0000","last_clean":"2024-03-18T23:06:08.249738+0000","last_became_active":"2024-03-18T23:06:04.211454+0000","last_became_peered":"2024-03-18T23:06:04.211454+0000","last_unstale":"2024-03-18T23:06:08.249738+0000","last_undegraded":"2024-03-18T23:06:08.249738+0000","last_fullsized":"2024-03-18T23:06:08.249738+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_clean_scrub_stamp":"2024-03-18T23:06:03.184668+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:52:14.010341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T23:06:06.398369+0000","last_change":"2024-03-18T23:06:04.911471+0000","last_active":"2024-03-18T23:06:06.398369+0000","last_peered":"2024-03-18T23:06:06.398369+0000","last_clean":"2024-03-18T23:06:06.398369+0000","last_became_active":"2024-03-18T23:06:04.910550+0000","last_became_peered":"2024-03-18T23:06:04.910550+0000","last_unstale":"2024-03-18T23:06:06.398369+0000","last_undegraded":"2024-03-18T23:06:06.398369+0000","last_fullsized":"2024-03-18T23:06:06.398369+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:34:55.084581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T23:06:06.397672+0000","last_change":"2024-03-18T23:06:04.215989+0000","last_active":"2024-03-18T23:06:06.397672+0000","last_peered":"2024-03-18T23:06:06.397672+0000","last_clean":"2024-03-18T23:06:06.397672+0000","last_became_active":"2024-03-18T23:06:04.215578+0000","last_became_peered":"2024-03-18T23:06:04.215578+0000","last_unstale":"2024-03-18T23:06:06.397672+0000","last_undegraded":"2024-03-18T23:06:06.397672+0000","last_fullsized":"2024-03-18T23:06:06.397672+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_clean_scrub_stamp":"2024-03-18T23:06:03.184668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:10:34.048011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T23:06:06.398256+0000","last_change":"2024-03-18T23:06:04.918955+0000","last_active":"2024-03-18T23:06:06.398256+0000","last_peered":"2024-03-18T23:06:06.398256+0000","last_clean":"2024-03-18T23:06:06.398256+0000","last_became_active":"2024-03-18T23:06:04.915475+0000","last_became_peered":"2024-03-18T23:06:04.915475+0000","last_unstale":"2024-03-18T23:06:06.398256+0000","last_undegraded":"2024-03-18T23:06:06.398256+0000","last_fullsized":"2024-03-18T23:06:06.398256+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:07:05.679235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"85'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T23:06:05.895852+0000","last_change":"2024-03-18T23:06:04.224237+0000","last_active":"2024-03-18T23:06:05.895852+0000","last_peered":"2024-03-18T23:06:05.895852+0000","last_clean":"2024-03-18T23:06:05.895852+0000","last_became_active":"2024-03-18T23:06:04.223820+0000","last_became_peered":"2024-03-18T23:06:04.223820+0000","last_unstale":"2024-03-18T23:06:05.895852+0000","last_undegraded":"2024-03-18T23:06:05.895852+0000","last_fullsized":"2024-03-18T23:06:05.895852+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_clean_scrub_stamp":"2024-03-18T23:06:03.184668+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:17:48.157499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T23:06:06.398205+0000","last_change":"2024-03-18T23:06:04.914846+0000","last_active":"2024-03-18T23:06:06.398205+0000","last_peered":"2024-03-18T23:06:06.398205+0000","last_clean":"2024-03-18T23:06:06.398205+0000","last_became_active":"2024-03-18T23:06:04.911629+0000","last_became_peered":"2024-03-18T23:06:04.911629+0000","last_unstale":"2024-03-18T23:06:06.398205+0000","last_undegraded":"2024-03-18T23:06:06.398205+0000","last_fullsized":"2024-03-18T23:06:06.398205+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:17:32.235568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T23:06:06.398483+0000","last_change":"2024-03-18T23:06:04.219594+0000","last_active":"2024-03-18T23:06:06.398483+0000","last_peered":"2024-03-18T23:06:06.398483+0000","last_clean":"2024-03-18T23:06:06.398483+0000","last_became_active":"2024-03-18T23:06:04.219241+0000","last_became_peered":"2024-03-18T23:06:04.219241+0000","last_unstale":"2024-03-18T23:06:06.398483+0000","last_undegraded":"2024-03-18T23:06:06.398483+0000","last_fullsized":"2024-03-18T23:06:06.398483+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_clean_scrub_stamp":"2024-03-18T23:06:03.184668+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:25:50.012659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T23:06:06.398537+0000","last_change":"2024-03-18T23:06:04.914463+0000","last_active":"2024-03-18T23:06:06.398537+0000","last_peered":"2024-03-18T23:06:06.398537+0000","last_clean":"2024-03-18T23:06:06.398537+0000","last_became_active":"2024-03-18T23:06:04.914130+0000","last_became_peered":"2024-03-18T23:06:04.914130+0000","last_unstale":"2024-03-18T23:06:06.398537+0000","last_undegraded":"2024-03-18T23:06:06.398537+0000","last_fullsized":"2024-03-18T23:06:06.398537+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:57:56.065100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T23:06:06.426705+0000","last_change":"2024-03-18T23:06:04.905951+0000","last_active":"2024-03-18T23:06:06.426705+0000","last_peered":"2024-03-18T23:06:06.426705+0000","last_clean":"2024-03-18T23:06:06.426705+0000","last_became_active":"2024-03-18T23:06:04.905614+0000","last_became_peered":"2024-03-18T23:06:04.905614+0000","last_unstale":"2024-03-18T23:06:06.426705+0000","last_undegraded":"2024-03-18T23:06:06.426705+0000","last_fullsized":"2024-03-18T23:06:06.426705+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:11:44.884671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T23:06:06.397633+0000","last_change":"2024-03-18T23:06:04.910738+0000","last_active":"2024-03-18T23:06:06.397633+0000","last_peered":"2024-03-18T23:06:06.397633+0000","last_clean":"2024-03-18T23:06:06.397633+0000","last_became_active":"2024-03-18T23:06:04.910369+0000","last_became_peered":"2024-03-18T23:06:04.910369+0000","last_unstale":"2024-03-18T23:06:06.397633+0000","last_undegraded":"2024-03-18T23:06:06.397633+0000","last_fullsized":"2024-03-18T23:06:06.397633+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:22:08.306825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T23:06:06.397551+0000","last_change":"2024-03-18T23:06:04.912818+0000","last_active":"2024-03-18T23:06:06.397551+0000","last_peered":"2024-03-18T23:06:06.397551+0000","last_clean":"2024-03-18T23:06:06.397551+0000","last_became_active":"2024-03-18T23:06:04.912511+0000","last_became_peered":"2024-03-18T23:06:04.912511+0000","last_unstale":"2024-03-18T23:06:06.397551+0000","last_undegraded":"2024-03-18T23:06:06.397551+0000","last_fullsized":"2024-03-18T23:06:06.397551+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T09:24:44.743041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T23:06:06.398123+0000","last_change":"2024-03-18T23:06:04.912177+0000","last_active":"2024-03-18T23:06:06.398123+0000","last_peered":"2024-03-18T23:06:06.398123+0000","last_clean":"2024-03-18T23:06:06.398123+0000","last_became_active":"2024-03-18T23:06:04.911638+0000","last_became_peered":"2024-03-18T23:06:04.911638+0000","last_unstale":"2024-03-18T23:06:06.398123+0000","last_undegraded":"2024-03-18T23:06:06.398123+0000","last_fullsized":"2024-03-18T23:06:06.398123+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:21:30.648246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T23:06:06.398565+0000","last_change":"2024-03-18T23:06:04.912670+0000","last_active":"2024-03-18T23:06:06.398565+0000","last_peered":"2024-03-18T23:06:06.398565+0000","last_clean":"2024-03-18T23:06:06.398565+0000","last_became_active":"2024-03-18T23:06:04.912241+0000","last_became_peered":"2024-03-18T23:06:04.912241+0000","last_unstale":"2024-03-18T23:06:06.398565+0000","last_undegraded":"2024-03-18T23:06:06.398565+0000","last_fullsized":"2024-03-18T23:06:06.398565+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:31:02.666360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T23:06:06.426849+0000","last_change":"2024-03-18T23:06:04.909618+0000","last_active":"2024-03-18T23:06:06.426849+0000","last_peered":"2024-03-18T23:06:06.426849+0000","last_clean":"2024-03-18T23:06:06.426849+0000","last_became_active":"2024-03-18T23:06:04.909301+0000","last_became_peered":"2024-03-18T23:06:04.909301+0000","last_unstale":"2024-03-18T23:06:06.426849+0000","last_undegraded":"2024-03-18T23:06:06.426849+0000","last_fullsized":"2024-03-18T23:06:06.426849+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:14:28.101360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T23:06:06.397902+0000","last_change":"2024-03-18T23:06:04.904607+0000","last_active":"2024-03-18T23:06:06.397902+0000","last_peered":"2024-03-18T23:06:06.397902+0000","last_clean":"2024-03-18T23:06:06.397902+0000","last_became_active":"2024-03-18T23:06:04.904280+0000","last_became_peered":"2024-03-18T23:06:04.904280+0000","last_unstale":"2024-03-18T23:06:06.397902+0000","last_undegraded":"2024-03-18T23:06:06.397902+0000","last_fullsized":"2024-03-18T23:06:06.397902+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:59:11.445200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T23:06:05.895775+0000","last_change":"2024-03-18T23:06:04.918342+0000","last_active":"2024-03-18T23:06:05.895775+0000","last_peered":"2024-03-18T23:06:05.895775+0000","last_clean":"2024-03-18T23:06:05.895775+0000","last_became_active":"2024-03-18T23:06:04.917947+0000","last_became_peered":"2024-03-18T23:06:04.917947+0000","last_unstale":"2024-03-18T23:06:05.895775+0000","last_undegraded":"2024-03-18T23:06:05.895775+0000","last_fullsized":"2024-03-18T23:06:05.895775+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:05:49.347374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T23:06:05.895787+0000","last_change":"2024-03-18T23:06:04.912457+0000","last_active":"2024-03-18T23:06:05.895787+0000","last_peered":"2024-03-18T23:06:05.895787+0000","last_clean":"2024-03-18T23:06:05.895787+0000","last_became_active":"2024-03-18T23:06:04.912141+0000","last_became_peered":"2024-03-18T23:06:04.912141+0000","last_unstale":"2024-03-18T23:06:05.895787+0000","last_undegraded":"2024-03-18T23:06:05.895787+0000","last_fullsized":"2024-03-18T23:06:05.895787+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:04:45.847314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T23:06:05.895715+0000","last_change":"2024-03-18T23:06:04.918723+0000","last_active":"2024-03-18T23:06:05.895715+0000","last_peered":"2024-03-18T23:06:05.895715+0000","last_clean":"2024-03-18T23:06:05.895715+0000","last_became_active":"2024-03-18T23:06:04.918375+0000","last_became_peered":"2024-03-18T23:06:04.918375+0000","last_unstale":"2024-03-18T23:06:05.895715+0000","last_undegraded":"2024-03-18T23:06:05.895715+0000","last_fullsized":"2024-03-18T23:06:05.895715+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:27:12.868420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T23:06:06.398427+0000","last_change":"2024-03-18T23:06:04.913935+0000","last_active":"2024-03-18T23:06:06.398427+0000","last_peered":"2024-03-18T23:06:06.398427+0000","last_clean":"2024-03-18T23:06:06.398427+0000","last_became_active":"2024-03-18T23:06:04.913617+0000","last_became_peered":"2024-03-18T23:06:04.913617+0000","last_unstale":"2024-03-18T23:06:06.398427+0000","last_undegraded":"2024-03-18T23:06:06.398427+0000","last_fullsized":"2024-03-18T23:06:06.398427+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:53:15.585608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T23:06:06.398230+0000","last_change":"2024-03-18T23:06:04.911951+0000","last_active":"2024-03-18T23:06:06.398230+0000","last_peered":"2024-03-18T23:06:06.398230+0000","last_clean":"2024-03-18T23:06:06.398230+0000","last_became_active":"2024-03-18T23:06:04.911621+0000","last_became_peered":"2024-03-18T23:06:04.911621+0000","last_unstale":"2024-03-18T23:06:06.398230+0000","last_undegraded":"2024-03-18T23:06:06.398230+0000","last_fullsized":"2024-03-18T23:06:06.398230+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:20:46.600106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T23:06:06.426650+0000","last_change":"2024-03-18T23:06:04.910377+0000","last_active":"2024-03-18T23:06:06.426650+0000","last_peered":"2024-03-18T23:06:06.426650+0000","last_clean":"2024-03-18T23:06:06.426650+0000","last_became_active":"2024-03-18T23:06:04.910053+0000","last_became_peered":"2024-03-18T23:06:04.910053+0000","last_unstale":"2024-03-18T23:06:06.426650+0000","last_undegraded":"2024-03-18T23:06:06.426650+0000","last_fullsized":"2024-03-18T23:06:06.426650+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T23:29:51.750365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":284006,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":23923,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":18,"num_read_kb":15,"num_write":146,"num_write_kb":58,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":45,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":266240,"data_stored":34832,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":126,"ondisk_log_size":126,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892350989,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":925696,"data_stored":444311,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612625,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28028,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717124,"statfs":{"total":95995035648,"available":95966334976,"internally_reserved":0,"allocated":1036288,"data_stored":1044627,"data_compressed":36115,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776151,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":950272,"data_stored":445219,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037789,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32100,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713052,"statfs":{"total":95995035648,"available":95962165248,"internally_reserved":0,"allocated":946176,"data_stored":445049,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332128,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":987136,"data_stored":1035883,"data_compressed":36115,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495655,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36244,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708908,"statfs":{"total":95995035648,"available":95957921792,"internally_reserved":0,"allocated":925696,"data_stored":444311,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574912,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32092,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713060,"statfs":{"total":95995035648,"available":95962173440,"internally_reserved":0,"allocated":937984,"data_stored":445923,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738437,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36272,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708880,"statfs":{"total":95995035648,"available":95957893120,"internally_reserved":0,"allocated":958464,"data_stored":446377,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411387,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":950272,"data_stored":446579,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182453,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":933888,"data_stored":444415,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920817,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32084,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713068,"statfs":{"total":95995035648,"available":95962181632,"internally_reserved":0,"allocated":929792,"data_stored":444333,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757291,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32152,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713000,"statfs":{"total":95995035648,"available":95962112000,"internally_reserved":0,"allocated":987136,"data_stored":1034861,"data_compressed":36115,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34477,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34477,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34477,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36115,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36115,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36115,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36115,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36115,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2268,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":22631,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-18T23:06:13.854 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-03-18T23:06:13.854 INFO:tasks.cephadm:Setting up client nodes... 2024-03-18T23:06:13.855 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c63591ea-e57a-11ee-95c9-87774f69a715 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-03-18T23:06:14.138 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T23:06:14.518 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:06:14 smithi049 ceph-mon[62991]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-03-18T23:06:14.519 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:06:14 smithi049 ceph-mon[62991]: from='client.14775 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-18T23:06:14.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:06:14 smithi139 ceph-mon[67282]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-03-18T23:06:14.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:06:14 smithi139 ceph-mon[67282]: from='client.14775 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-18T23:06:14.711 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:06:14 smithi175 ceph-mon[67352]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-03-18T23:06:14.711 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:06:14 smithi175 ceph-mon[67352]: from='client.14775 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-18T23:06:15.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:06:15 smithi139 ceph-mon[67282]: from='client.14781 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-18T23:06:15.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:06:15 smithi139 ceph-mon[67282]: pgmap v293: 97 pgs: 97 active+clean; 602 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 8.0 KiB/s wr, 23 op/s 2024-03-18T23:06:15.626 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:06:15 smithi049 ceph-mon[62991]: from='client.14781 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-18T23:06:15.627 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:06:15 smithi049 ceph-mon[62991]: pgmap v293: 97 pgs: 97 active+clean; 602 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 8.0 KiB/s wr, 23 op/s 2024-03-18T23:06:15.711 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:06:15 smithi175 ceph-mon[67352]: from='client.14781 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-18T23:06:15.711 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:06:15 smithi175 ceph-mon[67352]: pgmap v293: 97 pgs: 97 active+clean; 602 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 8.0 KiB/s wr, 23 op/s 2024-03-18T23:06:15.872 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:15.870+0000 7f76efeab700 1 -- 172.21.15.49:0/3965197620 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f76e8108080 msgr2=0x7f76e810a4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:15.872 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:15.870+0000 7f76efeab700 1 --2- 172.21.15.49:0/3965197620 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f76e8108080 0x7f76e810a4f0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f76e000b0a0 tx=0x7f76e000b3b0 comp rx=0 tx=0).stop 2024-03-18T23:06:15.872 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:15.870+0000 7f76efeab700 1 -- 172.21.15.49:0/3965197620 shutdown_connections 2024-03-18T23:06:15.872 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:15.870+0000 7f76efeab700 1 --2- 172.21.15.49:0/3965197620 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f76e8108080 0x7f76e810a4f0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:15.872 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:15.870+0000 7f76efeab700 1 --2- 172.21.15.49:0/3965197620 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f76e8102e20 0x7f76e8105210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:15.872 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:15.870+0000 7f76efeab700 1 --2- 172.21.15.49:0/3965197620 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f76e8105750 0x7f76e8107b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:15.873 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:15.870+0000 7f76efeab700 1 -- 172.21.15.49:0/3965197620 >> 172.21.15.49:0/3965197620 conn(0x7f76e80fc830 msgr2=0x7f76e80fec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:06:15.873 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:15.870+0000 7f76efeab700 1 -- 172.21.15.49:0/3965197620 shutdown_connections 2024-03-18T23:06:15.873 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:15.870+0000 7f76efeab700 1 -- 172.21.15.49:0/3965197620 wait complete. 2024-03-18T23:06:15.873 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:15.870+0000 7f76efeab700 1 Processor -- start 2024-03-18T23:06:15.873 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:15.871+0000 7f76efeab700 1 -- start start 2024-03-18T23:06:15.873 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:15.871+0000 7f76efeab700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f76e8102e20 0x7f76e8198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:06:15.873 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:15.871+0000 7f76efeab700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f76e8105750 0x7f76e8198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:06:15.873 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:15.871+0000 7f76efeab700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f76e8108080 0x7f76e819f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:06:15.873 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:15.871+0000 7f76efeab700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f76e810dfa0 con 0x7f76e8108080 2024-03-18T23:06:15.873 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:15.871+0000 7f76efeab700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f76e810de20 con 0x7f76e8102e20 2024-03-18T23:06:15.873 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:15.871+0000 7f76efeab700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f76e810e120 con 0x7f76e8105750 2024-03-18T23:06:15.873 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:15.871+0000 7f76ed446700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f76e8105750 0x7f76e8198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:06:15.873 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:15.871+0000 7f76ee448700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f76e8108080 0x7f76e819f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:06:15.874 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:15.871+0000 7f76ed446700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f76e8105750 0x7f76e8198970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.49:57554/0 (socket says 172.21.15.49:57554) 2024-03-18T23:06:15.874 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:15.871+0000 7f76edc47700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f76e8102e20 0x7f76e8198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:06:15.874 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:15.871+0000 7f76ed446700 1 -- 172.21.15.49:0/2586982596 learned_addr learned my addr 172.21.15.49:0/2586982596 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:06:15.874 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:15.871+0000 7f76ee448700 1 -- 172.21.15.49:0/2586982596 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f76e8105750 msgr2=0x7f76e8198970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:15.874 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:15.871+0000 7f76ee448700 1 --2- 172.21.15.49:0/2586982596 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f76e8105750 0x7f76e8198970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:15.874 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:15.871+0000 7f76ee448700 1 -- 172.21.15.49:0/2586982596 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f76e8102e20 msgr2=0x7f76e8198430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:15.874 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:15.871+0000 7f76ee448700 1 --2- 172.21.15.49:0/2586982596 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f76e8102e20 0x7f76e8198430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:15.874 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:15.871+0000 7f76ee448700 1 -- 172.21.15.49:0/2586982596 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76e81a0090 con 0x7f76e8108080 2024-03-18T23:06:15.874 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:15.871+0000 7f76ee448700 1 --2- 172.21.15.49:0/2586982596 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f76e8108080 0x7f76e819f9f0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f76e0007fa0 tx=0x7f76e0004ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:06:15.874 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:15.872+0000 7f76deffd700 1 -- 172.21.15.49:0/2586982596 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76e0021070 con 0x7f76e8108080 2024-03-18T23:06:15.874 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:15.872+0000 7f76efeab700 1 -- 172.21.15.49:0/2586982596 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76e81a0320 con 0x7f76e8108080 2024-03-18T23:06:15.874 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:15.872+0000 7f76deffd700 1 -- 172.21.15.49:0/2586982596 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f76e0007b80 con 0x7f76e8108080 2024-03-18T23:06:15.874 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:15.872+0000 7f76deffd700 1 -- 172.21.15.49:0/2586982596 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76e0004800 con 0x7f76e8108080 2024-03-18T23:06:15.875 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:15.872+0000 7f76efeab700 1 -- 172.21.15.49:0/2586982596 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76e81a07d0 con 0x7f76e8108080 2024-03-18T23:06:15.875 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:15.873+0000 7f76efeab700 1 -- 172.21.15.49:0/2586982596 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76e8050820 con 0x7f76e8108080 2024-03-18T23:06:15.876 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:15.874+0000 7f76deffd700 1 -- 172.21.15.49:0/2586982596 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 17) ==== 137341+0+0 (secure 0 0 0) 0x7f76e0003a00 con 0x7f76e8108080 2024-03-18T23:06:15.876 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:15.874+0000 7f76deffd700 1 --2- 172.21.15.49:0/2586982596 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f76d40a29d0 0x7f76d40a4e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:06:15.876 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:15.875+0000 7f76edc47700 1 --2- 172.21.15.49:0/2586982596 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f76d40a29d0 0x7f76d40a4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:06:15.876 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:15.875+0000 7f76deffd700 1 -- 172.21.15.49:0/2586982596 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f76e00cf980 con 0x7f76e8108080 2024-03-18T23:06:15.876 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:15.875+0000 7f76edc47700 1 --2- 172.21.15.49:0/2586982596 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f76d40a29d0 0x7f76d40a4e90 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f76e4009c60 tx=0x7f76e4009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:06:15.879 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:15.879+0000 7f76deffd700 1 -- 172.21.15.49:0/2586982596 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f76e009cb00 con 0x7f76e8108080 2024-03-18T23:06:16.171 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:16.171+0000 7f76efeab700 1 -- 172.21.15.49:0/2586982596 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7f76e802dbb0 con 0x7f76e8108080 2024-03-18T23:06:16.172 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:16.173+0000 7f76deffd700 1 -- 172.21.15.49:0/2586982596 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7f76e009c400 con 0x7f76e8108080 2024-03-18T23:06:16.173 INFO:teuthology.orchestra.run.smithi049.stdout:[client.0] 2024-03-18T23:06:16.173 INFO:teuthology.orchestra.run.smithi049.stdout: key = AQDoyPhlavFBChAAcEziZmyl2rkG7ON03ipi6g== 2024-03-18T23:06:16.175 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:16.175+0000 7f76efeab700 1 -- 172.21.15.49:0/2586982596 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f76d40a29d0 msgr2=0x7f76d40a4e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:16.175 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:16.175+0000 7f76efeab700 1 --2- 172.21.15.49:0/2586982596 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f76d40a29d0 0x7f76d40a4e90 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f76e4009c60 tx=0x7f76e4009260 comp rx=0 tx=0).stop 2024-03-18T23:06:16.175 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:16.175+0000 7f76efeab700 1 -- 172.21.15.49:0/2586982596 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f76e8108080 msgr2=0x7f76e819f9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:16.175 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:16.175+0000 7f76efeab700 1 --2- 172.21.15.49:0/2586982596 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f76e8108080 0x7f76e819f9f0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f76e0007fa0 tx=0x7f76e0004ce0 comp rx=0 tx=0).stop 2024-03-18T23:06:16.175 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:16.175+0000 7f76efeab700 1 -- 172.21.15.49:0/2586982596 shutdown_connections 2024-03-18T23:06:16.175 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:16.175+0000 7f76efeab700 1 --2- 172.21.15.49:0/2586982596 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f76e8108080 0x7f76e819f9f0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:16.176 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:16.175+0000 7f76efeab700 1 --2- 172.21.15.49:0/2586982596 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f76e8102e20 0x7f76e8198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:16.176 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:16.175+0000 7f76efeab700 1 --2- 172.21.15.49:0/2586982596 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f76d40a29d0 0x7f76d40a4e90 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:16.176 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:16.175+0000 7f76efeab700 1 --2- 172.21.15.49:0/2586982596 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f76e8105750 0x7f76e8198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:16.176 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:16.175+0000 7f76efeab700 1 -- 172.21.15.49:0/2586982596 >> 172.21.15.49:0/2586982596 conn(0x7f76e80fc830 msgr2=0x7f76e8103740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:06:16.176 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:16.175+0000 7f76efeab700 1 -- 172.21.15.49:0/2586982596 shutdown_connections 2024-03-18T23:06:16.176 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:16.175+0000 7f76efeab700 1 -- 172.21.15.49:0/2586982596 wait complete. 2024-03-18T23:06:16.565 INFO:journalctl@ceph.mds.b.smithi139.stdout:Mar 18 23:06:16 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mds-b[98118]: 2024-03-18T23:06:16.149+0000 7f80011e6700 -1 mds.pinger is_rank_lagging: rank=4 was never sent ping request. 2024-03-18T23:06:16.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:06:16 smithi139 ceph-mon[67282]: from='client.? 172.21.15.49:0/2586982596' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-18T23:06:16.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:06:16 smithi139 ceph-mon[67282]: from='client.? 172.21.15.49:0/2586982596' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-18T23:06:16.572 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:06:16 smithi049 ceph-mon[62991]: from='client.? 172.21.15.49:0/2586982596' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-18T23:06:16.572 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:06:16 smithi049 ceph-mon[62991]: from='client.? 172.21.15.49:0/2586982596' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-18T23:06:16.711 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:06:16 smithi175 ceph-mon[67352]: from='client.? 172.21.15.49:0/2586982596' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-18T23:06:16.711 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:06:16 smithi175 ceph-mon[67352]: from='client.? 172.21.15.49:0/2586982596' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-18T23:06:16.905 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-03-18T23:06:16.905 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-03-18T23:06:16.905 DEBUG:teuthology.orchestra.run.smithi049:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-03-18T23:06:16.958 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-03-18T23:06:16.958 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-03-18T23:06:16.958 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell --fsid c63591ea-e57a-11ee-95c9-87774f69a715 -- ceph mgr dump --format=json 2024-03-18T23:06:17.249 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T23:06:17.524 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:06:17 smithi049 ceph-mon[62991]: pgmap v294: 97 pgs: 97 active+clean; 604 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 8.6 KiB/s wr, 25 op/s 2024-03-18T23:06:17.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:06:17 smithi139 ceph-mon[67282]: pgmap v294: 97 pgs: 97 active+clean; 604 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 8.6 KiB/s wr, 25 op/s 2024-03-18T23:06:17.711 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:06:17 smithi175 ceph-mon[67352]: pgmap v294: 97 pgs: 97 active+clean; 604 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 8.6 KiB/s wr, 25 op/s 2024-03-18T23:06:19.021 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:19.019+0000 7fcf3c509700 1 -- 172.21.15.49:0/2668917183 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcf3406a960 msgr2=0x7fcf3410f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:19.021 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:19.019+0000 7fcf3c509700 1 --2- 172.21.15.49:0/2668917183 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcf3406a960 0x7fcf3410f1a0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fcf2c00b0d0 tx=0x7fcf2c00b3e0 comp rx=0 tx=0).stop 2024-03-18T23:06:19.021 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:19.020+0000 7fcf3c509700 1 -- 172.21.15.49:0/2668917183 shutdown_connections 2024-03-18T23:06:19.022 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:19.020+0000 7fcf3c509700 1 --2- 172.21.15.49:0/2668917183 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcf3406a960 0x7fcf3410f1a0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:19.022 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:19.020+0000 7fcf3c509700 1 --2- 172.21.15.49:0/2668917183 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcf34108360 0x7fcf34108740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:19.022 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:19.020+0000 7fcf3c509700 1 --2- 172.21.15.49:0/2668917183 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcf34069fc0 0x7fcf3406a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:19.022 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:19.020+0000 7fcf3c509700 1 -- 172.21.15.49:0/2668917183 >> 172.21.15.49:0/2668917183 conn(0x7fcf34078140 msgr2=0x7fcf34078550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:06:19.022 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:19.020+0000 7fcf3c509700 1 -- 172.21.15.49:0/2668917183 shutdown_connections 2024-03-18T23:06:19.022 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:19.020+0000 7fcf3c509700 1 -- 172.21.15.49:0/2668917183 wait complete. 2024-03-18T23:06:19.022 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:19.020+0000 7fcf3c509700 1 Processor -- start 2024-03-18T23:06:19.022 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:19.020+0000 7fcf3c509700 1 -- start start 2024-03-18T23:06:19.022 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:19.020+0000 7fcf3c509700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcf34069fc0 0x7fcf341012e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:06:19.022 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:19.020+0000 7fcf3c509700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcf3406a960 0x7fcf34101820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:06:19.022 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:19.020+0000 7fcf3c509700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcf34108360 0x7fcf34101d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:06:19.022 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:19.020+0000 7fcf3c509700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fcf341125b0 con 0x7fcf34108360 2024-03-18T23:06:19.023 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:19.020+0000 7fcf3c509700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fcf34112430 con 0x7fcf34069fc0 2024-03-18T23:06:19.023 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:19.020+0000 7fcf3c509700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fcf34112730 con 0x7fcf3406a960 2024-03-18T23:06:19.023 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:19.021+0000 7fcf3aaa6700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcf34108360 0x7fcf34101d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:06:19.023 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:19.021+0000 7fcf3aaa6700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcf34108360 0x7fcf34101d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:40100/0 (socket says 172.21.15.49:40100) 2024-03-18T23:06:19.023 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:19.021+0000 7fcf3aaa6700 1 -- 172.21.15.49:0/2731976003 learned_addr learned my addr 172.21.15.49:0/2731976003 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:06:19.023 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:19.021+0000 7fcf3a2a5700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcf34069fc0 0x7fcf341012e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:06:19.023 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:19.021+0000 7fcf39aa4700 1 --2- 172.21.15.49:0/2731976003 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcf3406a960 0x7fcf34101820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:06:19.023 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:19.021+0000 7fcf3aaa6700 1 -- 172.21.15.49:0/2731976003 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcf3406a960 msgr2=0x7fcf34101820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:19.023 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:19.021+0000 7fcf3aaa6700 1 --2- 172.21.15.49:0/2731976003 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcf3406a960 0x7fcf34101820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:19.023 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:19.021+0000 7fcf3aaa6700 1 -- 172.21.15.49:0/2731976003 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcf34069fc0 msgr2=0x7fcf341012e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:19.023 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:19.021+0000 7fcf3aaa6700 1 --2- 172.21.15.49:0/2731976003 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcf34069fc0 0x7fcf341012e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:19.023 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:19.021+0000 7fcf3aaa6700 1 -- 172.21.15.49:0/2731976003 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf341a6fe0 con 0x7fcf34108360 2024-03-18T23:06:19.023 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:19.021+0000 7fcf3a2a5700 1 --2- 172.21.15.49:0/2731976003 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcf34069fc0 0x7fcf341012e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:06:19.023 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:19.021+0000 7fcf3aaa6700 1 --2- 172.21.15.49:0/2731976003 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcf34108360 0x7fcf34101d60 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fcf2c006070 tx=0x7fcf2c007a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:06:19.023 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:19.021+0000 7fcf2b7fe700 1 -- 172.21.15.49:0/2731976003 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf2c003c00 con 0x7fcf34108360 2024-03-18T23:06:19.024 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:19.021+0000 7fcf3c509700 1 -- 172.21.15.49:0/2731976003 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf341a7270 con 0x7fcf34108360 2024-03-18T23:06:19.024 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:19.021+0000 7fcf2b7fe700 1 -- 172.21.15.49:0/2731976003 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf2c022070 con 0x7fcf34108360 2024-03-18T23:06:19.024 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:19.021+0000 7fcf2b7fe700 1 -- 172.21.15.49:0/2731976003 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf2c01b6c0 con 0x7fcf34108360 2024-03-18T23:06:19.025 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:19.021+0000 7fcf3c509700 1 -- 172.21.15.49:0/2731976003 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf341a7670 con 0x7fcf34108360 2024-03-18T23:06:19.025 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:19.022+0000 7fcf3c509700 1 -- 172.21.15.49:0/2731976003 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf34068fd0 con 0x7fcf34108360 2024-03-18T23:06:19.025 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:19.024+0000 7fcf2b7fe700 1 -- 172.21.15.49:0/2731976003 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 17) ==== 137341+0+0 (secure 0 0 0) 0x7fcf2c01d470 con 0x7fcf34108360 2024-03-18T23:06:19.025 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:19.024+0000 7fcf2b7fe700 1 --2- 172.21.15.49:0/2731976003 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fcf200a2920 0x7fcf200a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:06:19.025 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:19.024+0000 7fcf3a2a5700 1 --2- 172.21.15.49:0/2731976003 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fcf200a2920 0x7fcf200a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:06:19.025 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:19.024+0000 7fcf2b7fe700 1 -- 172.21.15.49:0/2731976003 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7fcf2c0cfc70 con 0x7fcf34108360 2024-03-18T23:06:19.025 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:19.025+0000 7fcf3a2a5700 1 --2- 172.21.15.49:0/2731976003 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fcf200a2920 0x7fcf200a4de0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fcf30009fd0 tx=0x7fcf30009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:06:19.027 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:19.028+0000 7fcf2b7fe700 1 -- 172.21.15.49:0/2731976003 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fcf2c09cdf0 con 0x7fcf34108360 2024-03-18T23:06:19.292 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:19.292+0000 7fcf3c509700 1 -- 172.21.15.49:0/2731976003 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7fcf3402dbb0 con 0x7fcf34108360 2024-03-18T23:06:19.297 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:19.297+0000 7fcf2b7fe700 1 -- 172.21.15.49:0/2731976003 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+262658 (secure 0 0 0) 0x7fcf2c09c6f0 con 0x7fcf34108360 2024-03-18T23:06:19.301 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-18T23:06:19.303 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:19.300+0000 7fcf3c509700 1 -- 172.21.15.49:0/2731976003 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fcf200a2920 msgr2=0x7fcf200a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:19.303 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:19.300+0000 7fcf3c509700 1 --2- 172.21.15.49:0/2731976003 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fcf200a2920 0x7fcf200a4de0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fcf30009fd0 tx=0x7fcf30009260 comp rx=0 tx=0).stop 2024-03-18T23:06:19.303 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:19.300+0000 7fcf3c509700 1 -- 172.21.15.49:0/2731976003 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcf34108360 msgr2=0x7fcf34101d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:19.303 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:19.300+0000 7fcf3c509700 1 --2- 172.21.15.49:0/2731976003 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcf34108360 0x7fcf34101d60 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fcf2c006070 tx=0x7fcf2c007a20 comp rx=0 tx=0).stop 2024-03-18T23:06:19.303 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:19.300+0000 7fcf3c509700 1 -- 172.21.15.49:0/2731976003 shutdown_connections 2024-03-18T23:06:19.303 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:19.300+0000 7fcf3c509700 1 --2- 172.21.15.49:0/2731976003 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcf34108360 0x7fcf34101d60 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:19.303 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:19.300+0000 7fcf3c509700 1 --2- 172.21.15.49:0/2731976003 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcf34069fc0 0x7fcf341012e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:19.303 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:19.300+0000 7fcf3c509700 1 --2- 172.21.15.49:0/2731976003 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fcf200a2920 0x7fcf200a4de0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:19.303 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:19.300+0000 7fcf3c509700 1 --2- 172.21.15.49:0/2731976003 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcf3406a960 0x7fcf34101820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:19.303 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:19.300+0000 7fcf3c509700 1 -- 172.21.15.49:0/2731976003 >> 172.21.15.49:0/2731976003 conn(0x7fcf34078140 msgr2=0x7fcf340ffe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:06:19.303 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:19.300+0000 7fcf3c509700 1 -- 172.21.15.49:0/2731976003 shutdown_connections 2024-03-18T23:06:19.304 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:19.300+0000 7fcf3c509700 1 -- 172.21.15.49:0/2731976003 wait complete. 2024-03-18T23:06:19.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:06:19 smithi139 ceph-mon[67282]: pgmap v295: 97 pgs: 97 active+clean; 626 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 9.1 KiB/s wr, 23 op/s 2024-03-18T23:06:19.629 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:06:19 smithi049 ceph-mon[62991]: pgmap v295: 97 pgs: 97 active+clean; 626 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 9.1 KiB/s wr, 23 op/s 2024-03-18T23:06:19.711 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:06:19 smithi175 ceph-mon[67352]: pgmap v295: 97 pgs: 97 active+clean; 626 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 9.1 KiB/s wr, 23 op/s 2024-03-18T23:06:19.937 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":17,"flags":0,"active_gid":14150,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":2083805211},{"type":"v1","addr":"172.21.15.49:6801","nonce":2083805211}]},"active_addr":"172.21.15.49:6801/2083805211","active_change":"2024-03-18T22:58:59.806422+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24106,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24113,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.49:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.49:0","nonce":94393705}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.49:0","nonce":861926960}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.49:0","nonce":3268662009}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.49:0","nonce":1976299520}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.49:0","nonce":2652109253}]}]} 2024-03-18T23:06:19.942 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-03-18T23:06:19.943 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-03-18T23:06:19.943 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell --fsid c63591ea-e57a-11ee-95c9-87774f69a715 -- ceph osd dump --format=json 2024-03-18T23:06:20.200 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T23:06:20.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:06:20 smithi139 ceph-mon[67282]: from='client.? 172.21.15.49:0/2731976003' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-03-18T23:06:20.619 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:06:20 smithi049 ceph-mon[62991]: from='client.? 172.21.15.49:0/2731976003' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-03-18T23:06:20.711 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:06:20 smithi175 ceph-mon[67352]: from='client.? 172.21.15.49:0/2731976003' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-03-18T23:06:21.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:06:21 smithi049 ceph-mon[62991]: pgmap v296: 97 pgs: 97 active+clean; 626 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 8.8 KiB/s wr, 24 op/s 2024-03-18T23:06:21.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:06:21 smithi139 ceph-mon[67282]: pgmap v296: 97 pgs: 97 active+clean; 626 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 8.8 KiB/s wr, 24 op/s 2024-03-18T23:06:21.711 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:06:21 smithi175 ceph-mon[67352]: pgmap v296: 97 pgs: 97 active+clean; 626 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 8.8 KiB/s wr, 24 op/s 2024-03-18T23:06:21.979 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:21.973+0000 7fec6758b700 1 -- 172.21.15.49:0/3526266217 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fec60101bb0 msgr2=0x7fec6010e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:21.979 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:21.973+0000 7fec6758b700 1 --2- 172.21.15.49:0/3526266217 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fec60101bb0 0x7fec6010e090 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fec50009990 tx=0x7fec50009ca0 comp rx=0 tx=0).stop 2024-03-18T23:06:21.979 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:21.973+0000 7fec6758b700 1 -- 172.21.15.49:0/3526266217 shutdown_connections 2024-03-18T23:06:21.979 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:21.973+0000 7fec6758b700 1 --2- 172.21.15.49:0/3526266217 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fec60101bb0 0x7fec6010e090 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:21.979 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:21.973+0000 7fec6758b700 1 --2- 172.21.15.49:0/3526266217 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fec60107250 0x7fec60107630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:21.979 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:21.973+0000 7fec6758b700 1 --2- 172.21.15.49:0/3526266217 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fec60101210 0x7fec60101670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:21.980 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:21.973+0000 7fec6758b700 1 -- 172.21.15.49:0/3526266217 >> 172.21.15.49:0/3526266217 conn(0x7fec60078140 msgr2=0x7fec60078550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:06:21.980 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:21.973+0000 7fec6758b700 1 -- 172.21.15.49:0/3526266217 shutdown_connections 2024-03-18T23:06:21.980 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:21.973+0000 7fec6758b700 1 -- 172.21.15.49:0/3526266217 wait complete. 2024-03-18T23:06:21.980 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:21.974+0000 7fec6758b700 1 Processor -- start 2024-03-18T23:06:21.980 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:21.974+0000 7fec6758b700 1 -- start start 2024-03-18T23:06:21.980 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:21.974+0000 7fec6758b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fec60101210 0x7fec6019e740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:06:21.980 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:21.974+0000 7fec6758b700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fec60101bb0 0x7fec6019ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:06:21.980 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:21.974+0000 7fec6758b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fec60107250 0x7fec60198900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:06:21.980 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:21.974+0000 7fec6758b700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fec601114d0 con 0x7fec60101210 2024-03-18T23:06:21.980 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:21.974+0000 7fec6758b700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fec60111350 con 0x7fec60101bb0 2024-03-18T23:06:21.980 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:21.974+0000 7fec6758b700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fec60111650 con 0x7fec60107250 2024-03-18T23:06:21.980 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:21.974+0000 7fec65327700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fec60101210 0x7fec6019e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:06:21.980 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:21.974+0000 7fec65327700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fec60101210 0x7fec6019e740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:40116/0 (socket says 172.21.15.49:40116) 2024-03-18T23:06:21.980 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:21.974+0000 7fec65327700 1 -- 172.21.15.49:0/3142351508 learned_addr learned my addr 172.21.15.49:0/3142351508 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:06:21.980 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:21.974+0000 7fec64b26700 1 --2- 172.21.15.49:0/3142351508 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fec60101bb0 0x7fec6019ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:06:21.981 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:21.974+0000 7fec65b28700 1 --2- 172.21.15.49:0/3142351508 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fec60107250 0x7fec60198900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:06:21.981 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:21.974+0000 7fec65327700 1 -- 172.21.15.49:0/3142351508 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fec60107250 msgr2=0x7fec60198900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:21.981 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:21.974+0000 7fec65327700 1 --2- 172.21.15.49:0/3142351508 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fec60107250 0x7fec60198900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:21.981 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:21.974+0000 7fec65327700 1 -- 172.21.15.49:0/3142351508 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fec60101bb0 msgr2=0x7fec6019ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:21.981 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:21.974+0000 7fec65327700 1 --2- 172.21.15.49:0/3142351508 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fec60101bb0 0x7fec6019ec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:21.981 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:21.974+0000 7fec65327700 1 -- 172.21.15.49:0/3142351508 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec60199060 con 0x7fec60101210 2024-03-18T23:06:21.981 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:21.975+0000 7fec65327700 1 --2- 172.21.15.49:0/3142351508 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fec60101210 0x7fec6019e740 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fec5800ef30 tx=0x7fec5800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:06:21.981 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:21.975+0000 7fec5e7fc700 1 -- 172.21.15.49:0/3142351508 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec580092e0 con 0x7fec60101210 2024-03-18T23:06:21.981 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:21.975+0000 7fec5e7fc700 1 -- 172.21.15.49:0/3142351508 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fec58004bb0 con 0x7fec60101210 2024-03-18T23:06:21.981 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:21.975+0000 7fec6758b700 1 -- 172.21.15.49:0/3142351508 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec601992f0 con 0x7fec60101210 2024-03-18T23:06:21.981 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:21.975+0000 7fec5e7fc700 1 -- 172.21.15.49:0/3142351508 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec58004280 con 0x7fec60101210 2024-03-18T23:06:21.981 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:21.975+0000 7fec6758b700 1 -- 172.21.15.49:0/3142351508 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec601a5610 con 0x7fec60101210 2024-03-18T23:06:21.981 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:21.976+0000 7fec6758b700 1 -- 172.21.15.49:0/3142351508 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec60068fd0 con 0x7fec60101210 2024-03-18T23:06:21.981 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:21.977+0000 7fec5e7fc700 1 -- 172.21.15.49:0/3142351508 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 17) ==== 137341+0+0 (secure 0 0 0) 0x7fec5801b030 con 0x7fec60101210 2024-03-18T23:06:21.981 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:21.978+0000 7fec5e7fc700 1 --2- 172.21.15.49:0/3142351508 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fec480a2920 0x7fec480a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:06:21.982 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:21.978+0000 7fec64b26700 1 --2- 172.21.15.49:0/3142351508 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fec480a2920 0x7fec480a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:06:21.982 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:21.978+0000 7fec5e7fc700 1 -- 172.21.15.49:0/3142351508 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7fec580ce520 con 0x7fec60101210 2024-03-18T23:06:21.982 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:21.978+0000 7fec64b26700 1 --2- 172.21.15.49:0/3142351508 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fec480a2920 0x7fec480a4de0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fec600786b0 tx=0x7fec54009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:06:21.982 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:21.982+0000 7fec5e7fc700 1 -- 172.21.15.49:0/3142351508 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fec58022020 con 0x7fec60101210 2024-03-18T23:06:22.187 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:22.187+0000 7fec6758b700 1 -- 172.21.15.49:0/3142351508 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fec60199e30 con 0x7fec60101210 2024-03-18T23:06:22.188 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:22.189+0000 7fec5e7fc700 1 -- 172.21.15.49:0/3142351508 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+22242 (secure 0 0 0) 0x7fec58010040 con 0x7fec60101210 2024-03-18T23:06:22.189 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-18T23:06:22.189 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":86,"fsid":"c63591ea-e57a-11ee-95c9-87774f69a715","created":"2024-03-18T22:57:58.734467+0000","modified":"2024-03-18T23:06:05.892182+0000","last_up_change":"2024-03-18T23:05:16.576310+0000","last_in_change":"2024-03-18T23:04:57.024067+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-18T23:01:28.363432+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-18T23:06:03.172963+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-18T23:06:03.973303+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e7fc9daa-4098-4ba5-8019-9aaef1aab3e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":1651162501},{"type":"v1","addr":"172.21.15.49:6803","nonce":1651162501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":1651162501},{"type":"v1","addr":"172.21.15.49:6805","nonce":1651162501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":1651162501},{"type":"v1","addr":"172.21.15.49:6809","nonce":1651162501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":1651162501},{"type":"v1","addr":"172.21.15.49:6807","nonce":1651162501}]},"public_addr":"172.21.15.49:6803/1651162501","cluster_addr":"172.21.15.49:6805/1651162501","heartbeat_back_addr":"172.21.15.49:6809/1651162501","heartbeat_front_addr":"172.21.15.49:6807/1651162501","state":["exists","up"]},{"osd":1,"uuid":"40ac929c-e90f-434c-a796-9aabd686f1fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":1766700359},{"type":"v1","addr":"172.21.15.139:6801","nonce":1766700359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":1766700359},{"type":"v1","addr":"172.21.15.139:6803","nonce":1766700359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":1766700359},{"type":"v1","addr":"172.21.15.139:6807","nonce":1766700359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":1766700359},{"type":"v1","addr":"172.21.15.139:6805","nonce":1766700359}]},"public_addr":"172.21.15.139:6801/1766700359","cluster_addr":"172.21.15.139:6803/1766700359","heartbeat_back_addr":"172.21.15.139:6807/1766700359","heartbeat_front_addr":"172.21.15.139:6805/1766700359","state":["exists","up"]},{"osd":2,"uuid":"8ae3b666-00b4-45ed-a5ee-4d0771edabb6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":3752671234},{"type":"v1","addr":"172.21.15.175:6801","nonce":3752671234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":3752671234},{"type":"v1","addr":"172.21.15.175:6803","nonce":3752671234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":3752671234},{"type":"v1","addr":"172.21.15.175:6807","nonce":3752671234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":3752671234},{"type":"v1","addr":"172.21.15.175:6805","nonce":3752671234}]},"public_addr":"172.21.15.175:6801/3752671234","cluster_addr":"172.21.15.175:6803/3752671234","heartbeat_back_addr":"172.21.15.175:6807/3752671234","heartbeat_front_addr":"172.21.15.175:6805/3752671234","state":["exists","up"]},{"osd":3,"uuid":"1a62e89d-45d4-4237-b813-7f96a59aca7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":956160536},{"type":"v1","addr":"172.21.15.49:6811","nonce":956160536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":956160536},{"type":"v1","addr":"172.21.15.49:6813","nonce":956160536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":956160536},{"type":"v1","addr":"172.21.15.49:6817","nonce":956160536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":956160536},{"type":"v1","addr":"172.21.15.49:6815","nonce":956160536}]},"public_addr":"172.21.15.49:6811/956160536","cluster_addr":"172.21.15.49:6813/956160536","heartbeat_back_addr":"172.21.15.49:6817/956160536","heartbeat_front_addr":"172.21.15.49:6815/956160536","state":["exists","up"]},{"osd":4,"uuid":"c715101c-0c3f-4b98-b142-d64a35e187df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":3129712668},{"type":"v1","addr":"172.21.15.139:6809","nonce":3129712668}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":3129712668},{"type":"v1","addr":"172.21.15.139:6811","nonce":3129712668}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":3129712668},{"type":"v1","addr":"172.21.15.139:6815","nonce":3129712668}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":3129712668},{"type":"v1","addr":"172.21.15.139:6813","nonce":3129712668}]},"public_addr":"172.21.15.139:6809/3129712668","cluster_addr":"172.21.15.139:6811/3129712668","heartbeat_back_addr":"172.21.15.139:6815/3129712668","heartbeat_front_addr":"172.21.15.139:6813/3129712668","state":["exists","up"]},{"osd":5,"uuid":"4b641f81-77eb-45fd-8464-511e7b3062b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":2774567215},{"type":"v1","addr":"172.21.15.175:6809","nonce":2774567215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":2774567215},{"type":"v1","addr":"172.21.15.175:6811","nonce":2774567215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":2774567215},{"type":"v1","addr":"172.21.15.175:6815","nonce":2774567215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":2774567215},{"type":"v1","addr":"172.21.15.175:6813","nonce":2774567215}]},"public_addr":"172.21.15.175:6809/2774567215","cluster_addr":"172.21.15.175:6811/2774567215","heartbeat_back_addr":"172.21.15.175:6815/2774567215","heartbeat_front_addr":"172.21.15.175:6813/2774567215","state":["exists","up"]},{"osd":6,"uuid":"653da8e0-1197-4b48-abed-470f2a9c4dce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":1472215461},{"type":"v1","addr":"172.21.15.49:6819","nonce":1472215461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":1472215461},{"type":"v1","addr":"172.21.15.49:6821","nonce":1472215461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":1472215461},{"type":"v1","addr":"172.21.15.49:6825","nonce":1472215461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":1472215461},{"type":"v1","addr":"172.21.15.49:6823","nonce":1472215461}]},"public_addr":"172.21.15.49:6819/1472215461","cluster_addr":"172.21.15.49:6821/1472215461","heartbeat_back_addr":"172.21.15.49:6825/1472215461","heartbeat_front_addr":"172.21.15.49:6823/1472215461","state":["exists","up"]},{"osd":7,"uuid":"ad157c9b-4b75-451c-8740-630f0975b7fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":2163986787},{"type":"v1","addr":"172.21.15.139:6817","nonce":2163986787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":2163986787},{"type":"v1","addr":"172.21.15.139:6819","nonce":2163986787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":2163986787},{"type":"v1","addr":"172.21.15.139:6823","nonce":2163986787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":2163986787},{"type":"v1","addr":"172.21.15.139:6821","nonce":2163986787}]},"public_addr":"172.21.15.139:6817/2163986787","cluster_addr":"172.21.15.139:6819/2163986787","heartbeat_back_addr":"172.21.15.139:6823/2163986787","heartbeat_front_addr":"172.21.15.139:6821/2163986787","state":["exists","up"]},{"osd":8,"uuid":"c8e75f15-df75-4d5d-9b92-a064bffb6c01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":1194026992},{"type":"v1","addr":"172.21.15.175:6817","nonce":1194026992}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1194026992},{"type":"v1","addr":"172.21.15.175:6819","nonce":1194026992}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":1194026992},{"type":"v1","addr":"172.21.15.175:6823","nonce":1194026992}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1194026992},{"type":"v1","addr":"172.21.15.175:6821","nonce":1194026992}]},"public_addr":"172.21.15.175:6817/1194026992","cluster_addr":"172.21.15.175:6819/1194026992","heartbeat_back_addr":"172.21.15.175:6823/1194026992","heartbeat_front_addr":"172.21.15.175:6821/1194026992","state":["exists","up"]},{"osd":9,"uuid":"edb5ffef-b6a2-4941-b5ab-40f982ad30d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":2742084884},{"type":"v1","addr":"172.21.15.49:6827","nonce":2742084884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":2742084884},{"type":"v1","addr":"172.21.15.49:6829","nonce":2742084884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":2742084884},{"type":"v1","addr":"172.21.15.49:6833","nonce":2742084884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":2742084884},{"type":"v1","addr":"172.21.15.49:6831","nonce":2742084884}]},"public_addr":"172.21.15.49:6827/2742084884","cluster_addr":"172.21.15.49:6829/2742084884","heartbeat_back_addr":"172.21.15.49:6833/2742084884","heartbeat_front_addr":"172.21.15.49:6831/2742084884","state":["exists","up"]},{"osd":10,"uuid":"3945e46d-766a-4c4a-8821-d43648dd1578","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":2404740201},{"type":"v1","addr":"172.21.15.139:6825","nonce":2404740201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":2404740201},{"type":"v1","addr":"172.21.15.139:6827","nonce":2404740201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":2404740201},{"type":"v1","addr":"172.21.15.139:6831","nonce":2404740201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":2404740201},{"type":"v1","addr":"172.21.15.139:6829","nonce":2404740201}]},"public_addr":"172.21.15.139:6825/2404740201","cluster_addr":"172.21.15.139:6827/2404740201","heartbeat_back_addr":"172.21.15.139:6831/2404740201","heartbeat_front_addr":"172.21.15.139:6829/2404740201","state":["exists","up"]},{"osd":11,"uuid":"9954ba15-d877-46cc-b24a-893f9168deb8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":3519503868},{"type":"v1","addr":"172.21.15.175:6825","nonce":3519503868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":3519503868},{"type":"v1","addr":"172.21.15.175:6827","nonce":3519503868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":3519503868},{"type":"v1","addr":"172.21.15.175:6831","nonce":3519503868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":3519503868},{"type":"v1","addr":"172.21.15.175:6829","nonce":3519503868}]},"public_addr":"172.21.15.175:6825/3519503868","cluster_addr":"172.21.15.175:6827/3519503868","heartbeat_back_addr":"172.21.15.175:6831/3519503868","heartbeat_front_addr":"172.21.15.175:6829/3519503868","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:00:38.367865+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:01:02.237687+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:01:25.609751+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:01:51.268572+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:02:16.514605+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:02:41.586523+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:03:07.029964+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:03:32.956250+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:03:57.547984+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:04:23.948972+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:04:49.667210+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:05:15.284693+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.49:0/563527416":"2024-03-19T22:58:59.806253+0000","172.21.15.49:0/1036383534":"2024-03-19T22:58:59.806253+0000","172.21.15.49:6801/2609458951":"2024-03-19T22:58:59.806253+0000","172.21.15.49:0/21283102":"2024-03-19T22:58:31.972474+0000","172.21.15.49:0/1292058922":"2024-03-19T22:58:31.972474+0000","172.21.15.49:6800/1985190312":"2024-03-19T22:58:31.972474+0000","172.21.15.49:0/1912446700":"2024-03-19T22:58:59.806253+0000","172.21.15.49:0/2176210039":"2024-03-19T22:58:31.972474+0000","172.21.15.49:6801/1985190312":"2024-03-19T22:58:31.972474+0000","172.21.15.49:6800/2609458951":"2024-03-19T22:58:59.806253+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-18T23:06:22.191 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:22.191+0000 7fec6758b700 1 -- 172.21.15.49:0/3142351508 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fec480a2920 msgr2=0x7fec480a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:22.191 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:22.191+0000 7fec6758b700 1 --2- 172.21.15.49:0/3142351508 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fec480a2920 0x7fec480a4de0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fec600786b0 tx=0x7fec54009260 comp rx=0 tx=0).stop 2024-03-18T23:06:22.191 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:22.191+0000 7fec6758b700 1 -- 172.21.15.49:0/3142351508 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fec60101210 msgr2=0x7fec6019e740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:22.191 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:22.191+0000 7fec6758b700 1 --2- 172.21.15.49:0/3142351508 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fec60101210 0x7fec6019e740 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fec5800ef30 tx=0x7fec5800c3a0 comp rx=0 tx=0).stop 2024-03-18T23:06:22.191 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:22.191+0000 7fec6758b700 1 -- 172.21.15.49:0/3142351508 shutdown_connections 2024-03-18T23:06:22.191 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:22.191+0000 7fec6758b700 1 --2- 172.21.15.49:0/3142351508 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fec480a2920 0x7fec480a4de0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:22.191 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:22.191+0000 7fec6758b700 1 --2- 172.21.15.49:0/3142351508 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fec60107250 0x7fec60198900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:22.191 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:22.191+0000 7fec6758b700 1 --2- 172.21.15.49:0/3142351508 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fec60101210 0x7fec6019e740 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:22.191 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:22.191+0000 7fec6758b700 1 --2- 172.21.15.49:0/3142351508 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fec60101bb0 0x7fec6019ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:22.192 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:22.191+0000 7fec6758b700 1 -- 172.21.15.49:0/3142351508 >> 172.21.15.49:0/3142351508 conn(0x7fec60078140 msgr2=0x7fec601054a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:06:22.192 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:22.191+0000 7fec6758b700 1 -- 172.21.15.49:0/3142351508 shutdown_connections 2024-03-18T23:06:22.192 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:22.191+0000 7fec6758b700 1 -- 172.21.15.49:0/3142351508 wait complete. 2024-03-18T23:06:22.561 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:06:22 smithi049 ceph-mon[62991]: from='client.? 172.21.15.49:0/3142351508' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T23:06:22.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:06:22 smithi139 ceph-mon[67282]: from='client.? 172.21.15.49:0/3142351508' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T23:06:22.711 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:06:22 smithi175 ceph-mon[67352]: from='client.? 172.21.15.49:0/3142351508' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T23:06:22.889 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-03-18T23:06:22.889 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell --fsid c63591ea-e57a-11ee-95c9-87774f69a715 -- ceph osd dump --format=json 2024-03-18T23:06:23.144 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T23:06:23.382 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:06:23 smithi049 ceph-mon[62991]: pgmap v297: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.7 KiB/s wr, 24 op/s 2024-03-18T23:06:23.566 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:06:23 smithi139 ceph-mon[67282]: pgmap v297: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.7 KiB/s wr, 24 op/s 2024-03-18T23:06:23.711 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:06:23 smithi175 ceph-mon[67352]: pgmap v297: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.7 KiB/s wr, 24 op/s 2024-03-18T23:06:24.866 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:24.863+0000 7f04fa6f8700 1 -- 172.21.15.49:0/1753452189 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f04f4074540 msgr2=0x7f04f4074920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:24.867 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:24.863+0000 7f04fa6f8700 1 --2- 172.21.15.49:0/1753452189 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f04f4074540 0x7f04f4074920 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f04e4009990 tx=0x7f04e4009ca0 comp rx=0 tx=0).stop 2024-03-18T23:06:24.867 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:24.863+0000 7f04fa6f8700 1 -- 172.21.15.49:0/1753452189 shutdown_connections 2024-03-18T23:06:24.867 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:24.863+0000 7f04fa6f8700 1 --2- 172.21.15.49:0/1753452189 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f04f410ed40 0x7f04f4111130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:24.867 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:24.863+0000 7f04fa6f8700 1 --2- 172.21.15.49:0/1753452189 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f04f4074540 0x7f04f4074920 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:24.867 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:24.863+0000 7f04fa6f8700 1 --2- 172.21.15.49:0/1753452189 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f04f4074e60 0x7f04f410e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:24.867 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:24.863+0000 7f04fa6f8700 1 -- 172.21.15.49:0/1753452189 >> 172.21.15.49:0/1753452189 conn(0x7f04f40793d0 msgr2=0x7f04f40797e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:06:24.867 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:24.863+0000 7f04fa6f8700 1 -- 172.21.15.49:0/1753452189 shutdown_connections 2024-03-18T23:06:24.867 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:24.863+0000 7f04fa6f8700 1 -- 172.21.15.49:0/1753452189 wait complete. 2024-03-18T23:06:24.867 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:24.864+0000 7f04fa6f8700 1 Processor -- start 2024-03-18T23:06:24.867 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:24.864+0000 7f04fa6f8700 1 -- start start 2024-03-18T23:06:24.867 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:24.865+0000 7f04fa6f8700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f04f4074540 0x7f04f419ad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:06:24.867 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:24.865+0000 7f04fa6f8700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f04f4074e60 0x7f04f419b290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:06:24.868 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:24.865+0000 7f04fa6f8700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f04f410ed40 0x7f04f419f620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:06:24.868 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:24.865+0000 7f04fa6f8700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f04f4114840 con 0x7f04f410ed40 2024-03-18T23:06:24.868 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:24.865+0000 7f04fa6f8700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f04f41146c0 con 0x7f04f4074540 2024-03-18T23:06:24.868 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:24.865+0000 7f04fa6f8700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f04f41149c0 con 0x7f04f4074e60 2024-03-18T23:06:24.868 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:24.865+0000 7f04f3fff700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f04f4074540 0x7f04f419ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:06:24.868 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:24.865+0000 7f04f3fff700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f04f4074540 0x7f04f419ad50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.49:43760/0 (socket says 172.21.15.49:43760) 2024-03-18T23:06:24.868 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:24.865+0000 7f04f37fe700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f04f4074e60 0x7f04f419b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:06:24.868 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:24.865+0000 7f04f3fff700 1 -- 172.21.15.49:0/3159479715 learned_addr learned my addr 172.21.15.49:0/3159479715 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:06:24.868 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:24.865+0000 7f04f8c95700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f04f410ed40 0x7f04f419f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:06:24.868 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:24.865+0000 7f04f37fe700 1 -- 172.21.15.49:0/3159479715 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f04f4074540 msgr2=0x7f04f419ad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:24.868 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:24.865+0000 7f04f37fe700 1 --2- 172.21.15.49:0/3159479715 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f04f4074540 0x7f04f419ad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:24.868 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:24.865+0000 7f04f37fe700 1 -- 172.21.15.49:0/3159479715 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f04f410ed40 msgr2=0x7f04f419f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:24.868 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:24.865+0000 7f04f37fe700 1 --2- 172.21.15.49:0/3159479715 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f04f410ed40 0x7f04f419f620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:24.869 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:24.865+0000 7f04f37fe700 1 -- 172.21.15.49:0/3159479715 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04f419fd60 con 0x7f04f4074e60 2024-03-18T23:06:24.869 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:24.865+0000 7f04f3fff700 1 --2- 172.21.15.49:0/3159479715 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f04f4074540 0x7f04f419ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:06:24.869 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:24.865+0000 7f04f37fe700 1 --2- 172.21.15.49:0/3159479715 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f04f4074e60 0x7f04f419b290 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f04e800e9f0 tx=0x7f04e800ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:06:24.869 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:24.865+0000 7f04f17fa700 1 -- 172.21.15.49:0/3159479715 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04e800c9c0 con 0x7f04f4074e60 2024-03-18T23:06:24.869 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:24.866+0000 7f04fa6f8700 1 -- 172.21.15.49:0/3159479715 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04f41a0050 con 0x7f04f4074e60 2024-03-18T23:06:24.869 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:24.866+0000 7f04f17fa700 1 -- 172.21.15.49:0/3159479715 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f04e800cb60 con 0x7f04f4074e60 2024-03-18T23:06:24.869 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:24.866+0000 7f04f17fa700 1 -- 172.21.15.49:0/3159479715 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04e8018680 con 0x7f04f4074e60 2024-03-18T23:06:24.869 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:24.866+0000 7f04fa6f8700 1 -- 172.21.15.49:0/3159479715 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04f41a78d0 con 0x7f04f4074e60 2024-03-18T23:06:24.870 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:24.866+0000 7f04fa6f8700 1 -- 172.21.15.49:0/3159479715 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04f4068fd0 con 0x7f04f4074e60 2024-03-18T23:06:24.870 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:24.868+0000 7f04f17fa700 1 -- 172.21.15.49:0/3159479715 <== mon.1 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 17) ==== 137341+0+0 (secure 0 0 0) 0x7f04e8010430 con 0x7f04f4074e60 2024-03-18T23:06:24.870 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:24.868+0000 7f04f17fa700 1 --2- 172.21.15.49:0/3159479715 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f04dc0a2920 0x7f04dc0a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:06:24.870 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:24.868+0000 7f04f3fff700 1 --2- 172.21.15.49:0/3159479715 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f04dc0a2920 0x7f04dc0a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:06:24.870 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:24.868+0000 7f04f17fa700 1 -- 172.21.15.49:0/3159479715 <== mon.1 v2:172.21.15.175:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f04e80cee70 con 0x7f04f4074e60 2024-03-18T23:06:24.870 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:24.869+0000 7f04f3fff700 1 --2- 172.21.15.49:0/3159479715 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f04dc0a2920 0x7f04dc0a4de0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f04e4009750 tx=0x7f04e400b280 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:06:24.877 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:24.876+0000 7f04f17fa700 1 -- 172.21.15.49:0/3159479715 <== mon.1 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f04e809be60 con 0x7f04f4074e60 2024-03-18T23:06:25.086 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:25.084+0000 7f04fa6f8700 1 -- 172.21.15.49:0/3159479715 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f04f41a7b60 con 0x7f04f4074e60 2024-03-18T23:06:25.087 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:25.086+0000 7f04f17fa700 1 -- 172.21.15.49:0/3159479715 <== mon.1 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+22242 (secure 0 0 0) 0x7f04e809b760 con 0x7f04f4074e60 2024-03-18T23:06:25.087 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-18T23:06:25.088 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":86,"fsid":"c63591ea-e57a-11ee-95c9-87774f69a715","created":"2024-03-18T22:57:58.734467+0000","modified":"2024-03-18T23:06:05.892182+0000","last_up_change":"2024-03-18T23:05:16.576310+0000","last_in_change":"2024-03-18T23:04:57.024067+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-18T23:01:28.363432+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-18T23:06:03.172963+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-18T23:06:03.973303+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e7fc9daa-4098-4ba5-8019-9aaef1aab3e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":1651162501},{"type":"v1","addr":"172.21.15.49:6803","nonce":1651162501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":1651162501},{"type":"v1","addr":"172.21.15.49:6805","nonce":1651162501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":1651162501},{"type":"v1","addr":"172.21.15.49:6809","nonce":1651162501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":1651162501},{"type":"v1","addr":"172.21.15.49:6807","nonce":1651162501}]},"public_addr":"172.21.15.49:6803/1651162501","cluster_addr":"172.21.15.49:6805/1651162501","heartbeat_back_addr":"172.21.15.49:6809/1651162501","heartbeat_front_addr":"172.21.15.49:6807/1651162501","state":["exists","up"]},{"osd":1,"uuid":"40ac929c-e90f-434c-a796-9aabd686f1fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":1766700359},{"type":"v1","addr":"172.21.15.139:6801","nonce":1766700359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":1766700359},{"type":"v1","addr":"172.21.15.139:6803","nonce":1766700359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":1766700359},{"type":"v1","addr":"172.21.15.139:6807","nonce":1766700359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":1766700359},{"type":"v1","addr":"172.21.15.139:6805","nonce":1766700359}]},"public_addr":"172.21.15.139:6801/1766700359","cluster_addr":"172.21.15.139:6803/1766700359","heartbeat_back_addr":"172.21.15.139:6807/1766700359","heartbeat_front_addr":"172.21.15.139:6805/1766700359","state":["exists","up"]},{"osd":2,"uuid":"8ae3b666-00b4-45ed-a5ee-4d0771edabb6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":3752671234},{"type":"v1","addr":"172.21.15.175:6801","nonce":3752671234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":3752671234},{"type":"v1","addr":"172.21.15.175:6803","nonce":3752671234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":3752671234},{"type":"v1","addr":"172.21.15.175:6807","nonce":3752671234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":3752671234},{"type":"v1","addr":"172.21.15.175:6805","nonce":3752671234}]},"public_addr":"172.21.15.175:6801/3752671234","cluster_addr":"172.21.15.175:6803/3752671234","heartbeat_back_addr":"172.21.15.175:6807/3752671234","heartbeat_front_addr":"172.21.15.175:6805/3752671234","state":["exists","up"]},{"osd":3,"uuid":"1a62e89d-45d4-4237-b813-7f96a59aca7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":956160536},{"type":"v1","addr":"172.21.15.49:6811","nonce":956160536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":956160536},{"type":"v1","addr":"172.21.15.49:6813","nonce":956160536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":956160536},{"type":"v1","addr":"172.21.15.49:6817","nonce":956160536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":956160536},{"type":"v1","addr":"172.21.15.49:6815","nonce":956160536}]},"public_addr":"172.21.15.49:6811/956160536","cluster_addr":"172.21.15.49:6813/956160536","heartbeat_back_addr":"172.21.15.49:6817/956160536","heartbeat_front_addr":"172.21.15.49:6815/956160536","state":["exists","up"]},{"osd":4,"uuid":"c715101c-0c3f-4b98-b142-d64a35e187df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":3129712668},{"type":"v1","addr":"172.21.15.139:6809","nonce":3129712668}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":3129712668},{"type":"v1","addr":"172.21.15.139:6811","nonce":3129712668}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":3129712668},{"type":"v1","addr":"172.21.15.139:6815","nonce":3129712668}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":3129712668},{"type":"v1","addr":"172.21.15.139:6813","nonce":3129712668}]},"public_addr":"172.21.15.139:6809/3129712668","cluster_addr":"172.21.15.139:6811/3129712668","heartbeat_back_addr":"172.21.15.139:6815/3129712668","heartbeat_front_addr":"172.21.15.139:6813/3129712668","state":["exists","up"]},{"osd":5,"uuid":"4b641f81-77eb-45fd-8464-511e7b3062b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":2774567215},{"type":"v1","addr":"172.21.15.175:6809","nonce":2774567215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":2774567215},{"type":"v1","addr":"172.21.15.175:6811","nonce":2774567215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":2774567215},{"type":"v1","addr":"172.21.15.175:6815","nonce":2774567215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":2774567215},{"type":"v1","addr":"172.21.15.175:6813","nonce":2774567215}]},"public_addr":"172.21.15.175:6809/2774567215","cluster_addr":"172.21.15.175:6811/2774567215","heartbeat_back_addr":"172.21.15.175:6815/2774567215","heartbeat_front_addr":"172.21.15.175:6813/2774567215","state":["exists","up"]},{"osd":6,"uuid":"653da8e0-1197-4b48-abed-470f2a9c4dce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":1472215461},{"type":"v1","addr":"172.21.15.49:6819","nonce":1472215461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":1472215461},{"type":"v1","addr":"172.21.15.49:6821","nonce":1472215461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":1472215461},{"type":"v1","addr":"172.21.15.49:6825","nonce":1472215461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":1472215461},{"type":"v1","addr":"172.21.15.49:6823","nonce":1472215461}]},"public_addr":"172.21.15.49:6819/1472215461","cluster_addr":"172.21.15.49:6821/1472215461","heartbeat_back_addr":"172.21.15.49:6825/1472215461","heartbeat_front_addr":"172.21.15.49:6823/1472215461","state":["exists","up"]},{"osd":7,"uuid":"ad157c9b-4b75-451c-8740-630f0975b7fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":2163986787},{"type":"v1","addr":"172.21.15.139:6817","nonce":2163986787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":2163986787},{"type":"v1","addr":"172.21.15.139:6819","nonce":2163986787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":2163986787},{"type":"v1","addr":"172.21.15.139:6823","nonce":2163986787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":2163986787},{"type":"v1","addr":"172.21.15.139:6821","nonce":2163986787}]},"public_addr":"172.21.15.139:6817/2163986787","cluster_addr":"172.21.15.139:6819/2163986787","heartbeat_back_addr":"172.21.15.139:6823/2163986787","heartbeat_front_addr":"172.21.15.139:6821/2163986787","state":["exists","up"]},{"osd":8,"uuid":"c8e75f15-df75-4d5d-9b92-a064bffb6c01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":1194026992},{"type":"v1","addr":"172.21.15.175:6817","nonce":1194026992}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1194026992},{"type":"v1","addr":"172.21.15.175:6819","nonce":1194026992}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":1194026992},{"type":"v1","addr":"172.21.15.175:6823","nonce":1194026992}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1194026992},{"type":"v1","addr":"172.21.15.175:6821","nonce":1194026992}]},"public_addr":"172.21.15.175:6817/1194026992","cluster_addr":"172.21.15.175:6819/1194026992","heartbeat_back_addr":"172.21.15.175:6823/1194026992","heartbeat_front_addr":"172.21.15.175:6821/1194026992","state":["exists","up"]},{"osd":9,"uuid":"edb5ffef-b6a2-4941-b5ab-40f982ad30d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":2742084884},{"type":"v1","addr":"172.21.15.49:6827","nonce":2742084884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":2742084884},{"type":"v1","addr":"172.21.15.49:6829","nonce":2742084884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":2742084884},{"type":"v1","addr":"172.21.15.49:6833","nonce":2742084884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":2742084884},{"type":"v1","addr":"172.21.15.49:6831","nonce":2742084884}]},"public_addr":"172.21.15.49:6827/2742084884","cluster_addr":"172.21.15.49:6829/2742084884","heartbeat_back_addr":"172.21.15.49:6833/2742084884","heartbeat_front_addr":"172.21.15.49:6831/2742084884","state":["exists","up"]},{"osd":10,"uuid":"3945e46d-766a-4c4a-8821-d43648dd1578","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":2404740201},{"type":"v1","addr":"172.21.15.139:6825","nonce":2404740201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":2404740201},{"type":"v1","addr":"172.21.15.139:6827","nonce":2404740201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":2404740201},{"type":"v1","addr":"172.21.15.139:6831","nonce":2404740201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":2404740201},{"type":"v1","addr":"172.21.15.139:6829","nonce":2404740201}]},"public_addr":"172.21.15.139:6825/2404740201","cluster_addr":"172.21.15.139:6827/2404740201","heartbeat_back_addr":"172.21.15.139:6831/2404740201","heartbeat_front_addr":"172.21.15.139:6829/2404740201","state":["exists","up"]},{"osd":11,"uuid":"9954ba15-d877-46cc-b24a-893f9168deb8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":3519503868},{"type":"v1","addr":"172.21.15.175:6825","nonce":3519503868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":3519503868},{"type":"v1","addr":"172.21.15.175:6827","nonce":3519503868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":3519503868},{"type":"v1","addr":"172.21.15.175:6831","nonce":3519503868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":3519503868},{"type":"v1","addr":"172.21.15.175:6829","nonce":3519503868}]},"public_addr":"172.21.15.175:6825/3519503868","cluster_addr":"172.21.15.175:6827/3519503868","heartbeat_back_addr":"172.21.15.175:6831/3519503868","heartbeat_front_addr":"172.21.15.175:6829/3519503868","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:00:38.367865+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:01:02.237687+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:01:25.609751+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:01:51.268572+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:02:16.514605+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:02:41.586523+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:03:07.029964+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:03:32.956250+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:03:57.547984+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:04:23.948972+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:04:49.667210+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:05:15.284693+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.49:0/563527416":"2024-03-19T22:58:59.806253+0000","172.21.15.49:0/1036383534":"2024-03-19T22:58:59.806253+0000","172.21.15.49:6801/2609458951":"2024-03-19T22:58:59.806253+0000","172.21.15.49:0/21283102":"2024-03-19T22:58:31.972474+0000","172.21.15.49:0/1292058922":"2024-03-19T22:58:31.972474+0000","172.21.15.49:6800/1985190312":"2024-03-19T22:58:31.972474+0000","172.21.15.49:0/1912446700":"2024-03-19T22:58:59.806253+0000","172.21.15.49:0/2176210039":"2024-03-19T22:58:31.972474+0000","172.21.15.49:6801/1985190312":"2024-03-19T22:58:31.972474+0000","172.21.15.49:6800/2609458951":"2024-03-19T22:58:59.806253+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-18T23:06:25.089 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:25.088+0000 7f04fa6f8700 1 -- 172.21.15.49:0/3159479715 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f04dc0a2920 msgr2=0x7f04dc0a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:25.089 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:25.088+0000 7f04fa6f8700 1 --2- 172.21.15.49:0/3159479715 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f04dc0a2920 0x7f04dc0a4de0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f04e4009750 tx=0x7f04e400b280 comp rx=0 tx=0).stop 2024-03-18T23:06:25.089 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:25.088+0000 7f04fa6f8700 1 -- 172.21.15.49:0/3159479715 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f04f4074e60 msgr2=0x7f04f419b290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:25.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:25.088+0000 7f04fa6f8700 1 --2- 172.21.15.49:0/3159479715 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f04f4074e60 0x7f04f419b290 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f04e800e9f0 tx=0x7f04e800ed00 comp rx=0 tx=0).stop 2024-03-18T23:06:25.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:25.088+0000 7f04fa6f8700 1 -- 172.21.15.49:0/3159479715 shutdown_connections 2024-03-18T23:06:25.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:25.088+0000 7f04fa6f8700 1 --2- 172.21.15.49:0/3159479715 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f04f410ed40 0x7f04f419f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:25.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:25.088+0000 7f04fa6f8700 1 --2- 172.21.15.49:0/3159479715 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f04f4074540 0x7f04f419ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:25.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:25.088+0000 7f04fa6f8700 1 --2- 172.21.15.49:0/3159479715 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f04dc0a2920 0x7f04dc0a4de0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:25.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:25.088+0000 7f04fa6f8700 1 --2- 172.21.15.49:0/3159479715 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f04f4074e60 0x7f04f419b290 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:25.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:25.088+0000 7f04fa6f8700 1 -- 172.21.15.49:0/3159479715 >> 172.21.15.49:0/3159479715 conn(0x7f04f40793d0 msgr2=0x7f04f41007f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:06:25.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:25.089+0000 7f04fa6f8700 1 -- 172.21.15.49:0/3159479715 shutdown_connections 2024-03-18T23:06:25.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:25.089+0000 7f04fa6f8700 1 -- 172.21.15.49:0/3159479715 wait complete. 2024-03-18T23:06:25.462 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:06:25 smithi049 ceph-mon[62991]: pgmap v298: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.9 KiB/s wr, 17 op/s 2024-03-18T23:06:25.462 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:06:25 smithi049 ceph-mon[62991]: from='client.? 172.21.15.49:0/3159479715' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T23:06:25.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:06:25 smithi139 ceph-mon[67282]: pgmap v298: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.9 KiB/s wr, 17 op/s 2024-03-18T23:06:25.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:06:25 smithi139 ceph-mon[67282]: from='client.? 172.21.15.49:0/3159479715' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T23:06:25.711 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:06:25 smithi175 ceph-mon[67352]: pgmap v298: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.9 KiB/s wr, 17 op/s 2024-03-18T23:06:25.711 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:06:25 smithi175 ceph-mon[67352]: from='client.? 172.21.15.49:0/3159479715' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T23:06:25.849 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell --fsid c63591ea-e57a-11ee-95c9-87774f69a715 -- ceph tell osd.0 flush_pg_stats 2024-03-18T23:06:25.850 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell --fsid c63591ea-e57a-11ee-95c9-87774f69a715 -- ceph tell osd.1 flush_pg_stats 2024-03-18T23:06:25.850 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell --fsid c63591ea-e57a-11ee-95c9-87774f69a715 -- ceph tell osd.2 flush_pg_stats 2024-03-18T23:06:25.850 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell --fsid c63591ea-e57a-11ee-95c9-87774f69a715 -- ceph tell osd.3 flush_pg_stats 2024-03-18T23:06:25.851 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell --fsid c63591ea-e57a-11ee-95c9-87774f69a715 -- ceph tell osd.4 flush_pg_stats 2024-03-18T23:06:25.851 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell --fsid c63591ea-e57a-11ee-95c9-87774f69a715 -- ceph tell osd.5 flush_pg_stats 2024-03-18T23:06:25.852 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell --fsid c63591ea-e57a-11ee-95c9-87774f69a715 -- ceph tell osd.6 flush_pg_stats 2024-03-18T23:06:25.852 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell --fsid c63591ea-e57a-11ee-95c9-87774f69a715 -- ceph tell osd.7 flush_pg_stats 2024-03-18T23:06:25.852 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell --fsid c63591ea-e57a-11ee-95c9-87774f69a715 -- ceph tell osd.8 flush_pg_stats 2024-03-18T23:06:25.853 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell --fsid c63591ea-e57a-11ee-95c9-87774f69a715 -- ceph tell osd.9 flush_pg_stats 2024-03-18T23:06:25.853 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell --fsid c63591ea-e57a-11ee-95c9-87774f69a715 -- ceph tell osd.10 flush_pg_stats 2024-03-18T23:06:25.854 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell --fsid c63591ea-e57a-11ee-95c9-87774f69a715 -- ceph tell osd.11 flush_pg_stats 2024-03-18T23:06:26.468 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T23:06:26.534 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T23:06:26.543 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T23:06:26.545 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T23:06:26.551 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T23:06:27.546 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:06:27 smithi049 ceph-mon[62991]: pgmap v299: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 4.8 KiB/s wr, 15 op/s 2024-03-18T23:06:27.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:06:27 smithi139 ceph-mon[67282]: pgmap v299: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 4.8 KiB/s wr, 15 op/s 2024-03-18T23:06:27.711 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:06:27 smithi175 ceph-mon[67352]: pgmap v299: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 4.8 KiB/s wr, 15 op/s 2024-03-18T23:06:28.122 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T23:06:29.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:06:29 smithi139 ceph-mon[67282]: pgmap v300: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 3.5 KiB/s wr, 11 op/s 2024-03-18T23:06:29.568 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T23:06:29.568 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T23:06:29.630 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:06:29 smithi049 ceph-mon[62991]: pgmap v300: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 3.5 KiB/s wr, 11 op/s 2024-03-18T23:06:29.711 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:06:29 smithi175 ceph-mon[67352]: pgmap v300: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 3.5 KiB/s wr, 11 op/s 2024-03-18T23:06:31.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:06:31 smithi139 ceph-mon[67282]: pgmap v301: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.6 KiB/s wr, 11 op/s 2024-03-18T23:06:31.630 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:06:31 smithi049 ceph-mon[62991]: pgmap v301: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.6 KiB/s wr, 11 op/s 2024-03-18T23:06:31.711 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:06:31 smithi175 ceph-mon[67352]: pgmap v301: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.6 KiB/s wr, 11 op/s 2024-03-18T23:06:32.245 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:32.243+0000 7fca1089a700 1 -- 172.21.15.49:0/1603774402 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fca0c101c00 msgr2=0x7fca0c10e190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:32.245 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:32.243+0000 7fca1089a700 1 --2- 172.21.15.49:0/1603774402 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fca0c101c00 0x7fca0c10e190 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fca04009f90 tx=0x7fca0400b330 comp rx=0 tx=0).stop 2024-03-18T23:06:32.245 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:32.243+0000 7fca1089a700 1 -- 172.21.15.49:0/1603774402 shutdown_connections 2024-03-18T23:06:32.245 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:32.243+0000 7fca1089a700 1 --2- 172.21.15.49:0/1603774402 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fca0c101c00 0x7fca0c10e190 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:32.245 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:32.243+0000 7fca1089a700 1 --2- 172.21.15.49:0/1603774402 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fca0c1031a0 0x7fca0c103580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:32.245 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:32.243+0000 7fca1089a700 1 --2- 172.21.15.49:0/1603774402 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fca0c101240 0x7fca0c1016c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:32.245 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:32.243+0000 7fca1089a700 1 -- 172.21.15.49:0/1603774402 >> 172.21.15.49:0/1603774402 conn(0x7fca0c077230 msgr2=0x7fca0c077640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:06:32.245 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:32.243+0000 7fca1089a700 1 -- 172.21.15.49:0/1603774402 shutdown_connections 2024-03-18T23:06:32.245 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:32.243+0000 7fca1089a700 1 -- 172.21.15.49:0/1603774402 wait complete. 2024-03-18T23:06:32.247 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:32.244+0000 7fca1089a700 1 Processor -- start 2024-03-18T23:06:32.247 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:32.244+0000 7fca1089a700 1 -- start start 2024-03-18T23:06:32.247 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:32.245+0000 7fca1089a700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fca0c101240 0x7fca0c1a0940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:06:32.248 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:32.245+0000 7fca1089a700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fca0c101c00 0x7fca0c1a0e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:06:32.248 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:32.245+0000 7fca1089a700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fca0c1031a0 0x7fca0c19ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:06:32.248 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:32.245+0000 7fca1089a700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fca0c1115e0 con 0x7fca0c101c00 2024-03-18T23:06:32.248 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:32.245+0000 7fca1089a700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fca0c111460 con 0x7fca0c101240 2024-03-18T23:06:32.248 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:32.245+0000 7fca1089a700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fca0c111760 con 0x7fca0c1031a0 2024-03-18T23:06:32.248 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:32.245+0000 7fca0a59c700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fca0c101240 0x7fca0c1a0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:06:32.248 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:32.245+0000 7fca09d9b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fca0c101c00 0x7fca0c1a0e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:06:32.248 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:32.245+0000 7fca0a59c700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fca0c101240 0x7fca0c1a0940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.49:43786/0 (socket says 172.21.15.49:43786) 2024-03-18T23:06:32.248 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:32.245+0000 7fca0a59c700 1 -- 172.21.15.49:0/2615044126 learned_addr learned my addr 172.21.15.49:0/2615044126 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:06:32.248 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:32.245+0000 7fca0ad9d700 1 --2- 172.21.15.49:0/2615044126 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fca0c1031a0 0x7fca0c19ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:06:32.248 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:32.245+0000 7fca0a59c700 1 -- 172.21.15.49:0/2615044126 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fca0c1031a0 msgr2=0x7fca0c19ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:32.248 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:32.245+0000 7fca0a59c700 1 --2- 172.21.15.49:0/2615044126 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fca0c1031a0 0x7fca0c19ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:32.248 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:32.245+0000 7fca0a59c700 1 -- 172.21.15.49:0/2615044126 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fca0c101c00 msgr2=0x7fca0c1a0e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:32.249 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:32.245+0000 7fca0a59c700 1 --2- 172.21.15.49:0/2615044126 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fca0c101c00 0x7fca0c1a0e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:32.249 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:32.245+0000 7fca0a59c700 1 -- 172.21.15.49:0/2615044126 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca0c19b260 con 0x7fca0c101240 2024-03-18T23:06:32.249 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:32.245+0000 7fca0ad9d700 1 --2- 172.21.15.49:0/2615044126 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fca0c1031a0 0x7fca0c19ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T23:06:32.249 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:32.246+0000 7fca0a59c700 1 --2- 172.21.15.49:0/2615044126 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fca0c101240 0x7fca0c1a0940 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fc9fc00ec10 tx=0x7fc9fc00ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:06:32.249 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:32.246+0000 7fc9fb7fe700 1 -- 172.21.15.49:0/2615044126 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9fc00cd40 con 0x7fca0c101240 2024-03-18T23:06:32.249 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:32.246+0000 7fc9fb7fe700 1 -- 172.21.15.49:0/2615044126 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc9fc004500 con 0x7fca0c101240 2024-03-18T23:06:32.249 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:32.246+0000 7fca1089a700 1 -- 172.21.15.49:0/2615044126 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca0c19b550 con 0x7fca0c101240 2024-03-18T23:06:32.249 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:32.246+0000 7fc9fb7fe700 1 -- 172.21.15.49:0/2615044126 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9fc010690 con 0x7fca0c101240 2024-03-18T23:06:32.249 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:32.246+0000 7fca1089a700 1 -- 172.21.15.49:0/2615044126 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca0c1a7810 con 0x7fca0c101240 2024-03-18T23:06:32.249 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:32.247+0000 7fca1089a700 1 -- 172.21.15.49:0/2615044126 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fca0c0ffad0 con 0x7fca0c101240 2024-03-18T23:06:32.250 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:32.248+0000 7fc9fb7fe700 1 -- 172.21.15.49:0/2615044126 <== mon.2 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 17) ==== 137341+0+0 (secure 0 0 0) 0x7fc9fc003f10 con 0x7fca0c101240 2024-03-18T23:06:32.250 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:32.248+0000 7fc9fb7fe700 1 --2- 172.21.15.49:0/2615044126 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fc9f40a2920 0x7fc9f40a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:06:32.251 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:32.249+0000 7fca09d9b700 1 --2- 172.21.15.49:0/2615044126 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fc9f40a2920 0x7fc9f40a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:06:32.251 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:32.249+0000 7fc9fb7fe700 1 -- 172.21.15.49:0/2615044126 <== mon.2 v2:172.21.15.139:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7fc9fc0cf160 con 0x7fca0c101240 2024-03-18T23:06:32.251 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:32.249+0000 7fc9fb7fe700 1 --2- 172.21.15.49:0/2615044126 >> [v2:172.21.15.139:6808/3129712668,v1:172.21.15.139:6809/3129712668] conn(0x7fc9f40ac100 0x7fc9f40ae520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:06:32.251 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:32.249+0000 7fc9fb7fe700 1 -- 172.21.15.49:0/2615044126 --> [v2:172.21.15.139:6808/3129712668,v1:172.21.15.139:6809/3129712668] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc9f40aebd0 con 0x7fc9f40ac100 2024-03-18T23:06:32.251 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:32.249+0000 7fc9fb7fe700 1 -- 172.21.15.49:0/2615044126 <== mon.2 v2:172.21.15.139:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7fc9fc0cf510 con 0x7fca0c101240 2024-03-18T23:06:32.251 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:32.249+0000 7fca0ad9d700 1 --2- 172.21.15.49:0/2615044126 >> [v2:172.21.15.139:6808/3129712668,v1:172.21.15.139:6809/3129712668] conn(0x7fc9f40ac100 0x7fc9f40ae520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:06:32.251 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:32.249+0000 7fca09d9b700 1 --2- 172.21.15.49:0/2615044126 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fc9f40a2920 0x7fc9f40a4de0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fca0c06baf0 tx=0x7fca00009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:06:32.251 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:32.249+0000 7fca0ad9d700 1 --2- 172.21.15.49:0/2615044126 >> [v2:172.21.15.139:6808/3129712668,v1:172.21.15.139:6809/3129712668] conn(0x7fc9f40ac100 0x7fc9f40ae520 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:06:32.252 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:32.250+0000 7fc9fb7fe700 1 -- 172.21.15.49:0/2615044126 <== osd.4 v2:172.21.15.139:6808/3129712668 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc9f40aebd0 con 0x7fc9f40ac100 2024-03-18T23:06:32.268 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:32.267+0000 7fca1089a700 1 -- 172.21.15.49:0/2615044126 --> [v2:172.21.15.139:6808/3129712668,v1:172.21.15.139:6809/3129712668] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fca0c102930 con 0x7fc9f40ac100 2024-03-18T23:06:32.268 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:32.267+0000 7fc9fb7fe700 1 -- 172.21.15.49:0/2615044126 <== osd.4 v2:172.21.15.139:6808/3129712668 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fca0c102930 con 0x7fc9f40ac100 2024-03-18T23:06:32.269 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:32.268+0000 7fca1089a700 1 -- 172.21.15.49:0/2615044126 >> [v2:172.21.15.139:6808/3129712668,v1:172.21.15.139:6809/3129712668] conn(0x7fc9f40ac100 msgr2=0x7fc9f40ae520 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:32.269 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:32.268+0000 7fca1089a700 1 --2- 172.21.15.49:0/2615044126 >> [v2:172.21.15.139:6808/3129712668,v1:172.21.15.139:6809/3129712668] conn(0x7fc9f40ac100 0x7fc9f40ae520 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:32.269 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:32.268+0000 7fca1089a700 1 -- 172.21.15.49:0/2615044126 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fc9f40a2920 msgr2=0x7fc9f40a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:32.269 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:32.268+0000 7fca1089a700 1 --2- 172.21.15.49:0/2615044126 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fc9f40a2920 0x7fc9f40a4de0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fca0c06baf0 tx=0x7fca00009260 comp rx=0 tx=0).stop 2024-03-18T23:06:32.269 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:32.268+0000 7fca1089a700 1 -- 172.21.15.49:0/2615044126 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fca0c101240 msgr2=0x7fca0c1a0940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:32.269 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:32.268+0000 7fca1089a700 1 --2- 172.21.15.49:0/2615044126 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fca0c101240 0x7fca0c1a0940 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fc9fc00ec10 tx=0x7fc9fc00ef20 comp rx=0 tx=0).stop 2024-03-18T23:06:32.270 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:32.268+0000 7fca0ad9d700 1 -- 172.21.15.49:0/2615044126 reap_dead start 2024-03-18T23:06:32.270 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:32.268+0000 7fca1089a700 1 -- 172.21.15.49:0/2615044126 shutdown_connections 2024-03-18T23:06:32.270 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:32.268+0000 7fca1089a700 1 -- 172.21.15.49:0/2615044126 >> 172.21.15.49:0/2615044126 conn(0x7fca0c077230 msgr2=0x7fca0c0ff0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:06:32.270 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:32.268+0000 7fca1089a700 1 -- 172.21.15.49:0/2615044126 shutdown_connections 2024-03-18T23:06:32.270 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:32.268+0000 7fca1089a700 1 -- 172.21.15.49:0/2615044126 wait complete. 2024-03-18T23:06:33.438 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:06:33 smithi049 ceph-mon[62991]: pgmap v302: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2024-03-18T23:06:33.486 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T23:06:33.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:06:33 smithi139 ceph-mon[67282]: pgmap v302: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2024-03-18T23:06:33.711 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:06:33 smithi175 ceph-mon[67352]: pgmap v302: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2024-03-18T23:06:35.201 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T23:06:35.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:06:35 smithi139 ceph-mon[67282]: pgmap v303: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 170 B/s wr, 9 op/s 2024-03-18T23:06:35.597 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:06:35 smithi049 ceph-mon[62991]: pgmap v303: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 170 B/s wr, 9 op/s 2024-03-18T23:06:35.711 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:06:35 smithi175 ceph-mon[67352]: pgmap v303: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 170 B/s wr, 9 op/s 2024-03-18T23:06:37.242 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:37.241+0000 7ff51217f700 1 -- 172.21.15.49:0/2949891071 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff50c101e50 msgr2=0x7ff50c105e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:37.242 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:37.241+0000 7ff51217f700 1 --2- 172.21.15.49:0/2949891071 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff50c101e50 0x7ff50c105e50 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7ff500009990 tx=0x7ff500009ca0 comp rx=0 tx=0).stop 2024-03-18T23:06:37.243 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:37.241+0000 7ff51217f700 1 -- 172.21.15.49:0/2949891071 shutdown_connections 2024-03-18T23:06:37.243 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:37.241+0000 7ff51217f700 1 --2- 172.21.15.49:0/2949891071 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff50c106530 0x7ff50c10e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:37.243 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:37.241+0000 7ff51217f700 1 --2- 172.21.15.49:0/2949891071 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff50c1014a0 0x7ff50c101880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:37.243 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:37.241+0000 7ff51217f700 1 --2- 172.21.15.49:0/2949891071 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff50c101e50 0x7ff50c105e50 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:37.243 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:37.241+0000 7ff51217f700 1 -- 172.21.15.49:0/2949891071 >> 172.21.15.49:0/2949891071 conn(0x7ff50c077230 msgr2=0x7ff50c077640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:06:37.243 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:37.241+0000 7ff51217f700 1 -- 172.21.15.49:0/2949891071 shutdown_connections 2024-03-18T23:06:37.243 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:37.241+0000 7ff51217f700 1 -- 172.21.15.49:0/2949891071 wait complete. 2024-03-18T23:06:37.245 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:37.242+0000 7ff51217f700 1 Processor -- start 2024-03-18T23:06:37.245 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:37.242+0000 7ff51217f700 1 -- start start 2024-03-18T23:06:37.245 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:37.243+0000 7ff51217f700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff50c1014a0 0x7ff50c199950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:06:37.245 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:37.243+0000 7ff51217f700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff50c101e50 0x7ff50c199e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:06:37.245 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:37.243+0000 7ff51217f700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff50c106530 0x7ff50c1a49d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:06:37.246 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:37.243+0000 7ff51217f700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7ff50c1114d0 con 0x7ff50c101e50 2024-03-18T23:06:37.246 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:37.243+0000 7ff51217f700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7ff50c111350 con 0x7ff50c1014a0 2024-03-18T23:06:37.246 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:37.243+0000 7ff51217f700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff50c111650 con 0x7ff50c106530 2024-03-18T23:06:37.246 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:37.243+0000 7ff50affd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff50c101e50 0x7ff50c199e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:06:37.246 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:37.243+0000 7ff50bfff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff50c106530 0x7ff50c1a49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:06:37.246 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:37.243+0000 7ff50b7fe700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff50c1014a0 0x7ff50c199950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:06:37.246 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:37.243+0000 7ff50affd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff50c101e50 0x7ff50c199e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:45386/0 (socket says 172.21.15.49:45386) 2024-03-18T23:06:37.246 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:37.243+0000 7ff50affd700 1 -- 172.21.15.49:0/4220220735 learned_addr learned my addr 172.21.15.49:0/4220220735 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:06:37.246 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:37.243+0000 7ff50affd700 1 -- 172.21.15.49:0/4220220735 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff50c106530 msgr2=0x7ff50c1a49d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:37.246 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:37.243+0000 7ff50affd700 1 --2- 172.21.15.49:0/4220220735 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff50c106530 0x7ff50c1a49d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:37.246 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:37.243+0000 7ff50affd700 1 -- 172.21.15.49:0/4220220735 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff50c1014a0 msgr2=0x7ff50c199950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:37.246 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:37.243+0000 7ff50affd700 1 --2- 172.21.15.49:0/4220220735 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff50c1014a0 0x7ff50c199950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:37.247 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:37.243+0000 7ff50affd700 1 -- 172.21.15.49:0/4220220735 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff50c1a4f10 con 0x7ff50c101e50 2024-03-18T23:06:37.247 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:37.243+0000 7ff50bfff700 1 --2- 172.21.15.49:0/4220220735 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff50c106530 0x7ff50c1a49d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:06:37.247 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:37.243+0000 7ff50b7fe700 1 --2- 172.21.15.49:0/4220220735 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff50c1014a0 0x7ff50c199950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:06:37.247 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:37.243+0000 7ff50affd700 1 --2- 172.21.15.49:0/4220220735 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff50c101e50 0x7ff50c199e90 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7ff500000c00 tx=0x7ff50000bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:06:37.247 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:37.243+0000 7ff508ff9700 1 -- 172.21.15.49:0/4220220735 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff500027070 con 0x7ff50c101e50 2024-03-18T23:06:37.247 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:37.243+0000 7ff508ff9700 1 -- 172.21.15.49:0/4220220735 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff500003d70 con 0x7ff50c101e50 2024-03-18T23:06:37.247 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:37.244+0000 7ff51217f700 1 -- 172.21.15.49:0/4220220735 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff50c1a50e0 con 0x7ff50c101e50 2024-03-18T23:06:37.247 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:37.244+0000 7ff508ff9700 1 -- 172.21.15.49:0/4220220735 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff50000f460 con 0x7ff50c101e50 2024-03-18T23:06:37.247 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:37.244+0000 7ff51217f700 1 -- 172.21.15.49:0/4220220735 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff50c1a5590 con 0x7ff50c101e50 2024-03-18T23:06:37.248 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:37.244+0000 7ff51217f700 1 -- 172.21.15.49:0/4220220735 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ff50c062a20 con 0x7ff50c101e50 2024-03-18T23:06:37.249 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:37.246+0000 7ff508ff9700 1 -- 172.21.15.49:0/4220220735 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 17) ==== 137341+0+0 (secure 0 0 0) 0x7ff50002a070 con 0x7ff50c101e50 2024-03-18T23:06:37.249 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:37.246+0000 7ff508ff9700 1 --2- 172.21.15.49:0/4220220735 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7ff4f40a29a0 0x7ff4f40a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:06:37.249 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:37.247+0000 7ff50b7fe700 1 --2- 172.21.15.49:0/4220220735 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7ff4f40a29a0 0x7ff4f40a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:06:37.249 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:37.247+0000 7ff508ff9700 1 -- 172.21.15.49:0/4220220735 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7ff5000cfd20 con 0x7ff50c101e50 2024-03-18T23:06:37.249 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:37.247+0000 7ff508ff9700 1 --2- 172.21.15.49:0/4220220735 >> [v2:172.21.15.49:6810/956160536,v1:172.21.15.49:6811/956160536] conn(0x7ff4f40ac180 0x7ff4f40ae5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:06:37.249 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:37.247+0000 7ff508ff9700 1 -- 172.21.15.49:0/4220220735 --> [v2:172.21.15.49:6810/956160536,v1:172.21.15.49:6811/956160536] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff4f40aec50 con 0x7ff4f40ac180 2024-03-18T23:06:37.249 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:37.247+0000 7ff508ff9700 1 -- 172.21.15.49:0/4220220735 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7ff5000d0200 con 0x7ff50c101e50 2024-03-18T23:06:37.249 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:37.247+0000 7ff50bfff700 1 --2- 172.21.15.49:0/4220220735 >> [v2:172.21.15.49:6810/956160536,v1:172.21.15.49:6811/956160536] conn(0x7ff4f40ac180 0x7ff4f40ae5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:06:37.249 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:37.247+0000 7ff50b7fe700 1 --2- 172.21.15.49:0/4220220735 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7ff4f40a29a0 0x7ff4f40a4e60 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7ff4fc011490 tx=0x7ff4fc009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:06:37.249 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:37.247+0000 7ff50bfff700 1 --2- 172.21.15.49:0/4220220735 >> [v2:172.21.15.49:6810/956160536,v1:172.21.15.49:6811/956160536] conn(0x7ff4f40ac180 0x7ff4f40ae5a0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:06:37.250 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:37.248+0000 7ff508ff9700 1 -- 172.21.15.49:0/4220220735 <== osd.3 v2:172.21.15.49:6810/956160536 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff4f40aec50 con 0x7ff4f40ac180 2024-03-18T23:06:37.265 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:37.264+0000 7ff51217f700 1 -- 172.21.15.49:0/4220220735 --> [v2:172.21.15.49:6810/956160536,v1:172.21.15.49:6811/956160536] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7ff50c062ea0 con 0x7ff4f40ac180 2024-03-18T23:06:37.265 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:37.265+0000 7ff508ff9700 1 -- 172.21.15.49:0/4220220735 <== osd.3 v2:172.21.15.49:6810/956160536 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7ff50c062ea0 con 0x7ff4f40ac180 2024-03-18T23:06:37.267 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:37.265+0000 7ff51217f700 1 -- 172.21.15.49:0/4220220735 >> [v2:172.21.15.49:6810/956160536,v1:172.21.15.49:6811/956160536] conn(0x7ff4f40ac180 msgr2=0x7ff4f40ae5a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:37.267 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:37.265+0000 7ff51217f700 1 --2- 172.21.15.49:0/4220220735 >> [v2:172.21.15.49:6810/956160536,v1:172.21.15.49:6811/956160536] conn(0x7ff4f40ac180 0x7ff4f40ae5a0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:37.267 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:37.265+0000 7ff51217f700 1 -- 172.21.15.49:0/4220220735 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7ff4f40a29a0 msgr2=0x7ff4f40a4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:37.267 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:37.265+0000 7ff51217f700 1 --2- 172.21.15.49:0/4220220735 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7ff4f40a29a0 0x7ff4f40a4e60 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7ff4fc011490 tx=0x7ff4fc009260 comp rx=0 tx=0).stop 2024-03-18T23:06:37.267 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:37.265+0000 7ff51217f700 1 -- 172.21.15.49:0/4220220735 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff50c101e50 msgr2=0x7ff50c199e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:37.267 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:37.265+0000 7ff51217f700 1 --2- 172.21.15.49:0/4220220735 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff50c101e50 0x7ff50c199e90 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7ff500000c00 tx=0x7ff50000bab0 comp rx=0 tx=0).stop 2024-03-18T23:06:37.267 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:37.265+0000 7ff50bfff700 1 -- 172.21.15.49:0/4220220735 reap_dead start 2024-03-18T23:06:37.267 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:37.265+0000 7ff51217f700 1 -- 172.21.15.49:0/4220220735 shutdown_connections 2024-03-18T23:06:37.267 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:37.265+0000 7ff51217f700 1 -- 172.21.15.49:0/4220220735 >> 172.21.15.49:0/4220220735 conn(0x7ff50c077230 msgr2=0x7ff50c1076b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:06:37.267 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:37.265+0000 7ff51217f700 1 -- 172.21.15.49:0/4220220735 shutdown_connections 2024-03-18T23:06:37.268 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:37.265+0000 7ff51217f700 1 -- 172.21.15.49:0/4220220735 wait complete. 2024-03-18T23:06:37.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:06:37 smithi049 ceph-mon[62991]: pgmap v304: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 170 B/s wr, 10 op/s 2024-03-18T23:06:37.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:06:37 smithi139 ceph-mon[67282]: pgmap v304: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 170 B/s wr, 10 op/s 2024-03-18T23:06:37.711 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:06:37 smithi175 ceph-mon[67352]: pgmap v304: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 170 B/s wr, 10 op/s 2024-03-18T23:06:39.043 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T23:06:39.323 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T23:06:39.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:06:39 smithi139 ceph-mon[67282]: pgmap v305: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 170 B/s wr, 10 op/s 2024-03-18T23:06:39.630 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:06:39 smithi049 ceph-mon[62991]: pgmap v305: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 170 B/s wr, 10 op/s 2024-03-18T23:06:39.712 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:06:39 smithi175 ceph-mon[67352]: pgmap v305: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 170 B/s wr, 10 op/s 2024-03-18T23:06:40.098 INFO:teuthology.orchestra.run.smithi049.stdout:141733920822 2024-03-18T23:06:40.099 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell --fsid c63591ea-e57a-11ee-95c9-87774f69a715 -- ceph osd last-stat-seq osd.4 2024-03-18T23:06:40.313 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:40.312+0000 7f32f9ab2700 1 -- 172.21.15.49:0/3285727340 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f32f4108400 msgr2=0x7f32f41087e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:40.313 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:40.312+0000 7f32f9ab2700 1 --2- 172.21.15.49:0/3285727340 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f32f4108400 0x7f32f41087e0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f32e4009990 tx=0x7f32e4009ca0 comp rx=0 tx=0).stop 2024-03-18T23:06:40.313 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:40.312+0000 7f32f9ab2700 1 -- 172.21.15.49:0/3285727340 shutdown_connections 2024-03-18T23:06:40.313 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:40.312+0000 7f32f9ab2700 1 --2- 172.21.15.49:0/3285727340 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f32f406b370 0x7f32f410f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:40.314 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:40.312+0000 7f32f9ab2700 1 --2- 172.21.15.49:0/3285727340 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f32f4108400 0x7f32f41087e0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:40.314 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:40.312+0000 7f32f9ab2700 1 --2- 172.21.15.49:0/3285727340 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f32f406a9b0 0x7f32f406ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:40.314 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:40.312+0000 7f32f9ab2700 1 -- 172.21.15.49:0/3285727340 >> 172.21.15.49:0/3285727340 conn(0x7f32f40780d0 msgr2=0x7f32f40784e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:06:40.314 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:40.312+0000 7f32f9ab2700 1 -- 172.21.15.49:0/3285727340 shutdown_connections 2024-03-18T23:06:40.314 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:40.312+0000 7f32f9ab2700 1 -- 172.21.15.49:0/3285727340 wait complete. 2024-03-18T23:06:40.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:40.313+0000 7f32f9ab2700 1 Processor -- start 2024-03-18T23:06:40.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:40.313+0000 7f32f9ab2700 1 -- start start 2024-03-18T23:06:40.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:40.313+0000 7f32f9ab2700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f32f406a9b0 0x7f32f419e740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:06:40.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:40.313+0000 7f32f9ab2700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f32f406b370 0x7f32f419ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:06:40.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:40.313+0000 7f32f9ab2700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f32f4108400 0x7f32f4198900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:06:40.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:40.313+0000 7f32f9ab2700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f32f4112620 con 0x7f32f406a9b0 2024-03-18T23:06:40.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:40.314+0000 7f32f9ab2700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f32f41124a0 con 0x7f32f4108400 2024-03-18T23:06:40.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:40.314+0000 7f32f9ab2700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f32f41127a0 con 0x7f32f406b370 2024-03-18T23:06:40.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:40.314+0000 7f32f37fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f32f406a9b0 0x7f32f419e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:06:40.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:40.314+0000 7f32f3fff700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f32f4108400 0x7f32f4198900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:06:40.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:40.314+0000 7f32f37fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f32f406a9b0 0x7f32f419e740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:45414/0 (socket says 172.21.15.49:45414) 2024-03-18T23:06:40.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:40.314+0000 7f32f3fff700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f32f4108400 0x7f32f4198900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.49:41356/0 (socket says 172.21.15.49:41356) 2024-03-18T23:06:40.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:40.314+0000 7f32f37fe700 1 -- 172.21.15.49:0/683899713 learned_addr learned my addr 172.21.15.49:0/683899713 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:06:40.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:40.314+0000 7f32f2ffd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f32f406b370 0x7f32f419ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:06:40.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:40.314+0000 7f32f37fe700 1 -- 172.21.15.49:0/683899713 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f32f406b370 msgr2=0x7f32f419ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:40.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:40.314+0000 7f32f37fe700 1 --2- 172.21.15.49:0/683899713 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f32f406b370 0x7f32f419ec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:40.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:40.314+0000 7f32f37fe700 1 -- 172.21.15.49:0/683899713 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f32f4108400 msgr2=0x7f32f4198900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:40.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:40.314+0000 7f32f37fe700 1 --2- 172.21.15.49:0/683899713 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f32f4108400 0x7f32f4198900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:40.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:40.314+0000 7f32f37fe700 1 -- 172.21.15.49:0/683899713 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32f4199060 con 0x7f32f406a9b0 2024-03-18T23:06:40.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:40.314+0000 7f32f3fff700 1 --2- 172.21.15.49:0/683899713 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f32f4108400 0x7f32f4198900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:06:40.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:40.314+0000 7f32f2ffd700 1 --2- 172.21.15.49:0/683899713 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f32f406b370 0x7f32f419ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:06:40.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:40.314+0000 7f32f37fe700 1 --2- 172.21.15.49:0/683899713 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f32f406a9b0 0x7f32f419e740 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f32e4000c00 tx=0x7f32e400bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:06:40.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:40.315+0000 7f32f0ff9700 1 -- 172.21.15.49:0/683899713 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32e4027070 con 0x7f32f406a9b0 2024-03-18T23:06:40.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:40.315+0000 7f32f0ff9700 1 -- 172.21.15.49:0/683899713 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f32e4003d70 con 0x7f32f406a9b0 2024-03-18T23:06:40.318 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:40.315+0000 7f32f9ab2700 1 -- 172.21.15.49:0/683899713 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32f41992f0 con 0x7f32f406a9b0 2024-03-18T23:06:40.318 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:40.315+0000 7f32f0ff9700 1 -- 172.21.15.49:0/683899713 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32e400f460 con 0x7f32f406a9b0 2024-03-18T23:06:40.318 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:40.315+0000 7f32f9ab2700 1 -- 172.21.15.49:0/683899713 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32f41a5610 con 0x7f32f406a9b0 2024-03-18T23:06:40.318 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:40.316+0000 7f32f9ab2700 1 -- 172.21.15.49:0/683899713 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f32f4062a20 con 0x7f32f406a9b0 2024-03-18T23:06:40.318 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:40.317+0000 7f32f0ff9700 1 -- 172.21.15.49:0/683899713 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 17) ==== 137341+0+0 (secure 0 0 0) 0x7f32e402a070 con 0x7f32f406a9b0 2024-03-18T23:06:40.319 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:40.318+0000 7f32f0ff9700 1 --2- 172.21.15.49:0/683899713 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f32dc0a29a0 0x7f32dc0a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:06:40.320 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:40.318+0000 7f32f0ff9700 1 -- 172.21.15.49:0/683899713 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f32e40cfd90 con 0x7f32f406a9b0 2024-03-18T23:06:40.320 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:40.318+0000 7f32f0ff9700 1 --2- 172.21.15.49:0/683899713 >> [v2:172.21.15.139:6816/2163986787,v1:172.21.15.139:6817/2163986787] conn(0x7f32dc0ac180 0x7f32dc0ae5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:06:40.320 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:40.318+0000 7f32f0ff9700 1 -- 172.21.15.49:0/683899713 --> [v2:172.21.15.139:6816/2163986787,v1:172.21.15.139:6817/2163986787] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f32e40056d0 con 0x7f32dc0ac180 2024-03-18T23:06:40.320 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:40.318+0000 7f32f0ff9700 1 -- 172.21.15.49:0/683899713 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f32e40d0270 con 0x7f32f406a9b0 2024-03-18T23:06:40.320 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:40.319+0000 7f32f3fff700 1 --2- 172.21.15.49:0/683899713 >> [v2:172.21.15.139:6816/2163986787,v1:172.21.15.139:6817/2163986787] conn(0x7f32dc0ac180 0x7f32dc0ae5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:06:40.320 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:40.319+0000 7f32f2ffd700 1 --2- 172.21.15.49:0/683899713 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f32dc0a29a0 0x7f32dc0a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:06:40.321 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:40.319+0000 7f32f3fff700 1 --2- 172.21.15.49:0/683899713 >> [v2:172.21.15.139:6816/2163986787,v1:172.21.15.139:6817/2163986787] conn(0x7f32dc0ac180 0x7f32dc0ae5a0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:06:40.321 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:40.319+0000 7f32f2ffd700 1 --2- 172.21.15.49:0/683899713 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f32dc0a29a0 0x7f32dc0a4e60 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f32f4078740 tx=0x7f32e8009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:06:40.321 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:40.321+0000 7f32f0ff9700 1 -- 172.21.15.49:0/683899713 <== osd.7 v2:172.21.15.139:6816/2163986787 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f32e40056d0 con 0x7f32dc0ac180 2024-03-18T23:06:40.339 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:40.338+0000 7f32f9ab2700 1 -- 172.21.15.49:0/683899713 --> [v2:172.21.15.139:6816/2163986787,v1:172.21.15.139:6817/2163986787] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f32f4062e20 con 0x7f32dc0ac180 2024-03-18T23:06:40.339 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:40.339+0000 7f32f0ff9700 1 -- 172.21.15.49:0/683899713 <== osd.7 v2:172.21.15.139:6816/2163986787 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f32f4062e20 con 0x7f32dc0ac180 2024-03-18T23:06:40.340 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:40.339+0000 7f32f9ab2700 1 -- 172.21.15.49:0/683899713 >> [v2:172.21.15.139:6816/2163986787,v1:172.21.15.139:6817/2163986787] conn(0x7f32dc0ac180 msgr2=0x7f32dc0ae5a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:40.340 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:40.339+0000 7f32f9ab2700 1 --2- 172.21.15.49:0/683899713 >> [v2:172.21.15.139:6816/2163986787,v1:172.21.15.139:6817/2163986787] conn(0x7f32dc0ac180 0x7f32dc0ae5a0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:40.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:40.339+0000 7f32f9ab2700 1 -- 172.21.15.49:0/683899713 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f32dc0a29a0 msgr2=0x7f32dc0a4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:40.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:40.339+0000 7f32f9ab2700 1 --2- 172.21.15.49:0/683899713 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f32dc0a29a0 0x7f32dc0a4e60 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f32f4078740 tx=0x7f32e8009260 comp rx=0 tx=0).stop 2024-03-18T23:06:40.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:40.339+0000 7f32f9ab2700 1 -- 172.21.15.49:0/683899713 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f32f406a9b0 msgr2=0x7f32f419e740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:40.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:40.339+0000 7f32f9ab2700 1 --2- 172.21.15.49:0/683899713 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f32f406a9b0 0x7f32f419e740 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f32e4000c00 tx=0x7f32e400bab0 comp rx=0 tx=0).stop 2024-03-18T23:06:40.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:40.339+0000 7f32f3fff700 1 -- 172.21.15.49:0/683899713 reap_dead start 2024-03-18T23:06:40.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:40.339+0000 7f32f9ab2700 1 -- 172.21.15.49:0/683899713 shutdown_connections 2024-03-18T23:06:40.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:40.339+0000 7f32f9ab2700 1 -- 172.21.15.49:0/683899713 >> 172.21.15.49:0/683899713 conn(0x7f32f40780d0 msgr2=0x7f32f41001c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:06:40.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:40.340+0000 7f32f9ab2700 1 -- 172.21.15.49:0/683899713 shutdown_connections 2024-03-18T23:06:40.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:40.340+0000 7f32f9ab2700 1 -- 172.21.15.49:0/683899713 wait complete. 2024-03-18T23:06:40.942 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:40.937+0000 7fd48be29700 1 -- 172.21.15.49:0/3823317757 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd4841025f0 msgr2=0x7fd4841029d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:40.942 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:40.937+0000 7fd48be29700 1 --2- 172.21.15.49:0/3823317757 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd4841025f0 0x7fd4841029d0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fd4800099c0 tx=0x7fd480009cd0 comp rx=0 tx=0).stop 2024-03-18T23:06:40.942 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:40.937+0000 7fd48be29700 1 -- 172.21.15.49:0/3823317757 shutdown_connections 2024-03-18T23:06:40.942 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:40.937+0000 7fd48be29700 1 --2- 172.21.15.49:0/3823317757 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd4841076a0 0x7fd48410f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:40.942 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:40.937+0000 7fd48be29700 1 --2- 172.21.15.49:0/3823317757 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd4841025f0 0x7fd4841029d0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:40.942 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:40.937+0000 7fd48be29700 1 --2- 172.21.15.49:0/3823317757 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd484102fa0 0x7fd484106f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:40.942 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:40.937+0000 7fd48be29700 1 -- 172.21.15.49:0/3823317757 >> 172.21.15.49:0/3823317757 conn(0x7fd484079430 msgr2=0x7fd484079840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:06:40.942 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:40.938+0000 7fd48be29700 1 -- 172.21.15.49:0/3823317757 shutdown_connections 2024-03-18T23:06:40.943 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:40.938+0000 7fd48be29700 1 -- 172.21.15.49:0/3823317757 wait complete. 2024-03-18T23:06:40.943 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:40.938+0000 7fd48be29700 1 Processor -- start 2024-03-18T23:06:40.943 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:40.938+0000 7fd48be29700 1 -- start start 2024-03-18T23:06:40.943 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:40.938+0000 7fd48be29700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd4841025f0 0x7fd4841a0a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:06:40.943 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:40.938+0000 7fd48be29700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd484102fa0 0x7fd4841a0f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:06:40.943 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:40.938+0000 7fd48be29700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd4841076a0 0x7fd48419ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:06:40.943 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:40.938+0000 7fd48be29700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fd484112650 con 0x7fd484102fa0 2024-03-18T23:06:40.943 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:40.938+0000 7fd48be29700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fd4841124d0 con 0x7fd4841076a0 2024-03-18T23:06:40.943 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:40.938+0000 7fd48be29700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd4841127d0 con 0x7fd4841025f0 2024-03-18T23:06:40.943 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:40.938+0000 7fd4893c4700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd484102fa0 0x7fd4841a0f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:06:40.943 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:40.939+0000 7fd4893c4700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd484102fa0 0x7fd4841a0f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:45438/0 (socket says 172.21.15.49:45438) 2024-03-18T23:06:40.943 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:40.939+0000 7fd4893c4700 1 -- 172.21.15.49:0/1164562344 learned_addr learned my addr 172.21.15.49:0/1164562344 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:06:40.943 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:40.939+0000 7fd489bc5700 1 --2- 172.21.15.49:0/1164562344 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd4841025f0 0x7fd4841a0a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:06:40.943 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:40.939+0000 7fd48a3c6700 1 --2- 172.21.15.49:0/1164562344 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd4841076a0 0x7fd48419ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:06:40.944 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:40.939+0000 7fd4893c4700 1 -- 172.21.15.49:0/1164562344 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd4841025f0 msgr2=0x7fd4841a0a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:40.944 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:40.939+0000 7fd4893c4700 1 --2- 172.21.15.49:0/1164562344 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd4841025f0 0x7fd4841a0a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:40.944 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:40.939+0000 7fd4893c4700 1 -- 172.21.15.49:0/1164562344 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd4841076a0 msgr2=0x7fd48419ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:40.944 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:40.939+0000 7fd4893c4700 1 --2- 172.21.15.49:0/1164562344 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd4841076a0 0x7fd48419ab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:40.944 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:40.939+0000 7fd4893c4700 1 -- 172.21.15.49:0/1164562344 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd48419b3f0 con 0x7fd484102fa0 2024-03-18T23:06:40.944 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:40.939+0000 7fd4893c4700 1 --2- 172.21.15.49:0/1164562344 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd484102fa0 0x7fd4841a0f80 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fd47400e9c0 tx=0x7fd47400ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:06:40.944 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:40.939+0000 7fd47affd700 1 -- 172.21.15.49:0/1164562344 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd474004a30 con 0x7fd484102fa0 2024-03-18T23:06:40.944 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:40.939+0000 7fd48be29700 1 -- 172.21.15.49:0/1164562344 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd48419b6e0 con 0x7fd484102fa0 2024-03-18T23:06:40.944 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:40.939+0000 7fd47affd700 1 -- 172.21.15.49:0/1164562344 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd47400cca0 con 0x7fd484102fa0 2024-03-18T23:06:40.944 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:40.939+0000 7fd47affd700 1 -- 172.21.15.49:0/1164562344 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd474005690 con 0x7fd484102fa0 2024-03-18T23:06:40.944 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:40.939+0000 7fd48be29700 1 -- 172.21.15.49:0/1164562344 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4841a79c0 con 0x7fd484102fa0 2024-03-18T23:06:40.944 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:40.940+0000 7fd48be29700 1 -- 172.21.15.49:0/1164562344 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fd484075860 con 0x7fd484102fa0 2024-03-18T23:06:40.945 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:40.941+0000 7fd47affd700 1 -- 172.21.15.49:0/1164562344 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 17) ==== 137341+0+0 (secure 0 0 0) 0x7fd474004cf0 con 0x7fd484102fa0 2024-03-18T23:06:40.946 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:40.942+0000 7fd47affd700 1 --2- 172.21.15.49:0/1164562344 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fd4700a2920 0x7fd4700a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:06:40.946 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:40.942+0000 7fd489bc5700 1 --2- 172.21.15.49:0/1164562344 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fd4700a2920 0x7fd4700a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:06:40.946 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:40.942+0000 7fd47affd700 1 -- 172.21.15.49:0/1164562344 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7fd4740cf010 con 0x7fd484102fa0 2024-03-18T23:06:40.946 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:40.942+0000 7fd47affd700 1 --2- 172.21.15.49:0/1164562344 >> [v2:172.21.15.49:6826/2742084884,v1:172.21.15.49:6827/2742084884] conn(0x7fd4700ac100 0x7fd4700ae520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:06:40.946 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:40.942+0000 7fd47affd700 1 -- 172.21.15.49:0/1164562344 --> [v2:172.21.15.49:6826/2742084884,v1:172.21.15.49:6827/2742084884] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd4700aebd0 con 0x7fd4700ac100 2024-03-18T23:06:40.946 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:40.942+0000 7fd47affd700 1 -- 172.21.15.49:0/1164562344 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7fd4740cf4f0 con 0x7fd484102fa0 2024-03-18T23:06:40.946 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:40.942+0000 7fd48a3c6700 1 --2- 172.21.15.49:0/1164562344 >> [v2:172.21.15.49:6826/2742084884,v1:172.21.15.49:6827/2742084884] conn(0x7fd4700ac100 0x7fd4700ae520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:06:40.946 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:40.943+0000 7fd489bc5700 1 --2- 172.21.15.49:0/1164562344 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fd4700a2920 0x7fd4700a4de0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fd480000c00 tx=0x7fd480005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:06:40.946 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:40.943+0000 7fd48a3c6700 1 --2- 172.21.15.49:0/1164562344 >> [v2:172.21.15.49:6826/2742084884,v1:172.21.15.49:6827/2742084884] conn(0x7fd4700ac100 0x7fd4700ae520 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:06:40.946 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:40.944+0000 7fd47affd700 1 -- 172.21.15.49:0/1164562344 <== osd.9 v2:172.21.15.49:6826/2742084884 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd4700aebd0 con 0x7fd4700ac100 2024-03-18T23:06:40.961 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:40.960+0000 7fd48be29700 1 -- 172.21.15.49:0/1164562344 --> [v2:172.21.15.49:6826/2742084884,v1:172.21.15.49:6827/2742084884] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fd484103a50 con 0x7fd4700ac100 2024-03-18T23:06:40.962 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:40.961+0000 7fd47affd700 1 -- 172.21.15.49:0/1164562344 <== osd.9 v2:172.21.15.49:6826/2742084884 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fd484103a50 con 0x7fd4700ac100 2024-03-18T23:06:40.962 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:40.961+0000 7fd48be29700 1 -- 172.21.15.49:0/1164562344 >> [v2:172.21.15.49:6826/2742084884,v1:172.21.15.49:6827/2742084884] conn(0x7fd4700ac100 msgr2=0x7fd4700ae520 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:40.963 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:40.961+0000 7fd48be29700 1 --2- 172.21.15.49:0/1164562344 >> [v2:172.21.15.49:6826/2742084884,v1:172.21.15.49:6827/2742084884] conn(0x7fd4700ac100 0x7fd4700ae520 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:40.963 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:40.961+0000 7fd48be29700 1 -- 172.21.15.49:0/1164562344 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fd4700a2920 msgr2=0x7fd4700a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:40.963 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:40.961+0000 7fd48be29700 1 --2- 172.21.15.49:0/1164562344 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fd4700a2920 0x7fd4700a4de0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fd480000c00 tx=0x7fd480005fb0 comp rx=0 tx=0).stop 2024-03-18T23:06:40.963 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:40.961+0000 7fd48be29700 1 -- 172.21.15.49:0/1164562344 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd484102fa0 msgr2=0x7fd4841a0f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:40.963 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:40.961+0000 7fd48be29700 1 --2- 172.21.15.49:0/1164562344 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd484102fa0 0x7fd4841a0f80 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fd47400e9c0 tx=0x7fd47400ecd0 comp rx=0 tx=0).stop 2024-03-18T23:06:40.963 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:40.961+0000 7fd48a3c6700 1 -- 172.21.15.49:0/1164562344 reap_dead start 2024-03-18T23:06:40.963 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:40.961+0000 7fd48be29700 1 -- 172.21.15.49:0/1164562344 shutdown_connections 2024-03-18T23:06:40.963 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:40.961+0000 7fd48be29700 1 -- 172.21.15.49:0/1164562344 >> 172.21.15.49:0/1164562344 conn(0x7fd484079430 msgr2=0x7fd484101b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:06:40.963 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:40.961+0000 7fd48be29700 1 -- 172.21.15.49:0/1164562344 shutdown_connections 2024-03-18T23:06:40.963 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:40.961+0000 7fd48be29700 1 -- 172.21.15.49:0/1164562344 wait complete. 2024-03-18T23:06:41.485 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:41.482+0000 7fc7bab01700 1 -- 172.21.15.49:0/2125938360 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc7b4108360 msgr2=0x7fc7b4108740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:41.485 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:41.482+0000 7fc7bab01700 1 --2- 172.21.15.49:0/2125938360 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc7b4108360 0x7fc7b4108740 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fc7a80099c0 tx=0x7fc7a8009cd0 comp rx=0 tx=0).stop 2024-03-18T23:06:41.486 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:41.482+0000 7fc7bab01700 1 -- 172.21.15.49:0/2125938360 shutdown_connections 2024-03-18T23:06:41.486 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:41.482+0000 7fc7bab01700 1 --2- 172.21.15.49:0/2125938360 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc7b406a960 0x7fc7b410f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:41.486 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:41.482+0000 7fc7bab01700 1 --2- 172.21.15.49:0/2125938360 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc7b4108360 0x7fc7b4108740 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:41.486 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:41.482+0000 7fc7bab01700 1 --2- 172.21.15.49:0/2125938360 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc7b4069fc0 0x7fc7b406a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:41.486 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:41.482+0000 7fc7bab01700 1 -- 172.21.15.49:0/2125938360 >> 172.21.15.49:0/2125938360 conn(0x7fc7b4078140 msgr2=0x7fc7b4078550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:06:41.486 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:41.482+0000 7fc7bab01700 1 -- 172.21.15.49:0/2125938360 shutdown_connections 2024-03-18T23:06:41.486 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:41.482+0000 7fc7bab01700 1 -- 172.21.15.49:0/2125938360 wait complete. 2024-03-18T23:06:41.486 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:41.483+0000 7fc7bab01700 1 Processor -- start 2024-03-18T23:06:41.486 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:41.483+0000 7fc7bab01700 1 -- start start 2024-03-18T23:06:41.486 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:41.483+0000 7fc7bab01700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc7b4069fc0 0x7fc7b41012e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:06:41.487 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:41.483+0000 7fc7bab01700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc7b406a960 0x7fc7b4101820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:06:41.487 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:41.483+0000 7fc7bab01700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc7b4108360 0x7fc7b4101d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:06:41.487 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:41.483+0000 7fc7bab01700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fc7b41125a0 con 0x7fc7b406a960 2024-03-18T23:06:41.487 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:41.483+0000 7fc7bab01700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fc7b4112420 con 0x7fc7b4069fc0 2024-03-18T23:06:41.487 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:41.483+0000 7fc7bab01700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc7b4112720 con 0x7fc7b4108360 2024-03-18T23:06:41.487 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:41.483+0000 7fc7b3fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc7b406a960 0x7fc7b4101820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:06:41.487 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:41.483+0000 7fc7b889d700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc7b4069fc0 0x7fc7b41012e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:06:41.488 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:41.483+0000 7fc7b3fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc7b406a960 0x7fc7b4101820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:45454/0 (socket says 172.21.15.49:45454) 2024-03-18T23:06:41.488 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:41.483+0000 7fc7b3fff700 1 -- 172.21.15.49:0/2545505324 learned_addr learned my addr 172.21.15.49:0/2545505324 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:06:41.488 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:41.483+0000 7fc7b909e700 1 --2- 172.21.15.49:0/2545505324 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc7b4108360 0x7fc7b4101d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:06:41.488 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:41.483+0000 7fc7b3fff700 1 -- 172.21.15.49:0/2545505324 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc7b4108360 msgr2=0x7fc7b4101d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:41.488 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:41.483+0000 7fc7b3fff700 1 --2- 172.21.15.49:0/2545505324 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc7b4108360 0x7fc7b4101d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:41.488 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:41.484+0000 7fc7b3fff700 1 -- 172.21.15.49:0/2545505324 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc7b4069fc0 msgr2=0x7fc7b41012e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:41.488 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:41.484+0000 7fc7b3fff700 1 --2- 172.21.15.49:0/2545505324 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc7b4069fc0 0x7fc7b41012e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:41.488 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:41.484+0000 7fc7b3fff700 1 -- 172.21.15.49:0/2545505324 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7b41a6f60 con 0x7fc7b406a960 2024-03-18T23:06:41.488 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:41.484+0000 7fc7b889d700 1 --2- 172.21.15.49:0/2545505324 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc7b4069fc0 0x7fc7b41012e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:06:41.488 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:41.484+0000 7fc7b3fff700 1 --2- 172.21.15.49:0/2545505324 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc7b406a960 0x7fc7b4101820 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fc7a000e9a0 tx=0x7fc7a000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:06:41.489 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:41.484+0000 7fc7b1ffb700 1 -- 172.21.15.49:0/2545505324 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7a000ca10 con 0x7fc7b406a960 2024-03-18T23:06:41.489 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:41.484+0000 7fc7bab01700 1 -- 172.21.15.49:0/2545505324 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7b41a7250 con 0x7fc7b406a960 2024-03-18T23:06:41.489 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:41.484+0000 7fc7b1ffb700 1 -- 172.21.15.49:0/2545505324 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc7a000cbb0 con 0x7fc7b406a960 2024-03-18T23:06:41.489 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:41.484+0000 7fc7b1ffb700 1 -- 172.21.15.49:0/2545505324 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7a0018680 con 0x7fc7b406a960 2024-03-18T23:06:41.489 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:41.484+0000 7fc7bab01700 1 -- 172.21.15.49:0/2545505324 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7b41a77e0 con 0x7fc7b406a960 2024-03-18T23:06:41.490 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:41.485+0000 7fc7bab01700 1 -- 172.21.15.49:0/2545505324 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fc7b4062a20 con 0x7fc7b406a960 2024-03-18T23:06:41.490 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:41.486+0000 7fc7b1ffb700 1 -- 172.21.15.49:0/2545505324 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 17) ==== 137341+0+0 (secure 0 0 0) 0x7fc7a0010430 con 0x7fc7b406a960 2024-03-18T23:06:41.490 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:41.487+0000 7fc7b1ffb700 1 --2- 172.21.15.49:0/2545505324 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fc7a40a2920 0x7fc7a40a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:06:41.490 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:41.487+0000 7fc7b889d700 1 --2- 172.21.15.49:0/2545505324 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fc7a40a2920 0x7fc7a40a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:06:41.490 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:41.487+0000 7fc7b1ffb700 1 -- 172.21.15.49:0/2545505324 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7fc7a00cf490 con 0x7fc7b406a960 2024-03-18T23:06:41.490 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:41.487+0000 7fc7b1ffb700 1 --2- 172.21.15.49:0/2545505324 >> [v2:172.21.15.175:6800/3752671234,v1:172.21.15.175:6801/3752671234] conn(0x7fc7a40ac100 0x7fc7a40ae520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:06:41.490 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:41.487+0000 7fc7b1ffb700 1 -- 172.21.15.49:0/2545505324 --> [v2:172.21.15.175:6800/3752671234,v1:172.21.15.175:6801/3752671234] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc7a001bec0 con 0x7fc7a40ac100 2024-03-18T23:06:41.491 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:41.487+0000 7fc7b1ffb700 1 -- 172.21.15.49:0/2545505324 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7fc7a00cf970 con 0x7fc7b406a960 2024-03-18T23:06:41.491 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:41.487+0000 7fc7b909e700 1 --2- 172.21.15.49:0/2545505324 >> [v2:172.21.15.175:6800/3752671234,v1:172.21.15.175:6801/3752671234] conn(0x7fc7a40ac100 0x7fc7a40ae520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:06:41.491 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:41.487+0000 7fc7b889d700 1 --2- 172.21.15.49:0/2545505324 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fc7a40a2920 0x7fc7a40a4de0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fc7a8000c00 tx=0x7fc7a8005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:06:41.491 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:41.488+0000 7fc7b909e700 1 --2- 172.21.15.49:0/2545505324 >> [v2:172.21.15.175:6800/3752671234,v1:172.21.15.175:6801/3752671234] conn(0x7fc7a40ac100 0x7fc7a40ae520 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:06:41.491 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:41.489+0000 7fc7b1ffb700 1 -- 172.21.15.49:0/2545505324 <== osd.2 v2:172.21.15.175:6800/3752671234 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc7a001bec0 con 0x7fc7a40ac100 2024-03-18T23:06:41.506 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:41.505+0000 7fc7bab01700 1 -- 172.21.15.49:0/2545505324 --> [v2:172.21.15.175:6800/3752671234,v1:172.21.15.175:6801/3752671234] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fc7b4062e20 con 0x7fc7a40ac100 2024-03-18T23:06:41.506 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:41.505+0000 7fc7b1ffb700 1 -- 172.21.15.49:0/2545505324 <== osd.2 v2:172.21.15.175:6800/3752671234 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fc7b4062e20 con 0x7fc7a40ac100 2024-03-18T23:06:41.507 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:41.505+0000 7fc7bab01700 1 -- 172.21.15.49:0/2545505324 >> [v2:172.21.15.175:6800/3752671234,v1:172.21.15.175:6801/3752671234] conn(0x7fc7a40ac100 msgr2=0x7fc7a40ae520 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:41.508 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:41.505+0000 7fc7bab01700 1 --2- 172.21.15.49:0/2545505324 >> [v2:172.21.15.175:6800/3752671234,v1:172.21.15.175:6801/3752671234] conn(0x7fc7a40ac100 0x7fc7a40ae520 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:41.508 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:41.506+0000 7fc7bab01700 1 -- 172.21.15.49:0/2545505324 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fc7a40a2920 msgr2=0x7fc7a40a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:41.508 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:41.506+0000 7fc7bab01700 1 --2- 172.21.15.49:0/2545505324 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fc7a40a2920 0x7fc7a40a4de0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fc7a8000c00 tx=0x7fc7a8005fb0 comp rx=0 tx=0).stop 2024-03-18T23:06:41.508 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:41.506+0000 7fc7bab01700 1 -- 172.21.15.49:0/2545505324 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc7b406a960 msgr2=0x7fc7b4101820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:41.508 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:41.506+0000 7fc7bab01700 1 --2- 172.21.15.49:0/2545505324 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc7b406a960 0x7fc7b4101820 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fc7a000e9a0 tx=0x7fc7a000ecb0 comp rx=0 tx=0).stop 2024-03-18T23:06:41.508 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:41.506+0000 7fc7b909e700 1 -- 172.21.15.49:0/2545505324 reap_dead start 2024-03-18T23:06:41.508 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:41.506+0000 7fc7bab01700 1 -- 172.21.15.49:0/2545505324 shutdown_connections 2024-03-18T23:06:41.508 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:41.506+0000 7fc7bab01700 1 -- 172.21.15.49:0/2545505324 >> 172.21.15.49:0/2545505324 conn(0x7fc7b4078140 msgr2=0x7fc7b40ffd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:06:41.508 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:41.506+0000 7fc7bab01700 1 -- 172.21.15.49:0/2545505324 shutdown_connections 2024-03-18T23:06:41.508 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:41.506+0000 7fc7bab01700 1 -- 172.21.15.49:0/2545505324 wait complete. 2024-03-18T23:06:41.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:06:41 smithi139 ceph-mon[67282]: pgmap v306: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 10 op/s 2024-03-18T23:06:41.630 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:06:41 smithi049 ceph-mon[62991]: pgmap v306: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 10 op/s 2024-03-18T23:06:41.711 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:06:41 smithi175 ceph-mon[67352]: pgmap v306: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 10 op/s 2024-03-18T23:06:42.418 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:42.415+0000 7fe7ad34d700 1 -- 172.21.15.49:0/530001849 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe7a8107630 msgr2=0x7fe7a810f200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:42.418 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:42.415+0000 7fe7ad34d700 1 --2- 172.21.15.49:0/530001849 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe7a8107630 0x7fe7a810f200 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fe7980099c0 tx=0x7fe798009cd0 comp rx=0 tx=0).stop 2024-03-18T23:06:42.418 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:42.415+0000 7fe7ad34d700 1 -- 172.21.15.49:0/530001849 shutdown_connections 2024-03-18T23:06:42.418 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:42.415+0000 7fe7ad34d700 1 --2- 172.21.15.49:0/530001849 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe7a8107630 0x7fe7a810f200 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:42.418 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:42.415+0000 7fe7ad34d700 1 --2- 172.21.15.49:0/530001849 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe7a8102550 0x7fe7a8102930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:42.418 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:42.415+0000 7fe7ad34d700 1 --2- 172.21.15.49:0/530001849 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe7a8102f00 0x7fe7a8106f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:42.419 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:42.415+0000 7fe7ad34d700 1 -- 172.21.15.49:0/530001849 >> 172.21.15.49:0/530001849 conn(0x7fe7a80fddc0 msgr2=0x7fe7a81001e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:06:42.419 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:42.415+0000 7fe7ad34d700 1 -- 172.21.15.49:0/530001849 shutdown_connections 2024-03-18T23:06:42.419 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:42.415+0000 7fe7ad34d700 1 -- 172.21.15.49:0/530001849 wait complete. 2024-03-18T23:06:42.419 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:42.416+0000 7fe7ad34d700 1 Processor -- start 2024-03-18T23:06:42.419 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:42.416+0000 7fe7ad34d700 1 -- start start 2024-03-18T23:06:42.419 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:42.416+0000 7fe7ad34d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe7a8102550 0x7fe7a819e730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:06:42.419 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:42.416+0000 7fe7ad34d700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe7a8102f00 0x7fe7a819ec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:06:42.419 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:42.416+0000 7fe7ad34d700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe7a8107630 0x7fe7a81988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:06:42.419 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:42.416+0000 7fe7ad34d700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fe7a81126b0 con 0x7fe7a8102550 2024-03-18T23:06:42.419 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:42.416+0000 7fe7ad34d700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fe7a8112530 con 0x7fe7a8107630 2024-03-18T23:06:42.419 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:42.416+0000 7fe7ad34d700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe7a8112830 con 0x7fe7a8102f00 2024-03-18T23:06:42.419 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:42.416+0000 7fe7a6ffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe7a8102550 0x7fe7a819e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:06:42.419 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:42.416+0000 7fe7a6ffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe7a8102550 0x7fe7a819e730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:45508/0 (socket says 172.21.15.49:45508) 2024-03-18T23:06:42.420 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:42.416+0000 7fe7a6ffd700 1 -- 172.21.15.49:0/2789868685 learned_addr learned my addr 172.21.15.49:0/2789868685 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:06:42.420 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:42.416+0000 7fe7a77fe700 1 --2- 172.21.15.49:0/2789868685 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe7a8107630 0x7fe7a81988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:06:42.420 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:42.416+0000 7fe7a6ffd700 1 -- 172.21.15.49:0/2789868685 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe7a8102f00 msgr2=0x7fe7a819ec70 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-03-18T23:06:42.420 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:42.416+0000 7fe7a6ffd700 1 --2- 172.21.15.49:0/2789868685 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe7a8102f00 0x7fe7a819ec70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:42.420 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:42.416+0000 7fe7a6ffd700 1 -- 172.21.15.49:0/2789868685 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe7a8107630 msgr2=0x7fe7a81988f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:42.420 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:42.416+0000 7fe7a6ffd700 1 --2- 172.21.15.49:0/2789868685 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe7a8107630 0x7fe7a81988f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:42.420 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:42.416+0000 7fe7a6ffd700 1 -- 172.21.15.49:0/2789868685 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7a8199050 con 0x7fe7a8102550 2024-03-18T23:06:42.420 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:42.417+0000 7fe7a6ffd700 1 --2- 172.21.15.49:0/2789868685 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe7a8102550 0x7fe7a819e730 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fe7a000b840 tx=0x7fe7a000bb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:06:42.420 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:42.417+0000 7fe79ffff700 1 -- 172.21.15.49:0/2789868685 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7a000f840 con 0x7fe7a8102550 2024-03-18T23:06:42.420 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:42.417+0000 7fe7ad34d700 1 -- 172.21.15.49:0/2789868685 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7a8199340 con 0x7fe7a8102550 2024-03-18T23:06:42.420 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:42.417+0000 7fe79ffff700 1 -- 172.21.15.49:0/2789868685 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe7a0017400 con 0x7fe7a8102550 2024-03-18T23:06:42.420 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:42.417+0000 7fe79ffff700 1 -- 172.21.15.49:0/2789868685 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7a000d630 con 0x7fe7a8102550 2024-03-18T23:06:42.420 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:42.417+0000 7fe7ad34d700 1 -- 172.21.15.49:0/2789868685 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7a81a5600 con 0x7fe7a8102550 2024-03-18T23:06:42.422 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:42.418+0000 7fe7ad34d700 1 -- 172.21.15.49:0/2789868685 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe7a8062aa0 con 0x7fe7a8102550 2024-03-18T23:06:42.422 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:42.419+0000 7fe79ffff700 1 -- 172.21.15.49:0/2789868685 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 17) ==== 137341+0+0 (secure 0 0 0) 0x7fe7a00175b0 con 0x7fe7a8102550 2024-03-18T23:06:42.422 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:42.420+0000 7fe79ffff700 1 --2- 172.21.15.49:0/2789868685 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fe7940a29a0 0x7fe7940a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:06:42.422 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:42.420+0000 7fe79ffff700 1 -- 172.21.15.49:0/2789868685 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7fe7a00ced20 con 0x7fe7a8102550 2024-03-18T23:06:42.422 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:42.420+0000 7fe7a67fc700 1 --2- 172.21.15.49:0/2789868685 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fe7940a29a0 0x7fe7940a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:06:42.422 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:42.421+0000 7fe79ffff700 1 --2- 172.21.15.49:0/2789868685 >> [v2:172.21.15.49:6802/1651162501,v1:172.21.15.49:6803/1651162501] conn(0x7fe7940ac180 0x7fe7940ae5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:06:42.422 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:42.421+0000 7fe79ffff700 1 -- 172.21.15.49:0/2789868685 --> [v2:172.21.15.49:6802/1651162501,v1:172.21.15.49:6803/1651162501] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe7940aec50 con 0x7fe7940ac180 2024-03-18T23:06:42.422 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:42.421+0000 7fe79ffff700 1 -- 172.21.15.49:0/2789868685 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7fe7a00cf110 con 0x7fe7a8102550 2024-03-18T23:06:42.422 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:42.421+0000 7fe7a77fe700 1 --2- 172.21.15.49:0/2789868685 >> [v2:172.21.15.49:6802/1651162501,v1:172.21.15.49:6803/1651162501] conn(0x7fe7940ac180 0x7fe7940ae5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:06:42.423 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:42.421+0000 7fe7a67fc700 1 --2- 172.21.15.49:0/2789868685 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fe7940a29a0 0x7fe7940a4e60 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fe7a8108b60 tx=0x7fe790009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:06:42.423 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:42.421+0000 7fe7a77fe700 1 --2- 172.21.15.49:0/2789868685 >> [v2:172.21.15.49:6802/1651162501,v1:172.21.15.49:6803/1651162501] conn(0x7fe7940ac180 0x7fe7940ae5a0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:06:42.423 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:42.422+0000 7fe79ffff700 1 -- 172.21.15.49:0/2789868685 <== osd.0 v2:172.21.15.49:6802/1651162501 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe7940aec50 con 0x7fe7940ac180 2024-03-18T23:06:42.440 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:42.439+0000 7fe7ad34d700 1 -- 172.21.15.49:0/2789868685 --> [v2:172.21.15.49:6802/1651162501,v1:172.21.15.49:6803/1651162501] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fe7a8062ea0 con 0x7fe7940ac180 2024-03-18T23:06:42.441 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:42.440+0000 7fe79ffff700 1 -- 172.21.15.49:0/2789868685 <== osd.0 v2:172.21.15.49:6802/1651162501 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fe7a8062ea0 con 0x7fe7940ac180 2024-03-18T23:06:42.442 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:42.440+0000 7fe7ad34d700 1 -- 172.21.15.49:0/2789868685 >> [v2:172.21.15.49:6802/1651162501,v1:172.21.15.49:6803/1651162501] conn(0x7fe7940ac180 msgr2=0x7fe7940ae5a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:42.442 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:42.440+0000 7fe7ad34d700 1 --2- 172.21.15.49:0/2789868685 >> [v2:172.21.15.49:6802/1651162501,v1:172.21.15.49:6803/1651162501] conn(0x7fe7940ac180 0x7fe7940ae5a0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:42.442 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:42.440+0000 7fe7ad34d700 1 -- 172.21.15.49:0/2789868685 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fe7940a29a0 msgr2=0x7fe7940a4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:42.442 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:42.440+0000 7fe7ad34d700 1 --2- 172.21.15.49:0/2789868685 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fe7940a29a0 0x7fe7940a4e60 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fe7a8108b60 tx=0x7fe790009260 comp rx=0 tx=0).stop 2024-03-18T23:06:42.442 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:42.440+0000 7fe7ad34d700 1 -- 172.21.15.49:0/2789868685 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe7a8102550 msgr2=0x7fe7a819e730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:42.443 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:42.440+0000 7fe7ad34d700 1 --2- 172.21.15.49:0/2789868685 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe7a8102550 0x7fe7a819e730 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fe7a000b840 tx=0x7fe7a000bb50 comp rx=0 tx=0).stop 2024-03-18T23:06:42.443 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:42.440+0000 7fe7a77fe700 1 -- 172.21.15.49:0/2789868685 reap_dead start 2024-03-18T23:06:42.443 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:42.440+0000 7fe7ad34d700 1 -- 172.21.15.49:0/2789868685 shutdown_connections 2024-03-18T23:06:42.443 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:42.440+0000 7fe7ad34d700 1 -- 172.21.15.49:0/2789868685 >> 172.21.15.49:0/2789868685 conn(0x7fe7a80fddc0 msgr2=0x7fe7a80fe8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:06:42.443 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:42.441+0000 7fe7ad34d700 1 -- 172.21.15.49:0/2789868685 shutdown_connections 2024-03-18T23:06:42.443 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:42.441+0000 7fe7ad34d700 1 -- 172.21.15.49:0/2789868685 wait complete. 2024-03-18T23:06:42.547 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:41.911+0000 7fc6cf006700 1 -- 172.21.15.49:0/3213702772 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc6c8101210 msgr2=0x7fc6c8101670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:42.547 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:41.911+0000 7fc6cf006700 1 --2- 172.21.15.49:0/3213702772 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc6c8101210 0x7fc6c8101670 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fc6b4009990 tx=0x7fc6b4009ca0 comp rx=0 tx=0).stop 2024-03-18T23:06:42.547 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:41.911+0000 7fc6cf006700 1 -- 172.21.15.49:0/3213702772 shutdown_connections 2024-03-18T23:06:42.547 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:41.911+0000 7fc6cf006700 1 --2- 172.21.15.49:0/3213702772 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc6c8101bb0 0x7fc6c810e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:42.547 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:41.911+0000 7fc6cf006700 1 --2- 172.21.15.49:0/3213702772 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc6c8107250 0x7fc6c8107630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:42.548 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:41.911+0000 7fc6cf006700 1 --2- 172.21.15.49:0/3213702772 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc6c8101210 0x7fc6c8101670 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:42.548 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:41.911+0000 7fc6cf006700 1 -- 172.21.15.49:0/3213702772 >> 172.21.15.49:0/3213702772 conn(0x7fc6c8078140 msgr2=0x7fc6c8078550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:06:42.548 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:41.911+0000 7fc6cf006700 1 -- 172.21.15.49:0/3213702772 shutdown_connections 2024-03-18T23:06:42.548 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:41.911+0000 7fc6cf006700 1 -- 172.21.15.49:0/3213702772 wait complete. 2024-03-18T23:06:42.548 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:41.912+0000 7fc6cf006700 1 Processor -- start 2024-03-18T23:06:42.548 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:41.912+0000 7fc6cf006700 1 -- start start 2024-03-18T23:06:42.548 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:41.912+0000 7fc6cf006700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc6c8101210 0x7fc6c81998b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:06:42.548 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:41.912+0000 7fc6cf006700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc6c8101bb0 0x7fc6c8199df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:06:42.548 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:41.912+0000 7fc6cf006700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc6c8107250 0x7fc6c81a4930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:06:42.548 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:41.912+0000 7fc6cf006700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fc6c81114b0 con 0x7fc6c8101bb0 2024-03-18T23:06:42.548 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:41.912+0000 7fc6cf006700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fc6c8111330 con 0x7fc6c8101210 2024-03-18T23:06:42.548 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:41.912+0000 7fc6cf006700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc6c8111630 con 0x7fc6c8107250 2024-03-18T23:06:42.549 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:41.912+0000 7fc6c7fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc6c8101bb0 0x7fc6c8199df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:06:42.549 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:41.912+0000 7fc6c7fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc6c8101bb0 0x7fc6c8199df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:45472/0 (socket says 172.21.15.49:45472) 2024-03-18T23:06:42.549 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:41.912+0000 7fc6c7fff700 1 -- 172.21.15.49:0/2977370664 learned_addr learned my addr 172.21.15.49:0/2977370664 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:06:42.549 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:41.912+0000 7fc6ccda2700 1 --2- 172.21.15.49:0/2977370664 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc6c8101210 0x7fc6c81998b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:06:42.549 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:41.912+0000 7fc6cd5a3700 1 --2- 172.21.15.49:0/2977370664 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc6c8107250 0x7fc6c81a4930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:06:42.549 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:41.913+0000 7fc6c7fff700 1 -- 172.21.15.49:0/2977370664 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc6c8107250 msgr2=0x7fc6c81a4930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:42.549 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:41.913+0000 7fc6c7fff700 1 --2- 172.21.15.49:0/2977370664 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc6c8107250 0x7fc6c81a4930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:42.549 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:41.913+0000 7fc6c7fff700 1 -- 172.21.15.49:0/2977370664 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc6c8101210 msgr2=0x7fc6c81998b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:42.549 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:41.913+0000 7fc6c7fff700 1 --2- 172.21.15.49:0/2977370664 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc6c8101210 0x7fc6c81998b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:42.550 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:41.913+0000 7fc6c7fff700 1 -- 172.21.15.49:0/2977370664 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6c81a4e70 con 0x7fc6c8101bb0 2024-03-18T23:06:42.550 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:41.913+0000 7fc6c7fff700 1 --2- 172.21.15.49:0/2977370664 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc6c8101bb0 0x7fc6c8199df0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fc6b400b2e0 tx=0x7fc6b400b610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:06:42.550 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:41.913+0000 7fc6c5ffb700 1 -- 172.21.15.49:0/2977370664 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6b4026070 con 0x7fc6c8101bb0 2024-03-18T23:06:42.550 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:41.913+0000 7fc6c5ffb700 1 -- 172.21.15.49:0/2977370664 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc6b400fde0 con 0x7fc6c8101bb0 2024-03-18T23:06:42.550 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:41.913+0000 7fc6cf006700 1 -- 172.21.15.49:0/2977370664 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6c81a5040 con 0x7fc6c8101bb0 2024-03-18T23:06:42.550 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:41.913+0000 7fc6c5ffb700 1 -- 172.21.15.49:0/2977370664 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6b40173f0 con 0x7fc6c8101bb0 2024-03-18T23:06:42.550 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:41.913+0000 7fc6cf006700 1 -- 172.21.15.49:0/2977370664 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6c81a5490 con 0x7fc6c8101bb0 2024-03-18T23:06:42.550 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:41.914+0000 7fc6cf006700 1 -- 172.21.15.49:0/2977370664 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fc6c8062a20 con 0x7fc6c8101bb0 2024-03-18T23:06:42.550 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:41.915+0000 7fc6c5ffb700 1 -- 172.21.15.49:0/2977370664 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 17) ==== 137341+0+0 (secure 0 0 0) 0x7fc6b4029070 con 0x7fc6c8101bb0 2024-03-18T23:06:42.550 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:41.916+0000 7fc6c5ffb700 1 --2- 172.21.15.49:0/2977370664 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fc6b80a2920 0x7fc6b80a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:06:42.550 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:41.916+0000 7fc6ccda2700 1 --2- 172.21.15.49:0/2977370664 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fc6b80a2920 0x7fc6b80a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:06:42.550 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:41.916+0000 7fc6c5ffb700 1 -- 172.21.15.49:0/2977370664 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7fc6b40cfe40 con 0x7fc6c8101bb0 2024-03-18T23:06:42.551 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:41.916+0000 7fc6c5ffb700 1 --2- 172.21.15.49:0/2977370664 >> [v2:172.21.15.139:6800/1766700359,v1:172.21.15.139:6801/1766700359] conn(0x7fc6b80abfd0 0x7fc6b80ae3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:06:42.551 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:41.916+0000 7fc6c5ffb700 1 -- 172.21.15.49:0/2977370664 --> [v2:172.21.15.139:6800/1766700359,v1:172.21.15.139:6801/1766700359] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc6b401ae50 con 0x7fc6b80abfd0 2024-03-18T23:06:42.551 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:41.916+0000 7fc6c5ffb700 1 -- 172.21.15.49:0/2977370664 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7fc6b40d0320 con 0x7fc6c8101bb0 2024-03-18T23:06:42.551 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:41.917+0000 7fc6cd5a3700 1 --2- 172.21.15.49:0/2977370664 >> [v2:172.21.15.139:6800/1766700359,v1:172.21.15.139:6801/1766700359] conn(0x7fc6b80abfd0 0x7fc6b80ae3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:06:42.551 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:41.917+0000 7fc6ccda2700 1 --2- 172.21.15.49:0/2977370664 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fc6b80a2920 0x7fc6b80a4de0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fc6bc009be0 tx=0x7fc6bc009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:06:42.551 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:41.917+0000 7fc6cd5a3700 1 --2- 172.21.15.49:0/2977370664 >> [v2:172.21.15.139:6800/1766700359,v1:172.21.15.139:6801/1766700359] conn(0x7fc6b80abfd0 0x7fc6b80ae3f0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:06:42.551 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:41.918+0000 7fc6c5ffb700 1 -- 172.21.15.49:0/2977370664 <== osd.1 v2:172.21.15.139:6800/1766700359 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc6b401ae50 con 0x7fc6b80abfd0 2024-03-18T23:06:42.551 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:41.934+0000 7fc6cf006700 1 -- 172.21.15.49:0/2977370664 --> [v2:172.21.15.139:6800/1766700359,v1:172.21.15.139:6801/1766700359] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fc6c8062ea0 con 0x7fc6b80abfd0 2024-03-18T23:06:42.551 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:41.947+0000 7fc6c5ffb700 1 -- 172.21.15.49:0/2977370664 <== osd.1 v2:172.21.15.139:6800/1766700359 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fc6c8062ea0 con 0x7fc6b80abfd0 2024-03-18T23:06:42.551 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:41.947+0000 7fc6cf006700 1 -- 172.21.15.49:0/2977370664 >> [v2:172.21.15.139:6800/1766700359,v1:172.21.15.139:6801/1766700359] conn(0x7fc6b80abfd0 msgr2=0x7fc6b80ae3f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:42.551 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:41.947+0000 7fc6cf006700 1 --2- 172.21.15.49:0/2977370664 >> [v2:172.21.15.139:6800/1766700359,v1:172.21.15.139:6801/1766700359] conn(0x7fc6b80abfd0 0x7fc6b80ae3f0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:42.551 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:41.948+0000 7fc6cf006700 1 -- 172.21.15.49:0/2977370664 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fc6b80a2920 msgr2=0x7fc6b80a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:42.551 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:41.948+0000 7fc6cf006700 1 --2- 172.21.15.49:0/2977370664 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fc6b80a2920 0x7fc6b80a4de0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fc6bc009be0 tx=0x7fc6bc009260 comp rx=0 tx=0).stop 2024-03-18T23:06:42.551 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:41.948+0000 7fc6cf006700 1 -- 172.21.15.49:0/2977370664 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc6c8101bb0 msgr2=0x7fc6c8199df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:42.552 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:41.948+0000 7fc6cf006700 1 --2- 172.21.15.49:0/2977370664 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc6c8101bb0 0x7fc6c8199df0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fc6b400b2e0 tx=0x7fc6b400b610 comp rx=0 tx=0).stop 2024-03-18T23:06:42.552 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:41.948+0000 7fc6cd5a3700 1 -- 172.21.15.49:0/2977370664 reap_dead start 2024-03-18T23:06:42.552 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:41.948+0000 7fc6cf006700 1 -- 172.21.15.49:0/2977370664 shutdown_connections 2024-03-18T23:06:42.552 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:41.948+0000 7fc6cf006700 1 -- 172.21.15.49:0/2977370664 >> 172.21.15.49:0/2977370664 conn(0x7fc6c8078140 msgr2=0x7fc6c80ffe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:06:42.552 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:41.948+0000 7fc6cf006700 1 -- 172.21.15.49:0/2977370664 shutdown_connections 2024-03-18T23:06:42.552 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:41.948+0000 7fc6cf006700 1 -- 172.21.15.49:0/2977370664 wait complete. 2024-03-18T23:06:43.474 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:06:43 smithi049 ceph-mon[62991]: pgmap v307: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 170 B/s wr, 11 op/s 2024-03-18T23:06:43.476 INFO:teuthology.orchestra.run.smithi049.stdout:55834574919 2024-03-18T23:06:43.476 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell --fsid c63591ea-e57a-11ee-95c9-87774f69a715 -- ceph osd last-stat-seq osd.1 2024-03-18T23:06:43.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:06:43 smithi139 ceph-mon[67282]: pgmap v307: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 170 B/s wr, 11 op/s 2024-03-18T23:06:43.711 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:06:43 smithi175 ceph-mon[67352]: pgmap v307: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 170 B/s wr, 11 op/s 2024-03-18T23:06:44.060 INFO:teuthology.orchestra.run.smithi049.stdout:219043332135 2024-03-18T23:06:44.060 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell --fsid c63591ea-e57a-11ee-95c9-87774f69a715 -- ceph osd last-stat-seq osd.7 2024-03-18T23:06:44.159 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:42.332+0000 7f88aa67a700 1 -- 172.21.15.49:0/3303944292 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f88a4108340 msgr2=0x7f88a4108720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:44.159 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:42.332+0000 7f88aa67a700 1 --2- 172.21.15.49:0/3303944292 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f88a4108340 0x7f88a4108720 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f88940099c0 tx=0x7f8894009cd0 comp rx=0 tx=0).stop 2024-03-18T23:06:44.159 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:42.332+0000 7f88aa67a700 1 -- 172.21.15.49:0/3303944292 shutdown_connections 2024-03-18T23:06:44.159 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:42.332+0000 7f88aa67a700 1 --2- 172.21.15.49:0/3303944292 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f88a4102d20 0x7f88a410f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:44.159 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:42.332+0000 7f88aa67a700 1 --2- 172.21.15.49:0/3303944292 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f88a4108340 0x7f88a4108720 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:44.160 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:42.332+0000 7f88aa67a700 1 --2- 172.21.15.49:0/3303944292 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f88a4102360 0x7f88a41027e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:44.160 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:42.332+0000 7f88aa67a700 1 -- 172.21.15.49:0/3303944292 >> 172.21.15.49:0/3303944292 conn(0x7f88a4079440 msgr2=0x7f88a4079850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:06:44.160 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:42.332+0000 7f88aa67a700 1 -- 172.21.15.49:0/3303944292 shutdown_connections 2024-03-18T23:06:44.160 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:42.332+0000 7f88aa67a700 1 -- 172.21.15.49:0/3303944292 wait complete. 2024-03-18T23:06:44.160 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:42.332+0000 7f88aa67a700 1 Processor -- start 2024-03-18T23:06:44.160 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:42.333+0000 7f88aa67a700 1 -- start start 2024-03-18T23:06:44.160 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:42.333+0000 7f88aa67a700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f88a4102360 0x7f88a41a08d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:06:44.160 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:42.333+0000 7f88aa67a700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f88a4102d20 0x7f88a41a0e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:06:44.160 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:42.333+0000 7f88aa67a700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f88a4108340 0x7f88a419aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:06:44.160 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:42.333+0000 7f88aa67a700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f88a41125f0 con 0x7f88a4102360 2024-03-18T23:06:44.160 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:42.333+0000 7f88aa67a700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f88a4112470 con 0x7f88a4108340 2024-03-18T23:06:44.160 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:42.333+0000 7f88aa67a700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f88a4112770 con 0x7f88a4102d20 2024-03-18T23:06:44.160 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:42.333+0000 7f88a3fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f88a4102360 0x7f88a41a08d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:06:44.160 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:42.333+0000 7f88a37fe700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f88a4102d20 0x7f88a41a0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:06:44.160 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:42.333+0000 7f88a3fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f88a4102360 0x7f88a41a08d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:45490/0 (socket says 172.21.15.49:45490) 2024-03-18T23:06:44.161 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:42.333+0000 7f88a3fff700 1 -- 172.21.15.49:0/2537071097 learned_addr learned my addr 172.21.15.49:0/2537071097 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:06:44.161 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:42.333+0000 7f88a8c17700 1 --2- 172.21.15.49:0/2537071097 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f88a4108340 0x7f88a419aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:06:44.161 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:42.333+0000 7f88a3fff700 1 -- 172.21.15.49:0/2537071097 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f88a4102d20 msgr2=0x7f88a41a0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:44.162 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:42.333+0000 7f88a3fff700 1 --2- 172.21.15.49:0/2537071097 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f88a4102d20 0x7f88a41a0e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:44.162 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:42.333+0000 7f88a3fff700 1 -- 172.21.15.49:0/2537071097 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f88a4108340 msgr2=0x7f88a419aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:44.162 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:42.333+0000 7f88a3fff700 1 --2- 172.21.15.49:0/2537071097 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f88a4108340 0x7f88a419aa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:44.162 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:42.333+0000 7f88a3fff700 1 -- 172.21.15.49:0/2537071097 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88a419b1f0 con 0x7f88a4102360 2024-03-18T23:06:44.162 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:42.333+0000 7f88a8c17700 1 --2- 172.21.15.49:0/2537071097 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f88a4108340 0x7f88a419aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:06:44.162 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:42.333+0000 7f88a37fe700 1 --2- 172.21.15.49:0/2537071097 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f88a4102d20 0x7f88a41a0e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:06:44.162 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:42.334+0000 7f88a3fff700 1 --2- 172.21.15.49:0/2537071097 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f88a4102360 0x7f88a41a08d0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f8894006010 tx=0x7f889400f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:06:44.162 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:42.334+0000 7f88a17fa700 1 -- 172.21.15.49:0/2537071097 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8894003680 con 0x7f88a4102360 2024-03-18T23:06:44.162 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:42.334+0000 7f88aa67a700 1 -- 172.21.15.49:0/2537071097 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88a419b480 con 0x7f88a4102360 2024-03-18T23:06:44.162 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:42.334+0000 7f88a17fa700 1 -- 172.21.15.49:0/2537071097 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f88940050d0 con 0x7f88a4102360 2024-03-18T23:06:44.162 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:42.334+0000 7f88a17fa700 1 -- 172.21.15.49:0/2537071097 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8894003db0 con 0x7f88a4102360 2024-03-18T23:06:44.162 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:42.334+0000 7f88aa67a700 1 -- 172.21.15.49:0/2537071097 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88a41a77a0 con 0x7f88a4102360 2024-03-18T23:06:44.162 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:42.335+0000 7f88aa67a700 1 -- 172.21.15.49:0/2537071097 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f88a4076270 con 0x7f88a4102360 2024-03-18T23:06:44.163 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:42.336+0000 7f88a17fa700 1 -- 172.21.15.49:0/2537071097 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 17) ==== 137341+0+0 (secure 0 0 0) 0x7f8894005280 con 0x7f88a4102360 2024-03-18T23:06:44.163 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:42.337+0000 7f88a17fa700 1 --2- 172.21.15.49:0/2537071097 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f888c0a2ac0 0x7f888c0a4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:06:44.163 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:42.337+0000 7f88a37fe700 1 --2- 172.21.15.49:0/2537071097 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f888c0a2ac0 0x7f888c0a4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:06:44.163 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:42.337+0000 7f88a17fa700 1 -- 172.21.15.49:0/2537071097 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f88940cfad0 con 0x7f88a4102360 2024-03-18T23:06:44.163 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:42.337+0000 7f88a17fa700 1 --2- 172.21.15.49:0/2537071097 >> [v2:172.21.15.175:6824/3519503868,v1:172.21.15.175:6825/3519503868] conn(0x7f888c0ac2a0 0x7f888c0ae6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:06:44.163 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:42.337+0000 7f88a17fa700 1 -- 172.21.15.49:0/2537071097 --> [v2:172.21.15.175:6824/3519503868,v1:172.21.15.175:6825/3519503868] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8894003680 con 0x7f888c0ac2a0 2024-03-18T23:06:44.163 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:42.337+0000 7f88a17fa700 1 -- 172.21.15.49:0/2537071097 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f88940cffb0 con 0x7f88a4102360 2024-03-18T23:06:44.163 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:42.337+0000 7f88a8c17700 1 --2- 172.21.15.49:0/2537071097 >> [v2:172.21.15.175:6824/3519503868,v1:172.21.15.175:6825/3519503868] conn(0x7f888c0ac2a0 0x7f888c0ae6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:06:44.163 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:42.337+0000 7f88a37fe700 1 --2- 172.21.15.49:0/2537071097 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f888c0a2ac0 0x7f888c0a4f80 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f88a4079d40 tx=0x7f8898009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:06:44.163 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:42.338+0000 7f88a8c17700 1 --2- 172.21.15.49:0/2537071097 >> [v2:172.21.15.175:6824/3519503868,v1:172.21.15.175:6825/3519503868] conn(0x7f888c0ac2a0 0x7f888c0ae6c0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:06:44.163 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:42.339+0000 7f88a17fa700 1 -- 172.21.15.49:0/2537071097 <== osd.11 v2:172.21.15.175:6824/3519503868 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8894003680 con 0x7f888c0ac2a0 2024-03-18T23:06:44.163 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:42.355+0000 7f88aa67a700 1 -- 172.21.15.49:0/2537071097 --> [v2:172.21.15.175:6824/3519503868,v1:172.21.15.175:6825/3519503868] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f88a41039f0 con 0x7f888c0ac2a0 2024-03-18T23:06:44.163 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:42.355+0000 7f88a17fa700 1 -- 172.21.15.49:0/2537071097 <== osd.11 v2:172.21.15.175:6824/3519503868 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f88a41039f0 con 0x7f888c0ac2a0 2024-03-18T23:06:44.163 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:42.356+0000 7f88aa67a700 1 -- 172.21.15.49:0/2537071097 >> [v2:172.21.15.175:6824/3519503868,v1:172.21.15.175:6825/3519503868] conn(0x7f888c0ac2a0 msgr2=0x7f888c0ae6c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:44.163 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:42.356+0000 7f88aa67a700 1 --2- 172.21.15.49:0/2537071097 >> [v2:172.21.15.175:6824/3519503868,v1:172.21.15.175:6825/3519503868] conn(0x7f888c0ac2a0 0x7f888c0ae6c0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:44.164 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:42.356+0000 7f88aa67a700 1 -- 172.21.15.49:0/2537071097 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f888c0a2ac0 msgr2=0x7f888c0a4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:44.164 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:42.356+0000 7f88aa67a700 1 --2- 172.21.15.49:0/2537071097 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f888c0a2ac0 0x7f888c0a4f80 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f88a4079d40 tx=0x7f8898009260 comp rx=0 tx=0).stop 2024-03-18T23:06:44.164 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:42.356+0000 7f88aa67a700 1 -- 172.21.15.49:0/2537071097 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f88a4102360 msgr2=0x7f88a41a08d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:44.164 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:42.356+0000 7f88aa67a700 1 --2- 172.21.15.49:0/2537071097 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f88a4102360 0x7f88a41a08d0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f8894006010 tx=0x7f889400f690 comp rx=0 tx=0).stop 2024-03-18T23:06:44.164 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:42.356+0000 7f88a8c17700 1 -- 172.21.15.49:0/2537071097 reap_dead start 2024-03-18T23:06:44.164 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:42.356+0000 7f88aa67a700 1 -- 172.21.15.49:0/2537071097 shutdown_connections 2024-03-18T23:06:44.164 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:42.356+0000 7f88aa67a700 1 -- 172.21.15.49:0/2537071097 >> 172.21.15.49:0/2537071097 conn(0x7f88a4079440 msgr2=0x7f88a4074840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:06:44.164 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:42.356+0000 7f88aa67a700 1 -- 172.21.15.49:0/2537071097 shutdown_connections 2024-03-18T23:06:44.164 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:42.356+0000 7f88aa67a700 1 -- 172.21.15.49:0/2537071097 wait complete. 2024-03-18T23:06:45.527 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:06:45 smithi049 ceph-mon[62991]: pgmap v308: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-18T23:06:45.556 INFO:teuthology.orchestra.run.smithi049.stdout:347892350996 2024-03-18T23:06:45.557 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell --fsid c63591ea-e57a-11ee-95c9-87774f69a715 -- ceph osd last-stat-seq osd.11 2024-03-18T23:06:45.666 INFO:teuthology.orchestra.run.smithi049.stdout:292057776158 2024-03-18T23:06:45.666 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell --fsid c63591ea-e57a-11ee-95c9-87774f69a715 -- ceph osd last-stat-seq osd.9 2024-03-18T23:06:45.711 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:06:45 smithi175 ceph-mon[67352]: pgmap v308: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-18T23:06:45.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:06:45 smithi139 ceph-mon[67282]: pgmap v308: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-18T23:06:46.647 INFO:teuthology.orchestra.run.smithi049.stdout:77309411394 2024-03-18T23:06:46.648 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell --fsid c63591ea-e57a-11ee-95c9-87774f69a715 -- ceph osd last-stat-seq osd.2 2024-03-18T23:06:47.077 INFO:teuthology.orchestra.run.smithi049.stdout:107374182459 2024-03-18T23:06:47.077 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell --fsid c63591ea-e57a-11ee-95c9-87774f69a715 -- ceph osd last-stat-seq osd.3 2024-03-18T23:06:47.630 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:06:47 smithi049 ceph-mon[62991]: pgmap v309: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-18T23:06:47.711 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:06:47 smithi175 ceph-mon[67352]: pgmap v309: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-18T23:06:47.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:06:47 smithi139 ceph-mon[67282]: pgmap v309: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-18T23:06:48.282 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:48.282+0000 7f367ca6d700 1 -- 172.21.15.49:0/3198476493 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3678107250 msgr2=0x7f3678107630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:48.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:48.282+0000 7f367ca6d700 1 --2- 172.21.15.49:0/3198476493 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3678107250 0x7f3678107630 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f3660009990 tx=0x7f3660009ca0 comp rx=0 tx=0).stop 2024-03-18T23:06:48.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:48.282+0000 7f367ca6d700 1 -- 172.21.15.49:0/3198476493 shutdown_connections 2024-03-18T23:06:48.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:48.282+0000 7f367ca6d700 1 --2- 172.21.15.49:0/3198476493 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3678101bb0 0x7f367810e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:48.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:48.282+0000 7f367ca6d700 1 --2- 172.21.15.49:0/3198476493 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3678107250 0x7f3678107630 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:48.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:48.282+0000 7f367ca6d700 1 --2- 172.21.15.49:0/3198476493 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3678101210 0x7f3678101670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:48.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:48.282+0000 7f367ca6d700 1 -- 172.21.15.49:0/3198476493 >> 172.21.15.49:0/3198476493 conn(0x7f3678078140 msgr2=0x7f3678078550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:06:48.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:48.282+0000 7f367ca6d700 1 -- 172.21.15.49:0/3198476493 shutdown_connections 2024-03-18T23:06:48.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:48.282+0000 7f367ca6d700 1 -- 172.21.15.49:0/3198476493 wait complete. 2024-03-18T23:06:48.285 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:48.283+0000 7f367ca6d700 1 Processor -- start 2024-03-18T23:06:48.285 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:48.283+0000 7f367ca6d700 1 -- start start 2024-03-18T23:06:48.285 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:48.283+0000 7f367ca6d700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3678101210 0x7f36781998b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:06:48.285 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:48.283+0000 7f367ca6d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3678101bb0 0x7f3678199df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:06:48.285 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:48.283+0000 7f367ca6d700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3678107250 0x7f36781a4930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:06:48.286 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:48.283+0000 7f367ca6d700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f36781114b0 con 0x7f3678101bb0 2024-03-18T23:06:48.286 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:48.283+0000 7f367ca6d700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f3678111330 con 0x7f3678101210 2024-03-18T23:06:48.286 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:48.283+0000 7f367ca6d700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f3678111630 con 0x7f3678107250 2024-03-18T23:06:48.286 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:48.284+0000 7f3675d9b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3678101bb0 0x7f3678199df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:06:48.286 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:48.284+0000 7f3675d9b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3678101bb0 0x7f3678199df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:52390/0 (socket says 172.21.15.49:52390) 2024-03-18T23:06:48.286 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:48.284+0000 7f3675d9b700 1 -- 172.21.15.49:0/1058436729 learned_addr learned my addr 172.21.15.49:0/1058436729 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:06:48.286 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:48.284+0000 7f367659c700 1 --2- 172.21.15.49:0/1058436729 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3678101210 0x7f36781998b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:06:48.286 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:48.284+0000 7f3676d9d700 1 --2- 172.21.15.49:0/1058436729 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3678107250 0x7f36781a4930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:06:48.286 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:48.284+0000 7f3675d9b700 1 -- 172.21.15.49:0/1058436729 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3678107250 msgr2=0x7f36781a4930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:48.286 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:48.284+0000 7f3675d9b700 1 --2- 172.21.15.49:0/1058436729 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3678107250 0x7f36781a4930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:48.286 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:48.284+0000 7f3675d9b700 1 -- 172.21.15.49:0/1058436729 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3678101210 msgr2=0x7f36781998b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:48.286 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:48.284+0000 7f3675d9b700 1 --2- 172.21.15.49:0/1058436729 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3678101210 0x7f36781998b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:48.287 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:48.284+0000 7f3675d9b700 1 -- 172.21.15.49:0/1058436729 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36781a4e70 con 0x7f3678101bb0 2024-03-18T23:06:48.287 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:48.284+0000 7f367659c700 1 --2- 172.21.15.49:0/1058436729 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3678101210 0x7f36781998b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:06:48.287 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:48.284+0000 7f3675d9b700 1 --2- 172.21.15.49:0/1058436729 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3678101bb0 0x7f3678199df0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f366800e9c0 tx=0x7f366800ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:06:48.287 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:48.284+0000 7f366f7fe700 1 -- 172.21.15.49:0/1058436729 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f366800cbd0 con 0x7f3678101bb0 2024-03-18T23:06:48.287 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:48.284+0000 7f367ca6d700 1 -- 172.21.15.49:0/1058436729 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36781a50a0 con 0x7f3678101bb0 2024-03-18T23:06:48.287 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:48.284+0000 7f366f7fe700 1 -- 172.21.15.49:0/1058436729 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f366800ee60 con 0x7f3678101bb0 2024-03-18T23:06:48.287 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:48.284+0000 7f366f7fe700 1 -- 172.21.15.49:0/1058436729 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3668005630 con 0x7f3678101bb0 2024-03-18T23:06:48.287 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:48.284+0000 7f367ca6d700 1 -- 172.21.15.49:0/1058436729 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36781a55e0 con 0x7f3678101bb0 2024-03-18T23:06:48.287 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:48.285+0000 7f367ca6d700 1 -- 172.21.15.49:0/1058436729 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f3678062a20 con 0x7f3678101bb0 2024-03-18T23:06:48.288 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:48.286+0000 7f366f7fe700 1 -- 172.21.15.49:0/1058436729 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 17) ==== 137341+0+0 (secure 0 0 0) 0x7f3668004750 con 0x7f3678101bb0 2024-03-18T23:06:48.288 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:48.287+0000 7f366f7fe700 1 --2- 172.21.15.49:0/1058436729 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f36640a2920 0x7f36640a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:06:48.288 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:48.287+0000 7f367659c700 1 --2- 172.21.15.49:0/1058436729 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f36640a2920 0x7f36640a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:06:48.289 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:48.287+0000 7f366f7fe700 1 -- 172.21.15.49:0/1058436729 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f36680cf0f0 con 0x7f3678101bb0 2024-03-18T23:06:48.289 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:48.287+0000 7f366f7fe700 1 --2- 172.21.15.49:0/1058436729 >> [v2:172.21.15.175:6808/2774567215,v1:172.21.15.175:6809/2774567215] conn(0x7f36640ac100 0x7f36640ae520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:06:48.289 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:48.287+0000 7f366f7fe700 1 -- 172.21.15.49:0/1058436729 --> [v2:172.21.15.175:6808/2774567215,v1:172.21.15.175:6809/2774567215] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f36640aebd0 con 0x7f36640ac100 2024-03-18T23:06:48.289 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:48.287+0000 7f366f7fe700 1 -- 172.21.15.49:0/1058436729 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f36680cf5d0 con 0x7f3678101bb0 2024-03-18T23:06:48.289 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:48.288+0000 7f367659c700 1 --2- 172.21.15.49:0/1058436729 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f36640a2920 0x7f36640a4de0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f366000b2e0 tx=0x7f3660005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:06:48.289 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:48.288+0000 7f3676d9d700 1 --2- 172.21.15.49:0/1058436729 >> [v2:172.21.15.175:6808/2774567215,v1:172.21.15.175:6809/2774567215] conn(0x7f36640ac100 0x7f36640ae520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:06:48.289 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:48.288+0000 7f3676d9d700 1 --2- 172.21.15.49:0/1058436729 >> [v2:172.21.15.175:6808/2774567215,v1:172.21.15.175:6809/2774567215] conn(0x7f36640ac100 0x7f36640ae520 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:06:48.289 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:48.289+0000 7f366f7fe700 1 -- 172.21.15.49:0/1058436729 <== osd.5 v2:172.21.15.175:6808/2774567215 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f36640aebd0 con 0x7f36640ac100 2024-03-18T23:06:48.305 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:48.305+0000 7f367ca6d700 1 -- 172.21.15.49:0/1058436729 --> [v2:172.21.15.175:6808/2774567215,v1:172.21.15.175:6809/2774567215] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f3678062ea0 con 0x7f36640ac100 2024-03-18T23:06:48.305 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:48.305+0000 7f366f7fe700 1 -- 172.21.15.49:0/1058436729 <== osd.5 v2:172.21.15.175:6808/2774567215 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f3678062ea0 con 0x7f36640ac100 2024-03-18T23:06:48.306 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:48.306+0000 7f367ca6d700 1 -- 172.21.15.49:0/1058436729 >> [v2:172.21.15.175:6808/2774567215,v1:172.21.15.175:6809/2774567215] conn(0x7f36640ac100 msgr2=0x7f36640ae520 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:48.306 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:48.306+0000 7f367ca6d700 1 --2- 172.21.15.49:0/1058436729 >> [v2:172.21.15.175:6808/2774567215,v1:172.21.15.175:6809/2774567215] conn(0x7f36640ac100 0x7f36640ae520 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:48.306 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:48.306+0000 7f367ca6d700 1 -- 172.21.15.49:0/1058436729 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f36640a2920 msgr2=0x7f36640a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:48.306 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:48.306+0000 7f367ca6d700 1 --2- 172.21.15.49:0/1058436729 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f36640a2920 0x7f36640a4de0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f366000b2e0 tx=0x7f3660005960 comp rx=0 tx=0).stop 2024-03-18T23:06:48.306 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:48.306+0000 7f367ca6d700 1 -- 172.21.15.49:0/1058436729 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3678101bb0 msgr2=0x7f3678199df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:48.306 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:48.306+0000 7f367ca6d700 1 --2- 172.21.15.49:0/1058436729 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3678101bb0 0x7f3678199df0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f366800e9c0 tx=0x7f366800ecd0 comp rx=0 tx=0).stop 2024-03-18T23:06:48.307 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:48.306+0000 7f3676d9d700 1 -- 172.21.15.49:0/1058436729 reap_dead start 2024-03-18T23:06:48.307 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:48.306+0000 7f367ca6d700 1 -- 172.21.15.49:0/1058436729 shutdown_connections 2024-03-18T23:06:48.307 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:48.306+0000 7f367ca6d700 1 -- 172.21.15.49:0/1058436729 >> 172.21.15.49:0/1058436729 conn(0x7f3678078140 msgr2=0x7f36780fff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:06:48.307 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:48.306+0000 7f367ca6d700 1 -- 172.21.15.49:0/1058436729 shutdown_connections 2024-03-18T23:06:48.307 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:48.306+0000 7f367ca6d700 1 -- 172.21.15.49:0/1058436729 wait complete. 2024-03-18T23:06:49.630 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:06:49 smithi049 ceph-mon[62991]: pgmap v310: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-03-18T23:06:49.712 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:06:49 smithi175 ceph-mon[67352]: pgmap v310: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-03-18T23:06:49.758 INFO:teuthology.orchestra.run.smithi049.stdout:34359738444 2024-03-18T23:06:49.758 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell --fsid c63591ea-e57a-11ee-95c9-87774f69a715 -- ceph osd last-stat-seq osd.0 2024-03-18T23:06:49.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:06:49 smithi139 ceph-mon[67282]: pgmap v310: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-03-18T23:06:50.407 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:50.405+0000 7f961deea700 1 -- 172.21.15.49:0/2593023957 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9618101e50 msgr2=0x7f9618105e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:50.407 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:50.405+0000 7f961deea700 1 --2- 172.21.15.49:0/2593023957 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9618101e50 0x7f9618105e50 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f960c009990 tx=0x7f960c009ca0 comp rx=0 tx=0).stop 2024-03-18T23:06:50.407 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:50.405+0000 7f961deea700 1 -- 172.21.15.49:0/2593023957 shutdown_connections 2024-03-18T23:06:50.407 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:50.405+0000 7f961deea700 1 --2- 172.21.15.49:0/2593023957 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9618106530 0x7f961810e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:50.407 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:50.405+0000 7f961deea700 1 --2- 172.21.15.49:0/2593023957 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f96181014a0 0x7f9618101880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:50.407 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:50.405+0000 7f961deea700 1 --2- 172.21.15.49:0/2593023957 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9618101e50 0x7f9618105e50 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:50.407 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:50.405+0000 7f961deea700 1 -- 172.21.15.49:0/2593023957 >> 172.21.15.49:0/2593023957 conn(0x7f9618077230 msgr2=0x7f9618077640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:06:50.407 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:50.405+0000 7f961deea700 1 -- 172.21.15.49:0/2593023957 shutdown_connections 2024-03-18T23:06:50.407 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:50.405+0000 7f961deea700 1 -- 172.21.15.49:0/2593023957 wait complete. 2024-03-18T23:06:50.408 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:50.406+0000 7f961deea700 1 Processor -- start 2024-03-18T23:06:50.408 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:50.406+0000 7f961deea700 1 -- start start 2024-03-18T23:06:50.408 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:50.406+0000 7f961deea700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f96181014a0 0x7f961819ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:06:50.408 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:50.406+0000 7f961deea700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9618101e50 0x7f961819b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:06:50.408 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:50.406+0000 7f961deea700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9618106530 0x7f961819f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:06:50.408 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:50.406+0000 7f961deea700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f96181114d0 con 0x7f9618106530 2024-03-18T23:06:50.408 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:50.406+0000 7f961deea700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f9618111350 con 0x7f96181014a0 2024-03-18T23:06:50.408 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:50.406+0000 7f961deea700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f9618111650 con 0x7f9618101e50 2024-03-18T23:06:50.408 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:50.406+0000 7f96177fe700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f96181014a0 0x7f961819ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:06:50.408 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:50.407+0000 7f9616ffd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9618101e50 0x7f961819b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:06:50.408 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:50.407+0000 7f96177fe700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f96181014a0 0x7f961819ad10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.49:47516/0 (socket says 172.21.15.49:47516) 2024-03-18T23:06:50.408 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:50.407+0000 7f96177fe700 1 -- 172.21.15.49:0/856287264 learned_addr learned my addr 172.21.15.49:0/856287264 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:06:50.408 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:50.407+0000 7f9617fff700 1 --2- 172.21.15.49:0/856287264 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9618106530 0x7f961819f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:06:50.408 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:50.407+0000 7f96177fe700 1 -- 172.21.15.49:0/856287264 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9618101e50 msgr2=0x7f961819b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:50.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:50.407+0000 7f96177fe700 1 --2- 172.21.15.49:0/856287264 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9618101e50 0x7f961819b250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:50.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:50.407+0000 7f96177fe700 1 -- 172.21.15.49:0/856287264 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9618106530 msgr2=0x7f961819f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:50.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:50.407+0000 7f96177fe700 1 --2- 172.21.15.49:0/856287264 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9618106530 0x7f961819f5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:50.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:50.407+0000 7f96177fe700 1 -- 172.21.15.49:0/856287264 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f961819fd20 con 0x7f96181014a0 2024-03-18T23:06:50.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:50.407+0000 7f96177fe700 1 --2- 172.21.15.49:0/856287264 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f96181014a0 0x7f961819ad10 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f960800e9a0 tx=0x7f960800ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:06:50.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:50.407+0000 7f9614ff9700 1 -- 172.21.15.49:0/856287264 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f960800ca10 con 0x7f96181014a0 2024-03-18T23:06:50.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:50.407+0000 7f9614ff9700 1 -- 172.21.15.49:0/856287264 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f960800cbb0 con 0x7f96181014a0 2024-03-18T23:06:50.410 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:50.407+0000 7f9614ff9700 1 -- 172.21.15.49:0/856287264 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9608018680 con 0x7f96181014a0 2024-03-18T23:06:50.410 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:50.408+0000 7f961deea700 1 -- 172.21.15.49:0/856287264 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96181a0010 con 0x7f96181014a0 2024-03-18T23:06:50.410 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:50.408+0000 7f961deea700 1 -- 172.21.15.49:0/856287264 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96181a7890 con 0x7f96181014a0 2024-03-18T23:06:50.410 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:50.409+0000 7f961deea700 1 -- 172.21.15.49:0/856287264 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f961810d7b0 con 0x7f96181014a0 2024-03-18T23:06:50.410 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:50.410+0000 7f9614ff9700 1 -- 172.21.15.49:0/856287264 <== mon.2 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 17) ==== 137341+0+0 (secure 0 0 0) 0x7f9608010430 con 0x7f96181014a0 2024-03-18T23:06:50.411 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:50.411+0000 7f9614ff9700 1 --2- 172.21.15.49:0/856287264 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f96000a29a0 0x7f96000a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:06:50.411 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:50.411+0000 7f9616ffd700 1 --2- 172.21.15.49:0/856287264 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f96000a29a0 0x7f96000a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:06:50.412 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:50.411+0000 7f9614ff9700 1 -- 172.21.15.49:0/856287264 <== mon.2 v2:172.21.15.139:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f96080cec70 con 0x7f96181014a0 2024-03-18T23:06:50.412 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:50.412+0000 7f9614ff9700 1 --2- 172.21.15.49:0/856287264 >> [v2:172.21.15.175:6816/1194026992,v1:172.21.15.175:6817/1194026992] conn(0x7f96000ac180 0x7f96000ae5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:06:50.412 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:50.412+0000 7f9614ff9700 1 -- 172.21.15.49:0/856287264 --> [v2:172.21.15.175:6816/1194026992,v1:172.21.15.175:6817/1194026992] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9608012ec0 con 0x7f96000ac180 2024-03-18T23:06:50.412 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:50.412+0000 7f9614ff9700 1 -- 172.21.15.49:0/856287264 <== mon.2 v2:172.21.15.139:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f96080d1340 con 0x7f96181014a0 2024-03-18T23:06:50.412 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:50.412+0000 7f9617fff700 1 --2- 172.21.15.49:0/856287264 >> [v2:172.21.15.175:6816/1194026992,v1:172.21.15.175:6817/1194026992] conn(0x7f96000ac180 0x7f96000ae5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:06:50.412 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:50.412+0000 7f9616ffd700 1 --2- 172.21.15.49:0/856287264 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f96000a29a0 0x7f96000a4e60 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f960c006010 tx=0x7f960c005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:06:50.412 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:50.412+0000 7f9617fff700 1 --2- 172.21.15.49:0/856287264 >> [v2:172.21.15.175:6816/1194026992,v1:172.21.15.175:6817/1194026992] conn(0x7f96000ac180 0x7f96000ae5a0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:06:50.413 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:50.413+0000 7f9614ff9700 1 -- 172.21.15.49:0/856287264 <== osd.8 v2:172.21.15.175:6816/1194026992 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9608012ec0 con 0x7f96000ac180 2024-03-18T23:06:50.430 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:50.430+0000 7f961deea700 1 -- 172.21.15.49:0/856287264 --> [v2:172.21.15.175:6816/1194026992,v1:172.21.15.175:6817/1194026992] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f9618102930 con 0x7f96000ac180 2024-03-18T23:06:50.430 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:50.430+0000 7f9614ff9700 1 -- 172.21.15.49:0/856287264 <== osd.8 v2:172.21.15.175:6816/1194026992 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f9618102930 con 0x7f96000ac180 2024-03-18T23:06:50.431 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:50.431+0000 7f961deea700 1 -- 172.21.15.49:0/856287264 >> [v2:172.21.15.175:6816/1194026992,v1:172.21.15.175:6817/1194026992] conn(0x7f96000ac180 msgr2=0x7f96000ae5a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:50.431 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:50.431+0000 7f961deea700 1 --2- 172.21.15.49:0/856287264 >> [v2:172.21.15.175:6816/1194026992,v1:172.21.15.175:6817/1194026992] conn(0x7f96000ac180 0x7f96000ae5a0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:50.431 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:50.431+0000 7f961deea700 1 -- 172.21.15.49:0/856287264 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f96000a29a0 msgr2=0x7f96000a4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:50.431 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:50.431+0000 7f961deea700 1 --2- 172.21.15.49:0/856287264 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f96000a29a0 0x7f96000a4e60 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f960c006010 tx=0x7f960c005960 comp rx=0 tx=0).stop 2024-03-18T23:06:50.431 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:50.431+0000 7f961deea700 1 -- 172.21.15.49:0/856287264 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f96181014a0 msgr2=0x7f961819ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:50.432 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:50.431+0000 7f961deea700 1 --2- 172.21.15.49:0/856287264 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f96181014a0 0x7f961819ad10 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f960800e9a0 tx=0x7f960800ecb0 comp rx=0 tx=0).stop 2024-03-18T23:06:50.432 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:50.431+0000 7f9617fff700 1 -- 172.21.15.49:0/856287264 reap_dead start 2024-03-18T23:06:50.432 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:50.431+0000 7f961deea700 1 -- 172.21.15.49:0/856287264 shutdown_connections 2024-03-18T23:06:50.432 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:50.431+0000 7f961deea700 1 -- 172.21.15.49:0/856287264 >> 172.21.15.49:0/856287264 conn(0x7f9618077230 msgr2=0x7f961806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:06:50.432 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:50.431+0000 7f961deea700 1 -- 172.21.15.49:0/856287264 shutdown_connections 2024-03-18T23:06:50.432 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:50.431+0000 7f961deea700 1 -- 172.21.15.49:0/856287264 wait complete. 2024-03-18T23:06:50.979 INFO:teuthology.orchestra.run.smithi049.stdout:163208757299 2024-03-18T23:06:50.981 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell --fsid c63591ea-e57a-11ee-95c9-87774f69a715 -- ceph osd last-stat-seq osd.5 2024-03-18T23:06:51.630 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:06:51 smithi049 ceph-mon[62991]: pgmap v311: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-18T23:06:51.711 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:06:51 smithi175 ceph-mon[67352]: pgmap v311: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-18T23:06:51.738 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:51.737+0000 7f2fcf3a9700 1 -- 172.21.15.49:0/2212985793 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2fc81014a0 msgr2=0x7f2fc8101880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:51.738 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:51.737+0000 7f2fcf3a9700 1 --2- 172.21.15.49:0/2212985793 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2fc81014a0 0x7f2fc8101880 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f2fc000b0d0 tx=0x7f2fc000b3e0 comp rx=0 tx=0).stop 2024-03-18T23:06:51.738 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:51.737+0000 7f2fcf3a9700 1 -- 172.21.15.49:0/2212985793 shutdown_connections 2024-03-18T23:06:51.738 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:51.737+0000 7f2fcf3a9700 1 --2- 172.21.15.49:0/2212985793 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2fc81064f0 0x7f2fc810e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:51.739 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:51.737+0000 7f2fcf3a9700 1 --2- 172.21.15.49:0/2212985793 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2fc81014a0 0x7f2fc8101880 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:51.739 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:51.737+0000 7f2fcf3a9700 1 --2- 172.21.15.49:0/2212985793 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2fc8101e50 0x7f2fc8105e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:51.739 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:51.737+0000 7f2fcf3a9700 1 -- 172.21.15.49:0/2212985793 >> 172.21.15.49:0/2212985793 conn(0x7f2fc8077260 msgr2=0x7f2fc8077670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:06:51.739 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:51.737+0000 7f2fcf3a9700 1 -- 172.21.15.49:0/2212985793 shutdown_connections 2024-03-18T23:06:51.739 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:51.738+0000 7f2fcf3a9700 1 -- 172.21.15.49:0/2212985793 wait complete. 2024-03-18T23:06:51.739 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:51.738+0000 7f2fcf3a9700 1 Processor -- start 2024-03-18T23:06:51.739 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:51.738+0000 7f2fcf3a9700 1 -- start start 2024-03-18T23:06:51.740 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:51.738+0000 7f2fcf3a9700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2fc81014a0 0x7f2fc8102eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:06:51.741 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:51.738+0000 7f2fcf3a9700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2fc8101e50 0x7f2fc81033f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:06:51.741 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:51.738+0000 7f2fcf3a9700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2fc81064f0 0x7f2fc8103930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:06:51.741 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:51.738+0000 7f2fcf3a9700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f2fc81114d0 con 0x7f2fc81064f0 2024-03-18T23:06:51.741 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:51.738+0000 7f2fcf3a9700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f2fc8111350 con 0x7f2fc8101e50 2024-03-18T23:06:51.741 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:51.738+0000 7f2fcf3a9700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f2fc8111650 con 0x7f2fc81014a0 2024-03-18T23:06:51.741 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:51.739+0000 7f2fcc944700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2fc8101e50 0x7f2fc81033f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:06:51.741 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:51.739+0000 7f2fcd946700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2fc81064f0 0x7f2fc8103930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:06:51.741 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:51.739+0000 7f2fcc944700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2fc8101e50 0x7f2fc81033f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.49:47540/0 (socket says 172.21.15.49:47540) 2024-03-18T23:06:51.741 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:51.739+0000 7f2fcc944700 1 -- 172.21.15.49:0/1286695270 learned_addr learned my addr 172.21.15.49:0/1286695270 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:06:51.741 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:51.739+0000 7f2fcd145700 1 --2- 172.21.15.49:0/1286695270 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2fc81014a0 0x7f2fc8102eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:06:51.741 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:51.739+0000 7f2fcc944700 1 -- 172.21.15.49:0/1286695270 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2fc81014a0 msgr2=0x7f2fc8102eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:51.741 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:51.739+0000 7f2fcc944700 1 --2- 172.21.15.49:0/1286695270 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2fc81014a0 0x7f2fc8102eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:51.741 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:51.739+0000 7f2fcc944700 1 -- 172.21.15.49:0/1286695270 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2fc81064f0 msgr2=0x7f2fc8103930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:51.742 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:51.739+0000 7f2fcc944700 1 --2- 172.21.15.49:0/1286695270 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2fc81064f0 0x7f2fc8103930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:51.742 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:51.739+0000 7f2fcc944700 1 -- 172.21.15.49:0/1286695270 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fc81a7020 con 0x7f2fc8101e50 2024-03-18T23:06:51.742 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:51.739+0000 7f2fcc944700 1 --2- 172.21.15.49:0/1286695270 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2fc8101e50 0x7f2fc81033f0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f2fbc00e970 tx=0x7f2fbc00ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:06:51.742 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:51.739+0000 7f2fc67fc700 1 -- 172.21.15.49:0/1286695270 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fbc004a30 con 0x7f2fc8101e50 2024-03-18T23:06:51.742 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:51.739+0000 7f2fcf3a9700 1 -- 172.21.15.49:0/1286695270 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2fc81a7310 con 0x7f2fc8101e50 2024-03-18T23:06:51.742 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:51.739+0000 7f2fc67fc700 1 -- 172.21.15.49:0/1286695270 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2fbc00cca0 con 0x7f2fc8101e50 2024-03-18T23:06:51.742 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:51.739+0000 7f2fc67fc700 1 -- 172.21.15.49:0/1286695270 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fbc005680 con 0x7f2fc8101e50 2024-03-18T23:06:51.742 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:51.739+0000 7f2fcf3a9700 1 -- 172.21.15.49:0/1286695270 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2fc81a78a0 con 0x7f2fc8101e50 2024-03-18T23:06:51.742 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:51.740+0000 7f2fcf3a9700 1 -- 172.21.15.49:0/1286695270 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f2fc81052c0 con 0x7f2fc8101e50 2024-03-18T23:06:51.743 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:51.741+0000 7f2fc67fc700 1 -- 172.21.15.49:0/1286695270 <== mon.2 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 17) ==== 137341+0+0 (secure 0 0 0) 0x7f2fbc00c7c0 con 0x7f2fc8101e50 2024-03-18T23:06:51.743 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:51.742+0000 7f2fc67fc700 1 --2- 172.21.15.49:0/1286695270 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f2fb00a2920 0x7f2fb00a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:06:51.743 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:51.742+0000 7f2fcd145700 1 --2- 172.21.15.49:0/1286695270 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f2fb00a2920 0x7f2fb00a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:06:51.744 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:51.742+0000 7f2fc67fc700 1 -- 172.21.15.49:0/1286695270 <== mon.2 v2:172.21.15.139:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f2fbc0cf0f0 con 0x7f2fc8101e50 2024-03-18T23:06:51.744 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:51.742+0000 7f2fc67fc700 1 --2- 172.21.15.49:0/1286695270 >> [v2:172.21.15.49:6818/1472215461,v1:172.21.15.49:6819/1472215461] conn(0x7f2fb00ac100 0x7f2fb00ae520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:06:51.744 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:51.742+0000 7f2fc67fc700 1 -- 172.21.15.49:0/1286695270 --> [v2:172.21.15.49:6818/1472215461,v1:172.21.15.49:6819/1472215461] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f2fb00aebd0 con 0x7f2fb00ac100 2024-03-18T23:06:51.744 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:51.742+0000 7f2fc67fc700 1 -- 172.21.15.49:0/1286695270 <== mon.2 v2:172.21.15.139:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f2fbc0cf4a0 con 0x7f2fc8101e50 2024-03-18T23:06:51.744 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:51.742+0000 7f2fcd946700 1 --2- 172.21.15.49:0/1286695270 >> [v2:172.21.15.49:6818/1472215461,v1:172.21.15.49:6819/1472215461] conn(0x7f2fb00ac100 0x7f2fb00ae520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:06:51.744 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:51.742+0000 7f2fcd145700 1 --2- 172.21.15.49:0/1286695270 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f2fb00a2920 0x7f2fb00a4de0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f2fc0009fd0 tx=0x7f2fc000b040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:06:51.744 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:51.743+0000 7f2fcd946700 1 --2- 172.21.15.49:0/1286695270 >> [v2:172.21.15.49:6818/1472215461,v1:172.21.15.49:6819/1472215461] conn(0x7f2fb00ac100 0x7f2fb00ae520 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:06:51.744 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:51.743+0000 7f2fc67fc700 1 -- 172.21.15.49:0/1286695270 <== osd.6 v2:172.21.15.49:6818/1472215461 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f2fb00aebd0 con 0x7f2fb00ac100 2024-03-18T23:06:51.761 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:51.761+0000 7f2fcf3a9700 1 -- 172.21.15.49:0/1286695270 --> [v2:172.21.15.49:6818/1472215461,v1:172.21.15.49:6819/1472215461] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f2fc81028f0 con 0x7f2fb00ac100 2024-03-18T23:06:51.761 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:51.762+0000 7f2fc67fc700 1 -- 172.21.15.49:0/1286695270 <== osd.6 v2:172.21.15.49:6818/1472215461 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f2fc81028f0 con 0x7f2fb00ac100 2024-03-18T23:06:51.762 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:51.762+0000 7f2fcf3a9700 1 -- 172.21.15.49:0/1286695270 >> [v2:172.21.15.49:6818/1472215461,v1:172.21.15.49:6819/1472215461] conn(0x7f2fb00ac100 msgr2=0x7f2fb00ae520 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:51.762 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:51.762+0000 7f2fcf3a9700 1 --2- 172.21.15.49:0/1286695270 >> [v2:172.21.15.49:6818/1472215461,v1:172.21.15.49:6819/1472215461] conn(0x7f2fb00ac100 0x7f2fb00ae520 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:51.762 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:51.762+0000 7f2fcf3a9700 1 -- 172.21.15.49:0/1286695270 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f2fb00a2920 msgr2=0x7f2fb00a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:51.762 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:51.762+0000 7f2fcf3a9700 1 --2- 172.21.15.49:0/1286695270 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f2fb00a2920 0x7f2fb00a4de0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f2fc0009fd0 tx=0x7f2fc000b040 comp rx=0 tx=0).stop 2024-03-18T23:06:51.763 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:51.762+0000 7f2fcf3a9700 1 -- 172.21.15.49:0/1286695270 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2fc8101e50 msgr2=0x7f2fc81033f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:51.763 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:51.762+0000 7f2fcf3a9700 1 --2- 172.21.15.49:0/1286695270 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2fc8101e50 0x7f2fc81033f0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f2fbc00e970 tx=0x7f2fbc00ec80 comp rx=0 tx=0).stop 2024-03-18T23:06:51.763 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:51.762+0000 7f2fcd946700 1 -- 172.21.15.49:0/1286695270 reap_dead start 2024-03-18T23:06:51.763 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:51.762+0000 7f2fcf3a9700 1 -- 172.21.15.49:0/1286695270 shutdown_connections 2024-03-18T23:06:51.763 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:51.762+0000 7f2fcf3a9700 1 -- 172.21.15.49:0/1286695270 >> 172.21.15.49:0/1286695270 conn(0x7f2fc8077260 msgr2=0x7f2fc8100a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:06:51.763 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:51.762+0000 7f2fcf3a9700 1 -- 172.21.15.49:0/1286695270 shutdown_connections 2024-03-18T23:06:51.763 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:51.762+0000 7f2fcf3a9700 1 -- 172.21.15.49:0/1286695270 wait complete. 2024-03-18T23:06:51.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:06:51 smithi139 ceph-mon[67282]: pgmap v311: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-18T23:06:52.717 INFO:teuthology.orchestra.run.smithi049.stdout:257698037797 2024-03-18T23:06:52.718 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell --fsid c63591ea-e57a-11ee-95c9-87774f69a715 -- ceph osd last-stat-seq osd.8 2024-03-18T23:06:53.380 INFO:teuthology.orchestra.run.smithi049.stdout:197568495664 2024-03-18T23:06:53.381 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell --fsid c63591ea-e57a-11ee-95c9-87774f69a715 -- ceph osd last-stat-seq osd.6 2024-03-18T23:06:53.392 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:06:53 smithi049 ceph-mon[62991]: pgmap v312: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 10 op/s 2024-03-18T23:06:53.414 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T23:06:53.417 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T23:06:53.437 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T23:06:53.439 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T23:06:53.446 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T23:06:53.713 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:06:53 smithi175 ceph-mon[67352]: pgmap v312: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 10 op/s 2024-03-18T23:06:53.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:06:53 smithi139 ceph-mon[67282]: pgmap v312: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 10 op/s 2024-03-18T23:06:55.630 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:06:55 smithi049 ceph-mon[62991]: pgmap v313: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-03-18T23:06:55.711 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:06:55 smithi175 ceph-mon[67352]: pgmap v313: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-03-18T23:06:55.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:06:55 smithi139 ceph-mon[67282]: pgmap v313: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-03-18T23:06:56.819 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T23:06:56.820 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T23:06:56.837 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T23:06:56.838 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T23:06:56.838 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:56.837+0000 7f55cb97a700 1 -- 172.21.15.49:0/2108155358 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f55c4113340 msgr2=0x7f55c4115730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:56.838 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:56.837+0000 7f55cb97a700 1 --2- 172.21.15.49:0/2108155358 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f55c4113340 0x7f55c4115730 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f55bc00b0a0 tx=0x7f55bc00b3b0 comp rx=0 tx=0).stop 2024-03-18T23:06:56.838 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:56.838+0000 7f55cb97a700 1 -- 172.21.15.49:0/2108155358 shutdown_connections 2024-03-18T23:06:56.838 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:56.838+0000 7f55cb97a700 1 --2- 172.21.15.49:0/2108155358 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f55c4113340 0x7f55c4115730 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:56.838 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:56.838+0000 7f55cb97a700 1 --2- 172.21.15.49:0/2108155358 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f55c40746b0 0x7f55c4074a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:56.838 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:56.838+0000 7f55cb97a700 1 --2- 172.21.15.49:0/2108155358 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f55c4074fd0 0x7f55c4112e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:56.839 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:56.838+0000 7f55cb97a700 1 -- 172.21.15.49:0/2108155358 >> 172.21.15.49:0/2108155358 conn(0x7f55c40fdde0 msgr2=0x7f55c4100200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:06:56.839 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:56.838+0000 7f55cb97a700 1 -- 172.21.15.49:0/2108155358 shutdown_connections 2024-03-18T23:06:56.841 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:56.838+0000 7f55cb97a700 1 -- 172.21.15.49:0/2108155358 wait complete. 2024-03-18T23:06:56.842 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:56.839+0000 7f55cb97a700 1 Processor -- start 2024-03-18T23:06:56.842 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:56.839+0000 7f55cb97a700 1 -- start start 2024-03-18T23:06:56.842 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:56.839+0000 7f55cb97a700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f55c40746b0 0x7f55c419efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:06:56.842 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:56.839+0000 7f55cb97a700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f55c4074fd0 0x7f55c419f500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:06:56.842 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:56.839+0000 7f55cb97a700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f55c4113340 0x7f55c41a3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:06:56.842 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:56.839+0000 7f55cb97a700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f55c4118c90 con 0x7f55c40746b0 2024-03-18T23:06:56.842 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:56.839+0000 7f55cb97a700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f55c4118b10 con 0x7f55c4074fd0 2024-03-18T23:06:56.842 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:56.839+0000 7f55cb97a700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f55c4118e10 con 0x7f55c4113340 2024-03-18T23:06:56.842 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:56.840+0000 7f55c9716700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f55c40746b0 0x7f55c419efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:06:56.842 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:56.840+0000 7f55c9716700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f55c40746b0 0x7f55c419efc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:60976/0 (socket says 172.21.15.49:60976) 2024-03-18T23:06:56.842 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:56.840+0000 7f55c9716700 1 -- 172.21.15.49:0/4214914267 learned_addr learned my addr 172.21.15.49:0/4214914267 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:06:56.842 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:56.840+0000 7f55c8f15700 1 --2- 172.21.15.49:0/4214914267 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f55c4074fd0 0x7f55c419f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:06:56.843 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:56.840+0000 7f55c9f17700 1 --2- 172.21.15.49:0/4214914267 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f55c4113340 0x7f55c41a3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:06:56.843 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:56.840+0000 7f55c9716700 1 -- 172.21.15.49:0/4214914267 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f55c4113340 msgr2=0x7f55c41a3890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:56.843 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:56.840+0000 7f55c9716700 1 --2- 172.21.15.49:0/4214914267 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f55c4113340 0x7f55c41a3890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:56.843 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:56.840+0000 7f55c9716700 1 -- 172.21.15.49:0/4214914267 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f55c4074fd0 msgr2=0x7f55c419f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:56.843 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:56.840+0000 7f55c9716700 1 --2- 172.21.15.49:0/4214914267 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f55c4074fd0 0x7f55c419f500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:56.843 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:56.840+0000 7f55c9716700 1 -- 172.21.15.49:0/4214914267 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55c41a3fd0 con 0x7f55c40746b0 2024-03-18T23:06:56.843 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:56.841+0000 7f55c9716700 1 --2- 172.21.15.49:0/4214914267 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f55c40746b0 0x7f55c419efc0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f55c000c9b0 tx=0x7f55c000ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:06:56.843 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:56.841+0000 7f55ba7fc700 1 -- 172.21.15.49:0/4214914267 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55c0007b00 con 0x7f55c40746b0 2024-03-18T23:06:56.843 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:56.841+0000 7f55ba7fc700 1 -- 172.21.15.49:0/4214914267 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f55c0007ca0 con 0x7f55c40746b0 2024-03-18T23:06:56.843 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:56.841+0000 7f55ba7fc700 1 -- 172.21.15.49:0/4214914267 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55c000e650 con 0x7f55c40746b0 2024-03-18T23:06:56.843 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:56.841+0000 7f55cb97a700 1 -- 172.21.15.49:0/4214914267 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55c41a42c0 con 0x7f55c40746b0 2024-03-18T23:06:56.843 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:56.841+0000 7f55cb97a700 1 -- 172.21.15.49:0/4214914267 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55c41abb40 con 0x7f55c40746b0 2024-03-18T23:06:56.844 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:56.844+0000 7f55cb97a700 1 -- 172.21.15.49:0/4214914267 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f55a8001030 con 0x7f55c40746b0 2024-03-18T23:06:56.846 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:56.846+0000 7f55ba7fc700 1 -- 172.21.15.49:0/4214914267 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 17) ==== 137341+0+0 (secure 0 0 0) 0x7f55c000f450 con 0x7f55c40746b0 2024-03-18T23:06:56.847 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:56.847+0000 7f55ba7fc700 1 --2- 172.21.15.49:0/4214914267 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f55b00a2aa0 0x7f55b00a4f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:06:56.847 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:56.847+0000 7f55ba7fc700 1 -- 172.21.15.49:0/4214914267 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f55c00cede0 con 0x7f55c40746b0 2024-03-18T23:06:56.848 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:56.848+0000 7f55ba7fc700 1 --2- 172.21.15.49:0/4214914267 >> [v2:172.21.15.139:6824/2404740201,v1:172.21.15.139:6825/2404740201] conn(0x7f55b00ac280 0x7f55b00ae6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:06:56.848 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:56.848+0000 7f55ba7fc700 1 -- 172.21.15.49:0/4214914267 --> [v2:172.21.15.139:6824/2404740201,v1:172.21.15.139:6825/2404740201] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f55c0012ec0 con 0x7f55b00ac280 2024-03-18T23:06:56.848 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:56.848+0000 7f55ba7fc700 1 -- 172.21.15.49:0/4214914267 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f55c00d1530 con 0x7f55c40746b0 2024-03-18T23:06:56.848 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:56.848+0000 7f55c8f15700 1 --2- 172.21.15.49:0/4214914267 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f55b00a2aa0 0x7f55b00a4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:06:56.848 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:56.848+0000 7f55c9f17700 1 --2- 172.21.15.49:0/4214914267 >> [v2:172.21.15.139:6824/2404740201,v1:172.21.15.139:6825/2404740201] conn(0x7f55b00ac280 0x7f55b00ae6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:06:56.849 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:56.849+0000 7f55c9f17700 1 --2- 172.21.15.49:0/4214914267 >> [v2:172.21.15.139:6824/2404740201,v1:172.21.15.139:6825/2404740201] conn(0x7f55b00ac280 0x7f55b00ae6a0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:06:56.849 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:56.849+0000 7f55c8f15700 1 --2- 172.21.15.49:0/4214914267 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f55b00a2aa0 0x7f55b00a4f60 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f55c41060f0 tx=0x7f55b400a380 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:06:56.849 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:56.849+0000 7f55ba7fc700 1 -- 172.21.15.49:0/4214914267 <== osd.10 v2:172.21.15.139:6824/2404740201 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f55c0012ec0 con 0x7f55b00ac280 2024-03-18T23:06:56.876 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:56.875+0000 7f55cb97a700 1 -- 172.21.15.49:0/4214914267 --> [v2:172.21.15.139:6824/2404740201,v1:172.21.15.139:6825/2404740201] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f55a8002e60 con 0x7f55b00ac280 2024-03-18T23:06:56.876 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:56.876+0000 7f55ba7fc700 1 -- 172.21.15.49:0/4214914267 <== osd.10 v2:172.21.15.139:6824/2404740201 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f55a8002e60 con 0x7f55b00ac280 2024-03-18T23:06:56.877 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:56.876+0000 7f55cb97a700 1 -- 172.21.15.49:0/4214914267 >> [v2:172.21.15.139:6824/2404740201,v1:172.21.15.139:6825/2404740201] conn(0x7f55b00ac280 msgr2=0x7f55b00ae6a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:56.877 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:56.876+0000 7f55cb97a700 1 --2- 172.21.15.49:0/4214914267 >> [v2:172.21.15.139:6824/2404740201,v1:172.21.15.139:6825/2404740201] conn(0x7f55b00ac280 0x7f55b00ae6a0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:56.877 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:56.876+0000 7f55cb97a700 1 -- 172.21.15.49:0/4214914267 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f55b00a2aa0 msgr2=0x7f55b00a4f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:56.877 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:56.876+0000 7f55cb97a700 1 --2- 172.21.15.49:0/4214914267 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f55b00a2aa0 0x7f55b00a4f60 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f55c41060f0 tx=0x7f55b400a380 comp rx=0 tx=0).stop 2024-03-18T23:06:56.877 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:56.876+0000 7f55cb97a700 1 -- 172.21.15.49:0/4214914267 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f55c40746b0 msgr2=0x7f55c419efc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:56.877 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:56.876+0000 7f55cb97a700 1 --2- 172.21.15.49:0/4214914267 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f55c40746b0 0x7f55c419efc0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f55c000c9b0 tx=0x7f55c000ccc0 comp rx=0 tx=0).stop 2024-03-18T23:06:56.877 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:56.876+0000 7f55c9f17700 1 -- 172.21.15.49:0/4214914267 reap_dead start 2024-03-18T23:06:56.877 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:56.876+0000 7f55cb97a700 1 -- 172.21.15.49:0/4214914267 shutdown_connections 2024-03-18T23:06:56.877 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:56.876+0000 7f55cb97a700 1 -- 172.21.15.49:0/4214914267 >> 172.21.15.49:0/4214914267 conn(0x7f55c40fdde0 msgr2=0x7f55c40fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:06:56.877 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:56.876+0000 7f55cb97a700 1 -- 172.21.15.49:0/4214914267 shutdown_connections 2024-03-18T23:06:56.877 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:06:56.876+0000 7f55cb97a700 1 -- 172.21.15.49:0/4214914267 wait complete. 2024-03-18T23:06:57.630 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:06:57 smithi049 ceph-mon[62991]: pgmap v314: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-18T23:06:57.711 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:06:57 smithi175 ceph-mon[67352]: pgmap v314: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-18T23:06:57.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:06:57 smithi139 ceph-mon[67282]: pgmap v314: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-18T23:06:59.630 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:06:59 smithi049 ceph-mon[62991]: pgmap v315: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-03-18T23:06:59.711 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:06:59 smithi175 ceph-mon[67352]: pgmap v315: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-03-18T23:06:59.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:06:59 smithi139 ceph-mon[67282]: pgmap v315: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-03-18T23:07:00.584 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:07:00 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]: dispatch 2024-03-18T23:07:00.584 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:07:00 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 4]}]: dispatch 2024-03-18T23:07:00.584 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:07:00 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 1]}]: dispatch 2024-03-18T23:07:00.584 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:07:00 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]: dispatch 2024-03-18T23:07:00.585 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:07:00 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 6]}]: dispatch 2024-03-18T23:07:00.585 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:07:00 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [5, 9]}]: dispatch 2024-03-18T23:07:00.585 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:07:00 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]: dispatch 2024-03-18T23:07:00.585 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:07:00 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]: dispatch 2024-03-18T23:07:00.585 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:07:00 smithi049 ceph-mon[62991]: from='client.? 172.21.15.49:0/2982872954' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-03-18T23:07:00.711 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:07:00 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]: dispatch 2024-03-18T23:07:00.711 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:07:00 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 4]}]: dispatch 2024-03-18T23:07:00.712 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:07:00 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 1]}]: dispatch 2024-03-18T23:07:00.712 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:07:00 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]: dispatch 2024-03-18T23:07:00.712 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:07:00 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 6]}]: dispatch 2024-03-18T23:07:00.712 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:07:00 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [5, 9]}]: dispatch 2024-03-18T23:07:00.712 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:07:00 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]: dispatch 2024-03-18T23:07:00.712 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:07:00 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]: dispatch 2024-03-18T23:07:00.712 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:07:00 smithi175 ceph-mon[67352]: from='client.? 172.21.15.49:0/2982872954' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-03-18T23:07:00.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:07:00 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]: dispatch 2024-03-18T23:07:00.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:07:00 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 4]}]: dispatch 2024-03-18T23:07:00.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:07:00 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 1]}]: dispatch 2024-03-18T23:07:00.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:07:00 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]: dispatch 2024-03-18T23:07:00.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:07:00 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 6]}]: dispatch 2024-03-18T23:07:00.816 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:07:00 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [5, 9]}]: dispatch 2024-03-18T23:07:00.816 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:07:00 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]: dispatch 2024-03-18T23:07:00.816 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:07:00 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]: dispatch 2024-03-18T23:07:00.816 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:07:00 smithi139 ceph-mon[67282]: from='client.? 172.21.15.49:0/2982872954' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-03-18T23:07:01.253 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:01.252+0000 7f55334e7700 1 -- 172.21.15.49:0/405582866 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f552c101e80 msgr2=0x7f552c105e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:01.253 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:01.252+0000 7f55334e7700 1 --2- 172.21.15.49:0/405582866 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f552c101e80 0x7f552c105e40 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f551c0099c0 tx=0x7f551c009cd0 comp rx=0 tx=0).stop 2024-03-18T23:07:01.253 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:01.252+0000 7f55334e7700 1 -- 172.21.15.49:0/405582866 shutdown_connections 2024-03-18T23:07:01.253 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:01.252+0000 7f55334e7700 1 --2- 172.21.15.49:0/405582866 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f552c106520 0x7f552c10e0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:01.253 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:01.252+0000 7f55334e7700 1 --2- 172.21.15.49:0/405582866 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f552c1014d0 0x7f552c1018b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:01.253 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:01.252+0000 7f55334e7700 1 --2- 172.21.15.49:0/405582866 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f552c101e80 0x7f552c105e40 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:01.254 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:01.252+0000 7f55334e7700 1 -- 172.21.15.49:0/405582866 >> 172.21.15.49:0/405582866 conn(0x7f552c077260 msgr2=0x7f552c077670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:07:01.254 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:01.252+0000 7f55334e7700 1 -- 172.21.15.49:0/405582866 shutdown_connections 2024-03-18T23:07:01.254 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:01.252+0000 7f55334e7700 1 -- 172.21.15.49:0/405582866 wait complete. 2024-03-18T23:07:01.255 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:01.253+0000 7f55334e7700 1 Processor -- start 2024-03-18T23:07:01.255 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:01.253+0000 7f55334e7700 1 -- start start 2024-03-18T23:07:01.256 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:01.253+0000 7f55334e7700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f552c1014d0 0x7f552c1998a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:01.256 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:01.253+0000 7f55334e7700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f552c101e80 0x7f552c199de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:01.256 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:01.253+0000 7f55334e7700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f552c106520 0x7f552c1a4920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:01.256 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:01.253+0000 7f55334e7700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f552c111500 con 0x7f552c106520 2024-03-18T23:07:01.256 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:01.253+0000 7f55334e7700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f552c111380 con 0x7f552c1014d0 2024-03-18T23:07:01.256 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:01.253+0000 7f55334e7700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f552c111680 con 0x7f552c101e80 2024-03-18T23:07:01.256 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:01.253+0000 7f5531a84700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f552c106520 0x7f552c1a4920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:01.256 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:01.253+0000 7f5530a82700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f552c101e80 0x7f552c199de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:01.256 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:01.253+0000 7f5531a84700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f552c106520 0x7f552c1a4920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:32768/0 (socket says 172.21.15.49:32768) 2024-03-18T23:07:01.256 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:01.253+0000 7f5531a84700 1 -- 172.21.15.49:0/1255551593 learned_addr learned my addr 172.21.15.49:0/1255551593 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:07:01.256 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:01.253+0000 7f5531283700 1 --2- 172.21.15.49:0/1255551593 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f552c1014d0 0x7f552c1998a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:01.256 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:01.253+0000 7f5530a82700 1 -- 172.21.15.49:0/1255551593 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f552c1014d0 msgr2=0x7f552c1998a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:01.256 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:01.253+0000 7f5530a82700 1 --2- 172.21.15.49:0/1255551593 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f552c1014d0 0x7f552c1998a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:01.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:01.253+0000 7f5530a82700 1 -- 172.21.15.49:0/1255551593 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f552c106520 msgr2=0x7f552c1a4920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:01.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:01.253+0000 7f5530a82700 1 --2- 172.21.15.49:0/1255551593 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f552c106520 0x7f552c1a4920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:01.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:01.253+0000 7f5530a82700 1 -- 172.21.15.49:0/1255551593 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f552c1a4e60 con 0x7f552c101e80 2024-03-18T23:07:01.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:01.253+0000 7f5531a84700 1 --2- 172.21.15.49:0/1255551593 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f552c106520 0x7f552c1a4920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:07:01.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:01.253+0000 7f5531283700 1 --2- 172.21.15.49:0/1255551593 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f552c1014d0 0x7f552c1998a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:07:01.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:01.254+0000 7f5530a82700 1 --2- 172.21.15.49:0/1255551593 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f552c101e80 0x7f552c199de0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f551c0098e0 tx=0x7f551c005f10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:07:01.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:01.254+0000 7f55227fc700 1 -- 172.21.15.49:0/1255551593 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f551c027070 con 0x7f552c101e80 2024-03-18T23:07:01.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:01.254+0000 7f55334e7700 1 -- 172.21.15.49:0/1255551593 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f552c1a5090 con 0x7f552c101e80 2024-03-18T23:07:01.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:01.254+0000 7f55227fc700 1 -- 172.21.15.49:0/1255551593 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f551c003680 con 0x7f552c101e80 2024-03-18T23:07:01.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:01.254+0000 7f55227fc700 1 -- 172.21.15.49:0/1255551593 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f551c022de0 con 0x7f552c101e80 2024-03-18T23:07:01.258 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:01.254+0000 7f55334e7700 1 -- 172.21.15.49:0/1255551593 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f552c1a5540 con 0x7f552c101e80 2024-03-18T23:07:01.258 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:01.255+0000 7f55334e7700 1 -- 172.21.15.49:0/1255551593 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f552c068fd0 con 0x7f552c101e80 2024-03-18T23:07:01.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:01.256+0000 7f55227fc700 1 -- 172.21.15.49:0/1255551593 <== mon.1 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 17) ==== 137341+0+0 (secure 0 0 0) 0x7f551c02a070 con 0x7f552c101e80 2024-03-18T23:07:01.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:01.257+0000 7f55227fc700 1 --2- 172.21.15.49:0/1255551593 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f55180a2920 0x7f55180a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:01.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:01.257+0000 7f5531283700 1 --2- 172.21.15.49:0/1255551593 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f55180a2920 0x7f55180a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:01.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:01.257+0000 7f55227fc700 1 -- 172.21.15.49:0/1255551593 <== mon.1 v2:172.21.15.175:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f551c0cfa50 con 0x7f552c101e80 2024-03-18T23:07:01.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:01.257+0000 7f5531283700 1 --2- 172.21.15.49:0/1255551593 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f55180a2920 0x7f55180a4de0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f5528011490 tx=0x7f5528009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:07:01.265 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:01.265+0000 7f55227fc700 1 -- 172.21.15.49:0/1255551593 <== mon.1 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f551c013070 con 0x7f552c101e80 2024-03-18T23:07:01.478 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:01.477+0000 7f55334e7700 1 -- 172.21.15.49:0/1255551593 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f552c02dbb0 con 0x7f552c101e80 2024-03-18T23:07:01.478 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:01.477+0000 7f55227fc700 1 -- 172.21.15.49:0/1255551593 <== mon.1 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f551c09ca30 con 0x7f552c101e80 2024-03-18T23:07:01.479 INFO:teuthology.orchestra.run.smithi049.stdout:141733920828 2024-03-18T23:07:01.480 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:01.479+0000 7f55334e7700 1 -- 172.21.15.49:0/1255551593 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f55180a2920 msgr2=0x7f55180a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:01.480 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:01.479+0000 7f55334e7700 1 --2- 172.21.15.49:0/1255551593 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f55180a2920 0x7f55180a4de0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f5528011490 tx=0x7f5528009260 comp rx=0 tx=0).stop 2024-03-18T23:07:01.481 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:01.479+0000 7f55334e7700 1 -- 172.21.15.49:0/1255551593 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f552c101e80 msgr2=0x7f552c199de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:01.481 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:01.479+0000 7f55334e7700 1 --2- 172.21.15.49:0/1255551593 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f552c101e80 0x7f552c199de0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f551c0098e0 tx=0x7f551c005f10 comp rx=0 tx=0).stop 2024-03-18T23:07:01.481 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:01.479+0000 7f55334e7700 1 -- 172.21.15.49:0/1255551593 shutdown_connections 2024-03-18T23:07:01.481 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:01.479+0000 7f55334e7700 1 --2- 172.21.15.49:0/1255551593 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f552c106520 0x7f552c1a4920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:01.481 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:01.479+0000 7f55334e7700 1 --2- 172.21.15.49:0/1255551593 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f552c1014d0 0x7f552c1998a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:01.481 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:01.479+0000 7f55334e7700 1 --2- 172.21.15.49:0/1255551593 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f55180a2920 0x7f55180a4de0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:01.481 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:01.479+0000 7f55334e7700 1 --2- 172.21.15.49:0/1255551593 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f552c101e80 0x7f552c199de0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:01.481 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:01.479+0000 7f55334e7700 1 -- 172.21.15.49:0/1255551593 >> 172.21.15.49:0/1255551593 conn(0x7f552c077260 msgr2=0x7f552c0ff280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:07:01.481 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:01.480+0000 7f55334e7700 1 -- 172.21.15.49:0/1255551593 shutdown_connections 2024-03-18T23:07:01.481 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:01.480+0000 7f55334e7700 1 -- 172.21.15.49:0/1255551593 wait complete. 2024-03-18T23:07:01.630 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:07:01 smithi049 ceph-mon[62991]: pgmap v316: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-18T23:07:01.630 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:07:01 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]': finished 2024-03-18T23:07:01.630 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:07:01 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 4]}]': finished 2024-03-18T23:07:01.630 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:07:01 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 1]}]': finished 2024-03-18T23:07:01.630 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:07:01 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]': finished 2024-03-18T23:07:01.630 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:07:01 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 6]}]': finished 2024-03-18T23:07:01.630 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:07:01 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [5, 9]}]': finished 2024-03-18T23:07:01.630 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:07:01 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]': finished 2024-03-18T23:07:01.630 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:07:01 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]': finished 2024-03-18T23:07:01.631 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:07:01 smithi049 ceph-mon[62991]: osdmap e87: 12 total, 12 up, 12 in 2024-03-18T23:07:01.711 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:07:01 smithi175 ceph-mon[67352]: pgmap v316: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-18T23:07:01.711 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:07:01 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]': finished 2024-03-18T23:07:01.712 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:07:01 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 4]}]': finished 2024-03-18T23:07:01.712 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:07:01 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 1]}]': finished 2024-03-18T23:07:01.712 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:07:01 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]': finished 2024-03-18T23:07:01.712 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:07:01 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 6]}]': finished 2024-03-18T23:07:01.712 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:07:01 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [5, 9]}]': finished 2024-03-18T23:07:01.712 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:07:01 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]': finished 2024-03-18T23:07:01.712 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:07:01 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]': finished 2024-03-18T23:07:01.712 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:07:01 smithi175 ceph-mon[67352]: osdmap e87: 12 total, 12 up, 12 in 2024-03-18T23:07:01.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:07:01 smithi139 ceph-mon[67282]: pgmap v316: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-18T23:07:01.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:07:01 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]': finished 2024-03-18T23:07:01.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:07:01 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 4]}]': finished 2024-03-18T23:07:01.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:07:01 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 1]}]': finished 2024-03-18T23:07:01.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:07:01 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]': finished 2024-03-18T23:07:01.816 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:07:01 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 6]}]': finished 2024-03-18T23:07:01.816 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:07:01 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [5, 9]}]': finished 2024-03-18T23:07:01.816 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:07:01 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]': finished 2024-03-18T23:07:01.816 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:07:01 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]': finished 2024-03-18T23:07:01.816 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:07:01 smithi139 ceph-mon[67282]: osdmap e87: 12 total, 12 up, 12 in 2024-03-18T23:07:02.630 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:07:02 smithi049 ceph-mon[62991]: osdmap e88: 12 total, 12 up, 12 in 2024-03-18T23:07:02.630 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:07:02 smithi049 ceph-mon[62991]: from='client.? 172.21.15.49:0/1255551593' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-03-18T23:07:02.711 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:07:02 smithi175 ceph-mon[67352]: osdmap e88: 12 total, 12 up, 12 in 2024-03-18T23:07:02.711 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:07:02 smithi175 ceph-mon[67352]: from='client.? 172.21.15.49:0/1255551593' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-03-18T23:07:02.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:07:02 smithi139 ceph-mon[67282]: osdmap e88: 12 total, 12 up, 12 in 2024-03-18T23:07:02.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:07:02 smithi139 ceph-mon[67282]: from='client.? 172.21.15.49:0/1255551593' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-03-18T23:07:03.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:00.000+0000 7f6e243d4700 1 -- 172.21.15.49:0/1385516948 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6e1c110ee0 msgr2=0x7f6e1c109850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:03.318 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:00.000+0000 7f6e243d4700 1 --2- 172.21.15.49:0/1385516948 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6e1c110ee0 0x7f6e1c109850 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f6e1400b0a0 tx=0x7f6e1400b3b0 comp rx=0 tx=0).stop 2024-03-18T23:07:03.318 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:00.001+0000 7f6e243d4700 1 -- 172.21.15.49:0/1385516948 shutdown_connections 2024-03-18T23:07:03.318 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:00.001+0000 7f6e243d4700 1 --2- 172.21.15.49:0/1385516948 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6e1c11baf0 0x7f6e1c11dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:03.318 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:00.001+0000 7f6e243d4700 1 --2- 172.21.15.49:0/1385516948 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6e1c110a70 0x7f6e1c109310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:03.318 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:00.001+0000 7f6e243d4700 1 --2- 172.21.15.49:0/1385516948 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6e1c110ee0 0x7f6e1c109850 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:03.318 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:00.001+0000 7f6e243d4700 1 -- 172.21.15.49:0/1385516948 >> 172.21.15.49:0/1385516948 conn(0x7f6e1c06d450 msgr2=0x7f6e1c06d860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:07:03.318 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:00.001+0000 7f6e243d4700 1 -- 172.21.15.49:0/1385516948 shutdown_connections 2024-03-18T23:07:03.318 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:00.001+0000 7f6e243d4700 1 -- 172.21.15.49:0/1385516948 wait complete. 2024-03-18T23:07:03.318 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:00.001+0000 7f6e243d4700 1 Processor -- start 2024-03-18T23:07:03.318 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:00.001+0000 7f6e243d4700 1 -- start start 2024-03-18T23:07:03.318 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:00.002+0000 7f6e243d4700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6e1c110a70 0x7f6e1c1ad660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:03.319 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:00.002+0000 7f6e243d4700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6e1c11baf0 0x7f6e1c1adba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:03.319 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:00.002+0000 7f6e243d4700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6e1c1a77e0 0x7f6e1c1a7c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:03.319 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:00.002+0000 7f6e243d4700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f6e1c121400 con 0x7f6e1c110a70 2024-03-18T23:07:03.319 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:00.002+0000 7f6e243d4700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f6e1c121280 con 0x7f6e1c11baf0 2024-03-18T23:07:03.319 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:00.002+0000 7f6e243d4700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f6e1c121580 con 0x7f6e1c1a77e0 2024-03-18T23:07:03.319 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:00.002+0000 7f6e22170700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6e1c110a70 0x7f6e1c1ad660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:03.319 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:00.002+0000 7f6e22170700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6e1c110a70 0x7f6e1c1ad660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:60980/0 (socket says 172.21.15.49:60980) 2024-03-18T23:07:03.319 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:00.002+0000 7f6e22170700 1 -- 172.21.15.49:0/2982872954 learned_addr learned my addr 172.21.15.49:0/2982872954 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:07:03.319 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:00.002+0000 7f6e2196f700 1 --2- 172.21.15.49:0/2982872954 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6e1c11baf0 0x7f6e1c1adba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:03.319 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:00.002+0000 7f6e22971700 1 --2- 172.21.15.49:0/2982872954 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6e1c1a77e0 0x7f6e1c1a7c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:03.319 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:00.002+0000 7f6e22170700 1 -- 172.21.15.49:0/2982872954 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6e1c1a77e0 msgr2=0x7f6e1c1a7c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:03.319 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:00.002+0000 7f6e22170700 1 --2- 172.21.15.49:0/2982872954 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6e1c1a77e0 0x7f6e1c1a7c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:03.319 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:00.002+0000 7f6e22170700 1 -- 172.21.15.49:0/2982872954 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6e1c11baf0 msgr2=0x7f6e1c1adba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:03.319 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:00.002+0000 7f6e22170700 1 --2- 172.21.15.49:0/2982872954 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6e1c11baf0 0x7f6e1c1adba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:03.320 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:00.002+0000 7f6e22170700 1 -- 172.21.15.49:0/2982872954 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e1c1a83a0 con 0x7f6e1c110a70 2024-03-18T23:07:03.320 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:00.002+0000 7f6e2196f700 1 --2- 172.21.15.49:0/2982872954 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6e1c11baf0 0x7f6e1c1adba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:07:03.320 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:00.002+0000 7f6e22170700 1 --2- 172.21.15.49:0/2982872954 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6e1c110a70 0x7f6e1c1ad660 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f6e1800e9a0 tx=0x7f6e1800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:07:03.320 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:00.003+0000 7f6e137fe700 1 -- 172.21.15.49:0/2982872954 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e1800c9c0 con 0x7f6e1c110a70 2024-03-18T23:07:03.320 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:00.003+0000 7f6e137fe700 1 -- 172.21.15.49:0/2982872954 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6e1800cb60 con 0x7f6e1c110a70 2024-03-18T23:07:03.320 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:00.003+0000 7f6e243d4700 1 -- 172.21.15.49:0/2982872954 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e1c1b44c0 con 0x7f6e1c110a70 2024-03-18T23:07:03.320 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:00.003+0000 7f6e137fe700 1 -- 172.21.15.49:0/2982872954 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e18018660 con 0x7f6e1c110a70 2024-03-18T23:07:03.320 INFO:teuthology.orchestra.run.smithi049.stdout:55834574922 2024-03-18T23:07:03.320 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:00.003+0000 7f6e243d4700 1 -- 172.21.15.49:0/2982872954 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e1c1b48b0 con 0x7f6e1c110a70 2024-03-18T23:07:03.321 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:00.004+0000 7f6e243d4700 1 -- 172.21.15.49:0/2982872954 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e1c06e9b0 con 0x7f6e1c110a70 2024-03-18T23:07:03.321 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:00.005+0000 7f6e137fe700 1 -- 172.21.15.49:0/2982872954 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 17) ==== 137341+0+0 (secure 0 0 0) 0x7f6e1800cd10 con 0x7f6e1c110a70 2024-03-18T23:07:03.321 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:00.006+0000 7f6e137fe700 1 --2- 172.21.15.49:0/2982872954 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f6e080a29a0 0x7f6e080a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:03.321 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:00.006+0000 7f6e2196f700 1 --2- 172.21.15.49:0/2982872954 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f6e080a29a0 0x7f6e080a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:03.321 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:00.006+0000 7f6e137fe700 1 -- 172.21.15.49:0/2982872954 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f6e180cec60 con 0x7f6e1c110a70 2024-03-18T23:07:03.321 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:00.007+0000 7f6e2196f700 1 --2- 172.21.15.49:0/2982872954 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f6e080a29a0 0x7f6e080a4e60 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f6e1c11dfc0 tx=0x7f6e1400b700 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:07:03.321 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:00.010+0000 7f6e137fe700 1 -- 172.21.15.49:0/2982872954 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f6e1809bc50 con 0x7f6e1c110a70 2024-03-18T23:07:03.321 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:00.229+0000 7f6e243d4700 1 -- 172.21.15.49:0/2982872954 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f6e1c110ee0 con 0x7f6e1c110a70 2024-03-18T23:07:03.321 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:00.229+0000 7f6e137fe700 1 -- 172.21.15.49:0/2982872954 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f6e1809b550 con 0x7f6e1c110a70 2024-03-18T23:07:03.321 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:00.231+0000 7f6e243d4700 1 -- 172.21.15.49:0/2982872954 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f6e080a29a0 msgr2=0x7f6e080a4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:03.321 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:00.231+0000 7f6e243d4700 1 --2- 172.21.15.49:0/2982872954 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f6e080a29a0 0x7f6e080a4e60 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f6e1c11dfc0 tx=0x7f6e1400b700 comp rx=0 tx=0).stop 2024-03-18T23:07:03.321 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:00.231+0000 7f6e243d4700 1 -- 172.21.15.49:0/2982872954 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6e1c110a70 msgr2=0x7f6e1c1ad660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:03.321 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:00.231+0000 7f6e243d4700 1 --2- 172.21.15.49:0/2982872954 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6e1c110a70 0x7f6e1c1ad660 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f6e1800e9a0 tx=0x7f6e1800ecb0 comp rx=0 tx=0).stop 2024-03-18T23:07:03.321 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:00.231+0000 7f6e243d4700 1 -- 172.21.15.49:0/2982872954 shutdown_connections 2024-03-18T23:07:03.321 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:00.231+0000 7f6e243d4700 1 --2- 172.21.15.49:0/2982872954 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f6e080a29a0 0x7f6e080a4e60 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:03.321 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:00.231+0000 7f6e243d4700 1 --2- 172.21.15.49:0/2982872954 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6e1c1a77e0 0x7f6e1c1a7c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:03.322 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:00.231+0000 7f6e243d4700 1 --2- 172.21.15.49:0/2982872954 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6e1c110a70 0x7f6e1c1ad660 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:03.322 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:00.231+0000 7f6e243d4700 1 --2- 172.21.15.49:0/2982872954 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6e1c11baf0 0x7f6e1c1adba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:03.322 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:00.231+0000 7f6e243d4700 1 -- 172.21.15.49:0/2982872954 >> 172.21.15.49:0/2982872954 conn(0x7f6e1c06d450 msgr2=0x7f6e1c119ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:07:03.322 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:00.231+0000 7f6e243d4700 1 -- 172.21.15.49:0/2982872954 shutdown_connections 2024-03-18T23:07:03.322 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:00.231+0000 7f6e243d4700 1 -- 172.21.15.49:0/2982872954 wait complete. 2024-03-18T23:07:03.630 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:07:03 smithi049 ceph-mon[62991]: pgmap v319: 97 pgs: 2 peering, 95 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 10 op/s 2024-03-18T23:07:03.711 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:07:03 smithi175 ceph-mon[67352]: pgmap v319: 97 pgs: 2 peering, 95 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 10 op/s 2024-03-18T23:07:03.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:07:03 smithi139 ceph-mon[67282]: pgmap v319: 97 pgs: 2 peering, 95 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 10 op/s 2024-03-18T23:07:04.251 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T23:07:04.255 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:04.254+0000 7fee085b7700 1 -- 172.21.15.49:0/2595390337 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fee00101040 msgr2=0x7fee0010a410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:04.255 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:04.254+0000 7fee085b7700 1 --2- 172.21.15.49:0/2595390337 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fee00101040 0x7fee0010a410 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fedf0009920 tx=0x7fedf0009c30 comp rx=0 tx=0).stop 2024-03-18T23:07:04.255 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:04.254+0000 7fee085b7700 1 -- 172.21.15.49:0/2595390337 shutdown_connections 2024-03-18T23:07:04.256 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:04.254+0000 7fee085b7700 1 --2- 172.21.15.49:0/2595390337 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fee0010aaf0 0x7fee001114c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:04.256 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:04.254+0000 7fee085b7700 1 --2- 172.21.15.49:0/2595390337 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fee00100690 0x7fee00100a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:04.256 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:04.254+0000 7fee085b7700 1 --2- 172.21.15.49:0/2595390337 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fee00101040 0x7fee0010a410 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:04.256 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:04.254+0000 7fee085b7700 1 -- 172.21.15.49:0/2595390337 >> 172.21.15.49:0/2595390337 conn(0x7fee000773e0 msgr2=0x7fee000fe760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:07:04.256 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:04.255+0000 7fee085b7700 1 -- 172.21.15.49:0/2595390337 shutdown_connections 2024-03-18T23:07:04.256 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:04.255+0000 7fee085b7700 1 -- 172.21.15.49:0/2595390337 wait complete. 2024-03-18T23:07:04.258 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:04.255+0000 7fee085b7700 1 Processor -- start 2024-03-18T23:07:04.258 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:04.255+0000 7fee085b7700 1 -- start start 2024-03-18T23:07:04.258 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:04.256+0000 7fee085b7700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fee00100690 0x7fee0019ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:04.258 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:04.256+0000 7fee085b7700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fee00101040 0x7fee0019b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:04.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:04.256+0000 7fee085b7700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fee0010aaf0 0x7fee0019f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:04.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:04.256+0000 7fee085b7700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fee00114810 con 0x7fee00101040 2024-03-18T23:07:04.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:04.256+0000 7fee085b7700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fee00114690 con 0x7fee00100690 2024-03-18T23:07:04.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:04.256+0000 7fee085b7700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fee00114990 con 0x7fee0010aaf0 2024-03-18T23:07:04.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:04.256+0000 7fee06353700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fee00100690 0x7fee0019ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:04.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:04.256+0000 7fee06353700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fee00100690 0x7fee0019ad60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.49:60562/0 (socket says 172.21.15.49:60562) 2024-03-18T23:07:04.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:04.256+0000 7fee06353700 1 -- 172.21.15.49:0/2646314802 learned_addr learned my addr 172.21.15.49:0/2646314802 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:07:04.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:04.256+0000 7fee05b52700 1 --2- 172.21.15.49:0/2646314802 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fee00101040 0x7fee0019b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:04.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:04.256+0000 7fee06b54700 1 --2- 172.21.15.49:0/2646314802 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fee0010aaf0 0x7fee0019f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:04.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:04.256+0000 7fee06353700 1 -- 172.21.15.49:0/2646314802 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fee0010aaf0 msgr2=0x7fee0019f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:04.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:04.257+0000 7fee06353700 1 --2- 172.21.15.49:0/2646314802 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fee0010aaf0 0x7fee0019f5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:04.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:04.257+0000 7fee06353700 1 -- 172.21.15.49:0/2646314802 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fee00101040 msgr2=0x7fee0019b2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:04.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:04.257+0000 7fee06353700 1 --2- 172.21.15.49:0/2646314802 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fee00101040 0x7fee0019b2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:04.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:04.257+0000 7fee06353700 1 -- 172.21.15.49:0/2646314802 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee0019fd20 con 0x7fee00100690 2024-03-18T23:07:04.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:04.257+0000 7fee05b52700 1 --2- 172.21.15.49:0/2646314802 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fee00101040 0x7fee0019b2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:07:04.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:04.257+0000 7fee06b54700 1 --2- 172.21.15.49:0/2646314802 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fee0010aaf0 0x7fee0019f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:07:04.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:04.257+0000 7fee06353700 1 --2- 172.21.15.49:0/2646314802 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fee00100690 0x7fee0019ad60 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fedfc00ec10 tx=0x7fedfc00ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:07:04.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:04.257+0000 7fedf77fe700 1 -- 172.21.15.49:0/2646314802 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedfc00cc90 con 0x7fee00100690 2024-03-18T23:07:04.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:04.257+0000 7fedf77fe700 1 -- 172.21.15.49:0/2646314802 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fedfc00ce30 con 0x7fee00100690 2024-03-18T23:07:04.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:04.257+0000 7fedf77fe700 1 -- 172.21.15.49:0/2646314802 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedfc010690 con 0x7fee00100690 2024-03-18T23:07:04.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:04.257+0000 7fee085b7700 1 -- 172.21.15.49:0/2646314802 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee001a0010 con 0x7fee00100690 2024-03-18T23:07:04.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:04.257+0000 7fee085b7700 1 -- 172.21.15.49:0/2646314802 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee001a7890 con 0x7fee00100690 2024-03-18T23:07:04.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:04.259+0000 7fee085b7700 1 -- 172.21.15.49:0/2646314802 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee00101800 con 0x7fee00100690 2024-03-18T23:07:04.261 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:04.260+0000 7fedf77fe700 1 -- 172.21.15.49:0/2646314802 <== mon.2 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 17) ==== 137341+0+0 (secure 0 0 0) 0x7fedfc003680 con 0x7fee00100690 2024-03-18T23:07:04.262 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:04.261+0000 7fedf77fe700 1 --2- 172.21.15.49:0/2646314802 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fedec0a29a0 0x7fedec0a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:04.262 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:04.261+0000 7fee05b52700 1 --2- 172.21.15.49:0/2646314802 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fedec0a29a0 0x7fedec0a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:04.262 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:04.261+0000 7fedf77fe700 1 -- 172.21.15.49:0/2646314802 <== mon.2 v2:172.21.15.139:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7fedfc0ceef0 con 0x7fee00100690 2024-03-18T23:07:04.262 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:04.262+0000 7fee05b52700 1 --2- 172.21.15.49:0/2646314802 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fedec0a29a0 0x7fedec0a4e60 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fedf000a010 tx=0x7fedf0019040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:07:04.267 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:04.265+0000 7fedf77fe700 1 -- 172.21.15.49:0/2646314802 <== mon.2 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fedfc09be40 con 0x7fee00100690 2024-03-18T23:07:04.483 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:04.482+0000 7fee085b7700 1 -- 172.21.15.49:0/2646314802 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7fee0002dbb0 con 0x7fee00100690 2024-03-18T23:07:04.484 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:04.483+0000 7fedf77fe700 1 -- 172.21.15.49:0/2646314802 <== mon.2 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fedfc09b740 con 0x7fee00100690 2024-03-18T23:07:04.484 INFO:teuthology.orchestra.run.smithi049.stdout:34359738448 2024-03-18T23:07:04.486 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:04.485+0000 7fee085b7700 1 -- 172.21.15.49:0/2646314802 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fedec0a29a0 msgr2=0x7fedec0a4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:04.486 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:04.485+0000 7fee085b7700 1 --2- 172.21.15.49:0/2646314802 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fedec0a29a0 0x7fedec0a4e60 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fedf000a010 tx=0x7fedf0019040 comp rx=0 tx=0).stop 2024-03-18T23:07:04.486 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:04.485+0000 7fee085b7700 1 -- 172.21.15.49:0/2646314802 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fee00100690 msgr2=0x7fee0019ad60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:04.486 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:04.485+0000 7fee085b7700 1 --2- 172.21.15.49:0/2646314802 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fee00100690 0x7fee0019ad60 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fedfc00ec10 tx=0x7fedfc00ef20 comp rx=0 tx=0).stop 2024-03-18T23:07:04.486 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:04.485+0000 7fee085b7700 1 -- 172.21.15.49:0/2646314802 shutdown_connections 2024-03-18T23:07:04.487 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:04.485+0000 7fee085b7700 1 --2- 172.21.15.49:0/2646314802 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fedec0a29a0 0x7fedec0a4e60 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:04.487 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:04.485+0000 7fee085b7700 1 --2- 172.21.15.49:0/2646314802 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fee0010aaf0 0x7fee0019f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:04.487 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:04.485+0000 7fee085b7700 1 --2- 172.21.15.49:0/2646314802 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fee00100690 0x7fee0019ad60 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:04.487 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:04.485+0000 7fee085b7700 1 --2- 172.21.15.49:0/2646314802 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fee00101040 0x7fee0019b2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:04.487 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:04.485+0000 7fee085b7700 1 -- 172.21.15.49:0/2646314802 >> 172.21.15.49:0/2646314802 conn(0x7fee000773e0 msgr2=0x7fee000fe760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:07:04.487 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:04.485+0000 7fee085b7700 1 -- 172.21.15.49:0/2646314802 shutdown_connections 2024-03-18T23:07:04.487 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:04.485+0000 7fee085b7700 1 -- 172.21.15.49:0/2646314802 wait complete. 2024-03-18T23:07:05.630 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:07:05 smithi049 ceph-mon[62991]: pgmap v320: 97 pgs: 2 peering, 95 active+clean; 641 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s; 72 B/s, 0 objects/s recovering 2024-03-18T23:07:05.630 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:07:05 smithi049 ceph-mon[62991]: from='client.? 172.21.15.49:0/1104799237' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-03-18T23:07:05.630 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:07:05 smithi049 ceph-mon[62991]: from='client.? 172.21.15.49:0/2646314802' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-03-18T23:07:05.630 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:07:05 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:07:05.711 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:07:05 smithi175 ceph-mon[67352]: pgmap v320: 97 pgs: 2 peering, 95 active+clean; 641 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s; 72 B/s, 0 objects/s recovering 2024-03-18T23:07:05.711 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:07:05 smithi175 ceph-mon[67352]: from='client.? 172.21.15.49:0/1104799237' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-03-18T23:07:05.711 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:07:05 smithi175 ceph-mon[67352]: from='client.? 172.21.15.49:0/2646314802' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-03-18T23:07:05.712 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:07:05 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:07:05.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:07:05 smithi139 ceph-mon[67282]: pgmap v320: 97 pgs: 2 peering, 95 active+clean; 641 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s; 72 B/s, 0 objects/s recovering 2024-03-18T23:07:05.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:07:05 smithi139 ceph-mon[67282]: from='client.? 172.21.15.49:0/1104799237' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-03-18T23:07:05.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:07:05 smithi139 ceph-mon[67282]: from='client.? 172.21.15.49:0/2646314802' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-03-18T23:07:05.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:07:05 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:07:05.994 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:04.155+0000 7fb1e4253700 1 -- 172.21.15.49:0/3080293800 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb1dc102390 msgr2=0x7fb1dc102810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:05.994 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:04.155+0000 7fb1e4253700 1 --2- 172.21.15.49:0/3080293800 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb1dc102390 0x7fb1dc102810 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fb1cc009990 tx=0x7fb1cc009ca0 comp rx=0 tx=0).stop 2024-03-18T23:07:05.994 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:04.155+0000 7fb1e4253700 1 -- 172.21.15.49:0/3080293800 shutdown_connections 2024-03-18T23:07:05.994 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:04.155+0000 7fb1e4253700 1 --2- 172.21.15.49:0/3080293800 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb1dc102d50 0x7fb1dc10f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:05.995 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:04.155+0000 7fb1e4253700 1 --2- 172.21.15.49:0/3080293800 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb1dc108470 0x7fb1dc108850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:05.995 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:04.155+0000 7fb1e4253700 1 --2- 172.21.15.49:0/3080293800 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb1dc102390 0x7fb1dc102810 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:05.995 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:04.155+0000 7fb1e4253700 1 -- 172.21.15.49:0/3080293800 >> 172.21.15.49:0/3080293800 conn(0x7fb1dc0793d0 msgr2=0x7fb1dc0797e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:07:05.995 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:04.155+0000 7fb1e4253700 1 -- 172.21.15.49:0/3080293800 shutdown_connections 2024-03-18T23:07:05.995 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:04.155+0000 7fb1e4253700 1 -- 172.21.15.49:0/3080293800 wait complete. 2024-03-18T23:07:05.995 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:04.156+0000 7fb1e4253700 1 Processor -- start 2024-03-18T23:07:05.995 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:04.156+0000 7fb1e4253700 1 -- start start 2024-03-18T23:07:05.995 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:04.156+0000 7fb1e4253700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb1dc102390 0x7fb1dc074610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:05.995 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:04.156+0000 7fb1e4253700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb1dc102d50 0x7fb1dc074b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:05.995 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:04.156+0000 7fb1e4253700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb1dc108470 0x7fb1dc075090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:05.995 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:04.156+0000 7fb1e4253700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fb1dc1125f0 con 0x7fb1dc102d50 2024-03-18T23:07:05.996 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:04.156+0000 7fb1e4253700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fb1dc112470 con 0x7fb1dc108470 2024-03-18T23:07:05.996 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:04.156+0000 7fb1e4253700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb1dc112770 con 0x7fb1dc102390 2024-03-18T23:07:05.996 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:04.156+0000 7fb1e17ee700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb1dc102d50 0x7fb1dc074b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:05.996 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:04.156+0000 7fb1e1fef700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb1dc102390 0x7fb1dc074610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:05.996 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:04.156+0000 7fb1e17ee700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb1dc102d50 0x7fb1dc074b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:32778/0 (socket says 172.21.15.49:32778) 2024-03-18T23:07:05.996 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:04.156+0000 7fb1e17ee700 1 -- 172.21.15.49:0/1104799237 learned_addr learned my addr 172.21.15.49:0/1104799237 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:07:05.996 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:04.156+0000 7fb1e27f0700 1 --2- 172.21.15.49:0/1104799237 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb1dc108470 0x7fb1dc075090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:05.997 INFO:teuthology.orchestra.run.smithi049.stdout:77309411399 2024-03-18T23:07:05.997 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:04.157+0000 7fb1e17ee700 1 -- 172.21.15.49:0/1104799237 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb1dc102390 msgr2=0x7fb1dc074610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:05.998 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:04.157+0000 7fb1e17ee700 1 --2- 172.21.15.49:0/1104799237 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb1dc102390 0x7fb1dc074610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:05.998 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:04.157+0000 7fb1e17ee700 1 -- 172.21.15.49:0/1104799237 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb1dc108470 msgr2=0x7fb1dc075090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:05.998 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:04.157+0000 7fb1e17ee700 1 --2- 172.21.15.49:0/1104799237 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb1dc108470 0x7fb1dc075090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:05.998 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:04.157+0000 7fb1e17ee700 1 -- 172.21.15.49:0/1104799237 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1dc1a6f70 con 0x7fb1dc102d50 2024-03-18T23:07:05.998 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:04.157+0000 7fb1e1fef700 1 --2- 172.21.15.49:0/1104799237 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb1dc102390 0x7fb1dc074610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:07:05.998 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:04.157+0000 7fb1e17ee700 1 --2- 172.21.15.49:0/1104799237 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb1dc102d50 0x7fb1dc074b50 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fb1cc000c00 tx=0x7fb1cc00bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:07:05.998 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:04.157+0000 7fb1d2ffd700 1 -- 172.21.15.49:0/1104799237 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1cc027070 con 0x7fb1dc102d50 2024-03-18T23:07:05.998 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:04.157+0000 7fb1e4253700 1 -- 172.21.15.49:0/1104799237 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1dc1a7200 con 0x7fb1dc102d50 2024-03-18T23:07:05.998 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:04.157+0000 7fb1d2ffd700 1 -- 172.21.15.49:0/1104799237 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb1cc003d70 con 0x7fb1dc102d50 2024-03-18T23:07:05.999 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:04.157+0000 7fb1d2ffd700 1 -- 172.21.15.49:0/1104799237 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1cc00f460 con 0x7fb1dc102d50 2024-03-18T23:07:05.999 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:04.157+0000 7fb1e4253700 1 -- 172.21.15.49:0/1104799237 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1dc1a7730 con 0x7fb1dc102d50 2024-03-18T23:07:05.999 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:04.158+0000 7fb1e4253700 1 -- 172.21.15.49:0/1104799237 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb1dc068fd0 con 0x7fb1dc102d50 2024-03-18T23:07:05.999 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:04.159+0000 7fb1d2ffd700 1 -- 172.21.15.49:0/1104799237 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 17) ==== 137341+0+0 (secure 0 0 0) 0x7fb1cc02a070 con 0x7fb1dc102d50 2024-03-18T23:07:05.999 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:04.160+0000 7fb1d2ffd700 1 --2- 172.21.15.49:0/1104799237 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fb1c80a2920 0x7fb1c80a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:05.999 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:04.160+0000 7fb1e1fef700 1 --2- 172.21.15.49:0/1104799237 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fb1c80a2920 0x7fb1c80a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:05.999 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:04.160+0000 7fb1d2ffd700 1 -- 172.21.15.49:0/1104799237 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7fb1cc0cf980 con 0x7fb1dc102d50 2024-03-18T23:07:05.999 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:04.160+0000 7fb1e1fef700 1 --2- 172.21.15.49:0/1104799237 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fb1c80a2920 0x7fb1c80a4de0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fb1d8009fd0 tx=0x7fb1d8009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:07:05.999 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:04.169+0000 7fb1d2ffd700 1 -- 172.21.15.49:0/1104799237 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fb1cc013070 con 0x7fb1dc102d50 2024-03-18T23:07:06.000 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:04.395+0000 7fb1e4253700 1 -- 172.21.15.49:0/1104799237 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7fb1dc02dbb0 con 0x7fb1dc102d50 2024-03-18T23:07:06.000 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:04.395+0000 7fb1d2ffd700 1 -- 172.21.15.49:0/1104799237 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fb1cc09c960 con 0x7fb1dc102d50 2024-03-18T23:07:06.000 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:04.397+0000 7fb1e4253700 1 -- 172.21.15.49:0/1104799237 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fb1c80a2920 msgr2=0x7fb1c80a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:06.000 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:04.397+0000 7fb1e4253700 1 --2- 172.21.15.49:0/1104799237 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fb1c80a2920 0x7fb1c80a4de0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fb1d8009fd0 tx=0x7fb1d8009260 comp rx=0 tx=0).stop 2024-03-18T23:07:06.000 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:04.397+0000 7fb1e4253700 1 -- 172.21.15.49:0/1104799237 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb1dc102d50 msgr2=0x7fb1dc074b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:06.000 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:04.397+0000 7fb1e4253700 1 --2- 172.21.15.49:0/1104799237 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb1dc102d50 0x7fb1dc074b50 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fb1cc000c00 tx=0x7fb1cc00bab0 comp rx=0 tx=0).stop 2024-03-18T23:07:06.000 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:04.398+0000 7fb1e4253700 1 -- 172.21.15.49:0/1104799237 shutdown_connections 2024-03-18T23:07:06.000 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:04.398+0000 7fb1e4253700 1 --2- 172.21.15.49:0/1104799237 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb1dc108470 0x7fb1dc075090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:06.000 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:04.398+0000 7fb1e4253700 1 --2- 172.21.15.49:0/1104799237 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fb1c80a2920 0x7fb1c80a4de0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:06.000 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:04.398+0000 7fb1e4253700 1 --2- 172.21.15.49:0/1104799237 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb1dc102390 0x7fb1dc074610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:06.001 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:04.398+0000 7fb1e4253700 1 --2- 172.21.15.49:0/1104799237 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb1dc102d50 0x7fb1dc074b50 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:06.001 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:04.398+0000 7fb1e4253700 1 -- 172.21.15.49:0/1104799237 >> 172.21.15.49:0/1104799237 conn(0x7fb1dc0793d0 msgr2=0x7fb1dc100f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:07:06.001 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:04.398+0000 7fb1e4253700 1 -- 172.21.15.49:0/1104799237 shutdown_connections 2024-03-18T23:07:06.001 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:04.398+0000 7fb1e4253700 1 -- 172.21.15.49:0/1104799237 wait complete. 2024-03-18T23:07:06.653 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411394 got 77309411399 for osd.2 2024-03-18T23:07:06.654 DEBUG:teuthology.parallel:result is None 2024-03-18T23:07:07.564 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:07:07 smithi049 ceph-mon[62991]: pgmap v321: 97 pgs: 1 peering, 96 active+clean; 641 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 3 op/s; 267 B/s, 1 objects/s recovering 2024-03-18T23:07:07.564 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:07:07 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:07:07.564 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:07:07 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:07:07.564 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:07:07 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:07:07.564 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:07:07 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:07:07.672 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:07.671+0000 7fb41b6df700 1 -- 172.21.15.49:0/1844827658 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb414102390 msgr2=0x7fb414102810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:07.672 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:07.671+0000 7fb41b6df700 1 --2- 172.21.15.49:0/1844827658 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb414102390 0x7fb414102810 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fb404009990 tx=0x7fb404009ca0 comp rx=0 tx=0).stop 2024-03-18T23:07:07.672 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:07.671+0000 7fb41b6df700 1 -- 172.21.15.49:0/1844827658 shutdown_connections 2024-03-18T23:07:07.672 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:07.671+0000 7fb41b6df700 1 --2- 172.21.15.49:0/1844827658 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb414102d50 0x7fb41410f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:07.673 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:07.671+0000 7fb41b6df700 1 --2- 172.21.15.49:0/1844827658 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb414108470 0x7fb414108850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:07.673 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:07.671+0000 7fb41b6df700 1 --2- 172.21.15.49:0/1844827658 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb414102390 0x7fb414102810 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:07.673 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:07.671+0000 7fb41b6df700 1 -- 172.21.15.49:0/1844827658 >> 172.21.15.49:0/1844827658 conn(0x7fb4140793d0 msgr2=0x7fb4140797e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:07:07.673 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:07.671+0000 7fb41b6df700 1 -- 172.21.15.49:0/1844827658 shutdown_connections 2024-03-18T23:07:07.673 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:07.671+0000 7fb41b6df700 1 -- 172.21.15.49:0/1844827658 wait complete. 2024-03-18T23:07:07.674 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:07.672+0000 7fb41b6df700 1 Processor -- start 2024-03-18T23:07:07.675 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:07.672+0000 7fb41b6df700 1 -- start start 2024-03-18T23:07:07.675 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:07.672+0000 7fb41b6df700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb414102390 0x7fb41419e730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:07.675 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:07.672+0000 7fb41b6df700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb414102d50 0x7fb41419ec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:07.675 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:07.672+0000 7fb41b6df700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb414108470 0x7fb4141988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:07.675 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:07.672+0000 7fb41b6df700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fb4141125d0 con 0x7fb414108470 2024-03-18T23:07:07.675 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:07.672+0000 7fb41b6df700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fb414112450 con 0x7fb414102d50 2024-03-18T23:07:07.675 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:07.672+0000 7fb41b6df700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb414112750 con 0x7fb414102390 2024-03-18T23:07:07.675 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:07.672+0000 7fb419c7c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb414108470 0x7fb4141988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:07.675 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:07.672+0000 7fb41947b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb414102390 0x7fb41419e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:07.675 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:07.672+0000 7fb419c7c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb414108470 0x7fb4141988f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47096/0 (socket says 172.21.15.49:47096) 2024-03-18T23:07:07.675 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:07.672+0000 7fb419c7c700 1 -- 172.21.15.49:0/2875317519 learned_addr learned my addr 172.21.15.49:0/2875317519 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:07:07.675 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:07.672+0000 7fb418c7a700 1 --2- 172.21.15.49:0/2875317519 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb414102d50 0x7fb41419ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:07.675 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:07.673+0000 7fb418c7a700 1 -- 172.21.15.49:0/2875317519 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb414102390 msgr2=0x7fb41419e730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:07.676 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:07.673+0000 7fb418c7a700 1 --2- 172.21.15.49:0/2875317519 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb414102390 0x7fb41419e730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:07.676 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:07.673+0000 7fb418c7a700 1 -- 172.21.15.49:0/2875317519 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb414108470 msgr2=0x7fb4141988f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:07.676 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:07.673+0000 7fb418c7a700 1 --2- 172.21.15.49:0/2875317519 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb414108470 0x7fb4141988f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:07.676 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:07.673+0000 7fb418c7a700 1 -- 172.21.15.49:0/2875317519 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb414199050 con 0x7fb414102d50 2024-03-18T23:07:07.676 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:07.673+0000 7fb41947b700 1 --2- 172.21.15.49:0/2875317519 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb414102390 0x7fb41419e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:07:07.676 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:07.673+0000 7fb418c7a700 1 --2- 172.21.15.49:0/2875317519 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb414102d50 0x7fb41419ec70 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fb404006010 tx=0x7fb404003890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:07:07.676 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:07.673+0000 7fb40a7fc700 1 -- 172.21.15.49:0/2875317519 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb404027070 con 0x7fb414102d50 2024-03-18T23:07:07.676 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:07.673+0000 7fb41b6df700 1 -- 172.21.15.49:0/2875317519 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4141992e0 con 0x7fb414102d50 2024-03-18T23:07:07.676 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:07.673+0000 7fb40a7fc700 1 -- 172.21.15.49:0/2875317519 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb4040056d0 con 0x7fb414102d50 2024-03-18T23:07:07.676 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:07.673+0000 7fb40a7fc700 1 -- 172.21.15.49:0/2875317519 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb404022bd0 con 0x7fb414102d50 2024-03-18T23:07:07.676 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:07.673+0000 7fb41b6df700 1 -- 172.21.15.49:0/2875317519 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4141a5600 con 0x7fb414102d50 2024-03-18T23:07:07.677 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:07.674+0000 7fb41b6df700 1 -- 172.21.15.49:0/2875317519 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb414068fd0 con 0x7fb414102d50 2024-03-18T23:07:07.677 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:07.675+0000 7fb40a7fc700 1 -- 172.21.15.49:0/2875317519 <== mon.2 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 17) ==== 137341+0+0 (secure 0 0 0) 0x7fb40402a070 con 0x7fb414102d50 2024-03-18T23:07:07.677 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:07.676+0000 7fb40a7fc700 1 --2- 172.21.15.49:0/2875317519 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fb4000a2920 0x7fb4000a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:07.677 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:07.676+0000 7fb41947b700 1 --2- 172.21.15.49:0/2875317519 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fb4000a2920 0x7fb4000a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:07.677 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:07.676+0000 7fb40a7fc700 1 -- 172.21.15.49:0/2875317519 <== mon.2 v2:172.21.15.139:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7fb4040cff80 con 0x7fb414102d50 2024-03-18T23:07:07.678 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:07.677+0000 7fb41947b700 1 --2- 172.21.15.49:0/2875317519 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fb4000a2920 0x7fb4000a4de0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fb414199a30 tx=0x7fb4100073d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:07:07.680 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:07.679+0000 7fb40a7fc700 1 -- 172.21.15.49:0/2875317519 <== mon.2 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fb404013070 con 0x7fb414102d50 2024-03-18T23:07:07.711 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:07:07 smithi175 ceph-mon[67352]: pgmap v321: 97 pgs: 1 peering, 96 active+clean; 641 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 3 op/s; 267 B/s, 1 objects/s recovering 2024-03-18T23:07:07.711 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:07:07 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:07:07.711 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:07:07 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:07:07.711 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:07:07 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:07:07.711 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:07:07 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:07:07.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:07:07 smithi139 ceph-mon[67282]: pgmap v321: 97 pgs: 1 peering, 96 active+clean; 641 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 3 op/s; 267 B/s, 1 objects/s recovering 2024-03-18T23:07:07.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:07:07 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:07:07.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:07:07 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:07:07.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:07:07 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:07:07.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:07:07 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:07:07.892 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:07.891+0000 7fb41b6df700 1 -- 172.21.15.49:0/2875317519 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7fb41402dbb0 con 0x7fb414102d50 2024-03-18T23:07:07.892 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:07.891+0000 7fb40a7fc700 1 -- 172.21.15.49:0/2875317519 <== mon.2 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fb40409cf00 con 0x7fb414102d50 2024-03-18T23:07:07.892 INFO:teuthology.orchestra.run.smithi049.stdout:292057776163 2024-03-18T23:07:07.895 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:07.894+0000 7fb41b6df700 1 -- 172.21.15.49:0/2875317519 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fb4000a2920 msgr2=0x7fb4000a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:07.895 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:07.894+0000 7fb41b6df700 1 --2- 172.21.15.49:0/2875317519 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fb4000a2920 0x7fb4000a4de0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fb414199a30 tx=0x7fb4100073d0 comp rx=0 tx=0).stop 2024-03-18T23:07:07.895 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:07.894+0000 7fb41b6df700 1 -- 172.21.15.49:0/2875317519 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb414102d50 msgr2=0x7fb41419ec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:07.895 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:07.894+0000 7fb41b6df700 1 --2- 172.21.15.49:0/2875317519 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb414102d50 0x7fb41419ec70 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fb404006010 tx=0x7fb404003890 comp rx=0 tx=0).stop 2024-03-18T23:07:07.895 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:07.894+0000 7fb41b6df700 1 -- 172.21.15.49:0/2875317519 shutdown_connections 2024-03-18T23:07:07.895 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:07.894+0000 7fb41b6df700 1 --2- 172.21.15.49:0/2875317519 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb414108470 0x7fb4141988f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:07.895 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:07.894+0000 7fb41b6df700 1 --2- 172.21.15.49:0/2875317519 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fb4000a2920 0x7fb4000a4de0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:07.895 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:07.894+0000 7fb41b6df700 1 --2- 172.21.15.49:0/2875317519 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb414102390 0x7fb41419e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:07.895 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:07.894+0000 7fb41b6df700 1 --2- 172.21.15.49:0/2875317519 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb414102d50 0x7fb41419ec70 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:07.895 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:07.894+0000 7fb41b6df700 1 -- 172.21.15.49:0/2875317519 >> 172.21.15.49:0/2875317519 conn(0x7fb4140793d0 msgr2=0x7fb414101060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:07:07.896 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:07.894+0000 7fb41b6df700 1 -- 172.21.15.49:0/2875317519 shutdown_connections 2024-03-18T23:07:07.896 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:07.894+0000 7fb41b6df700 1 -- 172.21.15.49:0/2875317519 wait complete. 2024-03-18T23:07:08.047 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738444 got 34359738448 for osd.0 2024-03-18T23:07:08.047 DEBUG:teuthology.parallel:result is None 2024-03-18T23:07:08.168 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574919 got 55834574922 for osd.1 2024-03-18T23:07:08.168 DEBUG:teuthology.parallel:result is None 2024-03-18T23:07:08.630 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:07:08 smithi049 ceph-mon[62991]: from='client.? 172.21.15.49:0/2875317519' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-03-18T23:07:08.705 INFO:teuthology.orchestra.run.smithi049.stdout:313532612635 2024-03-18T23:07:08.706 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell --fsid c63591ea-e57a-11ee-95c9-87774f69a715 -- ceph osd last-stat-seq osd.10 2024-03-18T23:07:08.711 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:07:08 smithi175 ceph-mon[67352]: from='client.? 172.21.15.49:0/2875317519' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-03-18T23:07:08.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:07:08 smithi139 ceph-mon[67282]: from='client.? 172.21.15.49:0/2875317519' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-03-18T23:07:09.148 INFO:tasks.cephadm.ceph_manager.ceph:need seq 141733920822 got 141733920828 for osd.4 2024-03-18T23:07:09.148 DEBUG:teuthology.parallel:result is None 2024-03-18T23:07:09.369 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:07:09 smithi049 ceph-mon[62991]: pgmap v322: 97 pgs: 97 active+clean; 641 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 3 op/s; 267 B/s, 1 objects/s recovering 2024-03-18T23:07:09.612 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:07:09 smithi175 ceph-mon[67352]: pgmap v322: 97 pgs: 97 active+clean; 641 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 3 op/s; 267 B/s, 1 objects/s recovering 2024-03-18T23:07:09.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:07:09 smithi139 ceph-mon[67282]: pgmap v322: 97 pgs: 97 active+clean; 641 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 3 op/s; 267 B/s, 1 objects/s recovering 2024-03-18T23:07:10.017 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:10.015+0000 7f0ccc92d700 1 -- 172.21.15.49:0/3040902870 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0cc8102f00 msgr2=0x7f0cc8106f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:10.017 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:10.015+0000 7f0ccc92d700 1 --2- 172.21.15.49:0/3040902870 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0cc8102f00 0x7f0cc8106f50 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f0cbc0099c0 tx=0x7f0cbc009cd0 comp rx=0 tx=0).stop 2024-03-18T23:07:10.017 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:10.015+0000 7f0ccc92d700 1 -- 172.21.15.49:0/3040902870 shutdown_connections 2024-03-18T23:07:10.018 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:10.015+0000 7f0ccc92d700 1 --2- 172.21.15.49:0/3040902870 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0cc8107630 0x7f0cc810f200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:10.018 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:10.015+0000 7f0ccc92d700 1 --2- 172.21.15.49:0/3040902870 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0cc8102550 0x7f0cc8102930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:10.018 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:10.015+0000 7f0ccc92d700 1 --2- 172.21.15.49:0/3040902870 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0cc8102f00 0x7f0cc8106f50 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:10.018 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:10.015+0000 7f0ccc92d700 1 -- 172.21.15.49:0/3040902870 >> 172.21.15.49:0/3040902870 conn(0x7f0cc80fddc0 msgr2=0x7f0cc81001e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:07:10.018 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:10.015+0000 7f0ccc92d700 1 -- 172.21.15.49:0/3040902870 shutdown_connections 2024-03-18T23:07:10.018 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:10.015+0000 7f0ccc92d700 1 -- 172.21.15.49:0/3040902870 wait complete. 2024-03-18T23:07:10.018 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:10.016+0000 7f0ccc92d700 1 Processor -- start 2024-03-18T23:07:10.018 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:10.016+0000 7f0ccc92d700 1 -- start start 2024-03-18T23:07:10.018 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:10.016+0000 7f0ccc92d700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0cc8102550 0x7f0cc819e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:10.018 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:10.016+0000 7f0ccc92d700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0cc8102f00 0x7f0cc819ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:10.018 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:10.016+0000 7f0ccc92d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0cc8107630 0x7f0cc81989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:10.018 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:10.016+0000 7f0ccc92d700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f0cc8112660 con 0x7f0cc8107630 2024-03-18T23:07:10.018 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:10.016+0000 7f0ccc92d700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f0cc81124e0 con 0x7f0cc8102f00 2024-03-18T23:07:10.019 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:10.016+0000 7f0ccc92d700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f0cc81127e0 con 0x7f0cc8102550 2024-03-18T23:07:10.019 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:10.016+0000 7f0cc6d9d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0cc8107630 0x7f0cc81989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:10.019 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:10.016+0000 7f0cc6d9d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0cc8107630 0x7f0cc81989a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47112/0 (socket says 172.21.15.49:47112) 2024-03-18T23:07:10.019 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:10.016+0000 7f0cc659c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0cc8102550 0x7f0cc819e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:10.019 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:10.016+0000 7f0cc6d9d700 1 -- 172.21.15.49:0/583421344 learned_addr learned my addr 172.21.15.49:0/583421344 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:07:10.019 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:10.016+0000 7f0cc5d9b700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0cc8102f00 0x7f0cc819ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:10.020 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:10.017+0000 7f0cc6d9d700 1 -- 172.21.15.49:0/583421344 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0cc8102550 msgr2=0x7f0cc819e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:10.020 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:10.017+0000 7f0cc6d9d700 1 --2- 172.21.15.49:0/583421344 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0cc8102550 0x7f0cc819e790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:10.020 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:10.017+0000 7f0cc6d9d700 1 -- 172.21.15.49:0/583421344 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0cc8102f00 msgr2=0x7f0cc819ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:10.021 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:10.017+0000 7f0cc6d9d700 1 --2- 172.21.15.49:0/583421344 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0cc8102f00 0x7f0cc819ecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:10.021 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:10.017+0000 7f0cc6d9d700 1 -- 172.21.15.49:0/583421344 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cc8199100 con 0x7f0cc8107630 2024-03-18T23:07:10.021 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:10.017+0000 7f0cc6d9d700 1 --2- 172.21.15.49:0/583421344 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0cc8107630 0x7f0cc81989a0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f0cc000ef30 tx=0x7f0cc000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:07:10.021 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:10.017+0000 7f0cb77fe700 1 -- 172.21.15.49:0/583421344 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cc00092e0 con 0x7f0cc8107630 2024-03-18T23:07:10.021 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:10.017+0000 7f0cb77fe700 1 -- 172.21.15.49:0/583421344 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0cc0004bb0 con 0x7f0cc8107630 2024-03-18T23:07:10.021 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:10.017+0000 7f0ccc92d700 1 -- 172.21.15.49:0/583421344 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0cc8199390 con 0x7f0cc8107630 2024-03-18T23:07:10.021 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:10.017+0000 7f0cb77fe700 1 -- 172.21.15.49:0/583421344 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cc0004280 con 0x7f0cc8107630 2024-03-18T23:07:10.021 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:10.017+0000 7f0ccc92d700 1 -- 172.21.15.49:0/583421344 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0cc81a5660 con 0x7f0cc8107630 2024-03-18T23:07:10.021 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:10.018+0000 7f0ccc92d700 1 -- 172.21.15.49:0/583421344 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0cc8068fd0 con 0x7f0cc8107630 2024-03-18T23:07:10.021 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:10.019+0000 7f0cb77fe700 1 -- 172.21.15.49:0/583421344 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 17) ==== 137341+0+0 (secure 0 0 0) 0x7f0cc001b030 con 0x7f0cc8107630 2024-03-18T23:07:10.021 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:10.020+0000 7f0cb77fe700 1 --2- 172.21.15.49:0/583421344 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f0cb00a2920 0x7f0cb00a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:10.022 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:10.020+0000 7f0cc659c700 1 --2- 172.21.15.49:0/583421344 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f0cb00a2920 0x7f0cb00a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:10.022 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:10.020+0000 7f0cb77fe700 1 -- 172.21.15.49:0/583421344 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f0cc00cefd0 con 0x7f0cc8107630 2024-03-18T23:07:10.022 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:10.020+0000 7f0cc659c700 1 --2- 172.21.15.49:0/583421344 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f0cb00a2920 0x7f0cb00a4de0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f0cc80fe260 tx=0x7f0cb8009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:07:10.024 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:10.024+0000 7f0cb77fe700 1 -- 172.21.15.49:0/583421344 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f0cc0022020 con 0x7f0cc8107630 2024-03-18T23:07:10.239 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:10.238+0000 7f0ccc92d700 1 -- 172.21.15.49:0/583421344 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f0cc802dbb0 con 0x7f0cc8107630 2024-03-18T23:07:10.239 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:10.239+0000 7f0cb77fe700 1 -- 172.21.15.49:0/583421344 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f0cc0010040 con 0x7f0cc8107630 2024-03-18T23:07:10.240 INFO:teuthology.orchestra.run.smithi049.stdout:219043332140 2024-03-18T23:07:10.242 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:10.241+0000 7f0ccc92d700 1 -- 172.21.15.49:0/583421344 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f0cb00a2920 msgr2=0x7f0cb00a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:10.242 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:10.241+0000 7f0ccc92d700 1 --2- 172.21.15.49:0/583421344 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f0cb00a2920 0x7f0cb00a4de0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f0cc80fe260 tx=0x7f0cb8009260 comp rx=0 tx=0).stop 2024-03-18T23:07:10.243 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:10.241+0000 7f0ccc92d700 1 -- 172.21.15.49:0/583421344 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0cc8107630 msgr2=0x7f0cc81989a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:10.243 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:10.241+0000 7f0ccc92d700 1 --2- 172.21.15.49:0/583421344 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0cc8107630 0x7f0cc81989a0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f0cc000ef30 tx=0x7f0cc000c3a0 comp rx=0 tx=0).stop 2024-03-18T23:07:10.243 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:10.241+0000 7f0ccc92d700 1 -- 172.21.15.49:0/583421344 shutdown_connections 2024-03-18T23:07:10.243 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:10.241+0000 7f0ccc92d700 1 --2- 172.21.15.49:0/583421344 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0cc8107630 0x7f0cc81989a0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:10.243 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:10.241+0000 7f0ccc92d700 1 --2- 172.21.15.49:0/583421344 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f0cb00a2920 0x7f0cb00a4de0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:10.243 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:10.241+0000 7f0ccc92d700 1 --2- 172.21.15.49:0/583421344 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0cc8102550 0x7f0cc819e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:10.243 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:10.241+0000 7f0ccc92d700 1 --2- 172.21.15.49:0/583421344 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0cc8102f00 0x7f0cc819ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:10.243 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:10.241+0000 7f0ccc92d700 1 -- 172.21.15.49:0/583421344 >> 172.21.15.49:0/583421344 conn(0x7f0cc80fddc0 msgr2=0x7f0cc8100090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:07:10.243 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:10.241+0000 7f0ccc92d700 1 -- 172.21.15.49:0/583421344 shutdown_connections 2024-03-18T23:07:10.243 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:10.241+0000 7f0ccc92d700 1 -- 172.21.15.49:0/583421344 wait complete. 2024-03-18T23:07:10.630 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:07:10 smithi049 ceph-mon[62991]: from='client.? 172.21.15.49:0/583421344' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-03-18T23:07:10.651 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:07:10 smithi175 ceph-mon[67352]: from='client.? 172.21.15.49:0/583421344' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-03-18T23:07:10.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:07:10 smithi139 ceph-mon[67282]: from='client.? 172.21.15.49:0/583421344' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-03-18T23:07:11.577 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:07:11 smithi049 ceph-mon[62991]: pgmap v323: 97 pgs: 97 active+clean; 641 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 4 op/s; 223 B/s, 1 objects/s recovering 2024-03-18T23:07:11.609 INFO:tasks.cephadm.ceph_manager.ceph:need seq 292057776158 got 292057776163 for osd.9 2024-03-18T23:07:11.610 DEBUG:teuthology.parallel:result is None 2024-03-18T23:07:11.712 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:07:11 smithi175 ceph-mon[67352]: pgmap v323: 97 pgs: 97 active+clean; 641 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 4 op/s; 223 B/s, 1 objects/s recovering 2024-03-18T23:07:11.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:07:11 smithi139 ceph-mon[67282]: pgmap v323: 97 pgs: 97 active+clean; 641 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 4 op/s; 223 B/s, 1 objects/s recovering 2024-03-18T23:07:12.249 INFO:tasks.cephadm.ceph_manager.ceph:need seq 219043332135 got 219043332140 for osd.7 2024-03-18T23:07:12.249 DEBUG:teuthology.parallel:result is None 2024-03-18T23:07:13.291 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:13.279+0000 7f6cc28ac700 1 -- 172.21.15.49:0/2920344891 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6cbc101210 msgr2=0x7f6cbc101670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:13.291 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:13.279+0000 7f6cc28ac700 1 --2- 172.21.15.49:0/2920344891 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6cbc101210 0x7f6cbc101670 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f6cb0009990 tx=0x7f6cb0009ca0 comp rx=0 tx=0).stop 2024-03-18T23:07:13.291 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:13.279+0000 7f6cc28ac700 1 -- 172.21.15.49:0/2920344891 shutdown_connections 2024-03-18T23:07:13.291 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:13.279+0000 7f6cc28ac700 1 --2- 172.21.15.49:0/2920344891 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6cbc101bb0 0x7f6cbc10e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:13.291 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:13.279+0000 7f6cc28ac700 1 --2- 172.21.15.49:0/2920344891 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6cbc107250 0x7f6cbc107630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:13.291 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:13.279+0000 7f6cc28ac700 1 --2- 172.21.15.49:0/2920344891 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6cbc101210 0x7f6cbc101670 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:13.292 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:13.279+0000 7f6cc28ac700 1 -- 172.21.15.49:0/2920344891 >> 172.21.15.49:0/2920344891 conn(0x7f6cbc078140 msgr2=0x7f6cbc078550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:07:13.292 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:13.279+0000 7f6cc28ac700 1 -- 172.21.15.49:0/2920344891 shutdown_connections 2024-03-18T23:07:13.292 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:13.279+0000 7f6cc28ac700 1 -- 172.21.15.49:0/2920344891 wait complete. 2024-03-18T23:07:13.292 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:13.280+0000 7f6cc28ac700 1 Processor -- start 2024-03-18T23:07:13.292 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:13.280+0000 7f6cc28ac700 1 -- start start 2024-03-18T23:07:13.292 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:13.281+0000 7f6cc28ac700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6cbc101210 0x7f6cbc1998d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:13.292 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:13.281+0000 7f6cc28ac700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6cbc101bb0 0x7f6cbc199e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:13.292 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:13.281+0000 7f6cc28ac700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6cbc107250 0x7f6cbc1a4950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:13.292 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:13.281+0000 7f6cc28ac700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f6cbc1114b0 con 0x7f6cbc107250 2024-03-18T23:07:13.292 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:13.281+0000 7f6cc28ac700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f6cbc111330 con 0x7f6cbc101210 2024-03-18T23:07:13.292 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:13.281+0000 7f6cc28ac700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f6cbc111630 con 0x7f6cbc101bb0 2024-03-18T23:07:13.292 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:13.281+0000 7f6cc0e49700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6cbc107250 0x7f6cbc1a4950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:13.292 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:13.281+0000 7f6cbb7fe700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6cbc101bb0 0x7f6cbc199e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:13.292 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:13.281+0000 7f6cbb7fe700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6cbc101bb0 0x7f6cbc199e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.49:35204/0 (socket says 172.21.15.49:35204) 2024-03-18T23:07:13.293 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:13.281+0000 7f6cbb7fe700 1 -- 172.21.15.49:0/418735770 learned_addr learned my addr 172.21.15.49:0/418735770 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:07:13.293 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:13.281+0000 7f6cbbfff700 1 --2- 172.21.15.49:0/418735770 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6cbc101210 0x7f6cbc1998d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:13.293 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:13.281+0000 7f6cbb7fe700 1 -- 172.21.15.49:0/418735770 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6cbc101210 msgr2=0x7f6cbc1998d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:13.293 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:13.281+0000 7f6cbb7fe700 1 --2- 172.21.15.49:0/418735770 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6cbc101210 0x7f6cbc1998d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:13.293 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:13.281+0000 7f6cbb7fe700 1 -- 172.21.15.49:0/418735770 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6cbc107250 msgr2=0x7f6cbc1a4950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:13.293 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:13.281+0000 7f6cbb7fe700 1 --2- 172.21.15.49:0/418735770 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6cbc107250 0x7f6cbc1a4950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:13.293 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:13.281+0000 7f6cbb7fe700 1 -- 172.21.15.49:0/418735770 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6cbc1a4e90 con 0x7f6cbc101bb0 2024-03-18T23:07:13.293 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:13.281+0000 7f6cbb7fe700 1 --2- 172.21.15.49:0/418735770 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6cbc101bb0 0x7f6cbc199e10 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f6cb0000c00 tx=0x7f6cb000bab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:07:13.293 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:13.282+0000 7f6cb97fa700 1 -- 172.21.15.49:0/418735770 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cb0027070 con 0x7f6cbc101bb0 2024-03-18T23:07:13.293 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:13.282+0000 7f6cb97fa700 1 -- 172.21.15.49:0/418735770 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6cb0003d70 con 0x7f6cbc101bb0 2024-03-18T23:07:13.293 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:13.282+0000 7f6cc28ac700 1 -- 172.21.15.49:0/418735770 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6cbc1a5060 con 0x7f6cbc101bb0 2024-03-18T23:07:13.293 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:13.282+0000 7f6cb97fa700 1 -- 172.21.15.49:0/418735770 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cb000f460 con 0x7f6cbc101bb0 2024-03-18T23:07:13.293 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:13.282+0000 7f6cc28ac700 1 -- 172.21.15.49:0/418735770 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6cbc1a54b0 con 0x7f6cbc101bb0 2024-03-18T23:07:13.294 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:13.283+0000 7f6cc28ac700 1 -- 172.21.15.49:0/418735770 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6cbc068fd0 con 0x7f6cbc101bb0 2024-03-18T23:07:13.294 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:13.284+0000 7f6cb97fa700 1 -- 172.21.15.49:0/418735770 <== mon.1 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 17) ==== 137341+0+0 (secure 0 0 0) 0x7f6cb002a070 con 0x7f6cbc101bb0 2024-03-18T23:07:13.294 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:13.284+0000 7f6cb97fa700 1 --2- 172.21.15.49:0/418735770 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f6ca40a2920 0x7f6ca40a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:13.294 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:13.284+0000 7f6cbbfff700 1 --2- 172.21.15.49:0/418735770 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f6ca40a2920 0x7f6ca40a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:13.294 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:13.284+0000 7f6cb97fa700 1 -- 172.21.15.49:0/418735770 <== mon.1 v2:172.21.15.175:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f6cb00cf960 con 0x7f6cbc101bb0 2024-03-18T23:07:13.294 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:13.285+0000 7f6cbbfff700 1 --2- 172.21.15.49:0/418735770 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f6ca40a2920 0x7f6ca40a4de0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f6cac009be0 tx=0x7f6cac009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:07:13.294 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:13.289+0000 7f6cb97fa700 1 -- 172.21.15.49:0/418735770 <== mon.1 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f6cb0013070 con 0x7f6cbc101bb0 2024-03-18T23:07:13.503 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:13.502+0000 7f6cc28ac700 1 -- 172.21.15.49:0/418735770 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f6cbc02dbb0 con 0x7f6cbc101bb0 2024-03-18T23:07:13.504 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:07:13 smithi049 ceph-mon[62991]: pgmap v324: 97 pgs: 97 active+clean; 641 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 6 op/s; 202 B/s, 0 objects/s recovering 2024-03-18T23:07:13.504 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:13.503+0000 7f6cb97fa700 1 -- 172.21.15.49:0/418735770 <== mon.1 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f6cb009c940 con 0x7f6cbc101bb0 2024-03-18T23:07:13.504 INFO:teuthology.orchestra.run.smithi049.stdout:163208757304 2024-03-18T23:07:13.506 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:13.505+0000 7f6cc28ac700 1 -- 172.21.15.49:0/418735770 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f6ca40a2920 msgr2=0x7f6ca40a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:13.506 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:13.505+0000 7f6cc28ac700 1 --2- 172.21.15.49:0/418735770 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f6ca40a2920 0x7f6ca40a4de0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f6cac009be0 tx=0x7f6cac009260 comp rx=0 tx=0).stop 2024-03-18T23:07:13.506 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:13.505+0000 7f6cc28ac700 1 -- 172.21.15.49:0/418735770 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6cbc101bb0 msgr2=0x7f6cbc199e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:13.506 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:13.505+0000 7f6cc28ac700 1 --2- 172.21.15.49:0/418735770 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6cbc101bb0 0x7f6cbc199e10 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f6cb0000c00 tx=0x7f6cb000bab0 comp rx=0 tx=0).stop 2024-03-18T23:07:13.506 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:13.505+0000 7f6cc28ac700 1 -- 172.21.15.49:0/418735770 shutdown_connections 2024-03-18T23:07:13.506 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:13.505+0000 7f6cc28ac700 1 --2- 172.21.15.49:0/418735770 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6cbc107250 0x7f6cbc1a4950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:13.506 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:13.505+0000 7f6cc28ac700 1 --2- 172.21.15.49:0/418735770 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6cbc101210 0x7f6cbc1998d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:13.506 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:13.505+0000 7f6cc28ac700 1 --2- 172.21.15.49:0/418735770 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f6ca40a2920 0x7f6ca40a4de0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:13.506 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:13.505+0000 7f6cc28ac700 1 --2- 172.21.15.49:0/418735770 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6cbc101bb0 0x7f6cbc199e10 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:13.507 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:13.505+0000 7f6cc28ac700 1 -- 172.21.15.49:0/418735770 >> 172.21.15.49:0/418735770 conn(0x7f6cbc078140 msgr2=0x7f6cbc0ffe50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:07:13.507 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:13.505+0000 7f6cc28ac700 1 -- 172.21.15.49:0/418735770 shutdown_connections 2024-03-18T23:07:13.507 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:13.505+0000 7f6cc28ac700 1 -- 172.21.15.49:0/418735770 wait complete. 2024-03-18T23:07:13.661 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:13.190+0000 7f2208d1c700 1 -- 172.21.15.49:0/310899284 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2204101bb0 msgr2=0x7f220410e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:13.661 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:13.190+0000 7f2208d1c700 1 --2- 172.21.15.49:0/310899284 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2204101bb0 0x7f220410e090 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f21fc00b0a0 tx=0x7f21fc00b3b0 comp rx=0 tx=0).stop 2024-03-18T23:07:13.661 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:13.190+0000 7f2208d1c700 1 -- 172.21.15.49:0/310899284 shutdown_connections 2024-03-18T23:07:13.661 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:13.190+0000 7f2208d1c700 1 --2- 172.21.15.49:0/310899284 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2204101bb0 0x7f220410e090 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:13.661 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:13.190+0000 7f2208d1c700 1 --2- 172.21.15.49:0/310899284 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2204107250 0x7f2204107630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:13.661 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:13.190+0000 7f2208d1c700 1 --2- 172.21.15.49:0/310899284 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2204101210 0x7f2204101670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:13.662 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:13.190+0000 7f2208d1c700 1 -- 172.21.15.49:0/310899284 >> 172.21.15.49:0/310899284 conn(0x7f2204078140 msgr2=0x7f2204078550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:07:13.662 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:13.190+0000 7f2208d1c700 1 -- 172.21.15.49:0/310899284 shutdown_connections 2024-03-18T23:07:13.662 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:13.190+0000 7f2208d1c700 1 -- 172.21.15.49:0/310899284 wait complete. 2024-03-18T23:07:13.662 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:13.191+0000 7f2208d1c700 1 Processor -- start 2024-03-18T23:07:13.662 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:13.191+0000 7f2208d1c700 1 -- start start 2024-03-18T23:07:13.662 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:13.191+0000 7f2208d1c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2204101210 0x7f22041998d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:13.662 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:13.191+0000 7f2208d1c700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2204101bb0 0x7f2204199e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:13.662 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:13.191+0000 7f2208d1c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2204107250 0x7f22041a4950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:13.662 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:13.191+0000 7f2208d1c700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f22041114d0 con 0x7f2204107250 2024-03-18T23:07:13.662 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:13.191+0000 7f2208d1c700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f2204111350 con 0x7f2204101bb0 2024-03-18T23:07:13.662 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:13.191+0000 7f2208d1c700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f2204111650 con 0x7f2204101210 2024-03-18T23:07:13.662 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:13.191+0000 7f2202d9d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2204107250 0x7f22041a4950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:13.663 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:13.191+0000 7f2202d9d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2204107250 0x7f22041a4950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47128/0 (socket says 172.21.15.49:47128) 2024-03-18T23:07:13.663 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:13.191+0000 7f220259c700 1 --2- 172.21.15.49:0/1935784325 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2204101210 0x7f22041998d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:13.663 INFO:teuthology.orchestra.run.smithi049.stdout:257698037802 2024-03-18T23:07:13.663 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:13.191+0000 7f2202d9d700 1 -- 172.21.15.49:0/1935784325 learned_addr learned my addr 172.21.15.49:0/1935784325 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:07:13.663 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:13.191+0000 7f21fbfff700 1 --2- 172.21.15.49:0/1935784325 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2204101bb0 0x7f2204199e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:13.664 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:13.191+0000 7f2202d9d700 1 -- 172.21.15.49:0/1935784325 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2204101210 msgr2=0x7f22041998d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:13.664 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:13.191+0000 7f2202d9d700 1 --2- 172.21.15.49:0/1935784325 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2204101210 0x7f22041998d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:13.664 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:13.191+0000 7f2202d9d700 1 -- 172.21.15.49:0/1935784325 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2204101bb0 msgr2=0x7f2204199e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:13.664 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:13.191+0000 7f2202d9d700 1 --2- 172.21.15.49:0/1935784325 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2204101bb0 0x7f2204199e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:13.664 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:13.191+0000 7f2202d9d700 1 -- 172.21.15.49:0/1935784325 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22041a4e90 con 0x7f2204107250 2024-03-18T23:07:13.664 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:13.191+0000 7f220259c700 1 --2- 172.21.15.49:0/1935784325 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2204101210 0x7f22041998d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:07:13.664 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:13.191+0000 7f21fbfff700 1 --2- 172.21.15.49:0/1935784325 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2204101bb0 0x7f2204199e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:07:13.664 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:13.192+0000 7f2202d9d700 1 --2- 172.21.15.49:0/1935784325 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2204107250 0x7f22041a4950 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f21fc00b780 tx=0x7f21fc004d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:07:13.664 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:13.192+0000 7f21fb7fe700 1 -- 172.21.15.49:0/1935784325 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21fc021070 con 0x7f2204107250 2024-03-18T23:07:13.664 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:13.192+0000 7f2208d1c700 1 -- 172.21.15.49:0/1935784325 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22041a5060 con 0x7f2204107250 2024-03-18T23:07:13.664 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:13.192+0000 7f21fb7fe700 1 -- 172.21.15.49:0/1935784325 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f21fc007b80 con 0x7f2204107250 2024-03-18T23:07:13.664 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:13.192+0000 7f21fb7fe700 1 -- 172.21.15.49:0/1935784325 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21fc0052a0 con 0x7f2204107250 2024-03-18T23:07:13.665 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:13.192+0000 7f2208d1c700 1 -- 172.21.15.49:0/1935784325 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22041a54c0 con 0x7f2204107250 2024-03-18T23:07:13.665 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:13.193+0000 7f2208d1c700 1 -- 172.21.15.49:0/1935784325 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2204068fd0 con 0x7f2204107250 2024-03-18T23:07:13.665 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:13.194+0000 7f21fb7fe700 1 -- 172.21.15.49:0/1935784325 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 17) ==== 137341+0+0 (secure 0 0 0) 0x7f21fc024070 con 0x7f2204107250 2024-03-18T23:07:13.665 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:13.195+0000 7f21fb7fe700 1 --2- 172.21.15.49:0/1935784325 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f21f00a2920 0x7f21f00a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:13.665 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:13.195+0000 7f220259c700 1 --2- 172.21.15.49:0/1935784325 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f21f00a2920 0x7f21f00a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:13.665 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:13.195+0000 7f21fb7fe700 1 -- 172.21.15.49:0/1935784325 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f21fc0cfbe0 con 0x7f2204107250 2024-03-18T23:07:13.665 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:13.195+0000 7f220259c700 1 --2- 172.21.15.49:0/1935784325 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f21f00a2920 0x7f21f00a4de0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f21f4009fd0 tx=0x7f21f4009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:07:13.665 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:13.198+0000 7f21fb7fe700 1 -- 172.21.15.49:0/1935784325 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f21fc007d30 con 0x7f2204107250 2024-03-18T23:07:13.665 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:13.410+0000 7f2208d1c700 1 -- 172.21.15.49:0/1935784325 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7f220402dd70 con 0x7f2204107250 2024-03-18T23:07:13.665 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:13.410+0000 7f21fb7fe700 1 -- 172.21.15.49:0/1935784325 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f21fc09cbc0 con 0x7f2204107250 2024-03-18T23:07:13.665 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:13.412+0000 7f2208d1c700 1 -- 172.21.15.49:0/1935784325 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f21f00a2920 msgr2=0x7f21f00a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:13.665 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:13.412+0000 7f2208d1c700 1 --2- 172.21.15.49:0/1935784325 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f21f00a2920 0x7f21f00a4de0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f21f4009fd0 tx=0x7f21f4009260 comp rx=0 tx=0).stop 2024-03-18T23:07:13.665 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:13.412+0000 7f2208d1c700 1 -- 172.21.15.49:0/1935784325 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2204107250 msgr2=0x7f22041a4950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:13.665 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:13.412+0000 7f2208d1c700 1 --2- 172.21.15.49:0/1935784325 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2204107250 0x7f22041a4950 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f21fc00b780 tx=0x7f21fc004d00 comp rx=0 tx=0).stop 2024-03-18T23:07:13.666 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:13.412+0000 7f2208d1c700 1 -- 172.21.15.49:0/1935784325 shutdown_connections 2024-03-18T23:07:13.666 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:13.412+0000 7f2208d1c700 1 --2- 172.21.15.49:0/1935784325 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2204107250 0x7f22041a4950 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:13.666 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:13.412+0000 7f2208d1c700 1 --2- 172.21.15.49:0/1935784325 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f21f00a2920 0x7f21f00a4de0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:13.666 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:13.412+0000 7f2208d1c700 1 --2- 172.21.15.49:0/1935784325 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2204101210 0x7f22041998d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:13.666 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:13.412+0000 7f2208d1c700 1 --2- 172.21.15.49:0/1935784325 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2204101bb0 0x7f2204199e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:13.666 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:13.412+0000 7f2208d1c700 1 -- 172.21.15.49:0/1935784325 >> 172.21.15.49:0/1935784325 conn(0x7f2204078140 msgr2=0x7f22040ffe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:07:13.666 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:13.413+0000 7f2208d1c700 1 -- 172.21.15.49:0/1935784325 shutdown_connections 2024-03-18T23:07:13.666 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:13.413+0000 7f2208d1c700 1 -- 172.21.15.49:0/1935784325 wait complete. 2024-03-18T23:07:13.711 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:07:13 smithi175 ceph-mon[67352]: pgmap v324: 97 pgs: 97 active+clean; 641 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 6 op/s; 202 B/s, 0 objects/s recovering 2024-03-18T23:07:13.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:07:13 smithi139 ceph-mon[67282]: pgmap v324: 97 pgs: 97 active+clean; 641 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 6 op/s; 202 B/s, 0 objects/s recovering 2024-03-18T23:07:14.398 INFO:tasks.cephadm.ceph_manager.ceph:need seq 257698037797 got 257698037802 for osd.8 2024-03-18T23:07:14.398 DEBUG:teuthology.parallel:result is None 2024-03-18T23:07:14.630 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:07:14 smithi049 ceph-mon[62991]: from='client.? 172.21.15.49:0/1935784325' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-03-18T23:07:14.630 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:07:14 smithi049 ceph-mon[62991]: from='client.? 172.21.15.49:0/418735770' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-03-18T23:07:14.711 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:07:14 smithi175 ceph-mon[67352]: from='client.? 172.21.15.49:0/1935784325' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-03-18T23:07:14.711 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:07:14 smithi175 ceph-mon[67352]: from='client.? 172.21.15.49:0/418735770' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-03-18T23:07:14.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:07:14 smithi139 ceph-mon[67282]: from='client.? 172.21.15.49:0/1935784325' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-03-18T23:07:14.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:07:14 smithi139 ceph-mon[67282]: from='client.? 172.21.15.49:0/418735770' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-03-18T23:07:14.920 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T23:07:14.929 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:14.927+0000 7f82fd97f700 1 -- 172.21.15.49:0/2621262728 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f82f81083a0 msgr2=0x7f82f8108780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:14.929 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:14.927+0000 7f82fd97f700 1 --2- 172.21.15.49:0/2621262728 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f82f81083a0 0x7f82f8108780 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f82e80098f0 tx=0x7f82e8009c00 comp rx=0 tx=0).stop 2024-03-18T23:07:14.929 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:14.928+0000 7f82fd97f700 1 -- 172.21.15.49:0/2621262728 shutdown_connections 2024-03-18T23:07:14.929 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:14.928+0000 7f82fd97f700 1 --2- 172.21.15.49:0/2621262728 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f82f8102d30 0x7f82f810f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:14.929 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:14.928+0000 7f82fd97f700 1 --2- 172.21.15.49:0/2621262728 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f82f81083a0 0x7f82f8108780 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:14.929 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:14.928+0000 7f82fd97f700 1 --2- 172.21.15.49:0/2621262728 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f82f8102390 0x7f82f81027f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:14.929 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:14.928+0000 7f82fd97f700 1 -- 172.21.15.49:0/2621262728 >> 172.21.15.49:0/2621262728 conn(0x7f82f80793e0 msgr2=0x7f82f80797f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:07:14.930 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:14.928+0000 7f82fd97f700 1 -- 172.21.15.49:0/2621262728 shutdown_connections 2024-03-18T23:07:14.930 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:14.929+0000 7f82fd97f700 1 -- 172.21.15.49:0/2621262728 wait complete. 2024-03-18T23:07:14.930 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:14.929+0000 7f82fd97f700 1 Processor -- start 2024-03-18T23:07:14.932 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:14.930+0000 7f82fd97f700 1 -- start start 2024-03-18T23:07:14.932 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:14.930+0000 7f82fd97f700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f82f8102390 0x7f82f819e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:14.932 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:14.930+0000 7f82fd97f700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f82f8102d30 0x7f82f819ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:14.932 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:14.930+0000 7f82fd97f700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f82f81083a0 0x7f82f8198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:14.932 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:14.930+0000 7f82fd97f700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f82f8112630 con 0x7f82f8102d30 2024-03-18T23:07:14.932 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:14.930+0000 7f82fd97f700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f82f81124b0 con 0x7f82f81083a0 2024-03-18T23:07:14.932 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:14.930+0000 7f82fd97f700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f82f81127b0 con 0x7f82f8102390 2024-03-18T23:07:14.933 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:14.930+0000 7f82f77fe700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f82f81083a0 0x7f82f8198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:14.933 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:14.930+0000 7f82f6ffd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f82f8102390 0x7f82f819e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:14.933 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:14.930+0000 7f82f77fe700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f82f81083a0 0x7f82f8198990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.49:59378/0 (socket says 172.21.15.49:59378) 2024-03-18T23:07:14.933 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:14.930+0000 7f82f77fe700 1 -- 172.21.15.49:0/1271779600 learned_addr learned my addr 172.21.15.49:0/1271779600 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:07:14.933 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:14.930+0000 7f82f67fc700 1 --2- 172.21.15.49:0/1271779600 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f82f8102d30 0x7f82f819ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:14.933 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:14.930+0000 7f82f77fe700 1 -- 172.21.15.49:0/1271779600 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f82f8102390 msgr2=0x7f82f819e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:14.933 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:14.930+0000 7f82f77fe700 1 --2- 172.21.15.49:0/1271779600 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f82f8102390 0x7f82f819e7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:14.933 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:14.930+0000 7f82f77fe700 1 -- 172.21.15.49:0/1271779600 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f82f8102d30 msgr2=0x7f82f819ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:14.933 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:14.930+0000 7f82f77fe700 1 --2- 172.21.15.49:0/1271779600 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f82f8102d30 0x7f82f819ed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:14.933 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:14.930+0000 7f82f77fe700 1 -- 172.21.15.49:0/1271779600 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82f81990f0 con 0x7f82f81083a0 2024-03-18T23:07:14.933 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:14.931+0000 7f82f6ffd700 1 --2- 172.21.15.49:0/1271779600 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f82f8102390 0x7f82f819e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:07:14.933 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:14.931+0000 7f82f77fe700 1 --2- 172.21.15.49:0/1271779600 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f82f81083a0 0x7f82f8198990 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f82f000ef30 tx=0x7f82f000c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:07:14.933 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:14.931+0000 7f82fc97d700 1 -- 172.21.15.49:0/1271779600 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82f00092e0 con 0x7f82f81083a0 2024-03-18T23:07:14.934 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:14.931+0000 7f82fc97d700 1 -- 172.21.15.49:0/1271779600 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f82f0004bb0 con 0x7f82f81083a0 2024-03-18T23:07:14.934 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:14.931+0000 7f82fd97f700 1 -- 172.21.15.49:0/1271779600 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82f8199380 con 0x7f82f81083a0 2024-03-18T23:07:14.934 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:14.931+0000 7f82fc97d700 1 -- 172.21.15.49:0/1271779600 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82f0004280 con 0x7f82f81083a0 2024-03-18T23:07:14.934 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:14.931+0000 7f82fd97f700 1 -- 172.21.15.49:0/1271779600 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82f81a56a0 con 0x7f82f81083a0 2024-03-18T23:07:14.934 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:14.933+0000 7f82fd97f700 1 -- 172.21.15.49:0/1271779600 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82f8068fd0 con 0x7f82f81083a0 2024-03-18T23:07:14.935 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:14.934+0000 7f82fc97d700 1 -- 172.21.15.49:0/1271779600 <== mon.2 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 17) ==== 137341+0+0 (secure 0 0 0) 0x7f82f001b030 con 0x7f82f81083a0 2024-03-18T23:07:14.935 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:14.935+0000 7f82fc97d700 1 --2- 172.21.15.49:0/1271779600 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f82e00a29a0 0x7f82e00a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:14.936 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:14.935+0000 7f82fc97d700 1 -- 172.21.15.49:0/1271779600 <== mon.2 v2:172.21.15.139:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f82f00ce640 con 0x7f82f81083a0 2024-03-18T23:07:14.936 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:14.935+0000 7f82f6ffd700 1 --2- 172.21.15.49:0/1271779600 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f82e00a29a0 0x7f82e00a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:14.936 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:14.936+0000 7f82f6ffd700 1 --2- 172.21.15.49:0/1271779600 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f82e00a29a0 0x7f82e00a4e60 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f82e80098c0 tx=0x7f82e8019040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:07:14.942 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:14.941+0000 7f82fc97d700 1 -- 172.21.15.49:0/1271779600 <== mon.2 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f82f0022020 con 0x7f82f81083a0 2024-03-18T23:07:15.154 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:15.153+0000 7f82fd97f700 1 -- 172.21.15.49:0/1271779600 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f82f802dbb0 con 0x7f82f81083a0 2024-03-18T23:07:15.154 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:15.153+0000 7f82fc97d700 1 -- 172.21.15.49:0/1271779600 <== mon.2 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f82f0010040 con 0x7f82f81083a0 2024-03-18T23:07:15.154 INFO:teuthology.orchestra.run.smithi049.stdout:107374182467 2024-03-18T23:07:15.157 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:15.156+0000 7f82fd97f700 1 -- 172.21.15.49:0/1271779600 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f82e00a29a0 msgr2=0x7f82e00a4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:15.157 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:15.156+0000 7f82fd97f700 1 --2- 172.21.15.49:0/1271779600 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f82e00a29a0 0x7f82e00a4e60 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f82e80098c0 tx=0x7f82e8019040 comp rx=0 tx=0).stop 2024-03-18T23:07:15.157 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:15.156+0000 7f82fd97f700 1 -- 172.21.15.49:0/1271779600 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f82f81083a0 msgr2=0x7f82f8198990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:15.157 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:15.156+0000 7f82fd97f700 1 --2- 172.21.15.49:0/1271779600 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f82f81083a0 0x7f82f8198990 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f82f000ef30 tx=0x7f82f000c3a0 comp rx=0 tx=0).stop 2024-03-18T23:07:15.157 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:15.156+0000 7f82fd97f700 1 -- 172.21.15.49:0/1271779600 shutdown_connections 2024-03-18T23:07:15.157 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:15.156+0000 7f82fd97f700 1 --2- 172.21.15.49:0/1271779600 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f82f81083a0 0x7f82f8198990 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:15.157 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:15.156+0000 7f82fd97f700 1 --2- 172.21.15.49:0/1271779600 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f82e00a29a0 0x7f82e00a4e60 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:15.157 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:15.156+0000 7f82fd97f700 1 --2- 172.21.15.49:0/1271779600 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f82f8102390 0x7f82f819e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:15.157 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:15.156+0000 7f82fd97f700 1 --2- 172.21.15.49:0/1271779600 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f82f8102d30 0x7f82f819ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:15.157 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:15.156+0000 7f82fd97f700 1 -- 172.21.15.49:0/1271779600 >> 172.21.15.49:0/1271779600 conn(0x7f82f80793e0 msgr2=0x7f82f8101150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:07:15.157 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:15.156+0000 7f82fd97f700 1 -- 172.21.15.49:0/1271779600 shutdown_connections 2024-03-18T23:07:15.157 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:15.156+0000 7f82fd97f700 1 -- 172.21.15.49:0/1271779600 wait complete. 2024-03-18T23:07:15.567 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:07:15 smithi049 ceph-mon[62991]: pgmap v325: 97 pgs: 97 active+clean; 641 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 4 op/s; 178 B/s, 0 objects/s recovering 2024-03-18T23:07:15.567 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:07:15 smithi049 ceph-mon[62991]: from='client.? 172.21.15.49:0/1271779600' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-03-18T23:07:15.676 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:15.672+0000 7f6e6ca45700 1 -- 172.21.15.49:0/1519656431 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6e68102590 msgr2=0x7f6e68102970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:15.676 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:15.672+0000 7f6e6ca45700 1 --2- 172.21.15.49:0/1519656431 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6e68102590 0x7f6e68102970 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f6e6000b0d0 tx=0x7f6e6000b3e0 comp rx=0 tx=0).stop 2024-03-18T23:07:15.676 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:15.672+0000 7f6e6ca45700 1 -- 172.21.15.49:0/1519656431 shutdown_connections 2024-03-18T23:07:15.676 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:15.672+0000 7f6e6ca45700 1 --2- 172.21.15.49:0/1519656431 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6e68107670 0x7f6e6810f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:15.677 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:15.672+0000 7f6e6ca45700 1 --2- 172.21.15.49:0/1519656431 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6e68102590 0x7f6e68102970 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:15.677 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:15.672+0000 7f6e6ca45700 1 --2- 172.21.15.49:0/1519656431 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6e68102f40 0x7f6e68106f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:15.677 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:15.672+0000 7f6e6ca45700 1 -- 172.21.15.49:0/1519656431 >> 172.21.15.49:0/1519656431 conn(0x7f6e680fdde0 msgr2=0x7f6e68100200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:07:15.677 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:15.672+0000 7f6e6ca45700 1 -- 172.21.15.49:0/1519656431 shutdown_connections 2024-03-18T23:07:15.677 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:15.672+0000 7f6e6ca45700 1 -- 172.21.15.49:0/1519656431 wait complete. 2024-03-18T23:07:15.677 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:15.673+0000 7f6e6ca45700 1 Processor -- start 2024-03-18T23:07:15.677 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:15.673+0000 7f6e6ca45700 1 -- start start 2024-03-18T23:07:15.677 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:15.673+0000 7f6e6ca45700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6e68102590 0x7f6e681a0a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:15.677 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:15.673+0000 7f6e6ca45700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6e68102f40 0x7f6e681a0f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:15.678 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:15.673+0000 7f6e6ca45700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6e68107670 0x7f6e6819ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:15.678 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:15.673+0000 7f6e6ca45700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f6e68112680 con 0x7f6e68102f40 2024-03-18T23:07:15.678 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:15.673+0000 7f6e6ca45700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f6e68112500 con 0x7f6e68102590 2024-03-18T23:07:15.678 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:15.673+0000 7f6e6ca45700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f6e68112800 con 0x7f6e68107670 2024-03-18T23:07:15.678 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:15.673+0000 7f6e65d9b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6e68102f40 0x7f6e681a0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:15.678 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:15.673+0000 7f6e65d9b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6e68102f40 0x7f6e681a0f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:56020/0 (socket says 172.21.15.49:56020) 2024-03-18T23:07:15.678 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:15.673+0000 7f6e65d9b700 1 -- 172.21.15.49:0/2896182544 learned_addr learned my addr 172.21.15.49:0/2896182544 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:07:15.678 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:15.673+0000 7f6e66d9d700 1 --2- 172.21.15.49:0/2896182544 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6e68107670 0x7f6e6819ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:15.678 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:15.674+0000 7f6e6659c700 1 --2- 172.21.15.49:0/2896182544 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6e68102590 0x7f6e681a0a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:15.679 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:15.674+0000 7f6e65d9b700 1 -- 172.21.15.49:0/2896182544 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6e68107670 msgr2=0x7f6e6819ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:15.679 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:15.674+0000 7f6e65d9b700 1 --2- 172.21.15.49:0/2896182544 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6e68107670 0x7f6e6819ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:15.679 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:15.674+0000 7f6e65d9b700 1 -- 172.21.15.49:0/2896182544 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6e68102590 msgr2=0x7f6e681a0a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:15.679 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:15.674+0000 7f6e65d9b700 1 --2- 172.21.15.49:0/2896182544 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6e68102590 0x7f6e681a0a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:15.679 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:15.674+0000 7f6e65d9b700 1 -- 172.21.15.49:0/2896182544 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e6819b400 con 0x7f6e68102f40 2024-03-18T23:07:15.679 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:15.674+0000 7f6e65d9b700 1 --2- 172.21.15.49:0/2896182544 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6e68102f40 0x7f6e681a0f90 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f6e5000e970 tx=0x7f6e5000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:07:15.679 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:15.674+0000 7f6e5f7fe700 1 -- 172.21.15.49:0/2896182544 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e50004a30 con 0x7f6e68102f40 2024-03-18T23:07:15.679 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:15.674+0000 7f6e5f7fe700 1 -- 172.21.15.49:0/2896182544 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6e5000ccc0 con 0x7f6e68102f40 2024-03-18T23:07:15.679 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:15.674+0000 7f6e6ca45700 1 -- 172.21.15.49:0/2896182544 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e6819b6f0 con 0x7f6e68102f40 2024-03-18T23:07:15.679 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:15.674+0000 7f6e5f7fe700 1 -- 172.21.15.49:0/2896182544 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e50005650 con 0x7f6e68102f40 2024-03-18T23:07:15.679 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:15.674+0000 7f6e6ca45700 1 -- 172.21.15.49:0/2896182544 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e681a79d0 con 0x7f6e68102f40 2024-03-18T23:07:15.680 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:15.675+0000 7f6e6ca45700 1 -- 172.21.15.49:0/2896182544 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e68103a70 con 0x7f6e68102f40 2024-03-18T23:07:15.680 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:15.676+0000 7f6e5f7fe700 1 -- 172.21.15.49:0/2896182544 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 17) ==== 137341+0+0 (secure 0 0 0) 0x7f6e50004bd0 con 0x7f6e68102f40 2024-03-18T23:07:15.680 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:15.677+0000 7f6e5f7fe700 1 --2- 172.21.15.49:0/2896182544 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f6e540a29a0 0x7f6e540a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:15.680 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:15.677+0000 7f6e5f7fe700 1 -- 172.21.15.49:0/2896182544 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f6e500ceff0 con 0x7f6e68102f40 2024-03-18T23:07:15.680 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:15.677+0000 7f6e6659c700 1 --2- 172.21.15.49:0/2896182544 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f6e540a29a0 0x7f6e540a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:15.681 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:15.677+0000 7f6e6659c700 1 --2- 172.21.15.49:0/2896182544 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f6e540a29a0 0x7f6e540a4e60 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f6e60009fd0 tx=0x7f6e6000b040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:07:15.681 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:15.680+0000 7f6e5f7fe700 1 -- 172.21.15.49:0/2896182544 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f6e50053740 con 0x7f6e68102f40 2024-03-18T23:07:15.711 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:07:15 smithi175 ceph-mon[67352]: pgmap v325: 97 pgs: 97 active+clean; 641 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 4 op/s; 178 B/s, 0 objects/s recovering 2024-03-18T23:07:15.711 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:07:15 smithi175 ceph-mon[67352]: from='client.? 172.21.15.49:0/1271779600' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-03-18T23:07:15.716 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:07:15 smithi139 ceph-mon[67282]: pgmap v325: 97 pgs: 97 active+clean; 641 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 4 op/s; 178 B/s, 0 objects/s recovering 2024-03-18T23:07:15.716 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:07:15 smithi139 ceph-mon[67282]: from='client.? 172.21.15.49:0/1271779600' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-03-18T23:07:15.891 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:15.890+0000 7f6e6ca45700 1 -- 172.21.15.49:0/2896182544 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7f6e681a7ba0 con 0x7f6e68102f40 2024-03-18T23:07:15.900 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:15.891+0000 7f6e5f7fe700 1 -- 172.21.15.49:0/2896182544 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f6e5009bee0 con 0x7f6e68102f40 2024-03-18T23:07:15.900 INFO:teuthology.orchestra.run.smithi049.stdout:347892351003 2024-03-18T23:07:15.901 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:15.893+0000 7f6e6ca45700 1 -- 172.21.15.49:0/2896182544 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f6e540a29a0 msgr2=0x7f6e540a4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:15.902 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:15.893+0000 7f6e6ca45700 1 --2- 172.21.15.49:0/2896182544 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f6e540a29a0 0x7f6e540a4e60 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f6e60009fd0 tx=0x7f6e6000b040 comp rx=0 tx=0).stop 2024-03-18T23:07:15.902 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:15.893+0000 7f6e6ca45700 1 -- 172.21.15.49:0/2896182544 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6e68102f40 msgr2=0x7f6e681a0f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:15.902 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:15.893+0000 7f6e6ca45700 1 --2- 172.21.15.49:0/2896182544 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6e68102f40 0x7f6e681a0f90 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f6e5000e970 tx=0x7f6e5000ec80 comp rx=0 tx=0).stop 2024-03-18T23:07:15.902 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:15.893+0000 7f6e6ca45700 1 -- 172.21.15.49:0/2896182544 shutdown_connections 2024-03-18T23:07:15.902 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:15.893+0000 7f6e6ca45700 1 --2- 172.21.15.49:0/2896182544 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f6e540a29a0 0x7f6e540a4e60 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:15.902 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:15.893+0000 7f6e6ca45700 1 --2- 172.21.15.49:0/2896182544 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6e68107670 0x7f6e6819ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:15.902 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:15.893+0000 7f6e6ca45700 1 --2- 172.21.15.49:0/2896182544 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6e68102590 0x7f6e681a0a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:15.902 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:15.893+0000 7f6e6ca45700 1 --2- 172.21.15.49:0/2896182544 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6e68102f40 0x7f6e681a0f90 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:15.902 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:15.893+0000 7f6e6ca45700 1 -- 172.21.15.49:0/2896182544 >> 172.21.15.49:0/2896182544 conn(0x7f6e680fdde0 msgr2=0x7f6e680feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:07:15.902 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:15.893+0000 7f6e6ca45700 1 -- 172.21.15.49:0/2896182544 shutdown_connections 2024-03-18T23:07:15.903 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:15.893+0000 7f6e6ca45700 1 -- 172.21.15.49:0/2896182544 wait complete. 2024-03-18T23:07:16.188 INFO:tasks.cephadm.ceph_manager.ceph:need seq 107374182459 got 107374182467 for osd.3 2024-03-18T23:07:16.188 DEBUG:teuthology.parallel:result is None 2024-03-18T23:07:16.558 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:07:16 smithi049 ceph-mon[62991]: from='client.? 172.21.15.49:0/2896182544' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-03-18T23:07:16.711 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:07:16 smithi175 ceph-mon[67352]: from='client.? 172.21.15.49:0/2896182544' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-03-18T23:07:16.741 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:07:16 smithi139 ceph-mon[67282]: from='client.? 172.21.15.49:0/2896182544' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-03-18T23:07:17.335 INFO:tasks.cephadm.ceph_manager.ceph:need seq 163208757299 got 163208757304 for osd.5 2024-03-18T23:07:17.335 DEBUG:teuthology.parallel:result is None 2024-03-18T23:07:17.630 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:07:17 smithi049 ceph-mon[62991]: pgmap v326: 97 pgs: 97 active+clean; 641 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 6 op/s; 130 B/s, 0 objects/s recovering 2024-03-18T23:07:17.711 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:07:17 smithi175 ceph-mon[67352]: pgmap v326: 97 pgs: 97 active+clean; 641 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 6 op/s; 130 B/s, 0 objects/s recovering 2024-03-18T23:07:17.813 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T23:07:17.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:07:17 smithi139 ceph-mon[67282]: pgmap v326: 97 pgs: 97 active+clean; 641 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 6 op/s; 130 B/s, 0 objects/s recovering 2024-03-18T23:07:18.271 INFO:tasks.cephadm.ceph_manager.ceph:need seq 347892350996 got 347892351003 for osd.11 2024-03-18T23:07:18.271 DEBUG:teuthology.parallel:result is None 2024-03-18T23:07:18.737 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:18.735+0000 7fd7add59700 1 -- 172.21.15.49:0/4004381265 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd7a81014a0 msgr2=0x7fd7a8101880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:18.737 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:18.735+0000 7fd7add59700 1 --2- 172.21.15.49:0/4004381265 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd7a81014a0 0x7fd7a8101880 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fd79c009990 tx=0x7fd79c009ca0 comp rx=0 tx=0).stop 2024-03-18T23:07:18.737 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:18.735+0000 7fd7add59700 1 -- 172.21.15.49:0/4004381265 shutdown_connections 2024-03-18T23:07:18.737 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:18.735+0000 7fd7add59700 1 --2- 172.21.15.49:0/4004381265 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd7a8106530 0x7fd7a810e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:18.737 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:18.735+0000 7fd7add59700 1 --2- 172.21.15.49:0/4004381265 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd7a81014a0 0x7fd7a8101880 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:18.737 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:18.735+0000 7fd7add59700 1 --2- 172.21.15.49:0/4004381265 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd7a8101e50 0x7fd7a8105e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:18.738 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:18.735+0000 7fd7add59700 1 -- 172.21.15.49:0/4004381265 >> 172.21.15.49:0/4004381265 conn(0x7fd7a8077230 msgr2=0x7fd7a8077640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:07:18.738 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:18.735+0000 7fd7add59700 1 -- 172.21.15.49:0/4004381265 shutdown_connections 2024-03-18T23:07:18.738 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:18.735+0000 7fd7add59700 1 -- 172.21.15.49:0/4004381265 wait complete. 2024-03-18T23:07:18.738 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:18.736+0000 7fd7add59700 1 Processor -- start 2024-03-18T23:07:18.738 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:18.736+0000 7fd7add59700 1 -- start start 2024-03-18T23:07:18.739 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:18.736+0000 7fd7add59700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd7a81014a0 0x7fd7a8102ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:18.739 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:18.736+0000 7fd7add59700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd7a8101e50 0x7fd7a8103430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:18.740 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:18.737+0000 7fd7add59700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd7a8106530 0x7fd7a8103970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:18.740 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:18.737+0000 7fd7add59700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fd7a8111510 con 0x7fd7a81014a0 2024-03-18T23:07:18.740 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:18.737+0000 7fd7add59700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fd7a8111390 con 0x7fd7a8106530 2024-03-18T23:07:18.740 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:18.737+0000 7fd7add59700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd7a8111690 con 0x7fd7a8101e50 2024-03-18T23:07:18.740 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:18.737+0000 7fd7a77fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd7a81014a0 0x7fd7a8102ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:18.740 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:18.737+0000 7fd7a77fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd7a81014a0 0x7fd7a8102ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:56034/0 (socket says 172.21.15.49:56034) 2024-03-18T23:07:18.740 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:18.737+0000 7fd7a77fe700 1 -- 172.21.15.49:0/4052810933 learned_addr learned my addr 172.21.15.49:0/4052810933 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:07:18.740 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:18.737+0000 7fd7a7fff700 1 --2- 172.21.15.49:0/4052810933 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd7a8106530 0x7fd7a8103970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:18.740 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:18.737+0000 7fd7a6ffd700 1 --2- 172.21.15.49:0/4052810933 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd7a8101e50 0x7fd7a8103430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:18.740 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:18.737+0000 7fd7a77fe700 1 -- 172.21.15.49:0/4052810933 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd7a8101e50 msgr2=0x7fd7a8103430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:18.740 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:18.737+0000 7fd7a77fe700 1 --2- 172.21.15.49:0/4052810933 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd7a8101e50 0x7fd7a8103430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:18.740 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:18.737+0000 7fd7a77fe700 1 -- 172.21.15.49:0/4052810933 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd7a8106530 msgr2=0x7fd7a8103970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:18.740 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:18.737+0000 7fd7a77fe700 1 --2- 172.21.15.49:0/4052810933 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd7a8106530 0x7fd7a8103970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:18.741 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:18.737+0000 7fd7a77fe700 1 -- 172.21.15.49:0/4052810933 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7a81a7040 con 0x7fd7a81014a0 2024-03-18T23:07:18.741 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:18.737+0000 7fd7a77fe700 1 --2- 172.21.15.49:0/4052810933 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd7a81014a0 0x7fd7a8102ef0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fd79c000c00 tx=0x7fd79c00bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:07:18.741 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:18.737+0000 7fd7a4ff9700 1 -- 172.21.15.49:0/4052810933 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd79c027070 con 0x7fd7a81014a0 2024-03-18T23:07:18.741 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:18.737+0000 7fd7add59700 1 -- 172.21.15.49:0/4052810933 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7a81a72d0 con 0x7fd7a81014a0 2024-03-18T23:07:18.741 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:18.737+0000 7fd7a4ff9700 1 -- 172.21.15.49:0/4052810933 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd79c003d70 con 0x7fd7a81014a0 2024-03-18T23:07:18.741 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:18.737+0000 7fd7a4ff9700 1 -- 172.21.15.49:0/4052810933 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd79c00f460 con 0x7fd7a81014a0 2024-03-18T23:07:18.741 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:18.737+0000 7fd7add59700 1 -- 172.21.15.49:0/4052810933 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7a81a7720 con 0x7fd7a81014a0 2024-03-18T23:07:18.741 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:18.738+0000 7fd7add59700 1 -- 172.21.15.49:0/4052810933 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7a8102930 con 0x7fd7a81014a0 2024-03-18T23:07:18.742 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:18.739+0000 7fd7a4ff9700 1 -- 172.21.15.49:0/4052810933 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 17) ==== 137341+0+0 (secure 0 0 0) 0x7fd79c02a070 con 0x7fd7a81014a0 2024-03-18T23:07:18.742 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:18.740+0000 7fd7a4ff9700 1 --2- 172.21.15.49:0/4052810933 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fd7900a2920 0x7fd7900a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:18.743 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:18.740+0000 7fd7a6ffd700 1 --2- 172.21.15.49:0/4052810933 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fd7900a2920 0x7fd7900a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:18.743 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:18.740+0000 7fd7a4ff9700 1 -- 172.21.15.49:0/4052810933 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7fd79c0cf960 con 0x7fd7a81014a0 2024-03-18T23:07:18.743 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:18.741+0000 7fd7a6ffd700 1 --2- 172.21.15.49:0/4052810933 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fd7900a2920 0x7fd7900a4de0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fd7a8101110 tx=0x7fd798009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:07:18.746 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:18.745+0000 7fd7a4ff9700 1 -- 172.21.15.49:0/4052810933 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fd79c013070 con 0x7fd7a81014a0 2024-03-18T23:07:18.958 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:18.957+0000 7fd7add59700 1 -- 172.21.15.49:0/4052810933 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7fd7a802dbb0 con 0x7fd7a81014a0 2024-03-18T23:07:18.958 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:18.957+0000 7fd7a4ff9700 1 -- 172.21.15.49:0/4052810933 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fd79c09c940 con 0x7fd7a81014a0 2024-03-18T23:07:18.958 INFO:teuthology.orchestra.run.smithi049.stdout:197568495669 2024-03-18T23:07:18.960 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:18.959+0000 7fd7add59700 1 -- 172.21.15.49:0/4052810933 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fd7900a2920 msgr2=0x7fd7900a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:18.960 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:18.959+0000 7fd7add59700 1 --2- 172.21.15.49:0/4052810933 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fd7900a2920 0x7fd7900a4de0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fd7a8101110 tx=0x7fd798009260 comp rx=0 tx=0).stop 2024-03-18T23:07:18.961 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:18.959+0000 7fd7add59700 1 -- 172.21.15.49:0/4052810933 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd7a81014a0 msgr2=0x7fd7a8102ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:18.961 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:18.959+0000 7fd7add59700 1 --2- 172.21.15.49:0/4052810933 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd7a81014a0 0x7fd7a8102ef0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fd79c000c00 tx=0x7fd79c00bab0 comp rx=0 tx=0).stop 2024-03-18T23:07:18.961 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:18.959+0000 7fd7add59700 1 -- 172.21.15.49:0/4052810933 shutdown_connections 2024-03-18T23:07:18.961 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:18.959+0000 7fd7add59700 1 --2- 172.21.15.49:0/4052810933 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd7a8106530 0x7fd7a8103970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:18.961 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:18.959+0000 7fd7add59700 1 --2- 172.21.15.49:0/4052810933 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd7a81014a0 0x7fd7a8102ef0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:18.961 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:18.959+0000 7fd7add59700 1 --2- 172.21.15.49:0/4052810933 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fd7900a2920 0x7fd7900a4de0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:18.961 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:18.959+0000 7fd7add59700 1 --2- 172.21.15.49:0/4052810933 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd7a8101e50 0x7fd7a8103430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:18.961 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:18.959+0000 7fd7add59700 1 -- 172.21.15.49:0/4052810933 >> 172.21.15.49:0/4052810933 conn(0x7fd7a8077230 msgr2=0x7fd7a806ad90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:07:18.961 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:18.960+0000 7fd7add59700 1 -- 172.21.15.49:0/4052810933 shutdown_connections 2024-03-18T23:07:18.961 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:18.960+0000 7fd7add59700 1 -- 172.21.15.49:0/4052810933 wait complete. 2024-03-18T23:07:19.658 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:07:19 smithi049 ceph-mon[62991]: pgmap v327: 97 pgs: 97 active+clean; 641 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-03-18T23:07:19.658 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:07:19 smithi049 ceph-mon[62991]: from='client.? 172.21.15.49:0/4052810933' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-03-18T23:07:19.659 INFO:tasks.cephadm.ceph_manager.ceph:need seq 197568495664 got 197568495669 for osd.6 2024-03-18T23:07:19.660 DEBUG:teuthology.parallel:result is None 2024-03-18T23:07:19.716 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:07:19 smithi175 ceph-mon[67352]: pgmap v327: 97 pgs: 97 active+clean; 641 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-03-18T23:07:19.716 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:07:19 smithi175 ceph-mon[67352]: from='client.? 172.21.15.49:0/4052810933' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-03-18T23:07:19.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:07:19 smithi139 ceph-mon[67282]: pgmap v327: 97 pgs: 97 active+clean; 641 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-03-18T23:07:19.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:07:19 smithi139 ceph-mon[67282]: from='client.? 172.21.15.49:0/4052810933' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-03-18T23:07:21.423 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:21.423+0000 7f6705771700 1 -- 172.21.15.49:0/1543905833 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f670006ac70 msgr2=0x7f670006b050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:21.423 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:21.423+0000 7f6705771700 1 --2- 172.21.15.49:0/1543905833 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f670006ac70 0x7f670006b050 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f66f00099c0 tx=0x7f66f0009cd0 comp rx=0 tx=0).stop 2024-03-18T23:07:21.423 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:21.423+0000 7f6705771700 1 -- 172.21.15.49:0/1543905833 shutdown_connections 2024-03-18T23:07:21.423 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:21.423+0000 7f6705771700 1 --2- 172.21.15.49:0/1543905833 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f67001076c0 0x7f670010f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:21.423 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:21.423+0000 7f6705771700 1 --2- 172.21.15.49:0/1543905833 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f670006ac70 0x7f670006b050 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:21.423 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:21.423+0000 7f6705771700 1 --2- 172.21.15.49:0/1543905833 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f670006b620 0x7f6700106f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:21.424 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:21.423+0000 7f6705771700 1 -- 172.21.15.49:0/1543905833 >> 172.21.15.49:0/1543905833 conn(0x7f6700077260 msgr2=0x7f6700077670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:07:21.424 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:21.423+0000 7f6705771700 1 -- 172.21.15.49:0/1543905833 shutdown_connections 2024-03-18T23:07:21.424 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:21.423+0000 7f6705771700 1 -- 172.21.15.49:0/1543905833 wait complete. 2024-03-18T23:07:21.424 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:21.423+0000 7f6705771700 1 Processor -- start 2024-03-18T23:07:21.426 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:21.423+0000 7f6705771700 1 -- start start 2024-03-18T23:07:21.426 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:21.424+0000 7f6705771700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f670006ac70 0x7f670019e730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:21.426 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:21.424+0000 7f6705771700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f670006b620 0x7f670019ec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:21.426 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:21.424+0000 7f6705771700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f67001076c0 0x7f67001988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:21.426 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:21.424+0000 7f6705771700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f6700112650 con 0x7f670006b620 2024-03-18T23:07:21.426 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:21.424+0000 7f6705771700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f67001124d0 con 0x7f670006ac70 2024-03-18T23:07:21.426 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:21.424+0000 7f6705771700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f67001127d0 con 0x7f67001076c0 2024-03-18T23:07:21.426 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:21.424+0000 7f66fe7fc700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f670006b620 0x7f670019ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:21.426 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:21.424+0000 7f66feffd700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f670006ac70 0x7f670019e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:21.427 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:21.424+0000 7f66fe7fc700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f670006b620 0x7f670019ec70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:56058/0 (socket says 172.21.15.49:56058) 2024-03-18T23:07:21.427 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:21.424+0000 7f66fe7fc700 1 -- 172.21.15.49:0/2348313978 learned_addr learned my addr 172.21.15.49:0/2348313978 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:07:21.427 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:21.424+0000 7f66ff7fe700 1 --2- 172.21.15.49:0/2348313978 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f67001076c0 0x7f67001988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:21.427 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:21.424+0000 7f66fe7fc700 1 -- 172.21.15.49:0/2348313978 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f67001076c0 msgr2=0x7f67001988f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:21.427 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:21.424+0000 7f66fe7fc700 1 --2- 172.21.15.49:0/2348313978 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f67001076c0 0x7f67001988f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:21.427 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:21.424+0000 7f66fe7fc700 1 -- 172.21.15.49:0/2348313978 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f670006ac70 msgr2=0x7f670019e730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:21.427 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:21.424+0000 7f66fe7fc700 1 --2- 172.21.15.49:0/2348313978 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f670006ac70 0x7f670019e730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:21.427 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:21.424+0000 7f66fe7fc700 1 -- 172.21.15.49:0/2348313978 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6700199050 con 0x7f670006b620 2024-03-18T23:07:21.427 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:21.424+0000 7f66feffd700 1 --2- 172.21.15.49:0/2348313978 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f670006ac70 0x7f670019e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:07:21.427 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:21.424+0000 7f66fe7fc700 1 --2- 172.21.15.49:0/2348313978 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f670006b620 0x7f670019ec70 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f66f400d950 tx=0x7f66f400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:07:21.427 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:21.424+0000 7f66e7fff700 1 -- 172.21.15.49:0/2348313978 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66f4016070 con 0x7f670006b620 2024-03-18T23:07:21.427 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:21.425+0000 7f6705771700 1 -- 172.21.15.49:0/2348313978 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6700199340 con 0x7f670006b620 2024-03-18T23:07:21.428 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:21.425+0000 7f66e7fff700 1 -- 172.21.15.49:0/2348313978 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f66f4004d10 con 0x7f670006b620 2024-03-18T23:07:21.428 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:21.425+0000 7f66e7fff700 1 -- 172.21.15.49:0/2348313978 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66f40056a0 con 0x7f670006b620 2024-03-18T23:07:21.428 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:21.425+0000 7f6705771700 1 -- 172.21.15.49:0/2348313978 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67001a5600 con 0x7f670006b620 2024-03-18T23:07:21.428 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:21.426+0000 7f6705771700 1 -- 172.21.15.49:0/2348313978 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f670010a890 con 0x7f670006b620 2024-03-18T23:07:21.429 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:21.427+0000 7f66e7fff700 1 -- 172.21.15.49:0/2348313978 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 17) ==== 137341+0+0 (secure 0 0 0) 0x7f66f4004220 con 0x7f670006b620 2024-03-18T23:07:21.429 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:21.427+0000 7f66e7fff700 1 --2- 172.21.15.49:0/2348313978 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f66e80a2920 0x7f66e80a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:21.429 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:21.427+0000 7f66feffd700 1 --2- 172.21.15.49:0/2348313978 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f66e80a2920 0x7f66e80a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:21.429 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:21.427+0000 7f66e7fff700 1 -- 172.21.15.49:0/2348313978 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f66f40ce910 con 0x7f670006b620 2024-03-18T23:07:21.429 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:21.428+0000 7f66feffd700 1 --2- 172.21.15.49:0/2348313978 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f66e80a2920 0x7f66e80a4de0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f66f0000c00 tx=0x7f66f0005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:07:21.432 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:21.432+0000 7f66e7fff700 1 -- 172.21.15.49:0/2348313978 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f66f401f020 con 0x7f670006b620 2024-03-18T23:07:21.485 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:07:21 smithi049 ceph-mon[62991]: pgmap v328: 97 pgs: 97 active+clean; 641 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-18T23:07:21.640 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:21.640+0000 7f6705771700 1 -- 172.21.15.49:0/2348313978 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7f670002dbb0 con 0x7f670006b620 2024-03-18T23:07:21.640 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:21.640+0000 7f66e7fff700 1 -- 172.21.15.49:0/2348313978 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f66f4022350 con 0x7f670006b620 2024-03-18T23:07:21.641 INFO:teuthology.orchestra.run.smithi049.stdout:313532612639 2024-03-18T23:07:21.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:21.642+0000 7f6705771700 1 -- 172.21.15.49:0/2348313978 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f66e80a2920 msgr2=0x7f66e80a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:21.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:21.642+0000 7f6705771700 1 --2- 172.21.15.49:0/2348313978 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f66e80a2920 0x7f66e80a4de0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f66f0000c00 tx=0x7f66f0005fb0 comp rx=0 tx=0).stop 2024-03-18T23:07:21.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:21.642+0000 7f6705771700 1 -- 172.21.15.49:0/2348313978 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f670006b620 msgr2=0x7f670019ec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:21.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:21.642+0000 7f6705771700 1 --2- 172.21.15.49:0/2348313978 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f670006b620 0x7f670019ec70 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f66f400d950 tx=0x7f66f400dc60 comp rx=0 tx=0).stop 2024-03-18T23:07:21.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:21.642+0000 7f6705771700 1 -- 172.21.15.49:0/2348313978 shutdown_connections 2024-03-18T23:07:21.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:21.642+0000 7f6705771700 1 --2- 172.21.15.49:0/2348313978 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f66e80a2920 0x7f66e80a4de0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:21.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:21.642+0000 7f6705771700 1 --2- 172.21.15.49:0/2348313978 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f67001076c0 0x7f67001988f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:21.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:21.642+0000 7f6705771700 1 --2- 172.21.15.49:0/2348313978 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f670006ac70 0x7f670019e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:21.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:21.642+0000 7f6705771700 1 --2- 172.21.15.49:0/2348313978 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f670006b620 0x7f670019ec70 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:21.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:21.642+0000 7f6705771700 1 -- 172.21.15.49:0/2348313978 >> 172.21.15.49:0/2348313978 conn(0x7f6700077260 msgr2=0x7f67001087f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:07:21.644 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:21.643+0000 7f6705771700 1 -- 172.21.15.49:0/2348313978 shutdown_connections 2024-03-18T23:07:21.644 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:21.643+0000 7f6705771700 1 -- 172.21.15.49:0/2348313978 wait complete. 2024-03-18T23:07:21.711 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:07:21 smithi175 ceph-mon[67352]: pgmap v328: 97 pgs: 97 active+clean; 641 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-18T23:07:21.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:07:21 smithi139 ceph-mon[67282]: pgmap v328: 97 pgs: 97 active+clean; 641 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-18T23:07:22.353 INFO:tasks.cephadm.ceph_manager.ceph:need seq 313532612635 got 313532612639 for osd.10 2024-03-18T23:07:22.353 DEBUG:teuthology.parallel:result is None 2024-03-18T23:07:22.353 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-03-18T23:07:22.353 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell --fsid c63591ea-e57a-11ee-95c9-87774f69a715 -- ceph pg dump --format=json 2024-03-18T23:07:22.630 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:07:22 smithi049 ceph-mon[62991]: from='client.? 172.21.15.49:0/2348313978' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-03-18T23:07:22.640 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T23:07:22.711 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:07:22 smithi175 ceph-mon[67352]: from='client.? 172.21.15.49:0/2348313978' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-03-18T23:07:22.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:07:22 smithi139 ceph-mon[67282]: from='client.? 172.21.15.49:0/2348313978' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-03-18T23:07:23.711 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:07:23 smithi175 ceph-mon[67352]: pgmap v329: 97 pgs: 97 active+clean; 641 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-03-18T23:07:23.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:07:23 smithi139 ceph-mon[67282]: pgmap v329: 97 pgs: 97 active+clean; 641 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-03-18T23:07:23.868 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:07:23 smithi049 ceph-mon[62991]: pgmap v329: 97 pgs: 97 active+clean; 641 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-03-18T23:07:24.446 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:24.439+0000 7f4dcb2b8700 1 -- 172.21.15.49:0/1806044958 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4dc4102390 msgr2=0x7f4dc4102810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:24.446 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:24.439+0000 7f4dcb2b8700 1 --2- 172.21.15.49:0/1806044958 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4dc4102390 0x7f4dc4102810 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f4db4009990 tx=0x7f4db4009ca0 comp rx=0 tx=0).stop 2024-03-18T23:07:24.446 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:24.439+0000 7f4dcb2b8700 1 -- 172.21.15.49:0/1806044958 shutdown_connections 2024-03-18T23:07:24.446 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:24.439+0000 7f4dcb2b8700 1 --2- 172.21.15.49:0/1806044958 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4dc4102d50 0x7f4dc410f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:24.447 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:24.439+0000 7f4dcb2b8700 1 --2- 172.21.15.49:0/1806044958 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4dc41083e0 0x7f4dc41087c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:24.447 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:24.439+0000 7f4dcb2b8700 1 --2- 172.21.15.49:0/1806044958 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4dc4102390 0x7f4dc4102810 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:24.447 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:24.439+0000 7f4dcb2b8700 1 -- 172.21.15.49:0/1806044958 >> 172.21.15.49:0/1806044958 conn(0x7f4dc4079460 msgr2=0x7f4dc4079870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:07:24.447 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:24.439+0000 7f4dcb2b8700 1 -- 172.21.15.49:0/1806044958 shutdown_connections 2024-03-18T23:07:24.447 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:24.439+0000 7f4dcb2b8700 1 -- 172.21.15.49:0/1806044958 wait complete. 2024-03-18T23:07:24.447 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:24.440+0000 7f4dcb2b8700 1 Processor -- start 2024-03-18T23:07:24.447 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:24.440+0000 7f4dcb2b8700 1 -- start start 2024-03-18T23:07:24.447 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:24.440+0000 7f4dcb2b8700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4dc4102390 0x7f4dc419e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:24.447 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:24.440+0000 7f4dcb2b8700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4dc4102d50 0x7f4dc419ec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:24.447 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:24.440+0000 7f4dcb2b8700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4dc41083e0 0x7f4dc41988c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:24.447 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:24.440+0000 7f4dc9054700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4dc4102390 0x7f4dc419e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:24.447 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:24.440+0000 7f4dc8853700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4dc4102d50 0x7f4dc419ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:24.447 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:24.440+0000 7f4dc8853700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4dc4102d50 0x7f4dc419ec40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.49:43476/0 (socket says 172.21.15.49:43476) 2024-03-18T23:07:24.448 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:24.440+0000 7f4dc8853700 1 -- 172.21.15.49:0/260896265 learned_addr learned my addr 172.21.15.49:0/260896265 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:07:24.448 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:24.440+0000 7f4dc9855700 1 --2- 172.21.15.49:0/260896265 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4dc41083e0 0x7f4dc41988c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:24.448 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:24.440+0000 7f4dcb2b8700 1 -- 172.21.15.49:0/260896265 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f4dc4112600 con 0x7f4dc41083e0 2024-03-18T23:07:24.448 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:24.441+0000 7f4dcb2b8700 1 -- 172.21.15.49:0/260896265 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f4dc4112480 con 0x7f4dc4102390 2024-03-18T23:07:24.448 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:24.441+0000 7f4dcb2b8700 1 -- 172.21.15.49:0/260896265 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f4dc4112780 con 0x7f4dc4102d50 2024-03-18T23:07:24.448 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:24.441+0000 7f4dc8853700 1 -- 172.21.15.49:0/260896265 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4dc4102390 msgr2=0x7f4dc419e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:24.448 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:24.441+0000 7f4dc8853700 1 --2- 172.21.15.49:0/260896265 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4dc4102390 0x7f4dc419e700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:24.448 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:24.441+0000 7f4dc8853700 1 -- 172.21.15.49:0/260896265 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4dc41083e0 msgr2=0x7f4dc41988c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:24.448 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:24.441+0000 7f4dc8853700 1 --2- 172.21.15.49:0/260896265 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4dc41083e0 0x7f4dc41988c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:24.448 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:24.441+0000 7f4dc8853700 1 -- 172.21.15.49:0/260896265 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4dc4199020 con 0x7f4dc4102d50 2024-03-18T23:07:24.448 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:24.441+0000 7f4dc9855700 1 --2- 172.21.15.49:0/260896265 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4dc41083e0 0x7f4dc41988c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:07:24.448 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:24.441+0000 7f4dc8853700 1 --2- 172.21.15.49:0/260896265 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4dc4102d50 0x7f4dc419ec40 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f4db40038b0 tx=0x7f4db4005ef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:07:24.449 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:24.441+0000 7f4dba7fc700 1 -- 172.21.15.49:0/260896265 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4db4027070 con 0x7f4dc4102d50 2024-03-18T23:07:24.449 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:24.441+0000 7f4dba7fc700 1 -- 172.21.15.49:0/260896265 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4db4004050 con 0x7f4dc4102d50 2024-03-18T23:07:24.449 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:24.441+0000 7f4dcb2b8700 1 -- 172.21.15.49:0/260896265 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4dc41992b0 con 0x7f4dc4102d50 2024-03-18T23:07:24.449 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:24.441+0000 7f4dba7fc700 1 -- 172.21.15.49:0/260896265 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4db4022d00 con 0x7f4dc4102d50 2024-03-18T23:07:24.449 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:24.441+0000 7f4dcb2b8700 1 -- 172.21.15.49:0/260896265 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4dc41a55d0 con 0x7f4dc4102d50 2024-03-18T23:07:24.449 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:24.442+0000 7f4dcb2b8700 1 -- 172.21.15.49:0/260896265 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4dc410a7c0 con 0x7f4dc4102d50 2024-03-18T23:07:24.449 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:24.444+0000 7f4dba7fc700 1 -- 172.21.15.49:0/260896265 <== mon.1 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 17) ==== 137341+0+0 (secure 0 0 0) 0x7f4db402a070 con 0x7f4dc4102d50 2024-03-18T23:07:24.449 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:24.444+0000 7f4dba7fc700 1 --2- 172.21.15.49:0/260896265 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f4db00a29a0 0x7f4db00a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:24.449 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:24.444+0000 7f4dc9054700 1 --2- 172.21.15.49:0/260896265 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f4db00a29a0 0x7f4db00a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:24.449 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:24.444+0000 7f4dba7fc700 1 -- 172.21.15.49:0/260896265 <== mon.1 v2:172.21.15.175:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f4db40d03a0 con 0x7f4dc4102d50 2024-03-18T23:07:24.449 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:24.445+0000 7f4dc9054700 1 --2- 172.21.15.49:0/260896265 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f4db00a29a0 0x7f4db00a4e60 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f4dc4199a00 tx=0x7f4dc00073d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:07:24.450 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:24.448+0000 7f4dba7fc700 1 -- 172.21.15.49:0/260896265 <== mon.1 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f4db4013070 con 0x7f4dc4102d50 2024-03-18T23:07:24.653 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:24.653+0000 7f4dcb2b8700 1 -- 172.21.15.49:0/260896265 --> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f4dc4199df0 con 0x7f4db00a29a0 2024-03-18T23:07:24.660 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:24.660+0000 7f4dba7fc700 1 -- 172.21.15.49:0/260896265 <== mgr.14150 v2:172.21.15.49:6800/2083805211 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258990 (secure 0 0 0) 0x7f4dc4199df0 con 0x7f4db00a29a0 2024-03-18T23:07:24.665 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-18T23:07:24.667 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:24.665+0000 7f4dcb2b8700 1 -- 172.21.15.49:0/260896265 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f4db00a29a0 msgr2=0x7f4db00a4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:24.667 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:24.665+0000 7f4dcb2b8700 1 --2- 172.21.15.49:0/260896265 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f4db00a29a0 0x7f4db00a4e60 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f4dc4199a00 tx=0x7f4dc00073d0 comp rx=0 tx=0).stop 2024-03-18T23:07:24.667 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:24.665+0000 7f4dcb2b8700 1 -- 172.21.15.49:0/260896265 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4dc4102d50 msgr2=0x7f4dc419ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:24.667 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:24.665+0000 7f4dcb2b8700 1 --2- 172.21.15.49:0/260896265 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4dc4102d50 0x7f4dc419ec40 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f4db40038b0 tx=0x7f4db4005ef0 comp rx=0 tx=0).stop 2024-03-18T23:07:24.667 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:24.665+0000 7f4dcb2b8700 1 -- 172.21.15.49:0/260896265 shutdown_connections 2024-03-18T23:07:24.667 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:24.665+0000 7f4dcb2b8700 1 --2- 172.21.15.49:0/260896265 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4dc41083e0 0x7f4dc41988c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:24.667 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:24.665+0000 7f4dcb2b8700 1 --2- 172.21.15.49:0/260896265 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4dc4102390 0x7f4dc419e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:24.667 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:24.665+0000 7f4dcb2b8700 1 --2- 172.21.15.49:0/260896265 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f4db00a29a0 0x7f4db00a4e60 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:24.667 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:24.665+0000 7f4dcb2b8700 1 --2- 172.21.15.49:0/260896265 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4dc4102d50 0x7f4dc419ec40 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:24.667 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:24.665+0000 7f4dcb2b8700 1 -- 172.21.15.49:0/260896265 >> 172.21.15.49:0/260896265 conn(0x7f4dc4079460 msgr2=0x7f4dc4101110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:07:24.667 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:24.665+0000 7f4dcb2b8700 1 -- 172.21.15.49:0/260896265 shutdown_connections 2024-03-18T23:07:24.668 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:24.665+0000 7f4dcb2b8700 1 -- 172.21.15.49:0/260896265 wait complete. 2024-03-18T23:07:24.668 INFO:teuthology.orchestra.run.smithi049.stderr:dumped all 2024-03-18T23:07:25.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:07:24 smithi139 ceph-mon[67282]: pgmap v330: 97 pgs: 97 active+clean; 641 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-18T23:07:25.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:07:24 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:07:25.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:07:24 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:07:25.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:07:24 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:07:25.316 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:07:24 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:07:25.316 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:07:24 smithi139 ceph-mon[67282]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:07:25.316 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:07:24 smithi139 ceph-mon[67282]: from='client.24766 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-18T23:07:25.326 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:07:24 smithi049 ceph-mon[62991]: pgmap v330: 97 pgs: 97 active+clean; 641 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-18T23:07:25.326 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:07:24 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:07:25.326 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:07:24 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:07:25.326 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:07:24 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:07:25.326 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:07:24 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:07:25.326 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:07:24 smithi049 ceph-mon[62991]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:07:25.326 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:07:24 smithi049 ceph-mon[62991]: from='client.24766 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-18T23:07:25.353 INFO:teuthology.orchestra.run.smithi049.stdout:{"pg_ready":true,"pg_map":{"version":330,"stamp":"2024-03-18T23:07:23.917620+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":656880,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":823,"num_read_kb":558,"num_write":418,"num_write_kb":3070,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":1182879,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":343,"ondisk_log_size":343,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":369396,"kb_used_data":11844,"kb_used_omap":18,"kb_used_meta":357357,"kb_avail":1124572428,"statfs":{"total":1151940427776,"available":1151562166272,"internally_reserved":0,"allocated":12128256,"data_stored":7507596,"data_compressed":113904,"data_compressed_allocated":147456,"data_compressed_original":1794048,"omap_allocated":19142,"internal_metadata":365933882},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":105,"num_read_kb":50,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003419"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:01.341507+0000","last_change":"2024-03-18T23:06:04.913788+0000","last_active":"2024-03-18T23:07:01.341507+0000","last_peered":"2024-03-18T23:07:01.341507+0000","last_clean":"2024-03-18T23:07:01.341507+0000","last_became_active":"2024-03-18T23:06:04.913103+0000","last_became_peered":"2024-03-18T23:06:04.913103+0000","last_unstale":"2024-03-18T23:07:01.341507+0000","last_undegraded":"2024-03-18T23:07:01.341507+0000","last_fullsized":"2024-03-18T23:07:01.341507+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:48:10.100291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:01.343786+0000","last_change":"2024-03-18T23:06:04.921126+0000","last_active":"2024-03-18T23:07:01.343786+0000","last_peered":"2024-03-18T23:07:01.343786+0000","last_clean":"2024-03-18T23:07:01.343786+0000","last_became_active":"2024-03-18T23:06:04.919934+0000","last_became_peered":"2024-03-18T23:06:04.919934+0000","last_unstale":"2024-03-18T23:07:01.343786+0000","last_undegraded":"2024-03-18T23:07:01.343786+0000","last_fullsized":"2024-03-18T23:07:01.343786+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:24:25.351428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:01.344010+0000","last_change":"2024-03-18T23:06:04.920933+0000","last_active":"2024-03-18T23:07:01.344010+0000","last_peered":"2024-03-18T23:07:01.344010+0000","last_clean":"2024-03-18T23:07:01.344010+0000","last_became_active":"2024-03-18T23:06:04.920548+0000","last_became_peered":"2024-03-18T23:06:04.920548+0000","last_unstale":"2024-03-18T23:07:01.344010+0000","last_undegraded":"2024-03-18T23:07:01.344010+0000","last_fullsized":"2024-03-18T23:07:01.344010+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:06:01.746208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:01.344330+0000","last_change":"2024-03-18T23:06:04.921219+0000","last_active":"2024-03-18T23:07:01.344330+0000","last_peered":"2024-03-18T23:07:01.344330+0000","last_clean":"2024-03-18T23:07:01.344330+0000","last_became_active":"2024-03-18T23:06:04.920069+0000","last_became_peered":"2024-03-18T23:06:04.920069+0000","last_unstale":"2024-03-18T23:07:01.344330+0000","last_undegraded":"2024-03-18T23:07:01.344330+0000","last_fullsized":"2024-03-18T23:07:01.344330+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:22:44.463061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:01.346564+0000","last_change":"2024-03-18T23:06:04.916741+0000","last_active":"2024-03-18T23:07:01.346564+0000","last_peered":"2024-03-18T23:07:01.346564+0000","last_clean":"2024-03-18T23:07:01.346564+0000","last_became_active":"2024-03-18T23:06:04.916434+0000","last_became_peered":"2024-03-18T23:06:04.916434+0000","last_unstale":"2024-03-18T23:07:01.346564+0000","last_undegraded":"2024-03-18T23:07:01.346564+0000","last_fullsized":"2024-03-18T23:07:01.346564+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:45:10.553306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:01.346188+0000","last_change":"2024-03-18T23:06:04.917198+0000","last_active":"2024-03-18T23:07:01.346188+0000","last_peered":"2024-03-18T23:07:01.346188+0000","last_clean":"2024-03-18T23:07:01.346188+0000","last_became_active":"2024-03-18T23:06:04.916897+0000","last_became_peered":"2024-03-18T23:06:04.916897+0000","last_unstale":"2024-03-18T23:07:01.346188+0000","last_undegraded":"2024-03-18T23:07:01.346188+0000","last_fullsized":"2024-03-18T23:07:01.346188+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T23:23:26.530668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:01.343844+0000","last_change":"2024-03-18T23:06:04.908446+0000","last_active":"2024-03-18T23:07:01.343844+0000","last_peered":"2024-03-18T23:07:01.343844+0000","last_clean":"2024-03-18T23:07:01.343844+0000","last_became_active":"2024-03-18T23:06:04.908093+0000","last_became_peered":"2024-03-18T23:06:04.908093+0000","last_unstale":"2024-03-18T23:07:01.343844+0000","last_undegraded":"2024-03-18T23:07:01.343844+0000","last_fullsized":"2024-03-18T23:07:01.343844+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T09:21:30.253229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:01.345242+0000","last_change":"2024-03-18T23:06:04.913080+0000","last_active":"2024-03-18T23:07:01.345242+0000","last_peered":"2024-03-18T23:07:01.345242+0000","last_clean":"2024-03-18T23:07:01.345242+0000","last_became_active":"2024-03-18T23:06:04.912255+0000","last_became_peered":"2024-03-18T23:06:04.912255+0000","last_unstale":"2024-03-18T23:07:01.345242+0000","last_undegraded":"2024-03-18T23:07:01.345242+0000","last_fullsized":"2024-03-18T23:07:01.345242+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:16:36.069545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:01.344342+0000","last_change":"2024-03-18T23:06:04.905284+0000","last_active":"2024-03-18T23:07:01.344342+0000","last_peered":"2024-03-18T23:07:01.344342+0000","last_clean":"2024-03-18T23:07:01.344342+0000","last_became_active":"2024-03-18T23:06:04.904388+0000","last_became_peered":"2024-03-18T23:06:04.904388+0000","last_unstale":"2024-03-18T23:07:01.344342+0000","last_undegraded":"2024-03-18T23:07:01.344342+0000","last_fullsized":"2024-03-18T23:07:01.344342+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:57:56.628599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:01.346070+0000","last_change":"2024-03-18T23:06:04.914602+0000","last_active":"2024-03-18T23:07:01.346070+0000","last_peered":"2024-03-18T23:07:01.346070+0000","last_clean":"2024-03-18T23:07:01.346070+0000","last_became_active":"2024-03-18T23:06:04.914288+0000","last_became_peered":"2024-03-18T23:06:04.914288+0000","last_unstale":"2024-03-18T23:07:01.346070+0000","last_undegraded":"2024-03-18T23:07:01.346070+0000","last_fullsized":"2024-03-18T23:07:01.346070+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:44:52.182950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:01.344985+0000","last_change":"2024-03-18T23:06:04.919821+0000","last_active":"2024-03-18T23:07:01.344985+0000","last_peered":"2024-03-18T23:07:01.344985+0000","last_clean":"2024-03-18T23:07:01.344985+0000","last_became_active":"2024-03-18T23:06:04.919525+0000","last_became_peered":"2024-03-18T23:06:04.919525+0000","last_unstale":"2024-03-18T23:07:01.344985+0000","last_undegraded":"2024-03-18T23:07:01.344985+0000","last_fullsized":"2024-03-18T23:07:01.344985+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:54:50.720229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:01.350429+0000","last_change":"2024-03-18T23:07:01.350429+0000","last_active":"2024-03-18T23:07:01.350429+0000","last_peered":"2024-03-18T23:07:01.350429+0000","last_clean":"2024-03-18T23:07:01.350429+0000","last_became_active":"2024-03-18T23:07:01.350040+0000","last_became_peered":"2024-03-18T23:07:01.350040+0000","last_unstale":"2024-03-18T23:07:01.350429+0000","last_undegraded":"2024-03-18T23:07:01.350429+0000","last_fullsized":"2024-03-18T23:07:01.350429+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:35:05.048875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:01.343529+0000","last_change":"2024-03-18T23:06:04.905399+0000","last_active":"2024-03-18T23:07:01.343529+0000","last_peered":"2024-03-18T23:07:01.343529+0000","last_clean":"2024-03-18T23:07:01.343529+0000","last_became_active":"2024-03-18T23:06:04.905117+0000","last_became_peered":"2024-03-18T23:06:04.905117+0000","last_unstale":"2024-03-18T23:07:01.343529+0000","last_undegraded":"2024-03-18T23:07:01.343529+0000","last_fullsized":"2024-03-18T23:07:01.343529+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:41:56.382244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:01.343615+0000","last_change":"2024-03-18T23:06:04.909993+0000","last_active":"2024-03-18T23:07:01.343615+0000","last_peered":"2024-03-18T23:07:01.343615+0000","last_clean":"2024-03-18T23:07:01.343615+0000","last_became_active":"2024-03-18T23:06:04.909483+0000","last_became_peered":"2024-03-18T23:06:04.909483+0000","last_unstale":"2024-03-18T23:07:01.343615+0000","last_undegraded":"2024-03-18T23:07:01.343615+0000","last_fullsized":"2024-03-18T23:07:01.343615+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:36:52.826582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:01.343772+0000","last_change":"2024-03-18T23:06:04.915273+0000","last_active":"2024-03-18T23:07:01.343772+0000","last_peered":"2024-03-18T23:07:01.343772+0000","last_clean":"2024-03-18T23:07:01.343772+0000","last_became_active":"2024-03-18T23:06:04.913510+0000","last_became_peered":"2024-03-18T23:06:04.913510+0000","last_unstale":"2024-03-18T23:07:01.343772+0000","last_undegraded":"2024-03-18T23:07:01.343772+0000","last_fullsized":"2024-03-18T23:07:01.343772+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:46:01.622963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:01.343699+0000","last_change":"2024-03-18T23:06:04.917599+0000","last_active":"2024-03-18T23:07:01.343699+0000","last_peered":"2024-03-18T23:07:01.343699+0000","last_clean":"2024-03-18T23:07:01.343699+0000","last_became_active":"2024-03-18T23:06:04.914961+0000","last_became_peered":"2024-03-18T23:06:04.914961+0000","last_unstale":"2024-03-18T23:07:01.343699+0000","last_undegraded":"2024-03-18T23:07:01.343699+0000","last_fullsized":"2024-03-18T23:07:01.343699+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T09:38:50.076075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:01.341718+0000","last_change":"2024-03-18T23:06:04.920814+0000","last_active":"2024-03-18T23:07:01.341718+0000","last_peered":"2024-03-18T23:07:01.341718+0000","last_clean":"2024-03-18T23:07:01.341718+0000","last_became_active":"2024-03-18T23:06:04.920484+0000","last_became_peered":"2024-03-18T23:06:04.920484+0000","last_unstale":"2024-03-18T23:07:01.341718+0000","last_undegraded":"2024-03-18T23:07:01.341718+0000","last_fullsized":"2024-03-18T23:07:01.341718+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:45:01.545059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:01.340689+0000","last_change":"2024-03-18T23:06:04.904684+0000","last_active":"2024-03-18T23:07:01.340689+0000","last_peered":"2024-03-18T23:07:01.340689+0000","last_clean":"2024-03-18T23:07:01.340689+0000","last_became_active":"2024-03-18T23:06:04.904296+0000","last_became_peered":"2024-03-18T23:06:04.904296+0000","last_unstale":"2024-03-18T23:07:01.340689+0000","last_undegraded":"2024-03-18T23:07:01.340689+0000","last_fullsized":"2024-03-18T23:07:01.340689+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:26:28.098286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:01.344254+0000","last_change":"2024-03-18T23:06:04.912803+0000","last_active":"2024-03-18T23:07:01.344254+0000","last_peered":"2024-03-18T23:07:01.344254+0000","last_clean":"2024-03-18T23:07:01.344254+0000","last_became_active":"2024-03-18T23:06:04.912495+0000","last_became_peered":"2024-03-18T23:06:04.912495+0000","last_unstale":"2024-03-18T23:07:01.344254+0000","last_undegraded":"2024-03-18T23:07:01.344254+0000","last_fullsized":"2024-03-18T23:07:01.344254+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:47:38.111260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"86'13","reported_seq":102,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:19.080834+0000","last_change":"2024-03-18T23:06:04.221492+0000","last_active":"2024-03-18T23:07:19.080834+0000","last_peered":"2024-03-18T23:07:19.080834+0000","last_clean":"2024-03-18T23:07:19.080834+0000","last_became_active":"2024-03-18T23:06:04.221190+0000","last_became_peered":"2024-03-18T23:06:04.221190+0000","last_unstale":"2024-03-18T23:07:19.080834+0000","last_undegraded":"2024-03-18T23:07:19.080834+0000","last_fullsized":"2024-03-18T23:07:19.080834+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_clean_scrub_stamp":"2024-03-18T23:06:03.184668+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:45:08.747471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":70,"num_read_kb":70,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:01.345953+0000","last_change":"2024-03-18T23:06:04.911680+0000","last_active":"2024-03-18T23:07:01.345953+0000","last_peered":"2024-03-18T23:07:01.345953+0000","last_clean":"2024-03-18T23:07:01.345953+0000","last_became_active":"2024-03-18T23:06:04.911356+0000","last_became_peered":"2024-03-18T23:06:04.911356+0000","last_unstale":"2024-03-18T23:07:01.345953+0000","last_undegraded":"2024-03-18T23:07:01.345953+0000","last_fullsized":"2024-03-18T23:07:01.345953+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:23:21.943304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:01.346096+0000","last_change":"2024-03-18T23:06:04.225792+0000","last_active":"2024-03-18T23:07:01.346096+0000","last_peered":"2024-03-18T23:07:01.346096+0000","last_clean":"2024-03-18T23:07:01.346096+0000","last_became_active":"2024-03-18T23:06:04.224248+0000","last_became_peered":"2024-03-18T23:06:04.224248+0000","last_unstale":"2024-03-18T23:07:01.346096+0000","last_undegraded":"2024-03-18T23:07:01.346096+0000","last_fullsized":"2024-03-18T23:07:01.346096+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_clean_scrub_stamp":"2024-03-18T23:06:03.184668+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:48:27.052831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:01.344246+0000","last_change":"2024-03-18T23:06:04.921139+0000","last_active":"2024-03-18T23:07:01.344246+0000","last_peered":"2024-03-18T23:07:01.344246+0000","last_clean":"2024-03-18T23:07:01.344246+0000","last_became_active":"2024-03-18T23:06:04.920700+0000","last_became_peered":"2024-03-18T23:06:04.920700+0000","last_unstale":"2024-03-18T23:07:01.344246+0000","last_undegraded":"2024-03-18T23:07:01.344246+0000","last_fullsized":"2024-03-18T23:07:01.344246+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T08:47:34.465176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"86'12","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:01.347823+0000","last_change":"2024-03-18T23:06:04.214134+0000","last_active":"2024-03-18T23:07:01.347823+0000","last_peered":"2024-03-18T23:07:01.347823+0000","last_clean":"2024-03-18T23:07:01.347823+0000","last_became_active":"2024-03-18T23:06:04.212256+0000","last_became_peered":"2024-03-18T23:06:04.212256+0000","last_unstale":"2024-03-18T23:07:01.347823+0000","last_undegraded":"2024-03-18T23:07:01.347823+0000","last_fullsized":"2024-03-18T23:07:01.347823+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_clean_scrub_stamp":"2024-03-18T23:06:03.184668+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:39:46.851964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:01.343878+0000","last_change":"2024-03-18T23:06:04.907465+0000","last_active":"2024-03-18T23:07:01.343878+0000","last_peered":"2024-03-18T23:07:01.343878+0000","last_clean":"2024-03-18T23:07:01.343878+0000","last_became_active":"2024-03-18T23:06:04.907142+0000","last_became_peered":"2024-03-18T23:06:04.907142+0000","last_unstale":"2024-03-18T23:07:01.343878+0000","last_undegraded":"2024-03-18T23:07:01.343878+0000","last_fullsized":"2024-03-18T23:07:01.343878+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:15:29.066708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"86'5","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:01.354353+0000","last_change":"2024-03-18T23:07:01.353908+0000","last_active":"2024-03-18T23:07:01.354353+0000","last_peered":"2024-03-18T23:07:01.354353+0000","last_clean":"2024-03-18T23:07:01.354353+0000","last_became_active":"2024-03-18T23:07:01.345254+0000","last_became_peered":"2024-03-18T23:07:01.345254+0000","last_unstale":"2024-03-18T23:07:01.354353+0000","last_undegraded":"2024-03-18T23:07:01.354353+0000","last_fullsized":"2024-03-18T23:07:01.354353+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_clean_scrub_stamp":"2024-03-18T23:06:03.184668+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:48:56.474966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:01.343607+0000","last_change":"2024-03-18T23:06:04.918596+0000","last_active":"2024-03-18T23:07:01.343607+0000","last_peered":"2024-03-18T23:07:01.343607+0000","last_clean":"2024-03-18T23:07:01.343607+0000","last_became_active":"2024-03-18T23:06:04.918273+0000","last_became_peered":"2024-03-18T23:06:04.918273+0000","last_unstale":"2024-03-18T23:07:01.343607+0000","last_undegraded":"2024-03-18T23:07:01.343607+0000","last_fullsized":"2024-03-18T23:07:01.343607+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T08:09:04.860241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"86'5","reported_seq":101,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:22.059077+0000","last_change":"2024-03-18T23:06:04.214573+0000","last_active":"2024-03-18T23:07:22.059077+0000","last_peered":"2024-03-18T23:07:22.059077+0000","last_clean":"2024-03-18T23:07:22.059077+0000","last_became_active":"2024-03-18T23:06:04.213891+0000","last_became_peered":"2024-03-18T23:06:04.213891+0000","last_unstale":"2024-03-18T23:07:22.059077+0000","last_undegraded":"2024-03-18T23:07:22.059077+0000","last_fullsized":"2024-03-18T23:07:22.059077+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_clean_scrub_stamp":"2024-03-18T23:06:03.184668+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:17:22.895024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":77,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:01.346271+0000","last_change":"2024-03-18T23:06:04.908336+0000","last_active":"2024-03-18T23:07:01.346271+0000","last_peered":"2024-03-18T23:07:01.346271+0000","last_clean":"2024-03-18T23:07:01.346271+0000","last_became_active":"2024-03-18T23:06:04.908023+0000","last_became_peered":"2024-03-18T23:06:04.908023+0000","last_unstale":"2024-03-18T23:07:01.346271+0000","last_undegraded":"2024-03-18T23:07:01.346271+0000","last_fullsized":"2024-03-18T23:07:01.346271+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:34:50.993715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"86'7","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:01.346058+0000","last_change":"2024-03-18T23:06:04.210036+0000","last_active":"2024-03-18T23:07:01.346058+0000","last_peered":"2024-03-18T23:07:01.346058+0000","last_clean":"2024-03-18T23:07:01.346058+0000","last_became_active":"2024-03-18T23:06:04.209658+0000","last_became_peered":"2024-03-18T23:06:04.209658+0000","last_unstale":"2024-03-18T23:07:01.346058+0000","last_undegraded":"2024-03-18T23:07:01.346058+0000","last_fullsized":"2024-03-18T23:07:01.346058+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_clean_scrub_stamp":"2024-03-18T23:06:03.184668+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T23:55:37.286542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:01.343424+0000","last_change":"2024-03-18T23:06:04.917909+0000","last_active":"2024-03-18T23:07:01.343424+0000","last_peered":"2024-03-18T23:07:01.343424+0000","last_clean":"2024-03-18T23:07:01.343424+0000","last_became_active":"2024-03-18T23:06:04.917305+0000","last_became_peered":"2024-03-18T23:06:04.917305+0000","last_unstale":"2024-03-18T23:07:01.343424+0000","last_undegraded":"2024-03-18T23:07:01.343424+0000","last_fullsized":"2024-03-18T23:07:01.343424+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:39:48.953530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"86'3","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:01.343454+0000","last_change":"2024-03-18T23:06:04.216030+0000","last_active":"2024-03-18T23:07:01.343454+0000","last_peered":"2024-03-18T23:07:01.343454+0000","last_clean":"2024-03-18T23:07:01.343454+0000","last_became_active":"2024-03-18T23:06:04.215695+0000","last_became_peered":"2024-03-18T23:06:04.215695+0000","last_unstale":"2024-03-18T23:07:01.343454+0000","last_undegraded":"2024-03-18T23:07:01.343454+0000","last_fullsized":"2024-03-18T23:07:01.343454+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_clean_scrub_stamp":"2024-03-18T23:06:03.184668+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:48:37.742916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:01.345348+0000","last_change":"2024-03-18T23:06:04.910749+0000","last_active":"2024-03-18T23:07:01.345348+0000","last_peered":"2024-03-18T23:07:01.345348+0000","last_clean":"2024-03-18T23:07:01.345348+0000","last_became_active":"2024-03-18T23:06:04.910313+0000","last_became_peered":"2024-03-18T23:06:04.910313+0000","last_unstale":"2024-03-18T23:07:01.345348+0000","last_undegraded":"2024-03-18T23:07:01.345348+0000","last_fullsized":"2024-03-18T23:07:01.345348+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:18:19.726537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"86'3","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:01.342902+0000","last_change":"2024-03-18T23:06:04.218263+0000","last_active":"2024-03-18T23:07:01.342902+0000","last_peered":"2024-03-18T23:07:01.342902+0000","last_clean":"2024-03-18T23:07:01.342902+0000","last_became_active":"2024-03-18T23:06:04.217943+0000","last_became_peered":"2024-03-18T23:06:04.217943+0000","last_unstale":"2024-03-18T23:07:01.342902+0000","last_undegraded":"2024-03-18T23:07:01.342902+0000","last_fullsized":"2024-03-18T23:07:01.342902+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_clean_scrub_stamp":"2024-03-18T23:06:03.184668+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:25:11.632767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:01.340341+0000","last_change":"2024-03-18T23:06:04.915905+0000","last_active":"2024-03-18T23:07:01.340341+0000","last_peered":"2024-03-18T23:07:01.340341+0000","last_clean":"2024-03-18T23:07:01.340341+0000","last_became_active":"2024-03-18T23:06:04.915584+0000","last_became_peered":"2024-03-18T23:06:04.915584+0000","last_unstale":"2024-03-18T23:07:01.340341+0000","last_undegraded":"2024-03-18T23:07:01.340341+0000","last_fullsized":"2024-03-18T23:07:01.340341+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:29:05.503861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"86'4","reported_seq":166,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:21.057547+0000","last_change":"2024-03-18T23:06:04.217077+0000","last_active":"2024-03-18T23:07:21.057547+0000","last_peered":"2024-03-18T23:07:21.057547+0000","last_clean":"2024-03-18T23:07:21.057547+0000","last_became_active":"2024-03-18T23:06:04.216308+0000","last_became_peered":"2024-03-18T23:06:04.216308+0000","last_unstale":"2024-03-18T23:07:21.057547+0000","last_undegraded":"2024-03-18T23:07:21.057547+0000","last_fullsized":"2024-03-18T23:07:21.057547+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_clean_scrub_stamp":"2024-03-18T23:06:03.184668+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T23:39:58.537740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":143,"num_read_kb":143,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:01.344160+0000","last_change":"2024-03-18T23:06:04.906669+0000","last_active":"2024-03-18T23:07:01.344160+0000","last_peered":"2024-03-18T23:07:01.344160+0000","last_clean":"2024-03-18T23:07:01.344160+0000","last_became_active":"2024-03-18T23:06:04.906338+0000","last_became_peered":"2024-03-18T23:06:04.906338+0000","last_unstale":"2024-03-18T23:07:01.344160+0000","last_undegraded":"2024-03-18T23:07:01.344160+0000","last_fullsized":"2024-03-18T23:07:01.344160+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:29:39.997671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"86'4","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:01.344846+0000","last_change":"2024-03-18T23:06:04.216177+0000","last_active":"2024-03-18T23:07:01.344846+0000","last_peered":"2024-03-18T23:07:01.344846+0000","last_clean":"2024-03-18T23:07:01.344846+0000","last_became_active":"2024-03-18T23:06:04.215890+0000","last_became_peered":"2024-03-18T23:06:04.215890+0000","last_unstale":"2024-03-18T23:07:01.344846+0000","last_undegraded":"2024-03-18T23:07:01.344846+0000","last_fullsized":"2024-03-18T23:07:01.344846+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_clean_scrub_stamp":"2024-03-18T23:06:03.184668+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T09:33:52.578996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:01.344216+0000","last_change":"2024-03-18T23:06:04.911928+0000","last_active":"2024-03-18T23:07:01.344216+0000","last_peered":"2024-03-18T23:07:01.344216+0000","last_clean":"2024-03-18T23:07:01.344216+0000","last_became_active":"2024-03-18T23:06:04.911501+0000","last_became_peered":"2024-03-18T23:06:04.911501+0000","last_unstale":"2024-03-18T23:07:01.344216+0000","last_undegraded":"2024-03-18T23:07:01.344216+0000","last_fullsized":"2024-03-18T23:07:01.344216+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:05:17.653574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"86'5","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:01.343657+0000","last_change":"2024-03-18T23:06:04.211035+0000","last_active":"2024-03-18T23:07:01.343657+0000","last_peered":"2024-03-18T23:07:01.343657+0000","last_clean":"2024-03-18T23:07:01.343657+0000","last_became_active":"2024-03-18T23:06:04.210665+0000","last_became_peered":"2024-03-18T23:06:04.210665+0000","last_unstale":"2024-03-18T23:07:01.343657+0000","last_undegraded":"2024-03-18T23:07:01.343657+0000","last_fullsized":"2024-03-18T23:07:01.343657+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_clean_scrub_stamp":"2024-03-18T23:06:03.184668+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:49:44.072275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:01.344075+0000","last_change":"2024-03-18T23:06:04.911856+0000","last_active":"2024-03-18T23:07:01.344075+0000","last_peered":"2024-03-18T23:07:01.344075+0000","last_clean":"2024-03-18T23:07:01.344075+0000","last_became_active":"2024-03-18T23:06:04.908034+0000","last_became_peered":"2024-03-18T23:06:04.908034+0000","last_unstale":"2024-03-18T23:07:01.344075+0000","last_undegraded":"2024-03-18T23:07:01.344075+0000","last_fullsized":"2024-03-18T23:07:01.344075+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:38:09.712683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"86'29","reported_seq":201,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:21.057452+0000","last_change":"2024-03-18T23:06:04.211152+0000","last_active":"2024-03-18T23:07:21.057452+0000","last_peered":"2024-03-18T23:07:21.057452+0000","last_clean":"2024-03-18T23:07:21.057452+0000","last_became_active":"2024-03-18T23:06:04.210841+0000","last_became_peered":"2024-03-18T23:06:04.210841+0000","last_unstale":"2024-03-18T23:07:21.057452+0000","last_undegraded":"2024-03-18T23:07:21.057452+0000","last_fullsized":"2024-03-18T23:07:21.057452+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_clean_scrub_stamp":"2024-03-18T23:06:03.184668+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:48:07.935314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":59465,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":153,"num_read_kb":134,"num_write":31,"num_write_kb":67,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:01.347556+0000","last_change":"2024-03-18T23:06:04.910303+0000","last_active":"2024-03-18T23:07:01.347556+0000","last_peered":"2024-03-18T23:07:01.347556+0000","last_clean":"2024-03-18T23:07:01.347556+0000","last_became_active":"2024-03-18T23:06:04.910004+0000","last_became_peered":"2024-03-18T23:06:04.910004+0000","last_unstale":"2024-03-18T23:07:01.347556+0000","last_undegraded":"2024-03-18T23:07:01.347556+0000","last_fullsized":"2024-03-18T23:07:01.347556+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:45:25.750741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"86'9","reported_seq":30,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:01.343486+0000","last_change":"2024-03-18T23:06:04.220318+0000","last_active":"2024-03-18T23:07:01.343486+0000","last_peered":"2024-03-18T23:07:01.343486+0000","last_clean":"2024-03-18T23:07:01.343486+0000","last_became_active":"2024-03-18T23:06:04.219985+0000","last_became_peered":"2024-03-18T23:06:04.219985+0000","last_unstale":"2024-03-18T23:07:01.343486+0000","last_undegraded":"2024-03-18T23:07:01.343486+0000","last_fullsized":"2024-03-18T23:07:01.343486+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_clean_scrub_stamp":"2024-03-18T23:06:03.184668+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:49:00.090356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:01.343340+0000","last_change":"2024-03-18T23:06:04.914456+0000","last_active":"2024-03-18T23:07:01.343340+0000","last_peered":"2024-03-18T23:07:01.343340+0000","last_clean":"2024-03-18T23:07:01.343340+0000","last_became_active":"2024-03-18T23:06:04.914083+0000","last_became_peered":"2024-03-18T23:06:04.914083+0000","last_unstale":"2024-03-18T23:07:01.343340+0000","last_undegraded":"2024-03-18T23:07:01.343340+0000","last_fullsized":"2024-03-18T23:07:01.343340+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:17:00.443254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"86'5","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:01.343123+0000","last_change":"2024-03-18T23:06:04.224808+0000","last_active":"2024-03-18T23:07:01.343123+0000","last_peered":"2024-03-18T23:07:01.343123+0000","last_clean":"2024-03-18T23:07:01.343123+0000","last_became_active":"2024-03-18T23:06:04.224112+0000","last_became_peered":"2024-03-18T23:06:04.224112+0000","last_unstale":"2024-03-18T23:07:01.343123+0000","last_undegraded":"2024-03-18T23:07:01.343123+0000","last_fullsized":"2024-03-18T23:07:01.343123+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_clean_scrub_stamp":"2024-03-18T23:06:03.184668+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:19:20.399875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:01.344917+0000","last_change":"2024-03-18T23:06:04.912271+0000","last_active":"2024-03-18T23:07:01.344917+0000","last_peered":"2024-03-18T23:07:01.344917+0000","last_clean":"2024-03-18T23:07:01.344917+0000","last_became_active":"2024-03-18T23:06:04.911985+0000","last_became_peered":"2024-03-18T23:06:04.911985+0000","last_unstale":"2024-03-18T23:07:01.344917+0000","last_undegraded":"2024-03-18T23:07:01.344917+0000","last_fullsized":"2024-03-18T23:07:01.344917+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T09:18:27.436136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"86'5","reported_seq":94,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:18.080228+0000","last_change":"2024-03-18T23:06:04.209434+0000","last_active":"2024-03-18T23:07:18.080228+0000","last_peered":"2024-03-18T23:07:18.080228+0000","last_clean":"2024-03-18T23:07:18.080228+0000","last_became_active":"2024-03-18T23:06:04.209086+0000","last_became_peered":"2024-03-18T23:06:04.209086+0000","last_unstale":"2024-03-18T23:07:18.080228+0000","last_undegraded":"2024-03-18T23:07:18.080228+0000","last_fullsized":"2024-03-18T23:07:18.080228+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_clean_scrub_stamp":"2024-03-18T23:06:03.184668+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:50:20.823467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":71,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:01.343494+0000","last_change":"2024-03-18T23:06:04.911069+0000","last_active":"2024-03-18T23:07:01.343494+0000","last_peered":"2024-03-18T23:07:01.343494+0000","last_clean":"2024-03-18T23:07:01.343494+0000","last_became_active":"2024-03-18T23:06:04.910760+0000","last_became_peered":"2024-03-18T23:06:04.910760+0000","last_unstale":"2024-03-18T23:07:01.343494+0000","last_undegraded":"2024-03-18T23:07:01.343494+0000","last_fullsized":"2024-03-18T23:07:01.343494+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:16:46.264972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"86'17","reported_seq":37,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:01.345915+0000","last_change":"2024-03-18T23:06:04.216653+0000","last_active":"2024-03-18T23:07:01.345915+0000","last_peered":"2024-03-18T23:07:01.345915+0000","last_clean":"2024-03-18T23:07:01.345915+0000","last_became_active":"2024-03-18T23:06:04.216358+0000","last_became_peered":"2024-03-18T23:06:04.216358+0000","last_unstale":"2024-03-18T23:07:01.345915+0000","last_undegraded":"2024-03-18T23:07:01.345915+0000","last_fullsized":"2024-03-18T23:07:01.345915+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_clean_scrub_stamp":"2024-03-18T23:06:03.184668+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:16:56.500368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:01.342941+0000","last_change":"2024-03-18T23:06:04.917377+0000","last_active":"2024-03-18T23:07:01.342941+0000","last_peered":"2024-03-18T23:07:01.342941+0000","last_clean":"2024-03-18T23:07:01.342941+0000","last_became_active":"2024-03-18T23:06:04.917064+0000","last_became_peered":"2024-03-18T23:06:04.917064+0000","last_unstale":"2024-03-18T23:07:01.342941+0000","last_undegraded":"2024-03-18T23:07:01.342941+0000","last_fullsized":"2024-03-18T23:07:01.342941+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:03:41.518177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"86'2","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:01.348134+0000","last_change":"2024-03-18T23:06:04.210191+0000","last_active":"2024-03-18T23:07:01.348134+0000","last_peered":"2024-03-18T23:07:01.348134+0000","last_clean":"2024-03-18T23:07:01.348134+0000","last_became_active":"2024-03-18T23:06:04.208084+0000","last_became_peered":"2024-03-18T23:06:04.208084+0000","last_unstale":"2024-03-18T23:07:01.348134+0000","last_undegraded":"2024-03-18T23:07:01.348134+0000","last_fullsized":"2024-03-18T23:07:01.348134+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_clean_scrub_stamp":"2024-03-18T23:06:03.184668+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:09:07.220171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:01.341378+0000","last_change":"2024-03-18T23:06:04.910631+0000","last_active":"2024-03-18T23:07:01.341378+0000","last_peered":"2024-03-18T23:07:01.341378+0000","last_clean":"2024-03-18T23:07:01.341378+0000","last_became_active":"2024-03-18T23:06:04.910322+0000","last_became_peered":"2024-03-18T23:06:04.910322+0000","last_unstale":"2024-03-18T23:07:01.341378+0000","last_undegraded":"2024-03-18T23:07:01.341378+0000","last_fullsized":"2024-03-18T23:07:01.341378+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:39:06.389101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"86'1","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:01.340500+0000","last_change":"2024-03-18T23:06:04.220530+0000","last_active":"2024-03-18T23:07:01.340500+0000","last_peered":"2024-03-18T23:07:01.340500+0000","last_clean":"2024-03-18T23:07:01.340500+0000","last_became_active":"2024-03-18T23:06:04.220136+0000","last_became_peered":"2024-03-18T23:06:04.220136+0000","last_unstale":"2024-03-18T23:07:01.340500+0000","last_undegraded":"2024-03-18T23:07:01.340500+0000","last_fullsized":"2024-03-18T23:07:01.340500+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_clean_scrub_stamp":"2024-03-18T23:06:03.184668+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:58:37.037982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"86'1","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:01.346267+0000","last_change":"2024-03-18T23:06:04.912789+0000","last_active":"2024-03-18T23:07:01.346267+0000","last_peered":"2024-03-18T23:07:01.346267+0000","last_clean":"2024-03-18T23:07:01.346267+0000","last_became_active":"2024-03-18T23:06:04.910451+0000","last_became_peered":"2024-03-18T23:06:04.910451+0000","last_unstale":"2024-03-18T23:07:01.346267+0000","last_undegraded":"2024-03-18T23:07:01.346267+0000","last_fullsized":"2024-03-18T23:07:01.346267+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:06:36.539369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"86'5","reported_seq":93,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:18.080179+0000","last_change":"2024-03-18T23:06:04.217992+0000","last_active":"2024-03-18T23:07:18.080179+0000","last_peered":"2024-03-18T23:07:18.080179+0000","last_clean":"2024-03-18T23:07:18.080179+0000","last_became_active":"2024-03-18T23:06:04.217651+0000","last_became_peered":"2024-03-18T23:06:04.217651+0000","last_unstale":"2024-03-18T23:07:18.080179+0000","last_undegraded":"2024-03-18T23:07:18.080179+0000","last_fullsized":"2024-03-18T23:07:18.080179+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_clean_scrub_stamp":"2024-03-18T23:06:03.184668+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:25:46.901251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":69,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:01.346198+0000","last_change":"2024-03-18T23:06:04.913349+0000","last_active":"2024-03-18T23:07:01.346198+0000","last_peered":"2024-03-18T23:07:01.346198+0000","last_clean":"2024-03-18T23:07:01.346198+0000","last_became_active":"2024-03-18T23:06:04.913026+0000","last_became_peered":"2024-03-18T23:06:04.913026+0000","last_unstale":"2024-03-18T23:07:01.346198+0000","last_undegraded":"2024-03-18T23:07:01.346198+0000","last_fullsized":"2024-03-18T23:07:01.346198+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T23:24:28.884524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:01.343112+0000","last_change":"2024-03-18T23:06:04.916541+0000","last_active":"2024-03-18T23:07:01.343112+0000","last_peered":"2024-03-18T23:07:01.343112+0000","last_clean":"2024-03-18T23:07:01.343112+0000","last_became_active":"2024-03-18T23:06:04.916179+0000","last_became_peered":"2024-03-18T23:06:04.916179+0000","last_unstale":"2024-03-18T23:07:01.343112+0000","last_undegraded":"2024-03-18T23:07:01.343112+0000","last_fullsized":"2024-03-18T23:07:01.343112+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:31:59.515767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"86'5","reported_seq":93,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:18.079689+0000","last_change":"2024-03-18T23:06:04.217581+0000","last_active":"2024-03-18T23:07:18.079689+0000","last_peered":"2024-03-18T23:07:18.079689+0000","last_clean":"2024-03-18T23:07:18.079689+0000","last_became_active":"2024-03-18T23:06:04.217191+0000","last_became_peered":"2024-03-18T23:06:04.217191+0000","last_unstale":"2024-03-18T23:07:18.079689+0000","last_undegraded":"2024-03-18T23:07:18.079689+0000","last_fullsized":"2024-03-18T23:07:18.079689+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_clean_scrub_stamp":"2024-03-18T23:06:03.184668+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:47:24.728136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":69,"num_read_kb":69,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:01.345960+0000","last_change":"2024-03-18T23:06:04.919784+0000","last_active":"2024-03-18T23:07:01.345960+0000","last_peered":"2024-03-18T23:07:01.345960+0000","last_clean":"2024-03-18T23:07:01.345960+0000","last_became_active":"2024-03-18T23:06:04.919483+0000","last_became_peered":"2024-03-18T23:06:04.919483+0000","last_unstale":"2024-03-18T23:07:01.345960+0000","last_undegraded":"2024-03-18T23:07:01.345960+0000","last_fullsized":"2024-03-18T23:07:01.345960+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:37:29.635750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'6","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:01.342844+0000","last_change":"2024-03-18T23:06:04.215152+0000","last_active":"2024-03-18T23:07:01.342844+0000","last_peered":"2024-03-18T23:07:01.342844+0000","last_clean":"2024-03-18T23:07:01.342844+0000","last_became_active":"2024-03-18T23:06:04.214789+0000","last_became_peered":"2024-03-18T23:06:04.214789+0000","last_unstale":"2024-03-18T23:07:01.342844+0000","last_undegraded":"2024-03-18T23:07:01.342844+0000","last_fullsized":"2024-03-18T23:07:01.342844+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_clean_scrub_stamp":"2024-03-18T23:06:03.184668+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:58:11.630840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":353,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:01.343583+0000","last_change":"2024-03-18T23:04:54.734000+0000","last_active":"2024-03-18T23:07:01.343583+0000","last_peered":"2024-03-18T23:07:01.343583+0000","last_clean":"2024-03-18T23:07:01.343583+0000","last_became_active":"2024-03-18T23:04:54.733516+0000","last_became_peered":"2024-03-18T23:04:54.733516+0000","last_unstale":"2024-03-18T23:07:01.343583+0000","last_undegraded":"2024-03-18T23:07:01.343583+0000","last_fullsized":"2024-03-18T23:07:01.343583+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:01:28.441034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:01:28.441034+0000","last_clean_scrub_stamp":"2024-03-18T23:01:28.441034+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T23:48:14.029479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:01.343133+0000","last_change":"2024-03-18T23:06:04.908549+0000","last_active":"2024-03-18T23:07:01.343133+0000","last_peered":"2024-03-18T23:07:01.343133+0000","last_clean":"2024-03-18T23:07:01.343133+0000","last_became_active":"2024-03-18T23:06:04.908202+0000","last_became_peered":"2024-03-18T23:06:04.908202+0000","last_unstale":"2024-03-18T23:07:01.343133+0000","last_undegraded":"2024-03-18T23:07:01.343133+0000","last_fullsized":"2024-03-18T23:07:01.343133+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:57:09.486103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'4","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:01.343728+0000","last_change":"2024-03-18T23:06:04.213819+0000","last_active":"2024-03-18T23:07:01.343728+0000","last_peered":"2024-03-18T23:07:01.343728+0000","last_clean":"2024-03-18T23:07:01.343728+0000","last_became_active":"2024-03-18T23:06:04.213418+0000","last_became_peered":"2024-03-18T23:06:04.213418+0000","last_unstale":"2024-03-18T23:07:01.343728+0000","last_undegraded":"2024-03-18T23:07:01.343728+0000","last_fullsized":"2024-03-18T23:07:01.343728+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_clean_scrub_stamp":"2024-03-18T23:06:03.184668+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:50:03.175779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"86'4","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:01.347657+0000","last_change":"2024-03-18T23:06:04.218660+0000","last_active":"2024-03-18T23:07:01.347657+0000","last_peered":"2024-03-18T23:07:01.347657+0000","last_clean":"2024-03-18T23:07:01.347657+0000","last_became_active":"2024-03-18T23:06:04.218183+0000","last_became_peered":"2024-03-18T23:06:04.218183+0000","last_unstale":"2024-03-18T23:07:01.347657+0000","last_undegraded":"2024-03-18T23:07:01.347657+0000","last_fullsized":"2024-03-18T23:07:01.347657+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_clean_scrub_stamp":"2024-03-18T23:06:03.184668+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T08:17:42.208572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:01.343312+0000","last_change":"2024-03-18T23:06:04.901124+0000","last_active":"2024-03-18T23:07:01.343312+0000","last_peered":"2024-03-18T23:07:01.343312+0000","last_clean":"2024-03-18T23:07:01.343312+0000","last_became_active":"2024-03-18T23:06:04.899992+0000","last_became_peered":"2024-03-18T23:06:04.899992+0000","last_unstale":"2024-03-18T23:07:01.343312+0000","last_undegraded":"2024-03-18T23:07:01.343312+0000","last_fullsized":"2024-03-18T23:07:01.343312+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:27:00.965945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"86'12","reported_seq":31,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:01.340611+0000","last_change":"2024-03-18T23:06:04.227413+0000","last_active":"2024-03-18T23:07:01.340611+0000","last_peered":"2024-03-18T23:07:01.340611+0000","last_clean":"2024-03-18T23:07:01.340611+0000","last_became_active":"2024-03-18T23:06:04.227071+0000","last_became_peered":"2024-03-18T23:06:04.227071+0000","last_unstale":"2024-03-18T23:07:01.340611+0000","last_undegraded":"2024-03-18T23:07:01.340611+0000","last_fullsized":"2024-03-18T23:07:01.340611+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_clean_scrub_stamp":"2024-03-18T23:06:03.184668+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T08:15:31.531168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:01.345733+0000","last_change":"2024-03-18T23:06:04.910220+0000","last_active":"2024-03-18T23:07:01.345733+0000","last_peered":"2024-03-18T23:07:01.345733+0000","last_clean":"2024-03-18T23:07:01.345733+0000","last_became_active":"2024-03-18T23:06:04.909855+0000","last_became_peered":"2024-03-18T23:06:04.909855+0000","last_unstale":"2024-03-18T23:07:01.345733+0000","last_undegraded":"2024-03-18T23:07:01.345733+0000","last_fullsized":"2024-03-18T23:07:01.345733+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:19:09.315343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"86'7","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:01.343638+0000","last_change":"2024-03-18T23:06:04.225473+0000","last_active":"2024-03-18T23:07:01.343638+0000","last_peered":"2024-03-18T23:07:01.343638+0000","last_clean":"2024-03-18T23:07:01.343638+0000","last_became_active":"2024-03-18T23:06:04.225126+0000","last_became_peered":"2024-03-18T23:06:04.225126+0000","last_unstale":"2024-03-18T23:07:01.343638+0000","last_undegraded":"2024-03-18T23:07:01.343638+0000","last_fullsized":"2024-03-18T23:07:01.343638+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_clean_scrub_stamp":"2024-03-18T23:06:03.184668+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T09:29:26.438597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:01.347970+0000","last_change":"2024-03-18T23:07:01.347970+0000","last_active":"2024-03-18T23:07:01.347970+0000","last_peered":"2024-03-18T23:07:01.347970+0000","last_clean":"2024-03-18T23:07:01.347970+0000","last_became_active":"2024-03-18T23:07:01.347617+0000","last_became_peered":"2024-03-18T23:07:01.347617+0000","last_unstale":"2024-03-18T23:07:01.347970+0000","last_undegraded":"2024-03-18T23:07:01.347970+0000","last_fullsized":"2024-03-18T23:07:01.347970+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:43:26.292403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:01.343830+0000","last_change":"2024-03-18T23:06:04.216750+0000","last_active":"2024-03-18T23:07:01.343830+0000","last_peered":"2024-03-18T23:07:01.343830+0000","last_clean":"2024-03-18T23:07:01.343830+0000","last_became_active":"2024-03-18T23:06:04.216393+0000","last_became_peered":"2024-03-18T23:06:04.216393+0000","last_unstale":"2024-03-18T23:07:01.343830+0000","last_undegraded":"2024-03-18T23:07:01.343830+0000","last_fullsized":"2024-03-18T23:07:01.343830+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_clean_scrub_stamp":"2024-03-18T23:06:03.184668+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:40:33.486235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:01.345809+0000","last_change":"2024-03-18T23:06:04.915156+0000","last_active":"2024-03-18T23:07:01.345809+0000","last_peered":"2024-03-18T23:07:01.345809+0000","last_clean":"2024-03-18T23:07:01.345809+0000","last_became_active":"2024-03-18T23:06:04.914821+0000","last_became_peered":"2024-03-18T23:06:04.914821+0000","last_unstale":"2024-03-18T23:07:01.345809+0000","last_undegraded":"2024-03-18T23:07:01.345809+0000","last_fullsized":"2024-03-18T23:07:01.345809+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:41:22.344360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"86'7","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:01.345613+0000","last_change":"2024-03-18T23:06:04.224533+0000","last_active":"2024-03-18T23:07:01.345613+0000","last_peered":"2024-03-18T23:07:01.345613+0000","last_clean":"2024-03-18T23:07:01.345613+0000","last_became_active":"2024-03-18T23:06:04.224122+0000","last_became_peered":"2024-03-18T23:06:04.224122+0000","last_unstale":"2024-03-18T23:07:01.345613+0000","last_undegraded":"2024-03-18T23:07:01.345613+0000","last_fullsized":"2024-03-18T23:07:01.345613+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_clean_scrub_stamp":"2024-03-18T23:06:03.184668+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T09:03:40.689085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:01.340600+0000","last_change":"2024-03-18T23:06:04.912574+0000","last_active":"2024-03-18T23:07:01.340600+0000","last_peered":"2024-03-18T23:07:01.340600+0000","last_clean":"2024-03-18T23:07:01.340600+0000","last_became_active":"2024-03-18T23:06:04.912284+0000","last_became_peered":"2024-03-18T23:06:04.912284+0000","last_unstale":"2024-03-18T23:07:01.340600+0000","last_undegraded":"2024-03-18T23:07:01.340600+0000","last_fullsized":"2024-03-18T23:07:01.340600+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:18:11.003763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"86'3","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:01.343310+0000","last_change":"2024-03-18T23:06:04.220364+0000","last_active":"2024-03-18T23:07:01.343310+0000","last_peered":"2024-03-18T23:07:01.343310+0000","last_clean":"2024-03-18T23:07:01.343310+0000","last_became_active":"2024-03-18T23:06:04.220074+0000","last_became_peered":"2024-03-18T23:06:04.220074+0000","last_unstale":"2024-03-18T23:07:01.343310+0000","last_undegraded":"2024-03-18T23:07:01.343310+0000","last_fullsized":"2024-03-18T23:07:01.343310+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_clean_scrub_stamp":"2024-03-18T23:06:03.184668+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:10:15.385724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:01.345915+0000","last_change":"2024-03-18T23:06:04.917448+0000","last_active":"2024-03-18T23:07:01.345915+0000","last_peered":"2024-03-18T23:07:01.345915+0000","last_clean":"2024-03-18T23:07:01.345915+0000","last_became_active":"2024-03-18T23:06:04.916921+0000","last_became_peered":"2024-03-18T23:06:04.916921+0000","last_unstale":"2024-03-18T23:07:01.345915+0000","last_undegraded":"2024-03-18T23:07:01.345915+0000","last_fullsized":"2024-03-18T23:07:01.345915+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:23:06.715597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:01.358216+0000","last_change":"2024-03-18T23:07:01.357779+0000","last_active":"2024-03-18T23:07:01.358216+0000","last_peered":"2024-03-18T23:07:01.358216+0000","last_clean":"2024-03-18T23:07:01.358216+0000","last_became_active":"2024-03-18T23:07:01.346447+0000","last_became_peered":"2024-03-18T23:07:01.346447+0000","last_unstale":"2024-03-18T23:07:01.358216+0000","last_undegraded":"2024-03-18T23:07:01.358216+0000","last_fullsized":"2024-03-18T23:07:01.358216+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_clean_scrub_stamp":"2024-03-18T23:06:03.184668+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:12:17.794463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:01.344313+0000","last_change":"2024-03-18T23:06:04.911471+0000","last_active":"2024-03-18T23:07:01.344313+0000","last_peered":"2024-03-18T23:07:01.344313+0000","last_clean":"2024-03-18T23:07:01.344313+0000","last_became_active":"2024-03-18T23:06:04.910550+0000","last_became_peered":"2024-03-18T23:06:04.910550+0000","last_unstale":"2024-03-18T23:07:01.344313+0000","last_undegraded":"2024-03-18T23:07:01.344313+0000","last_fullsized":"2024-03-18T23:07:01.344313+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:34:55.084581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"86'6","reported_seq":37,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:17.369452+0000","last_change":"2024-03-18T23:07:01.353076+0000","last_active":"2024-03-18T23:07:17.369452+0000","last_peered":"2024-03-18T23:07:17.369452+0000","last_clean":"2024-03-18T23:07:17.369452+0000","last_became_active":"2024-03-18T23:07:01.346752+0000","last_became_peered":"2024-03-18T23:07:01.346752+0000","last_unstale":"2024-03-18T23:07:17.369452+0000","last_undegraded":"2024-03-18T23:07:17.369452+0000","last_fullsized":"2024-03-18T23:07:17.369452+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_clean_scrub_stamp":"2024-03-18T23:06:03.184668+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:05:33.551956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1561,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":18,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":1561,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,0],"acting":[4,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:01.345083+0000","last_change":"2024-03-18T23:06:04.918955+0000","last_active":"2024-03-18T23:07:01.345083+0000","last_peered":"2024-03-18T23:07:01.345083+0000","last_clean":"2024-03-18T23:07:01.345083+0000","last_became_active":"2024-03-18T23:06:04.915475+0000","last_became_peered":"2024-03-18T23:06:04.915475+0000","last_unstale":"2024-03-18T23:07:01.345083+0000","last_undegraded":"2024-03-18T23:07:01.345083+0000","last_fullsized":"2024-03-18T23:07:01.345083+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:07:05.679235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"86'3","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:01.343552+0000","last_change":"2024-03-18T23:06:04.224237+0000","last_active":"2024-03-18T23:07:01.343552+0000","last_peered":"2024-03-18T23:07:01.343552+0000","last_clean":"2024-03-18T23:07:01.343552+0000","last_became_active":"2024-03-18T23:06:04.223820+0000","last_became_peered":"2024-03-18T23:06:04.223820+0000","last_unstale":"2024-03-18T23:07:01.343552+0000","last_undegraded":"2024-03-18T23:07:01.343552+0000","last_fullsized":"2024-03-18T23:07:01.343552+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_clean_scrub_stamp":"2024-03-18T23:06:03.184668+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:17:48.157499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"86'1","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:01.343424+0000","last_change":"2024-03-18T23:06:04.914846+0000","last_active":"2024-03-18T23:07:01.343424+0000","last_peered":"2024-03-18T23:07:01.343424+0000","last_clean":"2024-03-18T23:07:01.343424+0000","last_became_active":"2024-03-18T23:06:04.911629+0000","last_became_peered":"2024-03-18T23:06:04.911629+0000","last_unstale":"2024-03-18T23:07:01.343424+0000","last_undegraded":"2024-03-18T23:07:01.343424+0000","last_fullsized":"2024-03-18T23:07:01.343424+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:17:32.235568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"86'5","reported_seq":36,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:01.356827+0000","last_change":"2024-03-18T23:07:01.356827+0000","last_active":"2024-03-18T23:07:01.356827+0000","last_peered":"2024-03-18T23:07:01.356827+0000","last_clean":"2024-03-18T23:07:01.356827+0000","last_became_active":"2024-03-18T23:07:01.347750+0000","last_became_peered":"2024-03-18T23:07:01.347750+0000","last_unstale":"2024-03-18T23:07:01.356827+0000","last_undegraded":"2024-03-18T23:07:01.356827+0000","last_fullsized":"2024-03-18T23:07:01.356827+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_clean_scrub_stamp":"2024-03-18T23:06:03.184668+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:35:07.291013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,1],"acting":[6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:01.345190+0000","last_change":"2024-03-18T23:06:04.914463+0000","last_active":"2024-03-18T23:07:01.345190+0000","last_peered":"2024-03-18T23:07:01.345190+0000","last_clean":"2024-03-18T23:07:01.345190+0000","last_became_active":"2024-03-18T23:06:04.914130+0000","last_became_peered":"2024-03-18T23:06:04.914130+0000","last_unstale":"2024-03-18T23:07:01.345190+0000","last_undegraded":"2024-03-18T23:07:01.345190+0000","last_fullsized":"2024-03-18T23:07:01.345190+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:57:56.065100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:01.340664+0000","last_change":"2024-03-18T23:06:04.905951+0000","last_active":"2024-03-18T23:07:01.340664+0000","last_peered":"2024-03-18T23:07:01.340664+0000","last_clean":"2024-03-18T23:07:01.340664+0000","last_became_active":"2024-03-18T23:06:04.905614+0000","last_became_peered":"2024-03-18T23:06:04.905614+0000","last_unstale":"2024-03-18T23:07:01.340664+0000","last_undegraded":"2024-03-18T23:07:01.340664+0000","last_fullsized":"2024-03-18T23:07:01.340664+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:11:44.884671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:01.347053+0000","last_change":"2024-03-18T23:07:01.347053+0000","last_active":"2024-03-18T23:07:01.347053+0000","last_peered":"2024-03-18T23:07:01.347053+0000","last_clean":"2024-03-18T23:07:01.347053+0000","last_became_active":"2024-03-18T23:07:01.346483+0000","last_became_peered":"2024-03-18T23:07:01.346483+0000","last_unstale":"2024-03-18T23:07:01.347053+0000","last_undegraded":"2024-03-18T23:07:01.347053+0000","last_fullsized":"2024-03-18T23:07:01.347053+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:38:44.581164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,4],"acting":[2,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:01.343302+0000","last_change":"2024-03-18T23:06:04.912818+0000","last_active":"2024-03-18T23:07:01.343302+0000","last_peered":"2024-03-18T23:07:01.343302+0000","last_clean":"2024-03-18T23:07:01.343302+0000","last_became_active":"2024-03-18T23:06:04.912511+0000","last_became_peered":"2024-03-18T23:06:04.912511+0000","last_unstale":"2024-03-18T23:07:01.343302+0000","last_undegraded":"2024-03-18T23:07:01.343302+0000","last_fullsized":"2024-03-18T23:07:01.343302+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T09:24:44.743041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:01.343979+0000","last_change":"2024-03-18T23:06:04.912177+0000","last_active":"2024-03-18T23:07:01.343979+0000","last_peered":"2024-03-18T23:07:01.343979+0000","last_clean":"2024-03-18T23:07:01.343979+0000","last_became_active":"2024-03-18T23:06:04.911638+0000","last_became_peered":"2024-03-18T23:06:04.911638+0000","last_unstale":"2024-03-18T23:07:01.343979+0000","last_undegraded":"2024-03-18T23:07:01.343979+0000","last_fullsized":"2024-03-18T23:07:01.343979+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:21:30.648246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:01.343439+0000","last_change":"2024-03-18T23:06:04.912670+0000","last_active":"2024-03-18T23:07:01.343439+0000","last_peered":"2024-03-18T23:07:01.343439+0000","last_clean":"2024-03-18T23:07:01.343439+0000","last_became_active":"2024-03-18T23:06:04.912241+0000","last_became_peered":"2024-03-18T23:06:04.912241+0000","last_unstale":"2024-03-18T23:07:01.343439+0000","last_undegraded":"2024-03-18T23:07:01.343439+0000","last_fullsized":"2024-03-18T23:07:01.343439+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:31:02.666360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:01.341320+0000","last_change":"2024-03-18T23:06:04.909618+0000","last_active":"2024-03-18T23:07:01.341320+0000","last_peered":"2024-03-18T23:07:01.341320+0000","last_clean":"2024-03-18T23:07:01.341320+0000","last_became_active":"2024-03-18T23:06:04.909301+0000","last_became_peered":"2024-03-18T23:06:04.909301+0000","last_unstale":"2024-03-18T23:07:01.341320+0000","last_undegraded":"2024-03-18T23:07:01.341320+0000","last_fullsized":"2024-03-18T23:07:01.341320+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:14:28.101360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:01.343626+0000","last_change":"2024-03-18T23:06:04.904607+0000","last_active":"2024-03-18T23:07:01.343626+0000","last_peered":"2024-03-18T23:07:01.343626+0000","last_clean":"2024-03-18T23:07:01.343626+0000","last_became_active":"2024-03-18T23:06:04.904280+0000","last_became_peered":"2024-03-18T23:06:04.904280+0000","last_unstale":"2024-03-18T23:07:01.343626+0000","last_undegraded":"2024-03-18T23:07:01.343626+0000","last_fullsized":"2024-03-18T23:07:01.343626+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:59:11.445200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:01.345637+0000","last_change":"2024-03-18T23:06:04.918342+0000","last_active":"2024-03-18T23:07:01.345637+0000","last_peered":"2024-03-18T23:07:01.345637+0000","last_clean":"2024-03-18T23:07:01.345637+0000","last_became_active":"2024-03-18T23:06:04.917947+0000","last_became_peered":"2024-03-18T23:06:04.917947+0000","last_unstale":"2024-03-18T23:07:01.345637+0000","last_undegraded":"2024-03-18T23:07:01.345637+0000","last_fullsized":"2024-03-18T23:07:01.345637+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:05:49.347374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:01.343012+0000","last_change":"2024-03-18T23:06:04.912457+0000","last_active":"2024-03-18T23:07:01.343012+0000","last_peered":"2024-03-18T23:07:01.343012+0000","last_clean":"2024-03-18T23:07:01.343012+0000","last_became_active":"2024-03-18T23:06:04.912141+0000","last_became_peered":"2024-03-18T23:06:04.912141+0000","last_unstale":"2024-03-18T23:07:01.343012+0000","last_undegraded":"2024-03-18T23:07:01.343012+0000","last_fullsized":"2024-03-18T23:07:01.343012+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:04:45.847314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:01.345511+0000","last_change":"2024-03-18T23:06:04.918723+0000","last_active":"2024-03-18T23:07:01.345511+0000","last_peered":"2024-03-18T23:07:01.345511+0000","last_clean":"2024-03-18T23:07:01.345511+0000","last_became_active":"2024-03-18T23:06:04.918375+0000","last_became_peered":"2024-03-18T23:06:04.918375+0000","last_unstale":"2024-03-18T23:07:01.345511+0000","last_undegraded":"2024-03-18T23:07:01.345511+0000","last_fullsized":"2024-03-18T23:07:01.345511+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:27:12.868420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:01.344578+0000","last_change":"2024-03-18T23:06:04.913935+0000","last_active":"2024-03-18T23:07:01.344578+0000","last_peered":"2024-03-18T23:07:01.344578+0000","last_clean":"2024-03-18T23:07:01.344578+0000","last_became_active":"2024-03-18T23:06:04.913617+0000","last_became_peered":"2024-03-18T23:06:04.913617+0000","last_unstale":"2024-03-18T23:07:01.344578+0000","last_undegraded":"2024-03-18T23:07:01.344578+0000","last_fullsized":"2024-03-18T23:07:01.344578+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:53:15.585608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:01.343628+0000","last_change":"2024-03-18T23:06:04.911951+0000","last_active":"2024-03-18T23:07:01.343628+0000","last_peered":"2024-03-18T23:07:01.343628+0000","last_clean":"2024-03-18T23:07:01.343628+0000","last_became_active":"2024-03-18T23:06:04.911621+0000","last_became_peered":"2024-03-18T23:06:04.911621+0000","last_unstale":"2024-03-18T23:07:01.343628+0000","last_undegraded":"2024-03-18T23:07:01.343628+0000","last_fullsized":"2024-03-18T23:07:01.343628+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:20:46.600106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:01.349238+0000","last_change":"2024-03-18T23:07:01.349238+0000","last_active":"2024-03-18T23:07:01.349238+0000","last_peered":"2024-03-18T23:07:01.349238+0000","last_clean":"2024-03-18T23:07:01.349238+0000","last_became_active":"2024-03-18T23:07:01.348853+0000","last_became_peered":"2024-03-18T23:07:01.348853+0000","last_unstale":"2024-03-18T23:07:01.349238+0000","last_undegraded":"2024-03-18T23:07:01.349238+0000","last_fullsized":"2024-03-18T23:07:01.349238+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T08:16:35.883785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":284006,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":66278,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":687,"num_read_kb":439,"num_write":235,"num_write_kb":130,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2143,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":552960,"data_stored":198834,"data_compressed":5559,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":204,"ondisk_log_size":204,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351005,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":966656,"data_stored":463578,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612640,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28108,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717044,"statfs":{"total":95995035648,"available":95966253056,"internally_reserved":0,"allocated":1114112,"data_stored":1113099,"data_compressed":37968,"data_compressed_allocated":49152,"data_compressed_original":598016,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776166,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":987136,"data_stored":462535,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037804,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32140,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713012,"statfs":{"total":95995035648,"available":95962124288,"internally_reserved":0,"allocated":978944,"data_stored":462331,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332143,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1036288,"data_stored":1055040,"data_compressed":36115,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495670,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36292,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708860,"statfs":{"total":95995035648,"available":95957872640,"internally_reserved":0,"allocated":970752,"data_stored":462252,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574927,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32188,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712964,"statfs":{"total":95995035648,"available":95962075136,"internally_reserved":0,"allocated":1028096,"data_stored":522630,"data_compressed":1853,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738452,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36312,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708840,"statfs":{"total":95995035648,"available":95957852160,"internally_reserved":0,"allocated":991232,"data_stored":463982,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411403,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1040384,"data_stored":521680,"data_compressed":1853,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182469,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":987136,"data_stored":462077,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920832,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32140,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713012,"statfs":{"total":95995035648,"available":95962124288,"internally_reserved":0,"allocated":978944,"data_stored":463674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757307,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32204,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712948,"statfs":{"total":95995035648,"available":95962058752,"internally_reserved":0,"allocated":1048576,"data_stored":1054718,"data_compressed":36115,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34477,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34477,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34477,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36115,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36115,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36115,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36115,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36115,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":61217,"data_compressed":1853,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":60267,"data_compressed":1853,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2261,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":61201,"data_compressed":1853,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2165,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-18T23:07:25.358 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell --fsid c63591ea-e57a-11ee-95c9-87774f69a715 -- ceph pg dump --format=json 2024-03-18T23:07:25.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:07:24 smithi175 ceph-mon[67352]: pgmap v330: 97 pgs: 97 active+clean; 641 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-18T23:07:25.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:07:24 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:07:25.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:07:24 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:07:25.462 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:07:24 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:07:25.462 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:07:24 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:07:25.462 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:07:24 smithi175 ceph-mon[67352]: from='mgr.14150 172.21.15.49:0/2980968613' entity='mgr.x' 2024-03-18T23:07:25.462 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:07:24 smithi175 ceph-mon[67352]: from='client.24766 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-18T23:07:25.611 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T23:07:27.120 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:07:26 smithi049 ceph-mon[62991]: pgmap v331: 97 pgs: 97 active+clean; 641 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-03-18T23:07:27.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:07:26 smithi139 ceph-mon[67282]: pgmap v331: 97 pgs: 97 active+clean; 641 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-03-18T23:07:27.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:27.342+0000 7f0ababa4700 1 -- 172.21.15.49:0/904774750 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0ab4101bb0 msgr2=0x7f0ab410e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:27.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:27.342+0000 7f0ababa4700 1 --2- 172.21.15.49:0/904774750 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0ab4101bb0 0x7f0ab410e090 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f0a9c009990 tx=0x7f0a9c009ca0 comp rx=0 tx=0).stop 2024-03-18T23:07:27.343 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:27.342+0000 7f0ababa4700 1 -- 172.21.15.49:0/904774750 shutdown_connections 2024-03-18T23:07:27.343 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:27.342+0000 7f0ababa4700 1 --2- 172.21.15.49:0/904774750 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0ab4101bb0 0x7f0ab410e090 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:27.343 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:27.342+0000 7f0ababa4700 1 --2- 172.21.15.49:0/904774750 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0ab4107250 0x7f0ab4107630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:27.343 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:27.342+0000 7f0ababa4700 1 --2- 172.21.15.49:0/904774750 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0ab4101210 0x7f0ab4101670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:27.343 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:27.342+0000 7f0ababa4700 1 -- 172.21.15.49:0/904774750 >> 172.21.15.49:0/904774750 conn(0x7f0ab4078140 msgr2=0x7f0ab4078550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:07:27.343 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:27.342+0000 7f0ababa4700 1 -- 172.21.15.49:0/904774750 shutdown_connections 2024-03-18T23:07:27.343 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:27.342+0000 7f0ababa4700 1 -- 172.21.15.49:0/904774750 wait complete. 2024-03-18T23:07:27.345 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:27.343+0000 7f0ababa4700 1 Processor -- start 2024-03-18T23:07:27.345 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:27.343+0000 7f0ababa4700 1 -- start start 2024-03-18T23:07:27.345 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:27.343+0000 7f0ababa4700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0ab4101210 0x7f0ab419e720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:27.346 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:27.343+0000 7f0ababa4700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0ab4101bb0 0x7f0ab419ec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:27.346 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:27.343+0000 7f0ababa4700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0ab4107250 0x7f0ab41988e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:27.346 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:27.343+0000 7f0ababa4700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f0ab41114d0 con 0x7f0ab4101bb0 2024-03-18T23:07:27.346 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:27.343+0000 7f0ababa4700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f0ab4111350 con 0x7f0ab4101210 2024-03-18T23:07:27.346 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:27.343+0000 7f0ababa4700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f0ab4111650 con 0x7f0ab4107250 2024-03-18T23:07:27.346 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:27.343+0000 7f0ab3fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0ab4101bb0 0x7f0ab419ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:27.346 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:27.343+0000 7f0ab3fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0ab4101bb0 0x7f0ab419ec60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:44086/0 (socket says 172.21.15.49:44086) 2024-03-18T23:07:27.346 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:27.343+0000 7f0ab3fff700 1 -- 172.21.15.49:0/2048804010 learned_addr learned my addr 172.21.15.49:0/2048804010 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:07:27.346 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:27.343+0000 7f0ab9141700 1 --2- 172.21.15.49:0/2048804010 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0ab4107250 0x7f0ab41988e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:27.346 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:27.343+0000 7f0ab8940700 1 --2- 172.21.15.49:0/2048804010 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0ab4101210 0x7f0ab419e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:27.346 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:27.344+0000 7f0ab3fff700 1 -- 172.21.15.49:0/2048804010 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0ab4107250 msgr2=0x7f0ab41988e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:27.346 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:27.344+0000 7f0ab3fff700 1 --2- 172.21.15.49:0/2048804010 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0ab4107250 0x7f0ab41988e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:27.346 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:27.344+0000 7f0ab3fff700 1 -- 172.21.15.49:0/2048804010 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0ab4101210 msgr2=0x7f0ab419e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:27.347 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:27.344+0000 7f0ab3fff700 1 --2- 172.21.15.49:0/2048804010 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0ab4101210 0x7f0ab419e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:27.347 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:27.344+0000 7f0ab3fff700 1 -- 172.21.15.49:0/2048804010 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ab4199040 con 0x7f0ab4101bb0 2024-03-18T23:07:27.347 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:27.344+0000 7f0ab3fff700 1 --2- 172.21.15.49:0/2048804010 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0ab4101bb0 0x7f0ab419ec60 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f0aa400e9c0 tx=0x7f0aa400ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:07:27.347 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:27.344+0000 7f0ab1ffb700 1 -- 172.21.15.49:0/2048804010 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0aa400cbd0 con 0x7f0ab4101bb0 2024-03-18T23:07:27.347 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:27.344+0000 7f0ab1ffb700 1 -- 172.21.15.49:0/2048804010 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0aa400ee60 con 0x7f0ab4101bb0 2024-03-18T23:07:27.347 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:27.344+0000 7f0ababa4700 1 -- 172.21.15.49:0/2048804010 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ab4199330 con 0x7f0ab4101bb0 2024-03-18T23:07:27.347 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:27.344+0000 7f0ab1ffb700 1 -- 172.21.15.49:0/2048804010 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0aa4005630 con 0x7f0ab4101bb0 2024-03-18T23:07:27.347 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:27.344+0000 7f0ababa4700 1 -- 172.21.15.49:0/2048804010 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ab41a55f0 con 0x7f0ab4101bb0 2024-03-18T23:07:27.347 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:27.345+0000 7f0ababa4700 1 -- 172.21.15.49:0/2048804010 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ab4068fd0 con 0x7f0ab4101bb0 2024-03-18T23:07:27.348 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:27.346+0000 7f0ab1ffb700 1 -- 172.21.15.49:0/2048804010 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 17) ==== 137341+0+0 (secure 0 0 0) 0x7f0aa4004750 con 0x7f0ab4101bb0 2024-03-18T23:07:27.348 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:27.347+0000 7f0ab1ffb700 1 --2- 172.21.15.49:0/2048804010 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f0aa00a2920 0x7f0aa00a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:27.348 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:27.347+0000 7f0ab8940700 1 --2- 172.21.15.49:0/2048804010 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f0aa00a2920 0x7f0aa00a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:27.348 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:27.347+0000 7f0ab1ffb700 1 -- 172.21.15.49:0/2048804010 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f0aa40cee60 con 0x7f0ab4101bb0 2024-03-18T23:07:27.348 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:27.347+0000 7f0ab8940700 1 --2- 172.21.15.49:0/2048804010 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f0aa00a2920 0x7f0aa00a4de0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f0ab4199a20 tx=0x7f0aac00b040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:07:27.351 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:27.351+0000 7f0ab1ffb700 1 -- 172.21.15.49:0/2048804010 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f0aa409bdb0 con 0x7f0ab4101bb0 2024-03-18T23:07:27.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:07:26 smithi175 ceph-mon[67352]: pgmap v331: 97 pgs: 97 active+clean; 641 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-03-18T23:07:27.551 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:27.551+0000 7f0ababa4700 1 -- 172.21.15.49:0/2048804010 --> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f0ab4105000 con 0x7f0aa00a2920 2024-03-18T23:07:27.558 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:27.558+0000 7f0ab1ffb700 1 -- 172.21.15.49:0/2048804010 <== mgr.14150 v2:172.21.15.49:6800/2083805211 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258990 (secure 0 0 0) 0x7f0ab4105000 con 0x7f0aa00a2920 2024-03-18T23:07:27.562 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-18T23:07:27.564 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:27.562+0000 7f0ababa4700 1 -- 172.21.15.49:0/2048804010 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f0aa00a2920 msgr2=0x7f0aa00a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:27.564 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:27.562+0000 7f0ababa4700 1 --2- 172.21.15.49:0/2048804010 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f0aa00a2920 0x7f0aa00a4de0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f0ab4199a20 tx=0x7f0aac00b040 comp rx=0 tx=0).stop 2024-03-18T23:07:27.564 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:27.562+0000 7f0ababa4700 1 -- 172.21.15.49:0/2048804010 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0ab4101bb0 msgr2=0x7f0ab419ec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:27.564 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:27.562+0000 7f0ababa4700 1 --2- 172.21.15.49:0/2048804010 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0ab4101bb0 0x7f0ab419ec60 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f0aa400e9c0 tx=0x7f0aa400ecd0 comp rx=0 tx=0).stop 2024-03-18T23:07:27.564 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:27.562+0000 7f0ababa4700 1 -- 172.21.15.49:0/2048804010 shutdown_connections 2024-03-18T23:07:27.564 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:27.562+0000 7f0ababa4700 1 --2- 172.21.15.49:0/2048804010 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f0aa00a2920 0x7f0aa00a4de0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:27.564 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:27.562+0000 7f0ababa4700 1 --2- 172.21.15.49:0/2048804010 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0ab4107250 0x7f0ab41988e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:27.564 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:27.562+0000 7f0ababa4700 1 --2- 172.21.15.49:0/2048804010 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0ab4101210 0x7f0ab419e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:27.565 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:27.562+0000 7f0ababa4700 1 --2- 172.21.15.49:0/2048804010 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0ab4101bb0 0x7f0ab419ec60 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:27.565 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:27.562+0000 7f0ababa4700 1 -- 172.21.15.49:0/2048804010 >> 172.21.15.49:0/2048804010 conn(0x7f0ab4078140 msgr2=0x7f0ab41005d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:07:27.565 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:27.562+0000 7f0ababa4700 1 -- 172.21.15.49:0/2048804010 shutdown_connections 2024-03-18T23:07:27.565 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:27.562+0000 7f0ababa4700 1 -- 172.21.15.49:0/2048804010 wait complete. 2024-03-18T23:07:27.565 INFO:teuthology.orchestra.run.smithi049.stderr:dumped all 2024-03-18T23:07:28.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:07:28 smithi139 ceph-mon[67282]: from='client.14961 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-18T23:07:28.360 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:07:27 smithi049 ceph-mon[62991]: from='client.14961 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-18T23:07:28.390 INFO:teuthology.orchestra.run.smithi049.stdout:{"pg_ready":true,"pg_map":{"version":331,"stamp":"2024-03-18T23:07:25.918428+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":656880,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":848,"num_read_kb":568,"num_write":418,"num_write_kb":3070,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":1182879,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":343,"ondisk_log_size":343,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":369396,"kb_used_data":11844,"kb_used_omap":18,"kb_used_meta":357357,"kb_avail":1124572428,"statfs":{"total":1151940427776,"available":1151562166272,"internally_reserved":0,"allocated":12128256,"data_stored":7507596,"data_compressed":113904,"data_compressed_allocated":147456,"data_compressed_original":1794048,"omap_allocated":19142,"internal_metadata":365933882},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":125,"num_read_kb":60,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003637"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:01.341507+0000","last_change":"2024-03-18T23:06:04.913788+0000","last_active":"2024-03-18T23:07:01.341507+0000","last_peered":"2024-03-18T23:07:01.341507+0000","last_clean":"2024-03-18T23:07:01.341507+0000","last_became_active":"2024-03-18T23:06:04.913103+0000","last_became_peered":"2024-03-18T23:06:04.913103+0000","last_unstale":"2024-03-18T23:07:01.341507+0000","last_undegraded":"2024-03-18T23:07:01.341507+0000","last_fullsized":"2024-03-18T23:07:01.341507+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:48:10.100291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:01.343786+0000","last_change":"2024-03-18T23:06:04.921126+0000","last_active":"2024-03-18T23:07:01.343786+0000","last_peered":"2024-03-18T23:07:01.343786+0000","last_clean":"2024-03-18T23:07:01.343786+0000","last_became_active":"2024-03-18T23:06:04.919934+0000","last_became_peered":"2024-03-18T23:06:04.919934+0000","last_unstale":"2024-03-18T23:07:01.343786+0000","last_undegraded":"2024-03-18T23:07:01.343786+0000","last_fullsized":"2024-03-18T23:07:01.343786+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:24:25.351428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:01.344010+0000","last_change":"2024-03-18T23:06:04.920933+0000","last_active":"2024-03-18T23:07:01.344010+0000","last_peered":"2024-03-18T23:07:01.344010+0000","last_clean":"2024-03-18T23:07:01.344010+0000","last_became_active":"2024-03-18T23:06:04.920548+0000","last_became_peered":"2024-03-18T23:06:04.920548+0000","last_unstale":"2024-03-18T23:07:01.344010+0000","last_undegraded":"2024-03-18T23:07:01.344010+0000","last_fullsized":"2024-03-18T23:07:01.344010+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:06:01.746208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:01.344330+0000","last_change":"2024-03-18T23:06:04.921219+0000","last_active":"2024-03-18T23:07:01.344330+0000","last_peered":"2024-03-18T23:07:01.344330+0000","last_clean":"2024-03-18T23:07:01.344330+0000","last_became_active":"2024-03-18T23:06:04.920069+0000","last_became_peered":"2024-03-18T23:06:04.920069+0000","last_unstale":"2024-03-18T23:07:01.344330+0000","last_undegraded":"2024-03-18T23:07:01.344330+0000","last_fullsized":"2024-03-18T23:07:01.344330+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:22:44.463061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:01.346564+0000","last_change":"2024-03-18T23:06:04.916741+0000","last_active":"2024-03-18T23:07:01.346564+0000","last_peered":"2024-03-18T23:07:01.346564+0000","last_clean":"2024-03-18T23:07:01.346564+0000","last_became_active":"2024-03-18T23:06:04.916434+0000","last_became_peered":"2024-03-18T23:06:04.916434+0000","last_unstale":"2024-03-18T23:07:01.346564+0000","last_undegraded":"2024-03-18T23:07:01.346564+0000","last_fullsized":"2024-03-18T23:07:01.346564+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:45:10.553306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:01.346188+0000","last_change":"2024-03-18T23:06:04.917198+0000","last_active":"2024-03-18T23:07:01.346188+0000","last_peered":"2024-03-18T23:07:01.346188+0000","last_clean":"2024-03-18T23:07:01.346188+0000","last_became_active":"2024-03-18T23:06:04.916897+0000","last_became_peered":"2024-03-18T23:06:04.916897+0000","last_unstale":"2024-03-18T23:07:01.346188+0000","last_undegraded":"2024-03-18T23:07:01.346188+0000","last_fullsized":"2024-03-18T23:07:01.346188+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T23:23:26.530668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:01.343844+0000","last_change":"2024-03-18T23:06:04.908446+0000","last_active":"2024-03-18T23:07:01.343844+0000","last_peered":"2024-03-18T23:07:01.343844+0000","last_clean":"2024-03-18T23:07:01.343844+0000","last_became_active":"2024-03-18T23:06:04.908093+0000","last_became_peered":"2024-03-18T23:06:04.908093+0000","last_unstale":"2024-03-18T23:07:01.343844+0000","last_undegraded":"2024-03-18T23:07:01.343844+0000","last_fullsized":"2024-03-18T23:07:01.343844+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T09:21:30.253229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:01.345242+0000","last_change":"2024-03-18T23:06:04.913080+0000","last_active":"2024-03-18T23:07:01.345242+0000","last_peered":"2024-03-18T23:07:01.345242+0000","last_clean":"2024-03-18T23:07:01.345242+0000","last_became_active":"2024-03-18T23:06:04.912255+0000","last_became_peered":"2024-03-18T23:06:04.912255+0000","last_unstale":"2024-03-18T23:07:01.345242+0000","last_undegraded":"2024-03-18T23:07:01.345242+0000","last_fullsized":"2024-03-18T23:07:01.345242+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:16:36.069545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:01.344342+0000","last_change":"2024-03-18T23:06:04.905284+0000","last_active":"2024-03-18T23:07:01.344342+0000","last_peered":"2024-03-18T23:07:01.344342+0000","last_clean":"2024-03-18T23:07:01.344342+0000","last_became_active":"2024-03-18T23:06:04.904388+0000","last_became_peered":"2024-03-18T23:06:04.904388+0000","last_unstale":"2024-03-18T23:07:01.344342+0000","last_undegraded":"2024-03-18T23:07:01.344342+0000","last_fullsized":"2024-03-18T23:07:01.344342+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:57:56.628599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:01.346070+0000","last_change":"2024-03-18T23:06:04.914602+0000","last_active":"2024-03-18T23:07:01.346070+0000","last_peered":"2024-03-18T23:07:01.346070+0000","last_clean":"2024-03-18T23:07:01.346070+0000","last_became_active":"2024-03-18T23:06:04.914288+0000","last_became_peered":"2024-03-18T23:06:04.914288+0000","last_unstale":"2024-03-18T23:07:01.346070+0000","last_undegraded":"2024-03-18T23:07:01.346070+0000","last_fullsized":"2024-03-18T23:07:01.346070+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:44:52.182950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:01.344985+0000","last_change":"2024-03-18T23:06:04.919821+0000","last_active":"2024-03-18T23:07:01.344985+0000","last_peered":"2024-03-18T23:07:01.344985+0000","last_clean":"2024-03-18T23:07:01.344985+0000","last_became_active":"2024-03-18T23:06:04.919525+0000","last_became_peered":"2024-03-18T23:06:04.919525+0000","last_unstale":"2024-03-18T23:07:01.344985+0000","last_undegraded":"2024-03-18T23:07:01.344985+0000","last_fullsized":"2024-03-18T23:07:01.344985+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:54:50.720229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:01.350429+0000","last_change":"2024-03-18T23:07:01.350429+0000","last_active":"2024-03-18T23:07:01.350429+0000","last_peered":"2024-03-18T23:07:01.350429+0000","last_clean":"2024-03-18T23:07:01.350429+0000","last_became_active":"2024-03-18T23:07:01.350040+0000","last_became_peered":"2024-03-18T23:07:01.350040+0000","last_unstale":"2024-03-18T23:07:01.350429+0000","last_undegraded":"2024-03-18T23:07:01.350429+0000","last_fullsized":"2024-03-18T23:07:01.350429+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:35:05.048875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:01.343529+0000","last_change":"2024-03-18T23:06:04.905399+0000","last_active":"2024-03-18T23:07:01.343529+0000","last_peered":"2024-03-18T23:07:01.343529+0000","last_clean":"2024-03-18T23:07:01.343529+0000","last_became_active":"2024-03-18T23:06:04.905117+0000","last_became_peered":"2024-03-18T23:06:04.905117+0000","last_unstale":"2024-03-18T23:07:01.343529+0000","last_undegraded":"2024-03-18T23:07:01.343529+0000","last_fullsized":"2024-03-18T23:07:01.343529+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:41:56.382244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:01.343615+0000","last_change":"2024-03-18T23:06:04.909993+0000","last_active":"2024-03-18T23:07:01.343615+0000","last_peered":"2024-03-18T23:07:01.343615+0000","last_clean":"2024-03-18T23:07:01.343615+0000","last_became_active":"2024-03-18T23:06:04.909483+0000","last_became_peered":"2024-03-18T23:06:04.909483+0000","last_unstale":"2024-03-18T23:07:01.343615+0000","last_undegraded":"2024-03-18T23:07:01.343615+0000","last_fullsized":"2024-03-18T23:07:01.343615+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:36:52.826582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:01.343772+0000","last_change":"2024-03-18T23:06:04.915273+0000","last_active":"2024-03-18T23:07:01.343772+0000","last_peered":"2024-03-18T23:07:01.343772+0000","last_clean":"2024-03-18T23:07:01.343772+0000","last_became_active":"2024-03-18T23:06:04.913510+0000","last_became_peered":"2024-03-18T23:06:04.913510+0000","last_unstale":"2024-03-18T23:07:01.343772+0000","last_undegraded":"2024-03-18T23:07:01.343772+0000","last_fullsized":"2024-03-18T23:07:01.343772+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:46:01.622963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:01.343699+0000","last_change":"2024-03-18T23:06:04.917599+0000","last_active":"2024-03-18T23:07:01.343699+0000","last_peered":"2024-03-18T23:07:01.343699+0000","last_clean":"2024-03-18T23:07:01.343699+0000","last_became_active":"2024-03-18T23:06:04.914961+0000","last_became_peered":"2024-03-18T23:06:04.914961+0000","last_unstale":"2024-03-18T23:07:01.343699+0000","last_undegraded":"2024-03-18T23:07:01.343699+0000","last_fullsized":"2024-03-18T23:07:01.343699+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T09:38:50.076075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:01.341718+0000","last_change":"2024-03-18T23:06:04.920814+0000","last_active":"2024-03-18T23:07:01.341718+0000","last_peered":"2024-03-18T23:07:01.341718+0000","last_clean":"2024-03-18T23:07:01.341718+0000","last_became_active":"2024-03-18T23:06:04.920484+0000","last_became_peered":"2024-03-18T23:06:04.920484+0000","last_unstale":"2024-03-18T23:07:01.341718+0000","last_undegraded":"2024-03-18T23:07:01.341718+0000","last_fullsized":"2024-03-18T23:07:01.341718+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:45:01.545059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:01.340689+0000","last_change":"2024-03-18T23:06:04.904684+0000","last_active":"2024-03-18T23:07:01.340689+0000","last_peered":"2024-03-18T23:07:01.340689+0000","last_clean":"2024-03-18T23:07:01.340689+0000","last_became_active":"2024-03-18T23:06:04.904296+0000","last_became_peered":"2024-03-18T23:06:04.904296+0000","last_unstale":"2024-03-18T23:07:01.340689+0000","last_undegraded":"2024-03-18T23:07:01.340689+0000","last_fullsized":"2024-03-18T23:07:01.340689+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:26:28.098286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:01.344254+0000","last_change":"2024-03-18T23:06:04.912803+0000","last_active":"2024-03-18T23:07:01.344254+0000","last_peered":"2024-03-18T23:07:01.344254+0000","last_clean":"2024-03-18T23:07:01.344254+0000","last_became_active":"2024-03-18T23:06:04.912495+0000","last_became_peered":"2024-03-18T23:06:04.912495+0000","last_unstale":"2024-03-18T23:07:01.344254+0000","last_undegraded":"2024-03-18T23:07:01.344254+0000","last_fullsized":"2024-03-18T23:07:01.344254+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:47:38.111260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"86'13","reported_seq":107,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:24.086545+0000","last_change":"2024-03-18T23:06:04.221492+0000","last_active":"2024-03-18T23:07:24.086545+0000","last_peered":"2024-03-18T23:07:24.086545+0000","last_clean":"2024-03-18T23:07:24.086545+0000","last_became_active":"2024-03-18T23:06:04.221190+0000","last_became_peered":"2024-03-18T23:06:04.221190+0000","last_unstale":"2024-03-18T23:07:24.086545+0000","last_undegraded":"2024-03-18T23:07:24.086545+0000","last_fullsized":"2024-03-18T23:07:24.086545+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_clean_scrub_stamp":"2024-03-18T23:06:03.184668+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:45:08.747471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":75,"num_read_kb":75,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:01.345953+0000","last_change":"2024-03-18T23:06:04.911680+0000","last_active":"2024-03-18T23:07:01.345953+0000","last_peered":"2024-03-18T23:07:01.345953+0000","last_clean":"2024-03-18T23:07:01.345953+0000","last_became_active":"2024-03-18T23:06:04.911356+0000","last_became_peered":"2024-03-18T23:06:04.911356+0000","last_unstale":"2024-03-18T23:07:01.345953+0000","last_undegraded":"2024-03-18T23:07:01.345953+0000","last_fullsized":"2024-03-18T23:07:01.345953+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:23:21.943304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:01.346096+0000","last_change":"2024-03-18T23:06:04.225792+0000","last_active":"2024-03-18T23:07:01.346096+0000","last_peered":"2024-03-18T23:07:01.346096+0000","last_clean":"2024-03-18T23:07:01.346096+0000","last_became_active":"2024-03-18T23:06:04.224248+0000","last_became_peered":"2024-03-18T23:06:04.224248+0000","last_unstale":"2024-03-18T23:07:01.346096+0000","last_undegraded":"2024-03-18T23:07:01.346096+0000","last_fullsized":"2024-03-18T23:07:01.346096+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_clean_scrub_stamp":"2024-03-18T23:06:03.184668+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:48:27.052831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:01.344246+0000","last_change":"2024-03-18T23:06:04.921139+0000","last_active":"2024-03-18T23:07:01.344246+0000","last_peered":"2024-03-18T23:07:01.344246+0000","last_clean":"2024-03-18T23:07:01.344246+0000","last_became_active":"2024-03-18T23:06:04.920700+0000","last_became_peered":"2024-03-18T23:06:04.920700+0000","last_unstale":"2024-03-18T23:07:01.344246+0000","last_undegraded":"2024-03-18T23:07:01.344246+0000","last_fullsized":"2024-03-18T23:07:01.344246+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T08:47:34.465176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"86'12","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:01.347823+0000","last_change":"2024-03-18T23:06:04.214134+0000","last_active":"2024-03-18T23:07:01.347823+0000","last_peered":"2024-03-18T23:07:01.347823+0000","last_clean":"2024-03-18T23:07:01.347823+0000","last_became_active":"2024-03-18T23:06:04.212256+0000","last_became_peered":"2024-03-18T23:06:04.212256+0000","last_unstale":"2024-03-18T23:07:01.347823+0000","last_undegraded":"2024-03-18T23:07:01.347823+0000","last_fullsized":"2024-03-18T23:07:01.347823+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_clean_scrub_stamp":"2024-03-18T23:06:03.184668+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:39:46.851964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:01.343878+0000","last_change":"2024-03-18T23:06:04.907465+0000","last_active":"2024-03-18T23:07:01.343878+0000","last_peered":"2024-03-18T23:07:01.343878+0000","last_clean":"2024-03-18T23:07:01.343878+0000","last_became_active":"2024-03-18T23:06:04.907142+0000","last_became_peered":"2024-03-18T23:06:04.907142+0000","last_unstale":"2024-03-18T23:07:01.343878+0000","last_undegraded":"2024-03-18T23:07:01.343878+0000","last_fullsized":"2024-03-18T23:07:01.343878+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:15:29.066708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"86'5","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:01.354353+0000","last_change":"2024-03-18T23:07:01.353908+0000","last_active":"2024-03-18T23:07:01.354353+0000","last_peered":"2024-03-18T23:07:01.354353+0000","last_clean":"2024-03-18T23:07:01.354353+0000","last_became_active":"2024-03-18T23:07:01.345254+0000","last_became_peered":"2024-03-18T23:07:01.345254+0000","last_unstale":"2024-03-18T23:07:01.354353+0000","last_undegraded":"2024-03-18T23:07:01.354353+0000","last_fullsized":"2024-03-18T23:07:01.354353+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_clean_scrub_stamp":"2024-03-18T23:06:03.184668+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:48:56.474966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:01.343607+0000","last_change":"2024-03-18T23:06:04.918596+0000","last_active":"2024-03-18T23:07:01.343607+0000","last_peered":"2024-03-18T23:07:01.343607+0000","last_clean":"2024-03-18T23:07:01.343607+0000","last_became_active":"2024-03-18T23:06:04.918273+0000","last_became_peered":"2024-03-18T23:06:04.918273+0000","last_unstale":"2024-03-18T23:07:01.343607+0000","last_undegraded":"2024-03-18T23:07:01.343607+0000","last_fullsized":"2024-03-18T23:07:01.343607+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T08:09:04.860241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"86'5","reported_seq":101,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:22.059077+0000","last_change":"2024-03-18T23:06:04.214573+0000","last_active":"2024-03-18T23:07:22.059077+0000","last_peered":"2024-03-18T23:07:22.059077+0000","last_clean":"2024-03-18T23:07:22.059077+0000","last_became_active":"2024-03-18T23:06:04.213891+0000","last_became_peered":"2024-03-18T23:06:04.213891+0000","last_unstale":"2024-03-18T23:07:22.059077+0000","last_undegraded":"2024-03-18T23:07:22.059077+0000","last_fullsized":"2024-03-18T23:07:22.059077+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_clean_scrub_stamp":"2024-03-18T23:06:03.184668+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:17:22.895024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":77,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:01.346271+0000","last_change":"2024-03-18T23:06:04.908336+0000","last_active":"2024-03-18T23:07:01.346271+0000","last_peered":"2024-03-18T23:07:01.346271+0000","last_clean":"2024-03-18T23:07:01.346271+0000","last_became_active":"2024-03-18T23:06:04.908023+0000","last_became_peered":"2024-03-18T23:06:04.908023+0000","last_unstale":"2024-03-18T23:07:01.346271+0000","last_undegraded":"2024-03-18T23:07:01.346271+0000","last_fullsized":"2024-03-18T23:07:01.346271+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:34:50.993715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"86'7","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:01.346058+0000","last_change":"2024-03-18T23:06:04.210036+0000","last_active":"2024-03-18T23:07:01.346058+0000","last_peered":"2024-03-18T23:07:01.346058+0000","last_clean":"2024-03-18T23:07:01.346058+0000","last_became_active":"2024-03-18T23:06:04.209658+0000","last_became_peered":"2024-03-18T23:06:04.209658+0000","last_unstale":"2024-03-18T23:07:01.346058+0000","last_undegraded":"2024-03-18T23:07:01.346058+0000","last_fullsized":"2024-03-18T23:07:01.346058+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_clean_scrub_stamp":"2024-03-18T23:06:03.184668+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T23:55:37.286542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:01.343424+0000","last_change":"2024-03-18T23:06:04.917909+0000","last_active":"2024-03-18T23:07:01.343424+0000","last_peered":"2024-03-18T23:07:01.343424+0000","last_clean":"2024-03-18T23:07:01.343424+0000","last_became_active":"2024-03-18T23:06:04.917305+0000","last_became_peered":"2024-03-18T23:06:04.917305+0000","last_unstale":"2024-03-18T23:07:01.343424+0000","last_undegraded":"2024-03-18T23:07:01.343424+0000","last_fullsized":"2024-03-18T23:07:01.343424+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:39:48.953530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"86'3","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:01.343454+0000","last_change":"2024-03-18T23:06:04.216030+0000","last_active":"2024-03-18T23:07:01.343454+0000","last_peered":"2024-03-18T23:07:01.343454+0000","last_clean":"2024-03-18T23:07:01.343454+0000","last_became_active":"2024-03-18T23:06:04.215695+0000","last_became_peered":"2024-03-18T23:06:04.215695+0000","last_unstale":"2024-03-18T23:07:01.343454+0000","last_undegraded":"2024-03-18T23:07:01.343454+0000","last_fullsized":"2024-03-18T23:07:01.343454+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_clean_scrub_stamp":"2024-03-18T23:06:03.184668+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:48:37.742916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:01.345348+0000","last_change":"2024-03-18T23:06:04.910749+0000","last_active":"2024-03-18T23:07:01.345348+0000","last_peered":"2024-03-18T23:07:01.345348+0000","last_clean":"2024-03-18T23:07:01.345348+0000","last_became_active":"2024-03-18T23:06:04.910313+0000","last_became_peered":"2024-03-18T23:06:04.910313+0000","last_unstale":"2024-03-18T23:07:01.345348+0000","last_undegraded":"2024-03-18T23:07:01.345348+0000","last_fullsized":"2024-03-18T23:07:01.345348+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:18:19.726537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"86'3","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:01.342902+0000","last_change":"2024-03-18T23:06:04.218263+0000","last_active":"2024-03-18T23:07:01.342902+0000","last_peered":"2024-03-18T23:07:01.342902+0000","last_clean":"2024-03-18T23:07:01.342902+0000","last_became_active":"2024-03-18T23:06:04.217943+0000","last_became_peered":"2024-03-18T23:06:04.217943+0000","last_unstale":"2024-03-18T23:07:01.342902+0000","last_undegraded":"2024-03-18T23:07:01.342902+0000","last_fullsized":"2024-03-18T23:07:01.342902+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_clean_scrub_stamp":"2024-03-18T23:06:03.184668+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:25:11.632767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:01.340341+0000","last_change":"2024-03-18T23:06:04.915905+0000","last_active":"2024-03-18T23:07:01.340341+0000","last_peered":"2024-03-18T23:07:01.340341+0000","last_clean":"2024-03-18T23:07:01.340341+0000","last_became_active":"2024-03-18T23:06:04.915584+0000","last_became_peered":"2024-03-18T23:06:04.915584+0000","last_unstale":"2024-03-18T23:07:01.340341+0000","last_undegraded":"2024-03-18T23:07:01.340341+0000","last_fullsized":"2024-03-18T23:07:01.340341+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:29:05.503861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"86'4","reported_seq":166,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:21.057547+0000","last_change":"2024-03-18T23:06:04.217077+0000","last_active":"2024-03-18T23:07:21.057547+0000","last_peered":"2024-03-18T23:07:21.057547+0000","last_clean":"2024-03-18T23:07:21.057547+0000","last_became_active":"2024-03-18T23:06:04.216308+0000","last_became_peered":"2024-03-18T23:06:04.216308+0000","last_unstale":"2024-03-18T23:07:21.057547+0000","last_undegraded":"2024-03-18T23:07:21.057547+0000","last_fullsized":"2024-03-18T23:07:21.057547+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_clean_scrub_stamp":"2024-03-18T23:06:03.184668+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T23:39:58.537740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":143,"num_read_kb":143,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:01.344160+0000","last_change":"2024-03-18T23:06:04.906669+0000","last_active":"2024-03-18T23:07:01.344160+0000","last_peered":"2024-03-18T23:07:01.344160+0000","last_clean":"2024-03-18T23:07:01.344160+0000","last_became_active":"2024-03-18T23:06:04.906338+0000","last_became_peered":"2024-03-18T23:06:04.906338+0000","last_unstale":"2024-03-18T23:07:01.344160+0000","last_undegraded":"2024-03-18T23:07:01.344160+0000","last_fullsized":"2024-03-18T23:07:01.344160+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:29:39.997671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"86'4","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:01.344846+0000","last_change":"2024-03-18T23:06:04.216177+0000","last_active":"2024-03-18T23:07:01.344846+0000","last_peered":"2024-03-18T23:07:01.344846+0000","last_clean":"2024-03-18T23:07:01.344846+0000","last_became_active":"2024-03-18T23:06:04.215890+0000","last_became_peered":"2024-03-18T23:06:04.215890+0000","last_unstale":"2024-03-18T23:07:01.344846+0000","last_undegraded":"2024-03-18T23:07:01.344846+0000","last_fullsized":"2024-03-18T23:07:01.344846+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_clean_scrub_stamp":"2024-03-18T23:06:03.184668+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T09:33:52.578996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:01.344216+0000","last_change":"2024-03-18T23:06:04.911928+0000","last_active":"2024-03-18T23:07:01.344216+0000","last_peered":"2024-03-18T23:07:01.344216+0000","last_clean":"2024-03-18T23:07:01.344216+0000","last_became_active":"2024-03-18T23:06:04.911501+0000","last_became_peered":"2024-03-18T23:06:04.911501+0000","last_unstale":"2024-03-18T23:07:01.344216+0000","last_undegraded":"2024-03-18T23:07:01.344216+0000","last_fullsized":"2024-03-18T23:07:01.344216+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:05:17.653574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"86'5","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:01.343657+0000","last_change":"2024-03-18T23:06:04.211035+0000","last_active":"2024-03-18T23:07:01.343657+0000","last_peered":"2024-03-18T23:07:01.343657+0000","last_clean":"2024-03-18T23:07:01.343657+0000","last_became_active":"2024-03-18T23:06:04.210665+0000","last_became_peered":"2024-03-18T23:06:04.210665+0000","last_unstale":"2024-03-18T23:07:01.343657+0000","last_undegraded":"2024-03-18T23:07:01.343657+0000","last_fullsized":"2024-03-18T23:07:01.343657+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_clean_scrub_stamp":"2024-03-18T23:06:03.184668+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:49:44.072275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:01.344075+0000","last_change":"2024-03-18T23:06:04.911856+0000","last_active":"2024-03-18T23:07:01.344075+0000","last_peered":"2024-03-18T23:07:01.344075+0000","last_clean":"2024-03-18T23:07:01.344075+0000","last_became_active":"2024-03-18T23:06:04.908034+0000","last_became_peered":"2024-03-18T23:06:04.908034+0000","last_unstale":"2024-03-18T23:07:01.344075+0000","last_undegraded":"2024-03-18T23:07:01.344075+0000","last_fullsized":"2024-03-18T23:07:01.344075+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:38:09.712683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"86'29","reported_seq":201,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:21.057452+0000","last_change":"2024-03-18T23:06:04.211152+0000","last_active":"2024-03-18T23:07:21.057452+0000","last_peered":"2024-03-18T23:07:21.057452+0000","last_clean":"2024-03-18T23:07:21.057452+0000","last_became_active":"2024-03-18T23:06:04.210841+0000","last_became_peered":"2024-03-18T23:06:04.210841+0000","last_unstale":"2024-03-18T23:07:21.057452+0000","last_undegraded":"2024-03-18T23:07:21.057452+0000","last_fullsized":"2024-03-18T23:07:21.057452+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_clean_scrub_stamp":"2024-03-18T23:06:03.184668+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:48:07.935314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":59465,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":153,"num_read_kb":134,"num_write":31,"num_write_kb":67,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:01.347556+0000","last_change":"2024-03-18T23:06:04.910303+0000","last_active":"2024-03-18T23:07:01.347556+0000","last_peered":"2024-03-18T23:07:01.347556+0000","last_clean":"2024-03-18T23:07:01.347556+0000","last_became_active":"2024-03-18T23:06:04.910004+0000","last_became_peered":"2024-03-18T23:06:04.910004+0000","last_unstale":"2024-03-18T23:07:01.347556+0000","last_undegraded":"2024-03-18T23:07:01.347556+0000","last_fullsized":"2024-03-18T23:07:01.347556+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:45:25.750741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"86'9","reported_seq":30,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:01.343486+0000","last_change":"2024-03-18T23:06:04.220318+0000","last_active":"2024-03-18T23:07:01.343486+0000","last_peered":"2024-03-18T23:07:01.343486+0000","last_clean":"2024-03-18T23:07:01.343486+0000","last_became_active":"2024-03-18T23:06:04.219985+0000","last_became_peered":"2024-03-18T23:06:04.219985+0000","last_unstale":"2024-03-18T23:07:01.343486+0000","last_undegraded":"2024-03-18T23:07:01.343486+0000","last_fullsized":"2024-03-18T23:07:01.343486+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_clean_scrub_stamp":"2024-03-18T23:06:03.184668+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:49:00.090356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:01.343340+0000","last_change":"2024-03-18T23:06:04.914456+0000","last_active":"2024-03-18T23:07:01.343340+0000","last_peered":"2024-03-18T23:07:01.343340+0000","last_clean":"2024-03-18T23:07:01.343340+0000","last_became_active":"2024-03-18T23:06:04.914083+0000","last_became_peered":"2024-03-18T23:06:04.914083+0000","last_unstale":"2024-03-18T23:07:01.343340+0000","last_undegraded":"2024-03-18T23:07:01.343340+0000","last_fullsized":"2024-03-18T23:07:01.343340+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:17:00.443254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"86'5","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:01.343123+0000","last_change":"2024-03-18T23:06:04.224808+0000","last_active":"2024-03-18T23:07:01.343123+0000","last_peered":"2024-03-18T23:07:01.343123+0000","last_clean":"2024-03-18T23:07:01.343123+0000","last_became_active":"2024-03-18T23:06:04.224112+0000","last_became_peered":"2024-03-18T23:06:04.224112+0000","last_unstale":"2024-03-18T23:07:01.343123+0000","last_undegraded":"2024-03-18T23:07:01.343123+0000","last_fullsized":"2024-03-18T23:07:01.343123+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_clean_scrub_stamp":"2024-03-18T23:06:03.184668+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:19:20.399875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:01.344917+0000","last_change":"2024-03-18T23:06:04.912271+0000","last_active":"2024-03-18T23:07:01.344917+0000","last_peered":"2024-03-18T23:07:01.344917+0000","last_clean":"2024-03-18T23:07:01.344917+0000","last_became_active":"2024-03-18T23:06:04.911985+0000","last_became_peered":"2024-03-18T23:06:04.911985+0000","last_unstale":"2024-03-18T23:07:01.344917+0000","last_undegraded":"2024-03-18T23:07:01.344917+0000","last_fullsized":"2024-03-18T23:07:01.344917+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T09:18:27.436136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"86'5","reported_seq":99,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:23.085815+0000","last_change":"2024-03-18T23:06:04.209434+0000","last_active":"2024-03-18T23:07:23.085815+0000","last_peered":"2024-03-18T23:07:23.085815+0000","last_clean":"2024-03-18T23:07:23.085815+0000","last_became_active":"2024-03-18T23:06:04.209086+0000","last_became_peered":"2024-03-18T23:06:04.209086+0000","last_unstale":"2024-03-18T23:07:23.085815+0000","last_undegraded":"2024-03-18T23:07:23.085815+0000","last_fullsized":"2024-03-18T23:07:23.085815+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_clean_scrub_stamp":"2024-03-18T23:06:03.184668+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:50:20.823467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":76,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:01.343494+0000","last_change":"2024-03-18T23:06:04.911069+0000","last_active":"2024-03-18T23:07:01.343494+0000","last_peered":"2024-03-18T23:07:01.343494+0000","last_clean":"2024-03-18T23:07:01.343494+0000","last_became_active":"2024-03-18T23:06:04.910760+0000","last_became_peered":"2024-03-18T23:06:04.910760+0000","last_unstale":"2024-03-18T23:07:01.343494+0000","last_undegraded":"2024-03-18T23:07:01.343494+0000","last_fullsized":"2024-03-18T23:07:01.343494+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:16:46.264972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"86'17","reported_seq":37,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:01.345915+0000","last_change":"2024-03-18T23:06:04.216653+0000","last_active":"2024-03-18T23:07:01.345915+0000","last_peered":"2024-03-18T23:07:01.345915+0000","last_clean":"2024-03-18T23:07:01.345915+0000","last_became_active":"2024-03-18T23:06:04.216358+0000","last_became_peered":"2024-03-18T23:06:04.216358+0000","last_unstale":"2024-03-18T23:07:01.345915+0000","last_undegraded":"2024-03-18T23:07:01.345915+0000","last_fullsized":"2024-03-18T23:07:01.345915+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_clean_scrub_stamp":"2024-03-18T23:06:03.184668+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:16:56.500368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:01.342941+0000","last_change":"2024-03-18T23:06:04.917377+0000","last_active":"2024-03-18T23:07:01.342941+0000","last_peered":"2024-03-18T23:07:01.342941+0000","last_clean":"2024-03-18T23:07:01.342941+0000","last_became_active":"2024-03-18T23:06:04.917064+0000","last_became_peered":"2024-03-18T23:06:04.917064+0000","last_unstale":"2024-03-18T23:07:01.342941+0000","last_undegraded":"2024-03-18T23:07:01.342941+0000","last_fullsized":"2024-03-18T23:07:01.342941+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:03:41.518177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"86'2","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:01.348134+0000","last_change":"2024-03-18T23:06:04.210191+0000","last_active":"2024-03-18T23:07:01.348134+0000","last_peered":"2024-03-18T23:07:01.348134+0000","last_clean":"2024-03-18T23:07:01.348134+0000","last_became_active":"2024-03-18T23:06:04.208084+0000","last_became_peered":"2024-03-18T23:06:04.208084+0000","last_unstale":"2024-03-18T23:07:01.348134+0000","last_undegraded":"2024-03-18T23:07:01.348134+0000","last_fullsized":"2024-03-18T23:07:01.348134+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_clean_scrub_stamp":"2024-03-18T23:06:03.184668+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:09:07.220171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:01.341378+0000","last_change":"2024-03-18T23:06:04.910631+0000","last_active":"2024-03-18T23:07:01.341378+0000","last_peered":"2024-03-18T23:07:01.341378+0000","last_clean":"2024-03-18T23:07:01.341378+0000","last_became_active":"2024-03-18T23:06:04.910322+0000","last_became_peered":"2024-03-18T23:06:04.910322+0000","last_unstale":"2024-03-18T23:07:01.341378+0000","last_undegraded":"2024-03-18T23:07:01.341378+0000","last_fullsized":"2024-03-18T23:07:01.341378+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:39:06.389101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"86'1","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:01.340500+0000","last_change":"2024-03-18T23:06:04.220530+0000","last_active":"2024-03-18T23:07:01.340500+0000","last_peered":"2024-03-18T23:07:01.340500+0000","last_clean":"2024-03-18T23:07:01.340500+0000","last_became_active":"2024-03-18T23:06:04.220136+0000","last_became_peered":"2024-03-18T23:06:04.220136+0000","last_unstale":"2024-03-18T23:07:01.340500+0000","last_undegraded":"2024-03-18T23:07:01.340500+0000","last_fullsized":"2024-03-18T23:07:01.340500+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_clean_scrub_stamp":"2024-03-18T23:06:03.184668+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:58:37.037982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"86'1","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:01.346267+0000","last_change":"2024-03-18T23:06:04.912789+0000","last_active":"2024-03-18T23:07:01.346267+0000","last_peered":"2024-03-18T23:07:01.346267+0000","last_clean":"2024-03-18T23:07:01.346267+0000","last_became_active":"2024-03-18T23:06:04.910451+0000","last_became_peered":"2024-03-18T23:06:04.910451+0000","last_unstale":"2024-03-18T23:07:01.346267+0000","last_undegraded":"2024-03-18T23:07:01.346267+0000","last_fullsized":"2024-03-18T23:07:01.346267+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:06:36.539369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"86'5","reported_seq":98,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:23.085960+0000","last_change":"2024-03-18T23:06:04.217992+0000","last_active":"2024-03-18T23:07:23.085960+0000","last_peered":"2024-03-18T23:07:23.085960+0000","last_clean":"2024-03-18T23:07:23.085960+0000","last_became_active":"2024-03-18T23:06:04.217651+0000","last_became_peered":"2024-03-18T23:06:04.217651+0000","last_unstale":"2024-03-18T23:07:23.085960+0000","last_undegraded":"2024-03-18T23:07:23.085960+0000","last_fullsized":"2024-03-18T23:07:23.085960+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_clean_scrub_stamp":"2024-03-18T23:06:03.184668+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:25:46.901251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":74,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:01.346198+0000","last_change":"2024-03-18T23:06:04.913349+0000","last_active":"2024-03-18T23:07:01.346198+0000","last_peered":"2024-03-18T23:07:01.346198+0000","last_clean":"2024-03-18T23:07:01.346198+0000","last_became_active":"2024-03-18T23:06:04.913026+0000","last_became_peered":"2024-03-18T23:06:04.913026+0000","last_unstale":"2024-03-18T23:07:01.346198+0000","last_undegraded":"2024-03-18T23:07:01.346198+0000","last_fullsized":"2024-03-18T23:07:01.346198+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T23:24:28.884524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:01.343112+0000","last_change":"2024-03-18T23:06:04.916541+0000","last_active":"2024-03-18T23:07:01.343112+0000","last_peered":"2024-03-18T23:07:01.343112+0000","last_clean":"2024-03-18T23:07:01.343112+0000","last_became_active":"2024-03-18T23:06:04.916179+0000","last_became_peered":"2024-03-18T23:06:04.916179+0000","last_unstale":"2024-03-18T23:07:01.343112+0000","last_undegraded":"2024-03-18T23:07:01.343112+0000","last_fullsized":"2024-03-18T23:07:01.343112+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:31:59.515767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"86'5","reported_seq":98,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:23.085370+0000","last_change":"2024-03-18T23:06:04.217581+0000","last_active":"2024-03-18T23:07:23.085370+0000","last_peered":"2024-03-18T23:07:23.085370+0000","last_clean":"2024-03-18T23:07:23.085370+0000","last_became_active":"2024-03-18T23:06:04.217191+0000","last_became_peered":"2024-03-18T23:06:04.217191+0000","last_unstale":"2024-03-18T23:07:23.085370+0000","last_undegraded":"2024-03-18T23:07:23.085370+0000","last_fullsized":"2024-03-18T23:07:23.085370+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_clean_scrub_stamp":"2024-03-18T23:06:03.184668+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:47:24.728136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":74,"num_read_kb":74,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:01.345960+0000","last_change":"2024-03-18T23:06:04.919784+0000","last_active":"2024-03-18T23:07:01.345960+0000","last_peered":"2024-03-18T23:07:01.345960+0000","last_clean":"2024-03-18T23:07:01.345960+0000","last_became_active":"2024-03-18T23:06:04.919483+0000","last_became_peered":"2024-03-18T23:06:04.919483+0000","last_unstale":"2024-03-18T23:07:01.345960+0000","last_undegraded":"2024-03-18T23:07:01.345960+0000","last_fullsized":"2024-03-18T23:07:01.345960+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:37:29.635750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'6","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:01.342844+0000","last_change":"2024-03-18T23:06:04.215152+0000","last_active":"2024-03-18T23:07:01.342844+0000","last_peered":"2024-03-18T23:07:01.342844+0000","last_clean":"2024-03-18T23:07:01.342844+0000","last_became_active":"2024-03-18T23:06:04.214789+0000","last_became_peered":"2024-03-18T23:06:04.214789+0000","last_unstale":"2024-03-18T23:07:01.342844+0000","last_undegraded":"2024-03-18T23:07:01.342844+0000","last_fullsized":"2024-03-18T23:07:01.342844+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_clean_scrub_stamp":"2024-03-18T23:06:03.184668+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:58:11.630840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":353,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:01.343583+0000","last_change":"2024-03-18T23:04:54.734000+0000","last_active":"2024-03-18T23:07:01.343583+0000","last_peered":"2024-03-18T23:07:01.343583+0000","last_clean":"2024-03-18T23:07:01.343583+0000","last_became_active":"2024-03-18T23:04:54.733516+0000","last_became_peered":"2024-03-18T23:04:54.733516+0000","last_unstale":"2024-03-18T23:07:01.343583+0000","last_undegraded":"2024-03-18T23:07:01.343583+0000","last_fullsized":"2024-03-18T23:07:01.343583+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:01:28.441034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:01:28.441034+0000","last_clean_scrub_stamp":"2024-03-18T23:01:28.441034+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T23:48:14.029479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:01.343133+0000","last_change":"2024-03-18T23:06:04.908549+0000","last_active":"2024-03-18T23:07:01.343133+0000","last_peered":"2024-03-18T23:07:01.343133+0000","last_clean":"2024-03-18T23:07:01.343133+0000","last_became_active":"2024-03-18T23:06:04.908202+0000","last_became_peered":"2024-03-18T23:06:04.908202+0000","last_unstale":"2024-03-18T23:07:01.343133+0000","last_undegraded":"2024-03-18T23:07:01.343133+0000","last_fullsized":"2024-03-18T23:07:01.343133+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:57:09.486103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'4","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:01.343728+0000","last_change":"2024-03-18T23:06:04.213819+0000","last_active":"2024-03-18T23:07:01.343728+0000","last_peered":"2024-03-18T23:07:01.343728+0000","last_clean":"2024-03-18T23:07:01.343728+0000","last_became_active":"2024-03-18T23:06:04.213418+0000","last_became_peered":"2024-03-18T23:06:04.213418+0000","last_unstale":"2024-03-18T23:07:01.343728+0000","last_undegraded":"2024-03-18T23:07:01.343728+0000","last_fullsized":"2024-03-18T23:07:01.343728+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_clean_scrub_stamp":"2024-03-18T23:06:03.184668+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:50:03.175779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"86'4","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:01.347657+0000","last_change":"2024-03-18T23:06:04.218660+0000","last_active":"2024-03-18T23:07:01.347657+0000","last_peered":"2024-03-18T23:07:01.347657+0000","last_clean":"2024-03-18T23:07:01.347657+0000","last_became_active":"2024-03-18T23:06:04.218183+0000","last_became_peered":"2024-03-18T23:06:04.218183+0000","last_unstale":"2024-03-18T23:07:01.347657+0000","last_undegraded":"2024-03-18T23:07:01.347657+0000","last_fullsized":"2024-03-18T23:07:01.347657+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_clean_scrub_stamp":"2024-03-18T23:06:03.184668+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T08:17:42.208572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:01.343312+0000","last_change":"2024-03-18T23:06:04.901124+0000","last_active":"2024-03-18T23:07:01.343312+0000","last_peered":"2024-03-18T23:07:01.343312+0000","last_clean":"2024-03-18T23:07:01.343312+0000","last_became_active":"2024-03-18T23:06:04.899992+0000","last_became_peered":"2024-03-18T23:06:04.899992+0000","last_unstale":"2024-03-18T23:07:01.343312+0000","last_undegraded":"2024-03-18T23:07:01.343312+0000","last_fullsized":"2024-03-18T23:07:01.343312+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:27:00.965945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"86'12","reported_seq":31,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:01.340611+0000","last_change":"2024-03-18T23:06:04.227413+0000","last_active":"2024-03-18T23:07:01.340611+0000","last_peered":"2024-03-18T23:07:01.340611+0000","last_clean":"2024-03-18T23:07:01.340611+0000","last_became_active":"2024-03-18T23:06:04.227071+0000","last_became_peered":"2024-03-18T23:06:04.227071+0000","last_unstale":"2024-03-18T23:07:01.340611+0000","last_undegraded":"2024-03-18T23:07:01.340611+0000","last_fullsized":"2024-03-18T23:07:01.340611+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_clean_scrub_stamp":"2024-03-18T23:06:03.184668+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T08:15:31.531168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:01.345733+0000","last_change":"2024-03-18T23:06:04.910220+0000","last_active":"2024-03-18T23:07:01.345733+0000","last_peered":"2024-03-18T23:07:01.345733+0000","last_clean":"2024-03-18T23:07:01.345733+0000","last_became_active":"2024-03-18T23:06:04.909855+0000","last_became_peered":"2024-03-18T23:06:04.909855+0000","last_unstale":"2024-03-18T23:07:01.345733+0000","last_undegraded":"2024-03-18T23:07:01.345733+0000","last_fullsized":"2024-03-18T23:07:01.345733+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:19:09.315343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"86'7","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:01.343638+0000","last_change":"2024-03-18T23:06:04.225473+0000","last_active":"2024-03-18T23:07:01.343638+0000","last_peered":"2024-03-18T23:07:01.343638+0000","last_clean":"2024-03-18T23:07:01.343638+0000","last_became_active":"2024-03-18T23:06:04.225126+0000","last_became_peered":"2024-03-18T23:06:04.225126+0000","last_unstale":"2024-03-18T23:07:01.343638+0000","last_undegraded":"2024-03-18T23:07:01.343638+0000","last_fullsized":"2024-03-18T23:07:01.343638+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_clean_scrub_stamp":"2024-03-18T23:06:03.184668+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T09:29:26.438597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:01.347970+0000","last_change":"2024-03-18T23:07:01.347970+0000","last_active":"2024-03-18T23:07:01.347970+0000","last_peered":"2024-03-18T23:07:01.347970+0000","last_clean":"2024-03-18T23:07:01.347970+0000","last_became_active":"2024-03-18T23:07:01.347617+0000","last_became_peered":"2024-03-18T23:07:01.347617+0000","last_unstale":"2024-03-18T23:07:01.347970+0000","last_undegraded":"2024-03-18T23:07:01.347970+0000","last_fullsized":"2024-03-18T23:07:01.347970+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:43:26.292403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:01.343830+0000","last_change":"2024-03-18T23:06:04.216750+0000","last_active":"2024-03-18T23:07:01.343830+0000","last_peered":"2024-03-18T23:07:01.343830+0000","last_clean":"2024-03-18T23:07:01.343830+0000","last_became_active":"2024-03-18T23:06:04.216393+0000","last_became_peered":"2024-03-18T23:06:04.216393+0000","last_unstale":"2024-03-18T23:07:01.343830+0000","last_undegraded":"2024-03-18T23:07:01.343830+0000","last_fullsized":"2024-03-18T23:07:01.343830+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_clean_scrub_stamp":"2024-03-18T23:06:03.184668+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:40:33.486235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:01.345809+0000","last_change":"2024-03-18T23:06:04.915156+0000","last_active":"2024-03-18T23:07:01.345809+0000","last_peered":"2024-03-18T23:07:01.345809+0000","last_clean":"2024-03-18T23:07:01.345809+0000","last_became_active":"2024-03-18T23:06:04.914821+0000","last_became_peered":"2024-03-18T23:06:04.914821+0000","last_unstale":"2024-03-18T23:07:01.345809+0000","last_undegraded":"2024-03-18T23:07:01.345809+0000","last_fullsized":"2024-03-18T23:07:01.345809+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:41:22.344360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"86'7","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:01.345613+0000","last_change":"2024-03-18T23:06:04.224533+0000","last_active":"2024-03-18T23:07:01.345613+0000","last_peered":"2024-03-18T23:07:01.345613+0000","last_clean":"2024-03-18T23:07:01.345613+0000","last_became_active":"2024-03-18T23:06:04.224122+0000","last_became_peered":"2024-03-18T23:06:04.224122+0000","last_unstale":"2024-03-18T23:07:01.345613+0000","last_undegraded":"2024-03-18T23:07:01.345613+0000","last_fullsized":"2024-03-18T23:07:01.345613+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_clean_scrub_stamp":"2024-03-18T23:06:03.184668+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T09:03:40.689085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:01.340600+0000","last_change":"2024-03-18T23:06:04.912574+0000","last_active":"2024-03-18T23:07:01.340600+0000","last_peered":"2024-03-18T23:07:01.340600+0000","last_clean":"2024-03-18T23:07:01.340600+0000","last_became_active":"2024-03-18T23:06:04.912284+0000","last_became_peered":"2024-03-18T23:06:04.912284+0000","last_unstale":"2024-03-18T23:07:01.340600+0000","last_undegraded":"2024-03-18T23:07:01.340600+0000","last_fullsized":"2024-03-18T23:07:01.340600+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:18:11.003763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"86'3","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:01.343310+0000","last_change":"2024-03-18T23:06:04.220364+0000","last_active":"2024-03-18T23:07:01.343310+0000","last_peered":"2024-03-18T23:07:01.343310+0000","last_clean":"2024-03-18T23:07:01.343310+0000","last_became_active":"2024-03-18T23:06:04.220074+0000","last_became_peered":"2024-03-18T23:06:04.220074+0000","last_unstale":"2024-03-18T23:07:01.343310+0000","last_undegraded":"2024-03-18T23:07:01.343310+0000","last_fullsized":"2024-03-18T23:07:01.343310+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_clean_scrub_stamp":"2024-03-18T23:06:03.184668+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:10:15.385724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:01.345915+0000","last_change":"2024-03-18T23:06:04.917448+0000","last_active":"2024-03-18T23:07:01.345915+0000","last_peered":"2024-03-18T23:07:01.345915+0000","last_clean":"2024-03-18T23:07:01.345915+0000","last_became_active":"2024-03-18T23:06:04.916921+0000","last_became_peered":"2024-03-18T23:06:04.916921+0000","last_unstale":"2024-03-18T23:07:01.345915+0000","last_undegraded":"2024-03-18T23:07:01.345915+0000","last_fullsized":"2024-03-18T23:07:01.345915+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:23:06.715597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:01.358216+0000","last_change":"2024-03-18T23:07:01.357779+0000","last_active":"2024-03-18T23:07:01.358216+0000","last_peered":"2024-03-18T23:07:01.358216+0000","last_clean":"2024-03-18T23:07:01.358216+0000","last_became_active":"2024-03-18T23:07:01.346447+0000","last_became_peered":"2024-03-18T23:07:01.346447+0000","last_unstale":"2024-03-18T23:07:01.358216+0000","last_undegraded":"2024-03-18T23:07:01.358216+0000","last_fullsized":"2024-03-18T23:07:01.358216+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_clean_scrub_stamp":"2024-03-18T23:06:03.184668+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:12:17.794463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:01.344313+0000","last_change":"2024-03-18T23:06:04.911471+0000","last_active":"2024-03-18T23:07:01.344313+0000","last_peered":"2024-03-18T23:07:01.344313+0000","last_clean":"2024-03-18T23:07:01.344313+0000","last_became_active":"2024-03-18T23:06:04.910550+0000","last_became_peered":"2024-03-18T23:06:04.910550+0000","last_unstale":"2024-03-18T23:07:01.344313+0000","last_undegraded":"2024-03-18T23:07:01.344313+0000","last_fullsized":"2024-03-18T23:07:01.344313+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:34:55.084581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"86'6","reported_seq":42,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:22.375009+0000","last_change":"2024-03-18T23:07:01.353076+0000","last_active":"2024-03-18T23:07:22.375009+0000","last_peered":"2024-03-18T23:07:22.375009+0000","last_clean":"2024-03-18T23:07:22.375009+0000","last_became_active":"2024-03-18T23:07:01.346752+0000","last_became_peered":"2024-03-18T23:07:01.346752+0000","last_unstale":"2024-03-18T23:07:22.375009+0000","last_undegraded":"2024-03-18T23:07:22.375009+0000","last_fullsized":"2024-03-18T23:07:22.375009+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_clean_scrub_stamp":"2024-03-18T23:06:03.184668+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:05:33.551956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1561,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":23,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":1561,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,0],"acting":[4,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:01.345083+0000","last_change":"2024-03-18T23:06:04.918955+0000","last_active":"2024-03-18T23:07:01.345083+0000","last_peered":"2024-03-18T23:07:01.345083+0000","last_clean":"2024-03-18T23:07:01.345083+0000","last_became_active":"2024-03-18T23:06:04.915475+0000","last_became_peered":"2024-03-18T23:06:04.915475+0000","last_unstale":"2024-03-18T23:07:01.345083+0000","last_undegraded":"2024-03-18T23:07:01.345083+0000","last_fullsized":"2024-03-18T23:07:01.345083+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:07:05.679235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"86'3","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:01.343552+0000","last_change":"2024-03-18T23:06:04.224237+0000","last_active":"2024-03-18T23:07:01.343552+0000","last_peered":"2024-03-18T23:07:01.343552+0000","last_clean":"2024-03-18T23:07:01.343552+0000","last_became_active":"2024-03-18T23:06:04.223820+0000","last_became_peered":"2024-03-18T23:06:04.223820+0000","last_unstale":"2024-03-18T23:07:01.343552+0000","last_undegraded":"2024-03-18T23:07:01.343552+0000","last_fullsized":"2024-03-18T23:07:01.343552+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_clean_scrub_stamp":"2024-03-18T23:06:03.184668+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:17:48.157499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"86'1","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:01.343424+0000","last_change":"2024-03-18T23:06:04.914846+0000","last_active":"2024-03-18T23:07:01.343424+0000","last_peered":"2024-03-18T23:07:01.343424+0000","last_clean":"2024-03-18T23:07:01.343424+0000","last_became_active":"2024-03-18T23:06:04.911629+0000","last_became_peered":"2024-03-18T23:06:04.911629+0000","last_unstale":"2024-03-18T23:07:01.343424+0000","last_undegraded":"2024-03-18T23:07:01.343424+0000","last_fullsized":"2024-03-18T23:07:01.343424+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:17:32.235568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"86'5","reported_seq":36,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:01.356827+0000","last_change":"2024-03-18T23:07:01.356827+0000","last_active":"2024-03-18T23:07:01.356827+0000","last_peered":"2024-03-18T23:07:01.356827+0000","last_clean":"2024-03-18T23:07:01.356827+0000","last_became_active":"2024-03-18T23:07:01.347750+0000","last_became_peered":"2024-03-18T23:07:01.347750+0000","last_unstale":"2024-03-18T23:07:01.356827+0000","last_undegraded":"2024-03-18T23:07:01.356827+0000","last_fullsized":"2024-03-18T23:07:01.356827+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:03.184668+0000","last_clean_scrub_stamp":"2024-03-18T23:06:03.184668+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:35:07.291013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,1],"acting":[6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:01.345190+0000","last_change":"2024-03-18T23:06:04.914463+0000","last_active":"2024-03-18T23:07:01.345190+0000","last_peered":"2024-03-18T23:07:01.345190+0000","last_clean":"2024-03-18T23:07:01.345190+0000","last_became_active":"2024-03-18T23:06:04.914130+0000","last_became_peered":"2024-03-18T23:06:04.914130+0000","last_unstale":"2024-03-18T23:07:01.345190+0000","last_undegraded":"2024-03-18T23:07:01.345190+0000","last_fullsized":"2024-03-18T23:07:01.345190+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:57:56.065100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:01.340664+0000","last_change":"2024-03-18T23:06:04.905951+0000","last_active":"2024-03-18T23:07:01.340664+0000","last_peered":"2024-03-18T23:07:01.340664+0000","last_clean":"2024-03-18T23:07:01.340664+0000","last_became_active":"2024-03-18T23:06:04.905614+0000","last_became_peered":"2024-03-18T23:06:04.905614+0000","last_unstale":"2024-03-18T23:07:01.340664+0000","last_undegraded":"2024-03-18T23:07:01.340664+0000","last_fullsized":"2024-03-18T23:07:01.340664+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:11:44.884671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:01.347053+0000","last_change":"2024-03-18T23:07:01.347053+0000","last_active":"2024-03-18T23:07:01.347053+0000","last_peered":"2024-03-18T23:07:01.347053+0000","last_clean":"2024-03-18T23:07:01.347053+0000","last_became_active":"2024-03-18T23:07:01.346483+0000","last_became_peered":"2024-03-18T23:07:01.346483+0000","last_unstale":"2024-03-18T23:07:01.347053+0000","last_undegraded":"2024-03-18T23:07:01.347053+0000","last_fullsized":"2024-03-18T23:07:01.347053+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:38:44.581164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,4],"acting":[2,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:01.343302+0000","last_change":"2024-03-18T23:06:04.912818+0000","last_active":"2024-03-18T23:07:01.343302+0000","last_peered":"2024-03-18T23:07:01.343302+0000","last_clean":"2024-03-18T23:07:01.343302+0000","last_became_active":"2024-03-18T23:06:04.912511+0000","last_became_peered":"2024-03-18T23:06:04.912511+0000","last_unstale":"2024-03-18T23:07:01.343302+0000","last_undegraded":"2024-03-18T23:07:01.343302+0000","last_fullsized":"2024-03-18T23:07:01.343302+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T09:24:44.743041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:01.343979+0000","last_change":"2024-03-18T23:06:04.912177+0000","last_active":"2024-03-18T23:07:01.343979+0000","last_peered":"2024-03-18T23:07:01.343979+0000","last_clean":"2024-03-18T23:07:01.343979+0000","last_became_active":"2024-03-18T23:06:04.911638+0000","last_became_peered":"2024-03-18T23:06:04.911638+0000","last_unstale":"2024-03-18T23:07:01.343979+0000","last_undegraded":"2024-03-18T23:07:01.343979+0000","last_fullsized":"2024-03-18T23:07:01.343979+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:21:30.648246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:01.343439+0000","last_change":"2024-03-18T23:06:04.912670+0000","last_active":"2024-03-18T23:07:01.343439+0000","last_peered":"2024-03-18T23:07:01.343439+0000","last_clean":"2024-03-18T23:07:01.343439+0000","last_became_active":"2024-03-18T23:06:04.912241+0000","last_became_peered":"2024-03-18T23:06:04.912241+0000","last_unstale":"2024-03-18T23:07:01.343439+0000","last_undegraded":"2024-03-18T23:07:01.343439+0000","last_fullsized":"2024-03-18T23:07:01.343439+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:31:02.666360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:01.341320+0000","last_change":"2024-03-18T23:06:04.909618+0000","last_active":"2024-03-18T23:07:01.341320+0000","last_peered":"2024-03-18T23:07:01.341320+0000","last_clean":"2024-03-18T23:07:01.341320+0000","last_became_active":"2024-03-18T23:06:04.909301+0000","last_became_peered":"2024-03-18T23:06:04.909301+0000","last_unstale":"2024-03-18T23:07:01.341320+0000","last_undegraded":"2024-03-18T23:07:01.341320+0000","last_fullsized":"2024-03-18T23:07:01.341320+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:14:28.101360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:01.343626+0000","last_change":"2024-03-18T23:06:04.904607+0000","last_active":"2024-03-18T23:07:01.343626+0000","last_peered":"2024-03-18T23:07:01.343626+0000","last_clean":"2024-03-18T23:07:01.343626+0000","last_became_active":"2024-03-18T23:06:04.904280+0000","last_became_peered":"2024-03-18T23:06:04.904280+0000","last_unstale":"2024-03-18T23:07:01.343626+0000","last_undegraded":"2024-03-18T23:07:01.343626+0000","last_fullsized":"2024-03-18T23:07:01.343626+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:59:11.445200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:01.345637+0000","last_change":"2024-03-18T23:06:04.918342+0000","last_active":"2024-03-18T23:07:01.345637+0000","last_peered":"2024-03-18T23:07:01.345637+0000","last_clean":"2024-03-18T23:07:01.345637+0000","last_became_active":"2024-03-18T23:06:04.917947+0000","last_became_peered":"2024-03-18T23:06:04.917947+0000","last_unstale":"2024-03-18T23:07:01.345637+0000","last_undegraded":"2024-03-18T23:07:01.345637+0000","last_fullsized":"2024-03-18T23:07:01.345637+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:05:49.347374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:01.343012+0000","last_change":"2024-03-18T23:06:04.912457+0000","last_active":"2024-03-18T23:07:01.343012+0000","last_peered":"2024-03-18T23:07:01.343012+0000","last_clean":"2024-03-18T23:07:01.343012+0000","last_became_active":"2024-03-18T23:06:04.912141+0000","last_became_peered":"2024-03-18T23:06:04.912141+0000","last_unstale":"2024-03-18T23:07:01.343012+0000","last_undegraded":"2024-03-18T23:07:01.343012+0000","last_fullsized":"2024-03-18T23:07:01.343012+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:04:45.847314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:01.345511+0000","last_change":"2024-03-18T23:06:04.918723+0000","last_active":"2024-03-18T23:07:01.345511+0000","last_peered":"2024-03-18T23:07:01.345511+0000","last_clean":"2024-03-18T23:07:01.345511+0000","last_became_active":"2024-03-18T23:06:04.918375+0000","last_became_peered":"2024-03-18T23:06:04.918375+0000","last_unstale":"2024-03-18T23:07:01.345511+0000","last_undegraded":"2024-03-18T23:07:01.345511+0000","last_fullsized":"2024-03-18T23:07:01.345511+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:27:12.868420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:01.344578+0000","last_change":"2024-03-18T23:06:04.913935+0000","last_active":"2024-03-18T23:07:01.344578+0000","last_peered":"2024-03-18T23:07:01.344578+0000","last_clean":"2024-03-18T23:07:01.344578+0000","last_became_active":"2024-03-18T23:06:04.913617+0000","last_became_peered":"2024-03-18T23:06:04.913617+0000","last_unstale":"2024-03-18T23:07:01.344578+0000","last_undegraded":"2024-03-18T23:07:01.344578+0000","last_fullsized":"2024-03-18T23:07:01.344578+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:53:15.585608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:01.343628+0000","last_change":"2024-03-18T23:06:04.911951+0000","last_active":"2024-03-18T23:07:01.343628+0000","last_peered":"2024-03-18T23:07:01.343628+0000","last_clean":"2024-03-18T23:07:01.343628+0000","last_became_active":"2024-03-18T23:06:04.911621+0000","last_became_peered":"2024-03-18T23:06:04.911621+0000","last_unstale":"2024-03-18T23:07:01.343628+0000","last_undegraded":"2024-03-18T23:07:01.343628+0000","last_fullsized":"2024-03-18T23:07:01.343628+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:20:46.600106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:01.349238+0000","last_change":"2024-03-18T23:07:01.349238+0000","last_active":"2024-03-18T23:07:01.349238+0000","last_peered":"2024-03-18T23:07:01.349238+0000","last_clean":"2024-03-18T23:07:01.349238+0000","last_became_active":"2024-03-18T23:07:01.348853+0000","last_became_peered":"2024-03-18T23:07:01.348853+0000","last_unstale":"2024-03-18T23:07:01.349238+0000","last_undegraded":"2024-03-18T23:07:01.349238+0000","last_fullsized":"2024-03-18T23:07:01.349238+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:06:04.189329+0000","last_clean_scrub_stamp":"2024-03-18T23:06:04.189329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T08:16:35.883785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":284006,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":66278,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":712,"num_read_kb":449,"num_write":235,"num_write_kb":130,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2143,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":552960,"data_stored":198834,"data_compressed":5559,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":204,"ondisk_log_size":204,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351005,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":966656,"data_stored":463578,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612640,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28108,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717044,"statfs":{"total":95995035648,"available":95966253056,"internally_reserved":0,"allocated":1114112,"data_stored":1113099,"data_compressed":37968,"data_compressed_allocated":49152,"data_compressed_original":598016,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776167,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":987136,"data_stored":462535,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037805,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32140,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713012,"statfs":{"total":95995035648,"available":95962124288,"internally_reserved":0,"allocated":978944,"data_stored":462331,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332144,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1036288,"data_stored":1055040,"data_compressed":36115,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495671,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36292,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708860,"statfs":{"total":95995035648,"available":95957872640,"internally_reserved":0,"allocated":970752,"data_stored":462252,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574928,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32188,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712964,"statfs":{"total":95995035648,"available":95962075136,"internally_reserved":0,"allocated":1028096,"data_stored":522630,"data_compressed":1853,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738452,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36312,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708840,"statfs":{"total":95995035648,"available":95957852160,"internally_reserved":0,"allocated":991232,"data_stored":463982,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411403,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1040384,"data_stored":521680,"data_compressed":1853,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182469,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":987136,"data_stored":462077,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920833,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32140,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713012,"statfs":{"total":95995035648,"available":95962124288,"internally_reserved":0,"allocated":978944,"data_stored":463674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757307,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32204,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712948,"statfs":{"total":95995035648,"available":95962058752,"internally_reserved":0,"allocated":1048576,"data_stored":1054718,"data_compressed":36115,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34477,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34477,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34477,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36115,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36115,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36115,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36115,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36115,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":61217,"data_compressed":1853,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":60267,"data_compressed":1853,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2261,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":61201,"data_compressed":1853,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2165,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-18T23:07:28.394 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-03-18T23:07:28.394 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-03-18T23:07:28.395 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-03-18T23:07:28.395 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell --fsid c63591ea-e57a-11ee-95c9-87774f69a715 -- ceph health --format=json 2024-03-18T23:07:28.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:07:27 smithi175 ceph-mon[67352]: from='client.14961 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-18T23:07:28.651 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T23:07:29.039 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:07:29 smithi049 ceph-mon[62991]: pgmap v332: 97 pgs: 97 active+clean; 641 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-03-18T23:07:29.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:07:29 smithi139 ceph-mon[67282]: pgmap v332: 97 pgs: 97 active+clean; 641 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-03-18T23:07:29.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:07:29 smithi175 ceph-mon[67352]: pgmap v332: 97 pgs: 97 active+clean; 641 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-03-18T23:07:30.456 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:30.450+0000 7fe719942700 1 -- 172.21.15.49:0/2231504382 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe714108470 msgr2=0x7fe714108850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:30.457 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:30.450+0000 7fe719942700 1 --2- 172.21.15.49:0/2231504382 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe714108470 0x7fe714108850 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fe704009990 tx=0x7fe704009ca0 comp rx=0 tx=0).stop 2024-03-18T23:07:30.457 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:30.450+0000 7fe719942700 1 -- 172.21.15.49:0/2231504382 shutdown_connections 2024-03-18T23:07:30.457 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:30.450+0000 7fe719942700 1 --2- 172.21.15.49:0/2231504382 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe714102d50 0x7fe71410f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:30.457 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:30.450+0000 7fe719942700 1 --2- 172.21.15.49:0/2231504382 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe714108470 0x7fe714108850 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:30.457 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:30.450+0000 7fe719942700 1 --2- 172.21.15.49:0/2231504382 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe714102390 0x7fe714102810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:30.457 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:30.450+0000 7fe719942700 1 -- 172.21.15.49:0/2231504382 >> 172.21.15.49:0/2231504382 conn(0x7fe7140793d0 msgr2=0x7fe7140797e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:07:30.457 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:30.450+0000 7fe719942700 1 -- 172.21.15.49:0/2231504382 shutdown_connections 2024-03-18T23:07:30.457 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:30.450+0000 7fe719942700 1 -- 172.21.15.49:0/2231504382 wait complete. 2024-03-18T23:07:30.457 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:30.451+0000 7fe719942700 1 Processor -- start 2024-03-18T23:07:30.457 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:30.451+0000 7fe719942700 1 -- start start 2024-03-18T23:07:30.457 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:30.451+0000 7fe719942700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe714102390 0x7fe7141a08d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:30.458 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:30.451+0000 7fe719942700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe714102d50 0x7fe7141a0e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:30.458 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:30.451+0000 7fe719942700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe714108470 0x7fe71419aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:30.458 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:30.451+0000 7fe719942700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fe7141125d0 con 0x7fe714102d50 2024-03-18T23:07:30.458 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:30.451+0000 7fe719942700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fe714112450 con 0x7fe714102390 2024-03-18T23:07:30.458 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:30.451+0000 7fe719942700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe714112750 con 0x7fe714108470 2024-03-18T23:07:30.458 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:30.451+0000 7fe70bfff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe714102d50 0x7fe7141a0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:30.458 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:30.451+0000 7fe70bfff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe714102d50 0x7fe7141a0e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:44106/0 (socket says 172.21.15.49:44106) 2024-03-18T23:07:30.458 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:30.451+0000 7fe70bfff700 1 -- 172.21.15.49:0/63410600 learned_addr learned my addr 172.21.15.49:0/63410600 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:07:30.458 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:30.451+0000 7fe7137fe700 1 --2- 172.21.15.49:0/63410600 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe714108470 0x7fe71419aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:30.458 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:30.451+0000 7fe712ffd700 1 --2- 172.21.15.49:0/63410600 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe714102390 0x7fe7141a08d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:30.458 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:30.451+0000 7fe70bfff700 1 -- 172.21.15.49:0/63410600 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe714108470 msgr2=0x7fe71419aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:30.459 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:30.451+0000 7fe70bfff700 1 --2- 172.21.15.49:0/63410600 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe714108470 0x7fe71419aa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:30.459 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:30.451+0000 7fe70bfff700 1 -- 172.21.15.49:0/63410600 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe714102390 msgr2=0x7fe7141a08d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:30.459 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:30.451+0000 7fe70bfff700 1 --2- 172.21.15.49:0/63410600 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe714102390 0x7fe7141a08d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:30.459 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:30.451+0000 7fe70bfff700 1 -- 172.21.15.49:0/63410600 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe71419b1f0 con 0x7fe714102d50 2024-03-18T23:07:30.459 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:30.452+0000 7fe70bfff700 1 --2- 172.21.15.49:0/63410600 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe714102d50 0x7fe7141a0e10 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fe6fc00d950 tx=0x7fe6fc00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:07:30.459 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:30.452+0000 7fe718940700 1 -- 172.21.15.49:0/63410600 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6fc016070 con 0x7fe714102d50 2024-03-18T23:07:30.459 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:30.452+0000 7fe719942700 1 -- 172.21.15.49:0/63410600 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe71419b4e0 con 0x7fe714102d50 2024-03-18T23:07:30.459 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:30.452+0000 7fe718940700 1 -- 172.21.15.49:0/63410600 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe6fc00fd70 con 0x7fe714102d50 2024-03-18T23:07:30.459 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:30.452+0000 7fe718940700 1 -- 172.21.15.49:0/63410600 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6fc00e610 con 0x7fe714102d50 2024-03-18T23:07:30.459 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:30.452+0000 7fe719942700 1 -- 172.21.15.49:0/63410600 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7141a77a0 con 0x7fe714102d50 2024-03-18T23:07:30.459 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:30.453+0000 7fe719942700 1 -- 172.21.15.49:0/63410600 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe714068fd0 con 0x7fe714102d50 2024-03-18T23:07:30.459 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:30.454+0000 7fe718940700 1 -- 172.21.15.49:0/63410600 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 17) ==== 137341+0+0 (secure 0 0 0) 0x7fe6fc019070 con 0x7fe714102d50 2024-03-18T23:07:30.459 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:30.454+0000 7fe718940700 1 --2- 172.21.15.49:0/63410600 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fe7000a2920 0x7fe7000a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:30.460 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:30.455+0000 7fe712ffd700 1 --2- 172.21.15.49:0/63410600 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fe7000a2920 0x7fe7000a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:30.460 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:30.455+0000 7fe718940700 1 -- 172.21.15.49:0/63410600 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7fe6fc0ce650 con 0x7fe714102d50 2024-03-18T23:07:30.460 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:30.455+0000 7fe712ffd700 1 --2- 172.21.15.49:0/63410600 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fe7000a2920 0x7fe7000a4de0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fe71419bbd0 tx=0x7fe704005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:07:30.460 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:30.458+0000 7fe718940700 1 -- 172.21.15.49:0/63410600 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fe6fc09b5a0 con 0x7fe714102d50 2024-03-18T23:07:30.714 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:30.713+0000 7fe719942700 1 -- 172.21.15.49:0/63410600 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fe71402dbb0 con 0x7fe714102d50 2024-03-18T23:07:30.714 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:30.714+0000 7fe718940700 1 -- 172.21.15.49:0/63410600 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7fe6fc09aea0 con 0x7fe714102d50 2024-03-18T23:07:30.714 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-18T23:07:30.714 INFO:teuthology.orchestra.run.smithi049.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-03-18T23:07:30.716 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:30.716+0000 7fe719942700 1 -- 172.21.15.49:0/63410600 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fe7000a2920 msgr2=0x7fe7000a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:30.716 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:30.716+0000 7fe719942700 1 --2- 172.21.15.49:0/63410600 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fe7000a2920 0x7fe7000a4de0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fe71419bbd0 tx=0x7fe704005960 comp rx=0 tx=0).stop 2024-03-18T23:07:30.716 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:30.716+0000 7fe719942700 1 -- 172.21.15.49:0/63410600 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe714102d50 msgr2=0x7fe7141a0e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:30.716 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:30.716+0000 7fe719942700 1 --2- 172.21.15.49:0/63410600 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe714102d50 0x7fe7141a0e10 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fe6fc00d950 tx=0x7fe6fc00dc60 comp rx=0 tx=0).stop 2024-03-18T23:07:30.716 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:30.716+0000 7fe719942700 1 -- 172.21.15.49:0/63410600 shutdown_connections 2024-03-18T23:07:30.716 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:30.716+0000 7fe719942700 1 --2- 172.21.15.49:0/63410600 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fe7000a2920 0x7fe7000a4de0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:30.716 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:30.716+0000 7fe719942700 1 --2- 172.21.15.49:0/63410600 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe714108470 0x7fe71419aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:30.716 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:30.716+0000 7fe719942700 1 --2- 172.21.15.49:0/63410600 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe714102390 0x7fe7141a08d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:30.717 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:30.716+0000 7fe719942700 1 --2- 172.21.15.49:0/63410600 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe714102d50 0x7fe7141a0e10 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:30.717 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:30.716+0000 7fe719942700 1 -- 172.21.15.49:0/63410600 >> 172.21.15.49:0/63410600 conn(0x7fe7140793d0 msgr2=0x7fe71410d8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:07:30.717 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:30.716+0000 7fe719942700 1 -- 172.21.15.49:0/63410600 shutdown_connections 2024-03-18T23:07:30.717 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:30.716+0000 7fe719942700 1 -- 172.21.15.49:0/63410600 wait complete. 2024-03-18T23:07:31.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:07:31 smithi139 ceph-mon[67282]: pgmap v333: 97 pgs: 97 active+clean; 641 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-03-18T23:07:31.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:07:31 smithi139 ceph-mon[67282]: from='client.? 172.21.15.49:0/63410600' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-03-18T23:07:31.370 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:07:31 smithi049 ceph-mon[62991]: pgmap v333: 97 pgs: 97 active+clean; 641 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-03-18T23:07:31.370 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:07:31 smithi049 ceph-mon[62991]: from='client.? 172.21.15.49:0/63410600' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-03-18T23:07:31.401 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-03-18T23:07:31.402 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-18T23:07:31.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:07:31 smithi175 ceph-mon[67352]: pgmap v333: 97 pgs: 97 active+clean; 641 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-03-18T23:07:31.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:07:31 smithi175 ceph-mon[67352]: from='client.? 172.21.15.49:0/63410600' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-03-18T23:07:31.519 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:31.517+0000 7f94c540b640 1 -- 172.21.15.49:0/3505123017 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f94c0101850 msgr2=0x7f94c0101c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:31.519 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:31.517+0000 7f94c540b640 1 --2- 172.21.15.49:0/3505123017 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f94c0101850 0x7f94c0101c70 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f94a8009900 tx=0x7f94a802bc30 comp rx=0 tx=0).stop 2024-03-18T23:07:31.519 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:31.517+0000 7f94c540b640 1 -- 172.21.15.49:0/3505123017 shutdown_connections 2024-03-18T23:07:31.519 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:31.517+0000 7f94c540b640 1 --2- 172.21.15.49:0/3505123017 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f94c0103400 0x7f94c0109c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:31.519 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:31.517+0000 7f94c540b640 1 --2- 172.21.15.49:0/3505123017 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f94c0102a40 0x7f94c0102ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:31.519 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:31.517+0000 7f94c540b640 1 --2- 172.21.15.49:0/3505123017 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f94c0101850 0x7f94c0101c70 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:31.520 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:31.517+0000 7f94c540b640 1 -- 172.21.15.49:0/3505123017 >> 172.21.15.49:0/3505123017 conn(0x7f94c00fce00 msgr2=0x7f94c00ff220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:07:31.520 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:31.517+0000 7f94c540b640 1 -- 172.21.15.49:0/3505123017 shutdown_connections 2024-03-18T23:07:31.520 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:31.517+0000 7f94c540b640 1 -- 172.21.15.49:0/3505123017 wait complete. 2024-03-18T23:07:31.520 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:31.518+0000 7f94c540b640 1 Processor -- start 2024-03-18T23:07:31.520 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:31.518+0000 7f94c540b640 1 -- start start 2024-03-18T23:07:31.520 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:31.518+0000 7f94c540b640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f94c0101850 0x7f94c0197ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:31.520 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:31.518+0000 7f94c540b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f94c0102a40 0x7f94c01980e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:31.520 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:31.518+0000 7f94c540b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f94c0103400 0x7f94c019f150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:31.520 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:31.518+0000 7f94c540b640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f94c010d4c0 con 0x7f94c0102a40 2024-03-18T23:07:31.520 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:31.518+0000 7f94c540b640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f94c010d340 con 0x7f94c0103400 2024-03-18T23:07:31.520 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:31.518+0000 7f94c540b640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f94c010d640 con 0x7f94c0101850 2024-03-18T23:07:31.520 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:31.518+0000 7f94be7fc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f94c0102a40 0x7f94c01980e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:31.520 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:31.518+0000 7f94be7fc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f94c0102a40 0x7f94c01980e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:44114/0 (socket says 172.21.15.49:44114) 2024-03-18T23:07:31.520 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:31.518+0000 7f94be7fc640 1 -- 172.21.15.49:0/989350235 learned_addr learned my addr 172.21.15.49:0/989350235 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:07:31.521 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:31.518+0000 7f94beffd640 1 --2- 172.21.15.49:0/989350235 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f94c0101850 0x7f94c0197ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:31.521 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:31.518+0000 7f94bf7fe640 1 --2- 172.21.15.49:0/989350235 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f94c0103400 0x7f94c019f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:31.521 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:31.518+0000 7f94be7fc640 1 -- 172.21.15.49:0/989350235 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f94c0101850 msgr2=0x7f94c0197ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:31.521 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:31.518+0000 7f94be7fc640 1 --2- 172.21.15.49:0/989350235 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f94c0101850 0x7f94c0197ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:31.521 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:31.518+0000 7f94be7fc640 1 -- 172.21.15.49:0/989350235 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f94c0103400 msgr2=0x7f94c019f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:31.521 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:31.518+0000 7f94be7fc640 1 --2- 172.21.15.49:0/989350235 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f94c0103400 0x7f94c019f150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:31.521 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:31.518+0000 7f94be7fc640 1 -- 172.21.15.49:0/989350235 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94c019f7c0 con 0x7f94c0102a40 2024-03-18T23:07:31.521 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:31.519+0000 7f94be7fc640 1 --2- 172.21.15.49:0/989350235 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f94c0102a40 0x7f94c01980e0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f94b000e9f0 tx=0x7f94b000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:07:31.521 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:31.519+0000 7f94b7fff640 1 -- 172.21.15.49:0/989350235 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94b000cd80 con 0x7f94c0102a40 2024-03-18T23:07:31.521 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:31.519+0000 7f94c540b640 1 -- 172.21.15.49:0/989350235 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94c019fab0 con 0x7f94c0102a40 2024-03-18T23:07:31.521 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:31.519+0000 7f94b7fff640 1 -- 172.21.15.49:0/989350235 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f94b0005500 con 0x7f94c0102a40 2024-03-18T23:07:31.521 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:31.519+0000 7f94b7fff640 1 -- 172.21.15.49:0/989350235 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94b0010690 con 0x7f94c0102a40 2024-03-18T23:07:31.521 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:31.519+0000 7f94c540b640 1 -- 172.21.15.49:0/989350235 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94c019ffb0 con 0x7f94c0102a40 2024-03-18T23:07:31.522 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:31.520+0000 7f94c540b640 1 -- 172.21.15.49:0/989350235 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94c0108ce0 con 0x7f94c0102a40 2024-03-18T23:07:31.523 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:31.521+0000 7f94b7fff640 1 -- 172.21.15.49:0/989350235 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7f94b00046c0 con 0x7f94c0102a40 2024-03-18T23:07:31.523 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:31.522+0000 7f94b7fff640 1 --2- 172.21.15.49:0/989350235 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f94ac0a2800 0x7f94ac0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:31.523 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:31.522+0000 7f94beffd640 1 --2- 172.21.15.49:0/989350235 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f94ac0a2800 0x7f94ac0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:31.523 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:31.522+0000 7f94b7fff640 1 -- 172.21.15.49:0/989350235 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f94b00cf020 con 0x7f94c0102a40 2024-03-18T23:07:31.523 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:31.522+0000 7f94beffd640 1 --2- 172.21.15.49:0/989350235 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f94ac0a2800 0x7f94ac0a4cc0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f94a8000c00 tx=0x7f94a8005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:07:31.524 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:31.525+0000 7f94b7fff640 1 -- 172.21.15.49:0/989350235 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f94b0053740 con 0x7f94c0102a40 2024-03-18T23:07:31.710 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:31.710+0000 7f94c540b640 1 -- 172.21.15.49:0/989350235 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f94c002d440 con 0x7f94c0102a40 2024-03-18T23:07:31.711 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:31.711+0000 7f94b7fff640 1 -- 172.21.15.49:0/989350235 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+22674 (secure 0 0 0) 0x7f94b009c0d0 con 0x7f94c0102a40 2024-03-18T23:07:31.711 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-18T23:07:31.712 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":88,"fsid":"c63591ea-e57a-11ee-95c9-87774f69a715","created":"2024-03-18T22:57:58.734467+0000","modified":"2024-03-18T23:07:01.337701+0000","last_up_change":"2024-03-18T23:05:16.576310+0000","last_in_change":"2024-03-18T23:04:57.024067+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-18T23:01:28.363432+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-18T23:06:03.172963+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-18T23:06:03.973303+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e7fc9daa-4098-4ba5-8019-9aaef1aab3e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":1651162501},{"type":"v1","addr":"172.21.15.49:6803","nonce":1651162501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":1651162501},{"type":"v1","addr":"172.21.15.49:6805","nonce":1651162501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":1651162501},{"type":"v1","addr":"172.21.15.49:6809","nonce":1651162501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":1651162501},{"type":"v1","addr":"172.21.15.49:6807","nonce":1651162501}]},"public_addr":"172.21.15.49:6803/1651162501","cluster_addr":"172.21.15.49:6805/1651162501","heartbeat_back_addr":"172.21.15.49:6809/1651162501","heartbeat_front_addr":"172.21.15.49:6807/1651162501","state":["exists","up"]},{"osd":1,"uuid":"40ac929c-e90f-434c-a796-9aabd686f1fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":1766700359},{"type":"v1","addr":"172.21.15.139:6801","nonce":1766700359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":1766700359},{"type":"v1","addr":"172.21.15.139:6803","nonce":1766700359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":1766700359},{"type":"v1","addr":"172.21.15.139:6807","nonce":1766700359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":1766700359},{"type":"v1","addr":"172.21.15.139:6805","nonce":1766700359}]},"public_addr":"172.21.15.139:6801/1766700359","cluster_addr":"172.21.15.139:6803/1766700359","heartbeat_back_addr":"172.21.15.139:6807/1766700359","heartbeat_front_addr":"172.21.15.139:6805/1766700359","state":["exists","up"]},{"osd":2,"uuid":"8ae3b666-00b4-45ed-a5ee-4d0771edabb6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":3752671234},{"type":"v1","addr":"172.21.15.175:6801","nonce":3752671234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":3752671234},{"type":"v1","addr":"172.21.15.175:6803","nonce":3752671234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":3752671234},{"type":"v1","addr":"172.21.15.175:6807","nonce":3752671234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":3752671234},{"type":"v1","addr":"172.21.15.175:6805","nonce":3752671234}]},"public_addr":"172.21.15.175:6801/3752671234","cluster_addr":"172.21.15.175:6803/3752671234","heartbeat_back_addr":"172.21.15.175:6807/3752671234","heartbeat_front_addr":"172.21.15.175:6805/3752671234","state":["exists","up"]},{"osd":3,"uuid":"1a62e89d-45d4-4237-b813-7f96a59aca7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":956160536},{"type":"v1","addr":"172.21.15.49:6811","nonce":956160536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":956160536},{"type":"v1","addr":"172.21.15.49:6813","nonce":956160536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":956160536},{"type":"v1","addr":"172.21.15.49:6817","nonce":956160536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":956160536},{"type":"v1","addr":"172.21.15.49:6815","nonce":956160536}]},"public_addr":"172.21.15.49:6811/956160536","cluster_addr":"172.21.15.49:6813/956160536","heartbeat_back_addr":"172.21.15.49:6817/956160536","heartbeat_front_addr":"172.21.15.49:6815/956160536","state":["exists","up"]},{"osd":4,"uuid":"c715101c-0c3f-4b98-b142-d64a35e187df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":3129712668},{"type":"v1","addr":"172.21.15.139:6809","nonce":3129712668}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":3129712668},{"type":"v1","addr":"172.21.15.139:6811","nonce":3129712668}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":3129712668},{"type":"v1","addr":"172.21.15.139:6815","nonce":3129712668}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":3129712668},{"type":"v1","addr":"172.21.15.139:6813","nonce":3129712668}]},"public_addr":"172.21.15.139:6809/3129712668","cluster_addr":"172.21.15.139:6811/3129712668","heartbeat_back_addr":"172.21.15.139:6815/3129712668","heartbeat_front_addr":"172.21.15.139:6813/3129712668","state":["exists","up"]},{"osd":5,"uuid":"4b641f81-77eb-45fd-8464-511e7b3062b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":2774567215},{"type":"v1","addr":"172.21.15.175:6809","nonce":2774567215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":2774567215},{"type":"v1","addr":"172.21.15.175:6811","nonce":2774567215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":2774567215},{"type":"v1","addr":"172.21.15.175:6815","nonce":2774567215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":2774567215},{"type":"v1","addr":"172.21.15.175:6813","nonce":2774567215}]},"public_addr":"172.21.15.175:6809/2774567215","cluster_addr":"172.21.15.175:6811/2774567215","heartbeat_back_addr":"172.21.15.175:6815/2774567215","heartbeat_front_addr":"172.21.15.175:6813/2774567215","state":["exists","up"]},{"osd":6,"uuid":"653da8e0-1197-4b48-abed-470f2a9c4dce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":1472215461},{"type":"v1","addr":"172.21.15.49:6819","nonce":1472215461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":1472215461},{"type":"v1","addr":"172.21.15.49:6821","nonce":1472215461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":1472215461},{"type":"v1","addr":"172.21.15.49:6825","nonce":1472215461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":1472215461},{"type":"v1","addr":"172.21.15.49:6823","nonce":1472215461}]},"public_addr":"172.21.15.49:6819/1472215461","cluster_addr":"172.21.15.49:6821/1472215461","heartbeat_back_addr":"172.21.15.49:6825/1472215461","heartbeat_front_addr":"172.21.15.49:6823/1472215461","state":["exists","up"]},{"osd":7,"uuid":"ad157c9b-4b75-451c-8740-630f0975b7fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":2163986787},{"type":"v1","addr":"172.21.15.139:6817","nonce":2163986787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":2163986787},{"type":"v1","addr":"172.21.15.139:6819","nonce":2163986787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":2163986787},{"type":"v1","addr":"172.21.15.139:6823","nonce":2163986787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":2163986787},{"type":"v1","addr":"172.21.15.139:6821","nonce":2163986787}]},"public_addr":"172.21.15.139:6817/2163986787","cluster_addr":"172.21.15.139:6819/2163986787","heartbeat_back_addr":"172.21.15.139:6823/2163986787","heartbeat_front_addr":"172.21.15.139:6821/2163986787","state":["exists","up"]},{"osd":8,"uuid":"c8e75f15-df75-4d5d-9b92-a064bffb6c01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":1194026992},{"type":"v1","addr":"172.21.15.175:6817","nonce":1194026992}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1194026992},{"type":"v1","addr":"172.21.15.175:6819","nonce":1194026992}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":1194026992},{"type":"v1","addr":"172.21.15.175:6823","nonce":1194026992}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1194026992},{"type":"v1","addr":"172.21.15.175:6821","nonce":1194026992}]},"public_addr":"172.21.15.175:6817/1194026992","cluster_addr":"172.21.15.175:6819/1194026992","heartbeat_back_addr":"172.21.15.175:6823/1194026992","heartbeat_front_addr":"172.21.15.175:6821/1194026992","state":["exists","up"]},{"osd":9,"uuid":"edb5ffef-b6a2-4941-b5ab-40f982ad30d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":2742084884},{"type":"v1","addr":"172.21.15.49:6827","nonce":2742084884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":2742084884},{"type":"v1","addr":"172.21.15.49:6829","nonce":2742084884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":2742084884},{"type":"v1","addr":"172.21.15.49:6833","nonce":2742084884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":2742084884},{"type":"v1","addr":"172.21.15.49:6831","nonce":2742084884}]},"public_addr":"172.21.15.49:6827/2742084884","cluster_addr":"172.21.15.49:6829/2742084884","heartbeat_back_addr":"172.21.15.49:6833/2742084884","heartbeat_front_addr":"172.21.15.49:6831/2742084884","state":["exists","up"]},{"osd":10,"uuid":"3945e46d-766a-4c4a-8821-d43648dd1578","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":2404740201},{"type":"v1","addr":"172.21.15.139:6825","nonce":2404740201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":2404740201},{"type":"v1","addr":"172.21.15.139:6827","nonce":2404740201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":2404740201},{"type":"v1","addr":"172.21.15.139:6831","nonce":2404740201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":2404740201},{"type":"v1","addr":"172.21.15.139:6829","nonce":2404740201}]},"public_addr":"172.21.15.139:6825/2404740201","cluster_addr":"172.21.15.139:6827/2404740201","heartbeat_back_addr":"172.21.15.139:6831/2404740201","heartbeat_front_addr":"172.21.15.139:6829/2404740201","state":["exists","up"]},{"osd":11,"uuid":"9954ba15-d877-46cc-b24a-893f9168deb8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":3519503868},{"type":"v1","addr":"172.21.15.175:6825","nonce":3519503868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":3519503868},{"type":"v1","addr":"172.21.15.175:6827","nonce":3519503868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":3519503868},{"type":"v1","addr":"172.21.15.175:6831","nonce":3519503868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":3519503868},{"type":"v1","addr":"172.21.15.175:6829","nonce":3519503868}]},"public_addr":"172.21.15.175:6825/3519503868","cluster_addr":"172.21.15.175:6827/3519503868","heartbeat_back_addr":"172.21.15.175:6831/3519503868","heartbeat_front_addr":"172.21.15.175:6829/3519503868","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:00:38.367865+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:01:02.237687+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:01:25.609751+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:01:51.268572+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:02:16.514605+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:02:41.586523+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:03:07.029964+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:03:32.956250+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:03:57.547984+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:04:23.948972+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:04:49.667210+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:05:15.284693+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.6","mappings":[{"from":4,"to":6}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.49:0/563527416":"2024-03-19T22:58:59.806253+0000","172.21.15.49:0/1036383534":"2024-03-19T22:58:59.806253+0000","172.21.15.49:6801/2609458951":"2024-03-19T22:58:59.806253+0000","172.21.15.49:0/21283102":"2024-03-19T22:58:31.972474+0000","172.21.15.49:0/1292058922":"2024-03-19T22:58:31.972474+0000","172.21.15.49:6800/1985190312":"2024-03-19T22:58:31.972474+0000","172.21.15.49:0/1912446700":"2024-03-19T22:58:59.806253+0000","172.21.15.49:0/2176210039":"2024-03-19T22:58:31.972474+0000","172.21.15.49:6801/1985190312":"2024-03-19T22:58:31.972474+0000","172.21.15.49:6800/2609458951":"2024-03-19T22:58:59.806253+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-18T23:07:31.713 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:31.713+0000 7f94c540b640 1 -- 172.21.15.49:0/989350235 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f94ac0a2800 msgr2=0x7f94ac0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:31.713 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:31.713+0000 7f94c540b640 1 --2- 172.21.15.49:0/989350235 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f94ac0a2800 0x7f94ac0a4cc0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f94a8000c00 tx=0x7f94a8005fd0 comp rx=0 tx=0).stop 2024-03-18T23:07:31.713 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:31.713+0000 7f94c540b640 1 -- 172.21.15.49:0/989350235 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f94c0102a40 msgr2=0x7f94c01980e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:31.714 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:31.713+0000 7f94c540b640 1 --2- 172.21.15.49:0/989350235 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f94c0102a40 0x7f94c01980e0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f94b000e9f0 tx=0x7f94b000edf0 comp rx=0 tx=0).stop 2024-03-18T23:07:31.714 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:31.713+0000 7f94c540b640 1 -- 172.21.15.49:0/989350235 shutdown_connections 2024-03-18T23:07:31.714 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:31.713+0000 7f94c540b640 1 --2- 172.21.15.49:0/989350235 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f94ac0a2800 0x7f94ac0a4cc0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:31.714 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:31.713+0000 7f94c540b640 1 --2- 172.21.15.49:0/989350235 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f94c0103400 0x7f94c019f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:31.714 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:31.713+0000 7f94c540b640 1 --2- 172.21.15.49:0/989350235 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f94c0102a40 0x7f94c01980e0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:31.714 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:31.713+0000 7f94c540b640 1 --2- 172.21.15.49:0/989350235 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f94c0101850 0x7f94c0197ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:31.714 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:31.713+0000 7f94c540b640 1 -- 172.21.15.49:0/989350235 >> 172.21.15.49:0/989350235 conn(0x7f94c00fce00 msgr2=0x7f94c0105eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:07:31.714 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:31.713+0000 7f94c540b640 1 -- 172.21.15.49:0/989350235 shutdown_connections 2024-03-18T23:07:31.714 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:31.713+0000 7f94c540b640 1 -- 172.21.15.49:0/989350235 wait complete. 2024-03-18T23:07:31.725 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-18T23:01:28.363432+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-18T23:06:03.172963+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-18T23:06:03.973303+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-18T23:07:31.725 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-18T23:07:31.839 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:31.837+0000 7f47e21b9640 1 -- 172.21.15.49:0/3061759980 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f47dc101840 msgr2=0x7f47dc101c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:31.840 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:31.837+0000 7f47e21b9640 1 --2- 172.21.15.49:0/3061759980 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f47dc101840 0x7f47dc101c60 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f47cc009900 tx=0x7f47cc02bc30 comp rx=0 tx=0).stop 2024-03-18T23:07:31.840 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:31.837+0000 7f47e21b9640 1 -- 172.21.15.49:0/3061759980 shutdown_connections 2024-03-18T23:07:31.840 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:31.837+0000 7f47e21b9640 1 --2- 172.21.15.49:0/3061759980 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f47dc1033f0 0x7f47dc109c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:31.840 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:31.837+0000 7f47e21b9640 1 --2- 172.21.15.49:0/3061759980 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f47dc102a30 0x7f47dc102eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:31.840 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:31.837+0000 7f47e21b9640 1 --2- 172.21.15.49:0/3061759980 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f47dc101840 0x7f47dc101c60 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:31.840 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:31.837+0000 7f47e21b9640 1 -- 172.21.15.49:0/3061759980 >> 172.21.15.49:0/3061759980 conn(0x7f47dc0fcdb0 msgr2=0x7f47dc0ff1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:07:31.840 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:31.837+0000 7f47e21b9640 1 -- 172.21.15.49:0/3061759980 shutdown_connections 2024-03-18T23:07:31.840 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:31.837+0000 7f47e21b9640 1 -- 172.21.15.49:0/3061759980 wait complete. 2024-03-18T23:07:31.840 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:31.838+0000 7f47e21b9640 1 Processor -- start 2024-03-18T23:07:31.840 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:31.838+0000 7f47e21b9640 1 -- start start 2024-03-18T23:07:31.840 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:31.838+0000 7f47e21b9640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f47dc101840 0x7f47dc197b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:31.840 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:31.838+0000 7f47e21b9640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f47dc102a30 0x7f47dc198070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:31.840 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:31.838+0000 7f47e21b9640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f47dc1033f0 0x7f47dc19f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:31.840 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:31.838+0000 7f47e21b9640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f47dc10d4c0 con 0x7f47dc1033f0 2024-03-18T23:07:31.841 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:31.838+0000 7f47e21b9640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f47dc10d340 con 0x7f47dc102a30 2024-03-18T23:07:31.841 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:31.838+0000 7f47e21b9640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f47dc10d640 con 0x7f47dc101840 2024-03-18T23:07:31.841 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:31.838+0000 7f47dbfff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f47dc1033f0 0x7f47dc19f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:31.841 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:31.838+0000 7f47daffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f47dc102a30 0x7f47dc198070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:31.841 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:31.838+0000 7f47dbfff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f47dc1033f0 0x7f47dc19f0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:44128/0 (socket says 172.21.15.49:44128) 2024-03-18T23:07:31.841 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:31.838+0000 7f47dbfff640 1 -- 172.21.15.49:0/176251154 learned_addr learned my addr 172.21.15.49:0/176251154 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:07:31.841 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:31.838+0000 7f47daffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f47dc102a30 0x7f47dc198070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.49:33416/0 (socket says 172.21.15.49:33416) 2024-03-18T23:07:31.841 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:31.838+0000 7f47db7fe640 1 --2- 172.21.15.49:0/176251154 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f47dc101840 0x7f47dc197b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:31.841 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:31.839+0000 7f47dbfff640 1 -- 172.21.15.49:0/176251154 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f47dc101840 msgr2=0x7f47dc197b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:31.841 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:31.839+0000 7f47dbfff640 1 --2- 172.21.15.49:0/176251154 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f47dc101840 0x7f47dc197b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:31.841 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:31.839+0000 7f47dbfff640 1 -- 172.21.15.49:0/176251154 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f47dc102a30 msgr2=0x7f47dc198070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:31.841 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:31.839+0000 7f47dbfff640 1 --2- 172.21.15.49:0/176251154 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f47dc102a30 0x7f47dc198070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:31.841 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:31.839+0000 7f47dbfff640 1 -- 172.21.15.49:0/176251154 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47dc19f750 con 0x7f47dc1033f0 2024-03-18T23:07:31.841 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:31.839+0000 7f47db7fe640 1 --2- 172.21.15.49:0/176251154 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f47dc101840 0x7f47dc197b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:07:31.842 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:31.839+0000 7f47daffd640 1 --2- 172.21.15.49:0/176251154 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f47dc102a30 0x7f47dc198070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:07:31.842 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:31.839+0000 7f47dbfff640 1 --2- 172.21.15.49:0/176251154 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f47dc1033f0 0x7f47dc19f0e0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f47d400eb10 tx=0x7f47d400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:07:31.842 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:31.839+0000 7f47d8ff9640 1 -- 172.21.15.49:0/176251154 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47d4004b20 con 0x7f47dc1033f0 2024-03-18T23:07:31.842 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:31.839+0000 7f47d8ff9640 1 -- 172.21.15.49:0/176251154 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f47d4004cc0 con 0x7f47dc1033f0 2024-03-18T23:07:31.842 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:31.839+0000 7f47e21b9640 1 -- 172.21.15.49:0/176251154 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47dc19fa40 con 0x7f47dc1033f0 2024-03-18T23:07:31.842 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:31.839+0000 7f47d8ff9640 1 -- 172.21.15.49:0/176251154 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47d4010760 con 0x7f47dc1033f0 2024-03-18T23:07:31.842 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:31.839+0000 7f47e21b9640 1 -- 172.21.15.49:0/176251154 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47dc19ffe0 con 0x7f47dc1033f0 2024-03-18T23:07:31.843 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:31.840+0000 7f47e21b9640 1 -- 172.21.15.49:0/176251154 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47dc108be0 con 0x7f47dc1033f0 2024-03-18T23:07:31.843 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:31.841+0000 7f47d8ff9640 1 -- 172.21.15.49:0/176251154 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7f47d4005000 con 0x7f47dc1033f0 2024-03-18T23:07:31.843 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:31.842+0000 7f47d8ff9640 1 --2- 172.21.15.49:0/176251154 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f47c40a2880 0x7f47c40a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:31.843 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:31.842+0000 7f47db7fe640 1 --2- 172.21.15.49:0/176251154 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f47c40a2880 0x7f47c40a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:31.843 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:31.842+0000 7f47d8ff9640 1 -- 172.21.15.49:0/176251154 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f47d40cf420 con 0x7f47dc1033f0 2024-03-18T23:07:31.843 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:31.842+0000 7f47db7fe640 1 --2- 172.21.15.49:0/176251154 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f47c40a2880 0x7f47c40a4d40 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f47cc000c00 tx=0x7f47cc005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:07:31.845 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:31.845+0000 7f47d8ff9640 1 -- 172.21.15.49:0/176251154 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f47d409c4d0 con 0x7f47dc1033f0 2024-03-18T23:07:32.028 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:32.028+0000 7f47e21b9640 1 -- 172.21.15.49:0/176251154 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f47dc02d440 con 0x7f47dc1033f0 2024-03-18T23:07:32.028 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:32.028+0000 7f47d8ff9640 1 -- 172.21.15.49:0/176251154 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) ==== 93+0+10 (secure 0 0 0) 0x7f47d409bdd0 con 0x7f47dc1033f0 2024-03-18T23:07:32.028 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 1 2024-03-18T23:07:32.030 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:32.030+0000 7f47e21b9640 1 -- 172.21.15.49:0/176251154 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f47c40a2880 msgr2=0x7f47c40a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:32.031 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:32.030+0000 7f47e21b9640 1 --2- 172.21.15.49:0/176251154 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f47c40a2880 0x7f47c40a4d40 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f47cc000c00 tx=0x7f47cc005fd0 comp rx=0 tx=0).stop 2024-03-18T23:07:32.031 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:32.030+0000 7f47e21b9640 1 -- 172.21.15.49:0/176251154 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f47dc1033f0 msgr2=0x7f47dc19f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:32.031 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:32.030+0000 7f47e21b9640 1 --2- 172.21.15.49:0/176251154 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f47dc1033f0 0x7f47dc19f0e0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f47d400eb10 tx=0x7f47d400c2d0 comp rx=0 tx=0).stop 2024-03-18T23:07:32.031 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:32.030+0000 7f47e21b9640 1 -- 172.21.15.49:0/176251154 shutdown_connections 2024-03-18T23:07:32.031 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:32.030+0000 7f47e21b9640 1 --2- 172.21.15.49:0/176251154 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f47c40a2880 0x7f47c40a4d40 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:32.031 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:32.030+0000 7f47e21b9640 1 --2- 172.21.15.49:0/176251154 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f47dc1033f0 0x7f47dc19f0e0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:32.031 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:32.030+0000 7f47e21b9640 1 --2- 172.21.15.49:0/176251154 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f47dc102a30 0x7f47dc198070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:32.031 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:32.030+0000 7f47e21b9640 1 --2- 172.21.15.49:0/176251154 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f47dc101840 0x7f47dc197b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:32.031 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:32.030+0000 7f47e21b9640 1 -- 172.21.15.49:0/176251154 >> 172.21.15.49:0/176251154 conn(0x7f47dc0fcdb0 msgr2=0x7f47dc105db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:07:32.031 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:32.031+0000 7f47e21b9640 1 -- 172.21.15.49:0/176251154 shutdown_connections 2024-03-18T23:07:32.031 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:32.031+0000 7f47e21b9640 1 -- 172.21.15.49:0/176251154 wait complete. 2024-03-18T23:07:32.042 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-18T23:07:32.156 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:32.154+0000 7fa9f373b640 1 -- 172.21.15.49:0/2411864674 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa9ec101840 msgr2=0x7fa9ec101c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:32.156 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:32.154+0000 7fa9f373b640 1 --2- 172.21.15.49:0/2411864674 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa9ec101840 0x7fa9ec101c60 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fa9dc009900 tx=0x7fa9dc02bc30 comp rx=0 tx=0).stop 2024-03-18T23:07:32.156 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:32.154+0000 7fa9f373b640 1 -- 172.21.15.49:0/2411864674 shutdown_connections 2024-03-18T23:07:32.157 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:32.154+0000 7fa9f373b640 1 --2- 172.21.15.49:0/2411864674 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa9ec1033f0 0x7fa9ec109c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:32.157 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:32.154+0000 7fa9f373b640 1 --2- 172.21.15.49:0/2411864674 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa9ec102a30 0x7fa9ec102eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:32.157 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:32.154+0000 7fa9f373b640 1 --2- 172.21.15.49:0/2411864674 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa9ec101840 0x7fa9ec101c60 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:32.157 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:32.154+0000 7fa9f373b640 1 -- 172.21.15.49:0/2411864674 >> 172.21.15.49:0/2411864674 conn(0x7fa9ec0fcdb0 msgr2=0x7fa9ec0ff1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:07:32.157 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:32.154+0000 7fa9f373b640 1 -- 172.21.15.49:0/2411864674 shutdown_connections 2024-03-18T23:07:32.157 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:32.154+0000 7fa9f373b640 1 -- 172.21.15.49:0/2411864674 wait complete. 2024-03-18T23:07:32.157 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:32.154+0000 7fa9f373b640 1 Processor -- start 2024-03-18T23:07:32.157 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:32.154+0000 7fa9f373b640 1 -- start start 2024-03-18T23:07:32.157 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:32.155+0000 7fa9f373b640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa9ec101840 0x7fa9ec197b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:32.158 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:32.155+0000 7fa9f373b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa9ec102a30 0x7fa9ec198070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:32.158 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:32.155+0000 7fa9f373b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa9ec1033f0 0x7fa9ec19f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:32.158 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:32.155+0000 7fa9f373b640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fa9ec10d4c0 con 0x7fa9ec1033f0 2024-03-18T23:07:32.158 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:32.155+0000 7fa9f373b640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fa9ec10d340 con 0x7fa9ec102a30 2024-03-18T23:07:32.158 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:32.155+0000 7fa9f373b640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa9ec10d640 con 0x7fa9ec101840 2024-03-18T23:07:32.158 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:32.155+0000 7fa9f1cd8640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa9ec1033f0 0x7fa9ec19f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:32.158 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:32.155+0000 7fa9f1cd8640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa9ec1033f0 0x7fa9ec19f0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:44138/0 (socket says 172.21.15.49:44138) 2024-03-18T23:07:32.158 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:32.155+0000 7fa9f1cd8640 1 -- 172.21.15.49:0/3242945707 learned_addr learned my addr 172.21.15.49:0/3242945707 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:07:32.158 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:32.155+0000 7fa9f0cd6640 1 --2- 172.21.15.49:0/3242945707 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa9ec102a30 0x7fa9ec198070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:32.158 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:32.155+0000 7fa9f14d7640 1 --2- 172.21.15.49:0/3242945707 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa9ec101840 0x7fa9ec197b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:32.158 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:32.155+0000 7fa9f1cd8640 1 -- 172.21.15.49:0/3242945707 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa9ec101840 msgr2=0x7fa9ec197b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:32.158 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:32.155+0000 7fa9f1cd8640 1 --2- 172.21.15.49:0/3242945707 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa9ec101840 0x7fa9ec197b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:32.159 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:32.155+0000 7fa9f1cd8640 1 -- 172.21.15.49:0/3242945707 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa9ec102a30 msgr2=0x7fa9ec198070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:32.159 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:32.155+0000 7fa9f1cd8640 1 --2- 172.21.15.49:0/3242945707 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa9ec102a30 0x7fa9ec198070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:32.159 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:32.155+0000 7fa9f1cd8640 1 -- 172.21.15.49:0/3242945707 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9ec19f750 con 0x7fa9ec1033f0 2024-03-18T23:07:32.159 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:32.155+0000 7fa9f1cd8640 1 --2- 172.21.15.49:0/3242945707 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa9ec1033f0 0x7fa9ec19f0e0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fa9e400eb10 tx=0x7fa9e400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:07:32.159 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:32.155+0000 7fa9e27fc640 1 -- 172.21.15.49:0/3242945707 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9e4004b20 con 0x7fa9ec1033f0 2024-03-18T23:07:32.159 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:32.155+0000 7fa9f373b640 1 -- 172.21.15.49:0/3242945707 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9ec19fa40 con 0x7fa9ec1033f0 2024-03-18T23:07:32.159 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:32.155+0000 7fa9e27fc640 1 -- 172.21.15.49:0/3242945707 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa9e4004cc0 con 0x7fa9ec1033f0 2024-03-18T23:07:32.159 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:32.155+0000 7fa9e27fc640 1 -- 172.21.15.49:0/3242945707 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9e4010760 con 0x7fa9ec1033f0 2024-03-18T23:07:32.159 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:32.155+0000 7fa9f373b640 1 -- 172.21.15.49:0/3242945707 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9ec19ff90 con 0x7fa9ec1033f0 2024-03-18T23:07:32.159 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:32.156+0000 7fa9f373b640 1 -- 172.21.15.49:0/3242945707 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9ec108be0 con 0x7fa9ec1033f0 2024-03-18T23:07:32.160 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:32.157+0000 7fa9e27fc640 1 -- 172.21.15.49:0/3242945707 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7fa9e4005020 con 0x7fa9ec1033f0 2024-03-18T23:07:32.160 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:32.158+0000 7fa9e27fc640 1 --2- 172.21.15.49:0/3242945707 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fa9d80a2800 0x7fa9d80a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:32.160 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:32.158+0000 7fa9f14d7640 1 --2- 172.21.15.49:0/3242945707 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fa9d80a2800 0x7fa9d80a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:32.160 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:32.158+0000 7fa9e27fc640 1 -- 172.21.15.49:0/3242945707 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7fa9e40cf130 con 0x7fa9ec1033f0 2024-03-18T23:07:32.161 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:32.158+0000 7fa9f14d7640 1 --2- 172.21.15.49:0/3242945707 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fa9d80a2800 0x7fa9d80a4cc0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fa9dc000c00 tx=0x7fa9dc005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:07:32.162 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:32.162+0000 7fa9e27fc640 1 -- 172.21.15.49:0/3242945707 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fa9e409c1e0 con 0x7fa9ec1033f0 2024-03-18T23:07:32.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:07:32 smithi139 ceph-mon[67282]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-03-18T23:07:32.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:07:32 smithi139 ceph-mon[67282]: from='client.? 172.21.15.49:0/989350235' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T23:07:32.348 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:32.348+0000 7fa9f373b640 1 -- 172.21.15.49:0/3242945707 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fa9ec062420 con 0x7fa9ec1033f0 2024-03-18T23:07:32.348 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:32.348+0000 7fa9e27fc640 1 -- 172.21.15.49:0/3242945707 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) ==== 104+0+11 (secure 0 0 0) 0x7fa9e409bae0 con 0x7fa9ec1033f0 2024-03-18T23:07:32.348 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 32 2024-03-18T23:07:32.350 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:32.350+0000 7fa9f373b640 1 -- 172.21.15.49:0/3242945707 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fa9d80a2800 msgr2=0x7fa9d80a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:32.351 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:32.350+0000 7fa9f373b640 1 --2- 172.21.15.49:0/3242945707 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fa9d80a2800 0x7fa9d80a4cc0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fa9dc000c00 tx=0x7fa9dc005fd0 comp rx=0 tx=0).stop 2024-03-18T23:07:32.351 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:32.350+0000 7fa9f373b640 1 -- 172.21.15.49:0/3242945707 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa9ec1033f0 msgr2=0x7fa9ec19f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:32.351 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:32.350+0000 7fa9f373b640 1 --2- 172.21.15.49:0/3242945707 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa9ec1033f0 0x7fa9ec19f0e0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fa9e400eb10 tx=0x7fa9e400c2d0 comp rx=0 tx=0).stop 2024-03-18T23:07:32.351 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:32.350+0000 7fa9f373b640 1 -- 172.21.15.49:0/3242945707 shutdown_connections 2024-03-18T23:07:32.351 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:32.350+0000 7fa9f373b640 1 --2- 172.21.15.49:0/3242945707 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fa9d80a2800 0x7fa9d80a4cc0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:32.351 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:32.350+0000 7fa9f373b640 1 --2- 172.21.15.49:0/3242945707 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa9ec1033f0 0x7fa9ec19f0e0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:32.351 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:32.350+0000 7fa9f373b640 1 --2- 172.21.15.49:0/3242945707 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa9ec102a30 0x7fa9ec198070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:32.351 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:32.350+0000 7fa9f373b640 1 --2- 172.21.15.49:0/3242945707 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa9ec101840 0x7fa9ec197b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:32.351 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:32.350+0000 7fa9f373b640 1 -- 172.21.15.49:0/3242945707 >> 172.21.15.49:0/3242945707 conn(0x7fa9ec0fcdb0 msgr2=0x7fa9ec105db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:07:32.351 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:32.351+0000 7fa9f373b640 1 -- 172.21.15.49:0/3242945707 shutdown_connections 2024-03-18T23:07:32.351 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:32.351+0000 7fa9f373b640 1 -- 172.21.15.49:0/3242945707 wait complete. 2024-03-18T23:07:32.360 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:07:32 smithi049 ceph-mon[62991]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-03-18T23:07:32.361 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:07:32 smithi049 ceph-mon[62991]: from='client.? 172.21.15.49:0/989350235' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T23:07:32.362 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-18T23:07:32.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:07:32 smithi175 ceph-mon[67352]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-03-18T23:07:32.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:07:32 smithi175 ceph-mon[67352]: from='client.? 172.21.15.49:0/989350235' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T23:07:32.475 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:32.473+0000 7f1e518bf640 1 -- 172.21.15.49:0/2796200449 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1e4c1022f0 msgr2=0x7f1e4c108b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:32.475 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:32.473+0000 7f1e518bf640 1 --2- 172.21.15.49:0/2796200449 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1e4c1022f0 0x7f1e4c108b80 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f1e44009ca0 tx=0x7f1e4402c0f0 comp rx=0 tx=0).stop 2024-03-18T23:07:32.475 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:32.473+0000 7f1e518bf640 1 -- 172.21.15.49:0/2796200449 shutdown_connections 2024-03-18T23:07:32.475 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:32.473+0000 7f1e518bf640 1 --2- 172.21.15.49:0/2796200449 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1e4c1022f0 0x7f1e4c108b80 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:32.475 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:32.473+0000 7f1e518bf640 1 --2- 172.21.15.49:0/2796200449 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1e4c101930 0x7f1e4c101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:32.475 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:32.473+0000 7f1e518bf640 1 --2- 172.21.15.49:0/2796200449 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1e4c100740 0x7f1e4c100b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:32.476 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:32.473+0000 7f1e518bf640 1 -- 172.21.15.49:0/2796200449 >> 172.21.15.49:0/2796200449 conn(0x7f1e4c0fbcd0 msgr2=0x7f1e4c0fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:07:32.476 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:32.473+0000 7f1e518bf640 1 -- 172.21.15.49:0/2796200449 shutdown_connections 2024-03-18T23:07:32.476 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:32.473+0000 7f1e518bf640 1 -- 172.21.15.49:0/2796200449 wait complete. 2024-03-18T23:07:32.476 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:32.473+0000 7f1e518bf640 1 Processor -- start 2024-03-18T23:07:32.476 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:32.473+0000 7f1e518bf640 1 -- start start 2024-03-18T23:07:32.476 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:32.474+0000 7f1e518bf640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1e4c100740 0x7f1e4c1969e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:32.476 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:32.474+0000 7f1e518bf640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1e4c101930 0x7f1e4c196f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:32.476 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:32.474+0000 7f1e518bf640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1e4c1022f0 0x7f1e4c19df90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:32.476 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:32.474+0000 7f1e518bf640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f1e4c10c3c0 con 0x7f1e4c101930 2024-03-18T23:07:32.476 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:32.474+0000 7f1e518bf640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f1e4c10c240 con 0x7f1e4c100740 2024-03-18T23:07:32.476 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:32.474+0000 7f1e518bf640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f1e4c10c540 con 0x7f1e4c1022f0 2024-03-18T23:07:32.476 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:32.474+0000 7f1e4a7fc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1e4c101930 0x7f1e4c196f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:32.477 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:32.474+0000 7f1e4a7fc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1e4c101930 0x7f1e4c196f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:44160/0 (socket says 172.21.15.49:44160) 2024-03-18T23:07:32.477 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:32.474+0000 7f1e4a7fc640 1 -- 172.21.15.49:0/4126874327 learned_addr learned my addr 172.21.15.49:0/4126874327 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:07:32.477 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:32.474+0000 7f1e4affd640 1 --2- 172.21.15.49:0/4126874327 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1e4c100740 0x7f1e4c1969e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:32.477 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:32.474+0000 7f1e4b7fe640 1 --2- 172.21.15.49:0/4126874327 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1e4c1022f0 0x7f1e4c19df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:32.477 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:32.474+0000 7f1e4a7fc640 1 -- 172.21.15.49:0/4126874327 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1e4c1022f0 msgr2=0x7f1e4c19df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:32.477 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:32.474+0000 7f1e4a7fc640 1 --2- 172.21.15.49:0/4126874327 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1e4c1022f0 0x7f1e4c19df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:32.477 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:32.474+0000 7f1e4a7fc640 1 -- 172.21.15.49:0/4126874327 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1e4c100740 msgr2=0x7f1e4c1969e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:32.477 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:32.474+0000 7f1e4a7fc640 1 --2- 172.21.15.49:0/4126874327 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1e4c100740 0x7f1e4c1969e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:32.477 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:32.474+0000 7f1e4a7fc640 1 -- 172.21.15.49:0/4126874327 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e4c19e600 con 0x7f1e4c101930 2024-03-18T23:07:32.477 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:32.474+0000 7f1e4affd640 1 --2- 172.21.15.49:0/4126874327 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1e4c100740 0x7f1e4c1969e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:07:32.477 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:32.474+0000 7f1e4a7fc640 1 --2- 172.21.15.49:0/4126874327 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1e4c101930 0x7f1e4c196f20 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f1e3c00e9f0 tx=0x7f1e3c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:07:32.477 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:32.474+0000 7f1e508bd640 1 -- 172.21.15.49:0/4126874327 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e3c00cd80 con 0x7f1e4c101930 2024-03-18T23:07:32.477 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:32.474+0000 7f1e508bd640 1 -- 172.21.15.49:0/4126874327 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1e3c005500 con 0x7f1e4c101930 2024-03-18T23:07:32.477 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:32.474+0000 7f1e518bf640 1 -- 172.21.15.49:0/4126874327 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e4c19e8f0 con 0x7f1e4c101930 2024-03-18T23:07:32.478 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:32.474+0000 7f1e508bd640 1 -- 172.21.15.49:0/4126874327 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e3c010690 con 0x7f1e4c101930 2024-03-18T23:07:32.478 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:32.474+0000 7f1e518bf640 1 -- 172.21.15.49:0/4126874327 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e4c19ee40 con 0x7f1e4c101930 2024-03-18T23:07:32.478 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:32.475+0000 7f1e518bf640 1 -- 172.21.15.49:0/4126874327 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e4c107a60 con 0x7f1e4c101930 2024-03-18T23:07:32.478 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:32.476+0000 7f1e508bd640 1 -- 172.21.15.49:0/4126874327 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7f1e3c0046c0 con 0x7f1e4c101930 2024-03-18T23:07:32.479 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:32.477+0000 7f1e508bd640 1 --2- 172.21.15.49:0/4126874327 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f1e340a2880 0x7f1e340a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:32.479 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:32.477+0000 7f1e508bd640 1 -- 172.21.15.49:0/4126874327 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f1e3c0cedd0 con 0x7f1e4c101930 2024-03-18T23:07:32.479 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:32.477+0000 7f1e4affd640 1 --2- 172.21.15.49:0/4126874327 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f1e340a2880 0x7f1e340a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:32.479 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:32.477+0000 7f1e4affd640 1 --2- 172.21.15.49:0/4126874327 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f1e340a2880 0x7f1e340a4d40 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f1e4c101790 tx=0x7f1e40005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:07:32.481 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:32.481+0000 7f1e508bd640 1 -- 172.21.15.49:0/4126874327 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f1e3c09be80 con 0x7f1e4c101930 2024-03-18T23:07:32.664 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:32.664+0000 7f1e518bf640 1 -- 172.21.15.49:0/4126874327 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f1e4c062420 con 0x7f1e4c101930 2024-03-18T23:07:32.665 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:32.665+0000 7f1e508bd640 1 -- 172.21.15.49:0/4126874327 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) ==== 100+0+11 (secure 0 0 0) 0x7f1e3c09b780 con 0x7f1e4c101930 2024-03-18T23:07:32.665 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 64 2024-03-18T23:07:32.667 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:32.667+0000 7f1e518bf640 1 -- 172.21.15.49:0/4126874327 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f1e340a2880 msgr2=0x7f1e340a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:32.667 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:32.667+0000 7f1e518bf640 1 --2- 172.21.15.49:0/4126874327 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f1e340a2880 0x7f1e340a4d40 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f1e4c101790 tx=0x7f1e40005f90 comp rx=0 tx=0).stop 2024-03-18T23:07:32.667 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:32.667+0000 7f1e518bf640 1 -- 172.21.15.49:0/4126874327 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1e4c101930 msgr2=0x7f1e4c196f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:32.667 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:32.667+0000 7f1e518bf640 1 --2- 172.21.15.49:0/4126874327 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1e4c101930 0x7f1e4c196f20 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f1e3c00e9f0 tx=0x7f1e3c00edf0 comp rx=0 tx=0).stop 2024-03-18T23:07:32.667 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:32.667+0000 7f1e518bf640 1 -- 172.21.15.49:0/4126874327 shutdown_connections 2024-03-18T23:07:32.667 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:32.667+0000 7f1e518bf640 1 --2- 172.21.15.49:0/4126874327 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f1e340a2880 0x7f1e340a4d40 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:32.667 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:32.667+0000 7f1e518bf640 1 --2- 172.21.15.49:0/4126874327 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1e4c1022f0 0x7f1e4c19df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:32.667 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:32.667+0000 7f1e518bf640 1 --2- 172.21.15.49:0/4126874327 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1e4c101930 0x7f1e4c196f20 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:32.667 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:32.667+0000 7f1e518bf640 1 --2- 172.21.15.49:0/4126874327 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1e4c100740 0x7f1e4c1969e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:32.667 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:32.667+0000 7f1e518bf640 1 -- 172.21.15.49:0/4126874327 >> 172.21.15.49:0/4126874327 conn(0x7f1e4c0fbcd0 msgr2=0x7f1e4c104c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:07:32.667 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:32.667+0000 7f1e518bf640 1 -- 172.21.15.49:0/4126874327 shutdown_connections 2024-03-18T23:07:32.668 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:32.667+0000 7f1e518bf640 1 -- 172.21.15.49:0/4126874327 wait complete. 2024-03-18T23:07:32.678 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-18T23:07:32.788 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:32.786+0000 7f32db3a9640 1 -- 172.21.15.49:0/8689367 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f32d41075a0 msgr2=0x7f32d41099f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:32.788 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:32.786+0000 7f32db3a9640 1 --2- 172.21.15.49:0/8689367 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f32d41075a0 0x7f32d41099f0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f32cc009ca0 tx=0x7f32cc02c0f0 comp rx=0 tx=0).stop 2024-03-18T23:07:32.788 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:32.786+0000 7f32db3a9640 1 -- 172.21.15.49:0/8689367 shutdown_connections 2024-03-18T23:07:32.788 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:32.786+0000 7f32db3a9640 1 --2- 172.21.15.49:0/8689367 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f32d41075a0 0x7f32d41099f0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:32.789 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:32.786+0000 7f32db3a9640 1 --2- 172.21.15.49:0/8689367 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f32d4104c70 0x7f32d4107060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:32.789 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:32.786+0000 7f32db3a9640 1 --2- 172.21.15.49:0/8689367 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f32d4102340 0x7f32d4104730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:32.789 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:32.786+0000 7f32db3a9640 1 -- 172.21.15.49:0/8689367 >> 172.21.15.49:0/8689367 conn(0x7f32d40fbd50 msgr2=0x7f32d40fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:07:32.789 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:32.786+0000 7f32db3a9640 1 -- 172.21.15.49:0/8689367 shutdown_connections 2024-03-18T23:07:32.789 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:32.786+0000 7f32db3a9640 1 -- 172.21.15.49:0/8689367 wait complete. 2024-03-18T23:07:32.789 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:32.786+0000 7f32db3a9640 1 Processor -- start 2024-03-18T23:07:32.789 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:32.787+0000 7f32db3a9640 1 -- start start 2024-03-18T23:07:32.789 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:32.787+0000 7f32db3a9640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f32d4102340 0x7f32d4197af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:32.789 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:32.787+0000 7f32db3a9640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f32d4104c70 0x7f32d4198030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:32.789 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:32.787+0000 7f32db3a9640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f32d41075a0 0x7f32d419f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:32.789 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:32.787+0000 7f32db3a9640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f32d410d4f0 con 0x7f32d4102340 2024-03-18T23:07:32.789 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:32.787+0000 7f32db3a9640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f32d410d370 con 0x7f32d4104c70 2024-03-18T23:07:32.789 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:32.787+0000 7f32db3a9640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f32d410d670 con 0x7f32d41075a0 2024-03-18T23:07:32.790 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:32.787+0000 7f32d9145640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f32d4102340 0x7f32d4197af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:32.790 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:32.787+0000 7f32d9145640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f32d4102340 0x7f32d4197af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:44182/0 (socket says 172.21.15.49:44182) 2024-03-18T23:07:32.790 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:32.787+0000 7f32d9145640 1 -- 172.21.15.49:0/324824060 learned_addr learned my addr 172.21.15.49:0/324824060 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:07:32.790 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:32.787+0000 7f32d9946640 1 --2- 172.21.15.49:0/324824060 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f32d41075a0 0x7f32d419f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:32.790 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:32.787+0000 7f32d8944640 1 --2- 172.21.15.49:0/324824060 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f32d4104c70 0x7f32d4198030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:32.790 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:32.787+0000 7f32d9145640 1 -- 172.21.15.49:0/324824060 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f32d41075a0 msgr2=0x7f32d419f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:32.790 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:32.787+0000 7f32d9145640 1 --2- 172.21.15.49:0/324824060 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f32d41075a0 0x7f32d419f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:32.790 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:32.787+0000 7f32d9145640 1 -- 172.21.15.49:0/324824060 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f32d4104c70 msgr2=0x7f32d4198030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:32.790 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:32.787+0000 7f32d9145640 1 --2- 172.21.15.49:0/324824060 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f32d4104c70 0x7f32d4198030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:32.790 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:32.787+0000 7f32d9145640 1 -- 172.21.15.49:0/324824060 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32d419f710 con 0x7f32d4102340 2024-03-18T23:07:32.790 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:32.787+0000 7f32d9145640 1 --2- 172.21.15.49:0/324824060 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f32d4102340 0x7f32d4197af0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f32c400e9f0 tx=0x7f32c400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:07:32.790 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:32.787+0000 7f32d27fc640 1 -- 172.21.15.49:0/324824060 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32c400cd80 con 0x7f32d4102340 2024-03-18T23:07:32.791 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:32.787+0000 7f32d27fc640 1 -- 172.21.15.49:0/324824060 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f32c4005500 con 0x7f32d4102340 2024-03-18T23:07:32.791 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:32.787+0000 7f32d27fc640 1 -- 172.21.15.49:0/324824060 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32c4010690 con 0x7f32d4102340 2024-03-18T23:07:32.791 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:32.787+0000 7f32db3a9640 1 -- 172.21.15.49:0/324824060 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32d419fa00 con 0x7f32d4102340 2024-03-18T23:07:32.791 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:32.787+0000 7f32db3a9640 1 -- 172.21.15.49:0/324824060 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32d419ff00 con 0x7f32d4102340 2024-03-18T23:07:32.791 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:32.788+0000 7f32db3a9640 1 -- 172.21.15.49:0/324824060 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32d40fd840 con 0x7f32d4102340 2024-03-18T23:07:32.791 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:32.790+0000 7f32d27fc640 1 -- 172.21.15.49:0/324824060 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7f32c40046c0 con 0x7f32d4102340 2024-03-18T23:07:32.792 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:32.790+0000 7f32d27fc640 1 --2- 172.21.15.49:0/324824060 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f32bc0a2800 0x7f32bc0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:32.792 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:32.790+0000 7f32d8944640 1 --2- 172.21.15.49:0/324824060 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f32bc0a2800 0x7f32bc0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:32.792 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:32.791+0000 7f32d27fc640 1 -- 172.21.15.49:0/324824060 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f32c40cee40 con 0x7f32d4102340 2024-03-18T23:07:32.792 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:32.791+0000 7f32d8944640 1 --2- 172.21.15.49:0/324824060 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f32bc0a2800 0x7f32bc0a4cc0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f32d4199000 tx=0x7f32c8005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:07:32.795 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:32.795+0000 7f32d27fc640 1 -- 172.21.15.49:0/324824060 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f32c409bef0 con 0x7f32d4102340 2024-03-18T23:07:33.021 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:33.021+0000 7f32db3a9640 1 -- 172.21.15.49:0/324824060 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f32d402d1f0 con 0x7f32d4102340 2024-03-18T23:07:33.021 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:33.021+0000 7f32d27fc640 1 -- 172.21.15.49:0/324824060 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11936 (secure 0 0 0) 0x7f32c409b7f0 con 0x7f32d4102340 2024-03-18T23:07:33.021 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-18T23:07:33.022 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6835/3631301630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":3631301630},{"type":"v1","addr":"172.21.15.49:6835","nonce":3631301630}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6837/444305233","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":444305233},{"type":"v1","addr":"172.21.15.49:6837","nonce":444305233}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-18T23:06:04.575999+0000","modified":"2024-03-18T23:06:11.998736+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24488,"mds_1":24463,"mds_2":24430,"mds_3":24419,"mds_4":24391},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14622":{"gid":14622,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.49:6839/1934085093","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6838","nonce":1934085093},{"type":"v1","addr":"172.21.15.49:6839","nonce":1934085093}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14634":{"gid":14634,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.49:6841/2869298862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6840","nonce":2869298862},{"type":"v1","addr":"172.21.15.49:6841","nonce":2869298862}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24389":{"gid":24389,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6833/3766178603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3766178603},{"type":"v1","addr":"172.21.15.139:6833","nonce":3766178603}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24391":{"gid":24391,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.175:6833/1702997173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":1702997173},{"type":"v1","addr":"172.21.15.175:6833","nonce":1702997173}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":9,"addr":"172.21.15.139:6835/1663212937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":1663212937},{"type":"v1","addr":"172.21.15.139:6835","nonce":1663212937}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.175:6835/755474931","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":755474931},{"type":"v1","addr":"172.21.15.175:6835","nonce":755474931}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6837/379482756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":379482756},{"type":"v1","addr":"172.21.15.175:6837","nonce":379482756}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6837/2330317540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6836","nonce":2330317540},{"type":"v1","addr":"172.21.15.139:6837","nonce":2330317540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"172.21.15.175:6839/3640882084","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":3640882084},{"type":"v1","addr":"172.21.15.175:6839","nonce":3640882084}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.139:6839/4280128209","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6838","nonce":4280128209},{"type":"v1","addr":"172.21.15.139:6839","nonce":4280128209}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24463,24488,24430,24419,24391]},"id":1}]} 2024-03-18T23:07:33.022 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 27 2024-03-18T23:07:33.024 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:33.024+0000 7f32db3a9640 1 -- 172.21.15.49:0/324824060 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f32bc0a2800 msgr2=0x7f32bc0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:33.024 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:33.024+0000 7f32db3a9640 1 --2- 172.21.15.49:0/324824060 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f32bc0a2800 0x7f32bc0a4cc0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f32d4199000 tx=0x7f32c8005f90 comp rx=0 tx=0).stop 2024-03-18T23:07:33.024 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:33.024+0000 7f32db3a9640 1 -- 172.21.15.49:0/324824060 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f32d4102340 msgr2=0x7f32d4197af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:33.024 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:33.024+0000 7f32db3a9640 1 --2- 172.21.15.49:0/324824060 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f32d4102340 0x7f32d4197af0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f32c400e9f0 tx=0x7f32c400edf0 comp rx=0 tx=0).stop 2024-03-18T23:07:33.024 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:33.024+0000 7f32db3a9640 1 -- 172.21.15.49:0/324824060 shutdown_connections 2024-03-18T23:07:33.024 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:33.024+0000 7f32db3a9640 1 --2- 172.21.15.49:0/324824060 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f32bc0a2800 0x7f32bc0a4cc0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:33.024 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:33.024+0000 7f32db3a9640 1 --2- 172.21.15.49:0/324824060 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f32d41075a0 0x7f32d419f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:33.024 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:33.024+0000 7f32db3a9640 1 --2- 172.21.15.49:0/324824060 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f32d4104c70 0x7f32d4198030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:33.024 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:33.024+0000 7f32db3a9640 1 --2- 172.21.15.49:0/324824060 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f32d4102340 0x7f32d4197af0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:33.025 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:33.024+0000 7f32db3a9640 1 -- 172.21.15.49:0/324824060 >> 172.21.15.49:0/324824060 conn(0x7f32d40fbd50 msgr2=0x7f32d4102c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:07:33.025 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:33.024+0000 7f32db3a9640 1 -- 172.21.15.49:0/324824060 shutdown_connections 2024-03-18T23:07:33.025 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:33.024+0000 7f32db3a9640 1 -- 172.21.15.49:0/324824060 wait complete. 2024-03-18T23:07:33.035 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-18T23:07:33.145 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:33.143+0000 7fb6ceef1640 1 -- 172.21.15.49:0/3944672529 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb6c80ffb30 msgr2=0x7fb6c80fff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:33.145 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:33.143+0000 7fb6ceef1640 1 --2- 172.21.15.49:0/3944672529 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb6c80ffb30 0x7fb6c80fff50 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fb6bc009860 tx=0x7fb6bc02bb70 comp rx=0 tx=0).stop 2024-03-18T23:07:33.146 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:33.143+0000 7fb6ceef1640 1 -- 172.21.15.49:0/3944672529 shutdown_connections 2024-03-18T23:07:33.146 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:33.143+0000 7fb6ceef1640 1 --2- 172.21.15.49:0/3944672529 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb6c8103560 0x7fb6c8109df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:33.146 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:33.143+0000 7fb6ceef1640 1 --2- 172.21.15.49:0/3944672529 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb6c8102ba0 0x7fb6c8103020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:33.146 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:33.143+0000 7fb6ceef1640 1 --2- 172.21.15.49:0/3944672529 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb6c80ffb30 0x7fb6c80fff50 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:33.146 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:33.143+0000 7fb6ceef1640 1 -- 172.21.15.49:0/3944672529 >> 172.21.15.49:0/3944672529 conn(0x7fb6c80fb440 msgr2=0x7fb6c80fd880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:07:33.146 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:33.143+0000 7fb6ceef1640 1 -- 172.21.15.49:0/3944672529 shutdown_connections 2024-03-18T23:07:33.146 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:33.143+0000 7fb6ceef1640 1 -- 172.21.15.49:0/3944672529 wait complete. 2024-03-18T23:07:33.146 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:33.144+0000 7fb6ceef1640 1 Processor -- start 2024-03-18T23:07:33.146 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:33.144+0000 7fb6ceef1640 1 -- start start 2024-03-18T23:07:33.146 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:33.144+0000 7fb6ceef1640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb6c80ffb30 0x7fb6c8197b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:33.146 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:33.144+0000 7fb6ceef1640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb6c8102ba0 0x7fb6c81980d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:33.146 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:33.144+0000 7fb6ceef1640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb6c8103560 0x7fb6c819f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:33.146 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:33.144+0000 7fb6ceef1640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fb6c810d4e0 con 0x7fb6c8102ba0 2024-03-18T23:07:33.147 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:33.144+0000 7fb6ceef1640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fb6c810d360 con 0x7fb6c8103560 2024-03-18T23:07:33.147 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:33.144+0000 7fb6ceef1640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb6c810d660 con 0x7fb6c80ffb30 2024-03-18T23:07:33.147 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:33.144+0000 7fb6c7fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb6c8102ba0 0x7fb6c81980d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:33.147 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:33.144+0000 7fb6c7fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb6c8102ba0 0x7fb6c81980d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:44200/0 (socket says 172.21.15.49:44200) 2024-03-18T23:07:33.147 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:33.144+0000 7fb6c7fff640 1 -- 172.21.15.49:0/2446534496 learned_addr learned my addr 172.21.15.49:0/2446534496 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:07:33.147 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:33.144+0000 7fb6ccc8d640 1 --2- 172.21.15.49:0/2446534496 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb6c80ffb30 0x7fb6c8197b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:33.147 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:33.144+0000 7fb6cd48e640 1 --2- 172.21.15.49:0/2446534496 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb6c8103560 0x7fb6c819f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:33.147 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:33.144+0000 7fb6c7fff640 1 -- 172.21.15.49:0/2446534496 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb6c80ffb30 msgr2=0x7fb6c8197b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:33.147 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:33.145+0000 7fb6c7fff640 1 --2- 172.21.15.49:0/2446534496 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb6c80ffb30 0x7fb6c8197b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:33.147 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:33.145+0000 7fb6c7fff640 1 -- 172.21.15.49:0/2446534496 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb6c8103560 msgr2=0x7fb6c819f140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:33.147 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:33.145+0000 7fb6c7fff640 1 --2- 172.21.15.49:0/2446534496 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb6c8103560 0x7fb6c819f140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:33.147 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:33.145+0000 7fb6c7fff640 1 -- 172.21.15.49:0/2446534496 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6c819f7b0 con 0x7fb6c8102ba0 2024-03-18T23:07:33.147 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:33.145+0000 7fb6c7fff640 1 --2- 172.21.15.49:0/2446534496 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb6c8102ba0 0x7fb6c81980d0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fb6b400e9f0 tx=0x7fb6b400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:07:33.148 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:33.145+0000 7fb6c5ffb640 1 -- 172.21.15.49:0/2446534496 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6b400cd80 con 0x7fb6c8102ba0 2024-03-18T23:07:33.148 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:33.145+0000 7fb6ceef1640 1 -- 172.21.15.49:0/2446534496 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6c819faa0 con 0x7fb6c8102ba0 2024-03-18T23:07:33.148 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:33.145+0000 7fb6c5ffb640 1 -- 172.21.15.49:0/2446534496 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb6b4005500 con 0x7fb6c8102ba0 2024-03-18T23:07:33.148 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:33.145+0000 7fb6c5ffb640 1 -- 172.21.15.49:0/2446534496 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6b4010690 con 0x7fb6c8102ba0 2024-03-18T23:07:33.148 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:33.145+0000 7fb6ceef1640 1 -- 172.21.15.49:0/2446534496 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6c819ffa0 con 0x7fb6c8102ba0 2024-03-18T23:07:33.149 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:33.146+0000 7fb6ceef1640 1 -- 172.21.15.49:0/2446534496 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6c8108cd0 con 0x7fb6c8102ba0 2024-03-18T23:07:33.149 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:33.147+0000 7fb6c5ffb640 1 -- 172.21.15.49:0/2446534496 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7fb6b40046c0 con 0x7fb6c8102ba0 2024-03-18T23:07:33.149 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:33.147+0000 7fb6c5ffb640 1 --2- 172.21.15.49:0/2446534496 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fb6b80a2800 0x7fb6b80a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:33.149 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:33.148+0000 7fb6ccc8d640 1 --2- 172.21.15.49:0/2446534496 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fb6b80a2800 0x7fb6b80a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:33.149 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:33.148+0000 7fb6c5ffb640 1 -- 172.21.15.49:0/2446534496 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7fb6b40cee30 con 0x7fb6c8102ba0 2024-03-18T23:07:33.149 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:33.148+0000 7fb6ccc8d640 1 --2- 172.21.15.49:0/2446534496 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fb6b80a2800 0x7fb6b80a4cc0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fb6bc009830 tx=0x7fb6bc034b00 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:07:33.152 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:33.152+0000 7fb6c5ffb640 1 -- 172.21.15.49:0/2446534496 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fb6b409bee0 con 0x7fb6c8102ba0 2024-03-18T23:07:33.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:07:33 smithi139 ceph-mon[67282]: pgmap v334: 97 pgs: 97 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 341 B/s wr, 11 op/s 2024-03-18T23:07:33.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:07:33 smithi139 ceph-mon[67282]: from='client.? 172.21.15.49:0/176251154' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-18T23:07:33.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:07:33 smithi139 ceph-mon[67282]: from='client.? 172.21.15.49:0/3242945707' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-18T23:07:33.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:07:33 smithi139 ceph-mon[67282]: from='client.? 172.21.15.49:0/4126874327' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-18T23:07:33.336 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:33.336+0000 7fb6ceef1640 1 -- 172.21.15.49:0/2446534496 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb6c8062420 con 0x7fb6c8102ba0 2024-03-18T23:07:33.337 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:33.338+0000 7fb6c5ffb640 1 -- 172.21.15.49:0/2446534496 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+22674 (secure 0 0 0) 0x7fb6b409b7e0 con 0x7fb6c8102ba0 2024-03-18T23:07:33.338 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-18T23:07:33.338 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":88,"fsid":"c63591ea-e57a-11ee-95c9-87774f69a715","created":"2024-03-18T22:57:58.734467+0000","modified":"2024-03-18T23:07:01.337701+0000","last_up_change":"2024-03-18T23:05:16.576310+0000","last_in_change":"2024-03-18T23:04:57.024067+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-18T23:01:28.363432+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-18T23:06:03.172963+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-18T23:06:03.973303+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e7fc9daa-4098-4ba5-8019-9aaef1aab3e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":1651162501},{"type":"v1","addr":"172.21.15.49:6803","nonce":1651162501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":1651162501},{"type":"v1","addr":"172.21.15.49:6805","nonce":1651162501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":1651162501},{"type":"v1","addr":"172.21.15.49:6809","nonce":1651162501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":1651162501},{"type":"v1","addr":"172.21.15.49:6807","nonce":1651162501}]},"public_addr":"172.21.15.49:6803/1651162501","cluster_addr":"172.21.15.49:6805/1651162501","heartbeat_back_addr":"172.21.15.49:6809/1651162501","heartbeat_front_addr":"172.21.15.49:6807/1651162501","state":["exists","up"]},{"osd":1,"uuid":"40ac929c-e90f-434c-a796-9aabd686f1fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":1766700359},{"type":"v1","addr":"172.21.15.139:6801","nonce":1766700359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":1766700359},{"type":"v1","addr":"172.21.15.139:6803","nonce":1766700359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":1766700359},{"type":"v1","addr":"172.21.15.139:6807","nonce":1766700359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":1766700359},{"type":"v1","addr":"172.21.15.139:6805","nonce":1766700359}]},"public_addr":"172.21.15.139:6801/1766700359","cluster_addr":"172.21.15.139:6803/1766700359","heartbeat_back_addr":"172.21.15.139:6807/1766700359","heartbeat_front_addr":"172.21.15.139:6805/1766700359","state":["exists","up"]},{"osd":2,"uuid":"8ae3b666-00b4-45ed-a5ee-4d0771edabb6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":3752671234},{"type":"v1","addr":"172.21.15.175:6801","nonce":3752671234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":3752671234},{"type":"v1","addr":"172.21.15.175:6803","nonce":3752671234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":3752671234},{"type":"v1","addr":"172.21.15.175:6807","nonce":3752671234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":3752671234},{"type":"v1","addr":"172.21.15.175:6805","nonce":3752671234}]},"public_addr":"172.21.15.175:6801/3752671234","cluster_addr":"172.21.15.175:6803/3752671234","heartbeat_back_addr":"172.21.15.175:6807/3752671234","heartbeat_front_addr":"172.21.15.175:6805/3752671234","state":["exists","up"]},{"osd":3,"uuid":"1a62e89d-45d4-4237-b813-7f96a59aca7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":956160536},{"type":"v1","addr":"172.21.15.49:6811","nonce":956160536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":956160536},{"type":"v1","addr":"172.21.15.49:6813","nonce":956160536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":956160536},{"type":"v1","addr":"172.21.15.49:6817","nonce":956160536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":956160536},{"type":"v1","addr":"172.21.15.49:6815","nonce":956160536}]},"public_addr":"172.21.15.49:6811/956160536","cluster_addr":"172.21.15.49:6813/956160536","heartbeat_back_addr":"172.21.15.49:6817/956160536","heartbeat_front_addr":"172.21.15.49:6815/956160536","state":["exists","up"]},{"osd":4,"uuid":"c715101c-0c3f-4b98-b142-d64a35e187df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":3129712668},{"type":"v1","addr":"172.21.15.139:6809","nonce":3129712668}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":3129712668},{"type":"v1","addr":"172.21.15.139:6811","nonce":3129712668}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":3129712668},{"type":"v1","addr":"172.21.15.139:6815","nonce":3129712668}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":3129712668},{"type":"v1","addr":"172.21.15.139:6813","nonce":3129712668}]},"public_addr":"172.21.15.139:6809/3129712668","cluster_addr":"172.21.15.139:6811/3129712668","heartbeat_back_addr":"172.21.15.139:6815/3129712668","heartbeat_front_addr":"172.21.15.139:6813/3129712668","state":["exists","up"]},{"osd":5,"uuid":"4b641f81-77eb-45fd-8464-511e7b3062b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":2774567215},{"type":"v1","addr":"172.21.15.175:6809","nonce":2774567215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":2774567215},{"type":"v1","addr":"172.21.15.175:6811","nonce":2774567215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":2774567215},{"type":"v1","addr":"172.21.15.175:6815","nonce":2774567215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":2774567215},{"type":"v1","addr":"172.21.15.175:6813","nonce":2774567215}]},"public_addr":"172.21.15.175:6809/2774567215","cluster_addr":"172.21.15.175:6811/2774567215","heartbeat_back_addr":"172.21.15.175:6815/2774567215","heartbeat_front_addr":"172.21.15.175:6813/2774567215","state":["exists","up"]},{"osd":6,"uuid":"653da8e0-1197-4b48-abed-470f2a9c4dce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":1472215461},{"type":"v1","addr":"172.21.15.49:6819","nonce":1472215461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":1472215461},{"type":"v1","addr":"172.21.15.49:6821","nonce":1472215461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":1472215461},{"type":"v1","addr":"172.21.15.49:6825","nonce":1472215461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":1472215461},{"type":"v1","addr":"172.21.15.49:6823","nonce":1472215461}]},"public_addr":"172.21.15.49:6819/1472215461","cluster_addr":"172.21.15.49:6821/1472215461","heartbeat_back_addr":"172.21.15.49:6825/1472215461","heartbeat_front_addr":"172.21.15.49:6823/1472215461","state":["exists","up"]},{"osd":7,"uuid":"ad157c9b-4b75-451c-8740-630f0975b7fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":2163986787},{"type":"v1","addr":"172.21.15.139:6817","nonce":2163986787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":2163986787},{"type":"v1","addr":"172.21.15.139:6819","nonce":2163986787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":2163986787},{"type":"v1","addr":"172.21.15.139:6823","nonce":2163986787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":2163986787},{"type":"v1","addr":"172.21.15.139:6821","nonce":2163986787}]},"public_addr":"172.21.15.139:6817/2163986787","cluster_addr":"172.21.15.139:6819/2163986787","heartbeat_back_addr":"172.21.15.139:6823/2163986787","heartbeat_front_addr":"172.21.15.139:6821/2163986787","state":["exists","up"]},{"osd":8,"uuid":"c8e75f15-df75-4d5d-9b92-a064bffb6c01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":1194026992},{"type":"v1","addr":"172.21.15.175:6817","nonce":1194026992}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1194026992},{"type":"v1","addr":"172.21.15.175:6819","nonce":1194026992}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":1194026992},{"type":"v1","addr":"172.21.15.175:6823","nonce":1194026992}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1194026992},{"type":"v1","addr":"172.21.15.175:6821","nonce":1194026992}]},"public_addr":"172.21.15.175:6817/1194026992","cluster_addr":"172.21.15.175:6819/1194026992","heartbeat_back_addr":"172.21.15.175:6823/1194026992","heartbeat_front_addr":"172.21.15.175:6821/1194026992","state":["exists","up"]},{"osd":9,"uuid":"edb5ffef-b6a2-4941-b5ab-40f982ad30d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":2742084884},{"type":"v1","addr":"172.21.15.49:6827","nonce":2742084884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":2742084884},{"type":"v1","addr":"172.21.15.49:6829","nonce":2742084884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":2742084884},{"type":"v1","addr":"172.21.15.49:6833","nonce":2742084884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":2742084884},{"type":"v1","addr":"172.21.15.49:6831","nonce":2742084884}]},"public_addr":"172.21.15.49:6827/2742084884","cluster_addr":"172.21.15.49:6829/2742084884","heartbeat_back_addr":"172.21.15.49:6833/2742084884","heartbeat_front_addr":"172.21.15.49:6831/2742084884","state":["exists","up"]},{"osd":10,"uuid":"3945e46d-766a-4c4a-8821-d43648dd1578","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":2404740201},{"type":"v1","addr":"172.21.15.139:6825","nonce":2404740201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":2404740201},{"type":"v1","addr":"172.21.15.139:6827","nonce":2404740201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":2404740201},{"type":"v1","addr":"172.21.15.139:6831","nonce":2404740201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":2404740201},{"type":"v1","addr":"172.21.15.139:6829","nonce":2404740201}]},"public_addr":"172.21.15.139:6825/2404740201","cluster_addr":"172.21.15.139:6827/2404740201","heartbeat_back_addr":"172.21.15.139:6831/2404740201","heartbeat_front_addr":"172.21.15.139:6829/2404740201","state":["exists","up"]},{"osd":11,"uuid":"9954ba15-d877-46cc-b24a-893f9168deb8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":3519503868},{"type":"v1","addr":"172.21.15.175:6825","nonce":3519503868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":3519503868},{"type":"v1","addr":"172.21.15.175:6827","nonce":3519503868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":3519503868},{"type":"v1","addr":"172.21.15.175:6831","nonce":3519503868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":3519503868},{"type":"v1","addr":"172.21.15.175:6829","nonce":3519503868}]},"public_addr":"172.21.15.175:6825/3519503868","cluster_addr":"172.21.15.175:6827/3519503868","heartbeat_back_addr":"172.21.15.175:6831/3519503868","heartbeat_front_addr":"172.21.15.175:6829/3519503868","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:00:38.367865+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:01:02.237687+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:01:25.609751+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:01:51.268572+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:02:16.514605+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:02:41.586523+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:03:07.029964+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:03:32.956250+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:03:57.547984+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:04:23.948972+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:04:49.667210+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:05:15.284693+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.6","mappings":[{"from":4,"to":6}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.49:0/563527416":"2024-03-19T22:58:59.806253+0000","172.21.15.49:0/1036383534":"2024-03-19T22:58:59.806253+0000","172.21.15.49:6801/2609458951":"2024-03-19T22:58:59.806253+0000","172.21.15.49:0/21283102":"2024-03-19T22:58:31.972474+0000","172.21.15.49:0/1292058922":"2024-03-19T22:58:31.972474+0000","172.21.15.49:6800/1985190312":"2024-03-19T22:58:31.972474+0000","172.21.15.49:0/1912446700":"2024-03-19T22:58:59.806253+0000","172.21.15.49:0/2176210039":"2024-03-19T22:58:31.972474+0000","172.21.15.49:6801/1985190312":"2024-03-19T22:58:31.972474+0000","172.21.15.49:6800/2609458951":"2024-03-19T22:58:59.806253+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-18T23:07:33.340 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:33.340+0000 7fb6ceef1640 1 -- 172.21.15.49:0/2446534496 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fb6b80a2800 msgr2=0x7fb6b80a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:33.340 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:33.340+0000 7fb6ceef1640 1 --2- 172.21.15.49:0/2446534496 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fb6b80a2800 0x7fb6b80a4cc0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fb6bc009830 tx=0x7fb6bc034b00 comp rx=0 tx=0).stop 2024-03-18T23:07:33.340 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:33.340+0000 7fb6ceef1640 1 -- 172.21.15.49:0/2446534496 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb6c8102ba0 msgr2=0x7fb6c81980d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:33.340 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:33.340+0000 7fb6ceef1640 1 --2- 172.21.15.49:0/2446534496 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb6c8102ba0 0x7fb6c81980d0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fb6b400e9f0 tx=0x7fb6b400edf0 comp rx=0 tx=0).stop 2024-03-18T23:07:33.340 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:33.340+0000 7fb6ceef1640 1 -- 172.21.15.49:0/2446534496 shutdown_connections 2024-03-18T23:07:33.340 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:33.340+0000 7fb6ceef1640 1 --2- 172.21.15.49:0/2446534496 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fb6b80a2800 0x7fb6b80a4cc0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:33.340 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:33.340+0000 7fb6ceef1640 1 --2- 172.21.15.49:0/2446534496 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb6c8103560 0x7fb6c819f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:33.340 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:33.340+0000 7fb6ceef1640 1 --2- 172.21.15.49:0/2446534496 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb6c8102ba0 0x7fb6c81980d0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:33.340 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:33.340+0000 7fb6ceef1640 1 --2- 172.21.15.49:0/2446534496 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb6c80ffb30 0x7fb6c8197b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:33.340 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:33.340+0000 7fb6ceef1640 1 -- 172.21.15.49:0/2446534496 >> 172.21.15.49:0/2446534496 conn(0x7fb6c80fb440 msgr2=0x7fb6c8105ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:07:33.340 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:33.340+0000 7fb6ceef1640 1 -- 172.21.15.49:0/2446534496 shutdown_connections 2024-03-18T23:07:33.340 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:33.340+0000 7fb6ceef1640 1 -- 172.21.15.49:0/2446534496 wait complete. 2024-03-18T23:07:33.350 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:07:33 smithi049 ceph-mon[62991]: pgmap v334: 97 pgs: 97 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 341 B/s wr, 11 op/s 2024-03-18T23:07:33.350 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:07:33 smithi049 ceph-mon[62991]: from='client.? 172.21.15.49:0/176251154' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-18T23:07:33.350 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:07:33 smithi049 ceph-mon[62991]: from='client.? 172.21.15.49:0/3242945707' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-18T23:07:33.350 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:07:33 smithi049 ceph-mon[62991]: from='client.? 172.21.15.49:0/4126874327' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-18T23:07:33.351 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-18T23:01:28.363432+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-18T23:06:03.172963+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-18T23:06:03.973303+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-18T23:07:33.352 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-18T23:07:33.462 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:07:33 smithi175 ceph-mon[67352]: pgmap v334: 97 pgs: 97 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 341 B/s wr, 11 op/s 2024-03-18T23:07:33.462 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:07:33 smithi175 ceph-mon[67352]: from='client.? 172.21.15.49:0/176251154' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-18T23:07:33.462 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:07:33 smithi175 ceph-mon[67352]: from='client.? 172.21.15.49:0/3242945707' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-18T23:07:33.462 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:07:33 smithi175 ceph-mon[67352]: from='client.? 172.21.15.49:0/4126874327' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-18T23:07:33.463 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:33.463+0000 7fcd7a322640 1 -- 172.21.15.49:0/676520205 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcd74101930 msgr2=0x7fcd74101db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:33.463 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:33.463+0000 7fcd7a322640 1 --2- 172.21.15.49:0/676520205 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcd74101930 0x7fcd74101db0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fcd68009900 tx=0x7fcd6802bc30 comp rx=0 tx=0).stop 2024-03-18T23:07:33.463 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:33.463+0000 7fcd7a322640 1 -- 172.21.15.49:0/676520205 shutdown_connections 2024-03-18T23:07:33.464 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:33.463+0000 7fcd7a322640 1 --2- 172.21.15.49:0/676520205 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcd741022f0 0x7fcd74108b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:33.464 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:33.463+0000 7fcd7a322640 1 --2- 172.21.15.49:0/676520205 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcd74101930 0x7fcd74101db0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:33.464 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:33.463+0000 7fcd7a322640 1 --2- 172.21.15.49:0/676520205 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcd74100740 0x7fcd74100b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:33.464 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:33.463+0000 7fcd7a322640 1 -- 172.21.15.49:0/676520205 >> 172.21.15.49:0/676520205 conn(0x7fcd740fbcd0 msgr2=0x7fcd740fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:07:33.464 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:33.463+0000 7fcd7a322640 1 -- 172.21.15.49:0/676520205 shutdown_connections 2024-03-18T23:07:33.464 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:33.463+0000 7fcd7a322640 1 -- 172.21.15.49:0/676520205 wait complete. 2024-03-18T23:07:33.464 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:33.464+0000 7fcd7a322640 1 Processor -- start 2024-03-18T23:07:33.464 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:33.464+0000 7fcd7a322640 1 -- start start 2024-03-18T23:07:33.466 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:33.464+0000 7fcd7a322640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcd74100740 0x7fcd7406a980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:33.466 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:33.464+0000 7fcd7a322640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcd74101930 0x7fcd7419c890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:33.466 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:33.464+0000 7fcd7a322640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcd741022f0 0x7fcd7419eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:33.467 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:33.464+0000 7fcd7a322640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fcd7410c3c0 con 0x7fcd74101930 2024-03-18T23:07:33.467 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:33.464+0000 7fcd7a322640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fcd7410c240 con 0x7fcd74100740 2024-03-18T23:07:33.467 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:33.464+0000 7fcd7a322640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fcd7410c540 con 0x7fcd741022f0 2024-03-18T23:07:33.467 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:33.464+0000 7fcd737fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcd74101930 0x7fcd7419c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:33.467 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:33.464+0000 7fcd737fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcd74101930 0x7fcd7419c890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:44212/0 (socket says 172.21.15.49:44212) 2024-03-18T23:07:33.467 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:33.464+0000 7fcd737fe640 1 -- 172.21.15.49:0/2060004379 learned_addr learned my addr 172.21.15.49:0/2060004379 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:07:33.467 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:33.464+0000 7fcd788bf640 1 --2- 172.21.15.49:0/2060004379 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcd741022f0 0x7fcd7419eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:33.467 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:33.464+0000 7fcd73fff640 1 --2- 172.21.15.49:0/2060004379 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcd74100740 0x7fcd7406a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:33.467 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:33.464+0000 7fcd737fe640 1 -- 172.21.15.49:0/2060004379 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcd741022f0 msgr2=0x7fcd7419eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:33.467 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:33.464+0000 7fcd737fe640 1 --2- 172.21.15.49:0/2060004379 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcd741022f0 0x7fcd7419eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:33.467 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:33.464+0000 7fcd737fe640 1 -- 172.21.15.49:0/2060004379 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcd74100740 msgr2=0x7fcd7406a980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:33.467 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:33.464+0000 7fcd737fe640 1 --2- 172.21.15.49:0/2060004379 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcd74100740 0x7fcd7406a980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:33.467 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:33.464+0000 7fcd737fe640 1 -- 172.21.15.49:0/2060004379 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd7419f790 con 0x7fcd74101930 2024-03-18T23:07:33.468 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:33.464+0000 7fcd73fff640 1 --2- 172.21.15.49:0/2060004379 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcd74100740 0x7fcd7406a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T23:07:33.468 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:33.464+0000 7fcd737fe640 1 --2- 172.21.15.49:0/2060004379 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcd74101930 0x7fcd7419c890 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fcd68009230 tx=0x7fcd68005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:07:33.468 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:33.465+0000 7fcd717fa640 1 -- 172.21.15.49:0/2060004379 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd68041070 con 0x7fcd74101930 2024-03-18T23:07:33.468 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:33.465+0000 7fcd717fa640 1 -- 172.21.15.49:0/2060004379 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcd680044d0 con 0x7fcd74101930 2024-03-18T23:07:33.468 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:33.465+0000 7fcd7a322640 1 -- 172.21.15.49:0/2060004379 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd7419fa20 con 0x7fcd74101930 2024-03-18T23:07:33.468 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:33.465+0000 7fcd717fa640 1 -- 172.21.15.49:0/2060004379 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd68034440 con 0x7fcd74101930 2024-03-18T23:07:33.468 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:33.465+0000 7fcd7a322640 1 -- 172.21.15.49:0/2060004379 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd7419ff60 con 0x7fcd74101930 2024-03-18T23:07:33.468 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:33.466+0000 7fcd7a322640 1 -- 172.21.15.49:0/2060004379 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd74107a60 con 0x7fcd74101930 2024-03-18T23:07:33.468 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:33.467+0000 7fcd717fa640 1 -- 172.21.15.49:0/2060004379 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7fcd68044070 con 0x7fcd74101930 2024-03-18T23:07:33.469 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:33.467+0000 7fcd717fa640 1 --2- 172.21.15.49:0/2060004379 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fcd5c0a2880 0x7fcd5c0a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:33.469 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:33.468+0000 7fcd73fff640 1 --2- 172.21.15.49:0/2060004379 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fcd5c0a2880 0x7fcd5c0a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:33.469 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:33.468+0000 7fcd717fa640 1 -- 172.21.15.49:0/2060004379 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7fcd680f0660 con 0x7fcd74101930 2024-03-18T23:07:33.469 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:33.468+0000 7fcd73fff640 1 --2- 172.21.15.49:0/2060004379 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fcd5c0a2880 0x7fcd5c0a4d40 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fcd74101790 tx=0x7fcd64005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:07:33.472 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:33.472+0000 7fcd717fa640 1 -- 172.21.15.49:0/2060004379 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fcd680bd710 con 0x7fcd74101930 2024-03-18T23:07:33.654 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:33.654+0000 7fcd7a322640 1 -- 172.21.15.49:0/2060004379 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fcd74062420 con 0x7fcd74101930 2024-03-18T23:07:33.654 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:33.654+0000 7fcd717fa640 1 -- 172.21.15.49:0/2060004379 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) ==== 93+0+10 (secure 0 0 0) 0x7fcd680bd010 con 0x7fcd74101930 2024-03-18T23:07:33.655 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 1 2024-03-18T23:07:33.656 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:33.656+0000 7fcd7a322640 1 -- 172.21.15.49:0/2060004379 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fcd5c0a2880 msgr2=0x7fcd5c0a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:33.657 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:33.656+0000 7fcd7a322640 1 --2- 172.21.15.49:0/2060004379 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fcd5c0a2880 0x7fcd5c0a4d40 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fcd74101790 tx=0x7fcd64005f90 comp rx=0 tx=0).stop 2024-03-18T23:07:33.657 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:33.656+0000 7fcd7a322640 1 -- 172.21.15.49:0/2060004379 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcd74101930 msgr2=0x7fcd7419c890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:33.657 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:33.656+0000 7fcd7a322640 1 --2- 172.21.15.49:0/2060004379 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcd74101930 0x7fcd7419c890 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fcd68009230 tx=0x7fcd68005d50 comp rx=0 tx=0).stop 2024-03-18T23:07:33.657 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:33.656+0000 7fcd7a322640 1 -- 172.21.15.49:0/2060004379 shutdown_connections 2024-03-18T23:07:33.657 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:33.656+0000 7fcd7a322640 1 --2- 172.21.15.49:0/2060004379 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fcd5c0a2880 0x7fcd5c0a4d40 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:33.657 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:33.656+0000 7fcd7a322640 1 --2- 172.21.15.49:0/2060004379 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcd741022f0 0x7fcd7419eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:33.657 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:33.656+0000 7fcd7a322640 1 --2- 172.21.15.49:0/2060004379 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcd74101930 0x7fcd7419c890 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:33.657 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:33.656+0000 7fcd7a322640 1 --2- 172.21.15.49:0/2060004379 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcd74100740 0x7fcd7406a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:33.657 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:33.656+0000 7fcd7a322640 1 -- 172.21.15.49:0/2060004379 >> 172.21.15.49:0/2060004379 conn(0x7fcd740fbcd0 msgr2=0x7fcd74104c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:07:33.657 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:33.656+0000 7fcd7a322640 1 -- 172.21.15.49:0/2060004379 shutdown_connections 2024-03-18T23:07:33.657 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:33.656+0000 7fcd7a322640 1 -- 172.21.15.49:0/2060004379 wait complete. 2024-03-18T23:07:33.668 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-18T23:07:33.782 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:33.779+0000 7f4bb7d5b640 1 -- 172.21.15.49:0/473759744 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4bb0101930 msgr2=0x7f4bb0101db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:33.782 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:33.779+0000 7f4bb7d5b640 1 --2- 172.21.15.49:0/473759744 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4bb0101930 0x7f4bb0101db0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f4ba4009860 tx=0x7f4ba402bb70 comp rx=0 tx=0).stop 2024-03-18T23:07:33.782 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:33.779+0000 7f4bb7d5b640 1 -- 172.21.15.49:0/473759744 shutdown_connections 2024-03-18T23:07:33.782 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:33.779+0000 7f4bb7d5b640 1 --2- 172.21.15.49:0/473759744 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4bb01022f0 0x7f4bb0108b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:33.783 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:33.779+0000 7f4bb7d5b640 1 --2- 172.21.15.49:0/473759744 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4bb0101930 0x7f4bb0101db0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:33.783 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:33.779+0000 7f4bb7d5b640 1 --2- 172.21.15.49:0/473759744 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4bb0100740 0x7f4bb0100b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:33.783 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:33.779+0000 7f4bb7d5b640 1 -- 172.21.15.49:0/473759744 >> 172.21.15.49:0/473759744 conn(0x7f4bb00fbcd0 msgr2=0x7f4bb00fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:07:33.783 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:33.779+0000 7f4bb7d5b640 1 -- 172.21.15.49:0/473759744 shutdown_connections 2024-03-18T23:07:33.783 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:33.779+0000 7f4bb7d5b640 1 -- 172.21.15.49:0/473759744 wait complete. 2024-03-18T23:07:33.784 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:33.780+0000 7f4bb7d5b640 1 Processor -- start 2024-03-18T23:07:33.784 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:33.780+0000 7f4bb7d5b640 1 -- start start 2024-03-18T23:07:33.784 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:33.780+0000 7f4bb7d5b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4bb0100740 0x7f4bb010b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:33.784 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:33.780+0000 7f4bb7d5b640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4bb0101930 0x7f4bb0109700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:33.784 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:33.780+0000 7f4bb7d5b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4bb01022f0 0x7f4bb0109c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:33.784 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:33.780+0000 7f4bb7d5b640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f4bb010d4e0 con 0x7f4bb0100740 2024-03-18T23:07:33.784 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:33.780+0000 7f4bb7d5b640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f4bb010d360 con 0x7f4bb01022f0 2024-03-18T23:07:33.784 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:33.780+0000 7f4bb7d5b640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f4bb010d660 con 0x7f4bb0101930 2024-03-18T23:07:33.784 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:33.780+0000 7f4bb5af7640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4bb0100740 0x7f4bb010b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:33.785 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:33.780+0000 7f4bb52f6640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4bb0101930 0x7f4bb0109700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:33.785 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:33.780+0000 7f4bb5af7640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4bb0100740 0x7f4bb010b0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:44226/0 (socket says 172.21.15.49:44226) 2024-03-18T23:07:33.785 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:33.780+0000 7f4bb5af7640 1 -- 172.21.15.49:0/3118367973 learned_addr learned my addr 172.21.15.49:0/3118367973 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:07:33.785 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:33.780+0000 7f4bb62f8640 1 --2- 172.21.15.49:0/3118367973 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4bb01022f0 0x7f4bb0109c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:33.785 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:33.780+0000 7f4bb52f6640 1 -- 172.21.15.49:0/3118367973 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4bb01022f0 msgr2=0x7f4bb0109c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:33.785 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:33.780+0000 7f4bb52f6640 1 --2- 172.21.15.49:0/3118367973 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4bb01022f0 0x7f4bb0109c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:33.785 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:33.780+0000 7f4bb52f6640 1 -- 172.21.15.49:0/3118367973 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4bb0100740 msgr2=0x7f4bb010b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:33.785 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:33.780+0000 7f4bb52f6640 1 --2- 172.21.15.49:0/3118367973 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4bb0100740 0x7f4bb010b0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:33.786 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:33.780+0000 7f4bb52f6640 1 -- 172.21.15.49:0/3118367973 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bb010a4d0 con 0x7f4bb0101930 2024-03-18T23:07:33.786 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:33.780+0000 7f4bb5af7640 1 --2- 172.21.15.49:0/3118367973 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4bb0100740 0x7f4bb010b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:07:33.786 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:33.780+0000 7f4bb52f6640 1 --2- 172.21.15.49:0/3118367973 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4bb0101930 0x7f4bb0109700 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f4ba402c0b0 tx=0x7f4ba4032430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:07:33.786 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:33.781+0000 7f4baeffd640 1 -- 172.21.15.49:0/3118367973 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ba4041020 con 0x7f4bb0101930 2024-03-18T23:07:33.786 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:33.781+0000 7f4bb7d5b640 1 -- 172.21.15.49:0/3118367973 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4bb01a1c10 con 0x7f4bb0101930 2024-03-18T23:07:33.786 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:33.781+0000 7f4baeffd640 1 -- 172.21.15.49:0/3118367973 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ba402c9f0 con 0x7f4bb0101930 2024-03-18T23:07:33.786 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:33.781+0000 7f4bb7d5b640 1 -- 172.21.15.49:0/3118367973 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4bb01a21b0 con 0x7f4bb0101930 2024-03-18T23:07:33.786 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:33.781+0000 7f4baeffd640 1 -- 172.21.15.49:0/3118367973 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ba4039040 con 0x7f4bb0101930 2024-03-18T23:07:33.786 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:33.782+0000 7f4bb7d5b640 1 -- 172.21.15.49:0/3118367973 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4bb0107a60 con 0x7f4bb0101930 2024-03-18T23:07:33.787 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:33.783+0000 7f4baeffd640 1 -- 172.21.15.49:0/3118367973 <== mon.1 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7f4ba4044020 con 0x7f4bb0101930 2024-03-18T23:07:33.787 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:33.783+0000 7f4baeffd640 1 --2- 172.21.15.49:0/3118367973 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f4b980a2800 0x7f4b980a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:33.787 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:33.784+0000 7f4bb5af7640 1 --2- 172.21.15.49:0/3118367973 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f4b980a2800 0x7f4b980a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:33.787 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:33.784+0000 7f4baeffd640 1 -- 172.21.15.49:0/3118367973 <== mon.1 v2:172.21.15.175:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f4ba40f06e0 con 0x7f4bb0101930 2024-03-18T23:07:33.787 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:33.784+0000 7f4bb5af7640 1 --2- 172.21.15.49:0/3118367973 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f4b980a2800 0x7f4b980a4cc0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f4ba80058c0 tx=0x7f4ba8005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:07:33.788 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:33.788+0000 7f4baeffd640 1 -- 172.21.15.49:0/3118367973 <== mon.1 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f4ba404b020 con 0x7f4bb0101930 2024-03-18T23:07:33.971 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:33.971+0000 7f4bb7d5b640 1 -- 172.21.15.49:0/3118367973 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f4bb002d1f0 con 0x7f4bb0101930 2024-03-18T23:07:33.971 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:33.971+0000 7f4baeffd640 1 -- 172.21.15.49:0/3118367973 <== mon.1 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) ==== 104+0+11 (secure 0 0 0) 0x7f4ba40bd730 con 0x7f4bb0101930 2024-03-18T23:07:33.972 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 32 2024-03-18T23:07:33.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:33.973+0000 7f4bb7d5b640 1 -- 172.21.15.49:0/3118367973 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f4b980a2800 msgr2=0x7f4b980a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:33.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:33.973+0000 7f4bb7d5b640 1 --2- 172.21.15.49:0/3118367973 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f4b980a2800 0x7f4b980a4cc0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f4ba80058c0 tx=0x7f4ba8005f90 comp rx=0 tx=0).stop 2024-03-18T23:07:33.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:33.973+0000 7f4bb7d5b640 1 -- 172.21.15.49:0/3118367973 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4bb0101930 msgr2=0x7f4bb0109700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:33.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:33.973+0000 7f4bb7d5b640 1 --2- 172.21.15.49:0/3118367973 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4bb0101930 0x7f4bb0109700 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f4ba402c0b0 tx=0x7f4ba4032430 comp rx=0 tx=0).stop 2024-03-18T23:07:33.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:33.973+0000 7f4bb7d5b640 1 -- 172.21.15.49:0/3118367973 shutdown_connections 2024-03-18T23:07:33.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:33.973+0000 7f4bb7d5b640 1 --2- 172.21.15.49:0/3118367973 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f4b980a2800 0x7f4b980a4cc0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:33.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:33.973+0000 7f4bb7d5b640 1 --2- 172.21.15.49:0/3118367973 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4bb01022f0 0x7f4bb0109c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:33.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:33.973+0000 7f4bb7d5b640 1 --2- 172.21.15.49:0/3118367973 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4bb0101930 0x7f4bb0109700 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:33.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:33.973+0000 7f4bb7d5b640 1 --2- 172.21.15.49:0/3118367973 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4bb0100740 0x7f4bb010b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:33.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:33.973+0000 7f4bb7d5b640 1 -- 172.21.15.49:0/3118367973 >> 172.21.15.49:0/3118367973 conn(0x7f4bb00fbcd0 msgr2=0x7f4bb0104c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:07:33.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:33.973+0000 7f4bb7d5b640 1 -- 172.21.15.49:0/3118367973 shutdown_connections 2024-03-18T23:07:33.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:33.974+0000 7f4bb7d5b640 1 -- 172.21.15.49:0/3118367973 wait complete. 2024-03-18T23:07:33.985 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-18T23:07:34.099 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:34.097+0000 7f3300548640 1 -- 172.21.15.49:0/3664434798 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f32f8104c50 msgr2=0x7f32f8107040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:34.100 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:34.097+0000 7f3300548640 1 --2- 172.21.15.49:0/3664434798 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f32f8104c50 0x7f32f8107040 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f32f4009900 tx=0x7f32f402bc30 comp rx=0 tx=0).stop 2024-03-18T23:07:34.100 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:34.097+0000 7f3300548640 1 -- 172.21.15.49:0/3664434798 shutdown_connections 2024-03-18T23:07:34.100 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:34.097+0000 7f3300548640 1 --2- 172.21.15.49:0/3664434798 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f32f8107580 0x7f32f8109970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:34.100 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:34.097+0000 7f3300548640 1 --2- 172.21.15.49:0/3664434798 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f32f8104c50 0x7f32f8107040 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:34.100 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:34.097+0000 7f3300548640 1 --2- 172.21.15.49:0/3664434798 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f32f8102320 0x7f32f8104710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:34.100 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:34.097+0000 7f3300548640 1 -- 172.21.15.49:0/3664434798 >> 172.21.15.49:0/3664434798 conn(0x7f32f80fbcd0 msgr2=0x7f32f80fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:07:34.100 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:34.097+0000 7f3300548640 1 -- 172.21.15.49:0/3664434798 shutdown_connections 2024-03-18T23:07:34.100 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:34.097+0000 7f3300548640 1 -- 172.21.15.49:0/3664434798 wait complete. 2024-03-18T23:07:34.100 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:34.097+0000 7f3300548640 1 Processor -- start 2024-03-18T23:07:34.100 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:34.097+0000 7f3300548640 1 -- start start 2024-03-18T23:07:34.100 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:34.098+0000 7f3300548640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f32f8102320 0x7f32f8197ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:34.100 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:34.098+0000 7f3300548640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f32f8104c50 0x7f32f81980e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:34.100 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:34.098+0000 7f3300548640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f32f8107580 0x7f32f819f150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:34.101 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:34.098+0000 7f3300548640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f32f810d500 con 0x7f32f8107580 2024-03-18T23:07:34.101 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:34.098+0000 7f3300548640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f32f810d380 con 0x7f32f8102320 2024-03-18T23:07:34.101 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:34.098+0000 7f3300548640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f32f810d680 con 0x7f32f8104c50 2024-03-18T23:07:34.101 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:34.098+0000 7f32feae5640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f32f8107580 0x7f32f819f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:34.101 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:34.098+0000 7f32feae5640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f32f8107580 0x7f32f819f150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:44248/0 (socket says 172.21.15.49:44248) 2024-03-18T23:07:34.101 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:34.098+0000 7f32feae5640 1 -- 172.21.15.49:0/1459876548 learned_addr learned my addr 172.21.15.49:0/1459876548 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:07:34.101 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:34.098+0000 7f32fdae3640 1 --2- 172.21.15.49:0/1459876548 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f32f8104c50 0x7f32f81980e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:34.101 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:34.098+0000 7f32fe2e4640 1 --2- 172.21.15.49:0/1459876548 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f32f8102320 0x7f32f8197ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:34.101 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:34.098+0000 7f32feae5640 1 -- 172.21.15.49:0/1459876548 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f32f8104c50 msgr2=0x7f32f81980e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:34.101 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:34.098+0000 7f32feae5640 1 --2- 172.21.15.49:0/1459876548 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f32f8104c50 0x7f32f81980e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:34.101 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:34.098+0000 7f32feae5640 1 -- 172.21.15.49:0/1459876548 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f32f8102320 msgr2=0x7f32f8197ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:34.101 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:34.098+0000 7f32feae5640 1 --2- 172.21.15.49:0/1459876548 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f32f8102320 0x7f32f8197ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:34.102 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:34.098+0000 7f32feae5640 1 -- 172.21.15.49:0/1459876548 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32f819f7c0 con 0x7f32f8107580 2024-03-18T23:07:34.102 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:34.098+0000 7f32fdae3640 1 --2- 172.21.15.49:0/1459876548 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f32f8104c50 0x7f32f81980e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:07:34.102 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:34.098+0000 7f32feae5640 1 --2- 172.21.15.49:0/1459876548 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f32f8107580 0x7f32f819f150 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f32f0009fd0 tx=0x7f32f000b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:07:34.102 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:34.098+0000 7f32eb7fe640 1 -- 172.21.15.49:0/1459876548 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32f0017070 con 0x7f32f8107580 2024-03-18T23:07:34.102 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:34.098+0000 7f32eb7fe640 1 -- 172.21.15.49:0/1459876548 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f32f0003e50 con 0x7f32f8107580 2024-03-18T23:07:34.102 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:34.098+0000 7f3300548640 1 -- 172.21.15.49:0/1459876548 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32f819fab0 con 0x7f32f8107580 2024-03-18T23:07:34.102 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:34.098+0000 7f32eb7fe640 1 -- 172.21.15.49:0/1459876548 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32f00106c0 con 0x7f32f8107580 2024-03-18T23:07:34.102 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:34.099+0000 7f3300548640 1 -- 172.21.15.49:0/1459876548 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32f819ffd0 con 0x7f32f8107580 2024-03-18T23:07:34.102 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:34.099+0000 7f3300548640 1 -- 172.21.15.49:0/1459876548 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32f80fd7c0 con 0x7f32f8107580 2024-03-18T23:07:34.103 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:34.101+0000 7f32eb7fe640 1 -- 172.21.15.49:0/1459876548 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7f32f0003970 con 0x7f32f8107580 2024-03-18T23:07:34.103 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:34.101+0000 7f32eb7fe640 1 --2- 172.21.15.49:0/1459876548 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f32e40a2880 0x7f32e40a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:34.103 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:34.101+0000 7f32fe2e4640 1 --2- 172.21.15.49:0/1459876548 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f32e40a2880 0x7f32e40a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:34.103 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:34.101+0000 7f32eb7fe640 1 -- 172.21.15.49:0/1459876548 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f32f00ce6a0 con 0x7f32f8107580 2024-03-18T23:07:34.103 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:34.102+0000 7f32fe2e4640 1 --2- 172.21.15.49:0/1459876548 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f32e40a2880 0x7f32e40a4d40 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f32ec009bd0 tx=0x7f32ec005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:07:34.104 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:34.105+0000 7f32eb7fe640 1 -- 172.21.15.49:0/1459876548 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f32f009b180 con 0x7f32f8107580 2024-03-18T23:07:34.290 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:34.290+0000 7f3300548640 1 -- 172.21.15.49:0/1459876548 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f32f802d1f0 con 0x7f32f8107580 2024-03-18T23:07:34.291 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:34.291+0000 7f32eb7fe640 1 -- 172.21.15.49:0/1459876548 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) ==== 100+0+11 (secure 0 0 0) 0x7f32f009f220 con 0x7f32f8107580 2024-03-18T23:07:34.291 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 64 2024-03-18T23:07:34.293 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:34.293+0000 7f3300548640 1 -- 172.21.15.49:0/1459876548 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f32e40a2880 msgr2=0x7f32e40a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:34.293 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:34.293+0000 7f3300548640 1 --2- 172.21.15.49:0/1459876548 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f32e40a2880 0x7f32e40a4d40 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f32ec009bd0 tx=0x7f32ec005f90 comp rx=0 tx=0).stop 2024-03-18T23:07:34.293 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:34.293+0000 7f3300548640 1 -- 172.21.15.49:0/1459876548 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f32f8107580 msgr2=0x7f32f819f150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:34.293 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:34.293+0000 7f3300548640 1 --2- 172.21.15.49:0/1459876548 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f32f8107580 0x7f32f819f150 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f32f0009fd0 tx=0x7f32f000b260 comp rx=0 tx=0).stop 2024-03-18T23:07:34.293 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:34.293+0000 7f3300548640 1 -- 172.21.15.49:0/1459876548 shutdown_connections 2024-03-18T23:07:34.293 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:34.293+0000 7f3300548640 1 --2- 172.21.15.49:0/1459876548 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f32e40a2880 0x7f32e40a4d40 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:34.293 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:34.293+0000 7f3300548640 1 --2- 172.21.15.49:0/1459876548 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f32f8107580 0x7f32f819f150 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:34.293 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:34.293+0000 7f3300548640 1 --2- 172.21.15.49:0/1459876548 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f32f8104c50 0x7f32f81980e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:34.293 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:34.293+0000 7f3300548640 1 --2- 172.21.15.49:0/1459876548 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f32f8102320 0x7f32f8197ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:34.293 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:34.293+0000 7f3300548640 1 -- 172.21.15.49:0/1459876548 >> 172.21.15.49:0/1459876548 conn(0x7f32f80fbcd0 msgr2=0x7f32f8102be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:07:34.293 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:34.293+0000 7f3300548640 1 -- 172.21.15.49:0/1459876548 shutdown_connections 2024-03-18T23:07:34.294 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:34.293+0000 7f3300548640 1 -- 172.21.15.49:0/1459876548 wait complete. 2024-03-18T23:07:34.303 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:07:34 smithi049 ceph-mon[62991]: from='client.? 172.21.15.49:0/324824060' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T23:07:34.303 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:07:34 smithi049 ceph-mon[62991]: from='client.? 172.21.15.49:0/2446534496' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T23:07:34.303 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:07:34 smithi049 ceph-mon[62991]: from='client.? 172.21.15.49:0/2060004379' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-18T23:07:34.303 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:07:34 smithi049 ceph-mon[62991]: from='client.? 172.21.15.49:0/3118367973' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-18T23:07:34.304 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-18T23:07:34.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:07:34 smithi139 ceph-mon[67282]: from='client.? 172.21.15.49:0/324824060' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T23:07:34.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:07:34 smithi139 ceph-mon[67282]: from='client.? 172.21.15.49:0/2446534496' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T23:07:34.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:07:34 smithi139 ceph-mon[67282]: from='client.? 172.21.15.49:0/2060004379' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-18T23:07:34.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:07:34 smithi139 ceph-mon[67282]: from='client.? 172.21.15.49:0/3118367973' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-18T23:07:34.413 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:34.413+0000 7ffbf3fff640 1 -- 172.21.15.49:0/1607374145 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffbf4101760 msgr2=0x7ffbf4101be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:34.413 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:34.413+0000 7ffbf3fff640 1 --2- 172.21.15.49:0/1607374145 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffbf4101760 0x7ffbf4101be0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7ffbdc009900 tx=0x7ffbdc02bc30 comp rx=0 tx=0).stop 2024-03-18T23:07:34.413 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:34.413+0000 7ffbf3fff640 1 -- 172.21.15.49:0/1607374145 shutdown_connections 2024-03-18T23:07:34.414 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:34.413+0000 7ffbf3fff640 1 --2- 172.21.15.49:0/1607374145 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffbf4102120 0x7ffbf41089b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:34.414 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:34.413+0000 7ffbf3fff640 1 --2- 172.21.15.49:0/1607374145 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffbf4101760 0x7ffbf4101be0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:34.414 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:34.413+0000 7ffbf3fff640 1 --2- 172.21.15.49:0/1607374145 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffbf4100570 0x7ffbf4100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:34.414 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:34.413+0000 7ffbf3fff640 1 -- 172.21.15.49:0/1607374145 >> 172.21.15.49:0/1607374145 conn(0x7ffbf40fbb00 msgr2=0x7ffbf40fdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:07:34.414 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:34.413+0000 7ffbf3fff640 1 -- 172.21.15.49:0/1607374145 shutdown_connections 2024-03-18T23:07:34.414 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:34.413+0000 7ffbf3fff640 1 -- 172.21.15.49:0/1607374145 wait complete. 2024-03-18T23:07:34.414 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:34.413+0000 7ffbf3fff640 1 Processor -- start 2024-03-18T23:07:34.414 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:34.414+0000 7ffbf3fff640 1 -- start start 2024-03-18T23:07:34.416 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:34.414+0000 7ffbf3fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffbf4100570 0x7ffbf4104500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:34.416 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:34.414+0000 7ffbf3fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffbf4101760 0x7ffbf4102b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:34.416 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:34.414+0000 7ffbf3fff640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffbf4102120 0x7ffbf4103080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:34.416 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:34.414+0000 7ffbf3fff640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7ffbf410c200 con 0x7ffbf4100570 2024-03-18T23:07:34.416 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:34.414+0000 7ffbf3fff640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7ffbf410c080 con 0x7ffbf4101760 2024-03-18T23:07:34.416 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:34.414+0000 7ffbf3fff640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ffbf410c380 con 0x7ffbf4102120 2024-03-18T23:07:34.416 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:34.414+0000 7ffbf2ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffbf4100570 0x7ffbf4104500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:34.417 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:34.414+0000 7ffbf2ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffbf4100570 0x7ffbf4104500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:44266/0 (socket says 172.21.15.49:44266) 2024-03-18T23:07:34.417 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:34.414+0000 7ffbf2ffd640 1 -- 172.21.15.49:0/2549190071 learned_addr learned my addr 172.21.15.49:0/2549190071 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:07:34.417 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:34.414+0000 7ffbf37fe640 1 --2- 172.21.15.49:0/2549190071 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffbf4102120 0x7ffbf4103080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:34.417 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:34.414+0000 7ffbea7fc640 1 --2- 172.21.15.49:0/2549190071 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffbf4101760 0x7ffbf4102b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:34.417 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:34.414+0000 7ffbf2ffd640 1 -- 172.21.15.49:0/2549190071 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffbf4102120 msgr2=0x7ffbf4103080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:34.417 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:34.414+0000 7ffbf2ffd640 1 --2- 172.21.15.49:0/2549190071 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffbf4102120 0x7ffbf4103080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:34.417 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:34.414+0000 7ffbf2ffd640 1 -- 172.21.15.49:0/2549190071 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffbf4101760 msgr2=0x7ffbf4102b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:34.417 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:34.414+0000 7ffbf2ffd640 1 --2- 172.21.15.49:0/2549190071 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffbf4101760 0x7ffbf4102b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:34.417 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:34.414+0000 7ffbf2ffd640 1 -- 172.21.15.49:0/2549190071 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbf4103910 con 0x7ffbf4100570 2024-03-18T23:07:34.417 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:34.414+0000 7ffbf2ffd640 1 --2- 172.21.15.49:0/2549190071 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffbf4100570 0x7ffbf4104500 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7ffbe400e9c0 tx=0x7ffbe400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:07:34.417 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:34.415+0000 7ffbf0ff9640 1 -- 172.21.15.49:0/2549190071 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbe400cd80 con 0x7ffbf4100570 2024-03-18T23:07:34.417 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:34.415+0000 7ffbf3fff640 1 -- 172.21.15.49:0/2549190071 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffbf41a1a40 con 0x7ffbf4100570 2024-03-18T23:07:34.417 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:34.415+0000 7ffbf0ff9640 1 -- 172.21.15.49:0/2549190071 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffbe40035f0 con 0x7ffbf4100570 2024-03-18T23:07:34.417 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:34.415+0000 7ffbf0ff9640 1 -- 172.21.15.49:0/2549190071 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbe4010690 con 0x7ffbf4100570 2024-03-18T23:07:34.417 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:34.415+0000 7ffbf3fff640 1 -- 172.21.15.49:0/2549190071 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffbf41a1f60 con 0x7ffbf4100570 2024-03-18T23:07:34.418 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:34.416+0000 7ffbf3fff640 1 -- 172.21.15.49:0/2549190071 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffbf4068980 con 0x7ffbf4100570 2024-03-18T23:07:34.418 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:34.417+0000 7ffbf0ff9640 1 -- 172.21.15.49:0/2549190071 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7ffbe40037a0 con 0x7ffbf4100570 2024-03-18T23:07:34.418 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:34.418+0000 7ffbf0ff9640 1 --2- 172.21.15.49:0/2549190071 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7ffbe00a2800 0x7ffbe00a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:34.419 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:34.418+0000 7ffbea7fc640 1 --2- 172.21.15.49:0/2549190071 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7ffbe00a2800 0x7ffbe00a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:34.419 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:34.418+0000 7ffbf0ff9640 1 -- 172.21.15.49:0/2549190071 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7ffbe40cee60 con 0x7ffbf4100570 2024-03-18T23:07:34.419 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:34.418+0000 7ffbea7fc640 1 --2- 172.21.15.49:0/2549190071 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7ffbe00a2800 0x7ffbe00a4cc0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7ffbdc02c7b0 tx=0x7ffbdc005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:07:34.422 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:34.422+0000 7ffbf0ff9640 1 -- 172.21.15.49:0/2549190071 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7ffbe409bf10 con 0x7ffbf4100570 2024-03-18T23:07:34.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:07:34 smithi175 ceph-mon[67352]: from='client.? 172.21.15.49:0/324824060' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T23:07:34.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:07:34 smithi175 ceph-mon[67352]: from='client.? 172.21.15.49:0/2446534496' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T23:07:34.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:07:34 smithi175 ceph-mon[67352]: from='client.? 172.21.15.49:0/2060004379' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-18T23:07:34.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:07:34 smithi175 ceph-mon[67352]: from='client.? 172.21.15.49:0/3118367973' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-18T23:07:34.646 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:34.646+0000 7ffbf3fff640 1 -- 172.21.15.49:0/2549190071 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ffbf41a28c0 con 0x7ffbf4100570 2024-03-18T23:07:34.646 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:34.646+0000 7ffbf0ff9640 1 -- 172.21.15.49:0/2549190071 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11936 (secure 0 0 0) 0x7ffbe409b810 con 0x7ffbf4100570 2024-03-18T23:07:34.647 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-18T23:07:34.647 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6835/3631301630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":3631301630},{"type":"v1","addr":"172.21.15.49:6835","nonce":3631301630}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6837/444305233","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":444305233},{"type":"v1","addr":"172.21.15.49:6837","nonce":444305233}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-18T23:06:04.575999+0000","modified":"2024-03-18T23:06:11.998736+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24488,"mds_1":24463,"mds_2":24430,"mds_3":24419,"mds_4":24391},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14622":{"gid":14622,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.49:6839/1934085093","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6838","nonce":1934085093},{"type":"v1","addr":"172.21.15.49:6839","nonce":1934085093}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14634":{"gid":14634,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.49:6841/2869298862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6840","nonce":2869298862},{"type":"v1","addr":"172.21.15.49:6841","nonce":2869298862}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24389":{"gid":24389,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6833/3766178603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3766178603},{"type":"v1","addr":"172.21.15.139:6833","nonce":3766178603}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24391":{"gid":24391,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.175:6833/1702997173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":1702997173},{"type":"v1","addr":"172.21.15.175:6833","nonce":1702997173}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":9,"addr":"172.21.15.139:6835/1663212937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":1663212937},{"type":"v1","addr":"172.21.15.139:6835","nonce":1663212937}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.175:6835/755474931","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":755474931},{"type":"v1","addr":"172.21.15.175:6835","nonce":755474931}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6837/379482756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":379482756},{"type":"v1","addr":"172.21.15.175:6837","nonce":379482756}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6837/2330317540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6836","nonce":2330317540},{"type":"v1","addr":"172.21.15.139:6837","nonce":2330317540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"172.21.15.175:6839/3640882084","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":3640882084},{"type":"v1","addr":"172.21.15.175:6839","nonce":3640882084}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.139:6839/4280128209","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6838","nonce":4280128209},{"type":"v1","addr":"172.21.15.139:6839","nonce":4280128209}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24463,24488,24430,24419,24391]},"id":1}]} 2024-03-18T23:07:34.647 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 27 2024-03-18T23:07:34.649 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:34.648+0000 7ffbf3fff640 1 -- 172.21.15.49:0/2549190071 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7ffbe00a2800 msgr2=0x7ffbe00a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:34.649 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:34.648+0000 7ffbf3fff640 1 --2- 172.21.15.49:0/2549190071 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7ffbe00a2800 0x7ffbe00a4cc0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7ffbdc02c7b0 tx=0x7ffbdc005fd0 comp rx=0 tx=0).stop 2024-03-18T23:07:34.649 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:34.648+0000 7ffbf3fff640 1 -- 172.21.15.49:0/2549190071 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffbf4100570 msgr2=0x7ffbf4104500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:34.649 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:34.648+0000 7ffbf3fff640 1 --2- 172.21.15.49:0/2549190071 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffbf4100570 0x7ffbf4104500 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7ffbe400e9c0 tx=0x7ffbe400ee80 comp rx=0 tx=0).stop 2024-03-18T23:07:34.649 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:34.648+0000 7ffbf3fff640 1 -- 172.21.15.49:0/2549190071 shutdown_connections 2024-03-18T23:07:34.649 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:34.648+0000 7ffbf3fff640 1 --2- 172.21.15.49:0/2549190071 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7ffbe00a2800 0x7ffbe00a4cc0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:34.649 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:34.648+0000 7ffbf3fff640 1 --2- 172.21.15.49:0/2549190071 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffbf4102120 0x7ffbf4103080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:34.649 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:34.648+0000 7ffbf3fff640 1 --2- 172.21.15.49:0/2549190071 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffbf4101760 0x7ffbf4102b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:34.649 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:34.648+0000 7ffbf3fff640 1 --2- 172.21.15.49:0/2549190071 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffbf4100570 0x7ffbf4104500 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:34.649 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:34.649+0000 7ffbf3fff640 1 -- 172.21.15.49:0/2549190071 >> 172.21.15.49:0/2549190071 conn(0x7ffbf40fbb00 msgr2=0x7ffbf40fd5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:07:34.649 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:34.649+0000 7ffbf3fff640 1 -- 172.21.15.49:0/2549190071 shutdown_connections 2024-03-18T23:07:34.649 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:34.649+0000 7ffbf3fff640 1 -- 172.21.15.49:0/2549190071 wait complete. 2024-03-18T23:07:34.660 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-18T23:07:34.770 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:34.768+0000 7f9a46790640 1 -- 172.21.15.49:0/1382154063 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9a40104c70 msgr2=0x7f9a40107060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:34.770 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:34.768+0000 7f9a46790640 1 --2- 172.21.15.49:0/1382154063 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9a40104c70 0x7f9a40107060 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f9a30009900 tx=0x7f9a3002bc30 comp rx=0 tx=0).stop 2024-03-18T23:07:34.770 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:34.768+0000 7f9a46790640 1 -- 172.21.15.49:0/1382154063 shutdown_connections 2024-03-18T23:07:34.770 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:34.768+0000 7f9a46790640 1 --2- 172.21.15.49:0/1382154063 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9a401075a0 0x7f9a401099f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:34.770 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:34.768+0000 7f9a46790640 1 --2- 172.21.15.49:0/1382154063 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9a40104c70 0x7f9a40107060 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:34.770 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:34.768+0000 7f9a46790640 1 --2- 172.21.15.49:0/1382154063 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9a40102340 0x7f9a40104730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:34.771 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:34.768+0000 7f9a46790640 1 -- 172.21.15.49:0/1382154063 >> 172.21.15.49:0/1382154063 conn(0x7f9a400fbd50 msgr2=0x7f9a400fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:07:34.771 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:34.768+0000 7f9a46790640 1 -- 172.21.15.49:0/1382154063 shutdown_connections 2024-03-18T23:07:34.771 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:34.768+0000 7f9a46790640 1 -- 172.21.15.49:0/1382154063 wait complete. 2024-03-18T23:07:34.771 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:34.769+0000 7f9a46790640 1 Processor -- start 2024-03-18T23:07:34.771 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:34.769+0000 7f9a46790640 1 -- start start 2024-03-18T23:07:34.771 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:34.769+0000 7f9a46790640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9a40102340 0x7f9a40197b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:34.771 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:34.769+0000 7f9a46790640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9a40104c70 0x7f9a401980d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:34.771 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:34.769+0000 7f9a46790640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9a401075a0 0x7f9a4019f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:34.771 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:34.769+0000 7f9a46790640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f9a4010d4e0 con 0x7f9a401075a0 2024-03-18T23:07:34.771 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:34.769+0000 7f9a46790640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f9a4010d360 con 0x7f9a40104c70 2024-03-18T23:07:34.771 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:34.769+0000 7f9a46790640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f9a4010d660 con 0x7f9a40102340 2024-03-18T23:07:34.771 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:34.769+0000 7f9a44d2d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9a401075a0 0x7f9a4019f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:34.771 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:34.769+0000 7f9a3f7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9a40104c70 0x7f9a401980d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:34.772 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:34.769+0000 7f9a3ffff640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9a40102340 0x7f9a40197b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:34.772 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:34.769+0000 7f9a44d2d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9a401075a0 0x7f9a4019f140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53838/0 (socket says 172.21.15.49:53838) 2024-03-18T23:07:34.772 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:34.769+0000 7f9a44d2d640 1 -- 172.21.15.49:0/1684981628 learned_addr learned my addr 172.21.15.49:0/1684981628 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:07:34.772 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:34.769+0000 7f9a44d2d640 1 -- 172.21.15.49:0/1684981628 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9a40102340 msgr2=0x7f9a40197b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:34.772 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:34.769+0000 7f9a44d2d640 1 --2- 172.21.15.49:0/1684981628 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9a40102340 0x7f9a40197b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:34.772 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:34.769+0000 7f9a44d2d640 1 -- 172.21.15.49:0/1684981628 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9a40104c70 msgr2=0x7f9a401980d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:34.772 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:34.769+0000 7f9a44d2d640 1 --2- 172.21.15.49:0/1684981628 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9a40104c70 0x7f9a401980d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:34.772 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:34.769+0000 7f9a44d2d640 1 -- 172.21.15.49:0/1684981628 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a4019f7b0 con 0x7f9a401075a0 2024-03-18T23:07:34.772 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:34.770+0000 7f9a44d2d640 1 --2- 172.21.15.49:0/1684981628 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9a401075a0 0x7f9a4019f140 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f9a3800eb10 tx=0x7f9a3800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:07:34.772 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:34.770+0000 7f9a3d7fa640 1 -- 172.21.15.49:0/1684981628 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a38004b20 con 0x7f9a401075a0 2024-03-18T23:07:34.772 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:34.770+0000 7f9a3d7fa640 1 -- 172.21.15.49:0/1684981628 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9a38004cc0 con 0x7f9a401075a0 2024-03-18T23:07:34.772 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:34.770+0000 7f9a46790640 1 -- 172.21.15.49:0/1684981628 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a4019faa0 con 0x7f9a401075a0 2024-03-18T23:07:34.772 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:34.770+0000 7f9a3d7fa640 1 -- 172.21.15.49:0/1684981628 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a38010760 con 0x7f9a401075a0 2024-03-18T23:07:34.773 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:34.770+0000 7f9a46790640 1 -- 172.21.15.49:0/1684981628 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a4019ff70 con 0x7f9a401075a0 2024-03-18T23:07:34.773 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:34.771+0000 7f9a46790640 1 -- 172.21.15.49:0/1684981628 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a400fd840 con 0x7f9a401075a0 2024-03-18T23:07:34.773 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:34.772+0000 7f9a3d7fa640 1 -- 172.21.15.49:0/1684981628 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7f9a38005020 con 0x7f9a401075a0 2024-03-18T23:07:34.774 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:34.773+0000 7f9a3d7fa640 1 --2- 172.21.15.49:0/1684981628 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f9a280a2800 0x7f9a280a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:34.774 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:34.773+0000 7f9a3ffff640 1 --2- 172.21.15.49:0/1684981628 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f9a280a2800 0x7f9a280a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:34.774 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:34.773+0000 7f9a3d7fa640 1 -- 172.21.15.49:0/1684981628 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f9a380cefc0 con 0x7f9a401075a0 2024-03-18T23:07:34.774 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:34.773+0000 7f9a3ffff640 1 --2- 172.21.15.49:0/1684981628 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f9a280a2800 0x7f9a280a4cc0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f9a34009bd0 tx=0x7f9a34005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:07:34.777 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:34.777+0000 7f9a3d7fa640 1 -- 172.21.15.49:0/1684981628 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f9a3809c070 con 0x7f9a401075a0 2024-03-18T23:07:34.961 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:34.961+0000 7f9a46790640 1 -- 172.21.15.49:0/1684981628 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9a40062420 con 0x7f9a401075a0 2024-03-18T23:07:34.962 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:34.962+0000 7f9a3d7fa640 1 -- 172.21.15.49:0/1684981628 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+22674 (secure 0 0 0) 0x7f9a3809b970 con 0x7f9a401075a0 2024-03-18T23:07:34.963 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-18T23:07:34.963 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":88,"fsid":"c63591ea-e57a-11ee-95c9-87774f69a715","created":"2024-03-18T22:57:58.734467+0000","modified":"2024-03-18T23:07:01.337701+0000","last_up_change":"2024-03-18T23:05:16.576310+0000","last_in_change":"2024-03-18T23:04:57.024067+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-18T23:01:28.363432+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-18T23:06:03.172963+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-18T23:06:03.973303+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e7fc9daa-4098-4ba5-8019-9aaef1aab3e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":1651162501},{"type":"v1","addr":"172.21.15.49:6803","nonce":1651162501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":1651162501},{"type":"v1","addr":"172.21.15.49:6805","nonce":1651162501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":1651162501},{"type":"v1","addr":"172.21.15.49:6809","nonce":1651162501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":1651162501},{"type":"v1","addr":"172.21.15.49:6807","nonce":1651162501}]},"public_addr":"172.21.15.49:6803/1651162501","cluster_addr":"172.21.15.49:6805/1651162501","heartbeat_back_addr":"172.21.15.49:6809/1651162501","heartbeat_front_addr":"172.21.15.49:6807/1651162501","state":["exists","up"]},{"osd":1,"uuid":"40ac929c-e90f-434c-a796-9aabd686f1fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":1766700359},{"type":"v1","addr":"172.21.15.139:6801","nonce":1766700359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":1766700359},{"type":"v1","addr":"172.21.15.139:6803","nonce":1766700359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":1766700359},{"type":"v1","addr":"172.21.15.139:6807","nonce":1766700359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":1766700359},{"type":"v1","addr":"172.21.15.139:6805","nonce":1766700359}]},"public_addr":"172.21.15.139:6801/1766700359","cluster_addr":"172.21.15.139:6803/1766700359","heartbeat_back_addr":"172.21.15.139:6807/1766700359","heartbeat_front_addr":"172.21.15.139:6805/1766700359","state":["exists","up"]},{"osd":2,"uuid":"8ae3b666-00b4-45ed-a5ee-4d0771edabb6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":3752671234},{"type":"v1","addr":"172.21.15.175:6801","nonce":3752671234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":3752671234},{"type":"v1","addr":"172.21.15.175:6803","nonce":3752671234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":3752671234},{"type":"v1","addr":"172.21.15.175:6807","nonce":3752671234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":3752671234},{"type":"v1","addr":"172.21.15.175:6805","nonce":3752671234}]},"public_addr":"172.21.15.175:6801/3752671234","cluster_addr":"172.21.15.175:6803/3752671234","heartbeat_back_addr":"172.21.15.175:6807/3752671234","heartbeat_front_addr":"172.21.15.175:6805/3752671234","state":["exists","up"]},{"osd":3,"uuid":"1a62e89d-45d4-4237-b813-7f96a59aca7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":956160536},{"type":"v1","addr":"172.21.15.49:6811","nonce":956160536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":956160536},{"type":"v1","addr":"172.21.15.49:6813","nonce":956160536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":956160536},{"type":"v1","addr":"172.21.15.49:6817","nonce":956160536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":956160536},{"type":"v1","addr":"172.21.15.49:6815","nonce":956160536}]},"public_addr":"172.21.15.49:6811/956160536","cluster_addr":"172.21.15.49:6813/956160536","heartbeat_back_addr":"172.21.15.49:6817/956160536","heartbeat_front_addr":"172.21.15.49:6815/956160536","state":["exists","up"]},{"osd":4,"uuid":"c715101c-0c3f-4b98-b142-d64a35e187df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":3129712668},{"type":"v1","addr":"172.21.15.139:6809","nonce":3129712668}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":3129712668},{"type":"v1","addr":"172.21.15.139:6811","nonce":3129712668}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":3129712668},{"type":"v1","addr":"172.21.15.139:6815","nonce":3129712668}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":3129712668},{"type":"v1","addr":"172.21.15.139:6813","nonce":3129712668}]},"public_addr":"172.21.15.139:6809/3129712668","cluster_addr":"172.21.15.139:6811/3129712668","heartbeat_back_addr":"172.21.15.139:6815/3129712668","heartbeat_front_addr":"172.21.15.139:6813/3129712668","state":["exists","up"]},{"osd":5,"uuid":"4b641f81-77eb-45fd-8464-511e7b3062b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":2774567215},{"type":"v1","addr":"172.21.15.175:6809","nonce":2774567215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":2774567215},{"type":"v1","addr":"172.21.15.175:6811","nonce":2774567215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":2774567215},{"type":"v1","addr":"172.21.15.175:6815","nonce":2774567215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":2774567215},{"type":"v1","addr":"172.21.15.175:6813","nonce":2774567215}]},"public_addr":"172.21.15.175:6809/2774567215","cluster_addr":"172.21.15.175:6811/2774567215","heartbeat_back_addr":"172.21.15.175:6815/2774567215","heartbeat_front_addr":"172.21.15.175:6813/2774567215","state":["exists","up"]},{"osd":6,"uuid":"653da8e0-1197-4b48-abed-470f2a9c4dce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":1472215461},{"type":"v1","addr":"172.21.15.49:6819","nonce":1472215461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":1472215461},{"type":"v1","addr":"172.21.15.49:6821","nonce":1472215461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":1472215461},{"type":"v1","addr":"172.21.15.49:6825","nonce":1472215461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":1472215461},{"type":"v1","addr":"172.21.15.49:6823","nonce":1472215461}]},"public_addr":"172.21.15.49:6819/1472215461","cluster_addr":"172.21.15.49:6821/1472215461","heartbeat_back_addr":"172.21.15.49:6825/1472215461","heartbeat_front_addr":"172.21.15.49:6823/1472215461","state":["exists","up"]},{"osd":7,"uuid":"ad157c9b-4b75-451c-8740-630f0975b7fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":2163986787},{"type":"v1","addr":"172.21.15.139:6817","nonce":2163986787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":2163986787},{"type":"v1","addr":"172.21.15.139:6819","nonce":2163986787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":2163986787},{"type":"v1","addr":"172.21.15.139:6823","nonce":2163986787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":2163986787},{"type":"v1","addr":"172.21.15.139:6821","nonce":2163986787}]},"public_addr":"172.21.15.139:6817/2163986787","cluster_addr":"172.21.15.139:6819/2163986787","heartbeat_back_addr":"172.21.15.139:6823/2163986787","heartbeat_front_addr":"172.21.15.139:6821/2163986787","state":["exists","up"]},{"osd":8,"uuid":"c8e75f15-df75-4d5d-9b92-a064bffb6c01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":1194026992},{"type":"v1","addr":"172.21.15.175:6817","nonce":1194026992}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1194026992},{"type":"v1","addr":"172.21.15.175:6819","nonce":1194026992}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":1194026992},{"type":"v1","addr":"172.21.15.175:6823","nonce":1194026992}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1194026992},{"type":"v1","addr":"172.21.15.175:6821","nonce":1194026992}]},"public_addr":"172.21.15.175:6817/1194026992","cluster_addr":"172.21.15.175:6819/1194026992","heartbeat_back_addr":"172.21.15.175:6823/1194026992","heartbeat_front_addr":"172.21.15.175:6821/1194026992","state":["exists","up"]},{"osd":9,"uuid":"edb5ffef-b6a2-4941-b5ab-40f982ad30d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":2742084884},{"type":"v1","addr":"172.21.15.49:6827","nonce":2742084884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":2742084884},{"type":"v1","addr":"172.21.15.49:6829","nonce":2742084884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":2742084884},{"type":"v1","addr":"172.21.15.49:6833","nonce":2742084884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":2742084884},{"type":"v1","addr":"172.21.15.49:6831","nonce":2742084884}]},"public_addr":"172.21.15.49:6827/2742084884","cluster_addr":"172.21.15.49:6829/2742084884","heartbeat_back_addr":"172.21.15.49:6833/2742084884","heartbeat_front_addr":"172.21.15.49:6831/2742084884","state":["exists","up"]},{"osd":10,"uuid":"3945e46d-766a-4c4a-8821-d43648dd1578","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":2404740201},{"type":"v1","addr":"172.21.15.139:6825","nonce":2404740201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":2404740201},{"type":"v1","addr":"172.21.15.139:6827","nonce":2404740201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":2404740201},{"type":"v1","addr":"172.21.15.139:6831","nonce":2404740201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":2404740201},{"type":"v1","addr":"172.21.15.139:6829","nonce":2404740201}]},"public_addr":"172.21.15.139:6825/2404740201","cluster_addr":"172.21.15.139:6827/2404740201","heartbeat_back_addr":"172.21.15.139:6831/2404740201","heartbeat_front_addr":"172.21.15.139:6829/2404740201","state":["exists","up"]},{"osd":11,"uuid":"9954ba15-d877-46cc-b24a-893f9168deb8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":3519503868},{"type":"v1","addr":"172.21.15.175:6825","nonce":3519503868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":3519503868},{"type":"v1","addr":"172.21.15.175:6827","nonce":3519503868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":3519503868},{"type":"v1","addr":"172.21.15.175:6831","nonce":3519503868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":3519503868},{"type":"v1","addr":"172.21.15.175:6829","nonce":3519503868}]},"public_addr":"172.21.15.175:6825/3519503868","cluster_addr":"172.21.15.175:6827/3519503868","heartbeat_back_addr":"172.21.15.175:6831/3519503868","heartbeat_front_addr":"172.21.15.175:6829/3519503868","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:00:38.367865+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:01:02.237687+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:01:25.609751+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:01:51.268572+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:02:16.514605+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:02:41.586523+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:03:07.029964+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:03:32.956250+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:03:57.547984+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:04:23.948972+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:04:49.667210+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:05:15.284693+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.6","mappings":[{"from":4,"to":6}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.49:0/563527416":"2024-03-19T22:58:59.806253+0000","172.21.15.49:0/1036383534":"2024-03-19T22:58:59.806253+0000","172.21.15.49:6801/2609458951":"2024-03-19T22:58:59.806253+0000","172.21.15.49:0/21283102":"2024-03-19T22:58:31.972474+0000","172.21.15.49:0/1292058922":"2024-03-19T22:58:31.972474+0000","172.21.15.49:6800/1985190312":"2024-03-19T22:58:31.972474+0000","172.21.15.49:0/1912446700":"2024-03-19T22:58:59.806253+0000","172.21.15.49:0/2176210039":"2024-03-19T22:58:31.972474+0000","172.21.15.49:6801/1985190312":"2024-03-19T22:58:31.972474+0000","172.21.15.49:6800/2609458951":"2024-03-19T22:58:59.806253+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-18T23:07:34.964 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:34.964+0000 7f9a46790640 1 -- 172.21.15.49:0/1684981628 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f9a280a2800 msgr2=0x7f9a280a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:34.964 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:34.964+0000 7f9a46790640 1 --2- 172.21.15.49:0/1684981628 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f9a280a2800 0x7f9a280a4cc0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f9a34009bd0 tx=0x7f9a34005f90 comp rx=0 tx=0).stop 2024-03-18T23:07:34.965 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:34.964+0000 7f9a46790640 1 -- 172.21.15.49:0/1684981628 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9a401075a0 msgr2=0x7f9a4019f140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:34.965 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:34.964+0000 7f9a46790640 1 --2- 172.21.15.49:0/1684981628 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9a401075a0 0x7f9a4019f140 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f9a3800eb10 tx=0x7f9a3800c2d0 comp rx=0 tx=0).stop 2024-03-18T23:07:34.965 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:34.964+0000 7f9a46790640 1 -- 172.21.15.49:0/1684981628 shutdown_connections 2024-03-18T23:07:34.965 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:34.964+0000 7f9a46790640 1 --2- 172.21.15.49:0/1684981628 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f9a280a2800 0x7f9a280a4cc0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:34.965 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:34.964+0000 7f9a46790640 1 --2- 172.21.15.49:0/1684981628 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9a401075a0 0x7f9a4019f140 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:34.965 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:34.964+0000 7f9a46790640 1 --2- 172.21.15.49:0/1684981628 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9a40104c70 0x7f9a401980d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:34.965 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:34.964+0000 7f9a46790640 1 --2- 172.21.15.49:0/1684981628 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9a40102340 0x7f9a40197b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:34.965 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:34.964+0000 7f9a46790640 1 -- 172.21.15.49:0/1684981628 >> 172.21.15.49:0/1684981628 conn(0x7f9a400fbd50 msgr2=0x7f9a40102d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:07:34.965 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:34.965+0000 7f9a46790640 1 -- 172.21.15.49:0/1684981628 shutdown_connections 2024-03-18T23:07:34.965 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:34.965+0000 7f9a46790640 1 -- 172.21.15.49:0/1684981628 wait complete. 2024-03-18T23:07:34.976 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-18T23:07:35.087 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.085+0000 7f47261cd640 1 -- 172.21.15.49:0/712570804 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4720102a20 msgr2=0x7f4720102ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:35.087 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.085+0000 7f47261cd640 1 --2- 172.21.15.49:0/712570804 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4720102a20 0x7f4720102ea0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f4714009900 tx=0x7f471402bc30 comp rx=0 tx=0).stop 2024-03-18T23:07:35.087 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.085+0000 7f47261cd640 1 -- 172.21.15.49:0/712570804 shutdown_connections 2024-03-18T23:07:35.087 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.085+0000 7f47261cd640 1 --2- 172.21.15.49:0/712570804 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f47201033e0 0x7f4720109c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:35.087 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.085+0000 7f47261cd640 1 --2- 172.21.15.49:0/712570804 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4720102a20 0x7f4720102ea0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:35.087 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.085+0000 7f47261cd640 1 --2- 172.21.15.49:0/712570804 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4720101830 0x7f4720101c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:35.087 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.085+0000 7f47261cd640 1 -- 172.21.15.49:0/712570804 >> 172.21.15.49:0/712570804 conn(0x7f47200fce00 msgr2=0x7f47200ff220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:07:35.088 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.085+0000 7f47261cd640 1 -- 172.21.15.49:0/712570804 shutdown_connections 2024-03-18T23:07:35.088 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.085+0000 7f47261cd640 1 -- 172.21.15.49:0/712570804 wait complete. 2024-03-18T23:07:35.088 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.085+0000 7f47261cd640 1 Processor -- start 2024-03-18T23:07:35.088 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.086+0000 7f47261cd640 1 -- start start 2024-03-18T23:07:35.088 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.086+0000 7f47261cd640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4720101830 0x7f4720197b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:35.088 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.086+0000 7f47261cd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4720102a20 0x7f4720198040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:35.088 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.086+0000 7f47261cd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f47201033e0 0x7f472019f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:35.088 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.086+0000 7f47261cd640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f472010d4b0 con 0x7f4720102a20 2024-03-18T23:07:35.088 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.086+0000 7f47261cd640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f472010d330 con 0x7f47201033e0 2024-03-18T23:07:35.088 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.086+0000 7f47261cd640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f472010d630 con 0x7f4720101830 2024-03-18T23:07:35.088 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.086+0000 7f471effd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4720102a20 0x7f4720198040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:35.088 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.086+0000 7f471effd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4720102a20 0x7f4720198040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53858/0 (socket says 172.21.15.49:53858) 2024-03-18T23:07:35.088 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.086+0000 7f471effd640 1 -- 172.21.15.49:0/927676930 learned_addr learned my addr 172.21.15.49:0/927676930 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:07:35.089 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.086+0000 7f471f7fe640 1 --2- 172.21.15.49:0/927676930 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4720101830 0x7f4720197b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:35.089 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.086+0000 7f471ffff640 1 --2- 172.21.15.49:0/927676930 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f47201033e0 0x7f472019f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:35.089 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.086+0000 7f471effd640 1 -- 172.21.15.49:0/927676930 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4720101830 msgr2=0x7f4720197b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:35.089 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.086+0000 7f471effd640 1 --2- 172.21.15.49:0/927676930 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4720101830 0x7f4720197b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:35.089 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.086+0000 7f471effd640 1 -- 172.21.15.49:0/927676930 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f47201033e0 msgr2=0x7f472019f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:35.089 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.086+0000 7f471effd640 1 --2- 172.21.15.49:0/927676930 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f47201033e0 0x7f472019f0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:35.089 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.086+0000 7f471effd640 1 -- 172.21.15.49:0/927676930 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f472019f720 con 0x7f4720102a20 2024-03-18T23:07:35.089 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.086+0000 7f471effd640 1 --2- 172.21.15.49:0/927676930 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4720102a20 0x7f4720198040 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f4714009230 tx=0x7f4714005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:07:35.089 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.087+0000 7f471cff9640 1 -- 172.21.15.49:0/927676930 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4714041070 con 0x7f4720102a20 2024-03-18T23:07:35.089 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.087+0000 7f471cff9640 1 -- 172.21.15.49:0/927676930 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f47140044d0 con 0x7f4720102a20 2024-03-18T23:07:35.089 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.087+0000 7f47261cd640 1 -- 172.21.15.49:0/927676930 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f472019f9b0 con 0x7f4720102a20 2024-03-18T23:07:35.089 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.087+0000 7f471cff9640 1 -- 172.21.15.49:0/927676930 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4714034440 con 0x7f4720102a20 2024-03-18T23:07:35.089 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.087+0000 7f47261cd640 1 -- 172.21.15.49:0/927676930 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f472019fea0 con 0x7f4720102a20 2024-03-18T23:07:35.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.088+0000 7f47261cd640 1 -- 172.21.15.49:0/927676930 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4720108b50 con 0x7f4720102a20 2024-03-18T23:07:35.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.089+0000 7f471cff9640 1 -- 172.21.15.49:0/927676930 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7f4714044070 con 0x7f4720102a20 2024-03-18T23:07:35.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.089+0000 7f471cff9640 1 --2- 172.21.15.49:0/927676930 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f47080a2800 0x7f47080a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:35.091 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.089+0000 7f471f7fe640 1 --2- 172.21.15.49:0/927676930 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f47080a2800 0x7f47080a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:35.091 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.089+0000 7f471cff9640 1 -- 172.21.15.49:0/927676930 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f47140f06b0 con 0x7f4720102a20 2024-03-18T23:07:35.091 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.090+0000 7f471f7fe640 1 --2- 172.21.15.49:0/927676930 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f47080a2800 0x7f47080a4cc0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f4710005fd0 tx=0x7f471000b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:07:35.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.094+0000 7f471cff9640 1 -- 172.21.15.49:0/927676930 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f47140bd760 con 0x7f4720102a20 2024-03-18T23:07:35.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:07:35 smithi139 ceph-mon[67282]: pgmap v335: 97 pgs: 97 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 341 B/s wr, 9 op/s 2024-03-18T23:07:35.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:07:35 smithi139 ceph-mon[67282]: from='client.? 172.21.15.49:0/1459876548' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-18T23:07:35.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:07:35 smithi139 ceph-mon[67282]: from='client.? 172.21.15.49:0/2549190071' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T23:07:35.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:07:35 smithi139 ceph-mon[67282]: from='client.? 172.21.15.49:0/1684981628' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T23:07:35.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.316+0000 7f47261cd640 1 -- 172.21.15.49:0/927676930 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f472002d1f0 con 0x7f4720102a20 2024-03-18T23:07:35.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.317+0000 7f471cff9640 1 -- 172.21.15.49:0/927676930 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11936 (secure 0 0 0) 0x7f47140bd060 con 0x7f4720102a20 2024-03-18T23:07:35.318 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-18T23:07:35.318 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6835/3631301630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":3631301630},{"type":"v1","addr":"172.21.15.49:6835","nonce":3631301630}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6837/444305233","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":444305233},{"type":"v1","addr":"172.21.15.49:6837","nonce":444305233}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-18T23:06:04.575999+0000","modified":"2024-03-18T23:06:11.998736+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24488,"mds_1":24463,"mds_2":24430,"mds_3":24419,"mds_4":24391},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14622":{"gid":14622,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.49:6839/1934085093","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6838","nonce":1934085093},{"type":"v1","addr":"172.21.15.49:6839","nonce":1934085093}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14634":{"gid":14634,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.49:6841/2869298862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6840","nonce":2869298862},{"type":"v1","addr":"172.21.15.49:6841","nonce":2869298862}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24389":{"gid":24389,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6833/3766178603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3766178603},{"type":"v1","addr":"172.21.15.139:6833","nonce":3766178603}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24391":{"gid":24391,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.175:6833/1702997173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":1702997173},{"type":"v1","addr":"172.21.15.175:6833","nonce":1702997173}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":9,"addr":"172.21.15.139:6835/1663212937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":1663212937},{"type":"v1","addr":"172.21.15.139:6835","nonce":1663212937}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.175:6835/755474931","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":755474931},{"type":"v1","addr":"172.21.15.175:6835","nonce":755474931}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6837/379482756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":379482756},{"type":"v1","addr":"172.21.15.175:6837","nonce":379482756}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6837/2330317540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6836","nonce":2330317540},{"type":"v1","addr":"172.21.15.139:6837","nonce":2330317540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"172.21.15.175:6839/3640882084","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":3640882084},{"type":"v1","addr":"172.21.15.175:6839","nonce":3640882084}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.139:6839/4280128209","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6838","nonce":4280128209},{"type":"v1","addr":"172.21.15.139:6839","nonce":4280128209}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24463,24488,24430,24419,24391]},"id":1}]} 2024-03-18T23:07:35.318 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 27 2024-03-18T23:07:35.320 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.319+0000 7f47261cd640 1 -- 172.21.15.49:0/927676930 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f47080a2800 msgr2=0x7f47080a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:35.320 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.319+0000 7f47261cd640 1 --2- 172.21.15.49:0/927676930 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f47080a2800 0x7f47080a4cc0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f4710005fd0 tx=0x7f471000b410 comp rx=0 tx=0).stop 2024-03-18T23:07:35.320 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.319+0000 7f47261cd640 1 -- 172.21.15.49:0/927676930 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4720102a20 msgr2=0x7f4720198040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:35.320 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.319+0000 7f47261cd640 1 --2- 172.21.15.49:0/927676930 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4720102a20 0x7f4720198040 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f4714009230 tx=0x7f4714005d50 comp rx=0 tx=0).stop 2024-03-18T23:07:35.320 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.319+0000 7f47261cd640 1 -- 172.21.15.49:0/927676930 shutdown_connections 2024-03-18T23:07:35.320 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.319+0000 7f47261cd640 1 --2- 172.21.15.49:0/927676930 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f47080a2800 0x7f47080a4cc0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:35.320 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.319+0000 7f47261cd640 1 --2- 172.21.15.49:0/927676930 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f47201033e0 0x7f472019f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:35.320 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.319+0000 7f47261cd640 1 --2- 172.21.15.49:0/927676930 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4720102a20 0x7f4720198040 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:35.320 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.319+0000 7f47261cd640 1 --2- 172.21.15.49:0/927676930 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4720101830 0x7f4720197b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:35.320 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.319+0000 7f47261cd640 1 -- 172.21.15.49:0/927676930 >> 172.21.15.49:0/927676930 conn(0x7f47200fce00 msgr2=0x7f4720105d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:07:35.320 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.319+0000 7f47261cd640 1 -- 172.21.15.49:0/927676930 shutdown_connections 2024-03-18T23:07:35.320 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.319+0000 7f47261cd640 1 -- 172.21.15.49:0/927676930 wait complete. 2024-03-18T23:07:35.329 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:07:35 smithi049 ceph-mon[62991]: pgmap v335: 97 pgs: 97 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 341 B/s wr, 9 op/s 2024-03-18T23:07:35.330 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:07:35 smithi049 ceph-mon[62991]: from='client.? 172.21.15.49:0/1459876548' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-18T23:07:35.330 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:07:35 smithi049 ceph-mon[62991]: from='client.? 172.21.15.49:0/2549190071' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T23:07:35.330 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:07:35 smithi049 ceph-mon[62991]: from='client.? 172.21.15.49:0/1684981628' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T23:07:35.331 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 27, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-03-18T23:06:04.575999+0000', 'modified': '2024-03-18T23:06:11.998736+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24488, 'mds_1': 24463, 'mds_2': 24430, 'mds_3': 24419, 'mds_4': 24391}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14622': {'gid': 14622, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.49:6839/1934085093', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.49:6838', 'nonce': 1934085093}, {'type': 'v1', 'addr': '172.21.15.49:6839', 'nonce': 1934085093}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14634': {'gid': 14634, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.49:6841/2869298862', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.49:6840', 'nonce': 2869298862}, {'type': 'v1', 'addr': '172.21.15.49:6841', 'nonce': 2869298862}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24389': {'gid': 24389, 'name': 'k', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.139:6833/3766178603', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.139:6832', 'nonce': 3766178603}, {'type': 'v1', 'addr': '172.21.15.139:6833', 'nonce': 3766178603}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24391': {'gid': 24391, 'name': 'l', 'rank': 4, 'incarnation': 25, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.175:6833/1702997173', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6832', 'nonce': 1702997173}, {'type': 'v1', 'addr': '172.21.15.175:6833', 'nonce': 1702997173}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24419': {'gid': 24419, 'name': 'e', 'rank': 3, 'incarnation': 23, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.139:6835/1663212937', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.139:6834', 'nonce': 1663212937}, {'type': 'v1', 'addr': '172.21.15.139:6835', 'nonce': 1663212937}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24430': {'gid': 24430, 'name': 'f', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.175:6835/755474931', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6834', 'nonce': 755474931}, {'type': 'v1', 'addr': '172.21.15.175:6835', 'nonce': 755474931}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24439': {'gid': 24439, 'name': 'c', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.175:6837/379482756', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6836', 'nonce': 379482756}, {'type': 'v1', 'addr': '172.21.15.175:6837', 'nonce': 379482756}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24452': {'gid': 24452, 'name': 'h', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.139:6837/2330317540', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.139:6836', 'nonce': 2330317540}, {'type': 'v1', 'addr': '172.21.15.139:6837', 'nonce': 2330317540}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24463': {'gid': 24463, 'name': 'i', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.175:6839/3640882084', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6838', 'nonce': 3640882084}, {'type': 'v1', 'addr': '172.21.15.175:6839', 'nonce': 3640882084}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24488': {'gid': 24488, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.139:6839/4280128209', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.139:6838', 'nonce': 4280128209}, {'type': 'v1', 'addr': '172.21.15.139:6839', 'nonce': 4280128209}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24488, 'qdb_cluster': [24463, 24488, 24430, 24419, 24391]} 2024-03-18T23:07:35.331 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-03-18T23:07:35.331 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.l status 2024-03-18T23:07:35.442 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.439+0000 7f27cf444640 1 -- 172.21.15.49:0/3909810826 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f27c81022d0 msgr2=0x7f27c8108b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:35.442 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.439+0000 7f27cf444640 1 --2- 172.21.15.49:0/3909810826 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f27c81022d0 0x7f27c8108b60 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f27c4009ca0 tx=0x7f27c402c0f0 comp rx=0 tx=0).stop 2024-03-18T23:07:35.442 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.440+0000 7f27cf444640 1 -- 172.21.15.49:0/3909810826 shutdown_connections 2024-03-18T23:07:35.442 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.440+0000 7f27cf444640 1 --2- 172.21.15.49:0/3909810826 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f27c81022d0 0x7f27c8108b60 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:35.443 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.440+0000 7f27cf444640 1 --2- 172.21.15.49:0/3909810826 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f27c8101910 0x7f27c8101d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:35.443 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.440+0000 7f27cf444640 1 --2- 172.21.15.49:0/3909810826 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f27c8100720 0x7f27c8100b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:35.443 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.440+0000 7f27cf444640 1 -- 172.21.15.49:0/3909810826 >> 172.21.15.49:0/3909810826 conn(0x7f27c80fbcb0 msgr2=0x7f27c80fe0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:07:35.443 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.440+0000 7f27cf444640 1 -- 172.21.15.49:0/3909810826 shutdown_connections 2024-03-18T23:07:35.443 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.440+0000 7f27cf444640 1 -- 172.21.15.49:0/3909810826 wait complete. 2024-03-18T23:07:35.443 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.440+0000 7f27cf444640 1 Processor -- start 2024-03-18T23:07:35.443 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.440+0000 7f27cf444640 1 -- start start 2024-03-18T23:07:35.443 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.440+0000 7f27cf444640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f27c8100720 0x7f27c8199d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:35.443 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.440+0000 7f27cf444640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f27c8101910 0x7f27c819a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:35.443 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.440+0000 7f27cf444640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f27c81022d0 0x7f27c81a12f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:35.443 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.440+0000 7f27cf444640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f27c810b0c0 con 0x7f27c81022d0 2024-03-18T23:07:35.443 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.440+0000 7f27cf444640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f27c810af40 con 0x7f27c8101910 2024-03-18T23:07:35.443 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.440+0000 7f27cf444640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f27c810b240 con 0x7f27c8100720 2024-03-18T23:07:35.444 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.441+0000 7f27cd1e0640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f27c8100720 0x7f27c8199d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:35.444 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.441+0000 7f27cd9e1640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f27c81022d0 0x7f27c81a12f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:35.444 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.441+0000 7f27cc9df640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f27c8101910 0x7f27c819a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:35.444 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.441+0000 7f27cd1e0640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f27c8100720 0x7f27c8199d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.49:44792/0 (socket says 172.21.15.49:44792) 2024-03-18T23:07:35.444 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.441+0000 7f27cd1e0640 1 -- 172.21.15.49:0/3938689307 learned_addr learned my addr 172.21.15.49:0/3938689307 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:07:35.444 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.441+0000 7f27cc9df640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f27c8101910 0x7f27c819a280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.49:49210/0 (socket says 172.21.15.49:49210) 2024-03-18T23:07:35.444 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.441+0000 7f27cd9e1640 1 -- 172.21.15.49:0/3938689307 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f27c8100720 msgr2=0x7f27c8199d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:35.444 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.441+0000 7f27cd9e1640 1 --2- 172.21.15.49:0/3938689307 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f27c8100720 0x7f27c8199d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:35.444 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.441+0000 7f27cd9e1640 1 -- 172.21.15.49:0/3938689307 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f27c8101910 msgr2=0x7f27c819a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:35.444 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.441+0000 7f27cd9e1640 1 --2- 172.21.15.49:0/3938689307 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f27c8101910 0x7f27c819a280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:35.444 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.441+0000 7f27cd9e1640 1 -- 172.21.15.49:0/3938689307 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27c81a1960 con 0x7f27c81022d0 2024-03-18T23:07:35.444 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.441+0000 7f27cc9df640 1 --2- 172.21.15.49:0/3938689307 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f27c8101910 0x7f27c819a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:07:35.444 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.441+0000 7f27cd9e1640 1 --2- 172.21.15.49:0/3938689307 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f27c81022d0 0x7f27c81a12f0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f27c4037040 tx=0x7f27c4032ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:07:35.444 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.441+0000 7f27be7fc640 1 -- 172.21.15.49:0/3938689307 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27c4041020 con 0x7f27c81022d0 2024-03-18T23:07:35.445 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.441+0000 7f27cf444640 1 -- 172.21.15.49:0/3938689307 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27c81a1bf0 con 0x7f27c81022d0 2024-03-18T23:07:35.445 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.441+0000 7f27be7fc640 1 -- 172.21.15.49:0/3938689307 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f27c4034cd0 con 0x7f27c81022d0 2024-03-18T23:07:35.445 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.441+0000 7f27be7fc640 1 -- 172.21.15.49:0/3938689307 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27c4039040 con 0x7f27c81022d0 2024-03-18T23:07:35.445 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.441+0000 7f27cf444640 1 -- 172.21.15.49:0/3938689307 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27c81a20b0 con 0x7f27c81022d0 2024-03-18T23:07:35.450 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.443+0000 7f27be7fc640 1 -- 172.21.15.49:0/3938689307 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7f27c4044020 con 0x7f27c81022d0 2024-03-18T23:07:35.451 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.444+0000 7f27d202b740 1 Processor -- start 2024-03-18T23:07:35.451 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.444+0000 7f27d202b740 1 -- start start 2024-03-18T23:07:35.451 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.444+0000 7f27be7fc640 1 --2- 172.21.15.49:0/3938689307 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f27b40a2800 0x7f27b40a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:35.451 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.444+0000 7f27d202b740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560e563e2ff0 0x560e563e3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:35.451 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.444+0000 7f27d202b740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560e563e39e0 0x560e563e84e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:35.451 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.444+0000 7f27d202b740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560e563e8b50 0x560e563eafe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:35.451 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.444+0000 7f27d202b740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x560e56347de0 con 0x560e563e2ff0 2024-03-18T23:07:35.451 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.444+0000 7f27d202b740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x560e562f0c80 con 0x560e563e8b50 2024-03-18T23:07:35.451 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.444+0000 7f27d202b740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x560e56343e80 con 0x560e563e39e0 2024-03-18T23:07:35.451 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.444+0000 7f27be7fc640 1 -- 172.21.15.49:0/3938689307 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f27c40f0870 con 0x7f27c81022d0 2024-03-18T23:07:35.451 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.444+0000 7f27cd1e0640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560e563e2ff0 0x560e563e3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:35.451 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.444+0000 7f27cc9df640 1 --2- 172.21.15.49:0/3938689307 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f27b40a2800 0x7f27b40a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:35.451 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.444+0000 7f27cd1e0640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560e563e2ff0 0x560e563e3410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53876/0 (socket says 172.21.15.49:53876) 2024-03-18T23:07:35.451 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.444+0000 7f27cd1e0640 1 -- 172.21.15.49:0/3875681769 learned_addr learned my addr 172.21.15.49:0/3875681769 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:07:35.451 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.444+0000 7f27cd9e1640 1 --2- 172.21.15.49:0/3875681769 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560e563e8b50 0x560e563eafe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:35.452 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.444+0000 7f27cc9df640 1 --2- 172.21.15.49:0/3875681769 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560e563e39e0 0x560e563e84e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:35.452 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.445+0000 7f27cd1e0640 1 -- 172.21.15.49:0/3875681769 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560e563e39e0 msgr2=0x560e563e84e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:07:35.452 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.445+0000 7f27cd1e0640 1 --2- 172.21.15.49:0/3875681769 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560e563e39e0 0x560e563e84e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:35.452 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.445+0000 7f27cd1e0640 1 -- 172.21.15.49:0/3875681769 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560e563e8b50 msgr2=0x560e563eafe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:07:35.452 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.445+0000 7f27cd1e0640 1 --2- 172.21.15.49:0/3875681769 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560e563e8b50 0x560e563eafe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:35.452 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.445+0000 7f27cd1e0640 1 -- 172.21.15.49:0/3875681769 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560e56302fb0 con 0x560e563e2ff0 2024-03-18T23:07:35.452 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.445+0000 7f27cd1e0640 1 --2- 172.21.15.49:0/3875681769 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560e563e2ff0 0x560e563e3410 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f27c0009d60 tx=0x7f27c0011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a21b2414bf519ee5 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:07:35.452 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.445+0000 7f27cc9df640 1 --2- 172.21.15.49:0/3938689307 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f27b40a2800 0x7f27b40a4cc0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f27b8009fd0 tx=0x7f27b8009ca0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:07:35.452 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.445+0000 7f27b37fe640 1 -- 172.21.15.49:0/3875681769 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27c0004220 con 0x560e563e2ff0 2024-03-18T23:07:35.452 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.445+0000 7f27b37fe640 1 -- 172.21.15.49:0/3875681769 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f27c0009e50 con 0x560e563e2ff0 2024-03-18T23:07:35.452 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.445+0000 7f27b37fe640 1 -- 172.21.15.49:0/3875681769 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27c0018830 con 0x560e563e2ff0 2024-03-18T23:07:35.452 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.446+0000 7f27d202b740 1 -- 172.21.15.49:0/3875681769 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560e563e2ff0 msgr2=0x560e563e3410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:35.452 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.446+0000 7f27d202b740 1 --2- 172.21.15.49:0/3875681769 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560e563e2ff0 0x560e563e3410 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f27c0009d60 tx=0x7f27c0011810 comp rx=0 tx=0).stop 2024-03-18T23:07:35.452 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.446+0000 7f27d202b740 1 -- 172.21.15.49:0/3875681769 shutdown_connections 2024-03-18T23:07:35.452 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.446+0000 7f27d202b740 1 --2- 172.21.15.49:0/3875681769 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560e563e8b50 0x560e563eafe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:35.453 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.446+0000 7f27d202b740 1 --2- 172.21.15.49:0/3875681769 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560e563e39e0 0x560e563e84e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:35.453 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.446+0000 7f27d202b740 1 --2- 172.21.15.49:0/3875681769 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560e563e2ff0 0x560e563e3410 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:35.453 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.446+0000 7f27d202b740 1 -- 172.21.15.49:0/3875681769 >> 172.21.15.49:0/3875681769 conn(0x560e563de560 msgr2=0x560e563de960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:07:35.453 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.446+0000 7f27d202b740 1 -- 172.21.15.49:0/3875681769 shutdown_connections 2024-03-18T23:07:35.453 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.446+0000 7f27d202b740 1 -- 172.21.15.49:0/3875681769 wait complete. 2024-03-18T23:07:35.453 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.446+0000 7f27d202b740 1 -- start start 2024-03-18T23:07:35.453 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.446+0000 7f27d202b740 1 Processor -- start 2024-03-18T23:07:35.453 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.446+0000 7f27d202b740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560e563e2ff0 0x560e563ed500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:35.453 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.446+0000 7f27d202b740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560e563e39e0 0x560e563eda40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:35.453 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.447+0000 7f27d202b740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560e563e8b50 0x560e563f4a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:35.453 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.447+0000 7f27d202b740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x560e563ebb00 con 0x560e563e2ff0 2024-03-18T23:07:35.453 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.447+0000 7f27d202b740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x560e56303fc0 con 0x560e563e39e0 2024-03-18T23:07:35.453 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.447+0000 7f27d202b740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x560e563ebc80 con 0x560e563e8b50 2024-03-18T23:07:35.453 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.447+0000 7f27d202b740 10 client.0 _ll_register_callbacks cb 0x560e5633fa20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T23:07:35.453 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.447+0000 7f27cd1e0640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560e563e2ff0 0x560e563ed500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:35.454 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.447+0000 7f27cd1e0640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560e563e2ff0 0x560e563ed500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53880/0 (socket says 172.21.15.49:53880) 2024-03-18T23:07:35.454 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.447+0000 7f27cd1e0640 1 -- 172.21.15.49:0/3224283682 learned_addr learned my addr 172.21.15.49:0/3224283682 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:07:35.454 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.447+0000 7f27cd9e1640 1 --2- 172.21.15.49:0/3224283682 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560e563e8b50 0x560e563f4a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:35.454 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.447+0000 7f27cc9df640 1 --2- 172.21.15.49:0/3224283682 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560e563e39e0 0x560e563eda40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:35.454 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.447+0000 7f27cd1e0640 1 -- 172.21.15.49:0/3224283682 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560e563e8b50 msgr2=0x560e563f4a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:07:35.454 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.447+0000 7f27cd1e0640 1 --2- 172.21.15.49:0/3224283682 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560e563e8b50 0x560e563f4a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:35.454 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.447+0000 7f27cd1e0640 1 -- 172.21.15.49:0/3224283682 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560e563e39e0 msgr2=0x560e563eda40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:07:35.454 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.447+0000 7f27cd1e0640 1 --2- 172.21.15.49:0/3224283682 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560e563e39e0 0x560e563eda40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:35.454 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.447+0000 7f27cd1e0640 1 -- 172.21.15.49:0/3224283682 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560e5605c2d0 con 0x560e563e2ff0 2024-03-18T23:07:35.454 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.447+0000 7f27cd1e0640 1 --2- 172.21.15.49:0/3224283682 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560e563e2ff0 0x560e563ed500 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f27c0000c00 tx=0x7f27c0004250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be297a40b18f6a0a server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:07:35.454 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.447+0000 7f27b0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-18T23:07:35.454 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.447+0000 7f27b0ff9640 1 -- 172.21.15.49:0/3224283682 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27c001e040 con 0x560e563e2ff0 2024-03-18T23:07:35.454 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.447+0000 7f27b0ff9640 1 -- 172.21.15.49:0/3224283682 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f27c0005050 con 0x560e563e2ff0 2024-03-18T23:07:35.454 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.447+0000 7f27d202b740 1 -- 172.21.15.49:0/3224283682 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f27c0009e50 con 0x560e563e2ff0 2024-03-18T23:07:35.455 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.447+0000 7f27b0ff9640 1 -- 172.21.15.49:0/3224283682 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f27c0005c50 con 0x560e563e2ff0 2024-03-18T23:07:35.455 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.447+0000 7f27d202b740 10 client.15051 fetch_fsmap learned FSMap version 27 2024-03-18T23:07:35.455 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.447+0000 7f27d202b740 1 -- 172.21.15.49:0/3224283682 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x560e561266b0 con 0x560e563e2ff0 2024-03-18T23:07:35.455 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.448+0000 7f27b0ff9640 1 -- 172.21.15.49:0/3224283682 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f27c0009e50 con 0x560e563e2ff0 2024-03-18T23:07:35.455 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.448+0000 7f27b0ff9640 1 -- 172.21.15.49:0/3224283682 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f27c0018570 con 0x560e563e2ff0 2024-03-18T23:07:35.455 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.448+0000 7f27d202b740 10 client.15051 fetch_fsmap finished waiting for FSMap version 27 2024-03-18T23:07:35.455 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.448+0000 7f27d202b740 10 client.15051 resolve_mds: resolved name 'l' to daemon mds.l 2024-03-18T23:07:35.455 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.448+0000 7f27d202b740 20 client.15051 populate_metadata read hostname 'smithi049' 2024-03-18T23:07:35.455 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.448+0000 7f27d202b740 1 --2- 172.21.15.49:0/3224283682 >> [v2:172.21.15.175:6832/1702997173,v1:172.21.15.175:6833/1702997173] conn(0x560e563fa0f0 0x560e563fb560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:35.455 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.448+0000 7f27d202b740 4 client.15051 mds_command: new command op to 24391 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T23:07:35.455 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.448+0000 7f27d202b740 1 -- 172.21.15.49:0/3224283682 --> [v2:172.21.15.175:6832/1702997173,v1:172.21.15.175:6833/1702997173] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560e56355320 con 0x560e563fa0f0 2024-03-18T23:07:35.455 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.448+0000 7f27cc9df640 1 --2- 172.21.15.49:0/3224283682 >> [v2:172.21.15.175:6832/1702997173,v1:172.21.15.175:6833/1702997173] conn(0x560e563fa0f0 0x560e563fb560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:35.455 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.448+0000 7f27b0ff9640 10 client.15051 ms_handle_connect on v2:172.21.15.175:6832/1702997173 2024-03-18T23:07:35.455 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.448+0000 7f27cc9df640 1 --2- 172.21.15.49:0/3224283682 >> [v2:172.21.15.175:6832/1702997173,v1:172.21.15.175:6833/1702997173] conn(0x560e563fa0f0 0x560e563fb560 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=a638591ee1e9e83 server_cookie=1176674a23150cf4 in_seq=0 out_seq=0 2024-03-18T23:07:35.456 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.449+0000 7f27b0ff9640 1 -- 172.21.15.49:0/3224283682 <== mds.4 v2:172.21.15.175:6832/1702997173 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x560e56355320 con 0x560e563fa0f0 2024-03-18T23:07:35.456 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.449+0000 7f27b0ff9640 10 client.15051 handle_command_reply: tid=0 2024-03-18T23:07:35.456 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.449+0000 7f27d202b740 1 client.15051 shutdown 2024-03-18T23:07:35.456 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.449+0000 7f27d202b740 1 -- 172.21.15.49:0/3224283682 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560e563e2ff0 msgr2=0x560e563ed500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:35.456 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.449+0000 7f27d202b740 1 --2- 172.21.15.49:0/3224283682 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560e563e2ff0 0x560e563ed500 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f27c0000c00 tx=0x7f27c0004250 comp rx=0 tx=0).stop 2024-03-18T23:07:35.456 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.449+0000 7f27d202b740 1 -- 172.21.15.49:0/3224283682 shutdown_connections 2024-03-18T23:07:35.456 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.449+0000 7f27d202b740 1 --2- 172.21.15.49:0/3224283682 >> [v2:172.21.15.175:6832/1702997173,v1:172.21.15.175:6833/1702997173] conn(0x560e563fa0f0 0x560e563fb560 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:35.456 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.449+0000 7f27d202b740 1 --2- 172.21.15.49:0/3224283682 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560e563e8b50 0x560e563f4a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:35.456 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.449+0000 7f27d202b740 1 --2- 172.21.15.49:0/3224283682 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560e563e39e0 0x560e563eda40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:35.456 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.449+0000 7f27d202b740 1 --2- 172.21.15.49:0/3224283682 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560e563e2ff0 0x560e563ed500 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:35.456 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.449+0000 7f27d202b740 1 -- 172.21.15.49:0/3224283682 >> 172.21.15.49:0/3224283682 conn(0x560e563de560 msgr2=0x560e563dd7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:07:35.456 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.449+0000 7f27b0ff9640 0 client.15051 ms_handle_reset on v2:172.21.15.175:6832/1702997173 2024-03-18T23:07:35.456 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.449+0000 7f27d202b740 1 -- 172.21.15.49:0/3224283682 shutdown_connections 2024-03-18T23:07:35.456 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.449+0000 7f27d202b740 1 -- 172.21.15.49:0/3224283682 wait complete. 2024-03-18T23:07:35.456 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.450+0000 7f27d202b740 20 client.15051 trim_cache size 0 max 16384 2024-03-18T23:07:35.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:07:35 smithi175 ceph-mon[67352]: pgmap v335: 97 pgs: 97 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 341 B/s wr, 9 op/s 2024-03-18T23:07:35.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:07:35 smithi175 ceph-mon[67352]: from='client.? 172.21.15.49:0/1459876548' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-18T23:07:35.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:07:35 smithi175 ceph-mon[67352]: from='client.? 172.21.15.49:0/2549190071' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T23:07:35.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:07:35 smithi175 ceph-mon[67352]: from='client.? 172.21.15.49:0/1684981628' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T23:07:35.473 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.468+0000 7f27d202b740 1 Processor -- start 2024-03-18T23:07:35.474 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.468+0000 7f27d202b740 1 -- start start 2024-03-18T23:07:35.474 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.468+0000 7f27d202b740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560e563e2ff0 0x560e563f9020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:35.474 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.468+0000 7f27d202b740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560e563e39e0 0x560e563f4a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:35.474 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.468+0000 7f27d202b740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560e563e8b50 0x560e563dcf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:35.474 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.468+0000 7f27d202b740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x560e563f98a0 con 0x560e563e2ff0 2024-03-18T23:07:35.474 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.468+0000 7f27d202b740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x560e563f9f70 con 0x560e563e39e0 2024-03-18T23:07:35.474 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.468+0000 7f27d202b740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x560e563fa640 con 0x560e563e8b50 2024-03-18T23:07:35.474 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.468+0000 7f27cd1e0640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560e563e2ff0 0x560e563f9020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:35.474 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.468+0000 7f27cd1e0640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560e563e2ff0 0x560e563f9020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53882/0 (socket says 172.21.15.49:53882) 2024-03-18T23:07:35.474 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.468+0000 7f27cd1e0640 1 -- 172.21.15.49:0/68370556 learned_addr learned my addr 172.21.15.49:0/68370556 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:07:35.474 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.468+0000 7f27cd9e1640 1 --2- 172.21.15.49:0/68370556 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560e563e8b50 0x560e563dcf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:35.474 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.468+0000 7f27cc9df640 1 --2- 172.21.15.49:0/68370556 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560e563e39e0 0x560e563f4a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:35.474 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.468+0000 7f27cd1e0640 1 -- 172.21.15.49:0/68370556 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560e563e8b50 msgr2=0x560e563dcf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:07:35.475 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.468+0000 7f27cd1e0640 1 --2- 172.21.15.49:0/68370556 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560e563e8b50 0x560e563dcf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:35.475 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.468+0000 7f27cd1e0640 1 -- 172.21.15.49:0/68370556 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560e563e39e0 msgr2=0x560e563f4a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:07:35.475 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.468+0000 7f27cd1e0640 1 --2- 172.21.15.49:0/68370556 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560e563e39e0 0x560e563f4a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:35.475 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.468+0000 7f27cd1e0640 1 -- 172.21.15.49:0/68370556 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560e563df2d0 con 0x560e563e2ff0 2024-03-18T23:07:35.475 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.468+0000 7f27cc9df640 1 --2- 172.21.15.49:0/68370556 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560e563e39e0 0x560e563f4a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T23:07:35.475 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.468+0000 7f27cd1e0640 1 --2- 172.21.15.49:0/68370556 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560e563e2ff0 0x560e563f9020 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f27c00059f0 tx=0x7f27c00041b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75c7899da71b2115 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:07:35.475 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.468+0000 7f27b2ffd640 1 -- 172.21.15.49:0/68370556 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27c0009e50 con 0x560e563e2ff0 2024-03-18T23:07:35.475 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.468+0000 7f27b2ffd640 1 -- 172.21.15.49:0/68370556 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f27c0028070 con 0x560e563e2ff0 2024-03-18T23:07:35.475 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.468+0000 7f27b2ffd640 1 -- 172.21.15.49:0/68370556 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27c0005200 con 0x560e563e2ff0 2024-03-18T23:07:35.475 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.469+0000 7f27d202b740 1 -- 172.21.15.49:0/68370556 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560e563e2ff0 msgr2=0x560e563f9020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:35.475 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.469+0000 7f27d202b740 1 --2- 172.21.15.49:0/68370556 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560e563e2ff0 0x560e563f9020 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f27c00059f0 tx=0x7f27c00041b0 comp rx=0 tx=0).stop 2024-03-18T23:07:35.475 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.469+0000 7f27d202b740 1 -- 172.21.15.49:0/68370556 shutdown_connections 2024-03-18T23:07:35.475 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.469+0000 7f27d202b740 1 --2- 172.21.15.49:0/68370556 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560e563e8b50 0x560e563dcf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:35.475 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.469+0000 7f27d202b740 1 --2- 172.21.15.49:0/68370556 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560e563e39e0 0x560e563f4a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:35.476 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.469+0000 7f27d202b740 1 --2- 172.21.15.49:0/68370556 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560e563e2ff0 0x560e563f9020 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:35.476 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.469+0000 7f27d202b740 1 -- 172.21.15.49:0/68370556 >> 172.21.15.49:0/68370556 conn(0x560e563de560 msgr2=0x560e563de940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:07:35.476 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.469+0000 7f27d202b740 1 -- 172.21.15.49:0/68370556 shutdown_connections 2024-03-18T23:07:35.476 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.469+0000 7f27d202b740 1 -- 172.21.15.49:0/68370556 wait complete. 2024-03-18T23:07:35.476 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.469+0000 7f27d202b740 1 -- start start 2024-03-18T23:07:35.476 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.469+0000 7f27d202b740 1 Processor -- start 2024-03-18T23:07:35.476 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.470+0000 7f27d202b740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560e563e2ff0 0x560e563f0120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:35.476 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.470+0000 7f27d202b740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560e563e39e0 0x560e563f0660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:35.476 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.470+0000 7f27d202b740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560e563e8b50 0x560e563f3680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:35.476 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.470+0000 7f27d202b740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x560e563fb3e0 con 0x560e563e2ff0 2024-03-18T23:07:35.476 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.470+0000 7f27d202b740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x560e563fad10 con 0x560e563e8b50 2024-03-18T23:07:35.476 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.470+0000 7f27d202b740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x560e563fbab0 con 0x560e563e39e0 2024-03-18T23:07:35.476 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.470+0000 7f27d202b740 10 client.0 _ll_register_callbacks cb 0x7f27b80137d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T23:07:35.476 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.470+0000 7f27cd1e0640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560e563e2ff0 0x560e563f0120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:35.476 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.470+0000 7f27cd1e0640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560e563e2ff0 0x560e563f0120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53888/0 (socket says 172.21.15.49:53888) 2024-03-18T23:07:35.477 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.470+0000 7f27cd1e0640 1 -- 172.21.15.49:0/3042037413 learned_addr learned my addr 172.21.15.49:0/3042037413 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:07:35.477 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.470+0000 7f27cd9e1640 1 --2- 172.21.15.49:0/3042037413 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560e563e8b50 0x560e563f3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:35.477 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.470+0000 7f27cc9df640 1 --2- 172.21.15.49:0/3042037413 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560e563e39e0 0x560e563f0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:35.477 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.470+0000 7f27cd1e0640 1 -- 172.21.15.49:0/3042037413 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560e563e39e0 msgr2=0x560e563f0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:07:35.477 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.470+0000 7f27cd1e0640 1 --2- 172.21.15.49:0/3042037413 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560e563e39e0 0x560e563f0660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:35.477 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.470+0000 7f27cd1e0640 1 -- 172.21.15.49:0/3042037413 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560e563e8b50 msgr2=0x560e563f3680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:07:35.477 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.470+0000 7f27cd1e0640 1 --2- 172.21.15.49:0/3042037413 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560e563e8b50 0x560e563f3680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:35.477 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.470+0000 7f27cd1e0640 1 -- 172.21.15.49:0/3042037413 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560e563f3c70 con 0x560e563e2ff0 2024-03-18T23:07:35.477 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.470+0000 7f27cd1e0640 1 --2- 172.21.15.49:0/3042037413 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560e563e2ff0 0x560e563f0120 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f27c00058c0 tx=0x7f27c0009530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29fb9c4fe3bc64a server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:07:35.477 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.470+0000 7f27b0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-18T23:07:35.477 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.471+0000 7f27b0ff9640 1 -- 172.21.15.49:0/3042037413 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27c00185d0 con 0x560e563e2ff0 2024-03-18T23:07:35.477 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.471+0000 7f27b0ff9640 1 -- 172.21.15.49:0/3042037413 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f27c0004f80 con 0x560e563e2ff0 2024-03-18T23:07:35.477 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.471+0000 7f27d202b740 1 -- 172.21.15.49:0/3042037413 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f27c0028070 con 0x560e563e2ff0 2024-03-18T23:07:35.477 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.471+0000 7f27b0ff9640 1 -- 172.21.15.49:0/3042037413 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f27c0018770 con 0x560e563e2ff0 2024-03-18T23:07:35.478 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.471+0000 7f27d202b740 10 client.15057 fetch_fsmap learned FSMap version 27 2024-03-18T23:07:35.478 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.471+0000 7f27d202b740 1 -- 172.21.15.49:0/3042037413 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x560e5645fe60 con 0x560e563e2ff0 2024-03-18T23:07:35.478 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.471+0000 7f27b0ff9640 1 -- 172.21.15.49:0/3042037413 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f27c0028070 con 0x560e563e2ff0 2024-03-18T23:07:35.478 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.471+0000 7f27b0ff9640 1 -- 172.21.15.49:0/3042037413 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f27c004b020 con 0x560e563e2ff0 2024-03-18T23:07:35.478 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.471+0000 7f27d202b740 10 client.15057 fetch_fsmap finished waiting for FSMap version 27 2024-03-18T23:07:35.478 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.471+0000 7f27d202b740 10 client.15057 resolve_mds: resolved name 'l' to daemon mds.l 2024-03-18T23:07:35.478 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.471+0000 7f27d202b740 20 client.15057 populate_metadata read hostname 'smithi049' 2024-03-18T23:07:35.478 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.471+0000 7f27d202b740 1 --2- 172.21.15.49:0/3042037413 >> [v2:172.21.15.175:6832/1702997173,v1:172.21.15.175:6833/1702997173] conn(0x560e56464990 0x560e564636a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:35.478 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.471+0000 7f27d202b740 4 client.15057 mds_command: new command op to 24391 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-18T23:07:35.478 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.471+0000 7f27d202b740 1 -- 172.21.15.49:0/3042037413 --> [v2:172.21.15.175:6832/1702997173,v1:172.21.15.175:6833/1702997173] -- command(tid 0: {"prefix": "status"}) -- 0x560e5634f670 con 0x560e56464990 2024-03-18T23:07:35.478 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.471+0000 7f27cc9df640 1 --2- 172.21.15.49:0/3042037413 >> [v2:172.21.15.175:6832/1702997173,v1:172.21.15.175:6833/1702997173] conn(0x560e56464990 0x560e564636a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:35.478 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.472+0000 7f27b0ff9640 10 client.15057 ms_handle_connect on v2:172.21.15.175:6832/1702997173 2024-03-18T23:07:35.478 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.472+0000 7f27cc9df640 1 --2- 172.21.15.49:0/3042037413 >> [v2:172.21.15.175:6832/1702997173,v1:172.21.15.175:6833/1702997173] conn(0x560e56464990 0x560e564636a0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=c7346f116a016c62 server_cookie=2e36f8797d0ab253 in_seq=0 out_seq=0 2024-03-18T23:07:35.478 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.472+0000 7f27b0ff9640 1 -- 172.21.15.49:0/3042037413 <== mds.4 v2:172.21.15.175:6832/1702997173 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x560e5634f670 con 0x560e56464990 2024-03-18T23:07:35.479 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.472+0000 7f27b0ff9640 10 client.15057 handle_command_reply: tid=0 2024-03-18T23:07:35.479 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.472+0000 7f27d202b740 1 client.15057 shutdown 2024-03-18T23:07:35.479 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.472+0000 7f27d202b740 1 -- 172.21.15.49:0/3042037413 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560e563e2ff0 msgr2=0x560e563f0120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:35.479 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.472+0000 7f27d202b740 1 --2- 172.21.15.49:0/3042037413 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560e563e2ff0 0x560e563f0120 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f27c00058c0 tx=0x7f27c0009530 comp rx=0 tx=0).stop 2024-03-18T23:07:35.479 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.472+0000 7f27d202b740 1 -- 172.21.15.49:0/3042037413 shutdown_connections 2024-03-18T23:07:35.479 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.472+0000 7f27d202b740 1 --2- 172.21.15.49:0/3042037413 >> [v2:172.21.15.175:6832/1702997173,v1:172.21.15.175:6833/1702997173] conn(0x560e56464990 0x560e564636a0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:35.479 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.472+0000 7f27d202b740 1 --2- 172.21.15.49:0/3042037413 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560e563e8b50 0x560e563f3680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:35.479 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.472+0000 7f27d202b740 1 --2- 172.21.15.49:0/3042037413 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560e563e39e0 0x560e563f0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:35.479 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.472+0000 7f27d202b740 1 --2- 172.21.15.49:0/3042037413 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560e563e2ff0 0x560e563f0120 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:35.479 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.472+0000 7f27d202b740 1 -- 172.21.15.49:0/3042037413 >> 172.21.15.49:0/3042037413 conn(0x560e563de560 msgr2=0x560e563f9020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:07:35.479 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.472+0000 7f27b0ff9640 0 client.15057 ms_handle_reset on v2:172.21.15.175:6832/1702997173 2024-03-18T23:07:35.479 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.472+0000 7f27d202b740 1 -- 172.21.15.49:0/3042037413 shutdown_connections 2024-03-18T23:07:35.479 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.472+0000 7f27d202b740 1 -- 172.21.15.49:0/3042037413 wait complete. 2024-03-18T23:07:35.479 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.472+0000 7f27d202b740 20 client.15057 trim_cache size 0 max 16384 2024-03-18T23:07:35.480 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-03-18T23:07:35.480 INFO:teuthology.orchestra.run.smithi049.stdout: "cluster_fsid": "c63591ea-e57a-11ee-95c9-87774f69a715", 2024-03-18T23:07:35.480 INFO:teuthology.orchestra.run.smithi049.stdout: "whoami": 4, 2024-03-18T23:07:35.480 INFO:teuthology.orchestra.run.smithi049.stdout: "id": 24391, 2024-03-18T23:07:35.480 INFO:teuthology.orchestra.run.smithi049.stdout: "want_state": "up:active", 2024-03-18T23:07:35.480 INFO:teuthology.orchestra.run.smithi049.stdout: "state": "up:active", 2024-03-18T23:07:35.480 INFO:teuthology.orchestra.run.smithi049.stdout: "fs_name": "cephfs", 2024-03-18T23:07:35.480 INFO:teuthology.orchestra.run.smithi049.stdout: "rank_uptime": 84.477819049000004, 2024-03-18T23:07:35.480 INFO:teuthology.orchestra.run.smithi049.stdout: "mdsmap_epoch": 27, 2024-03-18T23:07:35.480 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_epoch": 88, 2024-03-18T23:07:35.480 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_epoch_barrier": 86, 2024-03-18T23:07:35.481 INFO:teuthology.orchestra.run.smithi049.stdout: "uptime": 126.003122562 2024-03-18T23:07:35.481 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-03-18T23:07:35.481 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.473+0000 7f27cf444640 1 -- 172.21.15.49:0/3938689307 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f27b40a2800 msgr2=0x7f27b40a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:35.481 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.473+0000 7f27cf444640 1 --2- 172.21.15.49:0/3938689307 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f27b40a2800 0x7f27b40a4cc0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f27b8009fd0 tx=0x7f27b8009ca0 comp rx=0 tx=0).stop 2024-03-18T23:07:35.482 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.473+0000 7f27cf444640 1 -- 172.21.15.49:0/3938689307 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f27c81022d0 msgr2=0x7f27c81a12f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:35.482 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.473+0000 7f27cf444640 1 --2- 172.21.15.49:0/3938689307 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f27c81022d0 0x7f27c81a12f0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f27c4037040 tx=0x7f27c4032ea0 comp rx=0 tx=0).stop 2024-03-18T23:07:35.482 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.473+0000 7f27cf444640 1 -- 172.21.15.49:0/3938689307 shutdown_connections 2024-03-18T23:07:35.482 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.473+0000 7f27cf444640 1 --2- 172.21.15.49:0/3938689307 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f27b40a2800 0x7f27b40a4cc0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:35.482 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.473+0000 7f27cf444640 1 --2- 172.21.15.49:0/3938689307 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f27c81022d0 0x7f27c81a12f0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:35.482 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.473+0000 7f27cf444640 1 --2- 172.21.15.49:0/3938689307 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f27c8101910 0x7f27c819a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:35.482 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.473+0000 7f27cf444640 1 --2- 172.21.15.49:0/3938689307 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f27c8100720 0x7f27c8199d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:35.482 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.473+0000 7f27cf444640 1 -- 172.21.15.49:0/3938689307 >> 172.21.15.49:0/3938689307 conn(0x7f27c80fbcb0 msgr2=0x7f27c8104c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:07:35.482 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.473+0000 7f27cf444640 1 -- 172.21.15.49:0/3938689307 shutdown_connections 2024-03-18T23:07:35.482 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.473+0000 7f27cf444640 1 -- 172.21.15.49:0/3938689307 wait complete. 2024-03-18T23:07:35.484 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.e status 2024-03-18T23:07:35.595 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.593+0000 7f30018ff640 1 -- 172.21.15.49:0/3175027227 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2ffc100730 msgr2=0x7f2ffc100b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:35.596 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.593+0000 7f30018ff640 1 --2- 172.21.15.49:0/3175027227 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2ffc100730 0x7f2ffc100b50 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f2fec009900 tx=0x7f2fec02bc30 comp rx=0 tx=0).stop 2024-03-18T23:07:35.596 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.593+0000 7f30018ff640 1 -- 172.21.15.49:0/3175027227 shutdown_connections 2024-03-18T23:07:35.596 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.593+0000 7f30018ff640 1 --2- 172.21.15.49:0/3175027227 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2ffc1022e0 0x7f2ffc108b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:35.596 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.593+0000 7f30018ff640 1 --2- 172.21.15.49:0/3175027227 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2ffc101920 0x7f2ffc101da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:35.596 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.593+0000 7f30018ff640 1 --2- 172.21.15.49:0/3175027227 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2ffc100730 0x7f2ffc100b50 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:35.596 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.593+0000 7f30018ff640 1 -- 172.21.15.49:0/3175027227 >> 172.21.15.49:0/3175027227 conn(0x7f2ffc0fbcc0 msgr2=0x7f2ffc0fe0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:07:35.596 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.593+0000 7f30018ff640 1 -- 172.21.15.49:0/3175027227 shutdown_connections 2024-03-18T23:07:35.596 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.593+0000 7f30018ff640 1 -- 172.21.15.49:0/3175027227 wait complete. 2024-03-18T23:07:35.596 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.593+0000 7f30018ff640 1 Processor -- start 2024-03-18T23:07:35.596 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.593+0000 7f30018ff640 1 -- start start 2024-03-18T23:07:35.596 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.594+0000 7f30018ff640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2ffc100730 0x7f2ffc197c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:35.596 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.594+0000 7f30018ff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2ffc101920 0x7f2ffc198170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:35.597 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.594+0000 7f30018ff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2ffc1022e0 0x7f2ffc19f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:35.597 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.594+0000 7f30018ff640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f2ffc10c3a0 con 0x7f2ffc101920 2024-03-18T23:07:35.597 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.594+0000 7f30018ff640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f2ffc10c220 con 0x7f2ffc1022e0 2024-03-18T23:07:35.597 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.594+0000 7f30018ff640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f2ffc10c520 con 0x7f2ffc100730 2024-03-18T23:07:35.597 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.594+0000 7f2ffa7fc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2ffc101920 0x7f2ffc198170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:35.597 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.594+0000 7f2ffaffd640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2ffc100730 0x7f2ffc197c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:35.597 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.594+0000 7f2ffa7fc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2ffc101920 0x7f2ffc198170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53906/0 (socket says 172.21.15.49:53906) 2024-03-18T23:07:35.597 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.594+0000 7f2ffa7fc640 1 -- 172.21.15.49:0/1124592864 learned_addr learned my addr 172.21.15.49:0/1124592864 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:07:35.597 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.594+0000 7f2ffb7fe640 1 --2- 172.21.15.49:0/1124592864 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2ffc1022e0 0x7f2ffc19f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:35.597 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.594+0000 7f2ffa7fc640 1 -- 172.21.15.49:0/1124592864 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2ffc100730 msgr2=0x7f2ffc197c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:35.598 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.594+0000 7f2ffa7fc640 1 --2- 172.21.15.49:0/1124592864 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2ffc100730 0x7f2ffc197c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:35.598 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.594+0000 7f2ffa7fc640 1 -- 172.21.15.49:0/1124592864 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2ffc1022e0 msgr2=0x7f2ffc19f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:35.598 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.594+0000 7f2ffa7fc640 1 --2- 172.21.15.49:0/1124592864 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2ffc1022e0 0x7f2ffc19f190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:35.598 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.594+0000 7f2ffa7fc640 1 -- 172.21.15.49:0/1124592864 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ffc19f800 con 0x7f2ffc101920 2024-03-18T23:07:35.598 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.594+0000 7f2ffa7fc640 1 --2- 172.21.15.49:0/1124592864 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2ffc101920 0x7f2ffc198170 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f2ff000e9f0 tx=0x7f2ff000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:07:35.598 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.594+0000 7f30008fd640 1 -- 172.21.15.49:0/1124592864 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ff000cd80 con 0x7f2ffc101920 2024-03-18T23:07:35.598 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.594+0000 7f30018ff640 1 -- 172.21.15.49:0/1124592864 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ffc19faf0 con 0x7f2ffc101920 2024-03-18T23:07:35.598 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.594+0000 7f30008fd640 1 -- 172.21.15.49:0/1124592864 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ff0005500 con 0x7f2ffc101920 2024-03-18T23:07:35.598 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.594+0000 7f30008fd640 1 -- 172.21.15.49:0/1124592864 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ff0010690 con 0x7f2ffc101920 2024-03-18T23:07:35.598 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.594+0000 7f30018ff640 1 -- 172.21.15.49:0/1124592864 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ffc1a0040 con 0x7f2ffc101920 2024-03-18T23:07:35.605 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.596+0000 7f30008fd640 1 -- 172.21.15.49:0/1124592864 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7f2ff00046c0 con 0x7f2ffc101920 2024-03-18T23:07:35.606 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.597+0000 7f30045ca740 1 Processor -- start 2024-03-18T23:07:35.606 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.597+0000 7f30045ca740 1 -- start start 2024-03-18T23:07:35.606 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.597+0000 7f30045ca740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564b91413ff0 0x564b91414410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:35.606 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.597+0000 7f30045ca740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564b914149e0 0x564b914194e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:35.606 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.597+0000 7f30045ca740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564b91419b50 0x564b9141bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:35.606 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.597+0000 7f30045ca740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x564b91378de0 con 0x564b91413ff0 2024-03-18T23:07:35.606 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.597+0000 7f30045ca740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x564b91321c80 con 0x564b914149e0 2024-03-18T23:07:35.606 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.597+0000 7f30045ca740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x564b91374e80 con 0x564b91419b50 2024-03-18T23:07:35.606 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.597+0000 7f2ffb7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564b91413ff0 0x564b91414410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:35.606 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.597+0000 7f2ffb7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564b91413ff0 0x564b91414410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53920/0 (socket says 172.21.15.49:53920) 2024-03-18T23:07:35.606 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.597+0000 7f2ffb7fe640 1 -- 172.21.15.49:0/1172921278 learned_addr learned my addr 172.21.15.49:0/1172921278 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:07:35.606 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.597+0000 7f30008fd640 1 --2- 172.21.15.49:0/1124592864 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f2fe40a2800 0x7f2fe40a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:35.607 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.597+0000 7f2ffaffd640 1 --2- 172.21.15.49:0/1172921278 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564b914149e0 0x564b914194e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:35.607 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.597+0000 7f2ffa7fc640 1 --2- 172.21.15.49:0/1172921278 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564b91419b50 0x564b9141bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:35.607 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.598+0000 7f2ffb7fe640 1 --2- 172.21.15.49:0/1124592864 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f2fe40a2800 0x7f2fe40a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:35.607 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.598+0000 7f2ffb7fe640 1 -- 172.21.15.49:0/1172921278 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564b91419b50 msgr2=0x564b9141bfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:07:35.607 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.598+0000 7f2ffb7fe640 1 --2- 172.21.15.49:0/1172921278 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564b91419b50 0x564b9141bfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:35.607 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.598+0000 7f2ffb7fe640 1 -- 172.21.15.49:0/1172921278 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564b914149e0 msgr2=0x564b914194e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:07:35.607 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.598+0000 7f2ffb7fe640 1 --2- 172.21.15.49:0/1172921278 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564b914149e0 0x564b914194e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:35.607 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.598+0000 7f2ffb7fe640 1 -- 172.21.15.49:0/1172921278 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564b91333fb0 con 0x564b91413ff0 2024-03-18T23:07:35.607 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.598+0000 7f2ffaffd640 1 --2- 172.21.15.49:0/1172921278 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564b914149e0 0x564b914194e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:07:35.607 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.598+0000 7f30008fd640 1 -- 172.21.15.49:0/1124592864 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f2ff00cee90 con 0x7f2ffc101920 2024-03-18T23:07:35.607 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.598+0000 7f2ffb7fe640 1 --2- 172.21.15.49:0/1172921278 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564b91413ff0 0x564b91414410 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f2ff4005c40 tx=0x7f2ff400f3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=292569f78a0af58c server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:07:35.607 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.598+0000 7f2fe1ffb640 1 -- 172.21.15.49:0/1172921278 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ff4022020 con 0x564b91413ff0 2024-03-18T23:07:35.607 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.598+0000 7f2fe1ffb640 1 -- 172.21.15.49:0/1172921278 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ff4013540 con 0x564b91413ff0 2024-03-18T23:07:35.607 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.598+0000 7f2ffb7fe640 1 --2- 172.21.15.49:0/1124592864 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f2fe40a2800 0x7f2fe40a4cc0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f2ff4023000 tx=0x7f2ff4021000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:07:35.607 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.598+0000 7f2fe1ffb640 1 -- 172.21.15.49:0/1172921278 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ff401ead0 con 0x564b91413ff0 2024-03-18T23:07:35.608 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.599+0000 7f30045ca740 1 -- 172.21.15.49:0/1172921278 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564b91413ff0 msgr2=0x564b91414410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:35.608 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.599+0000 7f30045ca740 1 --2- 172.21.15.49:0/1172921278 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564b91413ff0 0x564b91414410 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f2ff4005c40 tx=0x7f2ff400f3c0 comp rx=0 tx=0).stop 2024-03-18T23:07:35.608 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.599+0000 7f30045ca740 1 -- 172.21.15.49:0/1172921278 shutdown_connections 2024-03-18T23:07:35.608 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.599+0000 7f30045ca740 1 --2- 172.21.15.49:0/1172921278 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564b91419b50 0x564b9141bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:35.608 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.599+0000 7f30045ca740 1 --2- 172.21.15.49:0/1172921278 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564b914149e0 0x564b914194e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:35.608 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.599+0000 7f30045ca740 1 --2- 172.21.15.49:0/1172921278 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564b91413ff0 0x564b91414410 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:35.608 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.599+0000 7f30045ca740 1 -- 172.21.15.49:0/1172921278 >> 172.21.15.49:0/1172921278 conn(0x564b9140f560 msgr2=0x564b9140f960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:07:35.608 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.599+0000 7f30045ca740 1 -- 172.21.15.49:0/1172921278 shutdown_connections 2024-03-18T23:07:35.608 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.599+0000 7f30045ca740 1 -- 172.21.15.49:0/1172921278 wait complete. 2024-03-18T23:07:35.608 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.599+0000 7f30045ca740 1 -- start start 2024-03-18T23:07:35.608 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.599+0000 7f30045ca740 1 Processor -- start 2024-03-18T23:07:35.608 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.600+0000 7f30045ca740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564b91413ff0 0x564b9141e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:35.608 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.600+0000 7f30045ca740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564b914149e0 0x564b9141e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:35.608 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.600+0000 7f30045ca740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564b91419b50 0x564b91425a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:35.608 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.600+0000 7f30045ca740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x564b9141cb00 con 0x564b91419b50 2024-03-18T23:07:35.609 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.600+0000 7f30045ca740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x564b91334fc0 con 0x564b914149e0 2024-03-18T23:07:35.609 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.600+0000 7f30045ca740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x564b9141cc80 con 0x564b91413ff0 2024-03-18T23:07:35.609 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.600+0000 7f30045ca740 10 client.0 _ll_register_callbacks cb 0x564b91370a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T23:07:35.609 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.600+0000 7f2ffaffd640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564b91413ff0 0x564b9141e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:35.609 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.600+0000 7f2ffb7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564b91419b50 0x564b91425a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:35.609 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.600+0000 7f2ffb7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564b91419b50 0x564b91425a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53934/0 (socket says 172.21.15.49:53934) 2024-03-18T23:07:35.609 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.600+0000 7f2ffaffd640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564b91413ff0 0x564b9141e4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.49:44880/0 (socket says 172.21.15.49:44880) 2024-03-18T23:07:35.609 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.600+0000 7f2ffb7fe640 1 -- 172.21.15.49:0/2898525148 learned_addr learned my addr 172.21.15.49:0/2898525148 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:07:35.609 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.600+0000 7f2ffa7fc640 1 --2- 172.21.15.49:0/2898525148 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564b914149e0 0x564b9141e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:35.609 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.600+0000 7f2ffb7fe640 1 -- 172.21.15.49:0/2898525148 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564b91413ff0 msgr2=0x564b9141e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:07:35.609 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.600+0000 7f2ffb7fe640 1 --2- 172.21.15.49:0/2898525148 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564b91413ff0 0x564b9141e4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:35.609 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.600+0000 7f2ffb7fe640 1 -- 172.21.15.49:0/2898525148 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564b914149e0 msgr2=0x564b9141e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:07:35.609 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.600+0000 7f2ffb7fe640 1 --2- 172.21.15.49:0/2898525148 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564b914149e0 0x564b9141e9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:35.609 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.600+0000 7f2ffb7fe640 1 -- 172.21.15.49:0/2898525148 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564b9108d2d0 con 0x564b91419b50 2024-03-18T23:07:35.609 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.600+0000 7f2ffb7fe640 1 --2- 172.21.15.49:0/2898525148 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564b91419b50 0x564b91425a30 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f2ff4009c20 tx=0x7f2ff4009c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=344dca65ed0f6c6f server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:07:35.610 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.600+0000 7f2fdaffd640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-18T23:07:35.610 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.600+0000 7f2fdaffd640 1 -- 172.21.15.49:0/2898525148 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ff4022040 con 0x564b91419b50 2024-03-18T23:07:35.610 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.600+0000 7f2fdaffd640 1 -- 172.21.15.49:0/2898525148 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f2ff4013170 con 0x564b91419b50 2024-03-18T23:07:35.610 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.600+0000 7f30045ca740 1 -- 172.21.15.49:0/2898525148 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x564b91386320 con 0x564b91419b50 2024-03-18T23:07:35.610 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.601+0000 7f2fdaffd640 1 -- 172.21.15.49:0/2898525148 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f2ff401e420 con 0x564b91419b50 2024-03-18T23:07:35.610 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.601+0000 7f30045ca740 10 client.15069 fetch_fsmap learned FSMap version 27 2024-03-18T23:07:35.610 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.601+0000 7f30045ca740 1 -- 172.21.15.49:0/2898525148 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x564b911576b0 con 0x564b91419b50 2024-03-18T23:07:35.610 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.601+0000 7f2fdaffd640 1 -- 172.21.15.49:0/2898525148 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x564b91386320 con 0x564b91419b50 2024-03-18T23:07:35.610 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.601+0000 7f2fdaffd640 1 -- 172.21.15.49:0/2898525148 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f2ff4004a00 con 0x564b91419b50 2024-03-18T23:07:35.610 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.601+0000 7f30045ca740 10 client.15069 fetch_fsmap finished waiting for FSMap version 27 2024-03-18T23:07:35.610 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.601+0000 7f30045ca740 10 client.15069 resolve_mds: resolved name 'e' to daemon mds.e 2024-03-18T23:07:35.610 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.601+0000 7f30045ca740 20 client.15069 populate_metadata read hostname 'smithi049' 2024-03-18T23:07:35.610 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.601+0000 7f30045ca740 1 --2- 172.21.15.49:0/2898525148 >> [v2:172.21.15.139:6834/1663212937,v1:172.21.15.139:6835/1663212937] conn(0x564b9142b110 0x564b9142c580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:35.610 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.601+0000 7f30045ca740 4 client.15069 mds_command: new command op to 24419 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T23:07:35.611 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.601+0000 7f30045ca740 1 -- 172.21.15.49:0/2898525148 --> [v2:172.21.15.139:6834/1663212937,v1:172.21.15.139:6835/1663212937] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564b91380670 con 0x564b9142b110 2024-03-18T23:07:35.611 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.601+0000 7f2ffaffd640 1 --2- 172.21.15.49:0/2898525148 >> [v2:172.21.15.139:6834/1663212937,v1:172.21.15.139:6835/1663212937] conn(0x564b9142b110 0x564b9142c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:35.611 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.602+0000 7f2ffaffd640 1 --2- 172.21.15.49:0/2898525148 >> [v2:172.21.15.139:6834/1663212937,v1:172.21.15.139:6835/1663212937] conn(0x564b9142b110 0x564b9142c580 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=3563549f4304ceab server_cookie=758ff10423f277fd in_seq=0 out_seq=0 2024-03-18T23:07:35.611 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.602+0000 7f2fdaffd640 10 client.15069 ms_handle_connect on v2:172.21.15.139:6834/1663212937 2024-03-18T23:07:35.611 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.602+0000 7f2fdaffd640 1 -- 172.21.15.49:0/2898525148 <== mds.3 v2:172.21.15.139:6834/1663212937 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x564b91380670 con 0x564b9142b110 2024-03-18T23:07:35.611 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.602+0000 7f2fdaffd640 10 client.15069 handle_command_reply: tid=0 2024-03-18T23:07:35.611 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.602+0000 7f30045ca740 1 client.15069 shutdown 2024-03-18T23:07:35.611 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.602+0000 7f30045ca740 1 -- 172.21.15.49:0/2898525148 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564b91419b50 msgr2=0x564b91425a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:35.611 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.602+0000 7f30045ca740 1 --2- 172.21.15.49:0/2898525148 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564b91419b50 0x564b91425a30 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f2ff4009c20 tx=0x7f2ff4009c70 comp rx=0 tx=0).stop 2024-03-18T23:07:35.611 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.602+0000 7f30045ca740 1 -- 172.21.15.49:0/2898525148 shutdown_connections 2024-03-18T23:07:35.611 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.602+0000 7f30045ca740 1 --2- 172.21.15.49:0/2898525148 >> [v2:172.21.15.139:6834/1663212937,v1:172.21.15.139:6835/1663212937] conn(0x564b9142b110 0x564b9142c580 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:35.611 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.603+0000 7f30045ca740 1 --2- 172.21.15.49:0/2898525148 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564b91419b50 0x564b91425a30 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:35.611 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.603+0000 7f30045ca740 1 --2- 172.21.15.49:0/2898525148 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564b914149e0 0x564b9141e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:35.611 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.603+0000 7f2fdaffd640 0 client.15069 ms_handle_reset on v2:172.21.15.139:6834/1663212937 2024-03-18T23:07:35.612 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.603+0000 7f30045ca740 1 --2- 172.21.15.49:0/2898525148 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564b91413ff0 0x564b9141e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:35.612 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.603+0000 7f30045ca740 1 -- 172.21.15.49:0/2898525148 >> 172.21.15.49:0/2898525148 conn(0x564b9140f560 msgr2=0x564b9140e7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:07:35.612 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.603+0000 7f30045ca740 1 -- 172.21.15.49:0/2898525148 shutdown_connections 2024-03-18T23:07:35.612 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.603+0000 7f30045ca740 1 -- 172.21.15.49:0/2898525148 wait complete. 2024-03-18T23:07:35.612 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.603+0000 7f30045ca740 20 client.15069 trim_cache size 0 max 16384 2024-03-18T23:07:35.627 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.621+0000 7f30045ca740 1 Processor -- start 2024-03-18T23:07:35.627 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.621+0000 7f30045ca740 1 -- start start 2024-03-18T23:07:35.627 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.621+0000 7f30045ca740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564b91413ff0 0x564b9141e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:35.627 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.621+0000 7f30045ca740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564b914149e0 0x564b9141e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:35.627 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.621+0000 7f30045ca740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564b91419b50 0x564b91425950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:35.627 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.621+0000 7f30045ca740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x564b9142a8c0 con 0x564b91419b50 2024-03-18T23:07:35.627 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.621+0000 7f30045ca740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x564b9142af90 con 0x564b914149e0 2024-03-18T23:07:35.627 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.621+0000 7f30045ca740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x564b9142b660 con 0x564b91413ff0 2024-03-18T23:07:35.628 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.621+0000 7f2ffb7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564b91419b50 0x564b91425950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:35.628 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.621+0000 7f2ffb7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564b91419b50 0x564b91425950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53942/0 (socket says 172.21.15.49:53942) 2024-03-18T23:07:35.628 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.621+0000 7f2ffb7fe640 1 -- 172.21.15.49:0/3163362610 learned_addr learned my addr 172.21.15.49:0/3163362610 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:07:35.628 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.621+0000 7f2ffaffd640 1 --2- 172.21.15.49:0/3163362610 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564b91413ff0 0x564b9141e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:35.628 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.621+0000 7f2ffa7fc640 1 --2- 172.21.15.49:0/3163362610 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564b914149e0 0x564b9141e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:35.628 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.621+0000 7f2ffb7fe640 1 -- 172.21.15.49:0/3163362610 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564b91413ff0 msgr2=0x564b9141e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:07:35.628 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.621+0000 7f2ffb7fe640 1 --2- 172.21.15.49:0/3163362610 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564b91413ff0 0x564b9141e4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:35.628 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.621+0000 7f2ffb7fe640 1 -- 172.21.15.49:0/3163362610 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564b914149e0 msgr2=0x564b9141e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:07:35.628 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.621+0000 7f2ffb7fe640 1 --2- 172.21.15.49:0/3163362610 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564b914149e0 0x564b9141e9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:35.628 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.621+0000 7f2ffb7fe640 1 -- 172.21.15.49:0/3163362610 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564b9142c8b0 con 0x564b91419b50 2024-03-18T23:07:35.628 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.621+0000 7f2ffb7fe640 1 --2- 172.21.15.49:0/3163362610 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564b91419b50 0x564b91425950 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f2ff4004e40 tx=0x7f2ff400f3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8ffa7c52cc3087c server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:07:35.628 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.621+0000 7f2fe17fa640 1 -- 172.21.15.49:0/3163362610 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ff4042070 con 0x564b91419b50 2024-03-18T23:07:35.628 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.621+0000 7f2fe17fa640 1 -- 172.21.15.49:0/3163362610 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ff401eaa0 con 0x564b91419b50 2024-03-18T23:07:35.628 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.621+0000 7f2fe17fa640 1 -- 172.21.15.49:0/3163362610 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ff4022070 con 0x564b91419b50 2024-03-18T23:07:35.629 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.622+0000 7f30045ca740 1 -- 172.21.15.49:0/3163362610 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564b91419b50 msgr2=0x564b91425950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:35.629 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.622+0000 7f30045ca740 1 --2- 172.21.15.49:0/3163362610 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564b91419b50 0x564b91425950 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f2ff4004e40 tx=0x7f2ff400f3c0 comp rx=0 tx=0).stop 2024-03-18T23:07:35.629 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.622+0000 7f30045ca740 1 -- 172.21.15.49:0/3163362610 shutdown_connections 2024-03-18T23:07:35.629 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.622+0000 7f30045ca740 1 --2- 172.21.15.49:0/3163362610 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564b91419b50 0x564b91425950 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:35.629 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.622+0000 7f30045ca740 1 --2- 172.21.15.49:0/3163362610 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564b914149e0 0x564b9141e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:35.629 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.622+0000 7f30045ca740 1 --2- 172.21.15.49:0/3163362610 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564b91413ff0 0x564b9141e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:35.629 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.622+0000 7f30045ca740 1 -- 172.21.15.49:0/3163362610 >> 172.21.15.49:0/3163362610 conn(0x564b9140f560 msgr2=0x564b9140f940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:07:35.629 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.622+0000 7f30045ca740 1 -- 172.21.15.49:0/3163362610 shutdown_connections 2024-03-18T23:07:35.629 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.622+0000 7f30045ca740 1 -- 172.21.15.49:0/3163362610 wait complete. 2024-03-18T23:07:35.629 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.622+0000 7f30045ca740 1 -- start start 2024-03-18T23:07:35.629 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.622+0000 7f30045ca740 1 Processor -- start 2024-03-18T23:07:35.629 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.623+0000 7f30045ca740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564b91413ff0 0x564b91421430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:35.629 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.623+0000 7f30045ca740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564b914149e0 0x564b91421970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:35.629 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.623+0000 7f30045ca740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564b91419b50 0x564b91424920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:35.629 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.623+0000 7f30045ca740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x564b9142c400 con 0x564b91419b50 2024-03-18T23:07:35.630 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.623+0000 7f30045ca740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x564b9142bd30 con 0x564b91413ff0 2024-03-18T23:07:35.630 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.623+0000 7f30045ca740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x564b9142cad0 con 0x564b914149e0 2024-03-18T23:07:35.630 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.623+0000 7f30045ca740 10 client.0 _ll_register_callbacks cb 0x7f2ff0005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T23:07:35.630 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.623+0000 7f2ffb7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564b91419b50 0x564b91424920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:35.630 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.623+0000 7f2ffb7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564b91419b50 0x564b91424920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53952/0 (socket says 172.21.15.49:53952) 2024-03-18T23:07:35.630 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.623+0000 7f2ffb7fe640 1 -- 172.21.15.49:0/1845835184 learned_addr learned my addr 172.21.15.49:0/1845835184 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:07:35.630 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.623+0000 7f2ffa7fc640 1 --2- 172.21.15.49:0/1845835184 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564b914149e0 0x564b91421970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:35.630 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.623+0000 7f2ffaffd640 1 --2- 172.21.15.49:0/1845835184 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564b91413ff0 0x564b91421430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:35.630 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.623+0000 7f2ffb7fe640 1 -- 172.21.15.49:0/1845835184 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564b914149e0 msgr2=0x564b91421970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:07:35.630 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.623+0000 7f2ffb7fe640 1 --2- 172.21.15.49:0/1845835184 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564b914149e0 0x564b91421970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:35.630 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.623+0000 7f2ffb7fe640 1 -- 172.21.15.49:0/1845835184 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564b91413ff0 msgr2=0x564b91421430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:07:35.630 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.623+0000 7f2ffb7fe640 1 --2- 172.21.15.49:0/1845835184 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564b91413ff0 0x564b91421430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:35.630 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.623+0000 7f2ffb7fe640 1 -- 172.21.15.49:0/1845835184 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564b91424f10 con 0x564b91419b50 2024-03-18T23:07:35.631 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.623+0000 7f2ffaffd640 1 --2- 172.21.15.49:0/1845835184 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564b91413ff0 0x564b91421430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T23:07:35.631 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.623+0000 7f2ffb7fe640 1 --2- 172.21.15.49:0/1845835184 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564b91419b50 0x564b91424920 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f2ff4009cb0 tx=0x7f2ff4004fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc53388d2601e1d5 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:07:35.631 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.623+0000 7f2fdaffd640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-18T23:07:35.631 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.623+0000 7f2fdaffd640 1 -- 172.21.15.49:0/1845835184 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ff4042070 con 0x564b91419b50 2024-03-18T23:07:35.631 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.623+0000 7f2fdaffd640 1 -- 172.21.15.49:0/1845835184 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f2ff402f030 con 0x564b91419b50 2024-03-18T23:07:35.631 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.623+0000 7f30045ca740 1 -- 172.21.15.49:0/1845835184 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2ff401eaa0 con 0x564b91419b50 2024-03-18T23:07:35.631 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.624+0000 7f2fdaffd640 1 -- 172.21.15.49:0/1845835184 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f2ff4022040 con 0x564b91419b50 2024-03-18T23:07:35.631 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.624+0000 7f30045ca740 10 client.15075 fetch_fsmap learned FSMap version 27 2024-03-18T23:07:35.631 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.624+0000 7f30045ca740 1 -- 172.21.15.49:0/1845835184 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x564b91491170 con 0x564b91419b50 2024-03-18T23:07:35.631 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.624+0000 7f2fdaffd640 1 -- 172.21.15.49:0/1845835184 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ff401eaa0 con 0x564b91419b50 2024-03-18T23:07:35.631 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.624+0000 7f2fdaffd640 1 -- 172.21.15.49:0/1845835184 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f2ff40050a0 con 0x564b91419b50 2024-03-18T23:07:35.631 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.624+0000 7f30045ca740 10 client.15075 fetch_fsmap finished waiting for FSMap version 27 2024-03-18T23:07:35.631 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.624+0000 7f30045ca740 10 client.15075 resolve_mds: resolved name 'e' to daemon mds.e 2024-03-18T23:07:35.631 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.624+0000 7f30045ca740 20 client.15075 populate_metadata read hostname 'smithi049' 2024-03-18T23:07:35.632 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.624+0000 7f30045ca740 1 --2- 172.21.15.49:0/1845835184 >> [v2:172.21.15.139:6834/1663212937,v1:172.21.15.139:6835/1663212937] conn(0x564b91495aa0 0x564b914947b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:35.632 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.624+0000 7f30045ca740 4 client.15075 mds_command: new command op to 24419 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-18T23:07:35.632 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.624+0000 7f30045ca740 1 -- 172.21.15.49:0/1845835184 --> [v2:172.21.15.139:6834/1663212937,v1:172.21.15.139:6835/1663212937] -- command(tid 0: {"prefix": "status"}) -- 0x564b912fea00 con 0x564b91495aa0 2024-03-18T23:07:35.632 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.624+0000 7f2ffaffd640 1 --2- 172.21.15.49:0/1845835184 >> [v2:172.21.15.139:6834/1663212937,v1:172.21.15.139:6835/1663212937] conn(0x564b91495aa0 0x564b914947b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:35.632 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.625+0000 7f2fdaffd640 10 client.15075 ms_handle_connect on v2:172.21.15.139:6834/1663212937 2024-03-18T23:07:35.632 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.624+0000 7f2ffaffd640 1 --2- 172.21.15.49:0/1845835184 >> [v2:172.21.15.139:6834/1663212937,v1:172.21.15.139:6835/1663212937] conn(0x564b91495aa0 0x564b914947b0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=611e60cc9e524c7a server_cookie=b344ac78f61e8b0d in_seq=0 out_seq=0 2024-03-18T23:07:35.632 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.625+0000 7f2fdaffd640 1 -- 172.21.15.49:0/1845835184 <== mds.3 v2:172.21.15.139:6834/1663212937 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x564b912fea00 con 0x564b91495aa0 2024-03-18T23:07:35.632 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.625+0000 7f2fdaffd640 10 client.15075 handle_command_reply: tid=0 2024-03-18T23:07:35.632 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.625+0000 7f30045ca740 1 client.15075 shutdown 2024-03-18T23:07:35.632 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.625+0000 7f30045ca740 1 -- 172.21.15.49:0/1845835184 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564b91419b50 msgr2=0x564b91424920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:35.632 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.625+0000 7f30045ca740 1 --2- 172.21.15.49:0/1845835184 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564b91419b50 0x564b91424920 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f2ff4009cb0 tx=0x7f2ff4004fd0 comp rx=0 tx=0).stop 2024-03-18T23:07:35.632 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.625+0000 7f30045ca740 1 -- 172.21.15.49:0/1845835184 shutdown_connections 2024-03-18T23:07:35.632 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.625+0000 7f30045ca740 1 --2- 172.21.15.49:0/1845835184 >> [v2:172.21.15.139:6834/1663212937,v1:172.21.15.139:6835/1663212937] conn(0x564b91495aa0 0x564b914947b0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:35.632 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.625+0000 7f30045ca740 1 --2- 172.21.15.49:0/1845835184 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564b91419b50 0x564b91424920 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:35.632 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.625+0000 7f30045ca740 1 --2- 172.21.15.49:0/1845835184 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564b914149e0 0x564b91421970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:35.633 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.625+0000 7f2fdaffd640 0 client.15075 ms_handle_reset on v2:172.21.15.139:6834/1663212937 2024-03-18T23:07:35.633 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.625+0000 7f30045ca740 1 --2- 172.21.15.49:0/1845835184 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564b91413ff0 0x564b91421430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:35.633 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.625+0000 7f30045ca740 1 -- 172.21.15.49:0/1845835184 >> 172.21.15.49:0/1845835184 conn(0x564b9140f560 msgr2=0x564b9141e4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:07:35.633 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.625+0000 7f30045ca740 1 -- 172.21.15.49:0/1845835184 shutdown_connections 2024-03-18T23:07:35.633 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.625+0000 7f30045ca740 1 -- 172.21.15.49:0/1845835184 wait complete. 2024-03-18T23:07:35.633 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.625+0000 7f30045ca740 20 client.15075 trim_cache size 0 max 16384 2024-03-18T23:07:35.633 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-03-18T23:07:35.633 INFO:teuthology.orchestra.run.smithi049.stdout: "cluster_fsid": "c63591ea-e57a-11ee-95c9-87774f69a715", 2024-03-18T23:07:35.633 INFO:teuthology.orchestra.run.smithi049.stdout: "whoami": 3, 2024-03-18T23:07:35.633 INFO:teuthology.orchestra.run.smithi049.stdout: "id": 24419, 2024-03-18T23:07:35.634 INFO:teuthology.orchestra.run.smithi049.stdout: "want_state": "up:active", 2024-03-18T23:07:35.634 INFO:teuthology.orchestra.run.smithi049.stdout: "state": "up:active", 2024-03-18T23:07:35.634 INFO:teuthology.orchestra.run.smithi049.stdout: "fs_name": "cephfs", 2024-03-18T23:07:35.634 INFO:teuthology.orchestra.run.smithi049.stdout: "rank_uptime": 85.63605871, 2024-03-18T23:07:35.634 INFO:teuthology.orchestra.run.smithi049.stdout: "mdsmap_epoch": 27, 2024-03-18T23:07:35.634 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_epoch": 88, 2024-03-18T23:07:35.634 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_epoch_barrier": 86, 2024-03-18T23:07:35.634 INFO:teuthology.orchestra.run.smithi049.stdout: "uptime": 114.63341694099999 2024-03-18T23:07:35.634 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-03-18T23:07:35.635 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.626+0000 7f30018ff640 1 -- 172.21.15.49:0/1124592864 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f2fe40a2800 msgr2=0x7f2fe40a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:35.635 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.626+0000 7f30018ff640 1 --2- 172.21.15.49:0/1124592864 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f2fe40a2800 0x7f2fe40a4cc0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f2ff4023000 tx=0x7f2ff4021000 comp rx=0 tx=0).stop 2024-03-18T23:07:35.635 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.626+0000 7f30018ff640 1 -- 172.21.15.49:0/1124592864 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2ffc101920 msgr2=0x7f2ffc198170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:35.635 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.626+0000 7f30018ff640 1 --2- 172.21.15.49:0/1124592864 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2ffc101920 0x7f2ffc198170 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f2ff000e9f0 tx=0x7f2ff000edf0 comp rx=0 tx=0).stop 2024-03-18T23:07:35.635 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.626+0000 7f30018ff640 1 -- 172.21.15.49:0/1124592864 shutdown_connections 2024-03-18T23:07:35.635 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.626+0000 7f30018ff640 1 --2- 172.21.15.49:0/1124592864 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f2fe40a2800 0x7f2fe40a4cc0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:35.635 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.626+0000 7f30018ff640 1 --2- 172.21.15.49:0/1124592864 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2ffc1022e0 0x7f2ffc19f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:35.635 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.626+0000 7f30018ff640 1 --2- 172.21.15.49:0/1124592864 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2ffc101920 0x7f2ffc198170 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:35.635 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.626+0000 7f30018ff640 1 --2- 172.21.15.49:0/1124592864 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2ffc100730 0x7f2ffc197c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:35.636 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.626+0000 7f30018ff640 1 -- 172.21.15.49:0/1124592864 >> 172.21.15.49:0/1124592864 conn(0x7f2ffc0fbcc0 msgr2=0x7f2ffc0fd7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:07:35.636 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.626+0000 7f30018ff640 1 -- 172.21.15.49:0/1124592864 shutdown_connections 2024-03-18T23:07:35.636 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.626+0000 7f30018ff640 1 -- 172.21.15.49:0/1124592864 wait complete. 2024-03-18T23:07:35.638 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.f status 2024-03-18T23:07:35.748 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.745+0000 7f164d459640 1 -- 172.21.15.49:0/2323190017 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1648101830 msgr2=0x7f1648101c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:35.748 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.745+0000 7f164d459640 1 --2- 172.21.15.49:0/2323190017 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1648101830 0x7f1648101c50 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f1638009ca0 tx=0x7f163802c0f0 comp rx=0 tx=0).stop 2024-03-18T23:07:35.749 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.745+0000 7f164d459640 1 -- 172.21.15.49:0/2323190017 shutdown_connections 2024-03-18T23:07:35.749 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.745+0000 7f164d459640 1 --2- 172.21.15.49:0/2323190017 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f16481033e0 0x7f1648109c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:35.749 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.745+0000 7f164d459640 1 --2- 172.21.15.49:0/2323190017 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1648102a20 0x7f1648102ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:35.749 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.745+0000 7f164d459640 1 --2- 172.21.15.49:0/2323190017 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1648101830 0x7f1648101c50 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:35.749 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.745+0000 7f164d459640 1 -- 172.21.15.49:0/2323190017 >> 172.21.15.49:0/2323190017 conn(0x7f16480fcdc0 msgr2=0x7f16480ff1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:07:35.749 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.745+0000 7f164d459640 1 -- 172.21.15.49:0/2323190017 shutdown_connections 2024-03-18T23:07:35.749 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.745+0000 7f164d459640 1 -- 172.21.15.49:0/2323190017 wait complete. 2024-03-18T23:07:35.749 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.746+0000 7f164d459640 1 Processor -- start 2024-03-18T23:07:35.749 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.746+0000 7f164d459640 1 -- start start 2024-03-18T23:07:35.749 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.746+0000 7f164d459640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1648101830 0x7f16480755d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:35.749 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.746+0000 7f164d459640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1648102a20 0x7f1648073c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:35.749 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.746+0000 7f164d459640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f16481033e0 0x7f1648074150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:35.749 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.746+0000 7f164d459640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f164810d4b0 con 0x7f1648102a20 2024-03-18T23:07:35.749 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.746+0000 7f164d459640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f164810d330 con 0x7f16481033e0 2024-03-18T23:07:35.750 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.746+0000 7f164d459640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f164810d630 con 0x7f1648101830 2024-03-18T23:07:35.750 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.746+0000 7f16467fc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1648102a20 0x7f1648073c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:35.750 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.746+0000 7f16467fc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1648102a20 0x7f1648073c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53968/0 (socket says 172.21.15.49:53968) 2024-03-18T23:07:35.750 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.746+0000 7f16467fc640 1 -- 172.21.15.49:0/3778532918 learned_addr learned my addr 172.21.15.49:0/3778532918 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:07:35.750 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.746+0000 7f1646ffd640 1 --2- 172.21.15.49:0/3778532918 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1648101830 0x7f16480755d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:35.750 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.746+0000 7f16477fe640 1 --2- 172.21.15.49:0/3778532918 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f16481033e0 0x7f1648074150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:35.750 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.747+0000 7f16467fc640 1 -- 172.21.15.49:0/3778532918 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1648101830 msgr2=0x7f16480755d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:35.750 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.747+0000 7f16467fc640 1 --2- 172.21.15.49:0/3778532918 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1648101830 0x7f16480755d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:35.750 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.747+0000 7f16467fc640 1 -- 172.21.15.49:0/3778532918 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f16481033e0 msgr2=0x7f1648074150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:35.750 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.747+0000 7f16467fc640 1 --2- 172.21.15.49:0/3778532918 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f16481033e0 0x7f1648074150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:35.750 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.747+0000 7f16467fc640 1 -- 172.21.15.49:0/3778532918 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16480749e0 con 0x7f1648102a20 2024-03-18T23:07:35.750 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.747+0000 7f16467fc640 1 --2- 172.21.15.49:0/3778532918 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1648102a20 0x7f1648073c10 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f163000e9f0 tx=0x7f163000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:07:35.750 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.747+0000 7f163ffff640 1 -- 172.21.15.49:0/3778532918 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f163000cd80 con 0x7f1648102a20 2024-03-18T23:07:35.751 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.747+0000 7f163ffff640 1 -- 172.21.15.49:0/3778532918 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1630010910 con 0x7f1648102a20 2024-03-18T23:07:35.751 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.747+0000 7f164d459640 1 -- 172.21.15.49:0/3778532918 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16481a1cb0 con 0x7f1648102a20 2024-03-18T23:07:35.751 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.747+0000 7f163ffff640 1 -- 172.21.15.49:0/3778532918 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1630018990 con 0x7f1648102a20 2024-03-18T23:07:35.751 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.747+0000 7f164d459640 1 -- 172.21.15.49:0/3778532918 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16481a21a0 con 0x7f1648102a20 2024-03-18T23:07:35.758 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.749+0000 7f163ffff640 1 -- 172.21.15.49:0/3778532918 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7f1630010ac0 con 0x7f1648102a20 2024-03-18T23:07:35.758 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.750+0000 7f1650043740 1 Processor -- start 2024-03-18T23:07:35.758 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.750+0000 7f1650043740 1 -- start start 2024-03-18T23:07:35.758 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.750+0000 7f1650043740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c71288cff0 0x55c71288d410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:35.758 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.750+0000 7f1650043740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c71288d9e0 0x55c7128924e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:35.758 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.750+0000 7f163ffff640 1 --2- 172.21.15.49:0/3778532918 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f16340a2880 0x7f16340a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:35.758 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.750+0000 7f1650043740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c712892b50 0x55c712894fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:35.758 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.750+0000 7f1650043740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55c7127f1de0 con 0x55c71288d9e0 2024-03-18T23:07:35.758 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.750+0000 7f1650043740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55c71279ac80 con 0x55c71288cff0 2024-03-18T23:07:35.759 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.750+0000 7f1650043740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55c7127ede80 con 0x55c712892b50 2024-03-18T23:07:35.759 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.750+0000 7f16467fc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c71288d9e0 0x55c7128924e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:35.759 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.750+0000 7f1646ffd640 1 --2- 172.21.15.49:0/3778532918 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f16340a2880 0x7f16340a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:35.759 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.750+0000 7f16477fe640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c712892b50 0x55c712894fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:35.759 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.750+0000 7f16477fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c71288cff0 0x55c71288d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:35.759 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.750+0000 7f16467fc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c71288d9e0 0x55c7128924e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53978/0 (socket says 172.21.15.49:53978) 2024-03-18T23:07:35.759 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.750+0000 7f16467fc640 1 -- 172.21.15.49:0/2352032327 learned_addr learned my addr 172.21.15.49:0/2352032327 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:07:35.759 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.750+0000 7f163ffff640 1 -- 172.21.15.49:0/3778532918 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f16300ced10 con 0x7f1648102a20 2024-03-18T23:07:35.759 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.751+0000 7f16477fe640 1 -- 172.21.15.49:0/2352032327 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c712892b50 msgr2=0x55c712894fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:07:35.759 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.751+0000 7f16477fe640 1 --2- 172.21.15.49:0/2352032327 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c712892b50 0x55c712894fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:35.759 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.751+0000 7f16477fe640 1 -- 172.21.15.49:0/2352032327 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c71288d9e0 msgr2=0x55c7128924e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:07:35.759 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.751+0000 7f16477fe640 1 --2- 172.21.15.49:0/2352032327 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c71288d9e0 0x55c7128924e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:35.759 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.751+0000 7f16477fe640 1 -- 172.21.15.49:0/2352032327 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c7127acfb0 con 0x55c71288cff0 2024-03-18T23:07:35.759 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.751+0000 7f16467fc640 1 --2- 172.21.15.49:0/2352032327 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c71288d9e0 0x55c7128924e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:07:35.760 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.751+0000 7f1646ffd640 1 --2- 172.21.15.49:0/3778532918 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f16340a2880 0x7f16340a4d40 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f163802c700 tx=0x7f1638032590 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:07:35.760 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.751+0000 7f16477fe640 1 --2- 172.21.15.49:0/2352032327 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c71288cff0 0x55c71288d410 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f1640010860 tx=0x7f1640009f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f90f00f92bd9c63b server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:07:35.760 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.751+0000 7f163d7fa640 1 -- 172.21.15.49:0/2352032327 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1640016030 con 0x55c71288cff0 2024-03-18T23:07:35.760 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.751+0000 7f163d7fa640 1 -- 172.21.15.49:0/2352032327 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f164000f7c0 con 0x55c71288cff0 2024-03-18T23:07:35.760 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.751+0000 7f163d7fa640 1 -- 172.21.15.49:0/2352032327 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1640019040 con 0x55c71288cff0 2024-03-18T23:07:35.760 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.752+0000 7f1650043740 1 -- 172.21.15.49:0/2352032327 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c71288cff0 msgr2=0x55c71288d410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:35.760 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.752+0000 7f1650043740 1 --2- 172.21.15.49:0/2352032327 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c71288cff0 0x55c71288d410 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f1640010860 tx=0x7f1640009f90 comp rx=0 tx=0).stop 2024-03-18T23:07:35.760 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.752+0000 7f1650043740 1 -- 172.21.15.49:0/2352032327 shutdown_connections 2024-03-18T23:07:35.760 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.752+0000 7f1650043740 1 --2- 172.21.15.49:0/2352032327 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c712892b50 0x55c712894fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:35.760 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.752+0000 7f1650043740 1 --2- 172.21.15.49:0/2352032327 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c71288d9e0 0x55c7128924e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:35.760 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.752+0000 7f1650043740 1 --2- 172.21.15.49:0/2352032327 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c71288cff0 0x55c71288d410 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:35.760 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.752+0000 7f1650043740 1 -- 172.21.15.49:0/2352032327 >> 172.21.15.49:0/2352032327 conn(0x55c712888560 msgr2=0x55c712888960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:07:35.760 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.752+0000 7f1650043740 1 -- 172.21.15.49:0/2352032327 shutdown_connections 2024-03-18T23:07:35.760 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.752+0000 7f1650043740 1 -- 172.21.15.49:0/2352032327 wait complete. 2024-03-18T23:07:35.761 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.752+0000 7f1650043740 1 -- start start 2024-03-18T23:07:35.761 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.752+0000 7f1650043740 1 Processor -- start 2024-03-18T23:07:35.761 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.752+0000 7f1650043740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c71288cff0 0x55c712897540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:35.761 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.752+0000 7f1650043740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c71288d9e0 0x55c712897a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:35.761 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.752+0000 7f1650043740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c712892b50 0x55c71289ea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:35.761 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.752+0000 7f1650043740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55c712895b00 con 0x55c712892b50 2024-03-18T23:07:35.761 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.752+0000 7f1650043740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55c7127adfc0 con 0x55c71288cff0 2024-03-18T23:07:35.761 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.752+0000 7f1650043740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55c712895c80 con 0x55c71288d9e0 2024-03-18T23:07:35.761 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.752+0000 7f1650043740 10 client.0 _ll_register_callbacks cb 0x55c7127e9a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T23:07:35.761 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.753+0000 7f16477fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c712892b50 0x55c71289ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:35.761 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.753+0000 7f1646ffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c71288cff0 0x55c712897540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:35.761 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.753+0000 7f16467fc640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c71288d9e0 0x55c712897a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:35.761 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.753+0000 7f16477fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c712892b50 0x55c71289ea50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53990/0 (socket says 172.21.15.49:53990) 2024-03-18T23:07:35.761 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.753+0000 7f16477fe640 1 -- 172.21.15.49:0/453390004 learned_addr learned my addr 172.21.15.49:0/453390004 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:07:35.762 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.753+0000 7f16477fe640 1 -- 172.21.15.49:0/453390004 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c71288d9e0 msgr2=0x55c712897a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:07:35.762 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.753+0000 7f16477fe640 1 --2- 172.21.15.49:0/453390004 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c71288d9e0 0x55c712897a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:35.762 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.753+0000 7f16477fe640 1 -- 172.21.15.49:0/453390004 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c71288cff0 msgr2=0x55c712897540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:07:35.762 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.753+0000 7f16477fe640 1 --2- 172.21.15.49:0/453390004 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c71288cff0 0x55c712897540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:35.762 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.753+0000 7f16477fe640 1 -- 172.21.15.49:0/453390004 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c7125062d0 con 0x55c712892b50 2024-03-18T23:07:35.762 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.753+0000 7f16477fe640 1 --2- 172.21.15.49:0/453390004 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c712892b50 0x55c71289ea50 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f1640010830 tx=0x7f164000fe50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb148bd1177f6c26 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:07:35.762 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.753+0000 7f162e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-18T23:07:35.762 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.753+0000 7f162e7fc640 1 -- 172.21.15.49:0/453390004 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1640016070 con 0x55c712892b50 2024-03-18T23:07:35.762 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.753+0000 7f162e7fc640 1 -- 172.21.15.49:0/453390004 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f164001e3f0 con 0x55c712892b50 2024-03-18T23:07:35.762 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.753+0000 7f1650043740 1 -- 172.21.15.49:0/453390004 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f164000f7c0 con 0x55c712892b50 2024-03-18T23:07:35.762 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.753+0000 7f162e7fc640 1 -- 172.21.15.49:0/453390004 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f16400104d0 con 0x55c712892b50 2024-03-18T23:07:35.762 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.753+0000 7f1650043740 10 client.15087 fetch_fsmap learned FSMap version 27 2024-03-18T23:07:35.763 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.753+0000 7f1650043740 1 -- 172.21.15.49:0/453390004 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55c7125d06b0 con 0x55c712892b50 2024-03-18T23:07:35.763 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.754+0000 7f162e7fc640 1 -- 172.21.15.49:0/453390004 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f164000f7c0 con 0x55c712892b50 2024-03-18T23:07:35.763 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.754+0000 7f162e7fc640 1 -- 172.21.15.49:0/453390004 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f1640029790 con 0x55c712892b50 2024-03-18T23:07:35.763 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.754+0000 7f1650043740 10 client.15087 fetch_fsmap finished waiting for FSMap version 27 2024-03-18T23:07:35.763 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.754+0000 7f1650043740 10 client.15087 resolve_mds: resolved name 'f' to daemon mds.f 2024-03-18T23:07:35.763 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.754+0000 7f1650043740 20 client.15087 populate_metadata read hostname 'smithi049' 2024-03-18T23:07:35.763 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.754+0000 7f1650043740 1 --2- 172.21.15.49:0/453390004 >> [v2:172.21.15.175:6834/755474931,v1:172.21.15.175:6835/755474931] conn(0x55c7128a3f80 0x55c7128a53f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:35.763 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.754+0000 7f1650043740 4 client.15087 mds_command: new command op to 24430 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T23:07:35.763 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.754+0000 7f1650043740 1 -- 172.21.15.49:0/453390004 --> [v2:172.21.15.175:6834/755474931,v1:172.21.15.175:6835/755474931] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c7127ff320 con 0x55c7128a3f80 2024-03-18T23:07:35.763 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.754+0000 7f1646ffd640 1 --2- 172.21.15.49:0/453390004 >> [v2:172.21.15.175:6834/755474931,v1:172.21.15.175:6835/755474931] conn(0x55c7128a3f80 0x55c7128a53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:35.763 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.754+0000 7f1646ffd640 1 --2- 172.21.15.49:0/453390004 >> [v2:172.21.15.175:6834/755474931,v1:172.21.15.175:6835/755474931] conn(0x55c7128a3f80 0x55c7128a53f0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4e12213ed8422fe4 server_cookie=bf6a81b4552bad36 in_seq=0 out_seq=0 2024-03-18T23:07:35.763 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.754+0000 7f162e7fc640 10 client.15087 ms_handle_connect on v2:172.21.15.175:6834/755474931 2024-03-18T23:07:35.763 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.755+0000 7f162e7fc640 1 -- 172.21.15.49:0/453390004 <== mds.2 v2:172.21.15.175:6834/755474931 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55c7127ff320 con 0x55c7128a3f80 2024-03-18T23:07:35.764 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.755+0000 7f162e7fc640 10 client.15087 handle_command_reply: tid=0 2024-03-18T23:07:35.764 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.755+0000 7f1650043740 1 client.15087 shutdown 2024-03-18T23:07:35.764 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.755+0000 7f1650043740 1 -- 172.21.15.49:0/453390004 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c712892b50 msgr2=0x55c71289ea50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:35.764 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.755+0000 7f1650043740 1 --2- 172.21.15.49:0/453390004 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c712892b50 0x55c71289ea50 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f1640010830 tx=0x7f164000fe50 comp rx=0 tx=0).stop 2024-03-18T23:07:35.764 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.755+0000 7f1650043740 1 -- 172.21.15.49:0/453390004 shutdown_connections 2024-03-18T23:07:35.764 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.755+0000 7f1650043740 1 --2- 172.21.15.49:0/453390004 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c712892b50 0x55c71289ea50 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:35.764 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.755+0000 7f1650043740 1 --2- 172.21.15.49:0/453390004 >> [v2:172.21.15.175:6834/755474931,v1:172.21.15.175:6835/755474931] conn(0x55c7128a3f80 0x55c7128a53f0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:35.764 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.756+0000 7f1650043740 1 --2- 172.21.15.49:0/453390004 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c71288d9e0 0x55c712897a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:35.764 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.756+0000 7f162e7fc640 0 client.15087 ms_handle_reset on v2:172.21.15.175:6834/755474931 2024-03-18T23:07:35.764 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.756+0000 7f1650043740 1 --2- 172.21.15.49:0/453390004 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c71288cff0 0x55c712897540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:35.764 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.756+0000 7f1650043740 1 -- 172.21.15.49:0/453390004 >> 172.21.15.49:0/453390004 conn(0x55c712888560 msgr2=0x55c7128877e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:07:35.764 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.756+0000 7f1650043740 1 -- 172.21.15.49:0/453390004 shutdown_connections 2024-03-18T23:07:35.764 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.756+0000 7f1650043740 1 -- 172.21.15.49:0/453390004 wait complete. 2024-03-18T23:07:35.765 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.756+0000 7f1650043740 20 client.15087 trim_cache size 0 max 16384 2024-03-18T23:07:35.776 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.774+0000 7f1650043740 1 Processor -- start 2024-03-18T23:07:35.776 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.774+0000 7f1650043740 1 -- start start 2024-03-18T23:07:35.776 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.774+0000 7f1650043740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c71288cff0 0x55c712897500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:35.776 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.774+0000 7f1650043740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c71288d9e0 0x55c71289ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:35.776 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.774+0000 7f1650043740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c712892b50 0x55c712896830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:35.776 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.774+0000 7f1650043740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55c7128a3730 con 0x55c71288cff0 2024-03-18T23:07:35.776 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.774+0000 7f1650043740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55c7128a3e00 con 0x55c712892b50 2024-03-18T23:07:35.776 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.774+0000 7f1650043740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55c7128a44d0 con 0x55c71288d9e0 2024-03-18T23:07:35.776 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.774+0000 7f1646ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c71288cff0 0x55c712897500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:35.776 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.774+0000 7f1646ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c71288cff0 0x55c712897500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53996/0 (socket says 172.21.15.49:53996) 2024-03-18T23:07:35.777 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.774+0000 7f1646ffd640 1 -- 172.21.15.49:0/1687087728 learned_addr learned my addr 172.21.15.49:0/1687087728 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:07:35.777 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.774+0000 7f16467fc640 1 --2- 172.21.15.49:0/1687087728 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c71288d9e0 0x55c71289ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:35.777 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.774+0000 7f16477fe640 1 --2- 172.21.15.49:0/1687087728 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c712892b50 0x55c712896830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:35.777 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.774+0000 7f1646ffd640 1 -- 172.21.15.49:0/1687087728 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c71288d9e0 msgr2=0x55c71289ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:07:35.777 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.774+0000 7f1646ffd640 1 --2- 172.21.15.49:0/1687087728 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c71288d9e0 0x55c71289ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:35.777 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.774+0000 7f1646ffd640 1 -- 172.21.15.49:0/1687087728 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c712892b50 msgr2=0x55c712896830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:07:35.777 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.774+0000 7f1646ffd640 1 --2- 172.21.15.49:0/1687087728 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c712892b50 0x55c712896830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:35.777 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.774+0000 7f1646ffd640 1 -- 172.21.15.49:0/1687087728 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c712892970 con 0x55c71288cff0 2024-03-18T23:07:35.777 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.775+0000 7f1646ffd640 1 --2- 172.21.15.49:0/1687087728 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c71288cff0 0x55c712897500 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f163802cf30 tx=0x7f163803e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3197a53aef8cb20 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:07:35.777 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.775+0000 7f163cff9640 1 -- 172.21.15.49:0/1687087728 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1638039040 con 0x55c71288cff0 2024-03-18T23:07:35.777 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.775+0000 7f163cff9640 1 -- 172.21.15.49:0/1687087728 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1638003e50 con 0x55c71288cff0 2024-03-18T23:07:35.777 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.775+0000 7f163cff9640 1 -- 172.21.15.49:0/1687087728 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1638004040 con 0x55c71288cff0 2024-03-18T23:07:35.782 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.775+0000 7f1650043740 1 -- 172.21.15.49:0/1687087728 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c71288cff0 msgr2=0x55c712897500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:35.782 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.775+0000 7f1650043740 1 --2- 172.21.15.49:0/1687087728 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c71288cff0 0x55c712897500 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f163802cf30 tx=0x7f163803e000 comp rx=0 tx=0).stop 2024-03-18T23:07:35.782 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.775+0000 7f1650043740 1 -- 172.21.15.49:0/1687087728 shutdown_connections 2024-03-18T23:07:35.782 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.775+0000 7f1650043740 1 --2- 172.21.15.49:0/1687087728 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c712892b50 0x55c712896830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:35.782 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.776+0000 7f1650043740 1 --2- 172.21.15.49:0/1687087728 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c71288d9e0 0x55c71289ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:35.782 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.776+0000 7f1650043740 1 --2- 172.21.15.49:0/1687087728 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c71288cff0 0x55c712897500 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:35.782 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.776+0000 7f1650043740 1 -- 172.21.15.49:0/1687087728 >> 172.21.15.49:0/1687087728 conn(0x55c712888560 msgr2=0x55c712888940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:07:35.782 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.776+0000 7f1650043740 1 -- 172.21.15.49:0/1687087728 shutdown_connections 2024-03-18T23:07:35.782 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.776+0000 7f1650043740 1 -- 172.21.15.49:0/1687087728 wait complete. 2024-03-18T23:07:35.782 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.776+0000 7f1650043740 1 -- start start 2024-03-18T23:07:35.782 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.776+0000 7f1650043740 1 Processor -- start 2024-03-18T23:07:35.782 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.776+0000 7f1650043740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c71288cff0 0x55c71289aba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:35.783 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.776+0000 7f1650043740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c71288d9e0 0x55c71289b120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:35.783 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.776+0000 7f1650043740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c712892b50 0x55c71289de40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:35.783 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.776+0000 7f1650043740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55c7128a5270 con 0x55c71288d9e0 2024-03-18T23:07:35.783 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.776+0000 7f1650043740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55c7128a4ba0 con 0x55c712892b50 2024-03-18T23:07:35.783 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.776+0000 7f1650043740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55c7128a5940 con 0x55c71288cff0 2024-03-18T23:07:35.783 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.776+0000 7f1650043740 10 client.0 _ll_register_callbacks cb 0x7f163000cf20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T23:07:35.783 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.776+0000 7f16467fc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c71288d9e0 0x55c71289b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:35.783 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.776+0000 7f16477fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c712892b50 0x55c71289de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:35.783 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.776+0000 7f1646ffd640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c71288cff0 0x55c71289aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:35.783 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.776+0000 7f16467fc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c71288d9e0 0x55c71289b120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:54000/0 (socket says 172.21.15.49:54000) 2024-03-18T23:07:35.783 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.776+0000 7f16467fc640 1 -- 172.21.15.49:0/2201825155 learned_addr learned my addr 172.21.15.49:0/2201825155 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:07:35.783 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.777+0000 7f16467fc640 1 -- 172.21.15.49:0/2201825155 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c71288cff0 msgr2=0x55c71289aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:07:35.783 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.777+0000 7f16467fc640 1 --2- 172.21.15.49:0/2201825155 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c71288cff0 0x55c71289aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:35.784 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.777+0000 7f16467fc640 1 -- 172.21.15.49:0/2201825155 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c712892b50 msgr2=0x55c71289de40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:07:35.784 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.777+0000 7f16467fc640 1 --2- 172.21.15.49:0/2201825155 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c712892b50 0x55c71289de40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:35.784 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.777+0000 7f16467fc640 1 -- 172.21.15.49:0/2201825155 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c71289e430 con 0x55c71288d9e0 2024-03-18T23:07:35.784 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.777+0000 7f1646ffd640 1 --2- 172.21.15.49:0/2201825155 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c71288cff0 0x55c71289aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:07:35.784 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.777+0000 7f16477fe640 1 --2- 172.21.15.49:0/2201825155 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c712892b50 0x55c71289de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:07:35.784 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.777+0000 7f16467fc640 1 --2- 172.21.15.49:0/2201825155 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c71288d9e0 0x55c71289b120 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f1630000c00 tx=0x7f163006d4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c7f0f37e185c5b9 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:07:35.784 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.777+0000 7f162e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-18T23:07:35.784 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.777+0000 7f162e7fc640 1 -- 172.21.15.49:0/2201825155 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16300c5460 con 0x55c71288d9e0 2024-03-18T23:07:35.784 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.777+0000 7f162e7fc640 1 -- 172.21.15.49:0/2201825155 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f16300c5600 con 0x55c71288d9e0 2024-03-18T23:07:35.784 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.777+0000 7f1650043740 1 -- 172.21.15.49:0/2201825155 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1638003e50 con 0x55c71288d9e0 2024-03-18T23:07:35.784 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.777+0000 7f162e7fc640 1 -- 172.21.15.49:0/2201825155 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f16300a39d0 con 0x55c71288d9e0 2024-03-18T23:07:35.784 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.777+0000 7f1650043740 10 client.15093 fetch_fsmap learned FSMap version 27 2024-03-18T23:07:35.784 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.777+0000 7f1650043740 1 -- 172.21.15.49:0/2201825155 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55c71290a560 con 0x55c71288d9e0 2024-03-18T23:07:35.785 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.777+0000 7f162e7fc640 1 -- 172.21.15.49:0/2201825155 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1638003e50 con 0x55c71288d9e0 2024-03-18T23:07:35.785 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.777+0000 7f162e7fc640 1 -- 172.21.15.49:0/2201825155 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f16300ccce0 con 0x55c71288d9e0 2024-03-18T23:07:35.785 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.777+0000 7f1650043740 10 client.15093 fetch_fsmap finished waiting for FSMap version 27 2024-03-18T23:07:35.785 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.777+0000 7f1650043740 10 client.15093 resolve_mds: resolved name 'f' to daemon mds.f 2024-03-18T23:07:35.785 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.777+0000 7f1650043740 20 client.15093 populate_metadata read hostname 'smithi049' 2024-03-18T23:07:35.785 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.777+0000 7f1650043740 1 --2- 172.21.15.49:0/2201825155 >> [v2:172.21.15.175:6834/755474931,v1:172.21.15.175:6835/755474931] conn(0x55c71290f180 0x55c71290de90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:35.785 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.777+0000 7f1650043740 4 client.15093 mds_command: new command op to 24430 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-18T23:07:35.785 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.777+0000 7f1650043740 1 -- 172.21.15.49:0/2201825155 --> [v2:172.21.15.175:6834/755474931,v1:172.21.15.175:6835/755474931] -- command(tid 0: {"prefix": "status"}) -- 0x55c7127f9670 con 0x55c71290f180 2024-03-18T23:07:35.785 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.778+0000 7f1646ffd640 1 --2- 172.21.15.49:0/2201825155 >> [v2:172.21.15.175:6834/755474931,v1:172.21.15.175:6835/755474931] conn(0x55c71290f180 0x55c71290de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:35.785 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.778+0000 7f1646ffd640 1 --2- 172.21.15.49:0/2201825155 >> [v2:172.21.15.175:6834/755474931,v1:172.21.15.175:6835/755474931] conn(0x55c71290f180 0x55c71290de90 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=31d9661628157aca server_cookie=8bf5f96c53bef9f9 in_seq=0 out_seq=0 2024-03-18T23:07:35.785 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.778+0000 7f162e7fc640 10 client.15093 ms_handle_connect on v2:172.21.15.175:6834/755474931 2024-03-18T23:07:35.785 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.778+0000 7f162e7fc640 1 -- 172.21.15.49:0/2201825155 <== mds.2 v2:172.21.15.175:6834/755474931 1 ==== command_reply(tid 0: 0 ) ==== 8+0+326 (crc 0 0 0) 0x55c7127f9670 con 0x55c71290f180 2024-03-18T23:07:35.785 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.778+0000 7f162e7fc640 10 client.15093 handle_command_reply: tid=0 2024-03-18T23:07:35.785 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.778+0000 7f1650043740 1 client.15093 shutdown 2024-03-18T23:07:35.786 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.778+0000 7f1650043740 1 -- 172.21.15.49:0/2201825155 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c71288d9e0 msgr2=0x55c71289b120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:35.786 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.778+0000 7f1650043740 1 --2- 172.21.15.49:0/2201825155 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c71288d9e0 0x55c71289b120 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f1630000c00 tx=0x7f163006d4b0 comp rx=0 tx=0).stop 2024-03-18T23:07:35.786 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.778+0000 7f1650043740 1 -- 172.21.15.49:0/2201825155 shutdown_connections 2024-03-18T23:07:35.786 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.778+0000 7f1650043740 1 --2- 172.21.15.49:0/2201825155 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c712892b50 0x55c71289de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:35.786 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.778+0000 7f1650043740 1 --2- 172.21.15.49:0/2201825155 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c71288d9e0 0x55c71289b120 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:35.786 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.778+0000 7f1650043740 1 --2- 172.21.15.49:0/2201825155 >> [v2:172.21.15.175:6834/755474931,v1:172.21.15.175:6835/755474931] conn(0x55c71290f180 0x55c71290de90 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:35.786 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.778+0000 7f1650043740 1 --2- 172.21.15.49:0/2201825155 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c71288cff0 0x55c71289aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:35.786 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.778+0000 7f162e7fc640 0 client.15093 ms_handle_reset on v2:172.21.15.175:6834/755474931 2024-03-18T23:07:35.786 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.778+0000 7f1650043740 1 -- 172.21.15.49:0/2201825155 >> 172.21.15.49:0/2201825155 conn(0x55c712888560 msgr2=0x55c712883670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:07:35.786 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.778+0000 7f1650043740 1 -- 172.21.15.49:0/2201825155 shutdown_connections 2024-03-18T23:07:35.786 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.778+0000 7f1650043740 1 -- 172.21.15.49:0/2201825155 wait complete. 2024-03-18T23:07:35.786 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.779+0000 7f1650043740 20 client.15093 trim_cache size 0 max 16384 2024-03-18T23:07:35.787 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-03-18T23:07:35.787 INFO:teuthology.orchestra.run.smithi049.stdout: "cluster_fsid": "c63591ea-e57a-11ee-95c9-87774f69a715", 2024-03-18T23:07:35.787 INFO:teuthology.orchestra.run.smithi049.stdout: "whoami": 2, 2024-03-18T23:07:35.787 INFO:teuthology.orchestra.run.smithi049.stdout: "id": 24430, 2024-03-18T23:07:35.787 INFO:teuthology.orchestra.run.smithi049.stdout: "want_state": "up:active", 2024-03-18T23:07:35.787 INFO:teuthology.orchestra.run.smithi049.stdout: "state": "up:active", 2024-03-18T23:07:35.787 INFO:teuthology.orchestra.run.smithi049.stdout: "fs_name": "cephfs", 2024-03-18T23:07:35.787 INFO:teuthology.orchestra.run.smithi049.stdout: "rank_uptime": 86.793057133000005, 2024-03-18T23:07:35.787 INFO:teuthology.orchestra.run.smithi049.stdout: "mdsmap_epoch": 27, 2024-03-18T23:07:35.787 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_epoch": 88, 2024-03-18T23:07:35.787 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_epoch_barrier": 86, 2024-03-18T23:07:35.787 INFO:teuthology.orchestra.run.smithi049.stdout: "uptime": 111.97206509 2024-03-18T23:07:35.788 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-03-18T23:07:35.788 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.779+0000 7f164d459640 1 -- 172.21.15.49:0/3778532918 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f16340a2880 msgr2=0x7f16340a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:35.788 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.779+0000 7f164d459640 1 --2- 172.21.15.49:0/3778532918 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f16340a2880 0x7f16340a4d40 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f163802c700 tx=0x7f1638032590 comp rx=0 tx=0).stop 2024-03-18T23:07:35.788 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.779+0000 7f164d459640 1 -- 172.21.15.49:0/3778532918 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1648102a20 msgr2=0x7f1648073c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:35.789 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.779+0000 7f164d459640 1 --2- 172.21.15.49:0/3778532918 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1648102a20 0x7f1648073c10 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f163000e9f0 tx=0x7f163000eeb0 comp rx=0 tx=0).stop 2024-03-18T23:07:35.789 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.779+0000 7f164d459640 1 -- 172.21.15.49:0/3778532918 shutdown_connections 2024-03-18T23:07:35.789 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.779+0000 7f164d459640 1 --2- 172.21.15.49:0/3778532918 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f16340a2880 0x7f16340a4d40 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:35.789 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.779+0000 7f164d459640 1 --2- 172.21.15.49:0/3778532918 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f16481033e0 0x7f1648074150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:35.789 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.779+0000 7f164d459640 1 --2- 172.21.15.49:0/3778532918 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1648102a20 0x7f1648073c10 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:35.789 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.779+0000 7f164d459640 1 --2- 172.21.15.49:0/3778532918 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1648101830 0x7f16480755d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:35.789 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.779+0000 7f164d459640 1 -- 172.21.15.49:0/3778532918 >> 172.21.15.49:0/3778532918 conn(0x7f16480fcdc0 msgr2=0x7f1648105d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:07:35.789 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.779+0000 7f164d459640 1 -- 172.21.15.49:0/3778532918 shutdown_connections 2024-03-18T23:07:35.789 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.779+0000 7f164d459640 1 -- 172.21.15.49:0/3778532918 wait complete. 2024-03-18T23:07:35.791 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-03-18T23:07:35.902 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.899+0000 7f7195fda640 1 -- 172.21.15.49:0/1978410740 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7190101840 msgr2=0x7f7190101c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:35.903 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.899+0000 7f7195fda640 1 --2- 172.21.15.49:0/1978410740 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7190101840 0x7f7190101c60 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f7180009900 tx=0x7f718002bc30 comp rx=0 tx=0).stop 2024-03-18T23:07:35.903 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.899+0000 7f7195fda640 1 -- 172.21.15.49:0/1978410740 shutdown_connections 2024-03-18T23:07:35.903 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.899+0000 7f7195fda640 1 --2- 172.21.15.49:0/1978410740 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f71901033f0 0x7f7190109c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:35.903 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.899+0000 7f7195fda640 1 --2- 172.21.15.49:0/1978410740 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7190102a30 0x7f7190102eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:35.903 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.899+0000 7f7195fda640 1 --2- 172.21.15.49:0/1978410740 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7190101840 0x7f7190101c60 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:35.903 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.899+0000 7f7195fda640 1 -- 172.21.15.49:0/1978410740 >> 172.21.15.49:0/1978410740 conn(0x7f71900fcdb0 msgr2=0x7f71900ff1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:07:35.903 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.899+0000 7f7195fda640 1 -- 172.21.15.49:0/1978410740 shutdown_connections 2024-03-18T23:07:35.903 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.899+0000 7f7195fda640 1 -- 172.21.15.49:0/1978410740 wait complete. 2024-03-18T23:07:35.904 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.900+0000 7f7195fda640 1 Processor -- start 2024-03-18T23:07:35.904 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.900+0000 7f7195fda640 1 -- start start 2024-03-18T23:07:35.904 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.900+0000 7f7195fda640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7190101840 0x7f71900755d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:35.904 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.900+0000 7f7195fda640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7190102a30 0x7f7190073c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:35.904 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.900+0000 7f7195fda640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f71901033f0 0x7f7190074150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:35.904 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.900+0000 7f7195fda640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f719010d4e0 con 0x7f71901033f0 2024-03-18T23:07:35.904 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.900+0000 7f7195fda640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f719010d360 con 0x7f7190102a30 2024-03-18T23:07:35.904 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.900+0000 7f7195fda640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f719010d660 con 0x7f7190101840 2024-03-18T23:07:35.904 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.900+0000 7f718ffff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f71901033f0 0x7f7190074150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:35.904 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.900+0000 7f718ffff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f71901033f0 0x7f7190074150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:54020/0 (socket says 172.21.15.49:54020) 2024-03-18T23:07:35.904 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.900+0000 7f718ffff640 1 -- 172.21.15.49:0/114545921 learned_addr learned my addr 172.21.15.49:0/114545921 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:07:35.905 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.900+0000 7f718f7fe640 1 --2- 172.21.15.49:0/114545921 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7190101840 0x7f71900755d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:35.905 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.900+0000 7f718effd640 1 --2- 172.21.15.49:0/114545921 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7190102a30 0x7f7190073c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:35.905 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.901+0000 7f718ffff640 1 -- 172.21.15.49:0/114545921 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7190101840 msgr2=0x7f71900755d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:35.905 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.901+0000 7f718ffff640 1 --2- 172.21.15.49:0/114545921 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7190101840 0x7f71900755d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:35.905 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.901+0000 7f718ffff640 1 -- 172.21.15.49:0/114545921 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7190102a30 msgr2=0x7f7190073c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:35.905 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.901+0000 7f718ffff640 1 --2- 172.21.15.49:0/114545921 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7190102a30 0x7f7190073c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:35.905 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.901+0000 7f718ffff640 1 -- 172.21.15.49:0/114545921 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71900749e0 con 0x7f71901033f0 2024-03-18T23:07:35.905 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.901+0000 7f718effd640 1 --2- 172.21.15.49:0/114545921 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7190102a30 0x7f7190073c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:07:35.905 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.901+0000 7f718ffff640 1 --2- 172.21.15.49:0/114545921 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f71901033f0 0x7f7190074150 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f718800eb10 tx=0x7f718800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:07:35.905 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.901+0000 7f718cff9640 1 -- 172.21.15.49:0/114545921 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7188004b20 con 0x7f71901033f0 2024-03-18T23:07:35.905 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.901+0000 7f718cff9640 1 -- 172.21.15.49:0/114545921 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7188004cc0 con 0x7f71901033f0 2024-03-18T23:07:35.905 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.901+0000 7f7195fda640 1 -- 172.21.15.49:0/114545921 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71901a1c80 con 0x7f71901033f0 2024-03-18T23:07:35.905 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.901+0000 7f718cff9640 1 -- 172.21.15.49:0/114545921 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7188010760 con 0x7f71901033f0 2024-03-18T23:07:35.905 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.901+0000 7f7195fda640 1 -- 172.21.15.49:0/114545921 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71901a2180 con 0x7f71901033f0 2024-03-18T23:07:35.914 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.903+0000 7f718cff9640 1 -- 172.21.15.49:0/114545921 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7f7188005000 con 0x7f71901033f0 2024-03-18T23:07:35.914 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.903+0000 7f7198bbd740 1 Processor -- start 2024-03-18T23:07:35.914 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.904+0000 7f7198bbd740 1 -- start start 2024-03-18T23:07:35.914 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.904+0000 7f7198bbd740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555b69284ff0 0x555b69285410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:35.914 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.904+0000 7f7198bbd740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555b692859e0 0x555b6928a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:35.914 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.904+0000 7f7198bbd740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555b6928ab50 0x555b6928cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:35.915 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.904+0000 7f718cff9640 1 --2- 172.21.15.49:0/114545921 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f71780a2800 0x7f71780a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:35.915 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.904+0000 7f7198bbd740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x555b691e9de0 con 0x555b692859e0 2024-03-18T23:07:35.915 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.904+0000 7f7198bbd740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x555b69192c80 con 0x555b69284ff0 2024-03-18T23:07:35.915 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.904+0000 7f7198bbd740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x555b691e5e80 con 0x555b6928ab50 2024-03-18T23:07:35.915 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.904+0000 7f718f7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555b692859e0 0x555b6928a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:35.915 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.904+0000 7f718effd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555b69284ff0 0x555b69285410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:35.915 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.904+0000 7f718f7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555b692859e0 0x555b6928a4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:54036/0 (socket says 172.21.15.49:54036) 2024-03-18T23:07:35.915 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.904+0000 7f718f7fe640 1 -- 172.21.15.49:0/1736605650 learned_addr learned my addr 172.21.15.49:0/1736605650 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:07:35.915 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.904+0000 7f718ffff640 1 --2- 172.21.15.49:0/114545921 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f71780a2800 0x7f71780a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:35.915 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.904+0000 7f718effd640 1 --2- 172.21.15.49:0/1736605650 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555b6928ab50 0x555b6928cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:35.915 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.904+0000 7f718cff9640 1 -- 172.21.15.49:0/114545921 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f71880cf000 con 0x7f71901033f0 2024-03-18T23:07:35.915 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.904+0000 7f718f7fe640 1 -- 172.21.15.49:0/1736605650 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555b6928ab50 msgr2=0x555b6928cfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:07:35.915 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.904+0000 7f718f7fe640 1 --2- 172.21.15.49:0/1736605650 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555b6928ab50 0x555b6928cfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:35.916 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.904+0000 7f718f7fe640 1 -- 172.21.15.49:0/1736605650 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555b69284ff0 msgr2=0x555b69285410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:07:35.916 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.904+0000 7f718f7fe640 1 --2- 172.21.15.49:0/1736605650 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555b69284ff0 0x555b69285410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:35.916 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.904+0000 7f718f7fe640 1 -- 172.21.15.49:0/1736605650 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555b691a4fb0 con 0x555b692859e0 2024-03-18T23:07:35.916 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.904+0000 7f718effd640 1 --2- 172.21.15.49:0/1736605650 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555b69284ff0 0x555b69285410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:07:35.916 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.904+0000 7f718ffff640 1 --2- 172.21.15.49:0/114545921 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f71780a2800 0x7f71780a4cc0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f7188009150 tx=0x7f718806d040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:07:35.916 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.904+0000 7f718f7fe640 1 --2- 172.21.15.49:0/1736605650 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555b692859e0 0x555b6928a4e0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f7180000c00 tx=0x7f7180009560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76834ec273440d88 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:07:35.916 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.905+0000 7f7175ffb640 1 -- 172.21.15.49:0/1736605650 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f718002c980 con 0x555b692859e0 2024-03-18T23:07:35.916 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.905+0000 7f7175ffb640 1 -- 172.21.15.49:0/1736605650 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f718002cb20 con 0x555b692859e0 2024-03-18T23:07:35.916 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.905+0000 7f7175ffb640 1 -- 172.21.15.49:0/1736605650 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71800046c0 con 0x555b692859e0 2024-03-18T23:07:35.916 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.905+0000 7f7198bbd740 1 -- 172.21.15.49:0/1736605650 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555b692859e0 msgr2=0x555b6928a4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:35.916 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.905+0000 7f7198bbd740 1 --2- 172.21.15.49:0/1736605650 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555b692859e0 0x555b6928a4e0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f7180000c00 tx=0x7f7180009560 comp rx=0 tx=0).stop 2024-03-18T23:07:35.916 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.905+0000 7f7198bbd740 1 -- 172.21.15.49:0/1736605650 shutdown_connections 2024-03-18T23:07:35.916 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.905+0000 7f7198bbd740 1 --2- 172.21.15.49:0/1736605650 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555b6928ab50 0x555b6928cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:35.916 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.905+0000 7f7198bbd740 1 --2- 172.21.15.49:0/1736605650 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555b692859e0 0x555b6928a4e0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:35.917 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.905+0000 7f7198bbd740 1 --2- 172.21.15.49:0/1736605650 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555b69284ff0 0x555b69285410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:35.917 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.905+0000 7f7198bbd740 1 -- 172.21.15.49:0/1736605650 >> 172.21.15.49:0/1736605650 conn(0x555b69280560 msgr2=0x555b69280960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:07:35.917 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.905+0000 7f7198bbd740 1 -- 172.21.15.49:0/1736605650 shutdown_connections 2024-03-18T23:07:35.917 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.905+0000 7f7198bbd740 1 -- 172.21.15.49:0/1736605650 wait complete. 2024-03-18T23:07:35.917 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.906+0000 7f7198bbd740 1 -- start start 2024-03-18T23:07:35.917 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.906+0000 7f7198bbd740 1 Processor -- start 2024-03-18T23:07:35.917 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.906+0000 7f7198bbd740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555b69284ff0 0x555b6928f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:35.917 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.906+0000 7f7198bbd740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555b692859e0 0x555b6928f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:35.917 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.906+0000 7f7198bbd740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555b6928ab50 0x555b69296a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:35.917 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.906+0000 7f7198bbd740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x555b6928daa0 con 0x555b69284ff0 2024-03-18T23:07:35.917 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.906+0000 7f7198bbd740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x555b691a5fc0 con 0x555b6928ab50 2024-03-18T23:07:35.917 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.906+0000 7f7198bbd740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x555b6928dc20 con 0x555b692859e0 2024-03-18T23:07:35.917 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.906+0000 7f7198bbd740 10 client.0 _ll_register_callbacks cb 0x555b691e1a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T23:07:35.917 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.906+0000 7f718effd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555b69284ff0 0x555b6928f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:35.918 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.906+0000 7f718effd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555b69284ff0 0x555b6928f4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:54042/0 (socket says 172.21.15.49:54042) 2024-03-18T23:07:35.918 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.906+0000 7f718effd640 1 -- 172.21.15.49:0/3016630092 learned_addr learned my addr 172.21.15.49:0/3016630092 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:07:35.918 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.906+0000 7f718f7fe640 1 --2- 172.21.15.49:0/3016630092 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555b692859e0 0x555b6928f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:35.918 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.906+0000 7f718effd640 1 --2- 172.21.15.49:0/3016630092 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555b6928ab50 0x555b69296a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:35.918 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.906+0000 7f718effd640 1 -- 172.21.15.49:0/3016630092 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555b692859e0 msgr2=0x555b6928f9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:07:35.918 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.906+0000 7f718effd640 1 --2- 172.21.15.49:0/3016630092 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555b692859e0 0x555b6928f9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:35.918 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.906+0000 7f718effd640 1 -- 172.21.15.49:0/3016630092 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555b6928ab50 msgr2=0x555b69296a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:07:35.918 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.906+0000 7f718effd640 1 --2- 172.21.15.49:0/3016630092 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555b6928ab50 0x555b69296a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:35.918 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.906+0000 7f718effd640 1 -- 172.21.15.49:0/3016630092 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555b68efe2d0 con 0x555b69284ff0 2024-03-18T23:07:35.918 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.907+0000 7f718effd640 1 --2- 172.21.15.49:0/3016630092 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555b69284ff0 0x555b6928f4b0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f7184013b60 tx=0x7f7184009e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a9068d9d3669c63 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:07:35.918 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.907+0000 7f716effd640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-18T23:07:35.918 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.907+0000 7f716effd640 1 -- 172.21.15.49:0/3016630092 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71840118d0 con 0x555b69284ff0 2024-03-18T23:07:35.918 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.907+0000 7f716effd640 1 -- 172.21.15.49:0/3016630092 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f7184009040 con 0x555b69284ff0 2024-03-18T23:07:35.918 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.907+0000 7f7198bbd740 1 -- 172.21.15.49:0/3016630092 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f718002cb20 con 0x555b69284ff0 2024-03-18T23:07:35.919 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.907+0000 7f716effd640 1 -- 172.21.15.49:0/3016630092 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f7184011a70 con 0x555b69284ff0 2024-03-18T23:07:35.919 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.907+0000 7f7198bbd740 10 client.15105 fetch_fsmap learned FSMap version 27 2024-03-18T23:07:35.919 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.907+0000 7f7198bbd740 1 -- 172.21.15.49:0/3016630092 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x555b68fc86b0 con 0x555b69284ff0 2024-03-18T23:07:35.919 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.907+0000 7f716effd640 1 -- 172.21.15.49:0/3016630092 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f718002cb20 con 0x555b69284ff0 2024-03-18T23:07:35.919 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.907+0000 7f716effd640 1 -- 172.21.15.49:0/3016630092 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f7184020590 con 0x555b69284ff0 2024-03-18T23:07:35.919 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.907+0000 7f7198bbd740 10 client.15105 fetch_fsmap finished waiting for FSMap version 27 2024-03-18T23:07:35.919 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.907+0000 7f7198bbd740 10 client.15105 resolve_mds: resolved name 'i' to daemon mds.i 2024-03-18T23:07:35.919 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.907+0000 7f7198bbd740 20 client.15105 populate_metadata read hostname 'smithi049' 2024-03-18T23:07:35.919 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.907+0000 7f7198bbd740 1 --2- 172.21.15.49:0/3016630092 >> [v2:172.21.15.175:6838/3640882084,v1:172.21.15.175:6839/3640882084] conn(0x555b6929c2e0 0x555b6929d750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:35.919 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.907+0000 7f7198bbd740 4 client.15105 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T23:07:35.919 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.907+0000 7f7198bbd740 1 -- 172.21.15.49:0/3016630092 --> [v2:172.21.15.175:6838/3640882084,v1:172.21.15.175:6839/3640882084] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555b691f7320 con 0x555b6929c2e0 2024-03-18T23:07:35.919 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.908+0000 7f718f7fe640 1 --2- 172.21.15.49:0/3016630092 >> [v2:172.21.15.175:6838/3640882084,v1:172.21.15.175:6839/3640882084] conn(0x555b6929c2e0 0x555b6929d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:35.919 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.908+0000 7f716effd640 10 client.15105 ms_handle_connect on v2:172.21.15.175:6838/3640882084 2024-03-18T23:07:35.919 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.908+0000 7f718f7fe640 1 --2- 172.21.15.49:0/3016630092 >> [v2:172.21.15.175:6838/3640882084,v1:172.21.15.175:6839/3640882084] conn(0x555b6929c2e0 0x555b6929d750 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e63a5dd46ae69b38 server_cookie=b5baf523337ece5d in_seq=0 out_seq=0 2024-03-18T23:07:35.919 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.909+0000 7f716effd640 1 -- 172.21.15.49:0/3016630092 <== mds.1 v2:172.21.15.175:6838/3640882084 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x555b691f7320 con 0x555b6929c2e0 2024-03-18T23:07:35.920 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.909+0000 7f716effd640 10 client.15105 handle_command_reply: tid=0 2024-03-18T23:07:35.920 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.909+0000 7f7198bbd740 1 client.15105 shutdown 2024-03-18T23:07:35.920 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.909+0000 7f7198bbd740 1 -- 172.21.15.49:0/3016630092 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555b69284ff0 msgr2=0x555b6928f4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:35.920 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.909+0000 7f7198bbd740 1 --2- 172.21.15.49:0/3016630092 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555b69284ff0 0x555b6928f4b0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f7184013b60 tx=0x7f7184009e90 comp rx=0 tx=0).stop 2024-03-18T23:07:35.920 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.909+0000 7f7198bbd740 1 -- 172.21.15.49:0/3016630092 shutdown_connections 2024-03-18T23:07:35.920 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.909+0000 7f7198bbd740 1 --2- 172.21.15.49:0/3016630092 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555b6928ab50 0x555b69296a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:35.920 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.909+0000 7f7198bbd740 1 --2- 172.21.15.49:0/3016630092 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555b692859e0 0x555b6928f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:35.920 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.909+0000 7f7198bbd740 1 --2- 172.21.15.49:0/3016630092 >> [v2:172.21.15.175:6838/3640882084,v1:172.21.15.175:6839/3640882084] conn(0x555b6929c2e0 0x555b6929d750 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:35.920 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.909+0000 7f716effd640 0 client.15105 ms_handle_reset on v2:172.21.15.175:6838/3640882084 2024-03-18T23:07:35.920 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.909+0000 7f7198bbd740 1 --2- 172.21.15.49:0/3016630092 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555b69284ff0 0x555b6928f4b0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:35.920 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.909+0000 7f7198bbd740 1 -- 172.21.15.49:0/3016630092 >> 172.21.15.49:0/3016630092 conn(0x555b69280560 msgr2=0x555b6927f7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:07:35.920 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.909+0000 7f7198bbd740 1 -- 172.21.15.49:0/3016630092 shutdown_connections 2024-03-18T23:07:35.920 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.909+0000 7f7198bbd740 1 -- 172.21.15.49:0/3016630092 wait complete. 2024-03-18T23:07:35.920 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.909+0000 7f7198bbd740 20 client.15105 trim_cache size 0 max 16384 2024-03-18T23:07:35.933 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.927+0000 7f7198bbd740 1 Processor -- start 2024-03-18T23:07:35.933 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.927+0000 7f7198bbd740 1 -- start start 2024-03-18T23:07:35.933 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.927+0000 7f7198bbd740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555b69284ff0 0x555b69280940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:35.933 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.927+0000 7f7198bbd740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555b692859e0 0x555b692969f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:35.933 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.927+0000 7f7198bbd740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555b6928ab50 0x555b69286e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:35.933 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.927+0000 7f7198bbd740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x555b6929ba90 con 0x555b6928ab50 2024-03-18T23:07:35.934 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.927+0000 7f7198bbd740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x555b6929c160 con 0x555b692859e0 2024-03-18T23:07:35.934 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.927+0000 7f7198bbd740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x555b6929c830 con 0x555b69284ff0 2024-03-18T23:07:35.934 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.927+0000 7f718effd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555b6928ab50 0x555b69286e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:35.934 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.927+0000 7f718effd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555b6928ab50 0x555b69286e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:54058/0 (socket says 172.21.15.49:54058) 2024-03-18T23:07:35.934 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.927+0000 7f718effd640 1 -- 172.21.15.49:0/1211288481 learned_addr learned my addr 172.21.15.49:0/1211288481 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:07:35.934 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.927+0000 7f718f7fe640 1 --2- 172.21.15.49:0/1211288481 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555b692859e0 0x555b692969f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:35.934 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.928+0000 7f718effd640 1 --2- 172.21.15.49:0/1211288481 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555b69284ff0 0x555b69280940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:35.934 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.928+0000 7f718f7fe640 1 -- 172.21.15.49:0/1211288481 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555b69284ff0 msgr2=0x555b69280940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:07:35.934 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.928+0000 7f718f7fe640 1 --2- 172.21.15.49:0/1211288481 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555b69284ff0 0x555b69280940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:35.934 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.928+0000 7f718f7fe640 1 -- 172.21.15.49:0/1211288481 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555b6928ab50 msgr2=0x555b69286e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:07:35.934 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.928+0000 7f718f7fe640 1 --2- 172.21.15.49:0/1211288481 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555b6928ab50 0x555b69286e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:35.934 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.928+0000 7f718f7fe640 1 -- 172.21.15.49:0/1211288481 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555b6927b670 con 0x555b692859e0 2024-03-18T23:07:35.934 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.928+0000 7f718effd640 1 --2- 172.21.15.49:0/1211288481 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555b6928ab50 0x555b69286e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:07:35.935 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.928+0000 7f718f7fe640 1 --2- 172.21.15.49:0/1211288481 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555b692859e0 0x555b692969f0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x555b692814d0 tx=0x7f718002bc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8782f5bc2f9c92e3 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:07:35.935 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.928+0000 7f71757fa640 1 -- 172.21.15.49:0/1211288481 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7180005d50 con 0x555b692859e0 2024-03-18T23:07:35.935 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.928+0000 7f71757fa640 1 -- 172.21.15.49:0/1211288481 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f718002cae0 con 0x555b692859e0 2024-03-18T23:07:35.935 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.928+0000 7f71757fa640 1 -- 172.21.15.49:0/1211288481 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7180004b20 con 0x555b692859e0 2024-03-18T23:07:35.935 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.929+0000 7f7198bbd740 1 -- 172.21.15.49:0/1211288481 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555b692859e0 msgr2=0x555b692969f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:35.935 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.929+0000 7f7198bbd740 1 --2- 172.21.15.49:0/1211288481 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555b692859e0 0x555b692969f0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x555b692814d0 tx=0x7f718002bc30 comp rx=0 tx=0).stop 2024-03-18T23:07:35.935 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.929+0000 7f7198bbd740 1 -- 172.21.15.49:0/1211288481 shutdown_connections 2024-03-18T23:07:35.935 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.929+0000 7f7198bbd740 1 --2- 172.21.15.49:0/1211288481 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555b6928ab50 0x555b69286e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:35.935 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.929+0000 7f7198bbd740 1 --2- 172.21.15.49:0/1211288481 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555b692859e0 0x555b692969f0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:35.935 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.929+0000 7f7198bbd740 1 --2- 172.21.15.49:0/1211288481 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555b69284ff0 0x555b69280940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:35.935 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.929+0000 7f7198bbd740 1 -- 172.21.15.49:0/1211288481 >> 172.21.15.49:0/1211288481 conn(0x555b69280560 msgr2=0x555b6928e050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:07:35.935 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.929+0000 7f7198bbd740 1 -- 172.21.15.49:0/1211288481 shutdown_connections 2024-03-18T23:07:35.935 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.929+0000 7f7198bbd740 1 -- 172.21.15.49:0/1211288481 wait complete. 2024-03-18T23:07:35.935 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.929+0000 7f7198bbd740 1 -- start start 2024-03-18T23:07:35.935 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.929+0000 7f7198bbd740 1 Processor -- start 2024-03-18T23:07:35.936 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.929+0000 7f7198bbd740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555b69284ff0 0x555b692923e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:35.936 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.929+0000 7f7198bbd740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555b692859e0 0x555b69292920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:35.936 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.929+0000 7f7198bbd740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555b6928ab50 0x555b69295940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:35.936 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.929+0000 7f7198bbd740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x555b6929d5d0 con 0x555b69284ff0 2024-03-18T23:07:35.936 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.929+0000 7f7198bbd740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x555b6929cf00 con 0x555b6928ab50 2024-03-18T23:07:35.936 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.929+0000 7f7198bbd740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x555b6929dca0 con 0x555b692859e0 2024-03-18T23:07:35.936 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.930+0000 7f7198bbd740 10 client.0 _ll_register_callbacks cb 0x7f7184004040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T23:07:35.936 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.930+0000 7f718effd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555b69284ff0 0x555b692923e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:35.936 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.930+0000 7f718f7fe640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555b692859e0 0x555b69292920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:35.936 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.930+0000 7f718effd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555b69284ff0 0x555b692923e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:54070/0 (socket says 172.21.15.49:54070) 2024-03-18T23:07:35.936 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.930+0000 7f718effd640 1 -- 172.21.15.49:0/1386155261 learned_addr learned my addr 172.21.15.49:0/1386155261 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:07:35.936 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.930+0000 7f718effd640 1 --2- 172.21.15.49:0/1386155261 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555b6928ab50 0x555b69295940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:35.936 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.930+0000 7f718f7fe640 1 -- 172.21.15.49:0/1386155261 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555b6928ab50 msgr2=0x555b69295940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:07:35.936 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.930+0000 7f718f7fe640 1 --2- 172.21.15.49:0/1386155261 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555b6928ab50 0x555b69295940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:35.936 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.930+0000 7f718f7fe640 1 -- 172.21.15.49:0/1386155261 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555b69284ff0 msgr2=0x555b692923e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:07:35.937 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.930+0000 7f718f7fe640 1 --2- 172.21.15.49:0/1386155261 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555b69284ff0 0x555b692923e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:35.937 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.930+0000 7f718f7fe640 1 -- 172.21.15.49:0/1386155261 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555b69295f30 con 0x555b692859e0 2024-03-18T23:07:35.937 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.930+0000 7f718effd640 1 --2- 172.21.15.49:0/1386155261 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555b69284ff0 0x555b692923e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:07:35.937 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.930+0000 7f718f7fe640 1 --2- 172.21.15.49:0/1386155261 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555b692859e0 0x555b69292920 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f718002c210 tx=0x7f7180005be0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=13df877da0f2b8d0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:07:35.937 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.930+0000 7f716effd640 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-03-18T23:07:35.937 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.930+0000 7f716effd640 1 -- 172.21.15.49:0/1386155261 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7180005d50 con 0x555b692859e0 2024-03-18T23:07:35.937 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.930+0000 7f716effd640 1 -- 172.21.15.49:0/1386155261 <== mon.1 v2:172.21.15.175:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f7180005250 con 0x555b692859e0 2024-03-18T23:07:35.937 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.930+0000 7f7198bbd740 1 -- 172.21.15.49:0/1386155261 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f718002cae0 con 0x555b692859e0 2024-03-18T23:07:35.937 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.930+0000 7f716effd640 1 -- 172.21.15.49:0/1386155261 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f7180005450 con 0x555b692859e0 2024-03-18T23:07:35.937 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.930+0000 7f7198bbd740 10 client.24922 fetch_fsmap learned FSMap version 27 2024-03-18T23:07:35.937 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.930+0000 7f7198bbd740 1 -- 172.21.15.49:0/1386155261 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x555b69302300 con 0x555b692859e0 2024-03-18T23:07:35.937 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.931+0000 7f716effd640 1 -- 172.21.15.49:0/1386155261 <== mon.1 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f718002cae0 con 0x555b692859e0 2024-03-18T23:07:35.937 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.931+0000 7f716effd640 1 -- 172.21.15.49:0/1386155261 <== mon.1 v2:172.21.15.175:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f71800349a0 con 0x555b692859e0 2024-03-18T23:07:35.937 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.931+0000 7f7198bbd740 10 client.24922 fetch_fsmap finished waiting for FSMap version 27 2024-03-18T23:07:35.937 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.931+0000 7f7198bbd740 10 client.24922 resolve_mds: resolved name 'i' to daemon mds.i 2024-03-18T23:07:35.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.931+0000 7f7198bbd740 20 client.24922 populate_metadata read hostname 'smithi049' 2024-03-18T23:07:35.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.931+0000 7f7198bbd740 1 --2- 172.21.15.49:0/1386155261 >> [v2:172.21.15.175:6838/3640882084,v1:172.21.15.175:6839/3640882084] conn(0x555b693067e0 0x555b693054f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:35.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.931+0000 7f7198bbd740 4 client.24922 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-18T23:07:35.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.931+0000 7f7198bbd740 1 -- 172.21.15.49:0/1386155261 --> [v2:172.21.15.175:6838/3640882084,v1:172.21.15.175:6839/3640882084] -- command(tid 0: {"prefix": "status"}) -- 0x555b691f1670 con 0x555b693067e0 2024-03-18T23:07:35.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.931+0000 7f718effd640 1 --2- 172.21.15.49:0/1386155261 >> [v2:172.21.15.175:6838/3640882084,v1:172.21.15.175:6839/3640882084] conn(0x555b693067e0 0x555b693054f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:35.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.931+0000 7f716effd640 10 client.24922 ms_handle_connect on v2:172.21.15.175:6838/3640882084 2024-03-18T23:07:35.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.931+0000 7f718effd640 1 --2- 172.21.15.49:0/1386155261 >> [v2:172.21.15.175:6838/3640882084,v1:172.21.15.175:6839/3640882084] conn(0x555b693067e0 0x555b693054f0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=463dfb9c05acddbe server_cookie=c0bdd705dbeb4e48 in_seq=0 out_seq=0 2024-03-18T23:07:35.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.932+0000 7f716effd640 1 -- 172.21.15.49:0/1386155261 <== mds.1 v2:172.21.15.175:6838/3640882084 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x555b691f1670 con 0x555b693067e0 2024-03-18T23:07:35.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.932+0000 7f716effd640 10 client.24922 handle_command_reply: tid=0 2024-03-18T23:07:35.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.932+0000 7f7198bbd740 1 client.24922 shutdown 2024-03-18T23:07:35.939 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.932+0000 7f7198bbd740 1 -- 172.21.15.49:0/1386155261 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555b692859e0 msgr2=0x555b69292920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:35.939 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.932+0000 7f7198bbd740 1 --2- 172.21.15.49:0/1386155261 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555b692859e0 0x555b69292920 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f718002c210 tx=0x7f7180005be0 comp rx=0 tx=0).stop 2024-03-18T23:07:35.939 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.932+0000 7f7198bbd740 1 -- 172.21.15.49:0/1386155261 shutdown_connections 2024-03-18T23:07:35.939 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.932+0000 7f7198bbd740 1 --2- 172.21.15.49:0/1386155261 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555b6928ab50 0x555b69295940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:35.939 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.932+0000 7f7198bbd740 1 --2- 172.21.15.49:0/1386155261 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555b692859e0 0x555b69292920 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:35.939 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.932+0000 7f7198bbd740 1 --2- 172.21.15.49:0/1386155261 >> [v2:172.21.15.175:6838/3640882084,v1:172.21.15.175:6839/3640882084] conn(0x555b693067e0 0x555b693054f0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:35.939 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.932+0000 7f7198bbd740 1 --2- 172.21.15.49:0/1386155261 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555b69284ff0 0x555b692923e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:35.939 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.932+0000 7f716effd640 0 client.24922 ms_handle_reset on v2:172.21.15.175:6838/3640882084 2024-03-18T23:07:35.939 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.932+0000 7f7198bbd740 1 -- 172.21.15.49:0/1386155261 >> 172.21.15.49:0/1386155261 conn(0x555b69280560 msgr2=0x555b69280940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:07:35.939 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.932+0000 7f7198bbd740 1 -- 172.21.15.49:0/1386155261 shutdown_connections 2024-03-18T23:07:35.939 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.932+0000 7f7198bbd740 1 -- 172.21.15.49:0/1386155261 wait complete. 2024-03-18T23:07:35.939 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.932+0000 7f7198bbd740 20 client.24922 trim_cache size 0 max 16384 2024-03-18T23:07:35.940 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-03-18T23:07:35.940 INFO:teuthology.orchestra.run.smithi049.stdout: "cluster_fsid": "c63591ea-e57a-11ee-95c9-87774f69a715", 2024-03-18T23:07:35.940 INFO:teuthology.orchestra.run.smithi049.stdout: "whoami": 1, 2024-03-18T23:07:35.940 INFO:teuthology.orchestra.run.smithi049.stdout: "id": 24463, 2024-03-18T23:07:35.940 INFO:teuthology.orchestra.run.smithi049.stdout: "want_state": "up:active", 2024-03-18T23:07:35.940 INFO:teuthology.orchestra.run.smithi049.stdout: "state": "up:active", 2024-03-18T23:07:35.940 INFO:teuthology.orchestra.run.smithi049.stdout: "fs_name": "cephfs", 2024-03-18T23:07:35.940 INFO:teuthology.orchestra.run.smithi049.stdout: "rank_uptime": 87.692609472000001, 2024-03-18T23:07:35.940 INFO:teuthology.orchestra.run.smithi049.stdout: "mdsmap_epoch": 27, 2024-03-18T23:07:35.940 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_epoch": 88, 2024-03-18T23:07:35.940 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_epoch_barrier": 86, 2024-03-18T23:07:35.940 INFO:teuthology.orchestra.run.smithi049.stdout: "uptime": 103.623285235 2024-03-18T23:07:35.940 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-03-18T23:07:35.941 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.932+0000 7f7195fda640 1 -- 172.21.15.49:0/114545921 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f71780a2800 msgr2=0x7f71780a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:35.941 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.932+0000 7f7195fda640 1 --2- 172.21.15.49:0/114545921 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f71780a2800 0x7f71780a4cc0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f7188009150 tx=0x7f718806d040 comp rx=0 tx=0).stop 2024-03-18T23:07:35.941 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.932+0000 7f7195fda640 1 -- 172.21.15.49:0/114545921 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f71901033f0 msgr2=0x7f7190074150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:35.942 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.932+0000 7f7195fda640 1 --2- 172.21.15.49:0/114545921 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f71901033f0 0x7f7190074150 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f718800eb10 tx=0x7f718800c2d0 comp rx=0 tx=0).stop 2024-03-18T23:07:35.942 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.932+0000 7f7195fda640 1 -- 172.21.15.49:0/114545921 shutdown_connections 2024-03-18T23:07:35.942 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.932+0000 7f7195fda640 1 --2- 172.21.15.49:0/114545921 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f71780a2800 0x7f71780a4cc0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:35.942 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.932+0000 7f7195fda640 1 --2- 172.21.15.49:0/114545921 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f71901033f0 0x7f7190074150 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:35.942 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.932+0000 7f7195fda640 1 --2- 172.21.15.49:0/114545921 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7190102a30 0x7f7190073c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:35.942 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.932+0000 7f7195fda640 1 --2- 172.21.15.49:0/114545921 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7190101840 0x7f71900755d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:35.942 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.932+0000 7f7195fda640 1 -- 172.21.15.49:0/114545921 >> 172.21.15.49:0/114545921 conn(0x7f71900fcdb0 msgr2=0x7f7190105e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:07:35.942 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.933+0000 7f7195fda640 1 -- 172.21.15.49:0/114545921 shutdown_connections 2024-03-18T23:07:35.942 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:35.933+0000 7f7195fda640 1 -- 172.21.15.49:0/114545921 wait complete. 2024-03-18T23:07:35.944 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-03-18T23:07:36.057 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:36.054+0000 7f942ed3b640 1 -- 172.21.15.49:0/1496674178 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f94281022f0 msgr2=0x7f9428108b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:36.057 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:36.054+0000 7f942ed3b640 1 --2- 172.21.15.49:0/1496674178 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f94281022f0 0x7f9428108b80 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f9418009900 tx=0x7f941802bc30 comp rx=0 tx=0).stop 2024-03-18T23:07:36.057 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:36.054+0000 7f942ed3b640 1 -- 172.21.15.49:0/1496674178 shutdown_connections 2024-03-18T23:07:36.057 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:36.054+0000 7f942ed3b640 1 --2- 172.21.15.49:0/1496674178 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f94281022f0 0x7f9428108b80 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:36.057 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:36.054+0000 7f942ed3b640 1 --2- 172.21.15.49:0/1496674178 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9428101930 0x7f9428101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:36.057 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:36.054+0000 7f942ed3b640 1 --2- 172.21.15.49:0/1496674178 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9428100740 0x7f9428100b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:36.057 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:36.054+0000 7f942ed3b640 1 -- 172.21.15.49:0/1496674178 >> 172.21.15.49:0/1496674178 conn(0x7f94280fbcd0 msgr2=0x7f94280fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:07:36.057 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:36.054+0000 7f942ed3b640 1 -- 172.21.15.49:0/1496674178 shutdown_connections 2024-03-18T23:07:36.057 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:36.054+0000 7f942ed3b640 1 -- 172.21.15.49:0/1496674178 wait complete. 2024-03-18T23:07:36.057 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:36.054+0000 7f942ed3b640 1 Processor -- start 2024-03-18T23:07:36.057 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:36.054+0000 7f942ed3b640 1 -- start start 2024-03-18T23:07:36.057 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:36.054+0000 7f942ed3b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9428100740 0x7f94281046d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:36.058 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:36.054+0000 7f942ed3b640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9428101930 0x7f9428102d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:36.058 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:36.054+0000 7f942ed3b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f94281022f0 0x7f9428103250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:36.058 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:36.054+0000 7f942ed3b640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f942810c3c0 con 0x7f94281022f0 2024-03-18T23:07:36.058 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:36.054+0000 7f942ed3b640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f942810c240 con 0x7f9428100740 2024-03-18T23:07:36.058 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:36.054+0000 7f942ed3b640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f942810c540 con 0x7f9428101930 2024-03-18T23:07:36.058 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:36.055+0000 7f942d2d8640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f94281022f0 0x7f9428103250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:36.058 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:36.055+0000 7f942cad7640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9428100740 0x7f94281046d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:36.058 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:36.055+0000 7f942cad7640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9428100740 0x7f94281046d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.49:49418/0 (socket says 172.21.15.49:49418) 2024-03-18T23:07:36.058 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:36.055+0000 7f942d2d8640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f94281022f0 0x7f9428103250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:54092/0 (socket says 172.21.15.49:54092) 2024-03-18T23:07:36.058 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:36.055+0000 7f942cad7640 1 -- 172.21.15.49:0/4198426275 learned_addr learned my addr 172.21.15.49:0/4198426275 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:07:36.058 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:36.055+0000 7f9427fff640 1 --2- 172.21.15.49:0/4198426275 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9428101930 0x7f9428102d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:36.058 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:36.055+0000 7f942d2d8640 1 -- 172.21.15.49:0/4198426275 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9428101930 msgr2=0x7f9428102d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:36.058 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:36.055+0000 7f942d2d8640 1 --2- 172.21.15.49:0/4198426275 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9428101930 0x7f9428102d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:36.059 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:36.055+0000 7f942d2d8640 1 -- 172.21.15.49:0/4198426275 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9428100740 msgr2=0x7f94281046d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:36.059 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:36.055+0000 7f942d2d8640 1 --2- 172.21.15.49:0/4198426275 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9428100740 0x7f94281046d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:36.059 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:36.055+0000 7f942d2d8640 1 -- 172.21.15.49:0/4198426275 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9428103ae0 con 0x7f94281022f0 2024-03-18T23:07:36.059 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:36.055+0000 7f942cad7640 1 --2- 172.21.15.49:0/4198426275 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9428100740 0x7f94281046d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:07:36.059 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:36.055+0000 7f942d2d8640 1 --2- 172.21.15.49:0/4198426275 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f94281022f0 0x7f9428103250 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f941802c170 tx=0x7f941802c6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:07:36.059 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:36.055+0000 7f9425ffb640 1 -- 172.21.15.49:0/4198426275 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9418041070 con 0x7f94281022f0 2024-03-18T23:07:36.059 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:36.055+0000 7f9425ffb640 1 -- 172.21.15.49:0/4198426275 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9418004450 con 0x7f94281022f0 2024-03-18T23:07:36.059 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:36.055+0000 7f942ed3b640 1 -- 172.21.15.49:0/4198426275 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94281a1be0 con 0x7f94281022f0 2024-03-18T23:07:36.059 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:36.055+0000 7f9425ffb640 1 -- 172.21.15.49:0/4198426275 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9418034440 con 0x7f94281022f0 2024-03-18T23:07:36.059 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:36.055+0000 7f942ed3b640 1 -- 172.21.15.49:0/4198426275 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94281a2100 con 0x7f94281022f0 2024-03-18T23:07:36.066 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:36.058+0000 7f9425ffb640 1 -- 172.21.15.49:0/4198426275 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7f9418044070 con 0x7f94281022f0 2024-03-18T23:07:36.066 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:36.058+0000 7f9431a04740 1 Processor -- start 2024-03-18T23:07:36.066 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:36.058+0000 7f9431a04740 1 -- start start 2024-03-18T23:07:36.066 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:36.059+0000 7f9431a04740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56539545bff0 0x56539545c410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:36.066 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:36.059+0000 7f9431a04740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56539545c9e0 0x5653954614e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:36.066 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:36.059+0000 7f9431a04740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x565395461b50 0x565395463fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:36.066 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:36.059+0000 7f9431a04740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5653953c0de0 con 0x565395461b50 2024-03-18T23:07:36.066 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:36.059+0000 7f9431a04740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x565395369c80 con 0x56539545bff0 2024-03-18T23:07:36.066 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:36.059+0000 7f9431a04740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5653953bce80 con 0x56539545c9e0 2024-03-18T23:07:36.066 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:36.059+0000 7f9427fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x565395461b50 0x565395463fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:36.067 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:36.059+0000 7f942cad7640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56539545c9e0 0x5653954614e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:36.067 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:36.059+0000 7f942cad7640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56539545c9e0 0x5653954614e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.49:45028/0 (socket says 172.21.15.49:45028) 2024-03-18T23:07:36.067 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:36.059+0000 7f9427fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x565395461b50 0x565395463fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:54096/0 (socket says 172.21.15.49:54096) 2024-03-18T23:07:36.067 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:36.059+0000 7f942cad7640 1 -- 172.21.15.49:0/1200195919 learned_addr learned my addr 172.21.15.49:0/1200195919 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:07:36.067 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:36.059+0000 7f9427fff640 1 --2- 172.21.15.49:0/1200195919 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56539545bff0 0x56539545c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:36.067 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:36.059+0000 7f9425ffb640 1 --2- 172.21.15.49:0/4198426275 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f94140a2800 0x7f94140a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:36.067 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:36.059+0000 7f942d2d8640 1 --2- 172.21.15.49:0/4198426275 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f94140a2800 0x7f94140a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:36.067 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:36.059+0000 7f9427fff640 1 -- 172.21.15.49:0/1200195919 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56539545c9e0 msgr2=0x5653954614e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:07:36.067 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:36.059+0000 7f9427fff640 1 --2- 172.21.15.49:0/1200195919 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56539545c9e0 0x5653954614e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:36.067 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:36.059+0000 7f9427fff640 1 -- 172.21.15.49:0/1200195919 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56539545bff0 msgr2=0x56539545c410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:07:36.067 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:36.059+0000 7f9427fff640 1 --2- 172.21.15.49:0/1200195919 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56539545bff0 0x56539545c410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:36.067 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:36.059+0000 7f9427fff640 1 -- 172.21.15.49:0/1200195919 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56539537bfb0 con 0x565395461b50 2024-03-18T23:07:36.067 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:36.059+0000 7f942cad7640 1 --2- 172.21.15.49:0/1200195919 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56539545c9e0 0x5653954614e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:07:36.067 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:36.059+0000 7f9425ffb640 1 -- 172.21.15.49:0/4198426275 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f94180f05e0 con 0x7f94281022f0 2024-03-18T23:07:36.068 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:36.059+0000 7f9427fff640 1 --2- 172.21.15.49:0/1200195919 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x565395461b50 0x565395463fe0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f9410009fd0 tx=0x7f94100096c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9a83198fc0310fa server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:07:36.068 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:36.059+0000 7f942d2d8640 1 --2- 172.21.15.49:0/4198426275 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f94140a2800 0x7f94140a4cc0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f94180bcd40 tx=0x7f941808fd70 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:07:36.068 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:36.059+0000 7f941effd640 1 -- 172.21.15.49:0/1200195919 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9410004050 con 0x565395461b50 2024-03-18T23:07:36.068 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:36.060+0000 7f941effd640 1 -- 172.21.15.49:0/1200195919 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f94100041f0 con 0x565395461b50 2024-03-18T23:07:36.068 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:36.060+0000 7f941effd640 1 -- 172.21.15.49:0/1200195919 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f941001e690 con 0x565395461b50 2024-03-18T23:07:36.068 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:36.060+0000 7f9431a04740 1 -- 172.21.15.49:0/1200195919 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x565395461b50 msgr2=0x565395463fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:36.068 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:36.060+0000 7f9431a04740 1 --2- 172.21.15.49:0/1200195919 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x565395461b50 0x565395463fe0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f9410009fd0 tx=0x7f94100096c0 comp rx=0 tx=0).stop 2024-03-18T23:07:36.068 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:36.060+0000 7f9431a04740 1 -- 172.21.15.49:0/1200195919 shutdown_connections 2024-03-18T23:07:36.068 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:36.060+0000 7f9431a04740 1 --2- 172.21.15.49:0/1200195919 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x565395461b50 0x565395463fe0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:36.068 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:36.060+0000 7f9431a04740 1 --2- 172.21.15.49:0/1200195919 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56539545c9e0 0x5653954614e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:36.068 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:36.060+0000 7f9431a04740 1 --2- 172.21.15.49:0/1200195919 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56539545bff0 0x56539545c410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:36.068 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:36.060+0000 7f9431a04740 1 -- 172.21.15.49:0/1200195919 >> 172.21.15.49:0/1200195919 conn(0x565395457560 msgr2=0x565395457960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:07:36.068 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:36.060+0000 7f9431a04740 1 -- 172.21.15.49:0/1200195919 shutdown_connections 2024-03-18T23:07:36.068 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:36.060+0000 7f9431a04740 1 -- 172.21.15.49:0/1200195919 wait complete. 2024-03-18T23:07:36.069 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:36.061+0000 7f9431a04740 1 -- start start 2024-03-18T23:07:36.069 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:36.061+0000 7f9431a04740 1 Processor -- start 2024-03-18T23:07:36.069 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:36.061+0000 7f9431a04740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56539545bff0 0x565395466530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:36.069 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:36.061+0000 7f9431a04740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56539545c9e0 0x565395466a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:36.069 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:36.061+0000 7f9431a04740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x565395461b50 0x56539546dab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:36.069 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:36.061+0000 7f9431a04740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x565395464aa0 con 0x56539545bff0 2024-03-18T23:07:36.069 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:36.061+0000 7f9431a04740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x56539537cfc0 con 0x565395461b50 2024-03-18T23:07:36.069 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:36.061+0000 7f9431a04740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x565395464c20 con 0x56539545c9e0 2024-03-18T23:07:36.069 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:36.061+0000 7f9431a04740 10 client.0 _ll_register_callbacks cb 0x5653953b8a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T23:07:36.069 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:36.061+0000 7f9427fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56539545bff0 0x565395466530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:36.069 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:36.061+0000 7f9427fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56539545bff0 0x565395466530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:54110/0 (socket says 172.21.15.49:54110) 2024-03-18T23:07:36.069 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:36.061+0000 7f9427fff640 1 -- 172.21.15.49:0/2733126288 learned_addr learned my addr 172.21.15.49:0/2733126288 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:07:36.069 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:36.061+0000 7f942cad7640 1 --2- 172.21.15.49:0/2733126288 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56539545c9e0 0x565395466a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:36.069 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:36.061+0000 7f9427fff640 1 --2- 172.21.15.49:0/2733126288 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x565395461b50 0x56539546dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:36.070 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:36.061+0000 7f9427fff640 1 -- 172.21.15.49:0/2733126288 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56539545c9e0 msgr2=0x565395466a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:07:36.070 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:36.061+0000 7f9427fff640 1 --2- 172.21.15.49:0/2733126288 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56539545c9e0 0x565395466a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:36.070 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:36.061+0000 7f9427fff640 1 -- 172.21.15.49:0/2733126288 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x565395461b50 msgr2=0x56539546dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:07:36.070 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:36.061+0000 7f9427fff640 1 --2- 172.21.15.49:0/2733126288 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x565395461b50 0x56539546dab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:36.070 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:36.061+0000 7f9427fff640 1 -- 172.21.15.49:0/2733126288 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5653950d52d0 con 0x56539545bff0 2024-03-18T23:07:36.070 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:36.062+0000 7f9427fff640 1 --2- 172.21.15.49:0/2733126288 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56539545bff0 0x565395466530 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f9410005bc0 tx=0x7f9410011550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8b841fa8f0a02c2 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:07:36.070 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:36.062+0000 7f9407fff640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-18T23:07:36.070 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:36.062+0000 7f9407fff640 1 -- 172.21.15.49:0/2733126288 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f941001f3f0 con 0x56539545bff0 2024-03-18T23:07:36.070 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:36.062+0000 7f9407fff640 1 -- 172.21.15.49:0/2733126288 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f941001f590 con 0x56539545bff0 2024-03-18T23:07:36.070 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:36.062+0000 7f9431a04740 1 -- 172.21.15.49:0/2733126288 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f94100041f0 con 0x56539545bff0 2024-03-18T23:07:36.070 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:36.062+0000 7f9407fff640 1 -- 172.21.15.49:0/2733126288 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f9410005020 con 0x56539545bff0 2024-03-18T23:07:36.070 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:36.062+0000 7f9431a04740 10 client.15123 fetch_fsmap learned FSMap version 27 2024-03-18T23:07:36.070 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:36.062+0000 7f9431a04740 1 -- 172.21.15.49:0/2733126288 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56539519f6b0 con 0x56539545bff0 2024-03-18T23:07:36.070 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:36.062+0000 7f9407fff640 1 -- 172.21.15.49:0/2733126288 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f94100041f0 con 0x56539545bff0 2024-03-18T23:07:36.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:36.062+0000 7f9407fff640 1 -- 172.21.15.49:0/2733126288 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f94100046c0 con 0x56539545bff0 2024-03-18T23:07:36.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:36.062+0000 7f9431a04740 10 client.15123 fetch_fsmap finished waiting for FSMap version 27 2024-03-18T23:07:36.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:36.062+0000 7f9431a04740 10 client.15123 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-18T23:07:36.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:36.062+0000 7f9431a04740 20 client.15123 populate_metadata read hostname 'smithi049' 2024-03-18T23:07:36.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:36.062+0000 7f9431a04740 1 --2- 172.21.15.49:0/2733126288 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x565395473080 0x5653954744f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:36.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:36.062+0000 7f9431a04740 4 client.15123 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T23:07:36.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:36.062+0000 7f9431a04740 1 -- 172.21.15.49:0/2733126288 --> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5653953ce320 con 0x565395473080 2024-03-18T23:07:36.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:36.063+0000 7f942cad7640 1 --2- 172.21.15.49:0/2733126288 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x565395473080 0x5653954744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:36.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:36.063+0000 7f942cad7640 1 --2- 172.21.15.49:0/2733126288 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x565395473080 0x5653954744f0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3ebd513b834bb15a server_cookie=d191bdc928421e8e in_seq=0 out_seq=0 2024-03-18T23:07:36.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:36.063+0000 7f9407fff640 10 client.15123 ms_handle_connect on v2:172.21.15.139:6838/4280128209 2024-03-18T23:07:36.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:36.064+0000 7f9407fff640 1 -- 172.21.15.49:0/2733126288 <== mds.0 v2:172.21.15.139:6838/4280128209 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5653953ce320 con 0x565395473080 2024-03-18T23:07:36.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:36.064+0000 7f9407fff640 10 client.15123 handle_command_reply: tid=0 2024-03-18T23:07:36.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:36.064+0000 7f9431a04740 1 client.15123 shutdown 2024-03-18T23:07:36.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:36.064+0000 7f9431a04740 1 -- 172.21.15.49:0/2733126288 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56539545bff0 msgr2=0x565395466530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:36.072 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:36.064+0000 7f9431a04740 1 --2- 172.21.15.49:0/2733126288 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56539545bff0 0x565395466530 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f9410005bc0 tx=0x7f9410011550 comp rx=0 tx=0).stop 2024-03-18T23:07:36.072 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:36.064+0000 7f9431a04740 1 -- 172.21.15.49:0/2733126288 shutdown_connections 2024-03-18T23:07:36.072 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:36.064+0000 7f9431a04740 1 --2- 172.21.15.49:0/2733126288 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x565395473080 0x5653954744f0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:36.072 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:36.064+0000 7f9431a04740 1 --2- 172.21.15.49:0/2733126288 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x565395461b50 0x56539546dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:36.072 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:36.064+0000 7f9407fff640 0 client.15123 ms_handle_reset on v2:172.21.15.139:6838/4280128209 2024-03-18T23:07:36.072 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:36.064+0000 7f9431a04740 1 --2- 172.21.15.49:0/2733126288 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56539545c9e0 0x565395466a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:36.072 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:36.064+0000 7f9431a04740 1 --2- 172.21.15.49:0/2733126288 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56539545bff0 0x565395466530 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:36.072 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:36.064+0000 7f9431a04740 1 -- 172.21.15.49:0/2733126288 >> 172.21.15.49:0/2733126288 conn(0x565395457560 msgr2=0x5653954567e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:07:36.072 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:36.064+0000 7f9431a04740 1 -- 172.21.15.49:0/2733126288 shutdown_connections 2024-03-18T23:07:36.072 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:36.064+0000 7f9431a04740 1 -- 172.21.15.49:0/2733126288 wait complete. 2024-03-18T23:07:36.072 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:36.064+0000 7f9431a04740 20 client.15123 trim_cache size 0 max 16384 2024-03-18T23:07:36.088 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:36.082+0000 7f9431a04740 1 Processor -- start 2024-03-18T23:07:36.088 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:36.082+0000 7f9431a04740 1 -- start start 2024-03-18T23:07:36.088 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:36.082+0000 7f9431a04740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56539545bff0 0x56539545dee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:36.089 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:36.082+0000 7f9431a04740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56539545c9e0 0x56539546da40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:36.089 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:36.082+0000 7f9431a04740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x565395461b50 0x565395466140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:36.089 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:36.082+0000 7f9431a04740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x565395472830 con 0x565395461b50 2024-03-18T23:07:36.089 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:36.082+0000 7f9431a04740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x565395472f00 con 0x56539545bff0 2024-03-18T23:07:36.089 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:36.082+0000 7f9431a04740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5653954735d0 con 0x56539545c9e0 2024-03-18T23:07:36.089 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:36.083+0000 7f9427fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x565395461b50 0x565395466140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:36.089 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:36.083+0000 7f942cad7640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56539545c9e0 0x56539546da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:36.089 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:36.083+0000 7f9427fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x565395461b50 0x565395466140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:54122/0 (socket says 172.21.15.49:54122) 2024-03-18T23:07:36.089 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:36.083+0000 7f942cad7640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56539545c9e0 0x56539546da40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.49:45052/0 (socket says 172.21.15.49:45052) 2024-03-18T23:07:36.089 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:36.083+0000 7f9427fff640 1 -- 172.21.15.49:0/4054238590 learned_addr learned my addr 172.21.15.49:0/4054238590 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:07:36.089 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:36.083+0000 7f9427fff640 1 --2- 172.21.15.49:0/4054238590 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56539545bff0 0x56539545dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:36.089 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:36.083+0000 7f9427fff640 1 -- 172.21.15.49:0/4054238590 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56539545c9e0 msgr2=0x56539546da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:07:36.089 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:36.083+0000 7f9427fff640 1 --2- 172.21.15.49:0/4054238590 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56539545c9e0 0x56539546da40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:36.089 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:36.083+0000 7f9427fff640 1 -- 172.21.15.49:0/4054238590 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56539545bff0 msgr2=0x56539545dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:07:36.089 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:36.083+0000 7f9427fff640 1 --2- 172.21.15.49:0/4054238590 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56539545bff0 0x56539545dee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:36.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:36.083+0000 7f9427fff640 1 -- 172.21.15.49:0/4054238590 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565395467520 con 0x565395461b50 2024-03-18T23:07:36.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:36.083+0000 7f9427fff640 1 --2- 172.21.15.49:0/4054238590 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x565395461b50 0x565395466140 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f9410009dd0 tx=0x7f9410009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d3613823dde3a server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:07:36.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:36.083+0000 7f941e7fc640 1 -- 172.21.15.49:0/4054238590 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9410003ba0 con 0x565395461b50 2024-03-18T23:07:36.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:36.083+0000 7f941e7fc640 1 -- 172.21.15.49:0/4054238590 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9410003d40 con 0x565395461b50 2024-03-18T23:07:36.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:36.083+0000 7f941e7fc640 1 -- 172.21.15.49:0/4054238590 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9410004860 con 0x565395461b50 2024-03-18T23:07:36.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:36.084+0000 7f9431a04740 1 -- 172.21.15.49:0/4054238590 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x565395461b50 msgr2=0x565395466140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:36.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:36.084+0000 7f9431a04740 1 --2- 172.21.15.49:0/4054238590 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x565395461b50 0x565395466140 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f9410009dd0 tx=0x7f9410009bd0 comp rx=0 tx=0).stop 2024-03-18T23:07:36.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:36.084+0000 7f9431a04740 1 -- 172.21.15.49:0/4054238590 shutdown_connections 2024-03-18T23:07:36.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:36.084+0000 7f9431a04740 1 --2- 172.21.15.49:0/4054238590 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x565395461b50 0x565395466140 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:36.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:36.084+0000 7f9431a04740 1 --2- 172.21.15.49:0/4054238590 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56539545c9e0 0x56539546da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:36.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:36.084+0000 7f9431a04740 1 --2- 172.21.15.49:0/4054238590 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56539545bff0 0x56539545dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:36.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:36.084+0000 7f9431a04740 1 -- 172.21.15.49:0/4054238590 >> 172.21.15.49:0/4054238590 conn(0x565395457560 msgr2=0x565395471fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:07:36.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:36.084+0000 7f9431a04740 1 -- 172.21.15.49:0/4054238590 shutdown_connections 2024-03-18T23:07:36.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:36.084+0000 7f9431a04740 1 -- 172.21.15.49:0/4054238590 wait complete. 2024-03-18T23:07:36.091 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:36.084+0000 7f9431a04740 1 -- start start 2024-03-18T23:07:36.091 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:36.084+0000 7f9431a04740 1 Processor -- start 2024-03-18T23:07:36.091 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:36.085+0000 7f9431a04740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56539545bff0 0x565395469290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:36.091 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:36.085+0000 7f9431a04740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56539545c9e0 0x5653954697d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:36.091 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:36.085+0000 7f9431a04740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x565395461b50 0x56539546c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:36.091 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:36.085+0000 7f9431a04740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x565395474370 con 0x56539545c9e0 2024-03-18T23:07:36.091 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:36.085+0000 7f9431a04740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x565395473ca0 con 0x565395461b50 2024-03-18T23:07:36.091 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:36.085+0000 7f9431a04740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x565395474a40 con 0x56539545bff0 2024-03-18T23:07:36.091 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:36.085+0000 7f9431a04740 10 client.0 _ll_register_callbacks cb 0x7f9410009e00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T23:07:36.091 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:36.085+0000 7f942cad7640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56539545c9e0 0x5653954697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:36.091 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:36.085+0000 7f942cad7640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56539545c9e0 0x5653954697d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:54130/0 (socket says 172.21.15.49:54130) 2024-03-18T23:07:36.091 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:36.085+0000 7f942cad7640 1 -- 172.21.15.49:0/4215085332 learned_addr learned my addr 172.21.15.49:0/4215085332 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:07:36.091 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:36.085+0000 7f9427fff640 1 --2- 172.21.15.49:0/4215085332 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56539545bff0 0x565395469290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:36.091 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:36.085+0000 7f9427fff640 1 --2- 172.21.15.49:0/4215085332 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x565395461b50 0x56539546c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:36.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:36.085+0000 7f942cad7640 1 -- 172.21.15.49:0/4215085332 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56539545bff0 msgr2=0x565395469290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:07:36.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:36.085+0000 7f942cad7640 1 --2- 172.21.15.49:0/4215085332 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56539545bff0 0x565395469290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:36.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:36.085+0000 7f942cad7640 1 -- 172.21.15.49:0/4215085332 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x565395461b50 msgr2=0x56539546c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:07:36.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:36.085+0000 7f942cad7640 1 --2- 172.21.15.49:0/4215085332 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x565395461b50 0x56539546c7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:36.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:36.085+0000 7f942cad7640 1 -- 172.21.15.49:0/4215085332 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56539546cde0 con 0x56539545c9e0 2024-03-18T23:07:36.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:36.085+0000 7f9427fff640 1 --2- 172.21.15.49:0/4215085332 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x565395461b50 0x56539546c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T23:07:36.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:36.085+0000 7f942cad7640 1 --2- 172.21.15.49:0/4215085332 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56539545c9e0 0x5653954697d0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f9420004a80 tx=0x7f9420004850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4782437ad43080db server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:07:36.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:36.085+0000 7f9407fff640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-18T23:07:36.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:36.085+0000 7f9407fff640 1 -- 172.21.15.49:0/4215085332 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9420012bc0 con 0x56539545c9e0 2024-03-18T23:07:36.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:36.085+0000 7f9407fff640 1 -- 172.21.15.49:0/4215085332 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f9420012d60 con 0x56539545c9e0 2024-03-18T23:07:36.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:36.085+0000 7f9431a04740 1 -- 172.21.15.49:0/4215085332 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9410003d40 con 0x56539545c9e0 2024-03-18T23:07:36.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:36.085+0000 7f9407fff640 1 -- 172.21.15.49:0/4215085332 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f942002f050 con 0x56539545c9e0 2024-03-18T23:07:36.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:36.085+0000 7f9431a04740 10 client.15129 fetch_fsmap learned FSMap version 27 2024-03-18T23:07:36.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:36.085+0000 7f9431a04740 1 -- 172.21.15.49:0/4215085332 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5653954d8fe0 con 0x56539545c9e0 2024-03-18T23:07:36.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:36.086+0000 7f9407fff640 1 -- 172.21.15.49:0/4215085332 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9410003d40 con 0x56539545c9e0 2024-03-18T23:07:36.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:36.086+0000 7f9407fff640 1 -- 172.21.15.49:0/4215085332 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f942001d8c0 con 0x56539545c9e0 2024-03-18T23:07:36.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:36.086+0000 7f9431a04740 10 client.15129 fetch_fsmap finished waiting for FSMap version 27 2024-03-18T23:07:36.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:36.086+0000 7f9431a04740 10 client.15129 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-18T23:07:36.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:36.086+0000 7f9431a04740 20 client.15129 populate_metadata read hostname 'smithi049' 2024-03-18T23:07:36.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:36.086+0000 7f9431a04740 1 --2- 172.21.15.49:0/4215085332 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x5653954ddc40 0x5653954dc950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:36.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:36.086+0000 7f9431a04740 4 client.15129 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-18T23:07:36.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:36.086+0000 7f9431a04740 1 -- 172.21.15.49:0/4215085332 --> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] -- command(tid 0: {"prefix": "status"}) -- 0x5653953c8670 con 0x5653954ddc40 2024-03-18T23:07:36.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:36.086+0000 7f9427fff640 1 --2- 172.21.15.49:0/4215085332 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x5653954ddc40 0x5653954dc950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:36.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:36.086+0000 7f9407fff640 10 client.15129 ms_handle_connect on v2:172.21.15.139:6838/4280128209 2024-03-18T23:07:36.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:36.086+0000 7f9427fff640 1 --2- 172.21.15.49:0/4215085332 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x5653954ddc40 0x5653954dc950 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c9ef3147455ef325 server_cookie=87e3927cbbc0646b in_seq=0 out_seq=0 2024-03-18T23:07:36.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:36.087+0000 7f9407fff640 1 -- 172.21.15.49:0/4215085332 <== mds.0 v2:172.21.15.139:6838/4280128209 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x5653953c8670 con 0x5653954ddc40 2024-03-18T23:07:36.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:36.087+0000 7f9407fff640 10 client.15129 handle_command_reply: tid=0 2024-03-18T23:07:36.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:36.087+0000 7f9431a04740 1 client.15129 shutdown 2024-03-18T23:07:36.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:36.087+0000 7f9431a04740 1 -- 172.21.15.49:0/4215085332 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56539545c9e0 msgr2=0x5653954697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:36.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:36.087+0000 7f9431a04740 1 --2- 172.21.15.49:0/4215085332 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56539545c9e0 0x5653954697d0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f9420004a80 tx=0x7f9420004850 comp rx=0 tx=0).stop 2024-03-18T23:07:36.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:36.087+0000 7f9431a04740 1 -- 172.21.15.49:0/4215085332 shutdown_connections 2024-03-18T23:07:36.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:36.087+0000 7f9431a04740 1 --2- 172.21.15.49:0/4215085332 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x5653954ddc40 0x5653954dc950 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:36.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:36.087+0000 7f9431a04740 1 --2- 172.21.15.49:0/4215085332 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x565395461b50 0x56539546c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:36.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:36.087+0000 7f9431a04740 1 --2- 172.21.15.49:0/4215085332 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56539545c9e0 0x5653954697d0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:36.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:36.087+0000 7f9407fff640 0 client.15129 ms_handle_reset on v2:172.21.15.139:6838/4280128209 2024-03-18T23:07:36.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:36.087+0000 7f9431a04740 1 --2- 172.21.15.49:0/4215085332 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56539545bff0 0x565395469290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:36.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:36.087+0000 7f9431a04740 1 -- 172.21.15.49:0/4215085332 >> 172.21.15.49:0/4215085332 conn(0x565395457560 msgr2=0x565395452670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:07:36.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:36.087+0000 7f9431a04740 1 -- 172.21.15.49:0/4215085332 shutdown_connections 2024-03-18T23:07:36.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:36.087+0000 7f9431a04740 1 -- 172.21.15.49:0/4215085332 wait complete. 2024-03-18T23:07:36.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:36.087+0000 7f9431a04740 20 client.15129 trim_cache size 0 max 16384 2024-03-18T23:07:36.095 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-03-18T23:07:36.095 INFO:teuthology.orchestra.run.smithi049.stdout: "cluster_fsid": "c63591ea-e57a-11ee-95c9-87774f69a715", 2024-03-18T23:07:36.095 INFO:teuthology.orchestra.run.smithi049.stdout: "whoami": 0, 2024-03-18T23:07:36.095 INFO:teuthology.orchestra.run.smithi049.stdout: "id": 24488, 2024-03-18T23:07:36.095 INFO:teuthology.orchestra.run.smithi049.stdout: "want_state": "up:active", 2024-03-18T23:07:36.095 INFO:teuthology.orchestra.run.smithi049.stdout: "state": "up:active", 2024-03-18T23:07:36.095 INFO:teuthology.orchestra.run.smithi049.stdout: "fs_name": "cephfs", 2024-03-18T23:07:36.095 INFO:teuthology.orchestra.run.smithi049.stdout: "rank_uptime": 90.872553992999997, 2024-03-18T23:07:36.095 INFO:teuthology.orchestra.run.smithi049.stdout: "mdsmap_epoch": 27, 2024-03-18T23:07:36.095 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_epoch": 88, 2024-03-18T23:07:36.095 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_epoch_barrier": 86, 2024-03-18T23:07:36.095 INFO:teuthology.orchestra.run.smithi049.stdout: "uptime": 95.101621022000003 2024-03-18T23:07:36.096 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-03-18T23:07:36.096 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:36.087+0000 7f942ed3b640 1 -- 172.21.15.49:0/4198426275 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f94140a2800 msgr2=0x7f94140a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:36.096 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:36.087+0000 7f942ed3b640 1 --2- 172.21.15.49:0/4198426275 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f94140a2800 0x7f94140a4cc0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f94180bcd40 tx=0x7f941808fd70 comp rx=0 tx=0).stop 2024-03-18T23:07:36.096 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:36.087+0000 7f942ed3b640 1 -- 172.21.15.49:0/4198426275 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f94281022f0 msgr2=0x7f9428103250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:36.097 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:36.087+0000 7f942ed3b640 1 --2- 172.21.15.49:0/4198426275 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f94281022f0 0x7f9428103250 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f941802c170 tx=0x7f941802c6c0 comp rx=0 tx=0).stop 2024-03-18T23:07:36.097 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:36.087+0000 7f942ed3b640 1 -- 172.21.15.49:0/4198426275 shutdown_connections 2024-03-18T23:07:36.097 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:36.087+0000 7f942ed3b640 1 --2- 172.21.15.49:0/4198426275 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f94140a2800 0x7f94140a4cc0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:36.097 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:36.088+0000 7f942ed3b640 1 --2- 172.21.15.49:0/4198426275 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f94281022f0 0x7f9428103250 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:36.097 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:36.088+0000 7f942ed3b640 1 --2- 172.21.15.49:0/4198426275 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9428101930 0x7f9428102d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:36.097 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:36.088+0000 7f942ed3b640 1 --2- 172.21.15.49:0/4198426275 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9428100740 0x7f94281046d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:36.097 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:36.088+0000 7f942ed3b640 1 -- 172.21.15.49:0/4198426275 >> 172.21.15.49:0/4198426275 conn(0x7f94280fbcd0 msgr2=0x7f94280fd720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:07:36.097 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:36.088+0000 7f942ed3b640 1 -- 172.21.15.49:0/4198426275 shutdown_connections 2024-03-18T23:07:36.097 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:36.088+0000 7f942ed3b640 1 -- 172.21.15.49:0/4198426275 wait complete. 2024-03-18T23:07:36.099 INFO:tasks.cephadm:Setup complete, yielding 2024-03-18T23:07:36.100 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-03-18T23:07:36.109 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi049.front.sepia.ceph.com 2024-03-18T23:07:36.109 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c63591ea-e57a-11ee-95c9-87774f69a715 -- bash -c 'ceph orch status' 2024-03-18T23:07:36.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:07:36 smithi139 ceph-mon[67282]: from='client.? 172.21.15.49:0/927676930' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T23:07:36.359 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T23:07:36.379 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:07:36 smithi049 ceph-mon[62991]: from='client.? 172.21.15.49:0/927676930' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T23:07:36.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:07:36 smithi175 ceph-mon[67352]: from='client.? 172.21.15.49:0/927676930' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T23:07:37.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:07:37 smithi139 ceph-mon[67282]: pgmap v336: 97 pgs: 97 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 426 B/s wr, 11 op/s 2024-03-18T23:07:37.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:07:37 smithi049 ceph-mon[62991]: pgmap v336: 97 pgs: 97 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 426 B/s wr, 11 op/s 2024-03-18T23:07:37.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:07:37 smithi175 ceph-mon[67352]: pgmap v336: 97 pgs: 97 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 426 B/s wr, 11 op/s 2024-03-18T23:07:38.051 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:38.050+0000 7fd9105cf700 1 -- 172.21.15.49:0/3133329360 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd9081056f0 msgr2=0x7fd908107ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:38.051 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:38.050+0000 7fd9105cf700 1 --2- 172.21.15.49:0/3133329360 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd9081056f0 0x7fd908107ae0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fd8f8009990 tx=0x7fd8f8009ca0 comp rx=0 tx=0).stop 2024-03-18T23:07:38.051 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:38.050+0000 7fd9105cf700 1 -- 172.21.15.49:0/3133329360 shutdown_connections 2024-03-18T23:07:38.051 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:38.050+0000 7fd9105cf700 1 --2- 172.21.15.49:0/3133329360 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd908108020 0x7fd90810a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:38.052 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:38.050+0000 7fd9105cf700 1 --2- 172.21.15.49:0/3133329360 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd908102dc0 0x7fd9081051b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:38.052 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:38.050+0000 7fd9105cf700 1 --2- 172.21.15.49:0/3133329360 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd9081056f0 0x7fd908107ae0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:38.052 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:38.050+0000 7fd9105cf700 1 -- 172.21.15.49:0/3133329360 >> 172.21.15.49:0/3133329360 conn(0x7fd9080fc7d0 msgr2=0x7fd9080fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:07:38.052 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:38.050+0000 7fd9105cf700 1 -- 172.21.15.49:0/3133329360 shutdown_connections 2024-03-18T23:07:38.052 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:38.050+0000 7fd9105cf700 1 -- 172.21.15.49:0/3133329360 wait complete. 2024-03-18T23:07:38.054 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:38.051+0000 7fd9105cf700 1 Processor -- start 2024-03-18T23:07:38.054 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:38.051+0000 7fd9105cf700 1 -- start start 2024-03-18T23:07:38.054 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:38.051+0000 7fd9105cf700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd908102dc0 0x7fd908198560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:38.054 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:38.051+0000 7fd9105cf700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd9081056f0 0x7fd908198aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:38.054 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:38.051+0000 7fd9105cf700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd908108020 0x7fd90819fb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:38.054 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:38.051+0000 7fd9105cf700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fd90810df40 con 0x7fd908102dc0 2024-03-18T23:07:38.054 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:38.051+0000 7fd9105cf700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fd90810ddc0 con 0x7fd9081056f0 2024-03-18T23:07:38.054 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:38.051+0000 7fd9105cf700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd90810e0c0 con 0x7fd908108020 2024-03-18T23:07:38.054 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:38.051+0000 7fd90e36b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd908102dc0 0x7fd908198560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:38.054 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:38.051+0000 7fd90e36b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd908102dc0 0x7fd908198560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:54160/0 (socket says 172.21.15.49:54160) 2024-03-18T23:07:38.055 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:38.051+0000 7fd90e36b700 1 -- 172.21.15.49:0/2302801108 learned_addr learned my addr 172.21.15.49:0/2302801108 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:07:38.055 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:38.051+0000 7fd90eb6c700 1 --2- 172.21.15.49:0/2302801108 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd908108020 0x7fd90819fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:38.055 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:38.051+0000 7fd90db6a700 1 --2- 172.21.15.49:0/2302801108 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd9081056f0 0x7fd908198aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:38.055 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:38.051+0000 7fd90e36b700 1 -- 172.21.15.49:0/2302801108 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd908108020 msgr2=0x7fd90819fb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:38.055 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:38.051+0000 7fd90e36b700 1 --2- 172.21.15.49:0/2302801108 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd908108020 0x7fd90819fb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:38.055 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:38.051+0000 7fd90e36b700 1 -- 172.21.15.49:0/2302801108 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd9081056f0 msgr2=0x7fd908198aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:38.055 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:38.051+0000 7fd90e36b700 1 --2- 172.21.15.49:0/2302801108 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd9081056f0 0x7fd908198aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:38.055 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:38.051+0000 7fd90e36b700 1 -- 172.21.15.49:0/2302801108 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9081a01c0 con 0x7fd908102dc0 2024-03-18T23:07:38.055 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:38.052+0000 7fd90e36b700 1 --2- 172.21.15.49:0/2302801108 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd908102dc0 0x7fd908198560 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fd90400e9d0 tx=0x7fd90400ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:07:38.055 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:38.052+0000 7fd8ff7fe700 1 -- 172.21.15.49:0/2302801108 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd90400cbd0 con 0x7fd908102dc0 2024-03-18T23:07:38.055 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:38.052+0000 7fd8ff7fe700 1 -- 172.21.15.49:0/2302801108 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd90400ee70 con 0x7fd908102dc0 2024-03-18T23:07:38.055 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:38.052+0000 7fd8ff7fe700 1 -- 172.21.15.49:0/2302801108 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd904018660 con 0x7fd908102dc0 2024-03-18T23:07:38.055 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:38.052+0000 7fd9105cf700 1 -- 172.21.15.49:0/2302801108 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9081a04b0 con 0x7fd908102dc0 2024-03-18T23:07:38.055 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:38.052+0000 7fd9105cf700 1 -- 172.21.15.49:0/2302801108 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9081a09f0 con 0x7fd908102dc0 2024-03-18T23:07:38.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:38.053+0000 7fd9105cf700 1 -- 172.21.15.49:0/2302801108 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd908068fd0 con 0x7fd908102dc0 2024-03-18T23:07:38.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:38.054+0000 7fd8ff7fe700 1 -- 172.21.15.49:0/2302801108 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7fd904010430 con 0x7fd908102dc0 2024-03-18T23:07:38.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:38.055+0000 7fd8ff7fe700 1 --2- 172.21.15.49:0/2302801108 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fd8f40a2910 0x7fd8f40a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:38.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:38.055+0000 7fd90db6a700 1 --2- 172.21.15.49:0/2302801108 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fd8f40a2910 0x7fd8f40a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:38.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:38.055+0000 7fd8ff7fe700 1 -- 172.21.15.49:0/2302801108 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7fd9040cebf0 con 0x7fd908102dc0 2024-03-18T23:07:38.057 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:38.055+0000 7fd90db6a700 1 --2- 172.21.15.49:0/2302801108 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fd8f40a2910 0x7fd8f40a4dd0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fd8f800b2e0 tx=0x7fd8f8005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:07:38.060 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:38.059+0000 7fd8ff7fe700 1 -- 172.21.15.49:0/2302801108 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fd90409bca0 con 0x7fd908102dc0 2024-03-18T23:07:38.285 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:38.283+0000 7fd9105cf700 1 -- 172.21.15.49:0/2302801108 --> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7fd908062750 con 0x7fd8f40a2910 2024-03-18T23:07:38.286 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:38.284+0000 7fd8ff7fe700 1 -- 172.21.15.49:0/2302801108 <== mgr.14150 v2:172.21.15.49:6800/2083805211 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7fd908062750 con 0x7fd8f40a2910 2024-03-18T23:07:38.286 INFO:teuthology.orchestra.run.smithi049.stdout:Backend: cephadm 2024-03-18T23:07:38.286 INFO:teuthology.orchestra.run.smithi049.stdout:Available: Yes 2024-03-18T23:07:38.286 INFO:teuthology.orchestra.run.smithi049.stdout:Paused: No 2024-03-18T23:07:38.288 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:38.287+0000 7fd9105cf700 1 -- 172.21.15.49:0/2302801108 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fd8f40a2910 msgr2=0x7fd8f40a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:38.288 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:38.287+0000 7fd9105cf700 1 --2- 172.21.15.49:0/2302801108 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fd8f40a2910 0x7fd8f40a4dd0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fd8f800b2e0 tx=0x7fd8f8005fd0 comp rx=0 tx=0).stop 2024-03-18T23:07:38.288 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:38.287+0000 7fd9105cf700 1 -- 172.21.15.49:0/2302801108 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd908102dc0 msgr2=0x7fd908198560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:38.288 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:38.287+0000 7fd9105cf700 1 --2- 172.21.15.49:0/2302801108 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd908102dc0 0x7fd908198560 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fd90400e9d0 tx=0x7fd90400ece0 comp rx=0 tx=0).stop 2024-03-18T23:07:38.288 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:38.287+0000 7fd9105cf700 1 -- 172.21.15.49:0/2302801108 shutdown_connections 2024-03-18T23:07:38.288 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:38.287+0000 7fd9105cf700 1 --2- 172.21.15.49:0/2302801108 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fd8f40a2910 0x7fd8f40a4dd0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:38.288 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:38.287+0000 7fd9105cf700 1 --2- 172.21.15.49:0/2302801108 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd908108020 0x7fd90819fb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:38.288 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:38.287+0000 7fd9105cf700 1 --2- 172.21.15.49:0/2302801108 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd908102dc0 0x7fd908198560 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:38.288 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:38.287+0000 7fd9105cf700 1 --2- 172.21.15.49:0/2302801108 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd9081056f0 0x7fd908198aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:38.289 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:38.287+0000 7fd9105cf700 1 -- 172.21.15.49:0/2302801108 >> 172.21.15.49:0/2302801108 conn(0x7fd9080fc7d0 msgr2=0x7fd908103800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:07:38.289 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:38.287+0000 7fd9105cf700 1 -- 172.21.15.49:0/2302801108 shutdown_connections 2024-03-18T23:07:38.289 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:38.287+0000 7fd9105cf700 1 -- 172.21.15.49:0/2302801108 wait complete. 2024-03-18T23:07:38.958 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c63591ea-e57a-11ee-95c9-87774f69a715 -- bash -c 'ceph orch ps' 2024-03-18T23:07:39.215 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T23:07:39.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:07:39 smithi139 ceph-mon[67282]: pgmap v337: 97 pgs: 97 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 426 B/s wr, 10 op/s 2024-03-18T23:07:39.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:07:39 smithi139 ceph-mon[67282]: from='client.15135 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T23:07:39.344 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:07:39 smithi049 ceph-mon[62991]: pgmap v337: 97 pgs: 97 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 426 B/s wr, 10 op/s 2024-03-18T23:07:39.344 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:07:39 smithi049 ceph-mon[62991]: from='client.15135 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T23:07:39.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:07:39 smithi175 ceph-mon[67352]: pgmap v337: 97 pgs: 97 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 426 B/s wr, 10 op/s 2024-03-18T23:07:39.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:07:39 smithi175 ceph-mon[67352]: from='client.15135 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T23:07:41.033 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:41.007+0000 7f18d36ab700 1 -- 172.21.15.49:0/1275194782 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f18cc105700 msgr2=0x7f18cc107af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:41.034 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:41.007+0000 7f18d36ab700 1 --2- 172.21.15.49:0/1275194782 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f18cc105700 0x7f18cc107af0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f18c8009990 tx=0x7f18c8009ca0 comp rx=0 tx=0).stop 2024-03-18T23:07:41.034 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:41.007+0000 7f18d36ab700 1 -- 172.21.15.49:0/1275194782 shutdown_connections 2024-03-18T23:07:41.034 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:41.007+0000 7f18d36ab700 1 --2- 172.21.15.49:0/1275194782 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f18cc108030 0x7f18cc10a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:41.034 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:41.007+0000 7f18d36ab700 1 --2- 172.21.15.49:0/1275194782 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f18cc102dd0 0x7f18cc1051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:41.034 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:41.007+0000 7f18d36ab700 1 --2- 172.21.15.49:0/1275194782 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f18cc105700 0x7f18cc107af0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:41.034 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:41.007+0000 7f18d36ab700 1 -- 172.21.15.49:0/1275194782 >> 172.21.15.49:0/1275194782 conn(0x7f18cc0fc7e0 msgr2=0x7f18cc0fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:07:41.034 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:41.007+0000 7f18d36ab700 1 -- 172.21.15.49:0/1275194782 shutdown_connections 2024-03-18T23:07:41.034 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:41.007+0000 7f18d36ab700 1 -- 172.21.15.49:0/1275194782 wait complete. 2024-03-18T23:07:41.034 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:41.008+0000 7f18d36ab700 1 Processor -- start 2024-03-18T23:07:41.034 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:41.008+0000 7f18d36ab700 1 -- start start 2024-03-18T23:07:41.034 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:41.008+0000 7f18d36ab700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f18cc102dd0 0x7f18cc1985f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:41.034 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:41.008+0000 7f18d36ab700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f18cc105700 0x7f18cc198b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:41.035 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:41.008+0000 7f18d36ab700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f18cc108030 0x7f18cc19fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:41.035 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:41.008+0000 7f18d36ab700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f18cc10dfa0 con 0x7f18cc108030 2024-03-18T23:07:41.035 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:41.008+0000 7f18d36ab700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f18cc10de20 con 0x7f18cc105700 2024-03-18T23:07:41.035 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:41.008+0000 7f18d36ab700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f18cc10e120 con 0x7f18cc102dd0 2024-03-18T23:07:41.035 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:41.008+0000 7f18d1c48700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f18cc108030 0x7f18cc19fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:41.035 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:41.008+0000 7f18d1447700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f18cc102dd0 0x7f18cc1985f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:41.035 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:41.008+0000 7f18d1c48700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f18cc108030 0x7f18cc19fbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:54180/0 (socket says 172.21.15.49:54180) 2024-03-18T23:07:41.035 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:41.008+0000 7f18d1c48700 1 -- 172.21.15.49:0/2949887136 learned_addr learned my addr 172.21.15.49:0/2949887136 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:07:41.037 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:41.008+0000 7f18d0c46700 1 --2- 172.21.15.49:0/2949887136 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f18cc105700 0x7f18cc198b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:41.037 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:41.009+0000 7f18d1c48700 1 -- 172.21.15.49:0/2949887136 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f18cc102dd0 msgr2=0x7f18cc1985f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:41.037 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:41.009+0000 7f18d1c48700 1 --2- 172.21.15.49:0/2949887136 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f18cc102dd0 0x7f18cc1985f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:41.037 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:41.009+0000 7f18d1c48700 1 -- 172.21.15.49:0/2949887136 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f18cc105700 msgr2=0x7f18cc198b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:41.037 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:41.009+0000 7f18d1c48700 1 --2- 172.21.15.49:0/2949887136 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f18cc105700 0x7f18cc198b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:41.037 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:41.009+0000 7f18d1c48700 1 -- 172.21.15.49:0/2949887136 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18cc1a0250 con 0x7f18cc108030 2024-03-18T23:07:41.038 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:41.009+0000 7f18d1c48700 1 --2- 172.21.15.49:0/2949887136 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f18cc108030 0x7f18cc19fbb0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f18c400efc0 tx=0x7f18c400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:07:41.038 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:41.009+0000 7f18c27fc700 1 -- 172.21.15.49:0/2949887136 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18c40092e0 con 0x7f18cc108030 2024-03-18T23:07:41.038 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:41.009+0000 7f18d36ab700 1 -- 172.21.15.49:0/2949887136 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18cc1a04e0 con 0x7f18cc108030 2024-03-18T23:07:41.038 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:41.009+0000 7f18c27fc700 1 -- 172.21.15.49:0/2949887136 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f18c4004bb0 con 0x7f18cc108030 2024-03-18T23:07:41.038 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:41.009+0000 7f18c27fc700 1 -- 172.21.15.49:0/2949887136 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18c4004280 con 0x7f18cc108030 2024-03-18T23:07:41.038 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:41.009+0000 7f18d36ab700 1 -- 172.21.15.49:0/2949887136 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18cc1a0a70 con 0x7f18cc108030 2024-03-18T23:07:41.038 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:41.010+0000 7f18d36ab700 1 -- 172.21.15.49:0/2949887136 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18cc068fd0 con 0x7f18cc108030 2024-03-18T23:07:41.038 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:41.011+0000 7f18c27fc700 1 -- 172.21.15.49:0/2949887136 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7f18c401b030 con 0x7f18cc108030 2024-03-18T23:07:41.038 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:41.012+0000 7f18c27fc700 1 --2- 172.21.15.49:0/2949887136 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f18b80a2890 0x7f18b80a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:41.038 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:41.012+0000 7f18d1447700 1 --2- 172.21.15.49:0/2949887136 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f18b80a2890 0x7f18b80a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:41.038 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:41.012+0000 7f18c27fc700 1 -- 172.21.15.49:0/2949887136 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f18c40ce4a0 con 0x7f18cc108030 2024-03-18T23:07:41.038 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:41.012+0000 7f18d1447700 1 --2- 172.21.15.49:0/2949887136 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f18b80a2890 0x7f18b80a4d50 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f18bc007c60 tx=0x7f18bc0073d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:07:41.038 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:41.016+0000 7f18c27fc700 1 -- 172.21.15.49:0/2949887136 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f18c4022020 con 0x7f18cc108030 2024-03-18T23:07:41.234 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:41.233+0000 7f18d36ab700 1 -- 172.21.15.49:0/2949887136 --> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f18cc062750 con 0x7f18b80a2890 2024-03-18T23:07:41.247 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:41.246+0000 7f18c27fc700 1 -- 172.21.15.49:0/2949887136 <== mgr.14150 v2:172.21.15.49:6800/2083805211 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4185 (secure 0 0 0) 0x7f18cc062750 con 0x7f18b80a2890 2024-03-18T23:07:41.247 INFO:teuthology.orchestra.run.smithi049.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-18T23:07:41.247 INFO:teuthology.orchestra.run.smithi049.stdout:mds.a smithi049 running (103s) 17s ago 103s 17.3M - 19.0.0-2206-gb6ae2491 2312c8e42d00 e54358a09ccc 2024-03-18T23:07:41.248 INFO:teuthology.orchestra.run.smithi049.stdout:mds.b smithi139 running (100s) 34s ago 100s 25.8M - 19.0.0-2206-gb6ae2491 2312c8e42d00 4049f303e6e8 2024-03-18T23:07:41.248 INFO:teuthology.orchestra.run.smithi049.stdout:mds.c smithi175 running (115s) 34s ago 114s 19.3M - 19.0.0-2206-gb6ae2491 2312c8e42d00 1bbba6b30980 2024-03-18T23:07:41.248 INFO:teuthology.orchestra.run.smithi049.stdout:mds.d smithi049 running (2m) 17s ago 2m 12.9M - 19.0.0-2206-gb6ae2491 2312c8e42d00 004eafb644c5 2024-03-18T23:07:41.248 INFO:teuthology.orchestra.run.smithi049.stdout:mds.e smithi139 running (2m) 34s ago 2m 19.1M - 19.0.0-2206-gb6ae2491 2312c8e42d00 0431e396f283 2024-03-18T23:07:41.248 INFO:teuthology.orchestra.run.smithi049.stdout:mds.f smithi175 running (117s) 34s ago 117s 16.9M - 19.0.0-2206-gb6ae2491 2312c8e42d00 66fdf9cee264 2024-03-18T23:07:41.248 INFO:teuthology.orchestra.run.smithi049.stdout:mds.g smithi049 running (106s) 17s ago 105s 18.7M - 19.0.0-2206-gb6ae2491 2312c8e42d00 6536069ee407 2024-03-18T23:07:41.248 INFO:teuthology.orchestra.run.smithi049.stdout:mds.h smithi139 running (112s) 34s ago 111s 19.3M - 19.0.0-2206-gb6ae2491 2312c8e42d00 d7e908d6f032 2024-03-18T23:07:41.248 INFO:teuthology.orchestra.run.smithi049.stdout:mds.i smithi175 running (109s) 34s ago 108s 17.9M - 19.0.0-2206-gb6ae2491 2312c8e42d00 c9b962415920 2024-03-18T23:07:41.248 INFO:teuthology.orchestra.run.smithi049.stdout:mds.j smithi049 running (2m) 17s ago 2m 14.0M - 19.0.0-2206-gb6ae2491 2312c8e42d00 3a80eb2dbc25 2024-03-18T23:07:41.248 INFO:teuthology.orchestra.run.smithi049.stdout:mds.k smithi139 running (2m) 34s ago 2m 19.0M - 19.0.0-2206-gb6ae2491 2312c8e42d00 c4db76d479ee 2024-03-18T23:07:41.248 INFO:teuthology.orchestra.run.smithi049.stdout:mds.l smithi175 running (2m) 34s ago 2m 19.0M - 19.0.0-2206-gb6ae2491 2312c8e42d00 db3bb4b298f0 2024-03-18T23:07:41.248 INFO:teuthology.orchestra.run.smithi049.stdout:mgr.x smithi049 *:9283,8765 running (9m) 17s ago 9m 513M - 19.0.0-2206-gb6ae2491 2312c8e42d00 75469e5b5506 2024-03-18T23:07:41.248 INFO:teuthology.orchestra.run.smithi049.stdout:mgr.y smithi139 *:8443,8765 running (7m) 34s ago 7m 453M - 19.0.0-2206-gb6ae2491 2312c8e42d00 03a453950734 2024-03-18T23:07:41.248 INFO:teuthology.orchestra.run.smithi049.stdout:mgr.z smithi175 *:8443,8765 running (7m) 34s ago 7m 453M - 19.0.0-2206-gb6ae2491 2312c8e42d00 8a53c6563795 2024-03-18T23:07:41.249 INFO:teuthology.orchestra.run.smithi049.stdout:mon.a smithi049 running (9m) 17s ago 9m 52.1M 2048M 19.0.0-2206-gb6ae2491 2312c8e42d00 8521c70f9f18 2024-03-18T23:07:41.249 INFO:teuthology.orchestra.run.smithi049.stdout:mon.b smithi139 running (7m) 34s ago 7m 40.4M 2048M 19.0.0-2206-gb6ae2491 2312c8e42d00 127fbf6c9088 2024-03-18T23:07:41.249 INFO:teuthology.orchestra.run.smithi049.stdout:mon.c smithi175 running (7m) 34s ago 7m 40.0M 2048M 19.0.0-2206-gb6ae2491 2312c8e42d00 6f8dd756d022 2024-03-18T23:07:41.249 INFO:teuthology.orchestra.run.smithi049.stdout:osd.0 smithi049 running (7m) 17s ago 7m 56.1M 4272M 19.0.0-2206-gb6ae2491 2312c8e42d00 4f7da08d94bd 2024-03-18T23:07:41.249 INFO:teuthology.orchestra.run.smithi049.stdout:osd.1 smithi139 running (6m) 34s ago 6m 57.5M 4270M 19.0.0-2206-gb6ae2491 2312c8e42d00 3e2f5d678a0e 2024-03-18T23:07:41.249 INFO:teuthology.orchestra.run.smithi049.stdout:osd.2 smithi175 running (6m) 34s ago 6m 46.9M 4270M 19.0.0-2206-gb6ae2491 2312c8e42d00 17b1cf604deb 2024-03-18T23:07:41.249 INFO:teuthology.orchestra.run.smithi049.stdout:osd.3 smithi049 running (5m) 17s ago 5m 42.9M 4272M 19.0.0-2206-gb6ae2491 2312c8e42d00 4328db4d778b 2024-03-18T23:07:41.249 INFO:teuthology.orchestra.run.smithi049.stdout:osd.4 smithi139 running (5m) 34s ago 5m 52.0M 4270M 19.0.0-2206-gb6ae2491 2312c8e42d00 398fe5f95f4d 2024-03-18T23:07:41.249 INFO:teuthology.orchestra.run.smithi049.stdout:osd.5 smithi175 running (5m) 34s ago 5m 54.7M 4270M 19.0.0-2206-gb6ae2491 2312c8e42d00 35bce1dd2dd6 2024-03-18T23:07:41.249 INFO:teuthology.orchestra.run.smithi049.stdout:osd.6 smithi049 running (4m) 17s ago 4m 52.8M 4272M 19.0.0-2206-gb6ae2491 2312c8e42d00 12795277ad14 2024-03-18T23:07:41.249 INFO:teuthology.orchestra.run.smithi049.stdout:osd.7 smithi139 running (4m) 34s ago 4m 47.0M 4270M 19.0.0-2206-gb6ae2491 2312c8e42d00 e4866443a6ae 2024-03-18T23:07:41.249 INFO:teuthology.orchestra.run.smithi049.stdout:osd.8 smithi175 running (3m) 34s ago 3m 51.8M 4270M 19.0.0-2206-gb6ae2491 2312c8e42d00 61637f12f1c5 2024-03-18T23:07:41.249 INFO:teuthology.orchestra.run.smithi049.stdout:osd.9 smithi049 running (3m) 17s ago 3m 42.5M 4272M 19.0.0-2206-gb6ae2491 2312c8e42d00 0320517bd430 2024-03-18T23:07:41.249 INFO:teuthology.orchestra.run.smithi049.stdout:osd.10 smithi139 running (2m) 34s ago 2m 42.7M 4270M 19.0.0-2206-gb6ae2491 2312c8e42d00 15d5cddd4307 2024-03-18T23:07:41.249 INFO:teuthology.orchestra.run.smithi049.stdout:osd.11 smithi175 running (2m) 34s ago 2m 41.4M 4270M 19.0.0-2206-gb6ae2491 2312c8e42d00 2b7c27ba092f 2024-03-18T23:07:41.250 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:41.248+0000 7f18d36ab700 1 -- 172.21.15.49:0/2949887136 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f18b80a2890 msgr2=0x7f18b80a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:41.250 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:41.248+0000 7f18d36ab700 1 --2- 172.21.15.49:0/2949887136 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f18b80a2890 0x7f18b80a4d50 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f18bc007c60 tx=0x7f18bc0073d0 comp rx=0 tx=0).stop 2024-03-18T23:07:41.250 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:41.248+0000 7f18d36ab700 1 -- 172.21.15.49:0/2949887136 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f18cc108030 msgr2=0x7f18cc19fbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:41.250 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:41.248+0000 7f18d36ab700 1 --2- 172.21.15.49:0/2949887136 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f18cc108030 0x7f18cc19fbb0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f18c400efc0 tx=0x7f18c400c3a0 comp rx=0 tx=0).stop 2024-03-18T23:07:41.250 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:41.249+0000 7f18d36ab700 1 -- 172.21.15.49:0/2949887136 shutdown_connections 2024-03-18T23:07:41.250 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:41.249+0000 7f18d36ab700 1 --2- 172.21.15.49:0/2949887136 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f18cc108030 0x7f18cc19fbb0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:41.251 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:41.249+0000 7f18d36ab700 1 --2- 172.21.15.49:0/2949887136 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f18b80a2890 0x7f18b80a4d50 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:41.251 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:41.249+0000 7f18d36ab700 1 --2- 172.21.15.49:0/2949887136 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f18cc102dd0 0x7f18cc1985f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:41.251 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:41.249+0000 7f18d36ab700 1 --2- 172.21.15.49:0/2949887136 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f18cc105700 0x7f18cc198b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:41.251 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:41.249+0000 7f18d36ab700 1 -- 172.21.15.49:0/2949887136 >> 172.21.15.49:0/2949887136 conn(0x7f18cc0fc7e0 msgr2=0x7f18cc103810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:07:41.251 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:41.249+0000 7f18d36ab700 1 -- 172.21.15.49:0/2949887136 shutdown_connections 2024-03-18T23:07:41.251 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:41.249+0000 7f18d36ab700 1 -- 172.21.15.49:0/2949887136 wait complete. 2024-03-18T23:07:41.260 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:07:41 smithi049 ceph-mon[62991]: pgmap v338: 97 pgs: 97 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 11 op/s 2024-03-18T23:07:41.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:07:41 smithi139 ceph-mon[67282]: pgmap v338: 97 pgs: 97 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 11 op/s 2024-03-18T23:07:41.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:07:41 smithi175 ceph-mon[67352]: pgmap v338: 97 pgs: 97 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 11 op/s 2024-03-18T23:07:42.030 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c63591ea-e57a-11ee-95c9-87774f69a715 -- bash -c 'ceph orch ls' 2024-03-18T23:07:42.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:07:42 smithi139 ceph-mon[67282]: from='client.15141 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T23:07:42.379 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:07:42 smithi049 ceph-mon[62991]: from='client.15141 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T23:07:42.431 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T23:07:42.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:07:42 smithi175 ceph-mon[67352]: from='client.15141 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T23:07:43.229 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:07:43 smithi049 ceph-mon[62991]: pgmap v339: 97 pgs: 97 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 426 B/s wr, 11 op/s 2024-03-18T23:07:43.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:07:43 smithi139 ceph-mon[67282]: pgmap v339: 97 pgs: 97 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 426 B/s wr, 11 op/s 2024-03-18T23:07:43.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:07:43 smithi175 ceph-mon[67352]: pgmap v339: 97 pgs: 97 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 426 B/s wr, 11 op/s 2024-03-18T23:07:44.137 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:44.135+0000 7fd622863700 1 -- 172.21.15.49:0/3815010358 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd61c102dd0 msgr2=0x7fd61c109680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:44.137 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:44.135+0000 7fd622863700 1 --2- 172.21.15.49:0/3815010358 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd61c102dd0 0x7fd61c109680 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fd61400b0d0 tx=0x7fd61400b3e0 comp rx=0 tx=0).stop 2024-03-18T23:07:44.137 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:44.135+0000 7fd622863700 1 -- 172.21.15.49:0/3815010358 shutdown_connections 2024-03-18T23:07:44.137 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:44.135+0000 7fd622863700 1 --2- 172.21.15.49:0/3815010358 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd61c102dd0 0x7fd61c109680 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:44.137 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:44.135+0000 7fd622863700 1 --2- 172.21.15.49:0/3815010358 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd61c101230 0x7fd61c101650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:44.137 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:44.135+0000 7fd622863700 1 --2- 172.21.15.49:0/3815010358 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd61c102430 0x7fd61c102890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:44.137 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:44.135+0000 7fd622863700 1 -- 172.21.15.49:0/3815010358 >> 172.21.15.49:0/3815010358 conn(0x7fd61c0fc7d0 msgr2=0x7fd61c0fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:07:44.138 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:44.135+0000 7fd622863700 1 -- 172.21.15.49:0/3815010358 shutdown_connections 2024-03-18T23:07:44.138 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:44.135+0000 7fd622863700 1 -- 172.21.15.49:0/3815010358 wait complete. 2024-03-18T23:07:44.138 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:44.136+0000 7fd622863700 1 Processor -- start 2024-03-18T23:07:44.138 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:44.136+0000 7fd622863700 1 -- start start 2024-03-18T23:07:44.138 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:44.136+0000 7fd622863700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd61c101230 0x7fd61c1051c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:44.138 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:44.136+0000 7fd622863700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd61c102430 0x7fd61c103810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:44.138 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:44.136+0000 7fd622863700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd61c102dd0 0x7fd61c103d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:44.138 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:44.136+0000 7fd622863700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fd61c10ce10 con 0x7fd61c101230 2024-03-18T23:07:44.138 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:44.136+0000 7fd622863700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fd61c10cc90 con 0x7fd61c102dd0 2024-03-18T23:07:44.138 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:44.136+0000 7fd622863700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd61c10cf90 con 0x7fd61c102430 2024-03-18T23:07:44.138 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:44.136+0000 7fd61bfff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd61c101230 0x7fd61c1051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:44.138 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:44.136+0000 7fd61b7fe700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd61c102430 0x7fd61c103810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:44.138 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:44.136+0000 7fd61bfff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd61c101230 0x7fd61c1051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:54198/0 (socket says 172.21.15.49:54198) 2024-03-18T23:07:44.138 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:44.136+0000 7fd61bfff700 1 -- 172.21.15.49:0/858777053 learned_addr learned my addr 172.21.15.49:0/858777053 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:07:44.139 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:44.136+0000 7fd620e00700 1 --2- 172.21.15.49:0/858777053 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd61c102dd0 0x7fd61c103d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:44.140 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:44.136+0000 7fd61b7fe700 1 -- 172.21.15.49:0/858777053 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd61c102dd0 msgr2=0x7fd61c103d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:44.140 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:44.136+0000 7fd61b7fe700 1 --2- 172.21.15.49:0/858777053 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd61c102dd0 0x7fd61c103d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:44.140 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:44.136+0000 7fd61b7fe700 1 -- 172.21.15.49:0/858777053 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd61c101230 msgr2=0x7fd61c1051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:44.140 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:44.136+0000 7fd61b7fe700 1 --2- 172.21.15.49:0/858777053 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd61c101230 0x7fd61c1051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:44.140 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:44.137+0000 7fd61b7fe700 1 -- 172.21.15.49:0/858777053 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd61c104600 con 0x7fd61c102430 2024-03-18T23:07:44.140 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:44.137+0000 7fd61bfff700 1 --2- 172.21.15.49:0/858777053 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd61c101230 0x7fd61c1051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:07:44.140 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:44.137+0000 7fd620e00700 1 --2- 172.21.15.49:0/858777053 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd61c102dd0 0x7fd61c103d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:07:44.140 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:44.137+0000 7fd61b7fe700 1 --2- 172.21.15.49:0/858777053 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd61c102430 0x7fd61c103810 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fd60400e9f0 tx=0x7fd60400ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:07:44.140 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:44.137+0000 7fd6197fa700 1 -- 172.21.15.49:0/858777053 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd604004220 con 0x7fd61c102430 2024-03-18T23:07:44.140 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:44.137+0000 7fd6197fa700 1 -- 172.21.15.49:0/858777053 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd604004d10 con 0x7fd61c102430 2024-03-18T23:07:44.141 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:44.137+0000 7fd6197fa700 1 -- 172.21.15.49:0/858777053 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd604005020 con 0x7fd61c102430 2024-03-18T23:07:44.141 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:44.137+0000 7fd622863700 1 -- 172.21.15.49:0/858777053 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd61c1a2650 con 0x7fd61c102430 2024-03-18T23:07:44.141 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:44.137+0000 7fd622863700 1 -- 172.21.15.49:0/858777053 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd61c1a2b10 con 0x7fd61c102430 2024-03-18T23:07:44.141 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:44.138+0000 7fd622863700 1 -- 172.21.15.49:0/858777053 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd61c068fd0 con 0x7fd61c102430 2024-03-18T23:07:44.141 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:44.139+0000 7fd6197fa700 1 -- 172.21.15.49:0/858777053 <== mon.1 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7fd604004750 con 0x7fd61c102430 2024-03-18T23:07:44.142 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:44.140+0000 7fd6197fa700 1 --2- 172.21.15.49:0/858777053 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fd6080a2890 0x7fd6080a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:44.142 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:44.140+0000 7fd61bfff700 1 --2- 172.21.15.49:0/858777053 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fd6080a2890 0x7fd6080a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:44.142 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:44.140+0000 7fd6197fa700 1 -- 172.21.15.49:0/858777053 <== mon.1 v2:172.21.15.175:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7fd6040ce900 con 0x7fd61c102430 2024-03-18T23:07:44.142 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:44.140+0000 7fd61bfff700 1 --2- 172.21.15.49:0/858777053 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fd6080a2890 0x7fd6080a4d50 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fd60c005950 tx=0x7fd60c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:07:44.146 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:44.145+0000 7fd6197fa700 1 -- 172.21.15.49:0/858777053 <== mon.1 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fd60409b9b0 con 0x7fd61c102430 2024-03-18T23:07:44.361 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:44.360+0000 7fd622863700 1 -- 172.21.15.49:0/858777053 --> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7fd61c1a2dd0 con 0x7fd6080a2890 2024-03-18T23:07:44.369 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:44.368+0000 7fd6197fa700 1 -- 172.21.15.49:0/858777053 <== mgr.14150 v2:172.21.15.49:6800/2083805211 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7fd61c1a2dd0 con 0x7fd6080a2890 2024-03-18T23:07:44.370 INFO:teuthology.orchestra.run.smithi049.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-03-18T23:07:44.370 INFO:teuthology.orchestra.run.smithi049.stdout:mds.all 12/12 37s ago 2m smithi049=a;smithi049=d;smithi049=g;smithi049=j;smithi139=b;smithi139=e;smithi139=h;smithi139=k;smithi175=c;smithi175=f;smithi175=i;smithi175=l;count:12 2024-03-18T23:07:44.370 INFO:teuthology.orchestra.run.smithi049.stdout:mgr 3/3 37s ago 7m smithi049=x;smithi139=y;smithi175=z;count:3 2024-03-18T23:07:44.370 INFO:teuthology.orchestra.run.smithi049.stdout:mon 3/3 37s ago 8m smithi049:172.21.15.49=a;smithi139:172.21.15.139=b;smithi175:172.21.15.175=c;count:3 2024-03-18T23:07:44.370 INFO:teuthology.orchestra.run.smithi049.stdout:osd 12 37s ago - 2024-03-18T23:07:44.372 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:44.370+0000 7fd622863700 1 -- 172.21.15.49:0/858777053 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fd6080a2890 msgr2=0x7fd6080a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:44.372 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:44.370+0000 7fd622863700 1 --2- 172.21.15.49:0/858777053 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fd6080a2890 0x7fd6080a4d50 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fd60c005950 tx=0x7fd60c009260 comp rx=0 tx=0).stop 2024-03-18T23:07:44.372 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:44.370+0000 7fd622863700 1 -- 172.21.15.49:0/858777053 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd61c102430 msgr2=0x7fd61c103810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:44.372 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:44.370+0000 7fd622863700 1 --2- 172.21.15.49:0/858777053 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd61c102430 0x7fd61c103810 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fd60400e9f0 tx=0x7fd60400ed00 comp rx=0 tx=0).stop 2024-03-18T23:07:44.372 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:44.371+0000 7fd622863700 1 -- 172.21.15.49:0/858777053 shutdown_connections 2024-03-18T23:07:44.372 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:44.371+0000 7fd622863700 1 --2- 172.21.15.49:0/858777053 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd61c102dd0 0x7fd61c103d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:44.372 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:44.371+0000 7fd622863700 1 --2- 172.21.15.49:0/858777053 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd61c101230 0x7fd61c1051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:44.372 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:44.371+0000 7fd622863700 1 --2- 172.21.15.49:0/858777053 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fd6080a2890 0x7fd6080a4d50 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:44.372 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:44.371+0000 7fd622863700 1 --2- 172.21.15.49:0/858777053 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd61c102430 0x7fd61c103810 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:44.372 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:44.371+0000 7fd622863700 1 -- 172.21.15.49:0/858777053 >> 172.21.15.49:0/858777053 conn(0x7fd61c0fc7d0 msgr2=0x7fd61c0fe210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:07:44.372 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:44.371+0000 7fd622863700 1 -- 172.21.15.49:0/858777053 shutdown_connections 2024-03-18T23:07:44.372 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:44.371+0000 7fd622863700 1 -- 172.21.15.49:0/858777053 wait complete. 2024-03-18T23:07:45.304 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:07:45 smithi049 ceph-mon[62991]: pgmap v340: 97 pgs: 97 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-03-18T23:07:45.304 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:07:45 smithi049 ceph-mon[62991]: from='client.24958 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T23:07:45.306 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c63591ea-e57a-11ee-95c9-87774f69a715 -- bash -c 'ceph orch host ls' 2024-03-18T23:07:45.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:07:45 smithi139 ceph-mon[67282]: pgmap v340: 97 pgs: 97 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-03-18T23:07:45.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:07:45 smithi139 ceph-mon[67282]: from='client.24958 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T23:07:45.465 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:07:45 smithi175 ceph-mon[67352]: pgmap v340: 97 pgs: 97 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-03-18T23:07:45.465 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:07:45 smithi175 ceph-mon[67352]: from='client.24958 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T23:07:45.568 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T23:07:47.269 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:07:47 smithi049 ceph-mon[62991]: pgmap v341: 97 pgs: 97 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 85 B/s wr, 10 op/s 2024-03-18T23:07:47.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:07:47 smithi139 ceph-mon[67282]: pgmap v341: 97 pgs: 97 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 85 B/s wr, 10 op/s 2024-03-18T23:07:47.385 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:47.379+0000 7f563bde5700 1 -- 172.21.15.49:0/4217781071 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5634105750 msgr2=0x7f5634107b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:47.386 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:47.379+0000 7f563bde5700 1 --2- 172.21.15.49:0/4217781071 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5634105750 0x7f5634107b40 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f5628009990 tx=0x7f5628009ca0 comp rx=0 tx=0).stop 2024-03-18T23:07:47.386 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:47.379+0000 7f563bde5700 1 -- 172.21.15.49:0/4217781071 shutdown_connections 2024-03-18T23:07:47.386 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:47.379+0000 7f563bde5700 1 --2- 172.21.15.49:0/4217781071 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5634108080 0x7f563410a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:47.386 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:47.379+0000 7f563bde5700 1 --2- 172.21.15.49:0/4217781071 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5634102e20 0x7f5634105210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:47.386 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:47.379+0000 7f563bde5700 1 --2- 172.21.15.49:0/4217781071 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5634105750 0x7f5634107b40 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:47.386 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:47.379+0000 7f563bde5700 1 -- 172.21.15.49:0/4217781071 >> 172.21.15.49:0/4217781071 conn(0x7f56340fc7d0 msgr2=0x7f56340fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:07:47.386 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:47.379+0000 7f563bde5700 1 -- 172.21.15.49:0/4217781071 shutdown_connections 2024-03-18T23:07:47.386 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:47.379+0000 7f563bde5700 1 -- 172.21.15.49:0/4217781071 wait complete. 2024-03-18T23:07:47.386 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:47.380+0000 7f563bde5700 1 Processor -- start 2024-03-18T23:07:47.386 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:47.380+0000 7f563bde5700 1 -- start start 2024-03-18T23:07:47.387 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:47.380+0000 7f563bde5700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5634102e20 0x7f5634102780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:47.387 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:47.380+0000 7f563bde5700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5634105750 0x7f5634100dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:47.387 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:47.380+0000 7f563bde5700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5634108080 0x7f5634101310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:47.387 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:47.380+0000 7f563bde5700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f563410df40 con 0x7f5634105750 2024-03-18T23:07:47.387 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:47.380+0000 7f563bde5700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f563410ddc0 con 0x7f5634102e20 2024-03-18T23:07:47.387 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:47.380+0000 7f563bde5700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f563410e0c0 con 0x7f5634108080 2024-03-18T23:07:47.387 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:47.380+0000 7f5639380700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5634105750 0x7f5634100dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:47.387 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:47.380+0000 7f5639380700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5634105750 0x7f5634100dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53606/0 (socket says 172.21.15.49:53606) 2024-03-18T23:07:47.387 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:47.380+0000 7f5639380700 1 -- 172.21.15.49:0/2220076017 learned_addr learned my addr 172.21.15.49:0/2220076017 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:07:47.387 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:47.380+0000 7f5639b81700 1 --2- 172.21.15.49:0/2220076017 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5634102e20 0x7f5634102780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:47.388 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:47.380+0000 7f563a382700 1 --2- 172.21.15.49:0/2220076017 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5634108080 0x7f5634101310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:47.388 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:47.380+0000 7f5639b81700 1 -- 172.21.15.49:0/2220076017 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5634108080 msgr2=0x7f5634101310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:47.388 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:47.380+0000 7f5639b81700 1 --2- 172.21.15.49:0/2220076017 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5634108080 0x7f5634101310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:47.388 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:47.380+0000 7f5639b81700 1 -- 172.21.15.49:0/2220076017 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5634105750 msgr2=0x7f5634100dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:47.388 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:47.380+0000 7f5639b81700 1 --2- 172.21.15.49:0/2220076017 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5634105750 0x7f5634100dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:47.388 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:47.380+0000 7f5639b81700 1 -- 172.21.15.49:0/2220076017 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5634101bc0 con 0x7f5634102e20 2024-03-18T23:07:47.388 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:47.380+0000 7f563a382700 1 --2- 172.21.15.49:0/2220076017 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5634108080 0x7f5634101310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:07:47.388 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:47.381+0000 7f5639b81700 1 --2- 172.21.15.49:0/2220076017 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5634102e20 0x7f5634102780 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f562c00ef10 tx=0x7f562c00c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:07:47.388 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:47.381+0000 7f5632ffd700 1 -- 172.21.15.49:0/2220076017 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f562c00c9c0 con 0x7f5634102e20 2024-03-18T23:07:47.388 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:47.381+0000 7f5632ffd700 1 -- 172.21.15.49:0/2220076017 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f562c019070 con 0x7f5634102e20 2024-03-18T23:07:47.388 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:47.381+0000 7f563bde5700 1 -- 172.21.15.49:0/2220076017 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56341a2640 con 0x7f5634102e20 2024-03-18T23:07:47.388 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:47.381+0000 7f5632ffd700 1 -- 172.21.15.49:0/2220076017 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f562c0146e0 con 0x7f5634102e20 2024-03-18T23:07:47.389 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:47.381+0000 7f563bde5700 1 -- 172.21.15.49:0/2220076017 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56341a2b50 con 0x7f5634102e20 2024-03-18T23:07:47.389 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:47.382+0000 7f563bde5700 1 -- 172.21.15.49:0/2220076017 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5634068fd0 con 0x7f5634102e20 2024-03-18T23:07:47.389 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:47.383+0000 7f5632ffd700 1 -- 172.21.15.49:0/2220076017 <== mon.2 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7f562c012050 con 0x7f5634102e20 2024-03-18T23:07:47.389 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:47.384+0000 7f5632ffd700 1 --2- 172.21.15.49:0/2220076017 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f561c0a2890 0x7f561c0a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:47.389 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:47.384+0000 7f5639380700 1 --2- 172.21.15.49:0/2220076017 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f561c0a2890 0x7f561c0a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:47.389 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:47.384+0000 7f5632ffd700 1 -- 172.21.15.49:0/2220076017 <== mon.2 v2:172.21.15.139:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f562c0cf030 con 0x7f5634102e20 2024-03-18T23:07:47.389 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:47.384+0000 7f5639380700 1 --2- 172.21.15.49:0/2220076017 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f561c0a2890 0x7f561c0a4d50 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f562800b2e0 tx=0x7f562800b880 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:07:47.390 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:47.387+0000 7f5632ffd700 1 -- 172.21.15.49:0/2220076017 <== mon.2 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f562c010040 con 0x7f5634102e20 2024-03-18T23:07:47.462 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:07:47 smithi175 ceph-mon[67352]: pgmap v341: 97 pgs: 97 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 85 B/s wr, 10 op/s 2024-03-18T23:07:47.601 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:47.600+0000 7f563bde5700 1 -- 172.21.15.49:0/2220076017 --> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7f5634062750 con 0x7f561c0a2890 2024-03-18T23:07:47.602 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:47.601+0000 7f5632ffd700 1 -- 172.21.15.49:0/2220076017 <== mgr.14150 v2:172.21.15.49:6800/2083805211 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7f5634062750 con 0x7f561c0a2890 2024-03-18T23:07:47.602 INFO:teuthology.orchestra.run.smithi049.stdout:HOST ADDR LABELS STATUS 2024-03-18T23:07:47.602 INFO:teuthology.orchestra.run.smithi049.stdout:smithi049 172.21.15.49 2024-03-18T23:07:47.602 INFO:teuthology.orchestra.run.smithi049.stdout:smithi139 172.21.15.139 2024-03-18T23:07:47.603 INFO:teuthology.orchestra.run.smithi049.stdout:smithi175 172.21.15.175 2024-03-18T23:07:47.603 INFO:teuthology.orchestra.run.smithi049.stdout:3 hosts in cluster 2024-03-18T23:07:47.605 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:47.603+0000 7f563bde5700 1 -- 172.21.15.49:0/2220076017 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f561c0a2890 msgr2=0x7f561c0a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:47.605 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:47.603+0000 7f563bde5700 1 --2- 172.21.15.49:0/2220076017 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f561c0a2890 0x7f561c0a4d50 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f562800b2e0 tx=0x7f562800b880 comp rx=0 tx=0).stop 2024-03-18T23:07:47.605 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:47.603+0000 7f563bde5700 1 -- 172.21.15.49:0/2220076017 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5634102e20 msgr2=0x7f5634102780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:47.605 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:47.603+0000 7f563bde5700 1 --2- 172.21.15.49:0/2220076017 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5634102e20 0x7f5634102780 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f562c00ef10 tx=0x7f562c00c3a0 comp rx=0 tx=0).stop 2024-03-18T23:07:47.605 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:47.603+0000 7f563bde5700 1 -- 172.21.15.49:0/2220076017 shutdown_connections 2024-03-18T23:07:47.605 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:47.603+0000 7f563bde5700 1 --2- 172.21.15.49:0/2220076017 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f561c0a2890 0x7f561c0a4d50 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:47.605 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:47.603+0000 7f563bde5700 1 --2- 172.21.15.49:0/2220076017 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5634108080 0x7f5634101310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:47.605 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:47.603+0000 7f563bde5700 1 --2- 172.21.15.49:0/2220076017 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5634102e20 0x7f5634102780 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:47.605 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:47.603+0000 7f563bde5700 1 --2- 172.21.15.49:0/2220076017 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5634105750 0x7f5634100dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:47.605 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:47.603+0000 7f563bde5700 1 -- 172.21.15.49:0/2220076017 >> 172.21.15.49:0/2220076017 conn(0x7f56340fc7d0 msgr2=0x7f56341036c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:07:47.606 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:47.604+0000 7f563bde5700 1 -- 172.21.15.49:0/2220076017 shutdown_connections 2024-03-18T23:07:47.606 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:47.604+0000 7f563bde5700 1 -- 172.21.15.49:0/2220076017 wait complete. 2024-03-18T23:07:48.201 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:07:48 smithi049 ceph-mon[62991]: from='client.24974 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T23:07:48.230 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c63591ea-e57a-11ee-95c9-87774f69a715 -- bash -c 'ceph orch device ls' 2024-03-18T23:07:48.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:07:48 smithi139 ceph-mon[67282]: from='client.24974 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T23:07:48.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:07:48 smithi175 ceph-mon[67352]: from='client.24974 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T23:07:48.486 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T23:07:49.130 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:07:49 smithi049 ceph-mon[62991]: pgmap v342: 97 pgs: 97 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-03-18T23:07:49.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:07:49 smithi139 ceph-mon[67282]: pgmap v342: 97 pgs: 97 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-03-18T23:07:49.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:07:49 smithi175 ceph-mon[67352]: pgmap v342: 97 pgs: 97 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-03-18T23:07:50.212 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:50.206+0000 7fddbefef700 1 -- 172.21.15.49:0/3688112104 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fddb8102dd0 msgr2=0x7fddb8109680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:50.212 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:50.206+0000 7fddbefef700 1 --2- 172.21.15.49:0/3688112104 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fddb8102dd0 0x7fddb8109680 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fdda80099c0 tx=0x7fdda8009cd0 comp rx=0 tx=0).stop 2024-03-18T23:07:50.212 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:50.206+0000 7fddbefef700 1 -- 172.21.15.49:0/3688112104 shutdown_connections 2024-03-18T23:07:50.213 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:50.206+0000 7fddbefef700 1 --2- 172.21.15.49:0/3688112104 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fddb8102dd0 0x7fddb8109680 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:50.213 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:50.206+0000 7fddbefef700 1 --2- 172.21.15.49:0/3688112104 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fddb8101230 0x7fddb8101650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:50.213 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:50.206+0000 7fddbefef700 1 --2- 172.21.15.49:0/3688112104 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fddb8102430 0x7fddb8102890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:50.213 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:50.206+0000 7fddbefef700 1 -- 172.21.15.49:0/3688112104 >> 172.21.15.49:0/3688112104 conn(0x7fddb80fc7d0 msgr2=0x7fddb80fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:07:50.213 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:50.206+0000 7fddbefef700 1 -- 172.21.15.49:0/3688112104 shutdown_connections 2024-03-18T23:07:50.213 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:50.206+0000 7fddbefef700 1 -- 172.21.15.49:0/3688112104 wait complete. 2024-03-18T23:07:50.213 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:50.207+0000 7fddbefef700 1 Processor -- start 2024-03-18T23:07:50.213 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:50.207+0000 7fddbefef700 1 -- start start 2024-03-18T23:07:50.213 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:50.207+0000 7fddbefef700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fddb8101230 0x7fddb8197390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:50.213 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:50.207+0000 7fddbefef700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fddb8102430 0x7fddb81978d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:50.213 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:50.207+0000 7fddbefef700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fddb8102dd0 0x7fddb819e950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:50.213 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:50.207+0000 7fddbefef700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fddb810ce10 con 0x7fddb8102430 2024-03-18T23:07:50.213 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:50.207+0000 7fddbefef700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fddb810cc90 con 0x7fddb8102dd0 2024-03-18T23:07:50.213 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:50.207+0000 7fddbefef700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fddb810cf90 con 0x7fddb8101230 2024-03-18T23:07:50.214 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:50.207+0000 7fddb7fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fddb8102430 0x7fddb81978d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:50.214 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:50.207+0000 7fddb7fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fddb8102430 0x7fddb81978d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53632/0 (socket says 172.21.15.49:53632) 2024-03-18T23:07:50.214 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:50.207+0000 7fddb7fff700 1 -- 172.21.15.49:0/119931347 learned_addr learned my addr 172.21.15.49:0/119931347 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:07:50.214 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:50.207+0000 7fddbcd8b700 1 --2- 172.21.15.49:0/119931347 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fddb8101230 0x7fddb8197390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:50.214 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:50.207+0000 7fddbd58c700 1 --2- 172.21.15.49:0/119931347 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fddb8102dd0 0x7fddb819e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:50.214 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:50.207+0000 7fddb7fff700 1 -- 172.21.15.49:0/119931347 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fddb8101230 msgr2=0x7fddb8197390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:50.214 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:50.207+0000 7fddb7fff700 1 --2- 172.21.15.49:0/119931347 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fddb8101230 0x7fddb8197390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:50.214 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:50.207+0000 7fddb7fff700 1 -- 172.21.15.49:0/119931347 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fddb8102dd0 msgr2=0x7fddb819e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:50.214 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:50.207+0000 7fddb7fff700 1 --2- 172.21.15.49:0/119931347 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fddb8102dd0 0x7fddb819e950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:50.214 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:50.207+0000 7fddb7fff700 1 -- 172.21.15.49:0/119931347 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddb819eff0 con 0x7fddb8102430 2024-03-18T23:07:50.214 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:50.208+0000 7fddb7fff700 1 --2- 172.21.15.49:0/119931347 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fddb8102430 0x7fddb81978d0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fddac00e9d0 tx=0x7fddac00ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:07:50.214 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:50.208+0000 7fddb5ffb700 1 -- 172.21.15.49:0/119931347 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddac00cbd0 con 0x7fddb8102430 2024-03-18T23:07:50.214 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:50.208+0000 7fddb5ffb700 1 -- 172.21.15.49:0/119931347 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fddac00ee70 con 0x7fddb8102430 2024-03-18T23:07:50.215 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:50.208+0000 7fddbefef700 1 -- 172.21.15.49:0/119931347 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fddb819f2e0 con 0x7fddb8102430 2024-03-18T23:07:50.215 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:50.208+0000 7fddb5ffb700 1 -- 172.21.15.49:0/119931347 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddac018660 con 0x7fddb8102430 2024-03-18T23:07:50.215 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:50.208+0000 7fddbefef700 1 -- 172.21.15.49:0/119931347 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fddb819f820 con 0x7fddb8102430 2024-03-18T23:07:50.215 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:50.209+0000 7fddbefef700 1 -- 172.21.15.49:0/119931347 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fddb8068fd0 con 0x7fddb8102430 2024-03-18T23:07:50.215 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:50.210+0000 7fddb5ffb700 1 -- 172.21.15.49:0/119931347 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7fddac010430 con 0x7fddb8102430 2024-03-18T23:07:50.215 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:50.211+0000 7fddb5ffb700 1 --2- 172.21.15.49:0/119931347 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fdda00a2890 0x7fdda00a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:50.215 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:50.211+0000 7fddbcd8b700 1 --2- 172.21.15.49:0/119931347 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fdda00a2890 0x7fdda00a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:50.215 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:50.211+0000 7fddb5ffb700 1 -- 172.21.15.49:0/119931347 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7fddac0cec70 con 0x7fddb8102430 2024-03-18T23:07:50.215 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:50.211+0000 7fddbcd8b700 1 --2- 172.21.15.49:0/119931347 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fdda00a2890 0x7fdda00a4d50 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fddb8102290 tx=0x7fddb000b040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:07:50.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:50.215+0000 7fddb5ffb700 1 -- 172.21.15.49:0/119931347 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fddac09bd20 con 0x7fddb8102430 2024-03-18T23:07:50.430 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:50.429+0000 7fddbefef700 1 -- 172.21.15.49:0/119931347 --> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7fddb8062750 con 0x7fdda00a2890 2024-03-18T23:07:50.432 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:50.431+0000 7fddb5ffb700 1 -- 172.21.15.49:0/119931347 <== mgr.14150 v2:172.21.15.49:6800/2083805211 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7fddb8062750 con 0x7fdda00a2890 2024-03-18T23:07:50.432 INFO:teuthology.orchestra.run.smithi049.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-03-18T23:07:50.432 INFO:teuthology.orchestra.run.smithi049.stdout:smithi049 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB747500D2480DGN 447G No 3m ago Has a FileSystem, LVM detected 2024-03-18T23:07:50.432 INFO:teuthology.orchestra.run.smithi049.stdout:smithi139 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620500N6400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-03-18T23:07:50.432 INFO:teuthology.orchestra.run.smithi049.stdout:smithi175 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT623300M0400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-03-18T23:07:50.435 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:50.433+0000 7fddbefef700 1 -- 172.21.15.49:0/119931347 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fdda00a2890 msgr2=0x7fdda00a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:50.435 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:50.433+0000 7fddbefef700 1 --2- 172.21.15.49:0/119931347 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fdda00a2890 0x7fdda00a4d50 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fddb8102290 tx=0x7fddb000b040 comp rx=0 tx=0).stop 2024-03-18T23:07:50.435 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:50.433+0000 7fddbefef700 1 -- 172.21.15.49:0/119931347 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fddb8102430 msgr2=0x7fddb81978d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:50.435 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:50.433+0000 7fddbefef700 1 --2- 172.21.15.49:0/119931347 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fddb8102430 0x7fddb81978d0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fddac00e9d0 tx=0x7fddac00ece0 comp rx=0 tx=0).stop 2024-03-18T23:07:50.435 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:50.433+0000 7fddbefef700 1 -- 172.21.15.49:0/119931347 shutdown_connections 2024-03-18T23:07:50.435 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:50.433+0000 7fddbefef700 1 --2- 172.21.15.49:0/119931347 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fddb8102dd0 0x7fddb819e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:50.435 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:50.433+0000 7fddbefef700 1 --2- 172.21.15.49:0/119931347 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fdda00a2890 0x7fdda00a4d50 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:50.435 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:50.433+0000 7fddbefef700 1 --2- 172.21.15.49:0/119931347 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fddb8101230 0x7fddb8197390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:50.435 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:50.433+0000 7fddbefef700 1 --2- 172.21.15.49:0/119931347 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fddb8102430 0x7fddb81978d0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:50.436 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:50.433+0000 7fddbefef700 1 -- 172.21.15.49:0/119931347 >> 172.21.15.49:0/119931347 conn(0x7fddb80fc7d0 msgr2=0x7fddb81056d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:07:50.436 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:50.433+0000 7fddbefef700 1 -- 172.21.15.49:0/119931347 shutdown_connections 2024-03-18T23:07:50.436 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:50.433+0000 7fddbefef700 1 -- 172.21.15.49:0/119931347 wait complete. 2024-03-18T23:07:51.130 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:07:51 smithi049 ceph-mon[62991]: pgmap v343: 97 pgs: 97 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-03-18T23:07:51.130 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:07:51 smithi049 ceph-mon[62991]: from='client.15159 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T23:07:51.163 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-03-18T23:07:51.172 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi049.front.sepia.ceph.com 2024-03-18T23:07:51.172 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c63591ea-e57a-11ee-95c9-87774f69a715 -- bash -c 'ceph fs dump' 2024-03-18T23:07:51.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:07:51 smithi139 ceph-mon[67282]: pgmap v343: 97 pgs: 97 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-03-18T23:07:51.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:07:51 smithi139 ceph-mon[67282]: from='client.15159 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T23:07:51.423 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T23:07:51.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:07:51 smithi175 ceph-mon[67352]: pgmap v343: 97 pgs: 97 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-03-18T23:07:51.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:07:51 smithi175 ceph-mon[67352]: from='client.15159 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T23:07:53.256 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:53.255+0000 7f4a467ec700 1 -- 172.21.15.49:0/914347436 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4a40103540 msgr2=0x7f4a401039a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:53.256 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:53.255+0000 7f4a467ec700 1 --2- 172.21.15.49:0/914347436 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4a40103540 0x7f4a401039a0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f4a300099c0 tx=0x7f4a30009cd0 comp rx=0 tx=0).stop 2024-03-18T23:07:53.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:53.255+0000 7f4a467ec700 1 -- 172.21.15.49:0/914347436 shutdown_connections 2024-03-18T23:07:53.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:53.255+0000 7f4a467ec700 1 --2- 172.21.15.49:0/914347436 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4a40103ee0 0x7f4a4010a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:53.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:53.255+0000 7f4a467ec700 1 --2- 172.21.15.49:0/914347436 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4a40102340 0x7f4a40102760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:53.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:53.255+0000 7f4a467ec700 1 --2- 172.21.15.49:0/914347436 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4a40103540 0x7f4a401039a0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:53.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:53.255+0000 7f4a467ec700 1 -- 172.21.15.49:0/914347436 >> 172.21.15.49:0/914347436 conn(0x7f4a400fd8c0 msgr2=0x7f4a400ffd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:07:53.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:53.255+0000 7f4a467ec700 1 -- 172.21.15.49:0/914347436 shutdown_connections 2024-03-18T23:07:53.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:53.255+0000 7f4a467ec700 1 -- 172.21.15.49:0/914347436 wait complete. 2024-03-18T23:07:53.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:53.256+0000 7f4a467ec700 1 Processor -- start 2024-03-18T23:07:53.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:53.256+0000 7f4a467ec700 1 -- start start 2024-03-18T23:07:53.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:53.256+0000 7f4a467ec700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4a40102340 0x7f4a40198500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:53.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:53.256+0000 7f4a467ec700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4a40103540 0x7f4a40198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:53.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:53.256+0000 7f4a467ec700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4a40103ee0 0x7f4a4019fac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:53.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:53.256+0000 7f4a467ec700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f4a4010df10 con 0x7f4a40102340 2024-03-18T23:07:53.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:53.256+0000 7f4a467ec700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f4a4010dd90 con 0x7f4a40103ee0 2024-03-18T23:07:53.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:53.256+0000 7f4a467ec700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f4a4010e090 con 0x7f4a40103540 2024-03-18T23:07:53.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:53.256+0000 7f4a3ffff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4a40102340 0x7f4a40198500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:53.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:53.256+0000 7f4a3ffff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4a40102340 0x7f4a40198500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53660/0 (socket says 172.21.15.49:53660) 2024-03-18T23:07:53.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:53.256+0000 7f4a3f7fe700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4a40103540 0x7f4a40198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:53.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:53.256+0000 7f4a3ffff700 1 -- 172.21.15.49:0/2574738231 learned_addr learned my addr 172.21.15.49:0/2574738231 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:07:53.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:53.256+0000 7f4a44d89700 1 --2- 172.21.15.49:0/2574738231 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4a40103ee0 0x7f4a4019fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:53.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:53.257+0000 7f4a3ffff700 1 -- 172.21.15.49:0/2574738231 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4a40103540 msgr2=0x7f4a40198a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:53.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:53.257+0000 7f4a3ffff700 1 --2- 172.21.15.49:0/2574738231 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4a40103540 0x7f4a40198a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:53.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:53.257+0000 7f4a3ffff700 1 -- 172.21.15.49:0/2574738231 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4a40103ee0 msgr2=0x7f4a4019fac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:53.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:53.257+0000 7f4a3ffff700 1 --2- 172.21.15.49:0/2574738231 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4a40103ee0 0x7f4a4019fac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:53.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:53.257+0000 7f4a3ffff700 1 -- 172.21.15.49:0/2574738231 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a401a0160 con 0x7f4a40102340 2024-03-18T23:07:53.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:53.257+0000 7f4a3f7fe700 1 --2- 172.21.15.49:0/2574738231 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4a40103540 0x7f4a40198a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:07:53.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:53.257+0000 7f4a3ffff700 1 --2- 172.21.15.49:0/2574738231 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4a40102340 0x7f4a40198500 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f4a2800e9a0 tx=0x7f4a2800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:07:53.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:53.257+0000 7f4a3d7fa700 1 -- 172.21.15.49:0/2574738231 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a28004220 con 0x7f4a40102340 2024-03-18T23:07:53.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:53.257+0000 7f4a3d7fa700 1 -- 172.21.15.49:0/2574738231 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4a280043c0 con 0x7f4a40102340 2024-03-18T23:07:53.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:53.257+0000 7f4a3d7fa700 1 -- 172.21.15.49:0/2574738231 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a28003df0 con 0x7f4a40102340 2024-03-18T23:07:53.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:53.257+0000 7f4a467ec700 1 -- 172.21.15.49:0/2574738231 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a401a0450 con 0x7f4a40102340 2024-03-18T23:07:53.261 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:53.257+0000 7f4a467ec700 1 -- 172.21.15.49:0/2574738231 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a401a0990 con 0x7f4a40102340 2024-03-18T23:07:53.261 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:53.258+0000 7f4a467ec700 1 -- 172.21.15.49:0/2574738231 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a40068fd0 con 0x7f4a40102340 2024-03-18T23:07:53.261 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:53.259+0000 7f4a3d7fa700 1 -- 172.21.15.49:0/2574738231 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7f4a2800c810 con 0x7f4a40102340 2024-03-18T23:07:53.262 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:53.260+0000 7f4a3d7fa700 1 --2- 172.21.15.49:0/2574738231 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f4a2c0a2890 0x7f4a2c0a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:53.262 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:53.260+0000 7f4a3f7fe700 1 --2- 172.21.15.49:0/2574738231 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f4a2c0a2890 0x7f4a2c0a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:53.262 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:53.260+0000 7f4a3d7fa700 1 -- 172.21.15.49:0/2574738231 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f4a280cebf0 con 0x7f4a40102340 2024-03-18T23:07:53.262 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:53.260+0000 7f4a3f7fe700 1 --2- 172.21.15.49:0/2574738231 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f4a2c0a2890 0x7f4a2c0a4d50 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f4a30000c00 tx=0x7f4a30005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:07:53.265 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:53.264+0000 7f4a3d7fa700 1 -- 172.21.15.49:0/2574738231 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f4a2809bca0 con 0x7f4a40102340 2024-03-18T23:07:53.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:07:53 smithi139 ceph-mon[67282]: pgmap v344: 97 pgs: 97 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-03-18T23:07:53.332 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:07:53 smithi049 ceph-mon[62991]: pgmap v344: 97 pgs: 97 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-03-18T23:07:53.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:07:53 smithi175 ceph-mon[67352]: pgmap v344: 97 pgs: 97 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-03-18T23:07:53.546 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:53.544+0000 7f4a467ec700 1 -- 172.21.15.49:0/2574738231 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f4a40062f20 con 0x7f4a40102340 2024-03-18T23:07:53.546 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:53.545+0000 7f4a3d7fa700 1 -- 172.21.15.49:0/2574738231 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 27 v27) ==== 76+0+3093 (secure 0 0 0) 0x7f4a2809b5a0 con 0x7f4a40102340 2024-03-18T23:07:53.546 INFO:teuthology.orchestra.run.smithi049.stdout:e27 2024-03-18T23:07:53.546 INFO:teuthology.orchestra.run.smithi049.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-18T23:07:53.547 INFO:teuthology.orchestra.run.smithi049.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-03-18T23:07:53.547 INFO:teuthology.orchestra.run.smithi049.stdout:legacy client fscid: 1 2024-03-18T23:07:53.547 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-18T23:07:53.547 INFO:teuthology.orchestra.run.smithi049.stdout:Filesystem 'cephfs' (1) 2024-03-18T23:07:53.547 INFO:teuthology.orchestra.run.smithi049.stdout:fs_name cephfs 2024-03-18T23:07:53.547 INFO:teuthology.orchestra.run.smithi049.stdout:epoch 27 2024-03-18T23:07:53.547 INFO:teuthology.orchestra.run.smithi049.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-03-18T23:07:53.547 INFO:teuthology.orchestra.run.smithi049.stdout:created 2024-03-18T23:06:04.575999+0000 2024-03-18T23:07:53.547 INFO:teuthology.orchestra.run.smithi049.stdout:modified 2024-03-18T23:06:11.998736+0000 2024-03-18T23:07:53.547 INFO:teuthology.orchestra.run.smithi049.stdout:tableserver 0 2024-03-18T23:07:53.547 INFO:teuthology.orchestra.run.smithi049.stdout:root 0 2024-03-18T23:07:53.547 INFO:teuthology.orchestra.run.smithi049.stdout:session_timeout 300 2024-03-18T23:07:53.547 INFO:teuthology.orchestra.run.smithi049.stdout:session_autoclose 300 2024-03-18T23:07:53.547 INFO:teuthology.orchestra.run.smithi049.stdout:max_file_size 1099511627776 2024-03-18T23:07:53.547 INFO:teuthology.orchestra.run.smithi049.stdout:max_xattr_size 65536 2024-03-18T23:07:53.548 INFO:teuthology.orchestra.run.smithi049.stdout:required_client_features {} 2024-03-18T23:07:53.548 INFO:teuthology.orchestra.run.smithi049.stdout:last_failure 0 2024-03-18T23:07:53.548 INFO:teuthology.orchestra.run.smithi049.stdout:last_failure_osd_epoch 0 2024-03-18T23:07:53.548 INFO:teuthology.orchestra.run.smithi049.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-03-18T23:07:53.548 INFO:teuthology.orchestra.run.smithi049.stdout:max_mds 5 2024-03-18T23:07:53.548 INFO:teuthology.orchestra.run.smithi049.stdout:in 0,1,2,3,4 2024-03-18T23:07:53.548 INFO:teuthology.orchestra.run.smithi049.stdout:up {0=24488,1=24463,2=24430,3=24419,4=24391} 2024-03-18T23:07:53.548 INFO:teuthology.orchestra.run.smithi049.stdout:failed 2024-03-18T23:07:53.548 INFO:teuthology.orchestra.run.smithi049.stdout:damaged 2024-03-18T23:07:53.548 INFO:teuthology.orchestra.run.smithi049.stdout:stopped 2024-03-18T23:07:53.548 INFO:teuthology.orchestra.run.smithi049.stdout:data_pools [3] 2024-03-18T23:07:53.548 INFO:teuthology.orchestra.run.smithi049.stdout:metadata_pool 2 2024-03-18T23:07:53.548 INFO:teuthology.orchestra.run.smithi049.stdout:inline_data disabled 2024-03-18T23:07:53.548 INFO:teuthology.orchestra.run.smithi049.stdout:balancer 2024-03-18T23:07:53.548 INFO:teuthology.orchestra.run.smithi049.stdout:bal_rank_mask -1 2024-03-18T23:07:53.549 INFO:teuthology.orchestra.run.smithi049.stdout:standby_count_wanted 0 2024-03-18T23:07:53.549 INFO:teuthology.orchestra.run.smithi049.stdout:qdb_cluster leader: 24488 members: 24463,24488,24430,24419,24391 2024-03-18T23:07:53.549 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.b{0:24488} state up:active seq 3 addr [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] compat {c=[1],r=[1],i=[1fff]}] 2024-03-18T23:07:53.549 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.h{0:24452} state up:standby-replay seq 1 addr [v2:172.21.15.139:6836/2330317540,v1:172.21.15.139:6837/2330317540] compat {c=[1],r=[1],i=[1fff]}] 2024-03-18T23:07:53.549 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.i{1:24463} state up:active seq 5 addr [v2:172.21.15.175:6838/3640882084,v1:172.21.15.175:6839/3640882084] compat {c=[1],r=[1],i=[1fff]}] 2024-03-18T23:07:53.549 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.c{1:24439} state up:standby-replay seq 1 addr [v2:172.21.15.175:6836/379482756,v1:172.21.15.175:6837/379482756] compat {c=[1],r=[1],i=[1fff]}] 2024-03-18T23:07:53.549 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.f{2:24430} state up:active seq 8 addr [v2:172.21.15.175:6834/755474931,v1:172.21.15.175:6835/755474931] compat {c=[1],r=[1],i=[1fff]}] 2024-03-18T23:07:53.549 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.k{2:24389} state up:standby-replay seq 1 addr [v2:172.21.15.139:6832/3766178603,v1:172.21.15.139:6833/3766178603] compat {c=[1],r=[1],i=[1fff]}] 2024-03-18T23:07:53.549 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.e{3:24419} state up:active seq 9 addr [v2:172.21.15.139:6834/1663212937,v1:172.21.15.139:6835/1663212937] compat {c=[1],r=[1],i=[1fff]}] 2024-03-18T23:07:53.549 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.a{3:14634} state up:standby-replay seq 1 addr [v2:172.21.15.49:6840/2869298862,v1:172.21.15.49:6841/2869298862] compat {c=[1],r=[1],i=[1fff]}] 2024-03-18T23:07:53.549 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.l{4:24391} state up:active seq 12 addr [v2:172.21.15.175:6832/1702997173,v1:172.21.15.175:6833/1702997173] compat {c=[1],r=[1],i=[1fff]}] 2024-03-18T23:07:53.549 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.g{4:14622} state up:standby-replay seq 1 addr [v2:172.21.15.49:6838/1934085093,v1:172.21.15.49:6839/1934085093] compat {c=[1],r=[1],i=[1fff]}] 2024-03-18T23:07:53.549 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-18T23:07:53.549 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-18T23:07:53.549 INFO:teuthology.orchestra.run.smithi049.stdout:Standby daemons: 2024-03-18T23:07:53.549 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-18T23:07:53.549 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.j{-1:14556} state up:standby seq 1 addr [v2:172.21.15.49:6834/3631301630,v1:172.21.15.49:6835/3631301630] compat {c=[1],r=[1],i=[1fff]}] 2024-03-18T23:07:53.550 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.d{-1:14568} state up:standby seq 1 addr [v2:172.21.15.49:6836/444305233,v1:172.21.15.49:6837/444305233] compat {c=[1],r=[1],i=[1fff]}] 2024-03-18T23:07:53.550 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:53.548+0000 7f4a467ec700 1 -- 172.21.15.49:0/2574738231 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f4a2c0a2890 msgr2=0x7f4a2c0a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:53.551 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:53.548+0000 7f4a467ec700 1 --2- 172.21.15.49:0/2574738231 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f4a2c0a2890 0x7f4a2c0a4d50 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f4a30000c00 tx=0x7f4a30005fb0 comp rx=0 tx=0).stop 2024-03-18T23:07:53.551 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:53.548+0000 7f4a467ec700 1 -- 172.21.15.49:0/2574738231 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4a40102340 msgr2=0x7f4a40198500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:53.551 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:53.548+0000 7f4a467ec700 1 --2- 172.21.15.49:0/2574738231 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4a40102340 0x7f4a40198500 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f4a2800e9a0 tx=0x7f4a2800ecb0 comp rx=0 tx=0).stop 2024-03-18T23:07:53.551 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:53.548+0000 7f4a467ec700 1 -- 172.21.15.49:0/2574738231 shutdown_connections 2024-03-18T23:07:53.551 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:53.548+0000 7f4a467ec700 1 --2- 172.21.15.49:0/2574738231 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4a40103ee0 0x7f4a4019fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:53.551 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:53.548+0000 7f4a467ec700 1 --2- 172.21.15.49:0/2574738231 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4a40102340 0x7f4a40198500 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:53.551 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:53.548+0000 7f4a467ec700 1 --2- 172.21.15.49:0/2574738231 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f4a2c0a2890 0x7f4a2c0a4d50 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:53.551 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:53.548+0000 7f4a467ec700 1 --2- 172.21.15.49:0/2574738231 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4a40103540 0x7f4a40198a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:53.551 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:53.548+0000 7f4a467ec700 1 -- 172.21.15.49:0/2574738231 >> 172.21.15.49:0/2574738231 conn(0x7f4a400fd8c0 msgr2=0x7f4a40106840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:07:53.551 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:53.548+0000 7f4a467ec700 1 -- 172.21.15.49:0/2574738231 shutdown_connections 2024-03-18T23:07:53.551 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:53.548+0000 7f4a467ec700 1 -- 172.21.15.49:0/2574738231 wait complete. 2024-03-18T23:07:53.551 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 27 2024-03-18T23:07:54.213 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:07:54 smithi049 ceph-mon[62991]: from='client.? 172.21.15.49:0/2574738231' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-18T23:07:54.215 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c63591ea-e57a-11ee-95c9-87774f69a715 -- bash -c 'ceph osd dump' 2024-03-18T23:07:54.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:07:54 smithi139 ceph-mon[67282]: from='client.? 172.21.15.49:0/2574738231' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-18T23:07:54.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:07:54 smithi175 ceph-mon[67352]: from='client.? 172.21.15.49:0/2574738231' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-18T23:07:54.480 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T23:07:55.130 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:07:55 smithi049 ceph-mon[62991]: pgmap v345: 97 pgs: 97 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-18T23:07:55.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:07:55 smithi139 ceph-mon[67282]: pgmap v345: 97 pgs: 97 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-18T23:07:55.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:07:55 smithi175 ceph-mon[67352]: pgmap v345: 97 pgs: 97 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-18T23:07:56.393 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:56.361+0000 7f96e78ed700 1 -- 172.21.15.49:0/3523452432 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f96e0102360 msgr2=0x7f96e0102780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:56.394 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:56.361+0000 7f96e78ed700 1 --2- 172.21.15.49:0/3523452432 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f96e0102360 0x7f96e0102780 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f96d00099c0 tx=0x7f96d0009cd0 comp rx=0 tx=0).stop 2024-03-18T23:07:56.394 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:56.361+0000 7f96e78ed700 1 -- 172.21.15.49:0/3523452432 shutdown_connections 2024-03-18T23:07:56.394 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:56.361+0000 7f96e78ed700 1 --2- 172.21.15.49:0/3523452432 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f96e0103f00 0x7f96e010a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:56.394 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:56.361+0000 7f96e78ed700 1 --2- 172.21.15.49:0/3523452432 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f96e0102360 0x7f96e0102780 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:56.394 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:56.361+0000 7f96e78ed700 1 --2- 172.21.15.49:0/3523452432 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f96e0103560 0x7f96e01039c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:56.394 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:56.361+0000 7f96e78ed700 1 -- 172.21.15.49:0/3523452432 >> 172.21.15.49:0/3523452432 conn(0x7f96e00fd8c0 msgr2=0x7f96e00ffd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:07:56.394 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:56.361+0000 7f96e78ed700 1 -- 172.21.15.49:0/3523452432 shutdown_connections 2024-03-18T23:07:56.394 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:56.362+0000 7f96e78ed700 1 -- 172.21.15.49:0/3523452432 wait complete. 2024-03-18T23:07:56.394 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:56.362+0000 7f96e78ed700 1 Processor -- start 2024-03-18T23:07:56.394 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:56.362+0000 7f96e78ed700 1 -- start start 2024-03-18T23:07:56.395 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:56.362+0000 7f96e78ed700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f96e0102360 0x7f96e0198570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:56.395 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:56.362+0000 7f96e78ed700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f96e0103560 0x7f96e0198ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:56.395 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:56.362+0000 7f96e78ed700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f96e0103f00 0x7f96e019fb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:56.395 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:56.362+0000 7f96e78ed700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f96e010df50 con 0x7f96e0103f00 2024-03-18T23:07:56.395 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:56.362+0000 7f96e78ed700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f96e010ddd0 con 0x7f96e0102360 2024-03-18T23:07:56.395 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:56.362+0000 7f96e78ed700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f96e010e0d0 con 0x7f96e0103560 2024-03-18T23:07:56.395 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:56.363+0000 7f96e5689700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f96e0102360 0x7f96e0198570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:56.395 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:56.363+0000 7f96e4e88700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f96e0103560 0x7f96e0198ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:56.395 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:56.363+0000 7f96e5e8a700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f96e0103f00 0x7f96e019fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:56.395 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:56.363+0000 7f96e5689700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f96e0102360 0x7f96e0198570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.49:48298/0 (socket says 172.21.15.49:48298) 2024-03-18T23:07:56.395 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:56.363+0000 7f96e5689700 1 -- 172.21.15.49:0/65601876 learned_addr learned my addr 172.21.15.49:0/65601876 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:07:56.395 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:56.363+0000 7f96e4e88700 1 -- 172.21.15.49:0/65601876 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f96e0102360 msgr2=0x7f96e0198570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:56.395 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:56.363+0000 7f96e4e88700 1 --2- 172.21.15.49:0/65601876 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f96e0102360 0x7f96e0198570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:56.395 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:56.363+0000 7f96e4e88700 1 -- 172.21.15.49:0/65601876 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f96e0103f00 msgr2=0x7f96e019fb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:56.396 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:56.363+0000 7f96e4e88700 1 --2- 172.21.15.49:0/65601876 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f96e0103f00 0x7f96e019fb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:56.396 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:56.363+0000 7f96e4e88700 1 -- 172.21.15.49:0/65601876 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96e01a01d0 con 0x7f96e0103560 2024-03-18T23:07:56.396 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:56.363+0000 7f96e5e8a700 1 --2- 172.21.15.49:0/65601876 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f96e0103f00 0x7f96e019fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:07:56.396 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:56.363+0000 7f96e5689700 1 --2- 172.21.15.49:0/65601876 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f96e0102360 0x7f96e0198570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:07:56.396 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:56.363+0000 7f96e4e88700 1 --2- 172.21.15.49:0/65601876 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f96e0103560 0x7f96e0198ab0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f96dc00ea20 tx=0x7f96dc00ed30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:07:56.396 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:56.363+0000 7f96d67fc700 1 -- 172.21.15.49:0/65601876 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96dc00cbd0 con 0x7f96e0103560 2024-03-18T23:07:56.396 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:56.363+0000 7f96e78ed700 1 -- 172.21.15.49:0/65601876 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96e01a04c0 con 0x7f96e0103560 2024-03-18T23:07:56.396 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:56.363+0000 7f96d67fc700 1 -- 172.21.15.49:0/65601876 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f96dc00ee60 con 0x7f96e0103560 2024-03-18T23:07:56.396 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:56.363+0000 7f96d67fc700 1 -- 172.21.15.49:0/65601876 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96dc018680 con 0x7f96e0103560 2024-03-18T23:07:56.396 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:56.363+0000 7f96e78ed700 1 -- 172.21.15.49:0/65601876 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96e01a0a00 con 0x7f96e0103560 2024-03-18T23:07:56.396 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:56.364+0000 7f96e78ed700 1 -- 172.21.15.49:0/65601876 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96e0068fd0 con 0x7f96e0103560 2024-03-18T23:07:56.396 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:56.366+0000 7f96d67fc700 1 -- 172.21.15.49:0/65601876 <== mon.1 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7f96dc018820 con 0x7f96e0103560 2024-03-18T23:07:56.396 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:56.366+0000 7f96d67fc700 1 --2- 172.21.15.49:0/65601876 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f96cc0a2890 0x7f96cc0a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:56.397 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:56.367+0000 7f96e5689700 1 --2- 172.21.15.49:0/65601876 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f96cc0a2890 0x7f96cc0a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:56.397 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:56.367+0000 7f96d67fc700 1 -- 172.21.15.49:0/65601876 <== mon.1 v2:172.21.15.175:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f96dc0ceda0 con 0x7f96e0103560 2024-03-18T23:07:56.397 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:56.367+0000 7f96e5689700 1 --2- 172.21.15.49:0/65601876 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f96cc0a2890 0x7f96cc0a4d50 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f96d0000c00 tx=0x7f96d0005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:07:56.397 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:56.371+0000 7f96d67fc700 1 -- 172.21.15.49:0/65601876 <== mon.1 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f96dc09bdf0 con 0x7f96e0103560 2024-03-18T23:07:56.575 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:56.575+0000 7f96e78ed700 1 -- 172.21.15.49:0/65601876 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7f96e002dbe0 con 0x7f96e0103560 2024-03-18T23:07:56.576 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:56.576+0000 7f96d67fc700 1 -- 172.21.15.49:0/65601876 <== mon.1 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v88) ==== 56+0+5289 (secure 0 0 0) 0x7f96dc09b6f0 con 0x7f96e0103560 2024-03-18T23:07:56.576 INFO:teuthology.orchestra.run.smithi049.stdout:epoch 88 2024-03-18T23:07:56.576 INFO:teuthology.orchestra.run.smithi049.stdout:fsid c63591ea-e57a-11ee-95c9-87774f69a715 2024-03-18T23:07:56.576 INFO:teuthology.orchestra.run.smithi049.stdout:created 2024-03-18T22:57:58.734467+0000 2024-03-18T23:07:56.576 INFO:teuthology.orchestra.run.smithi049.stdout:modified 2024-03-18T23:07:01.337701+0000 2024-03-18T23:07:56.577 INFO:teuthology.orchestra.run.smithi049.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-03-18T23:07:56.577 INFO:teuthology.orchestra.run.smithi049.stdout:crush_version 26 2024-03-18T23:07:56.577 INFO:teuthology.orchestra.run.smithi049.stdout:full_ratio 0.95 2024-03-18T23:07:56.577 INFO:teuthology.orchestra.run.smithi049.stdout:backfillfull_ratio 0.9 2024-03-18T23:07:56.577 INFO:teuthology.orchestra.run.smithi049.stdout:nearfull_ratio 0.85 2024-03-18T23:07:56.577 INFO:teuthology.orchestra.run.smithi049.stdout:require_min_compat_client luminous 2024-03-18T23:07:56.577 INFO:teuthology.orchestra.run.smithi049.stdout:min_compat_client luminous 2024-03-18T23:07:56.577 INFO:teuthology.orchestra.run.smithi049.stdout:require_osd_release squid 2024-03-18T23:07:56.577 INFO:teuthology.orchestra.run.smithi049.stdout:stretch_mode_enabled false 2024-03-18T23:07:56.577 INFO:teuthology.orchestra.run.smithi049.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 21 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-03-18T23:07:56.577 INFO:teuthology.orchestra.run.smithi049.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 85 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-03-18T23:07:56.577 INFO:teuthology.orchestra.run.smithi049.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 86 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.9 application cephfs read_balance_score 1.50 2024-03-18T23:07:56.577 INFO:teuthology.orchestra.run.smithi049.stdout:max_osd 12 2024-03-18T23:07:56.577 INFO:teuthology.orchestra.run.smithi049.stdout:osd.0 up in weight 1 up_from 8 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.49:6802/1651162501,v1:172.21.15.49:6803/1651162501] [v2:172.21.15.49:6804/1651162501,v1:172.21.15.49:6805/1651162501] exists,up e7fc9daa-4098-4ba5-8019-9aaef1aab3e3 2024-03-18T23:07:56.578 INFO:teuthology.orchestra.run.smithi049.stdout:osd.1 up in weight 1 up_from 13 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.139:6800/1766700359,v1:172.21.15.139:6801/1766700359] [v2:172.21.15.139:6802/1766700359,v1:172.21.15.139:6803/1766700359] exists,up 40ac929c-e90f-434c-a796-9aabd686f1fa 2024-03-18T23:07:56.578 INFO:teuthology.orchestra.run.smithi049.stdout:osd.2 up in weight 1 up_from 18 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.175:6800/3752671234,v1:172.21.15.175:6801/3752671234] [v2:172.21.15.175:6802/3752671234,v1:172.21.15.175:6803/3752671234] exists,up 8ae3b666-00b4-45ed-a5ee-4d0771edabb6 2024-03-18T23:07:56.578 INFO:teuthology.orchestra.run.smithi049.stdout:osd.3 up in weight 1 up_from 25 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.49:6810/956160536,v1:172.21.15.49:6811/956160536] [v2:172.21.15.49:6812/956160536,v1:172.21.15.49:6813/956160536] exists,up 1a62e89d-45d4-4237-b813-7f96a59aca7c 2024-03-18T23:07:56.578 INFO:teuthology.orchestra.run.smithi049.stdout:osd.4 up in weight 1 up_from 33 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.139:6808/3129712668,v1:172.21.15.139:6809/3129712668] [v2:172.21.15.139:6810/3129712668,v1:172.21.15.139:6811/3129712668] exists,up c715101c-0c3f-4b98-b142-d64a35e187df 2024-03-18T23:07:56.578 INFO:teuthology.orchestra.run.smithi049.stdout:osd.5 up in weight 1 up_from 38 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.175:6808/2774567215,v1:172.21.15.175:6809/2774567215] [v2:172.21.15.175:6810/2774567215,v1:172.21.15.175:6811/2774567215] exists,up 4b641f81-77eb-45fd-8464-511e7b3062b3 2024-03-18T23:07:56.578 INFO:teuthology.orchestra.run.smithi049.stdout:osd.6 up in weight 1 up_from 46 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.49:6818/1472215461,v1:172.21.15.49:6819/1472215461] [v2:172.21.15.49:6820/1472215461,v1:172.21.15.49:6821/1472215461] exists,up 653da8e0-1197-4b48-abed-470f2a9c4dce 2024-03-18T23:07:56.578 INFO:teuthology.orchestra.run.smithi049.stdout:osd.7 up in weight 1 up_from 51 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.139:6816/2163986787,v1:172.21.15.139:6817/2163986787] [v2:172.21.15.139:6818/2163986787,v1:172.21.15.139:6819/2163986787] exists,up ad157c9b-4b75-451c-8740-630f0975b7fc 2024-03-18T23:07:56.578 INFO:teuthology.orchestra.run.smithi049.stdout:osd.8 up in weight 1 up_from 60 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.175:6816/1194026992,v1:172.21.15.175:6817/1194026992] [v2:172.21.15.175:6818/1194026992,v1:172.21.15.175:6819/1194026992] exists,up c8e75f15-df75-4d5d-9b92-a064bffb6c01 2024-03-18T23:07:56.578 INFO:teuthology.orchestra.run.smithi049.stdout:osd.9 up in weight 1 up_from 68 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.49:6826/2742084884,v1:172.21.15.49:6827/2742084884] [v2:172.21.15.49:6828/2742084884,v1:172.21.15.49:6829/2742084884] exists,up edb5ffef-b6a2-4941-b5ab-40f982ad30d5 2024-03-18T23:07:56.578 INFO:teuthology.orchestra.run.smithi049.stdout:osd.10 up in weight 1 up_from 73 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.139:6824/2404740201,v1:172.21.15.139:6825/2404740201] [v2:172.21.15.139:6826/2404740201,v1:172.21.15.139:6827/2404740201] exists,up 3945e46d-766a-4c4a-8821-d43648dd1578 2024-03-18T23:07:56.578 INFO:teuthology.orchestra.run.smithi049.stdout:osd.11 up in weight 1 up_from 81 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.175:6824/3519503868,v1:172.21.15.175:6825/3519503868] [v2:172.21.15.175:6826/3519503868,v1:172.21.15.175:6827/3519503868] exists,up 9954ba15-d877-46cc-b24a-893f9168deb8 2024-03-18T23:07:56.578 INFO:teuthology.orchestra.run.smithi049.stdout:pg_upmap_items 2.b [8,11] 2024-03-18T23:07:56.578 INFO:teuthology.orchestra.run.smithi049.stdout:pg_upmap_items 2.c [2,4] 2024-03-18T23:07:56.578 INFO:teuthology.orchestra.run.smithi049.stdout:pg_upmap_items 2.e [2,1] 2024-03-18T23:07:56.579 INFO:teuthology.orchestra.run.smithi049.stdout:pg_upmap_items 2.1c [8,11] 2024-03-18T23:07:56.579 INFO:teuthology.orchestra.run.smithi049.stdout:pg_upmap_items 3.6 [4,6] 2024-03-18T23:07:56.579 INFO:teuthology.orchestra.run.smithi049.stdout:pg_upmap_items 3.21 [5,9] 2024-03-18T23:07:56.579 INFO:teuthology.orchestra.run.smithi049.stdout:pg_upmap_items 3.2c [4,9] 2024-03-18T23:07:56.579 INFO:teuthology.orchestra.run.smithi049.stdout:pg_upmap_items 3.34 [4,9] 2024-03-18T23:07:56.579 INFO:teuthology.orchestra.run.smithi049.stdout:blocklist 172.21.15.49:0/563527416 expires 2024-03-19T22:58:59.806253+0000 2024-03-18T23:07:56.579 INFO:teuthology.orchestra.run.smithi049.stdout:blocklist 172.21.15.49:0/1036383534 expires 2024-03-19T22:58:59.806253+0000 2024-03-18T23:07:56.579 INFO:teuthology.orchestra.run.smithi049.stdout:blocklist 172.21.15.49:6801/2609458951 expires 2024-03-19T22:58:59.806253+0000 2024-03-18T23:07:56.579 INFO:teuthology.orchestra.run.smithi049.stdout:blocklist 172.21.15.49:0/21283102 expires 2024-03-19T22:58:31.972474+0000 2024-03-18T23:07:56.579 INFO:teuthology.orchestra.run.smithi049.stdout:blocklist 172.21.15.49:0/1292058922 expires 2024-03-19T22:58:31.972474+0000 2024-03-18T23:07:56.579 INFO:teuthology.orchestra.run.smithi049.stdout:blocklist 172.21.15.49:6800/1985190312 expires 2024-03-19T22:58:31.972474+0000 2024-03-18T23:07:56.579 INFO:teuthology.orchestra.run.smithi049.stdout:blocklist 172.21.15.49:0/1912446700 expires 2024-03-19T22:58:59.806253+0000 2024-03-18T23:07:56.579 INFO:teuthology.orchestra.run.smithi049.stdout:blocklist 172.21.15.49:0/2176210039 expires 2024-03-19T22:58:31.972474+0000 2024-03-18T23:07:56.579 INFO:teuthology.orchestra.run.smithi049.stdout:blocklist 172.21.15.49:6801/1985190312 expires 2024-03-19T22:58:31.972474+0000 2024-03-18T23:07:56.580 INFO:teuthology.orchestra.run.smithi049.stdout:blocklist 172.21.15.49:6800/2609458951 expires 2024-03-19T22:58:59.806253+0000 2024-03-18T23:07:56.580 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:56.578+0000 7f96e78ed700 1 -- 172.21.15.49:0/65601876 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f96cc0a2890 msgr2=0x7f96cc0a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:56.580 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:56.578+0000 7f96e78ed700 1 --2- 172.21.15.49:0/65601876 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f96cc0a2890 0x7f96cc0a4d50 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f96d0000c00 tx=0x7f96d0005fb0 comp rx=0 tx=0).stop 2024-03-18T23:07:56.580 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:56.578+0000 7f96e78ed700 1 -- 172.21.15.49:0/65601876 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f96e0103560 msgr2=0x7f96e0198ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:56.581 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:56.578+0000 7f96e78ed700 1 --2- 172.21.15.49:0/65601876 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f96e0103560 0x7f96e0198ab0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f96dc00ea20 tx=0x7f96dc00ed30 comp rx=0 tx=0).stop 2024-03-18T23:07:56.581 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:56.578+0000 7f96e78ed700 1 -- 172.21.15.49:0/65601876 shutdown_connections 2024-03-18T23:07:56.581 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:56.578+0000 7f96e78ed700 1 --2- 172.21.15.49:0/65601876 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f96e0103f00 0x7f96e019fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:56.581 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:56.578+0000 7f96e78ed700 1 --2- 172.21.15.49:0/65601876 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f96e0102360 0x7f96e0198570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:56.581 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:56.578+0000 7f96e78ed700 1 --2- 172.21.15.49:0/65601876 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f96cc0a2890 0x7f96cc0a4d50 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:56.581 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:56.578+0000 7f96e78ed700 1 --2- 172.21.15.49:0/65601876 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f96e0103560 0x7f96e0198ab0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:56.581 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:56.578+0000 7f96e78ed700 1 -- 172.21.15.49:0/65601876 >> 172.21.15.49:0/65601876 conn(0x7f96e00fd8c0 msgr2=0x7f96e01068b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:07:56.581 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:56.578+0000 7f96e78ed700 1 -- 172.21.15.49:0/65601876 shutdown_connections 2024-03-18T23:07:56.581 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:56.578+0000 7f96e78ed700 1 -- 172.21.15.49:0/65601876 wait complete. 2024-03-18T23:07:57.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:07:57 smithi139 ceph-mon[67282]: pgmap v346: 97 pgs: 97 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-03-18T23:07:57.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:07:57 smithi139 ceph-mon[67282]: from='client.? 172.21.15.49:0/65601876' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-03-18T23:07:57.324 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:07:57 smithi049 ceph-mon[62991]: pgmap v346: 97 pgs: 97 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-03-18T23:07:57.324 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:07:57 smithi049 ceph-mon[62991]: from='client.? 172.21.15.49:0/65601876' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-03-18T23:07:57.353 INFO:teuthology.run_tasks:Running task fs.ready... 2024-03-18T23:07:57.364 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-18T23:07:57.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:07:57 smithi175 ceph-mon[67352]: pgmap v346: 97 pgs: 97 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-03-18T23:07:57.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:07:57 smithi175 ceph-mon[67352]: from='client.? 172.21.15.49:0/65601876' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-03-18T23:07:57.479 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:57.477+0000 7f6e31c4b640 1 -- 172.21.15.49:0/2274347691 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6e2c1022f0 msgr2=0x7f6e2c108b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:57.479 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:57.477+0000 7f6e31c4b640 1 --2- 172.21.15.49:0/2274347691 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6e2c1022f0 0x7f6e2c108b80 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f6e1c009900 tx=0x7f6e1c02bc30 comp rx=0 tx=0).stop 2024-03-18T23:07:57.479 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:57.477+0000 7f6e31c4b640 1 -- 172.21.15.49:0/2274347691 shutdown_connections 2024-03-18T23:07:57.479 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:57.477+0000 7f6e31c4b640 1 --2- 172.21.15.49:0/2274347691 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6e2c1022f0 0x7f6e2c108b80 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:57.479 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:57.477+0000 7f6e31c4b640 1 --2- 172.21.15.49:0/2274347691 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6e2c101930 0x7f6e2c101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:57.479 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:57.477+0000 7f6e31c4b640 1 --2- 172.21.15.49:0/2274347691 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6e2c100740 0x7f6e2c100b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:57.479 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:57.477+0000 7f6e31c4b640 1 -- 172.21.15.49:0/2274347691 >> 172.21.15.49:0/2274347691 conn(0x7f6e2c0fbcb0 msgr2=0x7f6e2c0fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:07:57.479 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:57.477+0000 7f6e31c4b640 1 -- 172.21.15.49:0/2274347691 shutdown_connections 2024-03-18T23:07:57.479 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:57.477+0000 7f6e31c4b640 1 -- 172.21.15.49:0/2274347691 wait complete. 2024-03-18T23:07:57.479 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:57.477+0000 7f6e31c4b640 1 Processor -- start 2024-03-18T23:07:57.479 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:57.478+0000 7f6e31c4b640 1 -- start start 2024-03-18T23:07:57.480 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:57.478+0000 7f6e31c4b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6e2c100740 0x7f6e2c1958c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:57.480 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:57.478+0000 7f6e31c4b640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6e2c101930 0x7f6e2c195e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:57.480 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:57.478+0000 7f6e31c4b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6e2c1022f0 0x7f6e2c19ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:57.480 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:57.478+0000 7f6e31c4b640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f6e2c10b2d0 con 0x7f6e2c1022f0 2024-03-18T23:07:57.480 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:57.478+0000 7f6e31c4b640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f6e2c10b150 con 0x7f6e2c100740 2024-03-18T23:07:57.480 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:57.478+0000 7f6e31c4b640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f6e2c10b450 con 0x7f6e2c101930 2024-03-18T23:07:57.480 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:57.478+0000 7f6e2bfff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6e2c1022f0 0x7f6e2c19ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:57.480 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:57.478+0000 7f6e2bfff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6e2c1022f0 0x7f6e2c19ce70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:37524/0 (socket says 172.21.15.49:37524) 2024-03-18T23:07:57.480 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:57.478+0000 7f6e2bfff640 1 -- 172.21.15.49:0/1186157477 learned_addr learned my addr 172.21.15.49:0/1186157477 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:07:57.480 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:57.478+0000 7f6e2b7fe640 1 --2- 172.21.15.49:0/1186157477 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6e2c100740 0x7f6e2c1958c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:57.480 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:57.478+0000 7f6e2affd640 1 --2- 172.21.15.49:0/1186157477 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6e2c101930 0x7f6e2c195e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:57.480 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:57.478+0000 7f6e2bfff640 1 -- 172.21.15.49:0/1186157477 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6e2c101930 msgr2=0x7f6e2c195e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:57.480 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:57.478+0000 7f6e2bfff640 1 --2- 172.21.15.49:0/1186157477 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6e2c101930 0x7f6e2c195e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:57.480 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:57.478+0000 7f6e2bfff640 1 -- 172.21.15.49:0/1186157477 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6e2c100740 msgr2=0x7f6e2c1958c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:57.481 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:57.478+0000 7f6e2bfff640 1 --2- 172.21.15.49:0/1186157477 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6e2c100740 0x7f6e2c1958c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:57.481 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:57.478+0000 7f6e2bfff640 1 -- 172.21.15.49:0/1186157477 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e2c19d4e0 con 0x7f6e2c1022f0 2024-03-18T23:07:57.481 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:57.478+0000 7f6e2bfff640 1 --2- 172.21.15.49:0/1186157477 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6e2c1022f0 0x7f6e2c19ce70 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f6e1c005ab0 tx=0x7f6e1c02c6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:07:57.481 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:57.479+0000 7f6e28ff9640 1 -- 172.21.15.49:0/1186157477 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e1c041070 con 0x7f6e2c1022f0 2024-03-18T23:07:57.481 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:57.479+0000 7f6e28ff9640 1 -- 172.21.15.49:0/1186157477 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6e1c004450 con 0x7f6e2c1022f0 2024-03-18T23:07:57.481 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:57.479+0000 7f6e31c4b640 1 -- 172.21.15.49:0/1186157477 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e2c19d770 con 0x7f6e2c1022f0 2024-03-18T23:07:57.481 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:57.479+0000 7f6e28ff9640 1 -- 172.21.15.49:0/1186157477 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e1c003e60 con 0x7f6e2c1022f0 2024-03-18T23:07:57.481 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:57.479+0000 7f6e31c4b640 1 -- 172.21.15.49:0/1186157477 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e2c19dc60 con 0x7f6e2c1022f0 2024-03-18T23:07:57.482 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:57.480+0000 7f6e31c4b640 1 -- 172.21.15.49:0/1186157477 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e2c107a60 con 0x7f6e2c1022f0 2024-03-18T23:07:57.482 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:57.481+0000 7f6e28ff9640 1 -- 172.21.15.49:0/1186157477 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7f6e1c044070 con 0x7f6e2c1022f0 2024-03-18T23:07:57.482 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:57.481+0000 7f6e28ff9640 1 --2- 172.21.15.49:0/1186157477 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f6e180a2800 0x7f6e180a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:57.482 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:57.481+0000 7f6e2b7fe640 1 --2- 172.21.15.49:0/1186157477 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f6e180a2800 0x7f6e180a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:57.482 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:57.481+0000 7f6e28ff9640 1 -- 172.21.15.49:0/1186157477 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f6e1c0f0670 con 0x7f6e2c1022f0 2024-03-18T23:07:57.482 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:57.482+0000 7f6e2b7fe640 1 --2- 172.21.15.49:0/1186157477 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f6e180a2800 0x7f6e180a4cc0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f6e2c101790 tx=0x7f6e24011040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:07:57.485 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:57.486+0000 7f6e28ff9640 1 -- 172.21.15.49:0/1186157477 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f6e1c0bd0d0 con 0x7f6e2c1022f0 2024-03-18T23:07:57.670 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:57.670+0000 7f6e31c4b640 1 -- 172.21.15.49:0/1186157477 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6e2c062420 con 0x7f6e2c1022f0 2024-03-18T23:07:57.672 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:57.672+0000 7f6e28ff9640 1 -- 172.21.15.49:0/1186157477 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+22674 (secure 0 0 0) 0x7f6e1c0c1170 con 0x7f6e2c1022f0 2024-03-18T23:07:57.672 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-18T23:07:57.673 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":88,"fsid":"c63591ea-e57a-11ee-95c9-87774f69a715","created":"2024-03-18T22:57:58.734467+0000","modified":"2024-03-18T23:07:01.337701+0000","last_up_change":"2024-03-18T23:05:16.576310+0000","last_in_change":"2024-03-18T23:04:57.024067+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-18T23:01:28.363432+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-18T23:06:03.172963+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-18T23:06:03.973303+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e7fc9daa-4098-4ba5-8019-9aaef1aab3e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":1651162501},{"type":"v1","addr":"172.21.15.49:6803","nonce":1651162501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":1651162501},{"type":"v1","addr":"172.21.15.49:6805","nonce":1651162501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":1651162501},{"type":"v1","addr":"172.21.15.49:6809","nonce":1651162501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":1651162501},{"type":"v1","addr":"172.21.15.49:6807","nonce":1651162501}]},"public_addr":"172.21.15.49:6803/1651162501","cluster_addr":"172.21.15.49:6805/1651162501","heartbeat_back_addr":"172.21.15.49:6809/1651162501","heartbeat_front_addr":"172.21.15.49:6807/1651162501","state":["exists","up"]},{"osd":1,"uuid":"40ac929c-e90f-434c-a796-9aabd686f1fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":1766700359},{"type":"v1","addr":"172.21.15.139:6801","nonce":1766700359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":1766700359},{"type":"v1","addr":"172.21.15.139:6803","nonce":1766700359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":1766700359},{"type":"v1","addr":"172.21.15.139:6807","nonce":1766700359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":1766700359},{"type":"v1","addr":"172.21.15.139:6805","nonce":1766700359}]},"public_addr":"172.21.15.139:6801/1766700359","cluster_addr":"172.21.15.139:6803/1766700359","heartbeat_back_addr":"172.21.15.139:6807/1766700359","heartbeat_front_addr":"172.21.15.139:6805/1766700359","state":["exists","up"]},{"osd":2,"uuid":"8ae3b666-00b4-45ed-a5ee-4d0771edabb6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":3752671234},{"type":"v1","addr":"172.21.15.175:6801","nonce":3752671234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":3752671234},{"type":"v1","addr":"172.21.15.175:6803","nonce":3752671234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":3752671234},{"type":"v1","addr":"172.21.15.175:6807","nonce":3752671234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":3752671234},{"type":"v1","addr":"172.21.15.175:6805","nonce":3752671234}]},"public_addr":"172.21.15.175:6801/3752671234","cluster_addr":"172.21.15.175:6803/3752671234","heartbeat_back_addr":"172.21.15.175:6807/3752671234","heartbeat_front_addr":"172.21.15.175:6805/3752671234","state":["exists","up"]},{"osd":3,"uuid":"1a62e89d-45d4-4237-b813-7f96a59aca7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":956160536},{"type":"v1","addr":"172.21.15.49:6811","nonce":956160536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":956160536},{"type":"v1","addr":"172.21.15.49:6813","nonce":956160536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":956160536},{"type":"v1","addr":"172.21.15.49:6817","nonce":956160536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":956160536},{"type":"v1","addr":"172.21.15.49:6815","nonce":956160536}]},"public_addr":"172.21.15.49:6811/956160536","cluster_addr":"172.21.15.49:6813/956160536","heartbeat_back_addr":"172.21.15.49:6817/956160536","heartbeat_front_addr":"172.21.15.49:6815/956160536","state":["exists","up"]},{"osd":4,"uuid":"c715101c-0c3f-4b98-b142-d64a35e187df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":3129712668},{"type":"v1","addr":"172.21.15.139:6809","nonce":3129712668}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":3129712668},{"type":"v1","addr":"172.21.15.139:6811","nonce":3129712668}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":3129712668},{"type":"v1","addr":"172.21.15.139:6815","nonce":3129712668}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":3129712668},{"type":"v1","addr":"172.21.15.139:6813","nonce":3129712668}]},"public_addr":"172.21.15.139:6809/3129712668","cluster_addr":"172.21.15.139:6811/3129712668","heartbeat_back_addr":"172.21.15.139:6815/3129712668","heartbeat_front_addr":"172.21.15.139:6813/3129712668","state":["exists","up"]},{"osd":5,"uuid":"4b641f81-77eb-45fd-8464-511e7b3062b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":2774567215},{"type":"v1","addr":"172.21.15.175:6809","nonce":2774567215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":2774567215},{"type":"v1","addr":"172.21.15.175:6811","nonce":2774567215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":2774567215},{"type":"v1","addr":"172.21.15.175:6815","nonce":2774567215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":2774567215},{"type":"v1","addr":"172.21.15.175:6813","nonce":2774567215}]},"public_addr":"172.21.15.175:6809/2774567215","cluster_addr":"172.21.15.175:6811/2774567215","heartbeat_back_addr":"172.21.15.175:6815/2774567215","heartbeat_front_addr":"172.21.15.175:6813/2774567215","state":["exists","up"]},{"osd":6,"uuid":"653da8e0-1197-4b48-abed-470f2a9c4dce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":1472215461},{"type":"v1","addr":"172.21.15.49:6819","nonce":1472215461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":1472215461},{"type":"v1","addr":"172.21.15.49:6821","nonce":1472215461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":1472215461},{"type":"v1","addr":"172.21.15.49:6825","nonce":1472215461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":1472215461},{"type":"v1","addr":"172.21.15.49:6823","nonce":1472215461}]},"public_addr":"172.21.15.49:6819/1472215461","cluster_addr":"172.21.15.49:6821/1472215461","heartbeat_back_addr":"172.21.15.49:6825/1472215461","heartbeat_front_addr":"172.21.15.49:6823/1472215461","state":["exists","up"]},{"osd":7,"uuid":"ad157c9b-4b75-451c-8740-630f0975b7fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":2163986787},{"type":"v1","addr":"172.21.15.139:6817","nonce":2163986787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":2163986787},{"type":"v1","addr":"172.21.15.139:6819","nonce":2163986787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":2163986787},{"type":"v1","addr":"172.21.15.139:6823","nonce":2163986787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":2163986787},{"type":"v1","addr":"172.21.15.139:6821","nonce":2163986787}]},"public_addr":"172.21.15.139:6817/2163986787","cluster_addr":"172.21.15.139:6819/2163986787","heartbeat_back_addr":"172.21.15.139:6823/2163986787","heartbeat_front_addr":"172.21.15.139:6821/2163986787","state":["exists","up"]},{"osd":8,"uuid":"c8e75f15-df75-4d5d-9b92-a064bffb6c01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":1194026992},{"type":"v1","addr":"172.21.15.175:6817","nonce":1194026992}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1194026992},{"type":"v1","addr":"172.21.15.175:6819","nonce":1194026992}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":1194026992},{"type":"v1","addr":"172.21.15.175:6823","nonce":1194026992}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1194026992},{"type":"v1","addr":"172.21.15.175:6821","nonce":1194026992}]},"public_addr":"172.21.15.175:6817/1194026992","cluster_addr":"172.21.15.175:6819/1194026992","heartbeat_back_addr":"172.21.15.175:6823/1194026992","heartbeat_front_addr":"172.21.15.175:6821/1194026992","state":["exists","up"]},{"osd":9,"uuid":"edb5ffef-b6a2-4941-b5ab-40f982ad30d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":2742084884},{"type":"v1","addr":"172.21.15.49:6827","nonce":2742084884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":2742084884},{"type":"v1","addr":"172.21.15.49:6829","nonce":2742084884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":2742084884},{"type":"v1","addr":"172.21.15.49:6833","nonce":2742084884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":2742084884},{"type":"v1","addr":"172.21.15.49:6831","nonce":2742084884}]},"public_addr":"172.21.15.49:6827/2742084884","cluster_addr":"172.21.15.49:6829/2742084884","heartbeat_back_addr":"172.21.15.49:6833/2742084884","heartbeat_front_addr":"172.21.15.49:6831/2742084884","state":["exists","up"]},{"osd":10,"uuid":"3945e46d-766a-4c4a-8821-d43648dd1578","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":2404740201},{"type":"v1","addr":"172.21.15.139:6825","nonce":2404740201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":2404740201},{"type":"v1","addr":"172.21.15.139:6827","nonce":2404740201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":2404740201},{"type":"v1","addr":"172.21.15.139:6831","nonce":2404740201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":2404740201},{"type":"v1","addr":"172.21.15.139:6829","nonce":2404740201}]},"public_addr":"172.21.15.139:6825/2404740201","cluster_addr":"172.21.15.139:6827/2404740201","heartbeat_back_addr":"172.21.15.139:6831/2404740201","heartbeat_front_addr":"172.21.15.139:6829/2404740201","state":["exists","up"]},{"osd":11,"uuid":"9954ba15-d877-46cc-b24a-893f9168deb8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":3519503868},{"type":"v1","addr":"172.21.15.175:6825","nonce":3519503868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":3519503868},{"type":"v1","addr":"172.21.15.175:6827","nonce":3519503868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":3519503868},{"type":"v1","addr":"172.21.15.175:6831","nonce":3519503868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":3519503868},{"type":"v1","addr":"172.21.15.175:6829","nonce":3519503868}]},"public_addr":"172.21.15.175:6825/3519503868","cluster_addr":"172.21.15.175:6827/3519503868","heartbeat_back_addr":"172.21.15.175:6831/3519503868","heartbeat_front_addr":"172.21.15.175:6829/3519503868","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:00:38.367865+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:01:02.237687+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:01:25.609751+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:01:51.268572+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:02:16.514605+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:02:41.586523+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:03:07.029964+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:03:32.956250+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:03:57.547984+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:04:23.948972+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:04:49.667210+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:05:15.284693+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.6","mappings":[{"from":4,"to":6}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.49:0/563527416":"2024-03-19T22:58:59.806253+0000","172.21.15.49:0/1036383534":"2024-03-19T22:58:59.806253+0000","172.21.15.49:6801/2609458951":"2024-03-19T22:58:59.806253+0000","172.21.15.49:0/21283102":"2024-03-19T22:58:31.972474+0000","172.21.15.49:0/1292058922":"2024-03-19T22:58:31.972474+0000","172.21.15.49:6800/1985190312":"2024-03-19T22:58:31.972474+0000","172.21.15.49:0/1912446700":"2024-03-19T22:58:59.806253+0000","172.21.15.49:0/2176210039":"2024-03-19T22:58:31.972474+0000","172.21.15.49:6801/1985190312":"2024-03-19T22:58:31.972474+0000","172.21.15.49:6800/2609458951":"2024-03-19T22:58:59.806253+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-18T23:07:57.674 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:57.674+0000 7f6e31c4b640 1 -- 172.21.15.49:0/1186157477 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f6e180a2800 msgr2=0x7f6e180a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:57.674 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:57.674+0000 7f6e31c4b640 1 --2- 172.21.15.49:0/1186157477 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f6e180a2800 0x7f6e180a4cc0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f6e2c101790 tx=0x7f6e24011040 comp rx=0 tx=0).stop 2024-03-18T23:07:57.674 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:57.674+0000 7f6e31c4b640 1 -- 172.21.15.49:0/1186157477 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6e2c1022f0 msgr2=0x7f6e2c19ce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:57.674 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:57.674+0000 7f6e31c4b640 1 --2- 172.21.15.49:0/1186157477 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6e2c1022f0 0x7f6e2c19ce70 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f6e1c005ab0 tx=0x7f6e1c02c6c0 comp rx=0 tx=0).stop 2024-03-18T23:07:57.674 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:57.674+0000 7f6e31c4b640 1 -- 172.21.15.49:0/1186157477 shutdown_connections 2024-03-18T23:07:57.675 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:57.674+0000 7f6e31c4b640 1 --2- 172.21.15.49:0/1186157477 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f6e180a2800 0x7f6e180a4cc0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:57.675 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:57.674+0000 7f6e31c4b640 1 --2- 172.21.15.49:0/1186157477 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6e2c1022f0 0x7f6e2c19ce70 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:57.675 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:57.674+0000 7f6e31c4b640 1 --2- 172.21.15.49:0/1186157477 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6e2c101930 0x7f6e2c195e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:57.675 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:57.674+0000 7f6e31c4b640 1 --2- 172.21.15.49:0/1186157477 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6e2c100740 0x7f6e2c1958c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:57.675 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:57.674+0000 7f6e31c4b640 1 -- 172.21.15.49:0/1186157477 >> 172.21.15.49:0/1186157477 conn(0x7f6e2c0fbcb0 msgr2=0x7f6e2c104c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:07:57.675 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:57.674+0000 7f6e31c4b640 1 -- 172.21.15.49:0/1186157477 shutdown_connections 2024-03-18T23:07:57.675 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:57.674+0000 7f6e31c4b640 1 -- 172.21.15.49:0/1186157477 wait complete. 2024-03-18T23:07:57.685 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-18T23:01:28.363432+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-18T23:06:03.172963+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-18T23:06:03.973303+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-18T23:07:57.686 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-18T23:07:57.801 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:57.797+0000 7f88d82c6640 1 -- 172.21.15.49:0/1370666355 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f88d0100720 msgr2=0x7f88d0100b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:57.801 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:57.797+0000 7f88d82c6640 1 --2- 172.21.15.49:0/1370666355 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f88d0100720 0x7f88d0100b40 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f88cc009ca0 tx=0x7f88cc02c0f0 comp rx=0 tx=0).stop 2024-03-18T23:07:57.802 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:57.798+0000 7f88d82c6640 1 -- 172.21.15.49:0/1370666355 shutdown_connections 2024-03-18T23:07:57.802 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:57.798+0000 7f88d82c6640 1 --2- 172.21.15.49:0/1370666355 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f88d01022d0 0x7f88d0108b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:57.802 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:57.798+0000 7f88d82c6640 1 --2- 172.21.15.49:0/1370666355 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f88d0101910 0x7f88d0101d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:57.802 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:57.798+0000 7f88d82c6640 1 --2- 172.21.15.49:0/1370666355 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f88d0100720 0x7f88d0100b40 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:57.802 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:57.798+0000 7f88d82c6640 1 -- 172.21.15.49:0/1370666355 >> 172.21.15.49:0/1370666355 conn(0x7f88d00fbcb0 msgr2=0x7f88d00fe0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:07:57.802 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:57.798+0000 7f88d82c6640 1 -- 172.21.15.49:0/1370666355 shutdown_connections 2024-03-18T23:07:57.802 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:57.798+0000 7f88d82c6640 1 -- 172.21.15.49:0/1370666355 wait complete. 2024-03-18T23:07:57.802 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:57.798+0000 7f88d82c6640 1 Processor -- start 2024-03-18T23:07:57.802 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:57.798+0000 7f88d82c6640 1 -- start start 2024-03-18T23:07:57.802 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:57.798+0000 7f88d82c6640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f88d0100720 0x7f88d00694f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:57.802 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:57.798+0000 7f88d82c6640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f88d0101910 0x7f88d019c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:57.802 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:57.798+0000 7f88d82c6640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f88d01022d0 0x7f88d019eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:57.802 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:57.798+0000 7f88d82c6640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f88d010c3a0 con 0x7f88d0100720 2024-03-18T23:07:57.803 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:57.798+0000 7f88d82c6640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f88d010c220 con 0x7f88d01022d0 2024-03-18T23:07:57.803 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:57.798+0000 7f88d82c6640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f88d010c520 con 0x7f88d0101910 2024-03-18T23:07:57.803 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:57.798+0000 7f88d6062640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f88d0100720 0x7f88d00694f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:57.803 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:57.798+0000 7f88d6062640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f88d0100720 0x7f88d00694f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:37536/0 (socket says 172.21.15.49:37536) 2024-03-18T23:07:57.803 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:57.798+0000 7f88d6062640 1 -- 172.21.15.49:0/1127108568 learned_addr learned my addr 172.21.15.49:0/1127108568 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:07:57.803 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:57.798+0000 7f88d5861640 1 --2- 172.21.15.49:0/1127108568 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f88d0101910 0x7f88d019c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:57.803 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:57.798+0000 7f88d6863640 1 --2- 172.21.15.49:0/1127108568 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f88d01022d0 0x7f88d019eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:57.803 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:57.799+0000 7f88d6062640 1 -- 172.21.15.49:0/1127108568 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f88d0101910 msgr2=0x7f88d019c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:57.803 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:57.799+0000 7f88d6062640 1 --2- 172.21.15.49:0/1127108568 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f88d0101910 0x7f88d019c8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:57.803 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:57.799+0000 7f88d6062640 1 -- 172.21.15.49:0/1127108568 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f88d01022d0 msgr2=0x7f88d019eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:57.803 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:57.799+0000 7f88d6062640 1 --2- 172.21.15.49:0/1127108568 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f88d01022d0 0x7f88d019eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:57.803 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:57.799+0000 7f88d6062640 1 -- 172.21.15.49:0/1127108568 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88d019f7d0 con 0x7f88d0100720 2024-03-18T23:07:57.803 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:57.799+0000 7f88d5861640 1 --2- 172.21.15.49:0/1127108568 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f88d0101910 0x7f88d019c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T23:07:57.803 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:57.799+0000 7f88d6062640 1 --2- 172.21.15.49:0/1127108568 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f88d0100720 0x7f88d00694f0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f88cc0324c0 tx=0x7f88cc032f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:07:57.803 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:57.799+0000 7f88c77fe640 1 -- 172.21.15.49:0/1127108568 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88cc041020 con 0x7f88d0100720 2024-03-18T23:07:57.804 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:57.799+0000 7f88c77fe640 1 -- 172.21.15.49:0/1127108568 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f88cc034cd0 con 0x7f88d0100720 2024-03-18T23:07:57.804 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:57.799+0000 7f88c77fe640 1 -- 172.21.15.49:0/1127108568 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88cc039040 con 0x7f88d0100720 2024-03-18T23:07:57.804 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:57.799+0000 7f88d82c6640 1 -- 172.21.15.49:0/1127108568 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88d019fa00 con 0x7f88d0100720 2024-03-18T23:07:57.804 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:57.799+0000 7f88d82c6640 1 -- 172.21.15.49:0/1127108568 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88d019fea0 con 0x7f88d0100720 2024-03-18T23:07:57.804 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:57.800+0000 7f88d82c6640 1 -- 172.21.15.49:0/1127108568 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88d0107ab0 con 0x7f88d0100720 2024-03-18T23:07:57.805 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:57.801+0000 7f88c77fe640 1 -- 172.21.15.49:0/1127108568 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7f88cc044020 con 0x7f88d0100720 2024-03-18T23:07:57.805 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:57.802+0000 7f88c77fe640 1 --2- 172.21.15.49:0/1127108568 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f88bc0a2800 0x7f88bc0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:57.805 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:57.802+0000 7f88d5861640 1 --2- 172.21.15.49:0/1127108568 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f88bc0a2800 0x7f88bc0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:57.805 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:57.802+0000 7f88c77fe640 1 -- 172.21.15.49:0/1127108568 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f88cc0f0950 con 0x7f88d0100720 2024-03-18T23:07:57.805 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:57.802+0000 7f88d5861640 1 --2- 172.21.15.49:0/1127108568 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f88bc0a2800 0x7f88bc0a4cc0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f88c0005d00 tx=0x7f88c0005c70 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:07:57.806 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:57.806+0000 7f88c77fe640 1 -- 172.21.15.49:0/1127108568 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f88cc04b020 con 0x7f88d0100720 2024-03-18T23:07:57.988 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:57.988+0000 7f88d82c6640 1 -- 172.21.15.49:0/1127108568 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f88d00624c0 con 0x7f88d0100720 2024-03-18T23:07:57.989 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:57.989+0000 7f88c77fe640 1 -- 172.21.15.49:0/1127108568 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) ==== 93+0+10 (secure 0 0 0) 0x7f88cc0bda90 con 0x7f88d0100720 2024-03-18T23:07:57.989 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 1 2024-03-18T23:07:57.991 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:57.991+0000 7f88d82c6640 1 -- 172.21.15.49:0/1127108568 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f88bc0a2800 msgr2=0x7f88bc0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:57.991 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:57.991+0000 7f88d82c6640 1 --2- 172.21.15.49:0/1127108568 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f88bc0a2800 0x7f88bc0a4cc0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f88c0005d00 tx=0x7f88c0005c70 comp rx=0 tx=0).stop 2024-03-18T23:07:57.991 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:57.991+0000 7f88d82c6640 1 -- 172.21.15.49:0/1127108568 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f88d0100720 msgr2=0x7f88d00694f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:57.991 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:57.991+0000 7f88d82c6640 1 --2- 172.21.15.49:0/1127108568 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f88d0100720 0x7f88d00694f0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f88cc0324c0 tx=0x7f88cc032f40 comp rx=0 tx=0).stop 2024-03-18T23:07:57.991 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:57.991+0000 7f88d82c6640 1 -- 172.21.15.49:0/1127108568 shutdown_connections 2024-03-18T23:07:57.991 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:57.991+0000 7f88d82c6640 1 --2- 172.21.15.49:0/1127108568 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f88bc0a2800 0x7f88bc0a4cc0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:57.991 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:57.991+0000 7f88d82c6640 1 --2- 172.21.15.49:0/1127108568 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f88d01022d0 0x7f88d019eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:57.991 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:57.991+0000 7f88d82c6640 1 --2- 172.21.15.49:0/1127108568 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f88d0101910 0x7f88d019c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:57.991 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:57.991+0000 7f88d82c6640 1 --2- 172.21.15.49:0/1127108568 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f88d0100720 0x7f88d00694f0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:57.991 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:57.991+0000 7f88d82c6640 1 -- 172.21.15.49:0/1127108568 >> 172.21.15.49:0/1127108568 conn(0x7f88d00fbcb0 msgr2=0x7f88d0104c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:07:57.992 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:57.991+0000 7f88d82c6640 1 -- 172.21.15.49:0/1127108568 shutdown_connections 2024-03-18T23:07:57.992 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:57.991+0000 7f88d82c6640 1 -- 172.21.15.49:0/1127108568 wait complete. 2024-03-18T23:07:58.002 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-18T23:07:58.116 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:58.114+0000 7f41f288c640 1 -- 172.21.15.49:0/2924809397 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f41ec1022f0 msgr2=0x7f41ec108b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:58.116 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:58.114+0000 7f41f288c640 1 --2- 172.21.15.49:0/2924809397 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f41ec1022f0 0x7f41ec108b80 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f41e4009ca0 tx=0x7f41e402c0f0 comp rx=0 tx=0).stop 2024-03-18T23:07:58.116 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:58.114+0000 7f41f288c640 1 -- 172.21.15.49:0/2924809397 shutdown_connections 2024-03-18T23:07:58.117 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:58.114+0000 7f41f288c640 1 --2- 172.21.15.49:0/2924809397 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f41ec1022f0 0x7f41ec108b80 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:58.117 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:58.114+0000 7f41f288c640 1 --2- 172.21.15.49:0/2924809397 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f41ec101930 0x7f41ec101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:58.117 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:58.114+0000 7f41f288c640 1 --2- 172.21.15.49:0/2924809397 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f41ec100740 0x7f41ec100b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:58.117 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:58.114+0000 7f41f288c640 1 -- 172.21.15.49:0/2924809397 >> 172.21.15.49:0/2924809397 conn(0x7f41ec0fbcd0 msgr2=0x7f41ec0fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:07:58.117 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:58.114+0000 7f41f288c640 1 -- 172.21.15.49:0/2924809397 shutdown_connections 2024-03-18T23:07:58.117 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:58.114+0000 7f41f288c640 1 -- 172.21.15.49:0/2924809397 wait complete. 2024-03-18T23:07:58.117 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:58.114+0000 7f41f288c640 1 Processor -- start 2024-03-18T23:07:58.117 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:58.114+0000 7f41f288c640 1 -- start start 2024-03-18T23:07:58.117 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:58.115+0000 7f41f288c640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f41ec100740 0x7f41ec10b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:58.117 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:58.115+0000 7f41f288c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f41ec101930 0x7f41ec109700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:58.117 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:58.115+0000 7f41f288c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f41ec1022f0 0x7f41ec109c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:58.117 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:58.115+0000 7f41f288c640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f41ec10d4e0 con 0x7f41ec101930 2024-03-18T23:07:58.118 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:58.115+0000 7f41f288c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f41ec10d360 con 0x7f41ec1022f0 2024-03-18T23:07:58.118 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:58.115+0000 7f41f288c640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f41ec10d660 con 0x7f41ec100740 2024-03-18T23:07:58.118 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:58.115+0000 7f41eb7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f41ec101930 0x7f41ec109700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:58.118 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:58.115+0000 7f41eb7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f41ec101930 0x7f41ec109700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:37564/0 (socket says 172.21.15.49:37564) 2024-03-18T23:07:58.118 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:58.115+0000 7f41eb7fe640 1 -- 172.21.15.49:0/1100434796 learned_addr learned my addr 172.21.15.49:0/1100434796 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:07:58.118 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:58.115+0000 7f41f0e29640 1 --2- 172.21.15.49:0/1100434796 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f41ec1022f0 0x7f41ec109c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:58.118 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:58.115+0000 7f41ebfff640 1 --2- 172.21.15.49:0/1100434796 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f41ec100740 0x7f41ec10b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:58.118 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:58.115+0000 7f41eb7fe640 1 -- 172.21.15.49:0/1100434796 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f41ec100740 msgr2=0x7f41ec10b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:58.118 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:58.115+0000 7f41eb7fe640 1 --2- 172.21.15.49:0/1100434796 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f41ec100740 0x7f41ec10b0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:58.118 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:58.115+0000 7f41eb7fe640 1 -- 172.21.15.49:0/1100434796 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f41ec1022f0 msgr2=0x7f41ec109c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:58.118 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:58.115+0000 7f41eb7fe640 1 --2- 172.21.15.49:0/1100434796 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f41ec1022f0 0x7f41ec109c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:58.118 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:58.115+0000 7f41eb7fe640 1 -- 172.21.15.49:0/1100434796 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41ec10a4d0 con 0x7f41ec101930 2024-03-18T23:07:58.119 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:58.115+0000 7f41eb7fe640 1 --2- 172.21.15.49:0/1100434796 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f41ec101930 0x7f41ec109700 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f41e000ea60 tx=0x7f41e000ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:07:58.119 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:58.116+0000 7f41e97fa640 1 -- 172.21.15.49:0/1100434796 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41e000cde0 con 0x7f41ec101930 2024-03-18T23:07:58.119 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:58.116+0000 7f41e97fa640 1 -- 172.21.15.49:0/1100434796 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f41e00035f0 con 0x7f41ec101930 2024-03-18T23:07:58.119 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:58.116+0000 7f41f288c640 1 -- 172.21.15.49:0/1100434796 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41ec1a1c00 con 0x7f41ec101930 2024-03-18T23:07:58.119 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:58.116+0000 7f41e97fa640 1 -- 172.21.15.49:0/1100434796 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41e0010670 con 0x7f41ec101930 2024-03-18T23:07:58.119 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:58.116+0000 7f41f288c640 1 -- 172.21.15.49:0/1100434796 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41ec1a2150 con 0x7f41ec101930 2024-03-18T23:07:58.120 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:58.117+0000 7f41f288c640 1 -- 172.21.15.49:0/1100434796 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41ec107a60 con 0x7f41ec101930 2024-03-18T23:07:58.120 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:58.118+0000 7f41e97fa640 1 -- 172.21.15.49:0/1100434796 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7f41e00037a0 con 0x7f41ec101930 2024-03-18T23:07:58.120 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:58.118+0000 7f41e97fa640 1 --2- 172.21.15.49:0/1100434796 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f41d40a2800 0x7f41d40a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:58.120 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:58.119+0000 7f41ebfff640 1 --2- 172.21.15.49:0/1100434796 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f41d40a2800 0x7f41d40a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:58.120 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:58.119+0000 7f41e97fa640 1 -- 172.21.15.49:0/1100434796 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f41e00ced90 con 0x7f41ec101930 2024-03-18T23:07:58.120 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:58.119+0000 7f41ebfff640 1 --2- 172.21.15.49:0/1100434796 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f41d40a2800 0x7f41d40a4cc0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f41ec101790 tx=0x7f41dc005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:07:58.123 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:58.123+0000 7f41e97fa640 1 -- 172.21.15.49:0/1100434796 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f41e009be40 con 0x7f41ec101930 2024-03-18T23:07:58.308 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:58.308+0000 7f41f288c640 1 -- 172.21.15.49:0/1100434796 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f41ec0624c0 con 0x7f41ec101930 2024-03-18T23:07:58.308 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:58.308+0000 7f41e97fa640 1 -- 172.21.15.49:0/1100434796 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) ==== 104+0+11 (secure 0 0 0) 0x7f41e009b740 con 0x7f41ec101930 2024-03-18T23:07:58.308 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 32 2024-03-18T23:07:58.311 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:58.310+0000 7f41f288c640 1 -- 172.21.15.49:0/1100434796 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f41d40a2800 msgr2=0x7f41d40a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:58.311 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:58.310+0000 7f41f288c640 1 --2- 172.21.15.49:0/1100434796 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f41d40a2800 0x7f41d40a4cc0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f41ec101790 tx=0x7f41dc005f90 comp rx=0 tx=0).stop 2024-03-18T23:07:58.311 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:58.310+0000 7f41f288c640 1 -- 172.21.15.49:0/1100434796 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f41ec101930 msgr2=0x7f41ec109700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:58.311 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:58.310+0000 7f41f288c640 1 --2- 172.21.15.49:0/1100434796 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f41ec101930 0x7f41ec109700 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f41e000ea60 tx=0x7f41e000ef20 comp rx=0 tx=0).stop 2024-03-18T23:07:58.311 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:58.310+0000 7f41f288c640 1 -- 172.21.15.49:0/1100434796 shutdown_connections 2024-03-18T23:07:58.311 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:58.310+0000 7f41f288c640 1 --2- 172.21.15.49:0/1100434796 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f41d40a2800 0x7f41d40a4cc0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:58.311 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:58.310+0000 7f41f288c640 1 --2- 172.21.15.49:0/1100434796 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f41ec1022f0 0x7f41ec109c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:58.311 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:58.310+0000 7f41f288c640 1 --2- 172.21.15.49:0/1100434796 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f41ec101930 0x7f41ec109700 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:58.311 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:58.310+0000 7f41f288c640 1 --2- 172.21.15.49:0/1100434796 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f41ec100740 0x7f41ec10b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:58.311 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:58.310+0000 7f41f288c640 1 -- 172.21.15.49:0/1100434796 >> 172.21.15.49:0/1100434796 conn(0x7f41ec0fbcd0 msgr2=0x7f41ec104c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:07:58.311 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:58.310+0000 7f41f288c640 1 -- 172.21.15.49:0/1100434796 shutdown_connections 2024-03-18T23:07:58.312 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:58.310+0000 7f41f288c640 1 -- 172.21.15.49:0/1100434796 wait complete. 2024-03-18T23:07:58.320 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:07:58 smithi049 ceph-mon[62991]: from='client.? 172.21.15.49:0/1186157477' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T23:07:58.321 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:07:58 smithi049 ceph-mon[62991]: from='client.? 172.21.15.49:0/1127108568' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-18T23:07:58.321 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-18T23:07:58.433 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:58.433+0000 7f09c1da9640 1 -- 172.21.15.49:0/2103743378 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f09bc1022f0 msgr2=0x7f09bc108b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:58.433 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:58.433+0000 7f09c1da9640 1 --2- 172.21.15.49:0/2103743378 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f09bc1022f0 0x7f09bc108b80 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f09b4009ca0 tx=0x7f09b402c0f0 comp rx=0 tx=0).stop 2024-03-18T23:07:58.433 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:58.433+0000 7f09c1da9640 1 -- 172.21.15.49:0/2103743378 shutdown_connections 2024-03-18T23:07:58.433 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:58.433+0000 7f09c1da9640 1 --2- 172.21.15.49:0/2103743378 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f09bc1022f0 0x7f09bc108b80 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:58.433 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:58.433+0000 7f09c1da9640 1 --2- 172.21.15.49:0/2103743378 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f09bc101930 0x7f09bc101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:58.433 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:58.433+0000 7f09c1da9640 1 --2- 172.21.15.49:0/2103743378 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f09bc100740 0x7f09bc100b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:58.433 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:58.433+0000 7f09c1da9640 1 -- 172.21.15.49:0/2103743378 >> 172.21.15.49:0/2103743378 conn(0x7f09bc0fbcd0 msgr2=0x7f09bc0fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:07:58.433 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:58.433+0000 7f09c1da9640 1 -- 172.21.15.49:0/2103743378 shutdown_connections 2024-03-18T23:07:58.433 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:58.433+0000 7f09c1da9640 1 -- 172.21.15.49:0/2103743378 wait complete. 2024-03-18T23:07:58.433 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:58.433+0000 7f09c1da9640 1 Processor -- start 2024-03-18T23:07:58.435 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:58.433+0000 7f09c1da9640 1 -- start start 2024-03-18T23:07:58.435 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:58.434+0000 7f09c1da9640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f09bc100740 0x7f09bc06a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:58.435 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:58.434+0000 7f09c1da9640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f09bc101930 0x7f09bc19c800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:58.436 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:58.434+0000 7f09c1da9640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f09bc1022f0 0x7f09bc19ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:58.436 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:58.434+0000 7f09c1da9640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f09bc10c3a0 con 0x7f09bc1022f0 2024-03-18T23:07:58.436 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:58.434+0000 7f09c1da9640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f09bc10c220 con 0x7f09bc100740 2024-03-18T23:07:58.436 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:58.434+0000 7f09c1da9640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f09bc10c520 con 0x7f09bc101930 2024-03-18T23:07:58.436 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:58.434+0000 7f09bb7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f09bc100740 0x7f09bc06a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:58.436 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:58.434+0000 7f09bbfff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f09bc1022f0 0x7f09bc19ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:58.436 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:58.434+0000 7f09bb7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f09bc100740 0x7f09bc06a960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.49:48394/0 (socket says 172.21.15.49:48394) 2024-03-18T23:07:58.436 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:58.434+0000 7f09bb7fe640 1 -- 172.21.15.49:0/1369448245 learned_addr learned my addr 172.21.15.49:0/1369448245 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:07:58.436 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:58.434+0000 7f09baffd640 1 --2- 172.21.15.49:0/1369448245 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f09bc101930 0x7f09bc19c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:58.436 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:58.434+0000 7f09bb7fe640 1 -- 172.21.15.49:0/1369448245 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f09bc101930 msgr2=0x7f09bc19c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:58.436 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:58.434+0000 7f09bb7fe640 1 --2- 172.21.15.49:0/1369448245 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f09bc101930 0x7f09bc19c800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:58.436 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:58.434+0000 7f09bb7fe640 1 -- 172.21.15.49:0/1369448245 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f09bc1022f0 msgr2=0x7f09bc19ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:58.436 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:58.434+0000 7f09bb7fe640 1 --2- 172.21.15.49:0/1369448245 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f09bc1022f0 0x7f09bc19ee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:58.436 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:58.434+0000 7f09bb7fe640 1 -- 172.21.15.49:0/1369448245 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09bc19f730 con 0x7f09bc100740 2024-03-18T23:07:58.437 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:58.434+0000 7f09bbfff640 1 --2- 172.21.15.49:0/1369448245 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f09bc1022f0 0x7f09bc19ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:07:58.437 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:58.434+0000 7f09bb7fe640 1 --2- 172.21.15.49:0/1369448245 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f09bc100740 0x7f09bc06a960 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f09b000e9c0 tx=0x7f09b000ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:07:58.437 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:58.434+0000 7f09b8ff9640 1 -- 172.21.15.49:0/1369448245 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09b000cd80 con 0x7f09bc100740 2024-03-18T23:07:58.437 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:58.434+0000 7f09c1da9640 1 -- 172.21.15.49:0/1369448245 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09bc19f9c0 con 0x7f09bc100740 2024-03-18T23:07:58.437 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:58.434+0000 7f09b8ff9640 1 -- 172.21.15.49:0/1369448245 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f09b00035f0 con 0x7f09bc100740 2024-03-18T23:07:58.437 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:58.434+0000 7f09b8ff9640 1 -- 172.21.15.49:0/1369448245 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09b0010690 con 0x7f09bc100740 2024-03-18T23:07:58.437 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:58.434+0000 7f09c1da9640 1 -- 172.21.15.49:0/1369448245 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09bc19ff10 con 0x7f09bc100740 2024-03-18T23:07:58.437 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:58.435+0000 7f09c1da9640 1 -- 172.21.15.49:0/1369448245 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09bc107a00 con 0x7f09bc100740 2024-03-18T23:07:58.438 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:58.436+0000 7f09b8ff9640 1 -- 172.21.15.49:0/1369448245 <== mon.2 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7f09b00037a0 con 0x7f09bc100740 2024-03-18T23:07:58.438 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:58.437+0000 7f09b8ff9640 1 --2- 172.21.15.49:0/1369448245 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f09a40a2800 0x7f09a40a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:58.438 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:58.437+0000 7f09baffd640 1 --2- 172.21.15.49:0/1369448245 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f09a40a2800 0x7f09a40a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:58.438 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:58.437+0000 7f09b8ff9640 1 -- 172.21.15.49:0/1369448245 <== mon.2 v2:172.21.15.139:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f09b00cee00 con 0x7f09bc100740 2024-03-18T23:07:58.438 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:58.438+0000 7f09baffd640 1 --2- 172.21.15.49:0/1369448245 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f09a40a2800 0x7f09a40a4cc0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f09ac0095f0 tx=0x7f09ac005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:07:58.440 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:58.440+0000 7f09b8ff9640 1 -- 172.21.15.49:0/1369448245 <== mon.2 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f09b009beb0 con 0x7f09bc100740 2024-03-18T23:07:58.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:07:58 smithi175 ceph-mon[67352]: from='client.? 172.21.15.49:0/1186157477' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T23:07:58.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:07:58 smithi175 ceph-mon[67352]: from='client.? 172.21.15.49:0/1127108568' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-18T23:07:58.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:07:58 smithi139 ceph-mon[67282]: from='client.? 172.21.15.49:0/1186157477' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T23:07:58.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:07:58 smithi139 ceph-mon[67282]: from='client.? 172.21.15.49:0/1127108568' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-18T23:07:58.624 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:58.624+0000 7f09c1da9640 1 -- 172.21.15.49:0/1369448245 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f09bc02d1f0 con 0x7f09bc100740 2024-03-18T23:07:58.624 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:58.624+0000 7f09b8ff9640 1 -- 172.21.15.49:0/1369448245 <== mon.2 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) ==== 100+0+11 (secure 0 0 0) 0x7f09b009b7b0 con 0x7f09bc100740 2024-03-18T23:07:58.625 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 64 2024-03-18T23:07:58.626 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:58.626+0000 7f09c1da9640 1 -- 172.21.15.49:0/1369448245 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f09a40a2800 msgr2=0x7f09a40a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:58.627 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:58.626+0000 7f09c1da9640 1 --2- 172.21.15.49:0/1369448245 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f09a40a2800 0x7f09a40a4cc0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f09ac0095f0 tx=0x7f09ac005f90 comp rx=0 tx=0).stop 2024-03-18T23:07:58.627 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:58.626+0000 7f09c1da9640 1 -- 172.21.15.49:0/1369448245 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f09bc100740 msgr2=0x7f09bc06a960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:58.627 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:58.626+0000 7f09c1da9640 1 --2- 172.21.15.49:0/1369448245 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f09bc100740 0x7f09bc06a960 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f09b000e9c0 tx=0x7f09b000ee80 comp rx=0 tx=0).stop 2024-03-18T23:07:58.627 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:58.627+0000 7f09c1da9640 1 -- 172.21.15.49:0/1369448245 shutdown_connections 2024-03-18T23:07:58.627 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:58.627+0000 7f09c1da9640 1 --2- 172.21.15.49:0/1369448245 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f09a40a2800 0x7f09a40a4cc0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:58.627 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:58.627+0000 7f09c1da9640 1 --2- 172.21.15.49:0/1369448245 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f09bc1022f0 0x7f09bc19ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:58.627 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:58.627+0000 7f09c1da9640 1 --2- 172.21.15.49:0/1369448245 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f09bc101930 0x7f09bc19c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:58.627 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:58.627+0000 7f09c1da9640 1 --2- 172.21.15.49:0/1369448245 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f09bc100740 0x7f09bc06a960 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:58.627 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:58.627+0000 7f09c1da9640 1 -- 172.21.15.49:0/1369448245 >> 172.21.15.49:0/1369448245 conn(0x7f09bc0fbcd0 msgr2=0x7f09bc104bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:07:58.627 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:58.627+0000 7f09c1da9640 1 -- 172.21.15.49:0/1369448245 shutdown_connections 2024-03-18T23:07:58.627 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:58.627+0000 7f09c1da9640 1 -- 172.21.15.49:0/1369448245 wait complete. 2024-03-18T23:07:58.638 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-18T23:07:58.753 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:58.747+0000 7fa051a30640 1 -- 172.21.15.49:0/2301649154 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa04c104c50 msgr2=0x7fa04c107040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:58.753 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:58.747+0000 7fa051a30640 1 --2- 172.21.15.49:0/2301649154 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa04c104c50 0x7fa04c107040 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fa034009900 tx=0x7fa03402bc30 comp rx=0 tx=0).stop 2024-03-18T23:07:58.753 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:58.747+0000 7fa051a30640 1 -- 172.21.15.49:0/2301649154 shutdown_connections 2024-03-18T23:07:58.753 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:58.747+0000 7fa051a30640 1 --2- 172.21.15.49:0/2301649154 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa04c107580 0x7fa04c109970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:58.754 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:58.747+0000 7fa051a30640 1 --2- 172.21.15.49:0/2301649154 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa04c104c50 0x7fa04c107040 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:58.754 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:58.747+0000 7fa051a30640 1 --2- 172.21.15.49:0/2301649154 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa04c102320 0x7fa04c104710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:58.754 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:58.747+0000 7fa051a30640 1 -- 172.21.15.49:0/2301649154 >> 172.21.15.49:0/2301649154 conn(0x7fa04c0fbcb0 msgr2=0x7fa04c0fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:07:58.754 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:58.747+0000 7fa051a30640 1 -- 172.21.15.49:0/2301649154 shutdown_connections 2024-03-18T23:07:58.754 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:58.747+0000 7fa051a30640 1 -- 172.21.15.49:0/2301649154 wait complete. 2024-03-18T23:07:58.754 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:58.748+0000 7fa051a30640 1 Processor -- start 2024-03-18T23:07:58.755 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:58.748+0000 7fa051a30640 1 -- start start 2024-03-18T23:07:58.755 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:58.748+0000 7fa051a30640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa04c102320 0x7fa04c197b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:58.755 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:58.748+0000 7fa051a30640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa04c104c50 0x7fa04c1980d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:58.755 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:58.748+0000 7fa051a30640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa04c107580 0x7fa04c19f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:58.755 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:58.748+0000 7fa051a30640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fa04c10d4c0 con 0x7fa04c102320 2024-03-18T23:07:58.755 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:58.748+0000 7fa051a30640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fa04c10d340 con 0x7fa04c104c50 2024-03-18T23:07:58.756 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:58.748+0000 7fa051a30640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa04c10d640 con 0x7fa04c107580 2024-03-18T23:07:58.756 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:58.748+0000 7fa04affd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa04c102320 0x7fa04c197b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:58.756 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:58.748+0000 7fa04affd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa04c102320 0x7fa04c197b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:37608/0 (socket says 172.21.15.49:37608) 2024-03-18T23:07:58.756 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:58.748+0000 7fa04affd640 1 -- 172.21.15.49:0/165109298 learned_addr learned my addr 172.21.15.49:0/165109298 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:07:58.756 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:58.748+0000 7fa04b7fe640 1 --2- 172.21.15.49:0/165109298 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa04c107580 0x7fa04c19f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:58.756 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:58.748+0000 7fa04a7fc640 1 --2- 172.21.15.49:0/165109298 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa04c104c50 0x7fa04c1980d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:58.757 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:58.749+0000 7fa04affd640 1 -- 172.21.15.49:0/165109298 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa04c107580 msgr2=0x7fa04c19f140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:58.757 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:58.749+0000 7fa04affd640 1 --2- 172.21.15.49:0/165109298 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa04c107580 0x7fa04c19f140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:58.757 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:58.749+0000 7fa04affd640 1 -- 172.21.15.49:0/165109298 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa04c104c50 msgr2=0x7fa04c1980d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:58.757 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:58.749+0000 7fa04affd640 1 --2- 172.21.15.49:0/165109298 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa04c104c50 0x7fa04c1980d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:58.757 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:58.749+0000 7fa04affd640 1 -- 172.21.15.49:0/165109298 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa04c19f7b0 con 0x7fa04c102320 2024-03-18T23:07:58.757 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:58.749+0000 7fa04affd640 1 --2- 172.21.15.49:0/165109298 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa04c102320 0x7fa04c197b90 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fa04400e9f0 tx=0x7fa04400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:07:58.758 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:58.749+0000 7fa050a2e640 1 -- 172.21.15.49:0/165109298 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa04400cd80 con 0x7fa04c102320 2024-03-18T23:07:58.758 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:58.749+0000 7fa050a2e640 1 -- 172.21.15.49:0/165109298 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa044010910 con 0x7fa04c102320 2024-03-18T23:07:58.758 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:58.749+0000 7fa051a30640 1 -- 172.21.15.49:0/165109298 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa04c19faa0 con 0x7fa04c102320 2024-03-18T23:07:58.758 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:58.749+0000 7fa050a2e640 1 -- 172.21.15.49:0/165109298 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa044018990 con 0x7fa04c102320 2024-03-18T23:07:58.758 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:58.749+0000 7fa051a30640 1 -- 172.21.15.49:0/165109298 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa04c19ffc0 con 0x7fa04c102320 2024-03-18T23:07:58.759 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:58.750+0000 7fa051a30640 1 -- 172.21.15.49:0/165109298 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa04c068610 con 0x7fa04c102320 2024-03-18T23:07:58.759 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:58.751+0000 7fa050a2e640 1 -- 172.21.15.49:0/165109298 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7fa044010ac0 con 0x7fa04c102320 2024-03-18T23:07:58.759 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:58.752+0000 7fa050a2e640 1 --2- 172.21.15.49:0/165109298 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fa0380a2800 0x7fa0380a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:58.759 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:58.752+0000 7fa04a7fc640 1 --2- 172.21.15.49:0/165109298 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fa0380a2800 0x7fa0380a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:58.759 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:58.752+0000 7fa050a2e640 1 -- 172.21.15.49:0/165109298 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7fa0440cee00 con 0x7fa04c102320 2024-03-18T23:07:58.759 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:58.752+0000 7fa04a7fc640 1 --2- 172.21.15.49:0/165109298 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fa0380a2800 0x7fa0380a4cc0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fa034000c00 tx=0x7fa034005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:07:58.760 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:58.756+0000 7fa050a2e640 1 -- 172.21.15.49:0/165109298 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fa04409b800 con 0x7fa04c102320 2024-03-18T23:07:58.977 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:58.977+0000 7fa051a30640 1 -- 172.21.15.49:0/165109298 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa04c0fd7a0 con 0x7fa04c102320 2024-03-18T23:07:58.978 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:58.978+0000 7fa050a2e640 1 -- 172.21.15.49:0/165109298 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11936 (secure 0 0 0) 0x7fa04409f8a0 con 0x7fa04c102320 2024-03-18T23:07:58.978 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-18T23:07:58.979 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6835/3631301630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":3631301630},{"type":"v1","addr":"172.21.15.49:6835","nonce":3631301630}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6837/444305233","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":444305233},{"type":"v1","addr":"172.21.15.49:6837","nonce":444305233}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-18T23:06:04.575999+0000","modified":"2024-03-18T23:06:11.998736+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24488,"mds_1":24463,"mds_2":24430,"mds_3":24419,"mds_4":24391},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14622":{"gid":14622,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.49:6839/1934085093","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6838","nonce":1934085093},{"type":"v1","addr":"172.21.15.49:6839","nonce":1934085093}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14634":{"gid":14634,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.49:6841/2869298862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6840","nonce":2869298862},{"type":"v1","addr":"172.21.15.49:6841","nonce":2869298862}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24389":{"gid":24389,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6833/3766178603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3766178603},{"type":"v1","addr":"172.21.15.139:6833","nonce":3766178603}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24391":{"gid":24391,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.175:6833/1702997173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":1702997173},{"type":"v1","addr":"172.21.15.175:6833","nonce":1702997173}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":9,"addr":"172.21.15.139:6835/1663212937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":1663212937},{"type":"v1","addr":"172.21.15.139:6835","nonce":1663212937}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.175:6835/755474931","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":755474931},{"type":"v1","addr":"172.21.15.175:6835","nonce":755474931}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6837/379482756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":379482756},{"type":"v1","addr":"172.21.15.175:6837","nonce":379482756}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6837/2330317540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6836","nonce":2330317540},{"type":"v1","addr":"172.21.15.139:6837","nonce":2330317540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"172.21.15.175:6839/3640882084","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":3640882084},{"type":"v1","addr":"172.21.15.175:6839","nonce":3640882084}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.139:6839/4280128209","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6838","nonce":4280128209},{"type":"v1","addr":"172.21.15.139:6839","nonce":4280128209}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24463,24488,24430,24419,24391]},"id":1}]} 2024-03-18T23:07:58.979 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 27 2024-03-18T23:07:58.980 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:58.980+0000 7fa051a30640 1 -- 172.21.15.49:0/165109298 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fa0380a2800 msgr2=0x7fa0380a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:58.980 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:58.980+0000 7fa051a30640 1 --2- 172.21.15.49:0/165109298 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fa0380a2800 0x7fa0380a4cc0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fa034000c00 tx=0x7fa034005fd0 comp rx=0 tx=0).stop 2024-03-18T23:07:58.980 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:58.980+0000 7fa051a30640 1 -- 172.21.15.49:0/165109298 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa04c102320 msgr2=0x7fa04c197b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:58.980 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:58.980+0000 7fa051a30640 1 --2- 172.21.15.49:0/165109298 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa04c102320 0x7fa04c197b90 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fa04400e9f0 tx=0x7fa04400eeb0 comp rx=0 tx=0).stop 2024-03-18T23:07:58.980 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:58.980+0000 7fa051a30640 1 -- 172.21.15.49:0/165109298 shutdown_connections 2024-03-18T23:07:58.980 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:58.980+0000 7fa051a30640 1 --2- 172.21.15.49:0/165109298 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fa0380a2800 0x7fa0380a4cc0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:58.981 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:58.980+0000 7fa051a30640 1 --2- 172.21.15.49:0/165109298 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa04c107580 0x7fa04c19f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:58.981 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:58.980+0000 7fa051a30640 1 --2- 172.21.15.49:0/165109298 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa04c104c50 0x7fa04c1980d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:58.981 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:58.980+0000 7fa051a30640 1 --2- 172.21.15.49:0/165109298 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa04c102320 0x7fa04c197b90 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:58.981 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:58.980+0000 7fa051a30640 1 -- 172.21.15.49:0/165109298 >> 172.21.15.49:0/165109298 conn(0x7fa04c0fbcb0 msgr2=0x7fa04c102d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:07:58.981 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:58.980+0000 7fa051a30640 1 -- 172.21.15.49:0/165109298 shutdown_connections 2024-03-18T23:07:58.981 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:58.980+0000 7fa051a30640 1 -- 172.21.15.49:0/165109298 wait complete. 2024-03-18T23:07:58.992 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-18T23:07:59.107 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:59.102+0000 7fa3a72ea640 1 -- 172.21.15.49:0/1622900859 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa3a0104c70 msgr2=0x7fa3a0107060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:59.107 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:59.102+0000 7fa3a72ea640 1 --2- 172.21.15.49:0/1622900859 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa3a0104c70 0x7fa3a0107060 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fa390009900 tx=0x7fa39002bc30 comp rx=0 tx=0).stop 2024-03-18T23:07:59.107 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:59.102+0000 7fa3a72ea640 1 -- 172.21.15.49:0/1622900859 shutdown_connections 2024-03-18T23:07:59.108 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:59.102+0000 7fa3a72ea640 1 --2- 172.21.15.49:0/1622900859 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa3a01075a0 0x7fa3a01099f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:59.108 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:59.102+0000 7fa3a72ea640 1 --2- 172.21.15.49:0/1622900859 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa3a0104c70 0x7fa3a0107060 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:59.108 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:59.102+0000 7fa3a72ea640 1 --2- 172.21.15.49:0/1622900859 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa3a0102340 0x7fa3a0104730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:59.108 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:59.102+0000 7fa3a72ea640 1 -- 172.21.15.49:0/1622900859 >> 172.21.15.49:0/1622900859 conn(0x7fa3a00fbd50 msgr2=0x7fa3a00fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:07:59.108 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:59.102+0000 7fa3a72ea640 1 -- 172.21.15.49:0/1622900859 shutdown_connections 2024-03-18T23:07:59.108 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:59.102+0000 7fa3a72ea640 1 -- 172.21.15.49:0/1622900859 wait complete. 2024-03-18T23:07:59.109 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:59.103+0000 7fa3a72ea640 1 Processor -- start 2024-03-18T23:07:59.109 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:59.103+0000 7fa3a72ea640 1 -- start start 2024-03-18T23:07:59.109 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:59.103+0000 7fa3a72ea640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa3a0102340 0x7fa3a0197ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:59.109 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:59.103+0000 7fa3a72ea640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa3a0104c70 0x7fa3a01980e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:59.109 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:59.103+0000 7fa3a72ea640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa3a01075a0 0x7fa3a019f150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:59.109 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:59.103+0000 7fa3a72ea640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fa3a010d4c0 con 0x7fa3a0104c70 2024-03-18T23:07:59.109 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:59.103+0000 7fa3a72ea640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fa3a010d340 con 0x7fa3a0102340 2024-03-18T23:07:59.110 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:59.103+0000 7fa3a72ea640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa3a010d640 con 0x7fa3a01075a0 2024-03-18T23:07:59.110 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:59.103+0000 7fa3a4885640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa3a0104c70 0x7fa3a01980e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:59.110 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:59.103+0000 7fa3a4885640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa3a0104c70 0x7fa3a01980e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:37632/0 (socket says 172.21.15.49:37632) 2024-03-18T23:07:59.110 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:59.103+0000 7fa3a4885640 1 -- 172.21.15.49:0/4174881191 learned_addr learned my addr 172.21.15.49:0/4174881191 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:07:59.110 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:59.103+0000 7fa3a5887640 1 --2- 172.21.15.49:0/4174881191 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa3a01075a0 0x7fa3a019f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:59.110 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:59.103+0000 7fa3a5086640 1 --2- 172.21.15.49:0/4174881191 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa3a0102340 0x7fa3a0197ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:59.111 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:59.103+0000 7fa3a4885640 1 -- 172.21.15.49:0/4174881191 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa3a01075a0 msgr2=0x7fa3a019f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:59.111 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:59.103+0000 7fa3a4885640 1 --2- 172.21.15.49:0/4174881191 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa3a01075a0 0x7fa3a019f150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:59.111 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:59.103+0000 7fa3a4885640 1 -- 172.21.15.49:0/4174881191 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa3a0102340 msgr2=0x7fa3a0197ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:59.111 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:59.103+0000 7fa3a4885640 1 --2- 172.21.15.49:0/4174881191 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa3a0102340 0x7fa3a0197ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:59.111 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:59.103+0000 7fa3a4885640 1 -- 172.21.15.49:0/4174881191 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3a019f7c0 con 0x7fa3a0104c70 2024-03-18T23:07:59.111 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:59.103+0000 7fa3a5086640 1 --2- 172.21.15.49:0/4174881191 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa3a0102340 0x7fa3a0197ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T23:07:59.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:59.103+0000 7fa3a4885640 1 --2- 172.21.15.49:0/4174881191 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa3a0104c70 0x7fa3a01980e0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fa390005ab0 tx=0x7fa390005dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:07:59.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:59.104+0000 7fa3967fc640 1 -- 172.21.15.49:0/4174881191 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa390041070 con 0x7fa3a0104c70 2024-03-18T23:07:59.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:59.104+0000 7fa3967fc640 1 -- 172.21.15.49:0/4174881191 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa3900054e0 con 0x7fa3a0104c70 2024-03-18T23:07:59.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:59.104+0000 7fa3a72ea640 1 -- 172.21.15.49:0/4174881191 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3a019fa50 con 0x7fa3a0104c70 2024-03-18T23:07:59.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:59.104+0000 7fa3967fc640 1 -- 172.21.15.49:0/4174881191 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa390034440 con 0x7fa3a0104c70 2024-03-18T23:07:59.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:59.104+0000 7fa3a72ea640 1 -- 172.21.15.49:0/4174881191 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3a019ff40 con 0x7fa3a0104c70 2024-03-18T23:07:59.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:59.105+0000 7fa3a72ea640 1 -- 172.21.15.49:0/4174881191 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3a00fd840 con 0x7fa3a0104c70 2024-03-18T23:07:59.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:59.106+0000 7fa3967fc640 1 -- 172.21.15.49:0/4174881191 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7fa390044070 con 0x7fa3a0104c70 2024-03-18T23:07:59.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:59.106+0000 7fa3967fc640 1 --2- 172.21.15.49:0/4174881191 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fa38c0a2800 0x7fa38c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:59.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:59.106+0000 7fa3a5086640 1 --2- 172.21.15.49:0/4174881191 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fa38c0a2800 0x7fa38c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:59.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:59.106+0000 7fa3967fc640 1 -- 172.21.15.49:0/4174881191 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7fa3900f06c0 con 0x7fa3a0104c70 2024-03-18T23:07:59.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:59.107+0000 7fa3a5086640 1 --2- 172.21.15.49:0/4174881191 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fa38c0a2800 0x7fa38c0a4cc0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fa39c005d00 tx=0x7fa39c005c70 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:07:59.113 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:59.111+0000 7fa3967fc640 1 -- 172.21.15.49:0/4174881191 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fa3900bd770 con 0x7fa3a0104c70 2024-03-18T23:07:59.297 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:59.297+0000 7fa3a72ea640 1 -- 172.21.15.49:0/4174881191 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa3a00624c0 con 0x7fa3a0104c70 2024-03-18T23:07:59.298 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:59.299+0000 7fa3967fc640 1 -- 172.21.15.49:0/4174881191 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+22674 (secure 0 0 0) 0x7fa3900bd070 con 0x7fa3a0104c70 2024-03-18T23:07:59.299 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-18T23:07:59.299 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":88,"fsid":"c63591ea-e57a-11ee-95c9-87774f69a715","created":"2024-03-18T22:57:58.734467+0000","modified":"2024-03-18T23:07:01.337701+0000","last_up_change":"2024-03-18T23:05:16.576310+0000","last_in_change":"2024-03-18T23:04:57.024067+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-18T23:01:28.363432+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-18T23:06:03.172963+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-18T23:06:03.973303+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e7fc9daa-4098-4ba5-8019-9aaef1aab3e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":1651162501},{"type":"v1","addr":"172.21.15.49:6803","nonce":1651162501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":1651162501},{"type":"v1","addr":"172.21.15.49:6805","nonce":1651162501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":1651162501},{"type":"v1","addr":"172.21.15.49:6809","nonce":1651162501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":1651162501},{"type":"v1","addr":"172.21.15.49:6807","nonce":1651162501}]},"public_addr":"172.21.15.49:6803/1651162501","cluster_addr":"172.21.15.49:6805/1651162501","heartbeat_back_addr":"172.21.15.49:6809/1651162501","heartbeat_front_addr":"172.21.15.49:6807/1651162501","state":["exists","up"]},{"osd":1,"uuid":"40ac929c-e90f-434c-a796-9aabd686f1fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":1766700359},{"type":"v1","addr":"172.21.15.139:6801","nonce":1766700359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":1766700359},{"type":"v1","addr":"172.21.15.139:6803","nonce":1766700359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":1766700359},{"type":"v1","addr":"172.21.15.139:6807","nonce":1766700359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":1766700359},{"type":"v1","addr":"172.21.15.139:6805","nonce":1766700359}]},"public_addr":"172.21.15.139:6801/1766700359","cluster_addr":"172.21.15.139:6803/1766700359","heartbeat_back_addr":"172.21.15.139:6807/1766700359","heartbeat_front_addr":"172.21.15.139:6805/1766700359","state":["exists","up"]},{"osd":2,"uuid":"8ae3b666-00b4-45ed-a5ee-4d0771edabb6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":3752671234},{"type":"v1","addr":"172.21.15.175:6801","nonce":3752671234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":3752671234},{"type":"v1","addr":"172.21.15.175:6803","nonce":3752671234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":3752671234},{"type":"v1","addr":"172.21.15.175:6807","nonce":3752671234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":3752671234},{"type":"v1","addr":"172.21.15.175:6805","nonce":3752671234}]},"public_addr":"172.21.15.175:6801/3752671234","cluster_addr":"172.21.15.175:6803/3752671234","heartbeat_back_addr":"172.21.15.175:6807/3752671234","heartbeat_front_addr":"172.21.15.175:6805/3752671234","state":["exists","up"]},{"osd":3,"uuid":"1a62e89d-45d4-4237-b813-7f96a59aca7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":956160536},{"type":"v1","addr":"172.21.15.49:6811","nonce":956160536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":956160536},{"type":"v1","addr":"172.21.15.49:6813","nonce":956160536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":956160536},{"type":"v1","addr":"172.21.15.49:6817","nonce":956160536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":956160536},{"type":"v1","addr":"172.21.15.49:6815","nonce":956160536}]},"public_addr":"172.21.15.49:6811/956160536","cluster_addr":"172.21.15.49:6813/956160536","heartbeat_back_addr":"172.21.15.49:6817/956160536","heartbeat_front_addr":"172.21.15.49:6815/956160536","state":["exists","up"]},{"osd":4,"uuid":"c715101c-0c3f-4b98-b142-d64a35e187df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":3129712668},{"type":"v1","addr":"172.21.15.139:6809","nonce":3129712668}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":3129712668},{"type":"v1","addr":"172.21.15.139:6811","nonce":3129712668}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":3129712668},{"type":"v1","addr":"172.21.15.139:6815","nonce":3129712668}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":3129712668},{"type":"v1","addr":"172.21.15.139:6813","nonce":3129712668}]},"public_addr":"172.21.15.139:6809/3129712668","cluster_addr":"172.21.15.139:6811/3129712668","heartbeat_back_addr":"172.21.15.139:6815/3129712668","heartbeat_front_addr":"172.21.15.139:6813/3129712668","state":["exists","up"]},{"osd":5,"uuid":"4b641f81-77eb-45fd-8464-511e7b3062b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":2774567215},{"type":"v1","addr":"172.21.15.175:6809","nonce":2774567215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":2774567215},{"type":"v1","addr":"172.21.15.175:6811","nonce":2774567215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":2774567215},{"type":"v1","addr":"172.21.15.175:6815","nonce":2774567215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":2774567215},{"type":"v1","addr":"172.21.15.175:6813","nonce":2774567215}]},"public_addr":"172.21.15.175:6809/2774567215","cluster_addr":"172.21.15.175:6811/2774567215","heartbeat_back_addr":"172.21.15.175:6815/2774567215","heartbeat_front_addr":"172.21.15.175:6813/2774567215","state":["exists","up"]},{"osd":6,"uuid":"653da8e0-1197-4b48-abed-470f2a9c4dce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":1472215461},{"type":"v1","addr":"172.21.15.49:6819","nonce":1472215461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":1472215461},{"type":"v1","addr":"172.21.15.49:6821","nonce":1472215461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":1472215461},{"type":"v1","addr":"172.21.15.49:6825","nonce":1472215461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":1472215461},{"type":"v1","addr":"172.21.15.49:6823","nonce":1472215461}]},"public_addr":"172.21.15.49:6819/1472215461","cluster_addr":"172.21.15.49:6821/1472215461","heartbeat_back_addr":"172.21.15.49:6825/1472215461","heartbeat_front_addr":"172.21.15.49:6823/1472215461","state":["exists","up"]},{"osd":7,"uuid":"ad157c9b-4b75-451c-8740-630f0975b7fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":2163986787},{"type":"v1","addr":"172.21.15.139:6817","nonce":2163986787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":2163986787},{"type":"v1","addr":"172.21.15.139:6819","nonce":2163986787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":2163986787},{"type":"v1","addr":"172.21.15.139:6823","nonce":2163986787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":2163986787},{"type":"v1","addr":"172.21.15.139:6821","nonce":2163986787}]},"public_addr":"172.21.15.139:6817/2163986787","cluster_addr":"172.21.15.139:6819/2163986787","heartbeat_back_addr":"172.21.15.139:6823/2163986787","heartbeat_front_addr":"172.21.15.139:6821/2163986787","state":["exists","up"]},{"osd":8,"uuid":"c8e75f15-df75-4d5d-9b92-a064bffb6c01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":1194026992},{"type":"v1","addr":"172.21.15.175:6817","nonce":1194026992}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1194026992},{"type":"v1","addr":"172.21.15.175:6819","nonce":1194026992}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":1194026992},{"type":"v1","addr":"172.21.15.175:6823","nonce":1194026992}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1194026992},{"type":"v1","addr":"172.21.15.175:6821","nonce":1194026992}]},"public_addr":"172.21.15.175:6817/1194026992","cluster_addr":"172.21.15.175:6819/1194026992","heartbeat_back_addr":"172.21.15.175:6823/1194026992","heartbeat_front_addr":"172.21.15.175:6821/1194026992","state":["exists","up"]},{"osd":9,"uuid":"edb5ffef-b6a2-4941-b5ab-40f982ad30d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":2742084884},{"type":"v1","addr":"172.21.15.49:6827","nonce":2742084884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":2742084884},{"type":"v1","addr":"172.21.15.49:6829","nonce":2742084884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":2742084884},{"type":"v1","addr":"172.21.15.49:6833","nonce":2742084884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":2742084884},{"type":"v1","addr":"172.21.15.49:6831","nonce":2742084884}]},"public_addr":"172.21.15.49:6827/2742084884","cluster_addr":"172.21.15.49:6829/2742084884","heartbeat_back_addr":"172.21.15.49:6833/2742084884","heartbeat_front_addr":"172.21.15.49:6831/2742084884","state":["exists","up"]},{"osd":10,"uuid":"3945e46d-766a-4c4a-8821-d43648dd1578","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":2404740201},{"type":"v1","addr":"172.21.15.139:6825","nonce":2404740201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":2404740201},{"type":"v1","addr":"172.21.15.139:6827","nonce":2404740201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":2404740201},{"type":"v1","addr":"172.21.15.139:6831","nonce":2404740201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":2404740201},{"type":"v1","addr":"172.21.15.139:6829","nonce":2404740201}]},"public_addr":"172.21.15.139:6825/2404740201","cluster_addr":"172.21.15.139:6827/2404740201","heartbeat_back_addr":"172.21.15.139:6831/2404740201","heartbeat_front_addr":"172.21.15.139:6829/2404740201","state":["exists","up"]},{"osd":11,"uuid":"9954ba15-d877-46cc-b24a-893f9168deb8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":3519503868},{"type":"v1","addr":"172.21.15.175:6825","nonce":3519503868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":3519503868},{"type":"v1","addr":"172.21.15.175:6827","nonce":3519503868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":3519503868},{"type":"v1","addr":"172.21.15.175:6831","nonce":3519503868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":3519503868},{"type":"v1","addr":"172.21.15.175:6829","nonce":3519503868}]},"public_addr":"172.21.15.175:6825/3519503868","cluster_addr":"172.21.15.175:6827/3519503868","heartbeat_back_addr":"172.21.15.175:6831/3519503868","heartbeat_front_addr":"172.21.15.175:6829/3519503868","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:00:38.367865+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:01:02.237687+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:01:25.609751+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:01:51.268572+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:02:16.514605+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:02:41.586523+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:03:07.029964+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:03:32.956250+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:03:57.547984+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:04:23.948972+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:04:49.667210+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:05:15.284693+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.6","mappings":[{"from":4,"to":6}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.49:0/563527416":"2024-03-19T22:58:59.806253+0000","172.21.15.49:0/1036383534":"2024-03-19T22:58:59.806253+0000","172.21.15.49:6801/2609458951":"2024-03-19T22:58:59.806253+0000","172.21.15.49:0/21283102":"2024-03-19T22:58:31.972474+0000","172.21.15.49:0/1292058922":"2024-03-19T22:58:31.972474+0000","172.21.15.49:6800/1985190312":"2024-03-19T22:58:31.972474+0000","172.21.15.49:0/1912446700":"2024-03-19T22:58:59.806253+0000","172.21.15.49:0/2176210039":"2024-03-19T22:58:31.972474+0000","172.21.15.49:6801/1985190312":"2024-03-19T22:58:31.972474+0000","172.21.15.49:6800/2609458951":"2024-03-19T22:58:59.806253+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-18T23:07:59.301 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:59.301+0000 7fa3a72ea640 1 -- 172.21.15.49:0/4174881191 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fa38c0a2800 msgr2=0x7fa38c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:59.301 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:59.301+0000 7fa3a72ea640 1 --2- 172.21.15.49:0/4174881191 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fa38c0a2800 0x7fa38c0a4cc0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fa39c005d00 tx=0x7fa39c005c70 comp rx=0 tx=0).stop 2024-03-18T23:07:59.301 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:59.301+0000 7fa3a72ea640 1 -- 172.21.15.49:0/4174881191 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa3a0104c70 msgr2=0x7fa3a01980e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:59.301 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:59.301+0000 7fa3a72ea640 1 --2- 172.21.15.49:0/4174881191 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa3a0104c70 0x7fa3a01980e0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fa390005ab0 tx=0x7fa390005dc0 comp rx=0 tx=0).stop 2024-03-18T23:07:59.301 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:59.301+0000 7fa3a72ea640 1 -- 172.21.15.49:0/4174881191 shutdown_connections 2024-03-18T23:07:59.301 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:59.301+0000 7fa3a72ea640 1 --2- 172.21.15.49:0/4174881191 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fa38c0a2800 0x7fa38c0a4cc0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:59.301 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:59.301+0000 7fa3a72ea640 1 --2- 172.21.15.49:0/4174881191 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa3a01075a0 0x7fa3a019f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:59.301 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:59.301+0000 7fa3a72ea640 1 --2- 172.21.15.49:0/4174881191 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa3a0104c70 0x7fa3a01980e0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:59.301 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:59.301+0000 7fa3a72ea640 1 --2- 172.21.15.49:0/4174881191 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa3a0102340 0x7fa3a0197ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:59.302 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:59.301+0000 7fa3a72ea640 1 -- 172.21.15.49:0/4174881191 >> 172.21.15.49:0/4174881191 conn(0x7fa3a00fbd50 msgr2=0x7fa3a0102c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:07:59.302 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:59.301+0000 7fa3a72ea640 1 -- 172.21.15.49:0/4174881191 shutdown_connections 2024-03-18T23:07:59.302 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:59.301+0000 7fa3a72ea640 1 -- 172.21.15.49:0/4174881191 wait complete. 2024-03-18T23:07:59.310 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:07:59 smithi049 ceph-mon[62991]: pgmap v347: 97 pgs: 97 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-03-18T23:07:59.311 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:07:59 smithi049 ceph-mon[62991]: from='client.? 172.21.15.49:0/1100434796' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-18T23:07:59.311 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:07:59 smithi049 ceph-mon[62991]: from='client.? 172.21.15.49:0/1369448245' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-18T23:07:59.311 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:07:59 smithi049 ceph-mon[62991]: from='client.? 172.21.15.49:0/165109298' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T23:07:59.312 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-18T23:01:28.363432+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-18T23:06:03.172963+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-18T23:06:03.973303+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-18T23:07:59.312 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-18T23:07:59.426 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:59.423+0000 7fed2661f640 1 -- 172.21.15.49:0/4078715394 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fed201022d0 msgr2=0x7fed20108b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:59.426 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:59.423+0000 7fed2661f640 1 --2- 172.21.15.49:0/4078715394 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fed201022d0 0x7fed20108b60 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fed10009900 tx=0x7fed1002bc30 comp rx=0 tx=0).stop 2024-03-18T23:07:59.426 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:59.423+0000 7fed2661f640 1 -- 172.21.15.49:0/4078715394 shutdown_connections 2024-03-18T23:07:59.426 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:59.423+0000 7fed2661f640 1 --2- 172.21.15.49:0/4078715394 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fed201022d0 0x7fed20108b60 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:59.426 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:59.423+0000 7fed2661f640 1 --2- 172.21.15.49:0/4078715394 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fed20101910 0x7fed20101d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:59.426 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:59.423+0000 7fed2661f640 1 --2- 172.21.15.49:0/4078715394 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fed20100720 0x7fed20100b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:59.426 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:59.423+0000 7fed2661f640 1 -- 172.21.15.49:0/4078715394 >> 172.21.15.49:0/4078715394 conn(0x7fed200fbcb0 msgr2=0x7fed200fe0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:07:59.426 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:59.423+0000 7fed2661f640 1 -- 172.21.15.49:0/4078715394 shutdown_connections 2024-03-18T23:07:59.426 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:59.423+0000 7fed2661f640 1 -- 172.21.15.49:0/4078715394 wait complete. 2024-03-18T23:07:59.427 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:59.424+0000 7fed2661f640 1 Processor -- start 2024-03-18T23:07:59.427 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:59.424+0000 7fed2661f640 1 -- start start 2024-03-18T23:07:59.427 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:59.424+0000 7fed2661f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fed20100720 0x7fed2010b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:59.427 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:59.424+0000 7fed2661f640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fed20101910 0x7fed201096e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:59.427 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:59.424+0000 7fed2661f640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fed201022d0 0x7fed20109c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:59.427 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:59.424+0000 7fed2661f640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fed2010d4c0 con 0x7fed201022d0 2024-03-18T23:07:59.427 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:59.424+0000 7fed2661f640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fed2010d340 con 0x7fed20100720 2024-03-18T23:07:59.427 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:59.424+0000 7fed2661f640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fed2010d640 con 0x7fed20101910 2024-03-18T23:07:59.427 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:59.424+0000 7fed24bbc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fed201022d0 0x7fed20109c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:59.427 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:59.425+0000 7fed24bbc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fed201022d0 0x7fed20109c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:37654/0 (socket says 172.21.15.49:37654) 2024-03-18T23:07:59.427 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:59.425+0000 7fed24bbc640 1 -- 172.21.15.49:0/816836889 learned_addr learned my addr 172.21.15.49:0/816836889 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:07:59.427 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:59.425+0000 7fed1ffff640 1 --2- 172.21.15.49:0/816836889 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fed20100720 0x7fed2010b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:59.427 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:59.425+0000 7fed1f7fe640 1 --2- 172.21.15.49:0/816836889 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fed20101910 0x7fed201096e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:59.427 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:59.425+0000 7fed24bbc640 1 -- 172.21.15.49:0/816836889 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fed20101910 msgr2=0x7fed201096e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:59.428 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:59.425+0000 7fed24bbc640 1 --2- 172.21.15.49:0/816836889 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fed20101910 0x7fed201096e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:59.428 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:59.425+0000 7fed24bbc640 1 -- 172.21.15.49:0/816836889 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fed20100720 msgr2=0x7fed2010b0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:59.428 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:59.425+0000 7fed24bbc640 1 --2- 172.21.15.49:0/816836889 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fed20100720 0x7fed2010b0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:59.428 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:59.425+0000 7fed24bbc640 1 -- 172.21.15.49:0/816836889 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed2010a4b0 con 0x7fed201022d0 2024-03-18T23:07:59.428 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:59.425+0000 7fed24bbc640 1 --2- 172.21.15.49:0/816836889 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fed201022d0 0x7fed20109c20 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fed10005ab0 tx=0x7fed1002c6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:07:59.428 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:59.425+0000 7fed1d7fa640 1 -- 172.21.15.49:0/816836889 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed10041070 con 0x7fed201022d0 2024-03-18T23:07:59.428 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:59.425+0000 7fed1d7fa640 1 -- 172.21.15.49:0/816836889 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed10004450 con 0x7fed201022d0 2024-03-18T23:07:59.428 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:59.425+0000 7fed2661f640 1 -- 172.21.15.49:0/816836889 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed201a1bd0 con 0x7fed201022d0 2024-03-18T23:07:59.428 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:59.425+0000 7fed1d7fa640 1 -- 172.21.15.49:0/816836889 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed10003e60 con 0x7fed201022d0 2024-03-18T23:07:59.428 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:59.425+0000 7fed2661f640 1 -- 172.21.15.49:0/816836889 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed201a2120 con 0x7fed201022d0 2024-03-18T23:07:59.429 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:59.426+0000 7fed2661f640 1 -- 172.21.15.49:0/816836889 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed20107a40 con 0x7fed201022d0 2024-03-18T23:07:59.429 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:59.427+0000 7fed1d7fa640 1 -- 172.21.15.49:0/816836889 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7fed10044070 con 0x7fed201022d0 2024-03-18T23:07:59.429 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:59.428+0000 7fed1d7fa640 1 --2- 172.21.15.49:0/816836889 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fed080a2800 0x7fed080a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:59.429 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:59.428+0000 7fed1d7fa640 1 -- 172.21.15.49:0/816836889 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7fed100f0660 con 0x7fed201022d0 2024-03-18T23:07:59.429 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:59.428+0000 7fed1ffff640 1 --2- 172.21.15.49:0/816836889 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fed080a2800 0x7fed080a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:59.429 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:59.428+0000 7fed1ffff640 1 --2- 172.21.15.49:0/816836889 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fed080a2800 0x7fed080a4cc0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fed20101770 tx=0x7fed18009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:07:59.434 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:59.432+0000 7fed1d7fa640 1 -- 172.21.15.49:0/816836889 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fed100bd0c0 con 0x7fed201022d0 2024-03-18T23:07:59.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:07:59 smithi175 ceph-mon[67352]: pgmap v347: 97 pgs: 97 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-03-18T23:07:59.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:07:59 smithi175 ceph-mon[67352]: from='client.? 172.21.15.49:0/1100434796' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-18T23:07:59.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:07:59 smithi175 ceph-mon[67352]: from='client.? 172.21.15.49:0/1369448245' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-18T23:07:59.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:07:59 smithi175 ceph-mon[67352]: from='client.? 172.21.15.49:0/165109298' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T23:07:59.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:07:59 smithi139 ceph-mon[67282]: pgmap v347: 97 pgs: 97 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-03-18T23:07:59.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:07:59 smithi139 ceph-mon[67282]: from='client.? 172.21.15.49:0/1100434796' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-18T23:07:59.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:07:59 smithi139 ceph-mon[67282]: from='client.? 172.21.15.49:0/1369448245' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-18T23:07:59.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:07:59 smithi139 ceph-mon[67282]: from='client.? 172.21.15.49:0/165109298' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T23:07:59.617 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:59.617+0000 7fed2661f640 1 -- 172.21.15.49:0/816836889 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fed2002d1f0 con 0x7fed201022d0 2024-03-18T23:07:59.617 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:59.617+0000 7fed1d7fa640 1 -- 172.21.15.49:0/816836889 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) ==== 93+0+10 (secure 0 0 0) 0x7fed100c1160 con 0x7fed201022d0 2024-03-18T23:07:59.617 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 1 2024-03-18T23:07:59.620 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:59.619+0000 7fed2661f640 1 -- 172.21.15.49:0/816836889 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fed080a2800 msgr2=0x7fed080a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:59.620 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:59.619+0000 7fed2661f640 1 --2- 172.21.15.49:0/816836889 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fed080a2800 0x7fed080a4cc0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fed20101770 tx=0x7fed18009500 comp rx=0 tx=0).stop 2024-03-18T23:07:59.620 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:59.619+0000 7fed2661f640 1 -- 172.21.15.49:0/816836889 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fed201022d0 msgr2=0x7fed20109c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:59.620 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:59.619+0000 7fed2661f640 1 --2- 172.21.15.49:0/816836889 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fed201022d0 0x7fed20109c20 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fed10005ab0 tx=0x7fed1002c6c0 comp rx=0 tx=0).stop 2024-03-18T23:07:59.620 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:59.620+0000 7fed2661f640 1 -- 172.21.15.49:0/816836889 shutdown_connections 2024-03-18T23:07:59.620 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:59.620+0000 7fed2661f640 1 --2- 172.21.15.49:0/816836889 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fed080a2800 0x7fed080a4cc0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:59.620 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:59.620+0000 7fed2661f640 1 --2- 172.21.15.49:0/816836889 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fed201022d0 0x7fed20109c20 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:59.620 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:59.620+0000 7fed2661f640 1 --2- 172.21.15.49:0/816836889 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fed20101910 0x7fed201096e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:59.620 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:59.620+0000 7fed2661f640 1 --2- 172.21.15.49:0/816836889 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fed20100720 0x7fed2010b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:59.620 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:59.620+0000 7fed2661f640 1 -- 172.21.15.49:0/816836889 >> 172.21.15.49:0/816836889 conn(0x7fed200fbcb0 msgr2=0x7fed20104c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:07:59.621 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:59.620+0000 7fed2661f640 1 -- 172.21.15.49:0/816836889 shutdown_connections 2024-03-18T23:07:59.621 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:59.620+0000 7fed2661f640 1 -- 172.21.15.49:0/816836889 wait complete. 2024-03-18T23:07:59.631 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-18T23:07:59.744 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:59.742+0000 7fdb730de640 1 -- 172.21.15.49:0/1513136675 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdb6c100720 msgr2=0x7fdb6c100b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:59.745 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:59.742+0000 7fdb730de640 1 --2- 172.21.15.49:0/1513136675 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdb6c100720 0x7fdb6c100b40 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fdb5c009900 tx=0x7fdb5c02bc30 comp rx=0 tx=0).stop 2024-03-18T23:07:59.745 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:59.742+0000 7fdb730de640 1 -- 172.21.15.49:0/1513136675 shutdown_connections 2024-03-18T23:07:59.745 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:59.743+0000 7fdb730de640 1 --2- 172.21.15.49:0/1513136675 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdb6c1022d0 0x7fdb6c108b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:59.745 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:59.743+0000 7fdb730de640 1 --2- 172.21.15.49:0/1513136675 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdb6c101910 0x7fdb6c101d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:59.745 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:59.743+0000 7fdb730de640 1 --2- 172.21.15.49:0/1513136675 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdb6c100720 0x7fdb6c100b40 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:59.745 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:59.743+0000 7fdb730de640 1 -- 172.21.15.49:0/1513136675 >> 172.21.15.49:0/1513136675 conn(0x7fdb6c0fbcb0 msgr2=0x7fdb6c0fe0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:07:59.745 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:59.743+0000 7fdb730de640 1 -- 172.21.15.49:0/1513136675 shutdown_connections 2024-03-18T23:07:59.745 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:59.743+0000 7fdb730de640 1 -- 172.21.15.49:0/1513136675 wait complete. 2024-03-18T23:07:59.745 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:59.743+0000 7fdb730de640 1 Processor -- start 2024-03-18T23:07:59.745 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:59.743+0000 7fdb730de640 1 -- start start 2024-03-18T23:07:59.745 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:59.743+0000 7fdb730de640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdb6c100720 0x7fdb6c199dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:59.746 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:59.743+0000 7fdb730de640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdb6c101910 0x7fdb6c19a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:59.746 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:59.743+0000 7fdb730de640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdb6c1022d0 0x7fdb6c1a1370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:59.746 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:59.743+0000 7fdb730de640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fdb6c10c390 con 0x7fdb6c100720 2024-03-18T23:07:59.746 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:59.743+0000 7fdb730de640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fdb6c10c210 con 0x7fdb6c101910 2024-03-18T23:07:59.746 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:59.743+0000 7fdb730de640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fdb6c10c510 con 0x7fdb6c1022d0 2024-03-18T23:07:59.746 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:59.743+0000 7fdb70e7a640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdb6c100720 0x7fdb6c199dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:59.746 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:59.743+0000 7fdb70e7a640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdb6c100720 0x7fdb6c199dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:37680/0 (socket says 172.21.15.49:37680) 2024-03-18T23:07:59.746 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:59.743+0000 7fdb70e7a640 1 -- 172.21.15.49:0/654691200 learned_addr learned my addr 172.21.15.49:0/654691200 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:07:59.746 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:59.743+0000 7fdb6bfff640 1 --2- 172.21.15.49:0/654691200 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdb6c101910 0x7fdb6c19a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:59.746 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:59.743+0000 7fdb7167b640 1 --2- 172.21.15.49:0/654691200 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdb6c1022d0 0x7fdb6c1a1370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:59.746 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:59.744+0000 7fdb70e7a640 1 -- 172.21.15.49:0/654691200 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdb6c1022d0 msgr2=0x7fdb6c1a1370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:59.747 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:59.744+0000 7fdb70e7a640 1 --2- 172.21.15.49:0/654691200 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdb6c1022d0 0x7fdb6c1a1370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:59.747 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:59.744+0000 7fdb70e7a640 1 -- 172.21.15.49:0/654691200 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdb6c101910 msgr2=0x7fdb6c19a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:59.747 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:59.744+0000 7fdb70e7a640 1 --2- 172.21.15.49:0/654691200 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdb6c101910 0x7fdb6c19a300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:59.747 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:59.744+0000 7fdb70e7a640 1 -- 172.21.15.49:0/654691200 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb6c1a19e0 con 0x7fdb6c100720 2024-03-18T23:07:59.747 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:59.744+0000 7fdb6bfff640 1 --2- 172.21.15.49:0/654691200 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdb6c101910 0x7fdb6c19a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:07:59.747 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:59.744+0000 7fdb70e7a640 1 --2- 172.21.15.49:0/654691200 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdb6c100720 0x7fdb6c199dc0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fdb5c000c00 tx=0x7fdb5c02c640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:07:59.747 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:59.744+0000 7fdb69ffb640 1 -- 172.21.15.49:0/654691200 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb5c041070 con 0x7fdb6c100720 2024-03-18T23:07:59.747 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:59.744+0000 7fdb69ffb640 1 -- 172.21.15.49:0/654691200 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdb5c004450 con 0x7fdb6c100720 2024-03-18T23:07:59.747 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:59.744+0000 7fdb730de640 1 -- 172.21.15.49:0/654691200 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb6c1a1c70 con 0x7fdb6c100720 2024-03-18T23:07:59.747 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:59.744+0000 7fdb69ffb640 1 -- 172.21.15.49:0/654691200 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb5c034440 con 0x7fdb6c100720 2024-03-18T23:07:59.747 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:59.744+0000 7fdb730de640 1 -- 172.21.15.49:0/654691200 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb6c1a2160 con 0x7fdb6c100720 2024-03-18T23:07:59.748 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:59.745+0000 7fdb730de640 1 -- 172.21.15.49:0/654691200 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb6c1a2330 con 0x7fdb6c100720 2024-03-18T23:07:59.748 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:59.746+0000 7fdb69ffb640 1 -- 172.21.15.49:0/654691200 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7fdb5c044070 con 0x7fdb6c100720 2024-03-18T23:07:59.749 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:59.747+0000 7fdb69ffb640 1 --2- 172.21.15.49:0/654691200 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fdb540a2800 0x7fdb540a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:59.749 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:59.747+0000 7fdb6bfff640 1 --2- 172.21.15.49:0/654691200 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fdb540a2800 0x7fdb540a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:59.749 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:59.747+0000 7fdb69ffb640 1 -- 172.21.15.49:0/654691200 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7fdb5c0f0720 con 0x7fdb6c100720 2024-03-18T23:07:59.749 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:59.747+0000 7fdb6bfff640 1 --2- 172.21.15.49:0/654691200 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fdb540a2800 0x7fdb540a4cc0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fdb6c19b2d0 tx=0x7fdb60005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:07:59.750 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:59.750+0000 7fdb69ffb640 1 -- 172.21.15.49:0/654691200 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fdb5c0bd210 con 0x7fdb6c100720 2024-03-18T23:07:59.934 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:59.933+0000 7fdb730de640 1 -- 172.21.15.49:0/654691200 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fdb6c1a2620 con 0x7fdb6c100720 2024-03-18T23:07:59.934 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:59.934+0000 7fdb69ffb640 1 -- 172.21.15.49:0/654691200 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) ==== 104+0+11 (secure 0 0 0) 0x7fdb5c0c12b0 con 0x7fdb6c100720 2024-03-18T23:07:59.934 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 32 2024-03-18T23:07:59.936 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:59.936+0000 7fdb730de640 1 -- 172.21.15.49:0/654691200 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fdb540a2800 msgr2=0x7fdb540a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:59.936 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:59.936+0000 7fdb730de640 1 --2- 172.21.15.49:0/654691200 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fdb540a2800 0x7fdb540a4cc0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fdb6c19b2d0 tx=0x7fdb60005f90 comp rx=0 tx=0).stop 2024-03-18T23:07:59.936 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:59.936+0000 7fdb730de640 1 -- 172.21.15.49:0/654691200 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdb6c100720 msgr2=0x7fdb6c199dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:59.936 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:59.936+0000 7fdb730de640 1 --2- 172.21.15.49:0/654691200 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdb6c100720 0x7fdb6c199dc0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fdb5c000c00 tx=0x7fdb5c02c640 comp rx=0 tx=0).stop 2024-03-18T23:07:59.936 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:59.936+0000 7fdb730de640 1 -- 172.21.15.49:0/654691200 shutdown_connections 2024-03-18T23:07:59.936 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:59.936+0000 7fdb730de640 1 --2- 172.21.15.49:0/654691200 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fdb540a2800 0x7fdb540a4cc0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:59.937 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:59.936+0000 7fdb730de640 1 --2- 172.21.15.49:0/654691200 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdb6c1022d0 0x7fdb6c1a1370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:59.937 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:59.936+0000 7fdb730de640 1 --2- 172.21.15.49:0/654691200 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdb6c101910 0x7fdb6c19a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:59.937 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:59.936+0000 7fdb730de640 1 --2- 172.21.15.49:0/654691200 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdb6c100720 0x7fdb6c199dc0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:59.937 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:59.936+0000 7fdb730de640 1 -- 172.21.15.49:0/654691200 >> 172.21.15.49:0/654691200 conn(0x7fdb6c0fbcb0 msgr2=0x7fdb6c0fd7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:07:59.937 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:59.936+0000 7fdb730de640 1 -- 172.21.15.49:0/654691200 shutdown_connections 2024-03-18T23:07:59.937 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:07:59.936+0000 7fdb730de640 1 -- 172.21.15.49:0/654691200 wait complete. 2024-03-18T23:07:59.947 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-18T23:08:00.061 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:00.058+0000 7fc6714c6640 1 -- 172.21.15.49:0/507920452 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc66c1022d0 msgr2=0x7fc66c108b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:00.061 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:00.058+0000 7fc6714c6640 1 --2- 172.21.15.49:0/507920452 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc66c1022d0 0x7fc66c108b60 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fc65c009ca0 tx=0x7fc65c02c0f0 comp rx=0 tx=0).stop 2024-03-18T23:08:00.061 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:00.058+0000 7fc6714c6640 1 -- 172.21.15.49:0/507920452 shutdown_connections 2024-03-18T23:08:00.061 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:00.058+0000 7fc6714c6640 1 --2- 172.21.15.49:0/507920452 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc66c1022d0 0x7fc66c108b60 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:00.061 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:00.058+0000 7fc6714c6640 1 --2- 172.21.15.49:0/507920452 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc66c101910 0x7fc66c101d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:00.061 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:00.058+0000 7fc6714c6640 1 --2- 172.21.15.49:0/507920452 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc66c100720 0x7fc66c100b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:00.062 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:00.058+0000 7fc6714c6640 1 -- 172.21.15.49:0/507920452 >> 172.21.15.49:0/507920452 conn(0x7fc66c0fbcb0 msgr2=0x7fc66c0fe0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:08:00.062 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:00.058+0000 7fc6714c6640 1 -- 172.21.15.49:0/507920452 shutdown_connections 2024-03-18T23:08:00.062 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:00.058+0000 7fc6714c6640 1 -- 172.21.15.49:0/507920452 wait complete. 2024-03-18T23:08:00.062 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:00.059+0000 7fc6714c6640 1 Processor -- start 2024-03-18T23:08:00.062 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:00.059+0000 7fc6714c6640 1 -- start start 2024-03-18T23:08:00.062 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:00.059+0000 7fc6714c6640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc66c100720 0x7fc66c069510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:00.062 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:00.059+0000 7fc6714c6640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc66c101910 0x7fc66c19c820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:00.063 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:00.059+0000 7fc6714c6640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc66c1022d0 0x7fc66c19ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:00.063 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:00.059+0000 7fc6714c6640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fc66c10c380 con 0x7fc66c1022d0 2024-03-18T23:08:00.063 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:00.059+0000 7fc6714c6640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fc66c10c200 con 0x7fc66c100720 2024-03-18T23:08:00.063 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:00.059+0000 7fc6714c6640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc66c10c500 con 0x7fc66c101910 2024-03-18T23:08:00.063 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:00.059+0000 7fc66b7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc66c1022d0 0x7fc66c19ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:00.063 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:00.059+0000 7fc66b7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc66c1022d0 0x7fc66c19ee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:37700/0 (socket says 172.21.15.49:37700) 2024-03-18T23:08:00.063 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:00.059+0000 7fc66b7fe640 1 -- 172.21.15.49:0/3207631162 learned_addr learned my addr 172.21.15.49:0/3207631162 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:08:00.063 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:00.059+0000 7fc66affd640 1 --2- 172.21.15.49:0/3207631162 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc66c100720 0x7fc66c069510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:00.063 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:00.059+0000 7fc66a7fc640 1 --2- 172.21.15.49:0/3207631162 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc66c101910 0x7fc66c19c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:00.063 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:00.060+0000 7fc66b7fe640 1 -- 172.21.15.49:0/3207631162 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc66c101910 msgr2=0x7fc66c19c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:00.064 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:00.060+0000 7fc66b7fe640 1 --2- 172.21.15.49:0/3207631162 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc66c101910 0x7fc66c19c820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:00.064 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:00.060+0000 7fc66b7fe640 1 -- 172.21.15.49:0/3207631162 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc66c100720 msgr2=0x7fc66c069510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:00.064 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:00.060+0000 7fc66b7fe640 1 --2- 172.21.15.49:0/3207631162 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc66c100720 0x7fc66c069510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:00.064 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:00.060+0000 7fc66b7fe640 1 -- 172.21.15.49:0/3207631162 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc66c19f720 con 0x7fc66c1022d0 2024-03-18T23:08:00.064 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:00.060+0000 7fc66b7fe640 1 --2- 172.21.15.49:0/3207631162 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc66c1022d0 0x7fc66c19ee60 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fc65c037040 tx=0x7fc65c032f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:08:00.064 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:00.060+0000 7fc663fff640 1 -- 172.21.15.49:0/3207631162 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc65c041020 con 0x7fc66c1022d0 2024-03-18T23:08:00.064 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:00.060+0000 7fc663fff640 1 -- 172.21.15.49:0/3207631162 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc65c034cd0 con 0x7fc66c1022d0 2024-03-18T23:08:00.064 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:00.060+0000 7fc6714c6640 1 -- 172.21.15.49:0/3207631162 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc66c19f9b0 con 0x7fc66c1022d0 2024-03-18T23:08:00.064 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:00.060+0000 7fc663fff640 1 -- 172.21.15.49:0/3207631162 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc65c039040 con 0x7fc66c1022d0 2024-03-18T23:08:00.064 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:00.060+0000 7fc6714c6640 1 -- 172.21.15.49:0/3207631162 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc66c19fea0 con 0x7fc66c1022d0 2024-03-18T23:08:00.064 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:00.061+0000 7fc6714c6640 1 -- 172.21.15.49:0/3207631162 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc66c107a40 con 0x7fc66c1022d0 2024-03-18T23:08:00.065 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:00.062+0000 7fc663fff640 1 -- 172.21.15.49:0/3207631162 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7fc65c044020 con 0x7fc66c1022d0 2024-03-18T23:08:00.065 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:00.063+0000 7fc663fff640 1 --2- 172.21.15.49:0/3207631162 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fc6580a2800 0x7fc6580a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:00.066 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:00.063+0000 7fc66affd640 1 --2- 172.21.15.49:0/3207631162 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fc6580a2800 0x7fc6580a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:00.066 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:00.063+0000 7fc663fff640 1 -- 172.21.15.49:0/3207631162 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7fc65c0f08f0 con 0x7fc66c1022d0 2024-03-18T23:08:00.066 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:00.063+0000 7fc66affd640 1 --2- 172.21.15.49:0/3207631162 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fc6580a2800 0x7fc6580a4cc0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fc66c101770 tx=0x7fc664005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:08:00.067 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:00.068+0000 7fc663fff640 1 -- 172.21.15.49:0/3207631162 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fc65c04b020 con 0x7fc66c1022d0 2024-03-18T23:08:00.252 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:00.252+0000 7fc6714c6640 1 -- 172.21.15.49:0/3207631162 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fc66c02d410 con 0x7fc66c1022d0 2024-03-18T23:08:00.253 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:00.253+0000 7fc663fff640 1 -- 172.21.15.49:0/3207631162 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) ==== 100+0+11 (secure 0 0 0) 0x7fc65c0bda30 con 0x7fc66c1022d0 2024-03-18T23:08:00.253 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 64 2024-03-18T23:08:00.255 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:00.254+0000 7fc6714c6640 1 -- 172.21.15.49:0/3207631162 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fc6580a2800 msgr2=0x7fc6580a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:00.255 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:00.254+0000 7fc6714c6640 1 --2- 172.21.15.49:0/3207631162 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fc6580a2800 0x7fc6580a4cc0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fc66c101770 tx=0x7fc664005f90 comp rx=0 tx=0).stop 2024-03-18T23:08:00.255 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:00.254+0000 7fc6714c6640 1 -- 172.21.15.49:0/3207631162 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc66c1022d0 msgr2=0x7fc66c19ee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:00.255 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:00.254+0000 7fc6714c6640 1 --2- 172.21.15.49:0/3207631162 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc66c1022d0 0x7fc66c19ee60 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fc65c037040 tx=0x7fc65c032f40 comp rx=0 tx=0).stop 2024-03-18T23:08:00.255 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:00.255+0000 7fc6714c6640 1 -- 172.21.15.49:0/3207631162 shutdown_connections 2024-03-18T23:08:00.255 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:00.255+0000 7fc6714c6640 1 --2- 172.21.15.49:0/3207631162 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fc6580a2800 0x7fc6580a4cc0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:00.255 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:00.255+0000 7fc6714c6640 1 --2- 172.21.15.49:0/3207631162 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc66c1022d0 0x7fc66c19ee60 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:00.255 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:00.255+0000 7fc6714c6640 1 --2- 172.21.15.49:0/3207631162 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc66c101910 0x7fc66c19c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:00.255 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:00.255+0000 7fc6714c6640 1 --2- 172.21.15.49:0/3207631162 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc66c100720 0x7fc66c069510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:00.255 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:00.255+0000 7fc6714c6640 1 -- 172.21.15.49:0/3207631162 >> 172.21.15.49:0/3207631162 conn(0x7fc66c0fbcb0 msgr2=0x7fc66c104c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:08:00.255 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:00.255+0000 7fc6714c6640 1 -- 172.21.15.49:0/3207631162 shutdown_connections 2024-03-18T23:08:00.256 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:00.255+0000 7fc6714c6640 1 -- 172.21.15.49:0/3207631162 wait complete. 2024-03-18T23:08:00.264 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:08:00 smithi049 ceph-mon[62991]: from='client.? 172.21.15.49:0/4174881191' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T23:08:00.264 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:08:00 smithi049 ceph-mon[62991]: from='client.? 172.21.15.49:0/816836889' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-18T23:08:00.264 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:08:00 smithi049 ceph-mon[62991]: from='client.? 172.21.15.49:0/654691200' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-18T23:08:00.266 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-18T23:08:00.376 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:00.374+0000 7facc02dc640 1 -- 172.21.15.49:0/3692205015 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7facb81022f0 msgr2=0x7facb8108b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:00.376 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:00.374+0000 7facc02dc640 1 --2- 172.21.15.49:0/3692205015 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7facb81022f0 0x7facb8108b80 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7facb0009ca0 tx=0x7facb002c0f0 comp rx=0 tx=0).stop 2024-03-18T23:08:00.376 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:00.374+0000 7facc02dc640 1 -- 172.21.15.49:0/3692205015 shutdown_connections 2024-03-18T23:08:00.376 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:00.374+0000 7facc02dc640 1 --2- 172.21.15.49:0/3692205015 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7facb81022f0 0x7facb8108b80 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:00.376 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:00.374+0000 7facc02dc640 1 --2- 172.21.15.49:0/3692205015 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7facb8101930 0x7facb8101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:00.376 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:00.374+0000 7facc02dc640 1 --2- 172.21.15.49:0/3692205015 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7facb8100740 0x7facb8100b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:00.376 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:00.374+0000 7facc02dc640 1 -- 172.21.15.49:0/3692205015 >> 172.21.15.49:0/3692205015 conn(0x7facb80fbcb0 msgr2=0x7facb80fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:08:00.376 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:00.374+0000 7facc02dc640 1 -- 172.21.15.49:0/3692205015 shutdown_connections 2024-03-18T23:08:00.376 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:00.374+0000 7facc02dc640 1 -- 172.21.15.49:0/3692205015 wait complete. 2024-03-18T23:08:00.377 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:00.374+0000 7facc02dc640 1 Processor -- start 2024-03-18T23:08:00.377 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:00.374+0000 7facc02dc640 1 -- start start 2024-03-18T23:08:00.377 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:00.375+0000 7facc02dc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7facb8100740 0x7facb80694f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:00.377 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:00.375+0000 7facc02dc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7facb8101930 0x7facb819c880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:00.377 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:00.375+0000 7facc02dc640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7facb81022f0 0x7facb819eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:00.377 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:00.375+0000 7facc02dc640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7facb810c3c0 con 0x7facb8101930 2024-03-18T23:08:00.377 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:00.375+0000 7facc02dc640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7facb810c240 con 0x7facb8100740 2024-03-18T23:08:00.377 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:00.375+0000 7facc02dc640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7facb810c540 con 0x7facb81022f0 2024-03-18T23:08:00.377 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:00.375+0000 7facbd877640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7facb8101930 0x7facb819c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:00.377 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:00.375+0000 7facbd877640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7facb8101930 0x7facb819c880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:37716/0 (socket says 172.21.15.49:37716) 2024-03-18T23:08:00.377 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:00.375+0000 7facbd877640 1 -- 172.21.15.49:0/4167121602 learned_addr learned my addr 172.21.15.49:0/4167121602 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:08:00.377 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:00.375+0000 7facbe078640 1 --2- 172.21.15.49:0/4167121602 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7facb8100740 0x7facb80694f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:00.377 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:00.375+0000 7facbe879640 1 --2- 172.21.15.49:0/4167121602 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7facb81022f0 0x7facb819eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:00.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:00.375+0000 7facbd877640 1 -- 172.21.15.49:0/4167121602 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7facb81022f0 msgr2=0x7facb819eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:00.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:00.375+0000 7facbd877640 1 --2- 172.21.15.49:0/4167121602 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7facb81022f0 0x7facb819eea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:00.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:00.375+0000 7facbd877640 1 -- 172.21.15.49:0/4167121602 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7facb8100740 msgr2=0x7facb80694f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:00.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:00.375+0000 7facbd877640 1 --2- 172.21.15.49:0/4167121602 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7facb8100740 0x7facb80694f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:00.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:00.375+0000 7facbd877640 1 -- 172.21.15.49:0/4167121602 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facb819f7b0 con 0x7facb8101930 2024-03-18T23:08:00.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:00.375+0000 7facbd877640 1 --2- 172.21.15.49:0/4167121602 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7facb8101930 0x7facb819c880 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7faca800ea60 tx=0x7faca800ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:08:00.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:00.375+0000 7facaf7fe640 1 -- 172.21.15.49:0/4167121602 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faca800cde0 con 0x7facb8101930 2024-03-18T23:08:00.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:00.375+0000 7facaf7fe640 1 -- 172.21.15.49:0/4167121602 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faca80035f0 con 0x7facb8101930 2024-03-18T23:08:00.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:00.375+0000 7facc02dc640 1 -- 172.21.15.49:0/4167121602 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7facb819fa40 con 0x7facb8101930 2024-03-18T23:08:00.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:00.375+0000 7facaf7fe640 1 -- 172.21.15.49:0/4167121602 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faca8010670 con 0x7facb8101930 2024-03-18T23:08:00.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:00.375+0000 7facc02dc640 1 -- 172.21.15.49:0/4167121602 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7facb819ff90 con 0x7facb8101930 2024-03-18T23:08:00.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:00.376+0000 7facc02dc640 1 -- 172.21.15.49:0/4167121602 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7facb8107a60 con 0x7facb8101930 2024-03-18T23:08:00.379 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:00.377+0000 7facaf7fe640 1 -- 172.21.15.49:0/4167121602 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7faca80037a0 con 0x7facb8101930 2024-03-18T23:08:00.379 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:00.378+0000 7facaf7fe640 1 --2- 172.21.15.49:0/4167121602 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7faca40a2800 0x7faca40a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:00.379 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:00.378+0000 7facbe078640 1 --2- 172.21.15.49:0/4167121602 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7faca40a2800 0x7faca40a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:00.379 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:00.378+0000 7facaf7fe640 1 -- 172.21.15.49:0/4167121602 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7faca80ced40 con 0x7facb8101930 2024-03-18T23:08:00.379 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:00.378+0000 7facbe078640 1 --2- 172.21.15.49:0/4167121602 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7faca40a2800 0x7faca40a4cc0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7facb8101790 tx=0x7facb4005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:08:00.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:00.382+0000 7facaf7fe640 1 -- 172.21.15.49:0/4167121602 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7faca809bdf0 con 0x7facb8101930 2024-03-18T23:08:00.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:08:00 smithi175 ceph-mon[67352]: from='client.? 172.21.15.49:0/4174881191' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T23:08:00.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:08:00 smithi175 ceph-mon[67352]: from='client.? 172.21.15.49:0/816836889' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-18T23:08:00.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:08:00 smithi175 ceph-mon[67352]: from='client.? 172.21.15.49:0/654691200' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-18T23:08:00.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:08:00 smithi139 ceph-mon[67282]: from='client.? 172.21.15.49:0/4174881191' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T23:08:00.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:08:00 smithi139 ceph-mon[67282]: from='client.? 172.21.15.49:0/816836889' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-18T23:08:00.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:08:00 smithi139 ceph-mon[67282]: from='client.? 172.21.15.49:0/654691200' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-18T23:08:00.604 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:00.604+0000 7facc02dc640 1 -- 172.21.15.49:0/4167121602 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7facb802d1f0 con 0x7facb8101930 2024-03-18T23:08:00.605 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:00.605+0000 7facaf7fe640 1 -- 172.21.15.49:0/4167121602 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11936 (secure 0 0 0) 0x7faca809b6f0 con 0x7facb8101930 2024-03-18T23:08:00.606 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-18T23:08:00.606 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6835/3631301630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":3631301630},{"type":"v1","addr":"172.21.15.49:6835","nonce":3631301630}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6837/444305233","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":444305233},{"type":"v1","addr":"172.21.15.49:6837","nonce":444305233}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-18T23:06:04.575999+0000","modified":"2024-03-18T23:06:11.998736+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24488,"mds_1":24463,"mds_2":24430,"mds_3":24419,"mds_4":24391},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14622":{"gid":14622,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.49:6839/1934085093","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6838","nonce":1934085093},{"type":"v1","addr":"172.21.15.49:6839","nonce":1934085093}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14634":{"gid":14634,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.49:6841/2869298862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6840","nonce":2869298862},{"type":"v1","addr":"172.21.15.49:6841","nonce":2869298862}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24389":{"gid":24389,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6833/3766178603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3766178603},{"type":"v1","addr":"172.21.15.139:6833","nonce":3766178603}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24391":{"gid":24391,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.175:6833/1702997173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":1702997173},{"type":"v1","addr":"172.21.15.175:6833","nonce":1702997173}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":9,"addr":"172.21.15.139:6835/1663212937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":1663212937},{"type":"v1","addr":"172.21.15.139:6835","nonce":1663212937}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.175:6835/755474931","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":755474931},{"type":"v1","addr":"172.21.15.175:6835","nonce":755474931}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6837/379482756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":379482756},{"type":"v1","addr":"172.21.15.175:6837","nonce":379482756}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6837/2330317540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6836","nonce":2330317540},{"type":"v1","addr":"172.21.15.139:6837","nonce":2330317540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"172.21.15.175:6839/3640882084","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":3640882084},{"type":"v1","addr":"172.21.15.175:6839","nonce":3640882084}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.139:6839/4280128209","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6838","nonce":4280128209},{"type":"v1","addr":"172.21.15.139:6839","nonce":4280128209}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24463,24488,24430,24419,24391]},"id":1}]} 2024-03-18T23:08:00.606 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 27 2024-03-18T23:08:00.607 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:00.607+0000 7facc02dc640 1 -- 172.21.15.49:0/4167121602 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7faca40a2800 msgr2=0x7faca40a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:00.607 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:00.607+0000 7facc02dc640 1 --2- 172.21.15.49:0/4167121602 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7faca40a2800 0x7faca40a4cc0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7facb8101790 tx=0x7facb4005f90 comp rx=0 tx=0).stop 2024-03-18T23:08:00.607 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:00.607+0000 7facc02dc640 1 -- 172.21.15.49:0/4167121602 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7facb8101930 msgr2=0x7facb819c880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:00.608 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:00.607+0000 7facc02dc640 1 --2- 172.21.15.49:0/4167121602 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7facb8101930 0x7facb819c880 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7faca800ea60 tx=0x7faca800ef20 comp rx=0 tx=0).stop 2024-03-18T23:08:00.608 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:00.607+0000 7facc02dc640 1 -- 172.21.15.49:0/4167121602 shutdown_connections 2024-03-18T23:08:00.608 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:00.607+0000 7facc02dc640 1 --2- 172.21.15.49:0/4167121602 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7faca40a2800 0x7faca40a4cc0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:00.608 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:00.607+0000 7facc02dc640 1 --2- 172.21.15.49:0/4167121602 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7facb81022f0 0x7facb819eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:00.608 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:00.607+0000 7facc02dc640 1 --2- 172.21.15.49:0/4167121602 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7facb8101930 0x7facb819c880 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:00.608 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:00.607+0000 7facc02dc640 1 --2- 172.21.15.49:0/4167121602 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7facb8100740 0x7facb80694f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:00.608 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:00.607+0000 7facc02dc640 1 -- 172.21.15.49:0/4167121602 >> 172.21.15.49:0/4167121602 conn(0x7facb80fbcb0 msgr2=0x7facb8104c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:08:00.608 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:00.607+0000 7facc02dc640 1 -- 172.21.15.49:0/4167121602 shutdown_connections 2024-03-18T23:08:00.608 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:00.607+0000 7facc02dc640 1 -- 172.21.15.49:0/4167121602 wait complete. 2024-03-18T23:08:00.619 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-18T23:08:00.729 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:00.727+0000 7f141b0b2640 1 -- 172.21.15.49:0/3718209671 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1414107580 msgr2=0x7f1414109970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:00.730 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:00.727+0000 7f141b0b2640 1 --2- 172.21.15.49:0/3718209671 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1414107580 0x7f1414109970 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f140c009ca0 tx=0x7f140c02c0f0 comp rx=0 tx=0).stop 2024-03-18T23:08:00.730 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:00.727+0000 7f141b0b2640 1 -- 172.21.15.49:0/3718209671 shutdown_connections 2024-03-18T23:08:00.730 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:00.727+0000 7f141b0b2640 1 --2- 172.21.15.49:0/3718209671 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1414107580 0x7f1414109970 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:00.730 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:00.727+0000 7f141b0b2640 1 --2- 172.21.15.49:0/3718209671 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1414104c50 0x7f1414107040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:00.730 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:00.727+0000 7f141b0b2640 1 --2- 172.21.15.49:0/3718209671 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1414102320 0x7f1414104710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:00.730 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:00.727+0000 7f141b0b2640 1 -- 172.21.15.49:0/3718209671 >> 172.21.15.49:0/3718209671 conn(0x7f14140fbcb0 msgr2=0x7f14140fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:08:00.730 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:00.727+0000 7f141b0b2640 1 -- 172.21.15.49:0/3718209671 shutdown_connections 2024-03-18T23:08:00.730 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:00.727+0000 7f141b0b2640 1 -- 172.21.15.49:0/3718209671 wait complete. 2024-03-18T23:08:00.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:00.727+0000 7f141b0b2640 1 Processor -- start 2024-03-18T23:08:00.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:00.727+0000 7f141b0b2640 1 -- start start 2024-03-18T23:08:00.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:00.727+0000 7f141b0b2640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1414102320 0x7f1414197b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:00.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:00.727+0000 7f141b0b2640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1414104c50 0x7f1414198090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:00.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:00.727+0000 7f141b0b2640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1414107580 0x7f141419f100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:00.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:00.727+0000 7f141b0b2640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f141410d4e0 con 0x7f1414102320 2024-03-18T23:08:00.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:00.727+0000 7f141b0b2640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f141410d360 con 0x7f1414107580 2024-03-18T23:08:00.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:00.727+0000 7f141b0b2640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f141410d660 con 0x7f1414104c50 2024-03-18T23:08:00.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:00.728+0000 7f1418e4e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1414102320 0x7f1414197b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:00.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:00.728+0000 7f1418e4e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1414102320 0x7f1414197b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:37724/0 (socket says 172.21.15.49:37724) 2024-03-18T23:08:00.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:00.728+0000 7f1418e4e640 1 -- 172.21.15.49:0/2481216016 learned_addr learned my addr 172.21.15.49:0/2481216016 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:08:00.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:00.728+0000 7f1413fff640 1 --2- 172.21.15.49:0/2481216016 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1414104c50 0x7f1414198090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:00.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:00.728+0000 7f141964f640 1 --2- 172.21.15.49:0/2481216016 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1414107580 0x7f141419f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:00.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:00.728+0000 7f1418e4e640 1 -- 172.21.15.49:0/2481216016 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1414104c50 msgr2=0x7f1414198090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:00.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:00.728+0000 7f1418e4e640 1 --2- 172.21.15.49:0/2481216016 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1414104c50 0x7f1414198090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:00.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:00.728+0000 7f1418e4e640 1 -- 172.21.15.49:0/2481216016 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1414107580 msgr2=0x7f141419f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:00.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:00.728+0000 7f1418e4e640 1 --2- 172.21.15.49:0/2481216016 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1414107580 0x7f141419f100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:00.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:00.728+0000 7f1418e4e640 1 -- 172.21.15.49:0/2481216016 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f141419f770 con 0x7f1414102320 2024-03-18T23:08:00.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:00.728+0000 7f1418e4e640 1 --2- 172.21.15.49:0/2481216016 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1414102320 0x7f1414197b50 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f140800e9f0 tx=0x7f140800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:08:00.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:00.728+0000 7f1411ffb640 1 -- 172.21.15.49:0/2481216016 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f140800cd80 con 0x7f1414102320 2024-03-18T23:08:00.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:00.728+0000 7f1411ffb640 1 -- 172.21.15.49:0/2481216016 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1408010910 con 0x7f1414102320 2024-03-18T23:08:00.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:00.728+0000 7f141b0b2640 1 -- 172.21.15.49:0/2481216016 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f141419fa60 con 0x7f1414102320 2024-03-18T23:08:00.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:00.728+0000 7f1411ffb640 1 -- 172.21.15.49:0/2481216016 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14080188f0 con 0x7f1414102320 2024-03-18T23:08:00.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:00.728+0000 7f141b0b2640 1 -- 172.21.15.49:0/2481216016 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f141419ffb0 con 0x7f1414102320 2024-03-18T23:08:00.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:00.729+0000 7f141b0b2640 1 -- 172.21.15.49:0/2481216016 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14140fd7a0 con 0x7f1414102320 2024-03-18T23:08:00.733 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:00.730+0000 7f1411ffb640 1 -- 172.21.15.49:0/2481216016 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7f1408010ac0 con 0x7f1414102320 2024-03-18T23:08:00.733 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:00.731+0000 7f1411ffb640 1 --2- 172.21.15.49:0/2481216016 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f14040a2800 0x7f14040a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:00.733 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:00.731+0000 7f1413fff640 1 --2- 172.21.15.49:0/2481216016 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f14040a2800 0x7f14040a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:00.733 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:00.731+0000 7f1411ffb640 1 -- 172.21.15.49:0/2481216016 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f14080cf1c0 con 0x7f1414102320 2024-03-18T23:08:00.734 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:00.732+0000 7f1413fff640 1 --2- 172.21.15.49:0/2481216016 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f14040a2800 0x7f14040a4cc0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f1414199060 tx=0x7f1400005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:08:00.734 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:00.734+0000 7f1411ffb640 1 -- 172.21.15.49:0/2481216016 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f140809bc20 con 0x7f1414102320 2024-03-18T23:08:00.920 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:00.920+0000 7f141b0b2640 1 -- 172.21.15.49:0/2481216016 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f141402d3c0 con 0x7f1414102320 2024-03-18T23:08:00.922 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:00.922+0000 7f1411ffb640 1 -- 172.21.15.49:0/2481216016 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+22674 (secure 0 0 0) 0x7f1408086090 con 0x7f1414102320 2024-03-18T23:08:00.922 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-18T23:08:00.922 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":88,"fsid":"c63591ea-e57a-11ee-95c9-87774f69a715","created":"2024-03-18T22:57:58.734467+0000","modified":"2024-03-18T23:07:01.337701+0000","last_up_change":"2024-03-18T23:05:16.576310+0000","last_in_change":"2024-03-18T23:04:57.024067+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-18T23:01:28.363432+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-18T23:06:03.172963+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-18T23:06:03.973303+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e7fc9daa-4098-4ba5-8019-9aaef1aab3e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":1651162501},{"type":"v1","addr":"172.21.15.49:6803","nonce":1651162501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":1651162501},{"type":"v1","addr":"172.21.15.49:6805","nonce":1651162501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":1651162501},{"type":"v1","addr":"172.21.15.49:6809","nonce":1651162501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":1651162501},{"type":"v1","addr":"172.21.15.49:6807","nonce":1651162501}]},"public_addr":"172.21.15.49:6803/1651162501","cluster_addr":"172.21.15.49:6805/1651162501","heartbeat_back_addr":"172.21.15.49:6809/1651162501","heartbeat_front_addr":"172.21.15.49:6807/1651162501","state":["exists","up"]},{"osd":1,"uuid":"40ac929c-e90f-434c-a796-9aabd686f1fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":1766700359},{"type":"v1","addr":"172.21.15.139:6801","nonce":1766700359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":1766700359},{"type":"v1","addr":"172.21.15.139:6803","nonce":1766700359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":1766700359},{"type":"v1","addr":"172.21.15.139:6807","nonce":1766700359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":1766700359},{"type":"v1","addr":"172.21.15.139:6805","nonce":1766700359}]},"public_addr":"172.21.15.139:6801/1766700359","cluster_addr":"172.21.15.139:6803/1766700359","heartbeat_back_addr":"172.21.15.139:6807/1766700359","heartbeat_front_addr":"172.21.15.139:6805/1766700359","state":["exists","up"]},{"osd":2,"uuid":"8ae3b666-00b4-45ed-a5ee-4d0771edabb6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":3752671234},{"type":"v1","addr":"172.21.15.175:6801","nonce":3752671234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":3752671234},{"type":"v1","addr":"172.21.15.175:6803","nonce":3752671234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":3752671234},{"type":"v1","addr":"172.21.15.175:6807","nonce":3752671234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":3752671234},{"type":"v1","addr":"172.21.15.175:6805","nonce":3752671234}]},"public_addr":"172.21.15.175:6801/3752671234","cluster_addr":"172.21.15.175:6803/3752671234","heartbeat_back_addr":"172.21.15.175:6807/3752671234","heartbeat_front_addr":"172.21.15.175:6805/3752671234","state":["exists","up"]},{"osd":3,"uuid":"1a62e89d-45d4-4237-b813-7f96a59aca7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":956160536},{"type":"v1","addr":"172.21.15.49:6811","nonce":956160536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":956160536},{"type":"v1","addr":"172.21.15.49:6813","nonce":956160536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":956160536},{"type":"v1","addr":"172.21.15.49:6817","nonce":956160536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":956160536},{"type":"v1","addr":"172.21.15.49:6815","nonce":956160536}]},"public_addr":"172.21.15.49:6811/956160536","cluster_addr":"172.21.15.49:6813/956160536","heartbeat_back_addr":"172.21.15.49:6817/956160536","heartbeat_front_addr":"172.21.15.49:6815/956160536","state":["exists","up"]},{"osd":4,"uuid":"c715101c-0c3f-4b98-b142-d64a35e187df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":3129712668},{"type":"v1","addr":"172.21.15.139:6809","nonce":3129712668}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":3129712668},{"type":"v1","addr":"172.21.15.139:6811","nonce":3129712668}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":3129712668},{"type":"v1","addr":"172.21.15.139:6815","nonce":3129712668}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":3129712668},{"type":"v1","addr":"172.21.15.139:6813","nonce":3129712668}]},"public_addr":"172.21.15.139:6809/3129712668","cluster_addr":"172.21.15.139:6811/3129712668","heartbeat_back_addr":"172.21.15.139:6815/3129712668","heartbeat_front_addr":"172.21.15.139:6813/3129712668","state":["exists","up"]},{"osd":5,"uuid":"4b641f81-77eb-45fd-8464-511e7b3062b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":2774567215},{"type":"v1","addr":"172.21.15.175:6809","nonce":2774567215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":2774567215},{"type":"v1","addr":"172.21.15.175:6811","nonce":2774567215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":2774567215},{"type":"v1","addr":"172.21.15.175:6815","nonce":2774567215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":2774567215},{"type":"v1","addr":"172.21.15.175:6813","nonce":2774567215}]},"public_addr":"172.21.15.175:6809/2774567215","cluster_addr":"172.21.15.175:6811/2774567215","heartbeat_back_addr":"172.21.15.175:6815/2774567215","heartbeat_front_addr":"172.21.15.175:6813/2774567215","state":["exists","up"]},{"osd":6,"uuid":"653da8e0-1197-4b48-abed-470f2a9c4dce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":1472215461},{"type":"v1","addr":"172.21.15.49:6819","nonce":1472215461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":1472215461},{"type":"v1","addr":"172.21.15.49:6821","nonce":1472215461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":1472215461},{"type":"v1","addr":"172.21.15.49:6825","nonce":1472215461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":1472215461},{"type":"v1","addr":"172.21.15.49:6823","nonce":1472215461}]},"public_addr":"172.21.15.49:6819/1472215461","cluster_addr":"172.21.15.49:6821/1472215461","heartbeat_back_addr":"172.21.15.49:6825/1472215461","heartbeat_front_addr":"172.21.15.49:6823/1472215461","state":["exists","up"]},{"osd":7,"uuid":"ad157c9b-4b75-451c-8740-630f0975b7fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":2163986787},{"type":"v1","addr":"172.21.15.139:6817","nonce":2163986787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":2163986787},{"type":"v1","addr":"172.21.15.139:6819","nonce":2163986787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":2163986787},{"type":"v1","addr":"172.21.15.139:6823","nonce":2163986787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":2163986787},{"type":"v1","addr":"172.21.15.139:6821","nonce":2163986787}]},"public_addr":"172.21.15.139:6817/2163986787","cluster_addr":"172.21.15.139:6819/2163986787","heartbeat_back_addr":"172.21.15.139:6823/2163986787","heartbeat_front_addr":"172.21.15.139:6821/2163986787","state":["exists","up"]},{"osd":8,"uuid":"c8e75f15-df75-4d5d-9b92-a064bffb6c01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":1194026992},{"type":"v1","addr":"172.21.15.175:6817","nonce":1194026992}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1194026992},{"type":"v1","addr":"172.21.15.175:6819","nonce":1194026992}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":1194026992},{"type":"v1","addr":"172.21.15.175:6823","nonce":1194026992}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1194026992},{"type":"v1","addr":"172.21.15.175:6821","nonce":1194026992}]},"public_addr":"172.21.15.175:6817/1194026992","cluster_addr":"172.21.15.175:6819/1194026992","heartbeat_back_addr":"172.21.15.175:6823/1194026992","heartbeat_front_addr":"172.21.15.175:6821/1194026992","state":["exists","up"]},{"osd":9,"uuid":"edb5ffef-b6a2-4941-b5ab-40f982ad30d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":2742084884},{"type":"v1","addr":"172.21.15.49:6827","nonce":2742084884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":2742084884},{"type":"v1","addr":"172.21.15.49:6829","nonce":2742084884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":2742084884},{"type":"v1","addr":"172.21.15.49:6833","nonce":2742084884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":2742084884},{"type":"v1","addr":"172.21.15.49:6831","nonce":2742084884}]},"public_addr":"172.21.15.49:6827/2742084884","cluster_addr":"172.21.15.49:6829/2742084884","heartbeat_back_addr":"172.21.15.49:6833/2742084884","heartbeat_front_addr":"172.21.15.49:6831/2742084884","state":["exists","up"]},{"osd":10,"uuid":"3945e46d-766a-4c4a-8821-d43648dd1578","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":2404740201},{"type":"v1","addr":"172.21.15.139:6825","nonce":2404740201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":2404740201},{"type":"v1","addr":"172.21.15.139:6827","nonce":2404740201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":2404740201},{"type":"v1","addr":"172.21.15.139:6831","nonce":2404740201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":2404740201},{"type":"v1","addr":"172.21.15.139:6829","nonce":2404740201}]},"public_addr":"172.21.15.139:6825/2404740201","cluster_addr":"172.21.15.139:6827/2404740201","heartbeat_back_addr":"172.21.15.139:6831/2404740201","heartbeat_front_addr":"172.21.15.139:6829/2404740201","state":["exists","up"]},{"osd":11,"uuid":"9954ba15-d877-46cc-b24a-893f9168deb8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":3519503868},{"type":"v1","addr":"172.21.15.175:6825","nonce":3519503868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":3519503868},{"type":"v1","addr":"172.21.15.175:6827","nonce":3519503868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":3519503868},{"type":"v1","addr":"172.21.15.175:6831","nonce":3519503868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":3519503868},{"type":"v1","addr":"172.21.15.175:6829","nonce":3519503868}]},"public_addr":"172.21.15.175:6825/3519503868","cluster_addr":"172.21.15.175:6827/3519503868","heartbeat_back_addr":"172.21.15.175:6831/3519503868","heartbeat_front_addr":"172.21.15.175:6829/3519503868","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:00:38.367865+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:01:02.237687+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:01:25.609751+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:01:51.268572+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:02:16.514605+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:02:41.586523+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:03:07.029964+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:03:32.956250+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:03:57.547984+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:04:23.948972+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:04:49.667210+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:05:15.284693+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.6","mappings":[{"from":4,"to":6}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.49:0/563527416":"2024-03-19T22:58:59.806253+0000","172.21.15.49:0/1036383534":"2024-03-19T22:58:59.806253+0000","172.21.15.49:6801/2609458951":"2024-03-19T22:58:59.806253+0000","172.21.15.49:0/21283102":"2024-03-19T22:58:31.972474+0000","172.21.15.49:0/1292058922":"2024-03-19T22:58:31.972474+0000","172.21.15.49:6800/1985190312":"2024-03-19T22:58:31.972474+0000","172.21.15.49:0/1912446700":"2024-03-19T22:58:59.806253+0000","172.21.15.49:0/2176210039":"2024-03-19T22:58:31.972474+0000","172.21.15.49:6801/1985190312":"2024-03-19T22:58:31.972474+0000","172.21.15.49:6800/2609458951":"2024-03-19T22:58:59.806253+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-18T23:08:00.925 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:00.924+0000 7f141b0b2640 1 -- 172.21.15.49:0/2481216016 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f14040a2800 msgr2=0x7f14040a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:00.925 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:00.924+0000 7f141b0b2640 1 --2- 172.21.15.49:0/2481216016 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f14040a2800 0x7f14040a4cc0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f1414199060 tx=0x7f1400005f90 comp rx=0 tx=0).stop 2024-03-18T23:08:00.925 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:00.924+0000 7f141b0b2640 1 -- 172.21.15.49:0/2481216016 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1414102320 msgr2=0x7f1414197b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:00.925 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:00.924+0000 7f141b0b2640 1 --2- 172.21.15.49:0/2481216016 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1414102320 0x7f1414197b50 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f140800e9f0 tx=0x7f140800eeb0 comp rx=0 tx=0).stop 2024-03-18T23:08:00.925 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:00.924+0000 7f141b0b2640 1 -- 172.21.15.49:0/2481216016 shutdown_connections 2024-03-18T23:08:00.925 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:00.924+0000 7f141b0b2640 1 --2- 172.21.15.49:0/2481216016 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f14040a2800 0x7f14040a4cc0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:00.925 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:00.924+0000 7f141b0b2640 1 --2- 172.21.15.49:0/2481216016 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1414107580 0x7f141419f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:00.925 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:00.924+0000 7f141b0b2640 1 --2- 172.21.15.49:0/2481216016 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1414104c50 0x7f1414198090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:00.925 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:00.924+0000 7f141b0b2640 1 --2- 172.21.15.49:0/2481216016 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1414102320 0x7f1414197b50 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:00.925 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:00.924+0000 7f141b0b2640 1 -- 172.21.15.49:0/2481216016 >> 172.21.15.49:0/2481216016 conn(0x7f14140fbcb0 msgr2=0x7f1414102c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:08:00.925 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:00.924+0000 7f141b0b2640 1 -- 172.21.15.49:0/2481216016 shutdown_connections 2024-03-18T23:08:00.926 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:00.925+0000 7f141b0b2640 1 -- 172.21.15.49:0/2481216016 wait complete. 2024-03-18T23:08:00.936 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 27, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-03-18T23:06:04.575999+0000', 'modified': '2024-03-18T23:06:11.998736+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24488, 'mds_1': 24463, 'mds_2': 24430, 'mds_3': 24419, 'mds_4': 24391}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14622': {'gid': 14622, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.49:6839/1934085093', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.49:6838', 'nonce': 1934085093}, {'type': 'v1', 'addr': '172.21.15.49:6839', 'nonce': 1934085093}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14634': {'gid': 14634, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.49:6841/2869298862', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.49:6840', 'nonce': 2869298862}, {'type': 'v1', 'addr': '172.21.15.49:6841', 'nonce': 2869298862}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24389': {'gid': 24389, 'name': 'k', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.139:6833/3766178603', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.139:6832', 'nonce': 3766178603}, {'type': 'v1', 'addr': '172.21.15.139:6833', 'nonce': 3766178603}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24391': {'gid': 24391, 'name': 'l', 'rank': 4, 'incarnation': 25, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.175:6833/1702997173', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6832', 'nonce': 1702997173}, {'type': 'v1', 'addr': '172.21.15.175:6833', 'nonce': 1702997173}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24419': {'gid': 24419, 'name': 'e', 'rank': 3, 'incarnation': 23, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.139:6835/1663212937', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.139:6834', 'nonce': 1663212937}, {'type': 'v1', 'addr': '172.21.15.139:6835', 'nonce': 1663212937}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24430': {'gid': 24430, 'name': 'f', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.175:6835/755474931', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6834', 'nonce': 755474931}, {'type': 'v1', 'addr': '172.21.15.175:6835', 'nonce': 755474931}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24439': {'gid': 24439, 'name': 'c', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.175:6837/379482756', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6836', 'nonce': 379482756}, {'type': 'v1', 'addr': '172.21.15.175:6837', 'nonce': 379482756}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24452': {'gid': 24452, 'name': 'h', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.139:6837/2330317540', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.139:6836', 'nonce': 2330317540}, {'type': 'v1', 'addr': '172.21.15.139:6837', 'nonce': 2330317540}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24463': {'gid': 24463, 'name': 'i', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.175:6839/3640882084', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6838', 'nonce': 3640882084}, {'type': 'v1', 'addr': '172.21.15.175:6839', 'nonce': 3640882084}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24488': {'gid': 24488, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.139:6839/4280128209', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.139:6838', 'nonce': 4280128209}, {'type': 'v1', 'addr': '172.21.15.139:6839', 'nonce': 4280128209}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24488, 'qdb_cluster': [24463, 24488, 24430, 24419, 24391]} 2024-03-18T23:08:00.936 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-03-18T23:08:00.936 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.l status 2024-03-18T23:08:01.048 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.046+0000 7f8626dd4640 1 -- 172.21.15.49:0/276438251 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8620101930 msgr2=0x7f8620101db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:01.048 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.046+0000 7f8626dd4640 1 --2- 172.21.15.49:0/276438251 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8620101930 0x7f8620101db0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f8614009900 tx=0x7f861402bc30 comp rx=0 tx=0).stop 2024-03-18T23:08:01.048 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.046+0000 7f8626dd4640 1 -- 172.21.15.49:0/276438251 shutdown_connections 2024-03-18T23:08:01.049 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.046+0000 7f8626dd4640 1 --2- 172.21.15.49:0/276438251 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f86201022f0 0x7f8620108b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:01.049 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.046+0000 7f8626dd4640 1 --2- 172.21.15.49:0/276438251 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8620101930 0x7f8620101db0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:01.049 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.046+0000 7f8626dd4640 1 --2- 172.21.15.49:0/276438251 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8620100740 0x7f8620100b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:01.049 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.046+0000 7f8626dd4640 1 -- 172.21.15.49:0/276438251 >> 172.21.15.49:0/276438251 conn(0x7f86200fbcd0 msgr2=0x7f86200fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:08:01.049 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.046+0000 7f8626dd4640 1 -- 172.21.15.49:0/276438251 shutdown_connections 2024-03-18T23:08:01.049 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.046+0000 7f8626dd4640 1 -- 172.21.15.49:0/276438251 wait complete. 2024-03-18T23:08:01.049 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.047+0000 7f8626dd4640 1 Processor -- start 2024-03-18T23:08:01.049 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.047+0000 7f8626dd4640 1 -- start start 2024-03-18T23:08:01.049 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.047+0000 7f8626dd4640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8620100740 0x7f8620197b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:01.049 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.047+0000 7f8626dd4640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8620101930 0x7f8620198080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:01.049 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.047+0000 7f8626dd4640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f86201022f0 0x7f862019f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:01.049 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.047+0000 7f8626dd4640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f862010d4e0 con 0x7f8620100740 2024-03-18T23:08:01.049 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.047+0000 7f8626dd4640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f862010d360 con 0x7f86201022f0 2024-03-18T23:08:01.049 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.047+0000 7f8626dd4640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f862010d660 con 0x7f8620101930 2024-03-18T23:08:01.050 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.047+0000 7f8624b70640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8620100740 0x7f8620197b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:01.050 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.047+0000 7f8624b70640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8620100740 0x7f8620197b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:37736/0 (socket says 172.21.15.49:37736) 2024-03-18T23:08:01.050 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.047+0000 7f8624b70640 1 -- 172.21.15.49:0/4060928691 learned_addr learned my addr 172.21.15.49:0/4060928691 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:08:01.050 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.047+0000 7f861ffff640 1 --2- 172.21.15.49:0/4060928691 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8620101930 0x7f8620198080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:01.050 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.047+0000 7f8625371640 1 --2- 172.21.15.49:0/4060928691 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f86201022f0 0x7f862019f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:01.050 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.047+0000 7f8624b70640 1 -- 172.21.15.49:0/4060928691 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8620101930 msgr2=0x7f8620198080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:01.050 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.047+0000 7f8624b70640 1 --2- 172.21.15.49:0/4060928691 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8620101930 0x7f8620198080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:01.050 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.047+0000 7f8624b70640 1 -- 172.21.15.49:0/4060928691 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f86201022f0 msgr2=0x7f862019f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:01.050 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.047+0000 7f8624b70640 1 --2- 172.21.15.49:0/4060928691 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f86201022f0 0x7f862019f0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:01.050 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.048+0000 7f8624b70640 1 -- 172.21.15.49:0/4060928691 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f862019f760 con 0x7f8620100740 2024-03-18T23:08:01.050 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.048+0000 7f8625371640 1 --2- 172.21.15.49:0/4060928691 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f86201022f0 0x7f862019f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T23:08:01.050 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.048+0000 7f8624b70640 1 --2- 172.21.15.49:0/4060928691 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8620100740 0x7f8620197b40 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f861000e9c0 tx=0x7f861000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:08:01.050 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.048+0000 7f861dffb640 1 -- 172.21.15.49:0/4060928691 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f861000cde0 con 0x7f8620100740 2024-03-18T23:08:01.051 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.048+0000 7f8626dd4640 1 -- 172.21.15.49:0/4060928691 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f862019fa50 con 0x7f8620100740 2024-03-18T23:08:01.051 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.048+0000 7f861dffb640 1 -- 172.21.15.49:0/4060928691 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8610010910 con 0x7f8620100740 2024-03-18T23:08:01.051 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.048+0000 7f861dffb640 1 -- 172.21.15.49:0/4060928691 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8610018a10 con 0x7f8620100740 2024-03-18T23:08:01.051 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.048+0000 7f8626dd4640 1 -- 172.21.15.49:0/4060928691 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f862019ffc0 con 0x7f8620100740 2024-03-18T23:08:01.057 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.050+0000 7f861dffb640 1 -- 172.21.15.49:0/4060928691 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7f861001f020 con 0x7f8620100740 2024-03-18T23:08:01.057 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.051+0000 7f86299c3740 1 Processor -- start 2024-03-18T23:08:01.057 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.051+0000 7f86299c3740 1 -- start start 2024-03-18T23:08:01.057 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.051+0000 7f861dffb640 1 --2- 172.21.15.49:0/4060928691 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f86080a2800 0x7f86080a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:01.057 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.051+0000 7f86299c3740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5564ed4e3ff0 0x5564ed4e4410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:01.057 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.051+0000 7f86299c3740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5564ed4e49e0 0x5564ed4e94e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:01.057 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.051+0000 7f86299c3740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5564ed4e9b50 0x5564ed4ebfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:01.057 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.051+0000 7f86299c3740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5564ed448de0 con 0x5564ed4e3ff0 2024-03-18T23:08:01.057 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.051+0000 7f86299c3740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x5564ed3f1c80 con 0x5564ed4e49e0 2024-03-18T23:08:01.058 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.051+0000 7f86299c3740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5564ed444e80 con 0x5564ed4e9b50 2024-03-18T23:08:01.058 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.051+0000 7f8625371640 1 --2- 172.21.15.49:0/4060928691 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f86080a2800 0x7f86080a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:01.058 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.051+0000 7f861dffb640 1 -- 172.21.15.49:0/4060928691 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f86100ced20 con 0x7f8620100740 2024-03-18T23:08:01.058 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.051+0000 7f8624b70640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5564ed4e3ff0 0x5564ed4e4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:01.058 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.051+0000 7f861ffff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5564ed4e49e0 0x5564ed4e94e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:01.058 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.051+0000 7f8624b70640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5564ed4e3ff0 0x5564ed4e4410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:37740/0 (socket says 172.21.15.49:37740) 2024-03-18T23:08:01.058 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.051+0000 7f8624b70640 1 -- 172.21.15.49:0/3299287472 learned_addr learned my addr 172.21.15.49:0/3299287472 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:08:01.058 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.051+0000 7f861ffff640 1 --2- 172.21.15.49:0/3299287472 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5564ed4e49e0 0x5564ed4e94e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.49:48526/0 (socket says 172.21.15.49:48526) 2024-03-18T23:08:01.058 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.051+0000 7f8625371640 1 --2- 172.21.15.49:0/3299287472 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5564ed4e9b50 0x5564ed4ebfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:01.058 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.052+0000 7f8624b70640 1 -- 172.21.15.49:0/3299287472 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5564ed4e9b50 msgr2=0x5564ed4ebfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:08:01.058 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.052+0000 7f8624b70640 1 --2- 172.21.15.49:0/3299287472 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5564ed4e9b50 0x5564ed4ebfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:01.059 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.052+0000 7f8624b70640 1 -- 172.21.15.49:0/3299287472 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5564ed4e49e0 msgr2=0x5564ed4e94e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:08:01.059 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.052+0000 7f8624b70640 1 --2- 172.21.15.49:0/3299287472 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5564ed4e49e0 0x5564ed4e94e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:01.059 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.052+0000 7f8624b70640 1 -- 172.21.15.49:0/3299287472 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5564ed403fb0 con 0x5564ed4e3ff0 2024-03-18T23:08:01.059 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.052+0000 7f861ffff640 1 --2- 172.21.15.49:0/3299287472 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5564ed4e49e0 0x5564ed4e94e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:08:01.059 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.052+0000 7f8625371640 1 --2- 172.21.15.49:0/4060928691 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f86080a2800 0x7f86080a4cc0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f8618009fd0 tx=0x7f8618014cb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:08:01.059 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.052+0000 7f8624b70640 1 --2- 172.21.15.49:0/3299287472 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5564ed4e3ff0 0x5564ed4e4410 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f86100a38e0 tx=0x7f861009afd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f731fe9289bc5d8b server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:08:01.059 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.052+0000 7f8606ffd640 1 -- 172.21.15.49:0/3299287472 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86100bb050 con 0x5564ed4e3ff0 2024-03-18T23:08:01.059 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.052+0000 7f8606ffd640 1 -- 172.21.15.49:0/3299287472 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f861009b670 con 0x5564ed4e3ff0 2024-03-18T23:08:01.059 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.052+0000 7f8606ffd640 1 -- 172.21.15.49:0/3299287472 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86100cc8e0 con 0x5564ed4e3ff0 2024-03-18T23:08:01.059 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.053+0000 7f86299c3740 1 -- 172.21.15.49:0/3299287472 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5564ed4e3ff0 msgr2=0x5564ed4e4410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:01.059 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.053+0000 7f86299c3740 1 --2- 172.21.15.49:0/3299287472 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5564ed4e3ff0 0x5564ed4e4410 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f86100a38e0 tx=0x7f861009afd0 comp rx=0 tx=0).stop 2024-03-18T23:08:01.059 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.053+0000 7f86299c3740 1 -- 172.21.15.49:0/3299287472 shutdown_connections 2024-03-18T23:08:01.059 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.053+0000 7f86299c3740 1 --2- 172.21.15.49:0/3299287472 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5564ed4e9b50 0x5564ed4ebfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:01.059 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.053+0000 7f86299c3740 1 --2- 172.21.15.49:0/3299287472 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5564ed4e49e0 0x5564ed4e94e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:01.060 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.053+0000 7f86299c3740 1 --2- 172.21.15.49:0/3299287472 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5564ed4e3ff0 0x5564ed4e4410 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:01.060 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.053+0000 7f86299c3740 1 -- 172.21.15.49:0/3299287472 >> 172.21.15.49:0/3299287472 conn(0x5564ed4df560 msgr2=0x5564ed4df960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:08:01.060 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.053+0000 7f86299c3740 1 -- 172.21.15.49:0/3299287472 shutdown_connections 2024-03-18T23:08:01.060 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.053+0000 7f86299c3740 1 -- 172.21.15.49:0/3299287472 wait complete. 2024-03-18T23:08:01.060 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.053+0000 7f86299c3740 1 -- start start 2024-03-18T23:08:01.060 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.053+0000 7f86299c3740 1 Processor -- start 2024-03-18T23:08:01.060 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.053+0000 7f86299c3740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5564ed4e3ff0 0x5564ed4f28c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:01.060 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.054+0000 7f86299c3740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5564ed4e49e0 0x5564ed4f2e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:01.060 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.054+0000 7f86299c3740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5564ed4e9b50 0x5564ed4f9e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:01.060 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.054+0000 7f86299c3740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5564ed4f0eb0 con 0x5564ed4e3ff0 2024-03-18T23:08:01.060 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.054+0000 7f86299c3740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x5564ed404fc0 con 0x5564ed4e9b50 2024-03-18T23:08:01.060 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.054+0000 7f86299c3740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5564ed4f1030 con 0x5564ed4e49e0 2024-03-18T23:08:01.060 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.054+0000 7f86299c3740 10 client.0 _ll_register_callbacks cb 0x5564ed440a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T23:08:01.060 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.054+0000 7f8624b70640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5564ed4e3ff0 0x5564ed4f28c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:01.061 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.054+0000 7f8624b70640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5564ed4e3ff0 0x5564ed4f28c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:37746/0 (socket says 172.21.15.49:37746) 2024-03-18T23:08:01.061 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.054+0000 7f8624b70640 1 -- 172.21.15.49:0/336401068 learned_addr learned my addr 172.21.15.49:0/336401068 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:08:01.061 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.054+0000 7f861ffff640 1 --2- 172.21.15.49:0/336401068 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5564ed4e49e0 0x5564ed4f2e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:01.061 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.054+0000 7f8625371640 1 --2- 172.21.15.49:0/336401068 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5564ed4e9b50 0x5564ed4f9e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:01.061 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.054+0000 7f8624b70640 1 -- 172.21.15.49:0/336401068 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5564ed4e49e0 msgr2=0x5564ed4f2e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:08:01.061 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.054+0000 7f8624b70640 1 --2- 172.21.15.49:0/336401068 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5564ed4e49e0 0x5564ed4f2e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:01.061 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.054+0000 7f8624b70640 1 -- 172.21.15.49:0/336401068 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5564ed4e9b50 msgr2=0x5564ed4f9e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:08:01.061 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.054+0000 7f8624b70640 1 --2- 172.21.15.49:0/336401068 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5564ed4e9b50 0x5564ed4f9e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:01.061 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.054+0000 7f8624b70640 1 -- 172.21.15.49:0/336401068 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5564ed15d2d0 con 0x5564ed4e3ff0 2024-03-18T23:08:01.061 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.054+0000 7f8624b70640 1 --2- 172.21.15.49:0/336401068 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5564ed4e3ff0 0x5564ed4f28c0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f8610097ad0 tx=0x7f86100997a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb55dc9716b617a5 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:08:01.061 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.054+0000 7f85fffff640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-18T23:08:01.061 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.054+0000 7f85fffff640 1 -- 172.21.15.49:0/336401068 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8610094020 con 0x5564ed4e3ff0 2024-03-18T23:08:01.061 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.054+0000 7f85fffff640 1 -- 172.21.15.49:0/336401068 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f8610097ce0 con 0x5564ed4e3ff0 2024-03-18T23:08:01.061 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.054+0000 7f86299c3740 1 -- 172.21.15.49:0/336401068 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f861009b670 con 0x5564ed4e3ff0 2024-03-18T23:08:01.062 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.054+0000 7f85fffff640 1 -- 172.21.15.49:0/336401068 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f861008c030 con 0x5564ed4e3ff0 2024-03-18T23:08:01.062 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.054+0000 7f86299c3740 10 client.15249 fetch_fsmap learned FSMap version 27 2024-03-18T23:08:01.062 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.054+0000 7f86299c3740 1 -- 172.21.15.49:0/336401068 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5564ed2276b0 con 0x5564ed4e3ff0 2024-03-18T23:08:01.062 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.055+0000 7f85fffff640 1 -- 172.21.15.49:0/336401068 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f861009b670 con 0x5564ed4e3ff0 2024-03-18T23:08:01.062 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.055+0000 7f85fffff640 1 -- 172.21.15.49:0/336401068 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f8610090390 con 0x5564ed4e3ff0 2024-03-18T23:08:01.062 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.055+0000 7f86299c3740 10 client.15249 fetch_fsmap finished waiting for FSMap version 27 2024-03-18T23:08:01.062 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.055+0000 7f86299c3740 10 client.15249 resolve_mds: resolved name 'l' to daemon mds.l 2024-03-18T23:08:01.062 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.055+0000 7f86299c3740 20 client.15249 populate_metadata read hostname 'smithi049' 2024-03-18T23:08:01.062 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.055+0000 7f86299c3740 1 --2- 172.21.15.49:0/336401068 >> [v2:172.21.15.175:6832/1702997173,v1:172.21.15.175:6833/1702997173] conn(0x5564ed4ff110 0x5564ed500580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:01.062 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.055+0000 7f86299c3740 4 client.15249 mds_command: new command op to 24391 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T23:08:01.062 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.055+0000 7f86299c3740 1 -- 172.21.15.49:0/336401068 --> [v2:172.21.15.175:6832/1702997173,v1:172.21.15.175:6833/1702997173] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5564ed456320 con 0x5564ed4ff110 2024-03-18T23:08:01.062 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.055+0000 7f861ffff640 1 --2- 172.21.15.49:0/336401068 >> [v2:172.21.15.175:6832/1702997173,v1:172.21.15.175:6833/1702997173] conn(0x5564ed4ff110 0x5564ed500580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:01.063 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.055+0000 7f861ffff640 1 --2- 172.21.15.49:0/336401068 >> [v2:172.21.15.175:6832/1702997173,v1:172.21.15.175:6833/1702997173] conn(0x5564ed4ff110 0x5564ed500580 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=51fce3291b58ff8c server_cookie=8e07f92da2d2ace2 in_seq=0 out_seq=0 2024-03-18T23:08:01.063 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.055+0000 7f85fffff640 10 client.15249 ms_handle_connect on v2:172.21.15.175:6832/1702997173 2024-03-18T23:08:01.063 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.056+0000 7f85fffff640 1 -- 172.21.15.49:0/336401068 <== mds.4 v2:172.21.15.175:6832/1702997173 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5564ed456320 con 0x5564ed4ff110 2024-03-18T23:08:01.063 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.056+0000 7f85fffff640 10 client.15249 handle_command_reply: tid=0 2024-03-18T23:08:01.063 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.056+0000 7f86299c3740 1 client.15249 shutdown 2024-03-18T23:08:01.063 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.056+0000 7f86299c3740 1 -- 172.21.15.49:0/336401068 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5564ed4e3ff0 msgr2=0x5564ed4f28c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:01.063 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.056+0000 7f86299c3740 1 --2- 172.21.15.49:0/336401068 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5564ed4e3ff0 0x5564ed4f28c0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f8610097ad0 tx=0x7f86100997a0 comp rx=0 tx=0).stop 2024-03-18T23:08:01.063 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.056+0000 7f86299c3740 1 -- 172.21.15.49:0/336401068 shutdown_connections 2024-03-18T23:08:01.063 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.056+0000 7f86299c3740 1 --2- 172.21.15.49:0/336401068 >> [v2:172.21.15.175:6832/1702997173,v1:172.21.15.175:6833/1702997173] conn(0x5564ed4ff110 0x5564ed500580 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:01.063 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.056+0000 7f86299c3740 1 --2- 172.21.15.49:0/336401068 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5564ed4e9b50 0x5564ed4f9e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:01.063 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.056+0000 7f86299c3740 1 --2- 172.21.15.49:0/336401068 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5564ed4e49e0 0x5564ed4f2e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:01.063 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.056+0000 7f85fffff640 0 client.15249 ms_handle_reset on v2:172.21.15.175:6832/1702997173 2024-03-18T23:08:01.063 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.056+0000 7f86299c3740 1 --2- 172.21.15.49:0/336401068 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5564ed4e3ff0 0x5564ed4f28c0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:01.064 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.056+0000 7f86299c3740 1 -- 172.21.15.49:0/336401068 >> 172.21.15.49:0/336401068 conn(0x5564ed4df560 msgr2=0x5564ed4de7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:08:01.064 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.056+0000 7f86299c3740 1 -- 172.21.15.49:0/336401068 shutdown_connections 2024-03-18T23:08:01.064 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.056+0000 7f86299c3740 1 -- 172.21.15.49:0/336401068 wait complete. 2024-03-18T23:08:01.064 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.056+0000 7f86299c3740 20 client.15249 trim_cache size 0 max 16384 2024-03-18T23:08:01.076 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.075+0000 7f86299c3740 1 Processor -- start 2024-03-18T23:08:01.076 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.075+0000 7f86299c3740 1 -- start start 2024-03-18T23:08:01.077 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.075+0000 7f86299c3740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5564ed4e3ff0 0x5564ed4e9510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:01.077 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.075+0000 7f86299c3740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5564ed4e49e0 0x5564ed4f9e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:01.077 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.075+0000 7f86299c3740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5564ed4e9b50 0x5564ed4f2be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:01.077 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.075+0000 7f86299c3740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5564ed4fe8c0 con 0x5564ed4e3ff0 2024-03-18T23:08:01.077 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.075+0000 7f86299c3740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x5564ed4fef90 con 0x5564ed4e49e0 2024-03-18T23:08:01.077 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.075+0000 7f86299c3740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5564ed4ff660 con 0x5564ed4e9b50 2024-03-18T23:08:01.077 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.075+0000 7f8624b70640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5564ed4e3ff0 0x5564ed4e9510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:01.077 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.075+0000 7f8624b70640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5564ed4e3ff0 0x5564ed4e9510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:37758/0 (socket says 172.21.15.49:37758) 2024-03-18T23:08:01.077 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.075+0000 7f8625371640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5564ed4e9b50 0x5564ed4f2be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:01.077 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.075+0000 7f8624b70640 1 -- 172.21.15.49:0/434053613 learned_addr learned my addr 172.21.15.49:0/434053613 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:08:01.077 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.075+0000 7f861ffff640 1 --2- 172.21.15.49:0/434053613 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5564ed4e49e0 0x5564ed4f9e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:01.077 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.075+0000 7f8624b70640 1 -- 172.21.15.49:0/434053613 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5564ed4e9b50 msgr2=0x5564ed4f2be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:08:01.077 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.075+0000 7f8624b70640 1 --2- 172.21.15.49:0/434053613 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5564ed4e9b50 0x5564ed4f2be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:01.077 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.075+0000 7f8624b70640 1 -- 172.21.15.49:0/434053613 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5564ed4e49e0 msgr2=0x5564ed4f9e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:08:01.078 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.075+0000 7f8624b70640 1 --2- 172.21.15.49:0/434053613 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5564ed4e49e0 0x5564ed4f9e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:01.078 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.075+0000 7f8624b70640 1 -- 172.21.15.49:0/434053613 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5564ed4d9bf0 con 0x5564ed4e3ff0 2024-03-18T23:08:01.078 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.076+0000 7f8624b70640 1 --2- 172.21.15.49:0/434053613 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5564ed4e3ff0 0x5564ed4e9510 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f8610090750 tx=0x7f861009b100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf9a94f11ce9c61a server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:08:01.078 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.076+0000 7f86067fc640 1 -- 172.21.15.49:0/434053613 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8610094030 con 0x5564ed4e3ff0 2024-03-18T23:08:01.078 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.076+0000 7f86067fc640 1 -- 172.21.15.49:0/434053613 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f861009b670 con 0x5564ed4e3ff0 2024-03-18T23:08:01.078 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.076+0000 7f86067fc640 1 -- 172.21.15.49:0/434053613 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86100bb040 con 0x5564ed4e3ff0 2024-03-18T23:08:01.083 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.077+0000 7f86299c3740 1 -- 172.21.15.49:0/434053613 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5564ed4e3ff0 msgr2=0x5564ed4e9510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:01.083 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.077+0000 7f86299c3740 1 --2- 172.21.15.49:0/434053613 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5564ed4e3ff0 0x5564ed4e9510 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f8610090750 tx=0x7f861009b100 comp rx=0 tx=0).stop 2024-03-18T23:08:01.083 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.077+0000 7f86299c3740 1 -- 172.21.15.49:0/434053613 shutdown_connections 2024-03-18T23:08:01.083 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.077+0000 7f86299c3740 1 --2- 172.21.15.49:0/434053613 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5564ed4e9b50 0x5564ed4f2be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:01.083 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.077+0000 7f86299c3740 1 --2- 172.21.15.49:0/434053613 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5564ed4e49e0 0x5564ed4f9e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:01.084 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.077+0000 7f86299c3740 1 --2- 172.21.15.49:0/434053613 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5564ed4e3ff0 0x5564ed4e9510 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:01.084 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.077+0000 7f86299c3740 1 -- 172.21.15.49:0/434053613 >> 172.21.15.49:0/434053613 conn(0x5564ed4df560 msgr2=0x5564ed4f24d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:08:01.084 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.077+0000 7f86299c3740 1 -- 172.21.15.49:0/434053613 shutdown_connections 2024-03-18T23:08:01.084 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.077+0000 7f86299c3740 1 -- 172.21.15.49:0/434053613 wait complete. 2024-03-18T23:08:01.084 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.077+0000 7f86299c3740 1 -- start start 2024-03-18T23:08:01.084 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.078+0000 7f86299c3740 1 Processor -- start 2024-03-18T23:08:01.084 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.078+0000 7f86299c3740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5564ed4e3ff0 0x5564ed4f5a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:01.084 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.078+0000 7f86299c3740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5564ed4e49e0 0x5564ed4f5fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:01.084 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.078+0000 7f86299c3740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5564ed4e9b50 0x5564ed4f8fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:01.084 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.078+0000 7f86299c3740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5564ed500400 con 0x5564ed4e3ff0 2024-03-18T23:08:01.084 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.078+0000 7f86299c3740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x5564ed4ffd30 con 0x5564ed4e49e0 2024-03-18T23:08:01.084 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.078+0000 7f86299c3740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5564ed500ad0 con 0x5564ed4e9b50 2024-03-18T23:08:01.085 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.078+0000 7f86299c3740 10 client.0 _ll_register_callbacks cb 0x7f861008a070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T23:08:01.085 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.078+0000 7f8624b70640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5564ed4e3ff0 0x5564ed4f5a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:01.085 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.078+0000 7f8624b70640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5564ed4e3ff0 0x5564ed4f5a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:37760/0 (socket says 172.21.15.49:37760) 2024-03-18T23:08:01.085 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.078+0000 7f8624b70640 1 -- 172.21.15.49:0/596815932 learned_addr learned my addr 172.21.15.49:0/596815932 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:08:01.085 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.078+0000 7f861ffff640 1 --2- 172.21.15.49:0/596815932 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5564ed4e49e0 0x5564ed4f5fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:01.085 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.078+0000 7f8625371640 1 --2- 172.21.15.49:0/596815932 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5564ed4e9b50 0x5564ed4f8fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:01.085 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.078+0000 7f8624b70640 1 -- 172.21.15.49:0/596815932 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5564ed4e9b50 msgr2=0x5564ed4f8fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:08:01.085 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.078+0000 7f8624b70640 1 --2- 172.21.15.49:0/596815932 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5564ed4e9b50 0x5564ed4f8fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:01.085 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.078+0000 7f8624b70640 1 -- 172.21.15.49:0/596815932 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5564ed4e49e0 msgr2=0x5564ed4f5fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:08:01.085 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.078+0000 7f8624b70640 1 --2- 172.21.15.49:0/596815932 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5564ed4e49e0 0x5564ed4f5fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:01.085 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.078+0000 7f8624b70640 1 -- 172.21.15.49:0/596815932 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5564ed4f95c0 con 0x5564ed4e3ff0 2024-03-18T23:08:01.085 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.078+0000 7f861ffff640 1 --2- 172.21.15.49:0/596815932 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5564ed4e49e0 0x5564ed4f5fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:08:01.085 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.078+0000 7f8624b70640 1 --2- 172.21.15.49:0/596815932 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5564ed4e3ff0 0x5564ed4f5a70 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f861009b370 tx=0x7f8610099970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65766181e81f2440 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:08:01.085 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.078+0000 7f85fffff640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-18T23:08:01.086 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.079+0000 7f85fffff640 1 -- 172.21.15.49:0/596815932 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86100b9070 con 0x5564ed4e3ff0 2024-03-18T23:08:01.086 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.079+0000 7f85fffff640 1 -- 172.21.15.49:0/596815932 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f8610099710 con 0x5564ed4e3ff0 2024-03-18T23:08:01.086 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.079+0000 7f86299c3740 1 -- 172.21.15.49:0/596815932 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f861009b670 con 0x5564ed4e3ff0 2024-03-18T23:08:01.086 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.079+0000 7f85fffff640 1 -- 172.21.15.49:0/596815932 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f86100bb040 con 0x5564ed4e3ff0 2024-03-18T23:08:01.086 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.079+0000 7f86299c3740 10 client.15255 fetch_fsmap learned FSMap version 27 2024-03-18T23:08:01.086 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.079+0000 7f86299c3740 1 -- 172.21.15.49:0/596815932 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5564ed4f95c0 con 0x5564ed4e3ff0 2024-03-18T23:08:01.086 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.079+0000 7f85fffff640 1 -- 172.21.15.49:0/596815932 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f861009b670 con 0x5564ed4e3ff0 2024-03-18T23:08:01.086 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.079+0000 7f85fffff640 1 -- 172.21.15.49:0/596815932 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f86100a3a20 con 0x5564ed4e3ff0 2024-03-18T23:08:01.086 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.079+0000 7f86299c3740 10 client.15255 fetch_fsmap finished waiting for FSMap version 27 2024-03-18T23:08:01.087 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.079+0000 7f86299c3740 10 client.15255 resolve_mds: resolved name 'l' to daemon mds.l 2024-03-18T23:08:01.087 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.079+0000 7f86299c3740 20 client.15255 populate_metadata read hostname 'smithi049' 2024-03-18T23:08:01.087 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.079+0000 7f86299c3740 1 --2- 172.21.15.49:0/596815932 >> [v2:172.21.15.175:6832/1702997173,v1:172.21.15.175:6833/1702997173] conn(0x5564ed569a30 0x5564ed568740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:01.087 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.079+0000 7f86299c3740 4 client.15255 mds_command: new command op to 24391 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-18T23:08:01.087 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.079+0000 7f86299c3740 1 -- 172.21.15.49:0/596815932 --> [v2:172.21.15.175:6832/1702997173,v1:172.21.15.175:6833/1702997173] -- command(tid 0: {"prefix": "status"}) -- 0x5564ed450670 con 0x5564ed569a30 2024-03-18T23:08:01.087 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.079+0000 7f861ffff640 1 --2- 172.21.15.49:0/596815932 >> [v2:172.21.15.175:6832/1702997173,v1:172.21.15.175:6833/1702997173] conn(0x5564ed569a30 0x5564ed568740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:01.087 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.080+0000 7f85fffff640 10 client.15255 ms_handle_connect on v2:172.21.15.175:6832/1702997173 2024-03-18T23:08:01.087 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.080+0000 7f861ffff640 1 --2- 172.21.15.49:0/596815932 >> [v2:172.21.15.175:6832/1702997173,v1:172.21.15.175:6833/1702997173] conn(0x5564ed569a30 0x5564ed568740 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=f52bb1abba902d57 server_cookie=4d96f9d42f7d0977 in_seq=0 out_seq=0 2024-03-18T23:08:01.087 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.080+0000 7f85fffff640 1 -- 172.21.15.49:0/596815932 <== mds.4 v2:172.21.15.175:6832/1702997173 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x5564ed450670 con 0x5564ed569a30 2024-03-18T23:08:01.087 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.080+0000 7f85fffff640 10 client.15255 handle_command_reply: tid=0 2024-03-18T23:08:01.087 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.080+0000 7f86299c3740 1 client.15255 shutdown 2024-03-18T23:08:01.088 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.080+0000 7f86299c3740 1 -- 172.21.15.49:0/596815932 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5564ed4e3ff0 msgr2=0x5564ed4f5a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:01.088 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.080+0000 7f86299c3740 1 --2- 172.21.15.49:0/596815932 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5564ed4e3ff0 0x5564ed4f5a70 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f861009b370 tx=0x7f8610099970 comp rx=0 tx=0).stop 2024-03-18T23:08:01.088 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.080+0000 7f86299c3740 1 -- 172.21.15.49:0/596815932 shutdown_connections 2024-03-18T23:08:01.088 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.080+0000 7f86299c3740 1 --2- 172.21.15.49:0/596815932 >> [v2:172.21.15.175:6832/1702997173,v1:172.21.15.175:6833/1702997173] conn(0x5564ed569a30 0x5564ed568740 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:01.088 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.080+0000 7f86299c3740 1 --2- 172.21.15.49:0/596815932 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5564ed4e9b50 0x5564ed4f8fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:01.088 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.080+0000 7f85fffff640 0 client.15255 ms_handle_reset on v2:172.21.15.175:6832/1702997173 2024-03-18T23:08:01.088 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.080+0000 7f86299c3740 1 --2- 172.21.15.49:0/596815932 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5564ed4e49e0 0x5564ed4f5fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:01.088 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.080+0000 7f86299c3740 1 --2- 172.21.15.49:0/596815932 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5564ed4e3ff0 0x5564ed4f5a70 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:01.088 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.080+0000 7f86299c3740 1 -- 172.21.15.49:0/596815932 >> 172.21.15.49:0/596815932 conn(0x5564ed4df560 msgr2=0x5564ed4e3050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:08:01.088 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.080+0000 7f86299c3740 1 -- 172.21.15.49:0/596815932 shutdown_connections 2024-03-18T23:08:01.088 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.080+0000 7f86299c3740 1 -- 172.21.15.49:0/596815932 wait complete. 2024-03-18T23:08:01.088 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.080+0000 7f86299c3740 20 client.15255 trim_cache size 0 max 16384 2024-03-18T23:08:01.089 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-03-18T23:08:01.089 INFO:teuthology.orchestra.run.smithi049.stdout: "cluster_fsid": "c63591ea-e57a-11ee-95c9-87774f69a715", 2024-03-18T23:08:01.089 INFO:teuthology.orchestra.run.smithi049.stdout: "whoami": 4, 2024-03-18T23:08:01.089 INFO:teuthology.orchestra.run.smithi049.stdout: "id": 24391, 2024-03-18T23:08:01.089 INFO:teuthology.orchestra.run.smithi049.stdout: "want_state": "up:active", 2024-03-18T23:08:01.089 INFO:teuthology.orchestra.run.smithi049.stdout: "state": "up:active", 2024-03-18T23:08:01.089 INFO:teuthology.orchestra.run.smithi049.stdout: "fs_name": "cephfs", 2024-03-18T23:08:01.089 INFO:teuthology.orchestra.run.smithi049.stdout: "rank_uptime": 110.08486333899999, 2024-03-18T23:08:01.089 INFO:teuthology.orchestra.run.smithi049.stdout: "mdsmap_epoch": 27, 2024-03-18T23:08:01.090 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_epoch": 88, 2024-03-18T23:08:01.090 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_epoch_barrier": 86, 2024-03-18T23:08:01.090 INFO:teuthology.orchestra.run.smithi049.stdout: "uptime": 151.610167287 2024-03-18T23:08:01.090 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-03-18T23:08:01.091 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.081+0000 7f8626dd4640 1 -- 172.21.15.49:0/4060928691 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f86080a2800 msgr2=0x7f86080a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:01.091 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.081+0000 7f8626dd4640 1 --2- 172.21.15.49:0/4060928691 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f86080a2800 0x7f86080a4cc0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f8618009fd0 tx=0x7f8618014cb0 comp rx=0 tx=0).stop 2024-03-18T23:08:01.091 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.081+0000 7f8626dd4640 1 -- 172.21.15.49:0/4060928691 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8620100740 msgr2=0x7f8620197b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:01.091 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.081+0000 7f8626dd4640 1 --2- 172.21.15.49:0/4060928691 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8620100740 0x7f8620197b40 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f861000e9c0 tx=0x7f861000ee80 comp rx=0 tx=0).stop 2024-03-18T23:08:01.091 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.081+0000 7f8626dd4640 1 -- 172.21.15.49:0/4060928691 shutdown_connections 2024-03-18T23:08:01.091 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.081+0000 7f8626dd4640 1 --2- 172.21.15.49:0/4060928691 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f86080a2800 0x7f86080a4cc0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:01.091 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.081+0000 7f8626dd4640 1 --2- 172.21.15.49:0/4060928691 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f86201022f0 0x7f862019f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:01.091 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.081+0000 7f8626dd4640 1 --2- 172.21.15.49:0/4060928691 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8620101930 0x7f8620198080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:01.091 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.081+0000 7f8626dd4640 1 --2- 172.21.15.49:0/4060928691 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8620100740 0x7f8620197b40 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:01.091 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.081+0000 7f8626dd4640 1 -- 172.21.15.49:0/4060928691 >> 172.21.15.49:0/4060928691 conn(0x7f86200fbcd0 msgr2=0x7f8620104c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:08:01.091 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.081+0000 7f8626dd4640 1 -- 172.21.15.49:0/4060928691 shutdown_connections 2024-03-18T23:08:01.091 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.081+0000 7f8626dd4640 1 -- 172.21.15.49:0/4060928691 wait complete. 2024-03-18T23:08:01.092 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.e status 2024-03-18T23:08:01.203 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.201+0000 7f7892885640 1 -- 172.21.15.49:0/351437348 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f788c102a20 msgr2=0x7f788c102ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:01.204 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.201+0000 7f7892885640 1 --2- 172.21.15.49:0/351437348 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f788c102a20 0x7f788c102ea0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f7880009900 tx=0x7f788002bc30 comp rx=0 tx=0).stop 2024-03-18T23:08:01.204 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.202+0000 7f7892885640 1 -- 172.21.15.49:0/351437348 shutdown_connections 2024-03-18T23:08:01.204 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.202+0000 7f7892885640 1 --2- 172.21.15.49:0/351437348 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f788c1033e0 0x7f788c109c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:01.204 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.202+0000 7f7892885640 1 --2- 172.21.15.49:0/351437348 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f788c102a20 0x7f788c102ea0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:01.204 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.202+0000 7f7892885640 1 --2- 172.21.15.49:0/351437348 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f788c101830 0x7f788c101c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:01.204 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.202+0000 7f7892885640 1 -- 172.21.15.49:0/351437348 >> 172.21.15.49:0/351437348 conn(0x7f788c0fcda0 msgr2=0x7f788c0ff1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:08:01.204 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.202+0000 7f7892885640 1 -- 172.21.15.49:0/351437348 shutdown_connections 2024-03-18T23:08:01.204 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.202+0000 7f7892885640 1 -- 172.21.15.49:0/351437348 wait complete. 2024-03-18T23:08:01.204 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.202+0000 7f7892885640 1 Processor -- start 2024-03-18T23:08:01.204 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.202+0000 7f7892885640 1 -- start start 2024-03-18T23:08:01.205 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.202+0000 7f7892885640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f788c101830 0x7f788c197b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:01.205 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.202+0000 7f7892885640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f788c102a20 0x7f788c1980d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:01.205 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.202+0000 7f7892885640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f788c1033e0 0x7f788c19f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:01.205 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.202+0000 7f7892885640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f788c10d490 con 0x7f788c101830 2024-03-18T23:08:01.205 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.202+0000 7f7892885640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f788c10d310 con 0x7f788c102a20 2024-03-18T23:08:01.205 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.202+0000 7f7892885640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f788c10d610 con 0x7f788c1033e0 2024-03-18T23:08:01.205 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.203+0000 7f788bfff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f788c101830 0x7f788c197b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:01.205 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.203+0000 7f788bfff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f788c101830 0x7f788c197b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:37776/0 (socket says 172.21.15.49:37776) 2024-03-18T23:08:01.205 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.203+0000 7f788bfff640 1 -- 172.21.15.49:0/759313609 learned_addr learned my addr 172.21.15.49:0/759313609 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:08:01.205 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.203+0000 7f7890e22640 1 --2- 172.21.15.49:0/759313609 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f788c1033e0 0x7f788c19f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:01.205 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.203+0000 7f788b7fe640 1 --2- 172.21.15.49:0/759313609 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f788c102a20 0x7f788c1980d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:01.205 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.203+0000 7f7890e22640 1 -- 172.21.15.49:0/759313609 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f788c102a20 msgr2=0x7f788c1980d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:01.205 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.203+0000 7f7890e22640 1 --2- 172.21.15.49:0/759313609 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f788c102a20 0x7f788c1980d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:01.205 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.203+0000 7f7890e22640 1 -- 172.21.15.49:0/759313609 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f788c101830 msgr2=0x7f788c197b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:01.205 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.203+0000 7f7890e22640 1 --2- 172.21.15.49:0/759313609 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f788c101830 0x7f788c197b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:01.206 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.203+0000 7f7890e22640 1 -- 172.21.15.49:0/759313609 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f788c19f7b0 con 0x7f788c1033e0 2024-03-18T23:08:01.206 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.203+0000 7f788bfff640 1 --2- 172.21.15.49:0/759313609 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f788c101830 0x7f788c197b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:08:01.206 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.203+0000 7f7890e22640 1 --2- 172.21.15.49:0/759313609 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f788c1033e0 0x7f788c19f140 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f7884009fd0 tx=0x7f788400ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:08:01.206 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.203+0000 7f78897fa640 1 -- 172.21.15.49:0/759313609 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f788400ce20 con 0x7f788c1033e0 2024-03-18T23:08:01.206 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.203+0000 7f78897fa640 1 -- 172.21.15.49:0/759313609 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7884005020 con 0x7f788c1033e0 2024-03-18T23:08:01.206 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.203+0000 7f7892885640 1 -- 172.21.15.49:0/759313609 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f788c19faa0 con 0x7f788c1033e0 2024-03-18T23:08:01.206 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.203+0000 7f78897fa640 1 -- 172.21.15.49:0/759313609 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7884010720 con 0x7f788c1033e0 2024-03-18T23:08:01.206 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.203+0000 7f7892885640 1 -- 172.21.15.49:0/759313609 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f788c19ff70 con 0x7f788c1033e0 2024-03-18T23:08:01.213 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.205+0000 7f78897fa640 1 -- 172.21.15.49:0/759313609 <== mon.1 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7f7884005540 con 0x7f788c1033e0 2024-03-18T23:08:01.213 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.206+0000 7f789547a740 1 Processor -- start 2024-03-18T23:08:01.213 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.206+0000 7f789547a740 1 -- start start 2024-03-18T23:08:01.213 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.206+0000 7f789547a740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a29ac6fff0 0x55a29ac70410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:01.213 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.206+0000 7f78897fa640 1 --2- 172.21.15.49:0/759313609 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f78740a2800 0x7f78740a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:01.213 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.206+0000 7f789547a740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a29ac709e0 0x55a29ac754e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:01.213 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.206+0000 7f789547a740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a29ac75b50 0x55a29ac77fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:01.213 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.206+0000 7f789547a740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55a29abd4de0 con 0x55a29ac75b50 2024-03-18T23:08:01.213 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.206+0000 7f789547a740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55a29ab7dc80 con 0x55a29ac709e0 2024-03-18T23:08:01.213 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.206+0000 7f789547a740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55a29abd0e80 con 0x55a29ac6fff0 2024-03-18T23:08:01.214 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.206+0000 7f7890e22640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a29ac75b50 0x55a29ac77fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:01.214 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.206+0000 7f788b7fe640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a29ac6fff0 0x55a29ac70410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:01.214 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.206+0000 7f7890e22640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a29ac75b50 0x55a29ac77fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:37784/0 (socket says 172.21.15.49:37784) 2024-03-18T23:08:01.214 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.206+0000 7f7890e22640 1 -- 172.21.15.49:0/2966961436 learned_addr learned my addr 172.21.15.49:0/2966961436 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:08:01.214 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.206+0000 7f788bfff640 1 --2- 172.21.15.49:0/759313609 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f78740a2800 0x7f78740a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:01.214 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.206+0000 7f788b7fe640 1 --2- 172.21.15.49:0/2966961436 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a29ac709e0 0x55a29ac754e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:01.214 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.206+0000 7f78897fa640 1 -- 172.21.15.49:0/759313609 <== mon.1 v2:172.21.15.175:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f78840cf190 con 0x7f788c1033e0 2024-03-18T23:08:01.214 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.207+0000 7f788b7fe640 1 -- 172.21.15.49:0/2966961436 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a29ac709e0 msgr2=0x55a29ac754e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:08:01.214 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.207+0000 7f788b7fe640 1 --2- 172.21.15.49:0/2966961436 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a29ac709e0 0x55a29ac754e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:01.214 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.207+0000 7f788b7fe640 1 -- 172.21.15.49:0/2966961436 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a29ac75b50 msgr2=0x55a29ac77fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:08:01.214 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.207+0000 7f788b7fe640 1 --2- 172.21.15.49:0/2966961436 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a29ac75b50 0x55a29ac77fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:01.214 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.207+0000 7f788b7fe640 1 -- 172.21.15.49:0/2966961436 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a29ab8ffb0 con 0x55a29ac6fff0 2024-03-18T23:08:01.215 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.207+0000 7f788bfff640 1 --2- 172.21.15.49:0/759313609 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f78740a2800 0x7f78740a4cc0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f787c0095f0 tx=0x7f787c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:08:01.215 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.207+0000 7f788b7fe640 1 --2- 172.21.15.49:0/2966961436 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a29ac6fff0 0x55a29ac70410 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f7880009230 tx=0x7f7880005b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5e684c07a90f79c0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:08:01.215 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.207+0000 7f78727fc640 1 -- 172.21.15.49:0/2966961436 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7880004020 con 0x55a29ac6fff0 2024-03-18T23:08:01.215 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.207+0000 7f78727fc640 1 -- 172.21.15.49:0/2966961436 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f78800041c0 con 0x55a29ac6fff0 2024-03-18T23:08:01.215 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.207+0000 7f78727fc640 1 -- 172.21.15.49:0/2966961436 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7880034440 con 0x55a29ac6fff0 2024-03-18T23:08:01.215 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.208+0000 7f789547a740 1 -- 172.21.15.49:0/2966961436 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a29ac6fff0 msgr2=0x55a29ac70410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:01.215 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.208+0000 7f789547a740 1 --2- 172.21.15.49:0/2966961436 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a29ac6fff0 0x55a29ac70410 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f7880009230 tx=0x7f7880005b50 comp rx=0 tx=0).stop 2024-03-18T23:08:01.215 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.208+0000 7f789547a740 1 -- 172.21.15.49:0/2966961436 shutdown_connections 2024-03-18T23:08:01.215 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.208+0000 7f789547a740 1 --2- 172.21.15.49:0/2966961436 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a29ac75b50 0x55a29ac77fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:01.215 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.208+0000 7f789547a740 1 --2- 172.21.15.49:0/2966961436 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a29ac709e0 0x55a29ac754e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:01.215 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.208+0000 7f789547a740 1 --2- 172.21.15.49:0/2966961436 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a29ac6fff0 0x55a29ac70410 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:01.215 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.208+0000 7f789547a740 1 -- 172.21.15.49:0/2966961436 >> 172.21.15.49:0/2966961436 conn(0x55a29ac6b560 msgr2=0x55a29ac6b960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:08:01.215 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.208+0000 7f789547a740 1 -- 172.21.15.49:0/2966961436 shutdown_connections 2024-03-18T23:08:01.215 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.208+0000 7f789547a740 1 -- 172.21.15.49:0/2966961436 wait complete. 2024-03-18T23:08:01.215 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.208+0000 7f789547a740 1 -- start start 2024-03-18T23:08:01.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.208+0000 7f789547a740 1 Processor -- start 2024-03-18T23:08:01.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.208+0000 7f789547a740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a29ac6fff0 0x55a29ac7a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:01.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.208+0000 7f789547a740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a29ac709e0 0x55a29ac7aa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:01.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.208+0000 7f789547a740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a29ac75b50 0x55a29ac81a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:01.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.208+0000 7f789547a740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55a29ac78aa0 con 0x55a29ac75b50 2024-03-18T23:08:01.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.208+0000 7f789547a740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55a29ab90fc0 con 0x55a29ac6fff0 2024-03-18T23:08:01.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.208+0000 7f789547a740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55a29ac78c20 con 0x55a29ac709e0 2024-03-18T23:08:01.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.209+0000 7f789547a740 10 client.0 _ll_register_callbacks cb 0x55a29abcca20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T23:08:01.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.209+0000 7f7890e22640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a29ac75b50 0x55a29ac81a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:01.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.209+0000 7f788bfff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a29ac6fff0 0x55a29ac7a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:01.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.209+0000 7f788b7fe640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a29ac709e0 0x55a29ac7aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:01.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.209+0000 7f788bfff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a29ac6fff0 0x55a29ac7a4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.49:48586/0 (socket says 172.21.15.49:48586) 2024-03-18T23:08:01.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.209+0000 7f7890e22640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a29ac75b50 0x55a29ac81a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:37798/0 (socket says 172.21.15.49:37798) 2024-03-18T23:08:01.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.209+0000 7f788bfff640 1 -- 172.21.15.49:0/3220755613 learned_addr learned my addr 172.21.15.49:0/3220755613 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:08:01.217 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.209+0000 7f7890e22640 1 -- 172.21.15.49:0/3220755613 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a29ac709e0 msgr2=0x55a29ac7aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:08:01.217 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.209+0000 7f7890e22640 1 --2- 172.21.15.49:0/3220755613 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a29ac709e0 0x55a29ac7aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:01.217 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.209+0000 7f7890e22640 1 -- 172.21.15.49:0/3220755613 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a29ac6fff0 msgr2=0x55a29ac7a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:08:01.217 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.209+0000 7f7890e22640 1 --2- 172.21.15.49:0/3220755613 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a29ac6fff0 0x55a29ac7a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:01.217 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.209+0000 7f7890e22640 1 -- 172.21.15.49:0/3220755613 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a29a8e92d0 con 0x55a29ac75b50 2024-03-18T23:08:01.217 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.209+0000 7f788bfff640 1 --2- 172.21.15.49:0/3220755613 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a29ac6fff0 0x55a29ac7a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:08:01.217 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.209+0000 7f7890e22640 1 --2- 172.21.15.49:0/3220755613 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a29ac75b50 0x55a29ac81a60 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f788400c760 tx=0x7f788409b440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5f01f1cf7579d32 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:08:01.217 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.209+0000 7f786b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-18T23:08:01.217 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.209+0000 7f786b7fe640 1 -- 172.21.15.49:0/3220755613 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7884064e60 con 0x55a29ac75b50 2024-03-18T23:08:01.217 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.209+0000 7f786b7fe640 1 -- 172.21.15.49:0/3220755613 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f78840ccd50 con 0x55a29ac75b50 2024-03-18T23:08:01.217 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.209+0000 7f789547a740 1 -- 172.21.15.49:0/3220755613 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f78800041c0 con 0x55a29ac75b50 2024-03-18T23:08:01.217 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.209+0000 7f786b7fe640 1 -- 172.21.15.49:0/3220755613 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f788409c1e0 con 0x55a29ac75b50 2024-03-18T23:08:01.217 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.209+0000 7f789547a740 10 client.15267 fetch_fsmap learned FSMap version 27 2024-03-18T23:08:01.217 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.209+0000 7f789547a740 1 -- 172.21.15.49:0/3220755613 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a29a9b36b0 con 0x55a29ac75b50 2024-03-18T23:08:01.218 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.210+0000 7f786b7fe640 1 -- 172.21.15.49:0/3220755613 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f78800041c0 con 0x55a29ac75b50 2024-03-18T23:08:01.218 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.210+0000 7f786b7fe640 1 -- 172.21.15.49:0/3220755613 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f7884066910 con 0x55a29ac75b50 2024-03-18T23:08:01.218 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.210+0000 7f789547a740 10 client.15267 fetch_fsmap finished waiting for FSMap version 27 2024-03-18T23:08:01.218 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.210+0000 7f789547a740 10 client.15267 resolve_mds: resolved name 'e' to daemon mds.e 2024-03-18T23:08:01.218 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.210+0000 7f789547a740 20 client.15267 populate_metadata read hostname 'smithi049' 2024-03-18T23:08:01.218 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.210+0000 7f789547a740 1 --2- 172.21.15.49:0/3220755613 >> [v2:172.21.15.139:6834/1663212937,v1:172.21.15.139:6835/1663212937] conn(0x55a29ac87280 0x55a29ac886f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:01.218 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.210+0000 7f789547a740 4 client.15267 mds_command: new command op to 24419 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T23:08:01.218 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.210+0000 7f789547a740 1 -- 172.21.15.49:0/3220755613 --> [v2:172.21.15.139:6834/1663212937,v1:172.21.15.139:6835/1663212937] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a29abe2320 con 0x55a29ac87280 2024-03-18T23:08:01.218 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.210+0000 7f788bfff640 1 --2- 172.21.15.49:0/3220755613 >> [v2:172.21.15.139:6834/1663212937,v1:172.21.15.139:6835/1663212937] conn(0x55a29ac87280 0x55a29ac886f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:01.218 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.210+0000 7f788bfff640 1 --2- 172.21.15.49:0/3220755613 >> [v2:172.21.15.139:6834/1663212937,v1:172.21.15.139:6835/1663212937] conn(0x55a29ac87280 0x55a29ac886f0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=598c16f81240ee23 server_cookie=cde9f1903c3fe754 in_seq=0 out_seq=0 2024-03-18T23:08:01.218 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.210+0000 7f786b7fe640 10 client.15267 ms_handle_connect on v2:172.21.15.139:6834/1663212937 2024-03-18T23:08:01.218 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.211+0000 7f786b7fe640 1 -- 172.21.15.49:0/3220755613 <== mds.3 v2:172.21.15.139:6834/1663212937 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55a29abe2320 con 0x55a29ac87280 2024-03-18T23:08:01.218 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.211+0000 7f786b7fe640 10 client.15267 handle_command_reply: tid=0 2024-03-18T23:08:01.219 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.211+0000 7f789547a740 1 client.15267 shutdown 2024-03-18T23:08:01.219 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.211+0000 7f789547a740 1 -- 172.21.15.49:0/3220755613 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a29ac75b50 msgr2=0x55a29ac81a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:01.219 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.211+0000 7f789547a740 1 --2- 172.21.15.49:0/3220755613 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a29ac75b50 0x55a29ac81a60 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f788400c760 tx=0x7f788409b440 comp rx=0 tx=0).stop 2024-03-18T23:08:01.219 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.211+0000 7f789547a740 1 -- 172.21.15.49:0/3220755613 shutdown_connections 2024-03-18T23:08:01.219 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.211+0000 7f789547a740 1 --2- 172.21.15.49:0/3220755613 >> [v2:172.21.15.139:6834/1663212937,v1:172.21.15.139:6835/1663212937] conn(0x55a29ac87280 0x55a29ac886f0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:01.219 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.211+0000 7f789547a740 1 --2- 172.21.15.49:0/3220755613 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a29ac75b50 0x55a29ac81a60 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:01.219 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.211+0000 7f789547a740 1 --2- 172.21.15.49:0/3220755613 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a29ac709e0 0x55a29ac7aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:01.219 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.211+0000 7f786b7fe640 0 client.15267 ms_handle_reset on v2:172.21.15.139:6834/1663212937 2024-03-18T23:08:01.219 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.211+0000 7f789547a740 1 --2- 172.21.15.49:0/3220755613 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a29ac6fff0 0x55a29ac7a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:01.219 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.211+0000 7f789547a740 1 -- 172.21.15.49:0/3220755613 >> 172.21.15.49:0/3220755613 conn(0x55a29ac6b560 msgr2=0x55a29ac6a7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:08:01.219 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.211+0000 7f789547a740 1 -- 172.21.15.49:0/3220755613 shutdown_connections 2024-03-18T23:08:01.219 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.211+0000 7f789547a740 1 -- 172.21.15.49:0/3220755613 wait complete. 2024-03-18T23:08:01.219 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.212+0000 7f789547a740 20 client.15267 trim_cache size 0 max 16384 2024-03-18T23:08:01.235 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.230+0000 7f789547a740 1 Processor -- start 2024-03-18T23:08:01.235 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.230+0000 7f789547a740 1 -- start start 2024-03-18T23:08:01.235 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.230+0000 7f789547a740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a29ac6fff0 0x55a29ac6b940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:01.236 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.230+0000 7f789547a740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a29ac709e0 0x55a29ac81a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:01.236 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.230+0000 7f789547a740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a29ac75b50 0x55a29ac78fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:01.236 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.230+0000 7f789547a740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55a29ac86a30 con 0x55a29ac75b50 2024-03-18T23:08:01.236 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.230+0000 7f789547a740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55a29ac87100 con 0x55a29ac709e0 2024-03-18T23:08:01.236 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.230+0000 7f789547a740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55a29ac877d0 con 0x55a29ac6fff0 2024-03-18T23:08:01.236 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.230+0000 7f7890e22640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a29ac75b50 0x55a29ac78fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:01.236 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.230+0000 7f7890e22640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a29ac75b50 0x55a29ac78fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:37800/0 (socket says 172.21.15.49:37800) 2024-03-18T23:08:01.236 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.230+0000 7f7890e22640 1 -- 172.21.15.49:0/774529077 learned_addr learned my addr 172.21.15.49:0/774529077 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:08:01.236 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.230+0000 7f788bfff640 1 --2- 172.21.15.49:0/774529077 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a29ac6fff0 0x55a29ac6b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:01.236 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.230+0000 7f788b7fe640 1 --2- 172.21.15.49:0/774529077 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a29ac709e0 0x55a29ac81a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:01.236 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.230+0000 7f7890e22640 1 -- 172.21.15.49:0/774529077 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a29ac6fff0 msgr2=0x55a29ac6b940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:08:01.236 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.230+0000 7f7890e22640 1 --2- 172.21.15.49:0/774529077 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a29ac6fff0 0x55a29ac6b940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:01.236 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.230+0000 7f7890e22640 1 -- 172.21.15.49:0/774529077 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a29ac709e0 msgr2=0x55a29ac81a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:08:01.236 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.230+0000 7f7890e22640 1 --2- 172.21.15.49:0/774529077 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a29ac709e0 0x55a29ac81a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:01.237 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.230+0000 7f7890e22640 1 -- 172.21.15.49:0/774529077 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a29ac87c80 con 0x55a29ac75b50 2024-03-18T23:08:01.237 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.230+0000 7f7890e22640 1 --2- 172.21.15.49:0/774529077 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a29ac75b50 0x55a29ac78fc0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f7884066c80 tx=0x7f78840054b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cb6b0cd494ab032 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:08:01.237 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.230+0000 7f7871ffb640 1 -- 172.21.15.49:0/774529077 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78840ad030 con 0x55a29ac75b50 2024-03-18T23:08:01.237 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.230+0000 7f7871ffb640 1 -- 172.21.15.49:0/774529077 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f788409fc30 con 0x55a29ac75b50 2024-03-18T23:08:01.237 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.230+0000 7f7871ffb640 1 -- 172.21.15.49:0/774529077 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f788409b190 con 0x55a29ac75b50 2024-03-18T23:08:01.237 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.231+0000 7f789547a740 1 -- 172.21.15.49:0/774529077 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a29ac75b50 msgr2=0x55a29ac78fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:01.237 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.231+0000 7f789547a740 1 --2- 172.21.15.49:0/774529077 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a29ac75b50 0x55a29ac78fc0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f7884066c80 tx=0x7f78840054b0 comp rx=0 tx=0).stop 2024-03-18T23:08:01.237 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.231+0000 7f789547a740 1 -- 172.21.15.49:0/774529077 shutdown_connections 2024-03-18T23:08:01.237 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.231+0000 7f789547a740 1 --2- 172.21.15.49:0/774529077 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a29ac75b50 0x55a29ac78fc0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:01.237 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.231+0000 7f789547a740 1 --2- 172.21.15.49:0/774529077 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a29ac709e0 0x55a29ac81a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:01.237 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.231+0000 7f789547a740 1 --2- 172.21.15.49:0/774529077 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a29ac6fff0 0x55a29ac6b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:01.238 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.231+0000 7f789547a740 1 -- 172.21.15.49:0/774529077 >> 172.21.15.49:0/774529077 conn(0x55a29ac6b560 msgr2=0x55a29ac71270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:08:01.238 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.231+0000 7f789547a740 1 -- 172.21.15.49:0/774529077 shutdown_connections 2024-03-18T23:08:01.238 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.231+0000 7f789547a740 1 -- 172.21.15.49:0/774529077 wait complete. 2024-03-18T23:08:01.238 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.231+0000 7f789547a740 1 -- start start 2024-03-18T23:08:01.238 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.232+0000 7f789547a740 1 Processor -- start 2024-03-18T23:08:01.238 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.232+0000 7f789547a740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a29ac6fff0 0x55a29ac7fa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:01.238 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.232+0000 7f789547a740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a29ac709e0 0x55a29ace8fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:01.238 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.232+0000 7f789547a740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a29ac75b50 0x55a29aceb640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:01.238 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.232+0000 7f789547a740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55a29ac88570 con 0x55a29ac709e0 2024-03-18T23:08:01.238 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.232+0000 7f789547a740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55a29ac87ea0 con 0x55a29ac75b50 2024-03-18T23:08:01.238 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.232+0000 7f789547a740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55a29ac88c40 con 0x55a29ac6fff0 2024-03-18T23:08:01.238 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.232+0000 7f789547a740 10 client.0 _ll_register_callbacks cb 0x7f788409b8b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T23:08:01.238 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.232+0000 7f788b7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a29ac709e0 0x55a29ace8fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:01.238 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.232+0000 7f788b7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a29ac709e0 0x55a29ace8fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:37810/0 (socket says 172.21.15.49:37810) 2024-03-18T23:08:01.239 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.232+0000 7f788b7fe640 1 -- 172.21.15.49:0/4242480220 learned_addr learned my addr 172.21.15.49:0/4242480220 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:08:01.239 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.232+0000 7f788bfff640 1 --2- 172.21.15.49:0/4242480220 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a29ac6fff0 0x55a29ac7fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:01.239 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.232+0000 7f7890e22640 1 --2- 172.21.15.49:0/4242480220 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a29ac75b50 0x55a29aceb640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:01.239 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.232+0000 7f788b7fe640 1 -- 172.21.15.49:0/4242480220 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a29ac6fff0 msgr2=0x55a29ac7fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:08:01.239 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.232+0000 7f788b7fe640 1 --2- 172.21.15.49:0/4242480220 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a29ac6fff0 0x55a29ac7fa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:01.239 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.232+0000 7f788b7fe640 1 -- 172.21.15.49:0/4242480220 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a29ac75b50 msgr2=0x55a29aceb640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:08:01.239 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.232+0000 7f788b7fe640 1 --2- 172.21.15.49:0/4242480220 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a29ac75b50 0x55a29aceb640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:01.239 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.232+0000 7f788b7fe640 1 -- 172.21.15.49:0/4242480220 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a29acebe70 con 0x55a29ac709e0 2024-03-18T23:08:01.239 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.232+0000 7f788b7fe640 1 --2- 172.21.15.49:0/4242480220 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a29ac709e0 0x55a29ace8fd0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f7880009900 tx=0x7f7880009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dacda3f67a5bf9ae server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:08:01.239 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.232+0000 7f786b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-18T23:08:01.239 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.232+0000 7f786b7fe640 1 -- 172.21.15.49:0/4242480220 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f788004f020 con 0x55a29ac709e0 2024-03-18T23:08:01.239 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.233+0000 7f786b7fe640 1 -- 172.21.15.49:0/4242480220 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f7880003a00 con 0x55a29ac709e0 2024-03-18T23:08:01.239 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.233+0000 7f789547a740 1 -- 172.21.15.49:0/4242480220 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f788409fc30 con 0x55a29ac709e0 2024-03-18T23:08:01.239 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.233+0000 7f786b7fe640 1 -- 172.21.15.49:0/4242480220 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f78800057b0 con 0x55a29ac709e0 2024-03-18T23:08:01.240 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.233+0000 7f789547a740 10 client.15273 fetch_fsmap learned FSMap version 27 2024-03-18T23:08:01.240 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.233+0000 7f789547a740 1 -- 172.21.15.49:0/4242480220 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a29aced170 con 0x55a29ac709e0 2024-03-18T23:08:01.240 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.233+0000 7f786b7fe640 1 -- 172.21.15.49:0/4242480220 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f788409fc30 con 0x55a29ac709e0 2024-03-18T23:08:01.240 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.233+0000 7f786b7fe640 1 -- 172.21.15.49:0/4242480220 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f788004a030 con 0x55a29ac709e0 2024-03-18T23:08:01.240 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.233+0000 7f789547a740 10 client.15273 fetch_fsmap finished waiting for FSMap version 27 2024-03-18T23:08:01.240 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.233+0000 7f789547a740 10 client.15273 resolve_mds: resolved name 'e' to daemon mds.e 2024-03-18T23:08:01.240 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.233+0000 7f789547a740 20 client.15273 populate_metadata read hostname 'smithi049' 2024-03-18T23:08:01.240 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.233+0000 7f789547a740 1 --2- 172.21.15.49:0/4242480220 >> [v2:172.21.15.139:6834/1663212937,v1:172.21.15.139:6835/1663212937] conn(0x55a29acf1820 0x55a29acf0530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:01.240 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.233+0000 7f789547a740 4 client.15273 mds_command: new command op to 24419 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-18T23:08:01.240 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.233+0000 7f789547a740 1 -- 172.21.15.49:0/4242480220 --> [v2:172.21.15.139:6834/1663212937,v1:172.21.15.139:6835/1663212937] -- command(tid 0: {"prefix": "status"}) -- 0x55a29abdc670 con 0x55a29acf1820 2024-03-18T23:08:01.240 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.233+0000 7f788bfff640 1 --2- 172.21.15.49:0/4242480220 >> [v2:172.21.15.139:6834/1663212937,v1:172.21.15.139:6835/1663212937] conn(0x55a29acf1820 0x55a29acf0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:01.240 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.234+0000 7f788bfff640 1 --2- 172.21.15.49:0/4242480220 >> [v2:172.21.15.139:6834/1663212937,v1:172.21.15.139:6835/1663212937] conn(0x55a29acf1820 0x55a29acf0530 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=300429e102a2f297 server_cookie=cbb773bfc345d54e in_seq=0 out_seq=0 2024-03-18T23:08:01.240 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.234+0000 7f786b7fe640 10 client.15273 ms_handle_connect on v2:172.21.15.139:6834/1663212937 2024-03-18T23:08:01.240 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.234+0000 7f786b7fe640 1 -- 172.21.15.49:0/4242480220 <== mds.3 v2:172.21.15.139:6834/1663212937 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55a29abdc670 con 0x55a29acf1820 2024-03-18T23:08:01.241 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.234+0000 7f786b7fe640 10 client.15273 handle_command_reply: tid=0 2024-03-18T23:08:01.241 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.234+0000 7f789547a740 1 client.15273 shutdown 2024-03-18T23:08:01.241 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.234+0000 7f789547a740 1 -- 172.21.15.49:0/4242480220 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a29ac709e0 msgr2=0x55a29ace8fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:01.241 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.234+0000 7f789547a740 1 --2- 172.21.15.49:0/4242480220 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a29ac709e0 0x55a29ace8fd0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f7880009900 tx=0x7f7880009260 comp rx=0 tx=0).stop 2024-03-18T23:08:01.241 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.234+0000 7f789547a740 1 -- 172.21.15.49:0/4242480220 shutdown_connections 2024-03-18T23:08:01.241 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.234+0000 7f789547a740 1 --2- 172.21.15.49:0/4242480220 >> [v2:172.21.15.139:6834/1663212937,v1:172.21.15.139:6835/1663212937] conn(0x55a29acf1820 0x55a29acf0530 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:01.241 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.234+0000 7f789547a740 1 --2- 172.21.15.49:0/4242480220 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a29ac75b50 0x55a29aceb640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:01.241 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.234+0000 7f789547a740 1 --2- 172.21.15.49:0/4242480220 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a29ac709e0 0x55a29ace8fd0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:01.241 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.234+0000 7f786b7fe640 0 client.15273 ms_handle_reset on v2:172.21.15.139:6834/1663212937 2024-03-18T23:08:01.241 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.234+0000 7f789547a740 1 --2- 172.21.15.49:0/4242480220 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a29ac6fff0 0x55a29ac7fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:01.241 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.234+0000 7f789547a740 1 -- 172.21.15.49:0/4242480220 >> 172.21.15.49:0/4242480220 conn(0x55a29ac6b560 msgr2=0x55a29ac71f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:08:01.241 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.234+0000 7f789547a740 1 -- 172.21.15.49:0/4242480220 shutdown_connections 2024-03-18T23:08:01.241 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.234+0000 7f789547a740 1 -- 172.21.15.49:0/4242480220 wait complete. 2024-03-18T23:08:01.241 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.234+0000 7f789547a740 20 client.15273 trim_cache size 0 max 16384 2024-03-18T23:08:01.242 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-03-18T23:08:01.242 INFO:teuthology.orchestra.run.smithi049.stdout: "cluster_fsid": "c63591ea-e57a-11ee-95c9-87774f69a715", 2024-03-18T23:08:01.242 INFO:teuthology.orchestra.run.smithi049.stdout: "whoami": 3, 2024-03-18T23:08:01.242 INFO:teuthology.orchestra.run.smithi049.stdout: "id": 24419, 2024-03-18T23:08:01.242 INFO:teuthology.orchestra.run.smithi049.stdout: "want_state": "up:active", 2024-03-18T23:08:01.242 INFO:teuthology.orchestra.run.smithi049.stdout: "state": "up:active", 2024-03-18T23:08:01.242 INFO:teuthology.orchestra.run.smithi049.stdout: "fs_name": "cephfs", 2024-03-18T23:08:01.243 INFO:teuthology.orchestra.run.smithi049.stdout: "rank_uptime": 111.24412915000001, 2024-03-18T23:08:01.243 INFO:teuthology.orchestra.run.smithi049.stdout: "mdsmap_epoch": 27, 2024-03-18T23:08:01.243 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_epoch": 88, 2024-03-18T23:08:01.243 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_epoch_barrier": 86, 2024-03-18T23:08:01.243 INFO:teuthology.orchestra.run.smithi049.stdout: "uptime": 140.24148717099999 2024-03-18T23:08:01.243 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-03-18T23:08:01.244 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.235+0000 7f7892885640 1 -- 172.21.15.49:0/759313609 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f78740a2800 msgr2=0x7f78740a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:01.244 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.235+0000 7f7892885640 1 --2- 172.21.15.49:0/759313609 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f78740a2800 0x7f78740a4cc0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f787c0095f0 tx=0x7f787c005f90 comp rx=0 tx=0).stop 2024-03-18T23:08:01.244 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.235+0000 7f7892885640 1 -- 172.21.15.49:0/759313609 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f788c1033e0 msgr2=0x7f788c19f140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:01.244 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.235+0000 7f7892885640 1 --2- 172.21.15.49:0/759313609 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f788c1033e0 0x7f788c19f140 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f7884009fd0 tx=0x7f788400ef10 comp rx=0 tx=0).stop 2024-03-18T23:08:01.244 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.235+0000 7f7892885640 1 -- 172.21.15.49:0/759313609 shutdown_connections 2024-03-18T23:08:01.244 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.235+0000 7f7892885640 1 --2- 172.21.15.49:0/759313609 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f78740a2800 0x7f78740a4cc0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:01.244 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.235+0000 7f7892885640 1 --2- 172.21.15.49:0/759313609 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f788c1033e0 0x7f788c19f140 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:01.244 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.235+0000 7f7892885640 1 --2- 172.21.15.49:0/759313609 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f788c102a20 0x7f788c1980d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:01.244 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.235+0000 7f7892885640 1 --2- 172.21.15.49:0/759313609 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f788c101830 0x7f788c197b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:01.244 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.235+0000 7f7892885640 1 -- 172.21.15.49:0/759313609 >> 172.21.15.49:0/759313609 conn(0x7f788c0fcda0 msgr2=0x7f788c105d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:08:01.245 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.235+0000 7f7892885640 1 -- 172.21.15.49:0/759313609 shutdown_connections 2024-03-18T23:08:01.245 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.235+0000 7f7892885640 1 -- 172.21.15.49:0/759313609 wait complete. 2024-03-18T23:08:01.245 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:08:01 smithi049 ceph-mon[62991]: pgmap v348: 97 pgs: 97 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-03-18T23:08:01.245 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:08:01 smithi049 ceph-mon[62991]: from='client.? 172.21.15.49:0/3207631162' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-18T23:08:01.245 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:08:01 smithi049 ceph-mon[62991]: from='client.? 172.21.15.49:0/4167121602' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T23:08:01.245 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:08:01 smithi049 ceph-mon[62991]: from='client.? 172.21.15.49:0/2481216016' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T23:08:01.246 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.f status 2024-03-18T23:08:01.359 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.356+0000 7ffa8c485640 1 -- 172.21.15.49:0/3040779468 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffa841033f0 msgr2=0x7ffa84109c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:01.359 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.356+0000 7ffa8c485640 1 --2- 172.21.15.49:0/3040779468 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffa841033f0 0x7ffa84109c80 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ffa7c009be0 tx=0x7ffa7c02bef0 comp rx=0 tx=0).stop 2024-03-18T23:08:01.359 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.357+0000 7ffa8c485640 1 -- 172.21.15.49:0/3040779468 shutdown_connections 2024-03-18T23:08:01.359 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.357+0000 7ffa8c485640 1 --2- 172.21.15.49:0/3040779468 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffa841033f0 0x7ffa84109c80 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:01.359 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.357+0000 7ffa8c485640 1 --2- 172.21.15.49:0/3040779468 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffa84102a30 0x7ffa84102eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:01.359 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.357+0000 7ffa8c485640 1 --2- 172.21.15.49:0/3040779468 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffa84101840 0x7ffa84101c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:01.359 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.357+0000 7ffa8c485640 1 -- 172.21.15.49:0/3040779468 >> 172.21.15.49:0/3040779468 conn(0x7ffa840fcdb0 msgr2=0x7ffa840ff1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:08:01.359 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.357+0000 7ffa8c485640 1 -- 172.21.15.49:0/3040779468 shutdown_connections 2024-03-18T23:08:01.359 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.357+0000 7ffa8c485640 1 -- 172.21.15.49:0/3040779468 wait complete. 2024-03-18T23:08:01.360 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.357+0000 7ffa8c485640 1 Processor -- start 2024-03-18T23:08:01.360 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.357+0000 7ffa8c485640 1 -- start start 2024-03-18T23:08:01.360 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.357+0000 7ffa8c485640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffa84101840 0x7ffa84197ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:01.360 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.357+0000 7ffa8c485640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffa84102a30 0x7ffa84197ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:01.360 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.357+0000 7ffa8c485640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffa841033f0 0x7ffa8419f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:01.360 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.357+0000 7ffa8c485640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7ffa8410d4a0 con 0x7ffa84102a30 2024-03-18T23:08:01.360 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.357+0000 7ffa8c485640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7ffa8410d320 con 0x7ffa841033f0 2024-03-18T23:08:01.360 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.357+0000 7ffa8c485640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ffa8410d620 con 0x7ffa84101840 2024-03-18T23:08:01.360 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.357+0000 7ffa89a20640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffa84102a30 0x7ffa84197ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:01.360 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.357+0000 7ffa89a20640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffa84102a30 0x7ffa84197ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:37836/0 (socket says 172.21.15.49:37836) 2024-03-18T23:08:01.360 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.357+0000 7ffa89a20640 1 -- 172.21.15.49:0/783229343 learned_addr learned my addr 172.21.15.49:0/783229343 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:08:01.360 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.357+0000 7ffa8a221640 1 --2- 172.21.15.49:0/783229343 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffa84101840 0x7ffa84197ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:01.360 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.357+0000 7ffa8aa22640 1 --2- 172.21.15.49:0/783229343 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffa841033f0 0x7ffa8419f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:01.360 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.358+0000 7ffa89a20640 1 -- 172.21.15.49:0/783229343 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffa84101840 msgr2=0x7ffa84197ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:01.360 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.358+0000 7ffa89a20640 1 --2- 172.21.15.49:0/783229343 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffa84101840 0x7ffa84197ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:01.361 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.358+0000 7ffa89a20640 1 -- 172.21.15.49:0/783229343 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffa841033f0 msgr2=0x7ffa8419f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:01.361 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.358+0000 7ffa89a20640 1 --2- 172.21.15.49:0/783229343 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffa841033f0 0x7ffa8419f060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:01.361 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.358+0000 7ffa89a20640 1 -- 172.21.15.49:0/783229343 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa8419f6d0 con 0x7ffa84102a30 2024-03-18T23:08:01.361 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.358+0000 7ffa89a20640 1 --2- 172.21.15.49:0/783229343 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffa84102a30 0x7ffa84197ff0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7ffa8000e9f0 tx=0x7ffa8000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:08:01.361 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.358+0000 7ffa777fe640 1 -- 172.21.15.49:0/783229343 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa8000cd80 con 0x7ffa84102a30 2024-03-18T23:08:01.361 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.358+0000 7ffa777fe640 1 -- 172.21.15.49:0/783229343 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa80010910 con 0x7ffa84102a30 2024-03-18T23:08:01.361 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.358+0000 7ffa8c485640 1 -- 172.21.15.49:0/783229343 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa8419f9c0 con 0x7ffa84102a30 2024-03-18T23:08:01.361 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.358+0000 7ffa777fe640 1 -- 172.21.15.49:0/783229343 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa80018990 con 0x7ffa84102a30 2024-03-18T23:08:01.361 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.358+0000 7ffa8c485640 1 -- 172.21.15.49:0/783229343 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa8419ff10 con 0x7ffa84102a30 2024-03-18T23:08:01.368 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.360+0000 7ffa777fe640 1 -- 172.21.15.49:0/783229343 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7ffa80010ac0 con 0x7ffa84102a30 2024-03-18T23:08:01.368 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.361+0000 7ffa8f07c740 1 Processor -- start 2024-03-18T23:08:01.369 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.361+0000 7ffa8f07c740 1 -- start start 2024-03-18T23:08:01.369 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.361+0000 7ffa777fe640 1 --2- 172.21.15.49:0/783229343 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7ffa700a2800 0x7ffa700a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:01.369 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.361+0000 7ffa8f07c740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dff7fb9ff0 0x55dff7fba410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:01.369 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.361+0000 7ffa8f07c740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dff7fba9e0 0x55dff7fbf4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:01.369 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.361+0000 7ffa8f07c740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55dff7fbfb50 0x55dff7fc1fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:01.369 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.361+0000 7ffa8f07c740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55dff7f1ede0 con 0x55dff7fb9ff0 2024-03-18T23:08:01.369 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.361+0000 7ffa8f07c740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55dff7ec7c80 con 0x55dff7fbfb50 2024-03-18T23:08:01.369 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.361+0000 7ffa8f07c740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55dff7f1ae80 con 0x55dff7fba9e0 2024-03-18T23:08:01.369 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.361+0000 7ffa8a221640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dff7fb9ff0 0x55dff7fba410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:01.369 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.362+0000 7ffa8aa22640 1 --2- 172.21.15.49:0/783229343 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7ffa700a2800 0x7ffa700a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:01.369 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.362+0000 7ffa89a20640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dff7fba9e0 0x55dff7fbf4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:01.369 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.362+0000 7ffa8a221640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dff7fb9ff0 0x55dff7fba410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:37846/0 (socket says 172.21.15.49:37846) 2024-03-18T23:08:01.369 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.362+0000 7ffa8a221640 1 -- 172.21.15.49:0/165923543 learned_addr learned my addr 172.21.15.49:0/165923543 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:08:01.369 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.362+0000 7ffa777fe640 1 -- 172.21.15.49:0/783229343 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7ffa800ceda0 con 0x7ffa84102a30 2024-03-18T23:08:01.370 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.362+0000 7ffa8aa22640 1 --2- 172.21.15.49:0/165923543 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55dff7fbfb50 0x55dff7fc1fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:01.370 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.362+0000 7ffa89a20640 1 -- 172.21.15.49:0/165923543 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55dff7fbfb50 msgr2=0x55dff7fc1fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:08:01.370 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.362+0000 7ffa89a20640 1 --2- 172.21.15.49:0/165923543 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55dff7fbfb50 0x55dff7fc1fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:01.370 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.362+0000 7ffa89a20640 1 -- 172.21.15.49:0/165923543 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dff7fb9ff0 msgr2=0x55dff7fba410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:08:01.370 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.362+0000 7ffa89a20640 1 --2- 172.21.15.49:0/165923543 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dff7fb9ff0 0x55dff7fba410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:01.370 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.362+0000 7ffa89a20640 1 -- 172.21.15.49:0/165923543 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dff7ed9fb0 con 0x55dff7fba9e0 2024-03-18T23:08:01.370 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.362+0000 7ffa8a221640 1 --2- 172.21.15.49:0/165923543 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dff7fb9ff0 0x55dff7fba410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:08:01.370 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.362+0000 7ffa8aa22640 1 --2- 172.21.15.49:0/783229343 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7ffa700a2800 0x7ffa700a4cc0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7ffa7c005f00 tx=0x7ffa7c0098d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:08:01.370 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.362+0000 7ffa89a20640 1 --2- 172.21.15.49:0/165923543 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dff7fba9e0 0x55dff7fbf4e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ffa8009b500 tx=0x7ffa8009b160 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5b3148e8948fb27e server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:08:01.370 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.362+0000 7ffa74bfe640 1 -- 172.21.15.49:0/165923543 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa80026040 con 0x55dff7fba9e0 2024-03-18T23:08:01.370 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.362+0000 7ffa74bfe640 1 -- 172.21.15.49:0/165923543 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa8009f8a0 con 0x55dff7fba9e0 2024-03-18T23:08:01.371 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.362+0000 7ffa74bfe640 1 -- 172.21.15.49:0/165923543 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa80097e70 con 0x55dff7fba9e0 2024-03-18T23:08:01.371 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.363+0000 7ffa8f07c740 1 -- 172.21.15.49:0/165923543 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dff7fba9e0 msgr2=0x55dff7fbf4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:01.371 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.363+0000 7ffa8f07c740 1 --2- 172.21.15.49:0/165923543 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dff7fba9e0 0x55dff7fbf4e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ffa8009b500 tx=0x7ffa8009b160 comp rx=0 tx=0).stop 2024-03-18T23:08:01.371 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.363+0000 7ffa8f07c740 1 -- 172.21.15.49:0/165923543 shutdown_connections 2024-03-18T23:08:01.371 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.363+0000 7ffa8f07c740 1 --2- 172.21.15.49:0/165923543 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55dff7fbfb50 0x55dff7fc1fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:01.371 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.363+0000 7ffa8f07c740 1 --2- 172.21.15.49:0/165923543 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dff7fba9e0 0x55dff7fbf4e0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:01.371 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.363+0000 7ffa8f07c740 1 --2- 172.21.15.49:0/165923543 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dff7fb9ff0 0x55dff7fba410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:01.371 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.363+0000 7ffa8f07c740 1 -- 172.21.15.49:0/165923543 >> 172.21.15.49:0/165923543 conn(0x55dff7fb5560 msgr2=0x55dff7fb5960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:08:01.371 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.363+0000 7ffa8f07c740 1 -- 172.21.15.49:0/165923543 shutdown_connections 2024-03-18T23:08:01.371 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.363+0000 7ffa8f07c740 1 -- 172.21.15.49:0/165923543 wait complete. 2024-03-18T23:08:01.371 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.363+0000 7ffa8f07c740 1 -- start start 2024-03-18T23:08:01.371 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.363+0000 7ffa8f07c740 1 Processor -- start 2024-03-18T23:08:01.371 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.364+0000 7ffa8f07c740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55dff7fb9ff0 0x55dff7fc43e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:01.371 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.364+0000 7ffa8f07c740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dff7fba9e0 0x55dff7fc4920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:01.372 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.364+0000 7ffa8f07c740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dff7fbfb50 0x55dff7fcb960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:01.372 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.364+0000 7ffa8f07c740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55dff7fc2a40 con 0x55dff7fba9e0 2024-03-18T23:08:01.372 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.364+0000 7ffa8f07c740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55dff7edafc0 con 0x55dff7fb9ff0 2024-03-18T23:08:01.372 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.364+0000 7ffa8f07c740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55dff7fc2bc0 con 0x55dff7fbfb50 2024-03-18T23:08:01.372 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.364+0000 7ffa8f07c740 10 client.0 _ll_register_callbacks cb 0x55dff7f16a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T23:08:01.372 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.364+0000 7ffa89a20640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dff7fba9e0 0x55dff7fc4920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:01.372 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.364+0000 7ffa89a20640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dff7fba9e0 0x55dff7fc4920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:37854/0 (socket says 172.21.15.49:37854) 2024-03-18T23:08:01.372 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.364+0000 7ffa89a20640 1 -- 172.21.15.49:0/3069067344 learned_addr learned my addr 172.21.15.49:0/3069067344 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:08:01.372 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.364+0000 7ffa8a221640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55dff7fb9ff0 0x55dff7fc43e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:01.372 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.364+0000 7ffa8aa22640 1 --2- 172.21.15.49:0/3069067344 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dff7fbfb50 0x55dff7fcb960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:01.372 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.364+0000 7ffa89a20640 1 -- 172.21.15.49:0/3069067344 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dff7fbfb50 msgr2=0x55dff7fcb960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:08:01.372 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.364+0000 7ffa89a20640 1 --2- 172.21.15.49:0/3069067344 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dff7fbfb50 0x55dff7fcb960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:01.372 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.364+0000 7ffa89a20640 1 -- 172.21.15.49:0/3069067344 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55dff7fb9ff0 msgr2=0x55dff7fc43e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:08:01.372 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.364+0000 7ffa89a20640 1 --2- 172.21.15.49:0/3069067344 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55dff7fb9ff0 0x55dff7fc43e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:01.373 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.364+0000 7ffa89a20640 1 -- 172.21.15.49:0/3069067344 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dff7c332d0 con 0x55dff7fba9e0 2024-03-18T23:08:01.373 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.364+0000 7ffa89a20640 1 --2- 172.21.15.49:0/3069067344 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dff7fba9e0 0x55dff7fc4920 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7ffa8000e9c0 tx=0x7ffa8009b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ba63659abdc3271 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:08:01.373 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.364+0000 7ffa6dffb640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-18T23:08:01.373 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.364+0000 7ffa6dffb640 1 -- 172.21.15.49:0/3069067344 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa800a3ef0 con 0x55dff7fba9e0 2024-03-18T23:08:01.373 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.364+0000 7ffa6dffb640 1 -- 172.21.15.49:0/3069067344 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7ffa800afb30 con 0x55dff7fba9e0 2024-03-18T23:08:01.373 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.364+0000 7ffa8f07c740 1 -- 172.21.15.49:0/3069067344 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffa8009f8a0 con 0x55dff7fba9e0 2024-03-18T23:08:01.373 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.365+0000 7ffa6dffb640 1 -- 172.21.15.49:0/3069067344 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ffa800afd70 con 0x55dff7fba9e0 2024-03-18T23:08:01.373 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.365+0000 7ffa8f07c740 10 client.15285 fetch_fsmap learned FSMap version 27 2024-03-18T23:08:01.373 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.365+0000 7ffa8f07c740 1 -- 172.21.15.49:0/3069067344 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55dff7cfd6b0 con 0x55dff7fba9e0 2024-03-18T23:08:01.373 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.365+0000 7ffa6dffb640 1 -- 172.21.15.49:0/3069067344 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa8009f8a0 con 0x55dff7fba9e0 2024-03-18T23:08:01.373 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.365+0000 7ffa6dffb640 1 -- 172.21.15.49:0/3069067344 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7ffa8009a6c0 con 0x55dff7fba9e0 2024-03-18T23:08:01.373 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.365+0000 7ffa8f07c740 10 client.15285 fetch_fsmap finished waiting for FSMap version 27 2024-03-18T23:08:01.373 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.365+0000 7ffa8f07c740 10 client.15285 resolve_mds: resolved name 'f' to daemon mds.f 2024-03-18T23:08:01.373 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.365+0000 7ffa8f07c740 20 client.15285 populate_metadata read hostname 'smithi049' 2024-03-18T23:08:01.374 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.365+0000 7ffa8f07c740 1 --2- 172.21.15.49:0/3069067344 >> [v2:172.21.15.175:6834/755474931,v1:172.21.15.175:6835/755474931] conn(0x55dff7fd0cd0 0x55dff7fd2140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:01.374 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.365+0000 7ffa8f07c740 4 client.15285 mds_command: new command op to 24430 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T23:08:01.374 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.365+0000 7ffa8f07c740 1 -- 172.21.15.49:0/3069067344 --> [v2:172.21.15.175:6834/755474931,v1:172.21.15.175:6835/755474931] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dff7f2c320 con 0x55dff7fd0cd0 2024-03-18T23:08:01.374 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.365+0000 7ffa8a221640 1 --2- 172.21.15.49:0/3069067344 >> [v2:172.21.15.175:6834/755474931,v1:172.21.15.175:6835/755474931] conn(0x55dff7fd0cd0 0x55dff7fd2140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:01.374 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.366+0000 7ffa8a221640 1 --2- 172.21.15.49:0/3069067344 >> [v2:172.21.15.175:6834/755474931,v1:172.21.15.175:6835/755474931] conn(0x55dff7fd0cd0 0x55dff7fd2140 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5a370d23bfa49b8b server_cookie=5fe901ac75372a57 in_seq=0 out_seq=0 2024-03-18T23:08:01.374 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.366+0000 7ffa6dffb640 10 client.15285 ms_handle_connect on v2:172.21.15.175:6834/755474931 2024-03-18T23:08:01.374 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.366+0000 7ffa6dffb640 1 -- 172.21.15.49:0/3069067344 <== mds.2 v2:172.21.15.175:6834/755474931 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55dff7f2c320 con 0x55dff7fd0cd0 2024-03-18T23:08:01.374 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.366+0000 7ffa6dffb640 10 client.15285 handle_command_reply: tid=0 2024-03-18T23:08:01.374 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.366+0000 7ffa8f07c740 1 client.15285 shutdown 2024-03-18T23:08:01.374 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.366+0000 7ffa8f07c740 1 -- 172.21.15.49:0/3069067344 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dff7fba9e0 msgr2=0x55dff7fc4920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:01.375 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.366+0000 7ffa8f07c740 1 --2- 172.21.15.49:0/3069067344 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dff7fba9e0 0x55dff7fc4920 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7ffa8000e9c0 tx=0x7ffa8009b980 comp rx=0 tx=0).stop 2024-03-18T23:08:01.375 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.366+0000 7ffa8f07c740 1 -- 172.21.15.49:0/3069067344 shutdown_connections 2024-03-18T23:08:01.375 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.366+0000 7ffa8f07c740 1 --2- 172.21.15.49:0/3069067344 >> [v2:172.21.15.175:6834/755474931,v1:172.21.15.175:6835/755474931] conn(0x55dff7fd0cd0 0x55dff7fd2140 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:01.375 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.366+0000 7ffa8f07c740 1 --2- 172.21.15.49:0/3069067344 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dff7fbfb50 0x55dff7fcb960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:01.375 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.366+0000 7ffa8f07c740 1 --2- 172.21.15.49:0/3069067344 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dff7fba9e0 0x55dff7fc4920 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:01.375 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.366+0000 7ffa6dffb640 0 client.15285 ms_handle_reset on v2:172.21.15.175:6834/755474931 2024-03-18T23:08:01.375 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.366+0000 7ffa8f07c740 1 --2- 172.21.15.49:0/3069067344 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55dff7fb9ff0 0x55dff7fc43e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:01.375 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.366+0000 7ffa8f07c740 1 -- 172.21.15.49:0/3069067344 >> 172.21.15.49:0/3069067344 conn(0x55dff7fb5560 msgr2=0x55dff7fb47e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:08:01.375 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.366+0000 7ffa8f07c740 1 -- 172.21.15.49:0/3069067344 shutdown_connections 2024-03-18T23:08:01.375 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.366+0000 7ffa8f07c740 1 -- 172.21.15.49:0/3069067344 wait complete. 2024-03-18T23:08:01.375 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.367+0000 7ffa8f07c740 20 client.15285 trim_cache size 0 max 16384 2024-03-18T23:08:01.391 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.385+0000 7ffa8f07c740 1 Processor -- start 2024-03-18T23:08:01.391 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.385+0000 7ffa8f07c740 1 -- start start 2024-03-18T23:08:01.391 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.385+0000 7ffa8f07c740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55dff7fb9ff0 0x55dff7fbbe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:01.391 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.385+0000 7ffa8f07c740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dff7fba9e0 0x55dff7fbc340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:01.391 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.385+0000 7ffa8f07c740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dff7fbfb50 0x55dff7fcb920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:01.391 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.385+0000 7ffa8f07c740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55dff7fd0480 con 0x55dff7fba9e0 2024-03-18T23:08:01.391 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.385+0000 7ffa8f07c740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55dff7fd0b50 con 0x55dff7fb9ff0 2024-03-18T23:08:01.391 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.385+0000 7ffa8f07c740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55dff7fd1220 con 0x55dff7fbfb50 2024-03-18T23:08:01.391 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.385+0000 7ffa89a20640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dff7fba9e0 0x55dff7fbc340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:01.391 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.385+0000 7ffa89a20640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dff7fba9e0 0x55dff7fbc340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:37862/0 (socket says 172.21.15.49:37862) 2024-03-18T23:08:01.391 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.385+0000 7ffa89a20640 1 -- 172.21.15.49:0/2077250298 learned_addr learned my addr 172.21.15.49:0/2077250298 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:08:01.391 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.385+0000 7ffa8a221640 1 --2- 172.21.15.49:0/2077250298 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55dff7fb9ff0 0x55dff7fbbe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:01.392 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.385+0000 7ffa8aa22640 1 --2- 172.21.15.49:0/2077250298 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dff7fbfb50 0x55dff7fcb920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:01.392 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.385+0000 7ffa89a20640 1 -- 172.21.15.49:0/2077250298 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dff7fbfb50 msgr2=0x55dff7fcb920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:08:01.392 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.385+0000 7ffa89a20640 1 --2- 172.21.15.49:0/2077250298 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dff7fbfb50 0x55dff7fcb920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:01.392 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.385+0000 7ffa89a20640 1 -- 172.21.15.49:0/2077250298 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55dff7fb9ff0 msgr2=0x55dff7fbbe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:08:01.392 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.385+0000 7ffa89a20640 1 --2- 172.21.15.49:0/2077250298 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55dff7fb9ff0 0x55dff7fbbe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:01.392 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.385+0000 7ffa89a20640 1 -- 172.21.15.49:0/2077250298 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dff7fbadc0 con 0x55dff7fba9e0 2024-03-18T23:08:01.392 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.386+0000 7ffa89a20640 1 --2- 172.21.15.49:0/2077250298 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dff7fba9e0 0x55dff7fbc340 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7ffa80097c60 tx=0x7ffa8009b290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19d139fd07279644 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:08:01.392 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.386+0000 7ffa6ffff640 1 -- 172.21.15.49:0/2077250298 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa80097820 con 0x55dff7fba9e0 2024-03-18T23:08:01.392 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.386+0000 7ffa6ffff640 1 -- 172.21.15.49:0/2077250298 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa800a3ef0 con 0x55dff7fba9e0 2024-03-18T23:08:01.392 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.386+0000 7ffa6ffff640 1 -- 172.21.15.49:0/2077250298 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa8009ac50 con 0x55dff7fba9e0 2024-03-18T23:08:01.392 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.386+0000 7ffa8f07c740 1 -- 172.21.15.49:0/2077250298 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dff7fba9e0 msgr2=0x55dff7fbc340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:01.392 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.386+0000 7ffa8f07c740 1 --2- 172.21.15.49:0/2077250298 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dff7fba9e0 0x55dff7fbc340 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7ffa80097c60 tx=0x7ffa8009b290 comp rx=0 tx=0).stop 2024-03-18T23:08:01.392 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.386+0000 7ffa8f07c740 1 -- 172.21.15.49:0/2077250298 shutdown_connections 2024-03-18T23:08:01.392 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.386+0000 7ffa8f07c740 1 --2- 172.21.15.49:0/2077250298 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dff7fbfb50 0x55dff7fcb920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:01.393 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.386+0000 7ffa8f07c740 1 --2- 172.21.15.49:0/2077250298 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dff7fba9e0 0x55dff7fbc340 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:01.393 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.386+0000 7ffa8f07c740 1 --2- 172.21.15.49:0/2077250298 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55dff7fb9ff0 0x55dff7fbbe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:01.393 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.386+0000 7ffa8f07c740 1 -- 172.21.15.49:0/2077250298 >> 172.21.15.49:0/2077250298 conn(0x55dff7fb5560 msgr2=0x55dff7fb5940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:08:01.393 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.386+0000 7ffa8f07c740 1 -- 172.21.15.49:0/2077250298 shutdown_connections 2024-03-18T23:08:01.393 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.386+0000 7ffa8f07c740 1 -- 172.21.15.49:0/2077250298 wait complete. 2024-03-18T23:08:01.393 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.386+0000 7ffa8f07c740 1 -- start start 2024-03-18T23:08:01.393 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.387+0000 7ffa8f07c740 1 Processor -- start 2024-03-18T23:08:01.393 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.387+0000 7ffa8f07c740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55dff7fb9ff0 0x55dff7fc75b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:01.393 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.387+0000 7ffa8f07c740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dff7fba9e0 0x55dff7fc7af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:01.393 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.387+0000 7ffa8f07c740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dff7fbfb50 0x55dff7fcaaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:01.393 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.387+0000 7ffa8f07c740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55dff7fd1fc0 con 0x55dff7fbfb50 2024-03-18T23:08:01.393 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.387+0000 7ffa8f07c740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55dff7fd18f0 con 0x55dff7fb9ff0 2024-03-18T23:08:01.393 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.387+0000 7ffa8f07c740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55dff7fd2690 con 0x55dff7fba9e0 2024-03-18T23:08:01.393 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.387+0000 7ffa8f07c740 10 client.0 _ll_register_callbacks cb 0x7ffa78009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T23:08:01.394 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.387+0000 7ffa8aa22640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dff7fbfb50 0x55dff7fcaaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:01.394 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.387+0000 7ffa8a221640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55dff7fb9ff0 0x55dff7fc75b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:01.394 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.387+0000 7ffa8aa22640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dff7fbfb50 0x55dff7fcaaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:37878/0 (socket says 172.21.15.49:37878) 2024-03-18T23:08:01.394 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.387+0000 7ffa8aa22640 1 -- 172.21.15.49:0/2456647355 learned_addr learned my addr 172.21.15.49:0/2456647355 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:08:01.394 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.387+0000 7ffa89a20640 1 --2- 172.21.15.49:0/2456647355 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dff7fba9e0 0x55dff7fc7af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:01.394 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.387+0000 7ffa8aa22640 1 -- 172.21.15.49:0/2456647355 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dff7fba9e0 msgr2=0x55dff7fc7af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:08:01.394 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.387+0000 7ffa8aa22640 1 --2- 172.21.15.49:0/2456647355 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dff7fba9e0 0x55dff7fc7af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:01.394 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.387+0000 7ffa8aa22640 1 -- 172.21.15.49:0/2456647355 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55dff7fb9ff0 msgr2=0x55dff7fc75b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:08:01.394 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.387+0000 7ffa8aa22640 1 --2- 172.21.15.49:0/2456647355 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55dff7fb9ff0 0x55dff7fc75b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:01.394 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.387+0000 7ffa8aa22640 1 -- 172.21.15.49:0/2456647355 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dff7fcb090 con 0x55dff7fbfb50 2024-03-18T23:08:01.394 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.387+0000 7ffa8aa22640 1 --2- 172.21.15.49:0/2456647355 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dff7fbfb50 0x55dff7fcaaa0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7ffa7c035000 tx=0x7ffa7c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=955104fd1efb4cd6 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:08:01.394 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.387+0000 7ffa6dffb640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-18T23:08:01.394 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.388+0000 7ffa6dffb640 1 -- 172.21.15.49:0/2456647355 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa7c004040 con 0x55dff7fbfb50 2024-03-18T23:08:01.395 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.388+0000 7ffa6dffb640 1 -- 172.21.15.49:0/2456647355 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7ffa7c03a040 con 0x55dff7fbfb50 2024-03-18T23:08:01.395 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.388+0000 7ffa8f07c740 1 -- 172.21.15.49:0/2456647355 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffa800a3ef0 con 0x55dff7fbfb50 2024-03-18T23:08:01.395 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.388+0000 7ffa6dffb640 1 -- 172.21.15.49:0/2456647355 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ffa7c0041e0 con 0x55dff7fbfb50 2024-03-18T23:08:01.395 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.388+0000 7ffa8f07c740 10 client.15291 fetch_fsmap learned FSMap version 27 2024-03-18T23:08:01.395 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.388+0000 7ffa8f07c740 1 -- 172.21.15.49:0/2456647355 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55dff8036fe0 con 0x55dff7fbfb50 2024-03-18T23:08:01.395 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.388+0000 7ffa6dffb640 1 -- 172.21.15.49:0/2456647355 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa800a3ef0 con 0x55dff7fbfb50 2024-03-18T23:08:01.395 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.388+0000 7ffa6dffb640 1 -- 172.21.15.49:0/2456647355 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7ffa7c034ad0 con 0x55dff7fbfb50 2024-03-18T23:08:01.395 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.388+0000 7ffa8f07c740 10 client.15291 fetch_fsmap finished waiting for FSMap version 27 2024-03-18T23:08:01.395 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.388+0000 7ffa8f07c740 10 client.15291 resolve_mds: resolved name 'f' to daemon mds.f 2024-03-18T23:08:01.395 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.388+0000 7ffa8f07c740 20 client.15291 populate_metadata read hostname 'smithi049' 2024-03-18T23:08:01.395 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.388+0000 7ffa8f07c740 1 --2- 172.21.15.49:0/2456647355 >> [v2:172.21.15.175:6834/755474931,v1:172.21.15.175:6835/755474931] conn(0x55dff803bc90 0x55dff803a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:01.395 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.388+0000 7ffa8f07c740 4 client.15291 mds_command: new command op to 24430 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-18T23:08:01.395 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.388+0000 7ffa8f07c740 1 -- 172.21.15.49:0/2456647355 --> [v2:172.21.15.175:6834/755474931,v1:172.21.15.175:6835/755474931] -- command(tid 0: {"prefix": "status"}) -- 0x55dff7f26670 con 0x55dff803bc90 2024-03-18T23:08:01.395 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.388+0000 7ffa8a221640 1 --2- 172.21.15.49:0/2456647355 >> [v2:172.21.15.175:6834/755474931,v1:172.21.15.175:6835/755474931] conn(0x55dff803bc90 0x55dff803a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:01.396 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.389+0000 7ffa8a221640 1 --2- 172.21.15.49:0/2456647355 >> [v2:172.21.15.175:6834/755474931,v1:172.21.15.175:6835/755474931] conn(0x55dff803bc90 0x55dff803a9a0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ca5b5e953fd2f620 server_cookie=c5d98ccb83f230c2 in_seq=0 out_seq=0 2024-03-18T23:08:01.396 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.389+0000 7ffa6dffb640 10 client.15291 ms_handle_connect on v2:172.21.15.175:6834/755474931 2024-03-18T23:08:01.396 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.389+0000 7ffa6dffb640 1 -- 172.21.15.49:0/2456647355 <== mds.2 v2:172.21.15.175:6834/755474931 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55dff7f26670 con 0x55dff803bc90 2024-03-18T23:08:01.396 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.389+0000 7ffa6dffb640 10 client.15291 handle_command_reply: tid=0 2024-03-18T23:08:01.396 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.389+0000 7ffa8f07c740 1 client.15291 shutdown 2024-03-18T23:08:01.396 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.389+0000 7ffa8f07c740 1 -- 172.21.15.49:0/2456647355 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dff7fbfb50 msgr2=0x55dff7fcaaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:01.396 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.389+0000 7ffa8f07c740 1 --2- 172.21.15.49:0/2456647355 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dff7fbfb50 0x55dff7fcaaa0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7ffa7c035000 tx=0x7ffa7c004a40 comp rx=0 tx=0).stop 2024-03-18T23:08:01.396 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.389+0000 7ffa8f07c740 1 -- 172.21.15.49:0/2456647355 shutdown_connections 2024-03-18T23:08:01.396 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.389+0000 7ffa8f07c740 1 --2- 172.21.15.49:0/2456647355 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dff7fbfb50 0x55dff7fcaaa0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:01.396 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.389+0000 7ffa8f07c740 1 --2- 172.21.15.49:0/2456647355 >> [v2:172.21.15.175:6834/755474931,v1:172.21.15.175:6835/755474931] conn(0x55dff803bc90 0x55dff803a9a0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:01.396 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.389+0000 7ffa8f07c740 1 --2- 172.21.15.49:0/2456647355 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dff7fba9e0 0x55dff7fc7af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:01.396 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.389+0000 7ffa6dffb640 0 client.15291 ms_handle_reset on v2:172.21.15.175:6834/755474931 2024-03-18T23:08:01.396 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.389+0000 7ffa8f07c740 1 --2- 172.21.15.49:0/2456647355 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55dff7fb9ff0 0x55dff7fc75b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:01.396 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.389+0000 7ffa8f07c740 1 -- 172.21.15.49:0/2456647355 >> 172.21.15.49:0/2456647355 conn(0x55dff7fb5560 msgr2=0x55dff7fbbe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:08:01.397 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.389+0000 7ffa8f07c740 1 -- 172.21.15.49:0/2456647355 shutdown_connections 2024-03-18T23:08:01.397 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.389+0000 7ffa8f07c740 1 -- 172.21.15.49:0/2456647355 wait complete. 2024-03-18T23:08:01.397 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.389+0000 7ffa8f07c740 20 client.15291 trim_cache size 0 max 16384 2024-03-18T23:08:01.397 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-03-18T23:08:01.397 INFO:teuthology.orchestra.run.smithi049.stdout: "cluster_fsid": "c63591ea-e57a-11ee-95c9-87774f69a715", 2024-03-18T23:08:01.397 INFO:teuthology.orchestra.run.smithi049.stdout: "whoami": 2, 2024-03-18T23:08:01.397 INFO:teuthology.orchestra.run.smithi049.stdout: "id": 24430, 2024-03-18T23:08:01.397 INFO:teuthology.orchestra.run.smithi049.stdout: "want_state": "up:active", 2024-03-18T23:08:01.397 INFO:teuthology.orchestra.run.smithi049.stdout: "state": "up:active", 2024-03-18T23:08:01.397 INFO:teuthology.orchestra.run.smithi049.stdout: "fs_name": "cephfs", 2024-03-18T23:08:01.398 INFO:teuthology.orchestra.run.smithi049.stdout: "rank_uptime": 112.40273793999999, 2024-03-18T23:08:01.398 INFO:teuthology.orchestra.run.smithi049.stdout: "mdsmap_epoch": 27, 2024-03-18T23:08:01.398 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_epoch": 88, 2024-03-18T23:08:01.398 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_epoch_barrier": 86, 2024-03-18T23:08:01.398 INFO:teuthology.orchestra.run.smithi049.stdout: "uptime": 137.581744867 2024-03-18T23:08:01.398 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-03-18T23:08:01.399 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.390+0000 7ffa8c485640 1 -- 172.21.15.49:0/783229343 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7ffa700a2800 msgr2=0x7ffa700a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:01.399 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.390+0000 7ffa8c485640 1 --2- 172.21.15.49:0/783229343 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7ffa700a2800 0x7ffa700a4cc0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7ffa7c005f00 tx=0x7ffa7c0098d0 comp rx=0 tx=0).stop 2024-03-18T23:08:01.399 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.390+0000 7ffa8c485640 1 -- 172.21.15.49:0/783229343 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffa84102a30 msgr2=0x7ffa84197ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:01.399 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.390+0000 7ffa8c485640 1 --2- 172.21.15.49:0/783229343 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffa84102a30 0x7ffa84197ff0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7ffa8000e9f0 tx=0x7ffa8000eeb0 comp rx=0 tx=0).stop 2024-03-18T23:08:01.399 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.390+0000 7ffa8c485640 1 -- 172.21.15.49:0/783229343 shutdown_connections 2024-03-18T23:08:01.399 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.390+0000 7ffa8c485640 1 --2- 172.21.15.49:0/783229343 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7ffa700a2800 0x7ffa700a4cc0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:01.399 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.390+0000 7ffa8c485640 1 --2- 172.21.15.49:0/783229343 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffa841033f0 0x7ffa8419f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:01.399 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.390+0000 7ffa8c485640 1 --2- 172.21.15.49:0/783229343 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffa84102a30 0x7ffa84197ff0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:01.400 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.390+0000 7ffa8c485640 1 --2- 172.21.15.49:0/783229343 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffa84101840 0x7ffa84197ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:01.400 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.390+0000 7ffa8c485640 1 -- 172.21.15.49:0/783229343 >> 172.21.15.49:0/783229343 conn(0x7ffa840fcdb0 msgr2=0x7ffa84105cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:08:01.400 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.390+0000 7ffa8c485640 1 -- 172.21.15.49:0/783229343 shutdown_connections 2024-03-18T23:08:01.400 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.390+0000 7ffa8c485640 1 -- 172.21.15.49:0/783229343 wait complete. 2024-03-18T23:08:01.401 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-03-18T23:08:01.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:08:01 smithi175 ceph-mon[67352]: pgmap v348: 97 pgs: 97 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-03-18T23:08:01.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:08:01 smithi175 ceph-mon[67352]: from='client.? 172.21.15.49:0/3207631162' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-18T23:08:01.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:08:01 smithi175 ceph-mon[67352]: from='client.? 172.21.15.49:0/4167121602' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T23:08:01.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:08:01 smithi175 ceph-mon[67352]: from='client.? 172.21.15.49:0/2481216016' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T23:08:01.512 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.510+0000 7f285ac68640 1 -- 172.21.15.49:0/2742543986 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f28541022f0 msgr2=0x7f2854108b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:01.512 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.510+0000 7f285ac68640 1 --2- 172.21.15.49:0/2742543986 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f28541022f0 0x7f2854108b80 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f284c009ca0 tx=0x7f284c02c0f0 comp rx=0 tx=0).stop 2024-03-18T23:08:01.512 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.510+0000 7f285ac68640 1 -- 172.21.15.49:0/2742543986 shutdown_connections 2024-03-18T23:08:01.512 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.510+0000 7f285ac68640 1 --2- 172.21.15.49:0/2742543986 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f28541022f0 0x7f2854108b80 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:01.512 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.510+0000 7f285ac68640 1 --2- 172.21.15.49:0/2742543986 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2854101930 0x7f2854101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:01.513 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.510+0000 7f285ac68640 1 --2- 172.21.15.49:0/2742543986 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2854100740 0x7f2854100b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:01.513 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.510+0000 7f285ac68640 1 -- 172.21.15.49:0/2742543986 >> 172.21.15.49:0/2742543986 conn(0x7f28540fbcd0 msgr2=0x7f28540fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:08:01.513 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.510+0000 7f285ac68640 1 -- 172.21.15.49:0/2742543986 shutdown_connections 2024-03-18T23:08:01.513 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.510+0000 7f285ac68640 1 -- 172.21.15.49:0/2742543986 wait complete. 2024-03-18T23:08:01.513 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.511+0000 7f285ac68640 1 Processor -- start 2024-03-18T23:08:01.513 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.511+0000 7f285ac68640 1 -- start start 2024-03-18T23:08:01.513 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.511+0000 7f285ac68640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2854100740 0x7f2854199d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:01.513 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.511+0000 7f285ac68640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2854101930 0x7f285419a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:01.513 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.511+0000 7f285ac68640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f28541022f0 0x7f28541a1320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:01.513 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.511+0000 7f285ac68640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f285410c3a0 con 0x7f2854101930 2024-03-18T23:08:01.513 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.511+0000 7f285ac68640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f285410c220 con 0x7f2854100740 2024-03-18T23:08:01.513 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.511+0000 7f285ac68640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f285410c520 con 0x7f28541022f0 2024-03-18T23:08:01.513 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.511+0000 7f2853fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2854101930 0x7f285419a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:01.513 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.511+0000 7f2853fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2854101930 0x7f285419a2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:37896/0 (socket says 172.21.15.49:37896) 2024-03-18T23:08:01.514 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.511+0000 7f2853fff640 1 -- 172.21.15.49:0/1426723832 learned_addr learned my addr 172.21.15.49:0/1426723832 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:08:01.514 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.511+0000 7f2858a04640 1 --2- 172.21.15.49:0/1426723832 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2854100740 0x7f2854199d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:01.514 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.511+0000 7f2859205640 1 --2- 172.21.15.49:0/1426723832 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f28541022f0 0x7f28541a1320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:01.514 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.511+0000 7f2853fff640 1 -- 172.21.15.49:0/1426723832 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f28541022f0 msgr2=0x7f28541a1320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:01.514 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.511+0000 7f2853fff640 1 --2- 172.21.15.49:0/1426723832 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f28541022f0 0x7f28541a1320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:01.514 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.511+0000 7f2853fff640 1 -- 172.21.15.49:0/1426723832 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2854100740 msgr2=0x7f2854199d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:01.514 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.511+0000 7f2853fff640 1 --2- 172.21.15.49:0/1426723832 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2854100740 0x7f2854199d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:01.514 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.511+0000 7f2853fff640 1 -- 172.21.15.49:0/1426723832 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28541a1990 con 0x7f2854101930 2024-03-18T23:08:01.514 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.512+0000 7f2853fff640 1 --2- 172.21.15.49:0/1426723832 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2854101930 0x7f285419a2b0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f284000e9f0 tx=0x7f284000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:08:01.514 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.512+0000 7f2851ffb640 1 -- 172.21.15.49:0/1426723832 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f284000cd80 con 0x7f2854101930 2024-03-18T23:08:01.514 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.512+0000 7f285ac68640 1 -- 172.21.15.49:0/1426723832 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28541a1c80 con 0x7f2854101930 2024-03-18T23:08:01.514 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.512+0000 7f2851ffb640 1 -- 172.21.15.49:0/1426723832 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2840010910 con 0x7f2854101930 2024-03-18T23:08:01.515 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.512+0000 7f2851ffb640 1 -- 172.21.15.49:0/1426723832 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28400188f0 con 0x7f2854101930 2024-03-18T23:08:01.515 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.512+0000 7f285ac68640 1 -- 172.21.15.49:0/1426723832 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28541a21d0 con 0x7f2854101930 2024-03-18T23:08:01.521 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.514+0000 7f2851ffb640 1 -- 172.21.15.49:0/1426723832 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7f2840010ac0 con 0x7f2854101930 2024-03-18T23:08:01.521 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.514+0000 7f285d850740 1 Processor -- start 2024-03-18T23:08:01.521 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.514+0000 7f285d850740 1 -- start start 2024-03-18T23:08:01.521 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.515+0000 7f285d850740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557594771ff0 0x557594772410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:01.521 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.515+0000 7f285d850740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5575947729e0 0x5575947774e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:01.521 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.515+0000 7f285d850740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557594777b50 0x557594779fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:01.521 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.515+0000 7f285d850740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5575946d6de0 con 0x557594771ff0 2024-03-18T23:08:01.521 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.515+0000 7f285d850740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55759467fc80 con 0x5575947729e0 2024-03-18T23:08:01.521 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.515+0000 7f285d850740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5575946d2e80 con 0x557594777b50 2024-03-18T23:08:01.522 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.515+0000 7f2859205640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557594771ff0 0x557594772410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:01.522 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.515+0000 7f2853fff640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557594777b50 0x557594779fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:01.522 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.515+0000 7f2858a04640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5575947729e0 0x5575947774e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:01.522 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.515+0000 7f2859205640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557594771ff0 0x557594772410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:37906/0 (socket says 172.21.15.49:37906) 2024-03-18T23:08:01.522 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.515+0000 7f2859205640 1 -- 172.21.15.49:0/772059625 learned_addr learned my addr 172.21.15.49:0/772059625 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:08:01.522 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.515+0000 7f2851ffb640 1 --2- 172.21.15.49:0/1426723832 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f28440a2800 0x7f28440a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:01.522 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.515+0000 7f2859205640 1 -- 172.21.15.49:0/772059625 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557594777b50 msgr2=0x557594779fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:08:01.522 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.515+0000 7f2859205640 1 --2- 172.21.15.49:0/772059625 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557594777b50 0x557594779fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:01.522 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.515+0000 7f2859205640 1 -- 172.21.15.49:0/772059625 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5575947729e0 msgr2=0x5575947774e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:08:01.523 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.515+0000 7f2859205640 1 --2- 172.21.15.49:0/772059625 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5575947729e0 0x5575947774e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:01.523 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.515+0000 7f2859205640 1 -- 172.21.15.49:0/772059625 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557594691fb0 con 0x557594771ff0 2024-03-18T23:08:01.523 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.515+0000 7f2858a04640 1 --2- 172.21.15.49:0/772059625 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5575947729e0 0x5575947774e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:08:01.523 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.515+0000 7f2853fff640 1 --2- 172.21.15.49:0/772059625 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557594777b50 0x557594779fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:08:01.523 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.515+0000 7f2851ffb640 1 -- 172.21.15.49:0/1426723832 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f28400cedf0 con 0x7f2854101930 2024-03-18T23:08:01.523 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.515+0000 7f2859205640 1 --2- 172.21.15.49:0/1426723832 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f28440a2800 0x7f28440a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:01.523 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.515+0000 7f2859205640 1 --2- 172.21.15.49:0/772059625 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557594771ff0 0x557594772410 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f284c009ca0 tx=0x7f284c005f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a42099679fef03f6 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:08:01.523 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.516+0000 7f283effd640 1 -- 172.21.15.49:0/772059625 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f284c039040 con 0x557594771ff0 2024-03-18T23:08:01.523 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.516+0000 7f283effd640 1 -- 172.21.15.49:0/772059625 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f284c003e50 con 0x557594771ff0 2024-03-18T23:08:01.523 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.516+0000 7f283effd640 1 -- 172.21.15.49:0/772059625 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f284c034440 con 0x557594771ff0 2024-03-18T23:08:01.523 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.516+0000 7f285d850740 1 -- 172.21.15.49:0/772059625 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557594771ff0 msgr2=0x557594772410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:01.523 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.516+0000 7f2859205640 1 --2- 172.21.15.49:0/1426723832 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f28440a2800 0x7f28440a4cc0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f284c04c000 tx=0x7f284c045000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:08:01.523 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.516+0000 7f285d850740 1 --2- 172.21.15.49:0/772059625 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557594771ff0 0x557594772410 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f284c009ca0 tx=0x7f284c005f70 comp rx=0 tx=0).stop 2024-03-18T23:08:01.524 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.516+0000 7f285d850740 1 -- 172.21.15.49:0/772059625 shutdown_connections 2024-03-18T23:08:01.524 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.516+0000 7f285d850740 1 --2- 172.21.15.49:0/772059625 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557594777b50 0x557594779fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:01.524 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.516+0000 7f285d850740 1 --2- 172.21.15.49:0/772059625 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5575947729e0 0x5575947774e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:01.524 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.516+0000 7f285d850740 1 --2- 172.21.15.49:0/772059625 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557594771ff0 0x557594772410 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:01.524 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.516+0000 7f285d850740 1 -- 172.21.15.49:0/772059625 >> 172.21.15.49:0/772059625 conn(0x55759476d560 msgr2=0x55759476d960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:08:01.524 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.516+0000 7f285d850740 1 -- 172.21.15.49:0/772059625 shutdown_connections 2024-03-18T23:08:01.524 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.516+0000 7f285d850740 1 -- 172.21.15.49:0/772059625 wait complete. 2024-03-18T23:08:01.524 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.517+0000 7f285d850740 1 -- start start 2024-03-18T23:08:01.524 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.517+0000 7f285d850740 1 Processor -- start 2024-03-18T23:08:01.524 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.517+0000 7f285d850740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557594771ff0 0x55759477c570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:01.524 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.517+0000 7f285d850740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5575947729e0 0x55759477cab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:01.524 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.517+0000 7f285d850740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557594777b50 0x557594783af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:01.524 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.517+0000 7f285d850740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55759477ab00 con 0x5575947729e0 2024-03-18T23:08:01.525 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.517+0000 7f285d850740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x557594692fc0 con 0x557594771ff0 2024-03-18T23:08:01.525 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.517+0000 7f285d850740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55759477ac80 con 0x557594777b50 2024-03-18T23:08:01.525 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.517+0000 7f285d850740 10 client.0 _ll_register_callbacks cb 0x5575946cea20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T23:08:01.525 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.517+0000 7f2853fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5575947729e0 0x55759477cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:01.525 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.517+0000 7f2853fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5575947729e0 0x55759477cab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:37922/0 (socket says 172.21.15.49:37922) 2024-03-18T23:08:01.525 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.517+0000 7f2853fff640 1 -- 172.21.15.49:0/1887707585 learned_addr learned my addr 172.21.15.49:0/1887707585 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:08:01.525 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.517+0000 7f2859205640 1 --2- 172.21.15.49:0/1887707585 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557594777b50 0x557594783af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:01.525 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.517+0000 7f2858a04640 1 --2- 172.21.15.49:0/1887707585 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557594771ff0 0x55759477c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:01.525 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.518+0000 7f2853fff640 1 -- 172.21.15.49:0/1887707585 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557594777b50 msgr2=0x557594783af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:08:01.525 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.518+0000 7f2853fff640 1 --2- 172.21.15.49:0/1887707585 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557594777b50 0x557594783af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:01.525 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.518+0000 7f2853fff640 1 -- 172.21.15.49:0/1887707585 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557594771ff0 msgr2=0x55759477c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:08:01.525 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.518+0000 7f2853fff640 1 --2- 172.21.15.49:0/1887707585 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557594771ff0 0x55759477c570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:01.525 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.518+0000 7f2853fff640 1 -- 172.21.15.49:0/1887707585 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5575943eb2d0 con 0x5575947729e0 2024-03-18T23:08:01.525 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.518+0000 7f2859205640 1 --2- 172.21.15.49:0/1887707585 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557594777b50 0x557594783af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:08:01.526 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.518+0000 7f2853fff640 1 --2- 172.21.15.49:0/1887707585 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5575947729e0 0x55759477cab0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f28400cc900 tx=0x7f284009b680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e82bbbac7db145a6 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:08:01.526 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.518+0000 7f2833fff640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-18T23:08:01.526 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.518+0000 7f2833fff640 1 -- 172.21.15.49:0/1887707585 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28400a3f40 con 0x5575947729e0 2024-03-18T23:08:01.526 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.518+0000 7f2833fff640 1 -- 172.21.15.49:0/1887707585 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f28400860e0 con 0x5575947729e0 2024-03-18T23:08:01.526 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.518+0000 7f285d850740 1 -- 172.21.15.49:0/1887707585 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f284c003e50 con 0x5575947729e0 2024-03-18T23:08:01.526 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.518+0000 7f2833fff640 1 -- 172.21.15.49:0/1887707585 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f284009f8f0 con 0x5575947729e0 2024-03-18T23:08:01.526 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.518+0000 7f285d850740 10 client.15303 fetch_fsmap learned FSMap version 27 2024-03-18T23:08:01.526 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.518+0000 7f285d850740 1 -- 172.21.15.49:0/1887707585 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5575944b56b0 con 0x5575947729e0 2024-03-18T23:08:01.526 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.518+0000 7f2833fff640 1 -- 172.21.15.49:0/1887707585 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f284c003e50 con 0x5575947729e0 2024-03-18T23:08:01.526 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.518+0000 7f2833fff640 1 -- 172.21.15.49:0/1887707585 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f284009ad30 con 0x5575947729e0 2024-03-18T23:08:01.526 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.518+0000 7f285d850740 10 client.15303 fetch_fsmap finished waiting for FSMap version 27 2024-03-18T23:08:01.526 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.518+0000 7f285d850740 10 client.15303 resolve_mds: resolved name 'i' to daemon mds.i 2024-03-18T23:08:01.526 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.518+0000 7f285d850740 20 client.15303 populate_metadata read hostname 'smithi049' 2024-03-18T23:08:01.526 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.518+0000 7f285d850740 1 --2- 172.21.15.49:0/1887707585 >> [v2:172.21.15.175:6838/3640882084,v1:172.21.15.175:6839/3640882084] conn(0x5575947893e0 0x55759478a850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:01.527 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.518+0000 7f285d850740 4 client.15303 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T23:08:01.527 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.518+0000 7f285d850740 1 -- 172.21.15.49:0/1887707585 --> [v2:172.21.15.175:6838/3640882084,v1:172.21.15.175:6839/3640882084] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5575946e4320 con 0x5575947893e0 2024-03-18T23:08:01.527 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.519+0000 7f2858a04640 1 --2- 172.21.15.49:0/1887707585 >> [v2:172.21.15.175:6838/3640882084,v1:172.21.15.175:6839/3640882084] conn(0x5575947893e0 0x55759478a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:01.527 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.519+0000 7f2833fff640 10 client.15303 ms_handle_connect on v2:172.21.15.175:6838/3640882084 2024-03-18T23:08:01.527 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.519+0000 7f2858a04640 1 --2- 172.21.15.49:0/1887707585 >> [v2:172.21.15.175:6838/3640882084,v1:172.21.15.175:6839/3640882084] conn(0x5575947893e0 0x55759478a850 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a24878417e7efb16 server_cookie=2ed1ec11747be4e2 in_seq=0 out_seq=0 2024-03-18T23:08:01.527 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.520+0000 7f2833fff640 1 -- 172.21.15.49:0/1887707585 <== mds.1 v2:172.21.15.175:6838/3640882084 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5575946e4320 con 0x5575947893e0 2024-03-18T23:08:01.527 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.520+0000 7f2833fff640 10 client.15303 handle_command_reply: tid=0 2024-03-18T23:08:01.527 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.520+0000 7f285d850740 1 client.15303 shutdown 2024-03-18T23:08:01.527 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.520+0000 7f285d850740 1 -- 172.21.15.49:0/1887707585 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5575947729e0 msgr2=0x55759477cab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:01.528 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.520+0000 7f285d850740 1 --2- 172.21.15.49:0/1887707585 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5575947729e0 0x55759477cab0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f28400cc900 tx=0x7f284009b680 comp rx=0 tx=0).stop 2024-03-18T23:08:01.528 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.520+0000 7f285d850740 1 -- 172.21.15.49:0/1887707585 shutdown_connections 2024-03-18T23:08:01.528 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.520+0000 7f285d850740 1 --2- 172.21.15.49:0/1887707585 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557594777b50 0x557594783af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:01.528 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.520+0000 7f285d850740 1 --2- 172.21.15.49:0/1887707585 >> [v2:172.21.15.175:6838/3640882084,v1:172.21.15.175:6839/3640882084] conn(0x5575947893e0 0x55759478a850 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:01.528 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.520+0000 7f285d850740 1 --2- 172.21.15.49:0/1887707585 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5575947729e0 0x55759477cab0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:01.528 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.520+0000 7f285d850740 1 --2- 172.21.15.49:0/1887707585 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557594771ff0 0x55759477c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:01.528 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.520+0000 7f2833fff640 0 client.15303 ms_handle_reset on v2:172.21.15.175:6838/3640882084 2024-03-18T23:08:01.528 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.520+0000 7f285d850740 1 -- 172.21.15.49:0/1887707585 >> 172.21.15.49:0/1887707585 conn(0x55759476d560 msgr2=0x55759476c7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:08:01.528 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.520+0000 7f285d850740 1 -- 172.21.15.49:0/1887707585 shutdown_connections 2024-03-18T23:08:01.528 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.520+0000 7f285d850740 1 -- 172.21.15.49:0/1887707585 wait complete. 2024-03-18T23:08:01.528 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.520+0000 7f285d850740 20 client.15303 trim_cache size 0 max 16384 2024-03-18T23:08:01.544 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.538+0000 7f285d850740 1 Processor -- start 2024-03-18T23:08:01.544 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.538+0000 7f285d850740 1 -- start start 2024-03-18T23:08:01.544 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.538+0000 7f285d850740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557594771ff0 0x55759476bf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:01.544 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.538+0000 7f285d850740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5575947729e0 0x557594783ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:01.544 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.538+0000 7f285d850740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557594777b50 0x55759476c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:01.544 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.538+0000 7f285d850740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x557594788b90 con 0x557594771ff0 2024-03-18T23:08:01.544 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.538+0000 7f285d850740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x557594789260 con 0x557594777b50 2024-03-18T23:08:01.544 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.538+0000 7f285d850740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x557594789930 con 0x5575947729e0 2024-03-18T23:08:01.544 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.539+0000 7f2858a04640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557594771ff0 0x55759476bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:01.544 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.539+0000 7f2858a04640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557594771ff0 0x55759476bf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:37934/0 (socket says 172.21.15.49:37934) 2024-03-18T23:08:01.544 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.539+0000 7f2858a04640 1 -- 172.21.15.49:0/945539498 learned_addr learned my addr 172.21.15.49:0/945539498 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:08:01.544 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.539+0000 7f2853fff640 1 --2- 172.21.15.49:0/945539498 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5575947729e0 0x557594783ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:01.544 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.539+0000 7f2859205640 1 --2- 172.21.15.49:0/945539498 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557594777b50 0x55759476c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:01.545 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.539+0000 7f2858a04640 1 -- 172.21.15.49:0/945539498 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5575947729e0 msgr2=0x557594783ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:08:01.545 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.539+0000 7f2858a04640 1 --2- 172.21.15.49:0/945539498 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5575947729e0 0x557594783ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:01.545 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.539+0000 7f2858a04640 1 -- 172.21.15.49:0/945539498 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557594777b50 msgr2=0x55759476c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:08:01.545 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.539+0000 7f2858a04640 1 --2- 172.21.15.49:0/945539498 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557594777b50 0x55759476c490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:01.545 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.539+0000 7f2858a04640 1 -- 172.21.15.49:0/945539498 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557594773270 con 0x557594771ff0 2024-03-18T23:08:01.545 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.539+0000 7f2858a04640 1 --2- 172.21.15.49:0/945539498 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557594771ff0 0x55759476bf50 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x557594772de0 tx=0x7f2848004410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e32b0618ae930e6 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:08:01.545 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.539+0000 7f283e7fc640 1 -- 172.21.15.49:0/945539498 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2848003d80 con 0x557594771ff0 2024-03-18T23:08:01.545 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.539+0000 7f283e7fc640 1 -- 172.21.15.49:0/945539498 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f28480205d0 con 0x557594771ff0 2024-03-18T23:08:01.545 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.539+0000 7f283e7fc640 1 -- 172.21.15.49:0/945539498 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2848010d30 con 0x557594771ff0 2024-03-18T23:08:01.545 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.540+0000 7f285d850740 1 -- 172.21.15.49:0/945539498 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557594771ff0 msgr2=0x55759476bf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:01.545 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.540+0000 7f285d850740 1 --2- 172.21.15.49:0/945539498 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557594771ff0 0x55759476bf50 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x557594772de0 tx=0x7f2848004410 comp rx=0 tx=0).stop 2024-03-18T23:08:01.545 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.540+0000 7f285d850740 1 -- 172.21.15.49:0/945539498 shutdown_connections 2024-03-18T23:08:01.545 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.540+0000 7f285d850740 1 --2- 172.21.15.49:0/945539498 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557594777b50 0x55759476c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:01.545 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.540+0000 7f285d850740 1 --2- 172.21.15.49:0/945539498 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5575947729e0 0x557594783ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:01.545 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.540+0000 7f285d850740 1 --2- 172.21.15.49:0/945539498 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557594771ff0 0x55759476bf50 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:01.546 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.540+0000 7f285d850740 1 -- 172.21.15.49:0/945539498 >> 172.21.15.49:0/945539498 conn(0x55759476d560 msgr2=0x55759476d940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:08:01.546 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.540+0000 7f285d850740 1 -- 172.21.15.49:0/945539498 shutdown_connections 2024-03-18T23:08:01.546 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.540+0000 7f285d850740 1 -- 172.21.15.49:0/945539498 wait complete. 2024-03-18T23:08:01.546 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.540+0000 7f285d850740 1 -- start start 2024-03-18T23:08:01.546 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.540+0000 7f285d850740 1 Processor -- start 2024-03-18T23:08:01.546 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.540+0000 7f285d850740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557594771ff0 0x55759477fc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:01.546 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.540+0000 7f285d850740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5575947729e0 0x557594780170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:01.546 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.540+0000 7f285d850740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557594777b50 0x5575947830a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:01.546 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.540+0000 7f285d850740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55759478a6d0 con 0x5575947729e0 2024-03-18T23:08:01.546 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.540+0000 7f285d850740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55759478a000 con 0x557594771ff0 2024-03-18T23:08:01.546 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.540+0000 7f285d850740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55759478ada0 con 0x557594777b50 2024-03-18T23:08:01.546 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.540+0000 7f285d850740 10 client.0 _ll_register_callbacks cb 0x7f284000cf20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T23:08:01.546 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.541+0000 7f2853fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5575947729e0 0x557594780170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:01.547 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.541+0000 7f2853fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5575947729e0 0x557594780170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:37946/0 (socket says 172.21.15.49:37946) 2024-03-18T23:08:01.547 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.541+0000 7f2853fff640 1 -- 172.21.15.49:0/2465343086 learned_addr learned my addr 172.21.15.49:0/2465343086 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:08:01.547 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.541+0000 7f2858a04640 1 --2- 172.21.15.49:0/2465343086 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557594771ff0 0x55759477fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:01.547 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.541+0000 7f2859205640 1 --2- 172.21.15.49:0/2465343086 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557594777b50 0x5575947830a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:01.547 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.541+0000 7f2853fff640 1 -- 172.21.15.49:0/2465343086 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557594777b50 msgr2=0x5575947830a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:08:01.547 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.541+0000 7f2853fff640 1 --2- 172.21.15.49:0/2465343086 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557594777b50 0x5575947830a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:01.547 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.541+0000 7f2853fff640 1 -- 172.21.15.49:0/2465343086 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557594771ff0 msgr2=0x55759477fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:08:01.548 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.541+0000 7f2853fff640 1 --2- 172.21.15.49:0/2465343086 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557594771ff0 0x55759477fc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:01.548 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.541+0000 7f2853fff640 1 -- 172.21.15.49:0/2465343086 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557594783690 con 0x5575947729e0 2024-03-18T23:08:01.548 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.541+0000 7f2853fff640 1 --2- 172.21.15.49:0/2465343086 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5575947729e0 0x557594780170 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f2840097530 tx=0x7f284009b220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5dbedbd3c3aa6e0a server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:08:01.548 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.541+0000 7f2833fff640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-18T23:08:01.548 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.541+0000 7f2833fff640 1 -- 172.21.15.49:0/2465343086 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f284009aea0 con 0x5575947729e0 2024-03-18T23:08:01.548 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.541+0000 7f285d850740 1 -- 172.21.15.49:0/2465343086 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f28480205d0 con 0x5575947729e0 2024-03-18T23:08:01.548 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.541+0000 7f2833fff640 1 -- 172.21.15.49:0/2465343086 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f28400a3ab0 con 0x5575947729e0 2024-03-18T23:08:01.548 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.541+0000 7f2833fff640 1 -- 172.21.15.49:0/2465343086 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f284009b850 con 0x5575947729e0 2024-03-18T23:08:01.548 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.541+0000 7f285d850740 10 client.15309 fetch_fsmap learned FSMap version 27 2024-03-18T23:08:01.548 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.541+0000 7f285d850740 1 -- 172.21.15.49:0/2465343086 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5575947efaf0 con 0x5575947729e0 2024-03-18T23:08:01.548 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.542+0000 7f2833fff640 1 -- 172.21.15.49:0/2465343086 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f28480205d0 con 0x5575947729e0 2024-03-18T23:08:01.548 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.542+0000 7f2833fff640 1 -- 172.21.15.49:0/2465343086 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f284009a710 con 0x5575947729e0 2024-03-18T23:08:01.548 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.542+0000 7f285d850740 10 client.15309 fetch_fsmap finished waiting for FSMap version 27 2024-03-18T23:08:01.549 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.542+0000 7f285d850740 10 client.15309 resolve_mds: resolved name 'i' to daemon mds.i 2024-03-18T23:08:01.549 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.542+0000 7f285d850740 20 client.15309 populate_metadata read hostname 'smithi049' 2024-03-18T23:08:01.549 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.542+0000 7f285d850740 1 --2- 172.21.15.49:0/2465343086 >> [v2:172.21.15.175:6838/3640882084,v1:172.21.15.175:6839/3640882084] conn(0x5575947f4650 0x5575947f3360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:01.549 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.542+0000 7f285d850740 4 client.15309 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-18T23:08:01.549 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.542+0000 7f285d850740 1 -- 172.21.15.49:0/2465343086 --> [v2:172.21.15.175:6838/3640882084,v1:172.21.15.175:6839/3640882084] -- command(tid 0: {"prefix": "status"}) -- 0x5575946de670 con 0x5575947f4650 2024-03-18T23:08:01.549 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.542+0000 7f2858a04640 1 --2- 172.21.15.49:0/2465343086 >> [v2:172.21.15.175:6838/3640882084,v1:172.21.15.175:6839/3640882084] conn(0x5575947f4650 0x5575947f3360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:01.549 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.542+0000 7f2833fff640 10 client.15309 ms_handle_connect on v2:172.21.15.175:6838/3640882084 2024-03-18T23:08:01.549 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.542+0000 7f2858a04640 1 --2- 172.21.15.49:0/2465343086 >> [v2:172.21.15.175:6838/3640882084,v1:172.21.15.175:6839/3640882084] conn(0x5575947f4650 0x5575947f3360 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1f3519c9318bf269 server_cookie=cf27aa0f2d803a50 in_seq=0 out_seq=0 2024-03-18T23:08:01.549 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.542+0000 7f2833fff640 1 -- 172.21.15.49:0/2465343086 <== mds.1 v2:172.21.15.175:6838/3640882084 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x5575946de670 con 0x5575947f4650 2024-03-18T23:08:01.549 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.542+0000 7f2833fff640 10 client.15309 handle_command_reply: tid=0 2024-03-18T23:08:01.549 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.542+0000 7f285d850740 1 client.15309 shutdown 2024-03-18T23:08:01.549 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.543+0000 7f285d850740 1 -- 172.21.15.49:0/2465343086 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5575947729e0 msgr2=0x557594780170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:01.549 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.543+0000 7f285d850740 1 --2- 172.21.15.49:0/2465343086 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5575947729e0 0x557594780170 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f2840097530 tx=0x7f284009b220 comp rx=0 tx=0).stop 2024-03-18T23:08:01.549 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.543+0000 7f285d850740 1 -- 172.21.15.49:0/2465343086 shutdown_connections 2024-03-18T23:08:01.549 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.543+0000 7f285d850740 1 --2- 172.21.15.49:0/2465343086 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557594777b50 0x5575947830a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:01.550 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.543+0000 7f285d850740 1 --2- 172.21.15.49:0/2465343086 >> [v2:172.21.15.175:6838/3640882084,v1:172.21.15.175:6839/3640882084] conn(0x5575947f4650 0x5575947f3360 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:01.550 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.543+0000 7f285d850740 1 --2- 172.21.15.49:0/2465343086 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5575947729e0 0x557594780170 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:01.550 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.543+0000 7f285d850740 1 --2- 172.21.15.49:0/2465343086 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557594771ff0 0x55759477fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:01.550 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.543+0000 7f285d850740 1 -- 172.21.15.49:0/2465343086 >> 172.21.15.49:0/2465343086 conn(0x55759476d560 msgr2=0x5575947707c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:08:01.550 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.543+0000 7f2833fff640 0 client.15309 ms_handle_reset on v2:172.21.15.175:6838/3640882084 2024-03-18T23:08:01.550 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.543+0000 7f285d850740 1 -- 172.21.15.49:0/2465343086 shutdown_connections 2024-03-18T23:08:01.550 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.543+0000 7f285d850740 1 -- 172.21.15.49:0/2465343086 wait complete. 2024-03-18T23:08:01.550 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.543+0000 7f285d850740 20 client.15309 trim_cache size 0 max 16384 2024-03-18T23:08:01.550 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-03-18T23:08:01.550 INFO:teuthology.orchestra.run.smithi049.stdout: "cluster_fsid": "c63591ea-e57a-11ee-95c9-87774f69a715", 2024-03-18T23:08:01.551 INFO:teuthology.orchestra.run.smithi049.stdout: "whoami": 1, 2024-03-18T23:08:01.551 INFO:teuthology.orchestra.run.smithi049.stdout: "id": 24463, 2024-03-18T23:08:01.551 INFO:teuthology.orchestra.run.smithi049.stdout: "want_state": "up:active", 2024-03-18T23:08:01.551 INFO:teuthology.orchestra.run.smithi049.stdout: "state": "up:active", 2024-03-18T23:08:01.551 INFO:teuthology.orchestra.run.smithi049.stdout: "fs_name": "cephfs", 2024-03-18T23:08:01.551 INFO:teuthology.orchestra.run.smithi049.stdout: "rank_uptime": 113.302549578, 2024-03-18T23:08:01.551 INFO:teuthology.orchestra.run.smithi049.stdout: "mdsmap_epoch": 27, 2024-03-18T23:08:01.551 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_epoch": 88, 2024-03-18T23:08:01.551 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_epoch_barrier": 86, 2024-03-18T23:08:01.551 INFO:teuthology.orchestra.run.smithi049.stdout: "uptime": 129.23322457899999 2024-03-18T23:08:01.551 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-03-18T23:08:01.552 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.543+0000 7f285ac68640 1 -- 172.21.15.49:0/1426723832 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f28440a2800 msgr2=0x7f28440a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:01.552 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.543+0000 7f285ac68640 1 --2- 172.21.15.49:0/1426723832 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f28440a2800 0x7f28440a4cc0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f284c04c000 tx=0x7f284c045000 comp rx=0 tx=0).stop 2024-03-18T23:08:01.552 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.543+0000 7f285ac68640 1 -- 172.21.15.49:0/1426723832 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2854101930 msgr2=0x7f285419a2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:01.552 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.543+0000 7f285ac68640 1 --2- 172.21.15.49:0/1426723832 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2854101930 0x7f285419a2b0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f284000e9f0 tx=0x7f284000eeb0 comp rx=0 tx=0).stop 2024-03-18T23:08:01.552 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.543+0000 7f285ac68640 1 -- 172.21.15.49:0/1426723832 shutdown_connections 2024-03-18T23:08:01.552 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.543+0000 7f285ac68640 1 --2- 172.21.15.49:0/1426723832 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f28440a2800 0x7f28440a4cc0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:01.552 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.543+0000 7f285ac68640 1 --2- 172.21.15.49:0/1426723832 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f28541022f0 0x7f28541a1320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:01.552 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.543+0000 7f285ac68640 1 --2- 172.21.15.49:0/1426723832 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2854101930 0x7f285419a2b0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:01.552 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.543+0000 7f285ac68640 1 --2- 172.21.15.49:0/1426723832 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2854100740 0x7f2854199d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:01.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.543+0000 7f285ac68640 1 -- 172.21.15.49:0/1426723832 >> 172.21.15.49:0/1426723832 conn(0x7f28540fbcd0 msgr2=0x7f28540fd730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:08:01.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.544+0000 7f285ac68640 1 -- 172.21.15.49:0/1426723832 shutdown_connections 2024-03-18T23:08:01.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.544+0000 7f285ac68640 1 -- 172.21.15.49:0/1426723832 wait complete. 2024-03-18T23:08:01.554 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-03-18T23:08:01.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:08:01 smithi139 ceph-mon[67282]: pgmap v348: 97 pgs: 97 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-03-18T23:08:01.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:08:01 smithi139 ceph-mon[67282]: from='client.? 172.21.15.49:0/3207631162' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-18T23:08:01.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:08:01 smithi139 ceph-mon[67282]: from='client.? 172.21.15.49:0/4167121602' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T23:08:01.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:08:01 smithi139 ceph-mon[67282]: from='client.? 172.21.15.49:0/2481216016' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T23:08:01.665 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.664+0000 7fb2af1a9640 1 -- 172.21.15.49:0/4019608350 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb2a8103400 msgr2=0x7fb2a8109c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:01.666 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.664+0000 7fb2af1a9640 1 --2- 172.21.15.49:0/4019608350 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb2a8103400 0x7fb2a8109c90 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fb2a0009ca0 tx=0x7fb2a002c0f0 comp rx=0 tx=0).stop 2024-03-18T23:08:01.666 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.664+0000 7fb2af1a9640 1 -- 172.21.15.49:0/4019608350 shutdown_connections 2024-03-18T23:08:01.666 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.664+0000 7fb2af1a9640 1 --2- 172.21.15.49:0/4019608350 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb2a8103400 0x7fb2a8109c90 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:01.666 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.664+0000 7fb2af1a9640 1 --2- 172.21.15.49:0/4019608350 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb2a8102a40 0x7fb2a8102ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:01.666 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.664+0000 7fb2af1a9640 1 --2- 172.21.15.49:0/4019608350 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb2a8101850 0x7fb2a8101c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:01.666 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.664+0000 7fb2af1a9640 1 -- 172.21.15.49:0/4019608350 >> 172.21.15.49:0/4019608350 conn(0x7fb2a80fcdc0 msgr2=0x7fb2a80ff200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:08:01.666 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.664+0000 7fb2af1a9640 1 -- 172.21.15.49:0/4019608350 shutdown_connections 2024-03-18T23:08:01.666 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.664+0000 7fb2af1a9640 1 -- 172.21.15.49:0/4019608350 wait complete. 2024-03-18T23:08:01.666 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.664+0000 7fb2af1a9640 1 Processor -- start 2024-03-18T23:08:01.666 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.664+0000 7fb2af1a9640 1 -- start start 2024-03-18T23:08:01.666 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.664+0000 7fb2af1a9640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb2a8101850 0x7fb2a80755d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:01.667 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.664+0000 7fb2af1a9640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb2a8102a40 0x7fb2a8073c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:01.667 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.664+0000 7fb2af1a9640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb2a8103400 0x7fb2a8074150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:01.667 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.664+0000 7fb2af1a9640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fb2a810d4d0 con 0x7fb2a8103400 2024-03-18T23:08:01.667 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.664+0000 7fb2af1a9640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fb2a810d350 con 0x7fb2a8102a40 2024-03-18T23:08:01.667 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.664+0000 7fb2af1a9640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb2a810d650 con 0x7fb2a8101850 2024-03-18T23:08:01.667 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.664+0000 7fb2ad746640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb2a8103400 0x7fb2a8074150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:01.667 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.664+0000 7fb2acf45640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb2a8101850 0x7fb2a80755d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:01.667 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.664+0000 7fb2a7fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb2a8102a40 0x7fb2a8073c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:01.667 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.664+0000 7fb2ad746640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb2a8103400 0x7fb2a8074150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:37962/0 (socket says 172.21.15.49:37962) 2024-03-18T23:08:01.667 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.664+0000 7fb2acf45640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb2a8101850 0x7fb2a80755d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.49:52230/0 (socket says 172.21.15.49:52230) 2024-03-18T23:08:01.667 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.665+0000 7fb2ad746640 1 -- 172.21.15.49:0/1477141613 learned_addr learned my addr 172.21.15.49:0/1477141613 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:08:01.667 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.665+0000 7fb2ad746640 1 -- 172.21.15.49:0/1477141613 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb2a8101850 msgr2=0x7fb2a80755d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:01.667 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.665+0000 7fb2ad746640 1 --2- 172.21.15.49:0/1477141613 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb2a8101850 0x7fb2a80755d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:01.668 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.665+0000 7fb2ad746640 1 -- 172.21.15.49:0/1477141613 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb2a8102a40 msgr2=0x7fb2a8073c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:01.668 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.665+0000 7fb2ad746640 1 --2- 172.21.15.49:0/1477141613 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb2a8102a40 0x7fb2a8073c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:01.668 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.665+0000 7fb2ad746640 1 -- 172.21.15.49:0/1477141613 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2a80749e0 con 0x7fb2a8103400 2024-03-18T23:08:01.668 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.665+0000 7fb2ad746640 1 --2- 172.21.15.49:0/1477141613 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb2a8103400 0x7fb2a8074150 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fb2a0037040 tx=0x7fb2a0032f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:08:01.668 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.665+0000 7fb2a5ffb640 1 -- 172.21.15.49:0/1477141613 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2a0041020 con 0x7fb2a8103400 2024-03-18T23:08:01.668 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.665+0000 7fb2a5ffb640 1 -- 172.21.15.49:0/1477141613 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb2a0034cd0 con 0x7fb2a8103400 2024-03-18T23:08:01.668 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.665+0000 7fb2af1a9640 1 -- 172.21.15.49:0/1477141613 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2a81a1be0 con 0x7fb2a8103400 2024-03-18T23:08:01.668 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.665+0000 7fb2a5ffb640 1 -- 172.21.15.49:0/1477141613 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2a0039040 con 0x7fb2a8103400 2024-03-18T23:08:01.668 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.665+0000 7fb2af1a9640 1 -- 172.21.15.49:0/1477141613 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2a81a2130 con 0x7fb2a8103400 2024-03-18T23:08:01.675 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.667+0000 7fb2a5ffb640 1 -- 172.21.15.49:0/1477141613 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7fb2a0044020 con 0x7fb2a8103400 2024-03-18T23:08:01.675 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.668+0000 7fb2b1d91740 1 Processor -- start 2024-03-18T23:08:01.676 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.668+0000 7fb2b1d91740 1 -- start start 2024-03-18T23:08:01.676 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.668+0000 7fb2a5ffb640 1 --2- 172.21.15.49:0/1477141613 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fb2980a2800 0x7fb2980a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:01.676 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.668+0000 7fb2b1d91740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b178017ff0 0x55b178018410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:01.676 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.668+0000 7fb2b1d91740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b1780189e0 0x55b17801d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:01.676 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.668+0000 7fb2b1d91740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b17801db50 0x55b17801ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:01.676 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.668+0000 7fb2b1d91740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55b177f7cde0 con 0x55b1780189e0 2024-03-18T23:08:01.676 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.668+0000 7fb2b1d91740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55b177f25c80 con 0x55b178017ff0 2024-03-18T23:08:01.676 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.668+0000 7fb2b1d91740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55b177f78e80 con 0x55b17801db50 2024-03-18T23:08:01.676 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.668+0000 7fb2a7fff640 1 --2- 172.21.15.49:0/1477141613 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fb2980a2800 0x7fb2980a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:01.676 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.668+0000 7fb2a5ffb640 1 -- 172.21.15.49:0/1477141613 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7fb2a00f0870 con 0x7fb2a8103400 2024-03-18T23:08:01.676 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.668+0000 7fb2a7fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b1780189e0 0x55b17801d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:01.677 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.668+0000 7fb2acf45640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b178017ff0 0x55b178018410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:01.677 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.668+0000 7fb2ad746640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b17801db50 0x55b17801ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:01.677 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.668+0000 7fb2a7fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b1780189e0 0x55b17801d4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:37976/0 (socket says 172.21.15.49:37976) 2024-03-18T23:08:01.677 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.668+0000 7fb2acf45640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b178017ff0 0x55b178018410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.49:48746/0 (socket says 172.21.15.49:48746) 2024-03-18T23:08:01.677 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.668+0000 7fb2a7fff640 1 -- 172.21.15.49:0/1691851744 learned_addr learned my addr 172.21.15.49:0/1691851744 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:08:01.677 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.668+0000 7fb2acf45640 1 -- 172.21.15.49:0/1691851744 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b17801db50 msgr2=0x55b17801ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:08:01.677 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.668+0000 7fb2acf45640 1 --2- 172.21.15.49:0/1691851744 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b17801db50 0x55b17801ffe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:01.677 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.668+0000 7fb2acf45640 1 -- 172.21.15.49:0/1691851744 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b1780189e0 msgr2=0x55b17801d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:08:01.677 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.668+0000 7fb2acf45640 1 --2- 172.21.15.49:0/1691851744 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b1780189e0 0x55b17801d4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:01.677 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.668+0000 7fb2acf45640 1 -- 172.21.15.49:0/1691851744 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b177f37fb0 con 0x55b178017ff0 2024-03-18T23:08:01.677 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.668+0000 7fb2a7fff640 1 --2- 172.21.15.49:0/1691851744 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b1780189e0 0x55b17801d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:08:01.677 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.668+0000 7fb2ad746640 1 --2- 172.21.15.49:0/1691851744 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b17801db50 0x55b17801ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:08:01.678 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.669+0000 7fb2a7fff640 1 --2- 172.21.15.49:0/1477141613 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fb2980a2800 0x7fb2980a4cc0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fb2940119d0 tx=0x7fb294009c10 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:08:01.678 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.669+0000 7fb2acf45640 1 --2- 172.21.15.49:0/1691851744 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b178017ff0 0x55b178018410 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fb29c009bd0 tx=0x7fb29c011440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=49ce8c9787a77c99 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:08:01.678 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.669+0000 7fb292ffd640 1 -- 172.21.15.49:0/1691851744 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb29c004030 con 0x55b178017ff0 2024-03-18T23:08:01.678 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.669+0000 7fb292ffd640 1 -- 172.21.15.49:0/1691851744 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb29c0041d0 con 0x55b178017ff0 2024-03-18T23:08:01.678 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.669+0000 7fb292ffd640 1 -- 172.21.15.49:0/1691851744 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb29c019620 con 0x55b178017ff0 2024-03-18T23:08:01.678 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.669+0000 7fb2b1d91740 1 -- 172.21.15.49:0/1691851744 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b178017ff0 msgr2=0x55b178018410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:01.678 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.669+0000 7fb2b1d91740 1 --2- 172.21.15.49:0/1691851744 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b178017ff0 0x55b178018410 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fb29c009bd0 tx=0x7fb29c011440 comp rx=0 tx=0).stop 2024-03-18T23:08:01.678 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.669+0000 7fb2b1d91740 1 -- 172.21.15.49:0/1691851744 shutdown_connections 2024-03-18T23:08:01.678 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.669+0000 7fb2b1d91740 1 --2- 172.21.15.49:0/1691851744 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b17801db50 0x55b17801ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:01.678 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.669+0000 7fb2b1d91740 1 --2- 172.21.15.49:0/1691851744 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b1780189e0 0x55b17801d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:01.678 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.669+0000 7fb2b1d91740 1 --2- 172.21.15.49:0/1691851744 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b178017ff0 0x55b178018410 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:01.678 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.669+0000 7fb2b1d91740 1 -- 172.21.15.49:0/1691851744 >> 172.21.15.49:0/1691851744 conn(0x55b178013560 msgr2=0x55b178013960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:08:01.679 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.670+0000 7fb2b1d91740 1 -- 172.21.15.49:0/1691851744 shutdown_connections 2024-03-18T23:08:01.679 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.670+0000 7fb2b1d91740 1 -- 172.21.15.49:0/1691851744 wait complete. 2024-03-18T23:08:01.679 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.670+0000 7fb2b1d91740 1 -- start start 2024-03-18T23:08:01.679 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.670+0000 7fb2b1d91740 1 Processor -- start 2024-03-18T23:08:01.679 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.670+0000 7fb2b1d91740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b178017ff0 0x55b1780224e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:01.679 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.670+0000 7fb2b1d91740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b1780189e0 0x55b178022a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:01.679 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.670+0000 7fb2b1d91740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b17801db50 0x55b178029a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:01.679 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.670+0000 7fb2b1d91740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55b178020b00 con 0x55b1780189e0 2024-03-18T23:08:01.679 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.670+0000 7fb2b1d91740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55b177f38fc0 con 0x55b178017ff0 2024-03-18T23:08:01.679 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.670+0000 7fb2b1d91740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55b178020c80 con 0x55b17801db50 2024-03-18T23:08:01.679 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.670+0000 7fb2a7fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b1780189e0 0x55b178022a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:01.680 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.670+0000 7fb2acf45640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b178017ff0 0x55b1780224e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:01.680 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.670+0000 7fb2b1d91740 10 client.0 _ll_register_callbacks cb 0x55b177f74a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T23:08:01.680 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.670+0000 7fb2a7fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b1780189e0 0x55b178022a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:37992/0 (socket says 172.21.15.49:37992) 2024-03-18T23:08:01.680 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.670+0000 7fb2a7fff640 1 -- 172.21.15.49:0/1213837479 learned_addr learned my addr 172.21.15.49:0/1213837479 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:08:01.680 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.671+0000 7fb2ad746640 1 --2- 172.21.15.49:0/1213837479 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b17801db50 0x55b178029a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:01.680 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.671+0000 7fb2acf45640 1 -- 172.21.15.49:0/1213837479 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b17801db50 msgr2=0x55b178029a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:08:01.680 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.671+0000 7fb2acf45640 1 --2- 172.21.15.49:0/1213837479 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b17801db50 0x55b178029a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:01.680 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.671+0000 7fb2acf45640 1 -- 172.21.15.49:0/1213837479 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b1780189e0 msgr2=0x55b178022a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:08:01.680 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.671+0000 7fb2acf45640 1 --2- 172.21.15.49:0/1213837479 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b1780189e0 0x55b178022a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:01.680 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.671+0000 7fb2acf45640 1 -- 172.21.15.49:0/1213837479 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b177c912d0 con 0x55b178017ff0 2024-03-18T23:08:01.680 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.671+0000 7fb2a7fff640 1 --2- 172.21.15.49:0/1213837479 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b1780189e0 0x55b178022a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:08:01.680 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.671+0000 7fb2acf45640 1 --2- 172.21.15.49:0/1213837479 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b178017ff0 0x55b1780224e0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fb29c01b040 tx=0x7fb29c003ac0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3e38cc570cfc9aeb server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:08:01.680 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.671+0000 7fb287fff640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-03-18T23:08:01.680 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.671+0000 7fb287fff640 1 -- 172.21.15.49:0/1213837479 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb29c028070 con 0x55b178017ff0 2024-03-18T23:08:01.680 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.671+0000 7fb2b1d91740 1 -- 172.21.15.49:0/1213837479 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb29c0041d0 con 0x55b178017ff0 2024-03-18T23:08:01.681 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.671+0000 7fb287fff640 1 -- 172.21.15.49:0/1213837479 <== mon.2 v2:172.21.15.139:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7fb29c011ca0 con 0x55b178017ff0 2024-03-18T23:08:01.681 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.671+0000 7fb287fff640 1 -- 172.21.15.49:0/1213837479 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb29c034050 con 0x55b178017ff0 2024-03-18T23:08:01.681 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.671+0000 7fb2b1d91740 10 client.25142 fetch_fsmap learned FSMap version 27 2024-03-18T23:08:01.681 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.671+0000 7fb2b1d91740 1 -- 172.21.15.49:0/1213837479 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55b177d5b6b0 con 0x55b178017ff0 2024-03-18T23:08:01.681 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.672+0000 7fb287fff640 1 -- 172.21.15.49:0/1213837479 <== mon.2 v2:172.21.15.139:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb29c0041d0 con 0x55b178017ff0 2024-03-18T23:08:01.681 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.672+0000 7fb287fff640 1 -- 172.21.15.49:0/1213837479 <== mon.2 v2:172.21.15.139:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fb29c004a30 con 0x55b178017ff0 2024-03-18T23:08:01.681 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.672+0000 7fb2b1d91740 10 client.25142 fetch_fsmap finished waiting for FSMap version 27 2024-03-18T23:08:01.681 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.672+0000 7fb2b1d91740 10 client.25142 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-18T23:08:01.681 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.672+0000 7fb2b1d91740 20 client.25142 populate_metadata read hostname 'smithi049' 2024-03-18T23:08:01.681 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.672+0000 7fb2b1d91740 1 --2- 172.21.15.49:0/1213837479 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x55b17802f0c0 0x55b178030530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:01.681 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.672+0000 7fb2b1d91740 4 client.25142 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T23:08:01.681 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.672+0000 7fb2b1d91740 1 -- 172.21.15.49:0/1213837479 --> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b177f8a320 con 0x55b17802f0c0 2024-03-18T23:08:01.681 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.672+0000 7fb2a7fff640 1 --2- 172.21.15.49:0/1213837479 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x55b17802f0c0 0x55b178030530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:01.681 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.672+0000 7fb2a7fff640 1 --2- 172.21.15.49:0/1213837479 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x55b17802f0c0 0x55b178030530 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ecc950e1f698cc59 server_cookie=3682db3ab8034b45 in_seq=0 out_seq=0 2024-03-18T23:08:01.681 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.672+0000 7fb287fff640 10 client.25142 ms_handle_connect on v2:172.21.15.139:6838/4280128209 2024-03-18T23:08:01.682 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.673+0000 7fb287fff640 1 -- 172.21.15.49:0/1213837479 <== mds.0 v2:172.21.15.139:6838/4280128209 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55b177f8a320 con 0x55b17802f0c0 2024-03-18T23:08:01.682 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.673+0000 7fb287fff640 10 client.25142 handle_command_reply: tid=0 2024-03-18T23:08:01.682 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.673+0000 7fb2b1d91740 1 client.25142 shutdown 2024-03-18T23:08:01.682 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.673+0000 7fb2b1d91740 1 -- 172.21.15.49:0/1213837479 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b178017ff0 msgr2=0x55b1780224e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:01.682 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.673+0000 7fb2b1d91740 1 --2- 172.21.15.49:0/1213837479 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b178017ff0 0x55b1780224e0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fb29c01b040 tx=0x7fb29c003ac0 comp rx=0 tx=0).stop 2024-03-18T23:08:01.682 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.673+0000 7fb2b1d91740 1 -- 172.21.15.49:0/1213837479 shutdown_connections 2024-03-18T23:08:01.682 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.673+0000 7fb2b1d91740 1 --2- 172.21.15.49:0/1213837479 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x55b17802f0c0 0x55b178030530 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:01.682 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.673+0000 7fb2b1d91740 1 --2- 172.21.15.49:0/1213837479 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b17801db50 0x55b178029a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:01.682 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.673+0000 7fb2b1d91740 1 --2- 172.21.15.49:0/1213837479 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b1780189e0 0x55b178022a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:01.682 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.673+0000 7fb287fff640 0 client.25142 ms_handle_reset on v2:172.21.15.139:6838/4280128209 2024-03-18T23:08:01.682 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.673+0000 7fb2b1d91740 1 --2- 172.21.15.49:0/1213837479 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b178017ff0 0x55b1780224e0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:01.682 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.673+0000 7fb2b1d91740 1 -- 172.21.15.49:0/1213837479 >> 172.21.15.49:0/1213837479 conn(0x55b178013560 msgr2=0x55b1780127e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:08:01.682 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.674+0000 7fb2b1d91740 1 -- 172.21.15.49:0/1213837479 shutdown_connections 2024-03-18T23:08:01.682 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.674+0000 7fb2b1d91740 1 -- 172.21.15.49:0/1213837479 wait complete. 2024-03-18T23:08:01.683 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.674+0000 7fb2b1d91740 20 client.25142 trim_cache size 0 max 16384 2024-03-18T23:08:01.697 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.692+0000 7fb2b1d91740 1 Processor -- start 2024-03-18T23:08:01.698 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.692+0000 7fb2b1d91740 1 -- start start 2024-03-18T23:08:01.698 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.692+0000 7fb2b1d91740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b178017ff0 0x55b17801ff40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:01.698 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.692+0000 7fb2b1d91740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b1780189e0 0x55b1780299f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:01.698 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.692+0000 7fb2b1d91740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b17801db50 0x55b178019ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:01.698 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.692+0000 7fb2b1d91740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55b17802e870 con 0x55b1780189e0 2024-03-18T23:08:01.698 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.692+0000 7fb2b1d91740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55b17802ef40 con 0x55b17801db50 2024-03-18T23:08:01.698 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.692+0000 7fb2b1d91740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55b17802f610 con 0x55b178017ff0 2024-03-18T23:08:01.698 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.692+0000 7fb2a7fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b1780189e0 0x55b1780299f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:01.698 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.692+0000 7fb2a7fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b1780189e0 0x55b1780299f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:38000/0 (socket says 172.21.15.49:38000) 2024-03-18T23:08:01.698 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.692+0000 7fb2a7fff640 1 -- 172.21.15.49:0/3207787293 learned_addr learned my addr 172.21.15.49:0/3207787293 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:08:01.698 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.692+0000 7fb2acf45640 1 --2- 172.21.15.49:0/3207787293 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b178017ff0 0x55b17801ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:01.698 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.692+0000 7fb2ad746640 1 --2- 172.21.15.49:0/3207787293 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b17801db50 0x55b178019ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:01.698 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.693+0000 7fb2a7fff640 1 -- 172.21.15.49:0/3207787293 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b178017ff0 msgr2=0x55b17801ff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:08:01.699 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.693+0000 7fb2a7fff640 1 --2- 172.21.15.49:0/3207787293 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b178017ff0 0x55b17801ff40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:01.699 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.693+0000 7fb2a7fff640 1 -- 172.21.15.49:0/3207787293 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b17801db50 msgr2=0x55b178019ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:08:01.699 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.693+0000 7fb2a7fff640 1 --2- 172.21.15.49:0/3207787293 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b17801db50 0x55b178019ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:01.699 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.693+0000 7fb2a7fff640 1 -- 172.21.15.49:0/3207787293 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b178018dc0 con 0x55b1780189e0 2024-03-18T23:08:01.699 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.693+0000 7fb2ad746640 1 --2- 172.21.15.49:0/3207787293 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b17801db50 0x55b178019ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T23:08:01.699 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.693+0000 7fb2a7fff640 1 --2- 172.21.15.49:0/3207787293 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b1780189e0 0x55b1780299f0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fb29400fc80 tx=0x7fb294021000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86ade341e992151 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:08:01.699 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.693+0000 7fb2927fc640 1 -- 172.21.15.49:0/3207787293 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2940044c0 con 0x55b1780189e0 2024-03-18T23:08:01.699 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.693+0000 7fb2927fc640 1 -- 172.21.15.49:0/3207787293 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb29400fcb0 con 0x55b1780189e0 2024-03-18T23:08:01.699 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.693+0000 7fb2927fc640 1 -- 172.21.15.49:0/3207787293 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb294005280 con 0x55b1780189e0 2024-03-18T23:08:01.699 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.694+0000 7fb2b1d91740 1 -- 172.21.15.49:0/3207787293 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b1780189e0 msgr2=0x55b1780299f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:01.699 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.694+0000 7fb2b1d91740 1 --2- 172.21.15.49:0/3207787293 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b1780189e0 0x55b1780299f0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fb29400fc80 tx=0x7fb294021000 comp rx=0 tx=0).stop 2024-03-18T23:08:01.699 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.694+0000 7fb2b1d91740 1 -- 172.21.15.49:0/3207787293 shutdown_connections 2024-03-18T23:08:01.699 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.694+0000 7fb2b1d91740 1 --2- 172.21.15.49:0/3207787293 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b17801db50 0x55b178019ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:01.699 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.694+0000 7fb2b1d91740 1 --2- 172.21.15.49:0/3207787293 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b1780189e0 0x55b1780299f0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:01.700 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.694+0000 7fb2b1d91740 1 --2- 172.21.15.49:0/3207787293 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b178017ff0 0x55b17801ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:01.700 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.694+0000 7fb2b1d91740 1 -- 172.21.15.49:0/3207787293 >> 172.21.15.49:0/3207787293 conn(0x55b178013560 msgr2=0x55b17802e010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:08:01.700 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.694+0000 7fb2b1d91740 1 -- 172.21.15.49:0/3207787293 shutdown_connections 2024-03-18T23:08:01.700 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.694+0000 7fb2b1d91740 1 -- 172.21.15.49:0/3207787293 wait complete. 2024-03-18T23:08:01.700 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.694+0000 7fb2b1d91740 1 -- start start 2024-03-18T23:08:01.700 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.694+0000 7fb2b1d91740 1 Processor -- start 2024-03-18T23:08:01.700 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.694+0000 7fb2b1d91740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b178017ff0 0x55b1780279c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:01.700 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.694+0000 7fb2b1d91740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b1780189e0 0x55b178090e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:01.700 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.694+0000 7fb2b1d91740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b17801db50 0x55b178093480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:01.700 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.694+0000 7fb2b1d91740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55b1780303b0 con 0x55b1780189e0 2024-03-18T23:08:01.700 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.694+0000 7fb2b1d91740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55b17802fce0 con 0x55b17801db50 2024-03-18T23:08:01.700 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.694+0000 7fb2b1d91740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55b178030a80 con 0x55b178017ff0 2024-03-18T23:08:01.700 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.694+0000 7fb2b1d91740 10 client.0 _ll_register_callbacks cb 0x7fb294011520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T23:08:01.700 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.694+0000 7fb2a7fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b1780189e0 0x55b178090e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:01.700 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.694+0000 7fb2a7fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b1780189e0 0x55b178090e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:38016/0 (socket says 172.21.15.49:38016) 2024-03-18T23:08:01.701 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.694+0000 7fb2acf45640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b178017ff0 0x55b1780279c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:01.701 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.694+0000 7fb2a7fff640 1 -- 172.21.15.49:0/2229038990 learned_addr learned my addr 172.21.15.49:0/2229038990 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:08:01.701 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.694+0000 7fb2ad746640 1 --2- 172.21.15.49:0/2229038990 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b17801db50 0x55b178093480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:01.701 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.695+0000 7fb2a7fff640 1 -- 172.21.15.49:0/2229038990 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b178017ff0 msgr2=0x55b1780279c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:08:01.701 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.695+0000 7fb2a7fff640 1 --2- 172.21.15.49:0/2229038990 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b178017ff0 0x55b1780279c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:01.701 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.695+0000 7fb2a7fff640 1 -- 172.21.15.49:0/2229038990 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b17801db50 msgr2=0x55b178093480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:08:01.701 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.695+0000 7fb2a7fff640 1 --2- 172.21.15.49:0/2229038990 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b17801db50 0x55b178093480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:01.701 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.695+0000 7fb2a7fff640 1 -- 172.21.15.49:0/2229038990 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b178093cb0 con 0x55b1780189e0 2024-03-18T23:08:01.701 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.695+0000 7fb2a7fff640 1 --2- 172.21.15.49:0/2229038990 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b1780189e0 0x55b178090e10 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fb29400f790 tx=0x7fb294005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9c921fec08c6e89 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:08:01.701 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.695+0000 7fb287fff640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-18T23:08:01.701 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.695+0000 7fb287fff640 1 -- 172.21.15.49:0/2229038990 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2940044c0 con 0x55b1780189e0 2024-03-18T23:08:01.701 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.695+0000 7fb2b1d91740 1 -- 172.21.15.49:0/2229038990 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb29400fcb0 con 0x55b1780189e0 2024-03-18T23:08:01.701 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.695+0000 7fb287fff640 1 -- 172.21.15.49:0/2229038990 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7fb294003af0 con 0x55b1780189e0 2024-03-18T23:08:01.701 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.695+0000 7fb287fff640 1 -- 172.21.15.49:0/2229038990 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb29400fe60 con 0x55b1780189e0 2024-03-18T23:08:01.701 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.695+0000 7fb2b1d91740 10 client.15327 fetch_fsmap learned FSMap version 27 2024-03-18T23:08:01.702 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.695+0000 7fb2b1d91740 1 -- 172.21.15.49:0/2229038990 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55b178094fb0 con 0x55b1780189e0 2024-03-18T23:08:01.702 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.695+0000 7fb287fff640 1 -- 172.21.15.49:0/2229038990 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb29400fcb0 con 0x55b1780189e0 2024-03-18T23:08:01.702 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.696+0000 7fb287fff640 1 -- 172.21.15.49:0/2229038990 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fb29403b070 con 0x55b1780189e0 2024-03-18T23:08:01.702 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.696+0000 7fb2b1d91740 10 client.15327 fetch_fsmap finished waiting for FSMap version 27 2024-03-18T23:08:01.702 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.696+0000 7fb2b1d91740 10 client.15327 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-18T23:08:01.702 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.696+0000 7fb2b1d91740 20 client.15327 populate_metadata read hostname 'smithi049' 2024-03-18T23:08:01.702 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.696+0000 7fb2b1d91740 1 --2- 172.21.15.49:0/2229038990 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x55b17802f0c0 0x55b178098560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:01.702 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.696+0000 7fb2b1d91740 4 client.15327 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-18T23:08:01.702 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.696+0000 7fb2b1d91740 1 -- 172.21.15.49:0/2229038990 --> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] -- command(tid 0: {"prefix": "status"}) -- 0x55b177f84670 con 0x55b17802f0c0 2024-03-18T23:08:01.702 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.696+0000 7fb2acf45640 1 --2- 172.21.15.49:0/2229038990 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x55b17802f0c0 0x55b178098560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:01.702 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.696+0000 7fb2acf45640 1 --2- 172.21.15.49:0/2229038990 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x55b17802f0c0 0x55b178098560 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d3a79abbf2a457f0 server_cookie=14c82fa87244ee5 in_seq=0 out_seq=0 2024-03-18T23:08:01.702 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.696+0000 7fb287fff640 10 client.15327 ms_handle_connect on v2:172.21.15.139:6838/4280128209 2024-03-18T23:08:01.702 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.696+0000 7fb287fff640 1 -- 172.21.15.49:0/2229038990 <== mds.0 v2:172.21.15.139:6838/4280128209 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55b177f84670 con 0x55b17802f0c0 2024-03-18T23:08:01.702 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.696+0000 7fb287fff640 10 client.15327 handle_command_reply: tid=0 2024-03-18T23:08:01.703 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.696+0000 7fb2b1d91740 1 client.15327 shutdown 2024-03-18T23:08:01.703 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.696+0000 7fb2b1d91740 1 -- 172.21.15.49:0/2229038990 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b1780189e0 msgr2=0x55b178090e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:01.703 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.696+0000 7fb2b1d91740 1 --2- 172.21.15.49:0/2229038990 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b1780189e0 0x55b178090e10 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fb29400f790 tx=0x7fb294005850 comp rx=0 tx=0).stop 2024-03-18T23:08:01.703 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.697+0000 7fb2b1d91740 1 -- 172.21.15.49:0/2229038990 shutdown_connections 2024-03-18T23:08:01.703 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.697+0000 7fb2b1d91740 1 --2- 172.21.15.49:0/2229038990 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x55b17802f0c0 0x55b178098560 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:01.703 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.697+0000 7fb2b1d91740 1 --2- 172.21.15.49:0/2229038990 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b17801db50 0x55b178093480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:01.703 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.697+0000 7fb2b1d91740 1 --2- 172.21.15.49:0/2229038990 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b1780189e0 0x55b178090e10 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:01.703 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.697+0000 7fb2b1d91740 1 --2- 172.21.15.49:0/2229038990 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b178017ff0 0x55b1780279c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:01.703 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.697+0000 7fb287fff640 0 client.15327 ms_handle_reset on v2:172.21.15.139:6838/4280128209 2024-03-18T23:08:01.703 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.697+0000 7fb2b1d91740 1 -- 172.21.15.49:0/2229038990 >> 172.21.15.49:0/2229038990 conn(0x55b178013560 msgr2=0x55b17801d0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:08:01.703 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.697+0000 7fb2b1d91740 1 -- 172.21.15.49:0/2229038990 shutdown_connections 2024-03-18T23:08:01.703 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.697+0000 7fb2b1d91740 1 -- 172.21.15.49:0/2229038990 wait complete. 2024-03-18T23:08:01.703 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.697+0000 7fb2b1d91740 20 client.15327 trim_cache size 0 max 16384 2024-03-18T23:08:01.704 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-03-18T23:08:01.704 INFO:teuthology.orchestra.run.smithi049.stdout: "cluster_fsid": "c63591ea-e57a-11ee-95c9-87774f69a715", 2024-03-18T23:08:01.704 INFO:teuthology.orchestra.run.smithi049.stdout: "whoami": 0, 2024-03-18T23:08:01.704 INFO:teuthology.orchestra.run.smithi049.stdout: "id": 24488, 2024-03-18T23:08:01.704 INFO:teuthology.orchestra.run.smithi049.stdout: "want_state": "up:active", 2024-03-18T23:08:01.704 INFO:teuthology.orchestra.run.smithi049.stdout: "state": "up:active", 2024-03-18T23:08:01.704 INFO:teuthology.orchestra.run.smithi049.stdout: "fs_name": "cephfs", 2024-03-18T23:08:01.704 INFO:teuthology.orchestra.run.smithi049.stdout: "rank_uptime": 116.481391241, 2024-03-18T23:08:01.704 INFO:teuthology.orchestra.run.smithi049.stdout: "mdsmap_epoch": 27, 2024-03-18T23:08:01.704 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_epoch": 88, 2024-03-18T23:08:01.704 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_epoch_barrier": 86, 2024-03-18T23:08:01.704 INFO:teuthology.orchestra.run.smithi049.stdout: "uptime": 120.71045283799999 2024-03-18T23:08:01.704 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-03-18T23:08:01.705 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.697+0000 7fb2af1a9640 1 -- 172.21.15.49:0/1477141613 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fb2980a2800 msgr2=0x7fb2980a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:01.705 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.697+0000 7fb2af1a9640 1 --2- 172.21.15.49:0/1477141613 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fb2980a2800 0x7fb2980a4cc0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fb2940119d0 tx=0x7fb294009c10 comp rx=0 tx=0).stop 2024-03-18T23:08:01.705 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.697+0000 7fb2af1a9640 1 -- 172.21.15.49:0/1477141613 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb2a8103400 msgr2=0x7fb2a8074150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:01.705 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.697+0000 7fb2af1a9640 1 --2- 172.21.15.49:0/1477141613 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb2a8103400 0x7fb2a8074150 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fb2a0037040 tx=0x7fb2a0032f40 comp rx=0 tx=0).stop 2024-03-18T23:08:01.705 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.697+0000 7fb2af1a9640 1 -- 172.21.15.49:0/1477141613 shutdown_connections 2024-03-18T23:08:01.705 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.697+0000 7fb2af1a9640 1 --2- 172.21.15.49:0/1477141613 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fb2980a2800 0x7fb2980a4cc0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:01.706 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.697+0000 7fb2af1a9640 1 --2- 172.21.15.49:0/1477141613 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb2a8103400 0x7fb2a8074150 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:01.706 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.697+0000 7fb2af1a9640 1 --2- 172.21.15.49:0/1477141613 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb2a8102a40 0x7fb2a8073c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:01.706 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.697+0000 7fb2af1a9640 1 --2- 172.21.15.49:0/1477141613 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb2a8101850 0x7fb2a80755d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:01.706 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.697+0000 7fb2af1a9640 1 -- 172.21.15.49:0/1477141613 >> 172.21.15.49:0/1477141613 conn(0x7fb2a80fcdc0 msgr2=0x7fb2a8105d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:08:01.706 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.697+0000 7fb2af1a9640 1 -- 172.21.15.49:0/1477141613 shutdown_connections 2024-03-18T23:08:01.706 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:01.697+0000 7fb2af1a9640 1 -- 172.21.15.49:0/1477141613 wait complete. 2024-03-18T23:08:01.708 INFO:teuthology.run_tasks:Running task kclient... 2024-03-18T23:08:01.732 INFO:tasks.kclient:Mounting kernel clients... 2024-03-18T23:08:01.732 INFO:tasks.kclient:config is {'client.0': {}} 2024-03-18T23:08:01.733 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-18T23:08:01.733 DEBUG:teuthology.orchestra.run.smithi049:> ip netns list 2024-03-18T23:08:01.749 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-18T23:08:01.749 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip link delete ceph-brx 2024-03-18T23:08:01.821 INFO:teuthology.orchestra.run.smithi049.stderr:Cannot find device "ceph-brx" 2024-03-18T23:08:01.822 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-18T23:08:01.823 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=secure', 'nowsync'], 'syntax': 'v1', 'mount_subvol_num': 1} 2024-03-18T23:08:01.823 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-03-18T23:08:01.823 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-18T23:08:01.823 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell --fsid c63591ea-e57a-11ee-95c9-87774f69a715 -- ceph fs ls 2024-03-18T23:08:02.112 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T23:08:03.351 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:08:03 smithi049 ceph-mon[62991]: pgmap v349: 97 pgs: 97 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-03-18T23:08:03.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:08:03 smithi175 ceph-mon[67352]: pgmap v349: 97 pgs: 97 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-03-18T23:08:03.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:08:03 smithi139 ceph-mon[67282]: pgmap v349: 97 pgs: 97 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-03-18T23:08:03.937 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:03.937+0000 7ff7c46b2700 1 -- 172.21.15.49:0/11703753 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff7bc107750 msgr2=0x7ff7bc10f2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:03.937 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:03.937+0000 7ff7c46b2700 1 --2- 172.21.15.49:0/11703753 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff7bc107750 0x7ff7bc10f2d0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7ff7b400b0a0 tx=0x7ff7b400b3b0 comp rx=0 tx=0).stop 2024-03-18T23:08:03.937 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:03.937+0000 7ff7c46b2700 1 -- 172.21.15.49:0/11703753 shutdown_connections 2024-03-18T23:08:03.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:03.937+0000 7ff7c46b2700 1 --2- 172.21.15.49:0/11703753 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff7bc107750 0x7ff7bc10f2d0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:03.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:03.937+0000 7ff7c46b2700 1 --2- 172.21.15.49:0/11703753 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff7bc06acb0 0x7ff7bc06b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:03.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:03.937+0000 7ff7c46b2700 1 --2- 172.21.15.49:0/11703753 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff7bc06b660 0x7ff7bc107020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:03.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:03.937+0000 7ff7c46b2700 1 -- 172.21.15.49:0/11703753 >> 172.21.15.49:0/11703753 conn(0x7ff7bc078070 msgr2=0x7ff7bc078480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:08:03.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:03.937+0000 7ff7c46b2700 1 -- 172.21.15.49:0/11703753 shutdown_connections 2024-03-18T23:08:03.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:03.937+0000 7ff7c46b2700 1 -- 172.21.15.49:0/11703753 wait complete. 2024-03-18T23:08:03.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:03.937+0000 7ff7c46b2700 1 Processor -- start 2024-03-18T23:08:03.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:03.937+0000 7ff7c46b2700 1 -- start start 2024-03-18T23:08:03.939 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:03.938+0000 7ff7c46b2700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff7bc06acb0 0x7ff7bc19e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:03.939 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:03.938+0000 7ff7c46b2700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff7bc06b660 0x7ff7bc19ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:03.940 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:03.938+0000 7ff7c46b2700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff7bc107750 0x7ff7bc198870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:03.940 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:03.938+0000 7ff7c46b2700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7ff7bc1126b0 con 0x7ff7bc06b660 2024-03-18T23:08:03.940 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:03.938+0000 7ff7c46b2700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7ff7bc112530 con 0x7ff7bc06acb0 2024-03-18T23:08:03.940 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:03.938+0000 7ff7c46b2700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff7bc112830 con 0x7ff7bc107750 2024-03-18T23:08:03.940 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:03.938+0000 7ff7c1c4d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff7bc06b660 0x7ff7bc19ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:03.940 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:03.938+0000 7ff7c244e700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff7bc06acb0 0x7ff7bc19e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:03.940 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:03.938+0000 7ff7c1c4d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff7bc06b660 0x7ff7bc19ebf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:38038/0 (socket says 172.21.15.49:38038) 2024-03-18T23:08:03.940 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:03.938+0000 7ff7c1c4d700 1 -- 172.21.15.49:0/3270239979 learned_addr learned my addr 172.21.15.49:0/3270239979 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:08:03.940 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:03.938+0000 7ff7c2c4f700 1 --2- 172.21.15.49:0/3270239979 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff7bc107750 0x7ff7bc198870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:03.940 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:03.938+0000 7ff7c244e700 1 -- 172.21.15.49:0/3270239979 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff7bc107750 msgr2=0x7ff7bc198870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:03.940 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:03.938+0000 7ff7c244e700 1 --2- 172.21.15.49:0/3270239979 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff7bc107750 0x7ff7bc198870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:03.940 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:03.938+0000 7ff7c244e700 1 -- 172.21.15.49:0/3270239979 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff7bc06b660 msgr2=0x7ff7bc19ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:03.940 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:03.938+0000 7ff7c244e700 1 --2- 172.21.15.49:0/3270239979 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff7bc06b660 0x7ff7bc19ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:03.940 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:03.938+0000 7ff7c244e700 1 -- 172.21.15.49:0/3270239979 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7bc198fd0 con 0x7ff7bc06acb0 2024-03-18T23:08:03.941 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:03.938+0000 7ff7c2c4f700 1 --2- 172.21.15.49:0/3270239979 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff7bc107750 0x7ff7bc198870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:08:03.941 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:03.938+0000 7ff7c1c4d700 1 --2- 172.21.15.49:0/3270239979 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff7bc06b660 0x7ff7bc19ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:08:03.941 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:03.938+0000 7ff7c244e700 1 --2- 172.21.15.49:0/3270239979 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff7bc06acb0 0x7ff7bc19e6b0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7ff7b800e9f0 tx=0x7ff7b800ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:08:03.941 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:03.938+0000 7ff7b37fe700 1 -- 172.21.15.49:0/3270239979 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7b8004220 con 0x7ff7bc06acb0 2024-03-18T23:08:03.941 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:03.938+0000 7ff7c46b2700 1 -- 172.21.15.49:0/3270239979 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7bc1992c0 con 0x7ff7bc06acb0 2024-03-18T23:08:03.941 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:03.938+0000 7ff7b37fe700 1 -- 172.21.15.49:0/3270239979 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff7b8004d10 con 0x7ff7bc06acb0 2024-03-18T23:08:03.941 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:03.938+0000 7ff7b37fe700 1 -- 172.21.15.49:0/3270239979 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7b8005020 con 0x7ff7bc06acb0 2024-03-18T23:08:03.941 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:03.939+0000 7ff7c46b2700 1 -- 172.21.15.49:0/3270239979 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7bc1a5580 con 0x7ff7bc06acb0 2024-03-18T23:08:03.941 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:03.939+0000 7ff7c46b2700 1 -- 172.21.15.49:0/3270239979 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7bc103ad0 con 0x7ff7bc06acb0 2024-03-18T23:08:03.942 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:03.941+0000 7ff7b37fe700 1 -- 172.21.15.49:0/3270239979 <== mon.2 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7ff7b8004750 con 0x7ff7bc06acb0 2024-03-18T23:08:03.942 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:03.941+0000 7ff7b37fe700 1 --2- 172.21.15.49:0/3270239979 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7ff7a80a2890 0x7ff7a80a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:03.942 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:03.942+0000 7ff7c1c4d700 1 --2- 172.21.15.49:0/3270239979 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7ff7a80a2890 0x7ff7a80a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:03.942 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:03.942+0000 7ff7b37fe700 1 -- 172.21.15.49:0/3270239979 <== mon.2 v2:172.21.15.139:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7ff7b80cea10 con 0x7ff7bc06acb0 2024-03-18T23:08:03.942 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:03.942+0000 7ff7c1c4d700 1 --2- 172.21.15.49:0/3270239979 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7ff7a80a2890 0x7ff7a80a4d50 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7ff7bc1089c0 tx=0x7ff7ac009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:08:03.944 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:03.945+0000 7ff7b37fe700 1 -- 172.21.15.49:0/3270239979 <== mon.2 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7ff7b809baf0 con 0x7ff7bc06acb0 2024-03-18T23:08:04.200 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:04.200+0000 7ff7c46b2700 1 -- 172.21.15.49:0/3270239979 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7ff7bc1a5870 con 0x7ff7bc06acb0 2024-03-18T23:08:04.200 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:04.201+0000 7ff7b37fe700 1 -- 172.21.15.49:0/3270239979 <== mon.2 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v27) ==== 53+0+73 (secure 0 0 0) 0x7ff7b809b3f0 con 0x7ff7bc06acb0 2024-03-18T23:08:04.201 INFO:teuthology.orchestra.run.smithi049.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-03-18T23:08:04.203 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:04.203+0000 7ff7c46b2700 1 -- 172.21.15.49:0/3270239979 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7ff7a80a2890 msgr2=0x7ff7a80a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:04.203 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:04.203+0000 7ff7c46b2700 1 --2- 172.21.15.49:0/3270239979 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7ff7a80a2890 0x7ff7a80a4d50 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7ff7bc1089c0 tx=0x7ff7ac009260 comp rx=0 tx=0).stop 2024-03-18T23:08:04.203 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:04.203+0000 7ff7c46b2700 1 -- 172.21.15.49:0/3270239979 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff7bc06acb0 msgr2=0x7ff7bc19e6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:04.203 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:04.203+0000 7ff7c46b2700 1 --2- 172.21.15.49:0/3270239979 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff7bc06acb0 0x7ff7bc19e6b0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7ff7b800e9f0 tx=0x7ff7b800ed00 comp rx=0 tx=0).stop 2024-03-18T23:08:04.204 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:04.203+0000 7ff7c46b2700 1 -- 172.21.15.49:0/3270239979 shutdown_connections 2024-03-18T23:08:04.204 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:04.203+0000 7ff7c46b2700 1 --2- 172.21.15.49:0/3270239979 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7ff7a80a2890 0x7ff7a80a4d50 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:04.204 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:04.203+0000 7ff7c46b2700 1 --2- 172.21.15.49:0/3270239979 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff7bc107750 0x7ff7bc198870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:04.204 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:04.203+0000 7ff7c46b2700 1 --2- 172.21.15.49:0/3270239979 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff7bc06acb0 0x7ff7bc19e6b0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:04.204 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:04.203+0000 7ff7c46b2700 1 --2- 172.21.15.49:0/3270239979 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff7bc06b660 0x7ff7bc19ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:04.204 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:04.203+0000 7ff7c46b2700 1 -- 172.21.15.49:0/3270239979 >> 172.21.15.49:0/3270239979 conn(0x7ff7bc078070 msgr2=0x7ff7bc0ff460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:08:04.204 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:04.203+0000 7ff7c46b2700 1 -- 172.21.15.49:0/3270239979 shutdown_connections 2024-03-18T23:08:04.204 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:04.203+0000 7ff7c46b2700 1 -- 172.21.15.49:0/3270239979 wait complete. 2024-03-18T23:08:04.916 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-18T23:08:04.916 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-18T23:08:04.916 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi049.front.sepia.ceph.com 2024-03-18T23:08:04.916 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-03-18T23:08:04.916 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-18T23:08:04.917 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-18T23:08:04.917 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-18T23:08:04.917 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-18T23:08:04.917 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-18T23:08:04.917 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-18T23:08:04.917 DEBUG:teuthology.orchestra.run.smithi049:> ip addr 2024-03-18T23:08:04.934 INFO:teuthology.orchestra.run.smithi049.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-18T23:08:04.934 INFO:teuthology.orchestra.run.smithi049.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-18T23:08:04.934 INFO:teuthology.orchestra.run.smithi049.stdout: inet 127.0.0.1/8 scope host lo 2024-03-18T23:08:04.934 INFO:teuthology.orchestra.run.smithi049.stdout: valid_lft forever preferred_lft forever 2024-03-18T23:08:04.934 INFO:teuthology.orchestra.run.smithi049.stdout: inet6 ::1/128 scope host 2024-03-18T23:08:04.934 INFO:teuthology.orchestra.run.smithi049.stdout: valid_lft forever preferred_lft forever 2024-03-18T23:08:04.934 INFO:teuthology.orchestra.run.smithi049.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-18T23:08:04.935 INFO:teuthology.orchestra.run.smithi049.stdout: link/ether 0c:c4:7a:6b:ff:84 brd ff:ff:ff:ff:ff:ff 2024-03-18T23:08:04.935 INFO:teuthology.orchestra.run.smithi049.stdout: altname enp7s0f0 2024-03-18T23:08:04.935 INFO:teuthology.orchestra.run.smithi049.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-18T23:08:04.935 INFO:teuthology.orchestra.run.smithi049.stdout: link/ether 0c:c4:7a:6b:ff:85 brd ff:ff:ff:ff:ff:ff 2024-03-18T23:08:04.935 INFO:teuthology.orchestra.run.smithi049.stdout: altname enp7s0f1 2024-03-18T23:08:04.935 INFO:teuthology.orchestra.run.smithi049.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-18T23:08:04.935 INFO:teuthology.orchestra.run.smithi049.stdout: link/ether 0c:c4:7a:bd:15:66 brd ff:ff:ff:ff:ff:ff 2024-03-18T23:08:04.935 INFO:teuthology.orchestra.run.smithi049.stdout: altname enp1s0f0 2024-03-18T23:08:04.935 INFO:teuthology.orchestra.run.smithi049.stdout: inet 172.21.15.49/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-03-18T23:08:04.935 INFO:teuthology.orchestra.run.smithi049.stdout: valid_lft 38952sec preferred_lft 38952sec 2024-03-18T23:08:04.935 INFO:teuthology.orchestra.run.smithi049.stdout: inet6 fe80::ec4:7aff:febd:1566/64 scope link noprefixroute 2024-03-18T23:08:04.935 INFO:teuthology.orchestra.run.smithi049.stdout: valid_lft forever preferred_lft forever 2024-03-18T23:08:04.935 INFO:teuthology.orchestra.run.smithi049.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-18T23:08:04.935 INFO:teuthology.orchestra.run.smithi049.stdout: link/ether 0c:c4:7a:bd:15:67 brd ff:ff:ff:ff:ff:ff 2024-03-18T23:08:04.936 INFO:teuthology.orchestra.run.smithi049.stdout: altname enp1s0f1 2024-03-18T23:08:04.936 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-03-18T23:08:04.936 DEBUG:teuthology.orchestra.run.smithi049:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-18T23:08:04.936 DEBUG:teuthology.orchestra.run.smithi049:> set -e 2024-03-18T23:08:04.936 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip link add name ceph-brx type bridge 2024-03-18T23:08:04.936 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip addr flush dev ceph-brx 2024-03-18T23:08:04.936 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip link set ceph-brx up 2024-03-18T23:08:04.936 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-03-18T23:08:04.937 DEBUG:teuthology.orchestra.run.smithi049:> ') 2024-03-18T23:08:05.025 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:05 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-18T23:08:05.142 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:08:05 smithi049 ceph-mon[62991]: pgmap v350: 97 pgs: 97 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-18T23:08:05.142 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:08:05 smithi049 ceph-mon[62991]: from='client.? 172.21.15.49:0/3270239979' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-18T23:08:05.145 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-18T23:08:05.151 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-18T23:08:05.152 DEBUG:teuthology.orchestra.run.smithi049:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-18T23:08:05.183 INFO:teuthology.orchestra.run.smithi049.stdout:1 2024-03-18T23:08:05.184 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-18T23:08:05.184 DEBUG:teuthology.orchestra.run.smithi049:> route 2024-03-18T23:08:05.259 INFO:teuthology.orchestra.run.smithi049.stdout:Kernel IP routing table 2024-03-18T23:08:05.259 INFO:teuthology.orchestra.run.smithi049.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-18T23:08:05.259 INFO:teuthology.orchestra.run.smithi049.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-03-18T23:08:05.259 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-03-18T23:08:05.260 INFO:teuthology.orchestra.run.smithi049.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-18T23:08:05.260 DEBUG:teuthology.orchestra.run.smithi049:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-18T23:08:05.260 DEBUG:teuthology.orchestra.run.smithi049:> set -e 2024-03-18T23:08:05.260 DEBUG:teuthology.orchestra.run.smithi049:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-03-18T23:08:05.260 DEBUG:teuthology.orchestra.run.smithi049:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-03-18T23:08:05.260 DEBUG:teuthology.orchestra.run.smithi049:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-03-18T23:08:05.260 DEBUG:teuthology.orchestra.run.smithi049:> ') 2024-03-18T23:08:05.349 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:05 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-18T23:08:05.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:08:05 smithi175 ceph-mon[67352]: pgmap v350: 97 pgs: 97 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-18T23:08:05.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:08:05 smithi175 ceph-mon[67352]: from='client.? 172.21.15.49:0/3270239979' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-18T23:08:05.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:08:05 smithi139 ceph-mon[67282]: pgmap v350: 97 pgs: 97 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-18T23:08:05.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:08:05 smithi139 ceph-mon[67282]: from='client.? 172.21.15.49:0/3270239979' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-18T23:08:05.566 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-18T23:08:05.572 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-18T23:08:05.572 DEBUG:teuthology.orchestra.run.smithi049:> ip netns list 2024-03-18T23:08:05.588 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-18T23:08:05.588 DEBUG:teuthology.orchestra.run.smithi049:> ip netns list-id 2024-03-18T23:08:05.645 DEBUG:teuthology.orchestra.run.smithi049:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-18T23:08:05.645 DEBUG:teuthology.orchestra.run.smithi049:> set -e 2024-03-18T23:08:05.646 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-18T23:08:05.646 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-03-18T23:08:05.646 DEBUG:teuthology.orchestra.run.smithi049:> ') 2024-03-18T23:08:05.735 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:05 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-18T23:08:05.770 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-18T23:08:05.776 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-03-18T23:08:05.776 DEBUG:teuthology.orchestra.run.smithi049:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-18T23:08:05.776 DEBUG:teuthology.orchestra.run.smithi049:> set -e 2024-03-18T23:08:05.777 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-03-18T23:08:05.777 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-03-18T23:08:05.777 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-03-18T23:08:05.777 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-03-18T23:08:05.777 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-03-18T23:08:05.777 DEBUG:teuthology.orchestra.run.smithi049:> ') 2024-03-18T23:08:05.826 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:05 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-18T23:08:05.930 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-18T23:08:05.937 DEBUG:teuthology.orchestra.run.smithi049:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-18T23:08:05.937 DEBUG:teuthology.orchestra.run.smithi049:> set -e 2024-03-18T23:08:05.937 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip link set brx.0 up 2024-03-18T23:08:05.937 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip link set dev brx.0 master ceph-brx 2024-03-18T23:08:05.937 DEBUG:teuthology.orchestra.run.smithi049:> ') 2024-03-18T23:08:05.986 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:05 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-18T23:08:06.021 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-18T23:08:06.028 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-18T23:08:06.028 DEBUG:teuthology.orchestra.run.smithi049:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-03-18T23:08:06.043 INFO:teuthology.orchestra.run.smithi049.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-03-18T23:08:06.043 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-18T23:08:06.043 DEBUG:teuthology.orchestra.run.smithi049:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-03-18T23:08:06.099 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-03-18T23:08:06.099 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-03-18T23:08:06.099 INFO:tasks.cephfs.kernel_mount:mounting using device: :/volumes/qa/sv_1/b5d9a7f7-1d19-4c17-b495-a118d7fb5c7e 2024-03-18T23:08:06.099 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-18T23:08:06.100 DEBUG:teuthology.orchestra.run.smithi049:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/volumes/qa/sv_1/b5d9a7f7-1d19-4c17-b495-a118d7fb5c7e /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,nowsync 2024-03-18T23:08:06.188 INFO:teuthology.orchestra.run.smithi049.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,nowsync 2024-03-18T23:08:06.188 INFO:teuthology.orchestra.run.smithi049.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,nowsync". 2024-03-18T23:08:06.188 INFO:teuthology.orchestra.run.smithi049.stdout:invalid new device string format 2024-03-18T23:08:06.296 INFO:teuthology.orchestra.run.smithi049.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,nowsync 2024-03-18T23:08:06.297 INFO:teuthology.orchestra.run.smithi049.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,nowsync". 2024-03-18T23:08:06.297 INFO:teuthology.orchestra.run.smithi049.stdout:invalid new device string format 2024-03-18T23:08:06.297 INFO:teuthology.orchestra.run.smithi049.stdout:mount.ceph: resolved to: "172.21.15.49:3300,172.21.15.139:3300,172.21.15.175:3300" 2024-03-18T23:08:06.297 INFO:teuthology.orchestra.run.smithi049.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.49:3300,172.21.15.139:3300,172.21.15.175:3300:/volumes/qa/sv_1/b5d9a7f7-1d19-4c17-b495-a118d7fb5c7e 2024-03-18T23:08:06.297 INFO:teuthology.orchestra.run.smithi049.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,nowsync,key=0,fsid=c63591ea-e57a-11ee-95c9-87774f69a715" will pass to kernel 2024-03-18T23:08:06.298 INFO:teuthology.orchestra.run.smithi049.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-03-18T23:08:06.298 INFO:teuthology.orchestra.run.smithi049.stdout: You just mounted a file system that supports labels which does not 2024-03-18T23:08:06.298 INFO:teuthology.orchestra.run.smithi049.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-03-18T23:08:06.298 INFO:teuthology.orchestra.run.smithi049.stdout: applications will generate AVC messages and not be allowed access to 2024-03-18T23:08:06.298 INFO:teuthology.orchestra.run.smithi049.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-03-18T23:08:06.299 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-03-18T23:08:06.300 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-18T23:08:06.300 DEBUG:teuthology.orchestra.run.smithi049:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-03-18T23:08:06.335 DEBUG:teuthology.orchestra.run.smithi049:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-18T23:08:06.426 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:06 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-03-18T23:08:06.428 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-18T23:08:06.434 INFO:teuthology.orchestra.run.smithi049.stdout:client25157 2024-03-18T23:08:06.434 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-03-18T23:08:06.434 DEBUG:teuthology.orchestra.run.smithi049:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-18T23:08:06.451 DEBUG:teuthology.orchestra.run.smithi049:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-18T23:08:06.541 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:06 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-03-18T23:08:06.542 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-18T23:08:06.548 INFO:teuthology.orchestra.run.smithi049.stdout:client25157 2024-03-18T23:08:06.549 DEBUG:teuthology.orchestra.run.smithi049:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-18T23:08:06.598 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:06 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-03-18T23:08:06.600 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-18T23:08:06.606 INFO:teuthology.orchestra.run.smithi049.stdout:client25157 2024-03-18T23:08:06.606 DEBUG:teuthology.orchestra.run.smithi049:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/c63591ea-e57a-11ee-95c9-87774f69a715.client25157/status') 2024-03-18T23:08:06.655 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:06 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-18T23:08:06.671 INFO:teuthology.orchestra.run.smithi049.stdout:instance: client.25157 (3)172.21.15.49:0/2672657164 2024-03-18T23:08:06.671 INFO:teuthology.orchestra.run.smithi049.stdout:blocklisted: false 2024-03-18T23:08:06.672 INFO:teuthology.orchestra.run.smithi049.stderr:0+1 records in 2024-03-18T23:08:06.672 INFO:teuthology.orchestra.run.smithi049.stderr:0+1 records out 2024-03-18T23:08:06.672 INFO:teuthology.orchestra.run.smithi049.stderr:71 bytes copied, 2.2524e-05 s, 3.2 MB/s 2024-03-18T23:08:06.672 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-18T23:08:06.679 INFO:teuthology.run_tasks:Running task exec... 2024-03-18T23:08:06.690 INFO:teuthology.task.exec:Executing custom commands... 2024-03-18T23:08:06.690 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi049.front.sepia.ceph.com 2024-03-18T23:08:06.690 DEBUG:teuthology.orchestra.run.smithi049:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate false' 2024-03-18T23:08:06.801 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:06.799+0000 7f75ee5d4640 1 -- 172.21.15.49:0/487905196 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f75e8107570 msgr2=0x7f75e81099c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:06.802 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:06.799+0000 7f75ee5d4640 1 --2- 172.21.15.49:0/487905196 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f75e8107570 0x7f75e81099c0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f75e0009ca0 tx=0x7f75e002c0f0 comp rx=0 tx=0).stop 2024-03-18T23:08:06.802 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:06.799+0000 7f75ee5d4640 1 -- 172.21.15.49:0/487905196 shutdown_connections 2024-03-18T23:08:06.802 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:06.799+0000 7f75ee5d4640 1 --2- 172.21.15.49:0/487905196 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f75e8107570 0x7f75e81099c0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:06.802 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:06.799+0000 7f75ee5d4640 1 --2- 172.21.15.49:0/487905196 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f75e8104c40 0x7f75e8107030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:06.802 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:06.799+0000 7f75ee5d4640 1 --2- 172.21.15.49:0/487905196 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f75e8102310 0x7f75e8104700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:06.802 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:06.799+0000 7f75ee5d4640 1 -- 172.21.15.49:0/487905196 >> 172.21.15.49:0/487905196 conn(0x7f75e80fbd00 msgr2=0x7f75e80fe140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:08:06.802 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:06.799+0000 7f75ee5d4640 1 -- 172.21.15.49:0/487905196 shutdown_connections 2024-03-18T23:08:06.802 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:06.800+0000 7f75ee5d4640 1 -- 172.21.15.49:0/487905196 wait complete. 2024-03-18T23:08:06.802 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:06.800+0000 7f75ee5d4640 1 Processor -- start 2024-03-18T23:08:06.802 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:06.800+0000 7f75ee5d4640 1 -- start start 2024-03-18T23:08:06.803 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:06.800+0000 7f75ee5d4640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f75e8102310 0x7f75e8197a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:06.803 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:06.800+0000 7f75ee5d4640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f75e8104c40 0x7f75e8197fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:06.803 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:06.800+0000 7f75ee5d4640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f75e8107570 0x7f75e819f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:06.803 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:06.800+0000 7f75ee5d4640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f75e810d510 con 0x7f75e8107570 2024-03-18T23:08:06.803 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:06.800+0000 7f75ee5d4640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f75e810d390 con 0x7f75e8104c40 2024-03-18T23:08:06.803 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:06.800+0000 7f75ee5d4640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f75e810d690 con 0x7f75e8102310 2024-03-18T23:08:06.803 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:06.800+0000 7f75ecb71640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f75e8107570 0x7f75e819f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:06.803 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:06.801+0000 7f75ecb71640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f75e8107570 0x7f75e819f010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:60114/0 (socket says 172.21.15.49:60114) 2024-03-18T23:08:06.803 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:06.801+0000 7f75e7fff640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f75e8102310 0x7f75e8197a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:06.803 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:06.801+0000 7f75ecb71640 1 -- 172.21.15.49:0/672162808 learned_addr learned my addr 172.21.15.49:0/672162808 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:08:06.803 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:06.801+0000 7f75e77fe640 1 --2- 172.21.15.49:0/672162808 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f75e8104c40 0x7f75e8197fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:06.803 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:06.801+0000 7f75ecb71640 1 -- 172.21.15.49:0/672162808 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f75e8102310 msgr2=0x7f75e8197a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:06.803 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:06.801+0000 7f75ecb71640 1 --2- 172.21.15.49:0/672162808 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f75e8102310 0x7f75e8197a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:06.803 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:06.801+0000 7f75ecb71640 1 -- 172.21.15.49:0/672162808 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f75e8104c40 msgr2=0x7f75e8197fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:06.804 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:06.801+0000 7f75ecb71640 1 --2- 172.21.15.49:0/672162808 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f75e8104c40 0x7f75e8197fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:06.804 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:06.801+0000 7f75ecb71640 1 -- 172.21.15.49:0/672162808 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75e819f680 con 0x7f75e8107570 2024-03-18T23:08:06.804 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:06.801+0000 7f75ecb71640 1 --2- 172.21.15.49:0/672162808 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f75e8107570 0x7f75e819f010 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f75e0037040 tx=0x7f75e0032f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:08:06.804 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:06.801+0000 7f75e57fa640 1 -- 172.21.15.49:0/672162808 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75e0041020 con 0x7f75e8107570 2024-03-18T23:08:06.804 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:06.801+0000 7f75e57fa640 1 -- 172.21.15.49:0/672162808 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f75e0034cd0 con 0x7f75e8107570 2024-03-18T23:08:06.804 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:06.801+0000 7f75ee5d4640 1 -- 172.21.15.49:0/672162808 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75e819f910 con 0x7f75e8107570 2024-03-18T23:08:06.804 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:06.801+0000 7f75e57fa640 1 -- 172.21.15.49:0/672162808 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75e0039040 con 0x7f75e8107570 2024-03-18T23:08:06.804 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:06.801+0000 7f75ee5d4640 1 -- 172.21.15.49:0/672162808 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75e819fe50 con 0x7f75e8107570 2024-03-18T23:08:06.805 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:06.802+0000 7f75ee5d4640 1 -- 172.21.15.49:0/672162808 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75e80fd7f0 con 0x7f75e8107570 2024-03-18T23:08:06.805 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:06.803+0000 7f75e57fa640 1 -- 172.21.15.49:0/672162808 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7f75e0044020 con 0x7f75e8107570 2024-03-18T23:08:06.805 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:06.804+0000 7f75e57fa640 1 --2- 172.21.15.49:0/672162808 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f75d00a2920 0x7f75d00a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:06.805 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:06.804+0000 7f75e7fff640 1 --2- 172.21.15.49:0/672162808 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f75d00a2920 0x7f75d00a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:06.805 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:06.804+0000 7f75e57fa640 1 -- 172.21.15.49:0/672162808 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f75e00f0840 con 0x7f75e8107570 2024-03-18T23:08:06.805 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:06.805+0000 7f75e7fff640 1 --2- 172.21.15.49:0/672162808 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f75d00a2920 0x7f75d00a4de0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f75d8009bd0 tx=0x7f75d8005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:08:06.809 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:06.809+0000 7f75e57fa640 1 -- 172.21.15.49:0/672162808 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f75e004b020 con 0x7f75e8107570 2024-03-18T23:08:07.032 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:07.032+0000 7f75ee5d4640 1 -- 172.21.15.49:0/672162808 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"} v 0) -- 0x7f75e8062420 con 0x7f75e8107570 2024-03-18T23:08:07.085 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:07.085+0000 7f75e57fa640 1 -- 172.21.15.49:0/672162808 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]=0 v28) ==== 118+0+0 (secure 0 0 0) 0x7f75e00bd980 con 0x7f75e8107570 2024-03-18T23:08:07.088 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:07.087+0000 7f75ee5d4640 1 -- 172.21.15.49:0/672162808 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f75d00a2920 msgr2=0x7f75d00a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:07.088 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:07.087+0000 7f75ee5d4640 1 --2- 172.21.15.49:0/672162808 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f75d00a2920 0x7f75d00a4de0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f75d8009bd0 tx=0x7f75d8005f90 comp rx=0 tx=0).stop 2024-03-18T23:08:07.088 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:07.087+0000 7f75ee5d4640 1 -- 172.21.15.49:0/672162808 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f75e8107570 msgr2=0x7f75e819f010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:07.088 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:07.087+0000 7f75ee5d4640 1 --2- 172.21.15.49:0/672162808 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f75e8107570 0x7f75e819f010 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f75e0037040 tx=0x7f75e0032f40 comp rx=0 tx=0).stop 2024-03-18T23:08:07.088 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:07.088+0000 7f75ee5d4640 1 -- 172.21.15.49:0/672162808 shutdown_connections 2024-03-18T23:08:07.088 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:07.088+0000 7f75ee5d4640 1 --2- 172.21.15.49:0/672162808 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f75d00a2920 0x7f75d00a4de0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:07.088 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:07.088+0000 7f75ee5d4640 1 --2- 172.21.15.49:0/672162808 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f75e8107570 0x7f75e819f010 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:07.088 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:07.088+0000 7f75ee5d4640 1 --2- 172.21.15.49:0/672162808 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f75e8104c40 0x7f75e8197fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:07.088 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:07.088+0000 7f75ee5d4640 1 --2- 172.21.15.49:0/672162808 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f75e8102310 0x7f75e8197a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:07.088 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:07.088+0000 7f75ee5d4640 1 -- 172.21.15.49:0/672162808 >> 172.21.15.49:0/672162808 conn(0x7f75e80fbd00 msgr2=0x7f75e8102c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:08:07.088 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:07.088+0000 7f75ee5d4640 1 -- 172.21.15.49:0/672162808 shutdown_connections 2024-03-18T23:08:07.088 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:07.088+0000 7f75ee5d4640 1 -- 172.21.15.49:0/672162808 wait complete. 2024-03-18T23:08:07.099 DEBUG:teuthology.orchestra.run.smithi049:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs subvolumegroup pin cephfs qa random 0.10' 2024-03-18T23:08:07.212 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:07.209+0000 7fb79a0b2640 1 -- 172.21.15.49:0/1466241038 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb794101930 msgr2=0x7fb794101db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:07.212 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:07.209+0000 7fb79a0b2640 1 --2- 172.21.15.49:0/1466241038 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb794101930 0x7fb794101db0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fb77c009900 tx=0x7fb77c02bc30 comp rx=0 tx=0).stop 2024-03-18T23:08:07.212 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:07.210+0000 7fb79a0b2640 1 -- 172.21.15.49:0/1466241038 shutdown_connections 2024-03-18T23:08:07.213 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:07.210+0000 7fb79a0b2640 1 --2- 172.21.15.49:0/1466241038 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb7941022f0 0x7fb794108b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:07.213 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:07.210+0000 7fb79a0b2640 1 --2- 172.21.15.49:0/1466241038 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb794101930 0x7fb794101db0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:07.213 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:07.210+0000 7fb79a0b2640 1 --2- 172.21.15.49:0/1466241038 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb794100740 0x7fb794100b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:07.213 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:07.210+0000 7fb79a0b2640 1 -- 172.21.15.49:0/1466241038 >> 172.21.15.49:0/1466241038 conn(0x7fb7940fbcb0 msgr2=0x7fb7940fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:08:07.213 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:07.210+0000 7fb79a0b2640 1 -- 172.21.15.49:0/1466241038 shutdown_connections 2024-03-18T23:08:07.213 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:07.210+0000 7fb79a0b2640 1 -- 172.21.15.49:0/1466241038 wait complete. 2024-03-18T23:08:07.213 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:07.210+0000 7fb79a0b2640 1 Processor -- start 2024-03-18T23:08:07.213 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:07.210+0000 7fb79a0b2640 1 -- start start 2024-03-18T23:08:07.213 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:07.210+0000 7fb79a0b2640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb794100740 0x7fb794069a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:07.213 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:07.210+0000 7fb79a0b2640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb794101930 0x7fb79419c820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:07.213 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:07.210+0000 7fb79a0b2640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb7941022f0 0x7fb79419ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:07.213 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:07.210+0000 7fb79a0b2640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fb79410c3c0 con 0x7fb794101930 2024-03-18T23:08:07.213 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:07.210+0000 7fb79a0b2640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fb79410c240 con 0x7fb7941022f0 2024-03-18T23:08:07.214 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:07.211+0000 7fb79a0b2640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb79410c540 con 0x7fb794100740 2024-03-18T23:08:07.214 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:07.211+0000 7fb792ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb794101930 0x7fb79419c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:07.214 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:07.211+0000 7fb792ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb794101930 0x7fb79419c820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:60128/0 (socket says 172.21.15.49:60128) 2024-03-18T23:08:07.214 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:07.211+0000 7fb792ffd640 1 -- 172.21.15.49:0/4292837009 learned_addr learned my addr 172.21.15.49:0/4292837009 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:08:07.214 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:07.211+0000 7fb7937fe640 1 --2- 172.21.15.49:0/4292837009 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb794100740 0x7fb794069a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:07.214 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:07.211+0000 7fb793fff640 1 --2- 172.21.15.49:0/4292837009 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb7941022f0 0x7fb79419ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:07.214 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:07.211+0000 7fb792ffd640 1 -- 172.21.15.49:0/4292837009 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb794100740 msgr2=0x7fb794069a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:07.214 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:07.211+0000 7fb792ffd640 1 --2- 172.21.15.49:0/4292837009 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb794100740 0x7fb794069a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:07.214 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:07.211+0000 7fb792ffd640 1 -- 172.21.15.49:0/4292837009 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb7941022f0 msgr2=0x7fb79419ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:07.214 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:07.211+0000 7fb792ffd640 1 --2- 172.21.15.49:0/4292837009 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb7941022f0 0x7fb79419ee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:07.214 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:07.211+0000 7fb792ffd640 1 -- 172.21.15.49:0/4292837009 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb79419f750 con 0x7fb794101930 2024-03-18T23:08:07.214 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:07.211+0000 7fb792ffd640 1 --2- 172.21.15.49:0/4292837009 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb794101930 0x7fb79419c820 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fb77c009230 tx=0x7fb77c02c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:08:07.214 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:07.211+0000 7fb790ff9640 1 -- 172.21.15.49:0/4292837009 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb77c041070 con 0x7fb794101930 2024-03-18T23:08:07.214 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:07.211+0000 7fb790ff9640 1 -- 172.21.15.49:0/4292837009 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb77c004450 con 0x7fb794101930 2024-03-18T23:08:07.215 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:07.211+0000 7fb790ff9640 1 -- 172.21.15.49:0/4292837009 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb77c003e20 con 0x7fb794101930 2024-03-18T23:08:07.215 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:07.211+0000 7fb79a0b2640 1 -- 172.21.15.49:0/4292837009 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb79419f980 con 0x7fb794101930 2024-03-18T23:08:07.215 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:07.211+0000 7fb79a0b2640 1 -- 172.21.15.49:0/4292837009 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb79419fe70 con 0x7fb794101930 2024-03-18T23:08:07.215 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:07.212+0000 7fb79a0b2640 1 -- 172.21.15.49:0/4292837009 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb794107a60 con 0x7fb794101930 2024-03-18T23:08:07.215 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:07.213+0000 7fb790ff9640 1 -- 172.21.15.49:0/4292837009 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7fb77c044070 con 0x7fb794101930 2024-03-18T23:08:07.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:07.214+0000 7fb790ff9640 1 --2- 172.21.15.49:0/4292837009 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fb7800a28a0 0x7fb7800a4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:07.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:07.214+0000 7fb7937fe640 1 --2- 172.21.15.49:0/4292837009 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fb7800a28a0 0x7fb7800a4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:07.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:07.214+0000 7fb790ff9640 1 -- 172.21.15.49:0/4292837009 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7fb77c0f05d0 con 0x7fb794101930 2024-03-18T23:08:07.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:07.214+0000 7fb7937fe640 1 --2- 172.21.15.49:0/4292837009 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fb7800a28a0 0x7fb7800a4d60 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fb794101790 tx=0x7fb784009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:08:07.218 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:07.219+0000 7fb790ff9640 1 -- 172.21.15.49:0/4292837009 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fb77c0bd030 con 0x7fb794101930 2024-03-18T23:08:07.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:08:07 smithi049 ceph-mon[62991]: pgmap v351: 97 pgs: 97 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-03-18T23:08:07.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:08:07 smithi049 ceph-mon[62991]: from='client.? 172.21.15.49:0/672162808' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-03-18T23:08:07.431 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:07.431+0000 7fb79a0b2640 1 -- 172.21.15.49:0/4292837009 --> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}) -- 0x7fb794061dc0 con 0x7fb7800a28a0 2024-03-18T23:08:07.445 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:07.445+0000 7fb790ff9640 1 -- 172.21.15.49:0/4292837009 <== mgr.14150 v2:172.21.15.49:6800/2083805211 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3 (secure 0 0 0) 0x7fb794061dc0 con 0x7fb7800a28a0 2024-03-18T23:08:07.445 INFO:teuthology.orchestra.run.smithi049.stdout:{} 2024-03-18T23:08:07.447 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:07.447+0000 7fb79a0b2640 1 -- 172.21.15.49:0/4292837009 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fb7800a28a0 msgr2=0x7fb7800a4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:07.447 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:07.447+0000 7fb79a0b2640 1 --2- 172.21.15.49:0/4292837009 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fb7800a28a0 0x7fb7800a4d60 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fb794101790 tx=0x7fb784009500 comp rx=0 tx=0).stop 2024-03-18T23:08:07.447 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:07.447+0000 7fb79a0b2640 1 -- 172.21.15.49:0/4292837009 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb794101930 msgr2=0x7fb79419c820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:07.447 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:07.447+0000 7fb79a0b2640 1 --2- 172.21.15.49:0/4292837009 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb794101930 0x7fb79419c820 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fb77c009230 tx=0x7fb77c02c650 comp rx=0 tx=0).stop 2024-03-18T23:08:07.447 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:07.447+0000 7fb79a0b2640 1 -- 172.21.15.49:0/4292837009 shutdown_connections 2024-03-18T23:08:07.447 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:07.447+0000 7fb79a0b2640 1 --2- 172.21.15.49:0/4292837009 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7fb7800a28a0 0x7fb7800a4d60 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:07.447 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:07.447+0000 7fb79a0b2640 1 --2- 172.21.15.49:0/4292837009 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb7941022f0 0x7fb79419ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:07.447 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:07.447+0000 7fb79a0b2640 1 --2- 172.21.15.49:0/4292837009 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb794101930 0x7fb79419c820 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:07.447 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:07.447+0000 7fb79a0b2640 1 --2- 172.21.15.49:0/4292837009 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb794100740 0x7fb794069a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:07.448 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:07.447+0000 7fb79a0b2640 1 -- 172.21.15.49:0/4292837009 >> 172.21.15.49:0/4292837009 conn(0x7fb7940fbcb0 msgr2=0x7fb794104c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:08:07.448 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:07.447+0000 7fb79a0b2640 1 -- 172.21.15.49:0/4292837009 shutdown_connections 2024-03-18T23:08:07.448 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:07.447+0000 7fb79a0b2640 1 -- 172.21.15.49:0/4292837009 wait complete. 2024-03-18T23:08:07.458 INFO:teuthology.run_tasks:Running task check-counter... 2024-03-18T23:08:07.469 INFO:teuthology.run_tasks:Running task exec... 2024-03-18T23:08:07.502 INFO:teuthology.task.exec:Executing custom commands... 2024-03-18T23:08:07.502 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi049.front.sepia.ceph.com 2024-03-18T23:08:07.502 DEBUG:teuthology.orchestra.run.smithi049:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph mgr module enable snap_schedule' 2024-03-18T23:08:07.503 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:08:07 smithi175 ceph-mon[67352]: pgmap v351: 97 pgs: 97 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-03-18T23:08:07.503 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:08:07 smithi175 ceph-mon[67352]: from='client.? 172.21.15.49:0/672162808' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-03-18T23:08:07.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:08:07 smithi139 ceph-mon[67282]: pgmap v351: 97 pgs: 97 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-03-18T23:08:07.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:08:07 smithi139 ceph-mon[67282]: from='client.? 172.21.15.49:0/672162808' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-03-18T23:08:07.613 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:07.611+0000 7f401f47e640 1 -- 172.21.15.49:0/1701094758 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4018107570 msgr2=0x7f40181099c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:07.613 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:07.611+0000 7f401f47e640 1 --2- 172.21.15.49:0/1701094758 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4018107570 0x7f40181099c0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f4008009ca0 tx=0x7f400802c0f0 comp rx=0 tx=0).stop 2024-03-18T23:08:07.614 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:07.611+0000 7f401f47e640 1 -- 172.21.15.49:0/1701094758 shutdown_connections 2024-03-18T23:08:07.614 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:07.611+0000 7f401f47e640 1 --2- 172.21.15.49:0/1701094758 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4018107570 0x7f40181099c0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:07.614 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:07.611+0000 7f401f47e640 1 --2- 172.21.15.49:0/1701094758 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4018104c40 0x7f4018107030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:07.614 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:07.611+0000 7f401f47e640 1 --2- 172.21.15.49:0/1701094758 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4018102310 0x7f4018104700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:07.614 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:07.611+0000 7f401f47e640 1 -- 172.21.15.49:0/1701094758 >> 172.21.15.49:0/1701094758 conn(0x7f40180fbd00 msgr2=0x7f40180fe140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:08:07.614 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:07.611+0000 7f401f47e640 1 -- 172.21.15.49:0/1701094758 shutdown_connections 2024-03-18T23:08:07.614 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:07.611+0000 7f401f47e640 1 -- 172.21.15.49:0/1701094758 wait complete. 2024-03-18T23:08:07.614 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:07.612+0000 7f401f47e640 1 Processor -- start 2024-03-18T23:08:07.614 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:07.612+0000 7f401f47e640 1 -- start start 2024-03-18T23:08:07.614 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:07.612+0000 7f401f47e640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4018102310 0x7f4018197a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:07.614 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:07.612+0000 7f401f47e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4018104c40 0x7f4018197f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:07.614 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:07.612+0000 7f401f47e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4018107570 0x7f401819eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:07.615 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:07.612+0000 7f401f47e640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f401810d510 con 0x7f4018107570 2024-03-18T23:08:07.615 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:07.612+0000 7f401f47e640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f401810d390 con 0x7f4018104c40 2024-03-18T23:08:07.615 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:07.612+0000 7f401f47e640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f401810d690 con 0x7f4018102310 2024-03-18T23:08:07.615 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:07.612+0000 7f401da1b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4018107570 0x7f401819eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:07.615 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:07.612+0000 7f401da1b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4018107570 0x7f401819eff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:60164/0 (socket says 172.21.15.49:60164) 2024-03-18T23:08:07.615 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:07.612+0000 7f401da1b640 1 -- 172.21.15.49:0/2462774750 learned_addr learned my addr 172.21.15.49:0/2462774750 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:08:07.615 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:07.612+0000 7f401d21a640 1 --2- 172.21.15.49:0/2462774750 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4018102310 0x7f4018197a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:07.615 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:07.612+0000 7f401ca19640 1 --2- 172.21.15.49:0/2462774750 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4018104c40 0x7f4018197f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:07.615 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:07.612+0000 7f401da1b640 1 -- 172.21.15.49:0/2462774750 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4018102310 msgr2=0x7f4018197a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:07.615 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:07.612+0000 7f401da1b640 1 --2- 172.21.15.49:0/2462774750 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4018102310 0x7f4018197a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:07.615 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:07.612+0000 7f401da1b640 1 -- 172.21.15.49:0/2462774750 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4018104c40 msgr2=0x7f4018197f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:07.615 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:07.612+0000 7f401da1b640 1 --2- 172.21.15.49:0/2462774750 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4018104c40 0x7f4018197f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:07.615 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:07.612+0000 7f401da1b640 1 -- 172.21.15.49:0/2462774750 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f401819f660 con 0x7f4018107570 2024-03-18T23:08:07.615 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:07.612+0000 7f401da1b640 1 --2- 172.21.15.49:0/2462774750 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4018107570 0x7f401819eff0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f4008037040 tx=0x7f4008032f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:08:07.616 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:07.612+0000 7f40167fc640 1 -- 172.21.15.49:0/2462774750 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4008041020 con 0x7f4018107570 2024-03-18T23:08:07.616 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:07.612+0000 7f40167fc640 1 -- 172.21.15.49:0/2462774750 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4008034cd0 con 0x7f4018107570 2024-03-18T23:08:07.616 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:07.612+0000 7f401f47e640 1 -- 172.21.15.49:0/2462774750 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f401819f8f0 con 0x7f4018107570 2024-03-18T23:08:07.616 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:07.613+0000 7f40167fc640 1 -- 172.21.15.49:0/2462774750 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4008039040 con 0x7f4018107570 2024-03-18T23:08:07.616 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:07.613+0000 7f401f47e640 1 -- 172.21.15.49:0/2462774750 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f401819fe30 con 0x7f4018107570 2024-03-18T23:08:07.616 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:07.613+0000 7f401f47e640 1 -- 172.21.15.49:0/2462774750 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f40180fd7f0 con 0x7f4018107570 2024-03-18T23:08:07.617 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:07.615+0000 7f40167fc640 1 -- 172.21.15.49:0/2462774750 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7f4008044020 con 0x7f4018107570 2024-03-18T23:08:07.617 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:07.615+0000 7f40167fc640 1 --2- 172.21.15.49:0/2462774750 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f40000a2920 0x7f40000a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:07.617 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:07.615+0000 7f401d21a640 1 --2- 172.21.15.49:0/2462774750 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f40000a2920 0x7f40000a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:07.617 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:07.615+0000 7f40167fc640 1 -- 172.21.15.49:0/2462774750 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f40080f07f0 con 0x7f4018107570 2024-03-18T23:08:07.617 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:07.616+0000 7f401d21a640 1 --2- 172.21.15.49:0/2462774750 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f40000a2920 0x7f40000a4de0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f4010009bd0 tx=0x7f4010005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:08:07.619 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:07.620+0000 7f40167fc640 1 -- 172.21.15.49:0/2462774750 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f400804b020 con 0x7f4018107570 2024-03-18T23:08:07.854 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:07.854+0000 7f401f47e640 1 -- 172.21.15.49:0/2462774750 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) -- 0x7f4018062420 con 0x7f4018107570 2024-03-18T23:08:07.880 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 18 23:08:07 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-x[63224]: 2024-03-18T23:08:07.436+0000 7f1853516700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-18T23:08:07.880 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 18 23:08:07 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-x[63224]: 2024-03-18T23:08:07.436+0000 7f1853516700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-18T23:08:07.880 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 18 23:08:07 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-x[63224]: 2024-03-18T23:08:07.436+0000 7f1853516700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-18T23:08:07.880 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 18 23:08:07 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-x[63224]: 2024-03-18T23:08:07.436+0000 7f1853516700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-18T23:08:07.880 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 18 23:08:07 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-x[63224]: 2024-03-18T23:08:07.436+0000 7f1853516700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-18T23:08:08.091 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:08.091+0000 7f40167fc640 1 -- 172.21.15.49:0/2462774750 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v19) ==== 92+0+0 (secure 0 0 0) 0x7f40080bd930 con 0x7f4018107570 2024-03-18T23:08:08.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:08.094+0000 7f401f47e640 1 -- 172.21.15.49:0/2462774750 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f40000a2920 msgr2=0x7f40000a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:08.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:08.094+0000 7f401f47e640 1 --2- 172.21.15.49:0/2462774750 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f40000a2920 0x7f40000a4de0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f4010009bd0 tx=0x7f4010005f90 comp rx=0 tx=0).stop 2024-03-18T23:08:08.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:08.094+0000 7f401f47e640 1 -- 172.21.15.49:0/2462774750 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4018107570 msgr2=0x7f401819eff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:08.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:08.094+0000 7f401f47e640 1 --2- 172.21.15.49:0/2462774750 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4018107570 0x7f401819eff0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f4008037040 tx=0x7f4008032f40 comp rx=0 tx=0).stop 2024-03-18T23:08:08.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:08.094+0000 7f401f47e640 1 -- 172.21.15.49:0/2462774750 shutdown_connections 2024-03-18T23:08:08.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:08.094+0000 7f401f47e640 1 --2- 172.21.15.49:0/2462774750 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f40000a2920 0x7f40000a4de0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:08.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:08.094+0000 7f401f47e640 1 --2- 172.21.15.49:0/2462774750 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4018107570 0x7f401819eff0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:08.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:08.094+0000 7f401f47e640 1 --2- 172.21.15.49:0/2462774750 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4018104c40 0x7f4018197f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:08.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:08.094+0000 7f401f47e640 1 --2- 172.21.15.49:0/2462774750 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4018102310 0x7f4018197a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:08.095 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:08.094+0000 7f401f47e640 1 -- 172.21.15.49:0/2462774750 >> 172.21.15.49:0/2462774750 conn(0x7f40180fbd00 msgr2=0x7f4018102c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:08:08.095 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:08.095+0000 7f401f47e640 1 -- 172.21.15.49:0/2462774750 shutdown_connections 2024-03-18T23:08:08.095 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:08.095+0000 7f401f47e640 1 -- 172.21.15.49:0/2462774750 wait complete. 2024-03-18T23:08:08.111 DEBUG:teuthology.orchestra.run.smithi049:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/allow_m_granularity true' 2024-03-18T23:08:08.227 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:08.225+0000 7f6215e0b640 1 -- 172.21.15.49:0/839062990 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6210101930 msgr2=0x7f6210101db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:08.227 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:08.225+0000 7f6215e0b640 1 --2- 172.21.15.49:0/839062990 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6210101930 0x7f6210101db0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f6204009900 tx=0x7f620402bc30 comp rx=0 tx=0).stop 2024-03-18T23:08:08.227 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:08.225+0000 7f6215e0b640 1 -- 172.21.15.49:0/839062990 shutdown_connections 2024-03-18T23:08:08.228 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:08.225+0000 7f6215e0b640 1 --2- 172.21.15.49:0/839062990 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f62101022f0 0x7f6210108b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:08.228 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:08.225+0000 7f6215e0b640 1 --2- 172.21.15.49:0/839062990 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6210101930 0x7f6210101db0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:08.228 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:08.225+0000 7f6215e0b640 1 --2- 172.21.15.49:0/839062990 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6210100740 0x7f6210100b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:08.228 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:08.225+0000 7f6215e0b640 1 -- 172.21.15.49:0/839062990 >> 172.21.15.49:0/839062990 conn(0x7f62100fbcb0 msgr2=0x7f62100fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:08:08.228 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:08.225+0000 7f6215e0b640 1 -- 172.21.15.49:0/839062990 shutdown_connections 2024-03-18T23:08:08.228 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:08.225+0000 7f6215e0b640 1 -- 172.21.15.49:0/839062990 wait complete. 2024-03-18T23:08:08.228 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:08.226+0000 7f6215e0b640 1 Processor -- start 2024-03-18T23:08:08.228 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:08.226+0000 7f6215e0b640 1 -- start start 2024-03-18T23:08:08.228 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:08.226+0000 7f6215e0b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6210100740 0x7f6210197b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:08.228 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:08.226+0000 7f6215e0b640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6210101930 0x7f6210198050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:08.228 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:08.226+0000 7f6215e0b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f62101022f0 0x7f621019f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:08.229 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:08.226+0000 7f6215e0b640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f621010c3a0 con 0x7f62101022f0 2024-03-18T23:08:08.229 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:08.226+0000 7f6215e0b640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f621010c220 con 0x7f6210100740 2024-03-18T23:08:08.229 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:08.226+0000 7f6215e0b640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f621010c520 con 0x7f6210101930 2024-03-18T23:08:08.229 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:08.226+0000 7f620ffff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f62101022f0 0x7f621019f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:08.229 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:08.226+0000 7f620f7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6210100740 0x7f6210197b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:08.229 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:08.226+0000 7f620ffff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f62101022f0 0x7f621019f0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:60204/0 (socket says 172.21.15.49:60204) 2024-03-18T23:08:08.229 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:08.226+0000 7f620ffff640 1 -- 172.21.15.49:0/3412822680 learned_addr learned my addr 172.21.15.49:0/3412822680 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:08:08.229 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:08.226+0000 7f620f7fe640 1 --2- 172.21.15.49:0/3412822680 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6210100740 0x7f6210197b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.49:37470/0 (socket says 172.21.15.49:37470) 2024-03-18T23:08:08.229 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:08.226+0000 7f620effd640 1 --2- 172.21.15.49:0/3412822680 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6210101930 0x7f6210198050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:08.229 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:08.227+0000 7f620f7fe640 1 -- 172.21.15.49:0/3412822680 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6210101930 msgr2=0x7f6210198050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:08.229 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:08.227+0000 7f620f7fe640 1 --2- 172.21.15.49:0/3412822680 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6210101930 0x7f6210198050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:08.229 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:08.227+0000 7f620f7fe640 1 -- 172.21.15.49:0/3412822680 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f62101022f0 msgr2=0x7f621019f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:08.230 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:08.227+0000 7f620f7fe640 1 --2- 172.21.15.49:0/3412822680 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f62101022f0 0x7f621019f0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:08.230 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:08.227+0000 7f620f7fe640 1 -- 172.21.15.49:0/3412822680 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f621019f730 con 0x7f6210100740 2024-03-18T23:08:08.230 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:08.227+0000 7f620ffff640 1 --2- 172.21.15.49:0/3412822680 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f62101022f0 0x7f621019f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:08:08.230 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:08.227+0000 7f620f7fe640 1 --2- 172.21.15.49:0/3412822680 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6210100740 0x7f6210197b10 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f620000d6a0 tx=0x7f620000db60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:08:08.230 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:08.227+0000 7f620cff9640 1 -- 172.21.15.49:0/3412822680 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6200017070 con 0x7f6210100740 2024-03-18T23:08:08.230 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:08.227+0000 7f6215e0b640 1 -- 172.21.15.49:0/3412822680 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f621019fa20 con 0x7f6210100740 2024-03-18T23:08:08.230 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:08.227+0000 7f620cff9640 1 -- 172.21.15.49:0/3412822680 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6200009d00 con 0x7f6210100740 2024-03-18T23:08:08.230 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:08.227+0000 7f620cff9640 1 -- 172.21.15.49:0/3412822680 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6200003c60 con 0x7f6210100740 2024-03-18T23:08:08.230 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:08.227+0000 7f6215e0b640 1 -- 172.21.15.49:0/3412822680 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f621019ff90 con 0x7f6210100740 2024-03-18T23:08:08.231 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:08.228+0000 7f6215e0b640 1 -- 172.21.15.49:0/3412822680 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62101a01c0 con 0x7f6210100740 2024-03-18T23:08:08.231 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:08.230+0000 7f620cff9640 1 -- 172.21.15.49:0/3412822680 <== mon.2 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 19) ==== 137307+0+0 (secure 0 0 0) 0x7f6200009820 con 0x7f6210100740 2024-03-18T23:08:08.231 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:08.230+0000 7f620cff9640 1 --2- 172.21.15.49:0/3412822680 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f61f80a28f0 0x7f61f80a4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:08.231 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:08.230+0000 7f620effd640 1 -- 172.21.15.49:0/3412822680 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f61f80a28f0 msgr2=0x7f61f80a4db0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.49:6800/2083805211 2024-03-18T23:08:08.232 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:08.230+0000 7f620cff9640 1 -- 172.21.15.49:0/3412822680 <== mon.2 v2:172.21.15.139:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f62000ce730 con 0x7f6210100740 2024-03-18T23:08:08.232 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:08.230+0000 7f620effd640 1 --2- 172.21.15.49:0/3412822680 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f61f80a28f0 0x7f61f80a4db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-18T23:08:08.234 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:08.235+0000 7f620cff9640 1 -- 172.21.15.49:0/3412822680 <== mon.2 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f620009b190 con 0x7f6210100740 2024-03-18T23:08:08.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:08:08 smithi049 ceph-mon[62991]: from='client.? 172.21.15.49:0/672162808' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-03-18T23:08:08.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:08:08 smithi049 ceph-mon[62991]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T23:08:08.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:08:08 smithi049 ceph-mon[62991]: from='client.15345 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T23:08:08.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:08:08 smithi049 ceph-mon[62991]: from='client.? 172.21.15.49:0/2462774750' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-03-18T23:08:08.381 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 18 23:08:08 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-x[63224]: ignoring --setuser ceph since I am not root 2024-03-18T23:08:08.381 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 18 23:08:08 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-x[63224]: ignoring --setgroup ceph since I am not root 2024-03-18T23:08:08.421 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:08.421+0000 7f6215e0b640 1 -- 172.21.15.49:0/3412822680 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) -- 0x7f62101a0890 con 0x7f6210100740 2024-03-18T23:08:08.430 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:08.427+0000 7f620cff9640 1 -- 172.21.15.49:0/3412822680 <== mon.2 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v29)=0 v29) ==== 138+0+0 (secure 0 0 0) 0x7f620009f230 con 0x7f6210100740 2024-03-18T23:08:08.431 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:08.429+0000 7f6215e0b640 1 -- 172.21.15.49:0/3412822680 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f61f80a28f0 msgr2=0x7f61f80a4db0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-18T23:08:08.431 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:08.429+0000 7f6215e0b640 1 --2- 172.21.15.49:0/3412822680 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f61f80a28f0 0x7f61f80a4db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:08.431 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:08.429+0000 7f6215e0b640 1 -- 172.21.15.49:0/3412822680 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6210100740 msgr2=0x7f6210197b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:08.431 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:08.429+0000 7f6215e0b640 1 --2- 172.21.15.49:0/3412822680 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6210100740 0x7f6210197b10 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f620000d6a0 tx=0x7f620000db60 comp rx=0 tx=0).stop 2024-03-18T23:08:08.431 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:08.429+0000 7f6215e0b640 1 -- 172.21.15.49:0/3412822680 shutdown_connections 2024-03-18T23:08:08.431 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:08.429+0000 7f6215e0b640 1 --2- 172.21.15.49:0/3412822680 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f61f80a28f0 0x7f61f80a4db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:08.431 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:08.429+0000 7f6215e0b640 1 --2- 172.21.15.49:0/3412822680 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f62101022f0 0x7f621019f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:08.431 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:08.429+0000 7f6215e0b640 1 --2- 172.21.15.49:0/3412822680 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6210101930 0x7f6210198050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:08.431 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:08.429+0000 7f6215e0b640 1 --2- 172.21.15.49:0/3412822680 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6210100740 0x7f6210197b10 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:08.431 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:08.429+0000 7f6215e0b640 1 -- 172.21.15.49:0/3412822680 >> 172.21.15.49:0/3412822680 conn(0x7f62100fbcb0 msgr2=0x7f6210068ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:08:08.432 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:08.429+0000 7f6215e0b640 1 -- 172.21.15.49:0/3412822680 shutdown_connections 2024-03-18T23:08:08.432 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:08.429+0000 7f6215e0b640 1 -- 172.21.15.49:0/3412822680 wait complete. 2024-03-18T23:08:08.441 DEBUG:teuthology.orchestra.run.smithi049:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/dump_on_update true' 2024-03-18T23:08:08.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:08:08 smithi175 ceph-mon[67352]: from='client.? 172.21.15.49:0/672162808' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-03-18T23:08:08.462 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:08:08 smithi175 ceph-mon[67352]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T23:08:08.462 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:08:08 smithi175 ceph-mon[67352]: from='client.15345 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T23:08:08.462 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:08:08 smithi175 ceph-mon[67352]: from='client.? 172.21.15.49:0/2462774750' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-03-18T23:08:08.462 INFO:journalctl@ceph.mgr.z.smithi175.stdout:Mar 18 23:08:08 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-z[69214]: ignoring --setuser ceph since I am not root 2024-03-18T23:08:08.462 INFO:journalctl@ceph.mgr.z.smithi175.stdout:Mar 18 23:08:08 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-z[69214]: ignoring --setgroup ceph since I am not root 2024-03-18T23:08:08.462 INFO:journalctl@ceph.mgr.z.smithi175.stdout:Mar 18 23:08:08 smithi175 ceph-mgr[69218]: -- 172.21.15.175:0/3408256811 <== mon.0 v2:172.21.15.49:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f57d9205a0 con 0x55f57d8f5000 2024-03-18T23:08:08.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:08.552+0000 7f30065b9640 1 -- 172.21.15.49:0/2950538396 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f30001022c0 msgr2=0x7f30001046b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:08.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:08.552+0000 7f30065b9640 1 --2- 172.21.15.49:0/2950538396 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f30001022c0 0x7f30001046b0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f2ff0009900 tx=0x7f2ff002bc30 comp rx=0 tx=0).stop 2024-03-18T23:08:08.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:08.552+0000 7f30065b9640 1 -- 172.21.15.49:0/2950538396 shutdown_connections 2024-03-18T23:08:08.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:08.552+0000 7f30065b9640 1 --2- 172.21.15.49:0/2950538396 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3000107520 0x7f3000109970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:08.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:08.552+0000 7f30065b9640 1 --2- 172.21.15.49:0/2950538396 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3000104bf0 0x7f3000106fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:08.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:08.552+0000 7f30065b9640 1 --2- 172.21.15.49:0/2950538396 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f30001022c0 0x7f30001046b0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:08.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:08.552+0000 7f30065b9640 1 -- 172.21.15.49:0/2950538396 >> 172.21.15.49:0/2950538396 conn(0x7f30000fbcb0 msgr2=0x7f30000fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:08:08.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:08.552+0000 7f30065b9640 1 -- 172.21.15.49:0/2950538396 shutdown_connections 2024-03-18T23:08:08.554 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:08.552+0000 7f30065b9640 1 -- 172.21.15.49:0/2950538396 wait complete. 2024-03-18T23:08:08.554 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:08.553+0000 7f30065b9640 1 Processor -- start 2024-03-18T23:08:08.554 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:08.553+0000 7f30065b9640 1 -- start start 2024-03-18T23:08:08.557 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:08.553+0000 7f30065b9640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f30001022c0 0x7f3000197b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:08.557 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:08.553+0000 7f30065b9640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3000104bf0 0x7f3000198040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:08.557 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:08.553+0000 7f30065b9640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3000107520 0x7f300019f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:08.557 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:08.553+0000 7f30065b9640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f300010d4f0 con 0x7f3000107520 2024-03-18T23:08:08.557 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:08.553+0000 7f30065b9640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f300010d370 con 0x7f30001022c0 2024-03-18T23:08:08.557 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:08.553+0000 7f30065b9640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f300010d670 con 0x7f3000104bf0 2024-03-18T23:08:08.558 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:08.553+0000 7f3004b56640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3000107520 0x7f300019f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:08.558 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:08.553+0000 7f2ffffff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f30001022c0 0x7f3000197b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:08.558 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:08.553+0000 7f3004b56640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3000107520 0x7f300019f0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:60224/0 (socket says 172.21.15.49:60224) 2024-03-18T23:08:08.558 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:08.553+0000 7f3004b56640 1 -- 172.21.15.49:0/148518429 learned_addr learned my addr 172.21.15.49:0/148518429 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:08:08.558 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:08.553+0000 7f2fff7fe640 1 --2- 172.21.15.49:0/148518429 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3000104bf0 0x7f3000198040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:08.558 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:08.554+0000 7f3004b56640 1 -- 172.21.15.49:0/148518429 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3000104bf0 msgr2=0x7f3000198040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:08.558 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:08.554+0000 7f3004b56640 1 --2- 172.21.15.49:0/148518429 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3000104bf0 0x7f3000198040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:08.558 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:08.554+0000 7f3004b56640 1 -- 172.21.15.49:0/148518429 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f30001022c0 msgr2=0x7f3000197b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:08.558 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:08.554+0000 7f3004b56640 1 --2- 172.21.15.49:0/148518429 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f30001022c0 0x7f3000197b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:08.558 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:08.554+0000 7f3004b56640 1 -- 172.21.15.49:0/148518429 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f300019f720 con 0x7f3000107520 2024-03-18T23:08:08.559 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:08.554+0000 7f2ffffff640 1 --2- 172.21.15.49:0/148518429 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f30001022c0 0x7f3000197b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:08:08.559 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:08.554+0000 7f2fff7fe640 1 --2- 172.21.15.49:0/148518429 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3000104bf0 0x7f3000198040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:08:08.559 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:08.554+0000 7f3004b56640 1 --2- 172.21.15.49:0/148518429 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3000107520 0x7f300019f0b0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f2ff800eb10 tx=0x7f2ff800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:08:08.559 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:08.554+0000 7f2ffd7fa640 1 -- 172.21.15.49:0/148518429 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ff8004b20 con 0x7f3000107520 2024-03-18T23:08:08.559 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:08.554+0000 7f30065b9640 1 -- 172.21.15.49:0/148518429 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f300019fa10 con 0x7f3000107520 2024-03-18T23:08:08.559 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:08.554+0000 7f2ffd7fa640 1 -- 172.21.15.49:0/148518429 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ff8004cc0 con 0x7f3000107520 2024-03-18T23:08:08.559 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:08.554+0000 7f30065b9640 1 -- 172.21.15.49:0/148518429 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f300019ff60 con 0x7f3000107520 2024-03-18T23:08:08.559 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:08.554+0000 7f2ffd7fa640 1 -- 172.21.15.49:0/148518429 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ff8010760 con 0x7f3000107520 2024-03-18T23:08:08.559 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:08.555+0000 7f30065b9640 1 -- 172.21.15.49:0/148518429 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30000fd7a0 con 0x7f3000107520 2024-03-18T23:08:08.559 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:08.557+0000 7f2ffd7fa640 1 -- 172.21.15.49:0/148518429 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 19) ==== 137307+0+0 (secure 0 0 0) 0x7f2ff8005020 con 0x7f3000107520 2024-03-18T23:08:08.559 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:08.557+0000 7f2ffd7fa640 1 --2- 172.21.15.49:0/148518429 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f2fe80a28f0 0x7f2fe80a4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:08.559 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:08.557+0000 7f2ffffff640 1 -- 172.21.15.49:0/148518429 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f2fe80a28f0 msgr2=0x7f2fe80a4db0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.49:6800/2083805211 2024-03-18T23:08:08.559 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:08.557+0000 7f2ffd7fa640 1 -- 172.21.15.49:0/148518429 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f2ff80cf010 con 0x7f3000107520 2024-03-18T23:08:08.559 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:08.557+0000 7f2ffffff640 1 --2- 172.21.15.49:0/148518429 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f2fe80a28f0 0x7f2fe80a4db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-18T23:08:08.561 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:08.562+0000 7f2ffd7fa640 1 -- 172.21.15.49:0/148518429 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f2ff809c0c0 con 0x7f3000107520 2024-03-18T23:08:08.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:08:08 smithi139 ceph-mon[67282]: from='client.? 172.21.15.49:0/672162808' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-03-18T23:08:08.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:08:08 smithi139 ceph-mon[67282]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T23:08:08.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:08:08 smithi139 ceph-mon[67282]: from='client.15345 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T23:08:08.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:08:08 smithi139 ceph-mon[67282]: from='client.? 172.21.15.49:0/2462774750' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-03-18T23:08:08.565 INFO:journalctl@ceph.mgr.y.smithi139.stdout:Mar 18 23:08:08 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-y[68685]: ignoring --setuser ceph since I am not root 2024-03-18T23:08:08.566 INFO:journalctl@ceph.mgr.y.smithi139.stdout:Mar 18 23:08:08 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-y[68685]: ignoring --setgroup ceph since I am not root 2024-03-18T23:08:08.566 INFO:journalctl@ceph.mgr.y.smithi139.stdout:Mar 18 23:08:08 smithi139 ceph-mgr[68689]: -- 172.21.15.139:0/816483673 <== mon.2 v2:172.21.15.139:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x564dbe7a45a0 con 0x564dbe778c00 2024-03-18T23:08:08.747 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:08.747+0000 7f30065b9640 1 -- 172.21.15.49:0/148518429 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) -- 0x7f3000062490 con 0x7f3000107520 2024-03-18T23:08:08.750 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:08.750+0000 7f2ffd7fa640 1 -- 172.21.15.49:0/148518429 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v30)=0 v30) ==== 133+0+0 (secure 0 0 0) 0x7f2ff809b9c0 con 0x7f3000107520 2024-03-18T23:08:08.753 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:08.753+0000 7f30065b9640 1 -- 172.21.15.49:0/148518429 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f2fe80a28f0 msgr2=0x7f2fe80a4db0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-18T23:08:08.753 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:08.753+0000 7f30065b9640 1 --2- 172.21.15.49:0/148518429 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f2fe80a28f0 0x7f2fe80a4db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:08.753 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:08.753+0000 7f30065b9640 1 -- 172.21.15.49:0/148518429 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3000107520 msgr2=0x7f300019f0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:08.753 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:08.753+0000 7f30065b9640 1 --2- 172.21.15.49:0/148518429 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3000107520 0x7f300019f0b0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f2ff800eb10 tx=0x7f2ff800c2d0 comp rx=0 tx=0).stop 2024-03-18T23:08:08.753 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:08.753+0000 7f30065b9640 1 -- 172.21.15.49:0/148518429 shutdown_connections 2024-03-18T23:08:08.753 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:08.753+0000 7f30065b9640 1 --2- 172.21.15.49:0/148518429 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f2fe80a28f0 0x7f2fe80a4db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:08.753 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:08.753+0000 7f30065b9640 1 --2- 172.21.15.49:0/148518429 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3000107520 0x7f300019f0b0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:08.753 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:08.753+0000 7f30065b9640 1 --2- 172.21.15.49:0/148518429 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3000104bf0 0x7f3000198040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:08.754 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:08.753+0000 7f30065b9640 1 --2- 172.21.15.49:0/148518429 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f30001022c0 0x7f3000197b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:08.754 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:08.753+0000 7f30065b9640 1 -- 172.21.15.49:0/148518429 >> 172.21.15.49:0/148518429 conn(0x7f30000fbcb0 msgr2=0x7f3000102c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:08:08.754 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:08.753+0000 7f30065b9640 1 -- 172.21.15.49:0/148518429 shutdown_connections 2024-03-18T23:08:08.754 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:08.753+0000 7f30065b9640 1 -- 172.21.15.49:0/148518429 wait complete. 2024-03-18T23:08:08.764 DEBUG:teuthology.orchestra.run.smithi049:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m' 2024-03-18T23:08:08.879 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:08.877+0000 7f691e39d640 1 -- 172.21.15.49:0/1762316302 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6918100570 msgr2=0x7f6918100990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:08.879 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:08.877+0000 7f691e39d640 1 --2- 172.21.15.49:0/1762316302 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6918100570 0x7f6918100990 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f6914009900 tx=0x7f691402bc30 comp rx=0 tx=0).stop 2024-03-18T23:08:08.879 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:08.877+0000 7f691e39d640 1 -- 172.21.15.49:0/1762316302 shutdown_connections 2024-03-18T23:08:08.879 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:08.877+0000 7f691e39d640 1 --2- 172.21.15.49:0/1762316302 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6918102120 0x7f69181089b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:08.879 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:08.877+0000 7f691e39d640 1 --2- 172.21.15.49:0/1762316302 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6918101760 0x7f6918101be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:08.879 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:08.877+0000 7f691e39d640 1 --2- 172.21.15.49:0/1762316302 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6918100570 0x7f6918100990 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:08.880 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:08.877+0000 7f691e39d640 1 -- 172.21.15.49:0/1762316302 >> 172.21.15.49:0/1762316302 conn(0x7f69180fbb00 msgr2=0x7f69180fdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:08:08.880 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:08.877+0000 7f691e39d640 1 -- 172.21.15.49:0/1762316302 shutdown_connections 2024-03-18T23:08:08.880 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:08.877+0000 7f691e39d640 1 -- 172.21.15.49:0/1762316302 wait complete. 2024-03-18T23:08:08.880 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:08.877+0000 7f691e39d640 1 Processor -- start 2024-03-18T23:08:08.880 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:08.877+0000 7f691e39d640 1 -- start start 2024-03-18T23:08:08.880 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:08.878+0000 7f691e39d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6918100570 0x7f6918069a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:08.880 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:08.878+0000 7f691e39d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6918101760 0x7f691819eae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:08.880 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:08.878+0000 7f691e39d640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6918102120 0x7f69181a1030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:08.880 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:08.878+0000 7f691e39d640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f691810c1d0 con 0x7f6918101760 2024-03-18T23:08:08.880 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:08.878+0000 7f691e39d640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f691810c050 con 0x7f6918100570 2024-03-18T23:08:08.880 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:08.878+0000 7f691e39d640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f691810c350 con 0x7f6918102120 2024-03-18T23:08:08.880 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:08.878+0000 7f691cb9a640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6918101760 0x7f691819eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:08.880 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:08.878+0000 7f691cb9a640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6918101760 0x7f691819eae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:60248/0 (socket says 172.21.15.49:60248) 2024-03-18T23:08:08.880 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:08.878+0000 7f691cb9a640 1 -- 172.21.15.49:0/1869622753 learned_addr learned my addr 172.21.15.49:0/1869622753 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:08:08.881 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:08.878+0000 7f691d39b640 1 --2- 172.21.15.49:0/1869622753 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6918100570 0x7f6918069a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:08.881 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:08.878+0000 7f691db9c640 1 --2- 172.21.15.49:0/1869622753 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6918102120 0x7f69181a1030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:08.881 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:08.878+0000 7f691cb9a640 1 -- 172.21.15.49:0/1869622753 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6918102120 msgr2=0x7f69181a1030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:08.881 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:08.878+0000 7f691cb9a640 1 --2- 172.21.15.49:0/1869622753 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6918102120 0x7f69181a1030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:08.881 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:08.878+0000 7f691cb9a640 1 -- 172.21.15.49:0/1869622753 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6918100570 msgr2=0x7f6918069a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:08.881 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:08.878+0000 7f691cb9a640 1 --2- 172.21.15.49:0/1869622753 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6918100570 0x7f6918069a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:08.881 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:08.878+0000 7f691cb9a640 1 -- 172.21.15.49:0/1869622753 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69181a1820 con 0x7f6918101760 2024-03-18T23:08:08.881 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:08.878+0000 7f691cb9a640 1 --2- 172.21.15.49:0/1869622753 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6918101760 0x7f691819eae0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f690800e9f0 tx=0x7f690800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:08:08.881 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:08.878+0000 7f690e7fc640 1 -- 172.21.15.49:0/1869622753 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f690800cd80 con 0x7f6918101760 2024-03-18T23:08:08.881 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:08.878+0000 7f690e7fc640 1 -- 172.21.15.49:0/1869622753 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6908010910 con 0x7f6918101760 2024-03-18T23:08:08.881 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:08.878+0000 7f690e7fc640 1 -- 172.21.15.49:0/1869622753 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6908018990 con 0x7f6918101760 2024-03-18T23:08:08.881 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:08.878+0000 7f691e39d640 1 -- 172.21.15.49:0/1869622753 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69181a1ab0 con 0x7f6918101760 2024-03-18T23:08:08.881 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:08.878+0000 7f691e39d640 1 -- 172.21.15.49:0/1869622753 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69181a1fb0 con 0x7f6918101760 2024-03-18T23:08:08.882 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:08.879+0000 7f691e39d640 1 -- 172.21.15.49:0/1869622753 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6918107960 con 0x7f6918101760 2024-03-18T23:08:08.882 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:08.881+0000 7f690e7fc640 1 -- 172.21.15.49:0/1869622753 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 19) ==== 137307+0+0 (secure 0 0 0) 0x7f6908010ac0 con 0x7f6918101760 2024-03-18T23:08:08.882 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:08.881+0000 7f690e7fc640 1 --2- 172.21.15.49:0/1869622753 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f69040a28f0 0x7f69040a4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:08.883 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:08.881+0000 7f691d39b640 1 -- 172.21.15.49:0/1869622753 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f69040a28f0 msgr2=0x7f69040a4db0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.49:6800/2083805211 2024-03-18T23:08:08.883 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:08.882+0000 7f690e7fc640 1 -- 172.21.15.49:0/1869622753 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f69080cf0f0 con 0x7f6918101760 2024-03-18T23:08:08.883 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:08.882+0000 7f691d39b640 1 --2- 172.21.15.49:0/1869622753 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f69040a28f0 0x7f69040a4db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-18T23:08:08.889 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:08.889+0000 7f690e7fc640 1 -- 172.21.15.49:0/1869622753 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f690809bb50 con 0x7f6918101760 2024-03-18T23:08:09.065 INFO:journalctl@ceph.mgr.y.smithi139.stdout:Mar 18 23:08:08 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-y[68685]: 2024-03-18T23:08:08.739+0000 7f4235a6b200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-18T23:08:09.065 INFO:journalctl@ceph.mgr.y.smithi139.stdout:Mar 18 23:08:08 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-y[68685]: 2024-03-18T23:08:08.852+0000 7f4235a6b200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-18T23:08:09.082 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:09.083+0000 7f691d39b640 1 -- 172.21.15.49:0/1869622753 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f69040a28f0 msgr2=0x7f69040a4db0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.49:6800/2083805211 2024-03-18T23:08:09.083 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:09.083+0000 7f691d39b640 1 --2- 172.21.15.49:0/1869622753 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f69040a28f0 0x7f69040a4db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-18T23:08:09.092 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 18 23:08:08 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-x[63224]: 2024-03-18T23:08:08.699+0000 7f2e234bd200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-18T23:08:09.092 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 18 23:08:08 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-x[63224]: 2024-03-18T23:08:08.800+0000 7f2e234bd200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-18T23:08:09.094 INFO:journalctl@ceph.mgr.z.smithi175.stdout:Mar 18 23:08:08 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-z[69214]: 2024-03-18T23:08:08.732+0000 7fd447dba200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-18T23:08:09.094 INFO:journalctl@ceph.mgr.z.smithi175.stdout:Mar 18 23:08:08 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-z[69214]: 2024-03-18T23:08:08.854+0000 7fd447dba200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-18T23:08:09.096 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:09.097+0000 7f691e39d640 1 -- 172.21.15.49:0/1869622753 --> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) -- 0x7f6918061c60 con 0x7f69040a28f0 2024-03-18T23:08:09.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:08:09 smithi049 ceph-mon[62991]: from='client.? 172.21.15.49:0/2462774750' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-03-18T23:08:09.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:08:09 smithi049 ceph-mon[62991]: mgrmap e19: x(active, since 9m), standbys: z, y 2024-03-18T23:08:09.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:08:09 smithi049 ceph-mon[62991]: from='client.? ' entity='client.admin' 2024-03-18T23:08:09.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:08:09 smithi049 ceph-mon[62991]: from='client.? 172.21.15.49:0/148518429' entity='client.admin' 2024-03-18T23:08:09.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:08:09 smithi175 ceph-mon[67352]: from='client.? 172.21.15.49:0/2462774750' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-03-18T23:08:09.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:08:09 smithi175 ceph-mon[67352]: mgrmap e19: x(active, since 9m), standbys: z, y 2024-03-18T23:08:09.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:08:09 smithi175 ceph-mon[67352]: from='client.? ' entity='client.admin' 2024-03-18T23:08:09.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:08:09 smithi175 ceph-mon[67352]: from='client.? 172.21.15.49:0/148518429' entity='client.admin' 2024-03-18T23:08:09.483 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:09.484+0000 7f691d39b640 1 -- 172.21.15.49:0/1869622753 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f69040a28f0 msgr2=0x7f69040a4db0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.49:6800/2083805211 2024-03-18T23:08:09.483 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:09.484+0000 7f691d39b640 1 --2- 172.21.15.49:0/1869622753 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f69040a28f0 0x7f69040a4db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-18T23:08:09.506 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:08:09 smithi139 ceph-mon[67282]: from='client.? 172.21.15.49:0/2462774750' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-03-18T23:08:09.506 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:08:09 smithi139 ceph-mon[67282]: mgrmap e19: x(active, since 9m), standbys: z, y 2024-03-18T23:08:09.506 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:08:09 smithi139 ceph-mon[67282]: from='client.? ' entity='client.admin' 2024-03-18T23:08:09.506 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:08:09 smithi139 ceph-mon[67282]: from='client.? 172.21.15.49:0/148518429' entity='client.admin' 2024-03-18T23:08:09.815 INFO:journalctl@ceph.mgr.y.smithi139.stdout:Mar 18 23:08:09 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-y[68685]: 2024-03-18T23:08:09.505+0000 7f4235a6b200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-18T23:08:09.815 INFO:journalctl@ceph.mgr.y.smithi139.stdout:Mar 18 23:08:09 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-y[68685]: 2024-03-18T23:08:09.600+0000 7f4235a6b200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-18T23:08:09.815 INFO:journalctl@ceph.mgr.y.smithi139.stdout:Mar 18 23:08:09 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-y[68685]: 2024-03-18T23:08:09.745+0000 7f4235a6b200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-18T23:08:09.823 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 18 23:08:09 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-x[63224]: 2024-03-18T23:08:09.478+0000 7f2e234bd200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-18T23:08:09.823 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 18 23:08:09 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-x[63224]: 2024-03-18T23:08:09.575+0000 7f2e234bd200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-18T23:08:09.823 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 18 23:08:09 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-x[63224]: 2024-03-18T23:08:09.723+0000 7f2e234bd200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-18T23:08:09.823 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 18 23:08:09 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-x[63224]: 2024-03-18T23:08:09.822+0000 7f2e234bd200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-18T23:08:09.845 INFO:journalctl@ceph.mgr.z.smithi175.stdout:Mar 18 23:08:09 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-z[69214]: 2024-03-18T23:08:09.542+0000 7fd447dba200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-18T23:08:09.845 INFO:journalctl@ceph.mgr.z.smithi175.stdout:Mar 18 23:08:09 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-z[69214]: 2024-03-18T23:08:09.641+0000 7fd447dba200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-18T23:08:09.845 INFO:journalctl@ceph.mgr.z.smithi175.stdout:Mar 18 23:08:09 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-z[69214]: 2024-03-18T23:08:09.844+0000 7fd447dba200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-18T23:08:10.102 INFO:journalctl@ceph.mgr.z.smithi175.stdout:Mar 18 23:08:09 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-z[69214]: 2024-03-18T23:08:09.987+0000 7fd447dba200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-18T23:08:10.130 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 18 23:08:09 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-x[63224]: 2024-03-18T23:08:09.910+0000 7f2e234bd200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-18T23:08:10.222 INFO:journalctl@ceph.mgr.y.smithi139.stdout:Mar 18 23:08:09 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-y[68685]: 2024-03-18T23:08:09.842+0000 7f4235a6b200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-18T23:08:10.223 INFO:journalctl@ceph.mgr.y.smithi139.stdout:Mar 18 23:08:09 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-y[68685]: 2024-03-18T23:08:09.925+0000 7f4235a6b200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-18T23:08:10.223 INFO:journalctl@ceph.mgr.y.smithi139.stdout:Mar 18 23:08:10 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-y[68685]: 2024-03-18T23:08:10.222+0000 7f4235a6b200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-18T23:08:10.285 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:10.285+0000 7f691d39b640 1 -- 172.21.15.49:0/1869622753 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f69040a28f0 msgr2=0x7f69040a4db0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.49:6800/2083805211 2024-03-18T23:08:10.285 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:10.285+0000 7f691d39b640 1 --2- 172.21.15.49:0/1869622753 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f69040a28f0 0x7f69040a4db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-18T23:08:10.435 INFO:journalctl@ceph.mgr.z.smithi175.stdout:Mar 18 23:08:10 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-z[69214]: 2024-03-18T23:08:10.101+0000 7fd447dba200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-18T23:08:10.490 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 18 23:08:10 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-x[63224]: 2024-03-18T23:08:10.186+0000 7f2e234bd200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-18T23:08:10.490 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 18 23:08:10 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-x[63224]: 2024-03-18T23:08:10.300+0000 7f2e234bd200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-18T23:08:10.490 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 18 23:08:10 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-x[63224]: 2024-03-18T23:08:10.390+0000 7f2e234bd200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-18T23:08:10.490 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 18 23:08:10 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-x[63224]: 2024-03-18T23:08:10.489+0000 7f2e234bd200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-18T23:08:10.557 INFO:journalctl@ceph.mgr.y.smithi139.stdout:Mar 18 23:08:10 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-y[68685]: 2024-03-18T23:08:10.334+0000 7f4235a6b200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-18T23:08:10.557 INFO:journalctl@ceph.mgr.y.smithi139.stdout:Mar 18 23:08:10 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-y[68685]: 2024-03-18T23:08:10.421+0000 7f4235a6b200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-18T23:08:10.711 INFO:journalctl@ceph.mgr.z.smithi175.stdout:Mar 18 23:08:10 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-z[69214]: 2024-03-18T23:08:10.434+0000 7fd447dba200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-18T23:08:10.711 INFO:journalctl@ceph.mgr.z.smithi175.stdout:Mar 18 23:08:10 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-z[69214]: 2024-03-18T23:08:10.602+0000 7fd447dba200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-18T23:08:10.815 INFO:journalctl@ceph.mgr.y.smithi139.stdout:Mar 18 23:08:10 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-y[68685]: 2024-03-18T23:08:10.557+0000 7f4235a6b200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-18T23:08:10.815 INFO:journalctl@ceph.mgr.y.smithi139.stdout:Mar 18 23:08:10 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-y[68685]: 2024-03-18T23:08:10.778+0000 7f4235a6b200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-18T23:08:10.880 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 18 23:08:10 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-x[63224]: 2024-03-18T23:08:10.672+0000 7f2e234bd200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-18T23:08:11.104 INFO:journalctl@ceph.mgr.z.smithi175.stdout:Mar 18 23:08:10 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-z[69214]: 2024-03-18T23:08:10.737+0000 7fd447dba200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-18T23:08:11.105 INFO:journalctl@ceph.mgr.z.smithi175.stdout:Mar 18 23:08:10 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-z[69214]: 2024-03-18T23:08:10.867+0000 7fd447dba200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-18T23:08:11.105 INFO:journalctl@ceph.mgr.z.smithi175.stdout:Mar 18 23:08:11 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-z[69214]: 2024-03-18T23:08:11.103+0000 7fd447dba200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-18T23:08:11.886 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:11.885+0000 7f691d39b640 1 -- 172.21.15.49:0/1869622753 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f69040a28f0 msgr2=0x7f69040a4db0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.49:6800/2083805211 2024-03-18T23:08:11.886 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:11.885+0000 7f691d39b640 1 --2- 172.21.15.49:0/1869622753 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f69040a28f0 0x7f69040a4db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-03-18T23:08:12.065 INFO:journalctl@ceph.mgr.y.smithi139.stdout:Mar 18 23:08:11 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-y[68685]: 2024-03-18T23:08:11.799+0000 7f4235a6b200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-18T23:08:12.065 INFO:journalctl@ceph.mgr.y.smithi139.stdout:Mar 18 23:08:11 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-y[68685]: 2024-03-18T23:08:11.931+0000 7f4235a6b200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-18T23:08:12.130 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 18 23:08:11 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-x[63224]: 2024-03-18T23:08:11.695+0000 7f2e234bd200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-18T23:08:12.130 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 18 23:08:11 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-x[63224]: 2024-03-18T23:08:11.833+0000 7f2e234bd200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-18T23:08:12.485 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 18 23:08:12 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-x[63224]: 2024-03-18T23:08:12.182+0000 7f2e234bd200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-18T23:08:12.485 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 18 23:08:12 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-x[63224]: 2024-03-18T23:08:12.277+0000 7f2e234bd200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-18T23:08:12.486 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 18 23:08:12 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-x[63224]: 2024-03-18T23:08:12.484+0000 7f2e234bd200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-18T23:08:12.565 INFO:journalctl@ceph.mgr.y.smithi139.stdout:Mar 18 23:08:12 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-y[68685]: 2024-03-18T23:08:12.286+0000 7f4235a6b200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-18T23:08:12.565 INFO:journalctl@ceph.mgr.y.smithi139.stdout:Mar 18 23:08:12 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-y[68685]: 2024-03-18T23:08:12.394+0000 7f4235a6b200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-18T23:08:12.711 INFO:journalctl@ceph.mgr.z.smithi175.stdout:Mar 18 23:08:12 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-z[69214]: 2024-03-18T23:08:12.311+0000 7fd447dba200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-18T23:08:12.711 INFO:journalctl@ceph.mgr.z.smithi175.stdout:Mar 18 23:08:12 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-z[69214]: 2024-03-18T23:08:12.450+0000 7fd447dba200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-18T23:08:12.880 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 18 23:08:12 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-x[63224]: 2024-03-18T23:08:12.680+0000 7f2e234bd200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-18T23:08:13.065 INFO:journalctl@ceph.mgr.y.smithi139.stdout:Mar 18 23:08:12 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-y[68685]: 2024-03-18T23:08:12.604+0000 7f4235a6b200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-18T23:08:13.065 INFO:journalctl@ceph.mgr.y.smithi139.stdout:Mar 18 23:08:12 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-y[68685]: 2024-03-18T23:08:12.808+0000 7f4235a6b200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-18T23:08:13.170 INFO:journalctl@ceph.mgr.z.smithi175.stdout:Mar 18 23:08:12 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-z[69214]: 2024-03-18T23:08:12.823+0000 7fd447dba200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-18T23:08:13.170 INFO:journalctl@ceph.mgr.z.smithi175.stdout:Mar 18 23:08:12 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-z[69214]: 2024-03-18T23:08:12.933+0000 7fd447dba200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-18T23:08:13.256 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 18 23:08:12 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-x[63224]: 2024-03-18T23:08:12.984+0000 7f2e234bd200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-18T23:08:13.444 INFO:journalctl@ceph.mgr.y.smithi139.stdout:Mar 18 23:08:13 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-y[68685]: 2024-03-18T23:08:13.166+0000 7f4235a6b200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-18T23:08:13.445 INFO:journalctl@ceph.mgr.y.smithi139.stdout:Mar 18 23:08:13 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-y[68685]: 2024-03-18T23:08:13.444+0000 7f4235a6b200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-18T23:08:13.461 INFO:journalctl@ceph.mgr.z.smithi175.stdout:Mar 18 23:08:13 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-z[69214]: 2024-03-18T23:08:13.169+0000 7fd447dba200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-18T23:08:13.461 INFO:journalctl@ceph.mgr.z.smithi175.stdout:Mar 18 23:08:13 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-z[69214]: 2024-03-18T23:08:13.361+0000 7fd447dba200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-18T23:08:13.507 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 18 23:08:13 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-x[63224]: 2024-03-18T23:08:13.255+0000 7f2e234bd200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-18T23:08:13.880 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 18 23:08:13 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-x[63224]: 2024-03-18T23:08:13.506+0000 7f2e234bd200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-18T23:08:14.021 INFO:journalctl@ceph.mgr.z.smithi175.stdout:Mar 18 23:08:13 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-z[69214]: 2024-03-18T23:08:13.755+0000 7fd447dba200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-18T23:08:14.065 INFO:journalctl@ceph.mgr.y.smithi139.stdout:Mar 18 23:08:13 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-y[68685]: 2024-03-18T23:08:13.724+0000 7f4235a6b200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-18T23:08:14.334 INFO:journalctl@ceph.mgr.z.smithi175.stdout:Mar 18 23:08:14 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-z[69214]: 2024-03-18T23:08:14.020+0000 7fd447dba200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-18T23:08:14.630 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 18 23:08:14 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-x[63224]: 2024-03-18T23:08:14.360+0000 7f2e234bd200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-18T23:08:14.711 INFO:journalctl@ceph.mgr.z.smithi175.stdout:Mar 18 23:08:14 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-z[69214]: 2024-03-18T23:08:14.334+0000 7fd447dba200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-18T23:08:14.815 INFO:journalctl@ceph.mgr.y.smithi139.stdout:Mar 18 23:08:14 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-y[68685]: 2024-03-18T23:08:14.551+0000 7f4235a6b200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-18T23:08:15.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:15.089+0000 7f691d39b640 1 -- 172.21.15.49:0/1869622753 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f69040a28f0 msgr2=0x7f69040a4db0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.49:6800/2083805211 2024-03-18T23:08:15.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:15.089+0000 7f691d39b640 1 --2- 172.21.15.49:0/1869622753 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f69040a28f0 0x7f69040a4db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 6.400000 2024-03-18T23:08:15.299 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 18 23:08:14 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-x[63224]: 2024-03-18T23:08:14.959+0000 7f2e234bd200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-18T23:08:15.299 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 18 23:08:15 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-x[63224]: 2024-03-18T23:08:15.065+0000 7f2e234bd200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-18T23:08:15.322 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:15.321+0000 7f690e7fc640 1 -- 172.21.15.49:0/1869622753 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mgrmap(e 20) ==== 136975+0+0 (secure 0 0 0) 0x7f69080981c0 con 0x7f6918101760 2024-03-18T23:08:15.322 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:15.321+0000 7f690e7fc640 1 -- 172.21.15.49:0/1869622753 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f69040a28f0 msgr2=0x7f69040a4db0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-18T23:08:15.323 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:15.321+0000 7f690e7fc640 1 --2- 172.21.15.49:0/1869622753 >> [v2:172.21.15.49:6800/2083805211,v1:172.21.15.49:6801/2083805211] conn(0x7f69040a28f0 0x7f69040a4db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:15.482 INFO:journalctl@ceph.mgr.y.smithi139.stdout:Mar 18 23:08:15 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-y[68685]: 2024-03-18T23:08:15.165+0000 7f4235a6b200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-18T23:08:15.482 INFO:journalctl@ceph.mgr.y.smithi139.stdout:Mar 18 23:08:15 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-y[68685]: 2024-03-18T23:08:15.263+0000 7f4235a6b200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-18T23:08:15.483 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:08:15 smithi139 ceph-mon[67282]: Active manager daemon x restarted 2024-03-18T23:08:15.483 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:08:15 smithi139 ceph-mon[67282]: Activating manager daemon x 2024-03-18T23:08:15.483 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:08:15 smithi139 ceph-mon[67282]: osdmap e89: 12 total, 12 up, 12 in 2024-03-18T23:08:15.483 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:08:15 smithi139 ceph-mon[67282]: mgrmap e20: x(active, starting, since 0.0120627s), standbys: z, y 2024-03-18T23:08:15.484 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:08:15 smithi139 ceph-mon[67282]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-18T23:08:15.484 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:08:15 smithi139 ceph-mon[67282]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-18T23:08:15.484 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:08:15 smithi139 ceph-mon[67282]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-18T23:08:15.484 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:08:15 smithi139 ceph-mon[67282]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-18T23:08:15.484 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:08:15 smithi139 ceph-mon[67282]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-18T23:08:15.484 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:08:15 smithi139 ceph-mon[67282]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-18T23:08:15.485 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:08:15 smithi139 ceph-mon[67282]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-18T23:08:15.485 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:08:15 smithi139 ceph-mon[67282]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-18T23:08:15.485 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:08:15 smithi139 ceph-mon[67282]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-18T23:08:15.485 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:08:15 smithi139 ceph-mon[67282]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-18T23:08:15.485 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:08:15 smithi139 ceph-mon[67282]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-18T23:08:15.485 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:08:15 smithi139 ceph-mon[67282]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-18T23:08:15.486 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:08:15 smithi139 ceph-mon[67282]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-18T23:08:15.486 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:08:15 smithi139 ceph-mon[67282]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-18T23:08:15.486 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:08:15 smithi139 ceph-mon[67282]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-18T23:08:15.486 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:08:15 smithi139 ceph-mon[67282]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-18T23:08:15.486 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:08:15 smithi139 ceph-mon[67282]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-03-18T23:08:15.486 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:08:15 smithi139 ceph-mon[67282]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-18T23:08:15.487 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:08:15 smithi139 ceph-mon[67282]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-18T23:08:15.487 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:08:15 smithi139 ceph-mon[67282]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-18T23:08:15.487 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:08:15 smithi139 ceph-mon[67282]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-18T23:08:15.487 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:08:15 smithi139 ceph-mon[67282]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-18T23:08:15.487 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:08:15 smithi139 ceph-mon[67282]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-18T23:08:15.487 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:08:15 smithi139 ceph-mon[67282]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-18T23:08:15.487 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:08:15 smithi139 ceph-mon[67282]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-18T23:08:15.488 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:08:15 smithi139 ceph-mon[67282]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-18T23:08:15.488 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:08:15 smithi139 ceph-mon[67282]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-18T23:08:15.488 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:08:15 smithi139 ceph-mon[67282]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-18T23:08:15.488 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:08:15 smithi139 ceph-mon[67282]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-18T23:08:15.488 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:08:15 smithi139 ceph-mon[67282]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-18T23:08:15.488 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:08:15 smithi139 ceph-mon[67282]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-18T23:08:15.489 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:08:15 smithi139 ceph-mon[67282]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-18T23:08:15.489 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:08:15 smithi139 ceph-mon[67282]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-18T23:08:15.489 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:08:15 smithi139 ceph-mon[67282]: Manager daemon x is now available 2024-03-18T23:08:15.571 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 18 23:08:15 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-x[63224]: 2024-03-18T23:08:15.297+0000 7f2e234bd200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-18T23:08:15.572 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:08:15 smithi049 ceph-mon[62991]: Active manager daemon x restarted 2024-03-18T23:08:15.572 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:08:15 smithi049 ceph-mon[62991]: Activating manager daemon x 2024-03-18T23:08:15.572 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:08:15 smithi049 ceph-mon[62991]: osdmap e89: 12 total, 12 up, 12 in 2024-03-18T23:08:15.572 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:08:15 smithi049 ceph-mon[62991]: mgrmap e20: x(active, starting, since 0.0120627s), standbys: z, y 2024-03-18T23:08:15.572 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:08:15 smithi049 ceph-mon[62991]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-18T23:08:15.572 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:08:15 smithi049 ceph-mon[62991]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-18T23:08:15.573 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:08:15 smithi049 ceph-mon[62991]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-18T23:08:15.573 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:08:15 smithi049 ceph-mon[62991]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-18T23:08:15.573 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:08:15 smithi049 ceph-mon[62991]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-18T23:08:15.573 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:08:15 smithi049 ceph-mon[62991]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-18T23:08:15.573 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:08:15 smithi049 ceph-mon[62991]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-18T23:08:15.573 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:08:15 smithi049 ceph-mon[62991]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-18T23:08:15.574 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:08:15 smithi049 ceph-mon[62991]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-18T23:08:15.574 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:08:15 smithi049 ceph-mon[62991]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-18T23:08:15.574 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:08:15 smithi049 ceph-mon[62991]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-18T23:08:15.574 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:08:15 smithi049 ceph-mon[62991]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-18T23:08:15.574 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:08:15 smithi049 ceph-mon[62991]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-18T23:08:15.575 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:08:15 smithi049 ceph-mon[62991]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-18T23:08:15.575 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:08:15 smithi049 ceph-mon[62991]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-18T23:08:15.575 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:08:15 smithi049 ceph-mon[62991]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-18T23:08:15.575 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:08:15 smithi049 ceph-mon[62991]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-03-18T23:08:15.575 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:08:15 smithi049 ceph-mon[62991]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-18T23:08:15.575 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:08:15 smithi049 ceph-mon[62991]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-18T23:08:15.575 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:08:15 smithi049 ceph-mon[62991]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-18T23:08:15.576 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:08:15 smithi049 ceph-mon[62991]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-18T23:08:15.576 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:08:15 smithi049 ceph-mon[62991]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-18T23:08:15.576 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:08:15 smithi049 ceph-mon[62991]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-18T23:08:15.576 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:08:15 smithi049 ceph-mon[62991]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-18T23:08:15.576 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:08:15 smithi049 ceph-mon[62991]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-18T23:08:15.576 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:08:15 smithi049 ceph-mon[62991]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-18T23:08:15.577 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:08:15 smithi049 ceph-mon[62991]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-18T23:08:15.577 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:08:15 smithi049 ceph-mon[62991]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-18T23:08:15.577 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:08:15 smithi049 ceph-mon[62991]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-18T23:08:15.577 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:08:15 smithi049 ceph-mon[62991]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-18T23:08:15.577 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:08:15 smithi049 ceph-mon[62991]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-18T23:08:15.577 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:08:15 smithi049 ceph-mon[62991]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-18T23:08:15.578 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:08:15 smithi049 ceph-mon[62991]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-18T23:08:15.578 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:08:15 smithi049 ceph-mon[62991]: Manager daemon x is now available 2024-03-18T23:08:15.630 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:08:15 smithi175 ceph-mon[67352]: Active manager daemon x restarted 2024-03-18T23:08:15.630 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:08:15 smithi175 ceph-mon[67352]: Activating manager daemon x 2024-03-18T23:08:15.630 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:08:15 smithi175 ceph-mon[67352]: osdmap e89: 12 total, 12 up, 12 in 2024-03-18T23:08:15.630 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:08:15 smithi175 ceph-mon[67352]: mgrmap e20: x(active, starting, since 0.0120627s), standbys: z, y 2024-03-18T23:08:15.630 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:08:15 smithi175 ceph-mon[67352]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-18T23:08:15.630 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:08:15 smithi175 ceph-mon[67352]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-18T23:08:15.630 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:08:15 smithi175 ceph-mon[67352]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-18T23:08:15.630 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:08:15 smithi175 ceph-mon[67352]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-18T23:08:15.631 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:08:15 smithi175 ceph-mon[67352]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-18T23:08:15.631 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:08:15 smithi175 ceph-mon[67352]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-18T23:08:15.631 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:08:15 smithi175 ceph-mon[67352]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-18T23:08:15.631 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:08:15 smithi175 ceph-mon[67352]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-18T23:08:15.631 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:08:15 smithi175 ceph-mon[67352]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-18T23:08:15.631 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:08:15 smithi175 ceph-mon[67352]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-18T23:08:15.631 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:08:15 smithi175 ceph-mon[67352]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-18T23:08:15.631 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:08:15 smithi175 ceph-mon[67352]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-18T23:08:15.631 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:08:15 smithi175 ceph-mon[67352]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-18T23:08:15.631 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:08:15 smithi175 ceph-mon[67352]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-18T23:08:15.631 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:08:15 smithi175 ceph-mon[67352]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-18T23:08:15.631 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:08:15 smithi175 ceph-mon[67352]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-18T23:08:15.631 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:08:15 smithi175 ceph-mon[67352]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-03-18T23:08:15.631 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:08:15 smithi175 ceph-mon[67352]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-18T23:08:15.631 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:08:15 smithi175 ceph-mon[67352]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-18T23:08:15.632 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:08:15 smithi175 ceph-mon[67352]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-18T23:08:15.632 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:08:15 smithi175 ceph-mon[67352]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-18T23:08:15.632 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:08:15 smithi175 ceph-mon[67352]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-18T23:08:15.632 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:08:15 smithi175 ceph-mon[67352]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-18T23:08:15.632 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:08:15 smithi175 ceph-mon[67352]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-18T23:08:15.632 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:08:15 smithi175 ceph-mon[67352]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-18T23:08:15.632 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:08:15 smithi175 ceph-mon[67352]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-18T23:08:15.632 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:08:15 smithi175 ceph-mon[67352]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-18T23:08:15.632 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:08:15 smithi175 ceph-mon[67352]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-18T23:08:15.632 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:08:15 smithi175 ceph-mon[67352]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-18T23:08:15.632 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:08:15 smithi175 ceph-mon[67352]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-18T23:08:15.632 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:08:15 smithi175 ceph-mon[67352]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-18T23:08:15.632 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:08:15 smithi175 ceph-mon[67352]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-18T23:08:15.632 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:08:15 smithi175 ceph-mon[67352]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-18T23:08:15.632 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:08:15 smithi175 ceph-mon[67352]: Manager daemon x is now available 2024-03-18T23:08:15.633 INFO:journalctl@ceph.mgr.z.smithi175.stdout:Mar 18 23:08:15 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-z[69214]: 2024-03-18T23:08:15.320+0000 7fd447dba200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-18T23:08:15.815 INFO:journalctl@ceph.mgr.y.smithi139.stdout:Mar 18 23:08:15 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-y[68685]: 2024-03-18T23:08:15.514+0000 7f4235a6b200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-18T23:08:15.880 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 18 23:08:15 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-x[63224]: 2024-03-18T23:08:15.569+0000 7f2d82e8c700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-18T23:08:15.880 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 18 23:08:15 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-x[63224]: 2024-03-18T23:08:15.569+0000 7f2d82e8c700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-18T23:08:15.880 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 18 23:08:15 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-x[63224]: 2024-03-18T23:08:15.569+0000 7f2d82e8c700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-18T23:08:15.880 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 18 23:08:15 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-x[63224]: 2024-03-18T23:08:15.569+0000 7f2d82e8c700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-18T23:08:15.880 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 18 23:08:15 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-x[63224]: 2024-03-18T23:08:15.569+0000 7f2d82e8c700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-18T23:08:15.880 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 18 23:08:15 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-x[63224]: 2024-03-18T23:08:15.570+0000 7f2d85691700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-18T23:08:15.880 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 18 23:08:15 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-x[63224]: 2024-03-18T23:08:15.570+0000 7f2d85691700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-18T23:08:15.880 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 18 23:08:15 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-x[63224]: 2024-03-18T23:08:15.570+0000 7f2d85691700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-18T23:08:15.881 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 18 23:08:15 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-x[63224]: 2024-03-18T23:08:15.570+0000 7f2d85691700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-18T23:08:15.881 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 18 23:08:15 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-x[63224]: 2024-03-18T23:08:15.570+0000 7f2d85691700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-18T23:08:16.332 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:16.329+0000 7f690e7fc640 1 -- 172.21.15.49:0/1869622753 <== mon.0 v2:172.21.15.49:3300/0 8 ==== mgrmap(e 21) ==== 137102+0+0 (secure 0 0 0) 0x7f6908055a20 con 0x7f6918101760 2024-03-18T23:08:16.354 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:16.330+0000 7f690e7fc640 1 --2- 172.21.15.49:0/1869622753 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f6904051670 0x7f6904051a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:16.355 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:16.330+0000 7f690e7fc640 1 -- 172.21.15.49:0/1869622753 --> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) -- 0x7f690403c9c0 con 0x7f6904051670 2024-03-18T23:08:16.355 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:16.333+0000 7f691d39b640 1 --2- 172.21.15.49:0/1869622753 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f6904051670 0x7f6904051a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:16.355 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:16.336+0000 7f691d39b640 1 --2- 172.21.15.49:0/1869622753 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f6904051670 0x7f6904051a50 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f6914000c00 tx=0x7f6914005fd0 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:08:16.366 INFO:journalctl@ceph.mgr.z.smithi175.stdout:Mar 18 23:08:16 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-z[69214]: 2024-03-18T23:08:16.077+0000 7fd447dba200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-18T23:08:16.367 INFO:journalctl@ceph.mgr.z.smithi175.stdout:Mar 18 23:08:16 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-z[69214]: 2024-03-18T23:08:16.218+0000 7fd447dba200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-18T23:08:16.408 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:16.407+0000 7f690e7fc640 1 -- 172.21.15.49:0/1869622753 <== mgr.15366 v2:172.21.15.49:6800/1112476276 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f690403c9c0 con 0x7f6904051670 2024-03-18T23:08:16.409 INFO:teuthology.orchestra.run.smithi049.stdout:Schedule set for path / 2024-03-18T23:08:16.411 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:16.409+0000 7f691e39d640 1 -- 172.21.15.49:0/1869622753 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f6904051670 msgr2=0x7f6904051a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:16.411 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:16.409+0000 7f691e39d640 1 --2- 172.21.15.49:0/1869622753 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f6904051670 0x7f6904051a50 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f6914000c00 tx=0x7f6914005fd0 comp rx=0 tx=0).stop 2024-03-18T23:08:16.412 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:16.409+0000 7f691e39d640 1 -- 172.21.15.49:0/1869622753 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6918101760 msgr2=0x7f691819eae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:16.412 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:16.409+0000 7f691e39d640 1 --2- 172.21.15.49:0/1869622753 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6918101760 0x7f691819eae0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f690800e9f0 tx=0x7f690800eeb0 comp rx=0 tx=0).stop 2024-03-18T23:08:16.412 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:16.409+0000 7f691db9c640 1 -- 172.21.15.49:0/1869622753 reap_dead start 2024-03-18T23:08:16.412 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:16.410+0000 7f691e39d640 1 -- 172.21.15.49:0/1869622753 shutdown_connections 2024-03-18T23:08:16.412 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:16.410+0000 7f691e39d640 1 -- 172.21.15.49:0/1869622753 >> 172.21.15.49:0/1869622753 conn(0x7f69180fbb00 msgr2=0x7f6918104b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:08:16.412 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:16.410+0000 7f691e39d640 1 -- 172.21.15.49:0/1869622753 shutdown_connections 2024-03-18T23:08:16.412 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:16.410+0000 7f691e39d640 1 -- 172.21.15.49:0/1869622753 wait complete. 2024-03-18T23:08:16.422 DEBUG:teuthology.orchestra.run.smithi049:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h' 2024-03-18T23:08:16.537 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:16.534+0000 7fb5879c6640 1 -- 172.21.15.49:0/1273387714 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb580101980 msgr2=0x7fb580101e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:16.538 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:16.534+0000 7fb5879c6640 1 --2- 172.21.15.49:0/1273387714 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb580101980 0x7fb580101e00 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fb570009900 tx=0x7fb57002bc30 comp rx=0 tx=0).stop 2024-03-18T23:08:16.538 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:16.534+0000 7fb5879c6640 1 -- 172.21.15.49:0/1273387714 shutdown_connections 2024-03-18T23:08:16.538 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:16.534+0000 7fb5879c6640 1 --2- 172.21.15.49:0/1273387714 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb580102340 0x7fb580108bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:16.538 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:16.534+0000 7fb5879c6640 1 --2- 172.21.15.49:0/1273387714 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb580101980 0x7fb580101e00 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:16.538 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:16.534+0000 7fb5879c6640 1 --2- 172.21.15.49:0/1273387714 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb580100790 0x7fb580100bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:16.538 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:16.534+0000 7fb5879c6640 1 -- 172.21.15.49:0/1273387714 >> 172.21.15.49:0/1273387714 conn(0x7fb5800fbd00 msgr2=0x7fb5800fe140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:08:16.538 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:16.534+0000 7fb5879c6640 1 -- 172.21.15.49:0/1273387714 shutdown_connections 2024-03-18T23:08:16.538 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:16.534+0000 7fb5879c6640 1 -- 172.21.15.49:0/1273387714 wait complete. 2024-03-18T23:08:16.538 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:16.535+0000 7fb5879c6640 1 Processor -- start 2024-03-18T23:08:16.538 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:16.535+0000 7fb5879c6640 1 -- start start 2024-03-18T23:08:16.538 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:16.535+0000 7fb5879c6640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb580100790 0x7fb58006abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:16.538 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:16.535+0000 7fb5879c6640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb580101980 0x7fb58019c790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:16.539 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:16.535+0000 7fb5879c6640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb580102340 0x7fb58019ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:16.539 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:16.535+0000 7fb5879c6640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fb58010c410 con 0x7fb580102340 2024-03-18T23:08:16.539 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:16.535+0000 7fb5879c6640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fb58010c290 con 0x7fb580101980 2024-03-18T23:08:16.539 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:16.535+0000 7fb5879c6640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb58010c590 con 0x7fb580100790 2024-03-18T23:08:16.539 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:16.535+0000 7fb585f63640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb580102340 0x7fb58019ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:16.539 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:16.535+0000 7fb585f63640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb580102340 0x7fb58019ee00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53304/0 (socket says 172.21.15.49:53304) 2024-03-18T23:08:16.539 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:16.535+0000 7fb585f63640 1 -- 172.21.15.49:0/2395988150 learned_addr learned my addr 172.21.15.49:0/2395988150 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:08:16.539 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:16.535+0000 7fb585762640 1 --2- 172.21.15.49:0/2395988150 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb580100790 0x7fb58006abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:16.539 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:16.535+0000 7fb584f61640 1 --2- 172.21.15.49:0/2395988150 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb580101980 0x7fb58019c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:16.539 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:16.536+0000 7fb585f63640 1 -- 172.21.15.49:0/2395988150 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb580100790 msgr2=0x7fb58006abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:16.539 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:16.536+0000 7fb585f63640 1 --2- 172.21.15.49:0/2395988150 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb580100790 0x7fb58006abf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:16.539 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:16.536+0000 7fb585f63640 1 -- 172.21.15.49:0/2395988150 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb580101980 msgr2=0x7fb58019c790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:16.539 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:16.536+0000 7fb585f63640 1 --2- 172.21.15.49:0/2395988150 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb580101980 0x7fb58019c790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:16.540 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:16.536+0000 7fb585f63640 1 -- 172.21.15.49:0/2395988150 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb58019f6c0 con 0x7fb580102340 2024-03-18T23:08:16.540 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:16.536+0000 7fb585f63640 1 --2- 172.21.15.49:0/2395988150 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb580102340 0x7fb58019ee00 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fb57c00eb10 tx=0x7fb57c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:08:16.540 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:16.536+0000 7fb5767fc640 1 -- 172.21.15.49:0/2395988150 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb57c004b20 con 0x7fb580102340 2024-03-18T23:08:16.540 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:16.536+0000 7fb5767fc640 1 -- 172.21.15.49:0/2395988150 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb57c004cc0 con 0x7fb580102340 2024-03-18T23:08:16.540 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:16.536+0000 7fb5767fc640 1 -- 172.21.15.49:0/2395988150 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb57c010760 con 0x7fb580102340 2024-03-18T23:08:16.540 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:16.536+0000 7fb5879c6640 1 -- 172.21.15.49:0/2395988150 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb58019f9b0 con 0x7fb580102340 2024-03-18T23:08:16.540 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:16.536+0000 7fb5879c6640 1 -- 172.21.15.49:0/2395988150 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb58019fed0 con 0x7fb580102340 2024-03-18T23:08:16.541 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:16.537+0000 7fb5879c6640 1 -- 172.21.15.49:0/2395988150 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5801079e0 con 0x7fb580102340 2024-03-18T23:08:16.541 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:16.538+0000 7fb5767fc640 1 -- 172.21.15.49:0/2395988150 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 21) ==== 137102+0+0 (secure 0 0 0) 0x7fb57c005020 con 0x7fb580102340 2024-03-18T23:08:16.541 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:16.539+0000 7fb5767fc640 1 --2- 172.21.15.49:0/2395988150 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7fb56c0a26b0 0x7fb56c0a4b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:16.541 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:16.539+0000 7fb585762640 1 --2- 172.21.15.49:0/2395988150 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7fb56c0a26b0 0x7fb56c0a4b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:16.541 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:16.539+0000 7fb5767fc640 1 -- 172.21.15.49:0/2395988150 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7fb57c0cee30 con 0x7fb580102340 2024-03-18T23:08:16.541 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:16.539+0000 7fb585762640 1 --2- 172.21.15.49:0/2395988150 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7fb56c0a26b0 0x7fb56c0a4b70 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fb5780095f0 tx=0x7fb578005f90 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:08:16.544 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:16.543+0000 7fb5767fc640 1 -- 172.21.15.49:0/2395988150 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fb57c09c0d0 con 0x7fb580102340 2024-03-18T23:08:16.630 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:08:16 smithi049 ceph-mon[62991]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:08:16.630 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:08:16 smithi049 ceph-mon[62991]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:08:16.630 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:08:16 smithi049 ceph-mon[62991]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-18T23:08:16.630 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:08:16 smithi049 ceph-mon[62991]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-18T23:08:16.630 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:08:16 smithi049 ceph-mon[62991]: Standby manager daemon y restarted 2024-03-18T23:08:16.630 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:08:16 smithi049 ceph-mon[62991]: Standby manager daemon y started 2024-03-18T23:08:16.631 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:08:16 smithi049 ceph-mon[62991]: from='mgr.? 172.21.15.139:0/580931305' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-03-18T23:08:16.631 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:08:16 smithi049 ceph-mon[62991]: from='mgr.? 172.21.15.139:0/580931305' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-18T23:08:16.631 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:08:16 smithi049 ceph-mon[62991]: from='mgr.? 172.21.15.139:0/580931305' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-03-18T23:08:16.631 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:08:16 smithi049 ceph-mon[62991]: from='mgr.? 172.21.15.139:0/580931305' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-18T23:08:16.631 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:08:16 smithi049 ceph-mon[62991]: [18/Mar/2024:23:08:16] ENGINE Bus STARTING 2024-03-18T23:08:16.631 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:08:16 smithi049 ceph-mon[62991]: mgrmap e21: x(active, since 1.02053s), standbys: z, y 2024-03-18T23:08:16.631 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:08:16 smithi139 ceph-mon[67282]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:08:16.631 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:08:16 smithi139 ceph-mon[67282]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:08:16.631 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:08:16 smithi139 ceph-mon[67282]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-18T23:08:16.631 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:08:16 smithi139 ceph-mon[67282]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-18T23:08:16.632 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:08:16 smithi139 ceph-mon[67282]: Standby manager daemon y restarted 2024-03-18T23:08:16.632 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:08:16 smithi139 ceph-mon[67282]: Standby manager daemon y started 2024-03-18T23:08:16.632 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:08:16 smithi139 ceph-mon[67282]: from='mgr.? 172.21.15.139:0/580931305' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-03-18T23:08:16.632 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:08:16 smithi139 ceph-mon[67282]: from='mgr.? 172.21.15.139:0/580931305' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-18T23:08:16.632 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:08:16 smithi139 ceph-mon[67282]: from='mgr.? 172.21.15.139:0/580931305' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-03-18T23:08:16.632 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:08:16 smithi139 ceph-mon[67282]: from='mgr.? 172.21.15.139:0/580931305' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-18T23:08:16.632 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:08:16 smithi139 ceph-mon[67282]: [18/Mar/2024:23:08:16] ENGINE Bus STARTING 2024-03-18T23:08:16.632 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:08:16 smithi139 ceph-mon[67282]: mgrmap e21: x(active, since 1.02053s), standbys: z, y 2024-03-18T23:08:16.677 INFO:journalctl@ceph.mgr.z.smithi175.stdout:Mar 18 23:08:16 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-z[69214]: 2024-03-18T23:08:16.525+0000 7fd447dba200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-18T23:08:16.678 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:08:16 smithi175 ceph-mon[67352]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:08:16.678 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:08:16 smithi175 ceph-mon[67352]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:08:16.678 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:08:16 smithi175 ceph-mon[67352]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-18T23:08:16.678 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:08:16 smithi175 ceph-mon[67352]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-18T23:08:16.678 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:08:16 smithi175 ceph-mon[67352]: Standby manager daemon y restarted 2024-03-18T23:08:16.678 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:08:16 smithi175 ceph-mon[67352]: Standby manager daemon y started 2024-03-18T23:08:16.678 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:08:16 smithi175 ceph-mon[67352]: from='mgr.? 172.21.15.139:0/580931305' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-03-18T23:08:16.678 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:08:16 smithi175 ceph-mon[67352]: from='mgr.? 172.21.15.139:0/580931305' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-18T23:08:16.678 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:08:16 smithi175 ceph-mon[67352]: from='mgr.? 172.21.15.139:0/580931305' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-03-18T23:08:16.678 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:08:16 smithi175 ceph-mon[67352]: from='mgr.? 172.21.15.139:0/580931305' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-18T23:08:16.678 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:08:16 smithi175 ceph-mon[67352]: [18/Mar/2024:23:08:16] ENGINE Bus STARTING 2024-03-18T23:08:16.678 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:08:16 smithi175 ceph-mon[67352]: mgrmap e21: x(active, since 1.02053s), standbys: z, y 2024-03-18T23:08:16.762 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:16.761+0000 7fb5879c6640 1 -- 172.21.15.49:0/2395988150 --> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}) -- 0x7fb580061d00 con 0x7fb56c0a26b0 2024-03-18T23:08:16.818 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:16.817+0000 7fb5767fc640 1 -- 172.21.15.49:0/2395988150 <== mgr.15366 v2:172.21.15.49:6800/1112476276 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+26 (secure 0 0 0) 0x7fb580061d00 con 0x7fb56c0a26b0 2024-03-18T23:08:16.819 INFO:teuthology.orchestra.run.smithi049.stdout:Retention added to path / 2024-03-18T23:08:16.820 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:16.819+0000 7fb5879c6640 1 -- 172.21.15.49:0/2395988150 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7fb56c0a26b0 msgr2=0x7fb56c0a4b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:16.821 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:16.819+0000 7fb5879c6640 1 --2- 172.21.15.49:0/2395988150 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7fb56c0a26b0 0x7fb56c0a4b70 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fb5780095f0 tx=0x7fb578005f90 comp rx=0 tx=0).stop 2024-03-18T23:08:16.821 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:16.819+0000 7fb5879c6640 1 -- 172.21.15.49:0/2395988150 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb580102340 msgr2=0x7fb58019ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:16.821 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:16.819+0000 7fb5879c6640 1 --2- 172.21.15.49:0/2395988150 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb580102340 0x7fb58019ee00 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fb57c00eb10 tx=0x7fb57c00c2d0 comp rx=0 tx=0).stop 2024-03-18T23:08:16.821 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:16.819+0000 7fb5879c6640 1 -- 172.21.15.49:0/2395988150 shutdown_connections 2024-03-18T23:08:16.821 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:16.819+0000 7fb5879c6640 1 --2- 172.21.15.49:0/2395988150 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7fb56c0a26b0 0x7fb56c0a4b70 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:16.821 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:16.819+0000 7fb5879c6640 1 --2- 172.21.15.49:0/2395988150 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb580102340 0x7fb58019ee00 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:16.821 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:16.819+0000 7fb5879c6640 1 --2- 172.21.15.49:0/2395988150 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb580101980 0x7fb58019c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:16.821 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:16.819+0000 7fb5879c6640 1 --2- 172.21.15.49:0/2395988150 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb580100790 0x7fb58006abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:16.821 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:16.819+0000 7fb5879c6640 1 -- 172.21.15.49:0/2395988150 >> 172.21.15.49:0/2395988150 conn(0x7fb5800fbd00 msgr2=0x7fb580104bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:08:16.821 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:16.819+0000 7fb5879c6640 1 -- 172.21.15.49:0/2395988150 shutdown_connections 2024-03-18T23:08:16.821 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:16.820+0000 7fb5879c6640 1 -- 172.21.15.49:0/2395988150 wait complete. 2024-03-18T23:08:16.834 DEBUG:teuthology.orchestra.run.smithi049:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-03-18T23:08:16.947 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:16.944+0000 7f1565b52640 1 -- 172.21.15.49:0/1924044695 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1560104cd0 msgr2=0x7f15601070c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:16.947 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:16.944+0000 7f1565b52640 1 --2- 172.21.15.49:0/1924044695 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1560104cd0 0x7f15601070c0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f1548009900 tx=0x7f154802bc30 comp rx=0 tx=0).stop 2024-03-18T23:08:16.947 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:16.944+0000 7f1565b52640 1 -- 172.21.15.49:0/1924044695 shutdown_connections 2024-03-18T23:08:16.948 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:16.944+0000 7f1565b52640 1 --2- 172.21.15.49:0/1924044695 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1560107600 0x7f15601099f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:16.948 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:16.944+0000 7f1565b52640 1 --2- 172.21.15.49:0/1924044695 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1560104cd0 0x7f15601070c0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:16.948 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:16.944+0000 7f1565b52640 1 --2- 172.21.15.49:0/1924044695 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f15601023a0 0x7f1560104790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:16.948 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:16.944+0000 7f1565b52640 1 -- 172.21.15.49:0/1924044695 >> 172.21.15.49:0/1924044695 conn(0x7f15600fbcd0 msgr2=0x7f15600fe110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:08:16.948 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:16.944+0000 7f1565b52640 1 -- 172.21.15.49:0/1924044695 shutdown_connections 2024-03-18T23:08:16.948 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:16.944+0000 7f1565b52640 1 -- 172.21.15.49:0/1924044695 wait complete. 2024-03-18T23:08:16.948 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:16.944+0000 7f1565b52640 1 Processor -- start 2024-03-18T23:08:16.948 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:16.944+0000 7f1565b52640 1 -- start start 2024-03-18T23:08:16.948 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:16.945+0000 7f1565b52640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f15601023a0 0x7f1560101d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:16.948 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:16.945+0000 7f1565b52640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1560104cd0 0x7f1560100350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:16.949 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:16.945+0000 7f1565b52640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1560107600 0x7f1560100890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:16.949 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:16.945+0000 7f1565b52640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f156010d4c0 con 0x7f1560104cd0 2024-03-18T23:08:16.949 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:16.945+0000 7f1565b52640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f156010d340 con 0x7f15601023a0 2024-03-18T23:08:16.949 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:16.945+0000 7f1565b52640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f156010d640 con 0x7f1560107600 2024-03-18T23:08:16.949 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:16.945+0000 7f155effd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1560104cd0 0x7f1560100350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:16.949 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:16.945+0000 7f155effd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1560104cd0 0x7f1560100350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53316/0 (socket says 172.21.15.49:53316) 2024-03-18T23:08:16.949 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:16.945+0000 7f155effd640 1 -- 172.21.15.49:0/2897040497 learned_addr learned my addr 172.21.15.49:0/2897040497 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:08:16.949 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:16.945+0000 7f155f7fe640 1 --2- 172.21.15.49:0/2897040497 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f15601023a0 0x7f1560101d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:16.949 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:16.945+0000 7f155ffff640 1 --2- 172.21.15.49:0/2897040497 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1560107600 0x7f1560100890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:16.949 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:16.945+0000 7f155effd640 1 -- 172.21.15.49:0/2897040497 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1560107600 msgr2=0x7f1560100890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:16.949 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:16.945+0000 7f155effd640 1 --2- 172.21.15.49:0/2897040497 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1560107600 0x7f1560100890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:16.949 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:16.945+0000 7f155effd640 1 -- 172.21.15.49:0/2897040497 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f15601023a0 msgr2=0x7f1560101d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:16.950 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:16.945+0000 7f155effd640 1 --2- 172.21.15.49:0/2897040497 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f15601023a0 0x7f1560101d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:16.950 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:16.945+0000 7f155effd640 1 -- 172.21.15.49:0/2897040497 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1560101120 con 0x7f1560104cd0 2024-03-18T23:08:16.950 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:16.945+0000 7f155f7fe640 1 --2- 172.21.15.49:0/2897040497 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f15601023a0 0x7f1560101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T23:08:16.950 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:16.945+0000 7f155ffff640 1 --2- 172.21.15.49:0/2897040497 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1560107600 0x7f1560100890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T23:08:16.950 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:16.945+0000 7f155effd640 1 --2- 172.21.15.49:0/2897040497 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1560104cd0 0x7f1560100350 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f1548000c00 tx=0x7f1548005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:08:16.950 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:16.945+0000 7f155cff9640 1 -- 172.21.15.49:0/2897040497 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1548041070 con 0x7f1560104cd0 2024-03-18T23:08:16.950 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:16.945+0000 7f155cff9640 1 -- 172.21.15.49:0/2897040497 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f15480044d0 con 0x7f1560104cd0 2024-03-18T23:08:16.950 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:16.945+0000 7f1565b52640 1 -- 172.21.15.49:0/2897040497 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15601a1bf0 con 0x7f1560104cd0 2024-03-18T23:08:16.950 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:16.945+0000 7f155cff9640 1 -- 172.21.15.49:0/2897040497 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1548034440 con 0x7f1560104cd0 2024-03-18T23:08:16.950 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:16.946+0000 7f1565b52640 1 -- 172.21.15.49:0/2897040497 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15601a2140 con 0x7f1560104cd0 2024-03-18T23:08:16.951 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:16.947+0000 7f1565b52640 1 -- 172.21.15.49:0/2897040497 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15600fd7c0 con 0x7f1560104cd0 2024-03-18T23:08:16.951 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:16.948+0000 7f155cff9640 1 -- 172.21.15.49:0/2897040497 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 21) ==== 137102+0+0 (secure 0 0 0) 0x7f1548044070 con 0x7f1560104cd0 2024-03-18T23:08:16.952 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:16.948+0000 7f155cff9640 1 --2- 172.21.15.49:0/2897040497 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f154c0a26b0 0x7f154c0a4b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:16.952 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:16.948+0000 7f155cff9640 1 -- 172.21.15.49:0/2897040497 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f15480f0590 con 0x7f1560104cd0 2024-03-18T23:08:16.952 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:16.948+0000 7f155f7fe640 1 --2- 172.21.15.49:0/2897040497 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f154c0a26b0 0x7f154c0a4b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:16.952 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:16.949+0000 7f155f7fe640 1 --2- 172.21.15.49:0/2897040497 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f154c0a26b0 0x7f154c0a4b70 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f1550005fd0 tx=0x7f1550009600 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:08:16.952 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:16.952+0000 7f155cff9640 1 -- 172.21.15.49:0/2897040497 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f15480bd830 con 0x7f1560104cd0 2024-03-18T23:08:17.158 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:17.157+0000 7f1565b52640 1 -- 172.21.15.49:0/2897040497 --> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7f1560061c60 con 0x7f154c0a26b0 2024-03-18T23:08:17.167 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:17.166+0000 7f155cff9640 1 -- 172.21.15.49:0/2897040497 <== mgr.15366 v2:172.21.15.49:6800/1112476276 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+298 (secure 0 0 0) 0x7f1560061c60 con 0x7f154c0a26b0 2024-03-18T23:08:17.167 INFO:teuthology.orchestra.run.smithi049.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-03-18T00:00:00", "created": "2024-03-18T23:08:16", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true} 2024-03-18T23:08:17.169 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:17.168+0000 7f1565b52640 1 -- 172.21.15.49:0/2897040497 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f154c0a26b0 msgr2=0x7f154c0a4b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:17.170 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:17.168+0000 7f1565b52640 1 --2- 172.21.15.49:0/2897040497 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f154c0a26b0 0x7f154c0a4b70 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f1550005fd0 tx=0x7f1550009600 comp rx=0 tx=0).stop 2024-03-18T23:08:17.170 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:17.168+0000 7f1565b52640 1 -- 172.21.15.49:0/2897040497 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1560104cd0 msgr2=0x7f1560100350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:17.170 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:17.168+0000 7f1565b52640 1 --2- 172.21.15.49:0/2897040497 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1560104cd0 0x7f1560100350 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f1548000c00 tx=0x7f1548005d50 comp rx=0 tx=0).stop 2024-03-18T23:08:17.170 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:17.168+0000 7f1565b52640 1 -- 172.21.15.49:0/2897040497 shutdown_connections 2024-03-18T23:08:17.170 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:17.168+0000 7f1565b52640 1 --2- 172.21.15.49:0/2897040497 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f154c0a26b0 0x7f154c0a4b70 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:17.170 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:17.168+0000 7f1565b52640 1 --2- 172.21.15.49:0/2897040497 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1560107600 0x7f1560100890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:17.170 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:17.168+0000 7f1565b52640 1 --2- 172.21.15.49:0/2897040497 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1560104cd0 0x7f1560100350 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:17.170 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:17.168+0000 7f1565b52640 1 --2- 172.21.15.49:0/2897040497 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f15601023a0 0x7f1560101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:17.170 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:17.168+0000 7f1565b52640 1 -- 172.21.15.49:0/2897040497 >> 172.21.15.49:0/2897040497 conn(0x7f15600fbcd0 msgr2=0x7f1560102e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:08:17.170 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:17.169+0000 7f1565b52640 1 -- 172.21.15.49:0/2897040497 shutdown_connections 2024-03-18T23:08:17.170 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:17.169+0000 7f1565b52640 1 -- 172.21.15.49:0/2897040497 wait complete. 2024-03-18T23:08:17.182 DEBUG:teuthology.orchestra.run.smithi049:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-03-18T23:08:17.304 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:17.301+0000 7fe225ac6640 1 -- 172.21.15.49:0/2186759700 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe2201022f0 msgr2=0x7fe220108b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:17.305 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:17.301+0000 7fe225ac6640 1 --2- 172.21.15.49:0/2186759700 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe2201022f0 0x7fe220108b80 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fe218009ca0 tx=0x7fe21802c0f0 comp rx=0 tx=0).stop 2024-03-18T23:08:17.305 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:17.301+0000 7fe225ac6640 1 -- 172.21.15.49:0/2186759700 shutdown_connections 2024-03-18T23:08:17.305 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:17.301+0000 7fe225ac6640 1 --2- 172.21.15.49:0/2186759700 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe2201022f0 0x7fe220108b80 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:17.305 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:17.301+0000 7fe225ac6640 1 --2- 172.21.15.49:0/2186759700 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe220101930 0x7fe220101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:17.305 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:17.301+0000 7fe225ac6640 1 --2- 172.21.15.49:0/2186759700 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe220100740 0x7fe220100b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:17.305 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:17.301+0000 7fe225ac6640 1 -- 172.21.15.49:0/2186759700 >> 172.21.15.49:0/2186759700 conn(0x7fe2200fbcb0 msgr2=0x7fe2200fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:08:17.305 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:17.301+0000 7fe225ac6640 1 -- 172.21.15.49:0/2186759700 shutdown_connections 2024-03-18T23:08:17.305 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:17.301+0000 7fe225ac6640 1 -- 172.21.15.49:0/2186759700 wait complete. 2024-03-18T23:08:17.305 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:17.302+0000 7fe225ac6640 1 Processor -- start 2024-03-18T23:08:17.305 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:17.302+0000 7fe225ac6640 1 -- start start 2024-03-18T23:08:17.305 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:17.302+0000 7fe225ac6640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe220100740 0x7fe220197b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:17.305 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:17.302+0000 7fe225ac6640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe220101930 0x7fe220198040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:17.305 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:17.302+0000 7fe225ac6640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe2201022f0 0x7fe22019f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:17.306 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:17.302+0000 7fe225ac6640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fe22010c3c0 con 0x7fe2201022f0 2024-03-18T23:08:17.306 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:17.302+0000 7fe225ac6640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fe22010c240 con 0x7fe220101930 2024-03-18T23:08:17.306 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:17.302+0000 7fe225ac6640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe22010c540 con 0x7fe220100740 2024-03-18T23:08:17.306 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:17.303+0000 7fe21effd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe220101930 0x7fe220198040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:17.306 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:17.303+0000 7fe21ffff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe2201022f0 0x7fe22019f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:17.306 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:17.303+0000 7fe21f7fe640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe220100740 0x7fe220197b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:17.306 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:17.303+0000 7fe21effd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe220101930 0x7fe220198040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.49:52542/0 (socket says 172.21.15.49:52542) 2024-03-18T23:08:17.306 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:17.303+0000 7fe21effd640 1 -- 172.21.15.49:0/1987692769 learned_addr learned my addr 172.21.15.49:0/1987692769 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:08:17.306 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:17.303+0000 7fe21effd640 1 -- 172.21.15.49:0/1987692769 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe220100740 msgr2=0x7fe220197b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:17.306 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:17.303+0000 7fe21effd640 1 --2- 172.21.15.49:0/1987692769 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe220100740 0x7fe220197b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:17.306 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:17.303+0000 7fe21effd640 1 -- 172.21.15.49:0/1987692769 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe2201022f0 msgr2=0x7fe22019f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:17.306 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:17.303+0000 7fe21effd640 1 --2- 172.21.15.49:0/1987692769 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe2201022f0 0x7fe22019f060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:17.306 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:17.303+0000 7fe21effd640 1 -- 172.21.15.49:0/1987692769 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe22019f6d0 con 0x7fe220101930 2024-03-18T23:08:17.307 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:17.303+0000 7fe21ffff640 1 --2- 172.21.15.49:0/1987692769 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe2201022f0 0x7fe22019f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:08:17.307 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:17.303+0000 7fe21effd640 1 --2- 172.21.15.49:0/1987692769 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe220101930 0x7fe220198040 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fe21000d770 tx=0x7fe21000dc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:08:17.307 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:17.303+0000 7fe21cff9640 1 -- 172.21.15.49:0/1987692769 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe210017070 con 0x7fe220101930 2024-03-18T23:08:17.307 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:17.303+0000 7fe225ac6640 1 -- 172.21.15.49:0/1987692769 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe22019f9c0 con 0x7fe220101930 2024-03-18T23:08:17.307 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:17.303+0000 7fe21cff9640 1 -- 172.21.15.49:0/1987692769 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe210005500 con 0x7fe220101930 2024-03-18T23:08:17.307 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:17.303+0000 7fe21cff9640 1 -- 172.21.15.49:0/1987692769 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe210004e00 con 0x7fe220101930 2024-03-18T23:08:17.307 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:17.304+0000 7fe225ac6640 1 -- 172.21.15.49:0/1987692769 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe22019ff10 con 0x7fe220101930 2024-03-18T23:08:17.308 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:17.305+0000 7fe225ac6640 1 -- 172.21.15.49:0/1987692769 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2201a0200 con 0x7fe220101930 2024-03-18T23:08:17.308 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:17.306+0000 7fe21cff9640 1 -- 172.21.15.49:0/1987692769 <== mon.2 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 21) ==== 137102+0+0 (secure 0 0 0) 0x7fe210004fa0 con 0x7fe220101930 2024-03-18T23:08:17.308 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:17.306+0000 7fe21cff9640 1 --2- 172.21.15.49:0/1987692769 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7fe20c0a26b0 0x7fe20c0a4b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:17.308 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:17.307+0000 7fe21f7fe640 1 --2- 172.21.15.49:0/1987692769 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7fe20c0a26b0 0x7fe20c0a4b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:17.308 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:17.307+0000 7fe21cff9640 1 -- 172.21.15.49:0/1987692769 <== mon.2 v2:172.21.15.139:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7fe2100ce4e0 con 0x7fe220101930 2024-03-18T23:08:17.308 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:17.307+0000 7fe21f7fe640 1 --2- 172.21.15.49:0/1987692769 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7fe20c0a26b0 0x7fe20c0a4b70 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fe220101790 tx=0x7fe208005f90 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:08:17.311 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:17.310+0000 7fe21cff9640 1 -- 172.21.15.49:0/1987692769 <== mon.2 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fe21009b130 con 0x7fe220101930 2024-03-18T23:08:17.386 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:17.385+0000 7fe21cff9640 1 -- 172.21.15.49:0/1987692769 <== mon.2 v2:172.21.15.139:3300/0 7 ==== mgrmap(e 22) ==== 137410+0+0 (secure 0 0 0) 0x7fe2100977a0 con 0x7fe220101930 2024-03-18T23:08:17.575 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:17.575+0000 7fe225ac6640 1 -- 172.21.15.49:0/1987692769 --> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7fe2200fddc0 con 0x7fe20c0a26b0 2024-03-18T23:08:17.590 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:17.589+0000 7fe21cff9640 1 -- 172.21.15.49:0/1987692769 <== mgr.15366 v2:172.21.15.49:6800/1112476276 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7fe2200fddc0 con 0x7fe20c0a26b0 2024-03-18T23:08:17.590 INFO:teuthology.orchestra.run.smithi049.stdout:/ 1m 6m3h 2024-03-18T23:08:17.593 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:17.591+0000 7fe225ac6640 1 -- 172.21.15.49:0/1987692769 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7fe20c0a26b0 msgr2=0x7fe20c0a4b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:17.593 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:17.591+0000 7fe225ac6640 1 --2- 172.21.15.49:0/1987692769 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7fe20c0a26b0 0x7fe20c0a4b70 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fe220101790 tx=0x7fe208005f90 comp rx=0 tx=0).stop 2024-03-18T23:08:17.593 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:17.591+0000 7fe225ac6640 1 -- 172.21.15.49:0/1987692769 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe220101930 msgr2=0x7fe220198040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:17.593 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:17.591+0000 7fe225ac6640 1 --2- 172.21.15.49:0/1987692769 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe220101930 0x7fe220198040 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fe21000d770 tx=0x7fe21000dc30 comp rx=0 tx=0).stop 2024-03-18T23:08:17.593 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:17.592+0000 7fe225ac6640 1 -- 172.21.15.49:0/1987692769 shutdown_connections 2024-03-18T23:08:17.593 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:17.592+0000 7fe225ac6640 1 --2- 172.21.15.49:0/1987692769 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7fe20c0a26b0 0x7fe20c0a4b70 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:17.593 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:17.592+0000 7fe225ac6640 1 --2- 172.21.15.49:0/1987692769 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe2201022f0 0x7fe22019f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:17.593 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:17.592+0000 7fe225ac6640 1 --2- 172.21.15.49:0/1987692769 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe220101930 0x7fe220198040 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:17.593 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:17.592+0000 7fe225ac6640 1 --2- 172.21.15.49:0/1987692769 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe220100740 0x7fe220197b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:17.593 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:17.592+0000 7fe225ac6640 1 -- 172.21.15.49:0/1987692769 >> 172.21.15.49:0/1987692769 conn(0x7fe2200fbcb0 msgr2=0x7fe2200fd6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:08:17.593 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:17.592+0000 7fe225ac6640 1 -- 172.21.15.49:0/1987692769 shutdown_connections 2024-03-18T23:08:17.594 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:08:17.592+0000 7fe225ac6640 1 -- 172.21.15.49:0/1987692769 wait complete. 2024-03-18T23:08:17.603 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:08:17 smithi049 ceph-mon[62991]: [18/Mar/2024:23:08:16] ENGINE Serving on http://172.21.15.49:8765 2024-03-18T23:08:17.603 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:08:17 smithi049 ceph-mon[62991]: [18/Mar/2024:23:08:16] ENGINE Serving on https://172.21.15.49:7150 2024-03-18T23:08:17.603 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:08:17 smithi049 ceph-mon[62991]: [18/Mar/2024:23:08:16] ENGINE Bus STARTED 2024-03-18T23:08:17.603 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:08:17 smithi049 ceph-mon[62991]: from='client.15393 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T23:08:17.603 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:08:17 smithi049 ceph-mon[62991]: pgmap v3: 97 pgs: 97 active+clean; 1.1 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T23:08:17.603 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:08:17 smithi049 ceph-mon[62991]: from='mgr.? 172.21.15.175:0/2669361369' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-03-18T23:08:17.604 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:08:17 smithi049 ceph-mon[62991]: Standby manager daemon z restarted 2024-03-18T23:08:17.604 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:08:17 smithi049 ceph-mon[62991]: from='mgr.? 172.21.15.175:0/2669361369' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-18T23:08:17.604 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:08:17 smithi049 ceph-mon[62991]: Standby manager daemon z started 2024-03-18T23:08:17.604 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:08:17 smithi049 ceph-mon[62991]: from='mgr.? 172.21.15.175:0/2669361369' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-03-18T23:08:17.604 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:08:17 smithi049 ceph-mon[62991]: from='mgr.? 172.21.15.175:0/2669361369' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-18T23:08:17.604 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:08:17 smithi049 ceph-mon[62991]: from='client.15435 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T23:08:17.605 DEBUG:teuthology.orchestra.run.smithi049:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > START_TIME' 2024-03-18T23:08:17.639 INFO:teuthology.run_tasks:Running task full_sequential_finally... 2024-03-18T23:08:17.650 INFO:teuthology.run_tasks:Running task background_exec... 2024-03-18T23:08:17.685 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi049.front.sepia.ceph.com 2024-03-18T23:08:17.685 DEBUG:teuthology.orchestra.run.smithi049:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2024-03-18T23:08:17.688 INFO:teuthology.run_tasks:Running task workunit... 2024-03-18T23:08:17.719 INFO:tasks.workunit:Pulling workunits from ref e48b1f6c0608b2615e1890cc46286c3877d6e910 2024-03-18T23:08:17.719 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-03-18T23:08:17.719 INFO:tasks.workunit:timeout=3h 2024-03-18T23:08:17.720 INFO:tasks.workunit:cleanup=True 2024-03-18T23:08:17.720 DEBUG:teuthology.orchestra.run.smithi049:> stat -- /home/ubuntu/cephtest/mnt.0 2024-03-18T23:08:17.721 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:08:17 smithi175 ceph-mon[67352]: [18/Mar/2024:23:08:16] ENGINE Serving on http://172.21.15.49:8765 2024-03-18T23:08:17.721 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:08:17 smithi175 ceph-mon[67352]: [18/Mar/2024:23:08:16] ENGINE Serving on https://172.21.15.49:7150 2024-03-18T23:08:17.721 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:08:17 smithi175 ceph-mon[67352]: [18/Mar/2024:23:08:16] ENGINE Bus STARTED 2024-03-18T23:08:17.721 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:08:17 smithi175 ceph-mon[67352]: from='client.15393 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T23:08:17.721 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:08:17 smithi175 ceph-mon[67352]: pgmap v3: 97 pgs: 97 active+clean; 1.1 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T23:08:17.721 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:08:17 smithi175 ceph-mon[67352]: from='mgr.? 172.21.15.175:0/2669361369' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-03-18T23:08:17.722 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:08:17 smithi175 ceph-mon[67352]: Standby manager daemon z restarted 2024-03-18T23:08:17.722 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:08:17 smithi175 ceph-mon[67352]: from='mgr.? 172.21.15.175:0/2669361369' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-18T23:08:17.722 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:08:17 smithi175 ceph-mon[67352]: Standby manager daemon z started 2024-03-18T23:08:17.722 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:08:17 smithi175 ceph-mon[67352]: from='mgr.? 172.21.15.175:0/2669361369' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-03-18T23:08:17.722 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:08:17 smithi175 ceph-mon[67352]: from='mgr.? 172.21.15.175:0/2669361369' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-18T23:08:17.722 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:08:17 smithi175 ceph-mon[67352]: from='client.15435 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T23:08:17.736 INFO:teuthology.orchestra.run.smithi049.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-03-18T23:08:17.736 INFO:teuthology.orchestra.run.smithi049.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-03-18T23:08:17.736 INFO:teuthology.orchestra.run.smithi049.stdout:Device: 77h/119d Inode: 1099511628283 Links: 2 2024-03-18T23:08:17.736 INFO:teuthology.orchestra.run.smithi049.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-18T23:08:17.736 INFO:teuthology.orchestra.run.smithi049.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-03-18T23:08:17.736 INFO:teuthology.orchestra.run.smithi049.stdout:Access: 2024-03-18 23:06:12.097266729 +0000 2024-03-18T23:08:17.736 INFO:teuthology.orchestra.run.smithi049.stdout:Modify: 2024-03-18 23:06:12.097266729 +0000 2024-03-18T23:08:17.736 INFO:teuthology.orchestra.run.smithi049.stdout:Change: 2024-03-18 23:08:06.330121098 +0000 2024-03-18T23:08:17.736 INFO:teuthology.orchestra.run.smithi049.stdout: Birth: 2024-03-18 23:06:12.097266729 +0000 2024-03-18T23:08:17.737 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-03-18T23:08:17.737 DEBUG:teuthology.orchestra.run.smithi049:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-03-18T23:08:17.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:08:17 smithi139 ceph-mon[67282]: [18/Mar/2024:23:08:16] ENGINE Serving on http://172.21.15.49:8765 2024-03-18T23:08:17.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:08:17 smithi139 ceph-mon[67282]: [18/Mar/2024:23:08:16] ENGINE Serving on https://172.21.15.49:7150 2024-03-18T23:08:17.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:08:17 smithi139 ceph-mon[67282]: [18/Mar/2024:23:08:16] ENGINE Bus STARTED 2024-03-18T23:08:17.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:08:17 smithi139 ceph-mon[67282]: from='client.15393 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T23:08:17.816 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:08:17 smithi139 ceph-mon[67282]: pgmap v3: 97 pgs: 97 active+clean; 1.1 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T23:08:17.816 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:08:17 smithi139 ceph-mon[67282]: from='mgr.? 172.21.15.175:0/2669361369' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-03-18T23:08:17.816 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:08:17 smithi139 ceph-mon[67282]: Standby manager daemon z restarted 2024-03-18T23:08:17.816 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:08:17 smithi139 ceph-mon[67282]: from='mgr.? 172.21.15.175:0/2669361369' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-18T23:08:17.816 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:08:17 smithi139 ceph-mon[67282]: Standby manager daemon z started 2024-03-18T23:08:17.816 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:08:17 smithi139 ceph-mon[67282]: from='mgr.? 172.21.15.175:0/2669361369' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-03-18T23:08:17.816 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:08:17 smithi139 ceph-mon[67282]: from='mgr.? 172.21.15.175:0/2669361369' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-18T23:08:17.816 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:08:17 smithi139 ceph-mon[67282]: from='client.15435 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T23:08:17.820 DEBUG:teuthology.orchestra.run.smithi049:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://github.com/batrick/ceph /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout e48b1f6c0608b2615e1890cc46286c3877d6e910 2024-03-18T23:08:17.876 INFO:tasks.workunit.client.0.smithi049.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-03-18T23:08:18.630 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:08:18 smithi049 ceph-mon[62991]: from='client.15441 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T23:08:18.630 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:08:18 smithi049 ceph-mon[62991]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T23:08:18.630 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:08:18 smithi049 ceph-mon[62991]: mgrmap e22: x(active, since 2s), standbys: z, y 2024-03-18T23:08:18.630 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:08:18 smithi049 ceph-mon[62991]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' 2024-03-18T23:08:18.630 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:08:18 smithi049 ceph-mon[62991]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' 2024-03-18T23:08:18.630 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:08:18 smithi049 ceph-mon[62991]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' 2024-03-18T23:08:18.630 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:08:18 smithi049 ceph-mon[62991]: from='client.25265 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T23:08:18.630 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:08:18 smithi049 ceph-mon[62991]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' 2024-03-18T23:08:18.631 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:08:18 smithi049 ceph-mon[62991]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' 2024-03-18T23:08:18.631 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:08:18 smithi049 ceph-mon[62991]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' 2024-03-18T23:08:18.711 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:08:18 smithi175 ceph-mon[67352]: from='client.15441 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T23:08:18.711 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:08:18 smithi175 ceph-mon[67352]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T23:08:18.712 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:08:18 smithi175 ceph-mon[67352]: mgrmap e22: x(active, since 2s), standbys: z, y 2024-03-18T23:08:18.712 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:08:18 smithi175 ceph-mon[67352]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' 2024-03-18T23:08:18.712 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:08:18 smithi175 ceph-mon[67352]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' 2024-03-18T23:08:18.712 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:08:18 smithi175 ceph-mon[67352]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' 2024-03-18T23:08:18.712 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:08:18 smithi175 ceph-mon[67352]: from='client.25265 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T23:08:18.712 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:08:18 smithi175 ceph-mon[67352]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' 2024-03-18T23:08:18.712 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:08:18 smithi175 ceph-mon[67352]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' 2024-03-18T23:08:18.712 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:08:18 smithi175 ceph-mon[67352]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' 2024-03-18T23:08:18.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:08:18 smithi139 ceph-mon[67282]: from='client.15441 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T23:08:18.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:08:18 smithi139 ceph-mon[67282]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T23:08:18.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:08:18 smithi139 ceph-mon[67282]: mgrmap e22: x(active, since 2s), standbys: z, y 2024-03-18T23:08:18.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:08:18 smithi139 ceph-mon[67282]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' 2024-03-18T23:08:18.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:08:18 smithi139 ceph-mon[67282]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' 2024-03-18T23:08:18.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:08:18 smithi139 ceph-mon[67282]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' 2024-03-18T23:08:18.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:08:18 smithi139 ceph-mon[67282]: from='client.25265 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T23:08:18.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:08:18 smithi139 ceph-mon[67282]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' 2024-03-18T23:08:18.816 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:08:18 smithi139 ceph-mon[67282]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' 2024-03-18T23:08:18.816 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:08:18 smithi139 ceph-mon[67282]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' 2024-03-18T23:08:20.065 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:08:19 smithi139 ceph-mon[67282]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' 2024-03-18T23:08:20.066 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:08:19 smithi139 ceph-mon[67282]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' 2024-03-18T23:08:20.066 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:08:19 smithi139 ceph-mon[67282]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:08:20.066 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:08:19 smithi139 ceph-mon[67282]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:08:20.066 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:08:19 smithi139 ceph-mon[67282]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:08:20.066 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:08:19 smithi139 ceph-mon[67282]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:08:20.066 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:08:19 smithi139 ceph-mon[67282]: Adjusting osd_memory_target on smithi139 to 174.1M 2024-03-18T23:08:20.066 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:08:19 smithi139 ceph-mon[67282]: Unable to set osd_memory_target on smithi139 to 182618419: error parsing value: Value '182618419' is below minimum 939524096 2024-03-18T23:08:20.066 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:08:19 smithi139 ceph-mon[67282]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' 2024-03-18T23:08:20.066 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:08:19 smithi139 ceph-mon[67282]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' 2024-03-18T23:08:20.066 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:08:19 smithi139 ceph-mon[67282]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:08:20.066 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:08:19 smithi139 ceph-mon[67282]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:08:20.066 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:08:19 smithi139 ceph-mon[67282]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:08:20.067 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:08:19 smithi139 ceph-mon[67282]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:08:20.067 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:08:19 smithi139 ceph-mon[67282]: Adjusting osd_memory_target on smithi049 to 176.2M 2024-03-18T23:08:20.067 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:08:19 smithi139 ceph-mon[67282]: Unable to set osd_memory_target on smithi049 to 184762368: error parsing value: Value '184762368' is below minimum 939524096 2024-03-18T23:08:20.067 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:08:19 smithi139 ceph-mon[67282]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' 2024-03-18T23:08:20.067 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:08:19 smithi139 ceph-mon[67282]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' 2024-03-18T23:08:20.067 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:08:19 smithi139 ceph-mon[67282]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:08:20.067 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:08:19 smithi139 ceph-mon[67282]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:08:20.067 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:08:19 smithi139 ceph-mon[67282]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:08:20.067 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:08:19 smithi139 ceph-mon[67282]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:08:20.067 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:08:19 smithi139 ceph-mon[67282]: Adjusting osd_memory_target on smithi175 to 174.1M 2024-03-18T23:08:20.067 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:08:19 smithi139 ceph-mon[67282]: Unable to set osd_memory_target on smithi175 to 182618419: error parsing value: Value '182618419' is below minimum 939524096 2024-03-18T23:08:20.067 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:08:19 smithi139 ceph-mon[67282]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:08:20.067 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:08:19 smithi139 ceph-mon[67282]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:08:20.067 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:08:19 smithi139 ceph-mon[67282]: Updating smithi049:/etc/ceph/ceph.conf 2024-03-18T23:08:20.067 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:08:19 smithi139 ceph-mon[67282]: Updating smithi139:/etc/ceph/ceph.conf 2024-03-18T23:08:20.068 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:08:19 smithi139 ceph-mon[67282]: Updating smithi175:/etc/ceph/ceph.conf 2024-03-18T23:08:20.130 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:08:19 smithi049 ceph-mon[62991]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' 2024-03-18T23:08:20.130 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:08:19 smithi049 ceph-mon[62991]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' 2024-03-18T23:08:20.130 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:08:19 smithi049 ceph-mon[62991]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:08:20.130 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:08:19 smithi049 ceph-mon[62991]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:08:20.130 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:08:19 smithi049 ceph-mon[62991]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:08:20.130 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:08:19 smithi049 ceph-mon[62991]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:08:20.130 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:08:19 smithi049 ceph-mon[62991]: Adjusting osd_memory_target on smithi139 to 174.1M 2024-03-18T23:08:20.131 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:08:19 smithi049 ceph-mon[62991]: Unable to set osd_memory_target on smithi139 to 182618419: error parsing value: Value '182618419' is below minimum 939524096 2024-03-18T23:08:20.131 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:08:19 smithi049 ceph-mon[62991]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' 2024-03-18T23:08:20.131 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:08:19 smithi049 ceph-mon[62991]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' 2024-03-18T23:08:20.131 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:08:19 smithi049 ceph-mon[62991]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:08:20.131 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:08:19 smithi049 ceph-mon[62991]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:08:20.131 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:08:19 smithi049 ceph-mon[62991]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:08:20.131 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:08:19 smithi049 ceph-mon[62991]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:08:20.131 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:08:19 smithi049 ceph-mon[62991]: Adjusting osd_memory_target on smithi049 to 176.2M 2024-03-18T23:08:20.131 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:08:19 smithi049 ceph-mon[62991]: Unable to set osd_memory_target on smithi049 to 184762368: error parsing value: Value '184762368' is below minimum 939524096 2024-03-18T23:08:20.131 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:08:19 smithi049 ceph-mon[62991]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' 2024-03-18T23:08:20.131 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:08:19 smithi049 ceph-mon[62991]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' 2024-03-18T23:08:20.131 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:08:19 smithi049 ceph-mon[62991]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:08:20.131 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:08:19 smithi049 ceph-mon[62991]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:08:20.131 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:08:19 smithi049 ceph-mon[62991]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:08:20.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:08:19 smithi049 ceph-mon[62991]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:08:20.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:08:19 smithi049 ceph-mon[62991]: Adjusting osd_memory_target on smithi175 to 174.1M 2024-03-18T23:08:20.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:08:19 smithi049 ceph-mon[62991]: Unable to set osd_memory_target on smithi175 to 182618419: error parsing value: Value '182618419' is below minimum 939524096 2024-03-18T23:08:20.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:08:19 smithi049 ceph-mon[62991]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:08:20.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:08:19 smithi049 ceph-mon[62991]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:08:20.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:08:19 smithi049 ceph-mon[62991]: Updating smithi049:/etc/ceph/ceph.conf 2024-03-18T23:08:20.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:08:19 smithi049 ceph-mon[62991]: Updating smithi139:/etc/ceph/ceph.conf 2024-03-18T23:08:20.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:08:19 smithi049 ceph-mon[62991]: Updating smithi175:/etc/ceph/ceph.conf 2024-03-18T23:08:20.211 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:08:19 smithi175 ceph-mon[67352]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' 2024-03-18T23:08:20.212 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:08:19 smithi175 ceph-mon[67352]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' 2024-03-18T23:08:20.212 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:08:19 smithi175 ceph-mon[67352]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:08:20.212 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:08:19 smithi175 ceph-mon[67352]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:08:20.212 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:08:19 smithi175 ceph-mon[67352]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:08:20.212 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:08:19 smithi175 ceph-mon[67352]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:08:20.212 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:08:19 smithi175 ceph-mon[67352]: Adjusting osd_memory_target on smithi139 to 174.1M 2024-03-18T23:08:20.212 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:08:19 smithi175 ceph-mon[67352]: Unable to set osd_memory_target on smithi139 to 182618419: error parsing value: Value '182618419' is below minimum 939524096 2024-03-18T23:08:20.212 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:08:19 smithi175 ceph-mon[67352]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' 2024-03-18T23:08:20.212 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:08:19 smithi175 ceph-mon[67352]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' 2024-03-18T23:08:20.212 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:08:19 smithi175 ceph-mon[67352]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:08:20.212 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:08:19 smithi175 ceph-mon[67352]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:08:20.212 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:08:19 smithi175 ceph-mon[67352]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:08:20.212 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:08:19 smithi175 ceph-mon[67352]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:08:20.213 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:08:19 smithi175 ceph-mon[67352]: Adjusting osd_memory_target on smithi049 to 176.2M 2024-03-18T23:08:20.213 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:08:19 smithi175 ceph-mon[67352]: Unable to set osd_memory_target on smithi049 to 184762368: error parsing value: Value '184762368' is below minimum 939524096 2024-03-18T23:08:20.213 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:08:19 smithi175 ceph-mon[67352]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' 2024-03-18T23:08:20.213 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:08:19 smithi175 ceph-mon[67352]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' 2024-03-18T23:08:20.213 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:08:19 smithi175 ceph-mon[67352]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:08:20.213 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:08:19 smithi175 ceph-mon[67352]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:08:20.213 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:08:19 smithi175 ceph-mon[67352]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:08:20.213 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:08:19 smithi175 ceph-mon[67352]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:08:20.213 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:08:19 smithi175 ceph-mon[67352]: Adjusting osd_memory_target on smithi175 to 174.1M 2024-03-18T23:08:20.213 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:08:19 smithi175 ceph-mon[67352]: Unable to set osd_memory_target on smithi175 to 182618419: error parsing value: Value '182618419' is below minimum 939524096 2024-03-18T23:08:20.213 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:08:19 smithi175 ceph-mon[67352]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:08:20.213 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:08:19 smithi175 ceph-mon[67352]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:08:20.213 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:08:19 smithi175 ceph-mon[67352]: Updating smithi049:/etc/ceph/ceph.conf 2024-03-18T23:08:20.213 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:08:19 smithi175 ceph-mon[67352]: Updating smithi139:/etc/ceph/ceph.conf 2024-03-18T23:08:20.213 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:08:19 smithi175 ceph-mon[67352]: Updating smithi175:/etc/ceph/ceph.conf 2024-03-18T23:08:21.211 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:08:20 smithi175 ceph-mon[67352]: Updating smithi049:/var/lib/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-18T23:08:21.212 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:08:20 smithi175 ceph-mon[67352]: Updating smithi139:/var/lib/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-18T23:08:21.212 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:08:20 smithi175 ceph-mon[67352]: Updating smithi175:/var/lib/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-18T23:08:21.212 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:08:20 smithi175 ceph-mon[67352]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T23:08:21.212 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:08:20 smithi175 ceph-mon[67352]: Updating smithi049:/etc/ceph/ceph.client.admin.keyring 2024-03-18T23:08:21.212 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:08:20 smithi175 ceph-mon[67352]: Updating smithi139:/etc/ceph/ceph.client.admin.keyring 2024-03-18T23:08:21.212 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:08:20 smithi175 ceph-mon[67352]: Updating smithi175:/etc/ceph/ceph.client.admin.keyring 2024-03-18T23:08:21.212 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:08:20 smithi175 ceph-mon[67352]: Updating smithi049:/var/lib/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/config/ceph.client.admin.keyring 2024-03-18T23:08:21.212 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:08:20 smithi175 ceph-mon[67352]: Updating smithi175:/var/lib/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/config/ceph.client.admin.keyring 2024-03-18T23:08:21.212 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:08:20 smithi175 ceph-mon[67352]: Updating smithi139:/var/lib/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/config/ceph.client.admin.keyring 2024-03-18T23:08:21.212 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:08:20 smithi175 ceph-mon[67352]: mgrmap e23: x(active, since 4s), standbys: z, y 2024-03-18T23:08:21.212 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:08:20 smithi175 ceph-mon[67352]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' 2024-03-18T23:08:21.212 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:08:20 smithi175 ceph-mon[67352]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' 2024-03-18T23:08:21.212 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:08:20 smithi175 ceph-mon[67352]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' 2024-03-18T23:08:21.212 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:08:20 smithi175 ceph-mon[67352]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' 2024-03-18T23:08:21.213 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:08:20 smithi175 ceph-mon[67352]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' 2024-03-18T23:08:21.213 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:08:20 smithi175 ceph-mon[67352]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' 2024-03-18T23:08:21.213 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:08:20 smithi175 ceph-mon[67352]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' 2024-03-18T23:08:21.213 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:08:20 smithi175 ceph-mon[67352]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:08:21.213 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:08:20 smithi175 ceph-mon[67352]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:08:21.213 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:08:20 smithi175 ceph-mon[67352]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:08:21.213 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:08:20 smithi175 ceph-mon[67352]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' 2024-03-18T23:08:21.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:08:20 smithi139 ceph-mon[67282]: Updating smithi049:/var/lib/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-18T23:08:21.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:08:20 smithi139 ceph-mon[67282]: Updating smithi139:/var/lib/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-18T23:08:21.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:08:20 smithi139 ceph-mon[67282]: Updating smithi175:/var/lib/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-18T23:08:21.316 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:08:20 smithi139 ceph-mon[67282]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T23:08:21.316 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:08:20 smithi139 ceph-mon[67282]: Updating smithi049:/etc/ceph/ceph.client.admin.keyring 2024-03-18T23:08:21.316 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:08:20 smithi139 ceph-mon[67282]: Updating smithi139:/etc/ceph/ceph.client.admin.keyring 2024-03-18T23:08:21.316 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:08:20 smithi139 ceph-mon[67282]: Updating smithi175:/etc/ceph/ceph.client.admin.keyring 2024-03-18T23:08:21.316 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:08:20 smithi139 ceph-mon[67282]: Updating smithi049:/var/lib/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/config/ceph.client.admin.keyring 2024-03-18T23:08:21.316 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:08:20 smithi139 ceph-mon[67282]: Updating smithi175:/var/lib/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/config/ceph.client.admin.keyring 2024-03-18T23:08:21.316 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:08:20 smithi139 ceph-mon[67282]: Updating smithi139:/var/lib/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/config/ceph.client.admin.keyring 2024-03-18T23:08:21.316 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:08:20 smithi139 ceph-mon[67282]: mgrmap e23: x(active, since 4s), standbys: z, y 2024-03-18T23:08:21.316 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:08:20 smithi139 ceph-mon[67282]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' 2024-03-18T23:08:21.316 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:08:20 smithi139 ceph-mon[67282]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' 2024-03-18T23:08:21.316 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:08:20 smithi139 ceph-mon[67282]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' 2024-03-18T23:08:21.316 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:08:20 smithi139 ceph-mon[67282]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' 2024-03-18T23:08:21.316 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:08:20 smithi139 ceph-mon[67282]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' 2024-03-18T23:08:21.317 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:08:20 smithi139 ceph-mon[67282]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' 2024-03-18T23:08:21.317 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:08:20 smithi139 ceph-mon[67282]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' 2024-03-18T23:08:21.317 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:08:20 smithi139 ceph-mon[67282]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:08:21.317 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:08:20 smithi139 ceph-mon[67282]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:08:21.317 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:08:20 smithi139 ceph-mon[67282]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:08:21.317 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:08:20 smithi139 ceph-mon[67282]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' 2024-03-18T23:08:21.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:08:20 smithi049 ceph-mon[62991]: Updating smithi049:/var/lib/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-18T23:08:21.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:08:20 smithi049 ceph-mon[62991]: Updating smithi139:/var/lib/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-18T23:08:21.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:08:20 smithi049 ceph-mon[62991]: Updating smithi175:/var/lib/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-18T23:08:21.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:08:20 smithi049 ceph-mon[62991]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T23:08:21.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:08:20 smithi049 ceph-mon[62991]: Updating smithi049:/etc/ceph/ceph.client.admin.keyring 2024-03-18T23:08:21.381 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:08:20 smithi049 ceph-mon[62991]: Updating smithi139:/etc/ceph/ceph.client.admin.keyring 2024-03-18T23:08:21.381 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:08:20 smithi049 ceph-mon[62991]: Updating smithi175:/etc/ceph/ceph.client.admin.keyring 2024-03-18T23:08:21.381 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:08:20 smithi049 ceph-mon[62991]: Updating smithi049:/var/lib/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/config/ceph.client.admin.keyring 2024-03-18T23:08:21.381 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:08:20 smithi049 ceph-mon[62991]: Updating smithi175:/var/lib/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/config/ceph.client.admin.keyring 2024-03-18T23:08:21.381 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:08:20 smithi049 ceph-mon[62991]: Updating smithi139:/var/lib/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/config/ceph.client.admin.keyring 2024-03-18T23:08:21.381 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:08:20 smithi049 ceph-mon[62991]: mgrmap e23: x(active, since 4s), standbys: z, y 2024-03-18T23:08:21.381 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:08:20 smithi049 ceph-mon[62991]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' 2024-03-18T23:08:21.381 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:08:20 smithi049 ceph-mon[62991]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' 2024-03-18T23:08:21.381 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:08:20 smithi049 ceph-mon[62991]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' 2024-03-18T23:08:21.381 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:08:20 smithi049 ceph-mon[62991]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' 2024-03-18T23:08:21.381 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:08:20 smithi049 ceph-mon[62991]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' 2024-03-18T23:08:21.381 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:08:20 smithi049 ceph-mon[62991]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' 2024-03-18T23:08:21.381 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:08:20 smithi049 ceph-mon[62991]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' 2024-03-18T23:08:21.382 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:08:20 smithi049 ceph-mon[62991]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:08:21.382 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:08:20 smithi049 ceph-mon[62991]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:08:21.382 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:08:20 smithi049 ceph-mon[62991]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:08:21.382 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:08:20 smithi049 ceph-mon[62991]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' 2024-03-18T23:08:23.182 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:08:22 smithi049 ceph-mon[62991]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 1 op/s 2024-03-18T23:08:23.211 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:08:22 smithi175 ceph-mon[67352]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 1 op/s 2024-03-18T23:08:23.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:08:22 smithi139 ceph-mon[67282]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 1 op/s 2024-03-18T23:08:25.211 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:08:24 smithi175 ceph-mon[67352]: pgmap v7: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 132 KiB/s wr, 80 op/s 2024-03-18T23:08:25.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:08:24 smithi139 ceph-mon[67282]: pgmap v7: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 132 KiB/s wr, 80 op/s 2024-03-18T23:08:25.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:08:24 smithi049 ceph-mon[62991]: pgmap v7: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 132 KiB/s wr, 80 op/s 2024-03-18T23:08:27.211 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:08:26 smithi175 ceph-mon[67352]: pgmap v8: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 103 KiB/s wr, 62 op/s 2024-03-18T23:08:27.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:08:26 smithi139 ceph-mon[67282]: pgmap v8: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 103 KiB/s wr, 62 op/s 2024-03-18T23:08:27.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:08:26 smithi049 ceph-mon[62991]: pgmap v8: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 103 KiB/s wr, 62 op/s 2024-03-18T23:08:29.211 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:08:28 smithi175 ceph-mon[67352]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 84 KiB/s wr, 55 op/s 2024-03-18T23:08:29.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:08:28 smithi139 ceph-mon[67282]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 84 KiB/s wr, 55 op/s 2024-03-18T23:08:29.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:08:28 smithi049 ceph-mon[62991]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 84 KiB/s wr, 55 op/s 2024-03-18T23:08:30.831 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:30.828+0000 7f6480760640 1 -- 172.21.15.49:0/1412412856 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6478102a00 msgr2=0x7f6478102de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:30.831 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:30.828+0000 7f6480760640 1 --2- 172.21.15.49:0/1412412856 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6478102a00 0x7f6478102de0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f6474009900 tx=0x7f647402bc30 comp rx=0 tx=0).stop 2024-03-18T23:08:30.832 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:30.828+0000 7f6480760640 1 -- 172.21.15.49:0/1412412856 shutdown_connections 2024-03-18T23:08:30.832 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:30.828+0000 7f6480760640 1 --2- 172.21.15.49:0/1412412856 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f647810be70 0x7f647810e2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:30.832 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:30.828+0000 7f6480760640 1 --2- 172.21.15.49:0/1412412856 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6478103320 0x7f647810b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:30.832 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:30.828+0000 7f6480760640 1 --2- 172.21.15.49:0/1412412856 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6478102a00 0x7f6478102de0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:30.832 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:30.828+0000 7f6480760640 1 -- 172.21.15.49:0/1412412856 >> 172.21.15.49:0/1412412856 conn(0x7f6478077850 msgr2=0x7f6478077c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:08:30.832 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:30.828+0000 7f6480760640 1 -- 172.21.15.49:0/1412412856 shutdown_connections 2024-03-18T23:08:30.832 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:30.829+0000 7f6480760640 1 -- 172.21.15.49:0/1412412856 wait complete. 2024-03-18T23:08:30.832 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:30.829+0000 7f6480760640 1 Processor -- start 2024-03-18T23:08:30.832 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:30.829+0000 7f6480760640 1 -- start start 2024-03-18T23:08:30.832 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:30.829+0000 7f6480760640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6478102a00 0x7f6478100650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:30.832 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:30.829+0000 7f6480760640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6478103320 0x7f6478100b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:30.833 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:30.829+0000 7f6480760640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f647810be70 0x7f64781010d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:30.833 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:30.829+0000 7f6480760640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f6478111c40 con 0x7f647810be70 2024-03-18T23:08:30.833 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:30.829+0000 7f6480760640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f6478111ac0 con 0x7f6478103320 2024-03-18T23:08:30.833 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:30.829+0000 7f6480760640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f6478111dc0 con 0x7f6478102a00 2024-03-18T23:08:30.833 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:30.829+0000 7f647ecfd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f647810be70 0x7f64781010d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:30.833 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:30.830+0000 7f647e4fc640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6478102a00 0x7f6478100650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:30.834 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:30.830+0000 7f647dcfb640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6478103320 0x7f6478100b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:30.834 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:30.830+0000 7f647e4fc640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6478102a00 0x7f6478100650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.49:54360/0 (socket says 172.21.15.49:54360) 2024-03-18T23:08:30.834 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:30.830+0000 7f647e4fc640 1 -- 172.21.15.49:0/1329456510 learned_addr learned my addr 172.21.15.49:0/1329456510 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:08:30.834 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:30.830+0000 7f647ecfd640 1 -- 172.21.15.49:0/1329456510 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6478102a00 msgr2=0x7f6478100650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:30.834 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:30.830+0000 7f647ecfd640 1 --2- 172.21.15.49:0/1329456510 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6478102a00 0x7f6478100650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:30.834 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:30.830+0000 7f647ecfd640 1 -- 172.21.15.49:0/1329456510 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6478103320 msgr2=0x7f6478100b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:30.834 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:30.830+0000 7f647ecfd640 1 --2- 172.21.15.49:0/1329456510 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6478103320 0x7f6478100b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:30.834 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:30.830+0000 7f647ecfd640 1 -- 172.21.15.49:0/1329456510 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64781a6560 con 0x7f647810be70 2024-03-18T23:08:30.834 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:30.830+0000 7f647dcfb640 1 --2- 172.21.15.49:0/1329456510 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6478103320 0x7f6478100b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:08:30.834 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:30.830+0000 7f647ecfd640 1 --2- 172.21.15.49:0/1329456510 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f647810be70 0x7f64781010d0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f6470009fd0 tx=0x7f647000ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:08:30.834 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:30.830+0000 7f646f7fe640 1 -- 172.21.15.49:0/1329456510 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f647000ce20 con 0x7f647810be70 2024-03-18T23:08:30.834 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:30.830+0000 7f6480760640 1 -- 172.21.15.49:0/1329456510 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64781a6850 con 0x7f647810be70 2024-03-18T23:08:30.834 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:30.830+0000 7f646f7fe640 1 -- 172.21.15.49:0/1329456510 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6470005020 con 0x7f647810be70 2024-03-18T23:08:30.834 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:30.830+0000 7f646f7fe640 1 -- 172.21.15.49:0/1329456510 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6470010720 con 0x7f647810be70 2024-03-18T23:08:30.835 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:30.830+0000 7f6480760640 1 -- 172.21.15.49:0/1329456510 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64781a6da0 con 0x7f647810be70 2024-03-18T23:08:30.842 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:30.833+0000 7f646f7fe640 1 -- 172.21.15.49:0/1329456510 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 23) ==== 137466+0+0 (secure 0 0 0) 0x7f6470005540 con 0x7f647810be70 2024-03-18T23:08:30.842 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:30.833+0000 7f648341f740 1 Processor -- start 2024-03-18T23:08:30.842 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:30.833+0000 7f648341f740 1 -- start start 2024-03-18T23:08:30.842 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:30.834+0000 7f648341f740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f58d996ef0 0x55f58d9972d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:30.842 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:30.834+0000 7f648341f740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f58d997810 0x55f58d9a00a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:30.842 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:30.834+0000 7f648341f740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f58d9a05e0 0x55f58d9a29d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:30.842 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:30.834+0000 7f648341f740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55f58d8fabe0 con 0x55f58d996ef0 2024-03-18T23:08:30.842 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:30.834+0000 7f648341f740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55f58d8a3960 con 0x55f58d9a05e0 2024-03-18T23:08:30.842 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:30.834+0000 7f648341f740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55f58d8f6c80 con 0x55f58d997810 2024-03-18T23:08:30.842 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:30.834+0000 7f647dcfb640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f58d996ef0 0x55f58d9972d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:30.842 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:30.834+0000 7f647e4fc640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f58d997810 0x55f58d9a00a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:30.843 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:30.834+0000 7f647e4fc640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f58d997810 0x55f58d9a00a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.49:54374/0 (socket says 172.21.15.49:54374) 2024-03-18T23:08:30.843 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:30.834+0000 7f646f7fe640 1 --2- 172.21.15.49:0/1329456510 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f64640a2b30 0x7f64640a4ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:30.843 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:30.834+0000 7f647e4fc640 1 -- 172.21.15.49:0/4278422059 learned_addr learned my addr 172.21.15.49:0/4278422059 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:08:30.843 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:30.834+0000 7f647dcfb640 1 --2- 172.21.15.49:0/4278422059 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f58d9a05e0 0x55f58d9a29d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:30.843 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:30.834+0000 7f647ecfd640 1 --2- 172.21.15.49:0/1329456510 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f64640a2b30 0x7f64640a4ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:30.843 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:30.834+0000 7f646f7fe640 1 -- 172.21.15.49:0/1329456510 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f64700cf590 con 0x7f647810be70 2024-03-18T23:08:30.843 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:30.834+0000 7f647e4fc640 1 -- 172.21.15.49:0/4278422059 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f58d9a05e0 msgr2=0x55f58d9a29d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:08:30.843 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:30.834+0000 7f647e4fc640 1 --2- 172.21.15.49:0/4278422059 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f58d9a05e0 0x55f58d9a29d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:30.843 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:30.834+0000 7f647e4fc640 1 -- 172.21.15.49:0/4278422059 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f58d996ef0 msgr2=0x55f58d9972d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:08:30.843 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:30.834+0000 7f647e4fc640 1 --2- 172.21.15.49:0/4278422059 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f58d996ef0 0x55f58d9972d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:30.844 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:30.834+0000 7f647e4fc640 1 -- 172.21.15.49:0/4278422059 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f58d89ad20 con 0x55f58d997810 2024-03-18T23:08:30.844 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:30.834+0000 7f647dcfb640 1 --2- 172.21.15.49:0/4278422059 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f58d996ef0 0x55f58d9972d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:08:30.844 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:30.835+0000 7f647e4fc640 1 --2- 172.21.15.49:0/4278422059 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f58d997810 0x55f58d9a00a0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f6474036040 tx=0x7f6474009560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c1e16c59659e3416 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:08:30.844 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:30.835+0000 7f647ecfd640 1 --2- 172.21.15.49:0/1329456510 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f64640a2b30 0x7f64640a4ff0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f64700cccb0 tx=0x7f647004b040 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:08:30.844 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:30.835+0000 7f646cff9640 1 -- 172.21.15.49:0/4278422059 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6474042070 con 0x55f58d997810 2024-03-18T23:08:30.844 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:30.835+0000 7f646cff9640 1 -- 172.21.15.49:0/4278422059 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6474004030 con 0x55f58d997810 2024-03-18T23:08:30.844 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:30.835+0000 7f646cff9640 1 -- 172.21.15.49:0/4278422059 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6474004ee0 con 0x55f58d997810 2024-03-18T23:08:30.844 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:30.836+0000 7f648341f740 1 -- 172.21.15.49:0/4278422059 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f58d997810 msgr2=0x55f58d9a00a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:30.844 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:30.836+0000 7f648341f740 1 --2- 172.21.15.49:0/4278422059 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f58d997810 0x55f58d9a00a0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f6474036040 tx=0x7f6474009560 comp rx=0 tx=0).stop 2024-03-18T23:08:30.844 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:30.836+0000 7f648341f740 1 -- 172.21.15.49:0/4278422059 shutdown_connections 2024-03-18T23:08:30.845 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:30.836+0000 7f648341f740 1 --2- 172.21.15.49:0/4278422059 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f58d9a05e0 0x55f58d9a29d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:30.845 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:30.836+0000 7f648341f740 1 --2- 172.21.15.49:0/4278422059 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f58d997810 0x55f58d9a00a0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:30.845 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:30.836+0000 7f648341f740 1 --2- 172.21.15.49:0/4278422059 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f58d996ef0 0x55f58d9972d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:30.845 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:30.836+0000 7f648341f740 1 -- 172.21.15.49:0/4278422059 >> 172.21.15.49:0/4278422059 conn(0x55f58d991810 msgr2=0x55f58d991bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:08:30.845 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:30.836+0000 7f648341f740 1 -- 172.21.15.49:0/4278422059 shutdown_connections 2024-03-18T23:08:30.845 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:30.836+0000 7f648341f740 1 -- 172.21.15.49:0/4278422059 wait complete. 2024-03-18T23:08:30.845 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:30.836+0000 7f648341f740 1 -- start start 2024-03-18T23:08:30.845 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:30.836+0000 7f648341f740 1 Processor -- start 2024-03-18T23:08:30.845 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:30.836+0000 7f648341f740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f58d996ef0 0x55f58d99ff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:30.845 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:30.836+0000 7f648341f740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f58d997810 0x55f58d99d300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:30.845 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:30.836+0000 7f648341f740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f58d9a05e0 0x55f58d99d880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:30.845 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:30.836+0000 7f648341f740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55f58d9a31b0 con 0x55f58d9a05e0 2024-03-18T23:08:30.845 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:30.836+0000 7f648341f740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55f58d8b6db0 con 0x55f58d996ef0 2024-03-18T23:08:30.845 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:30.837+0000 7f648341f740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55f58d9a3330 con 0x55f58d997810 2024-03-18T23:08:30.846 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:30.837+0000 7f648341f740 10 client.0 _ll_register_callbacks cb 0x55f58d8bbe70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T23:08:30.846 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:30.837+0000 7f647dcfb640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f58d9a05e0 0x55f58d99d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:30.846 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:30.837+0000 7f647e4fc640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f58d997810 0x55f58d99d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:30.846 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:30.837+0000 7f647dcfb640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f58d996ef0 0x55f58d99ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:30.846 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:30.837+0000 7f647dcfb640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f58d996ef0 0x55f58d99ff70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.49:55258/0 (socket says 172.21.15.49:55258) 2024-03-18T23:08:30.846 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:30.837+0000 7f647dcfb640 1 -- 172.21.15.49:0/761521704 learned_addr learned my addr 172.21.15.49:0/761521704 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:08:30.846 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:30.837+0000 7f647e4fc640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f58d997810 0x55f58d99d300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.49:54378/0 (socket says 172.21.15.49:54378) 2024-03-18T23:08:30.846 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:30.837+0000 7f647dcfb640 1 -- 172.21.15.49:0/761521704 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f58d997810 msgr2=0x55f58d99d300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:08:30.847 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:30.837+0000 7f647dcfb640 1 --2- 172.21.15.49:0/761521704 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f58d997810 0x55f58d99d300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:30.847 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:30.837+0000 7f647dcfb640 1 -- 172.21.15.49:0/761521704 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f58d9a05e0 msgr2=0x55f58d99d880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:08:30.847 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:30.837+0000 7f647dcfb640 1 --2- 172.21.15.49:0/761521704 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f58d9a05e0 0x55f58d99d880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:30.847 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:30.837+0000 7f647dcfb640 1 -- 172.21.15.49:0/761521704 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f58d8b5c60 con 0x55f58d996ef0 2024-03-18T23:08:30.847 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:30.837+0000 7f647dcfb640 1 --2- 172.21.15.49:0/761521704 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f58d996ef0 0x55f58d99ff70 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f64781020e0 tx=0x7f6468005850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=69dd465454f5051d server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:08:30.847 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:30.837+0000 7f6461ffb640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-03-18T23:08:30.847 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:30.837+0000 7f6461ffb640 1 -- 172.21.15.49:0/761521704 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6468004320 con 0x55f58d996ef0 2024-03-18T23:08:30.848 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:30.837+0000 7f6461ffb640 1 -- 172.21.15.49:0/761521704 <== mon.2 v2:172.21.15.139:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f6468005020 con 0x55f58d996ef0 2024-03-18T23:08:30.848 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:30.837+0000 7f648341f740 1 -- 172.21.15.49:0/761521704 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6474004030 con 0x55f58d996ef0 2024-03-18T23:08:30.848 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:30.838+0000 7f6461ffb640 1 -- 172.21.15.49:0/761521704 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f646802b050 con 0x55f58d996ef0 2024-03-18T23:08:30.848 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:30.838+0000 7f648341f740 10 client.25283 fetch_fsmap learned FSMap version 28 2024-03-18T23:08:30.848 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:30.838+0000 7f648341f740 1 -- 172.21.15.49:0/761521704 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f58d60f210 con 0x55f58d996ef0 2024-03-18T23:08:30.848 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:30.838+0000 7f6461ffb640 1 -- 172.21.15.49:0/761521704 <== mon.2 v2:172.21.15.139:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6474004030 con 0x55f58d996ef0 2024-03-18T23:08:30.848 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:30.838+0000 7f6461ffb640 1 -- 172.21.15.49:0/761521704 <== mon.2 v2:172.21.15.139:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f646800ecf0 con 0x55f58d996ef0 2024-03-18T23:08:30.848 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:30.838+0000 7f648341f740 10 client.25283 fetch_fsmap finished waiting for FSMap version 28 2024-03-18T23:08:30.848 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:30.838+0000 7f648341f740 10 client.25283 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T23:08:30.848 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:30.838+0000 7f648341f740 20 client.25283 populate_metadata read hostname 'smithi049' 2024-03-18T23:08:30.848 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:30.838+0000 7f648341f740 1 --2- 172.21.15.49:0/761521704 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x55f58d9b0480 0x55f58d9b2870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:30.849 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:30.838+0000 7f648341f740 4 client.25283 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T23:08:30.849 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:30.838+0000 7f648341f740 1 -- 172.21.15.49:0/761521704 --> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f58d908120 con 0x55f58d9b0480 2024-03-18T23:08:30.849 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:30.838+0000 7f647e4fc640 1 --2- 172.21.15.49:0/761521704 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x55f58d9b0480 0x55f58d9b2870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:30.849 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:30.839+0000 7f647e4fc640 1 --2- 172.21.15.49:0/761521704 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x55f58d9b0480 0x55f58d9b2870 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=69ef1259f58110ec server_cookie=2d121d55cc402e64 in_seq=0 out_seq=0 2024-03-18T23:08:30.849 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:30.839+0000 7f6461ffb640 10 client.25283 ms_handle_connect on v2:172.21.15.139:6838/4280128209 2024-03-18T23:08:30.849 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:30.839+0000 7f6461ffb640 1 -- 172.21.15.49:0/761521704 <== mds.0 v2:172.21.15.139:6838/4280128209 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55f58d908120 con 0x55f58d9b0480 2024-03-18T23:08:30.849 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:30.839+0000 7f6461ffb640 10 client.25283 handle_command_reply: tid=0 2024-03-18T23:08:30.849 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:30.839+0000 7f648341f740 1 client.25283 shutdown 2024-03-18T23:08:30.849 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:30.840+0000 7f648341f740 1 -- 172.21.15.49:0/761521704 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f58d996ef0 msgr2=0x55f58d99ff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:30.849 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:30.840+0000 7f648341f740 1 --2- 172.21.15.49:0/761521704 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f58d996ef0 0x55f58d99ff70 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f64781020e0 tx=0x7f6468005850 comp rx=0 tx=0).stop 2024-03-18T23:08:30.849 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:30.840+0000 7f648341f740 1 -- 172.21.15.49:0/761521704 shutdown_connections 2024-03-18T23:08:30.849 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:30.840+0000 7f648341f740 1 --2- 172.21.15.49:0/761521704 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x55f58d9b0480 0x55f58d9b2870 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:30.850 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:30.840+0000 7f648341f740 1 --2- 172.21.15.49:0/761521704 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f58d9a05e0 0x55f58d99d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:30.850 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:30.840+0000 7f6461ffb640 0 client.25283 ms_handle_reset on v2:172.21.15.139:6838/4280128209 2024-03-18T23:08:30.850 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:30.840+0000 7f648341f740 1 --2- 172.21.15.49:0/761521704 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f58d997810 0x55f58d99d300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:30.850 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:30.840+0000 7f648341f740 1 --2- 172.21.15.49:0/761521704 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f58d996ef0 0x55f58d99ff70 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:30.850 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:30.840+0000 7f648341f740 1 -- 172.21.15.49:0/761521704 >> 172.21.15.49:0/761521704 conn(0x55f58d991810 msgr2=0x55f58d990ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:08:30.850 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:30.840+0000 7f648341f740 1 -- 172.21.15.49:0/761521704 shutdown_connections 2024-03-18T23:08:30.850 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:30.840+0000 7f648341f740 1 -- 172.21.15.49:0/761521704 wait complete. 2024-03-18T23:08:30.851 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:30.840+0000 7f648341f740 20 client.25283 trim_cache size 0 max 16384 2024-03-18T23:08:30.861 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:30.855+0000 7f648341f740 1 Processor -- start 2024-03-18T23:08:30.861 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:30.855+0000 7f648341f740 1 -- start start 2024-03-18T23:08:30.861 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:30.855+0000 7f648341f740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f58d996ef0 0x55f58d9abf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:30.861 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:30.855+0000 7f648341f740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f58d997810 0x55f58d9b8160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:30.862 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:30.855+0000 7f648341f740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f58d9a05e0 0x55f58d99fa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:30.862 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:30.855+0000 7f648341f740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f64781af4c0 con 0x55f58d997810 2024-03-18T23:08:30.862 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:30.855+0000 7f648341f740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f64781b1000 con 0x55f58d9a05e0 2024-03-18T23:08:30.862 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:30.855+0000 7f648341f740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f64781afb90 con 0x55f58d996ef0 2024-03-18T23:08:30.862 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:30.855+0000 7f647e4fc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f58d997810 0x55f58d9b8160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:30.862 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:30.855+0000 7f647dcfb640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f58d996ef0 0x55f58d9abf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:30.862 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:30.855+0000 7f647e4fc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f58d997810 0x55f58d9b8160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:41948/0 (socket says 172.21.15.49:41948) 2024-03-18T23:08:30.862 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:30.855+0000 7f647e4fc640 1 -- 172.21.15.49:0/4146696897 learned_addr learned my addr 172.21.15.49:0/4146696897 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:08:30.862 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:30.855+0000 7f647dcfb640 1 --2- 172.21.15.49:0/4146696897 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f58d9a05e0 0x55f58d99fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:30.862 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:30.856+0000 7f647e4fc640 1 -- 172.21.15.49:0/4146696897 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f58d996ef0 msgr2=0x55f58d9abf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:08:30.862 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:30.856+0000 7f647e4fc640 1 --2- 172.21.15.49:0/4146696897 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f58d996ef0 0x55f58d9abf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:30.862 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:30.856+0000 7f647e4fc640 1 -- 172.21.15.49:0/4146696897 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f58d9a05e0 msgr2=0x55f58d99fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:08:30.862 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:30.856+0000 7f647e4fc640 1 --2- 172.21.15.49:0/4146696897 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f58d9a05e0 0x55f58d99fa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:30.863 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:30.856+0000 7f647e4fc640 1 -- 172.21.15.49:0/4146696897 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f58d6db620 con 0x55f58d997810 2024-03-18T23:08:30.863 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:30.856+0000 7f647dcfb640 1 --2- 172.21.15.49:0/4146696897 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f58d996ef0 0x55f58d9abf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:08:30.863 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:30.856+0000 7f647e4fc640 1 --2- 172.21.15.49:0/4146696897 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f58d997810 0x55f58d9b8160 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f6474004940 tx=0x7f6474004970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a822699235fbfa8 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:08:30.863 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:30.856+0000 7f6463fff640 1 -- 172.21.15.49:0/4146696897 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6474042040 con 0x55f58d997810 2024-03-18T23:08:30.863 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:30.856+0000 7f6463fff640 1 -- 172.21.15.49:0/4146696897 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6474050d90 con 0x55f58d997810 2024-03-18T23:08:30.863 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:30.856+0000 7f6463fff640 1 -- 172.21.15.49:0/4146696897 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6474050020 con 0x55f58d997810 2024-03-18T23:08:30.863 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:30.857+0000 7f648341f740 1 -- 172.21.15.49:0/4146696897 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f58d997810 msgr2=0x55f58d9b8160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:30.863 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:30.857+0000 7f648341f740 1 --2- 172.21.15.49:0/4146696897 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f58d997810 0x55f58d9b8160 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f6474004940 tx=0x7f6474004970 comp rx=0 tx=0).stop 2024-03-18T23:08:30.863 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:30.857+0000 7f648341f740 1 -- 172.21.15.49:0/4146696897 shutdown_connections 2024-03-18T23:08:30.863 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:30.857+0000 7f648341f740 1 --2- 172.21.15.49:0/4146696897 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f58d9a05e0 0x55f58d99fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:30.864 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:30.857+0000 7f648341f740 1 --2- 172.21.15.49:0/4146696897 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f58d997810 0x55f58d9b8160 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:30.864 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:30.857+0000 7f648341f740 1 --2- 172.21.15.49:0/4146696897 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f58d996ef0 0x55f58d9abf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:30.864 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:30.857+0000 7f648341f740 1 -- 172.21.15.49:0/4146696897 >> 172.21.15.49:0/4146696897 conn(0x55f58d991810 msgr2=0x55f58d990ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:08:30.864 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:30.857+0000 7f648341f740 1 -- 172.21.15.49:0/4146696897 shutdown_connections 2024-03-18T23:08:30.864 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:30.857+0000 7f648341f740 1 -- 172.21.15.49:0/4146696897 wait complete. 2024-03-18T23:08:30.864 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:30.857+0000 7f648341f740 1 -- start start 2024-03-18T23:08:30.864 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:30.857+0000 7f648341f740 1 Processor -- start 2024-03-18T23:08:30.864 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:30.857+0000 7f648341f740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f58d996ef0 0x55f58d9a3fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:30.864 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:30.857+0000 7f648341f740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f58d997810 0x55f58d9a4520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:30.864 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:30.857+0000 7f648341f740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f58d9a05e0 0x55f58d9a4a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:30.864 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:30.857+0000 7f648341f740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f64781b0930 con 0x55f58d9a05e0 2024-03-18T23:08:30.865 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:30.857+0000 7f648341f740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f64781b0260 con 0x55f58d996ef0 2024-03-18T23:08:30.865 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:30.857+0000 7f648341f740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f64781ad4a0 con 0x55f58d997810 2024-03-18T23:08:30.865 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:30.857+0000 7f648341f740 10 client.0 _ll_register_callbacks cb 0x7f6468013560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T23:08:30.865 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:30.858+0000 7f647dcfb640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f58d996ef0 0x55f58d9a3fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:30.865 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:30.858+0000 7f647dcfb640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f58d9a05e0 0x55f58d9a4a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:30.865 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:30.858+0000 7f647e4fc640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f58d997810 0x55f58d9a4520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:30.865 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:30.858+0000 7f647e4fc640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f58d997810 0x55f58d9a4520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.49:54400/0 (socket says 172.21.15.49:54400) 2024-03-18T23:08:30.865 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:30.858+0000 7f647dcfb640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f58d9a05e0 0x55f58d9a4a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:41964/0 (socket says 172.21.15.49:41964) 2024-03-18T23:08:30.865 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:30.858+0000 7f647e4fc640 1 -- 172.21.15.49:0/3465162994 learned_addr learned my addr 172.21.15.49:0/3465162994 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:08:30.865 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:30.858+0000 7f647dcfb640 1 -- 172.21.15.49:0/3465162994 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f58d997810 msgr2=0x55f58d9a4520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:08:30.866 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:30.858+0000 7f647dcfb640 1 --2- 172.21.15.49:0/3465162994 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f58d997810 0x55f58d9a4520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:30.866 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:30.858+0000 7f647dcfb640 1 -- 172.21.15.49:0/3465162994 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f58d996ef0 msgr2=0x55f58d9a3fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:08:30.866 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:30.858+0000 7f647dcfb640 1 --2- 172.21.15.49:0/3465162994 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f58d996ef0 0x55f58d9a3fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:30.866 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:30.858+0000 7f647dcfb640 1 -- 172.21.15.49:0/3465162994 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f58da158f0 con 0x55f58d9a05e0 2024-03-18T23:08:30.866 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:30.858+0000 7f647dcfb640 1 --2- 172.21.15.49:0/3465162994 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f58d9a05e0 0x55f58d9a4a60 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f6468001eb0 tx=0x7f6468000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=457b9540531a325e server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:08:30.866 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:30.858+0000 7f6461ffb640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-18T23:08:30.866 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:30.858+0000 7f6461ffb640 1 -- 172.21.15.49:0/3465162994 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6468004c10 con 0x55f58d9a05e0 2024-03-18T23:08:30.866 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:30.858+0000 7f6461ffb640 1 -- 172.21.15.49:0/3465162994 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f6468013030 con 0x55f58d9a05e0 2024-03-18T23:08:30.866 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:30.858+0000 7f648341f740 1 -- 172.21.15.49:0/3465162994 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6474050d90 con 0x55f58d9a05e0 2024-03-18T23:08:30.866 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:30.858+0000 7f6461ffb640 1 -- 172.21.15.49:0/3465162994 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f6468043050 con 0x55f58d9a05e0 2024-03-18T23:08:30.866 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:30.858+0000 7f648341f740 10 client.15471 fetch_fsmap learned FSMap version 28 2024-03-18T23:08:30.866 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:30.858+0000 7f648341f740 1 -- 172.21.15.49:0/3465162994 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f58da16be0 con 0x55f58d9a05e0 2024-03-18T23:08:30.866 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:30.859+0000 7f6461ffb640 1 -- 172.21.15.49:0/3465162994 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6474050d90 con 0x55f58d9a05e0 2024-03-18T23:08:30.866 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:30.859+0000 7f6461ffb640 1 -- 172.21.15.49:0/3465162994 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f6468004f60 con 0x55f58d9a05e0 2024-03-18T23:08:30.867 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:30.859+0000 7f648341f740 10 client.15471 fetch_fsmap finished waiting for FSMap version 28 2024-03-18T23:08:30.867 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:30.859+0000 7f648341f740 10 client.15471 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T23:08:30.867 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:30.859+0000 7f648341f740 20 client.15471 populate_metadata read hostname 'smithi049' 2024-03-18T23:08:30.867 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:30.859+0000 7f648341f740 1 --2- 172.21.15.49:0/3465162994 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x55f58da171f0 0x55f58da195e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:30.867 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:30.859+0000 7f648341f740 4 client.15471 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T23:08:30.867 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:30.859+0000 7f648341f740 1 -- 172.21.15.49:0/3465162994 --> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f58d902470 con 0x55f58da171f0 2024-03-18T23:08:30.867 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:30.859+0000 7f647e4fc640 1 --2- 172.21.15.49:0/3465162994 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x55f58da171f0 0x55f58da195e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:30.867 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:30.859+0000 7f647e4fc640 1 --2- 172.21.15.49:0/3465162994 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x55f58da171f0 0x55f58da195e0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dfef7160423e9d96 server_cookie=1b50958221cd5ce5 in_seq=0 out_seq=0 2024-03-18T23:08:30.867 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:30.859+0000 7f6461ffb640 10 client.15471 ms_handle_connect on v2:172.21.15.139:6838/4280128209 2024-03-18T23:08:30.871 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:30.869+0000 7f6461ffb640 1 -- 172.21.15.49:0/3465162994 <== mds.0 v2:172.21.15.139:6838/4280128209 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f58d902470 con 0x55f58da171f0 2024-03-18T23:08:30.871 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:30.869+0000 7f6461ffb640 10 client.15471 handle_command_reply: tid=0 2024-03-18T23:08:30.871 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:30.869+0000 7f648341f740 1 client.15471 shutdown 2024-03-18T23:08:30.871 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:30.869+0000 7f648341f740 1 -- 172.21.15.49:0/3465162994 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f58d9a05e0 msgr2=0x55f58d9a4a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:30.871 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:30.869+0000 7f648341f740 1 --2- 172.21.15.49:0/3465162994 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f58d9a05e0 0x55f58d9a4a60 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f6468001eb0 tx=0x7f6468000bc0 comp rx=0 tx=0).stop 2024-03-18T23:08:30.871 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:30.869+0000 7f648341f740 1 -- 172.21.15.49:0/3465162994 shutdown_connections 2024-03-18T23:08:30.871 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:30.869+0000 7f648341f740 1 --2- 172.21.15.49:0/3465162994 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x55f58da171f0 0x55f58da195e0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:30.871 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:30.869+0000 7f648341f740 1 --2- 172.21.15.49:0/3465162994 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f58d9a05e0 0x55f58d9a4a60 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:30.871 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:30.869+0000 7f648341f740 1 --2- 172.21.15.49:0/3465162994 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f58d997810 0x55f58d9a4520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:30.871 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:30.869+0000 7f6461ffb640 0 client.15471 ms_handle_reset on v2:172.21.15.139:6838/4280128209 2024-03-18T23:08:30.872 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:30.869+0000 7f648341f740 1 --2- 172.21.15.49:0/3465162994 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f58d996ef0 0x55f58d9a3fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:30.872 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:30.870+0000 7f648341f740 1 -- 172.21.15.49:0/3465162994 >> 172.21.15.49:0/3465162994 conn(0x55f58d991810 msgr2=0x55f58d997d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:08:30.872 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:30.870+0000 7f648341f740 1 -- 172.21.15.49:0/3465162994 shutdown_connections 2024-03-18T23:08:30.872 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:30.870+0000 7f648341f740 1 -- 172.21.15.49:0/3465162994 wait complete. 2024-03-18T23:08:30.872 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:30.870+0000 7f648341f740 20 client.15471 trim_cache size 0 max 16384 2024-03-18T23:08:30.872 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:{ 2024-03-18T23:08:30.873 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "message": "", 2024-03-18T23:08:30.873 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "return_code": 0 2024-03-18T23:08:30.873 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:} 2024-03-18T23:08:30.874 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:30.870+0000 7f6480760640 1 -- 172.21.15.49:0/1329456510 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f64640a2b30 msgr2=0x7f64640a4ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:30.874 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:30.870+0000 7f6480760640 1 --2- 172.21.15.49:0/1329456510 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f64640a2b30 0x7f64640a4ff0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f64700cccb0 tx=0x7f647004b040 comp rx=0 tx=0).stop 2024-03-18T23:08:30.874 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:30.870+0000 7f6480760640 1 -- 172.21.15.49:0/1329456510 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f647810be70 msgr2=0x7f64781010d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:30.874 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:30.870+0000 7f6480760640 1 --2- 172.21.15.49:0/1329456510 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f647810be70 0x7f64781010d0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f6470009fd0 tx=0x7f647000ef10 comp rx=0 tx=0).stop 2024-03-18T23:08:30.874 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:30.870+0000 7f6480760640 1 -- 172.21.15.49:0/1329456510 shutdown_connections 2024-03-18T23:08:30.874 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:30.870+0000 7f6480760640 1 --2- 172.21.15.49:0/1329456510 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f64640a2b30 0x7f64640a4ff0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:30.875 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:30.870+0000 7f6480760640 1 --2- 172.21.15.49:0/1329456510 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f647810be70 0x7f64781010d0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:30.875 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:30.870+0000 7f6480760640 1 --2- 172.21.15.49:0/1329456510 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6478103320 0x7f6478100b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:30.875 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:30.870+0000 7f6480760640 1 --2- 172.21.15.49:0/1329456510 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6478102a00 0x7f6478100650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:30.875 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:30.870+0000 7f6480760640 1 -- 172.21.15.49:0/1329456510 >> 172.21.15.49:0/1329456510 conn(0x7f6478077850 msgr2=0x7f64780fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:08:30.875 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:30.871+0000 7f6480760640 1 -- 172.21.15.49:0/1329456510 shutdown_connections 2024-03-18T23:08:30.875 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:30.871+0000 7f6480760640 1 -- 172.21.15.49:0/1329456510 wait complete. 2024-03-18T23:08:31.211 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:08:30 smithi175 ceph-mon[67352]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 77 KiB/s wr, 50 op/s 2024-03-18T23:08:31.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:08:30 smithi139 ceph-mon[67282]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 77 KiB/s wr, 50 op/s 2024-03-18T23:08:31.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:08:30 smithi049 ceph-mon[62991]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 77 KiB/s wr, 50 op/s 2024-03-18T23:08:33.211 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:08:32 smithi175 ceph-mon[67352]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 77 KiB/s wr, 51 op/s 2024-03-18T23:08:33.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:08:32 smithi139 ceph-mon[67282]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 77 KiB/s wr, 51 op/s 2024-03-18T23:08:33.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:08:32 smithi049 ceph-mon[62991]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 77 KiB/s wr, 51 op/s 2024-03-18T23:08:35.211 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:08:34 smithi175 ceph-mon[67352]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 79 KiB/s wr, 59 op/s 2024-03-18T23:08:35.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:08:34 smithi139 ceph-mon[67282]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 79 KiB/s wr, 59 op/s 2024-03-18T23:08:35.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:08:34 smithi049 ceph-mon[62991]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 79 KiB/s wr, 59 op/s 2024-03-18T23:08:37.211 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:08:36 smithi175 ceph-mon[67352]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.6 KiB/s wr, 12 op/s 2024-03-18T23:08:37.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:08:36 smithi139 ceph-mon[67282]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.6 KiB/s wr, 12 op/s 2024-03-18T23:08:37.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:08:36 smithi049 ceph-mon[62991]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.6 KiB/s wr, 12 op/s 2024-03-18T23:08:39.211 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:08:38 smithi175 ceph-mon[67352]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.6 KiB/s wr, 16 op/s 2024-03-18T23:08:39.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:08:38 smithi139 ceph-mon[67282]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.6 KiB/s wr, 16 op/s 2024-03-18T23:08:39.381 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:08:38 smithi049 ceph-mon[62991]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.6 KiB/s wr, 16 op/s 2024-03-18T23:08:41.211 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:08:40 smithi175 ceph-mon[67352]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.6 KiB/s wr, 12 op/s 2024-03-18T23:08:41.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:08:40 smithi139 ceph-mon[67282]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.6 KiB/s wr, 12 op/s 2024-03-18T23:08:41.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:08:40 smithi049 ceph-mon[62991]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.6 KiB/s wr, 12 op/s 2024-03-18T23:08:43.211 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:08:42 smithi175 ceph-mon[67352]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.6 KiB/s wr, 13 op/s 2024-03-18T23:08:43.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:08:42 smithi139 ceph-mon[67282]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.6 KiB/s wr, 13 op/s 2024-03-18T23:08:43.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:08:42 smithi049 ceph-mon[62991]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.6 KiB/s wr, 13 op/s 2024-03-18T23:08:43.976 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:43.975+0000 7f24d8a44640 1 -- 172.21.15.49:0/2444011773 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f24d41066f0 msgr2=0x7f24d4106ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:43.976 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:43.975+0000 7f24d8a44640 1 --2- 172.21.15.49:0/2444011773 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f24d41066f0 0x7f24d4106ad0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f24c4009cd0 tx=0x7f24c402c0f0 comp rx=0 tx=0).stop 2024-03-18T23:08:43.976 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:43.975+0000 7f24d8a44640 1 -- 172.21.15.49:0/2444011773 shutdown_connections 2024-03-18T23:08:43.976 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:43.975+0000 7f24d8a44640 1 --2- 172.21.15.49:0/2444011773 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f24d41010a0 0x7f24d410d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:43.976 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:43.975+0000 7f24d8a44640 1 --2- 172.21.15.49:0/2444011773 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f24d41006e0 0x7f24d4100b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:43.976 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:43.975+0000 7f24d8a44640 1 --2- 172.21.15.49:0/2444011773 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f24d41066f0 0x7f24d4106ad0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:43.976 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:43.975+0000 7f24d8a44640 1 -- 172.21.15.49:0/2444011773 >> 172.21.15.49:0/2444011773 conn(0x7f24d4077800 msgr2=0x7f24d4077c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:08:43.976 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:43.975+0000 7f24d8a44640 1 -- 172.21.15.49:0/2444011773 shutdown_connections 2024-03-18T23:08:43.976 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:43.976+0000 7f24d8a44640 1 -- 172.21.15.49:0/2444011773 wait complete. 2024-03-18T23:08:43.979 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:43.976+0000 7f24d8a44640 1 Processor -- start 2024-03-18T23:08:43.979 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:43.976+0000 7f24d8a44640 1 -- start start 2024-03-18T23:08:43.979 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:43.977+0000 7f24d8a44640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f24d41006e0 0x7f24d4198050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:43.979 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:43.977+0000 7f24d8a44640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f24d41010a0 0x7f24d4198590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:43.979 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:43.977+0000 7f24d8a44640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f24d41066f0 0x7f24d419c910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:43.979 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:43.977+0000 7f24d8a44640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f24d4110ac0 con 0x7f24d41006e0 2024-03-18T23:08:43.979 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:43.977+0000 7f24d8a44640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f24d4110940 con 0x7f24d41066f0 2024-03-18T23:08:43.979 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:43.977+0000 7f24d8a44640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f24d4110c40 con 0x7f24d41010a0 2024-03-18T23:08:43.979 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:43.977+0000 7f24d259c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f24d41006e0 0x7f24d4198050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:43.980 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:43.977+0000 7f24d259c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f24d41006e0 0x7f24d4198050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:54134/0 (socket says 172.21.15.49:54134) 2024-03-18T23:08:43.980 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:43.977+0000 7f24d259c640 1 -- 172.21.15.49:0/941656878 learned_addr learned my addr 172.21.15.49:0/941656878 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:08:43.980 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:43.977+0000 7f24d1d9b640 1 --2- 172.21.15.49:0/941656878 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f24d41010a0 0x7f24d4198590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:43.980 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:43.977+0000 7f24d2d9d640 1 --2- 172.21.15.49:0/941656878 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f24d41066f0 0x7f24d419c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:43.980 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:43.977+0000 7f24d259c640 1 -- 172.21.15.49:0/941656878 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f24d41010a0 msgr2=0x7f24d4198590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:43.980 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:43.977+0000 7f24d259c640 1 --2- 172.21.15.49:0/941656878 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f24d41010a0 0x7f24d4198590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:43.980 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:43.977+0000 7f24d259c640 1 -- 172.21.15.49:0/941656878 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f24d41066f0 msgr2=0x7f24d419c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:43.980 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:43.977+0000 7f24d259c640 1 --2- 172.21.15.49:0/941656878 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f24d41066f0 0x7f24d419c910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:43.980 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:43.977+0000 7f24d259c640 1 -- 172.21.15.49:0/941656878 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24d419d040 con 0x7f24d41006e0 2024-03-18T23:08:43.980 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:43.977+0000 7f24d2d9d640 1 --2- 172.21.15.49:0/941656878 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f24d41066f0 0x7f24d419c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T23:08:43.980 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:43.977+0000 7f24d1d9b640 1 --2- 172.21.15.49:0/941656878 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f24d41010a0 0x7f24d4198590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:08:43.980 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:43.978+0000 7f24d259c640 1 --2- 172.21.15.49:0/941656878 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f24d41006e0 0x7f24d4198050 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f24c4004af0 tx=0x7f24c4032ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:08:43.980 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:43.978+0000 7f24cb7fe640 1 -- 172.21.15.49:0/941656878 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24c4041020 con 0x7f24d41006e0 2024-03-18T23:08:43.980 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:43.978+0000 7f24d8a44640 1 -- 172.21.15.49:0/941656878 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24d419d2d0 con 0x7f24d41006e0 2024-03-18T23:08:43.981 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:43.978+0000 7f24cb7fe640 1 -- 172.21.15.49:0/941656878 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f24c4034cd0 con 0x7f24d41006e0 2024-03-18T23:08:43.981 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:43.978+0000 7f24cb7fe640 1 -- 172.21.15.49:0/941656878 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24c4039040 con 0x7f24d41006e0 2024-03-18T23:08:43.981 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:43.978+0000 7f24d8a44640 1 -- 172.21.15.49:0/941656878 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24d41a4bc0 con 0x7f24d41006e0 2024-03-18T23:08:43.982 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:43.981+0000 7f24cb7fe640 1 -- 172.21.15.49:0/941656878 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 23) ==== 137466+0+0 (secure 0 0 0) 0x7f24c4044020 con 0x7f24d41006e0 2024-03-18T23:08:43.982 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:43.981+0000 7f24db62e740 1 Processor -- start 2024-03-18T23:08:43.983 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:43.982+0000 7f24db62e740 1 -- start start 2024-03-18T23:08:43.983 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:43.982+0000 7f24db62e740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5625b68e9ef0 0x5625b68ea2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:43.983 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:43.982+0000 7f24db62e740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5625b68ea810 0x5625b68f30a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:43.983 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:43.982+0000 7f24db62e740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5625b68f35e0 0x5625b68f59d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:43.983 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:43.982+0000 7f24db62e740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5625b684dbe0 con 0x5625b68f35e0 2024-03-18T23:08:43.983 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:43.982+0000 7f24db62e740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x5625b67f6960 con 0x5625b68e9ef0 2024-03-18T23:08:43.983 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:43.982+0000 7f24db62e740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5625b6849c80 con 0x5625b68ea810 2024-03-18T23:08:43.983 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:43.982+0000 7f24cb7fe640 1 --2- 172.21.15.49:0/941656878 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f24c00a2b30 0x7f24c00a4ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:43.983 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:43.982+0000 7f24d2d9d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5625b68e9ef0 0x5625b68ea2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:43.983 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:43.982+0000 7f24d259c640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5625b68ea810 0x5625b68f30a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:43.983 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:43.982+0000 7f24d1d9b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5625b68f35e0 0x5625b68f59d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:43.983 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:43.982+0000 7f24d2d9d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5625b68e9ef0 0x5625b68ea2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.49:38410/0 (socket says 172.21.15.49:38410) 2024-03-18T23:08:43.983 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:43.982+0000 7f24d2d9d640 1 -- 172.21.15.49:0/250771240 learned_addr learned my addr 172.21.15.49:0/250771240 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:08:43.983 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:43.982+0000 7f24d2d9d640 1 --2- 172.21.15.49:0/941656878 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f24c00a2b30 0x7f24c00a4ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:43.984 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:43.983+0000 7f24d1d9b640 1 -- 172.21.15.49:0/250771240 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5625b68ea810 msgr2=0x5625b68f30a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:08:43.984 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:43.983+0000 7f24d1d9b640 1 --2- 172.21.15.49:0/250771240 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5625b68ea810 0x5625b68f30a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:43.984 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:43.983+0000 7f24d1d9b640 1 -- 172.21.15.49:0/250771240 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5625b68e9ef0 msgr2=0x5625b68ea2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:08:43.984 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:43.983+0000 7f24d1d9b640 1 --2- 172.21.15.49:0/250771240 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5625b68e9ef0 0x5625b68ea2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:43.989 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:43.983+0000 7f24d1d9b640 1 -- 172.21.15.49:0/250771240 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5625b67edd20 con 0x5625b68f35e0 2024-03-18T23:08:43.989 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:43.983+0000 7f24cb7fe640 1 -- 172.21.15.49:0/941656878 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f24c40f0b20 con 0x7f24d41006e0 2024-03-18T23:08:43.989 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:43.983+0000 7f24d1d9b640 1 --2- 172.21.15.49:0/250771240 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5625b68f35e0 0x5625b68f59d0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f24bc009fa0 tx=0x7f24bc0118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2eaec0d76b32b7fa server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:08:43.989 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:43.983+0000 7f24d2d9d640 1 --2- 172.21.15.49:0/941656878 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f24c00a2b30 0x7f24c00a4ff0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f24cc0095f0 tx=0x7f24cc005f90 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:08:43.989 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:43.983+0000 7f24c8ff9640 1 -- 172.21.15.49:0/250771240 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24bc004030 con 0x5625b68f35e0 2024-03-18T23:08:43.989 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:43.983+0000 7f24c8ff9640 1 -- 172.21.15.49:0/250771240 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f24bc005500 con 0x5625b68f35e0 2024-03-18T23:08:43.989 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:43.983+0000 7f24c8ff9640 1 -- 172.21.15.49:0/250771240 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24bc004d80 con 0x5625b68f35e0 2024-03-18T23:08:43.990 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:43.984+0000 7f24db62e740 1 -- 172.21.15.49:0/250771240 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5625b68f35e0 msgr2=0x5625b68f59d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:43.990 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:43.984+0000 7f24db62e740 1 --2- 172.21.15.49:0/250771240 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5625b68f35e0 0x5625b68f59d0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f24bc009fa0 tx=0x7f24bc0118d0 comp rx=0 tx=0).stop 2024-03-18T23:08:43.990 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:43.984+0000 7f24db62e740 1 -- 172.21.15.49:0/250771240 shutdown_connections 2024-03-18T23:08:43.990 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:43.984+0000 7f24db62e740 1 --2- 172.21.15.49:0/250771240 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5625b68f35e0 0x5625b68f59d0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:43.990 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:43.984+0000 7f24db62e740 1 --2- 172.21.15.49:0/250771240 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5625b68ea810 0x5625b68f30a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:43.990 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:43.984+0000 7f24db62e740 1 --2- 172.21.15.49:0/250771240 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5625b68e9ef0 0x5625b68ea2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:43.990 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:43.984+0000 7f24db62e740 1 -- 172.21.15.49:0/250771240 >> 172.21.15.49:0/250771240 conn(0x5625b68e4810 msgr2=0x5625b68e4bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:08:43.990 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:43.984+0000 7f24db62e740 1 -- 172.21.15.49:0/250771240 shutdown_connections 2024-03-18T23:08:43.990 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:43.984+0000 7f24db62e740 1 -- 172.21.15.49:0/250771240 wait complete. 2024-03-18T23:08:43.990 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:43.984+0000 7f24db62e740 1 -- start start 2024-03-18T23:08:43.990 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:43.985+0000 7f24db62e740 1 Processor -- start 2024-03-18T23:08:43.990 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:43.985+0000 7f24db62e740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5625b68e9ef0 0x5625b68f3070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:43.990 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:43.985+0000 7f24db62e740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5625b68ea810 0x5625b68f03d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:43.991 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:43.985+0000 7f24db62e740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5625b68f0a20 0x5625b68f0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:43.991 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:43.985+0000 7f24db62e740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5625b68f6210 con 0x5625b68e9ef0 2024-03-18T23:08:43.991 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:43.985+0000 7f24db62e740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x5625b6809db0 con 0x5625b68ea810 2024-03-18T23:08:43.991 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:43.985+0000 7f24db62e740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5625b68f6390 con 0x5625b68f0a20 2024-03-18T23:08:43.991 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:43.985+0000 7f24db62e740 10 client.0 _ll_register_callbacks cb 0x5625b680ee70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T23:08:43.991 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:43.985+0000 7f24d259c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5625b68e9ef0 0x5625b68f3070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:43.991 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:43.985+0000 7f24d1d9b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5625b68ea810 0x5625b68f03d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:43.991 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:43.985+0000 7f24d1d9b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5625b68ea810 0x5625b68f03d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.49:38418/0 (socket says 172.21.15.49:38418) 2024-03-18T23:08:43.991 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:43.985+0000 7f24d259c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5625b68e9ef0 0x5625b68f3070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:54152/0 (socket says 172.21.15.49:54152) 2024-03-18T23:08:43.991 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:43.985+0000 7f24d2d9d640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5625b68f0a20 0x5625b68f0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:43.991 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:43.985+0000 7f24d1d9b640 1 -- 172.21.15.49:0/185193333 learned_addr learned my addr 172.21.15.49:0/185193333 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:08:43.991 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:43.985+0000 7f24d1d9b640 1 -- 172.21.15.49:0/185193333 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5625b68f0a20 msgr2=0x5625b68f0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:08:43.991 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:43.985+0000 7f24d1d9b640 1 --2- 172.21.15.49:0/185193333 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5625b68f0a20 0x5625b68f0ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:43.991 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:43.985+0000 7f24d1d9b640 1 -- 172.21.15.49:0/185193333 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5625b68e9ef0 msgr2=0x5625b68f3070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:08:43.991 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:43.985+0000 7f24d1d9b640 1 --2- 172.21.15.49:0/185193333 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5625b68e9ef0 0x5625b68f3070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:43.992 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:43.985+0000 7f24d1d9b640 1 -- 172.21.15.49:0/185193333 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5625b6808c60 con 0x5625b68ea810 2024-03-18T23:08:43.992 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:43.985+0000 7f24d259c640 1 --2- 172.21.15.49:0/185193333 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5625b68e9ef0 0x5625b68f3070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:08:43.992 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:43.986+0000 7f24d1d9b640 1 --2- 172.21.15.49:0/185193333 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5625b68ea810 0x5625b68f03d0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f24bc000c00 tx=0x7f24bc009310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=860ac04abf840a12 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:08:43.992 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:43.986+0000 7f24b9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-03-18T23:08:43.992 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:43.986+0000 7f24b9ffb640 1 -- 172.21.15.49:0/185193333 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24bc01d050 con 0x5625b68ea810 2024-03-18T23:08:43.992 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:43.986+0000 7f24b9ffb640 1 -- 172.21.15.49:0/185193333 <== mon.2 v2:172.21.15.139:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f24bc003800 con 0x5625b68ea810 2024-03-18T23:08:43.992 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:43.986+0000 7f24db62e740 1 -- 172.21.15.49:0/185193333 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f24bc005500 con 0x5625b68ea810 2024-03-18T23:08:43.992 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:43.986+0000 7f24b9ffb640 1 -- 172.21.15.49:0/185193333 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f24bc032050 con 0x5625b68ea810 2024-03-18T23:08:43.992 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:43.986+0000 7f24db62e740 10 client.25301 fetch_fsmap learned FSMap version 28 2024-03-18T23:08:43.992 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:43.986+0000 7f24db62e740 1 -- 172.21.15.49:0/185193333 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5625b6562210 con 0x5625b68ea810 2024-03-18T23:08:43.992 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:43.986+0000 7f24b9ffb640 1 -- 172.21.15.49:0/185193333 <== mon.2 v2:172.21.15.139:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f24bc005500 con 0x5625b68ea810 2024-03-18T23:08:43.992 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:43.986+0000 7f24b9ffb640 1 -- 172.21.15.49:0/185193333 <== mon.2 v2:172.21.15.139:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f24bc004fd0 con 0x5625b68ea810 2024-03-18T23:08:43.992 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:43.986+0000 7f24db62e740 10 client.25301 fetch_fsmap finished waiting for FSMap version 28 2024-03-18T23:08:43.993 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:43.986+0000 7f24db62e740 10 client.25301 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T23:08:43.993 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:43.986+0000 7f24db62e740 20 client.25301 populate_metadata read hostname 'smithi049' 2024-03-18T23:08:43.993 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:43.986+0000 7f24db62e740 1 --2- 172.21.15.49:0/185193333 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x5625b69038d0 0x5625b6905cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:43.993 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:43.986+0000 7f24db62e740 4 client.25301 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T23:08:43.993 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:43.986+0000 7f24db62e740 1 -- 172.21.15.49:0/185193333 --> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5625b685b120 con 0x5625b69038d0 2024-03-18T23:08:43.993 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:43.987+0000 7f24d259c640 1 --2- 172.21.15.49:0/185193333 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x5625b69038d0 0x5625b6905cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:43.993 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:43.987+0000 7f24b9ffb640 10 client.25301 ms_handle_connect on v2:172.21.15.139:6838/4280128209 2024-03-18T23:08:43.993 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:43.987+0000 7f24d259c640 1 --2- 172.21.15.49:0/185193333 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x5625b69038d0 0x5625b6905cc0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=882509a74ea79677 server_cookie=e10b623ee0941a8a in_seq=0 out_seq=0 2024-03-18T23:08:43.993 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:43.988+0000 7f24b9ffb640 1 -- 172.21.15.49:0/185193333 <== mds.0 v2:172.21.15.139:6838/4280128209 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5625b685b120 con 0x5625b69038d0 2024-03-18T23:08:43.993 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:43.988+0000 7f24b9ffb640 10 client.25301 handle_command_reply: tid=0 2024-03-18T23:08:43.993 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:43.988+0000 7f24db62e740 1 client.25301 shutdown 2024-03-18T23:08:43.993 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:43.988+0000 7f24db62e740 1 -- 172.21.15.49:0/185193333 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5625b68ea810 msgr2=0x5625b68f03d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:43.993 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:43.988+0000 7f24db62e740 1 --2- 172.21.15.49:0/185193333 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5625b68ea810 0x5625b68f03d0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f24bc000c00 tx=0x7f24bc009310 comp rx=0 tx=0).stop 2024-03-18T23:08:43.993 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:43.988+0000 7f24db62e740 1 -- 172.21.15.49:0/185193333 shutdown_connections 2024-03-18T23:08:43.994 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:43.988+0000 7f24db62e740 1 --2- 172.21.15.49:0/185193333 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x5625b69038d0 0x5625b6905cc0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:43.994 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:43.988+0000 7f24b9ffb640 0 client.25301 ms_handle_reset on v2:172.21.15.139:6838/4280128209 2024-03-18T23:08:43.994 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:43.988+0000 7f24db62e740 1 --2- 172.21.15.49:0/185193333 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5625b68f0a20 0x5625b68f0ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:43.994 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:43.988+0000 7f24db62e740 1 --2- 172.21.15.49:0/185193333 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5625b68ea810 0x5625b68f03d0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:43.994 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:43.988+0000 7f24db62e740 1 --2- 172.21.15.49:0/185193333 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5625b68e9ef0 0x5625b68f3070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:43.994 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:43.988+0000 7f24db62e740 1 -- 172.21.15.49:0/185193333 >> 172.21.15.49:0/185193333 conn(0x5625b68e4810 msgr2=0x5625b68e6560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:08:43.994 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:43.988+0000 7f24db62e740 1 -- 172.21.15.49:0/185193333 shutdown_connections 2024-03-18T23:08:43.994 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:43.988+0000 7f24db62e740 1 -- 172.21.15.49:0/185193333 wait complete. 2024-03-18T23:08:43.994 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:43.988+0000 7f24db62e740 20 client.25301 trim_cache size 0 max 16384 2024-03-18T23:08:44.004 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:44.003+0000 7f24db62e740 1 Processor -- start 2024-03-18T23:08:44.004 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:44.003+0000 7f24db62e740 1 -- start start 2024-03-18T23:08:44.004 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:44.003+0000 7f24db62e740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5625b68e9ef0 0x5625b68fefb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:44.004 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:44.003+0000 7f24db62e740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5625b68ea810 0x5625b68f2310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:44.004 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:44.003+0000 7f24db62e740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5625b68f0a20 0x5625b690b470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:44.004 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:44.003+0000 7f24db62e740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f24d41ab290 con 0x5625b68ea810 2024-03-18T23:08:44.004 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:44.003+0000 7f24db62e740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f24d41acdd0 con 0x5625b68e9ef0 2024-03-18T23:08:44.004 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:44.003+0000 7f24db62e740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f24d41ab960 con 0x5625b68f0a20 2024-03-18T23:08:44.004 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:44.003+0000 7f24d259c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5625b68e9ef0 0x5625b68fefb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:44.004 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:44.003+0000 7f24d259c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5625b68e9ef0 0x5625b68fefb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.49:38430/0 (socket says 172.21.15.49:38430) 2024-03-18T23:08:44.004 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:44.003+0000 7f24d259c640 1 -- 172.21.15.49:0/3185980603 learned_addr learned my addr 172.21.15.49:0/3185980603 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:08:44.004 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:44.003+0000 7f24d2d9d640 1 --2- 172.21.15.49:0/3185980603 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5625b68f0a20 0x5625b690b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:44.005 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:44.003+0000 7f24d1d9b640 1 --2- 172.21.15.49:0/3185980603 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5625b68ea810 0x5625b68f2310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:44.005 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:44.004+0000 7f24d259c640 1 -- 172.21.15.49:0/3185980603 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5625b68f0a20 msgr2=0x5625b690b470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:08:44.005 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:44.004+0000 7f24d259c640 1 --2- 172.21.15.49:0/3185980603 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5625b68f0a20 0x5625b690b470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:44.005 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:44.004+0000 7f24d259c640 1 -- 172.21.15.49:0/3185980603 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5625b68ea810 msgr2=0x5625b68f2310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:08:44.005 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:44.004+0000 7f24d259c640 1 --2- 172.21.15.49:0/3185980603 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5625b68ea810 0x5625b68f2310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:44.005 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:44.004+0000 7f24d259c640 1 -- 172.21.15.49:0/3185980603 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5625b662e620 con 0x5625b68e9ef0 2024-03-18T23:08:44.005 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:44.004+0000 7f24d259c640 1 --2- 172.21.15.49:0/3185980603 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5625b68e9ef0 0x5625b68fefb0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f24c40b9550 tx=0x7f24c40b9400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=24f922c0accd365e server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:08:44.005 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:44.004+0000 7f24bbfff640 1 -- 172.21.15.49:0/3185980603 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24c4034920 con 0x5625b68e9ef0 2024-03-18T23:08:44.005 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:44.004+0000 7f24bbfff640 1 -- 172.21.15.49:0/3185980603 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f24c40bd7f0 con 0x5625b68e9ef0 2024-03-18T23:08:44.005 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:44.004+0000 7f24bbfff640 1 -- 172.21.15.49:0/3185980603 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24c40ee430 con 0x5625b68e9ef0 2024-03-18T23:08:44.008 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:44.005+0000 7f24db62e740 1 -- 172.21.15.49:0/3185980603 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5625b68e9ef0 msgr2=0x5625b68fefb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:44.008 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:44.005+0000 7f24db62e740 1 --2- 172.21.15.49:0/3185980603 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5625b68e9ef0 0x5625b68fefb0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f24c40b9550 tx=0x7f24c40b9400 comp rx=0 tx=0).stop 2024-03-18T23:08:44.008 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:44.005+0000 7f24db62e740 1 -- 172.21.15.49:0/3185980603 shutdown_connections 2024-03-18T23:08:44.008 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:44.005+0000 7f24db62e740 1 --2- 172.21.15.49:0/3185980603 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5625b68f0a20 0x5625b690b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:44.009 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:44.005+0000 7f24db62e740 1 --2- 172.21.15.49:0/3185980603 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5625b68ea810 0x5625b68f2310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:44.009 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:44.005+0000 7f24db62e740 1 --2- 172.21.15.49:0/3185980603 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5625b68e9ef0 0x5625b68fefb0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:44.009 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:44.005+0000 7f24db62e740 1 -- 172.21.15.49:0/3185980603 >> 172.21.15.49:0/3185980603 conn(0x5625b68e4810 msgr2=0x5625b68f1c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:08:44.009 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:44.005+0000 7f24db62e740 1 -- 172.21.15.49:0/3185980603 shutdown_connections 2024-03-18T23:08:44.009 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:44.005+0000 7f24db62e740 1 -- 172.21.15.49:0/3185980603 wait complete. 2024-03-18T23:08:44.009 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:44.005+0000 7f24db62e740 1 -- start start 2024-03-18T23:08:44.009 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:44.005+0000 7f24db62e740 1 Processor -- start 2024-03-18T23:08:44.009 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:44.005+0000 7f24db62e740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5625b68e9ef0 0x5625b68edc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:44.009 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:44.005+0000 7f24db62e740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5625b68ea810 0x5625b68ec640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:44.009 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:44.005+0000 7f24db62e740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5625b68f0a20 0x5625b68ecb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:44.009 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:44.005+0000 7f24db62e740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f24d41ac700 con 0x5625b68f0a20 2024-03-18T23:08:44.009 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:44.005+0000 7f24db62e740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f24d41ac030 con 0x5625b68e9ef0 2024-03-18T23:08:44.009 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:44.006+0000 7f24d259c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5625b68e9ef0 0x5625b68edc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:44.009 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:44.006+0000 7f24d1d9b640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5625b68ea810 0x5625b68ec640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:44.010 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:44.006+0000 7f24db62e740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f24d41a9270 con 0x5625b68ea810 2024-03-18T23:08:44.010 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:44.006+0000 7f24d2d9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5625b68f0a20 0x5625b68ecb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:44.010 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:44.006+0000 7f24d2d9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5625b68f0a20 0x5625b68ecb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:54184/0 (socket says 172.21.15.49:54184) 2024-03-18T23:08:44.010 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:44.006+0000 7f24d1d9b640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5625b68ea810 0x5625b68ec640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.49:44670/0 (socket says 172.21.15.49:44670) 2024-03-18T23:08:44.010 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:44.006+0000 7f24d2d9d640 1 -- 172.21.15.49:0/3753165751 learned_addr learned my addr 172.21.15.49:0/3753165751 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:08:44.010 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:44.006+0000 7f24db62e740 10 client.0 _ll_register_callbacks cb 0x7f24cc009c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T23:08:44.010 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:44.006+0000 7f24d1d9b640 1 -- 172.21.15.49:0/3753165751 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5625b68e9ef0 msgr2=0x5625b68edc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:08:44.010 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:44.006+0000 7f24d1d9b640 1 --2- 172.21.15.49:0/3753165751 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5625b68e9ef0 0x5625b68edc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:44.010 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:44.006+0000 7f24d1d9b640 1 -- 172.21.15.49:0/3753165751 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5625b68f0a20 msgr2=0x5625b68ecb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:08:44.010 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:44.006+0000 7f24d1d9b640 1 --2- 172.21.15.49:0/3753165751 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5625b68f0a20 0x5625b68ecb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:44.010 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:44.006+0000 7f24d1d9b640 1 -- 172.21.15.49:0/3753165751 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5625b68fbab0 con 0x5625b68ea810 2024-03-18T23:08:44.010 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:44.006+0000 7f24d2d9d640 1 --2- 172.21.15.49:0/3753165751 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5625b68f0a20 0x5625b68ecb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:08:44.010 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:44.006+0000 7f24d1d9b640 1 --2- 172.21.15.49:0/3753165751 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5625b68ea810 0x5625b68ec640 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f24bc009380 tx=0x7f24bc011ee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e0ac458c19667710 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:08:44.010 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:44.006+0000 7f24b9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-03-18T23:08:44.010 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:44.006+0000 7f24b9ffb640 1 -- 172.21.15.49:0/3753165751 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24bc03a070 con 0x5625b68ea810 2024-03-18T23:08:44.011 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:44.006+0000 7f24b9ffb640 1 -- 172.21.15.49:0/3753165751 <== mon.1 v2:172.21.15.175:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f24bc00fca0 con 0x5625b68ea810 2024-03-18T23:08:44.011 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:44.006+0000 7f24db62e740 1 -- 172.21.15.49:0/3753165751 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f24c40bd7f0 con 0x5625b68ea810 2024-03-18T23:08:44.011 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:44.007+0000 7f24b9ffb640 1 -- 172.21.15.49:0/3753165751 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f24bc045050 con 0x5625b68ea810 2024-03-18T23:08:44.011 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:44.007+0000 7f24db62e740 10 client.25297 fetch_fsmap learned FSMap version 28 2024-03-18T23:08:44.011 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:44.007+0000 7f24db62e740 1 -- 172.21.15.49:0/3753165751 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5625b6969310 con 0x5625b68ea810 2024-03-18T23:08:44.011 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:44.007+0000 7f24b9ffb640 1 -- 172.21.15.49:0/3753165751 <== mon.1 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f24c40bd7f0 con 0x5625b68ea810 2024-03-18T23:08:44.011 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:44.007+0000 7f24b9ffb640 1 -- 172.21.15.49:0/3753165751 <== mon.1 v2:172.21.15.175:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f24bc019450 con 0x5625b68ea810 2024-03-18T23:08:44.011 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:44.007+0000 7f24db62e740 10 client.25297 fetch_fsmap finished waiting for FSMap version 28 2024-03-18T23:08:44.011 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:44.007+0000 7f24db62e740 10 client.25297 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T23:08:44.011 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:44.007+0000 7f24db62e740 20 client.25297 populate_metadata read hostname 'smithi049' 2024-03-18T23:08:44.011 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:44.007+0000 7f24db62e740 1 --2- 172.21.15.49:0/3753165751 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x5625b69038d0 0x5625b696ba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:44.011 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:44.007+0000 7f24db62e740 4 client.25297 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T23:08:44.011 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:44.007+0000 7f24db62e740 1 -- 172.21.15.49:0/3753165751 --> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5625b6855470 con 0x5625b69038d0 2024-03-18T23:08:44.011 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:44.007+0000 7f24d259c640 1 --2- 172.21.15.49:0/3753165751 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x5625b69038d0 0x5625b696ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:44.012 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:44.008+0000 7f24b9ffb640 10 client.25297 ms_handle_connect on v2:172.21.15.139:6838/4280128209 2024-03-18T23:08:44.012 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:44.008+0000 7f24d259c640 1 --2- 172.21.15.49:0/3753165751 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x5625b69038d0 0x5625b696ba70 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a29a556cfa562c7d server_cookie=d153737d363bc056 in_seq=0 out_seq=0 2024-03-18T23:08:44.013 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:44.011+0000 7f24b9ffb640 1 -- 172.21.15.49:0/3753165751 <== mds.0 v2:172.21.15.139:6838/4280128209 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5625b6855470 con 0x5625b69038d0 2024-03-18T23:08:44.013 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:44.011+0000 7f24b9ffb640 10 client.25297 handle_command_reply: tid=0 2024-03-18T23:08:44.013 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:44.011+0000 7f24db62e740 1 client.25297 shutdown 2024-03-18T23:08:44.013 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:44.011+0000 7f24db62e740 1 -- 172.21.15.49:0/3753165751 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5625b68ea810 msgr2=0x5625b68ec640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:44.013 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:44.011+0000 7f24db62e740 1 --2- 172.21.15.49:0/3753165751 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5625b68ea810 0x5625b68ec640 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f24bc009380 tx=0x7f24bc011ee0 comp rx=0 tx=0).stop 2024-03-18T23:08:44.013 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:44.011+0000 7f24db62e740 1 -- 172.21.15.49:0/3753165751 shutdown_connections 2024-03-18T23:08:44.013 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:44.011+0000 7f24db62e740 1 --2- 172.21.15.49:0/3753165751 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x5625b69038d0 0x5625b696ba70 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:44.013 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:44.011+0000 7f24db62e740 1 --2- 172.21.15.49:0/3753165751 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5625b68f0a20 0x5625b68ecb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:44.013 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:44.011+0000 7f24db62e740 1 --2- 172.21.15.49:0/3753165751 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5625b68ea810 0x5625b68ec640 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:44.013 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:44.011+0000 7f24db62e740 1 --2- 172.21.15.49:0/3753165751 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5625b68e9ef0 0x5625b68edc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:44.013 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:44.011+0000 7f24db62e740 1 -- 172.21.15.49:0/3753165751 >> 172.21.15.49:0/3753165751 conn(0x5625b68e4810 msgr2=0x5625b68e3ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:08:44.013 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:44.011+0000 7f24b9ffb640 0 client.25297 ms_handle_reset on v2:172.21.15.139:6838/4280128209 2024-03-18T23:08:44.013 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:44.011+0000 7f24db62e740 1 -- 172.21.15.49:0/3753165751 shutdown_connections 2024-03-18T23:08:44.013 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:44.011+0000 7f24db62e740 1 -- 172.21.15.49:0/3753165751 wait complete. 2024-03-18T23:08:44.014 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:44.011+0000 7f24db62e740 20 client.25297 trim_cache size 0 max 16384 2024-03-18T23:08:44.014 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:{ 2024-03-18T23:08:44.014 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "message": "", 2024-03-18T23:08:44.015 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "return_code": 0 2024-03-18T23:08:44.015 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:} 2024-03-18T23:08:44.016 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:44.012+0000 7f24d8a44640 1 -- 172.21.15.49:0/941656878 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f24c00a2b30 msgr2=0x7f24c00a4ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:44.016 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:44.012+0000 7f24d8a44640 1 --2- 172.21.15.49:0/941656878 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f24c00a2b30 0x7f24c00a4ff0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f24cc0095f0 tx=0x7f24cc005f90 comp rx=0 tx=0).stop 2024-03-18T23:08:44.016 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:44.012+0000 7f24d8a44640 1 -- 172.21.15.49:0/941656878 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f24d41006e0 msgr2=0x7f24d4198050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:44.016 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:44.012+0000 7f24d8a44640 1 --2- 172.21.15.49:0/941656878 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f24d41006e0 0x7f24d4198050 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f24c4004af0 tx=0x7f24c4032ea0 comp rx=0 tx=0).stop 2024-03-18T23:08:44.016 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:44.012+0000 7f24d8a44640 1 -- 172.21.15.49:0/941656878 shutdown_connections 2024-03-18T23:08:44.016 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:44.012+0000 7f24d8a44640 1 --2- 172.21.15.49:0/941656878 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f24c00a2b30 0x7f24c00a4ff0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:44.016 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:44.012+0000 7f24d8a44640 1 --2- 172.21.15.49:0/941656878 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f24d41066f0 0x7f24d419c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:44.016 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:44.012+0000 7f24d8a44640 1 --2- 172.21.15.49:0/941656878 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f24d41010a0 0x7f24d4198590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:44.017 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:44.012+0000 7f24d8a44640 1 --2- 172.21.15.49:0/941656878 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f24d41006e0 0x7f24d4198050 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:44.017 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:44.012+0000 7f24d8a44640 1 -- 172.21.15.49:0/941656878 >> 172.21.15.49:0/941656878 conn(0x7f24d4077800 msgr2=0x7f24d4069cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:08:44.017 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:44.012+0000 7f24d8a44640 1 -- 172.21.15.49:0/941656878 shutdown_connections 2024-03-18T23:08:44.017 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:44.012+0000 7f24d8a44640 1 -- 172.21.15.49:0/941656878 wait complete. 2024-03-18T23:08:45.211 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:08:44 smithi175 ceph-mon[67352]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 1.6 KiB/s wr, 16 op/s 2024-03-18T23:08:45.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:08:44 smithi139 ceph-mon[67282]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 1.6 KiB/s wr, 16 op/s 2024-03-18T23:08:45.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:08:44 smithi049 ceph-mon[62991]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 1.6 KiB/s wr, 16 op/s 2024-03-18T23:08:47.211 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:08:46 smithi175 ceph-mon[67352]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-18T23:08:47.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:08:46 smithi139 ceph-mon[67282]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-18T23:08:47.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:08:46 smithi049 ceph-mon[62991]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-18T23:08:49.211 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:08:48 smithi175 ceph-mon[67352]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-18T23:08:49.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:08:48 smithi139 ceph-mon[67282]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-18T23:08:49.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:08:48 smithi049 ceph-mon[62991]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-18T23:08:51.211 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:08:50 smithi175 ceph-mon[67352]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-18T23:08:51.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:08:50 smithi139 ceph-mon[67282]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-18T23:08:51.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:08:50 smithi049 ceph-mon[62991]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-18T23:08:53.211 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:08:52 smithi175 ceph-mon[67352]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-03-18T23:08:53.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:08:52 smithi139 ceph-mon[67282]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-03-18T23:08:53.379 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:08:52 smithi049 ceph-mon[62991]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-03-18T23:08:55.211 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:08:54 smithi175 ceph-mon[67352]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-03-18T23:08:55.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:08:54 smithi139 ceph-mon[67282]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-03-18T23:08:55.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:08:54 smithi049 ceph-mon[62991]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-03-18T23:08:57.103 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:57.101+0000 7f87d939d640 1 -- 172.21.15.49:0/1284581002 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f87d41065e0 msgr2=0x7f87d41069c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:57.103 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:57.101+0000 7f87d939d640 1 --2- 172.21.15.49:0/1284581002 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f87d41065e0 0x7f87d41069c0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f87c4009900 tx=0x7f87c402bc30 comp rx=0 tx=0).stop 2024-03-18T23:08:57.103 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:57.101+0000 7f87d939d640 1 -- 172.21.15.49:0/1284581002 shutdown_connections 2024-03-18T23:08:57.103 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:57.101+0000 7f87d939d640 1 --2- 172.21.15.49:0/1284581002 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f87d4100ee0 0x7f87d410d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:57.103 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:57.101+0000 7f87d939d640 1 --2- 172.21.15.49:0/1284581002 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f87d4100520 0x7f87d41009a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:57.103 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:57.101+0000 7f87d939d640 1 --2- 172.21.15.49:0/1284581002 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f87d41065e0 0x7f87d41069c0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:57.103 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:57.101+0000 7f87d939d640 1 -- 172.21.15.49:0/1284581002 >> 172.21.15.49:0/1284581002 conn(0x7f87d40fc030 msgr2=0x7f87d40fe450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:08:57.103 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:57.101+0000 7f87d939d640 1 -- 172.21.15.49:0/1284581002 shutdown_connections 2024-03-18T23:08:57.103 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:57.101+0000 7f87d939d640 1 -- 172.21.15.49:0/1284581002 wait complete. 2024-03-18T23:08:57.104 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:57.102+0000 7f87d939d640 1 Processor -- start 2024-03-18T23:08:57.104 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:57.102+0000 7f87d939d640 1 -- start start 2024-03-18T23:08:57.105 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:57.102+0000 7f87d939d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f87d4100520 0x7f87d4198c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:57.105 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:57.102+0000 7f87d939d640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f87d4100ee0 0x7f87d41991c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:57.106 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:57.103+0000 7f87d939d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f87d41065e0 0x7f87d41a3d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:57.106 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:57.103+0000 7f87d939d640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f87d41108c0 con 0x7f87d41065e0 2024-03-18T23:08:57.106 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:57.103+0000 7f87d939d640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f87d4110740 con 0x7f87d4100520 2024-03-18T23:08:57.106 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:57.103+0000 7f87d939d640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f87d4110a40 con 0x7f87d4100ee0 2024-03-18T23:08:57.106 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:57.103+0000 7f87d8b9c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f87d41065e0 0x7f87d41a3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:57.106 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:57.103+0000 7f87d3fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f87d4100520 0x7f87d4198c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:57.106 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:57.103+0000 7f87d8b9c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f87d41065e0 0x7f87d41a3d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:32968/0 (socket says 172.21.15.49:32968) 2024-03-18T23:08:57.106 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:57.103+0000 7f87d37fe640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f87d4100ee0 0x7f87d41991c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:57.106 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:57.103+0000 7f87d8b9c640 1 -- 172.21.15.49:0/1363679309 learned_addr learned my addr 172.21.15.49:0/1363679309 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:08:57.106 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:57.103+0000 7f87d8b9c640 1 -- 172.21.15.49:0/1363679309 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f87d4100ee0 msgr2=0x7f87d41991c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:57.106 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:57.103+0000 7f87d8b9c640 1 --2- 172.21.15.49:0/1363679309 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f87d4100ee0 0x7f87d41991c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:57.106 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:57.103+0000 7f87d8b9c640 1 -- 172.21.15.49:0/1363679309 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f87d4100520 msgr2=0x7f87d4198c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:57.107 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:57.103+0000 7f87d8b9c640 1 --2- 172.21.15.49:0/1363679309 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f87d4100520 0x7f87d4198c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:57.107 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:57.103+0000 7f87d8b9c640 1 -- 172.21.15.49:0/1363679309 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87d41a4240 con 0x7f87d41065e0 2024-03-18T23:08:57.107 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:57.103+0000 7f87d3fff640 1 --2- 172.21.15.49:0/1363679309 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f87d4100520 0x7f87d4198c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:08:57.107 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:57.103+0000 7f87d8b9c640 1 --2- 172.21.15.49:0/1363679309 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f87d41065e0 0x7f87d41a3d00 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f87cc00eb10 tx=0x7f87cc00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:08:57.107 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:57.103+0000 7f87d17fa640 1 -- 172.21.15.49:0/1363679309 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87cc004b20 con 0x7f87d41065e0 2024-03-18T23:08:57.107 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:57.103+0000 7f87d17fa640 1 -- 172.21.15.49:0/1363679309 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f87cc004cc0 con 0x7f87d41065e0 2024-03-18T23:08:57.107 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:57.103+0000 7f87d939d640 1 -- 172.21.15.49:0/1363679309 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87d41a4410 con 0x7f87d41065e0 2024-03-18T23:08:57.107 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:57.103+0000 7f87d17fa640 1 -- 172.21.15.49:0/1363679309 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87cc010760 con 0x7f87d41065e0 2024-03-18T23:08:57.107 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:57.103+0000 7f87d939d640 1 -- 172.21.15.49:0/1363679309 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87d41a4910 con 0x7f87d41065e0 2024-03-18T23:08:57.113 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:57.106+0000 7f87d17fa640 1 -- 172.21.15.49:0/1363679309 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 23) ==== 137466+0+0 (secure 0 0 0) 0x7f87cc005020 con 0x7f87d41065e0 2024-03-18T23:08:57.113 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:57.106+0000 7f87dd1e1740 1 Processor -- start 2024-03-18T23:08:57.113 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:57.106+0000 7f87dd1e1740 1 -- start start 2024-03-18T23:08:57.114 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:57.106+0000 7f87dd1e1740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560c512820d0 0x560c512824b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:57.114 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:57.106+0000 7f87dd1e1740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560c512829f0 0x560c5128b280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:57.114 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:57.106+0000 7f87dd1e1740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560c5128b7c0 0x560c5128dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:57.114 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:57.106+0000 7f87dd1e1740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x560c511e5be0 con 0x560c512820d0 2024-03-18T23:08:57.114 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:57.106+0000 7f87dd1e1740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x560c5118e960 con 0x560c5128b7c0 2024-03-18T23:08:57.114 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:57.106+0000 7f87dd1e1740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x560c511e1c80 con 0x560c512829f0 2024-03-18T23:08:57.114 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:57.106+0000 7f87d17fa640 1 --2- 172.21.15.49:0/1363679309 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f87bc0a29f0 0x7f87bc0a4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:57.114 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:57.106+0000 7f87d37fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560c512820d0 0x560c512824b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:57.114 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:57.106+0000 7f87d3fff640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560c512829f0 0x560c5128b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:57.114 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:57.106+0000 7f87d37fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560c512820d0 0x560c512824b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:32970/0 (socket says 172.21.15.49:32970) 2024-03-18T23:08:57.114 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:57.106+0000 7f87d37fe640 1 -- 172.21.15.49:0/1322688221 learned_addr learned my addr 172.21.15.49:0/1322688221 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:08:57.114 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:57.107+0000 7f87d37fe640 1 --2- 172.21.15.49:0/1322688221 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560c5128b7c0 0x560c5128dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:57.114 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:57.107+0000 7f87d8b9c640 1 --2- 172.21.15.49:0/1363679309 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f87bc0a29f0 0x7f87bc0a4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:57.115 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:57.107+0000 7f87d3fff640 1 -- 172.21.15.49:0/1322688221 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560c5128b7c0 msgr2=0x560c5128dbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:08:57.115 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:57.107+0000 7f87d3fff640 1 --2- 172.21.15.49:0/1322688221 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560c5128b7c0 0x560c5128dbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:57.115 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:57.107+0000 7f87d3fff640 1 -- 172.21.15.49:0/1322688221 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560c512820d0 msgr2=0x560c512824b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:08:57.115 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:57.107+0000 7f87d3fff640 1 --2- 172.21.15.49:0/1322688221 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560c512820d0 0x560c512824b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:57.115 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:57.107+0000 7f87d17fa640 1 -- 172.21.15.49:0/1363679309 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f87cc0cf4e0 con 0x7f87d41065e0 2024-03-18T23:08:57.115 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:57.107+0000 7f87d3fff640 1 -- 172.21.15.49:0/1322688221 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560c51185d20 con 0x560c512829f0 2024-03-18T23:08:57.115 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:57.107+0000 7f87d3fff640 1 --2- 172.21.15.49:0/1322688221 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560c512829f0 0x560c5128b280 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f87c4000c00 tx=0x7f87c4005e80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=49d7b1833830ab35 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:08:57.115 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:57.107+0000 7f87d8b9c640 1 --2- 172.21.15.49:0/1363679309 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f87bc0a29f0 0x7f87bc0a4eb0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f87cc0ccc00 tx=0x7f87cc086040 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:08:57.115 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:57.107+0000 7f87ba7fc640 1 -- 172.21.15.49:0/1322688221 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87c404c070 con 0x560c512829f0 2024-03-18T23:08:57.115 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:57.107+0000 7f87ba7fc640 1 -- 172.21.15.49:0/1322688221 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f87c402c5d0 con 0x560c512829f0 2024-03-18T23:08:57.115 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:57.107+0000 7f87ba7fc640 1 -- 172.21.15.49:0/1322688221 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87c4004f20 con 0x560c512829f0 2024-03-18T23:08:57.115 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:57.108+0000 7f87dd1e1740 1 -- 172.21.15.49:0/1322688221 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560c512829f0 msgr2=0x560c5128b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:57.115 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:57.108+0000 7f87dd1e1740 1 --2- 172.21.15.49:0/1322688221 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560c512829f0 0x560c5128b280 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f87c4000c00 tx=0x7f87c4005e80 comp rx=0 tx=0).stop 2024-03-18T23:08:57.115 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:57.108+0000 7f87dd1e1740 1 -- 172.21.15.49:0/1322688221 shutdown_connections 2024-03-18T23:08:57.116 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:57.108+0000 7f87dd1e1740 1 --2- 172.21.15.49:0/1322688221 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560c5128b7c0 0x560c5128dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:57.116 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:57.108+0000 7f87dd1e1740 1 --2- 172.21.15.49:0/1322688221 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560c512829f0 0x560c5128b280 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:57.116 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:57.108+0000 7f87dd1e1740 1 --2- 172.21.15.49:0/1322688221 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560c512820d0 0x560c512824b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:57.116 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:57.108+0000 7f87dd1e1740 1 -- 172.21.15.49:0/1322688221 >> 172.21.15.49:0/1322688221 conn(0x560c5127c9f0 msgr2=0x560c5127cdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:08:57.116 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:57.108+0000 7f87dd1e1740 1 -- 172.21.15.49:0/1322688221 shutdown_connections 2024-03-18T23:08:57.116 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:57.108+0000 7f87dd1e1740 1 -- 172.21.15.49:0/1322688221 wait complete. 2024-03-18T23:08:57.116 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:57.108+0000 7f87dd1e1740 1 -- start start 2024-03-18T23:08:57.116 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:57.109+0000 7f87dd1e1740 1 Processor -- start 2024-03-18T23:08:57.116 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:57.109+0000 7f87dd1e1740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560c512820d0 0x560c5128b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:57.116 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:57.109+0000 7f87dd1e1740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560c512829f0 0x560c51288550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:57.116 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:57.109+0000 7f87dd1e1740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560c5128b7c0 0x560c51288b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:57.116 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:57.109+0000 7f87dd1e1740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x560c5128e390 con 0x560c5128b7c0 2024-03-18T23:08:57.116 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:57.109+0000 7f87dd1e1740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x560c511a1db0 con 0x560c512820d0 2024-03-18T23:08:57.116 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:57.109+0000 7f87dd1e1740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x560c5128e510 con 0x560c512829f0 2024-03-18T23:08:57.117 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:57.109+0000 7f87dd1e1740 10 client.0 _ll_register_callbacks cb 0x560c511a6e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T23:08:57.117 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:57.109+0000 7f87d37fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560c512820d0 0x560c5128b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:57.117 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:57.109+0000 7f87d37fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560c512820d0 0x560c5128b1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.49:34754/0 (socket says 172.21.15.49:34754) 2024-03-18T23:08:57.117 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:57.109+0000 7f87d3fff640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560c512829f0 0x560c51288550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:57.117 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:57.109+0000 7f87d37fe640 1 -- 172.21.15.49:0/3408555978 learned_addr learned my addr 172.21.15.49:0/3408555978 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:08:57.117 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:57.109+0000 7f87d37fe640 1 --2- 172.21.15.49:0/3408555978 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560c5128b7c0 0x560c51288b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:57.117 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:57.109+0000 7f87d3fff640 1 -- 172.21.15.49:0/3408555978 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560c512820d0 msgr2=0x560c5128b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:08:57.117 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:57.109+0000 7f87d3fff640 1 --2- 172.21.15.49:0/3408555978 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560c512820d0 0x560c5128b1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:57.117 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:57.109+0000 7f87d3fff640 1 -- 172.21.15.49:0/3408555978 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560c5128b7c0 msgr2=0x560c51288b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:08:57.117 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:57.109+0000 7f87d3fff640 1 --2- 172.21.15.49:0/3408555978 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560c5128b7c0 0x560c51288b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:57.117 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:57.109+0000 7f87d3fff640 1 -- 172.21.15.49:0/3408555978 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560c511a0c60 con 0x560c512829f0 2024-03-18T23:08:57.117 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:57.110+0000 7f87d3fff640 1 --2- 172.21.15.49:0/3408555978 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560c512829f0 0x560c51288550 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f87c40096c0 tx=0x7f87c4004ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a2f7150b0b336a88 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:08:57.117 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:57.110+0000 7f87b37fe640 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-03-18T23:08:57.117 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:57.110+0000 7f87b37fe640 1 -- 172.21.15.49:0/3408555978 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87c404c070 con 0x560c512829f0 2024-03-18T23:08:57.118 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:57.110+0000 7f87b37fe640 1 -- 172.21.15.49:0/3408555978 <== mon.1 v2:172.21.15.175:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f87c4005130 con 0x560c512829f0 2024-03-18T23:08:57.118 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:57.110+0000 7f87dd1e1740 1 -- 172.21.15.49:0/3408555978 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f87c402c5d0 con 0x560c512829f0 2024-03-18T23:08:57.118 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:57.110+0000 7f87b37fe640 1 -- 172.21.15.49:0/3408555978 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f87c4055050 con 0x560c512829f0 2024-03-18T23:08:57.118 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:57.110+0000 7f87dd1e1740 10 client.25309 fetch_fsmap learned FSMap version 28 2024-03-18T23:08:57.118 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:57.110+0000 7f87dd1e1740 1 -- 172.21.15.49:0/3408555978 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x560c50efa210 con 0x560c512829f0 2024-03-18T23:08:57.118 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:57.110+0000 7f87b37fe640 1 -- 172.21.15.49:0/3408555978 <== mon.1 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f87c402c5d0 con 0x560c512829f0 2024-03-18T23:08:57.118 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:57.110+0000 7f87b37fe640 1 -- 172.21.15.49:0/3408555978 <== mon.1 v2:172.21.15.175:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f87c404f020 con 0x560c512829f0 2024-03-18T23:08:57.118 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:57.110+0000 7f87dd1e1740 10 client.25309 fetch_fsmap finished waiting for FSMap version 28 2024-03-18T23:08:57.118 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:57.110+0000 7f87dd1e1740 10 client.25309 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T23:08:57.118 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:57.110+0000 7f87dd1e1740 20 client.25309 populate_metadata read hostname 'smithi049' 2024-03-18T23:08:57.118 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:57.110+0000 7f87dd1e1740 1 --2- 172.21.15.49:0/3408555978 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x560c5129b4c0 0x560c5129d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:57.118 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:57.110+0000 7f87dd1e1740 4 client.25309 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T23:08:57.118 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:57.110+0000 7f87dd1e1740 1 -- 172.21.15.49:0/3408555978 --> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560c511f3120 con 0x560c5129b4c0 2024-03-18T23:08:57.118 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:57.111+0000 7f87d37fe640 1 --2- 172.21.15.49:0/3408555978 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x560c5129b4c0 0x560c5129d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:57.119 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:57.111+0000 7f87d37fe640 1 --2- 172.21.15.49:0/3408555978 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x560c5129b4c0 0x560c5129d8b0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9b063a367f9921d0 server_cookie=317137cd3a48ffeb in_seq=0 out_seq=0 2024-03-18T23:08:57.119 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:57.111+0000 7f87b37fe640 10 client.25309 ms_handle_connect on v2:172.21.15.139:6838/4280128209 2024-03-18T23:08:57.119 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:57.112+0000 7f87b37fe640 1 -- 172.21.15.49:0/3408555978 <== mds.0 v2:172.21.15.139:6838/4280128209 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x560c511f3120 con 0x560c5129b4c0 2024-03-18T23:08:57.119 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:57.112+0000 7f87b37fe640 10 client.25309 handle_command_reply: tid=0 2024-03-18T23:08:57.119 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:57.112+0000 7f87dd1e1740 1 client.25309 shutdown 2024-03-18T23:08:57.119 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:57.112+0000 7f87dd1e1740 1 -- 172.21.15.49:0/3408555978 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560c512829f0 msgr2=0x560c51288550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:57.119 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:57.112+0000 7f87dd1e1740 1 --2- 172.21.15.49:0/3408555978 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560c512829f0 0x560c51288550 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f87c40096c0 tx=0x7f87c4004ca0 comp rx=0 tx=0).stop 2024-03-18T23:08:57.119 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:57.112+0000 7f87dd1e1740 1 -- 172.21.15.49:0/3408555978 shutdown_connections 2024-03-18T23:08:57.119 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:57.112+0000 7f87dd1e1740 1 --2- 172.21.15.49:0/3408555978 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x560c5129b4c0 0x560c5129d8b0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:57.119 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:57.112+0000 7f87b37fe640 0 client.25309 ms_handle_reset on v2:172.21.15.139:6838/4280128209 2024-03-18T23:08:57.119 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:57.112+0000 7f87dd1e1740 1 --2- 172.21.15.49:0/3408555978 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560c5128b7c0 0x560c51288b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:57.119 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:57.112+0000 7f87dd1e1740 1 --2- 172.21.15.49:0/3408555978 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560c512829f0 0x560c51288550 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:57.119 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:57.112+0000 7f87dd1e1740 1 --2- 172.21.15.49:0/3408555978 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560c512820d0 0x560c5128b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:57.119 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:57.112+0000 7f87dd1e1740 1 -- 172.21.15.49:0/3408555978 >> 172.21.15.49:0/3408555978 conn(0x560c5127c9f0 msgr2=0x560c5127e740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:08:57.120 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:57.112+0000 7f87dd1e1740 1 -- 172.21.15.49:0/3408555978 shutdown_connections 2024-03-18T23:08:57.120 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:57.112+0000 7f87dd1e1740 1 -- 172.21.15.49:0/3408555978 wait complete. 2024-03-18T23:08:57.120 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:57.112+0000 7f87dd1e1740 20 client.25309 trim_cache size 0 max 16384 2024-03-18T23:08:57.130 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:57.127+0000 7f87dd1e1740 1 Processor -- start 2024-03-18T23:08:57.130 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:57.127+0000 7f87dd1e1740 1 -- start start 2024-03-18T23:08:57.130 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:57.127+0000 7f87dd1e1740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560c512820d0 0x560c51297130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:57.130 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:57.127+0000 7f87dd1e1740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560c512829f0 0x560c5127b930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:57.130 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:57.127+0000 7f87dd1e1740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560c5128b7c0 0x560c512a2e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:57.130 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:57.127+0000 7f87dd1e1740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f87d41ab0b0 con 0x560c512829f0 2024-03-18T23:08:57.130 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:57.127+0000 7f87dd1e1740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f87d41acbf0 con 0x560c5128b7c0 2024-03-18T23:08:57.130 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:57.127+0000 7f87dd1e1740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f87d41ab780 con 0x560c512820d0 2024-03-18T23:08:57.130 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:57.127+0000 7f87d3fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560c512829f0 0x560c5127b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:57.130 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:57.127+0000 7f87d37fe640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560c512820d0 0x560c51297130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:57.131 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:57.127+0000 7f87d37fe640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560c512820d0 0x560c51297130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.49:44498/0 (socket says 172.21.15.49:44498) 2024-03-18T23:08:57.131 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:57.127+0000 7f87d37fe640 1 -- 172.21.15.49:0/4100046057 learned_addr learned my addr 172.21.15.49:0/4100046057 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:08:57.131 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:57.127+0000 7f87d37fe640 1 --2- 172.21.15.49:0/4100046057 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560c5128b7c0 0x560c512a2e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:57.131 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:57.128+0000 7f87d37fe640 1 -- 172.21.15.49:0/4100046057 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560c5128b7c0 msgr2=0x560c512a2e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:08:57.131 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:57.128+0000 7f87d37fe640 1 --2- 172.21.15.49:0/4100046057 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560c5128b7c0 0x560c512a2e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:57.131 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:57.128+0000 7f87d37fe640 1 -- 172.21.15.49:0/4100046057 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560c512829f0 msgr2=0x560c5127b930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:08:57.131 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:57.128+0000 7f87d37fe640 1 --2- 172.21.15.49:0/4100046057 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560c512829f0 0x560c5127b930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:57.131 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:57.128+0000 7f87d37fe640 1 -- 172.21.15.49:0/4100046057 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560c5129dc20 con 0x560c512820d0 2024-03-18T23:08:57.131 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:57.128+0000 7f87d37fe640 1 --2- 172.21.15.49:0/4100046057 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560c512820d0 0x560c51297130 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f87c80045d0 tx=0x7f87c8015040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=54b4c2997bd62a5b server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:08:57.131 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:57.128+0000 7f87b9ffb640 1 -- 172.21.15.49:0/4100046057 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87c8005020 con 0x560c512820d0 2024-03-18T23:08:57.131 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:57.128+0000 7f87b9ffb640 1 -- 172.21.15.49:0/4100046057 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f87c80047a0 con 0x560c512820d0 2024-03-18T23:08:57.131 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:57.128+0000 7f87b9ffb640 1 -- 172.21.15.49:0/4100046057 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87c80139d0 con 0x560c512820d0 2024-03-18T23:08:57.131 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:57.129+0000 7f87dd1e1740 1 -- 172.21.15.49:0/4100046057 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560c512820d0 msgr2=0x560c51297130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:57.132 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:57.129+0000 7f87dd1e1740 1 --2- 172.21.15.49:0/4100046057 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560c512820d0 0x560c51297130 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f87c80045d0 tx=0x7f87c8015040 comp rx=0 tx=0).stop 2024-03-18T23:08:57.132 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:57.129+0000 7f87dd1e1740 1 -- 172.21.15.49:0/4100046057 shutdown_connections 2024-03-18T23:08:57.132 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:57.129+0000 7f87dd1e1740 1 --2- 172.21.15.49:0/4100046057 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560c5128b7c0 0x560c512a2e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:57.132 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:57.129+0000 7f87dd1e1740 1 --2- 172.21.15.49:0/4100046057 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560c512829f0 0x560c5127b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:57.132 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:57.129+0000 7f87dd1e1740 1 --2- 172.21.15.49:0/4100046057 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560c512820d0 0x560c51297130 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:57.132 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:57.129+0000 7f87dd1e1740 1 -- 172.21.15.49:0/4100046057 >> 172.21.15.49:0/4100046057 conn(0x560c5127c9f0 msgr2=0x560c5127b220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:08:57.132 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:57.129+0000 7f87dd1e1740 1 -- 172.21.15.49:0/4100046057 shutdown_connections 2024-03-18T23:08:57.132 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:57.129+0000 7f87dd1e1740 1 -- 172.21.15.49:0/4100046057 wait complete. 2024-03-18T23:08:57.132 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:57.129+0000 7f87dd1e1740 1 -- start start 2024-03-18T23:08:57.135 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:57.129+0000 7f87dd1e1740 1 Processor -- start 2024-03-18T23:08:57.135 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:57.130+0000 7f87dd1e1740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560c512820d0 0x560c51285040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:57.135 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:57.130+0000 7f87dd1e1740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560c512829f0 0x560c51285580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:57.135 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:57.130+0000 7f87dd1e1740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560c5128b7c0 0x560c51285ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:57.135 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:57.130+0000 7f87dd1e1740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f87d41ac520 con 0x560c5128b7c0 2024-03-18T23:08:57.135 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:57.130+0000 7f87dd1e1740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f87d41abe50 con 0x560c512829f0 2024-03-18T23:08:57.135 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:57.130+0000 7f87dd1e1740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f87d41a9090 con 0x560c512820d0 2024-03-18T23:08:57.135 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:57.130+0000 7f87dd1e1740 10 client.0 _ll_register_callbacks cb 0x7f87c404a070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T23:08:57.135 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:57.130+0000 7f87d37fe640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560c512820d0 0x560c51285040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:57.135 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:57.130+0000 7f87d37fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560c5128b7c0 0x560c51285ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:57.136 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:57.130+0000 7f87d37fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560c5128b7c0 0x560c51285ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:32984/0 (socket says 172.21.15.49:32984) 2024-03-18T23:08:57.136 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:57.130+0000 7f87d3fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560c512829f0 0x560c51285580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:57.136 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:57.130+0000 7f87d37fe640 1 -- 172.21.15.49:0/1866416675 learned_addr learned my addr 172.21.15.49:0/1866416675 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:08:57.136 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:57.130+0000 7f87d37fe640 1 -- 172.21.15.49:0/1866416675 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560c512820d0 msgr2=0x560c51285040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:08:57.136 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:57.130+0000 7f87d37fe640 1 --2- 172.21.15.49:0/1866416675 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560c512820d0 0x560c51285040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:57.136 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:57.130+0000 7f87d37fe640 1 -- 172.21.15.49:0/1866416675 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560c512829f0 msgr2=0x560c51285580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:08:57.136 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:57.130+0000 7f87d37fe640 1 --2- 172.21.15.49:0/1866416675 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560c512829f0 0x560c51285580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:57.136 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:57.130+0000 7f87d37fe640 1 -- 172.21.15.49:0/1866416675 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560c50fc6620 con 0x560c5128b7c0 2024-03-18T23:08:57.136 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:57.130+0000 7f87d37fe640 1 --2- 172.21.15.49:0/1866416675 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560c5128b7c0 0x560c51285ac0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f87c8009710 tx=0x7f87c8005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c3906dc9700da8d server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:08:57.136 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:57.130+0000 7f87b37fe640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-18T23:08:57.136 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:57.130+0000 7f87b37fe640 1 -- 172.21.15.49:0/1866416675 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87c802e040 con 0x560c5128b7c0 2024-03-18T23:08:57.136 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:57.130+0000 7f87dd1e1740 1 -- 172.21.15.49:0/1866416675 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f87c80047a0 con 0x560c5128b7c0 2024-03-18T23:08:57.136 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:57.130+0000 7f87b37fe640 1 -- 172.21.15.49:0/1866416675 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f87c8004b70 con 0x560c5128b7c0 2024-03-18T23:08:57.136 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:57.131+0000 7f87b37fe640 1 -- 172.21.15.49:0/1866416675 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f87c8004d70 con 0x560c5128b7c0 2024-03-18T23:08:57.137 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:57.131+0000 7f87dd1e1740 10 client.15507 fetch_fsmap learned FSMap version 28 2024-03-18T23:08:57.137 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:57.131+0000 7f87dd1e1740 1 -- 172.21.15.49:0/1866416675 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x560c513014a0 con 0x560c5128b7c0 2024-03-18T23:08:57.137 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:57.131+0000 7f87b37fe640 1 -- 172.21.15.49:0/1866416675 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f87c80047a0 con 0x560c5128b7c0 2024-03-18T23:08:57.137 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:57.131+0000 7f87b37fe640 1 -- 172.21.15.49:0/1866416675 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f87c8034040 con 0x560c5128b7c0 2024-03-18T23:08:57.137 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:57.131+0000 7f87dd1e1740 10 client.15507 fetch_fsmap finished waiting for FSMap version 28 2024-03-18T23:08:57.137 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:57.131+0000 7f87dd1e1740 10 client.15507 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T23:08:57.137 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:57.131+0000 7f87dd1e1740 20 client.15507 populate_metadata read hostname 'smithi049' 2024-03-18T23:08:57.137 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:57.131+0000 7f87dd1e1740 1 --2- 172.21.15.49:0/1866416675 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x560c5129b4c0 0x560c51303b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:57.137 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:57.131+0000 7f87dd1e1740 4 client.15507 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T23:08:57.137 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:57.131+0000 7f87dd1e1740 1 -- 172.21.15.49:0/1866416675 --> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560c511ed470 con 0x560c5129b4c0 2024-03-18T23:08:57.137 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:57.131+0000 7f87d3fff640 1 --2- 172.21.15.49:0/1866416675 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x560c5129b4c0 0x560c51303b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:57.137 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:57.132+0000 7f87b37fe640 10 client.15507 ms_handle_connect on v2:172.21.15.139:6838/4280128209 2024-03-18T23:08:57.137 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:57.132+0000 7f87d3fff640 1 --2- 172.21.15.49:0/1866416675 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x560c5129b4c0 0x560c51303b30 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e3fec3afc5cb1b3b server_cookie=347068862dfb81d2 in_seq=0 out_seq=0 2024-03-18T23:08:57.139 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:57.138+0000 7f87b37fe640 1 -- 172.21.15.49:0/1866416675 <== mds.0 v2:172.21.15.139:6838/4280128209 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x560c511ed470 con 0x560c5129b4c0 2024-03-18T23:08:57.139 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:57.138+0000 7f87b37fe640 10 client.15507 handle_command_reply: tid=0 2024-03-18T23:08:57.140 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:57.138+0000 7f87dd1e1740 1 client.15507 shutdown 2024-03-18T23:08:57.140 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:57.138+0000 7f87dd1e1740 1 -- 172.21.15.49:0/1866416675 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560c5128b7c0 msgr2=0x560c51285ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:57.140 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:57.138+0000 7f87dd1e1740 1 --2- 172.21.15.49:0/1866416675 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560c5128b7c0 0x560c51285ac0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f87c8009710 tx=0x7f87c8005f90 comp rx=0 tx=0).stop 2024-03-18T23:08:57.140 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:57.138+0000 7f87dd1e1740 1 -- 172.21.15.49:0/1866416675 shutdown_connections 2024-03-18T23:08:57.140 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:57.138+0000 7f87dd1e1740 1 --2- 172.21.15.49:0/1866416675 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x560c5129b4c0 0x560c51303b30 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:57.140 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:57.138+0000 7f87dd1e1740 1 --2- 172.21.15.49:0/1866416675 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560c5128b7c0 0x560c51285ac0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:57.140 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:57.138+0000 7f87dd1e1740 1 --2- 172.21.15.49:0/1866416675 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560c512829f0 0x560c51285580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:57.140 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:57.138+0000 7f87dd1e1740 1 --2- 172.21.15.49:0/1866416675 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560c512820d0 0x560c51285040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:57.140 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:57.138+0000 7f87b37fe640 0 client.15507 ms_handle_reset on v2:172.21.15.139:6838/4280128209 2024-03-18T23:08:57.140 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:57.138+0000 7f87dd1e1740 1 -- 172.21.15.49:0/1866416675 >> 172.21.15.49:0/1866416675 conn(0x560c5127c9f0 msgr2=0x560c5127bab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:08:57.140 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:57.138+0000 7f87dd1e1740 1 -- 172.21.15.49:0/1866416675 shutdown_connections 2024-03-18T23:08:57.140 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:57.138+0000 7f87dd1e1740 1 -- 172.21.15.49:0/1866416675 wait complete. 2024-03-18T23:08:57.140 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:57.138+0000 7f87dd1e1740 20 client.15507 trim_cache size 0 max 16384 2024-03-18T23:08:57.141 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:{ 2024-03-18T23:08:57.141 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "message": "", 2024-03-18T23:08:57.141 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "return_code": 0 2024-03-18T23:08:57.141 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:} 2024-03-18T23:08:57.142 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:57.139+0000 7f87d939d640 1 -- 172.21.15.49:0/1363679309 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f87bc0a29f0 msgr2=0x7f87bc0a4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:57.142 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:57.139+0000 7f87d939d640 1 --2- 172.21.15.49:0/1363679309 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f87bc0a29f0 0x7f87bc0a4eb0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f87cc0ccc00 tx=0x7f87cc086040 comp rx=0 tx=0).stop 2024-03-18T23:08:57.142 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:57.139+0000 7f87d939d640 1 -- 172.21.15.49:0/1363679309 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f87d41065e0 msgr2=0x7f87d41a3d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:57.142 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:57.139+0000 7f87d939d640 1 --2- 172.21.15.49:0/1363679309 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f87d41065e0 0x7f87d41a3d00 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f87cc00eb10 tx=0x7f87cc00c2d0 comp rx=0 tx=0).stop 2024-03-18T23:08:57.142 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:57.139+0000 7f87d939d640 1 -- 172.21.15.49:0/1363679309 shutdown_connections 2024-03-18T23:08:57.142 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:57.139+0000 7f87d939d640 1 --2- 172.21.15.49:0/1363679309 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f87bc0a29f0 0x7f87bc0a4eb0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:57.142 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:57.139+0000 7f87d939d640 1 --2- 172.21.15.49:0/1363679309 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f87d41065e0 0x7f87d41a3d00 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:57.142 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:57.139+0000 7f87d939d640 1 --2- 172.21.15.49:0/1363679309 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f87d4100ee0 0x7f87d41991c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:57.142 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:57.139+0000 7f87d939d640 1 --2- 172.21.15.49:0/1363679309 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f87d4100520 0x7f87d4198c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:57.142 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:57.139+0000 7f87d939d640 1 -- 172.21.15.49:0/1363679309 >> 172.21.15.49:0/1363679309 conn(0x7f87d40fc030 msgr2=0x7f87d40fd7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:08:57.142 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:57.139+0000 7f87d939d640 1 -- 172.21.15.49:0/1363679309 shutdown_connections 2024-03-18T23:08:57.142 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:08:57.139+0000 7f87d939d640 1 -- 172.21.15.49:0/1363679309 wait complete. 2024-03-18T23:08:57.211 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:08:56 smithi175 ceph-mon[67352]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-03-18T23:08:57.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:08:56 smithi139 ceph-mon[67282]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-03-18T23:08:57.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:08:56 smithi049 ceph-mon[62991]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-03-18T23:08:57.528 INFO:tasks.workunit.client.0.smithi049.stderr:Note: switching to 'e48b1f6c0608b2615e1890cc46286c3877d6e910'. 2024-03-18T23:08:57.528 INFO:tasks.workunit.client.0.smithi049.stderr: 2024-03-18T23:08:57.528 INFO:tasks.workunit.client.0.smithi049.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-03-18T23:08:57.529 INFO:tasks.workunit.client.0.smithi049.stderr:changes and commit them, and you can discard any commits you make in this 2024-03-18T23:08:57.529 INFO:tasks.workunit.client.0.smithi049.stderr:state without impacting any branches by switching back to a branch. 2024-03-18T23:08:57.529 INFO:tasks.workunit.client.0.smithi049.stderr: 2024-03-18T23:08:57.529 INFO:tasks.workunit.client.0.smithi049.stderr:If you want to create a new branch to retain commits you create, you may 2024-03-18T23:08:57.529 INFO:tasks.workunit.client.0.smithi049.stderr:do so (now or later) by using -c with the switch command. Example: 2024-03-18T23:08:57.529 INFO:tasks.workunit.client.0.smithi049.stderr: 2024-03-18T23:08:57.529 INFO:tasks.workunit.client.0.smithi049.stderr: git switch -c 2024-03-18T23:08:57.529 INFO:tasks.workunit.client.0.smithi049.stderr: 2024-03-18T23:08:57.529 INFO:tasks.workunit.client.0.smithi049.stderr:Or undo this operation with: 2024-03-18T23:08:57.529 INFO:tasks.workunit.client.0.smithi049.stderr: 2024-03-18T23:08:57.529 INFO:tasks.workunit.client.0.smithi049.stderr: git switch - 2024-03-18T23:08:57.529 INFO:tasks.workunit.client.0.smithi049.stderr: 2024-03-18T23:08:57.529 INFO:tasks.workunit.client.0.smithi049.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-03-18T23:08:57.529 INFO:tasks.workunit.client.0.smithi049.stderr: 2024-03-18T23:08:57.530 INFO:tasks.workunit.client.0.smithi049.stderr:HEAD is now at e48b1f6c060 testfix 2024-03-18T23:08:57.533 DEBUG:teuthology.orchestra.run.smithi049:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-03-18T23:08:57.595 INFO:tasks.workunit.client.0.smithi049.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-03-18T23:08:57.597 INFO:tasks.workunit.client.0.smithi049.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-03-18T23:08:57.597 INFO:tasks.workunit.client.0.smithi049.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-03-18T23:08:58.368 INFO:tasks.workunit.client.0.smithi049.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-03-18T23:08:58.437 INFO:tasks.workunit.client.0.smithi049.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-03-18T23:08:58.478 INFO:tasks.workunit.client.0.smithi049.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-03-18T23:08:58.480 INFO:tasks.workunit.client.0.smithi049.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-03-18T23:08:58.480 INFO:tasks.workunit.client.0.smithi049.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-03-18T23:08:58.520 INFO:tasks.workunit.client.0.smithi049.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-03-18T23:08:58.524 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-03-18T23:08:58.524 DEBUG:teuthology.orchestra.run.smithi049:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-03-18T23:08:58.580 INFO:tasks.workunit:Running workunits matching suites/dbench.sh on client.0... 2024-03-18T23:08:58.581 INFO:tasks.workunit:Running workunit suites/dbench.sh... 2024-03-18T23:08:58.581 DEBUG:teuthology.orchestra.run.smithi049:workunit test suites/dbench.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=e48b1f6c0608b2615e1890cc46286c3877d6e910 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh 2024-03-18T23:08:58.641 INFO:tasks.workunit.client.0.smithi049.stderr:+ dbench 1 2024-03-18T23:08:58.650 INFO:tasks.workunit.client.0.smithi049.stdout:dbench version 4.00 - Copyright Andrew Tridgell 1999-2004 2024-03-18T23:08:58.650 INFO:tasks.workunit.client.0.smithi049.stdout: 2024-03-18T23:08:58.650 INFO:tasks.workunit.client.0.smithi049.stdout:Running for 600 seconds with load '/usr/share/dbench/client.txt' and minimum warmup 120 secs 2024-03-18T23:08:58.657 INFO:tasks.workunit.client.0.smithi049.stdout:0 of 1 processes prepared for launch 0 sec 2024-03-18T23:08:58.758 INFO:tasks.workunit.client.0.smithi049.stdout:1 of 1 processes prepared for launch 0 sec 2024-03-18T23:08:58.758 INFO:tasks.workunit.client.0.smithi049.stdout:releasing clients 2024-03-18T23:08:59.211 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:08:58 smithi175 ceph-mon[67352]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.2 KiB/s wr, 13 op/s 2024-03-18T23:08:59.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:08:58 smithi139 ceph-mon[67282]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.2 KiB/s wr, 13 op/s 2024-03-18T23:08:59.379 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:08:58 smithi049 ceph-mon[62991]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.2 KiB/s wr, 13 op/s 2024-03-18T23:08:59.758 INFO:tasks.workunit.client.0.smithi049.stdout: 1 3329 35.37 MB/sec warmup 1 sec latency 6.957 ms 2024-03-18T23:09:00.758 INFO:tasks.workunit.client.0.smithi049.stdout: 1 5988 23.61 MB/sec warmup 2 sec latency 14.795 ms 2024-03-18T23:09:01.130 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:09:00 smithi049 ceph-mon[62991]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 938 B/s wr, 8 op/s 2024-03-18T23:09:01.130 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 18 23:09:00 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-x[63224]: 2024-03-18T23:09:00.825+0000 7f2d63a8e700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-18T23:09:01.130 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 18 23:09:00 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-x[63224]: 2024-03-18T23:09:00.825+0000 7f2d63a8e700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-18T23:09:01.130 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 18 23:09:00 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-x[63224]: 2024-03-18T23:09:00.825+0000 7f2d63a8e700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-18T23:09:01.130 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 18 23:09:00 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-x[63224]: 2024-03-18T23:09:00.825+0000 7f2d63a8e700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-18T23:09:01.130 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 18 23:09:00 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-x[63224]: 2024-03-18T23:09:00.825+0000 7f2d63a8e700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-18T23:09:01.211 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:09:00 smithi175 ceph-mon[67352]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 938 B/s wr, 8 op/s 2024-03-18T23:09:01.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:09:00 smithi139 ceph-mon[67282]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 938 B/s wr, 8 op/s 2024-03-18T23:09:01.758 INFO:tasks.workunit.client.0.smithi049.stdout: 1 7673 19.74 MB/sec warmup 3 sec latency 23.380 ms 2024-03-18T23:09:02.758 INFO:tasks.workunit.client.0.smithi049.stdout: 1 10718 19.21 MB/sec warmup 4 sec latency 16.660 ms 2024-03-18T23:09:03.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:09:02 smithi139 ceph-mon[67282]: pgmap v26: 97 pgs: 97 active+clean; 4.0 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 276 KiB/s wr, 67 op/s 2024-03-18T23:09:03.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:09:02 smithi139 ceph-mon[67282]: mgrmap e24: x(active, since 46s), standbys: z, y 2024-03-18T23:09:03.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:09:02 smithi139 ceph-mon[67282]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T23:09:03.379 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:09:02 smithi049 ceph-mon[62991]: pgmap v26: 97 pgs: 97 active+clean; 4.0 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 276 KiB/s wr, 67 op/s 2024-03-18T23:09:03.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:09:02 smithi049 ceph-mon[62991]: mgrmap e24: x(active, since 46s), standbys: z, y 2024-03-18T23:09:03.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:09:02 smithi049 ceph-mon[62991]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T23:09:03.462 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:09:02 smithi175 ceph-mon[67352]: pgmap v26: 97 pgs: 97 active+clean; 4.0 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 276 KiB/s wr, 67 op/s 2024-03-18T23:09:03.462 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:09:02 smithi175 ceph-mon[67352]: mgrmap e24: x(active, since 46s), standbys: z, y 2024-03-18T23:09:03.462 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:09:02 smithi175 ceph-mon[67352]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T23:09:03.758 INFO:tasks.workunit.client.0.smithi049.stdout: 1 13082 17.09 MB/sec warmup 5 sec latency 17.775 ms 2024-03-18T23:09:04.758 INFO:tasks.workunit.client.0.smithi049.stdout: 1 15417 16.59 MB/sec warmup 6 sec latency 23.760 ms 2024-03-18T23:09:05.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:09:04 smithi139 ceph-mon[67282]: pgmap v27: 97 pgs: 97 active+clean; 5.7 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 437 KiB/s wr, 85 op/s 2024-03-18T23:09:05.379 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:09:04 smithi049 ceph-mon[62991]: pgmap v27: 97 pgs: 97 active+clean; 5.7 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 437 KiB/s wr, 85 op/s 2024-03-18T23:09:05.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:09:04 smithi175 ceph-mon[67352]: pgmap v27: 97 pgs: 97 active+clean; 5.7 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 437 KiB/s wr, 85 op/s 2024-03-18T23:09:05.758 INFO:tasks.workunit.client.0.smithi049.stdout: 1 18289 16.65 MB/sec warmup 7 sec latency 11.605 ms 2024-03-18T23:09:06.758 INFO:tasks.workunit.client.0.smithi049.stdout: 1 21442 16.80 MB/sec warmup 8 sec latency 22.621 ms 2024-03-18T23:09:07.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:09:06 smithi139 ceph-mon[67282]: pgmap v28: 97 pgs: 97 active+clean; 5.7 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 436 KiB/s wr, 82 op/s 2024-03-18T23:09:07.379 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:09:06 smithi049 ceph-mon[62991]: pgmap v28: 97 pgs: 97 active+clean; 5.7 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 436 KiB/s wr, 82 op/s 2024-03-18T23:09:07.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:09:06 smithi175 ceph-mon[67352]: pgmap v28: 97 pgs: 97 active+clean; 5.7 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 436 KiB/s wr, 82 op/s 2024-03-18T23:09:07.758 INFO:tasks.workunit.client.0.smithi049.stdout: 1 23883 15.96 MB/sec warmup 9 sec latency 17.864 ms 2024-03-18T23:09:08.758 INFO:tasks.workunit.client.0.smithi049.stdout: 1 25946 15.68 MB/sec warmup 10 sec latency 70.933 ms 2024-03-18T23:09:09.262 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:09:08 smithi175 ceph-mon[67352]: pgmap v29: 97 pgs: 97 active+clean; 13 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 475 KiB/s rd, 1.1 MiB/s wr, 165 op/s 2024-03-18T23:09:09.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:09:08 smithi139 ceph-mon[67282]: pgmap v29: 97 pgs: 97 active+clean; 13 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 475 KiB/s rd, 1.1 MiB/s wr, 165 op/s 2024-03-18T23:09:09.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:09:08 smithi049 ceph-mon[62991]: pgmap v29: 97 pgs: 97 active+clean; 13 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 475 KiB/s rd, 1.1 MiB/s wr, 165 op/s 2024-03-18T23:09:09.758 INFO:tasks.workunit.client.0.smithi049.stdout: 1 28960 15.82 MB/sec warmup 11 sec latency 8.543 ms 2024-03-18T23:09:10.232 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:10.230+0000 7f35e85f4640 1 -- 172.21.15.49:0/1049813449 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f35e0100970 msgr2=0x7f35e0100df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:09:10.232 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:10.230+0000 7f35e85f4640 1 --2- 172.21.15.49:0/1049813449 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f35e0100970 0x7f35e0100df0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f35d0009900 tx=0x7f35d002bc30 comp rx=0 tx=0).stop 2024-03-18T23:09:10.232 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:10.230+0000 7f35e85f4640 1 -- 172.21.15.49:0/1049813449 shutdown_connections 2024-03-18T23:09:10.232 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:10.230+0000 7f35e85f4640 1 --2- 172.21.15.49:0/1049813449 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f35e0101330 0x7f35e010d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:10.232 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:10.230+0000 7f35e85f4640 1 --2- 172.21.15.49:0/1049813449 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f35e0100970 0x7f35e0100df0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:10.232 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:10.230+0000 7f35e85f4640 1 --2- 172.21.15.49:0/1049813449 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f35e01056c0 0x7f35e0105aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:10.232 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:10.230+0000 7f35e85f4640 1 -- 172.21.15.49:0/1049813449 >> 172.21.15.49:0/1049813449 conn(0x7f35e0077570 msgr2=0x7f35e0077980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:09:10.233 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:10.230+0000 7f35e85f4640 1 -- 172.21.15.49:0/1049813449 shutdown_connections 2024-03-18T23:09:10.233 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:10.230+0000 7f35e85f4640 1 -- 172.21.15.49:0/1049813449 wait complete. 2024-03-18T23:09:10.233 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:10.230+0000 7f35e85f4640 1 Processor -- start 2024-03-18T23:09:10.233 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:10.230+0000 7f35e85f4640 1 -- start start 2024-03-18T23:09:10.233 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:10.231+0000 7f35e85f4640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f35e0100970 0x7f35e0102390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:09:10.233 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:10.231+0000 7f35e85f4640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f35e0101330 0x7f35e01028d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:09:10.233 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:10.231+0000 7f35e85f4640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f35e01056c0 0x7f35e0102e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:09:10.233 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:10.231+0000 7f35e85f4640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f35e0110be0 con 0x7f35e0100970 2024-03-18T23:09:10.233 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:10.231+0000 7f35e85f4640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f35e0110a60 con 0x7f35e01056c0 2024-03-18T23:09:10.233 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:10.231+0000 7f35e85f4640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f35e0110d60 con 0x7f35e0101330 2024-03-18T23:09:10.233 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:10.231+0000 7f35e6390640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f35e0100970 0x7f35e0102390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:09:10.233 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:10.231+0000 7f35e6390640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f35e0100970 0x7f35e0102390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:59838/0 (socket says 172.21.15.49:59838) 2024-03-18T23:09:10.233 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:10.231+0000 7f35e6390640 1 -- 172.21.15.49:0/3860852564 learned_addr learned my addr 172.21.15.49:0/3860852564 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:09:10.233 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:10.231+0000 7f35e5b8f640 1 --2- 172.21.15.49:0/3860852564 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f35e0101330 0x7f35e01028d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:09:10.234 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:10.231+0000 7f35e6b91640 1 --2- 172.21.15.49:0/3860852564 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f35e01056c0 0x7f35e0102e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:09:10.234 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:10.231+0000 7f35e6390640 1 -- 172.21.15.49:0/3860852564 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f35e0101330 msgr2=0x7f35e01028d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:09:10.234 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:10.231+0000 7f35e6390640 1 --2- 172.21.15.49:0/3860852564 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f35e0101330 0x7f35e01028d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:10.234 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:10.231+0000 7f35e6390640 1 -- 172.21.15.49:0/3860852564 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f35e01056c0 msgr2=0x7f35e0102e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:09:10.234 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:10.231+0000 7f35e6390640 1 --2- 172.21.15.49:0/3860852564 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f35e01056c0 0x7f35e0102e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:10.234 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:10.231+0000 7f35e6390640 1 -- 172.21.15.49:0/3860852564 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35e01a6630 con 0x7f35e0100970 2024-03-18T23:09:10.234 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:10.231+0000 7f35e6390640 1 --2- 172.21.15.49:0/3860852564 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f35e0100970 0x7f35e0102390 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f35dc00e9f0 tx=0x7f35dc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:09:10.234 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:10.231+0000 7f35d77fe640 1 -- 172.21.15.49:0/3860852564 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35dc00cd80 con 0x7f35e0100970 2024-03-18T23:09:10.234 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:10.231+0000 7f35d77fe640 1 -- 172.21.15.49:0/3860852564 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f35dc010910 con 0x7f35e0100970 2024-03-18T23:09:10.234 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:10.231+0000 7f35e85f4640 1 -- 172.21.15.49:0/3860852564 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35e01a6920 con 0x7f35e0100970 2024-03-18T23:09:10.234 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:10.231+0000 7f35d77fe640 1 -- 172.21.15.49:0/3860852564 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35dc018990 con 0x7f35e0100970 2024-03-18T23:09:10.234 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:10.231+0000 7f35e85f4640 1 -- 172.21.15.49:0/3860852564 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35e01a6e90 con 0x7f35e0100970 2024-03-18T23:09:10.239 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:10.234+0000 7f35d77fe640 1 -- 172.21.15.49:0/3860852564 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 24) ==== 137523+0+0 (secure 0 0 0) 0x7f35dc010ac0 con 0x7f35e0100970 2024-03-18T23:09:10.240 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:10.234+0000 7f35eb1e6740 1 Processor -- start 2024-03-18T23:09:10.240 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:10.234+0000 7f35eb1e6740 1 -- start start 2024-03-18T23:09:10.240 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:10.235+0000 7f35eb1e6740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562dc8f3fef0 0x562dc8f402d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:09:10.240 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:10.235+0000 7f35eb1e6740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562dc8f40810 0x562dc8f490a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:09:10.240 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:10.235+0000 7f35eb1e6740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562dc8f495e0 0x562dc8f4b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:09:10.240 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:10.235+0000 7f35eb1e6740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x562dc8ea3be0 con 0x562dc8f40810 2024-03-18T23:09:10.240 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:10.235+0000 7f35eb1e6740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x562dc8e4c960 con 0x562dc8f3fef0 2024-03-18T23:09:10.240 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:10.235+0000 7f35eb1e6740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x562dc8e9fc80 con 0x562dc8f495e0 2024-03-18T23:09:10.240 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:10.235+0000 7f35d77fe640 1 --2- 172.21.15.49:0/3860852564 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f35cc0a2bb0 0x7f35cc0a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:09:10.240 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:10.235+0000 7f35e5b8f640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562dc8f495e0 0x562dc8f4b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:09:10.240 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:10.235+0000 7f35e6b91640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562dc8f3fef0 0x562dc8f402d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:09:10.240 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:10.235+0000 7f35e6390640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562dc8f40810 0x562dc8f490a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:09:10.240 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:10.235+0000 7f35e5b8f640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562dc8f495e0 0x562dc8f4b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.49:36152/0 (socket says 172.21.15.49:36152) 2024-03-18T23:09:10.240 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:10.235+0000 7f35e5b8f640 1 -- 172.21.15.49:0/911348663 learned_addr learned my addr 172.21.15.49:0/911348663 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:09:10.241 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:10.235+0000 7f35e6b91640 1 --2- 172.21.15.49:0/3860852564 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f35cc0a2bb0 0x7f35cc0a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:09:10.241 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:10.235+0000 7f35d77fe640 1 -- 172.21.15.49:0/3860852564 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f35dc0cf300 con 0x7f35e0100970 2024-03-18T23:09:10.241 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:10.235+0000 7f35e5b8f640 1 -- 172.21.15.49:0/911348663 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562dc8f3fef0 msgr2=0x562dc8f402d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:09:10.241 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:10.235+0000 7f35e5b8f640 1 --2- 172.21.15.49:0/911348663 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562dc8f3fef0 0x562dc8f402d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:10.241 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:10.235+0000 7f35e5b8f640 1 -- 172.21.15.49:0/911348663 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562dc8f40810 msgr2=0x562dc8f490a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:09:10.241 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:10.235+0000 7f35e5b8f640 1 --2- 172.21.15.49:0/911348663 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562dc8f40810 0x562dc8f490a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:10.241 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:10.235+0000 7f35e5b8f640 1 -- 172.21.15.49:0/911348663 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562dc8e43d20 con 0x562dc8f495e0 2024-03-18T23:09:10.241 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:10.236+0000 7f35e6b91640 1 --2- 172.21.15.49:0/3860852564 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f35cc0a2bb0 0x7f35cc0a5070 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f35d8005c40 tx=0x7f35d8004ab0 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:09:10.241 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:10.236+0000 7f35e5b8f640 1 --2- 172.21.15.49:0/911348663 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562dc8f495e0 0x562dc8f4b9d0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f35d0009900 tx=0x7f35d0005be0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f418d8e8c631badf server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:09:10.241 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:10.236+0000 7f35d4bfe640 1 -- 172.21.15.49:0/911348663 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35d0042070 con 0x562dc8f495e0 2024-03-18T23:09:10.241 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:10.236+0000 7f35d4bfe640 1 -- 172.21.15.49:0/911348663 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f35d002c8b0 con 0x562dc8f495e0 2024-03-18T23:09:10.241 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:10.236+0000 7f35d4bfe640 1 -- 172.21.15.49:0/911348663 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35d0004ec0 con 0x562dc8f495e0 2024-03-18T23:09:10.241 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:10.236+0000 7f35eb1e6740 1 -- 172.21.15.49:0/911348663 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562dc8f495e0 msgr2=0x562dc8f4b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:09:10.241 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:10.236+0000 7f35eb1e6740 1 --2- 172.21.15.49:0/911348663 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562dc8f495e0 0x562dc8f4b9d0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f35d0009900 tx=0x7f35d0005be0 comp rx=0 tx=0).stop 2024-03-18T23:09:10.242 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:10.236+0000 7f35eb1e6740 1 -- 172.21.15.49:0/911348663 shutdown_connections 2024-03-18T23:09:10.242 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:10.236+0000 7f35eb1e6740 1 --2- 172.21.15.49:0/911348663 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562dc8f495e0 0x562dc8f4b9d0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:10.242 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:10.236+0000 7f35eb1e6740 1 --2- 172.21.15.49:0/911348663 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562dc8f40810 0x562dc8f490a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:10.242 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:10.236+0000 7f35eb1e6740 1 --2- 172.21.15.49:0/911348663 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562dc8f3fef0 0x562dc8f402d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:10.242 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:10.236+0000 7f35eb1e6740 1 -- 172.21.15.49:0/911348663 >> 172.21.15.49:0/911348663 conn(0x562dc8f3a810 msgr2=0x562dc8f3abf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:09:10.242 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:10.237+0000 7f35eb1e6740 1 -- 172.21.15.49:0/911348663 shutdown_connections 2024-03-18T23:09:10.242 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:10.237+0000 7f35eb1e6740 1 -- 172.21.15.49:0/911348663 wait complete. 2024-03-18T23:09:10.242 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:10.237+0000 7f35eb1e6740 1 -- start start 2024-03-18T23:09:10.242 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:10.237+0000 7f35eb1e6740 1 Processor -- start 2024-03-18T23:09:10.242 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:10.237+0000 7f35eb1e6740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562dc8f3fef0 0x562dc8f48fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:09:10.242 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:10.237+0000 7f35eb1e6740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562dc8f40810 0x562dc8f46330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:09:10.242 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:10.237+0000 7f35eb1e6740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562dc8f495e0 0x562dc8f46940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:09:10.242 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:10.237+0000 7f35eb1e6740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x562dc8f4c1b0 con 0x562dc8f3fef0 2024-03-18T23:09:10.242 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:10.237+0000 7f35eb1e6740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x562dc8e5fdb0 con 0x562dc8f495e0 2024-03-18T23:09:10.243 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:10.237+0000 7f35eb1e6740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x562dc8f4c330 con 0x562dc8f40810 2024-03-18T23:09:10.243 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:10.237+0000 7f35eb1e6740 10 client.0 _ll_register_callbacks cb 0x562dc8e64e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T23:09:10.243 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:10.237+0000 7f35e6390640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562dc8f3fef0 0x562dc8f48fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:09:10.243 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:10.237+0000 7f35e6390640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562dc8f3fef0 0x562dc8f48fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:59850/0 (socket says 172.21.15.49:59850) 2024-03-18T23:09:10.243 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:10.237+0000 7f35e6390640 1 -- 172.21.15.49:0/1104398103 learned_addr learned my addr 172.21.15.49:0/1104398103 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:09:10.243 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:10.237+0000 7f35e6b91640 1 --2- 172.21.15.49:0/1104398103 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562dc8f495e0 0x562dc8f46940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:09:10.243 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:10.237+0000 7f35e5b8f640 1 --2- 172.21.15.49:0/1104398103 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562dc8f40810 0x562dc8f46330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:09:10.243 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:10.238+0000 7f35e5b8f640 1 -- 172.21.15.49:0/1104398103 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562dc8f495e0 msgr2=0x562dc8f46940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:09:10.243 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:10.238+0000 7f35e5b8f640 1 --2- 172.21.15.49:0/1104398103 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562dc8f495e0 0x562dc8f46940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:10.243 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:10.238+0000 7f35e5b8f640 1 -- 172.21.15.49:0/1104398103 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562dc8f3fef0 msgr2=0x562dc8f48fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:09:10.243 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:10.238+0000 7f35e5b8f640 1 --2- 172.21.15.49:0/1104398103 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562dc8f3fef0 0x562dc8f48fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:10.243 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:10.238+0000 7f35e5b8f640 1 -- 172.21.15.49:0/1104398103 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562dc8e5ec60 con 0x562dc8f40810 2024-03-18T23:09:10.243 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:10.238+0000 7f35e6390640 1 --2- 172.21.15.49:0/1104398103 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562dc8f3fef0 0x562dc8f48fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:09:10.243 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:10.238+0000 7f35e5b8f640 1 --2- 172.21.15.49:0/1104398103 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562dc8f40810 0x562dc8f46330 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x562dc8f3bc30 tx=0x7f35d0004840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7d2e1526c426daec server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:09:10.244 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:10.238+0000 7f35c9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-03-18T23:09:10.244 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:10.238+0000 7f35c9ffb640 1 -- 172.21.15.49:0/1104398103 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35d004b070 con 0x562dc8f40810 2024-03-18T23:09:10.244 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:10.238+0000 7f35c9ffb640 1 -- 172.21.15.49:0/1104398103 <== mon.1 v2:172.21.15.175:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f35d0005040 con 0x562dc8f40810 2024-03-18T23:09:10.244 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:10.238+0000 7f35eb1e6740 1 -- 172.21.15.49:0/1104398103 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f35d002c8b0 con 0x562dc8f40810 2024-03-18T23:09:10.244 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:10.238+0000 7f35c9ffb640 1 -- 172.21.15.49:0/1104398103 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f35d0057050 con 0x562dc8f40810 2024-03-18T23:09:10.244 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:10.238+0000 7f35eb1e6740 10 client.25333 fetch_fsmap learned FSMap version 29 2024-03-18T23:09:10.244 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:10.238+0000 7f35eb1e6740 1 -- 172.21.15.49:0/1104398103 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x562dc8bb8210 con 0x562dc8f40810 2024-03-18T23:09:10.244 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:10.239+0000 7f35c9ffb640 1 -- 172.21.15.49:0/1104398103 <== mon.1 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f35d002c8b0 con 0x562dc8f40810 2024-03-18T23:09:10.244 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:10.239+0000 7f35c9ffb640 1 -- 172.21.15.49:0/1104398103 <== mon.1 v2:172.21.15.175:3300/0 5 ==== fsmap(e 29) ==== 8358+0+0 (secure 0 0 0) 0x7f35d0047b10 con 0x562dc8f40810 2024-03-18T23:09:10.244 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:10.239+0000 7f35eb1e6740 10 client.25333 fetch_fsmap finished waiting for FSMap version 29 2024-03-18T23:09:10.244 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:10.239+0000 7f35eb1e6740 10 client.25333 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T23:09:10.244 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:10.239+0000 7f35eb1e6740 20 client.25333 populate_metadata read hostname 'smithi049' 2024-03-18T23:09:10.244 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:10.239+0000 7f35eb1e6740 1 --2- 172.21.15.49:0/1104398103 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x562dc8f592c0 0x562dc8f5b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:09:10.244 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:10.239+0000 7f35eb1e6740 4 client.25333 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T23:09:10.244 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:10.239+0000 7f35eb1e6740 1 -- 172.21.15.49:0/1104398103 --> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562dc8eb1120 con 0x562dc8f592c0 2024-03-18T23:09:10.245 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:10.239+0000 7f35e6390640 1 --2- 172.21.15.49:0/1104398103 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x562dc8f592c0 0x562dc8f5b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:09:10.245 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:10.240+0000 7f35e6390640 1 --2- 172.21.15.49:0/1104398103 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x562dc8f592c0 0x562dc8f5b6b0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=46ac48ae96ff9d08 server_cookie=19ff8a28c791472c in_seq=0 out_seq=0 2024-03-18T23:09:10.245 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:10.240+0000 7f35c9ffb640 10 client.25333 ms_handle_connect on v2:172.21.15.139:6838/4280128209 2024-03-18T23:09:10.246 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:10.242+0000 7f35c9ffb640 1 -- 172.21.15.49:0/1104398103 <== mds.0 v2:172.21.15.139:6838/4280128209 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x562dc8eb1120 con 0x562dc8f592c0 2024-03-18T23:09:10.246 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:10.242+0000 7f35c9ffb640 10 client.25333 handle_command_reply: tid=0 2024-03-18T23:09:10.246 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:10.242+0000 7f35eb1e6740 1 client.25333 shutdown 2024-03-18T23:09:10.246 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:10.242+0000 7f35eb1e6740 1 -- 172.21.15.49:0/1104398103 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562dc8f40810 msgr2=0x562dc8f46330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:09:10.246 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:10.242+0000 7f35eb1e6740 1 --2- 172.21.15.49:0/1104398103 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562dc8f40810 0x562dc8f46330 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x562dc8f3bc30 tx=0x7f35d0004840 comp rx=0 tx=0).stop 2024-03-18T23:09:10.246 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:10.242+0000 7f35eb1e6740 1 -- 172.21.15.49:0/1104398103 shutdown_connections 2024-03-18T23:09:10.246 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:10.242+0000 7f35eb1e6740 1 --2- 172.21.15.49:0/1104398103 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x562dc8f592c0 0x562dc8f5b6b0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:10.246 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:10.242+0000 7f35eb1e6740 1 --2- 172.21.15.49:0/1104398103 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562dc8f495e0 0x562dc8f46940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:10.246 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:10.242+0000 7f35eb1e6740 1 --2- 172.21.15.49:0/1104398103 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562dc8f40810 0x562dc8f46330 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:10.246 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:10.243+0000 7f35c9ffb640 0 client.25333 ms_handle_reset on v2:172.21.15.139:6838/4280128209 2024-03-18T23:09:10.247 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:10.242+0000 7f35eb1e6740 1 --2- 172.21.15.49:0/1104398103 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562dc8f3fef0 0x562dc8f48fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:10.247 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:10.243+0000 7f35eb1e6740 1 -- 172.21.15.49:0/1104398103 >> 172.21.15.49:0/1104398103 conn(0x562dc8f3a810 msgr2=0x562dc8f3c560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:09:10.247 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:10.243+0000 7f35eb1e6740 1 -- 172.21.15.49:0/1104398103 shutdown_connections 2024-03-18T23:09:10.247 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:10.243+0000 7f35eb1e6740 1 -- 172.21.15.49:0/1104398103 wait complete. 2024-03-18T23:09:10.247 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:10.243+0000 7f35eb1e6740 20 client.25333 trim_cache size 0 max 16384 2024-03-18T23:09:10.262 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:10.258+0000 7f35eb1e6740 1 Processor -- start 2024-03-18T23:09:10.262 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:10.258+0000 7f35eb1e6740 1 -- start start 2024-03-18T23:09:10.262 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:10.258+0000 7f35eb1e6740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562dc8f3fef0 0x562dc8f54f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:09:10.262 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:10.258+0000 7f35eb1e6740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562dc8f40810 0x562dc8f3abf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:09:10.262 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:10.258+0000 7f35eb1e6740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562dc8f495e0 0x562dc8f60d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:09:10.262 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:10.258+0000 7f35eb1e6740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f35e01af4e0 con 0x562dc8f40810 2024-03-18T23:09:10.263 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:10.258+0000 7f35eb1e6740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f35e01b1050 con 0x562dc8f495e0 2024-03-18T23:09:10.263 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:10.258+0000 7f35eb1e6740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f35e01afbb0 con 0x562dc8f3fef0 2024-03-18T23:09:10.263 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:10.258+0000 7f35e5b8f640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562dc8f40810 0x562dc8f3abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:09:10.263 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:10.258+0000 7f35e5b8f640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562dc8f40810 0x562dc8f3abf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:59862/0 (socket says 172.21.15.49:59862) 2024-03-18T23:09:10.263 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:10.258+0000 7f35e5b8f640 1 -- 172.21.15.49:0/1917370627 learned_addr learned my addr 172.21.15.49:0/1917370627 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:09:10.263 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:10.258+0000 7f35e6390640 1 --2- 172.21.15.49:0/1917370627 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562dc8f3fef0 0x562dc8f54f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:09:10.263 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:10.258+0000 7f35e6b91640 1 --2- 172.21.15.49:0/1917370627 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562dc8f495e0 0x562dc8f60d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:09:10.263 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:10.259+0000 7f35e5b8f640 1 -- 172.21.15.49:0/1917370627 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562dc8f3fef0 msgr2=0x562dc8f54f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:09:10.263 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:10.259+0000 7f35e5b8f640 1 --2- 172.21.15.49:0/1917370627 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562dc8f3fef0 0x562dc8f54f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:10.263 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:10.259+0000 7f35e5b8f640 1 -- 172.21.15.49:0/1917370627 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562dc8f495e0 msgr2=0x562dc8f60d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:09:10.263 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:10.259+0000 7f35e5b8f640 1 --2- 172.21.15.49:0/1917370627 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562dc8f495e0 0x562dc8f60d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:10.263 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:10.259+0000 7f35e5b8f640 1 -- 172.21.15.49:0/1917370627 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562dc8c84620 con 0x562dc8f40810 2024-03-18T23:09:10.263 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:10.259+0000 7f35e5b8f640 1 --2- 172.21.15.49:0/1917370627 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562dc8f40810 0x562dc8f3abf0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f35d00048b0 tx=0x7f35d0009800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e67a828f5055058 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:09:10.263 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:10.259+0000 7f35cbfff640 1 -- 172.21.15.49:0/1917370627 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35d004b070 con 0x562dc8f40810 2024-03-18T23:09:10.264 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:10.259+0000 7f35cbfff640 1 -- 172.21.15.49:0/1917370627 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f35d002c8b0 con 0x562dc8f40810 2024-03-18T23:09:10.264 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:10.259+0000 7f35cbfff640 1 -- 172.21.15.49:0/1917370627 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35d0042040 con 0x562dc8f40810 2024-03-18T23:09:10.264 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:10.260+0000 7f35eb1e6740 1 -- 172.21.15.49:0/1917370627 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562dc8f40810 msgr2=0x562dc8f3abf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:09:10.264 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:10.260+0000 7f35eb1e6740 1 --2- 172.21.15.49:0/1917370627 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562dc8f40810 0x562dc8f3abf0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f35d00048b0 tx=0x7f35d0009800 comp rx=0 tx=0).stop 2024-03-18T23:09:10.264 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:10.260+0000 7f35eb1e6740 1 -- 172.21.15.49:0/1917370627 shutdown_connections 2024-03-18T23:09:10.264 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:10.260+0000 7f35eb1e6740 1 --2- 172.21.15.49:0/1917370627 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562dc8f495e0 0x562dc8f60d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:10.264 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:10.260+0000 7f35eb1e6740 1 --2- 172.21.15.49:0/1917370627 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562dc8f40810 0x562dc8f3abf0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:10.264 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:10.260+0000 7f35eb1e6740 1 --2- 172.21.15.49:0/1917370627 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562dc8f3fef0 0x562dc8f54f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:10.264 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:10.260+0000 7f35eb1e6740 1 -- 172.21.15.49:0/1917370627 >> 172.21.15.49:0/1917370627 conn(0x562dc8f3a810 msgr2=0x562dc8f39ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:09:10.265 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:10.260+0000 7f35eb1e6740 1 -- 172.21.15.49:0/1917370627 shutdown_connections 2024-03-18T23:09:10.265 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:10.260+0000 7f35eb1e6740 1 -- 172.21.15.49:0/1917370627 wait complete. 2024-03-18T23:09:10.265 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:10.260+0000 7f35eb1e6740 1 -- start start 2024-03-18T23:09:10.265 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:10.260+0000 7f35eb1e6740 1 Processor -- start 2024-03-18T23:09:10.265 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:10.260+0000 7f35eb1e6740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562dc8f3fef0 0x562dc8fb76a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:09:10.265 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:10.260+0000 7f35eb1e6740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562dc8f40810 0x562dc8fb7be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:09:10.265 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:10.260+0000 7f35eb1e6740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562dc8f495e0 0x562dc8f4c640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:09:10.265 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:10.260+0000 7f35eb1e6740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f35e01b0950 con 0x562dc8f40810 2024-03-18T23:09:10.265 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:10.260+0000 7f35eb1e6740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f35e01b0280 con 0x562dc8f3fef0 2024-03-18T23:09:10.265 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:10.260+0000 7f35eb1e6740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f35e01ad4c0 con 0x562dc8f495e0 2024-03-18T23:09:10.265 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:10.260+0000 7f35eb1e6740 10 client.0 _ll_register_callbacks cb 0x7f35dc09b5c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T23:09:10.265 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:10.260+0000 7f35e5b8f640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562dc8f40810 0x562dc8fb7be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:09:10.265 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:10.261+0000 7f35e5b8f640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562dc8f40810 0x562dc8fb7be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:59866/0 (socket says 172.21.15.49:59866) 2024-03-18T23:09:10.265 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:10.261+0000 7f35e6390640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562dc8f3fef0 0x562dc8fb76a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:09:10.266 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:10.261+0000 7f35e5b8f640 1 -- 172.21.15.49:0/3681552302 learned_addr learned my addr 172.21.15.49:0/3681552302 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:09:10.266 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:10.261+0000 7f35e6b91640 1 --2- 172.21.15.49:0/3681552302 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562dc8f495e0 0x562dc8f4c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:09:10.266 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:10.261+0000 7f35e5b8f640 1 -- 172.21.15.49:0/3681552302 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562dc8f495e0 msgr2=0x562dc8f4c640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:09:10.266 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:10.261+0000 7f35e5b8f640 1 --2- 172.21.15.49:0/3681552302 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562dc8f495e0 0x562dc8f4c640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:10.266 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:10.261+0000 7f35e5b8f640 1 -- 172.21.15.49:0/3681552302 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562dc8f3fef0 msgr2=0x562dc8fb76a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:09:10.266 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:10.261+0000 7f35e5b8f640 1 --2- 172.21.15.49:0/3681552302 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562dc8f3fef0 0x562dc8fb76a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:10.266 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:10.261+0000 7f35e5b8f640 1 -- 172.21.15.49:0/3681552302 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562dc8f4cd30 con 0x562dc8f40810 2024-03-18T23:09:10.266 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:10.261+0000 7f35e5b8f640 1 --2- 172.21.15.49:0/3681552302 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562dc8f40810 0x562dc8fb7be0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f35d0048fc0 tx=0x7f35d0000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=916203522e0a4c5f server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:09:10.266 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:10.261+0000 7f35c9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-18T23:09:10.266 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:10.261+0000 7f35c9ffb640 1 -- 172.21.15.49:0/3681552302 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35d0042070 con 0x562dc8f40810 2024-03-18T23:09:10.266 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:10.261+0000 7f35c9ffb640 1 -- 172.21.15.49:0/3681552302 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f35d0034af0 con 0x562dc8f40810 2024-03-18T23:09:10.266 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:10.261+0000 7f35eb1e6740 1 -- 172.21.15.49:0/3681552302 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f35d002c8b0 con 0x562dc8f40810 2024-03-18T23:09:10.266 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:10.261+0000 7f35c9ffb640 1 -- 172.21.15.49:0/3681552302 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f35d004b040 con 0x562dc8f40810 2024-03-18T23:09:10.267 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:10.261+0000 7f35eb1e6740 10 client.15531 fetch_fsmap learned FSMap version 29 2024-03-18T23:09:10.267 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:10.261+0000 7f35eb1e6740 1 -- 172.21.15.49:0/3681552302 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x562dc8fbee50 con 0x562dc8f40810 2024-03-18T23:09:10.267 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:10.261+0000 7f35c9ffb640 1 -- 172.21.15.49:0/3681552302 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f35d002c8b0 con 0x562dc8f40810 2024-03-18T23:09:10.267 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:10.262+0000 7f35c9ffb640 1 -- 172.21.15.49:0/3681552302 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 29) ==== 8358+0+0 (secure 0 0 0) 0x7f35d0064070 con 0x562dc8f40810 2024-03-18T23:09:10.267 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:10.262+0000 7f35eb1e6740 10 client.15531 fetch_fsmap finished waiting for FSMap version 29 2024-03-18T23:09:10.267 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:10.262+0000 7f35eb1e6740 10 client.15531 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T23:09:10.267 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:10.262+0000 7f35eb1e6740 20 client.15531 populate_metadata read hostname 'smithi049' 2024-03-18T23:09:10.267 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:10.262+0000 7f35eb1e6740 1 --2- 172.21.15.49:0/3681552302 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x562dc8fbf460 0x562dc8fc1880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:09:10.267 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:10.262+0000 7f35eb1e6740 4 client.15531 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T23:09:10.267 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:10.262+0000 7f35eb1e6740 1 -- 172.21.15.49:0/3681552302 --> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562dc8eab470 con 0x562dc8fbf460 2024-03-18T23:09:10.267 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:10.262+0000 7f35e6390640 1 --2- 172.21.15.49:0/3681552302 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x562dc8fbf460 0x562dc8fc1880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:09:10.267 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:10.262+0000 7f35c9ffb640 10 client.15531 ms_handle_connect on v2:172.21.15.139:6838/4280128209 2024-03-18T23:09:10.267 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:10.262+0000 7f35e6390640 1 --2- 172.21.15.49:0/3681552302 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x562dc8fbf460 0x562dc8fc1880 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=64396279d5fc52ab server_cookie=a5532e0cda9e6be2 in_seq=0 out_seq=0 2024-03-18T23:09:10.435 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:10.435+0000 7f35c9ffb640 1 -- 172.21.15.49:0/3681552302 <== mds.0 v2:172.21.15.139:6838/4280128209 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x562dc8eab470 con 0x562dc8fbf460 2024-03-18T23:09:10.435 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:10.435+0000 7f35c9ffb640 10 client.15531 handle_command_reply: tid=0 2024-03-18T23:09:10.435 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:10.435+0000 7f35eb1e6740 1 client.15531 shutdown 2024-03-18T23:09:10.435 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:10.435+0000 7f35eb1e6740 1 -- 172.21.15.49:0/3681552302 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562dc8f40810 msgr2=0x562dc8fb7be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:09:10.435 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:10.435+0000 7f35eb1e6740 1 --2- 172.21.15.49:0/3681552302 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562dc8f40810 0x562dc8fb7be0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f35d0048fc0 tx=0x7f35d0000bc0 comp rx=0 tx=0).stop 2024-03-18T23:09:10.436 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:10.435+0000 7f35eb1e6740 1 -- 172.21.15.49:0/3681552302 shutdown_connections 2024-03-18T23:09:10.436 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:10.435+0000 7f35eb1e6740 1 --2- 172.21.15.49:0/3681552302 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x562dc8fbf460 0x562dc8fc1880 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:10.436 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:10.435+0000 7f35eb1e6740 1 --2- 172.21.15.49:0/3681552302 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562dc8f495e0 0x562dc8f4c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:10.436 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:10.435+0000 7f35eb1e6740 1 --2- 172.21.15.49:0/3681552302 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562dc8f40810 0x562dc8fb7be0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:10.436 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:10.435+0000 7f35c9ffb640 0 client.15531 ms_handle_reset on v2:172.21.15.139:6838/4280128209 2024-03-18T23:09:10.436 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:10.435+0000 7f35eb1e6740 1 --2- 172.21.15.49:0/3681552302 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562dc8f3fef0 0x562dc8fb76a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:10.436 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:10.435+0000 7f35eb1e6740 1 -- 172.21.15.49:0/3681552302 >> 172.21.15.49:0/3681552302 conn(0x562dc8f3a810 msgr2=0x562dc8f39ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:09:10.436 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:10.435+0000 7f35eb1e6740 1 -- 172.21.15.49:0/3681552302 shutdown_connections 2024-03-18T23:09:10.436 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:10.435+0000 7f35eb1e6740 1 -- 172.21.15.49:0/3681552302 wait complete. 2024-03-18T23:09:10.436 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:10.435+0000 7f35eb1e6740 20 client.15531 trim_cache size 0 max 16384 2024-03-18T23:09:10.436 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:{ 2024-03-18T23:09:10.436 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "message": "", 2024-03-18T23:09:10.436 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "return_code": 0 2024-03-18T23:09:10.437 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:} 2024-03-18T23:09:10.437 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:10.436+0000 7f35e85f4640 1 -- 172.21.15.49:0/3860852564 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f35cc0a2bb0 msgr2=0x7f35cc0a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:09:10.437 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:10.436+0000 7f35e85f4640 1 --2- 172.21.15.49:0/3860852564 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f35cc0a2bb0 0x7f35cc0a5070 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f35d8005c40 tx=0x7f35d8004ab0 comp rx=0 tx=0).stop 2024-03-18T23:09:10.437 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:10.436+0000 7f35e85f4640 1 -- 172.21.15.49:0/3860852564 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f35e0100970 msgr2=0x7f35e0102390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:09:10.438 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:10.436+0000 7f35e85f4640 1 --2- 172.21.15.49:0/3860852564 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f35e0100970 0x7f35e0102390 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f35dc00e9f0 tx=0x7f35dc00eeb0 comp rx=0 tx=0).stop 2024-03-18T23:09:10.438 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:10.436+0000 7f35e85f4640 1 -- 172.21.15.49:0/3860852564 shutdown_connections 2024-03-18T23:09:10.438 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:10.436+0000 7f35e85f4640 1 --2- 172.21.15.49:0/3860852564 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f35cc0a2bb0 0x7f35cc0a5070 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:10.438 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:10.436+0000 7f35e85f4640 1 --2- 172.21.15.49:0/3860852564 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f35e01056c0 0x7f35e0102e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:10.438 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:10.436+0000 7f35e85f4640 1 --2- 172.21.15.49:0/3860852564 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f35e0101330 0x7f35e01028d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:10.438 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:10.436+0000 7f35e85f4640 1 --2- 172.21.15.49:0/3860852564 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f35e0100970 0x7f35e0102390 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:10.438 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:10.436+0000 7f35e85f4640 1 -- 172.21.15.49:0/3860852564 >> 172.21.15.49:0/3860852564 conn(0x7f35e0077570 msgr2=0x7f35e00691e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:09:10.438 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:10.436+0000 7f35e85f4640 1 -- 172.21.15.49:0/3860852564 shutdown_connections 2024-03-18T23:09:10.438 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:10.436+0000 7f35e85f4640 1 -- 172.21.15.49:0/3860852564 wait complete. 2024-03-18T23:09:10.758 INFO:tasks.workunit.client.0.smithi049.stdout: 1 31340 15.54 MB/sec warmup 12 sec latency 101.426 ms 2024-03-18T23:09:11.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:09:10 smithi139 ceph-mon[67282]: pgmap v30: 97 pgs: 97 active+clean; 13 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 473 KiB/s rd, 1.1 MiB/s wr, 160 op/s 2024-03-18T23:09:11.379 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:09:10 smithi049 ceph-mon[62991]: pgmap v30: 97 pgs: 97 active+clean; 13 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 473 KiB/s rd, 1.1 MiB/s wr, 160 op/s 2024-03-18T23:09:11.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:09:10 smithi175 ceph-mon[67352]: pgmap v30: 97 pgs: 97 active+clean; 13 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 473 KiB/s rd, 1.1 MiB/s wr, 160 op/s 2024-03-18T23:09:11.758 INFO:tasks.workunit.client.0.smithi049.stdout: 1 33417 15.24 MB/sec warmup 13 sec latency 41.097 ms 2024-03-18T23:09:12.759 INFO:tasks.workunit.client.0.smithi049.stdout: 1 36169 15.17 MB/sec warmup 14 sec latency 8.223 ms 2024-03-18T23:09:13.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:09:12 smithi139 ceph-mon[67282]: pgmap v31: 97 pgs: 97 active+clean; 10 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 476 KiB/s rd, 1.3 MiB/s wr, 230 op/s 2024-03-18T23:09:13.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:09:12 smithi049 ceph-mon[62991]: pgmap v31: 97 pgs: 97 active+clean; 10 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 476 KiB/s rd, 1.3 MiB/s wr, 230 op/s 2024-03-18T23:09:13.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:09:12 smithi175 ceph-mon[67352]: pgmap v31: 97 pgs: 97 active+clean; 10 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 476 KiB/s rd, 1.3 MiB/s wr, 230 op/s 2024-03-18T23:09:13.759 INFO:tasks.workunit.client.0.smithi049.stdout: 1 39169 15.33 MB/sec warmup 15 sec latency 16.023 ms 2024-03-18T23:09:14.759 INFO:tasks.workunit.client.0.smithi049.stdout: 1 41816 15.15 MB/sec warmup 16 sec latency 14.461 ms 2024-03-18T23:09:15.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:09:14 smithi139 ceph-mon[67282]: pgmap v32: 97 pgs: 97 active+clean; 11 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 737 KiB/s rd, 1.5 MiB/s wr, 291 op/s 2024-03-18T23:09:15.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:09:14 smithi049 ceph-mon[62991]: pgmap v32: 97 pgs: 97 active+clean; 11 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 737 KiB/s rd, 1.5 MiB/s wr, 291 op/s 2024-03-18T23:09:15.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:09:14 smithi175 ceph-mon[67352]: pgmap v32: 97 pgs: 97 active+clean; 11 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 737 KiB/s rd, 1.5 MiB/s wr, 291 op/s 2024-03-18T23:09:15.759 INFO:tasks.workunit.client.0.smithi049.stdout: 1 44086 15.03 MB/sec warmup 17 sec latency 16.315 ms 2024-03-18T23:09:16.759 INFO:tasks.workunit.client.0.smithi049.stdout: 1 46687 14.98 MB/sec warmup 18 sec latency 25.438 ms 2024-03-18T23:09:17.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:09:16 smithi139 ceph-mon[67282]: pgmap v33: 97 pgs: 97 active+clean; 11 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 736 KiB/s rd, 1.4 MiB/s wr, 271 op/s 2024-03-18T23:09:17.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:09:16 smithi049 ceph-mon[62991]: pgmap v33: 97 pgs: 97 active+clean; 11 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 736 KiB/s rd, 1.4 MiB/s wr, 271 op/s 2024-03-18T23:09:17.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:09:16 smithi175 ceph-mon[67352]: pgmap v33: 97 pgs: 97 active+clean; 11 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 736 KiB/s rd, 1.4 MiB/s wr, 271 op/s 2024-03-18T23:09:17.759 INFO:tasks.workunit.client.0.smithi049.stdout: 1 49821 15.12 MB/sec warmup 19 sec latency 27.545 ms 2024-03-18T23:09:18.759 INFO:tasks.workunit.client.0.smithi049.stdout: 1 52261 14.83 MB/sec warmup 20 sec latency 18.311 ms 2024-03-18T23:09:19.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:09:18 smithi139 ceph-mon[67282]: pgmap v34: 97 pgs: 97 active+clean; 19 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.1 MiB/s wr, 367 op/s 2024-03-18T23:09:19.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:09:18 smithi049 ceph-mon[62991]: pgmap v34: 97 pgs: 97 active+clean; 19 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.1 MiB/s wr, 367 op/s 2024-03-18T23:09:19.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:09:18 smithi175 ceph-mon[67352]: pgmap v34: 97 pgs: 97 active+clean; 19 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.1 MiB/s wr, 367 op/s 2024-03-18T23:09:19.759 INFO:tasks.workunit.client.0.smithi049.stdout: 1 54391 14.75 MB/sec warmup 21 sec latency 25.558 ms 2024-03-18T23:09:20.759 INFO:tasks.workunit.client.0.smithi049.stdout: 1 57199 14.86 MB/sec warmup 22 sec latency 11.186 ms 2024-03-18T23:09:21.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:09:20 smithi139 ceph-mon[67282]: pgmap v35: 97 pgs: 97 active+clean; 19 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 822 KiB/s rd, 1.5 MiB/s wr, 284 op/s 2024-03-18T23:09:21.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:09:20 smithi139 ceph-mon[67282]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:09:21.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:09:20 smithi139 ceph-mon[67282]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' 2024-03-18T23:09:21.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:09:20 smithi139 ceph-mon[67282]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' 2024-03-18T23:09:21.379 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:09:20 smithi049 ceph-mon[62991]: pgmap v35: 97 pgs: 97 active+clean; 19 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 822 KiB/s rd, 1.5 MiB/s wr, 284 op/s 2024-03-18T23:09:21.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:09:20 smithi049 ceph-mon[62991]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:09:21.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:09:20 smithi049 ceph-mon[62991]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' 2024-03-18T23:09:21.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:09:20 smithi049 ceph-mon[62991]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' 2024-03-18T23:09:21.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:09:20 smithi175 ceph-mon[67352]: pgmap v35: 97 pgs: 97 active+clean; 19 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 822 KiB/s rd, 1.5 MiB/s wr, 284 op/s 2024-03-18T23:09:21.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:09:20 smithi175 ceph-mon[67352]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:09:21.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:09:20 smithi175 ceph-mon[67352]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' 2024-03-18T23:09:21.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:09:20 smithi175 ceph-mon[67352]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' 2024-03-18T23:09:21.759 INFO:tasks.workunit.client.0.smithi049.stdout: 1 59791 14.76 MB/sec warmup 23 sec latency 32.601 ms 2024-03-18T23:09:22.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:09:21 smithi139 ceph-mon[67282]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:09:22.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:09:21 smithi139 ceph-mon[67282]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:09:22.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:09:21 smithi139 ceph-mon[67282]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' 2024-03-18T23:09:22.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:09:21 smithi049 ceph-mon[62991]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:09:22.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:09:21 smithi049 ceph-mon[62991]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:09:22.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:09:21 smithi049 ceph-mon[62991]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' 2024-03-18T23:09:22.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:09:21 smithi175 ceph-mon[67352]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:09:22.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:09:21 smithi175 ceph-mon[67352]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:09:22.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:09:21 smithi175 ceph-mon[67352]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' 2024-03-18T23:09:22.759 INFO:tasks.workunit.client.0.smithi049.stdout: 1 62190 14.64 MB/sec warmup 24 sec latency 19.495 ms 2024-03-18T23:09:23.281 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:09:22 smithi049 ceph-mon[62991]: pgmap v36: 97 pgs: 97 active+clean; 25 MiB data, 469 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.1 MiB/s wr, 361 op/s 2024-03-18T23:09:23.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:09:22 smithi139 ceph-mon[67282]: pgmap v36: 97 pgs: 97 active+clean; 25 MiB data, 469 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.1 MiB/s wr, 361 op/s 2024-03-18T23:09:23.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:09:22 smithi175 ceph-mon[67352]: pgmap v36: 97 pgs: 97 active+clean; 25 MiB data, 469 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.1 MiB/s wr, 361 op/s 2024-03-18T23:09:23.533 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:23.530+0000 7fca440ac640 1 -- 172.21.15.49:0/1776277612 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fca3c102a00 msgr2=0x7fca3c102de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:09:23.533 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:23.530+0000 7fca440ac640 1 --2- 172.21.15.49:0/1776277612 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fca3c102a00 0x7fca3c102de0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fca38009900 tx=0x7fca3802bc30 comp rx=0 tx=0).stop 2024-03-18T23:09:23.534 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:23.530+0000 7fca440ac640 1 -- 172.21.15.49:0/1776277612 shutdown_connections 2024-03-18T23:09:23.534 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:23.530+0000 7fca440ac640 1 --2- 172.21.15.49:0/1776277612 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fca3c10be70 0x7fca3c10e2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:23.534 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:23.530+0000 7fca440ac640 1 --2- 172.21.15.49:0/1776277612 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fca3c103320 0x7fca3c10b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:23.534 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:23.530+0000 7fca440ac640 1 --2- 172.21.15.49:0/1776277612 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fca3c102a00 0x7fca3c102de0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:23.534 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:23.530+0000 7fca440ac640 1 -- 172.21.15.49:0/1776277612 >> 172.21.15.49:0/1776277612 conn(0x7fca3c077850 msgr2=0x7fca3c077c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:09:23.534 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:23.530+0000 7fca440ac640 1 -- 172.21.15.49:0/1776277612 shutdown_connections 2024-03-18T23:09:23.534 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:23.530+0000 7fca440ac640 1 -- 172.21.15.49:0/1776277612 wait complete. 2024-03-18T23:09:23.534 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:23.531+0000 7fca440ac640 1 Processor -- start 2024-03-18T23:09:23.534 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:23.531+0000 7fca440ac640 1 -- start start 2024-03-18T23:09:23.534 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:23.531+0000 7fca440ac640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fca3c102a00 0x7fca3c198020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:09:23.534 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:23.531+0000 7fca440ac640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fca3c103320 0x7fca3c198560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:09:23.535 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:23.531+0000 7fca440ac640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fca3c10be70 0x7fca3c19c890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:09:23.535 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:23.531+0000 7fca440ac640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fca3c111c40 con 0x7fca3c103320 2024-03-18T23:09:23.535 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:23.531+0000 7fca440ac640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fca3c111ac0 con 0x7fca3c102a00 2024-03-18T23:09:23.535 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:23.531+0000 7fca440ac640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fca3c111dc0 con 0x7fca3c10be70 2024-03-18T23:09:23.535 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:23.531+0000 7fca41647640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fca3c103320 0x7fca3c198560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:09:23.535 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:23.531+0000 7fca41e48640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fca3c102a00 0x7fca3c198020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:09:23.535 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:23.531+0000 7fca41647640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fca3c103320 0x7fca3c198560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:52476/0 (socket says 172.21.15.49:52476) 2024-03-18T23:09:23.535 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:23.531+0000 7fca41647640 1 -- 172.21.15.49:0/4271354409 learned_addr learned my addr 172.21.15.49:0/4271354409 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:09:23.535 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:23.531+0000 7fca41e48640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fca3c102a00 0x7fca3c198020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.49:55160/0 (socket says 172.21.15.49:55160) 2024-03-18T23:09:23.535 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:23.531+0000 7fca42649640 1 --2- 172.21.15.49:0/4271354409 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fca3c10be70 0x7fca3c19c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:09:23.535 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:23.532+0000 7fca41647640 1 -- 172.21.15.49:0/4271354409 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fca3c10be70 msgr2=0x7fca3c19c890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:09:23.535 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:23.532+0000 7fca41647640 1 --2- 172.21.15.49:0/4271354409 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fca3c10be70 0x7fca3c19c890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:23.535 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:23.532+0000 7fca41647640 1 -- 172.21.15.49:0/4271354409 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fca3c102a00 msgr2=0x7fca3c198020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:09:23.535 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:23.532+0000 7fca41647640 1 --2- 172.21.15.49:0/4271354409 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fca3c102a00 0x7fca3c198020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:23.538 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:23.532+0000 7fca41647640 1 -- 172.21.15.49:0/4271354409 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca3c19cfc0 con 0x7fca3c103320 2024-03-18T23:09:23.539 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:23.532+0000 7fca41647640 1 --2- 172.21.15.49:0/4271354409 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fca3c103320 0x7fca3c198560 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fca2c00e9f0 tx=0x7fca2c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:09:23.539 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:23.532+0000 7fca32ffd640 1 -- 172.21.15.49:0/4271354409 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca2c00ccc0 con 0x7fca3c103320 2024-03-18T23:09:23.539 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:23.532+0000 7fca32ffd640 1 -- 172.21.15.49:0/4271354409 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca2c010910 con 0x7fca3c103320 2024-03-18T23:09:23.539 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:23.532+0000 7fca32ffd640 1 -- 172.21.15.49:0/4271354409 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca2c0188a0 con 0x7fca3c103320 2024-03-18T23:09:23.539 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:23.532+0000 7fca440ac640 1 -- 172.21.15.49:0/4271354409 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca3c19d2b0 con 0x7fca3c103320 2024-03-18T23:09:23.539 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:23.532+0000 7fca440ac640 1 -- 172.21.15.49:0/4271354409 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca3c1a4b40 con 0x7fca3c103320 2024-03-18T23:09:23.539 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:23.535+0000 7fca32ffd640 1 -- 172.21.15.49:0/4271354409 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 24) ==== 137523+0+0 (secure 0 0 0) 0x7fca2c010ac0 con 0x7fca3c103320 2024-03-18T23:09:23.539 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:23.535+0000 7fca46d74740 1 Processor -- start 2024-03-18T23:09:23.539 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:23.535+0000 7fca46d74740 1 -- start start 2024-03-18T23:09:23.539 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:23.535+0000 7fca46d74740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fd0664fef0 0x55fd066502d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:09:23.539 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:23.535+0000 7fca46d74740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fd06650810 0x55fd066590a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:09:23.539 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:23.535+0000 7fca46d74740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55fd066595e0 0x55fd0665b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:09:23.540 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:23.535+0000 7fca46d74740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55fd065b3be0 con 0x55fd0664fef0 2024-03-18T23:09:23.540 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:23.535+0000 7fca46d74740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55fd0655c960 con 0x55fd066595e0 2024-03-18T23:09:23.540 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:23.535+0000 7fca46d74740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55fd065afc80 con 0x55fd06650810 2024-03-18T23:09:23.540 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:23.536+0000 7fca42649640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fd0664fef0 0x55fd066502d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:09:23.540 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:23.536+0000 7fca42649640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fd0664fef0 0x55fd066502d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:52482/0 (socket says 172.21.15.49:52482) 2024-03-18T23:09:23.540 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:23.536+0000 7fca42649640 1 -- 172.21.15.49:0/1557373090 learned_addr learned my addr 172.21.15.49:0/1557373090 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:09:23.540 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:23.536+0000 7fca41e48640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fd06650810 0x55fd066590a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:09:23.540 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:23.536+0000 7fca41647640 1 --2- 172.21.15.49:0/1557373090 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55fd066595e0 0x55fd0665b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:09:23.540 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:23.536+0000 7fca32ffd640 1 --2- 172.21.15.49:0/4271354409 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7fca280a2bb0 0x7fca280a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:09:23.540 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:23.536+0000 7fca42649640 1 -- 172.21.15.49:0/1557373090 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fd06650810 msgr2=0x55fd066590a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:09:23.540 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:23.536+0000 7fca42649640 1 --2- 172.21.15.49:0/1557373090 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fd06650810 0x55fd066590a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:23.540 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:23.536+0000 7fca42649640 1 -- 172.21.15.49:0/1557373090 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55fd066595e0 msgr2=0x55fd0665b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:09:23.540 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:23.536+0000 7fca42649640 1 --2- 172.21.15.49:0/1557373090 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55fd066595e0 0x55fd0665b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:23.541 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:23.536+0000 7fca42649640 1 -- 172.21.15.49:0/1557373090 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fd06553d20 con 0x55fd0664fef0 2024-03-18T23:09:23.541 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:23.536+0000 7fca32ffd640 1 -- 172.21.15.49:0/4271354409 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7fca2c0cf470 con 0x7fca3c103320 2024-03-18T23:09:23.541 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:23.536+0000 7fca42649640 1 --2- 172.21.15.49:0/4271354409 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7fca280a2bb0 0x7fca280a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:09:23.541 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:23.536+0000 7fca42649640 1 --2- 172.21.15.49:0/1557373090 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fd0664fef0 0x55fd066502d0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fca34011440 tx=0x7fca34011900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=723a1995bfe4e4c6 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:09:23.541 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:23.536+0000 7fca27fff640 1 -- 172.21.15.49:0/1557373090 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca34004250 con 0x55fd0664fef0 2024-03-18T23:09:23.541 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:23.536+0000 7fca27fff640 1 -- 172.21.15.49:0/1557373090 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca34003cd0 con 0x55fd0664fef0 2024-03-18T23:09:23.541 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:23.536+0000 7fca27fff640 1 -- 172.21.15.49:0/1557373090 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca3401bc60 con 0x55fd0664fef0 2024-03-18T23:09:23.541 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:23.537+0000 7fca46d74740 1 -- 172.21.15.49:0/1557373090 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fd0664fef0 msgr2=0x55fd066502d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:09:23.541 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:23.537+0000 7fca42649640 1 --2- 172.21.15.49:0/4271354409 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7fca280a2bb0 0x7fca280a5070 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fca34020000 tx=0x7fca34021000 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:09:23.541 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:23.537+0000 7fca46d74740 1 --2- 172.21.15.49:0/1557373090 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fd0664fef0 0x55fd066502d0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fca34011440 tx=0x7fca34011900 comp rx=0 tx=0).stop 2024-03-18T23:09:23.541 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:23.537+0000 7fca46d74740 1 -- 172.21.15.49:0/1557373090 shutdown_connections 2024-03-18T23:09:23.541 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:23.537+0000 7fca46d74740 1 --2- 172.21.15.49:0/1557373090 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55fd066595e0 0x55fd0665b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:23.542 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:23.537+0000 7fca46d74740 1 --2- 172.21.15.49:0/1557373090 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fd06650810 0x55fd066590a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:23.542 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:23.537+0000 7fca46d74740 1 --2- 172.21.15.49:0/1557373090 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fd0664fef0 0x55fd066502d0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:23.542 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:23.537+0000 7fca46d74740 1 -- 172.21.15.49:0/1557373090 >> 172.21.15.49:0/1557373090 conn(0x55fd0664a810 msgr2=0x55fd0664abf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:09:23.542 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:23.538+0000 7fca46d74740 1 -- 172.21.15.49:0/1557373090 shutdown_connections 2024-03-18T23:09:23.542 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:23.538+0000 7fca46d74740 1 -- 172.21.15.49:0/1557373090 wait complete. 2024-03-18T23:09:23.544 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:23.538+0000 7fca46d74740 1 -- start start 2024-03-18T23:09:23.545 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:23.538+0000 7fca46d74740 1 Processor -- start 2024-03-18T23:09:23.545 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:23.539+0000 7fca46d74740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fd0664fef0 0x55fd06658f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:09:23.545 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:23.539+0000 7fca46d74740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55fd06650810 0x55fd066562e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:09:23.545 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:23.539+0000 7fca46d74740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fd066595e0 0x55fd06656840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:09:23.545 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:23.539+0000 7fca46d74740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55fd0665c210 con 0x55fd066595e0 2024-03-18T23:09:23.545 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:23.539+0000 7fca46d74740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55fd0656fdb0 con 0x55fd06650810 2024-03-18T23:09:23.545 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:23.539+0000 7fca46d74740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55fd0665c390 con 0x55fd0664fef0 2024-03-18T23:09:23.545 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:23.539+0000 7fca46d74740 10 client.0 _ll_register_callbacks cb 0x55fd06574e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T23:09:23.545 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:23.539+0000 7fca42649640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fd066595e0 0x55fd06656840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:09:23.545 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:23.539+0000 7fca41647640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55fd06650810 0x55fd066562e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:09:23.545 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:23.539+0000 7fca42649640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fd066595e0 0x55fd06656840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:52494/0 (socket says 172.21.15.49:52494) 2024-03-18T23:09:23.546 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:23.539+0000 7fca41e48640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fd0664fef0 0x55fd06658f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:09:23.546 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:23.539+0000 7fca42649640 1 -- 172.21.15.49:0/2390674678 learned_addr learned my addr 172.21.15.49:0/2390674678 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:09:23.546 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:23.539+0000 7fca42649640 1 -- 172.21.15.49:0/2390674678 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fd0664fef0 msgr2=0x55fd06658f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:09:23.546 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:23.539+0000 7fca42649640 1 --2- 172.21.15.49:0/2390674678 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fd0664fef0 0x55fd06658f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:23.546 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:23.539+0000 7fca42649640 1 -- 172.21.15.49:0/2390674678 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55fd06650810 msgr2=0x55fd066562e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:09:23.546 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:23.539+0000 7fca42649640 1 --2- 172.21.15.49:0/2390674678 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55fd06650810 0x55fd066562e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:23.546 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:23.539+0000 7fca42649640 1 -- 172.21.15.49:0/2390674678 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fd0656ec60 con 0x55fd066595e0 2024-03-18T23:09:23.546 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:23.539+0000 7fca42649640 1 --2- 172.21.15.49:0/2390674678 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fd066595e0 0x55fd06656840 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fca3401b7a0 tx=0x7fca3401b940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7c5afad880dfe78 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:09:23.546 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:23.539+0000 7fca257fa640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-18T23:09:23.547 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:23.540+0000 7fca257fa640 1 -- 172.21.15.49:0/2390674678 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca34033020 con 0x55fd066595e0 2024-03-18T23:09:23.547 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:23.540+0000 7fca257fa640 1 -- 172.21.15.49:0/2390674678 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7fca34004250 con 0x55fd066595e0 2024-03-18T23:09:23.547 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:23.540+0000 7fca46d74740 1 -- 172.21.15.49:0/2390674678 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fca34003cd0 con 0x55fd066595e0 2024-03-18T23:09:23.547 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:23.540+0000 7fca257fa640 1 -- 172.21.15.49:0/2390674678 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fca3403e050 con 0x55fd066595e0 2024-03-18T23:09:23.547 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:23.540+0000 7fca46d74740 10 client.15543 fetch_fsmap learned FSMap version 29 2024-03-18T23:09:23.547 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:23.540+0000 7fca46d74740 1 -- 172.21.15.49:0/2390674678 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55fd062c8210 con 0x55fd066595e0 2024-03-18T23:09:23.547 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:23.540+0000 7fca257fa640 1 -- 172.21.15.49:0/2390674678 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca34003cd0 con 0x55fd066595e0 2024-03-18T23:09:23.547 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:23.541+0000 7fca257fa640 1 -- 172.21.15.49:0/2390674678 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 29) ==== 8358+0+0 (secure 0 0 0) 0x7fca34005050 con 0x55fd066595e0 2024-03-18T23:09:23.547 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:23.541+0000 7fca46d74740 10 client.15543 fetch_fsmap finished waiting for FSMap version 29 2024-03-18T23:09:23.547 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:23.541+0000 7fca46d74740 10 client.15543 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T23:09:23.547 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:23.541+0000 7fca46d74740 20 client.15543 populate_metadata read hostname 'smithi049' 2024-03-18T23:09:23.547 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:23.541+0000 7fca46d74740 1 --2- 172.21.15.49:0/2390674678 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x55fd06669370 0x55fd0666b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:09:23.547 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:23.541+0000 7fca46d74740 4 client.15543 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T23:09:23.548 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:23.541+0000 7fca46d74740 1 -- 172.21.15.49:0/2390674678 --> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fd065c1120 con 0x55fd06669370 2024-03-18T23:09:23.548 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:23.541+0000 7fca41e48640 1 --2- 172.21.15.49:0/2390674678 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x55fd06669370 0x55fd0666b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:09:23.548 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:23.541+0000 7fca41e48640 1 --2- 172.21.15.49:0/2390674678 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x55fd06669370 0x55fd0666b760 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6f7030681f34f09d server_cookie=a12c40b0bbe7b708 in_seq=0 out_seq=0 2024-03-18T23:09:23.548 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:23.541+0000 7fca257fa640 10 client.15543 ms_handle_connect on v2:172.21.15.139:6838/4280128209 2024-03-18T23:09:23.549 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:23.544+0000 7fca257fa640 1 -- 172.21.15.49:0/2390674678 <== mds.0 v2:172.21.15.139:6838/4280128209 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55fd065c1120 con 0x55fd06669370 2024-03-18T23:09:23.549 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:23.544+0000 7fca257fa640 10 client.15543 handle_command_reply: tid=0 2024-03-18T23:09:23.549 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:23.544+0000 7fca46d74740 1 client.15543 shutdown 2024-03-18T23:09:23.549 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:23.544+0000 7fca46d74740 1 -- 172.21.15.49:0/2390674678 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fd066595e0 msgr2=0x55fd06656840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:09:23.549 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:23.544+0000 7fca46d74740 1 --2- 172.21.15.49:0/2390674678 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fd066595e0 0x55fd06656840 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fca3401b7a0 tx=0x7fca3401b940 comp rx=0 tx=0).stop 2024-03-18T23:09:23.549 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:23.544+0000 7fca46d74740 1 -- 172.21.15.49:0/2390674678 shutdown_connections 2024-03-18T23:09:23.549 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:23.544+0000 7fca46d74740 1 --2- 172.21.15.49:0/2390674678 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x55fd06669370 0x55fd0666b760 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:23.550 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:23.544+0000 7fca46d74740 1 --2- 172.21.15.49:0/2390674678 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fd066595e0 0x55fd06656840 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:23.550 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:23.544+0000 7fca46d74740 1 --2- 172.21.15.49:0/2390674678 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55fd06650810 0x55fd066562e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:23.550 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:23.544+0000 7fca46d74740 1 --2- 172.21.15.49:0/2390674678 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fd0664fef0 0x55fd06658f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:23.550 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:23.544+0000 7fca257fa640 0 client.15543 ms_handle_reset on v2:172.21.15.139:6838/4280128209 2024-03-18T23:09:23.550 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:23.544+0000 7fca46d74740 1 -- 172.21.15.49:0/2390674678 >> 172.21.15.49:0/2390674678 conn(0x55fd0664a810 msgr2=0x55fd0664c560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:09:23.550 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:23.544+0000 7fca46d74740 1 -- 172.21.15.49:0/2390674678 shutdown_connections 2024-03-18T23:09:23.550 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:23.544+0000 7fca46d74740 1 -- 172.21.15.49:0/2390674678 wait complete. 2024-03-18T23:09:23.550 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:23.544+0000 7fca46d74740 20 client.15543 trim_cache size 0 max 16384 2024-03-18T23:09:23.563 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:23.560+0000 7fca46d74740 1 Processor -- start 2024-03-18T23:09:23.563 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:23.560+0000 7fca46d74740 1 -- start start 2024-03-18T23:09:23.563 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:23.560+0000 7fca46d74740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fd0664fef0 0x55fd06664fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:09:23.564 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:23.560+0000 7fca46d74740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55fd06650810 0x55fd06670f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:09:23.564 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:23.560+0000 7fca46d74740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fd066595e0 0x55fd06658a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:09:23.564 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:23.560+0000 7fca46d74740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fca3c1ad350 con 0x55fd0664fef0 2024-03-18T23:09:23.564 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:23.560+0000 7fca46d74740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fca3c1aeec0 con 0x55fd06650810 2024-03-18T23:09:23.564 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:23.560+0000 7fca46d74740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fca3c1ada20 con 0x55fd066595e0 2024-03-18T23:09:23.564 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:23.560+0000 7fca41e48640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fd0664fef0 0x55fd06664fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:09:23.564 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:23.560+0000 7fca41e48640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fd0664fef0 0x55fd06664fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:52510/0 (socket says 172.21.15.49:52510) 2024-03-18T23:09:23.564 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:23.560+0000 7fca41e48640 1 -- 172.21.15.49:0/2815943831 learned_addr learned my addr 172.21.15.49:0/2815943831 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:09:23.564 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:23.560+0000 7fca41647640 1 --2- 172.21.15.49:0/2815943831 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55fd06650810 0x55fd06670f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:09:23.564 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:23.560+0000 7fca42649640 1 --2- 172.21.15.49:0/2815943831 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fd066595e0 0x55fd06658a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:09:23.564 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:23.560+0000 7fca41e48640 1 -- 172.21.15.49:0/2815943831 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fd066595e0 msgr2=0x55fd06658a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:09:23.564 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:23.560+0000 7fca41e48640 1 --2- 172.21.15.49:0/2815943831 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fd066595e0 0x55fd06658a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:23.564 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:23.560+0000 7fca41e48640 1 -- 172.21.15.49:0/2815943831 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55fd06650810 msgr2=0x55fd06670f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:09:23.565 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:23.560+0000 7fca41e48640 1 --2- 172.21.15.49:0/2815943831 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55fd06650810 0x55fd06670f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:23.565 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:23.561+0000 7fca41e48640 1 -- 172.21.15.49:0/2815943831 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fd06394620 con 0x55fd0664fef0 2024-03-18T23:09:23.565 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:23.561+0000 7fca41e48640 1 --2- 172.21.15.49:0/2815943831 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fd0664fef0 0x55fd06664fb0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fca3802c170 tx=0x7fca3802e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79c606838894f1aa server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:09:23.565 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:23.561+0000 7fca277fe640 1 -- 172.21.15.49:0/2815943831 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca3804d020 con 0x55fd0664fef0 2024-03-18T23:09:23.565 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:23.561+0000 7fca277fe640 1 -- 172.21.15.49:0/2815943831 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca38049d10 con 0x55fd0664fef0 2024-03-18T23:09:23.565 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:23.561+0000 7fca277fe640 1 -- 172.21.15.49:0/2815943831 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca38004330 con 0x55fd0664fef0 2024-03-18T23:09:23.565 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:23.562+0000 7fca46d74740 1 -- 172.21.15.49:0/2815943831 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fd0664fef0 msgr2=0x55fd06664fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:09:23.565 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:23.562+0000 7fca46d74740 1 --2- 172.21.15.49:0/2815943831 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fd0664fef0 0x55fd06664fb0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fca3802c170 tx=0x7fca3802e040 comp rx=0 tx=0).stop 2024-03-18T23:09:23.565 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:23.562+0000 7fca46d74740 1 -- 172.21.15.49:0/2815943831 shutdown_connections 2024-03-18T23:09:23.565 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:23.562+0000 7fca46d74740 1 --2- 172.21.15.49:0/2815943831 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fd066595e0 0x55fd06658a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:23.565 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:23.562+0000 7fca46d74740 1 --2- 172.21.15.49:0/2815943831 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55fd06650810 0x55fd06670f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:23.565 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:23.562+0000 7fca46d74740 1 --2- 172.21.15.49:0/2815943831 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fd0664fef0 0x55fd06664fb0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:23.565 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:23.562+0000 7fca46d74740 1 -- 172.21.15.49:0/2815943831 >> 172.21.15.49:0/2815943831 conn(0x55fd0664a810 msgr2=0x55fd06649ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:09:23.565 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:23.562+0000 7fca46d74740 1 -- 172.21.15.49:0/2815943831 shutdown_connections 2024-03-18T23:09:23.565 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:23.562+0000 7fca46d74740 1 -- 172.21.15.49:0/2815943831 wait complete. 2024-03-18T23:09:23.568 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:23.562+0000 7fca46d74740 1 -- start start 2024-03-18T23:09:23.568 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:23.562+0000 7fca46d74740 1 Processor -- start 2024-03-18T23:09:23.568 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:23.562+0000 7fca46d74740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fd0664fef0 0x55fd066618b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:09:23.569 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:23.562+0000 7fca46d74740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55fd06650810 0x55fd06652b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:09:23.569 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:23.562+0000 7fca46d74740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fd066595e0 0x55fd066530b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:09:23.569 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:23.562+0000 7fca46d74740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fca3c1ae7c0 con 0x55fd066595e0 2024-03-18T23:09:23.569 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:23.562+0000 7fca46d74740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fca3c1ae0f0 con 0x55fd06650810 2024-03-18T23:09:23.569 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:23.562+0000 7fca46d74740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fca3c1ab330 con 0x55fd0664fef0 2024-03-18T23:09:23.569 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:23.563+0000 7fca46d74740 10 client.0 _ll_register_callbacks cb 0x7fca2c0d2e50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T23:09:23.569 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:23.563+0000 7fca42649640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fd066595e0 0x55fd066530b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:09:23.569 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:23.563+0000 7fca41e48640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fd0664fef0 0x55fd066618b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:09:23.569 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:23.563+0000 7fca42649640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fd066595e0 0x55fd066530b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:52518/0 (socket says 172.21.15.49:52518) 2024-03-18T23:09:23.569 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:23.563+0000 7fca42649640 1 -- 172.21.15.49:0/2994048871 learned_addr learned my addr 172.21.15.49:0/2994048871 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:09:23.569 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:23.563+0000 7fca41647640 1 --2- 172.21.15.49:0/2994048871 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55fd06650810 0x55fd06652b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:09:23.569 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:23.563+0000 7fca42649640 1 -- 172.21.15.49:0/2994048871 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fd0664fef0 msgr2=0x55fd066618b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:09:23.569 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:23.563+0000 7fca42649640 1 --2- 172.21.15.49:0/2994048871 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fd0664fef0 0x55fd066618b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:23.569 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:23.563+0000 7fca42649640 1 -- 172.21.15.49:0/2994048871 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55fd06650810 msgr2=0x55fd06652b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:09:23.570 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:23.563+0000 7fca42649640 1 --2- 172.21.15.49:0/2994048871 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55fd06650810 0x55fd06652b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:23.570 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:23.563+0000 7fca42649640 1 -- 172.21.15.49:0/2994048871 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fd066538e0 con 0x55fd066595e0 2024-03-18T23:09:23.570 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:23.563+0000 7fca42649640 1 --2- 172.21.15.49:0/2994048871 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fd066595e0 0x55fd066530b0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fca34005440 tx=0x7fca3401b7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=544912714190c65d server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:09:23.570 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:23.563+0000 7fca257fa640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-18T23:09:23.570 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:23.563+0000 7fca257fa640 1 -- 172.21.15.49:0/2994048871 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca34046030 con 0x55fd066595e0 2024-03-18T23:09:23.570 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:23.563+0000 7fca257fa640 1 -- 172.21.15.49:0/2994048871 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7fca340058c0 con 0x55fd066595e0 2024-03-18T23:09:23.570 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:23.563+0000 7fca46d74740 1 -- 172.21.15.49:0/2994048871 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fca38049d10 con 0x55fd066595e0 2024-03-18T23:09:23.570 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:23.563+0000 7fca257fa640 1 -- 172.21.15.49:0/2994048871 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fca34033050 con 0x55fd066595e0 2024-03-18T23:09:23.570 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:23.563+0000 7fca46d74740 10 client.15549 fetch_fsmap learned FSMap version 29 2024-03-18T23:09:23.570 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:23.563+0000 7fca46d74740 1 -- 172.21.15.49:0/2994048871 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55fd066cf600 con 0x55fd066595e0 2024-03-18T23:09:23.570 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:23.564+0000 7fca257fa640 1 -- 172.21.15.49:0/2994048871 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca38049d10 con 0x55fd066595e0 2024-03-18T23:09:23.570 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:23.564+0000 7fca257fa640 1 -- 172.21.15.49:0/2994048871 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 29) ==== 8358+0+0 (secure 0 0 0) 0x7fca340040f0 con 0x55fd066595e0 2024-03-18T23:09:23.570 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:23.564+0000 7fca46d74740 10 client.15549 fetch_fsmap finished waiting for FSMap version 29 2024-03-18T23:09:23.570 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:23.564+0000 7fca46d74740 10 client.15549 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T23:09:23.571 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:23.564+0000 7fca46d74740 20 client.15549 populate_metadata read hostname 'smithi049' 2024-03-18T23:09:23.571 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:23.564+0000 7fca46d74740 1 --2- 172.21.15.49:0/2994048871 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x55fd066cfc50 0x55fd066d2070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:09:23.571 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:23.564+0000 7fca46d74740 4 client.15549 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T23:09:23.571 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:23.564+0000 7fca46d74740 1 -- 172.21.15.49:0/2994048871 --> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55fd065bb470 con 0x55fd066cfc50 2024-03-18T23:09:23.571 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:23.564+0000 7fca41e48640 1 --2- 172.21.15.49:0/2994048871 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x55fd066cfc50 0x55fd066d2070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:09:23.571 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:23.564+0000 7fca257fa640 10 client.15549 ms_handle_connect on v2:172.21.15.139:6838/4280128209 2024-03-18T23:09:23.571 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:23.564+0000 7fca41e48640 1 --2- 172.21.15.49:0/2994048871 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x55fd066cfc50 0x55fd066d2070 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b53e211a2ee8e843 server_cookie=e5247f8469d12555 in_seq=0 out_seq=0 2024-03-18T23:09:23.751 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:23.749+0000 7fca257fa640 1 -- 172.21.15.49:0/2994048871 <== mds.0 v2:172.21.15.139:6838/4280128209 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55fd065bb470 con 0x55fd066cfc50 2024-03-18T23:09:23.752 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:23.749+0000 7fca257fa640 10 client.15549 handle_command_reply: tid=0 2024-03-18T23:09:23.752 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:23.749+0000 7fca46d74740 1 client.15549 shutdown 2024-03-18T23:09:23.752 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:23.749+0000 7fca46d74740 1 -- 172.21.15.49:0/2994048871 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fd066595e0 msgr2=0x55fd066530b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:09:23.752 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:23.749+0000 7fca46d74740 1 --2- 172.21.15.49:0/2994048871 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fd066595e0 0x55fd066530b0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fca34005440 tx=0x7fca3401b7d0 comp rx=0 tx=0).stop 2024-03-18T23:09:23.752 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:23.750+0000 7fca46d74740 1 -- 172.21.15.49:0/2994048871 shutdown_connections 2024-03-18T23:09:23.752 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:23.750+0000 7fca46d74740 1 --2- 172.21.15.49:0/2994048871 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x55fd066cfc50 0x55fd066d2070 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:23.752 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:23.750+0000 7fca46d74740 1 --2- 172.21.15.49:0/2994048871 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fd066595e0 0x55fd066530b0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:23.752 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:23.750+0000 7fca46d74740 1 --2- 172.21.15.49:0/2994048871 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55fd06650810 0x55fd06652b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:23.752 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:23.750+0000 7fca46d74740 1 --2- 172.21.15.49:0/2994048871 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fd0664fef0 0x55fd066618b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:23.752 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:23.750+0000 7fca257fa640 0 client.15549 ms_handle_reset on v2:172.21.15.139:6838/4280128209 2024-03-18T23:09:23.752 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:23.750+0000 7fca46d74740 1 -- 172.21.15.49:0/2994048871 >> 172.21.15.49:0/2994048871 conn(0x55fd0664a810 msgr2=0x55fd0664bb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:09:23.752 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:23.750+0000 7fca46d74740 1 -- 172.21.15.49:0/2994048871 shutdown_connections 2024-03-18T23:09:23.752 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:23.750+0000 7fca46d74740 1 -- 172.21.15.49:0/2994048871 wait complete. 2024-03-18T23:09:23.753 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:23.750+0000 7fca46d74740 20 client.15549 trim_cache size 0 max 16384 2024-03-18T23:09:23.753 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:{ 2024-03-18T23:09:23.753 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "message": "", 2024-03-18T23:09:23.753 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "return_code": 0 2024-03-18T23:09:23.753 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:} 2024-03-18T23:09:23.754 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:23.750+0000 7fca440ac640 1 -- 172.21.15.49:0/4271354409 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7fca280a2bb0 msgr2=0x7fca280a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:09:23.754 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:23.750+0000 7fca440ac640 1 --2- 172.21.15.49:0/4271354409 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7fca280a2bb0 0x7fca280a5070 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fca34020000 tx=0x7fca34021000 comp rx=0 tx=0).stop 2024-03-18T23:09:23.757 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:23.750+0000 7fca440ac640 1 -- 172.21.15.49:0/4271354409 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fca3c103320 msgr2=0x7fca3c198560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:09:23.758 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:23.750+0000 7fca440ac640 1 --2- 172.21.15.49:0/4271354409 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fca3c103320 0x7fca3c198560 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fca2c00e9f0 tx=0x7fca2c00edf0 comp rx=0 tx=0).stop 2024-03-18T23:09:23.758 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:23.751+0000 7fca440ac640 1 -- 172.21.15.49:0/4271354409 shutdown_connections 2024-03-18T23:09:23.758 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:23.751+0000 7fca440ac640 1 --2- 172.21.15.49:0/4271354409 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7fca280a2bb0 0x7fca280a5070 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:23.758 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:23.751+0000 7fca440ac640 1 --2- 172.21.15.49:0/4271354409 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fca3c10be70 0x7fca3c19c890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:23.758 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:23.751+0000 7fca440ac640 1 --2- 172.21.15.49:0/4271354409 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fca3c103320 0x7fca3c198560 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:23.758 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:23.751+0000 7fca440ac640 1 --2- 172.21.15.49:0/4271354409 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fca3c102a00 0x7fca3c198020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:23.758 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:23.751+0000 7fca440ac640 1 -- 172.21.15.49:0/4271354409 >> 172.21.15.49:0/4271354409 conn(0x7fca3c077850 msgr2=0x7fca3c0fd740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:09:23.758 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:23.751+0000 7fca440ac640 1 -- 172.21.15.49:0/4271354409 shutdown_connections 2024-03-18T23:09:23.758 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:23.751+0000 7fca440ac640 1 -- 172.21.15.49:0/4271354409 wait complete. 2024-03-18T23:09:23.759 INFO:tasks.workunit.client.0.smithi049.stdout: 1 64465 14.61 MB/sec warmup 25 sec latency 40.612 ms 2024-03-18T23:09:24.759 INFO:tasks.workunit.client.0.smithi049.stdout: 1 67396 14.71 MB/sec warmup 26 sec latency 28.039 ms 2024-03-18T23:09:25.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:09:24 smithi139 ceph-mon[67282]: pgmap v37: 97 pgs: 97 active+clean; 25 MiB data, 474 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.0 MiB/s wr, 295 op/s 2024-03-18T23:09:25.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:09:24 smithi049 ceph-mon[62991]: pgmap v37: 97 pgs: 97 active+clean; 25 MiB data, 474 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.0 MiB/s wr, 295 op/s 2024-03-18T23:09:25.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:09:24 smithi175 ceph-mon[67352]: pgmap v37: 97 pgs: 97 active+clean; 25 MiB data, 474 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.0 MiB/s wr, 295 op/s 2024-03-18T23:09:25.759 INFO:tasks.workunit.client.0.smithi049.stdout: 1 67506 14.18 MB/sec warmup 27 sec latency 535.406 ms 2024-03-18T23:09:26.759 INFO:tasks.workunit.client.0.smithi049.stdout: 1 68979 13.91 MB/sec warmup 28 sec latency 35.354 ms 2024-03-18T23:09:27.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:09:26 smithi139 ceph-mon[67282]: pgmap v38: 97 pgs: 97 active+clean; 25 MiB data, 474 MiB used, 1.0 TiB / 1.0 TiB avail; 828 KiB/s rd, 1.5 MiB/s wr, 176 op/s 2024-03-18T23:09:27.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:09:26 smithi049 ceph-mon[62991]: pgmap v38: 97 pgs: 97 active+clean; 25 MiB data, 474 MiB used, 1.0 TiB / 1.0 TiB avail; 828 KiB/s rd, 1.5 MiB/s wr, 176 op/s 2024-03-18T23:09:27.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:09:26 smithi175 ceph-mon[67352]: pgmap v38: 97 pgs: 97 active+clean; 25 MiB data, 474 MiB used, 1.0 TiB / 1.0 TiB avail; 828 KiB/s rd, 1.5 MiB/s wr, 176 op/s 2024-03-18T23:09:27.759 INFO:tasks.workunit.client.0.smithi049.stdout: 1 71545 13.91 MB/sec warmup 29 sec latency 8.235 ms 2024-03-18T23:09:28.759 INFO:tasks.workunit.client.0.smithi049.stdout: 1 74624 14.04 MB/sec warmup 30 sec latency 23.803 ms 2024-03-18T23:09:29.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:09:28 smithi139 ceph-mon[67282]: pgmap v39: 97 pgs: 97 active+clean; 78 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 8.0 MiB/s wr, 458 op/s 2024-03-18T23:09:29.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:09:28 smithi049 ceph-mon[62991]: pgmap v39: 97 pgs: 97 active+clean; 78 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 8.0 MiB/s wr, 458 op/s 2024-03-18T23:09:29.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:09:28 smithi175 ceph-mon[67352]: pgmap v39: 97 pgs: 97 active+clean; 78 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 8.0 MiB/s wr, 458 op/s 2024-03-18T23:09:29.759 INFO:tasks.workunit.client.0.smithi049.stdout: 1 76743 13.82 MB/sec warmup 31 sec latency 22.688 ms 2024-03-18T23:09:30.759 INFO:tasks.workunit.client.0.smithi049.stdout: 1 78753 13.81 MB/sec warmup 32 sec latency 18.468 ms 2024-03-18T23:09:31.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:09:30 smithi139 ceph-mon[67282]: pgmap v40: 97 pgs: 97 active+clean; 78 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 891 KiB/s rd, 7.2 MiB/s wr, 362 op/s 2024-03-18T23:09:31.379 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:09:30 smithi049 ceph-mon[62991]: pgmap v40: 97 pgs: 97 active+clean; 78 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 891 KiB/s rd, 7.2 MiB/s wr, 362 op/s 2024-03-18T23:09:31.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:09:30 smithi175 ceph-mon[67352]: pgmap v40: 97 pgs: 97 active+clean; 78 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 891 KiB/s rd, 7.2 MiB/s wr, 362 op/s 2024-03-18T23:09:31.759 INFO:tasks.workunit.client.0.smithi049.stdout: 1 81769 13.92 MB/sec warmup 33 sec latency 15.593 ms 2024-03-18T23:09:32.759 INFO:tasks.workunit.client.0.smithi049.stdout: 1 84033 13.76 MB/sec warmup 34 sec latency 18.518 ms 2024-03-18T23:09:33.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:09:32 smithi139 ceph-mon[67282]: pgmap v41: 97 pgs: 97 active+clean; 44 MiB data, 488 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 7.9 MiB/s wr, 488 op/s 2024-03-18T23:09:33.379 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:09:32 smithi049 ceph-mon[62991]: pgmap v41: 97 pgs: 97 active+clean; 44 MiB data, 488 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 7.9 MiB/s wr, 488 op/s 2024-03-18T23:09:33.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:09:32 smithi175 ceph-mon[67352]: pgmap v41: 97 pgs: 97 active+clean; 44 MiB data, 488 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 7.9 MiB/s wr, 488 op/s 2024-03-18T23:09:33.760 INFO:tasks.workunit.client.0.smithi049.stdout: 1 85957 13.72 MB/sec warmup 35 sec latency 28.046 ms 2024-03-18T23:09:34.759 INFO:tasks.workunit.client.0.smithi049.stdout: 1 88961 13.85 MB/sec warmup 36 sec latency 16.194 ms 2024-03-18T23:09:35.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:09:35 smithi139 ceph-mon[67282]: pgmap v42: 97 pgs: 97 active+clean; 36 MiB data, 535 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 7.4 MiB/s wr, 430 op/s 2024-03-18T23:09:35.379 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:09:35 smithi049 ceph-mon[62991]: pgmap v42: 97 pgs: 97 active+clean; 36 MiB data, 535 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 7.4 MiB/s wr, 430 op/s 2024-03-18T23:09:35.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:09:35 smithi175 ceph-mon[67352]: pgmap v42: 97 pgs: 97 active+clean; 36 MiB data, 535 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 7.4 MiB/s wr, 430 op/s 2024-03-18T23:09:35.760 INFO:tasks.workunit.client.0.smithi049.stdout: 1 91328 13.71 MB/sec warmup 37 sec latency 18.278 ms 2024-03-18T23:09:36.760 INFO:tasks.workunit.client.0.smithi049.stdout: 1 93373 13.69 MB/sec warmup 38 sec latency 22.583 ms 2024-03-18T23:09:36.844 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:36.841+0000 7eff2155b640 1 -- 172.21.15.49:0/3315681279 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7eff1c1056c0 msgr2=0x7eff1c105aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:09:36.844 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:36.841+0000 7eff2155b640 1 --2- 172.21.15.49:0/3315681279 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7eff1c1056c0 0x7eff1c105aa0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7eff0c009900 tx=0x7eff0c02bc30 comp rx=0 tx=0).stop 2024-03-18T23:09:36.844 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:36.841+0000 7eff2155b640 1 -- 172.21.15.49:0/3315681279 shutdown_connections 2024-03-18T23:09:36.845 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:36.841+0000 7eff2155b640 1 --2- 172.21.15.49:0/3315681279 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7eff1c101330 0x7eff1c10d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:36.845 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:36.841+0000 7eff2155b640 1 --2- 172.21.15.49:0/3315681279 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7eff1c100970 0x7eff1c100df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:36.845 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:36.841+0000 7eff2155b640 1 --2- 172.21.15.49:0/3315681279 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7eff1c1056c0 0x7eff1c105aa0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:36.845 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:36.841+0000 7eff2155b640 1 -- 172.21.15.49:0/3315681279 >> 172.21.15.49:0/3315681279 conn(0x7eff1c077570 msgr2=0x7eff1c077980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:09:36.845 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:36.841+0000 7eff2155b640 1 -- 172.21.15.49:0/3315681279 shutdown_connections 2024-03-18T23:09:36.845 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:36.841+0000 7eff2155b640 1 -- 172.21.15.49:0/3315681279 wait complete. 2024-03-18T23:09:36.845 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:36.842+0000 7eff2155b640 1 Processor -- start 2024-03-18T23:09:36.845 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:36.842+0000 7eff2155b640 1 -- start start 2024-03-18T23:09:36.845 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:36.842+0000 7eff2155b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7eff1c100970 0x7eff1c198120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:09:36.845 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:36.842+0000 7eff2155b640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7eff1c101330 0x7eff1c198660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:09:36.845 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:36.842+0000 7eff2155b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7eff1c1056c0 0x7eff1c19c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:09:36.845 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:36.842+0000 7eff2155b640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7eff1c111d00 con 0x7eff1c1056c0 2024-03-18T23:09:36.845 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:36.842+0000 7eff2155b640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7eff1c111b80 con 0x7eff1c100970 2024-03-18T23:09:36.845 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:36.842+0000 7eff2155b640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7eff1c111e80 con 0x7eff1c101330 2024-03-18T23:09:36.846 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:36.842+0000 7eff1b7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7eff1c1056c0 0x7eff1c19c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:09:36.846 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:36.842+0000 7eff1affd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7eff1c100970 0x7eff1c198120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:09:36.846 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:36.842+0000 7eff127fc640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7eff1c101330 0x7eff1c198660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:09:36.846 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:36.842+0000 7eff1affd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7eff1c100970 0x7eff1c198120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.49:52880/0 (socket says 172.21.15.49:52880) 2024-03-18T23:09:36.846 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:36.842+0000 7eff1affd640 1 -- 172.21.15.49:0/1821106709 learned_addr learned my addr 172.21.15.49:0/1821106709 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:09:36.846 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:36.842+0000 7eff1b7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7eff1c1056c0 0x7eff1c19c9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:44068/0 (socket says 172.21.15.49:44068) 2024-03-18T23:09:36.846 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:36.842+0000 7eff127fc640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7eff1c101330 0x7eff1c198660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.49:52910/0 (socket says 172.21.15.49:52910) 2024-03-18T23:09:36.846 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:36.843+0000 7eff1b7fe640 1 -- 172.21.15.49:0/1821106709 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7eff1c101330 msgr2=0x7eff1c198660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:09:36.846 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:36.843+0000 7eff1b7fe640 1 --2- 172.21.15.49:0/1821106709 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7eff1c101330 0x7eff1c198660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:36.846 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:36.843+0000 7eff1b7fe640 1 -- 172.21.15.49:0/1821106709 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7eff1c100970 msgr2=0x7eff1c198120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:09:36.846 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:36.843+0000 7eff1b7fe640 1 --2- 172.21.15.49:0/1821106709 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7eff1c100970 0x7eff1c198120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:36.846 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:36.843+0000 7eff1b7fe640 1 -- 172.21.15.49:0/1821106709 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff1c19d110 con 0x7eff1c1056c0 2024-03-18T23:09:36.846 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:36.843+0000 7eff127fc640 1 --2- 172.21.15.49:0/1821106709 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7eff1c101330 0x7eff1c198660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:09:36.847 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:36.843+0000 7eff1affd640 1 --2- 172.21.15.49:0/1821106709 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7eff1c100970 0x7eff1c198120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:09:36.847 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:36.843+0000 7eff1b7fe640 1 --2- 172.21.15.49:0/1821106709 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7eff1c1056c0 0x7eff1c19c9e0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7eff14005fd0 tx=0x7eff1400ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:09:36.847 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:36.843+0000 7eff18ff9640 1 -- 172.21.15.49:0/1821106709 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff1400cd80 con 0x7eff1c1056c0 2024-03-18T23:09:36.847 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:36.843+0000 7eff18ff9640 1 -- 172.21.15.49:0/1821106709 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff14005020 con 0x7eff1c1056c0 2024-03-18T23:09:36.847 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:36.843+0000 7eff2155b640 1 -- 172.21.15.49:0/1821106709 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff1c19d400 con 0x7eff1c1056c0 2024-03-18T23:09:36.847 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:36.843+0000 7eff18ff9640 1 -- 172.21.15.49:0/1821106709 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff14010720 con 0x7eff1c1056c0 2024-03-18T23:09:36.847 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:36.843+0000 7eff2155b640 1 -- 172.21.15.49:0/1821106709 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff1c1a4c90 con 0x7eff1c1056c0 2024-03-18T23:09:36.852 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:36.845+0000 7eff18ff9640 1 -- 172.21.15.49:0/1821106709 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 24) ==== 137523+0+0 (secure 0 0 0) 0x7eff140051d0 con 0x7eff1c1056c0 2024-03-18T23:09:36.852 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:36.846+0000 7eff24223740 1 Processor -- start 2024-03-18T23:09:36.852 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:36.846+0000 7eff24223740 1 -- start start 2024-03-18T23:09:36.852 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:36.846+0000 7eff18ff9640 1 --2- 172.21.15.49:0/1821106709 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7eff080a2b30 0x7eff080a4ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:09:36.853 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:36.846+0000 7eff24223740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558b69905ef0 0x558b699062d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:09:36.853 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:36.846+0000 7eff24223740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558b69906810 0x558b6990f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:09:36.853 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:36.846+0000 7eff24223740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558b6990f5e0 0x558b699119d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:09:36.853 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:36.846+0000 7eff24223740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x558b69869be0 con 0x558b6990f5e0 2024-03-18T23:09:36.853 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:36.846+0000 7eff24223740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x558b69812960 con 0x558b69906810 2024-03-18T23:09:36.853 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:36.846+0000 7eff24223740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x558b69865c80 con 0x558b69905ef0 2024-03-18T23:09:36.853 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:36.846+0000 7eff18ff9640 1 -- 172.21.15.49:0/1821106709 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7eff140cf670 con 0x7eff1c1056c0 2024-03-18T23:09:36.853 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:36.846+0000 7eff1affd640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558b69905ef0 0x558b699062d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:09:36.853 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:36.846+0000 7eff1b7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558b6990f5e0 0x558b699119d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:09:36.853 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:36.846+0000 7eff1b7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558b6990f5e0 0x558b699119d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:44076/0 (socket says 172.21.15.49:44076) 2024-03-18T23:09:36.853 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:36.846+0000 7eff127fc640 1 --2- 172.21.15.49:0/1821106709 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7eff080a2b30 0x7eff080a4ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:09:36.853 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:36.846+0000 7eff1affd640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558b69905ef0 0x558b699062d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.49:52924/0 (socket says 172.21.15.49:52924) 2024-03-18T23:09:36.854 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:36.846+0000 7eff1b7fe640 1 -- 172.21.15.49:0/3283594094 learned_addr learned my addr 172.21.15.49:0/3283594094 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:09:36.854 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:36.847+0000 7eff127fc640 1 --2- 172.21.15.49:0/3283594094 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558b69906810 0x558b6990f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:09:36.854 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:36.847+0000 7eff1b7fe640 1 -- 172.21.15.49:0/3283594094 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558b69905ef0 msgr2=0x558b699062d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:09:36.854 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:36.847+0000 7eff1b7fe640 1 --2- 172.21.15.49:0/3283594094 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558b69905ef0 0x558b699062d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:36.854 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:36.847+0000 7eff1b7fe640 1 -- 172.21.15.49:0/3283594094 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558b69906810 msgr2=0x558b6990f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:09:36.854 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:36.847+0000 7eff1b7fe640 1 --2- 172.21.15.49:0/3283594094 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558b69906810 0x558b6990f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:36.854 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:36.847+0000 7eff1b7fe640 1 -- 172.21.15.49:0/3283594094 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558b69809d20 con 0x558b6990f5e0 2024-03-18T23:09:36.854 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:36.847+0000 7eff127fc640 1 --2- 172.21.15.49:0/1821106709 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7eff080a2b30 0x7eff080a4ff0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x558b699059a0 tx=0x7eff04009d30 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:09:36.854 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:36.847+0000 7eff1b7fe640 1 --2- 172.21.15.49:0/3283594094 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558b6990f5e0 0x558b699119d0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7eff1400cfd0 tx=0x7eff1409b950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4ccb61a245f4bdf server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:09:36.854 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:36.847+0000 7eff117fa640 1 -- 172.21.15.49:0/3283594094 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff14067a80 con 0x558b6990f5e0 2024-03-18T23:09:36.854 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:36.847+0000 7eff117fa640 1 -- 172.21.15.49:0/3283594094 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff1409ba50 con 0x558b6990f5e0 2024-03-18T23:09:36.854 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:36.847+0000 7eff117fa640 1 -- 172.21.15.49:0/3283594094 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff14098250 con 0x558b6990f5e0 2024-03-18T23:09:36.854 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:36.848+0000 7eff24223740 1 -- 172.21.15.49:0/3283594094 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558b6990f5e0 msgr2=0x558b699119d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:09:36.855 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:36.848+0000 7eff24223740 1 --2- 172.21.15.49:0/3283594094 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558b6990f5e0 0x558b699119d0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7eff1400cfd0 tx=0x7eff1409b950 comp rx=0 tx=0).stop 2024-03-18T23:09:36.855 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:36.848+0000 7eff24223740 1 -- 172.21.15.49:0/3283594094 shutdown_connections 2024-03-18T23:09:36.855 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:36.848+0000 7eff24223740 1 --2- 172.21.15.49:0/3283594094 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558b6990f5e0 0x558b699119d0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:36.855 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:36.848+0000 7eff24223740 1 --2- 172.21.15.49:0/3283594094 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558b69906810 0x558b6990f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:36.855 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:36.848+0000 7eff24223740 1 --2- 172.21.15.49:0/3283594094 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558b69905ef0 0x558b699062d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:36.855 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:36.848+0000 7eff24223740 1 -- 172.21.15.49:0/3283594094 >> 172.21.15.49:0/3283594094 conn(0x558b69900810 msgr2=0x558b69900bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:09:36.855 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:36.848+0000 7eff24223740 1 -- 172.21.15.49:0/3283594094 shutdown_connections 2024-03-18T23:09:36.855 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:36.848+0000 7eff24223740 1 -- 172.21.15.49:0/3283594094 wait complete. 2024-03-18T23:09:36.855 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:36.848+0000 7eff24223740 1 -- start start 2024-03-18T23:09:36.855 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:36.848+0000 7eff24223740 1 Processor -- start 2024-03-18T23:09:36.855 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:36.849+0000 7eff24223740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558b69905ef0 0x558b6990f030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:09:36.855 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:36.849+0000 7eff24223740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558b69906810 0x558b6990c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:09:36.855 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:36.849+0000 7eff24223740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558b6990f5e0 0x558b6990cb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:09:36.855 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:36.849+0000 7eff24223740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x558b69912210 con 0x558b6990f5e0 2024-03-18T23:09:36.855 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:36.849+0000 7eff24223740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x558b69825db0 con 0x558b69905ef0 2024-03-18T23:09:36.856 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:36.849+0000 7eff24223740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x558b69912390 con 0x558b69906810 2024-03-18T23:09:36.856 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:36.849+0000 7eff24223740 10 client.0 _ll_register_callbacks cb 0x558b6982ae70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T23:09:36.856 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:36.849+0000 7eff1b7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558b6990f5e0 0x558b6990cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:09:36.856 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:36.849+0000 7eff1b7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558b6990f5e0 0x558b6990cb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:44090/0 (socket says 172.21.15.49:44090) 2024-03-18T23:09:36.856 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:36.849+0000 7eff1b7fe640 1 -- 172.21.15.49:0/92169317 learned_addr learned my addr 172.21.15.49:0/92169317 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:09:36.856 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:36.849+0000 7eff1affd640 1 --2- 172.21.15.49:0/92169317 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558b69905ef0 0x558b6990f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:09:36.856 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:36.849+0000 7eff127fc640 1 --2- 172.21.15.49:0/92169317 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558b69906810 0x558b6990c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:09:36.856 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:36.849+0000 7eff1b7fe640 1 -- 172.21.15.49:0/92169317 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558b69906810 msgr2=0x558b6990c3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:09:36.856 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:36.849+0000 7eff1b7fe640 1 --2- 172.21.15.49:0/92169317 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558b69906810 0x558b6990c3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:36.856 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:36.849+0000 7eff1b7fe640 1 -- 172.21.15.49:0/92169317 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558b69905ef0 msgr2=0x558b6990f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:09:36.856 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:36.849+0000 7eff1b7fe640 1 --2- 172.21.15.49:0/92169317 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558b69905ef0 0x558b6990f030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:36.856 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:36.849+0000 7eff1b7fe640 1 -- 172.21.15.49:0/92169317 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558b69824c60 con 0x558b6990f5e0 2024-03-18T23:09:36.856 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:36.849+0000 7eff1b7fe640 1 --2- 172.21.15.49:0/92169317 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558b6990f5e0 0x558b6990cb00 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7eff14009fd0 tx=0x7eff140980c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26d60006bbe00083 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:09:36.856 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:36.849+0000 7eff027fc640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-18T23:09:36.857 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:36.849+0000 7eff027fc640 1 -- 172.21.15.49:0/92169317 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff140b9060 con 0x558b6990f5e0 2024-03-18T23:09:36.857 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:36.849+0000 7eff027fc640 1 -- 172.21.15.49:0/92169317 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7eff1409bcb0 con 0x558b6990f5e0 2024-03-18T23:09:36.857 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:36.849+0000 7eff24223740 1 -- 172.21.15.49:0/92169317 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7eff1409ba50 con 0x558b6990f5e0 2024-03-18T23:09:36.857 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:36.850+0000 7eff027fc640 1 -- 172.21.15.49:0/92169317 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7eff14098230 con 0x558b6990f5e0 2024-03-18T23:09:36.857 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:36.850+0000 7eff24223740 10 client.15561 fetch_fsmap learned FSMap version 29 2024-03-18T23:09:36.857 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:36.850+0000 7eff24223740 1 -- 172.21.15.49:0/92169317 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x558b6957e210 con 0x558b6990f5e0 2024-03-18T23:09:36.857 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:36.850+0000 7eff027fc640 1 -- 172.21.15.49:0/92169317 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff1409ba50 con 0x558b6990f5e0 2024-03-18T23:09:36.857 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:36.850+0000 7eff027fc640 1 -- 172.21.15.49:0/92169317 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 29) ==== 8358+0+0 (secure 0 0 0) 0x7eff140a3df0 con 0x558b6990f5e0 2024-03-18T23:09:36.857 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:36.850+0000 7eff24223740 10 client.15561 fetch_fsmap finished waiting for FSMap version 29 2024-03-18T23:09:36.857 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:36.850+0000 7eff24223740 10 client.15561 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T23:09:36.857 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:36.850+0000 7eff24223740 20 client.15561 populate_metadata read hostname 'smithi049' 2024-03-18T23:09:36.857 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:36.850+0000 7eff24223740 1 --2- 172.21.15.49:0/92169317 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x558b6991f3a0 0x558b69921790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:09:36.857 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:36.850+0000 7eff24223740 4 client.15561 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T23:09:36.857 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:36.850+0000 7eff24223740 1 -- 172.21.15.49:0/92169317 --> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558b69877120 con 0x558b6991f3a0 2024-03-18T23:09:36.857 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:36.850+0000 7eff1affd640 1 --2- 172.21.15.49:0/92169317 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x558b6991f3a0 0x558b69921790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:09:36.858 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:36.851+0000 7eff1affd640 1 --2- 172.21.15.49:0/92169317 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x558b6991f3a0 0x558b69921790 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=66e4967937151bc3 server_cookie=6a0f2cce6c52543a in_seq=0 out_seq=0 2024-03-18T23:09:36.858 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:36.851+0000 7eff027fc640 10 client.15561 ms_handle_connect on v2:172.21.15.139:6838/4280128209 2024-03-18T23:09:36.859 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:36.852+0000 7eff027fc640 1 -- 172.21.15.49:0/92169317 <== mds.0 v2:172.21.15.139:6838/4280128209 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x558b69877120 con 0x558b6991f3a0 2024-03-18T23:09:36.859 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:36.852+0000 7eff027fc640 10 client.15561 handle_command_reply: tid=0 2024-03-18T23:09:36.859 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:36.852+0000 7eff24223740 1 client.15561 shutdown 2024-03-18T23:09:36.859 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:36.853+0000 7eff24223740 1 -- 172.21.15.49:0/92169317 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558b6990f5e0 msgr2=0x558b6990cb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:09:36.859 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:36.853+0000 7eff24223740 1 --2- 172.21.15.49:0/92169317 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558b6990f5e0 0x558b6990cb00 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7eff14009fd0 tx=0x7eff140980c0 comp rx=0 tx=0).stop 2024-03-18T23:09:36.859 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:36.853+0000 7eff24223740 1 -- 172.21.15.49:0/92169317 shutdown_connections 2024-03-18T23:09:36.859 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:36.853+0000 7eff24223740 1 --2- 172.21.15.49:0/92169317 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x558b6991f3a0 0x558b69921790 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:36.859 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:36.853+0000 7eff24223740 1 --2- 172.21.15.49:0/92169317 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558b6990f5e0 0x558b6990cb00 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:36.859 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:36.853+0000 7eff24223740 1 --2- 172.21.15.49:0/92169317 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558b69906810 0x558b6990c3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:36.859 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:36.853+0000 7eff027fc640 0 client.15561 ms_handle_reset on v2:172.21.15.139:6838/4280128209 2024-03-18T23:09:36.859 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:36.853+0000 7eff24223740 1 --2- 172.21.15.49:0/92169317 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558b69905ef0 0x558b6990f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:36.859 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:36.853+0000 7eff24223740 1 -- 172.21.15.49:0/92169317 >> 172.21.15.49:0/92169317 conn(0x558b69900810 msgr2=0x558b698ffab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:09:36.860 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:36.853+0000 7eff24223740 1 -- 172.21.15.49:0/92169317 shutdown_connections 2024-03-18T23:09:36.860 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:36.853+0000 7eff24223740 1 -- 172.21.15.49:0/92169317 wait complete. 2024-03-18T23:09:36.860 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:36.853+0000 7eff24223740 20 client.15561 trim_cache size 0 max 16384 2024-03-18T23:09:36.869 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:36.869+0000 7eff24223740 1 Processor -- start 2024-03-18T23:09:36.870 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:36.869+0000 7eff24223740 1 -- start start 2024-03-18T23:09:36.874 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:36.869+0000 7eff24223740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558b69905ef0 0x558b699247b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:09:36.874 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:36.869+0000 7eff24223740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558b69906810 0x558b6990eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:09:36.875 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:36.869+0000 7eff24223740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558b6990f5e0 0x558b6990f030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:09:36.875 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:36.869+0000 7eff24223740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7eff1c1ad400 con 0x558b6990f5e0 2024-03-18T23:09:36.875 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:36.869+0000 7eff24223740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7eff1c1aef70 con 0x558b69905ef0 2024-03-18T23:09:36.875 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:36.869+0000 7eff24223740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7eff1c1adad0 con 0x558b69906810 2024-03-18T23:09:36.875 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:36.869+0000 7eff1b7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558b6990f5e0 0x558b6990f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:09:36.875 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:36.869+0000 7eff1b7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558b6990f5e0 0x558b6990f030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:44096/0 (socket says 172.21.15.49:44096) 2024-03-18T23:09:36.875 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:36.869+0000 7eff1b7fe640 1 -- 172.21.15.49:0/1386011655 learned_addr learned my addr 172.21.15.49:0/1386011655 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:09:36.875 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:36.869+0000 7eff127fc640 1 --2- 172.21.15.49:0/1386011655 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558b69906810 0x558b6990eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:09:36.875 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:36.869+0000 7eff1affd640 1 --2- 172.21.15.49:0/1386011655 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558b69905ef0 0x558b699247b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:09:36.875 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:36.870+0000 7eff1b7fe640 1 -- 172.21.15.49:0/1386011655 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558b69906810 msgr2=0x558b6990eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:09:36.875 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:36.870+0000 7eff1b7fe640 1 --2- 172.21.15.49:0/1386011655 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558b69906810 0x558b6990eaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:36.875 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:36.870+0000 7eff1b7fe640 1 -- 172.21.15.49:0/1386011655 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558b69905ef0 msgr2=0x558b699247b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:09:36.875 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:36.870+0000 7eff1b7fe640 1 --2- 172.21.15.49:0/1386011655 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558b69905ef0 0x558b699247b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:36.876 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:36.870+0000 7eff1b7fe640 1 -- 172.21.15.49:0/1386011655 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558b6964a620 con 0x558b6990f5e0 2024-03-18T23:09:36.876 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:36.870+0000 7eff1b7fe640 1 --2- 172.21.15.49:0/1386011655 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558b6990f5e0 0x558b6990f030 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7eff1409b9e0 tx=0x7eff14005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2de44d45978998f6 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:09:36.876 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:36.870+0000 7eff10ff9640 1 -- 172.21.15.49:0/1386011655 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff140b9070 con 0x558b6990f5e0 2024-03-18T23:09:36.876 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:36.870+0000 7eff10ff9640 1 -- 172.21.15.49:0/1386011655 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff1409fba0 con 0x558b6990f5e0 2024-03-18T23:09:36.876 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:36.870+0000 7eff10ff9640 1 -- 172.21.15.49:0/1386011655 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff140a3fa0 con 0x558b6990f5e0 2024-03-18T23:09:36.876 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:36.871+0000 7eff24223740 1 -- 172.21.15.49:0/1386011655 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558b6990f5e0 msgr2=0x558b6990f030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:09:36.876 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:36.871+0000 7eff24223740 1 --2- 172.21.15.49:0/1386011655 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558b6990f5e0 0x558b6990f030 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7eff1409b9e0 tx=0x7eff14005840 comp rx=0 tx=0).stop 2024-03-18T23:09:36.876 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:36.871+0000 7eff24223740 1 -- 172.21.15.49:0/1386011655 shutdown_connections 2024-03-18T23:09:36.876 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:36.871+0000 7eff24223740 1 --2- 172.21.15.49:0/1386011655 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558b6990f5e0 0x558b6990f030 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:36.876 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:36.871+0000 7eff24223740 1 --2- 172.21.15.49:0/1386011655 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558b69906810 0x558b6990eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:36.876 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:36.871+0000 7eff24223740 1 --2- 172.21.15.49:0/1386011655 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558b69905ef0 0x558b699247b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:36.876 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:36.871+0000 7eff24223740 1 -- 172.21.15.49:0/1386011655 >> 172.21.15.49:0/1386011655 conn(0x558b69900810 msgr2=0x558b69900bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:09:36.876 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:36.871+0000 7eff24223740 1 -- 172.21.15.49:0/1386011655 shutdown_connections 2024-03-18T23:09:36.876 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:36.871+0000 7eff24223740 1 -- 172.21.15.49:0/1386011655 wait complete. 2024-03-18T23:09:36.876 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:36.871+0000 7eff24223740 1 -- start start 2024-03-18T23:09:36.877 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:36.871+0000 7eff24223740 1 Processor -- start 2024-03-18T23:09:36.877 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:36.871+0000 7eff24223740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558b69905ef0 0x558b699224b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:09:36.877 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:36.871+0000 7eff24223740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558b69906810 0x558b699229f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:09:36.877 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:36.871+0000 7eff24223740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558b6990f5e0 0x558b69984340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:09:36.877 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:36.871+0000 7eff24223740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7eff1c1ae870 con 0x558b69905ef0 2024-03-18T23:09:36.877 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:36.871+0000 7eff24223740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7eff1c1ae1a0 con 0x558b69906810 2024-03-18T23:09:36.877 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:36.871+0000 7eff24223740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7eff1c1ab3e0 con 0x558b6990f5e0 2024-03-18T23:09:36.877 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:36.871+0000 7eff24223740 10 client.0 _ll_register_callbacks cb 0x7eff04011860 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T23:09:36.877 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:36.871+0000 7eff1affd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558b69905ef0 0x558b699224b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:09:36.877 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:36.871+0000 7eff1affd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558b69905ef0 0x558b699224b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:44110/0 (socket says 172.21.15.49:44110) 2024-03-18T23:09:36.877 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:36.871+0000 7eff1affd640 1 -- 172.21.15.49:0/3883222546 learned_addr learned my addr 172.21.15.49:0/3883222546 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:09:36.877 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:36.872+0000 7eff1b7fe640 1 --2- 172.21.15.49:0/3883222546 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558b6990f5e0 0x558b69984340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:09:36.877 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:36.872+0000 7eff127fc640 1 --2- 172.21.15.49:0/3883222546 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558b69906810 0x558b699229f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:09:36.877 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:36.872+0000 7eff1affd640 1 -- 172.21.15.49:0/3883222546 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558b6990f5e0 msgr2=0x558b69984340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:09:36.878 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:36.872+0000 7eff1affd640 1 --2- 172.21.15.49:0/3883222546 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558b6990f5e0 0x558b69984340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:36.878 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:36.872+0000 7eff1affd640 1 -- 172.21.15.49:0/3883222546 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558b69906810 msgr2=0x558b699229f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:09:36.878 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:36.872+0000 7eff1affd640 1 --2- 172.21.15.49:0/3883222546 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558b69906810 0x558b699229f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:36.878 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:36.872+0000 7eff1affd640 1 -- 172.21.15.49:0/3883222546 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558b69923290 con 0x558b69905ef0 2024-03-18T23:09:36.878 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:36.872+0000 7eff1affd640 1 --2- 172.21.15.49:0/3883222546 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558b69905ef0 0x558b699224b0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7eff0c0096c0 tx=0x7eff0c004be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4aac3455348ff2c3 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:09:36.878 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:36.872+0000 7eff027fc640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-18T23:09:36.878 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:36.872+0000 7eff027fc640 1 -- 172.21.15.49:0/3883222546 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff0c004450 con 0x558b69905ef0 2024-03-18T23:09:36.878 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:36.872+0000 7eff027fc640 1 -- 172.21.15.49:0/3883222546 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7eff0c02c8b0 con 0x558b69905ef0 2024-03-18T23:09:36.878 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:36.872+0000 7eff24223740 1 -- 172.21.15.49:0/3883222546 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7eff1409fba0 con 0x558b69905ef0 2024-03-18T23:09:36.878 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:36.872+0000 7eff027fc640 1 -- 172.21.15.49:0/3883222546 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7eff0c05a050 con 0x558b69905ef0 2024-03-18T23:09:36.878 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:36.872+0000 7eff24223740 10 client.15567 fetch_fsmap learned FSMap version 29 2024-03-18T23:09:36.878 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:36.872+0000 7eff24223740 1 -- 172.21.15.49:0/3883222546 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x558b699859c0 con 0x558b69905ef0 2024-03-18T23:09:36.878 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:36.872+0000 7eff027fc640 1 -- 172.21.15.49:0/3883222546 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff1409fba0 con 0x558b69905ef0 2024-03-18T23:09:36.878 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:36.873+0000 7eff027fc640 1 -- 172.21.15.49:0/3883222546 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 29) ==== 8358+0+0 (secure 0 0 0) 0x7eff0c04a030 con 0x558b69905ef0 2024-03-18T23:09:36.879 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:36.873+0000 7eff24223740 10 client.15567 fetch_fsmap finished waiting for FSMap version 29 2024-03-18T23:09:36.879 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:36.873+0000 7eff24223740 10 client.15567 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T23:09:36.879 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:36.873+0000 7eff24223740 20 client.15567 populate_metadata read hostname 'smithi049' 2024-03-18T23:09:36.879 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:36.873+0000 7eff24223740 1 --2- 172.21.15.49:0/3883222546 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x558b699860e0 0x558b69988500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:09:36.879 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:36.873+0000 7eff24223740 4 client.15567 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T23:09:36.879 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:36.873+0000 7eff24223740 1 -- 172.21.15.49:0/3883222546 --> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558b69871470 con 0x558b699860e0 2024-03-18T23:09:36.879 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:36.873+0000 7eff127fc640 1 --2- 172.21.15.49:0/3883222546 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x558b699860e0 0x558b69988500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:09:36.879 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:36.873+0000 7eff127fc640 1 --2- 172.21.15.49:0/3883222546 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x558b699860e0 0x558b69988500 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=638a89b02a828b39 server_cookie=9283fbabe3f2258d in_seq=0 out_seq=0 2024-03-18T23:09:36.879 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:36.873+0000 7eff027fc640 10 client.15567 ms_handle_connect on v2:172.21.15.139:6838/4280128209 2024-03-18T23:09:37.003 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:37.001+0000 7eff027fc640 1 -- 172.21.15.49:0/3883222546 <== mds.0 v2:172.21.15.139:6838/4280128209 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x558b69871470 con 0x558b699860e0 2024-03-18T23:09:37.003 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:37.001+0000 7eff027fc640 10 client.15567 handle_command_reply: tid=0 2024-03-18T23:09:37.003 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:37.001+0000 7eff24223740 1 client.15567 shutdown 2024-03-18T23:09:37.003 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:37.001+0000 7eff24223740 1 -- 172.21.15.49:0/3883222546 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558b69905ef0 msgr2=0x558b699224b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:09:37.003 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:37.001+0000 7eff24223740 1 --2- 172.21.15.49:0/3883222546 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558b69905ef0 0x558b699224b0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7eff0c0096c0 tx=0x7eff0c004be0 comp rx=0 tx=0).stop 2024-03-18T23:09:37.004 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:37.001+0000 7eff24223740 1 -- 172.21.15.49:0/3883222546 shutdown_connections 2024-03-18T23:09:37.004 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:37.001+0000 7eff24223740 1 --2- 172.21.15.49:0/3883222546 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x558b699860e0 0x558b69988500 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:37.004 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:37.001+0000 7eff24223740 1 --2- 172.21.15.49:0/3883222546 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558b6990f5e0 0x558b69984340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:37.004 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:37.001+0000 7eff24223740 1 --2- 172.21.15.49:0/3883222546 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558b69906810 0x558b699229f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:37.004 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:37.001+0000 7eff24223740 1 --2- 172.21.15.49:0/3883222546 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558b69905ef0 0x558b699224b0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:37.004 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:37.001+0000 7eff24223740 1 -- 172.21.15.49:0/3883222546 >> 172.21.15.49:0/3883222546 conn(0x558b69900810 msgr2=0x558b6991a5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:09:37.004 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:37.001+0000 7eff027fc640 0 client.15567 ms_handle_reset on v2:172.21.15.139:6838/4280128209 2024-03-18T23:09:37.004 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:37.002+0000 7eff24223740 1 -- 172.21.15.49:0/3883222546 shutdown_connections 2024-03-18T23:09:37.004 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:37.002+0000 7eff24223740 1 -- 172.21.15.49:0/3883222546 wait complete. 2024-03-18T23:09:37.004 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:37.002+0000 7eff24223740 20 client.15567 trim_cache size 0 max 16384 2024-03-18T23:09:37.004 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:{ 2024-03-18T23:09:37.005 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "message": "", 2024-03-18T23:09:37.005 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "return_code": 0 2024-03-18T23:09:37.005 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:} 2024-03-18T23:09:37.005 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:37.002+0000 7eff2155b640 1 -- 172.21.15.49:0/1821106709 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7eff080a2b30 msgr2=0x7eff080a4ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:09:37.006 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:37.002+0000 7eff2155b640 1 --2- 172.21.15.49:0/1821106709 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7eff080a2b30 0x7eff080a4ff0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x558b699059a0 tx=0x7eff04009d30 comp rx=0 tx=0).stop 2024-03-18T23:09:37.006 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:37.002+0000 7eff2155b640 1 -- 172.21.15.49:0/1821106709 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7eff1c1056c0 msgr2=0x7eff1c19c9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:09:37.006 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:37.002+0000 7eff2155b640 1 --2- 172.21.15.49:0/1821106709 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7eff1c1056c0 0x7eff1c19c9e0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7eff14005fd0 tx=0x7eff1400ee60 comp rx=0 tx=0).stop 2024-03-18T23:09:37.006 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:37.002+0000 7eff2155b640 1 -- 172.21.15.49:0/1821106709 shutdown_connections 2024-03-18T23:09:37.006 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:37.002+0000 7eff2155b640 1 --2- 172.21.15.49:0/1821106709 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7eff080a2b30 0x7eff080a4ff0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:37.006 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:37.002+0000 7eff2155b640 1 --2- 172.21.15.49:0/1821106709 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7eff1c1056c0 0x7eff1c19c9e0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:37.006 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:37.002+0000 7eff2155b640 1 --2- 172.21.15.49:0/1821106709 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7eff1c101330 0x7eff1c198660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:37.006 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:37.002+0000 7eff2155b640 1 --2- 172.21.15.49:0/1821106709 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7eff1c100970 0x7eff1c198120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:37.006 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:37.003+0000 7eff2155b640 1 -- 172.21.15.49:0/1821106709 >> 172.21.15.49:0/1821106709 conn(0x7eff1c077570 msgr2=0x7eff1c0fe980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:09:37.006 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:37.003+0000 7eff2155b640 1 -- 172.21.15.49:0/1821106709 shutdown_connections 2024-03-18T23:09:37.006 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:37.003+0000 7eff2155b640 1 -- 172.21.15.49:0/1821106709 wait complete. 2024-03-18T23:09:37.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:09:37 smithi139 ceph-mon[67282]: pgmap v43: 97 pgs: 97 active+clean; 36 MiB data, 535 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 7.3 MiB/s wr, 426 op/s 2024-03-18T23:09:37.379 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:09:37 smithi049 ceph-mon[62991]: pgmap v43: 97 pgs: 97 active+clean; 36 MiB data, 535 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 7.3 MiB/s wr, 426 op/s 2024-03-18T23:09:37.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:09:37 smithi175 ceph-mon[67352]: pgmap v43: 97 pgs: 97 active+clean; 36 MiB data, 535 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 7.3 MiB/s wr, 426 op/s 2024-03-18T23:09:37.760 INFO:tasks.workunit.client.0.smithi049.stdout: 1 96050 13.77 MB/sec warmup 39 sec latency 59.760 ms 2024-03-18T23:09:38.760 INFO:tasks.workunit.client.0.smithi049.stdout: 1 98407 13.63 MB/sec warmup 40 sec latency 20.168 ms 2024-03-18T23:09:39.263 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:09:39 smithi175 ceph-mon[67352]: pgmap v44: 97 pgs: 97 active+clean; 42 MiB data, 518 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 7.9 MiB/s wr, 511 op/s 2024-03-18T23:09:39.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:09:39 smithi139 ceph-mon[67282]: pgmap v44: 97 pgs: 97 active+clean; 42 MiB data, 518 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 7.9 MiB/s wr, 511 op/s 2024-03-18T23:09:39.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:09:39 smithi049 ceph-mon[62991]: pgmap v44: 97 pgs: 97 active+clean; 42 MiB data, 518 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 7.9 MiB/s wr, 511 op/s 2024-03-18T23:09:39.760 INFO:tasks.workunit.client.0.smithi049.stdout: 1 100421 13.62 MB/sec warmup 41 sec latency 17.369 ms 2024-03-18T23:09:40.760 INFO:tasks.workunit.client.0.smithi049.stdout: 1 103149 13.70 MB/sec warmup 42 sec latency 104.795 ms 2024-03-18T23:09:41.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:09:41 smithi139 ceph-mon[67282]: pgmap v45: 97 pgs: 97 active+clean; 42 MiB data, 518 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 1.4 MiB/s wr, 229 op/s 2024-03-18T23:09:41.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:09:41 smithi049 ceph-mon[62991]: pgmap v45: 97 pgs: 97 active+clean; 42 MiB data, 518 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 1.4 MiB/s wr, 229 op/s 2024-03-18T23:09:41.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:09:41 smithi175 ceph-mon[67352]: pgmap v45: 97 pgs: 97 active+clean; 42 MiB data, 518 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 1.4 MiB/s wr, 229 op/s 2024-03-18T23:09:41.760 INFO:tasks.workunit.client.0.smithi049.stdout: 1 105463 13.57 MB/sec warmup 43 sec latency 16.369 ms 2024-03-18T23:09:42.760 INFO:tasks.workunit.client.0.smithi049.stdout: 1 107440 13.54 MB/sec warmup 44 sec latency 15.467 ms 2024-03-18T23:09:43.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:09:43 smithi139 ceph-mon[67282]: pgmap v46: 97 pgs: 97 active+clean; 34 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 1.9 MiB/s wr, 353 op/s 2024-03-18T23:09:43.379 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:09:43 smithi049 ceph-mon[62991]: pgmap v46: 97 pgs: 97 active+clean; 34 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 1.9 MiB/s wr, 353 op/s 2024-03-18T23:09:43.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:09:43 smithi175 ceph-mon[67352]: pgmap v46: 97 pgs: 97 active+clean; 34 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 1.9 MiB/s wr, 353 op/s 2024-03-18T23:09:43.760 INFO:tasks.workunit.client.0.smithi049.stdout: 1 110321 13.63 MB/sec warmup 45 sec latency 15.633 ms 2024-03-18T23:09:44.760 INFO:tasks.workunit.client.0.smithi049.stdout: 1 112899 13.60 MB/sec warmup 46 sec latency 20.207 ms 2024-03-18T23:09:45.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:09:45 smithi139 ceph-mon[67282]: pgmap v47: 97 pgs: 97 active+clean; 36 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 1.4 MiB/s wr, 268 op/s 2024-03-18T23:09:45.379 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:09:45 smithi049 ceph-mon[62991]: pgmap v47: 97 pgs: 97 active+clean; 36 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 1.4 MiB/s wr, 268 op/s 2024-03-18T23:09:45.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:09:45 smithi175 ceph-mon[67352]: pgmap v47: 97 pgs: 97 active+clean; 36 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 1.4 MiB/s wr, 268 op/s 2024-03-18T23:09:45.760 INFO:tasks.workunit.client.0.smithi049.stdout: 1 114694 13.51 MB/sec warmup 47 sec latency 22.383 ms 2024-03-18T23:09:46.760 INFO:tasks.workunit.client.0.smithi049.stdout: 1 117411 13.58 MB/sec warmup 48 sec latency 13.248 ms 2024-03-18T23:09:47.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:09:47 smithi139 ceph-mon[67282]: pgmap v48: 97 pgs: 97 active+clean; 36 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 1.3 MiB/s wr, 250 op/s 2024-03-18T23:09:47.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:09:47 smithi139 ceph-mon[67282]: mgrmap e25: x(active, since 90s), standbys: z, y 2024-03-18T23:09:47.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:09:47 smithi049 ceph-mon[62991]: pgmap v48: 97 pgs: 97 active+clean; 36 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 1.3 MiB/s wr, 250 op/s 2024-03-18T23:09:47.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:09:47 smithi049 ceph-mon[62991]: mgrmap e25: x(active, since 90s), standbys: z, y 2024-03-18T23:09:47.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:09:47 smithi175 ceph-mon[67352]: pgmap v48: 97 pgs: 97 active+clean; 36 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 1.3 MiB/s wr, 250 op/s 2024-03-18T23:09:47.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:09:47 smithi175 ceph-mon[67352]: mgrmap e25: x(active, since 90s), standbys: z, y 2024-03-18T23:09:47.760 INFO:tasks.workunit.client.0.smithi049.stdout: 1 119869 13.53 MB/sec warmup 49 sec latency 18.717 ms 2024-03-18T23:09:48.760 INFO:tasks.workunit.client.0.smithi049.stdout: 1 121741 13.46 MB/sec warmup 50 sec latency 22.483 ms 2024-03-18T23:09:49.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:09:49 smithi139 ceph-mon[67282]: pgmap v49: 97 pgs: 97 active+clean; 42 MiB data, 505 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 1.9 MiB/s wr, 335 op/s 2024-03-18T23:09:49.379 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:09:49 smithi049 ceph-mon[62991]: pgmap v49: 97 pgs: 97 active+clean; 42 MiB data, 505 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 1.9 MiB/s wr, 335 op/s 2024-03-18T23:09:49.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:09:49 smithi175 ceph-mon[67352]: pgmap v49: 97 pgs: 97 active+clean; 42 MiB data, 505 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 1.9 MiB/s wr, 335 op/s 2024-03-18T23:09:49.761 INFO:tasks.workunit.client.0.smithi049.stdout: 1 124489 13.53 MB/sec warmup 51 sec latency 13.330 ms 2024-03-18T23:09:50.096 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:50.094+0000 7f41c1827640 1 -- 172.21.15.49:0/3471758633 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f41bc102700 msgr2=0x7f41bc068ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:09:50.096 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:50.094+0000 7f41c1827640 1 --2- 172.21.15.49:0/3471758633 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f41bc102700 0x7f41bc068ea0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f41b4009900 tx=0x7f41b402bcd0 comp rx=0 tx=0).stop 2024-03-18T23:09:50.097 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:50.094+0000 7f41c1827640 1 -- 172.21.15.49:0/3471758633 shutdown_connections 2024-03-18T23:09:50.097 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:50.094+0000 7f41c1827640 1 --2- 172.21.15.49:0/3471758633 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f41bc10e070 0x7f41bc110480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:50.097 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:50.094+0000 7f41c1827640 1 --2- 172.21.15.49:0/3471758633 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f41bc0693e0 0x7f41bc069820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:50.097 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:50.094+0000 7f41c1827640 1 --2- 172.21.15.49:0/3471758633 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f41bc102700 0x7f41bc068ea0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:50.097 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:50.094+0000 7f41c1827640 1 -- 172.21.15.49:0/3471758633 >> 172.21.15.49:0/3471758633 conn(0x7f41bc077570 msgr2=0x7f41bc077980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:09:50.097 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:50.094+0000 7f41c1827640 1 -- 172.21.15.49:0/3471758633 shutdown_connections 2024-03-18T23:09:50.097 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:50.094+0000 7f41c1827640 1 -- 172.21.15.49:0/3471758633 wait complete. 2024-03-18T23:09:50.097 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:50.095+0000 7f41c1827640 1 Processor -- start 2024-03-18T23:09:50.097 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:50.095+0000 7f41c1827640 1 -- start start 2024-03-18T23:09:50.097 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:50.095+0000 7f41c1827640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f41bc0693e0 0x7f41bc19a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:09:50.097 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:50.095+0000 7f41c1827640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f41bc102700 0x7f41bc19a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:09:50.097 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:50.095+0000 7f41c1827640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f41bc10e070 0x7f41bc19eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:09:50.097 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:50.095+0000 7f41c1827640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f41bc113de0 con 0x7f41bc102700 2024-03-18T23:09:50.098 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:50.095+0000 7f41c1827640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f41bc113c60 con 0x7f41bc0693e0 2024-03-18T23:09:50.098 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:50.095+0000 7f41c1827640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f41bc113f60 con 0x7f41bc10e070 2024-03-18T23:09:50.098 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:50.096+0000 7f41ba7fc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f41bc102700 0x7f41bc19a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:09:50.098 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:50.096+0000 7f41ba7fc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f41bc102700 0x7f41bc19a7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:32986/0 (socket says 172.21.15.49:32986) 2024-03-18T23:09:50.098 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:50.096+0000 7f41ba7fc640 1 -- 172.21.15.49:0/4113075218 learned_addr learned my addr 172.21.15.49:0/4113075218 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:09:50.098 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:50.096+0000 7f41baffd640 1 --2- 172.21.15.49:0/4113075218 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f41bc0693e0 0x7f41bc19a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:09:50.098 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:50.096+0000 7f41bb7fe640 1 --2- 172.21.15.49:0/4113075218 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f41bc10e070 0x7f41bc19eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:09:50.098 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:50.096+0000 7f41ba7fc640 1 -- 172.21.15.49:0/4113075218 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f41bc10e070 msgr2=0x7f41bc19eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:09:50.098 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:50.096+0000 7f41ba7fc640 1 --2- 172.21.15.49:0/4113075218 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f41bc10e070 0x7f41bc19eaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:50.098 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:50.096+0000 7f41ba7fc640 1 -- 172.21.15.49:0/4113075218 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f41bc0693e0 msgr2=0x7f41bc19a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:09:50.098 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:50.096+0000 7f41ba7fc640 1 --2- 172.21.15.49:0/4113075218 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f41bc0693e0 0x7f41bc19a280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:50.098 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:50.096+0000 7f41ba7fc640 1 -- 172.21.15.49:0/4113075218 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41bc19f220 con 0x7f41bc102700 2024-03-18T23:09:50.098 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:50.096+0000 7f41ba7fc640 1 --2- 172.21.15.49:0/4113075218 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f41bc102700 0x7f41bc19a7c0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f41a400e9f0 tx=0x7f41a400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:09:50.098 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:50.096+0000 7f41c0825640 1 -- 172.21.15.49:0/4113075218 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41a400cd80 con 0x7f41bc102700 2024-03-18T23:09:50.099 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:50.096+0000 7f41c1827640 1 -- 172.21.15.49:0/4113075218 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41bc19f510 con 0x7f41bc102700 2024-03-18T23:09:50.099 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:50.096+0000 7f41c0825640 1 -- 172.21.15.49:0/4113075218 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f41a4005500 con 0x7f41bc102700 2024-03-18T23:09:50.099 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:50.096+0000 7f41c0825640 1 -- 172.21.15.49:0/4113075218 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41a4010690 con 0x7f41bc102700 2024-03-18T23:09:50.099 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:50.096+0000 7f41c1827640 1 -- 172.21.15.49:0/4113075218 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41bc1a6da0 con 0x7f41bc102700 2024-03-18T23:09:50.102 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:50.099+0000 7f41c0825640 1 -- 172.21.15.49:0/4113075218 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 25) ==== 137421+0+0 (secure 0 0 0) 0x7f41a40046c0 con 0x7f41bc102700 2024-03-18T23:09:50.102 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:50.100+0000 7f41c0825640 1 --2- 172.21.15.49:0/4113075218 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f41a80a2a10 0x7f41a80a4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:09:50.102 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:50.100+0000 7f41c0825640 1 -- 172.21.15.49:0/4113075218 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f41a40cf220 con 0x7f41bc102700 2024-03-18T23:09:50.102 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:50.100+0000 7f41baffd640 1 --2- 172.21.15.49:0/4113075218 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f41a80a2a10 0x7f41a80a4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:09:50.102 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:50.100+0000 7f41c440e740 1 Processor -- start 2024-03-18T23:09:50.103 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:50.100+0000 7f41c440e740 1 -- start start 2024-03-18T23:09:50.103 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:50.100+0000 7f41c440e740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c16991eef0 0x55c16991f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:09:50.103 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:50.100+0000 7f41c440e740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c16991f810 0x55c1699280a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:09:50.103 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:50.100+0000 7f41c440e740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c1699285e0 0x55c16992a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:09:50.103 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:50.100+0000 7f41baffd640 1 --2- 172.21.15.49:0/4113075218 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f41a80a2a10 0x7f41a80a4ed0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f41b402efd0 tx=0x7f41b402cf90 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:09:50.103 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:50.100+0000 7f41baffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c16991eef0 0x55c16991f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:09:50.103 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:50.100+0000 7f41c440e740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55c169882be0 con 0x55c16991eef0 2024-03-18T23:09:50.103 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:50.100+0000 7f41baffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c16991eef0 0x55c16991f2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:32992/0 (socket says 172.21.15.49:32992) 2024-03-18T23:09:50.103 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:50.100+0000 7f41c440e740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55c16982b960 con 0x55c1699285e0 2024-03-18T23:09:50.103 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:50.100+0000 7f41baffd640 1 -- 172.21.15.49:0/749193455 learned_addr learned my addr 172.21.15.49:0/749193455 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:09:50.103 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:50.100+0000 7f41c440e740 1 -- 172.21.15.49:0/749193455 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55c16987ec80 con 0x55c16991f810 2024-03-18T23:09:50.103 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:50.101+0000 7f41ba7fc640 1 --2- 172.21.15.49:0/749193455 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c16991f810 0x55c1699280a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:09:50.103 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:50.101+0000 7f41bb7fe640 1 --2- 172.21.15.49:0/749193455 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c1699285e0 0x55c16992a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:09:50.104 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:50.101+0000 7f41baffd640 1 -- 172.21.15.49:0/749193455 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c16991f810 msgr2=0x55c1699280a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:09:50.104 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:50.101+0000 7f41baffd640 1 --2- 172.21.15.49:0/749193455 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c16991f810 0x55c1699280a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:50.104 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:50.101+0000 7f41baffd640 1 -- 172.21.15.49:0/749193455 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c1699285e0 msgr2=0x55c16992a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:09:50.104 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:50.101+0000 7f41baffd640 1 --2- 172.21.15.49:0/749193455 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c1699285e0 0x55c16992a9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:50.104 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:50.101+0000 7f41baffd640 1 -- 172.21.15.49:0/749193455 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c169822d20 con 0x55c16991eef0 2024-03-18T23:09:50.104 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:50.101+0000 7f41bb7fe640 1 --2- 172.21.15.49:0/749193455 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c1699285e0 0x55c16992a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T23:09:50.104 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:50.101+0000 7f41baffd640 1 --2- 172.21.15.49:0/749193455 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c16991eef0 0x55c16991f2d0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f41b4009230 tx=0x7f41b403f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b400926cd31ae056 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:09:50.104 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:50.101+0000 7f41b1ffb640 1 -- 172.21.15.49:0/749193455 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41b4004300 con 0x55c16991eef0 2024-03-18T23:09:50.104 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:50.101+0000 7f41b1ffb640 1 -- 172.21.15.49:0/749193455 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f41b4050070 con 0x55c16991eef0 2024-03-18T23:09:50.104 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:50.101+0000 7f41b1ffb640 1 -- 172.21.15.49:0/749193455 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41b40055f0 con 0x55c16991eef0 2024-03-18T23:09:50.104 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:50.102+0000 7f41c440e740 1 -- 172.21.15.49:0/749193455 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c16991eef0 msgr2=0x55c16991f2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:09:50.104 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:50.102+0000 7f41c440e740 1 --2- 172.21.15.49:0/749193455 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c16991eef0 0x55c16991f2d0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f41b4009230 tx=0x7f41b403f000 comp rx=0 tx=0).stop 2024-03-18T23:09:50.104 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:50.102+0000 7f41c440e740 1 -- 172.21.15.49:0/749193455 shutdown_connections 2024-03-18T23:09:50.104 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:50.102+0000 7f41c440e740 1 --2- 172.21.15.49:0/749193455 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c1699285e0 0x55c16992a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:50.105 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:50.102+0000 7f41c440e740 1 --2- 172.21.15.49:0/749193455 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c16991f810 0x55c1699280a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:50.105 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:50.102+0000 7f41c440e740 1 --2- 172.21.15.49:0/749193455 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c16991eef0 0x55c16991f2d0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:50.105 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:50.102+0000 7f41c440e740 1 -- 172.21.15.49:0/749193455 >> 172.21.15.49:0/749193455 conn(0x55c169919810 msgr2=0x55c169919bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:09:50.105 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:50.102+0000 7f41c440e740 1 -- 172.21.15.49:0/749193455 shutdown_connections 2024-03-18T23:09:50.105 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:50.102+0000 7f41c440e740 1 -- 172.21.15.49:0/749193455 wait complete. 2024-03-18T23:09:50.105 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:50.102+0000 7f41c440e740 1 -- start start 2024-03-18T23:09:50.105 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:50.103+0000 7f41c440e740 1 Processor -- start 2024-03-18T23:09:50.109 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:50.103+0000 7f41c440e740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c16991eef0 0x55c169927fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:09:50.109 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:50.103+0000 7f41c440e740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c16991f810 0x55c169925370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:09:50.109 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:50.103+0000 7f41c440e740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c1699285e0 0x55c169925980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:09:50.109 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:50.103+0000 7f41c440e740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55c16992b1b0 con 0x55c16991f810 2024-03-18T23:09:50.109 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:50.103+0000 7f41c440e740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55c16983edb0 con 0x55c16991eef0 2024-03-18T23:09:50.109 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:50.103+0000 7f41c440e740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55c16992b330 con 0x55c1699285e0 2024-03-18T23:09:50.109 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:50.103+0000 7f41c440e740 10 client.0 _ll_register_callbacks cb 0x55c169843e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T23:09:50.109 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:50.103+0000 7f41ba7fc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c16991f810 0x55c169925370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:09:50.109 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:50.103+0000 7f41ba7fc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c16991f810 0x55c169925370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:33002/0 (socket says 172.21.15.49:33002) 2024-03-18T23:09:50.109 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:50.103+0000 7f41ba7fc640 1 -- 172.21.15.49:0/276630131 learned_addr learned my addr 172.21.15.49:0/276630131 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:09:50.109 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:50.103+0000 7f41baffd640 1 --2- 172.21.15.49:0/276630131 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c16991eef0 0x55c169927fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:09:50.109 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:50.103+0000 7f41bb7fe640 1 --2- 172.21.15.49:0/276630131 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c1699285e0 0x55c169925980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:09:50.110 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:50.103+0000 7f41ba7fc640 1 -- 172.21.15.49:0/276630131 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c1699285e0 msgr2=0x55c169925980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:09:50.110 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:50.103+0000 7f41ba7fc640 1 --2- 172.21.15.49:0/276630131 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c1699285e0 0x55c169925980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:50.110 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:50.103+0000 7f41ba7fc640 1 -- 172.21.15.49:0/276630131 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c16991eef0 msgr2=0x55c169927fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:09:50.110 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:50.103+0000 7f41ba7fc640 1 --2- 172.21.15.49:0/276630131 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c16991eef0 0x55c169927fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:50.110 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:50.103+0000 7f41ba7fc640 1 -- 172.21.15.49:0/276630131 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c16983dc60 con 0x55c16991f810 2024-03-18T23:09:50.110 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:50.103+0000 7f41baffd640 1 --2- 172.21.15.49:0/276630131 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c16991eef0 0x55c169927fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:09:50.110 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:50.104+0000 7f41ba7fc640 1 --2- 172.21.15.49:0/276630131 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c16991f810 0x55c169925370 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f41a409b5b0 tx=0x7f41a409b2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cff6184f0ba9506 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:09:50.110 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:50.104+0000 7f419affd640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-18T23:09:50.110 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:50.104+0000 7f419affd640 1 -- 172.21.15.49:0/276630131 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41a40b3070 con 0x55c16991f810 2024-03-18T23:09:50.110 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:50.104+0000 7f419affd640 1 -- 172.21.15.49:0/276630131 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f41a40860b0 con 0x55c16991f810 2024-03-18T23:09:50.110 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:50.104+0000 7f41c440e740 1 -- 172.21.15.49:0/276630131 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f41b4050070 con 0x55c16991f810 2024-03-18T23:09:50.110 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:50.104+0000 7f419affd640 1 -- 172.21.15.49:0/276630131 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f41a40862b0 con 0x55c16991f810 2024-03-18T23:09:50.110 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:50.104+0000 7f41c440e740 10 client.15579 fetch_fsmap learned FSMap version 29 2024-03-18T23:09:50.110 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:50.104+0000 7f41c440e740 1 -- 172.21.15.49:0/276630131 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c169597210 con 0x55c16991f810 2024-03-18T23:09:50.111 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:50.104+0000 7f419affd640 1 -- 172.21.15.49:0/276630131 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f41b4050070 con 0x55c16991f810 2024-03-18T23:09:50.111 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:50.104+0000 7f419affd640 1 -- 172.21.15.49:0/276630131 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 29) ==== 8358+0+0 (secure 0 0 0) 0x7f41a40a3bc0 con 0x55c16991f810 2024-03-18T23:09:50.111 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:50.104+0000 7f41c440e740 10 client.15579 fetch_fsmap finished waiting for FSMap version 29 2024-03-18T23:09:50.111 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:50.104+0000 7f41c440e740 10 client.15579 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T23:09:50.111 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:50.104+0000 7f41c440e740 20 client.15579 populate_metadata read hostname 'smithi049' 2024-03-18T23:09:50.111 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:50.104+0000 7f41c440e740 1 --2- 172.21.15.49:0/276630131 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x55c1699384b0 0x55c16993a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:09:50.111 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:50.104+0000 7f41c440e740 4 client.15579 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T23:09:50.111 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:50.104+0000 7f41c440e740 1 -- 172.21.15.49:0/276630131 --> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c169890120 con 0x55c1699384b0 2024-03-18T23:09:50.111 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:50.105+0000 7f41baffd640 1 --2- 172.21.15.49:0/276630131 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x55c1699384b0 0x55c16993a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:09:50.111 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:50.105+0000 7f41baffd640 1 --2- 172.21.15.49:0/276630131 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x55c1699384b0 0x55c16993a8a0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e618fee44dac1810 server_cookie=823fd1344e82a7e in_seq=0 out_seq=0 2024-03-18T23:09:50.111 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:50.105+0000 7f419affd640 10 client.15579 ms_handle_connect on v2:172.21.15.139:6838/4280128209 2024-03-18T23:09:50.111 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:50.108+0000 7f419affd640 1 -- 172.21.15.49:0/276630131 <== mds.0 v2:172.21.15.139:6838/4280128209 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55c169890120 con 0x55c1699384b0 2024-03-18T23:09:50.111 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:50.108+0000 7f419affd640 10 client.15579 handle_command_reply: tid=0 2024-03-18T23:09:50.111 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:50.108+0000 7f41c440e740 1 client.15579 shutdown 2024-03-18T23:09:50.111 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:50.109+0000 7f41c440e740 1 -- 172.21.15.49:0/276630131 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c16991f810 msgr2=0x55c169925370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:09:50.112 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:50.109+0000 7f41c440e740 1 --2- 172.21.15.49:0/276630131 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c16991f810 0x55c169925370 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f41a409b5b0 tx=0x7f41a409b2d0 comp rx=0 tx=0).stop 2024-03-18T23:09:50.112 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:50.109+0000 7f41c440e740 1 -- 172.21.15.49:0/276630131 shutdown_connections 2024-03-18T23:09:50.112 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:50.109+0000 7f41c440e740 1 --2- 172.21.15.49:0/276630131 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x55c1699384b0 0x55c16993a8a0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:50.112 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:50.109+0000 7f41c440e740 1 --2- 172.21.15.49:0/276630131 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c1699285e0 0x55c169925980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:50.112 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:50.109+0000 7f41c440e740 1 --2- 172.21.15.49:0/276630131 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c16991f810 0x55c169925370 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:50.112 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:50.109+0000 7f419affd640 0 client.15579 ms_handle_reset on v2:172.21.15.139:6838/4280128209 2024-03-18T23:09:50.112 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:50.109+0000 7f41c440e740 1 --2- 172.21.15.49:0/276630131 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c16991eef0 0x55c169927fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:50.112 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:50.109+0000 7f41c440e740 1 -- 172.21.15.49:0/276630131 >> 172.21.15.49:0/276630131 conn(0x55c169919810 msgr2=0x55c169918ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:09:50.112 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:50.109+0000 7f41c440e740 1 -- 172.21.15.49:0/276630131 shutdown_connections 2024-03-18T23:09:50.112 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:50.109+0000 7f41c440e740 1 -- 172.21.15.49:0/276630131 wait complete. 2024-03-18T23:09:50.112 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:50.109+0000 7f41c440e740 20 client.15579 trim_cache size 0 max 16384 2024-03-18T23:09:50.125 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:50.125+0000 7f41c440e740 1 Processor -- start 2024-03-18T23:09:50.125 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:50.125+0000 7f41c440e740 1 -- start start 2024-03-18T23:09:50.126 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:50.125+0000 7f41c440e740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c16991eef0 0x55c169933f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:09:50.126 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:50.125+0000 7f41c440e740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c16991f810 0x55c1699401b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:09:50.126 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:50.125+0000 7f41c440e740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c1699285e0 0x55c169927aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:09:50.127 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:50.125+0000 7f41c440e740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f41bc1af650 con 0x55c1699285e0 2024-03-18T23:09:50.127 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:50.125+0000 7f41c440e740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f41bc1b11c0 con 0x55c16991f810 2024-03-18T23:09:50.127 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:50.125+0000 7f41c440e740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f41bc1afd20 con 0x55c16991eef0 2024-03-18T23:09:50.127 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:50.126+0000 7f41bb7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c1699285e0 0x55c169927aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:09:50.127 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:50.126+0000 7f41baffd640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c16991eef0 0x55c169933f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:09:50.127 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:50.126+0000 7f41ba7fc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c16991f810 0x55c1699401b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:09:50.127 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:50.126+0000 7f41bb7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c1699285e0 0x55c169927aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:33016/0 (socket says 172.21.15.49:33016) 2024-03-18T23:09:50.127 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:50.126+0000 7f41bb7fe640 1 -- 172.21.15.49:0/1896080607 learned_addr learned my addr 172.21.15.49:0/1896080607 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:09:50.127 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:50.126+0000 7f41baffd640 1 -- 172.21.15.49:0/1896080607 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c16991f810 msgr2=0x55c1699401b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:09:50.127 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:50.126+0000 7f41baffd640 1 --2- 172.21.15.49:0/1896080607 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c16991f810 0x55c1699401b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:50.127 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:50.126+0000 7f41baffd640 1 -- 172.21.15.49:0/1896080607 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c1699285e0 msgr2=0x55c169927aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:09:50.127 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:50.126+0000 7f41baffd640 1 --2- 172.21.15.49:0/1896080607 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c1699285e0 0x55c169927aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:50.127 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:50.126+0000 7f41baffd640 1 -- 172.21.15.49:0/1896080607 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c169663620 con 0x55c16991eef0 2024-03-18T23:09:50.127 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:50.126+0000 7f41ba7fc640 1 --2- 172.21.15.49:0/1896080607 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c16991f810 0x55c1699401b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:09:50.128 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:50.126+0000 7f41bb7fe640 1 --2- 172.21.15.49:0/1896080607 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c1699285e0 0x55c169927aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:09:50.128 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:50.126+0000 7f41baffd640 1 --2- 172.21.15.49:0/1896080607 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c16991eef0 0x55c169933f50 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f41b4005ab0 tx=0x7f41b402e040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=274c7a4f7eb68de server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:09:50.128 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:50.127+0000 7f41b17fa640 1 -- 172.21.15.49:0/1896080607 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41b4065070 con 0x55c16991eef0 2024-03-18T23:09:50.128 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:50.127+0000 7f41b17fa640 1 -- 172.21.15.49:0/1896080607 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f41b4050040 con 0x55c16991eef0 2024-03-18T23:09:50.128 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:50.127+0000 7f41b17fa640 1 -- 172.21.15.49:0/1896080607 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41b405d070 con 0x55c16991eef0 2024-03-18T23:09:50.131 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:50.127+0000 7f41c440e740 1 -- 172.21.15.49:0/1896080607 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c16991eef0 msgr2=0x55c169933f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:09:50.131 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:50.127+0000 7f41c440e740 1 --2- 172.21.15.49:0/1896080607 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c16991eef0 0x55c169933f50 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f41b4005ab0 tx=0x7f41b402e040 comp rx=0 tx=0).stop 2024-03-18T23:09:50.132 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:50.127+0000 7f41c440e740 1 -- 172.21.15.49:0/1896080607 shutdown_connections 2024-03-18T23:09:50.132 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:50.127+0000 7f41c440e740 1 --2- 172.21.15.49:0/1896080607 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c1699285e0 0x55c169927aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:50.132 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:50.127+0000 7f41c440e740 1 --2- 172.21.15.49:0/1896080607 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c16991f810 0x55c1699401b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:50.132 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:50.127+0000 7f41c440e740 1 --2- 172.21.15.49:0/1896080607 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c16991eef0 0x55c169933f50 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:50.132 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:50.128+0000 7f41c440e740 1 -- 172.21.15.49:0/1896080607 >> 172.21.15.49:0/1896080607 conn(0x55c169919810 msgr2=0x55c169918ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:09:50.132 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:50.128+0000 7f41c440e740 1 -- 172.21.15.49:0/1896080607 shutdown_connections 2024-03-18T23:09:50.132 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:50.128+0000 7f41c440e740 1 -- 172.21.15.49:0/1896080607 wait complete. 2024-03-18T23:09:50.132 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:50.128+0000 7f41c440e740 1 -- start start 2024-03-18T23:09:50.132 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:50.128+0000 7f41c440e740 1 Processor -- start 2024-03-18T23:09:50.132 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:50.129+0000 7f41c440e740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c16991eef0 0x55c16992c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:09:50.132 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:50.129+0000 7f41c440e740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c16991f810 0x55c16992caf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:09:50.132 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:50.129+0000 7f41c440e740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c1699285e0 0x55c16999d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:09:50.132 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:50.129+0000 7f41c440e740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f41bc1b0ac0 con 0x55c16991eef0 2024-03-18T23:09:50.133 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:50.129+0000 7f41c440e740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f41bc1b03f0 con 0x55c1699285e0 2024-03-18T23:09:50.133 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:50.129+0000 7f41c440e740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f41bc1ad630 con 0x55c16991f810 2024-03-18T23:09:50.133 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:50.129+0000 7f41c440e740 10 client.0 _ll_register_callbacks cb 0x7f41a400e940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T23:09:50.133 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:50.129+0000 7f41baffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c16991eef0 0x55c16992c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:09:50.133 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:50.129+0000 7f41bb7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c1699285e0 0x55c16999d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:09:50.133 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:50.129+0000 7f41baffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c16991eef0 0x55c16992c5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:33028/0 (socket says 172.21.15.49:33028) 2024-03-18T23:09:50.133 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:50.129+0000 7f41baffd640 1 -- 172.21.15.49:0/963744555 learned_addr learned my addr 172.21.15.49:0/963744555 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:09:50.133 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:50.129+0000 7f41ba7fc640 1 --2- 172.21.15.49:0/963744555 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c16991f810 0x55c16992caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:09:50.133 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:50.129+0000 7f41baffd640 1 -- 172.21.15.49:0/963744555 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c16991f810 msgr2=0x55c16992caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:09:50.133 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:50.129+0000 7f41baffd640 1 --2- 172.21.15.49:0/963744555 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c16991f810 0x55c16992caf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:50.133 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:50.129+0000 7f41baffd640 1 -- 172.21.15.49:0/963744555 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c1699285e0 msgr2=0x55c16999d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:09:50.133 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:50.129+0000 7f41baffd640 1 --2- 172.21.15.49:0/963744555 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c1699285e0 0x55c16999d7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:50.133 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:50.129+0000 7f41baffd640 1 -- 172.21.15.49:0/963744555 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c16999dd20 con 0x55c16991eef0 2024-03-18T23:09:50.134 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:50.130+0000 7f41baffd640 1 --2- 172.21.15.49:0/963744555 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c16991eef0 0x55c16992c5b0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f41b402c210 tx=0x7f41b4009680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5a9959d2a985418 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:09:50.134 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:50.130+0000 7f419affd640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-18T23:09:50.134 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:50.130+0000 7f419affd640 1 -- 172.21.15.49:0/963744555 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41b4065070 con 0x55c16991eef0 2024-03-18T23:09:50.134 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:50.130+0000 7f419affd640 1 -- 172.21.15.49:0/963744555 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f41b405f690 con 0x55c16991eef0 2024-03-18T23:09:50.134 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:50.130+0000 7f41c440e740 1 -- 172.21.15.49:0/963744555 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f41b4050040 con 0x55c16991eef0 2024-03-18T23:09:50.134 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:50.130+0000 7f419affd640 1 -- 172.21.15.49:0/963744555 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f41b4071050 con 0x55c16991eef0 2024-03-18T23:09:50.134 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:50.130+0000 7f41c440e740 10 client.15585 fetch_fsmap learned FSMap version 29 2024-03-18T23:09:50.134 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:50.130+0000 7f41c440e740 1 -- 172.21.15.49:0/963744555 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c16999efe0 con 0x55c16991eef0 2024-03-18T23:09:50.134 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:50.130+0000 7f419affd640 1 -- 172.21.15.49:0/963744555 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f41b4050040 con 0x55c16991eef0 2024-03-18T23:09:50.134 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:50.131+0000 7f419affd640 1 -- 172.21.15.49:0/963744555 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 29) ==== 8358+0+0 (secure 0 0 0) 0x7f41b40049c0 con 0x55c16991eef0 2024-03-18T23:09:50.134 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:50.131+0000 7f41c440e740 10 client.15585 fetch_fsmap finished waiting for FSMap version 29 2024-03-18T23:09:50.134 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:50.131+0000 7f41c440e740 10 client.15585 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T23:09:50.134 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:50.131+0000 7f41c440e740 20 client.15585 populate_metadata read hostname 'smithi049' 2024-03-18T23:09:50.134 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:50.131+0000 7f41c440e740 1 --2- 172.21.15.49:0/963744555 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x55c16999f2b0 0x55c1699a16a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:09:50.135 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:50.131+0000 7f41c440e740 4 client.15585 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T23:09:50.135 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:50.131+0000 7f41c440e740 1 -- 172.21.15.49:0/963744555 --> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c16988a470 con 0x55c16999f2b0 2024-03-18T23:09:50.135 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:50.131+0000 7f41ba7fc640 1 --2- 172.21.15.49:0/963744555 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x55c16999f2b0 0x55c1699a16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:09:50.135 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:50.131+0000 7f41ba7fc640 1 --2- 172.21.15.49:0/963744555 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x55c16999f2b0 0x55c1699a16a0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eb312e1626d057be server_cookie=53a1cbc5780c53f9 in_seq=0 out_seq=0 2024-03-18T23:09:50.135 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:50.131+0000 7f419affd640 10 client.15585 ms_handle_connect on v2:172.21.15.139:6838/4280128209 2024-03-18T23:09:50.299 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:50.298+0000 7f419affd640 1 -- 172.21.15.49:0/963744555 <== mds.0 v2:172.21.15.139:6838/4280128209 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c16988a470 con 0x55c16999f2b0 2024-03-18T23:09:50.299 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:50.298+0000 7f419affd640 10 client.15585 handle_command_reply: tid=0 2024-03-18T23:09:50.299 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:50.298+0000 7f41c440e740 1 client.15585 shutdown 2024-03-18T23:09:50.299 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:50.298+0000 7f41c440e740 1 -- 172.21.15.49:0/963744555 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c16991eef0 msgr2=0x55c16992c5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:09:50.299 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:50.298+0000 7f41c440e740 1 --2- 172.21.15.49:0/963744555 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c16991eef0 0x55c16992c5b0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f41b402c210 tx=0x7f41b4009680 comp rx=0 tx=0).stop 2024-03-18T23:09:50.299 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:50.298+0000 7f41c440e740 1 -- 172.21.15.49:0/963744555 shutdown_connections 2024-03-18T23:09:50.299 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:50.298+0000 7f41c440e740 1 --2- 172.21.15.49:0/963744555 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x55c16999f2b0 0x55c1699a16a0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:50.299 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:50.298+0000 7f41c440e740 1 --2- 172.21.15.49:0/963744555 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c1699285e0 0x55c16999d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:50.300 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:50.298+0000 7f419affd640 0 client.15585 ms_handle_reset on v2:172.21.15.139:6838/4280128209 2024-03-18T23:09:50.300 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:50.298+0000 7f41c440e740 1 --2- 172.21.15.49:0/963744555 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c16991f810 0x55c16992caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:50.300 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:50.298+0000 7f41c440e740 1 --2- 172.21.15.49:0/963744555 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c16991eef0 0x55c16992c5b0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:50.300 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:50.298+0000 7f41c440e740 1 -- 172.21.15.49:0/963744555 >> 172.21.15.49:0/963744555 conn(0x55c169919810 msgr2=0x55c16991fea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:09:50.300 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:50.298+0000 7f41c440e740 1 -- 172.21.15.49:0/963744555 shutdown_connections 2024-03-18T23:09:50.300 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:50.298+0000 7f41c440e740 1 -- 172.21.15.49:0/963744555 wait complete. 2024-03-18T23:09:50.300 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:50.299+0000 7f41c440e740 20 client.15585 trim_cache size 0 max 16384 2024-03-18T23:09:50.301 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:{ 2024-03-18T23:09:50.301 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "message": "", 2024-03-18T23:09:50.301 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "return_code": 0 2024-03-18T23:09:50.301 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:} 2024-03-18T23:09:50.303 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:50.299+0000 7f41c1827640 1 -- 172.21.15.49:0/4113075218 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f41a80a2a10 msgr2=0x7f41a80a4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:09:50.303 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:50.299+0000 7f41c1827640 1 --2- 172.21.15.49:0/4113075218 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f41a80a2a10 0x7f41a80a4ed0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f41b402efd0 tx=0x7f41b402cf90 comp rx=0 tx=0).stop 2024-03-18T23:09:50.303 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:50.299+0000 7f41c1827640 1 -- 172.21.15.49:0/4113075218 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f41bc102700 msgr2=0x7f41bc19a7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:09:50.303 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:50.299+0000 7f41c1827640 1 --2- 172.21.15.49:0/4113075218 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f41bc102700 0x7f41bc19a7c0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f41a400e9f0 tx=0x7f41a400edf0 comp rx=0 tx=0).stop 2024-03-18T23:09:50.303 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:50.299+0000 7f41c1827640 1 -- 172.21.15.49:0/4113075218 shutdown_connections 2024-03-18T23:09:50.304 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:50.299+0000 7f41c1827640 1 --2- 172.21.15.49:0/4113075218 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f41a80a2a10 0x7f41a80a4ed0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:50.304 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:50.299+0000 7f41c1827640 1 --2- 172.21.15.49:0/4113075218 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f41bc10e070 0x7f41bc19eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:50.304 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:50.299+0000 7f41c1827640 1 --2- 172.21.15.49:0/4113075218 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f41bc102700 0x7f41bc19a7c0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:50.304 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:50.299+0000 7f41c1827640 1 --2- 172.21.15.49:0/4113075218 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f41bc0693e0 0x7f41bc19a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:50.304 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:50.299+0000 7f41c1827640 1 -- 172.21.15.49:0/4113075218 >> 172.21.15.49:0/4113075218 conn(0x7f41bc077570 msgr2=0x7f41bc0ff960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:09:50.304 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:50.299+0000 7f41c1827640 1 -- 172.21.15.49:0/4113075218 shutdown_connections 2024-03-18T23:09:50.304 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:09:50.299+0000 7f41c1827640 1 -- 172.21.15.49:0/4113075218 wait complete. 2024-03-18T23:09:50.760 INFO:tasks.workunit.client.0.smithi049.stdout: 1 126388 13.39 MB/sec warmup 52 sec latency 64.866 ms 2024-03-18T23:09:51.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:09:51 smithi139 ceph-mon[67282]: pgmap v50: 97 pgs: 97 active+clean; 42 MiB data, 505 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 1.3 MiB/s wr, 249 op/s 2024-03-18T23:09:51.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:09:51 smithi049 ceph-mon[62991]: pgmap v50: 97 pgs: 97 active+clean; 42 MiB data, 505 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 1.3 MiB/s wr, 249 op/s 2024-03-18T23:09:51.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:09:51 smithi175 ceph-mon[67352]: pgmap v50: 97 pgs: 97 active+clean; 42 MiB data, 505 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 1.3 MiB/s wr, 249 op/s 2024-03-18T23:09:51.760 INFO:tasks.workunit.client.0.smithi049.stdout: 1 128478 13.39 MB/sec warmup 53 sec latency 19.283 ms 2024-03-18T23:09:52.760 INFO:tasks.workunit.client.0.smithi049.stdout: 1 131395 13.47 MB/sec warmup 54 sec latency 10.289 ms 2024-03-18T23:09:53.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:09:53 smithi139 ceph-mon[67282]: pgmap v51: 97 pgs: 97 active+clean; 36 MiB data, 541 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 2.0 MiB/s wr, 445 op/s 2024-03-18T23:09:53.379 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:09:53 smithi049 ceph-mon[62991]: pgmap v51: 97 pgs: 97 active+clean; 36 MiB data, 541 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 2.0 MiB/s wr, 445 op/s 2024-03-18T23:09:53.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:09:53 smithi175 ceph-mon[67352]: pgmap v51: 97 pgs: 97 active+clean; 36 MiB data, 541 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 2.0 MiB/s wr, 445 op/s 2024-03-18T23:09:53.760 INFO:tasks.workunit.client.0.smithi049.stdout: 1 133655 13.37 MB/sec warmup 55 sec latency 14.363 ms 2024-03-18T23:09:54.760 INFO:tasks.workunit.client.0.smithi049.stdout: 1 135610 13.36 MB/sec warmup 56 sec latency 16.539 ms 2024-03-18T23:09:55.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:09:55 smithi139 ceph-mon[67282]: pgmap v52: 97 pgs: 97 active+clean; 36 MiB data, 541 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 1.5 MiB/s wr, 321 op/s 2024-03-18T23:09:55.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:09:55 smithi049 ceph-mon[62991]: pgmap v52: 97 pgs: 97 active+clean; 36 MiB data, 541 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 1.5 MiB/s wr, 321 op/s 2024-03-18T23:09:55.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:09:55 smithi175 ceph-mon[67352]: pgmap v52: 97 pgs: 97 active+clean; 36 MiB data, 541 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 1.5 MiB/s wr, 321 op/s 2024-03-18T23:09:55.760 INFO:tasks.workunit.client.0.smithi049.stdout: 1 138606 13.45 MB/sec warmup 57 sec latency 9.144 ms 2024-03-18T23:09:56.760 INFO:tasks.workunit.client.0.smithi049.stdout: 1 140963 13.36 MB/sec warmup 58 sec latency 20.290 ms 2024-03-18T23:09:57.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:09:57 smithi139 ceph-mon[67282]: pgmap v53: 97 pgs: 97 active+clean; 36 MiB data, 541 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 1.3 MiB/s wr, 280 op/s 2024-03-18T23:09:57.379 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:09:57 smithi049 ceph-mon[62991]: pgmap v53: 97 pgs: 97 active+clean; 36 MiB data, 541 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 1.3 MiB/s wr, 280 op/s 2024-03-18T23:09:57.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:09:57 smithi175 ceph-mon[67352]: pgmap v53: 97 pgs: 97 active+clean; 36 MiB data, 541 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 1.3 MiB/s wr, 280 op/s 2024-03-18T23:09:57.760 INFO:tasks.workunit.client.0.smithi049.stdout: 1 142961 13.35 MB/sec warmup 59 sec latency 34.998 ms 2024-03-18T23:09:58.761 INFO:tasks.workunit.client.0.smithi049.stdout: 1 145762 13.41 MB/sec warmup 60 sec latency 11.860 ms 2024-03-18T23:09:59.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:09:59 smithi139 ceph-mon[67282]: pgmap v54: 97 pgs: 97 active+clean; 42 MiB data, 560 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 2.0 MiB/s wr, 366 op/s 2024-03-18T23:09:59.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:09:59 smithi049 ceph-mon[62991]: pgmap v54: 97 pgs: 97 active+clean; 42 MiB data, 560 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 2.0 MiB/s wr, 366 op/s 2024-03-18T23:09:59.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:09:59 smithi175 ceph-mon[67352]: pgmap v54: 97 pgs: 97 active+clean; 42 MiB data, 560 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 2.0 MiB/s wr, 366 op/s 2024-03-18T23:09:59.761 INFO:tasks.workunit.client.0.smithi049.stdout: 1 148360 13.39 MB/sec warmup 61 sec latency 27.004 ms 2024-03-18T23:10:00.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:10:00 smithi139 ceph-mon[67282]: overall HEALTH_OK 2024-03-18T23:10:00.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:10:00 smithi049 ceph-mon[62991]: overall HEALTH_OK 2024-03-18T23:10:00.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:10:00 smithi175 ceph-mon[67352]: overall HEALTH_OK 2024-03-18T23:10:00.761 INFO:tasks.workunit.client.0.smithi049.stdout: 1 149909 13.30 MB/sec warmup 62 sec latency 72.984 ms 2024-03-18T23:10:01.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:10:01 smithi139 ceph-mon[67282]: pgmap v55: 97 pgs: 97 active+clean; 42 MiB data, 560 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 1.3 MiB/s wr, 281 op/s 2024-03-18T23:10:01.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:10:01 smithi049 ceph-mon[62991]: pgmap v55: 97 pgs: 97 active+clean; 42 MiB data, 560 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 1.3 MiB/s wr, 281 op/s 2024-03-18T23:10:01.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:10:01 smithi175 ceph-mon[67352]: pgmap v55: 97 pgs: 97 active+clean; 42 MiB data, 560 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 1.3 MiB/s wr, 281 op/s 2024-03-18T23:10:01.761 INFO:tasks.workunit.client.0.smithi049.stdout: 1 151930 13.28 MB/sec warmup 63 sec latency 24.994 ms 2024-03-18T23:10:02.761 INFO:tasks.workunit.client.0.smithi049.stdout: 1 153808 13.21 MB/sec warmup 64 sec latency 18.787 ms 2024-03-18T23:10:03.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:10:03 smithi139 ceph-mon[67282]: pgmap v56: 97 pgs: 97 active+clean; 57 MiB data, 588 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 2.8 MiB/s wr, 402 op/s 2024-03-18T23:10:03.379 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:10:03 smithi049 ceph-mon[62991]: pgmap v56: 97 pgs: 97 active+clean; 57 MiB data, 588 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 2.8 MiB/s wr, 402 op/s 2024-03-18T23:10:03.391 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:03.390+0000 7f5619e97640 1 -- 172.21.15.49:0/3486614562 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5614068ea0 msgr2=0x7f5614069300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:10:03.392 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:03.390+0000 7f5619e97640 1 --2- 172.21.15.49:0/3486614562 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5614068ea0 0x7f5614069300 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f5608009860 tx=0x7f560802bb70 comp rx=0 tx=0).stop 2024-03-18T23:10:03.392 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:03.390+0000 7f5619e97640 1 -- 172.21.15.49:0/3486614562 shutdown_connections 2024-03-18T23:10:03.392 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:03.390+0000 7f5619e97640 1 --2- 172.21.15.49:0/3486614562 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5614069840 0x7f561410e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:03.392 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:03.390+0000 7f5619e97640 1 --2- 172.21.15.49:0/3486614562 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5614068ea0 0x7f5614069300 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:03.392 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:03.390+0000 7f5619e97640 1 --2- 172.21.15.49:0/3486614562 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5614107910 0x7f5614107cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:03.392 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:03.390+0000 7f5619e97640 1 -- 172.21.15.49:0/3486614562 >> 172.21.15.49:0/3486614562 conn(0x7f5614077570 msgr2=0x7f5614077980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:10:03.392 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:03.390+0000 7f5619e97640 1 -- 172.21.15.49:0/3486614562 shutdown_connections 2024-03-18T23:10:03.392 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:03.390+0000 7f5619e97640 1 -- 172.21.15.49:0/3486614562 wait complete. 2024-03-18T23:10:03.392 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:03.391+0000 7f5619e97640 1 Processor -- start 2024-03-18T23:10:03.394 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:03.391+0000 7f5619e97640 1 -- start start 2024-03-18T23:10:03.394 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:03.391+0000 7f5619e97640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5614068ea0 0x7f5614198070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:10:03.394 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:03.391+0000 7f5619e97640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5614069840 0x7f56141985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:10:03.394 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:03.391+0000 7f5619e97640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5614107910 0x7f561419c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:10:03.394 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:03.391+0000 7f5619e97640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f5614111c90 con 0x7f5614107910 2024-03-18T23:10:03.394 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:03.391+0000 7f5619e97640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f5614111b10 con 0x7f5614068ea0 2024-03-18T23:10:03.395 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:03.391+0000 7f5619e97640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f5614111e10 con 0x7f5614069840 2024-03-18T23:10:03.395 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:03.391+0000 7f5613fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5614107910 0x7f561419c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:10:03.395 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:03.391+0000 7f56137fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5614068ea0 0x7f5614198070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:10:03.395 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:03.391+0000 7f5613fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5614107910 0x7f561419c930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:44998/0 (socket says 172.21.15.49:44998) 2024-03-18T23:10:03.395 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:03.391+0000 7f5613fff640 1 -- 172.21.15.49:0/1651307945 learned_addr learned my addr 172.21.15.49:0/1651307945 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:10:03.395 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:03.391+0000 7f5612ffd640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5614069840 0x7f56141985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:10:03.395 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:03.392+0000 7f5613fff640 1 -- 172.21.15.49:0/1651307945 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5614069840 msgr2=0x7f56141985b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:10:03.395 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:03.392+0000 7f5613fff640 1 --2- 172.21.15.49:0/1651307945 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5614069840 0x7f56141985b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:03.395 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:03.392+0000 7f5613fff640 1 -- 172.21.15.49:0/1651307945 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5614068ea0 msgr2=0x7f5614198070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:10:03.395 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:03.392+0000 7f5613fff640 1 --2- 172.21.15.49:0/1651307945 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5614068ea0 0x7f5614198070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:03.395 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:03.392+0000 7f5613fff640 1 -- 172.21.15.49:0/1651307945 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f561419d060 con 0x7f5614107910 2024-03-18T23:10:03.395 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:03.392+0000 7f56137fe640 1 --2- 172.21.15.49:0/1651307945 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5614068ea0 0x7f5614198070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:10:03.395 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:03.392+0000 7f5612ffd640 1 --2- 172.21.15.49:0/1651307945 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5614069840 0x7f56141985b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:10:03.395 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:03.392+0000 7f5613fff640 1 --2- 172.21.15.49:0/1651307945 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5614107910 0x7f561419c930 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f560c00eee0 tx=0x7f560c00c740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:10:03.396 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:03.392+0000 7f5610ff9640 1 -- 172.21.15.49:0/1651307945 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f560c004b20 con 0x7f5614107910 2024-03-18T23:10:03.396 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:03.392+0000 7f5619e97640 1 -- 172.21.15.49:0/1651307945 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f561419d350 con 0x7f5614107910 2024-03-18T23:10:03.396 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:03.392+0000 7f5610ff9640 1 -- 172.21.15.49:0/1651307945 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f560c003e50 con 0x7f5614107910 2024-03-18T23:10:03.396 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:03.392+0000 7f5610ff9640 1 -- 172.21.15.49:0/1651307945 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f560c005280 con 0x7f5614107910 2024-03-18T23:10:03.396 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:03.392+0000 7f5619e97640 1 -- 172.21.15.49:0/1651307945 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56141a4be0 con 0x7f5614107910 2024-03-18T23:10:03.398 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:03.395+0000 7f5610ff9640 1 -- 172.21.15.49:0/1651307945 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 25) ==== 137421+0+0 (secure 0 0 0) 0x7f560c01d020 con 0x7f5614107910 2024-03-18T23:10:03.399 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:03.395+0000 7f561ca84740 1 Processor -- start 2024-03-18T23:10:03.399 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:03.395+0000 7f561ca84740 1 -- start start 2024-03-18T23:10:03.399 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:03.396+0000 7f561ca84740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55868a366ef0 0x55868a3672d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:10:03.399 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:03.396+0000 7f561ca84740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55868a367810 0x55868a3700a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:10:03.399 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:03.396+0000 7f561ca84740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55868a3705e0 0x55868a3729d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:10:03.399 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:03.396+0000 7f561ca84740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55868a2cabe0 con 0x55868a3705e0 2024-03-18T23:10:03.399 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:03.396+0000 7f561ca84740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55868a273960 con 0x55868a367810 2024-03-18T23:10:03.399 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:03.396+0000 7f561ca84740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55868a2c6c80 con 0x55868a366ef0 2024-03-18T23:10:03.399 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:03.396+0000 7f5612ffd640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55868a366ef0 0x55868a3672d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:10:03.399 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:03.396+0000 7f56137fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55868a367810 0x55868a3700a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:10:03.399 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:03.396+0000 7f5612ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55868a3705e0 0x55868a3729d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:10:03.399 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:03.396+0000 7f56137fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55868a367810 0x55868a3700a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.49:34538/0 (socket says 172.21.15.49:34538) 2024-03-18T23:10:03.400 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:03.396+0000 7f56137fe640 1 -- 172.21.15.49:0/1478017947 learned_addr learned my addr 172.21.15.49:0/1478017947 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:10:03.400 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:03.396+0000 7f5610ff9640 1 --2- 172.21.15.49:0/1651307945 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f55fc0a2a10 0x7f55fc0a4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:10:03.400 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:03.396+0000 7f5613fff640 1 --2- 172.21.15.49:0/1651307945 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f55fc0a2a10 0x7f55fc0a4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:10:03.400 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:03.396+0000 7f5610ff9640 1 -- 172.21.15.49:0/1651307945 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f560c0cef60 con 0x7f5614107910 2024-03-18T23:10:03.400 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:03.396+0000 7f56137fe640 1 -- 172.21.15.49:0/1478017947 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55868a366ef0 msgr2=0x55868a3672d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:10:03.400 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:03.396+0000 7f56137fe640 1 --2- 172.21.15.49:0/1478017947 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55868a366ef0 0x55868a3672d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:03.400 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:03.396+0000 7f56137fe640 1 -- 172.21.15.49:0/1478017947 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55868a3705e0 msgr2=0x55868a3729d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:10:03.400 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:03.396+0000 7f56137fe640 1 --2- 172.21.15.49:0/1478017947 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55868a3705e0 0x55868a3729d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:03.400 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:03.396+0000 7f56137fe640 1 -- 172.21.15.49:0/1478017947 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55868a26ad20 con 0x55868a367810 2024-03-18T23:10:03.400 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:03.396+0000 7f5612ffd640 1 --2- 172.21.15.49:0/1478017947 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55868a3705e0 0x55868a3729d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:10:03.400 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:03.396+0000 7f56137fe640 1 --2- 172.21.15.49:0/1478017947 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55868a367810 0x55868a3700a0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f56040115c0 tx=0x7f5604011a80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5250c77fa1714463 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:10:03.400 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:03.396+0000 7f5613fff640 1 --2- 172.21.15.49:0/1651307945 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f55fc0a2a10 0x7f55fc0a4ed0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f560c09b250 tx=0x7f560c086040 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:10:03.400 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:03.397+0000 7f55f9ffb640 1 -- 172.21.15.49:0/1478017947 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f560400f640 con 0x55868a367810 2024-03-18T23:10:03.400 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:03.397+0000 7f55f9ffb640 1 -- 172.21.15.49:0/1478017947 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f560400fcc0 con 0x55868a367810 2024-03-18T23:10:03.401 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:03.397+0000 7f55f9ffb640 1 -- 172.21.15.49:0/1478017947 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5604019cd0 con 0x55868a367810 2024-03-18T23:10:03.401 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:03.397+0000 7f561ca84740 1 -- 172.21.15.49:0/1478017947 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55868a367810 msgr2=0x55868a3700a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:10:03.401 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:03.397+0000 7f561ca84740 1 --2- 172.21.15.49:0/1478017947 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55868a367810 0x55868a3700a0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f56040115c0 tx=0x7f5604011a80 comp rx=0 tx=0).stop 2024-03-18T23:10:03.401 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:03.397+0000 7f561ca84740 1 -- 172.21.15.49:0/1478017947 shutdown_connections 2024-03-18T23:10:03.401 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:03.397+0000 7f561ca84740 1 --2- 172.21.15.49:0/1478017947 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55868a3705e0 0x55868a3729d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:03.401 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:03.397+0000 7f561ca84740 1 --2- 172.21.15.49:0/1478017947 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55868a367810 0x55868a3700a0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:03.401 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:03.397+0000 7f561ca84740 1 --2- 172.21.15.49:0/1478017947 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55868a366ef0 0x55868a3672d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:03.401 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:03.397+0000 7f561ca84740 1 -- 172.21.15.49:0/1478017947 >> 172.21.15.49:0/1478017947 conn(0x55868a361810 msgr2=0x55868a361bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:10:03.401 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:03.398+0000 7f561ca84740 1 -- 172.21.15.49:0/1478017947 shutdown_connections 2024-03-18T23:10:03.401 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:03.398+0000 7f561ca84740 1 -- 172.21.15.49:0/1478017947 wait complete. 2024-03-18T23:10:03.401 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:03.398+0000 7f561ca84740 1 -- start start 2024-03-18T23:10:03.401 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:03.398+0000 7f561ca84740 1 Processor -- start 2024-03-18T23:10:03.405 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:03.398+0000 7f561ca84740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55868a366ef0 0x55868a36ff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:10:03.405 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:03.398+0000 7f561ca84740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55868a367810 0x55868a36d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:10:03.405 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:03.398+0000 7f561ca84740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55868a3705e0 0x55868a36d870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:10:03.405 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:03.398+0000 7f561ca84740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55868a373210 con 0x55868a3705e0 2024-03-18T23:10:03.405 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:03.398+0000 7f561ca84740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55868a286db0 con 0x55868a366ef0 2024-03-18T23:10:03.405 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:03.398+0000 7f561ca84740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55868a373390 con 0x55868a367810 2024-03-18T23:10:03.405 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:03.398+0000 7f561ca84740 10 client.0 _ll_register_callbacks cb 0x55868a28be70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T23:10:03.406 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:03.398+0000 7f56137fe640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55868a367810 0x55868a36d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:10:03.406 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:03.398+0000 7f5612ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55868a3705e0 0x55868a36d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:10:03.406 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:03.398+0000 7f56137fe640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55868a367810 0x55868a36d2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.49:49412/0 (socket says 172.21.15.49:49412) 2024-03-18T23:10:03.406 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:03.398+0000 7f56137fe640 1 -- 172.21.15.49:0/590450486 learned_addr learned my addr 172.21.15.49:0/590450486 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:10:03.406 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:03.398+0000 7f5612ffd640 1 --2- 172.21.15.49:0/590450486 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55868a366ef0 0x55868a36ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:10:03.406 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:03.399+0000 7f5612ffd640 1 -- 172.21.15.49:0/590450486 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55868a367810 msgr2=0x55868a36d2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:10:03.406 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:03.399+0000 7f5612ffd640 1 --2- 172.21.15.49:0/590450486 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55868a367810 0x55868a36d2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:03.406 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:03.399+0000 7f5612ffd640 1 -- 172.21.15.49:0/590450486 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55868a366ef0 msgr2=0x55868a36ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:10:03.406 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:03.399+0000 7f5612ffd640 1 --2- 172.21.15.49:0/590450486 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55868a366ef0 0x55868a36ff60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:03.406 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:03.399+0000 7f5612ffd640 1 -- 172.21.15.49:0/590450486 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55868a285c60 con 0x55868a3705e0 2024-03-18T23:10:03.406 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:03.399+0000 7f5612ffd640 1 --2- 172.21.15.49:0/590450486 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55868a3705e0 0x55868a36d870 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f5608005940 tx=0x7f560802bb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9201aa508ea4ce14 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:10:03.406 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:03.399+0000 7f55f2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-18T23:10:03.406 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:03.399+0000 7f55f2ffd640 1 -- 172.21.15.49:0/590450486 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5608039040 con 0x55868a3705e0 2024-03-18T23:10:03.407 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:03.399+0000 7f55f2ffd640 1 -- 172.21.15.49:0/590450486 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f560802c710 con 0x55868a3705e0 2024-03-18T23:10:03.407 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:03.399+0000 7f561ca84740 1 -- 172.21.15.49:0/590450486 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f560400fcc0 con 0x55868a3705e0 2024-03-18T23:10:03.407 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:03.399+0000 7f55f2ffd640 1 -- 172.21.15.49:0/590450486 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f56080348e0 con 0x55868a3705e0 2024-03-18T23:10:03.407 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:03.399+0000 7f561ca84740 10 client.15597 fetch_fsmap learned FSMap version 29 2024-03-18T23:10:03.407 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:03.399+0000 7f561ca84740 1 -- 172.21.15.49:0/590450486 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x558689fdf210 con 0x55868a3705e0 2024-03-18T23:10:03.407 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:03.399+0000 7f55f2ffd640 1 -- 172.21.15.49:0/590450486 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f560400fcc0 con 0x55868a3705e0 2024-03-18T23:10:03.407 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:03.400+0000 7f55f2ffd640 1 -- 172.21.15.49:0/590450486 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 29) ==== 8358+0+0 (secure 0 0 0) 0x7f560803f040 con 0x55868a3705e0 2024-03-18T23:10:03.407 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:03.400+0000 7f561ca84740 10 client.15597 fetch_fsmap finished waiting for FSMap version 29 2024-03-18T23:10:03.407 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:03.400+0000 7f561ca84740 10 client.15597 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T23:10:03.407 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:03.400+0000 7f561ca84740 20 client.15597 populate_metadata read hostname 'smithi049' 2024-03-18T23:10:03.407 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:03.400+0000 7f561ca84740 1 --2- 172.21.15.49:0/590450486 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x55868a380400 0x55868a3827f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:10:03.407 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:03.400+0000 7f561ca84740 4 client.15597 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T23:10:03.407 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:03.400+0000 7f561ca84740 1 -- 172.21.15.49:0/590450486 --> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55868a2d8120 con 0x55868a380400 2024-03-18T23:10:03.408 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:03.400+0000 7f56137fe640 1 --2- 172.21.15.49:0/590450486 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x55868a380400 0x55868a3827f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:10:03.408 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:03.400+0000 7f55f2ffd640 10 client.15597 ms_handle_connect on v2:172.21.15.139:6838/4280128209 2024-03-18T23:10:03.408 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:03.400+0000 7f56137fe640 1 --2- 172.21.15.49:0/590450486 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x55868a380400 0x55868a3827f0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b41023e6cf0c5baa server_cookie=3b6150867656da96 in_seq=0 out_seq=0 2024-03-18T23:10:03.408 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:03.401+0000 7f55f2ffd640 1 -- 172.21.15.49:0/590450486 <== mds.0 v2:172.21.15.139:6838/4280128209 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55868a2d8120 con 0x55868a380400 2024-03-18T23:10:03.408 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:03.401+0000 7f55f2ffd640 10 client.15597 handle_command_reply: tid=0 2024-03-18T23:10:03.408 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:03.401+0000 7f561ca84740 1 client.15597 shutdown 2024-03-18T23:10:03.408 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:03.402+0000 7f561ca84740 1 -- 172.21.15.49:0/590450486 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55868a3705e0 msgr2=0x55868a36d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:10:03.408 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:03.402+0000 7f561ca84740 1 --2- 172.21.15.49:0/590450486 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55868a3705e0 0x55868a36d870 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f5608005940 tx=0x7f560802bb70 comp rx=0 tx=0).stop 2024-03-18T23:10:03.408 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:03.402+0000 7f561ca84740 1 -- 172.21.15.49:0/590450486 shutdown_connections 2024-03-18T23:10:03.408 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:03.402+0000 7f561ca84740 1 --2- 172.21.15.49:0/590450486 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x55868a380400 0x55868a3827f0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:03.408 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:03.402+0000 7f561ca84740 1 --2- 172.21.15.49:0/590450486 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55868a3705e0 0x55868a36d870 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:03.408 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:03.402+0000 7f55f2ffd640 0 client.15597 ms_handle_reset on v2:172.21.15.139:6838/4280128209 2024-03-18T23:10:03.408 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:03.402+0000 7f561ca84740 1 --2- 172.21.15.49:0/590450486 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55868a367810 0x55868a36d2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:03.408 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:03.402+0000 7f561ca84740 1 --2- 172.21.15.49:0/590450486 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55868a366ef0 0x55868a36ff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:03.409 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:03.402+0000 7f561ca84740 1 -- 172.21.15.49:0/590450486 >> 172.21.15.49:0/590450486 conn(0x55868a361810 msgr2=0x55868a360ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:10:03.409 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:03.402+0000 7f561ca84740 1 -- 172.21.15.49:0/590450486 shutdown_connections 2024-03-18T23:10:03.409 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:03.402+0000 7f561ca84740 1 -- 172.21.15.49:0/590450486 wait complete. 2024-03-18T23:10:03.409 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:03.402+0000 7f561ca84740 20 client.15597 trim_cache size 0 max 16384 2024-03-18T23:10:03.422 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:03.417+0000 7f561ca84740 1 Processor -- start 2024-03-18T23:10:03.422 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:03.417+0000 7f561ca84740 1 -- start start 2024-03-18T23:10:03.423 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:03.417+0000 7f561ca84740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55868a361810 0x55868a37bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:10:03.423 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:03.417+0000 7f561ca84740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55868a366ef0 0x55868a3880c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:10:03.423 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:03.417+0000 7f561ca84740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55868a367810 0x55868a36fa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:10:03.423 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:03.417+0000 7f561ca84740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f56141ad420 con 0x55868a367810 2024-03-18T23:10:03.423 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:03.417+0000 7f561ca84740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f56141aef90 con 0x55868a361810 2024-03-18T23:10:03.423 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:03.417+0000 7f561ca84740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f56141adaf0 con 0x55868a366ef0 2024-03-18T23:10:03.423 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:03.417+0000 7f5612ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55868a367810 0x55868a36fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:10:03.423 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:03.417+0000 7f5612ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55868a367810 0x55868a36fa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:45026/0 (socket says 172.21.15.49:45026) 2024-03-18T23:10:03.423 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:03.417+0000 7f5612ffd640 1 -- 172.21.15.49:0/2058660795 learned_addr learned my addr 172.21.15.49:0/2058660795 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:10:03.423 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:03.417+0000 7f56137fe640 1 --2- 172.21.15.49:0/2058660795 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55868a366ef0 0x55868a3880c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:10:03.423 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:03.417+0000 7f5612ffd640 1 --2- 172.21.15.49:0/2058660795 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55868a361810 0x55868a37bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:10:03.423 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:03.417+0000 7f5612ffd640 1 -- 172.21.15.49:0/2058660795 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55868a366ef0 msgr2=0x55868a3880c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:10:03.423 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:03.417+0000 7f5612ffd640 1 --2- 172.21.15.49:0/2058660795 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55868a366ef0 0x55868a3880c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:03.423 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:03.417+0000 7f5612ffd640 1 -- 172.21.15.49:0/2058660795 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55868a361810 msgr2=0x55868a37bfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:10:03.424 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:03.417+0000 7f5612ffd640 1 --2- 172.21.15.49:0/2058660795 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55868a361810 0x55868a37bfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:03.424 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:03.417+0000 7f5612ffd640 1 -- 172.21.15.49:0/2058660795 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55868a382b60 con 0x55868a367810 2024-03-18T23:10:03.424 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:03.418+0000 7f5612ffd640 1 --2- 172.21.15.49:0/2058660795 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55868a367810 0x55868a36fa20 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f560802cd70 tx=0x7f5608009490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d9f837b10664f5e server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:10:03.424 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:03.418+0000 7f55f97fa640 1 -- 172.21.15.49:0/2058660795 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5608039050 con 0x55868a367810 2024-03-18T23:10:03.424 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:03.418+0000 7f55f97fa640 1 -- 172.21.15.49:0/2058660795 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5608003fd0 con 0x55868a367810 2024-03-18T23:10:03.424 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:03.418+0000 7f55f97fa640 1 -- 172.21.15.49:0/2058660795 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5608032640 con 0x55868a367810 2024-03-18T23:10:03.424 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:03.418+0000 7f561ca84740 1 -- 172.21.15.49:0/2058660795 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55868a367810 msgr2=0x55868a36fa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:10:03.424 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:03.418+0000 7f561ca84740 1 --2- 172.21.15.49:0/2058660795 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55868a367810 0x55868a36fa20 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f560802cd70 tx=0x7f5608009490 comp rx=0 tx=0).stop 2024-03-18T23:10:03.424 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:03.418+0000 7f561ca84740 1 -- 172.21.15.49:0/2058660795 shutdown_connections 2024-03-18T23:10:03.424 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:03.418+0000 7f561ca84740 1 --2- 172.21.15.49:0/2058660795 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55868a367810 0x55868a36fa20 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:03.424 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:03.418+0000 7f561ca84740 1 --2- 172.21.15.49:0/2058660795 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55868a366ef0 0x55868a3880c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:03.424 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:03.418+0000 7f561ca84740 1 --2- 172.21.15.49:0/2058660795 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55868a361810 0x55868a37bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:03.424 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:03.418+0000 7f561ca84740 1 -- 172.21.15.49:0/2058660795 >> 172.21.15.49:0/2058660795 conn(0x55868a360e40 msgr2=0x55868a36bbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:10:03.425 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:03.419+0000 7f561ca84740 1 -- 172.21.15.49:0/2058660795 shutdown_connections 2024-03-18T23:10:03.425 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:03.419+0000 7f561ca84740 1 -- 172.21.15.49:0/2058660795 wait complete. 2024-03-18T23:10:03.425 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:03.419+0000 7f561ca84740 1 -- start start 2024-03-18T23:10:03.425 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:03.419+0000 7f561ca84740 1 Processor -- start 2024-03-18T23:10:03.425 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:03.419+0000 7f561ca84740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55868a361810 0x55868a369dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:10:03.425 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:03.419+0000 7f561ca84740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55868a366ef0 0x55868a36a310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:10:03.425 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:03.419+0000 7f561ca84740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55868a367810 0x55868a36a850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:10:03.425 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:03.419+0000 7f561ca84740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f56141ae890 con 0x55868a361810 2024-03-18T23:10:03.425 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:03.419+0000 7f561ca84740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f56141ae1c0 con 0x55868a367810 2024-03-18T23:10:03.425 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:03.419+0000 7f561ca84740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f56141ab400 con 0x55868a366ef0 2024-03-18T23:10:03.425 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:03.419+0000 7f561ca84740 10 client.0 _ll_register_callbacks cb 0x7f56080096b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T23:10:03.425 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:03.419+0000 7f5612ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55868a361810 0x55868a369dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:10:03.425 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:03.420+0000 7f5612ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55868a361810 0x55868a369dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:45034/0 (socket says 172.21.15.49:45034) 2024-03-18T23:10:03.425 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:03.420+0000 7f5612ffd640 1 -- 172.21.15.49:0/1637727547 learned_addr learned my addr 172.21.15.49:0/1637727547 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:10:03.426 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:03.420+0000 7f56137fe640 1 --2- 172.21.15.49:0/1637727547 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55868a366ef0 0x55868a36a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:10:03.426 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:03.420+0000 7f5612ffd640 1 --2- 172.21.15.49:0/1637727547 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55868a367810 0x55868a36a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:10:03.426 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:03.420+0000 7f5612ffd640 1 -- 172.21.15.49:0/1637727547 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55868a366ef0 msgr2=0x55868a36a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:10:03.426 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:03.420+0000 7f5612ffd640 1 --2- 172.21.15.49:0/1637727547 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55868a366ef0 0x55868a36a310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:03.426 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:03.420+0000 7f5612ffd640 1 -- 172.21.15.49:0/1637727547 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55868a367810 msgr2=0x55868a36a850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:10:03.426 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:03.420+0000 7f5612ffd640 1 --2- 172.21.15.49:0/1637727547 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55868a367810 0x55868a36a850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:03.426 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:03.420+0000 7f5612ffd640 1 -- 172.21.15.49:0/1637727547 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55868a0ab620 con 0x55868a361810 2024-03-18T23:10:03.426 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:03.420+0000 7f5612ffd640 1 --2- 172.21.15.49:0/1637727547 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55868a361810 0x55868a369dd0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f5608005e00 tx=0x7f560802bb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=491ee227f0ac3d0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:10:03.426 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:03.420+0000 7f55f2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-18T23:10:03.426 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:03.420+0000 7f55f2ffd640 1 -- 172.21.15.49:0/1637727547 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5608069060 con 0x55868a361810 2024-03-18T23:10:03.426 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:03.420+0000 7f561ca84740 1 -- 172.21.15.49:0/1637727547 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5608003fd0 con 0x55868a361810 2024-03-18T23:10:03.426 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:03.420+0000 7f55f2ffd640 1 -- 172.21.15.49:0/1637727547 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f560802c510 con 0x55868a361810 2024-03-18T23:10:03.426 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:03.420+0000 7f55f2ffd640 1 -- 172.21.15.49:0/1637727547 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f5608039070 con 0x55868a361810 2024-03-18T23:10:03.427 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:03.420+0000 7f561ca84740 10 client.15603 fetch_fsmap learned FSMap version 29 2024-03-18T23:10:03.427 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:03.420+0000 7f561ca84740 1 -- 172.21.15.49:0/1637727547 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55868a3e6750 con 0x55868a361810 2024-03-18T23:10:03.427 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:03.421+0000 7f55f2ffd640 1 -- 172.21.15.49:0/1637727547 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5608003fd0 con 0x55868a361810 2024-03-18T23:10:03.427 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:03.421+0000 7f55f2ffd640 1 -- 172.21.15.49:0/1637727547 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 29) ==== 8358+0+0 (secure 0 0 0) 0x7f560803f070 con 0x55868a361810 2024-03-18T23:10:03.427 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:03.421+0000 7f561ca84740 10 client.15603 fetch_fsmap finished waiting for FSMap version 29 2024-03-18T23:10:03.427 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:03.421+0000 7f561ca84740 10 client.15603 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T23:10:03.427 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:03.421+0000 7f561ca84740 20 client.15603 populate_metadata read hostname 'smithi049' 2024-03-18T23:10:03.427 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:03.421+0000 7f561ca84740 1 --2- 172.21.15.49:0/1637727547 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x55868a3705e0 0x55868a3e8f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:10:03.427 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:03.421+0000 7f561ca84740 4 client.15603 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T23:10:03.427 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:03.421+0000 7f561ca84740 1 -- 172.21.15.49:0/1637727547 --> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55868a2d2470 con 0x55868a3705e0 2024-03-18T23:10:03.427 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:03.421+0000 7f56137fe640 1 --2- 172.21.15.49:0/1637727547 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x55868a3705e0 0x55868a3e8f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:10:03.427 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:03.421+0000 7f56137fe640 1 --2- 172.21.15.49:0/1637727547 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x55868a3705e0 0x55868a3e8f20 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5528d2b501a0e545 server_cookie=552fe91824251ac in_seq=0 out_seq=0 2024-03-18T23:10:03.427 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:03.421+0000 7f55f2ffd640 10 client.15603 ms_handle_connect on v2:172.21.15.139:6838/4280128209 2024-03-18T23:10:03.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:10:03 smithi175 ceph-mon[67352]: pgmap v56: 97 pgs: 97 active+clean; 57 MiB data, 588 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 2.8 MiB/s wr, 402 op/s 2024-03-18T23:10:03.579 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:03.577+0000 7f55f2ffd640 1 -- 172.21.15.49:0/1637727547 <== mds.0 v2:172.21.15.139:6838/4280128209 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55868a2d2470 con 0x55868a3705e0 2024-03-18T23:10:03.579 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:03.577+0000 7f55f2ffd640 10 client.15603 handle_command_reply: tid=0 2024-03-18T23:10:03.579 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:03.577+0000 7f561ca84740 1 client.15603 shutdown 2024-03-18T23:10:03.579 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:03.578+0000 7f561ca84740 1 -- 172.21.15.49:0/1637727547 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55868a361810 msgr2=0x55868a369dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:10:03.579 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:03.578+0000 7f561ca84740 1 --2- 172.21.15.49:0/1637727547 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55868a361810 0x55868a369dd0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f5608005e00 tx=0x7f560802bb70 comp rx=0 tx=0).stop 2024-03-18T23:10:03.579 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:03.578+0000 7f561ca84740 1 -- 172.21.15.49:0/1637727547 shutdown_connections 2024-03-18T23:10:03.579 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:03.578+0000 7f561ca84740 1 --2- 172.21.15.49:0/1637727547 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x55868a3705e0 0x55868a3e8f20 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:03.580 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:03.578+0000 7f561ca84740 1 --2- 172.21.15.49:0/1637727547 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55868a367810 0x55868a36a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:03.580 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:03.578+0000 7f55f2ffd640 0 client.15603 ms_handle_reset on v2:172.21.15.139:6838/4280128209 2024-03-18T23:10:03.580 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:03.578+0000 7f561ca84740 1 --2- 172.21.15.49:0/1637727547 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55868a366ef0 0x55868a36a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:03.580 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:03.578+0000 7f561ca84740 1 --2- 172.21.15.49:0/1637727547 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55868a361810 0x55868a369dd0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:03.580 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:03.578+0000 7f561ca84740 1 -- 172.21.15.49:0/1637727547 >> 172.21.15.49:0/1637727547 conn(0x55868a360e40 msgr2=0x55868a36c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:10:03.580 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:03.578+0000 7f561ca84740 1 -- 172.21.15.49:0/1637727547 shutdown_connections 2024-03-18T23:10:03.580 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:03.578+0000 7f561ca84740 1 -- 172.21.15.49:0/1637727547 wait complete. 2024-03-18T23:10:03.580 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:03.578+0000 7f561ca84740 20 client.15603 trim_cache size 0 max 16384 2024-03-18T23:10:03.580 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:{ 2024-03-18T23:10:03.580 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "message": "", 2024-03-18T23:10:03.580 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "return_code": 0 2024-03-18T23:10:03.581 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:} 2024-03-18T23:10:03.581 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:03.578+0000 7f5619e97640 1 -- 172.21.15.49:0/1651307945 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f55fc0a2a10 msgr2=0x7f55fc0a4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:10:03.581 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:03.578+0000 7f5619e97640 1 --2- 172.21.15.49:0/1651307945 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f55fc0a2a10 0x7f55fc0a4ed0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f560c09b250 tx=0x7f560c086040 comp rx=0 tx=0).stop 2024-03-18T23:10:03.581 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:03.578+0000 7f5619e97640 1 -- 172.21.15.49:0/1651307945 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5614107910 msgr2=0x7f561419c930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:10:03.582 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:03.578+0000 7f5619e97640 1 --2- 172.21.15.49:0/1651307945 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5614107910 0x7f561419c930 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f560c00eee0 tx=0x7f560c00c740 comp rx=0 tx=0).stop 2024-03-18T23:10:03.582 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:03.578+0000 7f5619e97640 1 -- 172.21.15.49:0/1651307945 shutdown_connections 2024-03-18T23:10:03.582 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:03.578+0000 7f5619e97640 1 --2- 172.21.15.49:0/1651307945 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f55fc0a2a10 0x7f55fc0a4ed0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:03.582 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:03.578+0000 7f5619e97640 1 --2- 172.21.15.49:0/1651307945 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5614107910 0x7f561419c930 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:03.582 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:03.578+0000 7f5619e97640 1 --2- 172.21.15.49:0/1651307945 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5614069840 0x7f56141985b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:03.582 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:03.578+0000 7f5619e97640 1 --2- 172.21.15.49:0/1651307945 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5614068ea0 0x7f5614198070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:03.582 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:03.578+0000 7f5619e97640 1 -- 172.21.15.49:0/1651307945 >> 172.21.15.49:0/1651307945 conn(0x7f5614077570 msgr2=0x7f56140fea20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:10:03.582 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:03.579+0000 7f5619e97640 1 -- 172.21.15.49:0/1651307945 shutdown_connections 2024-03-18T23:10:03.582 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:03.579+0000 7f5619e97640 1 -- 172.21.15.49:0/1651307945 wait complete. 2024-03-18T23:10:03.761 INFO:tasks.workunit.client.0.smithi049.stdout: 1 156322 13.26 MB/sec warmup 65 sec latency 64.398 ms 2024-03-18T23:10:04.761 INFO:tasks.workunit.client.0.smithi049.stdout: 1 158652 13.19 MB/sec warmup 66 sec latency 28.710 ms 2024-03-18T23:10:05.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:10:05 smithi139 ceph-mon[67282]: pgmap v57: 97 pgs: 97 active+clean; 57 MiB data, 588 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.2 MiB/s wr, 206 op/s 2024-03-18T23:10:05.379 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:10:05 smithi049 ceph-mon[62991]: pgmap v57: 97 pgs: 97 active+clean; 57 MiB data, 588 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.2 MiB/s wr, 206 op/s 2024-03-18T23:10:05.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:10:05 smithi175 ceph-mon[67352]: pgmap v57: 97 pgs: 97 active+clean; 57 MiB data, 588 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.2 MiB/s wr, 206 op/s 2024-03-18T23:10:05.761 INFO:tasks.workunit.client.0.smithi049.stdout: 1 160337 13.16 MB/sec warmup 67 sec latency 62.600 ms 2024-03-18T23:10:06.761 INFO:tasks.workunit.client.0.smithi049.stdout: 1 163152 13.22 MB/sec warmup 68 sec latency 101.896 ms 2024-03-18T23:10:07.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:10:07 smithi139 ceph-mon[67282]: pgmap v58: 97 pgs: 97 active+clean; 57 MiB data, 588 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.2 MiB/s wr, 206 op/s 2024-03-18T23:10:07.379 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:10:07 smithi049 ceph-mon[62991]: pgmap v58: 97 pgs: 97 active+clean; 57 MiB data, 588 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.2 MiB/s wr, 206 op/s 2024-03-18T23:10:07.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:10:07 smithi175 ceph-mon[67352]: pgmap v58: 97 pgs: 97 active+clean; 57 MiB data, 588 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.2 MiB/s wr, 206 op/s 2024-03-18T23:10:07.761 INFO:tasks.workunit.client.0.smithi049.stdout: 1 164878 13.13 MB/sec warmup 69 sec latency 15.291 ms 2024-03-18T23:10:08.761 INFO:tasks.workunit.client.0.smithi049.stdout: 1 167279 13.14 MB/sec warmup 70 sec latency 14.706 ms 2024-03-18T23:10:09.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:10:09 smithi139 ceph-mon[67282]: pgmap v59: 97 pgs: 97 active+clean; 50 MiB data, 615 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 3.6 MiB/s wr, 428 op/s 2024-03-18T23:10:09.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:10:09 smithi049 ceph-mon[62991]: pgmap v59: 97 pgs: 97 active+clean; 50 MiB data, 615 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 3.6 MiB/s wr, 428 op/s 2024-03-18T23:10:09.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:10:09 smithi175 ceph-mon[67352]: pgmap v59: 97 pgs: 97 active+clean; 50 MiB data, 615 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 3.6 MiB/s wr, 428 op/s 2024-03-18T23:10:09.761 INFO:tasks.workunit.client.0.smithi049.stdout: 1 170105 13.16 MB/sec warmup 71 sec latency 24.043 ms 2024-03-18T23:10:10.761 INFO:tasks.workunit.client.0.smithi049.stdout: 1 172099 13.11 MB/sec warmup 72 sec latency 16.549 ms 2024-03-18T23:10:11.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:10:11 smithi139 ceph-mon[67282]: pgmap v60: 97 pgs: 97 active+clean; 50 MiB data, 615 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.0 MiB/s wr, 342 op/s 2024-03-18T23:10:11.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:10:11 smithi049 ceph-mon[62991]: pgmap v60: 97 pgs: 97 active+clean; 50 MiB data, 615 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.0 MiB/s wr, 342 op/s 2024-03-18T23:10:11.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:10:11 smithi175 ceph-mon[67352]: pgmap v60: 97 pgs: 97 active+clean; 50 MiB data, 615 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.0 MiB/s wr, 342 op/s 2024-03-18T23:10:11.761 INFO:tasks.workunit.client.0.smithi049.stdout: 1 174160 13.12 MB/sec warmup 73 sec latency 122.746 ms 2024-03-18T23:10:12.761 INFO:tasks.workunit.client.0.smithi049.stdout: 1 176699 13.10 MB/sec warmup 74 sec latency 17.449 ms 2024-03-18T23:10:13.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:10:13 smithi139 ceph-mon[67282]: pgmap v61: 97 pgs: 97 active+clean; 58 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 4.6 MiB/s wr, 450 op/s 2024-03-18T23:10:13.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:10:13 smithi049 ceph-mon[62991]: pgmap v61: 97 pgs: 97 active+clean; 58 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 4.6 MiB/s wr, 450 op/s 2024-03-18T23:10:13.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:10:13 smithi175 ceph-mon[67352]: pgmap v61: 97 pgs: 97 active+clean; 58 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 4.6 MiB/s wr, 450 op/s 2024-03-18T23:10:13.761 INFO:tasks.workunit.client.0.smithi049.stdout: 1 178437 13.05 MB/sec warmup 75 sec latency 18.526 ms 2024-03-18T23:10:14.761 INFO:tasks.workunit.client.0.smithi049.stdout: 1 181213 13.10 MB/sec warmup 76 sec latency 10.352 ms 2024-03-18T23:10:15.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:10:15 smithi139 ceph-mon[67282]: pgmap v62: 97 pgs: 97 active+clean; 58 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.1 MiB/s wr, 330 op/s 2024-03-18T23:10:15.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:10:15 smithi049 ceph-mon[62991]: pgmap v62: 97 pgs: 97 active+clean; 58 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.1 MiB/s wr, 330 op/s 2024-03-18T23:10:15.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:10:15 smithi175 ceph-mon[67352]: pgmap v62: 97 pgs: 97 active+clean; 58 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.1 MiB/s wr, 330 op/s 2024-03-18T23:10:15.762 INFO:tasks.workunit.client.0.smithi049.stdout: 1 183470 13.04 MB/sec warmup 77 sec latency 47.489 ms 2024-03-18T23:10:16.689 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:16.687+0000 7f087a8e8640 1 -- 172.21.15.49:0/1287276358 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0874068ea0 msgr2=0x7f0874069300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:10:16.689 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:16.687+0000 7f087a8e8640 1 --2- 172.21.15.49:0/1287276358 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0874068ea0 0x7f0874069300 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f0868009860 tx=0x7f086802bb70 comp rx=0 tx=0).stop 2024-03-18T23:10:16.689 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:16.688+0000 7f087a8e8640 1 -- 172.21.15.49:0/1287276358 shutdown_connections 2024-03-18T23:10:16.689 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:16.688+0000 7f087a8e8640 1 --2- 172.21.15.49:0/1287276358 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0874069840 0x7f087410e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:16.689 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:16.688+0000 7f087a8e8640 1 --2- 172.21.15.49:0/1287276358 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0874068ea0 0x7f0874069300 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:16.689 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:16.688+0000 7f087a8e8640 1 --2- 172.21.15.49:0/1287276358 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0874107890 0x7f0874107c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:16.689 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:16.688+0000 7f087a8e8640 1 -- 172.21.15.49:0/1287276358 >> 172.21.15.49:0/1287276358 conn(0x7f0874077570 msgr2=0x7f0874077980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:10:16.690 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:16.688+0000 7f087a8e8640 1 -- 172.21.15.49:0/1287276358 shutdown_connections 2024-03-18T23:10:16.690 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:16.688+0000 7f087a8e8640 1 -- 172.21.15.49:0/1287276358 wait complete. 2024-03-18T23:10:16.692 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:16.689+0000 7f087a8e8640 1 Processor -- start 2024-03-18T23:10:16.692 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:16.689+0000 7f087a8e8640 1 -- start start 2024-03-18T23:10:16.692 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:16.689+0000 7f087a8e8640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0874068ea0 0x7f0874100a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:10:16.692 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:16.689+0000 7f087a8e8640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0874069840 0x7f0874100f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:10:16.692 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:16.690+0000 7f087a8e8640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0874107890 0x7f08741014c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:10:16.692 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:16.690+0000 7f087a8e8640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f0874111bc0 con 0x7f0874068ea0 2024-03-18T23:10:16.692 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:16.690+0000 7f087a8e8640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f0874111a40 con 0x7f0874069840 2024-03-18T23:10:16.692 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:16.690+0000 7f087a8e8640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f0874111d40 con 0x7f0874107890 2024-03-18T23:10:16.692 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:16.690+0000 7f0873fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0874068ea0 0x7f0874100a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:10:16.693 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:16.690+0000 7f0878e85640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0874107890 0x7f08741014c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:10:16.693 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:16.690+0000 7f08737fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0874069840 0x7f0874100f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:10:16.693 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:16.690+0000 7f0873fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0874068ea0 0x7f0874100a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:44364/0 (socket says 172.21.15.49:44364) 2024-03-18T23:10:16.693 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:16.690+0000 7f0873fff640 1 -- 172.21.15.49:0/4235104185 learned_addr learned my addr 172.21.15.49:0/4235104185 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:10:16.693 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:16.690+0000 7f0878e85640 1 -- 172.21.15.49:0/4235104185 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0874069840 msgr2=0x7f0874100f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:10:16.693 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:16.690+0000 7f0878e85640 1 --2- 172.21.15.49:0/4235104185 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0874069840 0x7f0874100f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:16.693 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:16.690+0000 7f0878e85640 1 -- 172.21.15.49:0/4235104185 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0874068ea0 msgr2=0x7f0874100a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:10:16.693 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:16.690+0000 7f0878e85640 1 --2- 172.21.15.49:0/4235104185 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0874068ea0 0x7f0874100a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:16.693 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:16.690+0000 7f0878e85640 1 -- 172.21.15.49:0/4235104185 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08741a64a0 con 0x7f0874107890 2024-03-18T23:10:16.693 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:16.690+0000 7f0873fff640 1 --2- 172.21.15.49:0/4235104185 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0874068ea0 0x7f0874100a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:10:16.693 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:16.690+0000 7f0878e85640 1 --2- 172.21.15.49:0/4235104185 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0874107890 0x7f08741014c0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f086c005fd0 tx=0x7f086c00ee60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:10:16.693 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:16.691+0000 7f08717fa640 1 -- 172.21.15.49:0/4235104185 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f086c00cd80 con 0x7f0874107890 2024-03-18T23:10:16.693 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:16.691+0000 7f08717fa640 1 -- 172.21.15.49:0/4235104185 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f086c005020 con 0x7f0874107890 2024-03-18T23:10:16.693 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:16.691+0000 7f087a8e8640 1 -- 172.21.15.49:0/4235104185 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08741a6790 con 0x7f0874107890 2024-03-18T23:10:16.694 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:16.691+0000 7f08717fa640 1 -- 172.21.15.49:0/4235104185 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f086c010720 con 0x7f0874107890 2024-03-18T23:10:16.694 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:16.691+0000 7f087a8e8640 1 -- 172.21.15.49:0/4235104185 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08741a6c60 con 0x7f0874107890 2024-03-18T23:10:16.695 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:16.694+0000 7f087d5ac740 1 Processor -- start 2024-03-18T23:10:16.695 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:16.694+0000 7f08717fa640 1 -- 172.21.15.49:0/4235104185 <== mon.1 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 25) ==== 137421+0+0 (secure 0 0 0) 0x7f086c0051d0 con 0x7f0874107890 2024-03-18T23:10:16.695 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:16.695+0000 7f087d5ac740 1 -- start start 2024-03-18T23:10:16.696 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:16.695+0000 7f08717fa640 1 --2- 172.21.15.49:0/4235104185 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f085c0a2a90 0x7f085c0a4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:10:16.697 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:16.695+0000 7f08717fa640 1 -- 172.21.15.49:0/4235104185 <== mon.1 v2:172.21.15.175:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f086c0cf450 con 0x7f0874107890 2024-03-18T23:10:16.697 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:16.695+0000 7f08737fe640 1 --2- 172.21.15.49:0/4235104185 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f085c0a2a90 0x7f085c0a4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:10:16.698 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:16.695+0000 7f087d5ac740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f09b4fcef0 0x55f09b4fd2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:10:16.698 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:16.696+0000 7f087d5ac740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f09b4fd810 0x55f09b5060a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:10:16.698 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:16.696+0000 7f087d5ac740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f09b5065e0 0x55f09b5089d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:10:16.698 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:16.696+0000 7f087d5ac740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55f09b460be0 con 0x55f09b5065e0 2024-03-18T23:10:16.698 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:16.696+0000 7f087d5ac740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55f09b409960 con 0x55f09b4fd810 2024-03-18T23:10:16.698 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:16.696+0000 7f087d5ac740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55f09b45cc80 con 0x55f09b4fcef0 2024-03-18T23:10:16.698 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:16.696+0000 7f0873fff640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f09b4fcef0 0x55f09b4fd2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:10:16.698 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:16.696+0000 7f08737fe640 1 --2- 172.21.15.49:0/4235104185 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f085c0a2a90 0x7f085c0a4f50 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f086802c180 tx=0x7f086802c090 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:10:16.698 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:16.696+0000 7f0873fff640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f09b4fcef0 0x55f09b4fd2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.49:48818/0 (socket says 172.21.15.49:48818) 2024-03-18T23:10:16.698 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:16.696+0000 7f0873fff640 1 -- 172.21.15.49:0/2401644580 learned_addr learned my addr 172.21.15.49:0/2401644580 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:10:16.698 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:16.696+0000 7f0878e85640 1 --2- 172.21.15.49:0/2401644580 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f09b5065e0 0x55f09b5089d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:10:16.698 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:16.696+0000 7f08737fe640 1 --2- 172.21.15.49:0/2401644580 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f09b4fd810 0x55f09b5060a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:10:16.699 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:16.696+0000 7f0873fff640 1 -- 172.21.15.49:0/2401644580 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f09b4fd810 msgr2=0x55f09b5060a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:10:16.699 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:16.696+0000 7f0873fff640 1 --2- 172.21.15.49:0/2401644580 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f09b4fd810 0x55f09b5060a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:16.699 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:16.696+0000 7f0873fff640 1 -- 172.21.15.49:0/2401644580 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f09b5065e0 msgr2=0x55f09b5089d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:10:16.699 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:16.696+0000 7f0873fff640 1 --2- 172.21.15.49:0/2401644580 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f09b5065e0 0x55f09b5089d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:16.699 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:16.696+0000 7f0873fff640 1 -- 172.21.15.49:0/2401644580 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f09b400d20 con 0x55f09b4fcef0 2024-03-18T23:10:16.699 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:16.697+0000 7f0873fff640 1 --2- 172.21.15.49:0/2401644580 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f09b4fcef0 0x55f09b4fd2d0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f0864011540 tx=0x7f0864011a00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=72f818e430759c14 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:10:16.703 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:16.697+0000 7f085a7fc640 1 -- 172.21.15.49:0/2401644580 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f086400f640 con 0x55f09b4fcef0 2024-03-18T23:10:16.703 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:16.697+0000 7f085a7fc640 1 -- 172.21.15.49:0/2401644580 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f086400fcc0 con 0x55f09b4fcef0 2024-03-18T23:10:16.703 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:16.697+0000 7f085a7fc640 1 -- 172.21.15.49:0/2401644580 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08640183b0 con 0x55f09b4fcef0 2024-03-18T23:10:16.703 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:16.698+0000 7f087d5ac740 1 -- 172.21.15.49:0/2401644580 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f09b4fcef0 msgr2=0x55f09b4fd2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:10:16.703 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:16.698+0000 7f087d5ac740 1 --2- 172.21.15.49:0/2401644580 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f09b4fcef0 0x55f09b4fd2d0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f0864011540 tx=0x7f0864011a00 comp rx=0 tx=0).stop 2024-03-18T23:10:16.703 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:16.698+0000 7f087d5ac740 1 -- 172.21.15.49:0/2401644580 shutdown_connections 2024-03-18T23:10:16.704 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:16.698+0000 7f087d5ac740 1 --2- 172.21.15.49:0/2401644580 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f09b5065e0 0x55f09b5089d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:16.704 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:16.698+0000 7f087d5ac740 1 --2- 172.21.15.49:0/2401644580 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f09b4fd810 0x55f09b5060a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:16.704 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:16.698+0000 7f087d5ac740 1 --2- 172.21.15.49:0/2401644580 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f09b4fcef0 0x55f09b4fd2d0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:16.704 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:16.698+0000 7f087d5ac740 1 -- 172.21.15.49:0/2401644580 >> 172.21.15.49:0/2401644580 conn(0x55f09b4f7810 msgr2=0x55f09b4f7bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:10:16.704 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:16.698+0000 7f087d5ac740 1 -- 172.21.15.49:0/2401644580 shutdown_connections 2024-03-18T23:10:16.704 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:16.698+0000 7f087d5ac740 1 -- 172.21.15.49:0/2401644580 wait complete. 2024-03-18T23:10:16.704 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:16.699+0000 7f087d5ac740 1 -- start start 2024-03-18T23:10:16.704 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:16.699+0000 7f087d5ac740 1 Processor -- start 2024-03-18T23:10:16.704 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:16.699+0000 7f087d5ac740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f09b4fcef0 0x55f09b506000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:10:16.704 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:16.699+0000 7f087d5ac740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f09b4fd810 0x55f09b503390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:10:16.704 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:16.699+0000 7f087d5ac740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f09b5065e0 0x55f09b5039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:10:16.704 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:16.699+0000 7f087d5ac740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55f09b509210 con 0x55f09b4fd810 2024-03-18T23:10:16.704 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:16.699+0000 7f087d5ac740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55f09b41cdb0 con 0x55f09b5065e0 2024-03-18T23:10:16.705 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:16.699+0000 7f0873fff640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f09b4fcef0 0x55f09b506000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:10:16.705 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:16.699+0000 7f087d5ac740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55f09b509390 con 0x55f09b4fcef0 2024-03-18T23:10:16.705 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:16.699+0000 7f0873fff640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f09b4fcef0 0x55f09b506000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.49:48832/0 (socket says 172.21.15.49:48832) 2024-03-18T23:10:16.705 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:16.700+0000 7f0873fff640 1 -- 172.21.15.49:0/1147202354 learned_addr learned my addr 172.21.15.49:0/1147202354 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:10:16.705 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:16.699+0000 7f08737fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f09b4fd810 0x55f09b503390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:10:16.705 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:16.700+0000 7f087d5ac740 10 client.0 _ll_register_callbacks cb 0x55f09b421e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T23:10:16.705 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:16.700+0000 7f0878e85640 1 --2- 172.21.15.49:0/1147202354 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f09b5065e0 0x55f09b5039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:10:16.705 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:16.700+0000 7f0873fff640 1 -- 172.21.15.49:0/1147202354 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f09b5065e0 msgr2=0x55f09b5039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:10:16.705 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:16.700+0000 7f0873fff640 1 --2- 172.21.15.49:0/1147202354 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f09b5065e0 0x55f09b5039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:16.705 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:16.700+0000 7f0873fff640 1 -- 172.21.15.49:0/1147202354 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f09b4fd810 msgr2=0x55f09b503390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:10:16.705 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:16.700+0000 7f0873fff640 1 --2- 172.21.15.49:0/1147202354 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f09b4fd810 0x55f09b503390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:16.705 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:16.700+0000 7f0873fff640 1 -- 172.21.15.49:0/1147202354 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f09b41bc60 con 0x55f09b4fcef0 2024-03-18T23:10:16.705 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:16.700+0000 7f08737fe640 1 --2- 172.21.15.49:0/1147202354 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f09b4fd810 0x55f09b503390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:10:16.705 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:16.700+0000 7f0873fff640 1 --2- 172.21.15.49:0/1147202354 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f09b4fcef0 0x55f09b506000 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f086401b040 tx=0x7f0864019640 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e45b8c3cdb404f3 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:10:16.706 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:16.700+0000 7f08537fe640 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-03-18T23:10:16.706 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:16.700+0000 7f08537fe640 1 -- 172.21.15.49:0/1147202354 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0864027020 con 0x55f09b4fcef0 2024-03-18T23:10:16.706 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:16.700+0000 7f08537fe640 1 -- 172.21.15.49:0/1147202354 <== mon.1 v2:172.21.15.175:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f0864004040 con 0x55f09b4fcef0 2024-03-18T23:10:16.706 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:16.700+0000 7f087d5ac740 1 -- 172.21.15.49:0/1147202354 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f086400fcc0 con 0x55f09b4fcef0 2024-03-18T23:10:16.706 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:16.701+0000 7f08537fe640 1 -- 172.21.15.49:0/1147202354 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f0864032050 con 0x55f09b4fcef0 2024-03-18T23:10:16.706 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:16.701+0000 7f087d5ac740 10 client.25423 fetch_fsmap learned FSMap version 29 2024-03-18T23:10:16.706 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:16.701+0000 7f087d5ac740 1 -- 172.21.15.49:0/1147202354 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55f09b175210 con 0x55f09b4fcef0 2024-03-18T23:10:16.706 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:16.701+0000 7f08537fe640 1 -- 172.21.15.49:0/1147202354 <== mon.1 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f086400fcc0 con 0x55f09b4fcef0 2024-03-18T23:10:16.706 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:16.701+0000 7f08537fe640 1 -- 172.21.15.49:0/1147202354 <== mon.1 v2:172.21.15.175:3300/0 5 ==== fsmap(e 29) ==== 8358+0+0 (secure 0 0 0) 0x7f0864005240 con 0x55f09b4fcef0 2024-03-18T23:10:16.706 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:16.701+0000 7f087d5ac740 10 client.25423 fetch_fsmap finished waiting for FSMap version 29 2024-03-18T23:10:16.706 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:16.701+0000 7f087d5ac740 10 client.25423 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T23:10:16.706 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:16.701+0000 7f087d5ac740 20 client.25423 populate_metadata read hostname 'smithi049' 2024-03-18T23:10:16.706 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:16.701+0000 7f087d5ac740 1 --2- 172.21.15.49:0/1147202354 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x55f09b516440 0x55f09b518830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:10:16.707 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:16.701+0000 7f087d5ac740 4 client.25423 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T23:10:16.707 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:16.701+0000 7f087d5ac740 1 -- 172.21.15.49:0/1147202354 --> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f09b46e120 con 0x55f09b516440 2024-03-18T23:10:16.707 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:16.701+0000 7f08737fe640 1 --2- 172.21.15.49:0/1147202354 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x55f09b516440 0x55f09b518830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:10:16.707 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:16.702+0000 7f08737fe640 1 --2- 172.21.15.49:0/1147202354 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x55f09b516440 0x55f09b518830 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e73692ea42f7f4f1 server_cookie=4ce7ccc658cc1a5d in_seq=0 out_seq=0 2024-03-18T23:10:16.707 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:16.702+0000 7f08537fe640 10 client.25423 ms_handle_connect on v2:172.21.15.139:6838/4280128209 2024-03-18T23:10:16.708 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:16.705+0000 7f08537fe640 1 -- 172.21.15.49:0/1147202354 <== mds.0 v2:172.21.15.139:6838/4280128209 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55f09b46e120 con 0x55f09b516440 2024-03-18T23:10:16.708 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:16.705+0000 7f08537fe640 10 client.25423 handle_command_reply: tid=0 2024-03-18T23:10:16.708 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:16.705+0000 7f087d5ac740 1 client.25423 shutdown 2024-03-18T23:10:16.708 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:16.706+0000 7f087d5ac740 1 -- 172.21.15.49:0/1147202354 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f09b4fcef0 msgr2=0x55f09b506000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:10:16.708 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:16.706+0000 7f087d5ac740 1 --2- 172.21.15.49:0/1147202354 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f09b4fcef0 0x55f09b506000 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f086401b040 tx=0x7f0864019640 comp rx=0 tx=0).stop 2024-03-18T23:10:16.708 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:16.706+0000 7f087d5ac740 1 -- 172.21.15.49:0/1147202354 shutdown_connections 2024-03-18T23:10:16.708 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:16.706+0000 7f087d5ac740 1 --2- 172.21.15.49:0/1147202354 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x55f09b516440 0x55f09b518830 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:16.709 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:16.706+0000 7f087d5ac740 1 --2- 172.21.15.49:0/1147202354 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f09b5065e0 0x55f09b5039a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:16.709 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:16.706+0000 7f08537fe640 0 client.25423 ms_handle_reset on v2:172.21.15.139:6838/4280128209 2024-03-18T23:10:16.709 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:16.706+0000 7f087d5ac740 1 --2- 172.21.15.49:0/1147202354 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f09b4fd810 0x55f09b503390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:16.709 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:16.706+0000 7f087d5ac740 1 --2- 172.21.15.49:0/1147202354 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f09b4fcef0 0x55f09b506000 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:16.709 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:16.706+0000 7f087d5ac740 1 -- 172.21.15.49:0/1147202354 >> 172.21.15.49:0/1147202354 conn(0x55f09b4f7810 msgr2=0x55f09b4f6ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:10:16.709 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:16.706+0000 7f087d5ac740 1 -- 172.21.15.49:0/1147202354 shutdown_connections 2024-03-18T23:10:16.709 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:16.706+0000 7f087d5ac740 1 -- 172.21.15.49:0/1147202354 wait complete. 2024-03-18T23:10:16.709 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:16.706+0000 7f087d5ac740 20 client.25423 trim_cache size 0 max 16384 2024-03-18T23:10:16.723 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:16.721+0000 7f087d5ac740 1 Processor -- start 2024-03-18T23:10:16.723 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:16.721+0000 7f087d5ac740 1 -- start start 2024-03-18T23:10:16.724 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:16.721+0000 7f087d5ac740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f09b4fcef0 0x55f09b511fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:10:16.724 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:16.721+0000 7f087d5ac740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f09b4fd810 0x55f09b4f2940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:10:16.724 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:16.721+0000 7f087d5ac740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f09b5065e0 0x55f09b51df20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:10:16.724 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:16.721+0000 7f087d5ac740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f08741af680 con 0x55f09b4fcef0 2024-03-18T23:10:16.724 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:16.721+0000 7f087d5ac740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f08741b11f0 con 0x55f09b5065e0 2024-03-18T23:10:16.724 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:16.721+0000 7f087d5ac740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f08741afd50 con 0x55f09b4fd810 2024-03-18T23:10:16.724 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:16.721+0000 7f0873fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f09b4fcef0 0x55f09b511fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:10:16.724 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:16.721+0000 7f0873fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f09b4fcef0 0x55f09b511fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:44378/0 (socket says 172.21.15.49:44378) 2024-03-18T23:10:16.724 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:16.721+0000 7f0873fff640 1 -- 172.21.15.49:0/2538362952 learned_addr learned my addr 172.21.15.49:0/2538362952 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:10:16.724 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:16.721+0000 7f0878e85640 1 --2- 172.21.15.49:0/2538362952 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f09b5065e0 0x55f09b51df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:10:16.724 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:16.721+0000 7f08737fe640 1 --2- 172.21.15.49:0/2538362952 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f09b4fd810 0x55f09b4f2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:10:16.724 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:16.721+0000 7f0873fff640 1 -- 172.21.15.49:0/2538362952 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f09b4fd810 msgr2=0x55f09b4f2940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:10:16.724 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:16.721+0000 7f0873fff640 1 --2- 172.21.15.49:0/2538362952 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f09b4fd810 0x55f09b4f2940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:16.724 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:16.721+0000 7f0873fff640 1 -- 172.21.15.49:0/2538362952 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f09b5065e0 msgr2=0x55f09b51df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:10:16.725 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:16.721+0000 7f0873fff640 1 --2- 172.21.15.49:0/2538362952 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f09b5065e0 0x55f09b51df20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:16.725 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:16.721+0000 7f0873fff640 1 -- 172.21.15.49:0/2538362952 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f09b241620 con 0x55f09b4fcef0 2024-03-18T23:10:16.725 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:16.721+0000 7f0873fff640 1 --2- 172.21.15.49:0/2538362952 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f09b4fcef0 0x55f09b511fb0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f08640055e0 tx=0x7f0864011a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1231d6dac3953a2 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:10:16.725 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:16.722+0000 7f0859ffb640 1 -- 172.21.15.49:0/2538362952 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0864027040 con 0x55f09b4fcef0 2024-03-18T23:10:16.725 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:16.722+0000 7f0859ffb640 1 -- 172.21.15.49:0/2538362952 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0864016020 con 0x55f09b4fcef0 2024-03-18T23:10:16.725 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:16.722+0000 7f0859ffb640 1 -- 172.21.15.49:0/2538362952 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f086400fb80 con 0x55f09b4fcef0 2024-03-18T23:10:16.725 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:16.722+0000 7f087d5ac740 1 -- 172.21.15.49:0/2538362952 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f09b4fcef0 msgr2=0x55f09b511fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:10:16.725 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:16.722+0000 7f087d5ac740 1 --2- 172.21.15.49:0/2538362952 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f09b4fcef0 0x55f09b511fb0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f08640055e0 tx=0x7f0864011a00 comp rx=0 tx=0).stop 2024-03-18T23:10:16.725 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:16.722+0000 7f087d5ac740 1 -- 172.21.15.49:0/2538362952 shutdown_connections 2024-03-18T23:10:16.725 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:16.722+0000 7f087d5ac740 1 --2- 172.21.15.49:0/2538362952 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f09b5065e0 0x55f09b51df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:16.725 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:16.722+0000 7f087d5ac740 1 --2- 172.21.15.49:0/2538362952 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f09b4fd810 0x55f09b4f2940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:16.725 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:16.722+0000 7f087d5ac740 1 --2- 172.21.15.49:0/2538362952 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f09b4fcef0 0x55f09b511fb0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:16.725 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:16.722+0000 7f087d5ac740 1 -- 172.21.15.49:0/2538362952 >> 172.21.15.49:0/2538362952 conn(0x55f09b4f7810 msgr2=0x55f09b4f6ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:10:16.726 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:16.722+0000 7f087d5ac740 1 -- 172.21.15.49:0/2538362952 shutdown_connections 2024-03-18T23:10:16.726 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:16.722+0000 7f087d5ac740 1 -- 172.21.15.49:0/2538362952 wait complete. 2024-03-18T23:10:16.726 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:16.723+0000 7f087d5ac740 1 -- start start 2024-03-18T23:10:16.726 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:16.723+0000 7f087d5ac740 1 Processor -- start 2024-03-18T23:10:16.728 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:16.723+0000 7f087d5ac740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f09b4fcef0 0x55f09b50e810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:10:16.729 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:16.723+0000 7f087d5ac740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f09b4fd810 0x55f09b509b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:10:16.729 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:16.723+0000 7f087d5ac740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f09b5065e0 0x55f09b50a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:10:16.729 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:16.723+0000 7f087d5ac740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f08741b0af0 con 0x55f09b4fd810 2024-03-18T23:10:16.729 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:16.723+0000 7f087d5ac740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f08741b0420 con 0x55f09b4fcef0 2024-03-18T23:10:16.729 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:16.723+0000 7f087d5ac740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f08741ad660 con 0x55f09b5065e0 2024-03-18T23:10:16.729 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:16.723+0000 7f087d5ac740 10 client.0 _ll_register_callbacks cb 0x7f086c097d80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T23:10:16.729 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:16.723+0000 7f08737fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f09b4fd810 0x55f09b509b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:10:16.729 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:16.723+0000 7f08737fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f09b4fd810 0x55f09b509b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:44390/0 (socket says 172.21.15.49:44390) 2024-03-18T23:10:16.729 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:16.723+0000 7f0873fff640 1 --2- 172.21.15.49:0/4042073446 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f09b4fcef0 0x55f09b50e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:10:16.729 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:16.723+0000 7f08737fe640 1 -- 172.21.15.49:0/4042073446 learned_addr learned my addr 172.21.15.49:0/4042073446 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:10:16.729 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:16.723+0000 7f0878e85640 1 --2- 172.21.15.49:0/4042073446 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f09b5065e0 0x55f09b50a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:10:16.729 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:16.723+0000 7f08737fe640 1 -- 172.21.15.49:0/4042073446 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f09b5065e0 msgr2=0x55f09b50a0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:10:16.729 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:16.723+0000 7f08737fe640 1 --2- 172.21.15.49:0/4042073446 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f09b5065e0 0x55f09b50a0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:16.729 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:16.723+0000 7f08737fe640 1 -- 172.21.15.49:0/4042073446 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f09b4fcef0 msgr2=0x55f09b50e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:10:16.730 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:16.723+0000 7f08737fe640 1 --2- 172.21.15.49:0/4042073446 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f09b4fcef0 0x55f09b50e810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:16.730 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:16.724+0000 7f08737fe640 1 -- 172.21.15.49:0/4042073446 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f09b50a8c0 con 0x55f09b4fd810 2024-03-18T23:10:16.730 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:16.724+0000 7f08737fe640 1 --2- 172.21.15.49:0/4042073446 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f09b4fd810 0x55f09b509b80 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f0868004150 tx=0x7f0868005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53a8dee018de6929 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:10:16.730 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:16.724+0000 7f08537fe640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-18T23:10:16.730 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:16.724+0000 7f08537fe640 1 -- 172.21.15.49:0/4042073446 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0868039040 con 0x55f09b4fd810 2024-03-18T23:10:16.730 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:16.724+0000 7f08537fe640 1 -- 172.21.15.49:0/4042073446 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f0868055590 con 0x55f09b4fd810 2024-03-18T23:10:16.730 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:16.724+0000 7f087d5ac740 1 -- 172.21.15.49:0/4042073446 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0864016020 con 0x55f09b4fd810 2024-03-18T23:10:16.730 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:16.724+0000 7f08537fe640 1 -- 172.21.15.49:0/4042073446 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f0868065050 con 0x55f09b4fd810 2024-03-18T23:10:16.730 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:16.724+0000 7f087d5ac740 10 client.15621 fetch_fsmap learned FSMap version 29 2024-03-18T23:10:16.730 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:16.724+0000 7f087d5ac740 1 -- 172.21.15.49:0/4042073446 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55f09b57c550 con 0x55f09b4fd810 2024-03-18T23:10:16.730 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:16.724+0000 7f08537fe640 1 -- 172.21.15.49:0/4042073446 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0864016020 con 0x55f09b4fd810 2024-03-18T23:10:16.730 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:16.724+0000 7f08537fe640 1 -- 172.21.15.49:0/4042073446 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 29) ==== 8358+0+0 (secure 0 0 0) 0x7f0868055070 con 0x55f09b4fd810 2024-03-18T23:10:16.731 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:16.724+0000 7f087d5ac740 10 client.15621 fetch_fsmap finished waiting for FSMap version 29 2024-03-18T23:10:16.731 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:16.724+0000 7f087d5ac740 10 client.15621 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T23:10:16.731 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:16.725+0000 7f087d5ac740 20 client.15621 populate_metadata read hostname 'smithi049' 2024-03-18T23:10:16.731 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:16.725+0000 7f087d5ac740 1 --2- 172.21.15.49:0/4042073446 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x55f09b57ccb0 0x55f09b57f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:10:16.731 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:16.725+0000 7f087d5ac740 4 client.15621 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T23:10:16.731 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:16.725+0000 7f087d5ac740 1 -- 172.21.15.49:0/4042073446 --> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f09b468470 con 0x55f09b57ccb0 2024-03-18T23:10:16.731 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:16.725+0000 7f0873fff640 1 --2- 172.21.15.49:0/4042073446 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x55f09b57ccb0 0x55f09b57f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:10:16.731 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:16.725+0000 7f08537fe640 10 client.15621 ms_handle_connect on v2:172.21.15.139:6838/4280128209 2024-03-18T23:10:16.731 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:16.725+0000 7f0873fff640 1 --2- 172.21.15.49:0/4042073446 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x55f09b57ccb0 0x55f09b57f0d0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1a7256892755bd07 server_cookie=fb8300c43d469a6d in_seq=0 out_seq=0 2024-03-18T23:10:16.761 INFO:tasks.workunit.client.0.smithi049.stdout: 1 185025 13.02 MB/sec warmup 78 sec latency 70.481 ms 2024-03-18T23:10:16.992 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:16.990+0000 7f08537fe640 1 -- 172.21.15.49:0/4042073446 <== mds.0 v2:172.21.15.139:6838/4280128209 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f09b468470 con 0x55f09b57ccb0 2024-03-18T23:10:16.992 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:16.990+0000 7f08537fe640 10 client.15621 handle_command_reply: tid=0 2024-03-18T23:10:16.992 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:16.990+0000 7f087d5ac740 1 client.15621 shutdown 2024-03-18T23:10:16.992 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:16.990+0000 7f087d5ac740 1 -- 172.21.15.49:0/4042073446 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f09b4fd810 msgr2=0x55f09b509b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:10:16.992 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:16.990+0000 7f087d5ac740 1 --2- 172.21.15.49:0/4042073446 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f09b4fd810 0x55f09b509b80 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f0868004150 tx=0x7f0868005f90 comp rx=0 tx=0).stop 2024-03-18T23:10:16.992 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:16.990+0000 7f087d5ac740 1 -- 172.21.15.49:0/4042073446 shutdown_connections 2024-03-18T23:10:16.992 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:16.990+0000 7f087d5ac740 1 --2- 172.21.15.49:0/4042073446 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x55f09b57ccb0 0x55f09b57f0d0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:16.992 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:16.990+0000 7f087d5ac740 1 --2- 172.21.15.49:0/4042073446 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f09b5065e0 0x55f09b50a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:16.992 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:16.990+0000 7f087d5ac740 1 --2- 172.21.15.49:0/4042073446 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f09b4fd810 0x55f09b509b80 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:16.992 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:16.990+0000 7f087d5ac740 1 --2- 172.21.15.49:0/4042073446 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f09b4fcef0 0x55f09b50e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:16.992 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:16.990+0000 7f087d5ac740 1 -- 172.21.15.49:0/4042073446 >> 172.21.15.49:0/4042073446 conn(0x55f09b4f7810 msgr2=0x55f09b4f6ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:10:16.992 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:16.990+0000 7f08537fe640 0 client.15621 ms_handle_reset on v2:172.21.15.139:6838/4280128209 2024-03-18T23:10:16.993 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:16.991+0000 7f087d5ac740 1 -- 172.21.15.49:0/4042073446 shutdown_connections 2024-03-18T23:10:16.993 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:16.991+0000 7f087d5ac740 1 -- 172.21.15.49:0/4042073446 wait complete. 2024-03-18T23:10:16.993 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:16.991+0000 7f087d5ac740 20 client.15621 trim_cache size 0 max 16384 2024-03-18T23:10:16.993 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:{ 2024-03-18T23:10:16.993 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "message": "", 2024-03-18T23:10:16.993 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "return_code": 0 2024-03-18T23:10:16.993 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:} 2024-03-18T23:10:16.994 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:16.991+0000 7f087a8e8640 1 -- 172.21.15.49:0/4235104185 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f085c0a2a90 msgr2=0x7f085c0a4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:10:16.994 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:16.991+0000 7f087a8e8640 1 --2- 172.21.15.49:0/4235104185 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f085c0a2a90 0x7f085c0a4f50 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f086802c180 tx=0x7f086802c090 comp rx=0 tx=0).stop 2024-03-18T23:10:16.994 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:16.991+0000 7f087a8e8640 1 -- 172.21.15.49:0/4235104185 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0874107890 msgr2=0x7f08741014c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:10:16.994 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:16.991+0000 7f087a8e8640 1 --2- 172.21.15.49:0/4235104185 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0874107890 0x7f08741014c0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f086c005fd0 tx=0x7f086c00ee60 comp rx=0 tx=0).stop 2024-03-18T23:10:16.994 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:16.992+0000 7f087a8e8640 1 -- 172.21.15.49:0/4235104185 shutdown_connections 2024-03-18T23:10:16.994 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:16.992+0000 7f087a8e8640 1 --2- 172.21.15.49:0/4235104185 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f085c0a2a90 0x7f085c0a4f50 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:16.994 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:16.992+0000 7f087a8e8640 1 --2- 172.21.15.49:0/4235104185 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0874107890 0x7f08741014c0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:16.995 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:16.992+0000 7f087a8e8640 1 --2- 172.21.15.49:0/4235104185 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0874069840 0x7f0874100f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:16.995 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:16.992+0000 7f087a8e8640 1 --2- 172.21.15.49:0/4235104185 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0874068ea0 0x7f0874100a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:16.995 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:16.992+0000 7f087a8e8640 1 -- 172.21.15.49:0/4235104185 >> 172.21.15.49:0/4235104185 conn(0x7f0874077570 msgr2=0x7f08740ff550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:10:16.995 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:16.992+0000 7f087a8e8640 1 -- 172.21.15.49:0/4235104185 shutdown_connections 2024-03-18T23:10:16.995 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:16.992+0000 7f087a8e8640 1 -- 172.21.15.49:0/4235104185 wait complete. 2024-03-18T23:10:17.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:10:17 smithi139 ceph-mon[67282]: pgmap v63: 97 pgs: 97 active+clean; 58 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.1 MiB/s wr, 330 op/s 2024-03-18T23:10:17.379 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:10:17 smithi049 ceph-mon[62991]: pgmap v63: 97 pgs: 97 active+clean; 58 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.1 MiB/s wr, 330 op/s 2024-03-18T23:10:17.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:10:17 smithi175 ceph-mon[67352]: pgmap v63: 97 pgs: 97 active+clean; 58 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.1 MiB/s wr, 330 op/s 2024-03-18T23:10:17.761 INFO:tasks.workunit.client.0.smithi049.stdout: 1 187387 13.01 MB/sec warmup 79 sec latency 103.702 ms 2024-03-18T23:10:18.762 INFO:tasks.workunit.client.0.smithi049.stdout: 1 189285 12.96 MB/sec warmup 80 sec latency 18.554 ms 2024-03-18T23:10:19.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:10:19 smithi139 ceph-mon[67282]: pgmap v64: 97 pgs: 97 active+clean; 61 MiB data, 689 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 4.5 MiB/s wr, 436 op/s 2024-03-18T23:10:19.379 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:10:19 smithi049 ceph-mon[62991]: pgmap v64: 97 pgs: 97 active+clean; 61 MiB data, 689 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 4.5 MiB/s wr, 436 op/s 2024-03-18T23:10:19.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:10:19 smithi175 ceph-mon[67352]: pgmap v64: 97 pgs: 97 active+clean; 61 MiB data, 689 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 4.5 MiB/s wr, 436 op/s 2024-03-18T23:10:19.762 INFO:tasks.workunit.client.0.smithi049.stdout: 1 191958 13.01 MB/sec warmup 81 sec latency 14.917 ms 2024-03-18T23:10:20.762 INFO:tasks.workunit.client.0.smithi049.stdout: 1 194735 13.01 MB/sec warmup 82 sec latency 39.979 ms 2024-03-18T23:10:21.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:10:21 smithi139 ceph-mon[67282]: pgmap v65: 97 pgs: 97 active+clean; 61 MiB data, 689 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.0 MiB/s wr, 214 op/s 2024-03-18T23:10:21.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:10:21 smithi049 ceph-mon[62991]: pgmap v65: 97 pgs: 97 active+clean; 61 MiB data, 689 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.0 MiB/s wr, 214 op/s 2024-03-18T23:10:21.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:10:21 smithi175 ceph-mon[67352]: pgmap v65: 97 pgs: 97 active+clean; 61 MiB data, 689 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.0 MiB/s wr, 214 op/s 2024-03-18T23:10:21.762 INFO:tasks.workunit.client.0.smithi049.stdout: 1 195980 12.93 MB/sec warmup 83 sec latency 68.502 ms 2024-03-18T23:10:22.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:10:22 smithi139 ceph-mon[67282]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:10:22.316 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:10:22 smithi175 ceph-mon[67352]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:10:22.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:10:22 smithi049 ceph-mon[62991]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:10:22.762 INFO:tasks.workunit.client.0.smithi049.stdout: 1 198746 12.98 MB/sec warmup 84 sec latency 57.908 ms 2024-03-18T23:10:23.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:10:23 smithi049 ceph-mon[62991]: pgmap v66: 97 pgs: 97 active+clean; 55 MiB data, 658 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 4.6 MiB/s wr, 419 op/s 2024-03-18T23:10:23.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:10:23 smithi049 ceph-mon[62991]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' 2024-03-18T23:10:23.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:10:23 smithi049 ceph-mon[62991]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' 2024-03-18T23:10:23.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:10:23 smithi049 ceph-mon[62991]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' 2024-03-18T23:10:23.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:10:23 smithi049 ceph-mon[62991]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' 2024-03-18T23:10:23.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:10:23 smithi175 ceph-mon[67352]: pgmap v66: 97 pgs: 97 active+clean; 55 MiB data, 658 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 4.6 MiB/s wr, 419 op/s 2024-03-18T23:10:23.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:10:23 smithi175 ceph-mon[67352]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' 2024-03-18T23:10:23.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:10:23 smithi175 ceph-mon[67352]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' 2024-03-18T23:10:23.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:10:23 smithi175 ceph-mon[67352]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' 2024-03-18T23:10:23.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:10:23 smithi175 ceph-mon[67352]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' 2024-03-18T23:10:23.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:10:23 smithi139 ceph-mon[67282]: pgmap v66: 97 pgs: 97 active+clean; 55 MiB data, 658 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 4.6 MiB/s wr, 419 op/s 2024-03-18T23:10:23.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:10:23 smithi139 ceph-mon[67282]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' 2024-03-18T23:10:23.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:10:23 smithi139 ceph-mon[67282]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' 2024-03-18T23:10:23.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:10:23 smithi139 ceph-mon[67282]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' 2024-03-18T23:10:23.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:10:23 smithi139 ceph-mon[67282]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' 2024-03-18T23:10:23.762 INFO:tasks.workunit.client.0.smithi049.stdout: 1 200932 12.92 MB/sec warmup 85 sec latency 26.634 ms 2024-03-18T23:10:24.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:10:24 smithi049 ceph-mon[62991]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:10:24.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:10:24 smithi049 ceph-mon[62991]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:10:24.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:10:24 smithi049 ceph-mon[62991]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' 2024-03-18T23:10:24.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:10:24 smithi175 ceph-mon[67352]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:10:24.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:10:24 smithi175 ceph-mon[67352]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:10:24.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:10:24 smithi175 ceph-mon[67352]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' 2024-03-18T23:10:24.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:10:24 smithi139 ceph-mon[67282]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:10:24.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:10:24 smithi139 ceph-mon[67282]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:10:24.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:10:24 smithi139 ceph-mon[67282]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' 2024-03-18T23:10:24.762 INFO:tasks.workunit.client.0.smithi049.stdout: 1 202903 12.92 MB/sec warmup 86 sec latency 20.080 ms 2024-03-18T23:10:25.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:10:25 smithi049 ceph-mon[62991]: pgmap v67: 97 pgs: 97 active+clean; 55 MiB data, 658 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.0 MiB/s wr, 311 op/s 2024-03-18T23:10:25.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:10:25 smithi175 ceph-mon[67352]: pgmap v67: 97 pgs: 97 active+clean; 55 MiB data, 658 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.0 MiB/s wr, 311 op/s 2024-03-18T23:10:25.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:10:25 smithi139 ceph-mon[67282]: pgmap v67: 97 pgs: 97 active+clean; 55 MiB data, 658 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.0 MiB/s wr, 311 op/s 2024-03-18T23:10:25.762 INFO:tasks.workunit.client.0.smithi049.stdout: 1 205871 12.97 MB/sec warmup 87 sec latency 9.637 ms 2024-03-18T23:10:26.762 INFO:tasks.workunit.client.0.smithi049.stdout: 1 207369 12.90 MB/sec warmup 88 sec latency 55.231 ms 2024-03-18T23:10:27.379 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:10:27 smithi049 ceph-mon[62991]: pgmap v68: 97 pgs: 97 active+clean; 55 MiB data, 658 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.0 MiB/s wr, 311 op/s 2024-03-18T23:10:27.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:10:27 smithi175 ceph-mon[67352]: pgmap v68: 97 pgs: 97 active+clean; 55 MiB data, 658 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.0 MiB/s wr, 311 op/s 2024-03-18T23:10:27.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:10:27 smithi139 ceph-mon[67282]: pgmap v68: 97 pgs: 97 active+clean; 55 MiB data, 658 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.0 MiB/s wr, 311 op/s 2024-03-18T23:10:27.762 INFO:tasks.workunit.client.0.smithi049.stdout: 1 209720 12.91 MB/sec warmup 89 sec latency 42.459 ms 2024-03-18T23:10:28.762 INFO:tasks.workunit.client.0.smithi049.stdout: 1 212613 12.93 MB/sec warmup 90 sec latency 26.230 ms 2024-03-18T23:10:29.379 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:10:29 smithi049 ceph-mon[62991]: pgmap v69: 97 pgs: 97 active+clean; 59 MiB data, 707 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 4.5 MiB/s wr, 421 op/s 2024-03-18T23:10:29.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:10:29 smithi175 ceph-mon[67352]: pgmap v69: 97 pgs: 97 active+clean; 59 MiB data, 707 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 4.5 MiB/s wr, 421 op/s 2024-03-18T23:10:29.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:10:29 smithi139 ceph-mon[67282]: pgmap v69: 97 pgs: 97 active+clean; 59 MiB data, 707 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 4.5 MiB/s wr, 421 op/s 2024-03-18T23:10:29.762 INFO:tasks.workunit.client.0.smithi049.stdout: 1 214469 12.90 MB/sec warmup 91 sec latency 24.582 ms 2024-03-18T23:10:30.088 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:30.086+0000 7f0150390640 1 -- 172.21.15.49:0/737882229 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0148068ea0 msgr2=0x7f0148069320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:10:30.089 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:30.086+0000 7f0150390640 1 --2- 172.21.15.49:0/737882229 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0148068ea0 0x7f0148069320 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f0138009900 tx=0x7f013802bc30 comp rx=0 tx=0).stop 2024-03-18T23:10:30.089 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:30.086+0000 7f0150390640 1 -- 172.21.15.49:0/737882229 shutdown_connections 2024-03-18T23:10:30.089 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:30.086+0000 7f0150390640 1 --2- 172.21.15.49:0/737882229 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0148069860 0x7f014810e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:30.089 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:30.086+0000 7f0150390640 1 --2- 172.21.15.49:0/737882229 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0148068ea0 0x7f0148069320 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:30.089 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:30.086+0000 7f0150390640 1 --2- 172.21.15.49:0/737882229 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0148107820 0x7f0148107c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:30.089 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:30.086+0000 7f0150390640 1 -- 172.21.15.49:0/737882229 >> 172.21.15.49:0/737882229 conn(0x7f01480774b0 msgr2=0x7f01480778c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:10:30.089 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:30.086+0000 7f0150390640 1 -- 172.21.15.49:0/737882229 shutdown_connections 2024-03-18T23:10:30.089 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:30.087+0000 7f0150390640 1 -- 172.21.15.49:0/737882229 wait complete. 2024-03-18T23:10:30.089 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:30.087+0000 7f0150390640 1 Processor -- start 2024-03-18T23:10:30.089 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:30.087+0000 7f0150390640 1 -- start start 2024-03-18T23:10:30.089 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:30.087+0000 7f0150390640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0148068ea0 0x7f0148197fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:10:30.089 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:30.087+0000 7f0150390640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0148069860 0x7f01481984f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:10:30.089 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:30.087+0000 7f0150390640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0148107820 0x7f014819c820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:10:30.090 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:30.087+0000 7f0150390640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f0148111bb0 con 0x7f0148069860 2024-03-18T23:10:30.090 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:30.087+0000 7f0150390640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f0148111a30 con 0x7f0148107820 2024-03-18T23:10:30.090 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:30.087+0000 7f0150390640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f0148111d30 con 0x7f0148068ea0 2024-03-18T23:10:30.090 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:30.087+0000 7f014d92b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0148069860 0x7f01481984f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:10:30.090 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:30.087+0000 7f014d92b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0148069860 0x7f01481984f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49376/0 (socket says 172.21.15.49:49376) 2024-03-18T23:10:30.090 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:30.087+0000 7f014e12c640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0148068ea0 0x7f0148197fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:10:30.090 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:30.088+0000 7f014d92b640 1 -- 172.21.15.49:0/3976930039 learned_addr learned my addr 172.21.15.49:0/3976930039 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:10:30.090 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:30.088+0000 7f014e92d640 1 --2- 172.21.15.49:0/3976930039 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0148107820 0x7f014819c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:10:30.090 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:30.088+0000 7f014d92b640 1 -- 172.21.15.49:0/3976930039 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0148068ea0 msgr2=0x7f0148197fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:10:30.090 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:30.088+0000 7f014d92b640 1 --2- 172.21.15.49:0/3976930039 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0148068ea0 0x7f0148197fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:30.090 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:30.088+0000 7f014d92b640 1 -- 172.21.15.49:0/3976930039 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0148107820 msgr2=0x7f014819c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:10:30.090 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:30.088+0000 7f014d92b640 1 --2- 172.21.15.49:0/3976930039 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0148107820 0x7f014819c820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:30.090 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:30.088+0000 7f014d92b640 1 -- 172.21.15.49:0/3976930039 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f014819cf50 con 0x7f0148069860 2024-03-18T23:10:30.091 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:30.088+0000 7f014d92b640 1 --2- 172.21.15.49:0/3976930039 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0148069860 0x7f01481984f0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f0138005940 tx=0x7f01380042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:10:30.091 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:30.088+0000 7f013f7fe640 1 -- 172.21.15.49:0/3976930039 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0138041070 con 0x7f0148069860 2024-03-18T23:10:30.091 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:30.088+0000 7f0150390640 1 -- 172.21.15.49:0/3976930039 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f014819d1e0 con 0x7f0148069860 2024-03-18T23:10:30.091 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:30.088+0000 7f013f7fe640 1 -- 172.21.15.49:0/3976930039 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f013802ccc0 con 0x7f0148069860 2024-03-18T23:10:30.091 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:30.088+0000 7f013f7fe640 1 -- 172.21.15.49:0/3976930039 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0138003d80 con 0x7f0148069860 2024-03-18T23:10:30.091 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:30.088+0000 7f0150390640 1 -- 172.21.15.49:0/3976930039 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01481a4ad0 con 0x7f0148069860 2024-03-18T23:10:30.097 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:30.090+0000 7f013f7fe640 1 -- 172.21.15.49:0/3976930039 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 25) ==== 137421+0+0 (secure 0 0 0) 0x7f0138044070 con 0x7f0148069860 2024-03-18T23:10:30.098 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:30.091+0000 7f0152f76740 1 Processor -- start 2024-03-18T23:10:30.098 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:30.091+0000 7f0152f76740 1 -- start start 2024-03-18T23:10:30.098 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:30.091+0000 7f0152f76740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563aa6f20ef0 0x563aa6f212d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:10:30.098 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:30.091+0000 7f0152f76740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563aa6f21810 0x563aa6f2a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:10:30.098 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:30.091+0000 7f0152f76740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563aa6f2a5e0 0x563aa6f2c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:10:30.098 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:30.091+0000 7f0152f76740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x563aa6e84be0 con 0x563aa6f21810 2024-03-18T23:10:30.098 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:30.091+0000 7f0152f76740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x563aa6e2d960 con 0x563aa6f2a5e0 2024-03-18T23:10:30.098 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:30.091+0000 7f0152f76740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x563aa6e80c80 con 0x563aa6f20ef0 2024-03-18T23:10:30.098 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:30.091+0000 7f013f7fe640 1 --2- 172.21.15.49:0/3976930039 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f01340a2a10 0x7f01340a4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:10:30.098 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:30.091+0000 7f014e12c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563aa6f21810 0x563aa6f2a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:10:30.098 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:30.091+0000 7f014e92d640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563aa6f20ef0 0x563aa6f212d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:10:30.098 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:30.091+0000 7f014e12c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563aa6f21810 0x563aa6f2a0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49382/0 (socket says 172.21.15.49:49382) 2024-03-18T23:10:30.099 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:30.091+0000 7f014e12c640 1 -- 172.21.15.49:0/2858635828 learned_addr learned my addr 172.21.15.49:0/2858635828 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:10:30.099 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:30.091+0000 7f014d92b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563aa6f2a5e0 0x563aa6f2c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:10:30.099 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:30.091+0000 7f014e92d640 1 --2- 172.21.15.49:0/3976930039 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f01340a2a10 0x7f01340a4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:10:30.099 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:30.092+0000 7f013f7fe640 1 -- 172.21.15.49:0/3976930039 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f01380f0800 con 0x7f0148069860 2024-03-18T23:10:30.099 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:30.092+0000 7f014e92d640 1 -- 172.21.15.49:0/2858635828 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563aa6f2a5e0 msgr2=0x563aa6f2c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:10:30.099 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:30.092+0000 7f014e92d640 1 --2- 172.21.15.49:0/2858635828 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563aa6f2a5e0 0x563aa6f2c9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:30.099 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:30.092+0000 7f014e92d640 1 -- 172.21.15.49:0/2858635828 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563aa6f21810 msgr2=0x563aa6f2a0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:10:30.099 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:30.092+0000 7f014e92d640 1 --2- 172.21.15.49:0/2858635828 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563aa6f21810 0x563aa6f2a0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:30.099 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:30.092+0000 7f014e92d640 1 -- 172.21.15.49:0/2858635828 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563aa6e24d20 con 0x563aa6f20ef0 2024-03-18T23:10:30.099 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:30.092+0000 7f014e12c640 1 --2- 172.21.15.49:0/2858635828 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563aa6f21810 0x563aa6f2a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:10:30.099 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:30.092+0000 7f014e92d640 1 --2- 172.21.15.49:0/2858635828 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563aa6f20ef0 0x563aa6f212d0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f0140005fd0 tx=0x7f0140010740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9d828b875527a95 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:10:30.099 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:30.092+0000 7f014e92d640 1 --2- 172.21.15.49:0/3976930039 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f01340a2a10 0x7f01340a4ed0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f0140010c40 tx=0x7f0140017000 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:10:30.099 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:30.092+0000 7f013cbfe640 1 -- 172.21.15.49:0/2858635828 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0140024070 con 0x563aa6f20ef0 2024-03-18T23:10:30.099 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:30.092+0000 7f013cbfe640 1 -- 172.21.15.49:0/2858635828 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0140003e50 con 0x563aa6f20ef0 2024-03-18T23:10:30.099 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:30.092+0000 7f013cbfe640 1 -- 172.21.15.49:0/2858635828 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f014000dea0 con 0x563aa6f20ef0 2024-03-18T23:10:30.100 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:30.093+0000 7f0152f76740 1 -- 172.21.15.49:0/2858635828 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563aa6f20ef0 msgr2=0x563aa6f212d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:10:30.100 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:30.093+0000 7f0152f76740 1 --2- 172.21.15.49:0/2858635828 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563aa6f20ef0 0x563aa6f212d0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f0140005fd0 tx=0x7f0140010740 comp rx=0 tx=0).stop 2024-03-18T23:10:30.100 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:30.093+0000 7f0152f76740 1 -- 172.21.15.49:0/2858635828 shutdown_connections 2024-03-18T23:10:30.100 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:30.093+0000 7f0152f76740 1 --2- 172.21.15.49:0/2858635828 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563aa6f2a5e0 0x563aa6f2c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:30.100 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:30.093+0000 7f0152f76740 1 --2- 172.21.15.49:0/2858635828 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563aa6f21810 0x563aa6f2a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:30.100 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:30.093+0000 7f0152f76740 1 --2- 172.21.15.49:0/2858635828 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563aa6f20ef0 0x563aa6f212d0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:30.100 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:30.093+0000 7f0152f76740 1 -- 172.21.15.49:0/2858635828 >> 172.21.15.49:0/2858635828 conn(0x563aa6f1b810 msgr2=0x563aa6f1bbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:10:30.100 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:30.093+0000 7f0152f76740 1 -- 172.21.15.49:0/2858635828 shutdown_connections 2024-03-18T23:10:30.100 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:30.093+0000 7f0152f76740 1 -- 172.21.15.49:0/2858635828 wait complete. 2024-03-18T23:10:30.100 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:30.093+0000 7f0152f76740 1 -- start start 2024-03-18T23:10:30.100 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:30.093+0000 7f0152f76740 1 Processor -- start 2024-03-18T23:10:30.101 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:30.094+0000 7f0152f76740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563aa6f20ef0 0x563aa6f29fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:10:30.101 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:30.094+0000 7f0152f76740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563aa6f21810 0x563aa6f273c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:10:30.101 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:30.094+0000 7f0152f76740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563aa6f2a5e0 0x563aa6f27a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:10:30.101 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:30.094+0000 7f0152f76740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x563aa6f2d1b0 con 0x563aa6f21810 2024-03-18T23:10:30.101 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:30.094+0000 7f0152f76740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x563aa6e40db0 con 0x563aa6f2a5e0 2024-03-18T23:10:30.101 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:30.094+0000 7f0152f76740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x563aa6f2d330 con 0x563aa6f20ef0 2024-03-18T23:10:30.101 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:30.094+0000 7f0152f76740 10 client.0 _ll_register_callbacks cb 0x563aa6e45e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T23:10:30.101 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:30.094+0000 7f014d92b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563aa6f21810 0x563aa6f273c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:10:30.101 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:30.094+0000 7f014d92b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563aa6f21810 0x563aa6f273c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49396/0 (socket says 172.21.15.49:49396) 2024-03-18T23:10:30.101 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:30.094+0000 7f014d92b640 1 -- 172.21.15.49:0/1786243185 learned_addr learned my addr 172.21.15.49:0/1786243185 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:10:30.101 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:30.094+0000 7f014e92d640 1 --2- 172.21.15.49:0/1786243185 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563aa6f2a5e0 0x563aa6f27a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:10:30.101 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:30.094+0000 7f014e12c640 1 --2- 172.21.15.49:0/1786243185 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563aa6f20ef0 0x563aa6f29fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:10:30.101 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:30.094+0000 7f014d92b640 1 -- 172.21.15.49:0/1786243185 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563aa6f20ef0 msgr2=0x563aa6f29fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:10:30.101 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:30.094+0000 7f014d92b640 1 --2- 172.21.15.49:0/1786243185 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563aa6f20ef0 0x563aa6f29fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:30.102 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:30.094+0000 7f014d92b640 1 -- 172.21.15.49:0/1786243185 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563aa6f2a5e0 msgr2=0x563aa6f27a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:10:30.102 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:30.094+0000 7f014d92b640 1 --2- 172.21.15.49:0/1786243185 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563aa6f2a5e0 0x563aa6f27a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:30.102 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:30.094+0000 7f014d92b640 1 -- 172.21.15.49:0/1786243185 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563aa6e3fc60 con 0x563aa6f21810 2024-03-18T23:10:30.102 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:30.094+0000 7f014d92b640 1 --2- 172.21.15.49:0/1786243185 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563aa6f21810 0x563aa6f273c0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f01380b9a20 tx=0x7f0138004560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c075ccbae9d3f350 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:10:30.102 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:30.094+0000 7f0131ffb640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-18T23:10:30.102 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:30.094+0000 7f0131ffb640 1 -- 172.21.15.49:0/1786243185 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f013804b070 con 0x563aa6f21810 2024-03-18T23:10:30.102 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:30.094+0000 7f0152f76740 1 -- 172.21.15.49:0/1786243185 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0140003e50 con 0x563aa6f21810 2024-03-18T23:10:30.102 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:30.094+0000 7f0131ffb640 1 -- 172.21.15.49:0/1786243185 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f01380c52b0 con 0x563aa6f21810 2024-03-18T23:10:30.102 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:30.095+0000 7f0131ffb640 1 -- 172.21.15.49:0/1786243185 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f01380bce20 con 0x563aa6f21810 2024-03-18T23:10:30.102 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:30.095+0000 7f0152f76740 10 client.15633 fetch_fsmap learned FSMap version 29 2024-03-18T23:10:30.102 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:30.095+0000 7f0152f76740 1 -- 172.21.15.49:0/1786243185 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x563aa6b99210 con 0x563aa6f21810 2024-03-18T23:10:30.102 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:30.095+0000 7f0131ffb640 1 -- 172.21.15.49:0/1786243185 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0140003e50 con 0x563aa6f21810 2024-03-18T23:10:30.102 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:30.095+0000 7f0131ffb640 1 -- 172.21.15.49:0/1786243185 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 29) ==== 8358+0+0 (secure 0 0 0) 0x7f013808cb90 con 0x563aa6f21810 2024-03-18T23:10:30.103 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:30.095+0000 7f0152f76740 10 client.15633 fetch_fsmap finished waiting for FSMap version 29 2024-03-18T23:10:30.103 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:30.095+0000 7f0152f76740 10 client.15633 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T23:10:30.103 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:30.095+0000 7f0152f76740 20 client.15633 populate_metadata read hostname 'smithi049' 2024-03-18T23:10:30.103 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:30.095+0000 7f0152f76740 1 --2- 172.21.15.49:0/1786243185 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x563aa6f3a460 0x563aa6f3c850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:10:30.103 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:30.095+0000 7f0152f76740 4 client.15633 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T23:10:30.103 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:30.095+0000 7f0152f76740 1 -- 172.21.15.49:0/1786243185 --> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563aa6e92120 con 0x563aa6f3a460 2024-03-18T23:10:30.103 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:30.095+0000 7f014e12c640 1 --2- 172.21.15.49:0/1786243185 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x563aa6f3a460 0x563aa6f3c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:10:30.103 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:30.096+0000 7f014e12c640 1 --2- 172.21.15.49:0/1786243185 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x563aa6f3a460 0x563aa6f3c850 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d8ca3ac8a68d47f7 server_cookie=85048cbc47573654 in_seq=0 out_seq=0 2024-03-18T23:10:30.103 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:30.096+0000 7f0131ffb640 10 client.15633 ms_handle_connect on v2:172.21.15.139:6838/4280128209 2024-03-18T23:10:30.103 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:30.097+0000 7f0131ffb640 1 -- 172.21.15.49:0/1786243185 <== mds.0 v2:172.21.15.139:6838/4280128209 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x563aa6e92120 con 0x563aa6f3a460 2024-03-18T23:10:30.103 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:30.097+0000 7f0131ffb640 10 client.15633 handle_command_reply: tid=0 2024-03-18T23:10:30.103 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:30.097+0000 7f0152f76740 1 client.15633 shutdown 2024-03-18T23:10:30.103 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:30.097+0000 7f0152f76740 1 -- 172.21.15.49:0/1786243185 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563aa6f21810 msgr2=0x563aa6f273c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:10:30.103 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:30.097+0000 7f0152f76740 1 --2- 172.21.15.49:0/1786243185 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563aa6f21810 0x563aa6f273c0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f01380b9a20 tx=0x7f0138004560 comp rx=0 tx=0).stop 2024-03-18T23:10:30.104 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:30.097+0000 7f0152f76740 1 -- 172.21.15.49:0/1786243185 shutdown_connections 2024-03-18T23:10:30.104 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:30.097+0000 7f0152f76740 1 --2- 172.21.15.49:0/1786243185 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x563aa6f3a460 0x563aa6f3c850 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:30.104 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:30.097+0000 7f0152f76740 1 --2- 172.21.15.49:0/1786243185 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563aa6f2a5e0 0x563aa6f27a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:30.104 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:30.097+0000 7f0152f76740 1 --2- 172.21.15.49:0/1786243185 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563aa6f21810 0x563aa6f273c0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:30.104 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:30.097+0000 7f0152f76740 1 --2- 172.21.15.49:0/1786243185 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563aa6f20ef0 0x563aa6f29fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:30.104 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:30.097+0000 7f0131ffb640 0 client.15633 ms_handle_reset on v2:172.21.15.139:6838/4280128209 2024-03-18T23:10:30.104 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:30.097+0000 7f0152f76740 1 -- 172.21.15.49:0/1786243185 >> 172.21.15.49:0/1786243185 conn(0x563aa6f1b810 msgr2=0x563aa6f1aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:10:30.104 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:30.097+0000 7f0152f76740 1 -- 172.21.15.49:0/1786243185 shutdown_connections 2024-03-18T23:10:30.104 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:30.098+0000 7f0152f76740 1 -- 172.21.15.49:0/1786243185 wait complete. 2024-03-18T23:10:30.104 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:30.098+0000 7f0152f76740 20 client.15633 trim_cache size 0 max 16384 2024-03-18T23:10:30.114 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:30.113+0000 7f0152f76740 1 Processor -- start 2024-03-18T23:10:30.114 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:30.113+0000 7f0152f76740 1 -- start start 2024-03-18T23:10:30.114 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:30.113+0000 7f0152f76740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563aa6f20ef0 0x563aa6f35f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:10:30.114 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:30.113+0000 7f0152f76740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563aa6f21810 0x563aa6f26ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:10:30.114 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:30.113+0000 7f0152f76740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563aa6f2a5e0 0x563aa6f291f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:10:30.114 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:30.113+0000 7f0152f76740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f01481ad2a0 con 0x563aa6f21810 2024-03-18T23:10:30.115 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:30.113+0000 7f0152f76740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f01481aee10 con 0x563aa6f20ef0 2024-03-18T23:10:30.115 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:30.113+0000 7f0152f76740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f01481ad970 con 0x563aa6f2a5e0 2024-03-18T23:10:30.115 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:30.113+0000 7f014d92b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563aa6f21810 0x563aa6f26ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:10:30.115 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:30.113+0000 7f014d92b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563aa6f21810 0x563aa6f26ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49404/0 (socket says 172.21.15.49:49404) 2024-03-18T23:10:30.115 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:30.113+0000 7f014d92b640 1 -- 172.21.15.49:0/67366755 learned_addr learned my addr 172.21.15.49:0/67366755 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:10:30.115 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:30.113+0000 7f014e92d640 1 --2- 172.21.15.49:0/67366755 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563aa6f2a5e0 0x563aa6f291f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:10:30.115 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:30.113+0000 7f014e12c640 1 --2- 172.21.15.49:0/67366755 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563aa6f20ef0 0x563aa6f35f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:10:30.115 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:30.114+0000 7f014d92b640 1 -- 172.21.15.49:0/67366755 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563aa6f2a5e0 msgr2=0x563aa6f291f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:10:30.115 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:30.114+0000 7f014d92b640 1 --2- 172.21.15.49:0/67366755 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563aa6f2a5e0 0x563aa6f291f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:30.115 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:30.114+0000 7f014d92b640 1 -- 172.21.15.49:0/67366755 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563aa6f20ef0 msgr2=0x563aa6f35f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:10:30.115 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:30.114+0000 7f014d92b640 1 --2- 172.21.15.49:0/67366755 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563aa6f20ef0 0x563aa6f35f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:30.115 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:30.114+0000 7f014d92b640 1 -- 172.21.15.49:0/67366755 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563aa6c65620 con 0x563aa6f21810 2024-03-18T23:10:30.115 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:30.114+0000 7f014e92d640 1 --2- 172.21.15.49:0/67366755 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563aa6f2a5e0 0x563aa6f291f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:10:30.115 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:30.114+0000 7f014d92b640 1 --2- 172.21.15.49:0/67366755 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563aa6f21810 0x563aa6f26ca0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f01380b94b0 tx=0x7f013802ce70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95041bdc2d7447a3 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:10:30.116 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:30.114+0000 7f0133fff640 1 -- 172.21.15.49:0/67366755 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f013804b070 con 0x563aa6f21810 2024-03-18T23:10:30.116 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:30.114+0000 7f0133fff640 1 -- 172.21.15.49:0/67366755 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f01380bc7c0 con 0x563aa6f21810 2024-03-18T23:10:30.116 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:30.114+0000 7f0133fff640 1 -- 172.21.15.49:0/67366755 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01380bc9b0 con 0x563aa6f21810 2024-03-18T23:10:30.119 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:30.115+0000 7f0152f76740 1 -- 172.21.15.49:0/67366755 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563aa6f21810 msgr2=0x563aa6f26ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:10:30.119 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:30.115+0000 7f0152f76740 1 --2- 172.21.15.49:0/67366755 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563aa6f21810 0x563aa6f26ca0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f01380b94b0 tx=0x7f013802ce70 comp rx=0 tx=0).stop 2024-03-18T23:10:30.119 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:30.115+0000 7f0152f76740 1 -- 172.21.15.49:0/67366755 shutdown_connections 2024-03-18T23:10:30.119 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:30.115+0000 7f0152f76740 1 --2- 172.21.15.49:0/67366755 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563aa6f2a5e0 0x563aa6f291f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:30.119 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:30.115+0000 7f0152f76740 1 --2- 172.21.15.49:0/67366755 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563aa6f21810 0x563aa6f26ca0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:30.119 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:30.115+0000 7f0152f76740 1 --2- 172.21.15.49:0/67366755 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563aa6f20ef0 0x563aa6f35f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:30.119 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:30.115+0000 7f0152f76740 1 -- 172.21.15.49:0/67366755 >> 172.21.15.49:0/67366755 conn(0x563aa6f1b810 msgr2=0x563aa6f28ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:10:30.120 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:30.115+0000 7f0152f76740 1 -- 172.21.15.49:0/67366755 shutdown_connections 2024-03-18T23:10:30.120 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:30.115+0000 7f0152f76740 1 -- 172.21.15.49:0/67366755 wait complete. 2024-03-18T23:10:30.120 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:30.115+0000 7f0152f76740 1 -- start start 2024-03-18T23:10:30.120 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:30.115+0000 7f0152f76740 1 Processor -- start 2024-03-18T23:10:30.120 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:30.115+0000 7f0152f76740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563aa6f20ef0 0x563aa6f32580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:10:30.120 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:30.115+0000 7f0152f76740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563aa6f21810 0x563aa6f237b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:10:30.120 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:30.116+0000 7f0152f76740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563aa6f2a5e0 0x563aa6f23cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:10:30.120 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:30.116+0000 7f0152f76740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f01481ae710 con 0x563aa6f21810 2024-03-18T23:10:30.120 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:30.116+0000 7f0152f76740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f01481ae040 con 0x563aa6f20ef0 2024-03-18T23:10:30.120 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:30.116+0000 7f0152f76740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f01481ab280 con 0x563aa6f2a5e0 2024-03-18T23:10:30.120 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:30.116+0000 7f0152f76740 10 client.0 _ll_register_callbacks cb 0x7f0138005b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T23:10:30.120 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:30.116+0000 7f014d92b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563aa6f21810 0x563aa6f237b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:10:30.120 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:30.116+0000 7f014e12c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563aa6f20ef0 0x563aa6f32580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:10:30.120 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:30.116+0000 7f014d92b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563aa6f21810 0x563aa6f237b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49408/0 (socket says 172.21.15.49:49408) 2024-03-18T23:10:30.121 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:30.116+0000 7f014e12c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563aa6f20ef0 0x563aa6f32580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.49:56454/0 (socket says 172.21.15.49:56454) 2024-03-18T23:10:30.121 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:30.116+0000 7f014d92b640 1 -- 172.21.15.49:0/1801700675 learned_addr learned my addr 172.21.15.49:0/1801700675 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:10:30.121 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:30.116+0000 7f014e92d640 1 --2- 172.21.15.49:0/1801700675 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563aa6f2a5e0 0x563aa6f23cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:10:30.121 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:30.116+0000 7f014d92b640 1 -- 172.21.15.49:0/1801700675 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563aa6f2a5e0 msgr2=0x563aa6f23cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:10:30.121 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:30.116+0000 7f014d92b640 1 --2- 172.21.15.49:0/1801700675 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563aa6f2a5e0 0x563aa6f23cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:30.121 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:30.116+0000 7f014d92b640 1 -- 172.21.15.49:0/1801700675 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563aa6f20ef0 msgr2=0x563aa6f32580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:10:30.121 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:30.116+0000 7f014d92b640 1 --2- 172.21.15.49:0/1801700675 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563aa6f20ef0 0x563aa6f32580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:30.121 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:30.116+0000 7f014d92b640 1 -- 172.21.15.49:0/1801700675 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563aa6f24230 con 0x563aa6f21810 2024-03-18T23:10:30.121 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:30.116+0000 7f014d92b640 1 --2- 172.21.15.49:0/1801700675 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563aa6f21810 0x563aa6f237b0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f0138049fd0 tx=0x7f01380c5600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32d367fcdf533d91 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:10:30.121 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:30.116+0000 7f0131ffb640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-18T23:10:30.121 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:30.116+0000 7f0131ffb640 1 -- 172.21.15.49:0/1801700675 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f013804b070 con 0x563aa6f21810 2024-03-18T23:10:30.121 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:30.116+0000 7f0131ffb640 1 -- 172.21.15.49:0/1801700675 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f01380b8da0 con 0x563aa6f21810 2024-03-18T23:10:30.121 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:30.116+0000 7f0152f76740 1 -- 172.21.15.49:0/1801700675 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f01380bc7c0 con 0x563aa6f21810 2024-03-18T23:10:30.122 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:30.116+0000 7f0131ffb640 1 -- 172.21.15.49:0/1801700675 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f01380bcb90 con 0x563aa6f21810 2024-03-18T23:10:30.122 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:30.116+0000 7f0152f76740 10 client.15639 fetch_fsmap learned FSMap version 29 2024-03-18T23:10:30.122 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:30.116+0000 7f0152f76740 1 -- 172.21.15.49:0/1801700675 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x563aa6f24230 con 0x563aa6f21810 2024-03-18T23:10:30.122 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:30.117+0000 7f0131ffb640 1 -- 172.21.15.49:0/1801700675 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f01380bc7c0 con 0x563aa6f21810 2024-03-18T23:10:30.122 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:30.117+0000 7f0131ffb640 1 -- 172.21.15.49:0/1801700675 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 29) ==== 8358+0+0 (secure 0 0 0) 0x7f01380dbaa0 con 0x563aa6f21810 2024-03-18T23:10:30.122 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:30.117+0000 7f0152f76740 10 client.15639 fetch_fsmap finished waiting for FSMap version 29 2024-03-18T23:10:30.122 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:30.117+0000 7f0152f76740 10 client.15639 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T23:10:30.122 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:30.117+0000 7f0152f76740 20 client.15639 populate_metadata read hostname 'smithi049' 2024-03-18T23:10:30.122 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:30.117+0000 7f0152f76740 1 --2- 172.21.15.49:0/1801700675 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x563aa6fa4a20 0x563aa6fa6e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:10:30.122 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:30.117+0000 7f0152f76740 4 client.15639 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T23:10:30.122 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:30.117+0000 7f0152f76740 1 -- 172.21.15.49:0/1801700675 --> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563aa6e8c470 con 0x563aa6fa4a20 2024-03-18T23:10:30.122 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:30.117+0000 7f014e12c640 1 --2- 172.21.15.49:0/1801700675 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x563aa6fa4a20 0x563aa6fa6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:10:30.122 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:30.117+0000 7f0131ffb640 10 client.15639 ms_handle_connect on v2:172.21.15.139:6838/4280128209 2024-03-18T23:10:30.122 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:30.117+0000 7f014e12c640 1 --2- 172.21.15.49:0/1801700675 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x563aa6fa4a20 0x563aa6fa6e10 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7686f56152959482 server_cookie=e0d2c1fdd6348486 in_seq=0 out_seq=0 2024-03-18T23:10:30.272 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:30.271+0000 7f0131ffb640 1 -- 172.21.15.49:0/1801700675 <== mds.0 v2:172.21.15.139:6838/4280128209 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x563aa6e8c470 con 0x563aa6fa4a20 2024-03-18T23:10:30.272 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:30.271+0000 7f0131ffb640 10 client.15639 handle_command_reply: tid=0 2024-03-18T23:10:30.272 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:30.271+0000 7f0152f76740 1 client.15639 shutdown 2024-03-18T23:10:30.272 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:30.271+0000 7f0152f76740 1 -- 172.21.15.49:0/1801700675 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563aa6f21810 msgr2=0x563aa6f237b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:10:30.272 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:30.271+0000 7f0152f76740 1 --2- 172.21.15.49:0/1801700675 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563aa6f21810 0x563aa6f237b0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f0138049fd0 tx=0x7f01380c5600 comp rx=0 tx=0).stop 2024-03-18T23:10:30.272 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:30.271+0000 7f0152f76740 1 -- 172.21.15.49:0/1801700675 shutdown_connections 2024-03-18T23:10:30.273 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:30.271+0000 7f0152f76740 1 --2- 172.21.15.49:0/1801700675 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x563aa6fa4a20 0x563aa6fa6e10 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:30.273 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:30.271+0000 7f0131ffb640 0 client.15639 ms_handle_reset on v2:172.21.15.139:6838/4280128209 2024-03-18T23:10:30.273 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:30.271+0000 7f0152f76740 1 --2- 172.21.15.49:0/1801700675 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563aa6f2a5e0 0x563aa6f23cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:30.273 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:30.271+0000 7f0152f76740 1 --2- 172.21.15.49:0/1801700675 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563aa6f21810 0x563aa6f237b0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:30.273 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:30.271+0000 7f0152f76740 1 --2- 172.21.15.49:0/1801700675 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563aa6f20ef0 0x563aa6f32580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:30.273 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:30.271+0000 7f0152f76740 1 -- 172.21.15.49:0/1801700675 >> 172.21.15.49:0/1801700675 conn(0x563aa6f1b810 msgr2=0x563aa6f29300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:10:30.273 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:30.271+0000 7f0152f76740 1 -- 172.21.15.49:0/1801700675 shutdown_connections 2024-03-18T23:10:30.273 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:30.271+0000 7f0152f76740 1 -- 172.21.15.49:0/1801700675 wait complete. 2024-03-18T23:10:30.273 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:30.271+0000 7f0152f76740 20 client.15639 trim_cache size 0 max 16384 2024-03-18T23:10:30.273 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:{ 2024-03-18T23:10:30.274 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "message": "", 2024-03-18T23:10:30.274 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "return_code": 0 2024-03-18T23:10:30.274 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:} 2024-03-18T23:10:30.275 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:30.272+0000 7f0150390640 1 -- 172.21.15.49:0/3976930039 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f01340a2a10 msgr2=0x7f01340a4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:10:30.275 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:30.272+0000 7f0150390640 1 --2- 172.21.15.49:0/3976930039 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f01340a2a10 0x7f01340a4ed0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f0140010c40 tx=0x7f0140017000 comp rx=0 tx=0).stop 2024-03-18T23:10:30.275 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:30.272+0000 7f0150390640 1 -- 172.21.15.49:0/3976930039 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0148069860 msgr2=0x7f01481984f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:10:30.275 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:30.272+0000 7f0150390640 1 --2- 172.21.15.49:0/3976930039 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0148069860 0x7f01481984f0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f0138005940 tx=0x7f01380042f0 comp rx=0 tx=0).stop 2024-03-18T23:10:30.275 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:30.272+0000 7f0150390640 1 -- 172.21.15.49:0/3976930039 shutdown_connections 2024-03-18T23:10:30.275 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:30.272+0000 7f0150390640 1 --2- 172.21.15.49:0/3976930039 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f01340a2a10 0x7f01340a4ed0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:30.275 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:30.272+0000 7f0150390640 1 --2- 172.21.15.49:0/3976930039 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0148107820 0x7f014819c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:30.275 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:30.272+0000 7f0150390640 1 --2- 172.21.15.49:0/3976930039 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0148069860 0x7f01481984f0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:30.275 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:30.272+0000 7f0150390640 1 --2- 172.21.15.49:0/3976930039 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0148068ea0 0x7f0148197fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:30.275 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:30.272+0000 7f0150390640 1 -- 172.21.15.49:0/3976930039 >> 172.21.15.49:0/3976930039 conn(0x7f01480774b0 msgr2=0x7f01480fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:10:30.275 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:30.273+0000 7f0150390640 1 -- 172.21.15.49:0/3976930039 shutdown_connections 2024-03-18T23:10:30.276 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:30.273+0000 7f0150390640 1 -- 172.21.15.49:0/3976930039 wait complete. 2024-03-18T23:10:30.762 INFO:tasks.workunit.client.0.smithi049.stdout: 1 216765 12.91 MB/sec warmup 92 sec latency 65.984 ms 2024-03-18T23:10:31.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:10:31 smithi049 ceph-mon[62991]: pgmap v70: 97 pgs: 97 active+clean; 59 MiB data, 707 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.0 MiB/s wr, 314 op/s 2024-03-18T23:10:31.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:10:31 smithi175 ceph-mon[67352]: pgmap v70: 97 pgs: 97 active+clean; 59 MiB data, 707 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.0 MiB/s wr, 314 op/s 2024-03-18T23:10:31.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:10:31 smithi139 ceph-mon[67282]: pgmap v70: 97 pgs: 97 active+clean; 59 MiB data, 707 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.0 MiB/s wr, 314 op/s 2024-03-18T23:10:31.762 INFO:tasks.workunit.client.0.smithi049.stdout: 1 218761 12.84 MB/sec warmup 93 sec latency 121.018 ms 2024-03-18T23:10:32.762 INFO:tasks.workunit.client.0.smithi049.stdout: 1 220632 12.84 MB/sec warmup 94 sec latency 33.055 ms 2024-03-18T23:10:33.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:10:33 smithi049 ceph-mon[62991]: pgmap v71: 97 pgs: 97 active+clean; 48 MiB data, 666 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 4.5 MiB/s wr, 525 op/s 2024-03-18T23:10:33.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:10:33 smithi175 ceph-mon[67352]: pgmap v71: 97 pgs: 97 active+clean; 48 MiB data, 666 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 4.5 MiB/s wr, 525 op/s 2024-03-18T23:10:33.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:10:33 smithi139 ceph-mon[67282]: pgmap v71: 97 pgs: 97 active+clean; 48 MiB data, 666 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 4.5 MiB/s wr, 525 op/s 2024-03-18T23:10:33.763 INFO:tasks.workunit.client.0.smithi049.stdout: 1 223630 12.89 MB/sec warmup 95 sec latency 33.656 ms 2024-03-18T23:10:34.762 INFO:tasks.workunit.client.0.smithi049.stdout: 1 225873 12.84 MB/sec warmup 96 sec latency 22.065 ms 2024-03-18T23:10:35.379 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:10:35 smithi049 ceph-mon[62991]: pgmap v72: 97 pgs: 97 active+clean; 48 MiB data, 666 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.0 MiB/s wr, 320 op/s 2024-03-18T23:10:35.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:10:35 smithi175 ceph-mon[67352]: pgmap v72: 97 pgs: 97 active+clean; 48 MiB data, 666 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.0 MiB/s wr, 320 op/s 2024-03-18T23:10:35.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:10:35 smithi139 ceph-mon[67282]: pgmap v72: 97 pgs: 97 active+clean; 48 MiB data, 666 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.0 MiB/s wr, 320 op/s 2024-03-18T23:10:35.763 INFO:tasks.workunit.client.0.smithi049.stdout: 1 227766 12.84 MB/sec warmup 97 sec latency 28.161 ms 2024-03-18T23:10:36.762 INFO:tasks.workunit.client.0.smithi049.stdout: 1 230262 12.84 MB/sec warmup 98 sec latency 75.695 ms 2024-03-18T23:10:37.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:10:37 smithi049 ceph-mon[62991]: pgmap v73: 97 pgs: 97 active+clean; 48 MiB data, 666 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.0 MiB/s wr, 320 op/s 2024-03-18T23:10:37.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:10:37 smithi175 ceph-mon[67352]: pgmap v73: 97 pgs: 97 active+clean; 48 MiB data, 666 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.0 MiB/s wr, 320 op/s 2024-03-18T23:10:37.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:10:37 smithi139 ceph-mon[67282]: pgmap v73: 97 pgs: 97 active+clean; 48 MiB data, 666 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.0 MiB/s wr, 320 op/s 2024-03-18T23:10:37.762 INFO:tasks.workunit.client.0.smithi049.stdout: 1 232195 12.82 MB/sec warmup 99 sec latency 15.687 ms 2024-03-18T23:10:38.762 INFO:tasks.workunit.client.0.smithi049.stdout: 1 234625 12.83 MB/sec warmup 100 sec latency 9.711 ms 2024-03-18T23:10:39.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:10:39 smithi049 ceph-mon[62991]: pgmap v74: 97 pgs: 97 active+clean; 45 MiB data, 686 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 3.7 MiB/s wr, 415 op/s 2024-03-18T23:10:39.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:10:39 smithi175 ceph-mon[67352]: pgmap v74: 97 pgs: 97 active+clean; 45 MiB data, 686 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 3.7 MiB/s wr, 415 op/s 2024-03-18T23:10:39.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:10:39 smithi139 ceph-mon[67282]: pgmap v74: 97 pgs: 97 active+clean; 45 MiB data, 686 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 3.7 MiB/s wr, 415 op/s 2024-03-18T23:10:39.763 INFO:tasks.workunit.client.0.smithi049.stdout: 1 237511 12.85 MB/sec warmup 101 sec latency 15.328 ms 2024-03-18T23:10:40.763 INFO:tasks.workunit.client.0.smithi049.stdout: 1 239355 12.82 MB/sec warmup 102 sec latency 25.595 ms 2024-03-18T23:10:41.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:10:41 smithi049 ceph-mon[62991]: pgmap v75: 97 pgs: 97 active+clean; 45 MiB data, 686 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 2.2 MiB/s wr, 305 op/s 2024-03-18T23:10:41.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:10:41 smithi175 ceph-mon[67352]: pgmap v75: 97 pgs: 97 active+clean; 45 MiB data, 686 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 2.2 MiB/s wr, 305 op/s 2024-03-18T23:10:41.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:10:41 smithi139 ceph-mon[67282]: pgmap v75: 97 pgs: 97 active+clean; 45 MiB data, 686 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 2.2 MiB/s wr, 305 op/s 2024-03-18T23:10:41.763 INFO:tasks.workunit.client.0.smithi049.stdout: 1 241600 12.83 MB/sec warmup 103 sec latency 55.141 ms 2024-03-18T23:10:42.763 INFO:tasks.workunit.client.0.smithi049.stdout: 1 244170 12.83 MB/sec warmup 104 sec latency 24.102 ms 2024-03-18T23:10:43.365 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:43.362+0000 7f9612390640 1 -- 172.21.15.49:0/3206542038 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f960c06a0a0 msgr2=0x7f960c06a520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:10:43.365 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:43.362+0000 7f9612390640 1 --2- 172.21.15.49:0/3206542038 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f960c06a0a0 0x7f960c06a520 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f9600009930 tx=0x7f960002bc30 comp rx=0 tx=0).stop 2024-03-18T23:10:43.365 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:43.362+0000 7f9612390640 1 -- 172.21.15.49:0/3206542038 shutdown_connections 2024-03-18T23:10:43.365 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:43.362+0000 7f9612390640 1 --2- 172.21.15.49:0/3206542038 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f960c06aa60 0x7f960c10e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:43.365 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:43.362+0000 7f9612390640 1 --2- 172.21.15.49:0/3206542038 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f960c06a0a0 0x7f960c06a520 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:43.365 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:43.362+0000 7f9612390640 1 --2- 172.21.15.49:0/3206542038 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f960c1078a0 0x7f960c107c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:43.365 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:43.362+0000 7f9612390640 1 -- 172.21.15.49:0/3206542038 >> 172.21.15.49:0/3206542038 conn(0x7f960c077560 msgr2=0x7f960c077970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:10:43.366 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:43.362+0000 7f9612390640 1 -- 172.21.15.49:0/3206542038 shutdown_connections 2024-03-18T23:10:43.366 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:43.362+0000 7f9612390640 1 -- 172.21.15.49:0/3206542038 wait complete. 2024-03-18T23:10:43.366 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:43.363+0000 7f9612390640 1 Processor -- start 2024-03-18T23:10:43.366 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:43.363+0000 7f9612390640 1 -- start start 2024-03-18T23:10:43.366 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:43.363+0000 7f9612390640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f960c06a0a0 0x7f960c19a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:10:43.366 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:43.363+0000 7f9612390640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f960c06aa60 0x7f960c19a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:10:43.366 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:43.363+0000 7f9612390640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f960c1078a0 0x7f960c19eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:10:43.366 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:43.363+0000 7f9612390640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f960c102150 con 0x7f960c1078a0 2024-03-18T23:10:43.366 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:43.363+0000 7f9612390640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f960c101fd0 con 0x7f960c06a0a0 2024-03-18T23:10:43.366 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:43.363+0000 7f9612390640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f960c1022d0 con 0x7f960c06aa60 2024-03-18T23:10:43.366 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:43.363+0000 7f961092d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f960c1078a0 0x7f960c19eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:10:43.366 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:43.363+0000 7f960bfff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f960c06a0a0 0x7f960c19a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:10:43.366 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:43.363+0000 7f960bfff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f960c06a0a0 0x7f960c19a1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.49:51770/0 (socket says 172.21.15.49:51770) 2024-03-18T23:10:43.367 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:43.363+0000 7f960b7fe640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f960c06aa60 0x7f960c19a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:10:43.367 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:43.363+0000 7f960bfff640 1 -- 172.21.15.49:0/514834255 learned_addr learned my addr 172.21.15.49:0/514834255 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:10:43.367 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:43.364+0000 7f960bfff640 1 -- 172.21.15.49:0/514834255 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f960c06aa60 msgr2=0x7f960c19a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:10:43.367 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:43.364+0000 7f960bfff640 1 --2- 172.21.15.49:0/514834255 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f960c06aa60 0x7f960c19a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:43.367 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:43.364+0000 7f960bfff640 1 -- 172.21.15.49:0/514834255 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f960c1078a0 msgr2=0x7f960c19eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:10:43.367 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:43.364+0000 7f960bfff640 1 --2- 172.21.15.49:0/514834255 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f960c1078a0 0x7f960c19eaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:43.367 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:43.364+0000 7f960bfff640 1 -- 172.21.15.49:0/514834255 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f960c19f1d0 con 0x7f960c06a0a0 2024-03-18T23:10:43.367 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:43.364+0000 7f960b7fe640 1 --2- 172.21.15.49:0/514834255 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f960c06aa60 0x7f960c19a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:10:43.367 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:43.364+0000 7f960bfff640 1 --2- 172.21.15.49:0/514834255 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f960c06a0a0 0x7f960c19a1e0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f95fc00e9f0 tx=0x7f95fc00edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:10:43.367 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:43.364+0000 7f96097fa640 1 -- 172.21.15.49:0/514834255 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95fc00ccc0 con 0x7f960c06a0a0 2024-03-18T23:10:43.367 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:43.364+0000 7f96097fa640 1 -- 172.21.15.49:0/514834255 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f95fc00ce60 con 0x7f960c06a0a0 2024-03-18T23:10:43.367 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:43.364+0000 7f9612390640 1 -- 172.21.15.49:0/514834255 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f960c19f4c0 con 0x7f960c06a0a0 2024-03-18T23:10:43.368 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:43.364+0000 7f96097fa640 1 -- 172.21.15.49:0/514834255 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95fc010690 con 0x7f960c06a0a0 2024-03-18T23:10:43.368 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:43.364+0000 7f9612390640 1 -- 172.21.15.49:0/514834255 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f960c1a6d50 con 0x7f960c06a0a0 2024-03-18T23:10:43.374 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:43.366+0000 7f96097fa640 1 -- 172.21.15.49:0/514834255 <== mon.2 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 25) ==== 137421+0+0 (secure 0 0 0) 0x7f95fc0046c0 con 0x7f960c06a0a0 2024-03-18T23:10:43.374 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:43.367+0000 7f9614f76740 1 Processor -- start 2024-03-18T23:10:43.374 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:43.367+0000 7f9614f76740 1 -- start start 2024-03-18T23:10:43.374 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:43.367+0000 7f96097fa640 1 --2- 172.21.15.49:0/514834255 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f95f40a2a10 0x7f95f40a4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:10:43.374 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:43.367+0000 7f9614f76740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5637a6fd7ef0 0x5637a6fd82d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:10:43.374 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:43.367+0000 7f9614f76740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5637a6fd8810 0x5637a6fe10a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:10:43.374 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:43.367+0000 7f9614f76740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5637a6fe15e0 0x5637a6fe39d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:10:43.374 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:43.367+0000 7f9614f76740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5637a6f3bbe0 con 0x5637a6fd8810 2024-03-18T23:10:43.375 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:43.367+0000 7f9614f76740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x5637a6ee4960 con 0x5637a6fe15e0 2024-03-18T23:10:43.375 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:43.367+0000 7f9614f76740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5637a6f37c80 con 0x5637a6fd7ef0 2024-03-18T23:10:43.375 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:43.367+0000 7f961092d640 1 --2- 172.21.15.49:0/514834255 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f95f40a2a10 0x7f95f40a4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:10:43.375 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:43.367+0000 7f96097fa640 1 -- 172.21.15.49:0/514834255 <== mon.2 v2:172.21.15.139:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f95fc0cf250 con 0x7f960c06a0a0 2024-03-18T23:10:43.375 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:43.367+0000 7f961092d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5637a6fe15e0 0x5637a6fe39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:10:43.375 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:43.367+0000 7f960bfff640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5637a6fd7ef0 0x5637a6fd82d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:10:43.375 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:43.367+0000 7f960b7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5637a6fd8810 0x5637a6fe10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:10:43.375 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:43.367+0000 7f960bfff640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5637a6fd7ef0 0x5637a6fd82d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.49:34710/0 (socket says 172.21.15.49:34710) 2024-03-18T23:10:43.375 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:43.367+0000 7f961092d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5637a6fe15e0 0x5637a6fe39d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.49:51780/0 (socket says 172.21.15.49:51780) 2024-03-18T23:10:43.375 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:43.367+0000 7f960bfff640 1 -- 172.21.15.49:0/1525511555 learned_addr learned my addr 172.21.15.49:0/1525511555 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:10:43.375 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:43.368+0000 7f960b7fe640 1 -- 172.21.15.49:0/1525511555 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5637a6fd7ef0 msgr2=0x5637a6fd82d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:10:43.375 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:43.368+0000 7f960b7fe640 1 --2- 172.21.15.49:0/1525511555 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5637a6fd7ef0 0x5637a6fd82d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:43.375 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:43.368+0000 7f960b7fe640 1 -- 172.21.15.49:0/1525511555 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5637a6fe15e0 msgr2=0x5637a6fe39d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:10:43.376 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:43.368+0000 7f960b7fe640 1 --2- 172.21.15.49:0/1525511555 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5637a6fe15e0 0x5637a6fe39d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:43.376 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:43.368+0000 7f960b7fe640 1 -- 172.21.15.49:0/1525511555 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5637a6edbd20 con 0x5637a6fd8810 2024-03-18T23:10:43.376 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:43.368+0000 7f960bfff640 1 --2- 172.21.15.49:0/1525511555 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5637a6fd7ef0 0x5637a6fd82d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:10:43.376 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:43.368+0000 7f961092d640 1 --2- 172.21.15.49:0/514834255 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f95f40a2a10 0x7f95f40a4ed0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f9604005fd0 tx=0x7f9604009e10 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:10:43.376 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:43.368+0000 7f960b7fe640 1 --2- 172.21.15.49:0/1525511555 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5637a6fd8810 0x5637a6fe10a0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f9600009230 tx=0x7f9600005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a289718b50e893f server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:10:43.376 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:43.368+0000 7f95f27fc640 1 -- 172.21.15.49:0/1525511555 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9600042070 con 0x5637a6fd8810 2024-03-18T23:10:43.376 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:43.368+0000 7f95f27fc640 1 -- 172.21.15.49:0/1525511555 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9600004a70 con 0x5637a6fd8810 2024-03-18T23:10:43.376 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:43.368+0000 7f95f27fc640 1 -- 172.21.15.49:0/1525511555 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9600004ef0 con 0x5637a6fd8810 2024-03-18T23:10:43.376 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:43.369+0000 7f9614f76740 1 -- 172.21.15.49:0/1525511555 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5637a6fd8810 msgr2=0x5637a6fe10a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:10:43.376 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:43.369+0000 7f9614f76740 1 --2- 172.21.15.49:0/1525511555 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5637a6fd8810 0x5637a6fe10a0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f9600009230 tx=0x7f9600005d50 comp rx=0 tx=0).stop 2024-03-18T23:10:43.377 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:43.369+0000 7f9614f76740 1 -- 172.21.15.49:0/1525511555 shutdown_connections 2024-03-18T23:10:43.377 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:43.369+0000 7f9614f76740 1 --2- 172.21.15.49:0/1525511555 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5637a6fe15e0 0x5637a6fe39d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:43.377 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:43.369+0000 7f9614f76740 1 --2- 172.21.15.49:0/1525511555 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5637a6fd8810 0x5637a6fe10a0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:43.377 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:43.369+0000 7f9614f76740 1 --2- 172.21.15.49:0/1525511555 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5637a6fd7ef0 0x5637a6fd82d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:43.377 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:43.369+0000 7f9614f76740 1 -- 172.21.15.49:0/1525511555 >> 172.21.15.49:0/1525511555 conn(0x5637a6fd2810 msgr2=0x5637a6fd2bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:10:43.377 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:43.369+0000 7f9614f76740 1 -- 172.21.15.49:0/1525511555 shutdown_connections 2024-03-18T23:10:43.377 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:43.369+0000 7f9614f76740 1 -- 172.21.15.49:0/1525511555 wait complete. 2024-03-18T23:10:43.377 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:43.369+0000 7f9614f76740 1 -- start start 2024-03-18T23:10:43.377 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:43.369+0000 7f9614f76740 1 Processor -- start 2024-03-18T23:10:43.377 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:43.369+0000 7f9614f76740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5637a6fd7ef0 0x5637a6fe0fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:10:43.377 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:43.370+0000 7f9614f76740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5637a6fd8810 0x5637a6fde370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:10:43.377 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:43.370+0000 7f9614f76740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5637a6fe15e0 0x5637a6fde980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:10:43.377 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:43.370+0000 7f9614f76740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5637a6fe41b0 con 0x5637a6fd7ef0 2024-03-18T23:10:43.377 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:43.370+0000 7f9614f76740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x5637a6ef7db0 con 0x5637a6fd8810 2024-03-18T23:10:43.378 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:43.370+0000 7f9614f76740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5637a6fe4330 con 0x5637a6fe15e0 2024-03-18T23:10:43.378 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:43.370+0000 7f9614f76740 10 client.0 _ll_register_callbacks cb 0x5637a6efce70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T23:10:43.378 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:43.370+0000 7f960bfff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5637a6fd7ef0 0x5637a6fe0fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:10:43.378 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:43.370+0000 7f960bfff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5637a6fd7ef0 0x5637a6fe0fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:46662/0 (socket says 172.21.15.49:46662) 2024-03-18T23:10:43.378 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:43.370+0000 7f960bfff640 1 -- 172.21.15.49:0/3161027074 learned_addr learned my addr 172.21.15.49:0/3161027074 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:10:43.378 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:43.370+0000 7f960b7fe640 1 --2- 172.21.15.49:0/3161027074 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5637a6fd8810 0x5637a6fde370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:10:43.378 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:43.370+0000 7f961092d640 1 --2- 172.21.15.49:0/3161027074 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5637a6fe15e0 0x5637a6fde980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:10:43.378 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:43.370+0000 7f960bfff640 1 -- 172.21.15.49:0/3161027074 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5637a6fe15e0 msgr2=0x5637a6fde980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:10:43.378 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:43.370+0000 7f960bfff640 1 --2- 172.21.15.49:0/3161027074 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5637a6fe15e0 0x5637a6fde980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:43.378 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:43.370+0000 7f960bfff640 1 -- 172.21.15.49:0/3161027074 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5637a6fd8810 msgr2=0x5637a6fde370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:10:43.378 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:43.370+0000 7f960bfff640 1 --2- 172.21.15.49:0/3161027074 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5637a6fd8810 0x5637a6fde370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:43.378 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:43.370+0000 7f960bfff640 1 -- 172.21.15.49:0/3161027074 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5637a6ef6c60 con 0x5637a6fd7ef0 2024-03-18T23:10:43.379 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:43.370+0000 7f960bfff640 1 --2- 172.21.15.49:0/3161027074 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5637a6fd7ef0 0x5637a6fe0fe0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f95fc00c6a0 tx=0x7f95fc09c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9990afd7edd0c580 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:10:43.379 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:43.370+0000 7f95eb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-18T23:10:43.379 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:43.370+0000 7f95eb7fe640 1 -- 172.21.15.49:0/3161027074 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95fc0b3020 con 0x5637a6fd7ef0 2024-03-18T23:10:43.379 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:43.370+0000 7f95eb7fe640 1 -- 172.21.15.49:0/3161027074 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f95fc0ccbf0 con 0x5637a6fd7ef0 2024-03-18T23:10:43.379 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:43.370+0000 7f9614f76740 1 -- 172.21.15.49:0/3161027074 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9600004a70 con 0x5637a6fd7ef0 2024-03-18T23:10:43.379 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:43.370+0000 7f95eb7fe640 1 -- 172.21.15.49:0/3161027074 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f95fc0cf050 con 0x5637a6fd7ef0 2024-03-18T23:10:43.379 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:43.370+0000 7f9614f76740 10 client.15651 fetch_fsmap learned FSMap version 29 2024-03-18T23:10:43.379 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:43.370+0000 7f9614f76740 1 -- 172.21.15.49:0/3161027074 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5637a6c50210 con 0x5637a6fd7ef0 2024-03-18T23:10:43.379 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:43.371+0000 7f95eb7fe640 1 -- 172.21.15.49:0/3161027074 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9600004a70 con 0x5637a6fd7ef0 2024-03-18T23:10:43.379 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:43.371+0000 7f95eb7fe640 1 -- 172.21.15.49:0/3161027074 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 29) ==== 8358+0+0 (secure 0 0 0) 0x7f95fc013cb0 con 0x5637a6fd7ef0 2024-03-18T23:10:43.379 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:43.371+0000 7f9614f76740 10 client.15651 fetch_fsmap finished waiting for FSMap version 29 2024-03-18T23:10:43.379 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:43.371+0000 7f9614f76740 10 client.15651 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T23:10:43.379 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:43.371+0000 7f9614f76740 20 client.15651 populate_metadata read hostname 'smithi049' 2024-03-18T23:10:43.380 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:43.371+0000 7f9614f76740 1 --2- 172.21.15.49:0/3161027074 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x5637a6ff14b0 0x5637a6ff38a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:10:43.380 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:43.371+0000 7f9614f76740 4 client.15651 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T23:10:43.380 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:43.371+0000 7f9614f76740 1 -- 172.21.15.49:0/3161027074 --> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5637a6f49120 con 0x5637a6ff14b0 2024-03-18T23:10:43.380 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:43.371+0000 7f960b7fe640 1 --2- 172.21.15.49:0/3161027074 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x5637a6ff14b0 0x5637a6ff38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:10:43.380 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:43.371+0000 7f95eb7fe640 10 client.15651 ms_handle_connect on v2:172.21.15.139:6838/4280128209 2024-03-18T23:10:43.380 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:43.371+0000 7f960b7fe640 1 --2- 172.21.15.49:0/3161027074 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x5637a6ff14b0 0x5637a6ff38a0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=13e7769d329c3ec server_cookie=a70626f2edb853c2 in_seq=0 out_seq=0 2024-03-18T23:10:43.380 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:43.373+0000 7f95eb7fe640 1 -- 172.21.15.49:0/3161027074 <== mds.0 v2:172.21.15.139:6838/4280128209 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5637a6f49120 con 0x5637a6ff14b0 2024-03-18T23:10:43.380 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:43.373+0000 7f95eb7fe640 10 client.15651 handle_command_reply: tid=0 2024-03-18T23:10:43.380 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:43.373+0000 7f9614f76740 1 client.15651 shutdown 2024-03-18T23:10:43.380 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:43.373+0000 7f9614f76740 1 -- 172.21.15.49:0/3161027074 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5637a6fd7ef0 msgr2=0x5637a6fe0fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:10:43.380 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:43.373+0000 7f9614f76740 1 --2- 172.21.15.49:0/3161027074 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5637a6fd7ef0 0x5637a6fe0fe0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f95fc00c6a0 tx=0x7f95fc09c040 comp rx=0 tx=0).stop 2024-03-18T23:10:43.380 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:43.373+0000 7f9614f76740 1 -- 172.21.15.49:0/3161027074 shutdown_connections 2024-03-18T23:10:43.380 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:43.373+0000 7f9614f76740 1 --2- 172.21.15.49:0/3161027074 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x5637a6ff14b0 0x5637a6ff38a0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:43.380 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:43.373+0000 7f9614f76740 1 --2- 172.21.15.49:0/3161027074 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5637a6fe15e0 0x5637a6fde980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:43.380 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:43.373+0000 7f9614f76740 1 --2- 172.21.15.49:0/3161027074 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5637a6fd8810 0x5637a6fde370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:43.381 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:43.373+0000 7f95eb7fe640 0 client.15651 ms_handle_reset on v2:172.21.15.139:6838/4280128209 2024-03-18T23:10:43.381 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:43.373+0000 7f9614f76740 1 --2- 172.21.15.49:0/3161027074 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5637a6fd7ef0 0x5637a6fe0fe0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:43.381 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:43.373+0000 7f9614f76740 1 -- 172.21.15.49:0/3161027074 >> 172.21.15.49:0/3161027074 conn(0x5637a6fd2810 msgr2=0x5637a6fd1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:10:43.381 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:43.373+0000 7f9614f76740 1 -- 172.21.15.49:0/3161027074 shutdown_connections 2024-03-18T23:10:43.381 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:43.373+0000 7f9614f76740 1 -- 172.21.15.49:0/3161027074 wait complete. 2024-03-18T23:10:43.381 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:43.373+0000 7f9614f76740 20 client.15651 trim_cache size 0 max 16384 2024-03-18T23:10:43.381 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:10:43 smithi049 ceph-mon[62991]: pgmap v76: 97 pgs: 97 active+clean; 50 MiB data, 750 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 3.6 MiB/s wr, 413 op/s 2024-03-18T23:10:43.393 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:43.388+0000 7f9614f76740 1 Processor -- start 2024-03-18T23:10:43.393 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:43.388+0000 7f9614f76740 1 -- start start 2024-03-18T23:10:43.393 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:43.388+0000 7f9614f76740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5637a6fd7ef0 0x5637a6fecf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:10:43.394 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:43.388+0000 7f9614f76740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5637a6fd8810 0x5637a6ff9190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:10:43.394 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:43.388+0000 7f9614f76740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5637a6fe15e0 0x5637a6fe0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:10:43.394 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:43.388+0000 7f9614f76740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f960c1af470 con 0x5637a6fd7ef0 2024-03-18T23:10:43.394 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:43.388+0000 7f9614f76740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f960c1b0fe0 con 0x5637a6fe15e0 2024-03-18T23:10:43.394 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:43.388+0000 7f9614f76740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f960c1afb40 con 0x5637a6fd8810 2024-03-18T23:10:43.394 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:43.388+0000 7f960bfff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5637a6fd7ef0 0x5637a6fecf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:10:43.394 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:43.388+0000 7f960bfff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5637a6fd7ef0 0x5637a6fecf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:46674/0 (socket says 172.21.15.49:46674) 2024-03-18T23:10:43.394 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:43.388+0000 7f960bfff640 1 -- 172.21.15.49:0/1347133648 learned_addr learned my addr 172.21.15.49:0/1347133648 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:10:43.394 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:43.388+0000 7f960b7fe640 1 --2- 172.21.15.49:0/1347133648 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5637a6fd8810 0x5637a6ff9190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:10:43.394 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:43.388+0000 7f961092d640 1 --2- 172.21.15.49:0/1347133648 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5637a6fe15e0 0x5637a6fe0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:10:43.394 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:43.388+0000 7f960bfff640 1 -- 172.21.15.49:0/1347133648 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5637a6fd8810 msgr2=0x5637a6ff9190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:10:43.394 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:43.388+0000 7f960bfff640 1 --2- 172.21.15.49:0/1347133648 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5637a6fd8810 0x5637a6ff9190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:43.394 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:43.389+0000 7f960bfff640 1 -- 172.21.15.49:0/1347133648 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5637a6fe15e0 msgr2=0x5637a6fe0aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:10:43.394 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:43.389+0000 7f960bfff640 1 --2- 172.21.15.49:0/1347133648 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5637a6fe15e0 0x5637a6fe0aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:43.395 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:43.389+0000 7f960bfff640 1 -- 172.21.15.49:0/1347133648 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5637a6fdecf0 con 0x5637a6fd7ef0 2024-03-18T23:10:43.395 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:43.389+0000 7f960b7fe640 1 --2- 172.21.15.49:0/1347133648 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5637a6fd8810 0x5637a6ff9190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T23:10:43.395 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:43.389+0000 7f960bfff640 1 --2- 172.21.15.49:0/1347133648 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5637a6fd7ef0 0x5637a6fecf50 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f95fc09b580 tx=0x7f95fc09b780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f18f2e2f98097225 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:10:43.395 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:43.389+0000 7f95f1ffb640 1 -- 172.21.15.49:0/1347133648 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95fc0b3040 con 0x5637a6fd7ef0 2024-03-18T23:10:43.395 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:43.389+0000 7f95f1ffb640 1 -- 172.21.15.49:0/1347133648 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f95fc010830 con 0x5637a6fd7ef0 2024-03-18T23:10:43.395 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:43.389+0000 7f95f1ffb640 1 -- 172.21.15.49:0/1347133648 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95fc097b20 con 0x5637a6fd7ef0 2024-03-18T23:10:43.395 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:43.389+0000 7f9614f76740 1 -- 172.21.15.49:0/1347133648 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5637a6fd7ef0 msgr2=0x5637a6fecf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:10:43.395 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:43.389+0000 7f9614f76740 1 --2- 172.21.15.49:0/1347133648 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5637a6fd7ef0 0x5637a6fecf50 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f95fc09b580 tx=0x7f95fc09b780 comp rx=0 tx=0).stop 2024-03-18T23:10:43.395 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:43.389+0000 7f9614f76740 1 -- 172.21.15.49:0/1347133648 shutdown_connections 2024-03-18T23:10:43.395 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:43.389+0000 7f9614f76740 1 --2- 172.21.15.49:0/1347133648 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5637a6fe15e0 0x5637a6fe0aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:43.395 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:43.389+0000 7f9614f76740 1 --2- 172.21.15.49:0/1347133648 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5637a6fd8810 0x5637a6ff9190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:43.395 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:43.389+0000 7f9614f76740 1 --2- 172.21.15.49:0/1347133648 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5637a6fd7ef0 0x5637a6fecf50 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:43.395 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:43.389+0000 7f9614f76740 1 -- 172.21.15.49:0/1347133648 >> 172.21.15.49:0/1347133648 conn(0x5637a6fd2810 msgr2=0x5637a6fd1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:10:43.395 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:43.389+0000 7f9614f76740 1 -- 172.21.15.49:0/1347133648 shutdown_connections 2024-03-18T23:10:43.396 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:43.390+0000 7f9614f76740 1 -- 172.21.15.49:0/1347133648 wait complete. 2024-03-18T23:10:43.396 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:43.390+0000 7f9614f76740 1 -- start start 2024-03-18T23:10:43.396 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:43.390+0000 7f9614f76740 1 Processor -- start 2024-03-18T23:10:43.396 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:43.390+0000 7f9614f76740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5637a6fd7ef0 0x5637a6fe56f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:10:43.396 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:43.390+0000 7f9614f76740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5637a6fd8810 0x5637a6fe5c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:10:43.396 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:43.390+0000 7f9614f76740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5637a6fe15e0 0x5637a7058fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:10:43.396 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:43.390+0000 7f9614f76740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f960c1b08e0 con 0x5637a6fd7ef0 2024-03-18T23:10:43.396 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:43.390+0000 7f9614f76740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f960c1b0210 con 0x5637a6fe15e0 2024-03-18T23:10:43.396 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:43.390+0000 7f9614f76740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f960c1ad450 con 0x5637a6fd8810 2024-03-18T23:10:43.396 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:43.390+0000 7f9614f76740 10 client.0 _ll_register_callbacks cb 0x7f95fc0cc910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T23:10:43.396 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:43.390+0000 7f960bfff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5637a6fd7ef0 0x5637a6fe56f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:10:43.396 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:43.390+0000 7f960bfff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5637a6fd7ef0 0x5637a6fe56f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:46680/0 (socket says 172.21.15.49:46680) 2024-03-18T23:10:43.396 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:43.390+0000 7f960bfff640 1 -- 172.21.15.49:0/1247166723 learned_addr learned my addr 172.21.15.49:0/1247166723 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:10:43.396 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:43.390+0000 7f961092d640 1 --2- 172.21.15.49:0/1247166723 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5637a6fe15e0 0x5637a7058fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:10:43.396 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:43.390+0000 7f960b7fe640 1 --2- 172.21.15.49:0/1247166723 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5637a6fd8810 0x5637a6fe5c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:10:43.397 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:43.391+0000 7f960bfff640 1 -- 172.21.15.49:0/1247166723 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5637a6fd8810 msgr2=0x5637a6fe5c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:10:43.397 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:43.391+0000 7f960bfff640 1 --2- 172.21.15.49:0/1247166723 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5637a6fd8810 0x5637a6fe5c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:43.397 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:43.391+0000 7f960bfff640 1 -- 172.21.15.49:0/1247166723 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5637a6fe15e0 msgr2=0x5637a7058fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:10:43.397 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:43.391+0000 7f960bfff640 1 --2- 172.21.15.49:0/1247166723 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5637a6fe15e0 0x5637a7058fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:43.397 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:43.391+0000 7f960bfff640 1 -- 172.21.15.49:0/1247166723 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5637a6ff3c10 con 0x5637a6fd7ef0 2024-03-18T23:10:43.397 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:43.391+0000 7f960bfff640 1 --2- 172.21.15.49:0/1247166723 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5637a6fd7ef0 0x5637a6fe56f0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f95fc097ce0 tx=0x7f95fc0cce80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb577d13ec4f37bb server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:10:43.397 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:43.391+0000 7f95eb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-18T23:10:43.397 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:43.391+0000 7f95eb7fe640 1 -- 172.21.15.49:0/1247166723 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95fc0b3070 con 0x5637a6fd7ef0 2024-03-18T23:10:43.397 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:43.391+0000 7f95eb7fe640 1 -- 172.21.15.49:0/1247166723 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f95fc09cbb0 con 0x5637a6fd7ef0 2024-03-18T23:10:43.397 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:43.391+0000 7f9614f76740 1 -- 172.21.15.49:0/1247166723 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f95fc010830 con 0x5637a6fd7ef0 2024-03-18T23:10:43.397 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:43.391+0000 7f95eb7fe640 1 -- 172.21.15.49:0/1247166723 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f95fc097b20 con 0x5637a6fd7ef0 2024-03-18T23:10:43.397 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:43.391+0000 7f9614f76740 10 client.15657 fetch_fsmap learned FSMap version 29 2024-03-18T23:10:43.397 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:43.391+0000 7f9614f76740 1 -- 172.21.15.49:0/1247166723 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5637a6d1c620 con 0x5637a6fd7ef0 2024-03-18T23:10:43.397 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:43.391+0000 7f95eb7fe640 1 -- 172.21.15.49:0/1247166723 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f95fc010830 con 0x5637a6fd7ef0 2024-03-18T23:10:43.398 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:43.392+0000 7f95eb7fe640 1 -- 172.21.15.49:0/1247166723 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 29) ==== 8358+0+0 (secure 0 0 0) 0x7f95fc09ad60 con 0x5637a6fd7ef0 2024-03-18T23:10:43.398 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:43.392+0000 7f9614f76740 10 client.15657 fetch_fsmap finished waiting for FSMap version 29 2024-03-18T23:10:43.398 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:43.392+0000 7f9614f76740 10 client.15657 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T23:10:43.398 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:43.392+0000 7f9614f76740 20 client.15657 populate_metadata read hostname 'smithi049' 2024-03-18T23:10:43.398 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:43.392+0000 7f9614f76740 1 --2- 172.21.15.49:0/1247166723 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x5637a7060a00 0x5637a7062df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:10:43.398 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:43.392+0000 7f9614f76740 4 client.15657 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T23:10:43.398 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:43.392+0000 7f9614f76740 1 -- 172.21.15.49:0/1247166723 --> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5637a6f43470 con 0x5637a7060a00 2024-03-18T23:10:43.398 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:43.392+0000 7f960b7fe640 1 --2- 172.21.15.49:0/1247166723 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x5637a7060a00 0x5637a7062df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:10:43.398 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:43.392+0000 7f95eb7fe640 10 client.15657 ms_handle_connect on v2:172.21.15.139:6838/4280128209 2024-03-18T23:10:43.398 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:43.392+0000 7f960b7fe640 1 --2- 172.21.15.49:0/1247166723 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x5637a7060a00 0x5637a7062df0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4f3f49fa4b27e9cc server_cookie=213d7ef013bc26c5 in_seq=0 out_seq=0 2024-03-18T23:10:43.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:10:43 smithi175 ceph-mon[67352]: pgmap v76: 97 pgs: 97 active+clean; 50 MiB data, 750 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 3.6 MiB/s wr, 413 op/s 2024-03-18T23:10:43.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:10:43 smithi139 ceph-mon[67282]: pgmap v76: 97 pgs: 97 active+clean; 50 MiB data, 750 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 3.6 MiB/s wr, 413 op/s 2024-03-18T23:10:43.673 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:43.672+0000 7f95eb7fe640 1 -- 172.21.15.49:0/1247166723 <== mds.0 v2:172.21.15.139:6838/4280128209 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5637a6f43470 con 0x5637a7060a00 2024-03-18T23:10:43.673 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:43.672+0000 7f95eb7fe640 10 client.15657 handle_command_reply: tid=0 2024-03-18T23:10:43.673 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:43.672+0000 7f9614f76740 1 client.15657 shutdown 2024-03-18T23:10:43.674 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:43.672+0000 7f9614f76740 1 -- 172.21.15.49:0/1247166723 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5637a6fd7ef0 msgr2=0x5637a6fe56f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:10:43.674 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:43.672+0000 7f9614f76740 1 --2- 172.21.15.49:0/1247166723 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5637a6fd7ef0 0x5637a6fe56f0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f95fc097ce0 tx=0x7f95fc0cce80 comp rx=0 tx=0).stop 2024-03-18T23:10:43.674 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:43.672+0000 7f9614f76740 1 -- 172.21.15.49:0/1247166723 shutdown_connections 2024-03-18T23:10:43.674 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:43.672+0000 7f9614f76740 1 --2- 172.21.15.49:0/1247166723 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x5637a7060a00 0x5637a7062df0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:43.674 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:43.672+0000 7f9614f76740 1 --2- 172.21.15.49:0/1247166723 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5637a6fe15e0 0x5637a7058fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:43.674 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:43.672+0000 7f9614f76740 1 --2- 172.21.15.49:0/1247166723 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5637a6fd8810 0x5637a6fe5c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:43.674 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:43.672+0000 7f95eb7fe640 0 client.15657 ms_handle_reset on v2:172.21.15.139:6838/4280128209 2024-03-18T23:10:43.674 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:43.672+0000 7f9614f76740 1 --2- 172.21.15.49:0/1247166723 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5637a6fd7ef0 0x5637a6fe56f0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:43.674 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:43.672+0000 7f9614f76740 1 -- 172.21.15.49:0/1247166723 >> 172.21.15.49:0/1247166723 conn(0x5637a6fd2810 msgr2=0x5637a6fd3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:10:43.674 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:43.672+0000 7f9614f76740 1 -- 172.21.15.49:0/1247166723 shutdown_connections 2024-03-18T23:10:43.674 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:43.672+0000 7f9614f76740 1 -- 172.21.15.49:0/1247166723 wait complete. 2024-03-18T23:10:43.674 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:43.672+0000 7f9614f76740 20 client.15657 trim_cache size 0 max 16384 2024-03-18T23:10:43.675 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:{ 2024-03-18T23:10:43.675 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "message": "", 2024-03-18T23:10:43.675 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "return_code": 0 2024-03-18T23:10:43.675 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:} 2024-03-18T23:10:43.676 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:43.673+0000 7f9612390640 1 -- 172.21.15.49:0/514834255 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f95f40a2a10 msgr2=0x7f95f40a4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:10:43.676 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:43.673+0000 7f9612390640 1 --2- 172.21.15.49:0/514834255 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f95f40a2a10 0x7f95f40a4ed0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f9604005fd0 tx=0x7f9604009e10 comp rx=0 tx=0).stop 2024-03-18T23:10:43.676 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:43.673+0000 7f9612390640 1 -- 172.21.15.49:0/514834255 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f960c06a0a0 msgr2=0x7f960c19a1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:10:43.676 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:43.673+0000 7f9612390640 1 --2- 172.21.15.49:0/514834255 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f960c06a0a0 0x7f960c19a1e0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f95fc00e9f0 tx=0x7f95fc00edf0 comp rx=0 tx=0).stop 2024-03-18T23:10:43.676 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:43.673+0000 7f9612390640 1 -- 172.21.15.49:0/514834255 shutdown_connections 2024-03-18T23:10:43.676 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:43.673+0000 7f9612390640 1 --2- 172.21.15.49:0/514834255 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f95f40a2a10 0x7f95f40a4ed0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:43.676 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:43.673+0000 7f9612390640 1 --2- 172.21.15.49:0/514834255 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f960c1078a0 0x7f960c19eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:43.676 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:43.673+0000 7f9612390640 1 --2- 172.21.15.49:0/514834255 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f960c06aa60 0x7f960c19a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:43.676 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:43.673+0000 7f9612390640 1 --2- 172.21.15.49:0/514834255 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f960c06a0a0 0x7f960c19a1e0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:43.676 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:43.673+0000 7f9612390640 1 -- 172.21.15.49:0/514834255 >> 172.21.15.49:0/514834255 conn(0x7f960c077560 msgr2=0x7f960c0fdfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:10:43.676 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:43.673+0000 7f9612390640 1 -- 172.21.15.49:0/514834255 shutdown_connections 2024-03-18T23:10:43.677 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:43.673+0000 7f9612390640 1 -- 172.21.15.49:0/514834255 wait complete. 2024-03-18T23:10:43.763 INFO:tasks.workunit.client.0.smithi049.stdout: 1 245606 12.77 MB/sec warmup 105 sec latency 83.298 ms 2024-03-18T23:10:44.763 INFO:tasks.workunit.client.0.smithi049.stdout: 1 248570 12.82 MB/sec warmup 106 sec latency 10.058 ms 2024-03-18T23:10:45.366 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:10:45 smithi139 ceph-mon[67282]: pgmap v77: 97 pgs: 97 active+clean; 50 MiB data, 750 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.1 MiB/s wr, 203 op/s 2024-03-18T23:10:45.379 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:10:45 smithi049 ceph-mon[62991]: pgmap v77: 97 pgs: 97 active+clean; 50 MiB data, 750 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.1 MiB/s wr, 203 op/s 2024-03-18T23:10:45.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:10:45 smithi175 ceph-mon[67352]: pgmap v77: 97 pgs: 97 active+clean; 50 MiB data, 750 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.1 MiB/s wr, 203 op/s 2024-03-18T23:10:45.763 INFO:tasks.workunit.client.0.smithi049.stdout: 1 250936 12.78 MB/sec warmup 107 sec latency 18.711 ms 2024-03-18T23:10:46.763 INFO:tasks.workunit.client.0.smithi049.stdout: 1 252660 12.77 MB/sec warmup 108 sec latency 81.878 ms 2024-03-18T23:10:47.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:10:47 smithi049 ceph-mon[62991]: pgmap v78: 97 pgs: 97 active+clean; 50 MiB data, 750 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.1 MiB/s wr, 203 op/s 2024-03-18T23:10:47.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:10:47 smithi175 ceph-mon[67352]: pgmap v78: 97 pgs: 97 active+clean; 50 MiB data, 750 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.1 MiB/s wr, 203 op/s 2024-03-18T23:10:47.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:10:47 smithi139 ceph-mon[67282]: pgmap v78: 97 pgs: 97 active+clean; 50 MiB data, 750 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.1 MiB/s wr, 203 op/s 2024-03-18T23:10:47.763 INFO:tasks.workunit.client.0.smithi049.stdout: 1 255484 12.81 MB/sec warmup 109 sec latency 50.427 ms 2024-03-18T23:10:48.763 INFO:tasks.workunit.client.0.smithi049.stdout: 1 257116 12.76 MB/sec warmup 110 sec latency 27.046 ms 2024-03-18T23:10:49.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:10:49 smithi049 ceph-mon[62991]: pgmap v79: 97 pgs: 97 active+clean; 43 MiB data, 765 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 3.7 MiB/s wr, 411 op/s 2024-03-18T23:10:49.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:10:49 smithi175 ceph-mon[67352]: pgmap v79: 97 pgs: 97 active+clean; 43 MiB data, 765 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 3.7 MiB/s wr, 411 op/s 2024-03-18T23:10:49.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:10:49 smithi139 ceph-mon[67282]: pgmap v79: 97 pgs: 97 active+clean; 43 MiB data, 765 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 3.7 MiB/s wr, 411 op/s 2024-03-18T23:10:49.763 INFO:tasks.workunit.client.0.smithi049.stdout: 1 259539 12.77 MB/sec warmup 111 sec latency 17.496 ms 2024-03-18T23:10:50.763 INFO:tasks.workunit.client.0.smithi049.stdout: 1 262494 12.81 MB/sec warmup 112 sec latency 23.116 ms 2024-03-18T23:10:51.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:10:51 smithi049 ceph-mon[62991]: pgmap v80: 97 pgs: 97 active+clean; 43 MiB data, 765 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.0 MiB/s wr, 316 op/s 2024-03-18T23:10:51.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:10:51 smithi175 ceph-mon[67352]: pgmap v80: 97 pgs: 97 active+clean; 43 MiB data, 765 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.0 MiB/s wr, 316 op/s 2024-03-18T23:10:51.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:10:51 smithi139 ceph-mon[67282]: pgmap v80: 97 pgs: 97 active+clean; 43 MiB data, 765 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.0 MiB/s wr, 316 op/s 2024-03-18T23:10:51.764 INFO:tasks.workunit.client.0.smithi049.stdout: 1 264121 12.75 MB/sec warmup 113 sec latency 23.991 ms 2024-03-18T23:10:52.763 INFO:tasks.workunit.client.0.smithi049.stdout: 1 266378 12.76 MB/sec warmup 114 sec latency 113.745 ms 2024-03-18T23:10:53.379 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:10:53 smithi049 ceph-mon[62991]: pgmap v81: 97 pgs: 97 active+clean; 48 MiB data, 785 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 4.4 MiB/s wr, 421 op/s 2024-03-18T23:10:53.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:10:53 smithi175 ceph-mon[67352]: pgmap v81: 97 pgs: 97 active+clean; 48 MiB data, 785 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 4.4 MiB/s wr, 421 op/s 2024-03-18T23:10:53.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:10:53 smithi139 ceph-mon[67282]: pgmap v81: 97 pgs: 97 active+clean; 48 MiB data, 785 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 4.4 MiB/s wr, 421 op/s 2024-03-18T23:10:53.763 INFO:tasks.workunit.client.0.smithi049.stdout: 1 268827 12.75 MB/sec warmup 115 sec latency 18.179 ms 2024-03-18T23:10:54.763 INFO:tasks.workunit.client.0.smithi049.stdout: 1 270710 12.73 MB/sec warmup 116 sec latency 16.559 ms 2024-03-18T23:10:55.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:10:55 smithi049 ceph-mon[62991]: pgmap v82: 97 pgs: 97 active+clean; 48 MiB data, 785 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.0 MiB/s wr, 312 op/s 2024-03-18T23:10:55.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:10:55 smithi175 ceph-mon[67352]: pgmap v82: 97 pgs: 97 active+clean; 48 MiB data, 785 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.0 MiB/s wr, 312 op/s 2024-03-18T23:10:55.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:10:55 smithi139 ceph-mon[67282]: pgmap v82: 97 pgs: 97 active+clean; 48 MiB data, 785 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.0 MiB/s wr, 312 op/s 2024-03-18T23:10:55.763 INFO:tasks.workunit.client.0.smithi049.stdout: 1 273447 12.76 MB/sec warmup 117 sec latency 20.319 ms 2024-03-18T23:10:56.763 INFO:tasks.workunit.client.0.smithi049.stdout: 1 275747 12.72 MB/sec warmup 118 sec latency 12.320 ms 2024-03-18T23:10:56.767 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:56.767+0000 7fa4a96dc640 1 -- 172.21.15.49:0/1624219397 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa4a41021d0 msgr2=0x7fa4a410e6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:10:56.767 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:56.767+0000 7fa4a96dc640 1 --2- 172.21.15.49:0/1624219397 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa4a41021d0 0x7fa4a410e6e0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fa494009930 tx=0x7fa49402bc30 comp rx=0 tx=0).stop 2024-03-18T23:10:56.767 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:56.767+0000 7fa4a96dc640 1 -- 172.21.15.49:0/1624219397 shutdown_connections 2024-03-18T23:10:56.767 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:56.767+0000 7fa4a96dc640 1 --2- 172.21.15.49:0/1624219397 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa4a41021d0 0x7fa4a410e6e0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:56.767 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:56.767+0000 7fa4a96dc640 1 --2- 172.21.15.49:0/1624219397 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa4a4101810 0x7fa4a4101c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:56.767 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:56.767+0000 7fa4a96dc640 1 --2- 172.21.15.49:0/1624219397 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa4a4107820 0x7fa4a4107c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:56.767 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:56.767+0000 7fa4a96dc640 1 -- 172.21.15.49:0/1624219397 >> 172.21.15.49:0/1624219397 conn(0x7fa4a40fd2e0 msgr2=0x7fa4a40ff700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:10:56.767 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:56.767+0000 7fa4a96dc640 1 -- 172.21.15.49:0/1624219397 shutdown_connections 2024-03-18T23:10:56.768 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:56.767+0000 7fa4a96dc640 1 -- 172.21.15.49:0/1624219397 wait complete. 2024-03-18T23:10:56.768 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:56.768+0000 7fa4a96dc640 1 Processor -- start 2024-03-18T23:10:56.768 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:56.769+0000 7fa4a96dc640 1 -- start start 2024-03-18T23:10:56.769 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:56.769+0000 7fa4a96dc640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa4a4101810 0x7fa4a419a220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:10:56.769 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:56.769+0000 7fa4a96dc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa4a41021d0 0x7fa4a419a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:10:56.769 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:56.769+0000 7fa4a96dc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa4a4107820 0x7fa4a419eae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:10:56.769 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:56.769+0000 7fa4a96dc640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fa4a4111c30 con 0x7fa4a4107820 2024-03-18T23:10:56.769 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:56.769+0000 7fa4a96dc640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fa4a4111ab0 con 0x7fa4a41021d0 2024-03-18T23:10:56.769 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:56.769+0000 7fa4a96dc640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa4a4111db0 con 0x7fa4a4101810 2024-03-18T23:10:56.769 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:56.769+0000 7fa4a2ffd640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa4a4101810 0x7fa4a419a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:10:56.769 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:56.769+0000 7fa4a27fc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa4a41021d0 0x7fa4a419a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:10:56.770 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:56.769+0000 7fa4a2ffd640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa4a4101810 0x7fa4a419a220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.49:56852/0 (socket says 172.21.15.49:56852) 2024-03-18T23:10:56.770 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:56.769+0000 7fa4a2ffd640 1 -- 172.21.15.49:0/3165281464 learned_addr learned my addr 172.21.15.49:0/3165281464 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:10:56.770 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:56.769+0000 7fa4a37fe640 1 --2- 172.21.15.49:0/3165281464 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa4a4107820 0x7fa4a419eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:10:56.771 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:56.770+0000 7fa4a2ffd640 1 -- 172.21.15.49:0/3165281464 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa4a41021d0 msgr2=0x7fa4a419a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:10:56.771 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:56.770+0000 7fa4a2ffd640 1 --2- 172.21.15.49:0/3165281464 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa4a41021d0 0x7fa4a419a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:56.771 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:56.770+0000 7fa4a2ffd640 1 -- 172.21.15.49:0/3165281464 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa4a4107820 msgr2=0x7fa4a419eae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:10:56.771 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:56.770+0000 7fa4a2ffd640 1 --2- 172.21.15.49:0/3165281464 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa4a4107820 0x7fa4a419eae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:56.771 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:56.770+0000 7fa4a2ffd640 1 -- 172.21.15.49:0/3165281464 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4a419f210 con 0x7fa4a4101810 2024-03-18T23:10:56.771 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:56.770+0000 7fa4a2ffd640 1 --2- 172.21.15.49:0/3165281464 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa4a4101810 0x7fa4a419a220 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fa49c00eb90 tx=0x7fa49c00c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:10:56.771 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:56.770+0000 7fa48bfff640 1 -- 172.21.15.49:0/3165281464 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa49c004d20 con 0x7fa4a4101810 2024-03-18T23:10:56.771 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:56.770+0000 7fa48bfff640 1 -- 172.21.15.49:0/3165281464 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa49c00ee90 con 0x7fa4a4101810 2024-03-18T23:10:56.771 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:56.770+0000 7fa48bfff640 1 -- 172.21.15.49:0/3165281464 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa49c010830 con 0x7fa4a4101810 2024-03-18T23:10:56.771 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:56.771+0000 7fa4a96dc640 1 -- 172.21.15.49:0/3165281464 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4a419f500 con 0x7fa4a4101810 2024-03-18T23:10:56.771 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:56.771+0000 7fa4a96dc640 1 -- 172.21.15.49:0/3165281464 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4a41a6d90 con 0x7fa4a4101810 2024-03-18T23:10:56.773 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:56.774+0000 7fa48bfff640 1 -- 172.21.15.49:0/3165281464 <== mon.1 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 25) ==== 137421+0+0 (secure 0 0 0) 0x7fa49c005020 con 0x7fa4a4101810 2024-03-18T23:10:56.780 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:56.774+0000 7fa4ac3a2740 1 Processor -- start 2024-03-18T23:10:56.780 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:56.774+0000 7fa4ac3a2740 1 -- start start 2024-03-18T23:10:56.780 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:56.775+0000 7fa48bfff640 1 --2- 172.21.15.49:0/3165281464 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7fa48c0a2a90 0x7fa48c0a4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:10:56.780 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:56.775+0000 7fa4ac3a2740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56271f697ef0 0x56271f6982d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:10:56.780 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:56.775+0000 7fa4ac3a2740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56271f698810 0x56271f6a10a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:10:56.780 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:56.775+0000 7fa4ac3a2740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56271f6a15e0 0x56271f6a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:10:56.780 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:56.775+0000 7fa4ac3a2740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x56271f5fbbe0 con 0x56271f6a15e0 2024-03-18T23:10:56.780 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:56.775+0000 7fa4ac3a2740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x56271f5a4960 con 0x56271f697ef0 2024-03-18T23:10:56.780 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:56.775+0000 7fa4ac3a2740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x56271f5f7c80 con 0x56271f698810 2024-03-18T23:10:56.780 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:56.775+0000 7fa48bfff640 1 -- 172.21.15.49:0/3165281464 <== mon.1 v2:172.21.15.175:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7fa49c0cf350 con 0x7fa4a4101810 2024-03-18T23:10:56.781 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:56.775+0000 7fa4a37fe640 1 --2- 172.21.15.49:0/3165281464 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7fa48c0a2a90 0x7fa48c0a4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:10:56.781 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:56.775+0000 7fa4a2ffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56271f697ef0 0x56271f6982d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:10:56.781 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:56.775+0000 7fa4a27fc640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56271f698810 0x56271f6a10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:10:56.781 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:56.775+0000 7fa4a27fc640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56271f698810 0x56271f6a10a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.49:56866/0 (socket says 172.21.15.49:56866) 2024-03-18T23:10:56.781 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:56.775+0000 7fa4a27fc640 1 -- 172.21.15.49:0/3865199867 learned_addr learned my addr 172.21.15.49:0/3865199867 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:10:56.781 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:56.775+0000 7fa4a2ffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56271f697ef0 0x56271f6982d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.49:42850/0 (socket says 172.21.15.49:42850) 2024-03-18T23:10:56.781 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:56.775+0000 7fa4a37fe640 1 --2- 172.21.15.49:0/3865199867 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56271f6a15e0 0x56271f6a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:10:56.781 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:56.775+0000 7fa4a2ffd640 1 -- 172.21.15.49:0/3865199867 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56271f698810 msgr2=0x56271f6a10a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:10:56.781 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:56.775+0000 7fa4a2ffd640 1 --2- 172.21.15.49:0/3865199867 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56271f698810 0x56271f6a10a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:56.781 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:56.775+0000 7fa4a2ffd640 1 -- 172.21.15.49:0/3865199867 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56271f6a15e0 msgr2=0x56271f6a39d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:10:56.781 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:56.775+0000 7fa4a2ffd640 1 --2- 172.21.15.49:0/3865199867 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56271f6a15e0 0x56271f6a39d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:56.781 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:56.775+0000 7fa4a2ffd640 1 -- 172.21.15.49:0/3865199867 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56271f59bd20 con 0x56271f697ef0 2024-03-18T23:10:56.781 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:56.775+0000 7fa4a27fc640 1 --2- 172.21.15.49:0/3865199867 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56271f698810 0x56271f6a10a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:10:56.781 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:56.775+0000 7fa4a37fe640 1 --2- 172.21.15.49:0/3165281464 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7fa48c0a2a90 0x7fa48c0a4f50 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fa494005940 tx=0x7fa494009680 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:10:56.782 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:56.775+0000 7fa4a2ffd640 1 --2- 172.21.15.49:0/3865199867 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56271f697ef0 0x56271f6982d0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fa49c0ccad0 tx=0x7fa49c097d40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7b55325fbf697681 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:10:56.782 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:56.776+0000 7fa4897fa640 1 -- 172.21.15.49:0/3865199867 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa49c09b340 con 0x56271f697ef0 2024-03-18T23:10:56.782 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:56.776+0000 7fa4897fa640 1 -- 172.21.15.49:0/3865199867 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa49c09c0e0 con 0x56271f697ef0 2024-03-18T23:10:56.782 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:56.776+0000 7fa4897fa640 1 -- 172.21.15.49:0/3865199867 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa49c01c1c0 con 0x56271f697ef0 2024-03-18T23:10:56.782 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:56.776+0000 7fa4ac3a2740 1 -- 172.21.15.49:0/3865199867 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56271f697ef0 msgr2=0x56271f6982d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:10:56.782 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:56.776+0000 7fa4ac3a2740 1 --2- 172.21.15.49:0/3865199867 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56271f697ef0 0x56271f6982d0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fa49c0ccad0 tx=0x7fa49c097d40 comp rx=0 tx=0).stop 2024-03-18T23:10:56.782 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:56.776+0000 7fa4ac3a2740 1 -- 172.21.15.49:0/3865199867 shutdown_connections 2024-03-18T23:10:56.782 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:56.776+0000 7fa4ac3a2740 1 --2- 172.21.15.49:0/3865199867 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56271f6a15e0 0x56271f6a39d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:56.782 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:56.776+0000 7fa4ac3a2740 1 --2- 172.21.15.49:0/3865199867 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56271f698810 0x56271f6a10a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:56.782 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:56.776+0000 7fa4ac3a2740 1 --2- 172.21.15.49:0/3865199867 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56271f697ef0 0x56271f6982d0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:56.782 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:56.776+0000 7fa4ac3a2740 1 -- 172.21.15.49:0/3865199867 >> 172.21.15.49:0/3865199867 conn(0x56271f692810 msgr2=0x56271f692bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:10:56.782 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:56.777+0000 7fa4ac3a2740 1 -- 172.21.15.49:0/3865199867 shutdown_connections 2024-03-18T23:10:56.782 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:56.777+0000 7fa4ac3a2740 1 -- 172.21.15.49:0/3865199867 wait complete. 2024-03-18T23:10:56.783 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:56.777+0000 7fa4ac3a2740 1 -- start start 2024-03-18T23:10:56.783 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:56.777+0000 7fa4ac3a2740 1 Processor -- start 2024-03-18T23:10:56.783 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:56.777+0000 7fa4ac3a2740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56271f697ef0 0x56271f6a0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:10:56.783 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:56.777+0000 7fa4ac3a2740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56271f698810 0x56271f69e350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:10:56.783 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:56.777+0000 7fa4ac3a2740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56271f6a15e0 0x56271f69e960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:10:56.783 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:56.777+0000 7fa4ac3a2740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x56271f6a4210 con 0x56271f697ef0 2024-03-18T23:10:56.783 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:56.777+0000 7fa4ac3a2740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x56271f5b7db0 con 0x56271f698810 2024-03-18T23:10:56.783 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:56.777+0000 7fa4ac3a2740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x56271f6a4390 con 0x56271f6a15e0 2024-03-18T23:10:56.783 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:56.777+0000 7fa4ac3a2740 10 client.0 _ll_register_callbacks cb 0x56271f5bce70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T23:10:56.783 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:56.777+0000 7fa4a2ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56271f697ef0 0x56271f6a0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:10:56.783 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:56.777+0000 7fa4a2ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56271f697ef0 0x56271f6a0fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:33374/0 (socket says 172.21.15.49:33374) 2024-03-18T23:10:56.783 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:56.777+0000 7fa4a2ffd640 1 -- 172.21.15.49:0/3309435596 learned_addr learned my addr 172.21.15.49:0/3309435596 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:10:56.783 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:56.777+0000 7fa4a27fc640 1 --2- 172.21.15.49:0/3309435596 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56271f698810 0x56271f69e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:10:56.783 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:56.777+0000 7fa4a37fe640 1 --2- 172.21.15.49:0/3309435596 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56271f6a15e0 0x56271f69e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:10:56.784 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:56.777+0000 7fa4a2ffd640 1 -- 172.21.15.49:0/3309435596 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56271f6a15e0 msgr2=0x56271f69e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:10:56.784 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:56.778+0000 7fa4a2ffd640 1 --2- 172.21.15.49:0/3309435596 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56271f6a15e0 0x56271f69e960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:56.784 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:56.778+0000 7fa4a2ffd640 1 -- 172.21.15.49:0/3309435596 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56271f698810 msgr2=0x56271f69e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:10:56.784 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:56.778+0000 7fa4a2ffd640 1 --2- 172.21.15.49:0/3309435596 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56271f698810 0x56271f69e350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:56.784 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:56.778+0000 7fa4a2ffd640 1 -- 172.21.15.49:0/3309435596 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56271f5b6c60 con 0x56271f697ef0 2024-03-18T23:10:56.784 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:56.778+0000 7fa4a2ffd640 1 --2- 172.21.15.49:0/3309435596 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56271f697ef0 0x56271f6a0fc0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fa49c099c90 tx=0x7fa49c09ba60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a23e21deb172053 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:10:56.784 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:56.778+0000 7fa486ff5640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-18T23:10:56.784 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:56.778+0000 7fa486ff5640 1 -- 172.21.15.49:0/3309435596 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa49c0b4020 con 0x56271f697ef0 2024-03-18T23:10:56.784 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:56.778+0000 7fa486ff5640 1 -- 172.21.15.49:0/3309435596 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7fa49c005590 con 0x56271f697ef0 2024-03-18T23:10:56.784 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:56.778+0000 7fa4ac3a2740 1 -- 172.21.15.49:0/3309435596 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa49c09c0e0 con 0x56271f697ef0 2024-03-18T23:10:56.784 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:56.778+0000 7fa486ff5640 1 -- 172.21.15.49:0/3309435596 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fa49c09fb30 con 0x56271f697ef0 2024-03-18T23:10:56.784 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:56.778+0000 7fa4ac3a2740 10 client.15669 fetch_fsmap learned FSMap version 29 2024-03-18T23:10:56.784 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:56.778+0000 7fa4ac3a2740 1 -- 172.21.15.49:0/3309435596 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56271f310210 con 0x56271f697ef0 2024-03-18T23:10:56.784 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:56.778+0000 7fa486ff5640 1 -- 172.21.15.49:0/3309435596 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa49c09c0e0 con 0x56271f697ef0 2024-03-18T23:10:56.785 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:56.778+0000 7fa486ff5640 1 -- 172.21.15.49:0/3309435596 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 29) ==== 8358+0+0 (secure 0 0 0) 0x7fa49c09bc40 con 0x56271f697ef0 2024-03-18T23:10:56.785 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:56.778+0000 7fa4ac3a2740 10 client.15669 fetch_fsmap finished waiting for FSMap version 29 2024-03-18T23:10:56.785 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:56.778+0000 7fa4ac3a2740 10 client.15669 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T23:10:56.785 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:56.778+0000 7fa4ac3a2740 20 client.15669 populate_metadata read hostname 'smithi049' 2024-03-18T23:10:56.785 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:56.778+0000 7fa4ac3a2740 1 --2- 172.21.15.49:0/3309435596 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x56271f6b1340 0x56271f6b3730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:10:56.785 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:56.779+0000 7fa4ac3a2740 4 client.15669 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T23:10:56.785 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:56.779+0000 7fa4ac3a2740 1 -- 172.21.15.49:0/3309435596 --> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56271f609120 con 0x56271f6b1340 2024-03-18T23:10:56.785 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:56.779+0000 7fa4a27fc640 1 --2- 172.21.15.49:0/3309435596 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x56271f6b1340 0x56271f6b3730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:10:56.785 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:56.779+0000 7fa486ff5640 10 client.15669 ms_handle_connect on v2:172.21.15.139:6838/4280128209 2024-03-18T23:10:56.785 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:56.779+0000 7fa4a27fc640 1 --2- 172.21.15.49:0/3309435596 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x56271f6b1340 0x56271f6b3730 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=131c28c4ef00ed33 server_cookie=f5cf5845281ea138 in_seq=0 out_seq=0 2024-03-18T23:10:56.786 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:56.780+0000 7fa486ff5640 1 -- 172.21.15.49:0/3309435596 <== mds.0 v2:172.21.15.139:6838/4280128209 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x56271f609120 con 0x56271f6b1340 2024-03-18T23:10:56.786 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:56.780+0000 7fa486ff5640 10 client.15669 handle_command_reply: tid=0 2024-03-18T23:10:56.787 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:56.781+0000 7fa4ac3a2740 1 client.15669 shutdown 2024-03-18T23:10:56.787 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:56.781+0000 7fa4ac3a2740 1 -- 172.21.15.49:0/3309435596 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56271f697ef0 msgr2=0x56271f6a0fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:10:56.787 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:56.781+0000 7fa4ac3a2740 1 --2- 172.21.15.49:0/3309435596 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56271f697ef0 0x56271f6a0fc0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fa49c099c90 tx=0x7fa49c09ba60 comp rx=0 tx=0).stop 2024-03-18T23:10:56.787 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:56.781+0000 7fa4ac3a2740 1 -- 172.21.15.49:0/3309435596 shutdown_connections 2024-03-18T23:10:56.787 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:56.781+0000 7fa4ac3a2740 1 --2- 172.21.15.49:0/3309435596 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x56271f6b1340 0x56271f6b3730 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:56.787 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:56.781+0000 7fa4ac3a2740 1 --2- 172.21.15.49:0/3309435596 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56271f6a15e0 0x56271f69e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:56.787 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:56.781+0000 7fa4ac3a2740 1 --2- 172.21.15.49:0/3309435596 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56271f698810 0x56271f69e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:56.787 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:56.781+0000 7fa4ac3a2740 1 --2- 172.21.15.49:0/3309435596 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56271f697ef0 0x56271f6a0fc0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:56.787 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:56.781+0000 7fa486ff5640 0 client.15669 ms_handle_reset on v2:172.21.15.139:6838/4280128209 2024-03-18T23:10:56.787 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:56.781+0000 7fa4ac3a2740 1 -- 172.21.15.49:0/3309435596 >> 172.21.15.49:0/3309435596 conn(0x56271f692810 msgr2=0x56271f691ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:10:56.787 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:56.781+0000 7fa4ac3a2740 1 -- 172.21.15.49:0/3309435596 shutdown_connections 2024-03-18T23:10:56.787 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:56.781+0000 7fa4ac3a2740 1 -- 172.21.15.49:0/3309435596 wait complete. 2024-03-18T23:10:56.787 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:56.781+0000 7fa4ac3a2740 20 client.15669 trim_cache size 0 max 16384 2024-03-18T23:10:56.803 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:56.797+0000 7fa4ac3a2740 1 Processor -- start 2024-03-18T23:10:56.803 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:56.797+0000 7fa4ac3a2740 1 -- start start 2024-03-18T23:10:56.804 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:56.797+0000 7fa4ac3a2740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56271f697ef0 0x56271f6acfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:10:56.804 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:56.797+0000 7fa4ac3a2740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56271f698810 0x56271f69d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:10:56.804 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:56.797+0000 7fa4ac3a2740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56271f6a15e0 0x56271f691930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:10:56.804 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:56.797+0000 7fa4ac3a2740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fa4a41af610 con 0x56271f6a15e0 2024-03-18T23:10:56.804 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:56.797+0000 7fa4ac3a2740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fa4a41b1180 con 0x56271f697ef0 2024-03-18T23:10:56.804 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:56.797+0000 7fa4ac3a2740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa4a41afce0 con 0x56271f698810 2024-03-18T23:10:56.804 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:56.798+0000 7fa4a37fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56271f6a15e0 0x56271f691930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:10:56.804 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:56.798+0000 7fa4a37fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56271f6a15e0 0x56271f691930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:33384/0 (socket says 172.21.15.49:33384) 2024-03-18T23:10:56.804 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:56.798+0000 7fa4a27fc640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56271f698810 0x56271f69d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:10:56.804 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:56.798+0000 7fa4a37fe640 1 -- 172.21.15.49:0/999035134 learned_addr learned my addr 172.21.15.49:0/999035134 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:10:56.804 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:56.798+0000 7fa4a2ffd640 1 --2- 172.21.15.49:0/999035134 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56271f697ef0 0x56271f6acfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:10:56.804 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:56.798+0000 7fa4a37fe640 1 -- 172.21.15.49:0/999035134 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56271f698810 msgr2=0x56271f69d4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:10:56.804 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:56.798+0000 7fa4a37fe640 1 --2- 172.21.15.49:0/999035134 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56271f698810 0x56271f69d4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:56.805 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:56.798+0000 7fa4a37fe640 1 -- 172.21.15.49:0/999035134 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56271f697ef0 msgr2=0x56271f6acfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:10:56.805 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:56.798+0000 7fa4a37fe640 1 --2- 172.21.15.49:0/999035134 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56271f697ef0 0x56271f6acfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:56.805 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:56.798+0000 7fa4a37fe640 1 -- 172.21.15.49:0/999035134 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56271f698640 con 0x56271f6a15e0 2024-03-18T23:10:56.805 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:56.798+0000 7fa4a37fe640 1 --2- 172.21.15.49:0/999035134 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56271f6a15e0 0x56271f691930 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fa494035000 tx=0x7fa49402bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19d73ac01b47d94e server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:10:56.805 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:56.798+0000 7fa488ff9640 1 -- 172.21.15.49:0/999035134 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4940046c0 con 0x56271f6a15e0 2024-03-18T23:10:56.805 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:56.798+0000 7fa488ff9640 1 -- 172.21.15.49:0/999035134 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa494004d40 con 0x56271f6a15e0 2024-03-18T23:10:56.805 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:56.798+0000 7fa488ff9640 1 -- 172.21.15.49:0/999035134 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa494003850 con 0x56271f6a15e0 2024-03-18T23:10:56.805 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:56.799+0000 7fa4ac3a2740 1 -- 172.21.15.49:0/999035134 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56271f6a15e0 msgr2=0x56271f691930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:10:56.805 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:56.799+0000 7fa4ac3a2740 1 --2- 172.21.15.49:0/999035134 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56271f6a15e0 0x56271f691930 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fa494035000 tx=0x7fa49402bc30 comp rx=0 tx=0).stop 2024-03-18T23:10:56.805 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:56.799+0000 7fa4ac3a2740 1 -- 172.21.15.49:0/999035134 shutdown_connections 2024-03-18T23:10:56.805 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:56.799+0000 7fa4ac3a2740 1 --2- 172.21.15.49:0/999035134 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56271f6a15e0 0x56271f691930 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:56.805 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:56.799+0000 7fa4ac3a2740 1 --2- 172.21.15.49:0/999035134 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56271f698810 0x56271f69d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:56.805 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:56.799+0000 7fa4ac3a2740 1 --2- 172.21.15.49:0/999035134 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56271f697ef0 0x56271f6acfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:56.805 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:56.799+0000 7fa4ac3a2740 1 -- 172.21.15.49:0/999035134 >> 172.21.15.49:0/999035134 conn(0x56271f692810 msgr2=0x56271f691220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:10:56.805 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:56.799+0000 7fa4ac3a2740 1 -- 172.21.15.49:0/999035134 shutdown_connections 2024-03-18T23:10:56.806 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:56.799+0000 7fa4ac3a2740 1 -- 172.21.15.49:0/999035134 wait complete. 2024-03-18T23:10:56.806 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:56.799+0000 7fa4ac3a2740 1 -- start start 2024-03-18T23:10:56.806 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:56.799+0000 7fa4ac3a2740 1 Processor -- start 2024-03-18T23:10:56.806 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:56.800+0000 7fa4ac3a2740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56271f697ef0 0x56271f6a95e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:10:56.806 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:56.800+0000 7fa4ac3a2740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56271f698810 0x56271f69a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:10:56.806 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:56.800+0000 7fa4ac3a2740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56271f6a15e0 0x56271f69aef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:10:56.806 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:56.800+0000 7fa4ac3a2740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fa4a41b0a80 con 0x56271f6a15e0 2024-03-18T23:10:56.806 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:56.800+0000 7fa4ac3a2740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fa4a41b03b0 con 0x56271f697ef0 2024-03-18T23:10:56.806 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:56.800+0000 7fa4ac3a2740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa4a41ad5f0 con 0x56271f698810 2024-03-18T23:10:56.806 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:56.800+0000 7fa4ac3a2740 10 client.0 _ll_register_callbacks cb 0x7fa49c0093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T23:10:56.806 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:56.800+0000 7fa4a37fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56271f6a15e0 0x56271f69aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:10:56.806 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:56.800+0000 7fa4a37fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56271f6a15e0 0x56271f69aef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:33398/0 (socket says 172.21.15.49:33398) 2024-03-18T23:10:56.806 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:56.800+0000 7fa4a37fe640 1 -- 172.21.15.49:0/1480702073 learned_addr learned my addr 172.21.15.49:0/1480702073 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:10:56.807 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:56.800+0000 7fa4a2ffd640 1 --2- 172.21.15.49:0/1480702073 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56271f697ef0 0x56271f6a95e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:10:56.807 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:56.800+0000 7fa4a27fc640 1 --2- 172.21.15.49:0/1480702073 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56271f698810 0x56271f69a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:10:56.807 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:56.800+0000 7fa4a37fe640 1 -- 172.21.15.49:0/1480702073 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56271f698810 msgr2=0x56271f69a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:10:56.807 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:56.800+0000 7fa4a37fe640 1 --2- 172.21.15.49:0/1480702073 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56271f698810 0x56271f69a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:56.807 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:56.800+0000 7fa4a37fe640 1 -- 172.21.15.49:0/1480702073 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56271f697ef0 msgr2=0x56271f6a95e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:10:56.807 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:56.800+0000 7fa4a37fe640 1 --2- 172.21.15.49:0/1480702073 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56271f697ef0 0x56271f6a95e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:56.807 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:56.800+0000 7fa4a37fe640 1 -- 172.21.15.49:0/1480702073 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56271f3dc620 con 0x56271f6a15e0 2024-03-18T23:10:56.807 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:56.800+0000 7fa4a2ffd640 1 --2- 172.21.15.49:0/1480702073 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56271f697ef0 0x56271f6a95e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:10:56.807 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:56.800+0000 7fa4a37fe640 1 --2- 172.21.15.49:0/1480702073 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56271f6a15e0 0x56271f69aef0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fa4940055e0 tx=0x7fa4940095d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a5cb7e250e6ce23 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:10:56.807 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:56.800+0000 7fa486ff5640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-18T23:10:56.807 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:56.801+0000 7fa486ff5640 1 -- 172.21.15.49:0/1480702073 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa494004ad0 con 0x56271f6a15e0 2024-03-18T23:10:56.807 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:56.801+0000 7fa486ff5640 1 -- 172.21.15.49:0/1480702073 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7fa4940035f0 con 0x56271f6a15e0 2024-03-18T23:10:56.807 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:56.801+0000 7fa4ac3a2740 1 -- 172.21.15.49:0/1480702073 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa494004d40 con 0x56271f6a15e0 2024-03-18T23:10:56.807 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:56.801+0000 7fa486ff5640 1 -- 172.21.15.49:0/1480702073 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fa494005680 con 0x56271f6a15e0 2024-03-18T23:10:56.808 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:56.801+0000 7fa4ac3a2740 10 client.15675 fetch_fsmap learned FSMap version 29 2024-03-18T23:10:56.808 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:56.801+0000 7fa4ac3a2740 1 -- 172.21.15.49:0/1480702073 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56271f716e90 con 0x56271f6a15e0 2024-03-18T23:10:56.808 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:56.801+0000 7fa486ff5640 1 -- 172.21.15.49:0/1480702073 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa494004d40 con 0x56271f6a15e0 2024-03-18T23:10:56.808 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:56.801+0000 7fa486ff5640 1 -- 172.21.15.49:0/1480702073 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 29) ==== 8358+0+0 (secure 0 0 0) 0x7fa4940617d0 con 0x56271f6a15e0 2024-03-18T23:10:56.808 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:56.801+0000 7fa4ac3a2740 10 client.15675 fetch_fsmap finished waiting for FSMap version 29 2024-03-18T23:10:56.808 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:56.801+0000 7fa4ac3a2740 10 client.15675 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T23:10:56.808 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:56.801+0000 7fa4ac3a2740 20 client.15675 populate_metadata read hostname 'smithi049' 2024-03-18T23:10:56.808 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:56.801+0000 7fa4ac3a2740 1 --2- 172.21.15.49:0/1480702073 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x56271f7175c0 0x56271f7199e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:10:56.808 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:56.801+0000 7fa4ac3a2740 4 client.15675 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T23:10:56.808 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:56.801+0000 7fa4ac3a2740 1 -- 172.21.15.49:0/1480702073 --> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56271f603470 con 0x56271f7175c0 2024-03-18T23:10:56.808 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:56.801+0000 7fa4a2ffd640 1 --2- 172.21.15.49:0/1480702073 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x56271f7175c0 0x56271f7199e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:10:56.808 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:56.802+0000 7fa486ff5640 10 client.15675 ms_handle_connect on v2:172.21.15.139:6838/4280128209 2024-03-18T23:10:56.808 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:56.802+0000 7fa4a2ffd640 1 --2- 172.21.15.49:0/1480702073 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x56271f7175c0 0x56271f7199e0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bd69ca376311aa06 server_cookie=3ea0380396fb00b2 in_seq=0 out_seq=0 2024-03-18T23:10:57.026 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:57.024+0000 7fa486ff5640 1 -- 172.21.15.49:0/1480702073 <== mds.0 v2:172.21.15.139:6838/4280128209 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56271f603470 con 0x56271f7175c0 2024-03-18T23:10:57.026 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:57.024+0000 7fa486ff5640 10 client.15675 handle_command_reply: tid=0 2024-03-18T23:10:57.026 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:57.024+0000 7fa4ac3a2740 1 client.15675 shutdown 2024-03-18T23:10:57.026 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:57.025+0000 7fa4ac3a2740 1 -- 172.21.15.49:0/1480702073 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56271f6a15e0 msgr2=0x56271f69aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:10:57.027 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:57.025+0000 7fa4ac3a2740 1 --2- 172.21.15.49:0/1480702073 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56271f6a15e0 0x56271f69aef0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fa4940055e0 tx=0x7fa4940095d0 comp rx=0 tx=0).stop 2024-03-18T23:10:57.027 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:57.025+0000 7fa4ac3a2740 1 -- 172.21.15.49:0/1480702073 shutdown_connections 2024-03-18T23:10:57.027 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:57.025+0000 7fa4ac3a2740 1 --2- 172.21.15.49:0/1480702073 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x56271f7175c0 0x56271f7199e0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:57.027 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:57.025+0000 7fa4ac3a2740 1 --2- 172.21.15.49:0/1480702073 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56271f6a15e0 0x56271f69aef0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:57.027 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:57.025+0000 7fa486ff5640 0 client.15675 ms_handle_reset on v2:172.21.15.139:6838/4280128209 2024-03-18T23:10:57.027 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:57.025+0000 7fa4ac3a2740 1 --2- 172.21.15.49:0/1480702073 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56271f698810 0x56271f69a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:57.027 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:57.025+0000 7fa4ac3a2740 1 --2- 172.21.15.49:0/1480702073 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56271f697ef0 0x56271f6a95e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:57.028 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:57.025+0000 7fa4ac3a2740 1 -- 172.21.15.49:0/1480702073 >> 172.21.15.49:0/1480702073 conn(0x56271f692810 msgr2=0x56271f691ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:10:57.028 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:57.025+0000 7fa4ac3a2740 1 -- 172.21.15.49:0/1480702073 shutdown_connections 2024-03-18T23:10:57.028 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:57.025+0000 7fa4ac3a2740 1 -- 172.21.15.49:0/1480702073 wait complete. 2024-03-18T23:10:57.028 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:57.025+0000 7fa4ac3a2740 20 client.15675 trim_cache size 0 max 16384 2024-03-18T23:10:57.028 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:{ 2024-03-18T23:10:57.029 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "message": "", 2024-03-18T23:10:57.029 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "return_code": 0 2024-03-18T23:10:57.029 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:} 2024-03-18T23:10:57.030 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:57.026+0000 7fa4a96dc640 1 -- 172.21.15.49:0/3165281464 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7fa48c0a2a90 msgr2=0x7fa48c0a4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:10:57.030 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:57.026+0000 7fa4a96dc640 1 --2- 172.21.15.49:0/3165281464 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7fa48c0a2a90 0x7fa48c0a4f50 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fa494005940 tx=0x7fa494009680 comp rx=0 tx=0).stop 2024-03-18T23:10:57.030 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:57.026+0000 7fa4a96dc640 1 -- 172.21.15.49:0/3165281464 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa4a4101810 msgr2=0x7fa4a419a220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:10:57.031 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:57.026+0000 7fa4a96dc640 1 --2- 172.21.15.49:0/3165281464 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa4a4101810 0x7fa4a419a220 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fa49c00eb90 tx=0x7fa49c00c6a0 comp rx=0 tx=0).stop 2024-03-18T23:10:57.031 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:57.026+0000 7fa4a96dc640 1 -- 172.21.15.49:0/3165281464 shutdown_connections 2024-03-18T23:10:57.031 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:57.026+0000 7fa4a96dc640 1 --2- 172.21.15.49:0/3165281464 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7fa48c0a2a90 0x7fa48c0a4f50 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:57.031 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:57.026+0000 7fa4a96dc640 1 --2- 172.21.15.49:0/3165281464 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa4a4107820 0x7fa4a419eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:57.031 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:57.026+0000 7fa4a96dc640 1 --2- 172.21.15.49:0/3165281464 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa4a41021d0 0x7fa4a419a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:57.031 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:57.026+0000 7fa4a96dc640 1 --2- 172.21.15.49:0/3165281464 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa4a4101810 0x7fa4a419a220 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:57.032 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:57.026+0000 7fa4a96dc640 1 -- 172.21.15.49:0/3165281464 >> 172.21.15.49:0/3165281464 conn(0x7fa4a40fd2e0 msgr2=0x7fa4a410cfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:10:57.032 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:57.026+0000 7fa4a96dc640 1 -- 172.21.15.49:0/3165281464 shutdown_connections 2024-03-18T23:10:57.032 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:10:57.026+0000 7fa4a96dc640 1 -- 172.21.15.49:0/3165281464 wait complete. 2024-03-18T23:10:57.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:10:57 smithi049 ceph-mon[62991]: pgmap v83: 97 pgs: 97 active+clean; 48 MiB data, 785 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.0 MiB/s wr, 312 op/s 2024-03-18T23:10:57.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:10:57 smithi175 ceph-mon[67352]: pgmap v83: 97 pgs: 97 active+clean; 48 MiB data, 785 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.0 MiB/s wr, 312 op/s 2024-03-18T23:10:57.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:10:57 smithi139 ceph-mon[67282]: pgmap v83: 97 pgs: 97 active+clean; 48 MiB data, 785 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.0 MiB/s wr, 312 op/s 2024-03-18T23:10:57.763 INFO:tasks.workunit.client.0.smithi049.stdout: 1 277143 12.71 MB/sec warmup 119 sec latency 81.936 ms 2024-03-18T23:10:59.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:10:59 smithi049 ceph-mon[62991]: pgmap v84: 97 pgs: 97 active+clean; 53 MiB data, 779 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 4.4 MiB/s wr, 424 op/s 2024-03-18T23:10:59.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:10:59 smithi175 ceph-mon[67352]: pgmap v84: 97 pgs: 97 active+clean; 53 MiB data, 779 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 4.4 MiB/s wr, 424 op/s 2024-03-18T23:10:59.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:10:59 smithi139 ceph-mon[67282]: pgmap v84: 97 pgs: 97 active+clean; 53 MiB data, 779 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 4.4 MiB/s wr, 424 op/s 2024-03-18T23:10:59.763 INFO:tasks.workunit.client.0.smithi049.stdout: 1 281761 11.07 MB/sec execute 1 sec latency 26.229 ms 2024-03-18T23:11:00.763 INFO:tasks.workunit.client.0.smithi049.stdout: 1 284248 12.53 MB/sec execute 2 sec latency 14.814 ms 2024-03-18T23:11:01.379 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:11:01 smithi049 ceph-mon[62991]: pgmap v85: 97 pgs: 97 active+clean; 53 MiB data, 779 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 2.8 MiB/s wr, 216 op/s 2024-03-18T23:11:01.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:11:01 smithi175 ceph-mon[67352]: pgmap v85: 97 pgs: 97 active+clean; 53 MiB data, 779 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 2.8 MiB/s wr, 216 op/s 2024-03-18T23:11:01.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:11:01 smithi139 ceph-mon[67282]: pgmap v85: 97 pgs: 97 active+clean; 53 MiB data, 779 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 2.8 MiB/s wr, 216 op/s 2024-03-18T23:11:01.764 INFO:tasks.workunit.client.0.smithi049.stdout: 1 285927 10.26 MB/sec execute 3 sec latency 26.283 ms 2024-03-18T23:11:02.764 INFO:tasks.workunit.client.0.smithi049.stdout: 1 287705 11.05 MB/sec execute 4 sec latency 87.363 ms 2024-03-18T23:11:03.379 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:11:03 smithi049 ceph-mon[62991]: pgmap v86: 97 pgs: 97 active+clean; 46 MiB data, 794 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 4.3 MiB/s wr, 452 op/s 2024-03-18T23:11:03.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:11:03 smithi175 ceph-mon[67352]: pgmap v86: 97 pgs: 97 active+clean; 46 MiB data, 794 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 4.3 MiB/s wr, 452 op/s 2024-03-18T23:11:03.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:11:03 smithi139 ceph-mon[67282]: pgmap v86: 97 pgs: 97 active+clean; 46 MiB data, 794 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 4.3 MiB/s wr, 452 op/s 2024-03-18T23:11:03.764 INFO:tasks.workunit.client.0.smithi049.stdout: 1 289891 10.43 MB/sec execute 5 sec latency 17.259 ms 2024-03-18T23:11:04.764 INFO:tasks.workunit.client.0.smithi049.stdout: 1 291809 10.69 MB/sec execute 6 sec latency 24.819 ms 2024-03-18T23:11:05.379 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:11:05 smithi049 ceph-mon[62991]: pgmap v87: 97 pgs: 97 active+clean; 46 MiB data, 794 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 2.9 MiB/s wr, 347 op/s 2024-03-18T23:11:05.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:11:05 smithi175 ceph-mon[67352]: pgmap v87: 97 pgs: 97 active+clean; 46 MiB data, 794 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 2.9 MiB/s wr, 347 op/s 2024-03-18T23:11:05.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:11:05 smithi139 ceph-mon[67282]: pgmap v87: 97 pgs: 97 active+clean; 46 MiB data, 794 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 2.9 MiB/s wr, 347 op/s 2024-03-18T23:11:05.764 INFO:tasks.workunit.client.0.smithi049.stdout: 1 294623 11.73 MB/sec execute 7 sec latency 13.368 ms 2024-03-18T23:11:06.764 INFO:tasks.workunit.client.0.smithi049.stdout: 1 296768 11.17 MB/sec execute 8 sec latency 19.541 ms 2024-03-18T23:11:07.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:11:07 smithi049 ceph-mon[62991]: pgmap v88: 97 pgs: 97 active+clean; 46 MiB data, 794 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 2.9 MiB/s wr, 347 op/s 2024-03-18T23:11:07.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:11:07 smithi049 ceph-mon[62991]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T23:11:07.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:11:07 smithi175 ceph-mon[67352]: pgmap v88: 97 pgs: 97 active+clean; 46 MiB data, 794 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 2.9 MiB/s wr, 347 op/s 2024-03-18T23:11:07.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:11:07 smithi175 ceph-mon[67352]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T23:11:07.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:11:07 smithi139 ceph-mon[67282]: pgmap v88: 97 pgs: 97 active+clean; 46 MiB data, 794 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 2.9 MiB/s wr, 347 op/s 2024-03-18T23:11:07.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:11:07 smithi139 ceph-mon[67282]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T23:11:07.764 INFO:tasks.workunit.client.0.smithi049.stdout: 1 298432 11.28 MB/sec execute 9 sec latency 50.445 ms 2024-03-18T23:11:08.764 INFO:tasks.workunit.client.0.smithi049.stdout: 1 301169 11.49 MB/sec execute 10 sec latency 41.054 ms 2024-03-18T23:11:09.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:11:09 smithi049 ceph-mon[62991]: pgmap v89: 97 pgs: 97 active+clean; 53 MiB data, 842 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 4.6 MiB/s wr, 456 op/s 2024-03-18T23:11:09.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:11:09 smithi175 ceph-mon[67352]: pgmap v89: 97 pgs: 97 active+clean; 53 MiB data, 842 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 4.6 MiB/s wr, 456 op/s 2024-03-18T23:11:09.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:11:09 smithi139 ceph-mon[67282]: pgmap v89: 97 pgs: 97 active+clean; 53 MiB data, 842 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 4.6 MiB/s wr, 456 op/s 2024-03-18T23:11:09.764 INFO:tasks.workunit.client.0.smithi049.stdout: 1 303015 11.43 MB/sec execute 11 sec latency 18.860 ms 2024-03-18T23:11:10.122 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:10.119+0000 7fb3af0c6640 1 -- 172.21.15.49:0/883473940 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb3a8107910 msgr2=0x7fb3a8107cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:11:10.122 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:10.119+0000 7fb3af0c6640 1 --2- 172.21.15.49:0/883473940 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb3a8107910 0x7fb3a8107cf0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fb39c009900 tx=0x7fb39c02bc30 comp rx=0 tx=0).stop 2024-03-18T23:11:10.122 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:10.119+0000 7fb3af0c6640 1 -- 172.21.15.49:0/883473940 shutdown_connections 2024-03-18T23:11:10.122 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:10.119+0000 7fb3af0c6640 1 --2- 172.21.15.49:0/883473940 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb3a8069840 0x7fb3a810e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:10.122 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:10.119+0000 7fb3af0c6640 1 --2- 172.21.15.49:0/883473940 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb3a8068ea0 0x7fb3a8069300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:10.122 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:10.119+0000 7fb3af0c6640 1 --2- 172.21.15.49:0/883473940 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb3a8107910 0x7fb3a8107cf0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:10.123 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:10.119+0000 7fb3af0c6640 1 -- 172.21.15.49:0/883473940 >> 172.21.15.49:0/883473940 conn(0x7fb3a8077570 msgr2=0x7fb3a8077980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:11:10.123 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:10.119+0000 7fb3af0c6640 1 -- 172.21.15.49:0/883473940 shutdown_connections 2024-03-18T23:11:10.123 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:10.119+0000 7fb3af0c6640 1 -- 172.21.15.49:0/883473940 wait complete. 2024-03-18T23:11:10.123 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:10.119+0000 7fb3af0c6640 1 Processor -- start 2024-03-18T23:11:10.123 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:10.119+0000 7fb3af0c6640 1 -- start start 2024-03-18T23:11:10.123 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:10.120+0000 7fb3af0c6640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb3a8068ea0 0x7fb3a8100a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:11:10.123 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:10.120+0000 7fb3af0c6640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb3a8069840 0x7fb3a8100f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:11:10.123 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:10.120+0000 7fb3af0c6640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb3a8107910 0x7fb3a81014c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:11:10.123 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:10.120+0000 7fb3af0c6640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fb3a8111c90 con 0x7fb3a8069840 2024-03-18T23:11:10.123 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:10.120+0000 7fb3af0c6640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fb3a8111b10 con 0x7fb3a8107910 2024-03-18T23:11:10.123 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:10.120+0000 7fb3af0c6640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb3a8111e10 con 0x7fb3a8068ea0 2024-03-18T23:11:10.123 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:10.120+0000 7fb3a7fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb3a8069840 0x7fb3a8100f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:11:10.123 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:10.120+0000 7fb3ad663640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb3a8107910 0x7fb3a81014c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:11:10.124 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:10.120+0000 7fb3ace62640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb3a8068ea0 0x7fb3a8100a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:11:10.124 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:10.120+0000 7fb3a7fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb3a8069840 0x7fb3a8100f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:36812/0 (socket says 172.21.15.49:36812) 2024-03-18T23:11:10.124 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:10.120+0000 7fb3a7fff640 1 -- 172.21.15.49:0/3282562410 learned_addr learned my addr 172.21.15.49:0/3282562410 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:11:10.124 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:10.120+0000 7fb3ace62640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb3a8068ea0 0x7fb3a8100a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.49:51244/0 (socket says 172.21.15.49:51244) 2024-03-18T23:11:10.124 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:10.120+0000 7fb3ad663640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb3a8107910 0x7fb3a81014c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.49:57050/0 (socket says 172.21.15.49:57050) 2024-03-18T23:11:10.124 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:10.120+0000 7fb3ad663640 1 -- 172.21.15.49:0/3282562410 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb3a8068ea0 msgr2=0x7fb3a8100a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:11:10.124 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:10.120+0000 7fb3ad663640 1 --2- 172.21.15.49:0/3282562410 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb3a8068ea0 0x7fb3a8100a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:10.124 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:10.120+0000 7fb3ad663640 1 -- 172.21.15.49:0/3282562410 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb3a8069840 msgr2=0x7fb3a8100f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:11:10.124 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:10.120+0000 7fb3ad663640 1 --2- 172.21.15.49:0/3282562410 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb3a8069840 0x7fb3a8100f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:10.124 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:10.120+0000 7fb3ad663640 1 -- 172.21.15.49:0/3282562410 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3a81a6510 con 0x7fb3a8107910 2024-03-18T23:11:10.124 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:10.120+0000 7fb3a7fff640 1 --2- 172.21.15.49:0/3282562410 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb3a8069840 0x7fb3a8100f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:11:10.125 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:10.120+0000 7fb3ad663640 1 --2- 172.21.15.49:0/3282562410 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb3a8107910 0x7fb3a81014c0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fb3a0005fd0 tx=0x7fb3a000ee60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:11:10.125 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:10.121+0000 7fb3a5ffb640 1 -- 172.21.15.49:0/3282562410 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3a000cd80 con 0x7fb3a8107910 2024-03-18T23:11:10.125 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:10.121+0000 7fb3a5ffb640 1 -- 172.21.15.49:0/3282562410 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb3a0005020 con 0x7fb3a8107910 2024-03-18T23:11:10.125 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:10.121+0000 7fb3a5ffb640 1 -- 172.21.15.49:0/3282562410 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3a0010720 con 0x7fb3a8107910 2024-03-18T23:11:10.125 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:10.121+0000 7fb3af0c6640 1 -- 172.21.15.49:0/3282562410 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3a81a6800 con 0x7fb3a8107910 2024-03-18T23:11:10.125 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:10.121+0000 7fb3af0c6640 1 -- 172.21.15.49:0/3282562410 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3a81a6d00 con 0x7fb3a8107910 2024-03-18T23:11:10.132 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:10.123+0000 7fb3a5ffb640 1 -- 172.21.15.49:0/3282562410 <== mon.2 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 25) ==== 137421+0+0 (secure 0 0 0) 0x7fb3a00051d0 con 0x7fb3a8107910 2024-03-18T23:11:10.132 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:10.123+0000 7fb3b1d8c740 1 Processor -- start 2024-03-18T23:11:10.132 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:10.123+0000 7fb3b1d8c740 1 -- start start 2024-03-18T23:11:10.133 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:10.124+0000 7fb3a5ffb640 1 --2- 172.21.15.49:0/3282562410 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7fb3980a2a10 0x7fb3980a4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:11:10.133 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:10.124+0000 7fb3b1d8c740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5573be101ef0 0x5573be1022d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:11:10.133 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:10.124+0000 7fb3b1d8c740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5573be102810 0x5573be10b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:11:10.133 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:10.124+0000 7fb3b1d8c740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5573be10b5e0 0x5573be10d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:11:10.133 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:10.124+0000 7fb3b1d8c740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5573be065be0 con 0x5573be101ef0 2024-03-18T23:11:10.133 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:10.124+0000 7fb3b1d8c740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x5573be00e960 con 0x5573be10b5e0 2024-03-18T23:11:10.133 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:10.124+0000 7fb3b1d8c740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5573be061c80 con 0x5573be102810 2024-03-18T23:11:10.133 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:10.124+0000 7fb3ace62640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5573be101ef0 0x5573be1022d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:11:10.133 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:10.124+0000 7fb3a5ffb640 1 -- 172.21.15.49:0/3282562410 <== mon.2 v2:172.21.15.139:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7fb3a00cf360 con 0x7fb3a8107910 2024-03-18T23:11:10.133 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:10.124+0000 7fb3a7fff640 1 --2- 172.21.15.49:0/3282562410 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7fb3980a2a10 0x7fb3980a4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:11:10.133 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:10.124+0000 7fb3ace62640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5573be101ef0 0x5573be1022d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:36824/0 (socket says 172.21.15.49:36824) 2024-03-18T23:11:10.133 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:10.124+0000 7fb3ace62640 1 -- 172.21.15.49:0/964043762 learned_addr learned my addr 172.21.15.49:0/964043762 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:11:10.133 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:10.124+0000 7fb3ad663640 1 --2- 172.21.15.49:0/964043762 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5573be10b5e0 0x5573be10d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:11:10.133 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:10.124+0000 7fb3a7fff640 1 --2- 172.21.15.49:0/964043762 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5573be102810 0x5573be10b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:11:10.134 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:10.124+0000 7fb3ace62640 1 -- 172.21.15.49:0/964043762 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5573be102810 msgr2=0x5573be10b0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:11:10.134 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:10.124+0000 7fb3ace62640 1 --2- 172.21.15.49:0/964043762 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5573be102810 0x5573be10b0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:10.134 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:10.124+0000 7fb3ace62640 1 -- 172.21.15.49:0/964043762 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5573be10b5e0 msgr2=0x5573be10d9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:11:10.134 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:10.124+0000 7fb3ace62640 1 --2- 172.21.15.49:0/964043762 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5573be10b5e0 0x5573be10d9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:10.134 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:10.124+0000 7fb3ace62640 1 -- 172.21.15.49:0/964043762 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5573be005d20 con 0x5573be101ef0 2024-03-18T23:11:10.134 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:10.124+0000 7fb3ad663640 1 --2- 172.21.15.49:0/964043762 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5573be10b5e0 0x5573be10d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:11:10.134 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:10.124+0000 7fb3a7fff640 1 --2- 172.21.15.49:0/3282562410 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7fb3980a2a10 0x7fb3980a4ed0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fb3940097f0 tx=0x7fb394013460 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:11:10.134 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:10.124+0000 7fb3ace62640 1 --2- 172.21.15.49:0/964043762 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5573be101ef0 0x5573be1022d0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fb39c02cfd0 tx=0x7fb39c005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22346dc9b7567f21 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:11:10.134 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:10.125+0000 7fb392ffd640 1 -- 172.21.15.49:0/964043762 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb39c042070 con 0x5573be101ef0 2024-03-18T23:11:10.134 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:10.125+0000 7fb392ffd640 1 -- 172.21.15.49:0/964043762 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb39c02ca80 con 0x5573be101ef0 2024-03-18T23:11:10.134 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:10.125+0000 7fb392ffd640 1 -- 172.21.15.49:0/964043762 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb39c004f00 con 0x5573be101ef0 2024-03-18T23:11:10.134 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:10.125+0000 7fb3b1d8c740 1 -- 172.21.15.49:0/964043762 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5573be101ef0 msgr2=0x5573be1022d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:11:10.135 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:10.125+0000 7fb3b1d8c740 1 --2- 172.21.15.49:0/964043762 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5573be101ef0 0x5573be1022d0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fb39c02cfd0 tx=0x7fb39c005c00 comp rx=0 tx=0).stop 2024-03-18T23:11:10.135 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:10.125+0000 7fb3b1d8c740 1 -- 172.21.15.49:0/964043762 shutdown_connections 2024-03-18T23:11:10.135 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:10.125+0000 7fb3b1d8c740 1 --2- 172.21.15.49:0/964043762 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5573be10b5e0 0x5573be10d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:10.135 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:10.125+0000 7fb3b1d8c740 1 --2- 172.21.15.49:0/964043762 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5573be102810 0x5573be10b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:10.135 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:10.125+0000 7fb3b1d8c740 1 --2- 172.21.15.49:0/964043762 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5573be101ef0 0x5573be1022d0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:10.135 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:10.125+0000 7fb3b1d8c740 1 -- 172.21.15.49:0/964043762 >> 172.21.15.49:0/964043762 conn(0x5573be0fc810 msgr2=0x5573be0fcbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:11:10.135 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:10.125+0000 7fb3b1d8c740 1 -- 172.21.15.49:0/964043762 shutdown_connections 2024-03-18T23:11:10.135 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:10.125+0000 7fb3b1d8c740 1 -- 172.21.15.49:0/964043762 wait complete. 2024-03-18T23:11:10.135 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:10.126+0000 7fb3b1d8c740 1 -- start start 2024-03-18T23:11:10.135 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:10.126+0000 7fb3b1d8c740 1 Processor -- start 2024-03-18T23:11:10.135 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:10.126+0000 7fb3b1d8c740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5573be101ef0 0x5573be10b000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:11:10.135 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:10.126+0000 7fb3b1d8c740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5573be102810 0x5573be108390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:11:10.135 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:10.126+0000 7fb3b1d8c740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5573be10b5e0 0x5573be1089a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:11:10.135 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:10.126+0000 7fb3b1d8c740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5573be10e1b0 con 0x5573be10b5e0 2024-03-18T23:11:10.135 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:10.126+0000 7fb3b1d8c740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x5573be021db0 con 0x5573be101ef0 2024-03-18T23:11:10.136 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:10.126+0000 7fb3b1d8c740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5573be10e330 con 0x5573be102810 2024-03-18T23:11:10.136 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:10.126+0000 7fb3b1d8c740 10 client.0 _ll_register_callbacks cb 0x5573be026e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T23:11:10.136 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:10.126+0000 7fb3ad663640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5573be10b5e0 0x5573be1089a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:11:10.136 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:10.126+0000 7fb3ad663640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5573be10b5e0 0x5573be1089a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:36834/0 (socket says 172.21.15.49:36834) 2024-03-18T23:11:10.136 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:10.126+0000 7fb3ace62640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5573be101ef0 0x5573be10b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:11:10.136 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:10.126+0000 7fb3a7fff640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5573be102810 0x5573be108390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:11:10.136 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:10.126+0000 7fb3ad663640 1 -- 172.21.15.49:0/229533093 learned_addr learned my addr 172.21.15.49:0/229533093 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:11:10.136 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:10.127+0000 7fb3ad663640 1 -- 172.21.15.49:0/229533093 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5573be102810 msgr2=0x5573be108390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:11:10.136 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:10.127+0000 7fb3ad663640 1 --2- 172.21.15.49:0/229533093 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5573be102810 0x5573be108390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:10.136 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:10.127+0000 7fb3ad663640 1 -- 172.21.15.49:0/229533093 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5573be101ef0 msgr2=0x5573be10b000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:11:10.136 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:10.127+0000 7fb3ad663640 1 --2- 172.21.15.49:0/229533093 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5573be101ef0 0x5573be10b000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:10.136 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:10.127+0000 7fb3ad663640 1 -- 172.21.15.49:0/229533093 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5573be020c60 con 0x5573be10b5e0 2024-03-18T23:11:10.136 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:10.127+0000 7fb3a7fff640 1 --2- 172.21.15.49:0/229533093 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5573be102810 0x5573be108390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:11:10.136 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:10.127+0000 7fb3ad663640 1 --2- 172.21.15.49:0/229533093 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5573be10b5e0 0x5573be1089a0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fb3a00cd0e0 tx=0x7fb3a009c0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42491f7e9441e0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:11:10.136 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:10.127+0000 7fb387fff640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-18T23:11:10.137 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:10.127+0000 7fb387fff640 1 -- 172.21.15.49:0/229533093 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3a009fb40 con 0x5573be10b5e0 2024-03-18T23:11:10.137 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:10.127+0000 7fb387fff640 1 -- 172.21.15.49:0/229533093 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7fb3a001cca0 con 0x5573be10b5e0 2024-03-18T23:11:10.137 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:10.127+0000 7fb3b1d8c740 1 -- 172.21.15.49:0/229533093 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb39c02ca80 con 0x5573be10b5e0 2024-03-18T23:11:10.137 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:10.127+0000 7fb387fff640 1 -- 172.21.15.49:0/229533093 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fb3a001cea0 con 0x5573be10b5e0 2024-03-18T23:11:10.137 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:10.127+0000 7fb3b1d8c740 10 client.15687 fetch_fsmap learned FSMap version 30 2024-03-18T23:11:10.137 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:10.127+0000 7fb3b1d8c740 1 -- 172.21.15.49:0/229533093 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5573bdd7a210 con 0x5573be10b5e0 2024-03-18T23:11:10.137 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:10.127+0000 7fb387fff640 1 -- 172.21.15.49:0/229533093 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb39c02ca80 con 0x5573be10b5e0 2024-03-18T23:11:10.137 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:10.127+0000 7fb387fff640 1 -- 172.21.15.49:0/229533093 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7fb3a009bce0 con 0x5573be10b5e0 2024-03-18T23:11:10.137 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:10.128+0000 7fb3b1d8c740 10 client.15687 fetch_fsmap finished waiting for FSMap version 30 2024-03-18T23:11:10.137 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:10.128+0000 7fb3b1d8c740 10 client.15687 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T23:11:10.137 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:10.128+0000 7fb3b1d8c740 20 client.15687 populate_metadata read hostname 'smithi049' 2024-03-18T23:11:10.137 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:10.128+0000 7fb3b1d8c740 1 --2- 172.21.15.49:0/229533093 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x5573be11b4a0 0x5573be11d890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:11:10.137 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:10.128+0000 7fb3b1d8c740 4 client.15687 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T23:11:10.137 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:10.128+0000 7fb3b1d8c740 1 -- 172.21.15.49:0/229533093 --> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5573be073120 con 0x5573be11b4a0 2024-03-18T23:11:10.137 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:10.128+0000 7fb3ace62640 1 --2- 172.21.15.49:0/229533093 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x5573be11b4a0 0x5573be11d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:11:10.138 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:10.128+0000 7fb3ace62640 1 --2- 172.21.15.49:0/229533093 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x5573be11b4a0 0x5573be11d890 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5c3dfba461b15044 server_cookie=d657dfdbecf7b5f2 in_seq=0 out_seq=0 2024-03-18T23:11:10.138 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:10.128+0000 7fb387fff640 10 client.15687 ms_handle_connect on v2:172.21.15.139:6838/4280128209 2024-03-18T23:11:10.138 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:10.129+0000 7fb387fff640 1 -- 172.21.15.49:0/229533093 <== mds.0 v2:172.21.15.139:6838/4280128209 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5573be073120 con 0x5573be11b4a0 2024-03-18T23:11:10.138 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:10.129+0000 7fb387fff640 10 client.15687 handle_command_reply: tid=0 2024-03-18T23:11:10.138 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:10.129+0000 7fb3b1d8c740 1 client.15687 shutdown 2024-03-18T23:11:10.138 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:10.130+0000 7fb3b1d8c740 1 -- 172.21.15.49:0/229533093 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5573be10b5e0 msgr2=0x5573be1089a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:11:10.138 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:10.130+0000 7fb3b1d8c740 1 --2- 172.21.15.49:0/229533093 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5573be10b5e0 0x5573be1089a0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fb3a00cd0e0 tx=0x7fb3a009c0f0 comp rx=0 tx=0).stop 2024-03-18T23:11:10.138 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:10.130+0000 7fb3b1d8c740 1 -- 172.21.15.49:0/229533093 shutdown_connections 2024-03-18T23:11:10.138 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:10.130+0000 7fb3b1d8c740 1 --2- 172.21.15.49:0/229533093 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x5573be11b4a0 0x5573be11d890 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:10.138 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:10.130+0000 7fb3b1d8c740 1 --2- 172.21.15.49:0/229533093 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5573be10b5e0 0x5573be1089a0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:10.138 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:10.130+0000 7fb3b1d8c740 1 --2- 172.21.15.49:0/229533093 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5573be102810 0x5573be108390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:10.138 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:10.130+0000 7fb3b1d8c740 1 --2- 172.21.15.49:0/229533093 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5573be101ef0 0x5573be10b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:10.138 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:10.130+0000 7fb3b1d8c740 1 -- 172.21.15.49:0/229533093 >> 172.21.15.49:0/229533093 conn(0x5573be0fc810 msgr2=0x5573be0fbab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:11:10.139 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:10.130+0000 7fb387fff640 0 client.15687 ms_handle_reset on v2:172.21.15.139:6838/4280128209 2024-03-18T23:11:10.139 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:10.130+0000 7fb3b1d8c740 1 -- 172.21.15.49:0/229533093 shutdown_connections 2024-03-18T23:11:10.139 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:10.130+0000 7fb3b1d8c740 1 -- 172.21.15.49:0/229533093 wait complete. 2024-03-18T23:11:10.139 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:10.130+0000 7fb3b1d8c740 20 client.15687 trim_cache size 0 max 16384 2024-03-18T23:11:10.147 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:10.146+0000 7fb3b1d8c740 1 Processor -- start 2024-03-18T23:11:10.147 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:10.146+0000 7fb3b1d8c740 1 -- start start 2024-03-18T23:11:10.148 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:10.146+0000 7fb3b1d8c740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5573be101ef0 0x5573be116f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:11:10.148 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:10.146+0000 7fb3b1d8c740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5573be102810 0x5573be0fb930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:11:10.148 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:10.146+0000 7fb3b1d8c740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5573be10b5e0 0x5573be1231a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:11:10.148 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:10.146+0000 7fb3b1d8c740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fb3a81af520 con 0x5573be101ef0 2024-03-18T23:11:10.148 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:10.146+0000 7fb3b1d8c740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fb3a81b1060 con 0x5573be102810 2024-03-18T23:11:10.148 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:10.146+0000 7fb3b1d8c740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb3a81afbf0 con 0x5573be10b5e0 2024-03-18T23:11:10.148 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:10.146+0000 7fb3ace62640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5573be101ef0 0x5573be116f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:11:10.148 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:10.146+0000 7fb3ace62640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5573be101ef0 0x5573be116f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:36844/0 (socket says 172.21.15.49:36844) 2024-03-18T23:11:10.148 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:10.146+0000 7fb3ace62640 1 -- 172.21.15.49:0/3837007915 learned_addr learned my addr 172.21.15.49:0/3837007915 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:11:10.148 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:10.146+0000 7fb3a7fff640 1 --2- 172.21.15.49:0/3837007915 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5573be102810 0x5573be0fb930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:11:10.148 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:10.146+0000 7fb3ad663640 1 --2- 172.21.15.49:0/3837007915 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5573be10b5e0 0x5573be1231a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:11:10.148 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:10.146+0000 7fb3ace62640 1 -- 172.21.15.49:0/3837007915 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5573be10b5e0 msgr2=0x5573be1231a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:11:10.148 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:10.147+0000 7fb3ace62640 1 --2- 172.21.15.49:0/3837007915 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5573be10b5e0 0x5573be1231a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:10.149 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:10.147+0000 7fb3ace62640 1 -- 172.21.15.49:0/3837007915 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5573be102810 msgr2=0x5573be0fb930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:11:10.149 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:10.147+0000 7fb3ace62640 1 --2- 172.21.15.49:0/3837007915 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5573be102810 0x5573be0fb930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:10.149 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:10.147+0000 7fb3ace62640 1 -- 172.21.15.49:0/3837007915 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5573bde46620 con 0x5573be101ef0 2024-03-18T23:11:10.149 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:10.147+0000 7fb3ace62640 1 --2- 172.21.15.49:0/3837007915 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5573be101ef0 0x5573be116f50 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fb39c02f020 tx=0x7fb39c005dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ce7bcdd30a2b24d server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:11:10.149 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:10.147+0000 7fb3927fc640 1 -- 172.21.15.49:0/3837007915 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb39c053070 con 0x5573be101ef0 2024-03-18T23:11:10.149 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:10.147+0000 7fb3927fc640 1 -- 172.21.15.49:0/3837007915 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb39c02cb80 con 0x5573be101ef0 2024-03-18T23:11:10.149 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:10.147+0000 7fb3927fc640 1 -- 172.21.15.49:0/3837007915 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb39c042040 con 0x5573be101ef0 2024-03-18T23:11:10.152 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:10.147+0000 7fb3b1d8c740 1 -- 172.21.15.49:0/3837007915 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5573be101ef0 msgr2=0x5573be116f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:11:10.152 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:10.147+0000 7fb3b1d8c740 1 --2- 172.21.15.49:0/3837007915 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5573be101ef0 0x5573be116f50 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fb39c02f020 tx=0x7fb39c005dd0 comp rx=0 tx=0).stop 2024-03-18T23:11:10.152 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:10.148+0000 7fb3b1d8c740 1 -- 172.21.15.49:0/3837007915 shutdown_connections 2024-03-18T23:11:10.152 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:10.148+0000 7fb3b1d8c740 1 --2- 172.21.15.49:0/3837007915 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5573be10b5e0 0x5573be1231a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:10.153 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:10.148+0000 7fb3b1d8c740 1 --2- 172.21.15.49:0/3837007915 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5573be102810 0x5573be0fb930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:10.153 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:10.148+0000 7fb3b1d8c740 1 --2- 172.21.15.49:0/3837007915 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5573be101ef0 0x5573be116f50 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:10.153 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:10.148+0000 7fb3b1d8c740 1 -- 172.21.15.49:0/3837007915 >> 172.21.15.49:0/3837007915 conn(0x5573be0fc810 msgr2=0x5573be0fb220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:11:10.153 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:10.148+0000 7fb3b1d8c740 1 -- 172.21.15.49:0/3837007915 shutdown_connections 2024-03-18T23:11:10.153 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:10.148+0000 7fb3b1d8c740 1 -- 172.21.15.49:0/3837007915 wait complete. 2024-03-18T23:11:10.153 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:10.148+0000 7fb3b1d8c740 1 -- start start 2024-03-18T23:11:10.153 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:10.148+0000 7fb3b1d8c740 1 Processor -- start 2024-03-18T23:11:10.153 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:10.148+0000 7fb3b1d8c740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5573be101ef0 0x5573be113580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:11:10.153 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:10.148+0000 7fb3b1d8c740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5573be102810 0x5573be104600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:11:10.154 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:10.148+0000 7fb3b1d8c740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5573be10b5e0 0x5573be104b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:11:10.154 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:10.148+0000 7fb3b1d8c740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fb3a81b0990 con 0x5573be10b5e0 2024-03-18T23:11:10.154 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:10.148+0000 7fb3b1d8c740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fb3a81b02c0 con 0x5573be102810 2024-03-18T23:11:10.154 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:10.148+0000 7fb3b1d8c740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb3a81ad500 con 0x5573be101ef0 2024-03-18T23:11:10.154 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:10.148+0000 7fb3b1d8c740 10 client.0 _ll_register_callbacks cb 0x7fb3a009b350 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T23:11:10.154 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:10.149+0000 7fb3ad663640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5573be10b5e0 0x5573be104b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:11:10.154 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:10.149+0000 7fb3ace62640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5573be101ef0 0x5573be113580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:11:10.154 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:10.149+0000 7fb3ad663640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5573be10b5e0 0x5573be104b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:36850/0 (socket says 172.21.15.49:36850) 2024-03-18T23:11:10.154 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:10.149+0000 7fb3a7fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5573be102810 0x5573be104600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:11:10.154 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:10.149+0000 7fb3ace62640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5573be101ef0 0x5573be113580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.49:51286/0 (socket says 172.21.15.49:51286) 2024-03-18T23:11:10.154 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:10.149+0000 7fb3ad663640 1 -- 172.21.15.49:0/3937679901 learned_addr learned my addr 172.21.15.49:0/3937679901 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:11:10.154 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:10.149+0000 7fb3ad663640 1 -- 172.21.15.49:0/3937679901 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5573be101ef0 msgr2=0x5573be113580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:11:10.154 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:10.149+0000 7fb3ad663640 1 --2- 172.21.15.49:0/3937679901 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5573be101ef0 0x5573be113580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:10.155 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:10.149+0000 7fb3ad663640 1 -- 172.21.15.49:0/3937679901 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5573be102810 msgr2=0x5573be104600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:11:10.155 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:10.149+0000 7fb3ad663640 1 --2- 172.21.15.49:0/3937679901 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5573be102810 0x5573be104600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:10.155 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:10.149+0000 7fb3ad663640 1 -- 172.21.15.49:0/3937679901 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5573be113b90 con 0x5573be10b5e0 2024-03-18T23:11:10.155 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:10.149+0000 7fb3ace62640 1 --2- 172.21.15.49:0/3937679901 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5573be101ef0 0x5573be113580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:11:10.155 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:10.149+0000 7fb3ad663640 1 --2- 172.21.15.49:0/3937679901 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5573be10b5e0 0x5573be104b40 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fb3a00a3c60 tx=0x7fb3a000ef90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ad972cb489b1795 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:11:10.155 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:10.149+0000 7fb387fff640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-18T23:11:10.155 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:10.149+0000 7fb387fff640 1 -- 172.21.15.49:0/3937679901 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3a00b8070 con 0x5573be10b5e0 2024-03-18T23:11:10.155 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:10.149+0000 7fb387fff640 1 -- 172.21.15.49:0/3937679901 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7fb3a005d370 con 0x5573be10b5e0 2024-03-18T23:11:10.155 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:10.149+0000 7fb3b1d8c740 1 -- 172.21.15.49:0/3937679901 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb39c02cb80 con 0x5573be10b5e0 2024-03-18T23:11:10.155 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:10.149+0000 7fb387fff640 1 -- 172.21.15.49:0/3937679901 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fb3a0081050 con 0x5573be10b5e0 2024-03-18T23:11:10.155 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:10.149+0000 7fb3b1d8c740 10 client.15693 fetch_fsmap learned FSMap version 30 2024-03-18T23:11:10.155 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:10.149+0000 7fb3b1d8c740 1 -- 172.21.15.49:0/3937679901 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5573be1811c0 con 0x5573be10b5e0 2024-03-18T23:11:10.156 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:10.149+0000 7fb387fff640 1 -- 172.21.15.49:0/3937679901 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb39c02cb80 con 0x5573be10b5e0 2024-03-18T23:11:10.156 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:10.150+0000 7fb387fff640 1 -- 172.21.15.49:0/3937679901 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7fb3a00a3d00 con 0x5573be10b5e0 2024-03-18T23:11:10.156 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:10.150+0000 7fb3b1d8c740 10 client.15693 fetch_fsmap finished waiting for FSMap version 30 2024-03-18T23:11:10.156 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:10.150+0000 7fb3b1d8c740 10 client.15693 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T23:11:10.156 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:10.150+0000 7fb3b1d8c740 20 client.15693 populate_metadata read hostname 'smithi049' 2024-03-18T23:11:10.156 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:10.150+0000 7fb3b1d8c740 1 --2- 172.21.15.49:0/3937679901 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x5573be181910 0x5573be183d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:11:10.156 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:10.150+0000 7fb3b1d8c740 4 client.15693 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T23:11:10.156 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:10.150+0000 7fb3b1d8c740 1 -- 172.21.15.49:0/3937679901 --> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5573be06d470 con 0x5573be181910 2024-03-18T23:11:10.156 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:10.150+0000 7fb3ace62640 1 --2- 172.21.15.49:0/3937679901 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x5573be181910 0x5573be183d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:11:10.156 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:10.151+0000 7fb387fff640 10 client.15693 ms_handle_connect on v2:172.21.15.139:6838/4280128209 2024-03-18T23:11:10.156 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:10.151+0000 7fb3ace62640 1 --2- 172.21.15.49:0/3937679901 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x5573be181910 0x5573be183d30 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6ef2657e9c02cc1d server_cookie=6b46cb643c8d723 in_seq=0 out_seq=0 2024-03-18T23:11:10.295 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:10.293+0000 7fb387fff640 1 -- 172.21.15.49:0/3937679901 <== mds.0 v2:172.21.15.139:6838/4280128209 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5573be06d470 con 0x5573be181910 2024-03-18T23:11:10.295 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:10.293+0000 7fb387fff640 10 client.15693 handle_command_reply: tid=0 2024-03-18T23:11:10.295 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:10.293+0000 7fb3b1d8c740 1 client.15693 shutdown 2024-03-18T23:11:10.295 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:10.293+0000 7fb3b1d8c740 1 -- 172.21.15.49:0/3937679901 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5573be10b5e0 msgr2=0x5573be104b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:11:10.295 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:10.293+0000 7fb3b1d8c740 1 --2- 172.21.15.49:0/3937679901 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5573be10b5e0 0x5573be104b40 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fb3a00a3c60 tx=0x7fb3a000ef90 comp rx=0 tx=0).stop 2024-03-18T23:11:10.295 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:10.293+0000 7fb3b1d8c740 1 -- 172.21.15.49:0/3937679901 shutdown_connections 2024-03-18T23:11:10.295 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:10.293+0000 7fb3b1d8c740 1 --2- 172.21.15.49:0/3937679901 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x5573be181910 0x5573be183d30 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:10.295 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:10.293+0000 7fb3b1d8c740 1 --2- 172.21.15.49:0/3937679901 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5573be10b5e0 0x5573be104b40 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:10.295 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:10.293+0000 7fb3b1d8c740 1 --2- 172.21.15.49:0/3937679901 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5573be102810 0x5573be104600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:10.295 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:10.293+0000 7fb3b1d8c740 1 --2- 172.21.15.49:0/3937679901 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5573be101ef0 0x5573be113580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:10.295 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:10.293+0000 7fb387fff640 0 client.15693 ms_handle_reset on v2:172.21.15.139:6838/4280128209 2024-03-18T23:11:10.296 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:10.293+0000 7fb3b1d8c740 1 -- 172.21.15.49:0/3937679901 >> 172.21.15.49:0/3937679901 conn(0x5573be0fc810 msgr2=0x5573be0fbab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:11:10.296 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:10.293+0000 7fb3b1d8c740 1 -- 172.21.15.49:0/3937679901 shutdown_connections 2024-03-18T23:11:10.296 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:10.293+0000 7fb3b1d8c740 1 -- 172.21.15.49:0/3937679901 wait complete. 2024-03-18T23:11:10.296 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:10.293+0000 7fb3b1d8c740 20 client.15693 trim_cache size 0 max 16384 2024-03-18T23:11:10.296 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:{ 2024-03-18T23:11:10.296 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "message": "", 2024-03-18T23:11:10.296 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "return_code": 0 2024-03-18T23:11:10.296 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:} 2024-03-18T23:11:10.297 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:10.294+0000 7fb3af0c6640 1 -- 172.21.15.49:0/3282562410 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7fb3980a2a10 msgr2=0x7fb3980a4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:11:10.297 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:10.294+0000 7fb3af0c6640 1 --2- 172.21.15.49:0/3282562410 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7fb3980a2a10 0x7fb3980a4ed0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fb3940097f0 tx=0x7fb394013460 comp rx=0 tx=0).stop 2024-03-18T23:11:10.297 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:10.294+0000 7fb3af0c6640 1 -- 172.21.15.49:0/3282562410 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb3a8107910 msgr2=0x7fb3a81014c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:11:10.297 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:10.294+0000 7fb3af0c6640 1 --2- 172.21.15.49:0/3282562410 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb3a8107910 0x7fb3a81014c0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fb3a0005fd0 tx=0x7fb3a000ee60 comp rx=0 tx=0).stop 2024-03-18T23:11:10.297 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:10.294+0000 7fb3af0c6640 1 -- 172.21.15.49:0/3282562410 shutdown_connections 2024-03-18T23:11:10.298 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:10.294+0000 7fb3af0c6640 1 --2- 172.21.15.49:0/3282562410 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7fb3980a2a10 0x7fb3980a4ed0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:10.298 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:10.294+0000 7fb3af0c6640 1 --2- 172.21.15.49:0/3282562410 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb3a8107910 0x7fb3a81014c0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:10.298 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:10.294+0000 7fb3af0c6640 1 --2- 172.21.15.49:0/3282562410 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb3a8069840 0x7fb3a8100f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:10.298 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:10.294+0000 7fb3af0c6640 1 --2- 172.21.15.49:0/3282562410 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb3a8068ea0 0x7fb3a8100a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:10.298 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:10.294+0000 7fb3af0c6640 1 -- 172.21.15.49:0/3282562410 >> 172.21.15.49:0/3282562410 conn(0x7fb3a8077570 msgr2=0x7fb3a80fe980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:11:10.298 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:10.294+0000 7fb3af0c6640 1 -- 172.21.15.49:0/3282562410 shutdown_connections 2024-03-18T23:11:10.298 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:10.294+0000 7fb3af0c6640 1 -- 172.21.15.49:0/3282562410 wait complete. 2024-03-18T23:11:10.764 INFO:tasks.workunit.client.0.smithi049.stdout: 1 305405 11.65 MB/sec execute 12 sec latency 57.092 ms 2024-03-18T23:11:11.379 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:11:11 smithi049 ceph-mon[62991]: pgmap v90: 97 pgs: 97 active+clean; 53 MiB data, 842 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.2 MiB/s wr, 344 op/s 2024-03-18T23:11:11.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:11:11 smithi175 ceph-mon[67352]: pgmap v90: 97 pgs: 97 active+clean; 53 MiB data, 842 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.2 MiB/s wr, 344 op/s 2024-03-18T23:11:11.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:11:11 smithi139 ceph-mon[67282]: pgmap v90: 97 pgs: 97 active+clean; 53 MiB data, 842 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.2 MiB/s wr, 344 op/s 2024-03-18T23:11:11.764 INFO:tasks.workunit.client.0.smithi049.stdout: 1 308086 11.72 MB/sec execute 13 sec latency 19.275 ms 2024-03-18T23:11:12.764 INFO:tasks.workunit.client.0.smithi049.stdout: 1 309592 11.41 MB/sec execute 14 sec latency 103.860 ms 2024-03-18T23:11:13.379 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:11:13 smithi049 ceph-mon[62991]: pgmap v91: 97 pgs: 97 active+clean; 43 MiB data, 844 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 4.7 MiB/s wr, 547 op/s 2024-03-18T23:11:13.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:11:13 smithi175 ceph-mon[67352]: pgmap v91: 97 pgs: 97 active+clean; 43 MiB data, 844 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 4.7 MiB/s wr, 547 op/s 2024-03-18T23:11:13.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:11:13 smithi139 ceph-mon[67282]: pgmap v91: 97 pgs: 97 active+clean; 43 MiB data, 844 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 4.7 MiB/s wr, 547 op/s 2024-03-18T23:11:13.764 INFO:tasks.workunit.client.0.smithi049.stdout: 1 312384 11.85 MB/sec execute 15 sec latency 15.730 ms 2024-03-18T23:11:14.764 INFO:tasks.workunit.client.0.smithi049.stdout: 1 314714 11.62 MB/sec execute 16 sec latency 23.960 ms 2024-03-18T23:11:15.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:11:15 smithi049 ceph-mon[62991]: pgmap v92: 97 pgs: 97 active+clean; 43 MiB data, 844 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.2 MiB/s wr, 311 op/s 2024-03-18T23:11:15.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:11:15 smithi175 ceph-mon[67352]: pgmap v92: 97 pgs: 97 active+clean; 43 MiB data, 844 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.2 MiB/s wr, 311 op/s 2024-03-18T23:11:15.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:11:15 smithi139 ceph-mon[67282]: pgmap v92: 97 pgs: 97 active+clean; 43 MiB data, 844 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.2 MiB/s wr, 311 op/s 2024-03-18T23:11:15.764 INFO:tasks.workunit.client.0.smithi049.stdout: 1 316514 11.63 MB/sec execute 17 sec latency 27.416 ms 2024-03-18T23:11:16.764 INFO:tasks.workunit.client.0.smithi049.stdout: 1 319406 11.94 MB/sec execute 18 sec latency 11.814 ms 2024-03-18T23:11:17.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:11:17 smithi049 ceph-mon[62991]: pgmap v93: 97 pgs: 97 active+clean; 43 MiB data, 844 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.2 MiB/s wr, 311 op/s 2024-03-18T23:11:17.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:11:17 smithi175 ceph-mon[67352]: pgmap v93: 97 pgs: 97 active+clean; 43 MiB data, 844 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.2 MiB/s wr, 311 op/s 2024-03-18T23:11:17.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:11:17 smithi139 ceph-mon[67282]: pgmap v93: 97 pgs: 97 active+clean; 43 MiB data, 844 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.2 MiB/s wr, 311 op/s 2024-03-18T23:11:17.764 INFO:tasks.workunit.client.0.smithi049.stdout: 1 320840 11.66 MB/sec execute 19 sec latency 117.853 ms 2024-03-18T23:11:18.764 INFO:tasks.workunit.client.0.smithi049.stdout: 1 323237 11.77 MB/sec execute 20 sec latency 17.349 ms 2024-03-18T23:11:19.379 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:11:19 smithi049 ceph-mon[62991]: pgmap v94: 97 pgs: 97 active+clean; 48 MiB data, 902 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 4.6 MiB/s wr, 418 op/s 2024-03-18T23:11:19.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:11:19 smithi175 ceph-mon[67352]: pgmap v94: 97 pgs: 97 active+clean; 48 MiB data, 902 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 4.6 MiB/s wr, 418 op/s 2024-03-18T23:11:19.566 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:11:19 smithi139 ceph-mon[67282]: pgmap v94: 97 pgs: 97 active+clean; 48 MiB data, 902 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 4.6 MiB/s wr, 418 op/s 2024-03-18T23:11:19.764 INFO:tasks.workunit.client.0.smithi049.stdout: 1 326130 11.90 MB/sec execute 21 sec latency 19.548 ms 2024-03-18T23:11:20.764 INFO:tasks.workunit.client.0.smithi049.stdout: 1 327946 11.81 MB/sec execute 22 sec latency 16.248 ms 2024-03-18T23:11:21.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:11:21 smithi049 ceph-mon[62991]: pgmap v95: 97 pgs: 97 active+clean; 48 MiB data, 902 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.0 MiB/s wr, 309 op/s 2024-03-18T23:11:21.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:11:21 smithi175 ceph-mon[67352]: pgmap v95: 97 pgs: 97 active+clean; 48 MiB data, 902 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.0 MiB/s wr, 309 op/s 2024-03-18T23:11:21.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:11:21 smithi139 ceph-mon[67282]: pgmap v95: 97 pgs: 97 active+clean; 48 MiB data, 902 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.0 MiB/s wr, 309 op/s 2024-03-18T23:11:21.764 INFO:tasks.workunit.client.0.smithi049.stdout: 1 330383 11.90 MB/sec execute 23 sec latency 17.659 ms 2024-03-18T23:11:22.764 INFO:tasks.workunit.client.0.smithi049.stdout: 1 332883 11.93 MB/sec execute 24 sec latency 40.686 ms 2024-03-18T23:11:23.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:11:23 smithi049 ceph-mon[62991]: pgmap v96: 97 pgs: 97 active+clean; 54 MiB data, 974 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 4.4 MiB/s wr, 421 op/s 2024-03-18T23:11:23.389 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:23.386+0000 7f0d1b962640 1 -- 172.21.15.49:0/2652704175 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0d141051b0 msgr2=0x7f0d1410cd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:11:23.389 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:23.386+0000 7f0d1b962640 1 --2- 172.21.15.49:0/2652704175 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0d141051b0 0x7f0d1410cd80 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f0d0c009ca0 tx=0x7f0d0c02c0f0 comp rx=0 tx=0).stop 2024-03-18T23:11:23.389 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:23.386+0000 7f0d1b962640 1 -- 172.21.15.49:0/2652704175 shutdown_connections 2024-03-18T23:11:23.389 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:23.386+0000 7f0d1b962640 1 --2- 172.21.15.49:0/2652704175 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0d141051b0 0x7f0d1410cd80 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:23.389 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:23.386+0000 7f0d1b962640 1 --2- 172.21.15.49:0/2652704175 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0d14100a90 0x7f0d14104ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:23.389 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:23.386+0000 7f0d1b962640 1 --2- 172.21.15.49:0/2652704175 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0d141000e0 0x7f0d141004c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:23.389 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:23.386+0000 7f0d1b962640 1 -- 172.21.15.49:0/2652704175 >> 172.21.15.49:0/2652704175 conn(0x7f0d14076ce0 msgr2=0x7f0d140fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:11:23.390 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:23.386+0000 7f0d1b962640 1 -- 172.21.15.49:0/2652704175 shutdown_connections 2024-03-18T23:11:23.390 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:23.386+0000 7f0d1b962640 1 -- 172.21.15.49:0/2652704175 wait complete. 2024-03-18T23:11:23.390 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:23.387+0000 7f0d1b962640 1 Processor -- start 2024-03-18T23:11:23.390 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:23.387+0000 7f0d1b962640 1 -- start start 2024-03-18T23:11:23.390 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:23.387+0000 7f0d1b962640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0d141000e0 0x7f0d14198000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:11:23.390 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:23.387+0000 7f0d1b962640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0d141051b0 0x7f0d14198540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:11:23.390 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:23.387+0000 7f0d1b962640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0d1419c8c0 0x7f0d1419cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:11:23.390 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:23.387+0000 7f0d1b962640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f0d14111c00 con 0x7f0d141000e0 2024-03-18T23:11:23.390 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:23.387+0000 7f0d1b962640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f0d14111a80 con 0x7f0d1419c8c0 2024-03-18T23:11:23.390 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:23.387+0000 7f0d1b962640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f0d14111d80 con 0x7f0d141051b0 2024-03-18T23:11:23.390 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:23.387+0000 7f0d196fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0d141000e0 0x7f0d14198000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:11:23.391 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:23.387+0000 7f0d19eff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0d1419c8c0 0x7f0d1419cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:11:23.391 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:23.387+0000 7f0d196fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0d141000e0 0x7f0d14198000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:54362/0 (socket says 172.21.15.49:54362) 2024-03-18T23:11:23.391 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:23.387+0000 7f0d196fe640 1 -- 172.21.15.49:0/2755074613 learned_addr learned my addr 172.21.15.49:0/2755074613 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:11:23.391 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:23.387+0000 7f0d18efd640 1 --2- 172.21.15.49:0/2755074613 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0d141051b0 0x7f0d14198540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:11:23.391 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:23.387+0000 7f0d19eff640 1 -- 172.21.15.49:0/2755074613 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0d141051b0 msgr2=0x7f0d14198540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:11:23.391 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:23.387+0000 7f0d19eff640 1 --2- 172.21.15.49:0/2755074613 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0d141051b0 0x7f0d14198540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:23.391 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:23.387+0000 7f0d19eff640 1 -- 172.21.15.49:0/2755074613 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0d141000e0 msgr2=0x7f0d14198000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:11:23.391 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:23.387+0000 7f0d19eff640 1 --2- 172.21.15.49:0/2755074613 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0d141000e0 0x7f0d14198000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:23.391 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:23.387+0000 7f0d19eff640 1 -- 172.21.15.49:0/2755074613 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d1419d3b0 con 0x7f0d1419c8c0 2024-03-18T23:11:23.391 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:23.387+0000 7f0d196fe640 1 --2- 172.21.15.49:0/2755074613 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0d141000e0 0x7f0d14198000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:11:23.391 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:23.388+0000 7f0d19eff640 1 --2- 172.21.15.49:0/2755074613 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0d1419c8c0 0x7f0d1419cd70 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f0d0c009bf0 tx=0x7f0d0c032ea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:11:23.391 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:23.388+0000 7f0d0a7fc640 1 -- 172.21.15.49:0/2755074613 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d0c041020 con 0x7f0d1419c8c0 2024-03-18T23:11:23.392 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:23.388+0000 7f0d0a7fc640 1 -- 172.21.15.49:0/2755074613 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d0c034cd0 con 0x7f0d1419c8c0 2024-03-18T23:11:23.392 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:23.388+0000 7f0d1b962640 1 -- 172.21.15.49:0/2755074613 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d1419d640 con 0x7f0d1419c8c0 2024-03-18T23:11:23.392 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:23.388+0000 7f0d0a7fc640 1 -- 172.21.15.49:0/2755074613 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d0c039040 con 0x7f0d1419c8c0 2024-03-18T23:11:23.392 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:23.388+0000 7f0d1b962640 1 -- 172.21.15.49:0/2755074613 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d141a4da0 con 0x7f0d1419c8c0 2024-03-18T23:11:23.399 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:23.390+0000 7f0d0a7fc640 1 -- 172.21.15.49:0/2755074613 <== mon.2 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 25) ==== 137421+0+0 (secure 0 0 0) 0x7f0d0c044020 con 0x7f0d1419c8c0 2024-03-18T23:11:23.399 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:23.390+0000 7f0d1e62d740 1 Processor -- start 2024-03-18T23:11:23.399 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:23.391+0000 7f0d1e62d740 1 -- start start 2024-03-18T23:11:23.399 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:23.391+0000 7f0d1e62d740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563823b57990 0x563823b57d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:11:23.399 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:23.391+0000 7f0d1e62d740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563823b582b0 0x563823b654b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:11:23.400 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:23.391+0000 7f0d1e62d740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563823b659f0 0x563823b67de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:11:23.400 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:23.391+0000 7f0d1e62d740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x563823abbbe0 con 0x563823b57990 2024-03-18T23:11:23.400 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:23.391+0000 7f0d1e62d740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x563823a64960 con 0x563823b582b0 2024-03-18T23:11:23.400 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:23.391+0000 7f0d1e62d740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x563823ab7c80 con 0x563823b659f0 2024-03-18T23:11:23.400 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:23.391+0000 7f0d0a7fc640 1 --2- 172.21.15.49:0/2755074613 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f0d000a2a10 0x7f0d000a4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:11:23.400 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:23.391+0000 7f0d18efd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563823b57990 0x563823b57d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:11:23.400 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:23.391+0000 7f0d18efd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563823b57990 0x563823b57d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:54374/0 (socket says 172.21.15.49:54374) 2024-03-18T23:11:23.400 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:23.391+0000 7f0d18efd640 1 -- 172.21.15.49:0/3526477776 learned_addr learned my addr 172.21.15.49:0/3526477776 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:11:23.400 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:23.391+0000 7f0d196fe640 1 --2- 172.21.15.49:0/3526477776 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563823b582b0 0x563823b654b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:11:23.400 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:23.391+0000 7f0d18efd640 1 --2- 172.21.15.49:0/3526477776 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563823b659f0 0x563823b67de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:11:23.400 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:23.391+0000 7f0d0a7fc640 1 -- 172.21.15.49:0/2755074613 <== mon.2 v2:172.21.15.139:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f0d0c0f0930 con 0x7f0d1419c8c0 2024-03-18T23:11:23.400 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:23.391+0000 7f0d19eff640 1 --2- 172.21.15.49:0/2755074613 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f0d000a2a10 0x7f0d000a4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:11:23.400 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:23.391+0000 7f0d196fe640 1 -- 172.21.15.49:0/3526477776 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563823b659f0 msgr2=0x563823b67de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:11:23.401 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:23.391+0000 7f0d196fe640 1 --2- 172.21.15.49:0/3526477776 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563823b659f0 0x563823b67de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:23.401 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:23.391+0000 7f0d196fe640 1 -- 172.21.15.49:0/3526477776 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563823b57990 msgr2=0x563823b57d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:11:23.401 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:23.391+0000 7f0d196fe640 1 --2- 172.21.15.49:0/3526477776 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563823b57990 0x563823b57d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:23.401 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:23.391+0000 7f0d196fe640 1 -- 172.21.15.49:0/3526477776 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563823a5bd20 con 0x563823b582b0 2024-03-18T23:11:23.401 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:23.391+0000 7f0d18efd640 1 --2- 172.21.15.49:0/3526477776 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563823b57990 0x563823b57d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:11:23.401 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:23.392+0000 7f0d19eff640 1 --2- 172.21.15.49:0/2755074613 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f0d000a2a10 0x7f0d000a4ed0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f0d0c0b9aa0 tx=0x7f0d0c02c0f0 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:11:23.401 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:23.392+0000 7f0d196fe640 1 --2- 172.21.15.49:0/3526477776 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563823b582b0 0x563823b654b0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f0d10011540 tx=0x7f0d10011a00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2c69acebea097113 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:11:23.401 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:23.392+0000 7f0cff7fe640 1 -- 172.21.15.49:0/3526477776 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d1000f640 con 0x563823b582b0 2024-03-18T23:11:23.401 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:23.392+0000 7f0cff7fe640 1 -- 172.21.15.49:0/3526477776 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d1000fcc0 con 0x563823b582b0 2024-03-18T23:11:23.401 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:23.392+0000 7f0cff7fe640 1 -- 172.21.15.49:0/3526477776 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d10019d30 con 0x563823b582b0 2024-03-18T23:11:23.401 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:23.392+0000 7f0d1e62d740 1 -- 172.21.15.49:0/3526477776 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563823b582b0 msgr2=0x563823b654b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:11:23.401 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:23.392+0000 7f0d1e62d740 1 --2- 172.21.15.49:0/3526477776 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563823b582b0 0x563823b654b0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f0d10011540 tx=0x7f0d10011a00 comp rx=0 tx=0).stop 2024-03-18T23:11:23.401 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:23.392+0000 7f0d1e62d740 1 -- 172.21.15.49:0/3526477776 shutdown_connections 2024-03-18T23:11:23.402 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:23.392+0000 7f0d1e62d740 1 --2- 172.21.15.49:0/3526477776 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563823b659f0 0x563823b67de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:23.402 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:23.392+0000 7f0d1e62d740 1 --2- 172.21.15.49:0/3526477776 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563823b582b0 0x563823b654b0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:23.402 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:23.392+0000 7f0d1e62d740 1 --2- 172.21.15.49:0/3526477776 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563823b57990 0x563823b57d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:23.402 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:23.392+0000 7f0d1e62d740 1 -- 172.21.15.49:0/3526477776 >> 172.21.15.49:0/3526477776 conn(0x563823b52810 msgr2=0x563823b52bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:11:23.402 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:23.393+0000 7f0d1e62d740 1 -- 172.21.15.49:0/3526477776 shutdown_connections 2024-03-18T23:11:23.402 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:23.393+0000 7f0d1e62d740 1 -- 172.21.15.49:0/3526477776 wait complete. 2024-03-18T23:11:23.402 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:23.393+0000 7f0d1e62d740 1 -- start start 2024-03-18T23:11:23.402 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:23.393+0000 7f0d1e62d740 1 Processor -- start 2024-03-18T23:11:23.402 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:23.393+0000 7f0d1e62d740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563823b57990 0x563823b653b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:11:23.402 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:23.393+0000 7f0d1e62d740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563823b582b0 0x563823b62740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:11:23.402 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:23.393+0000 7f0d1e62d740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563823b659f0 0x563823b62d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:11:23.402 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:23.393+0000 7f0d1e62d740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x563823b68620 con 0x563823b57990 2024-03-18T23:11:23.402 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:23.393+0000 7f0d1e62d740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x563823a77db0 con 0x563823b659f0 2024-03-18T23:11:23.403 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:23.393+0000 7f0d1e62d740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x563823b687a0 con 0x563823b582b0 2024-03-18T23:11:23.403 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:23.393+0000 7f0d1e62d740 10 client.0 _ll_register_callbacks cb 0x563823a7ce70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T23:11:23.403 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:23.393+0000 7f0d18efd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563823b57990 0x563823b653b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:11:23.403 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:23.394+0000 7f0d18efd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563823b57990 0x563823b653b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:54386/0 (socket says 172.21.15.49:54386) 2024-03-18T23:11:23.403 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:23.394+0000 7f0d196fe640 1 --2- 172.21.15.49:0/4115929009 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563823b582b0 0x563823b62740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:11:23.403 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:23.394+0000 7f0d18efd640 1 -- 172.21.15.49:0/4115929009 learned_addr learned my addr 172.21.15.49:0/4115929009 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:11:23.403 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:23.394+0000 7f0d18efd640 1 --2- 172.21.15.49:0/4115929009 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563823b659f0 0x563823b62d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:11:23.403 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:23.394+0000 7f0d18efd640 1 -- 172.21.15.49:0/4115929009 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563823b582b0 msgr2=0x563823b62740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:11:23.403 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:23.394+0000 7f0d18efd640 1 --2- 172.21.15.49:0/4115929009 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563823b582b0 0x563823b62740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:23.403 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:23.394+0000 7f0d18efd640 1 -- 172.21.15.49:0/4115929009 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563823b659f0 msgr2=0x563823b62d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:11:23.403 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:23.394+0000 7f0d18efd640 1 --2- 172.21.15.49:0/4115929009 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563823b659f0 0x563823b62d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:23.403 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:23.394+0000 7f0d18efd640 1 -- 172.21.15.49:0/4115929009 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563823a76c60 con 0x563823b57990 2024-03-18T23:11:23.403 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:23.394+0000 7f0d196fe640 1 --2- 172.21.15.49:0/4115929009 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563823b582b0 0x563823b62740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:11:23.404 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:23.394+0000 7f0d18efd640 1 --2- 172.21.15.49:0/4115929009 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563823b57990 0x563823b653b0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f0d040133f0 tx=0x7f0d04005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=637229cbaa64e593 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:11:23.404 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:23.394+0000 7f0cfcff9640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-18T23:11:23.404 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:23.394+0000 7f0cfcff9640 1 -- 172.21.15.49:0/4115929009 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d04004320 con 0x563823b57990 2024-03-18T23:11:23.404 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:23.394+0000 7f0cfcff9640 1 -- 172.21.15.49:0/4115929009 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f0d0400f980 con 0x563823b57990 2024-03-18T23:11:23.404 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:23.394+0000 7f0d1e62d740 1 -- 172.21.15.49:0/4115929009 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0d1000fcc0 con 0x563823b57990 2024-03-18T23:11:23.404 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:23.394+0000 7f0cfcff9640 1 -- 172.21.15.49:0/4115929009 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f0d04029050 con 0x563823b57990 2024-03-18T23:11:23.404 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:23.394+0000 7f0d1e62d740 10 client.15705 fetch_fsmap learned FSMap version 30 2024-03-18T23:11:23.404 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:23.394+0000 7f0d1e62d740 1 -- 172.21.15.49:0/4115929009 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5638237d0210 con 0x563823b57990 2024-03-18T23:11:23.404 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:23.395+0000 7f0cfcff9640 1 -- 172.21.15.49:0/4115929009 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d1000fcc0 con 0x563823b57990 2024-03-18T23:11:23.404 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:23.395+0000 7f0cfcff9640 1 -- 172.21.15.49:0/4115929009 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f0d04003ce0 con 0x563823b57990 2024-03-18T23:11:23.404 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:23.395+0000 7f0d1e62d740 10 client.15705 fetch_fsmap finished waiting for FSMap version 30 2024-03-18T23:11:23.404 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:23.395+0000 7f0d1e62d740 10 client.15705 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T23:11:23.404 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:23.395+0000 7f0d1e62d740 20 client.15705 populate_metadata read hostname 'smithi049' 2024-03-18T23:11:23.405 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:23.395+0000 7f0d1e62d740 1 --2- 172.21.15.49:0/4115929009 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x563823b759c0 0x563823b77db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:11:23.405 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:23.395+0000 7f0d1e62d740 4 client.15705 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T23:11:23.405 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:23.395+0000 7f0d1e62d740 1 -- 172.21.15.49:0/4115929009 --> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563823ac9120 con 0x563823b759c0 2024-03-18T23:11:23.405 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:23.395+0000 7f0d196fe640 1 --2- 172.21.15.49:0/4115929009 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x563823b759c0 0x563823b77db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:11:23.405 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:23.395+0000 7f0cfcff9640 10 client.15705 ms_handle_connect on v2:172.21.15.139:6838/4280128209 2024-03-18T23:11:23.405 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:23.395+0000 7f0d196fe640 1 --2- 172.21.15.49:0/4115929009 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x563823b759c0 0x563823b77db0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=341560dafebec907 server_cookie=2dd69e9b1333dc1f in_seq=0 out_seq=0 2024-03-18T23:11:23.405 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:23.397+0000 7f0cfcff9640 1 -- 172.21.15.49:0/4115929009 <== mds.0 v2:172.21.15.139:6838/4280128209 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x563823ac9120 con 0x563823b759c0 2024-03-18T23:11:23.405 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:23.397+0000 7f0cfcff9640 10 client.15705 handle_command_reply: tid=0 2024-03-18T23:11:23.405 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:23.397+0000 7f0d1e62d740 1 client.15705 shutdown 2024-03-18T23:11:23.405 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:23.397+0000 7f0d1e62d740 1 -- 172.21.15.49:0/4115929009 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563823b57990 msgr2=0x563823b653b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:11:23.405 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:23.397+0000 7f0d1e62d740 1 --2- 172.21.15.49:0/4115929009 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563823b57990 0x563823b653b0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f0d040133f0 tx=0x7f0d04005f90 comp rx=0 tx=0).stop 2024-03-18T23:11:23.405 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:23.397+0000 7f0d1e62d740 1 -- 172.21.15.49:0/4115929009 shutdown_connections 2024-03-18T23:11:23.405 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:23.397+0000 7f0d1e62d740 1 --2- 172.21.15.49:0/4115929009 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x563823b759c0 0x563823b77db0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:23.405 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:23.397+0000 7f0d1e62d740 1 --2- 172.21.15.49:0/4115929009 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563823b659f0 0x563823b62d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:23.406 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:23.397+0000 7f0d1e62d740 1 --2- 172.21.15.49:0/4115929009 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563823b582b0 0x563823b62740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:23.406 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:23.397+0000 7f0d1e62d740 1 --2- 172.21.15.49:0/4115929009 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563823b57990 0x563823b653b0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:23.406 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:23.397+0000 7f0d1e62d740 1 -- 172.21.15.49:0/4115929009 >> 172.21.15.49:0/4115929009 conn(0x563823b52810 msgr2=0x563823b54560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:11:23.406 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:23.397+0000 7f0cfcff9640 0 client.15705 ms_handle_reset on v2:172.21.15.139:6838/4280128209 2024-03-18T23:11:23.406 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:23.397+0000 7f0d1e62d740 1 -- 172.21.15.49:0/4115929009 shutdown_connections 2024-03-18T23:11:23.406 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:23.397+0000 7f0d1e62d740 1 -- 172.21.15.49:0/4115929009 wait complete. 2024-03-18T23:11:23.406 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:23.397+0000 7f0d1e62d740 20 client.15705 trim_cache size 0 max 16384 2024-03-18T23:11:23.417 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:23.412+0000 7f0d1e62d740 1 Processor -- start 2024-03-18T23:11:23.417 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:23.412+0000 7f0d1e62d740 1 -- start start 2024-03-18T23:11:23.418 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:23.412+0000 7f0d1e62d740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563823b57990 0x563823b71430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:11:23.418 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:23.412+0000 7f0d1e62d740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563823b582b0 0x563823b618c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:11:23.418 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:23.412+0000 7f0d1e62d740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563823b659f0 0x563823b5db60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:11:23.418 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:23.412+0000 7f0d1e62d740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f0d141ab120 con 0x563823b582b0 2024-03-18T23:11:23.418 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:23.412+0000 7f0d1e62d740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f0d141acc60 con 0x563823b659f0 2024-03-18T23:11:23.418 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:23.412+0000 7f0d1e62d740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f0d141ab7f0 con 0x563823b57990 2024-03-18T23:11:23.418 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:23.412+0000 7f0d196fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563823b582b0 0x563823b618c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:11:23.418 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:23.412+0000 7f0d196fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563823b582b0 0x563823b618c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:54388/0 (socket says 172.21.15.49:54388) 2024-03-18T23:11:23.418 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:23.412+0000 7f0d196fe640 1 -- 172.21.15.49:0/1954349606 learned_addr learned my addr 172.21.15.49:0/1954349606 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:11:23.418 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:23.412+0000 7f0d18efd640 1 --2- 172.21.15.49:0/1954349606 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563823b57990 0x563823b71430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:11:23.418 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:23.412+0000 7f0d18efd640 1 --2- 172.21.15.49:0/1954349606 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563823b659f0 0x563823b5db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:11:23.418 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:23.412+0000 7f0d196fe640 1 -- 172.21.15.49:0/1954349606 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563823b57990 msgr2=0x563823b71430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:11:23.419 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:23.412+0000 7f0d196fe640 1 --2- 172.21.15.49:0/1954349606 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563823b57990 0x563823b71430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:23.419 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:23.412+0000 7f0d196fe640 1 -- 172.21.15.49:0/1954349606 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563823b659f0 msgr2=0x563823b5db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:11:23.419 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:23.412+0000 7f0d196fe640 1 --2- 172.21.15.49:0/1954349606 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563823b659f0 0x563823b5db60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:23.419 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:23.412+0000 7f0d196fe640 1 -- 172.21.15.49:0/1954349606 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56382389c620 con 0x563823b582b0 2024-03-18T23:11:23.419 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:23.412+0000 7f0d18efd640 1 --2- 172.21.15.49:0/1954349606 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563823b659f0 0x563823b5db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T23:11:23.419 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:23.413+0000 7f0d196fe640 1 --2- 172.21.15.49:0/1954349606 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563823b582b0 0x563823b618c0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f0d100198a0 tx=0x7f0d10019830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45be7b14220c1deb server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:11:23.419 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:23.413+0000 7f0cfeffd640 1 -- 172.21.15.49:0/1954349606 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d10019bd0 con 0x563823b582b0 2024-03-18T23:11:23.419 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:23.413+0000 7f0cfeffd640 1 -- 172.21.15.49:0/1954349606 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d10019d70 con 0x563823b582b0 2024-03-18T23:11:23.419 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:23.413+0000 7f0cfeffd640 1 -- 172.21.15.49:0/1954349606 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d1000f850 con 0x563823b582b0 2024-03-18T23:11:23.419 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:23.413+0000 7f0d1e62d740 1 -- 172.21.15.49:0/1954349606 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563823b582b0 msgr2=0x563823b618c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:11:23.419 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:23.413+0000 7f0d1e62d740 1 --2- 172.21.15.49:0/1954349606 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563823b582b0 0x563823b618c0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f0d100198a0 tx=0x7f0d10019830 comp rx=0 tx=0).stop 2024-03-18T23:11:23.419 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:23.413+0000 7f0d1e62d740 1 -- 172.21.15.49:0/1954349606 shutdown_connections 2024-03-18T23:11:23.419 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:23.413+0000 7f0d1e62d740 1 --2- 172.21.15.49:0/1954349606 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563823b659f0 0x563823b5db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:23.419 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:23.413+0000 7f0d1e62d740 1 --2- 172.21.15.49:0/1954349606 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563823b582b0 0x563823b618c0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:23.420 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:23.413+0000 7f0d1e62d740 1 --2- 172.21.15.49:0/1954349606 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563823b57990 0x563823b71430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:23.420 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:23.413+0000 7f0d1e62d740 1 -- 172.21.15.49:0/1954349606 >> 172.21.15.49:0/1954349606 conn(0x563823b52810 msgr2=0x563823b5d450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:11:23.420 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:23.413+0000 7f0d1e62d740 1 -- 172.21.15.49:0/1954349606 shutdown_connections 2024-03-18T23:11:23.420 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:23.413+0000 7f0d1e62d740 1 -- 172.21.15.49:0/1954349606 wait complete. 2024-03-18T23:11:23.420 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:23.414+0000 7f0d1e62d740 1 -- start start 2024-03-18T23:11:23.420 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:23.414+0000 7f0d1e62d740 1 Processor -- start 2024-03-18T23:11:23.420 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:23.414+0000 7f0d1e62d740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563823b57990 0x563823b5fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:11:23.420 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:23.414+0000 7f0d1e62d740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563823b582b0 0x563823b60210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:11:23.420 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:23.414+0000 7f0d1e62d740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563823b659f0 0x563823b5e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:11:23.420 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:23.414+0000 7f0d1e62d740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f0d141ac590 con 0x563823b582b0 2024-03-18T23:11:23.420 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:23.414+0000 7f0d1e62d740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f0d141abec0 con 0x563823b659f0 2024-03-18T23:11:23.420 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:23.414+0000 7f0d1e62d740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f0d141a9100 con 0x563823b57990 2024-03-18T23:11:23.420 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:23.414+0000 7f0d196fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563823b582b0 0x563823b60210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:11:23.420 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:23.414+0000 7f0d1e62d740 10 client.0 _ll_register_callbacks cb 0x7f0d04013550 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T23:11:23.421 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:23.414+0000 7f0d18efd640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563823b57990 0x563823b5fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:11:23.421 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:23.414+0000 7f0d196fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563823b582b0 0x563823b60210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:54392/0 (socket says 172.21.15.49:54392) 2024-03-18T23:11:23.421 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:23.414+0000 7f0d196fe640 1 -- 172.21.15.49:0/359301977 learned_addr learned my addr 172.21.15.49:0/359301977 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:11:23.421 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:23.414+0000 7f0d18efd640 1 --2- 172.21.15.49:0/359301977 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563823b659f0 0x563823b5e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:11:23.421 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:23.415+0000 7f0d196fe640 1 -- 172.21.15.49:0/359301977 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563823b57990 msgr2=0x563823b5fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:11:23.421 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:23.415+0000 7f0d196fe640 1 --2- 172.21.15.49:0/359301977 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563823b57990 0x563823b5fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:23.421 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:23.415+0000 7f0d196fe640 1 -- 172.21.15.49:0/359301977 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563823b659f0 msgr2=0x563823b5e9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:11:23.421 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:23.415+0000 7f0d196fe640 1 --2- 172.21.15.49:0/359301977 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563823b659f0 0x563823b5e9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:23.421 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:23.415+0000 7f0d196fe640 1 -- 172.21.15.49:0/359301977 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563823b6e180 con 0x563823b582b0 2024-03-18T23:11:23.421 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:23.415+0000 7f0d18efd640 1 --2- 172.21.15.49:0/359301977 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563823b57990 0x563823b5fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:11:23.421 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:23.415+0000 7f0d196fe640 1 --2- 172.21.15.49:0/359301977 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563823b582b0 0x563823b60210 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f0d10005bf0 tx=0x7f0d1000f640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b6a4c3bb1bed63f server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:11:23.421 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:23.415+0000 7f0cfcff9640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-18T23:11:23.421 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:23.415+0000 7f0cfcff9640 1 -- 172.21.15.49:0/359301977 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d10033070 con 0x563823b582b0 2024-03-18T23:11:23.421 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:23.415+0000 7f0cfcff9640 1 -- 172.21.15.49:0/359301977 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f0d1002a040 con 0x563823b582b0 2024-03-18T23:11:23.422 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:23.415+0000 7f0d1e62d740 1 -- 172.21.15.49:0/359301977 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0d10019d70 con 0x563823b582b0 2024-03-18T23:11:23.422 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:23.415+0000 7f0cfcff9640 1 -- 172.21.15.49:0/359301977 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f0d1003f050 con 0x563823b582b0 2024-03-18T23:11:23.422 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:23.415+0000 7f0d1e62d740 10 client.15711 fetch_fsmap learned FSMap version 30 2024-03-18T23:11:23.422 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:23.415+0000 7f0d1e62d740 1 -- 172.21.15.49:0/359301977 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x563823bdb240 con 0x563823b582b0 2024-03-18T23:11:23.422 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:23.415+0000 7f0cfcff9640 1 -- 172.21.15.49:0/359301977 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d10019d70 con 0x563823b582b0 2024-03-18T23:11:23.422 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:23.416+0000 7f0cfcff9640 1 -- 172.21.15.49:0/359301977 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f0d1000fa90 con 0x563823b582b0 2024-03-18T23:11:23.422 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:23.416+0000 7f0d1e62d740 10 client.15711 fetch_fsmap finished waiting for FSMap version 30 2024-03-18T23:11:23.422 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:23.416+0000 7f0d1e62d740 10 client.15711 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T23:11:23.422 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:23.416+0000 7f0d1e62d740 20 client.15711 populate_metadata read hostname 'smithi049' 2024-03-18T23:11:23.422 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:23.416+0000 7f0d1e62d740 1 --2- 172.21.15.49:0/359301977 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x563823bdb800 0x563823bddc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:11:23.422 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:23.416+0000 7f0d1e62d740 4 client.15711 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T23:11:23.422 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:23.416+0000 7f0d1e62d740 1 -- 172.21.15.49:0/359301977 --> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563823ac3470 con 0x563823bdb800 2024-03-18T23:11:23.422 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:23.416+0000 7f0d18efd640 1 --2- 172.21.15.49:0/359301977 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x563823bdb800 0x563823bddc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:11:23.422 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:23.416+0000 7f0cfcff9640 10 client.15711 ms_handle_connect on v2:172.21.15.139:6838/4280128209 2024-03-18T23:11:23.423 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:23.416+0000 7f0d18efd640 1 --2- 172.21.15.49:0/359301977 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x563823bdb800 0x563823bddc20 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=16238c6c17223394 server_cookie=91a3ffe9855ed6ee in_seq=0 out_seq=0 2024-03-18T23:11:23.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:11:23 smithi175 ceph-mon[67352]: pgmap v96: 97 pgs: 97 active+clean; 54 MiB data, 974 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 4.4 MiB/s wr, 421 op/s 2024-03-18T23:11:23.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:11:23 smithi139 ceph-mon[67282]: pgmap v96: 97 pgs: 97 active+clean; 54 MiB data, 974 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 4.4 MiB/s wr, 421 op/s 2024-03-18T23:11:23.711 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:23.709+0000 7f0cfcff9640 1 -- 172.21.15.49:0/359301977 <== mds.0 v2:172.21.15.139:6838/4280128209 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x563823ac3470 con 0x563823bdb800 2024-03-18T23:11:23.711 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:23.709+0000 7f0cfcff9640 10 client.15711 handle_command_reply: tid=0 2024-03-18T23:11:23.711 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:23.709+0000 7f0d1e62d740 1 client.15711 shutdown 2024-03-18T23:11:23.711 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:23.709+0000 7f0d1e62d740 1 -- 172.21.15.49:0/359301977 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563823b582b0 msgr2=0x563823b60210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:11:23.712 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:23.709+0000 7f0d1e62d740 1 --2- 172.21.15.49:0/359301977 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563823b582b0 0x563823b60210 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f0d10005bf0 tx=0x7f0d1000f640 comp rx=0 tx=0).stop 2024-03-18T23:11:23.712 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:23.710+0000 7f0d1e62d740 1 -- 172.21.15.49:0/359301977 shutdown_connections 2024-03-18T23:11:23.712 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:23.710+0000 7f0d1e62d740 1 --2- 172.21.15.49:0/359301977 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x563823bdb800 0x563823bddc20 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:23.712 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:23.710+0000 7f0d1e62d740 1 --2- 172.21.15.49:0/359301977 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563823b659f0 0x563823b5e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:23.712 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:23.710+0000 7f0d1e62d740 1 --2- 172.21.15.49:0/359301977 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563823b582b0 0x563823b60210 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:23.712 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:23.710+0000 7f0d1e62d740 1 --2- 172.21.15.49:0/359301977 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563823b57990 0x563823b5fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:23.712 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:23.710+0000 7f0d1e62d740 1 -- 172.21.15.49:0/359301977 >> 172.21.15.49:0/359301977 conn(0x563823b52810 msgr2=0x563823b5d450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:11:23.712 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:23.710+0000 7f0cfcff9640 0 client.15711 ms_handle_reset on v2:172.21.15.139:6838/4280128209 2024-03-18T23:11:23.712 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:23.710+0000 7f0d1e62d740 1 -- 172.21.15.49:0/359301977 shutdown_connections 2024-03-18T23:11:23.712 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:23.710+0000 7f0d1e62d740 1 -- 172.21.15.49:0/359301977 wait complete. 2024-03-18T23:11:23.712 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:23.710+0000 7f0d1e62d740 20 client.15711 trim_cache size 0 max 16384 2024-03-18T23:11:23.713 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:{ 2024-03-18T23:11:23.713 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "message": "", 2024-03-18T23:11:23.713 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "return_code": 0 2024-03-18T23:11:23.713 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:} 2024-03-18T23:11:23.713 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:23.711+0000 7f0d1b962640 1 -- 172.21.15.49:0/2755074613 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f0d000a2a10 msgr2=0x7f0d000a4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:11:23.713 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:23.711+0000 7f0d1b962640 1 --2- 172.21.15.49:0/2755074613 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f0d000a2a10 0x7f0d000a4ed0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f0d0c0b9aa0 tx=0x7f0d0c02c0f0 comp rx=0 tx=0).stop 2024-03-18T23:11:23.713 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:23.711+0000 7f0d1b962640 1 -- 172.21.15.49:0/2755074613 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0d1419c8c0 msgr2=0x7f0d1419cd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:11:23.713 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:23.712+0000 7f0d1b962640 1 --2- 172.21.15.49:0/2755074613 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0d1419c8c0 0x7f0d1419cd70 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f0d0c009bf0 tx=0x7f0d0c032ea0 comp rx=0 tx=0).stop 2024-03-18T23:11:23.714 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:23.712+0000 7f0d1b962640 1 -- 172.21.15.49:0/2755074613 shutdown_connections 2024-03-18T23:11:23.714 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:23.712+0000 7f0d1b962640 1 --2- 172.21.15.49:0/2755074613 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f0d000a2a10 0x7f0d000a4ed0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:23.714 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:23.712+0000 7f0d1b962640 1 --2- 172.21.15.49:0/2755074613 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0d1419c8c0 0x7f0d1419cd70 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:23.715 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:23.712+0000 7f0d1b962640 1 --2- 172.21.15.49:0/2755074613 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0d141051b0 0x7f0d14198540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:23.715 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:23.712+0000 7f0d1b962640 1 --2- 172.21.15.49:0/2755074613 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0d141000e0 0x7f0d14198000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:23.715 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:23.712+0000 7f0d1b962640 1 -- 172.21.15.49:0/2755074613 >> 172.21.15.49:0/2755074613 conn(0x7f0d14076ce0 msgr2=0x7f0d141038b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:11:23.715 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:23.713+0000 7f0d1b962640 1 -- 172.21.15.49:0/2755074613 shutdown_connections 2024-03-18T23:11:23.715 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:23.713+0000 7f0d1b962640 1 -- 172.21.15.49:0/2755074613 wait complete. 2024-03-18T23:11:23.764 INFO:tasks.workunit.client.0.smithi049.stdout: 1 334220 11.73 MB/sec execute 25 sec latency 114.881 ms 2024-03-18T23:11:24.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:11:24 smithi175 ceph-mon[67352]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:11:24.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:11:24 smithi139 ceph-mon[67282]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:11:24.630 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:11:24 smithi049 ceph-mon[62991]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:11:24.764 INFO:tasks.workunit.client.0.smithi049.stdout: 1 337062 11.94 MB/sec execute 26 sec latency 15.740 ms 2024-03-18T23:11:25.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:11:25 smithi175 ceph-mon[67352]: pgmap v97: 97 pgs: 97 active+clean; 54 MiB data, 974 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.9 MiB/s wr, 218 op/s 2024-03-18T23:11:25.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:11:25 smithi175 ceph-mon[67352]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:11:25.462 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:11:25 smithi175 ceph-mon[67352]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:11:25.462 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:11:25 smithi175 ceph-mon[67352]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' 2024-03-18T23:11:25.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:11:25 smithi139 ceph-mon[67282]: pgmap v97: 97 pgs: 97 active+clean; 54 MiB data, 974 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.9 MiB/s wr, 218 op/s 2024-03-18T23:11:25.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:11:25 smithi139 ceph-mon[67282]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:11:25.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:11:25 smithi139 ceph-mon[67282]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:11:25.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:11:25 smithi139 ceph-mon[67282]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' 2024-03-18T23:11:25.629 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:11:25 smithi049 ceph-mon[62991]: pgmap v97: 97 pgs: 97 active+clean; 54 MiB data, 974 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.9 MiB/s wr, 218 op/s 2024-03-18T23:11:25.630 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:11:25 smithi049 ceph-mon[62991]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:11:25.630 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:11:25 smithi049 ceph-mon[62991]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:11:25.630 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:11:25 smithi049 ceph-mon[62991]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' 2024-03-18T23:11:25.764 INFO:tasks.workunit.client.0.smithi049.stdout: 1 339118 11.76 MB/sec execute 27 sec latency 19.621 ms 2024-03-18T23:11:26.765 INFO:tasks.workunit.client.0.smithi049.stdout: 1 341115 11.82 MB/sec execute 28 sec latency 10.191 ms 2024-03-18T23:11:27.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:11:27 smithi175 ceph-mon[67352]: pgmap v98: 97 pgs: 97 active+clean; 54 MiB data, 974 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.9 MiB/s wr, 218 op/s 2024-03-18T23:11:27.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:11:27 smithi139 ceph-mon[67282]: pgmap v98: 97 pgs: 97 active+clean; 54 MiB data, 974 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.9 MiB/s wr, 218 op/s 2024-03-18T23:11:27.630 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:11:27 smithi049 ceph-mon[62991]: pgmap v98: 97 pgs: 97 active+clean; 54 MiB data, 974 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.9 MiB/s wr, 218 op/s 2024-03-18T23:11:27.765 INFO:tasks.workunit.client.0.smithi049.stdout: 1 343506 11.85 MB/sec execute 29 sec latency 78.329 ms 2024-03-18T23:11:28.765 INFO:tasks.workunit.client.0.smithi049.stdout: 1 345175 11.74 MB/sec execute 30 sec latency 27.215 ms 2024-03-18T23:11:29.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:11:29 smithi175 ceph-mon[67352]: pgmap v99: 97 pgs: 97 active+clean; 43 MiB data, 971 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 4.6 MiB/s wr, 433 op/s 2024-03-18T23:11:29.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:11:29 smithi139 ceph-mon[67282]: pgmap v99: 97 pgs: 97 active+clean; 43 MiB data, 971 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 4.6 MiB/s wr, 433 op/s 2024-03-18T23:11:29.630 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:11:29 smithi049 ceph-mon[62991]: pgmap v99: 97 pgs: 97 active+clean; 43 MiB data, 971 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 4.6 MiB/s wr, 433 op/s 2024-03-18T23:11:29.765 INFO:tasks.workunit.client.0.smithi049.stdout: 1 347937 11.90 MB/sec execute 31 sec latency 11.303 ms 2024-03-18T23:11:30.765 INFO:tasks.workunit.client.0.smithi049.stdout: 1 350328 11.88 MB/sec execute 32 sec latency 23.698 ms 2024-03-18T23:11:31.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:11:31 smithi175 ceph-mon[67352]: pgmap v100: 97 pgs: 97 active+clean; 43 MiB data, 971 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.1 MiB/s wr, 326 op/s 2024-03-18T23:11:31.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:11:31 smithi139 ceph-mon[67282]: pgmap v100: 97 pgs: 97 active+clean; 43 MiB data, 971 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.1 MiB/s wr, 326 op/s 2024-03-18T23:11:31.629 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:11:31 smithi049 ceph-mon[62991]: pgmap v100: 97 pgs: 97 active+clean; 43 MiB data, 971 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.1 MiB/s wr, 326 op/s 2024-03-18T23:11:31.765 INFO:tasks.workunit.client.0.smithi049.stdout: 1 352137 11.79 MB/sec execute 33 sec latency 27.123 ms 2024-03-18T23:11:32.765 INFO:tasks.workunit.client.0.smithi049.stdout: 1 354666 11.93 MB/sec execute 34 sec latency 82.459 ms 2024-03-18T23:11:33.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:11:33 smithi175 ceph-mon[67352]: pgmap v101: 97 pgs: 97 active+clean; 49 MiB data, 1017 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 4.5 MiB/s wr, 426 op/s 2024-03-18T23:11:33.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:11:33 smithi139 ceph-mon[67282]: pgmap v101: 97 pgs: 97 active+clean; 49 MiB data, 1017 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 4.5 MiB/s wr, 426 op/s 2024-03-18T23:11:33.630 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:11:33 smithi049 ceph-mon[62991]: pgmap v101: 97 pgs: 97 active+clean; 49 MiB data, 1017 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 4.5 MiB/s wr, 426 op/s 2024-03-18T23:11:33.765 INFO:tasks.workunit.client.0.smithi049.stdout: 1 356110 11.79 MB/sec execute 35 sec latency 115.163 ms 2024-03-18T23:11:34.765 INFO:tasks.workunit.client.0.smithi049.stdout: 1 358680 11.85 MB/sec execute 36 sec latency 18.386 ms 2024-03-18T23:11:35.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:11:35 smithi175 ceph-mon[67352]: pgmap v102: 97 pgs: 97 active+clean; 49 MiB data, 1017 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.0 MiB/s wr, 314 op/s 2024-03-18T23:11:35.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:11:35 smithi139 ceph-mon[67282]: pgmap v102: 97 pgs: 97 active+clean; 49 MiB data, 1017 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.0 MiB/s wr, 314 op/s 2024-03-18T23:11:35.629 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:11:35 smithi049 ceph-mon[62991]: pgmap v102: 97 pgs: 97 active+clean; 49 MiB data, 1017 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.0 MiB/s wr, 314 op/s 2024-03-18T23:11:35.765 INFO:tasks.workunit.client.0.smithi049.stdout: 1 361259 11.87 MB/sec execute 37 sec latency 27.346 ms 2024-03-18T23:11:36.765 INFO:tasks.workunit.client.0.smithi049.stdout: 1 363201 11.86 MB/sec execute 38 sec latency 27.620 ms 2024-03-18T23:11:36.812 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:36.809+0000 7f7cab80b640 1 -- 172.21.15.49:0/1448689905 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7ca41056c0 msgr2=0x7f7ca4105aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:11:36.812 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:36.809+0000 7f7cab80b640 1 --2- 172.21.15.49:0/1448689905 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7ca41056c0 0x7f7ca4105aa0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f7c9c009900 tx=0x7f7c9c02bc30 comp rx=0 tx=0).stop 2024-03-18T23:11:36.812 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:36.809+0000 7f7cab80b640 1 -- 172.21.15.49:0/1448689905 shutdown_connections 2024-03-18T23:11:36.812 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:36.809+0000 7f7cab80b640 1 --2- 172.21.15.49:0/1448689905 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7ca4101330 0x7f7ca410d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:36.812 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:36.809+0000 7f7cab80b640 1 --2- 172.21.15.49:0/1448689905 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7ca4100970 0x7f7ca4100df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:36.812 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:36.809+0000 7f7cab80b640 1 --2- 172.21.15.49:0/1448689905 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7ca41056c0 0x7f7ca4105aa0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:36.812 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:36.809+0000 7f7cab80b640 1 -- 172.21.15.49:0/1448689905 >> 172.21.15.49:0/1448689905 conn(0x7f7ca4077570 msgr2=0x7f7ca4077980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:11:36.813 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:36.809+0000 7f7cab80b640 1 -- 172.21.15.49:0/1448689905 shutdown_connections 2024-03-18T23:11:36.813 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:36.809+0000 7f7cab80b640 1 -- 172.21.15.49:0/1448689905 wait complete. 2024-03-18T23:11:36.813 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:36.810+0000 7f7cab80b640 1 Processor -- start 2024-03-18T23:11:36.813 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:36.810+0000 7f7cab80b640 1 -- start start 2024-03-18T23:11:36.813 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:36.810+0000 7f7cab80b640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7ca4100970 0x7f7ca4195e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:11:36.813 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:36.810+0000 7f7cab80b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7ca4101330 0x7f7ca41963c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:11:36.813 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:36.810+0000 7f7cab80b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7ca41056c0 0x7f7ca419a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:11:36.813 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:36.810+0000 7f7cab80b640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f7ca410fb20 con 0x7f7ca41056c0 2024-03-18T23:11:36.814 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:36.810+0000 7f7cab80b640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f7ca410f9a0 con 0x7f7ca4101330 2024-03-18T23:11:36.814 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:36.810+0000 7f7cab80b640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f7ca410fca0 con 0x7f7ca4100970 2024-03-18T23:11:36.814 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:36.810+0000 7f7ca9da8640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7ca41056c0 0x7f7ca419a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:11:36.814 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:36.810+0000 7f7ca9da8640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7ca41056c0 0x7f7ca419a740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58594/0 (socket says 172.21.15.49:58594) 2024-03-18T23:11:36.814 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:36.810+0000 7f7ca9da8640 1 -- 172.21.15.49:0/1872957627 learned_addr learned my addr 172.21.15.49:0/1872957627 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:11:36.814 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:36.810+0000 7f7ca95a7640 1 --2- 172.21.15.49:0/1872957627 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7ca4100970 0x7f7ca4195e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:11:36.814 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:36.810+0000 7f7ca8da6640 1 --2- 172.21.15.49:0/1872957627 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7ca4101330 0x7f7ca41963c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:11:36.814 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:36.810+0000 7f7ca9da8640 1 -- 172.21.15.49:0/1872957627 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7ca4100970 msgr2=0x7f7ca4195e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:11:36.814 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:36.810+0000 7f7ca9da8640 1 --2- 172.21.15.49:0/1872957627 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7ca4100970 0x7f7ca4195e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:36.814 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:36.810+0000 7f7ca9da8640 1 -- 172.21.15.49:0/1872957627 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7ca4101330 msgr2=0x7f7ca41963c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:11:36.814 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:36.810+0000 7f7ca9da8640 1 --2- 172.21.15.49:0/1872957627 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7ca4101330 0x7f7ca41963c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:36.815 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:36.810+0000 7f7ca9da8640 1 -- 172.21.15.49:0/1872957627 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ca419ae70 con 0x7f7ca41056c0 2024-03-18T23:11:36.815 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:36.810+0000 7f7ca9da8640 1 --2- 172.21.15.49:0/1872957627 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7ca41056c0 0x7f7ca419a740 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f7c9400ea60 tx=0x7f7c9400ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:11:36.815 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:36.811+0000 7f7ca27fc640 1 -- 172.21.15.49:0/1872957627 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c9400ce40 con 0x7f7ca41056c0 2024-03-18T23:11:36.815 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:36.811+0000 7f7ca27fc640 1 -- 172.21.15.49:0/1872957627 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c94005020 con 0x7f7ca41056c0 2024-03-18T23:11:36.815 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:36.811+0000 7f7cab80b640 1 -- 172.21.15.49:0/1872957627 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ca419b160 con 0x7f7ca41056c0 2024-03-18T23:11:36.815 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:36.811+0000 7f7ca27fc640 1 -- 172.21.15.49:0/1872957627 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c94010690 con 0x7f7ca41056c0 2024-03-18T23:11:36.815 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:36.811+0000 7f7cab80b640 1 -- 172.21.15.49:0/1872957627 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ca41a29f0 con 0x7f7ca41056c0 2024-03-18T23:11:36.824 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:36.813+0000 7f7ca27fc640 1 -- 172.21.15.49:0/1872957627 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 25) ==== 137421+0+0 (secure 0 0 0) 0x7f7c94005520 con 0x7f7ca41056c0 2024-03-18T23:11:36.824 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:36.813+0000 7f7cae3f9740 1 Processor -- start 2024-03-18T23:11:36.824 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:36.814+0000 7f7cae3f9740 1 -- start start 2024-03-18T23:11:36.824 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:36.814+0000 7f7cae3f9740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5565ddf61ef0 0x5565ddf622d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:11:36.824 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:36.814+0000 7f7cae3f9740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5565ddf62810 0x5565ddf6b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:11:36.824 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:36.814+0000 7f7cae3f9740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5565ddf6b5e0 0x5565ddf6d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:11:36.824 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:36.814+0000 7f7cae3f9740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5565ddec5be0 con 0x5565ddf6b5e0 2024-03-18T23:11:36.824 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:36.814+0000 7f7cae3f9740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x5565dde6e960 con 0x5565ddf61ef0 2024-03-18T23:11:36.824 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:36.814+0000 7f7cae3f9740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5565ddec1c80 con 0x5565ddf62810 2024-03-18T23:11:36.824 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:36.814+0000 7f7ca27fc640 1 --2- 172.21.15.49:0/1872957627 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f7c8c0a2a10 0x7f7c8c0a4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:11:36.825 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:36.814+0000 7f7ca8da6640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5565ddf61ef0 0x5565ddf622d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:11:36.825 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:36.814+0000 7f7ca95a7640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5565ddf62810 0x5565ddf6b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:11:36.825 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:36.814+0000 7f7ca95a7640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5565ddf62810 0x5565ddf6b0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.49:33002/0 (socket says 172.21.15.49:33002) 2024-03-18T23:11:36.825 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:36.814+0000 7f7ca95a7640 1 -- 172.21.15.49:0/1847610343 learned_addr learned my addr 172.21.15.49:0/1847610343 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:11:36.825 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:36.814+0000 7f7ca8da6640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5565ddf61ef0 0x5565ddf622d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.49:56682/0 (socket says 172.21.15.49:56682) 2024-03-18T23:11:36.825 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:36.814+0000 7f7ca8da6640 1 --2- 172.21.15.49:0/1847610343 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5565ddf6b5e0 0x5565ddf6d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:11:36.825 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:36.814+0000 7f7ca9da8640 1 --2- 172.21.15.49:0/1872957627 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f7c8c0a2a10 0x7f7c8c0a4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:11:36.825 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:36.814+0000 7f7ca27fc640 1 -- 172.21.15.49:0/1872957627 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f7c940cf2d0 con 0x7f7ca41056c0 2024-03-18T23:11:36.825 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:36.814+0000 7f7ca95a7640 1 -- 172.21.15.49:0/1847610343 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5565ddf61ef0 msgr2=0x5565ddf622d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:11:36.825 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:36.814+0000 7f7ca95a7640 1 --2- 172.21.15.49:0/1847610343 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5565ddf61ef0 0x5565ddf622d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:36.825 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:36.814+0000 7f7ca95a7640 1 -- 172.21.15.49:0/1847610343 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5565ddf6b5e0 msgr2=0x5565ddf6d9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:11:36.825 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:36.814+0000 7f7ca95a7640 1 --2- 172.21.15.49:0/1847610343 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5565ddf6b5e0 0x5565ddf6d9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:36.826 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:36.814+0000 7f7ca95a7640 1 -- 172.21.15.49:0/1847610343 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5565dde65d20 con 0x5565ddf62810 2024-03-18T23:11:36.826 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:36.814+0000 7f7ca8da6640 1 --2- 172.21.15.49:0/1847610343 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5565ddf61ef0 0x5565ddf622d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:11:36.826 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:36.815+0000 7f7ca9da8640 1 --2- 172.21.15.49:0/1872957627 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f7c8c0a2a10 0x7f7c8c0a4ed0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f7c940ccb00 tx=0x7f7c9401bb30 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:11:36.826 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:36.815+0000 7f7ca95a7640 1 --2- 172.21.15.49:0/1847610343 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5565ddf62810 0x5565ddf6b0a0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f7c9c009230 tx=0x7f7c9c005df0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b8fdcbf758bf66fd server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:11:36.826 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:36.815+0000 7f7c8b7fe640 1 -- 172.21.15.49:0/1847610343 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c9c042070 con 0x5565ddf62810 2024-03-18T23:11:36.826 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:36.815+0000 7f7c8b7fe640 1 -- 172.21.15.49:0/1847610343 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c9c02c690 con 0x5565ddf62810 2024-03-18T23:11:36.826 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:36.815+0000 7f7c8b7fe640 1 -- 172.21.15.49:0/1847610343 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c9c004b50 con 0x5565ddf62810 2024-03-18T23:11:36.826 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:36.816+0000 7f7cae3f9740 1 -- 172.21.15.49:0/1847610343 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5565ddf62810 msgr2=0x5565ddf6b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:11:36.826 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:36.816+0000 7f7cae3f9740 1 --2- 172.21.15.49:0/1847610343 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5565ddf62810 0x5565ddf6b0a0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f7c9c009230 tx=0x7f7c9c005df0 comp rx=0 tx=0).stop 2024-03-18T23:11:36.826 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:36.816+0000 7f7cae3f9740 1 -- 172.21.15.49:0/1847610343 shutdown_connections 2024-03-18T23:11:36.826 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:36.816+0000 7f7cae3f9740 1 --2- 172.21.15.49:0/1847610343 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5565ddf6b5e0 0x5565ddf6d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:36.826 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:36.816+0000 7f7cae3f9740 1 --2- 172.21.15.49:0/1847610343 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5565ddf62810 0x5565ddf6b0a0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:36.827 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:36.816+0000 7f7cae3f9740 1 --2- 172.21.15.49:0/1847610343 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5565ddf61ef0 0x5565ddf622d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:36.827 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:36.816+0000 7f7cae3f9740 1 -- 172.21.15.49:0/1847610343 >> 172.21.15.49:0/1847610343 conn(0x5565ddf5c810 msgr2=0x5565ddf5cbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:11:36.827 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:36.816+0000 7f7cae3f9740 1 -- 172.21.15.49:0/1847610343 shutdown_connections 2024-03-18T23:11:36.827 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:36.816+0000 7f7cae3f9740 1 -- 172.21.15.49:0/1847610343 wait complete. 2024-03-18T23:11:36.827 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:36.816+0000 7f7cae3f9740 1 -- start start 2024-03-18T23:11:36.827 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:36.816+0000 7f7cae3f9740 1 Processor -- start 2024-03-18T23:11:36.827 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:36.817+0000 7f7cae3f9740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5565ddf61ef0 0x5565ddf6af60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:11:36.827 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:36.817+0000 7f7cae3f9740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5565ddf62810 0x5565ddf682f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:11:36.827 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:36.817+0000 7f7cae3f9740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5565ddf6b5e0 0x5565ddf68870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:11:36.827 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:36.817+0000 7f7cae3f9740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5565ddf6e1b0 con 0x5565ddf61ef0 2024-03-18T23:11:36.827 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:36.817+0000 7f7cae3f9740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x5565dde81db0 con 0x5565ddf6b5e0 2024-03-18T23:11:36.827 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:36.817+0000 7f7cae3f9740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5565ddf6e330 con 0x5565ddf62810 2024-03-18T23:11:36.827 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:36.817+0000 7f7cae3f9740 10 client.0 _ll_register_callbacks cb 0x5565dde86e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T23:11:36.827 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:36.817+0000 7f7ca8da6640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5565ddf61ef0 0x5565ddf6af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:11:36.828 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:36.817+0000 7f7ca8da6640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5565ddf61ef0 0x5565ddf6af60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58604/0 (socket says 172.21.15.49:58604) 2024-03-18T23:11:36.828 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:36.817+0000 7f7ca95a7640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5565ddf62810 0x5565ddf682f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:11:36.828 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:36.817+0000 7f7ca8da6640 1 -- 172.21.15.49:0/2007717565 learned_addr learned my addr 172.21.15.49:0/2007717565 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:11:36.828 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:36.817+0000 7f7ca8da6640 1 --2- 172.21.15.49:0/2007717565 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5565ddf6b5e0 0x5565ddf68870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:11:36.828 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:36.817+0000 7f7ca95a7640 1 -- 172.21.15.49:0/2007717565 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5565ddf6b5e0 msgr2=0x5565ddf68870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:11:36.828 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:36.817+0000 7f7ca95a7640 1 --2- 172.21.15.49:0/2007717565 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5565ddf6b5e0 0x5565ddf68870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:36.828 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:36.817+0000 7f7ca95a7640 1 -- 172.21.15.49:0/2007717565 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5565ddf61ef0 msgr2=0x5565ddf6af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:11:36.828 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:36.817+0000 7f7ca95a7640 1 --2- 172.21.15.49:0/2007717565 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5565ddf61ef0 0x5565ddf6af60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:36.828 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:36.817+0000 7f7ca95a7640 1 -- 172.21.15.49:0/2007717565 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5565dde80c60 con 0x5565ddf62810 2024-03-18T23:11:36.828 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:36.817+0000 7f7ca8da6640 1 --2- 172.21.15.49:0/2007717565 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5565ddf61ef0 0x5565ddf6af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:11:36.828 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:36.817+0000 7f7ca95a7640 1 --2- 172.21.15.49:0/2007717565 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5565ddf62810 0x5565ddf682f0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f7c9c036040 tx=0x7f7c9c02bc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2a8b4495905b02a7 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:11:36.828 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:36.817+0000 7f7c88ff9640 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-03-18T23:11:36.828 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:36.818+0000 7f7c88ff9640 1 -- 172.21.15.49:0/2007717565 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c9c042040 con 0x5565ddf62810 2024-03-18T23:11:36.828 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:36.818+0000 7f7cae3f9740 1 -- 172.21.15.49:0/2007717565 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7c9c02c690 con 0x5565ddf62810 2024-03-18T23:11:36.828 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:36.818+0000 7f7c88ff9640 1 -- 172.21.15.49:0/2007717565 <== mon.1 v2:172.21.15.175:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f7c9c004f80 con 0x5565ddf62810 2024-03-18T23:11:36.829 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:36.818+0000 7f7c88ff9640 1 -- 172.21.15.49:0/2007717565 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f7c9c05a050 con 0x5565ddf62810 2024-03-18T23:11:36.829 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:36.818+0000 7f7cae3f9740 10 client.25531 fetch_fsmap learned FSMap version 30 2024-03-18T23:11:36.829 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:36.818+0000 7f7cae3f9740 1 -- 172.21.15.49:0/2007717565 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5565ddbda210 con 0x5565ddf62810 2024-03-18T23:11:36.829 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:36.818+0000 7f7c88ff9640 1 -- 172.21.15.49:0/2007717565 <== mon.1 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c9c02c690 con 0x5565ddf62810 2024-03-18T23:11:36.829 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:36.818+0000 7f7c88ff9640 1 -- 172.21.15.49:0/2007717565 <== mon.1 v2:172.21.15.175:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f7c9c0479c0 con 0x5565ddf62810 2024-03-18T23:11:36.829 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:36.818+0000 7f7cae3f9740 10 client.25531 fetch_fsmap finished waiting for FSMap version 30 2024-03-18T23:11:36.829 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:36.818+0000 7f7cae3f9740 10 client.25531 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T23:11:36.829 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:36.818+0000 7f7cae3f9740 20 client.25531 populate_metadata read hostname 'smithi049' 2024-03-18T23:11:36.829 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:36.818+0000 7f7cae3f9740 1 --2- 172.21.15.49:0/2007717565 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x5565ddf7b2c0 0x5565ddf7d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:11:36.829 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:36.818+0000 7f7cae3f9740 4 client.25531 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T23:11:36.829 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:36.818+0000 7f7cae3f9740 1 -- 172.21.15.49:0/2007717565 --> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5565dded3120 con 0x5565ddf7b2c0 2024-03-18T23:11:36.829 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:36.818+0000 7f7ca8da6640 1 --2- 172.21.15.49:0/2007717565 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x5565ddf7b2c0 0x5565ddf7d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:11:36.829 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:36.819+0000 7f7c88ff9640 10 client.25531 ms_handle_connect on v2:172.21.15.139:6838/4280128209 2024-03-18T23:11:36.829 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:36.819+0000 7f7ca8da6640 1 --2- 172.21.15.49:0/2007717565 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x5565ddf7b2c0 0x5565ddf7d6b0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f3c2ff83bf223455 server_cookie=646e28f7e6d02d4d in_seq=0 out_seq=0 2024-03-18T23:11:36.830 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:36.820+0000 7f7c88ff9640 1 -- 172.21.15.49:0/2007717565 <== mds.0 v2:172.21.15.139:6838/4280128209 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5565dded3120 con 0x5565ddf7b2c0 2024-03-18T23:11:36.830 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:36.820+0000 7f7c88ff9640 10 client.25531 handle_command_reply: tid=0 2024-03-18T23:11:36.830 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:36.820+0000 7f7cae3f9740 1 client.25531 shutdown 2024-03-18T23:11:36.830 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:36.821+0000 7f7cae3f9740 1 -- 172.21.15.49:0/2007717565 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5565ddf62810 msgr2=0x5565ddf682f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:11:36.830 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:36.821+0000 7f7cae3f9740 1 --2- 172.21.15.49:0/2007717565 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5565ddf62810 0x5565ddf682f0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f7c9c036040 tx=0x7f7c9c02bc30 comp rx=0 tx=0).stop 2024-03-18T23:11:36.830 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:36.821+0000 7f7cae3f9740 1 -- 172.21.15.49:0/2007717565 shutdown_connections 2024-03-18T23:11:36.830 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:36.821+0000 7f7cae3f9740 1 --2- 172.21.15.49:0/2007717565 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x5565ddf7b2c0 0x5565ddf7d6b0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:36.830 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:36.821+0000 7f7cae3f9740 1 --2- 172.21.15.49:0/2007717565 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5565ddf6b5e0 0x5565ddf68870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:36.830 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:36.821+0000 7f7c88ff9640 0 client.25531 ms_handle_reset on v2:172.21.15.139:6838/4280128209 2024-03-18T23:11:36.830 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:36.821+0000 7f7cae3f9740 1 --2- 172.21.15.49:0/2007717565 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5565ddf62810 0x5565ddf682f0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:36.830 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:36.821+0000 7f7cae3f9740 1 --2- 172.21.15.49:0/2007717565 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5565ddf61ef0 0x5565ddf6af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:36.830 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:36.821+0000 7f7cae3f9740 1 -- 172.21.15.49:0/2007717565 >> 172.21.15.49:0/2007717565 conn(0x5565ddf5c810 msgr2=0x5565ddf5bab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:11:36.830 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:36.821+0000 7f7cae3f9740 1 -- 172.21.15.49:0/2007717565 shutdown_connections 2024-03-18T23:11:36.830 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:36.821+0000 7f7cae3f9740 1 -- 172.21.15.49:0/2007717565 wait complete. 2024-03-18T23:11:36.831 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:36.821+0000 7f7cae3f9740 20 client.25531 trim_cache size 0 max 16384 2024-03-18T23:11:36.839 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:36.837+0000 7f7cae3f9740 1 Processor -- start 2024-03-18T23:11:36.839 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:36.837+0000 7f7cae3f9740 1 -- start start 2024-03-18T23:11:36.840 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:36.838+0000 7f7cae3f9740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5565ddf61ef0 0x5565ddf76f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:11:36.840 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:36.838+0000 7f7cae3f9740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5565ddf62810 0x5565ddf82cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:11:36.840 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:36.838+0000 7f7cae3f9740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5565ddf6b5e0 0x5565ddf6aa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:11:36.840 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:36.838+0000 7f7cae3f9740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f7ca41ab0a0 con 0x5565ddf61ef0 2024-03-18T23:11:36.840 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:36.838+0000 7f7cae3f9740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f7ca41acbe0 con 0x5565ddf6b5e0 2024-03-18T23:11:36.840 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:36.838+0000 7f7cae3f9740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f7ca41ab770 con 0x5565ddf62810 2024-03-18T23:11:36.840 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:36.838+0000 7f7ca8da6640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5565ddf61ef0 0x5565ddf76f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:11:36.840 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:36.838+0000 7f7ca8da6640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5565ddf61ef0 0x5565ddf76f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58620/0 (socket says 172.21.15.49:58620) 2024-03-18T23:11:36.840 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:36.838+0000 7f7ca8da6640 1 -- 172.21.15.49:0/621085155 learned_addr learned my addr 172.21.15.49:0/621085155 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:11:36.840 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:36.838+0000 7f7ca95a7640 1 --2- 172.21.15.49:0/621085155 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5565ddf62810 0x5565ddf82cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:11:36.840 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:36.838+0000 7f7ca8da6640 1 --2- 172.21.15.49:0/621085155 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5565ddf6b5e0 0x5565ddf6aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:11:36.840 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:36.838+0000 7f7ca8da6640 1 -- 172.21.15.49:0/621085155 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5565ddf62810 msgr2=0x5565ddf82cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:11:36.841 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:36.838+0000 7f7ca8da6640 1 --2- 172.21.15.49:0/621085155 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5565ddf62810 0x5565ddf82cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:36.841 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:36.838+0000 7f7ca8da6640 1 -- 172.21.15.49:0/621085155 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5565ddf6b5e0 msgr2=0x5565ddf6aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:11:36.841 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:36.838+0000 7f7ca8da6640 1 --2- 172.21.15.49:0/621085155 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5565ddf6b5e0 0x5565ddf6aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:36.841 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:36.838+0000 7f7ca8da6640 1 -- 172.21.15.49:0/621085155 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5565ddca6620 con 0x5565ddf61ef0 2024-03-18T23:11:36.841 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:36.838+0000 7f7ca8da6640 1 --2- 172.21.15.49:0/621085155 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5565ddf61ef0 0x5565ddf76f50 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x5565ddf7dc50 tx=0x7f7c98022040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62f01180725b3cba server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:11:36.841 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:36.838+0000 7f7c8affd640 1 -- 172.21.15.49:0/621085155 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c98029040 con 0x5565ddf61ef0 2024-03-18T23:11:36.841 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:36.839+0000 7f7c8affd640 1 -- 172.21.15.49:0/621085155 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c98003ac0 con 0x5565ddf61ef0 2024-03-18T23:11:36.841 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:36.839+0000 7f7c8affd640 1 -- 172.21.15.49:0/621085155 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c98009a70 con 0x5565ddf61ef0 2024-03-18T23:11:36.841 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:36.839+0000 7f7cae3f9740 1 -- 172.21.15.49:0/621085155 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5565ddf61ef0 msgr2=0x5565ddf76f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:11:36.841 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:36.839+0000 7f7cae3f9740 1 --2- 172.21.15.49:0/621085155 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5565ddf61ef0 0x5565ddf76f50 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x5565ddf7dc50 tx=0x7f7c98022040 comp rx=0 tx=0).stop 2024-03-18T23:11:36.842 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:36.839+0000 7f7cae3f9740 1 -- 172.21.15.49:0/621085155 shutdown_connections 2024-03-18T23:11:36.842 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:36.839+0000 7f7cae3f9740 1 --2- 172.21.15.49:0/621085155 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5565ddf6b5e0 0x5565ddf6aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:36.842 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:36.839+0000 7f7cae3f9740 1 --2- 172.21.15.49:0/621085155 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5565ddf62810 0x5565ddf82cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:36.842 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:36.839+0000 7f7cae3f9740 1 --2- 172.21.15.49:0/621085155 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5565ddf61ef0 0x5565ddf76f50 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:36.842 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:36.839+0000 7f7cae3f9740 1 -- 172.21.15.49:0/621085155 >> 172.21.15.49:0/621085155 conn(0x5565ddf5c810 msgr2=0x5565ddf5bab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:11:36.842 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:36.839+0000 7f7cae3f9740 1 -- 172.21.15.49:0/621085155 shutdown_connections 2024-03-18T23:11:36.842 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:36.839+0000 7f7cae3f9740 1 -- 172.21.15.49:0/621085155 wait complete. 2024-03-18T23:11:36.842 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:36.840+0000 7f7cae3f9740 1 -- start start 2024-03-18T23:11:36.842 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:36.840+0000 7f7cae3f9740 1 Processor -- start 2024-03-18T23:11:36.846 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:36.840+0000 7f7cae3f9740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5565ddf61ef0 0x5565ddf73970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:11:36.846 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:36.840+0000 7f7cae3f9740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5565ddf62810 0x5565ddf64d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:11:36.846 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:36.840+0000 7f7cae3f9740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5565ddf6b5e0 0x5565ddf65460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:11:36.846 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:36.840+0000 7f7cae3f9740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f7ca41ac510 con 0x5565ddf6b5e0 2024-03-18T23:11:36.846 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:36.840+0000 7f7cae3f9740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f7ca41abe40 con 0x5565ddf61ef0 2024-03-18T23:11:36.846 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:36.840+0000 7f7cae3f9740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f7ca41a9080 con 0x5565ddf62810 2024-03-18T23:11:36.846 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:36.840+0000 7f7cae3f9740 10 client.0 _ll_register_callbacks cb 0x7f7c9800de60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T23:11:36.846 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:36.840+0000 7f7ca8da6640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5565ddf61ef0 0x5565ddf73970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:11:36.846 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:36.840+0000 7f7ca95a7640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5565ddf62810 0x5565ddf64d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:11:36.847 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:36.840+0000 7f7ca8da6640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5565ddf61ef0 0x5565ddf73970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.49:56708/0 (socket says 172.21.15.49:56708) 2024-03-18T23:11:36.847 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:36.840+0000 7f7ca8da6640 1 -- 172.21.15.49:0/243724685 learned_addr learned my addr 172.21.15.49:0/243724685 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:11:36.847 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:36.840+0000 7f7ca95a7640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5565ddf62810 0x5565ddf64d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.49:33036/0 (socket says 172.21.15.49:33036) 2024-03-18T23:11:36.847 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:36.841+0000 7f7ca8da6640 1 --2- 172.21.15.49:0/243724685 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5565ddf6b5e0 0x5565ddf65460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:11:36.847 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:36.841+0000 7f7ca8da6640 1 -- 172.21.15.49:0/243724685 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5565ddf62810 msgr2=0x5565ddf64d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:11:36.847 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:36.841+0000 7f7ca8da6640 1 --2- 172.21.15.49:0/243724685 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5565ddf62810 0x5565ddf64d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:36.847 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:36.841+0000 7f7ca8da6640 1 -- 172.21.15.49:0/243724685 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5565ddf6b5e0 msgr2=0x5565ddf65460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:11:36.847 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:36.841+0000 7f7ca8da6640 1 --2- 172.21.15.49:0/243724685 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5565ddf6b5e0 0x5565ddf65460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:36.847 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:36.841+0000 7f7ca8da6640 1 -- 172.21.15.49:0/243724685 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5565ddfe02e0 con 0x5565ddf61ef0 2024-03-18T23:11:36.847 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:36.841+0000 7f7ca95a7640 1 --2- 172.21.15.49:0/243724685 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5565ddf62810 0x5565ddf64d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:11:36.847 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:36.841+0000 7f7ca8da6640 1 --2- 172.21.15.49:0/243724685 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5565ddf61ef0 0x5565ddf73970 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f7c9800d100 tx=0x7f7c9800f040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b1204eaf3b9d501c server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:11:36.848 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:36.841+0000 7f7c88ff9640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-03-18T23:11:36.848 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:36.841+0000 7f7c88ff9640 1 -- 172.21.15.49:0/243724685 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c98003c70 con 0x5565ddf61ef0 2024-03-18T23:11:36.848 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:36.841+0000 7f7cae3f9740 1 -- 172.21.15.49:0/243724685 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7c98003ac0 con 0x5565ddf61ef0 2024-03-18T23:11:36.848 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:36.841+0000 7f7c88ff9640 1 -- 172.21.15.49:0/243724685 <== mon.2 v2:172.21.15.139:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f7c98003e10 con 0x5565ddf61ef0 2024-03-18T23:11:36.848 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:36.841+0000 7f7c88ff9640 1 -- 172.21.15.49:0/243724685 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f7c98029ab0 con 0x5565ddf61ef0 2024-03-18T23:11:36.848 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:36.841+0000 7f7cae3f9740 10 client.25547 fetch_fsmap learned FSMap version 30 2024-03-18T23:11:36.848 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:36.842+0000 7f7cae3f9740 1 -- 172.21.15.49:0/243724685 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5565ddfe1420 con 0x5565ddf61ef0 2024-03-18T23:11:36.848 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:36.842+0000 7f7c88ff9640 1 -- 172.21.15.49:0/243724685 <== mon.2 v2:172.21.15.139:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c98003ac0 con 0x5565ddf61ef0 2024-03-18T23:11:36.848 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:36.842+0000 7f7c88ff9640 1 -- 172.21.15.49:0/243724685 <== mon.2 v2:172.21.15.139:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f7c9801c040 con 0x5565ddf61ef0 2024-03-18T23:11:36.848 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:36.842+0000 7f7cae3f9740 10 client.25547 fetch_fsmap finished waiting for FSMap version 30 2024-03-18T23:11:36.848 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:36.842+0000 7f7cae3f9740 10 client.25547 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T23:11:36.848 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:36.842+0000 7f7cae3f9740 20 client.25547 populate_metadata read hostname 'smithi049' 2024-03-18T23:11:36.849 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:36.842+0000 7f7cae3f9740 1 --2- 172.21.15.49:0/243724685 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x5565ddfe1ab0 0x5565ddfe3ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:11:36.849 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:36.842+0000 7f7cae3f9740 4 client.25547 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T23:11:36.849 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:36.842+0000 7f7cae3f9740 1 -- 172.21.15.49:0/243724685 --> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5565ddecd470 con 0x5565ddfe1ab0 2024-03-18T23:11:36.849 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:36.842+0000 7f7ca95a7640 1 --2- 172.21.15.49:0/243724685 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x5565ddfe1ab0 0x5565ddfe3ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:11:36.849 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:36.843+0000 7f7c88ff9640 10 client.25547 ms_handle_connect on v2:172.21.15.139:6838/4280128209 2024-03-18T23:11:36.849 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:36.843+0000 7f7ca95a7640 1 --2- 172.21.15.49:0/243724685 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x5565ddfe1ab0 0x5565ddfe3ed0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bacc7b0ca216058f server_cookie=fb816fbb6f9b12d1 in_seq=0 out_seq=0 2024-03-18T23:11:37.016 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:37.015+0000 7f7c88ff9640 1 -- 172.21.15.49:0/243724685 <== mds.0 v2:172.21.15.139:6838/4280128209 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5565ddecd470 con 0x5565ddfe1ab0 2024-03-18T23:11:37.016 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:37.015+0000 7f7c88ff9640 10 client.25547 handle_command_reply: tid=0 2024-03-18T23:11:37.016 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:37.015+0000 7f7cae3f9740 1 client.25547 shutdown 2024-03-18T23:11:37.017 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:37.016+0000 7f7cae3f9740 1 -- 172.21.15.49:0/243724685 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5565ddf61ef0 msgr2=0x5565ddf73970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:11:37.017 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:37.016+0000 7f7cae3f9740 1 --2- 172.21.15.49:0/243724685 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5565ddf61ef0 0x5565ddf73970 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f7c9800d100 tx=0x7f7c9800f040 comp rx=0 tx=0).stop 2024-03-18T23:11:37.017 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:37.016+0000 7f7cae3f9740 1 -- 172.21.15.49:0/243724685 shutdown_connections 2024-03-18T23:11:37.017 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:37.016+0000 7f7cae3f9740 1 --2- 172.21.15.49:0/243724685 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x5565ddfe1ab0 0x5565ddfe3ed0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:37.017 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:37.016+0000 7f7cae3f9740 1 --2- 172.21.15.49:0/243724685 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5565ddf6b5e0 0x5565ddf65460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:37.017 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:37.016+0000 7f7c88ff9640 0 client.25547 ms_handle_reset on v2:172.21.15.139:6838/4280128209 2024-03-18T23:11:37.017 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:37.016+0000 7f7cae3f9740 1 --2- 172.21.15.49:0/243724685 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5565ddf62810 0x5565ddf64d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:37.017 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:37.016+0000 7f7cae3f9740 1 --2- 172.21.15.49:0/243724685 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5565ddf61ef0 0x5565ddf73970 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:37.017 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:37.016+0000 7f7cae3f9740 1 -- 172.21.15.49:0/243724685 >> 172.21.15.49:0/243724685 conn(0x5565ddf5c810 msgr2=0x5565ddf62f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:11:37.017 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:37.016+0000 7f7cae3f9740 1 -- 172.21.15.49:0/243724685 shutdown_connections 2024-03-18T23:11:37.017 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:37.016+0000 7f7cae3f9740 1 -- 172.21.15.49:0/243724685 wait complete. 2024-03-18T23:11:37.018 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:37.016+0000 7f7cae3f9740 20 client.25547 trim_cache size 0 max 16384 2024-03-18T23:11:37.018 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:{ 2024-03-18T23:11:37.018 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "message": "", 2024-03-18T23:11:37.018 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "return_code": 0 2024-03-18T23:11:37.018 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:} 2024-03-18T23:11:37.019 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:37.017+0000 7f7cab80b640 1 -- 172.21.15.49:0/1872957627 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f7c8c0a2a10 msgr2=0x7f7c8c0a4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:11:37.019 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:37.017+0000 7f7cab80b640 1 --2- 172.21.15.49:0/1872957627 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f7c8c0a2a10 0x7f7c8c0a4ed0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f7c940ccb00 tx=0x7f7c9401bb30 comp rx=0 tx=0).stop 2024-03-18T23:11:37.019 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:37.017+0000 7f7cab80b640 1 -- 172.21.15.49:0/1872957627 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7ca41056c0 msgr2=0x7f7ca419a740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:11:37.019 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:37.017+0000 7f7cab80b640 1 --2- 172.21.15.49:0/1872957627 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7ca41056c0 0x7f7ca419a740 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f7c9400ea60 tx=0x7f7c9400ef20 comp rx=0 tx=0).stop 2024-03-18T23:11:37.020 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:37.017+0000 7f7cab80b640 1 -- 172.21.15.49:0/1872957627 shutdown_connections 2024-03-18T23:11:37.020 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:37.017+0000 7f7cab80b640 1 --2- 172.21.15.49:0/1872957627 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f7c8c0a2a10 0x7f7c8c0a4ed0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:37.020 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:37.017+0000 7f7cab80b640 1 --2- 172.21.15.49:0/1872957627 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7ca41056c0 0x7f7ca419a740 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:37.020 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:37.017+0000 7f7cab80b640 1 --2- 172.21.15.49:0/1872957627 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7ca4101330 0x7f7ca41963c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:37.020 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:37.017+0000 7f7cab80b640 1 --2- 172.21.15.49:0/1872957627 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7ca4100970 0x7f7ca4195e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:37.020 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:37.017+0000 7f7cab80b640 1 -- 172.21.15.49:0/1872957627 >> 172.21.15.49:0/1872957627 conn(0x7f7ca4077570 msgr2=0x7f7ca40fe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:11:37.020 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:37.017+0000 7f7cab80b640 1 -- 172.21.15.49:0/1872957627 shutdown_connections 2024-03-18T23:11:37.020 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:37.017+0000 7f7cab80b640 1 -- 172.21.15.49:0/1872957627 wait complete. 2024-03-18T23:11:37.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:11:37 smithi175 ceph-mon[67352]: pgmap v103: 97 pgs: 97 active+clean; 49 MiB data, 1017 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.0 MiB/s wr, 314 op/s 2024-03-18T23:11:37.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:11:37 smithi139 ceph-mon[67282]: pgmap v103: 97 pgs: 97 active+clean; 49 MiB data, 1017 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.0 MiB/s wr, 314 op/s 2024-03-18T23:11:37.630 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:11:37 smithi049 ceph-mon[62991]: pgmap v103: 97 pgs: 97 active+clean; 49 MiB data, 1017 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.0 MiB/s wr, 314 op/s 2024-03-18T23:11:37.765 INFO:tasks.workunit.client.0.smithi049.stdout: 1 365568 11.91 MB/sec execute 39 sec latency 83.578 ms 2024-03-18T23:11:38.066 INFO:journalctl@ceph.mds.h.smithi139.stdout:Mar 18 23:11:37 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mds-h[97809]: ignoring --setuser ceph since I am not root 2024-03-18T23:11:38.066 INFO:journalctl@ceph.mds.h.smithi139.stdout:Mar 18 23:11:37 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mds-h[97809]: ignoring --setgroup ceph since I am not root 2024-03-18T23:11:38.066 INFO:journalctl@ceph.mds.h.smithi139.stdout:Mar 18 23:11:37 smithi139 ceph-mds[97813]: -- 172.21.15.139:0/3877514325 <== mon.0 v2:172.21.15.49:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55fce30f5a40 con 0x55fce30c8800 2024-03-18T23:11:38.066 INFO:journalctl@ceph.mds.h.smithi139.stdout:Mar 18 23:11:37 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mds-h[97809]: starting mds.h at 2024-03-18T23:11:38.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:11:38 smithi175 ceph-mon[67352]: daemon mds.h restarted 2024-03-18T23:11:38.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:11:38 smithi139 ceph-mon[67282]: daemon mds.h restarted 2024-03-18T23:11:38.630 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:11:38 smithi049 ceph-mon[62991]: daemon mds.h restarted 2024-03-18T23:11:38.765 INFO:tasks.workunit.client.0.smithi049.stdout: 1 367026 11.75 MB/sec execute 40 sec latency 118.284 ms 2024-03-18T23:11:39.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:11:39 smithi175 ceph-mon[67352]: pgmap v104: 97 pgs: 97 active+clean; 55 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 4.5 MiB/s wr, 417 op/s 2024-03-18T23:11:39.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:11:39 smithi175 ceph-mon[67352]: osdmap e90: 12 total, 12 up, 12 in 2024-03-18T23:11:39.462 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:11:39 smithi175 ceph-mon[67352]: mds.? [v2:172.21.15.139:6836/2891095267,v1:172.21.15.139:6837/2891095267] up:boot 2024-03-18T23:11:39.462 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:11:39 smithi175 ceph-mon[67352]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-03-18T23:11:39.462 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:11:39 smithi175 ceph-mon[67352]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-18T23:11:39.462 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:11:39 smithi175 ceph-mon[67352]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T23:11:39.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:11:39 smithi139 ceph-mon[67282]: pgmap v104: 97 pgs: 97 active+clean; 55 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 4.5 MiB/s wr, 417 op/s 2024-03-18T23:11:39.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:11:39 smithi139 ceph-mon[67282]: osdmap e90: 12 total, 12 up, 12 in 2024-03-18T23:11:39.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:11:39 smithi139 ceph-mon[67282]: mds.? [v2:172.21.15.139:6836/2891095267,v1:172.21.15.139:6837/2891095267] up:boot 2024-03-18T23:11:39.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:11:39 smithi139 ceph-mon[67282]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-03-18T23:11:39.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:11:39 smithi139 ceph-mon[67282]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-18T23:11:39.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:11:39 smithi139 ceph-mon[67282]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T23:11:39.630 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:11:39 smithi049 ceph-mon[62991]: pgmap v104: 97 pgs: 97 active+clean; 55 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 4.5 MiB/s wr, 417 op/s 2024-03-18T23:11:39.630 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:11:39 smithi049 ceph-mon[62991]: osdmap e90: 12 total, 12 up, 12 in 2024-03-18T23:11:39.630 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:11:39 smithi049 ceph-mon[62991]: mds.? [v2:172.21.15.139:6836/2891095267,v1:172.21.15.139:6837/2891095267] up:boot 2024-03-18T23:11:39.630 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:11:39 smithi049 ceph-mon[62991]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-03-18T23:11:39.630 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:11:39 smithi049 ceph-mon[62991]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-18T23:11:39.630 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:11:39 smithi049 ceph-mon[62991]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T23:11:39.765 INFO:tasks.workunit.client.0.smithi049.stdout: 1 369354 11.80 MB/sec execute 41 sec latency 19.380 ms 2024-03-18T23:11:40.765 INFO:tasks.workunit.client.0.smithi049.stdout: 1 372224 11.87 MB/sec execute 42 sec latency 27.616 ms 2024-03-18T23:11:41.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:11:41 smithi175 ceph-mon[67352]: pgmap v106: 97 pgs: 97 active+clean; 55 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.5 MiB/s wr, 242 op/s 2024-03-18T23:11:41.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:11:41 smithi139 ceph-mon[67282]: pgmap v106: 97 pgs: 97 active+clean; 55 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.5 MiB/s wr, 242 op/s 2024-03-18T23:11:41.629 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:11:41 smithi049 ceph-mon[62991]: pgmap v106: 97 pgs: 97 active+clean; 55 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.5 MiB/s wr, 242 op/s 2024-03-18T23:11:41.765 INFO:tasks.workunit.client.0.smithi049.stdout: 1 374117 11.82 MB/sec execute 43 sec latency 22.382 ms 2024-03-18T23:11:42.765 INFO:tasks.workunit.client.0.smithi049.stdout: 1 376512 11.87 MB/sec execute 44 sec latency 16.385 ms 2024-03-18T23:11:43.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:11:43 smithi175 ceph-mon[67352]: pgmap v107: 97 pgs: 97 active+clean; 44 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.5 MiB/s wr, 395 op/s 2024-03-18T23:11:43.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:11:43 smithi139 ceph-mon[67282]: pgmap v107: 97 pgs: 97 active+clean; 44 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.5 MiB/s wr, 395 op/s 2024-03-18T23:11:43.630 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:11:43 smithi049 ceph-mon[62991]: pgmap v107: 97 pgs: 97 active+clean; 44 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.5 MiB/s wr, 395 op/s 2024-03-18T23:11:43.765 INFO:tasks.workunit.client.0.smithi049.stdout: 1 378914 11.87 MB/sec execute 45 sec latency 87.159 ms 2024-03-18T23:11:44.766 INFO:tasks.workunit.client.0.smithi049.stdout: 1 380542 11.79 MB/sec execute 46 sec latency 26.106 ms 2024-03-18T23:11:45.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:11:45 smithi175 ceph-mon[67352]: pgmap v108: 97 pgs: 97 active+clean; 44 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.5 MiB/s wr, 395 op/s 2024-03-18T23:11:45.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:11:45 smithi139 ceph-mon[67282]: pgmap v108: 97 pgs: 97 active+clean; 44 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.5 MiB/s wr, 395 op/s 2024-03-18T23:11:45.630 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:11:45 smithi049 ceph-mon[62991]: pgmap v108: 97 pgs: 97 active+clean; 44 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.5 MiB/s wr, 395 op/s 2024-03-18T23:11:45.765 INFO:tasks.workunit.client.0.smithi049.stdout: 1 383348 11.92 MB/sec execute 47 sec latency 11.236 ms 2024-03-18T23:11:46.766 INFO:tasks.workunit.client.0.smithi049.stdout: 1 385531 11.82 MB/sec execute 48 sec latency 28.865 ms 2024-03-18T23:11:47.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:11:47 smithi175 ceph-mon[67352]: pgmap v109: 97 pgs: 97 active+clean; 44 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.5 MiB/s wr, 395 op/s 2024-03-18T23:11:47.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:11:47 smithi139 ceph-mon[67282]: pgmap v109: 97 pgs: 97 active+clean; 44 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.5 MiB/s wr, 395 op/s 2024-03-18T23:11:47.629 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:11:47 smithi049 ceph-mon[62991]: pgmap v109: 97 pgs: 97 active+clean; 44 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.5 MiB/s wr, 395 op/s 2024-03-18T23:11:47.766 INFO:tasks.workunit.client.0.smithi049.stdout: 1 387393 11.83 MB/sec execute 49 sec latency 18.419 ms 2024-03-18T23:11:48.766 INFO:tasks.workunit.client.0.smithi049.stdout: 1 390045 11.88 MB/sec execute 50 sec latency 54.179 ms 2024-03-18T23:11:49.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:11:49 smithi175 ceph-mon[67352]: pgmap v110: 97 pgs: 97 active+clean; 50 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.5 MiB/s wr, 404 op/s 2024-03-18T23:11:49.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:11:49 smithi139 ceph-mon[67282]: pgmap v110: 97 pgs: 97 active+clean; 50 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.5 MiB/s wr, 404 op/s 2024-03-18T23:11:49.629 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:11:49 smithi049 ceph-mon[62991]: pgmap v110: 97 pgs: 97 active+clean; 50 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.5 MiB/s wr, 404 op/s 2024-03-18T23:11:49.766 INFO:tasks.workunit.client.0.smithi049.stdout: 1 391707 11.84 MB/sec execute 51 sec latency 26.815 ms 2024-03-18T23:11:50.109 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:50.108+0000 7fa7c96e8640 1 -- 172.21.15.49:0/3354689077 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa7c4101330 msgr2=0x7fa7c410d550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:11:50.109 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:50.108+0000 7fa7c96e8640 1 --2- 172.21.15.49:0/3354689077 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa7c4101330 0x7fa7c410d550 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fa7bc009ca0 tx=0x7fa7bc02c0f0 comp rx=0 tx=0).stop 2024-03-18T23:11:50.109 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:50.108+0000 7fa7c96e8640 1 -- 172.21.15.49:0/3354689077 shutdown_connections 2024-03-18T23:11:50.109 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:50.108+0000 7fa7c96e8640 1 --2- 172.21.15.49:0/3354689077 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa7c4101330 0x7fa7c410d550 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:50.110 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:50.108+0000 7fa7c96e8640 1 --2- 172.21.15.49:0/3354689077 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa7c4100970 0x7fa7c4100df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:50.110 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:50.108+0000 7fa7c96e8640 1 --2- 172.21.15.49:0/3354689077 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa7c41056c0 0x7fa7c4105aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:50.110 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:50.108+0000 7fa7c96e8640 1 -- 172.21.15.49:0/3354689077 >> 172.21.15.49:0/3354689077 conn(0x7fa7c4077570 msgr2=0x7fa7c4077980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:11:50.110 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:50.108+0000 7fa7c96e8640 1 -- 172.21.15.49:0/3354689077 shutdown_connections 2024-03-18T23:11:50.110 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:50.108+0000 7fa7c96e8640 1 -- 172.21.15.49:0/3354689077 wait complete. 2024-03-18T23:11:50.110 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:50.108+0000 7fa7c96e8640 1 Processor -- start 2024-03-18T23:11:50.110 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:50.109+0000 7fa7c96e8640 1 -- start start 2024-03-18T23:11:50.112 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:50.109+0000 7fa7c96e8640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa7c4100970 0x7fa7c4198ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:11:50.112 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:50.109+0000 7fa7c96e8640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa7c4101330 0x7fa7c4199420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:11:50.112 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:50.109+0000 7fa7c96e8640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa7c41056c0 0x7fa7c41a3f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:11:50.112 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:50.109+0000 7fa7c96e8640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fa7c4110be0 con 0x7fa7c41056c0 2024-03-18T23:11:50.112 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:50.109+0000 7fa7c96e8640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fa7c4110a60 con 0x7fa7c4101330 2024-03-18T23:11:50.112 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:50.109+0000 7fa7c96e8640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa7c4110d60 con 0x7fa7c4100970 2024-03-18T23:11:50.112 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:50.109+0000 7fa7c37fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa7c41056c0 0x7fa7c41a3f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:11:50.112 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:50.109+0000 7fa7c2ffd640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa7c4100970 0x7fa7c4198ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:11:50.112 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:50.109+0000 7fa7c37fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa7c41056c0 0x7fa7c41a3f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:39774/0 (socket says 172.21.15.49:39774) 2024-03-18T23:11:50.112 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:50.109+0000 7fa7c27fc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa7c4101330 0x7fa7c4199420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:11:50.112 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:50.109+0000 7fa7c37fe640 1 -- 172.21.15.49:0/766891538 learned_addr learned my addr 172.21.15.49:0/766891538 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:11:50.112 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:50.109+0000 7fa7c37fe640 1 -- 172.21.15.49:0/766891538 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa7c4100970 msgr2=0x7fa7c4198ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:11:50.112 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:50.109+0000 7fa7c37fe640 1 --2- 172.21.15.49:0/766891538 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa7c4100970 0x7fa7c4198ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:50.112 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:50.109+0000 7fa7c37fe640 1 -- 172.21.15.49:0/766891538 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa7c4101330 msgr2=0x7fa7c4199420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:11:50.113 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:50.109+0000 7fa7c37fe640 1 --2- 172.21.15.49:0/766891538 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa7c4101330 0x7fa7c4199420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:50.113 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:50.109+0000 7fa7c37fe640 1 -- 172.21.15.49:0/766891538 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7c41a44a0 con 0x7fa7c41056c0 2024-03-18T23:11:50.113 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:50.109+0000 7fa7c2ffd640 1 --2- 172.21.15.49:0/766891538 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa7c4100970 0x7fa7c4198ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:11:50.113 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:50.109+0000 7fa7c27fc640 1 --2- 172.21.15.49:0/766891538 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa7c4101330 0x7fa7c4199420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:11:50.113 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:50.110+0000 7fa7c37fe640 1 --2- 172.21.15.49:0/766891538 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa7c41056c0 0x7fa7c41a3f60 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fa7bc037040 tx=0x7fa7bc032ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:11:50.113 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:50.110+0000 7fa7abfff640 1 -- 172.21.15.49:0/766891538 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7bc042020 con 0x7fa7c41056c0 2024-03-18T23:11:50.113 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:50.110+0000 7fa7c96e8640 1 -- 172.21.15.49:0/766891538 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7c41a4670 con 0x7fa7c41056c0 2024-03-18T23:11:50.113 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:50.110+0000 7fa7abfff640 1 -- 172.21.15.49:0/766891538 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa7bc034640 con 0x7fa7c41056c0 2024-03-18T23:11:50.113 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:50.110+0000 7fa7abfff640 1 -- 172.21.15.49:0/766891538 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7bc039040 con 0x7fa7c41056c0 2024-03-18T23:11:50.113 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:50.110+0000 7fa7c96e8640 1 -- 172.21.15.49:0/766891538 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7c41a4b30 con 0x7fa7c41056c0 2024-03-18T23:11:50.118 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:50.112+0000 7fa7abfff640 1 -- 172.21.15.49:0/766891538 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 25) ==== 137421+0+0 (secure 0 0 0) 0x7fa7bc045020 con 0x7fa7c41056c0 2024-03-18T23:11:50.118 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:50.112+0000 7fa7cc3ab740 1 Processor -- start 2024-03-18T23:11:50.118 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:50.112+0000 7fa7cc3ab740 1 -- start start 2024-03-18T23:11:50.118 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:50.113+0000 7fa7cc3ab740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5563d822cef0 0x5563d822d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:11:50.118 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:50.113+0000 7fa7cc3ab740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5563d822d810 0x5563d82360a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:11:50.119 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:50.113+0000 7fa7cc3ab740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5563d82365e0 0x5563d82389d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:11:50.119 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:50.113+0000 7fa7cc3ab740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5563d8190be0 con 0x5563d822d810 2024-03-18T23:11:50.119 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:50.113+0000 7fa7cc3ab740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x5563d8139960 con 0x5563d822cef0 2024-03-18T23:11:50.119 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:50.113+0000 7fa7cc3ab740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5563d818cc80 con 0x5563d82365e0 2024-03-18T23:11:50.119 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:50.113+0000 7fa7c2ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5563d822d810 0x5563d82360a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:11:50.119 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:50.113+0000 7fa7c27fc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5563d822cef0 0x5563d822d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:11:50.119 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:50.113+0000 7fa7c2ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5563d822d810 0x5563d82360a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:39786/0 (socket says 172.21.15.49:39786) 2024-03-18T23:11:50.119 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:50.113+0000 7fa7c2ffd640 1 -- 172.21.15.49:0/422369176 learned_addr learned my addr 172.21.15.49:0/422369176 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:11:50.119 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:50.113+0000 7fa7abfff640 1 --2- 172.21.15.49:0/766891538 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7fa7ac0a2a10 0x7fa7ac0a4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:11:50.119 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:50.113+0000 7fa7c27fc640 1 --2- 172.21.15.49:0/422369176 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5563d82365e0 0x5563d82389d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:11:50.119 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:50.113+0000 7fa7c37fe640 1 --2- 172.21.15.49:0/766891538 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7fa7ac0a2a10 0x7fa7ac0a4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:11:50.119 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:50.113+0000 7fa7abfff640 1 -- 172.21.15.49:0/766891538 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8469+0+0 (secure 0 0 0) 0x7fa7bc0f0850 con 0x7fa7c41056c0 2024-03-18T23:11:50.119 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:50.113+0000 7fa7c2ffd640 1 -- 172.21.15.49:0/422369176 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5563d82365e0 msgr2=0x5563d82389d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:11:50.119 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:50.113+0000 7fa7c2ffd640 1 --2- 172.21.15.49:0/422369176 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5563d82365e0 0x5563d82389d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:50.119 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:50.113+0000 7fa7c2ffd640 1 -- 172.21.15.49:0/422369176 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5563d822cef0 msgr2=0x5563d822d2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:11:50.120 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:50.113+0000 7fa7c2ffd640 1 --2- 172.21.15.49:0/422369176 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5563d822cef0 0x5563d822d2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:50.120 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:50.113+0000 7fa7c2ffd640 1 -- 172.21.15.49:0/422369176 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5563d8130d20 con 0x5563d822d810 2024-03-18T23:11:50.120 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:50.113+0000 7fa7c2ffd640 1 --2- 172.21.15.49:0/422369176 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5563d822d810 0x5563d82360a0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fa7b4013800 tx=0x7fa7b4013cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8b24e174183b600 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:11:50.120 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:50.114+0000 7fa7c37fe640 1 --2- 172.21.15.49:0/766891538 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7fa7ac0a2a10 0x7fa7ac0a4ed0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fa7bc0ee190 tx=0x7fa7bc0dd040 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:11:50.120 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:50.114+0000 7fa7a97fa640 1 -- 172.21.15.49:0/422369176 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7b4021070 con 0x5563d822d810 2024-03-18T23:11:50.120 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:50.114+0000 7fa7a97fa640 1 -- 172.21.15.49:0/422369176 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa7b401c410 con 0x5563d822d810 2024-03-18T23:11:50.120 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:50.114+0000 7fa7a97fa640 1 -- 172.21.15.49:0/422369176 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7b401b5d0 con 0x5563d822d810 2024-03-18T23:11:50.120 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:50.114+0000 7fa7cc3ab740 1 -- 172.21.15.49:0/422369176 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5563d822d810 msgr2=0x5563d82360a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:11:50.120 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:50.114+0000 7fa7cc3ab740 1 --2- 172.21.15.49:0/422369176 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5563d822d810 0x5563d82360a0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fa7b4013800 tx=0x7fa7b4013cc0 comp rx=0 tx=0).stop 2024-03-18T23:11:50.120 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:50.115+0000 7fa7cc3ab740 1 -- 172.21.15.49:0/422369176 shutdown_connections 2024-03-18T23:11:50.120 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:50.115+0000 7fa7cc3ab740 1 --2- 172.21.15.49:0/422369176 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5563d82365e0 0x5563d82389d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:50.120 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:50.115+0000 7fa7cc3ab740 1 --2- 172.21.15.49:0/422369176 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5563d822d810 0x5563d82360a0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:50.120 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:50.115+0000 7fa7cc3ab740 1 --2- 172.21.15.49:0/422369176 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5563d822cef0 0x5563d822d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:50.120 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:50.115+0000 7fa7cc3ab740 1 -- 172.21.15.49:0/422369176 >> 172.21.15.49:0/422369176 conn(0x5563d8227810 msgr2=0x5563d8227bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:11:50.120 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:50.115+0000 7fa7cc3ab740 1 -- 172.21.15.49:0/422369176 shutdown_connections 2024-03-18T23:11:50.121 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:50.115+0000 7fa7cc3ab740 1 -- 172.21.15.49:0/422369176 wait complete. 2024-03-18T23:11:50.121 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:50.115+0000 7fa7cc3ab740 1 -- start start 2024-03-18T23:11:50.121 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:50.115+0000 7fa7cc3ab740 1 Processor -- start 2024-03-18T23:11:50.121 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:50.115+0000 7fa7cc3ab740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5563d822cef0 0x5563d8236040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:11:50.121 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:50.115+0000 7fa7cc3ab740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5563d822d810 0x5563d82333d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:11:50.121 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:50.115+0000 7fa7cc3ab740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5563d82365e0 0x5563d8233af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:11:50.121 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:50.115+0000 7fa7cc3ab740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5563d8239210 con 0x5563d822cef0 2024-03-18T23:11:50.121 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:50.115+0000 7fa7cc3ab740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x5563d814cdb0 con 0x5563d82365e0 2024-03-18T23:11:50.121 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:50.115+0000 7fa7cc3ab740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5563d8239390 con 0x5563d822d810 2024-03-18T23:11:50.121 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:50.115+0000 7fa7c27fc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5563d822cef0 0x5563d8236040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:11:50.121 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:50.115+0000 7fa7cc3ab740 10 client.0 _ll_register_callbacks cb 0x5563d8151e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T23:11:50.121 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:50.115+0000 7fa7c27fc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5563d822cef0 0x5563d8236040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:39790/0 (socket says 172.21.15.49:39790) 2024-03-18T23:11:50.121 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:50.115+0000 7fa7c27fc640 1 -- 172.21.15.49:0/1205007572 learned_addr learned my addr 172.21.15.49:0/1205007572 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:11:50.121 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:50.115+0000 7fa7c2ffd640 1 --2- 172.21.15.49:0/1205007572 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5563d822d810 0x5563d82333d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:11:50.121 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:50.116+0000 7fa7c27fc640 1 --2- 172.21.15.49:0/1205007572 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5563d82365e0 0x5563d8233af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:11:50.121 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:50.116+0000 7fa7c27fc640 1 -- 172.21.15.49:0/1205007572 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5563d822d810 msgr2=0x5563d82333d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:11:50.122 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:50.116+0000 7fa7c27fc640 1 --2- 172.21.15.49:0/1205007572 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5563d822d810 0x5563d82333d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:50.122 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:50.116+0000 7fa7c27fc640 1 -- 172.21.15.49:0/1205007572 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5563d82365e0 msgr2=0x5563d8233af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:11:50.122 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:50.116+0000 7fa7c27fc640 1 --2- 172.21.15.49:0/1205007572 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5563d82365e0 0x5563d8233af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:50.122 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:50.116+0000 7fa7c27fc640 1 -- 172.21.15.49:0/1205007572 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5563d814bc60 con 0x5563d822cef0 2024-03-18T23:11:50.122 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:50.116+0000 7fa7c27fc640 1 --2- 172.21.15.49:0/1205007572 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5563d822cef0 0x5563d8236040 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fa7b80043e0 tx=0x7fa7b8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17d7554d14e5695d server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:11:50.122 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:50.116+0000 7fa7a6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-18T23:11:50.122 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:50.116+0000 7fa7a6ff5640 1 -- 172.21.15.49:0/1205007572 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7b8003ce0 con 0x5563d822cef0 2024-03-18T23:11:50.122 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:50.116+0000 7fa7a6ff5640 1 -- 172.21.15.49:0/1205007572 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8469+0+0 (secure 0 0 0) 0x7fa7b8005020 con 0x5563d822cef0 2024-03-18T23:11:50.122 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:50.116+0000 7fa7cc3ab740 1 -- 172.21.15.49:0/1205007572 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa7b401c410 con 0x5563d822cef0 2024-03-18T23:11:50.122 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:50.116+0000 7fa7a6ff5640 1 -- 172.21.15.49:0/1205007572 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fa7b8005220 con 0x5563d822cef0 2024-03-18T23:11:50.122 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:50.116+0000 7fa7cc3ab740 10 client.15747 fetch_fsmap learned FSMap version 32 2024-03-18T23:11:50.122 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:50.116+0000 7fa7cc3ab740 1 -- 172.21.15.49:0/1205007572 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5563d7ea5210 con 0x5563d822cef0 2024-03-18T23:11:50.122 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:50.116+0000 7fa7a6ff5640 1 -- 172.21.15.49:0/1205007572 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa7b401c410 con 0x5563d822cef0 2024-03-18T23:11:50.122 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:50.117+0000 7fa7a6ff5640 1 -- 172.21.15.49:0/1205007572 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7fa7b801d030 con 0x5563d822cef0 2024-03-18T23:11:50.122 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:50.117+0000 7fa7cc3ab740 10 client.15747 fetch_fsmap finished waiting for FSMap version 32 2024-03-18T23:11:50.123 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:50.117+0000 7fa7cc3ab740 10 client.15747 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T23:11:50.123 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:50.117+0000 7fa7cc3ab740 20 client.15747 populate_metadata read hostname 'smithi049' 2024-03-18T23:11:50.123 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:50.117+0000 7fa7cc3ab740 1 --2- 172.21.15.49:0/1205007572 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x5563d8246540 0x5563d8248930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:11:50.123 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:50.117+0000 7fa7cc3ab740 4 client.15747 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T23:11:50.123 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:50.117+0000 7fa7cc3ab740 1 -- 172.21.15.49:0/1205007572 --> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5563d819e120 con 0x5563d8246540 2024-03-18T23:11:50.123 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:50.117+0000 7fa7c2ffd640 1 --2- 172.21.15.49:0/1205007572 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x5563d8246540 0x5563d8248930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:11:50.123 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:50.117+0000 7fa7c2ffd640 1 --2- 172.21.15.49:0/1205007572 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x5563d8246540 0x5563d8248930 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7060de3a8db9e7ed server_cookie=2c8d2a7f9d8e385a in_seq=0 out_seq=0 2024-03-18T23:11:50.123 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:50.117+0000 7fa7a6ff5640 10 client.15747 ms_handle_connect on v2:172.21.15.139:6838/4280128209 2024-03-18T23:11:50.124 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:50.119+0000 7fa7a6ff5640 1 -- 172.21.15.49:0/1205007572 <== mds.0 v2:172.21.15.139:6838/4280128209 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5563d819e120 con 0x5563d8246540 2024-03-18T23:11:50.124 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:50.119+0000 7fa7a6ff5640 10 client.15747 handle_command_reply: tid=0 2024-03-18T23:11:50.124 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:50.119+0000 7fa7cc3ab740 1 client.15747 shutdown 2024-03-18T23:11:50.124 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:50.119+0000 7fa7cc3ab740 1 -- 172.21.15.49:0/1205007572 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5563d822cef0 msgr2=0x5563d8236040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:11:50.124 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:50.119+0000 7fa7cc3ab740 1 --2- 172.21.15.49:0/1205007572 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5563d822cef0 0x5563d8236040 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fa7b80043e0 tx=0x7fa7b8004730 comp rx=0 tx=0).stop 2024-03-18T23:11:50.125 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:50.119+0000 7fa7cc3ab740 1 -- 172.21.15.49:0/1205007572 shutdown_connections 2024-03-18T23:11:50.125 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:50.119+0000 7fa7cc3ab740 1 --2- 172.21.15.49:0/1205007572 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x5563d8246540 0x5563d8248930 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:50.125 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:50.119+0000 7fa7cc3ab740 1 --2- 172.21.15.49:0/1205007572 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5563d82365e0 0x5563d8233af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:50.125 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:50.119+0000 7fa7a6ff5640 0 client.15747 ms_handle_reset on v2:172.21.15.139:6838/4280128209 2024-03-18T23:11:50.125 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:50.119+0000 7fa7cc3ab740 1 --2- 172.21.15.49:0/1205007572 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5563d822d810 0x5563d82333d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:50.125 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:50.119+0000 7fa7cc3ab740 1 --2- 172.21.15.49:0/1205007572 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5563d822cef0 0x5563d8236040 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:50.125 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:50.119+0000 7fa7cc3ab740 1 -- 172.21.15.49:0/1205007572 >> 172.21.15.49:0/1205007572 conn(0x5563d8227810 msgr2=0x5563d8226ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:11:50.125 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:50.119+0000 7fa7cc3ab740 1 -- 172.21.15.49:0/1205007572 shutdown_connections 2024-03-18T23:11:50.125 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:50.119+0000 7fa7cc3ab740 1 -- 172.21.15.49:0/1205007572 wait complete. 2024-03-18T23:11:50.125 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:50.119+0000 7fa7cc3ab740 20 client.15747 trim_cache size 0 max 16384 2024-03-18T23:11:50.137 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:50.135+0000 7fa7cc3ab740 1 Processor -- start 2024-03-18T23:11:50.137 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:50.135+0000 7fa7cc3ab740 1 -- start start 2024-03-18T23:11:50.137 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:50.135+0000 7fa7cc3ab740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5563d8227810 0x5563d8241fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:11:50.137 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:50.135+0000 7fa7cc3ab740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5563d822cef0 0x5563d824e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:11:50.137 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:50.135+0000 7fa7cc3ab740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5563d822d810 0x5563d8235b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:11:50.137 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:50.135+0000 7fa7cc3ab740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fa7c41ab250 con 0x5563d822cef0 2024-03-18T23:11:50.137 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:50.135+0000 7fa7cc3ab740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fa7c41acd90 con 0x5563d8227810 2024-03-18T23:11:50.137 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:50.135+0000 7fa7cc3ab740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa7c41ab920 con 0x5563d822d810 2024-03-18T23:11:50.137 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:50.135+0000 7fa7c2ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5563d822cef0 0x5563d824e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:11:50.138 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:50.135+0000 7fa7c2ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5563d822cef0 0x5563d824e1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:39806/0 (socket says 172.21.15.49:39806) 2024-03-18T23:11:50.138 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:50.135+0000 7fa7c2ffd640 1 -- 172.21.15.49:0/4054592588 learned_addr learned my addr 172.21.15.49:0/4054592588 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:11:50.138 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:50.135+0000 7fa7c27fc640 1 --2- 172.21.15.49:0/4054592588 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5563d8227810 0x5563d8241fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:11:50.138 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:50.135+0000 7fa7c27fc640 1 --2- 172.21.15.49:0/4054592588 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5563d822d810 0x5563d8235b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:11:50.138 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:50.135+0000 7fa7c2ffd640 1 -- 172.21.15.49:0/4054592588 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5563d822d810 msgr2=0x5563d8235b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:11:50.138 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:50.135+0000 7fa7c2ffd640 1 --2- 172.21.15.49:0/4054592588 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5563d822d810 0x5563d8235b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:50.138 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:50.136+0000 7fa7c2ffd640 1 -- 172.21.15.49:0/4054592588 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5563d8227810 msgr2=0x5563d8241fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:11:50.138 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:50.136+0000 7fa7c2ffd640 1 --2- 172.21.15.49:0/4054592588 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5563d8227810 0x5563d8241fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:50.138 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:50.136+0000 7fa7c2ffd640 1 -- 172.21.15.49:0/4054592588 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5563d822d640 con 0x5563d822cef0 2024-03-18T23:11:50.138 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:50.136+0000 7fa7c2ffd640 1 --2- 172.21.15.49:0/4054592588 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5563d822cef0 0x5563d824e1f0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fa7b4000c00 tx=0x7fa7b40096c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f6c6e4231c6e99 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:11:50.138 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:50.136+0000 7fa7a8ff9640 1 -- 172.21.15.49:0/4054592588 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7b4021040 con 0x5563d822cef0 2024-03-18T23:11:50.138 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:50.136+0000 7fa7a8ff9640 1 -- 172.21.15.49:0/4054592588 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa7b4022040 con 0x5563d822cef0 2024-03-18T23:11:50.139 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:50.136+0000 7fa7a8ff9640 1 -- 172.21.15.49:0/4054592588 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7b400ee20 con 0x5563d822cef0 2024-03-18T23:11:50.142 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:50.137+0000 7fa7cc3ab740 1 -- 172.21.15.49:0/4054592588 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5563d822cef0 msgr2=0x5563d824e1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:11:50.142 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:50.137+0000 7fa7cc3ab740 1 --2- 172.21.15.49:0/4054592588 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5563d822cef0 0x5563d824e1f0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fa7b4000c00 tx=0x7fa7b40096c0 comp rx=0 tx=0).stop 2024-03-18T23:11:50.142 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:50.137+0000 7fa7cc3ab740 1 -- 172.21.15.49:0/4054592588 shutdown_connections 2024-03-18T23:11:50.142 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:50.137+0000 7fa7cc3ab740 1 --2- 172.21.15.49:0/4054592588 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5563d822d810 0x5563d8235b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:50.142 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:50.137+0000 7fa7cc3ab740 1 --2- 172.21.15.49:0/4054592588 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5563d822cef0 0x5563d824e1f0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:50.142 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:50.137+0000 7fa7cc3ab740 1 --2- 172.21.15.49:0/4054592588 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5563d8227810 0x5563d8241fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:50.142 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:50.137+0000 7fa7cc3ab740 1 -- 172.21.15.49:0/4054592588 >> 172.21.15.49:0/4054592588 conn(0x5563d8226e40 msgr2=0x5563d8227bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:11:50.142 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:50.137+0000 7fa7cc3ab740 1 -- 172.21.15.49:0/4054592588 shutdown_connections 2024-03-18T23:11:50.142 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:50.137+0000 7fa7cc3ab740 1 -- 172.21.15.49:0/4054592588 wait complete. 2024-03-18T23:11:50.142 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:50.137+0000 7fa7cc3ab740 1 -- start start 2024-03-18T23:11:50.142 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:50.137+0000 7fa7cc3ab740 1 Processor -- start 2024-03-18T23:11:50.142 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:50.137+0000 7fa7cc3ab740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5563d8227810 0x5563d823a200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:11:50.143 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:50.137+0000 7fa7cc3ab740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5563d822cef0 0x5563d823a740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:11:50.143 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:50.137+0000 7fa7cc3ab740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5563d822d810 0x5563d82ab820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:11:50.143 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:50.137+0000 7fa7cc3ab740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fa7c41ac6c0 con 0x5563d8227810 2024-03-18T23:11:50.143 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:50.137+0000 7fa7cc3ab740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fa7c41abff0 con 0x5563d822cef0 2024-03-18T23:11:50.143 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:50.137+0000 7fa7cc3ab740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa7c41a9230 con 0x5563d822d810 2024-03-18T23:11:50.143 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:50.137+0000 7fa7cc3ab740 10 client.0 _ll_register_callbacks cb 0x7fa7b8004040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T23:11:50.143 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:50.137+0000 7fa7c27fc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5563d8227810 0x5563d823a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:11:50.143 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:50.137+0000 7fa7c27fc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5563d8227810 0x5563d823a200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:39812/0 (socket says 172.21.15.49:39812) 2024-03-18T23:11:50.143 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:50.137+0000 7fa7c27fc640 1 -- 172.21.15.49:0/3250592473 learned_addr learned my addr 172.21.15.49:0/3250592473 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:11:50.143 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:50.137+0000 7fa7c2ffd640 1 --2- 172.21.15.49:0/3250592473 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5563d822cef0 0x5563d823a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:11:50.143 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:50.138+0000 7fa7c27fc640 1 --2- 172.21.15.49:0/3250592473 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5563d822d810 0x5563d82ab820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:11:50.143 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:50.138+0000 7fa7c27fc640 1 -- 172.21.15.49:0/3250592473 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5563d822d810 msgr2=0x5563d82ab820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:11:50.143 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:50.138+0000 7fa7c27fc640 1 --2- 172.21.15.49:0/3250592473 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5563d822d810 0x5563d82ab820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:50.143 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:50.138+0000 7fa7c27fc640 1 -- 172.21.15.49:0/3250592473 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5563d822cef0 msgr2=0x5563d823a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:11:50.144 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:50.138+0000 7fa7c27fc640 1 --2- 172.21.15.49:0/3250592473 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5563d822cef0 0x5563d823a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:50.144 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:50.138+0000 7fa7c27fc640 1 -- 172.21.15.49:0/3250592473 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5563d7f71620 con 0x5563d8227810 2024-03-18T23:11:50.144 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:50.138+0000 7fa7c27fc640 1 --2- 172.21.15.49:0/3250592473 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5563d8227810 0x5563d823a200 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fa7b8003c60 tx=0x7fa7b8009eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30132b07b48ffb47 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:11:50.144 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:50.138+0000 7fa7a6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-18T23:11:50.144 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:50.138+0000 7fa7a6ff5640 1 -- 172.21.15.49:0/3250592473 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7b80053c0 con 0x5563d8227810 2024-03-18T23:11:50.144 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:50.138+0000 7fa7a6ff5640 1 -- 172.21.15.49:0/3250592473 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8469+0+0 (secure 0 0 0) 0x7fa7b8011030 con 0x5563d8227810 2024-03-18T23:11:50.144 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:50.138+0000 7fa7cc3ab740 1 -- 172.21.15.49:0/3250592473 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa7b4022040 con 0x5563d8227810 2024-03-18T23:11:50.144 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:50.138+0000 7fa7a6ff5640 1 -- 172.21.15.49:0/3250592473 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fa7b8005560 con 0x5563d8227810 2024-03-18T23:11:50.144 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:50.138+0000 7fa7cc3ab740 10 client.15753 fetch_fsmap learned FSMap version 32 2024-03-18T23:11:50.144 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:50.138+0000 7fa7cc3ab740 1 -- 172.21.15.49:0/3250592473 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5563d82acc60 con 0x5563d8227810 2024-03-18T23:11:50.144 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:50.138+0000 7fa7a6ff5640 1 -- 172.21.15.49:0/3250592473 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa7b4022040 con 0x5563d8227810 2024-03-18T23:11:50.144 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:50.138+0000 7fa7a6ff5640 1 -- 172.21.15.49:0/3250592473 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7fa7b801d070 con 0x5563d8227810 2024-03-18T23:11:50.144 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:50.139+0000 7fa7cc3ab740 10 client.15753 fetch_fsmap finished waiting for FSMap version 32 2024-03-18T23:11:50.144 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:50.139+0000 7fa7cc3ab740 10 client.15753 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T23:11:50.144 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:50.139+0000 7fa7cc3ab740 20 client.15753 populate_metadata read hostname 'smithi049' 2024-03-18T23:11:50.144 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:50.139+0000 7fa7cc3ab740 1 --2- 172.21.15.49:0/3250592473 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x5563d82365e0 0x5563d82af2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:11:50.145 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:50.139+0000 7fa7cc3ab740 4 client.15753 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T23:11:50.145 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:50.139+0000 7fa7cc3ab740 1 -- 172.21.15.49:0/3250592473 --> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5563d8198470 con 0x5563d82365e0 2024-03-18T23:11:50.145 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:50.139+0000 7fa7c2ffd640 1 --2- 172.21.15.49:0/3250592473 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x5563d82365e0 0x5563d82af2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:11:50.145 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:50.139+0000 7fa7c2ffd640 1 --2- 172.21.15.49:0/3250592473 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x5563d82365e0 0x5563d82af2d0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a3fc9de62bd50be2 server_cookie=fbcaac897ee4940 in_seq=0 out_seq=0 2024-03-18T23:11:50.145 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:50.139+0000 7fa7a6ff5640 10 client.15753 ms_handle_connect on v2:172.21.15.139:6838/4280128209 2024-03-18T23:11:50.303 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:50.301+0000 7fa7a6ff5640 1 -- 172.21.15.49:0/3250592473 <== mds.0 v2:172.21.15.139:6838/4280128209 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5563d8198470 con 0x5563d82365e0 2024-03-18T23:11:50.303 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:50.301+0000 7fa7a6ff5640 10 client.15753 handle_command_reply: tid=0 2024-03-18T23:11:50.303 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:50.301+0000 7fa7cc3ab740 1 client.15753 shutdown 2024-03-18T23:11:50.303 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:50.301+0000 7fa7cc3ab740 1 -- 172.21.15.49:0/3250592473 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5563d8227810 msgr2=0x5563d823a200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:11:50.303 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:50.301+0000 7fa7cc3ab740 1 --2- 172.21.15.49:0/3250592473 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5563d8227810 0x5563d823a200 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fa7b8003c60 tx=0x7fa7b8009eb0 comp rx=0 tx=0).stop 2024-03-18T23:11:50.303 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:50.301+0000 7fa7cc3ab740 1 -- 172.21.15.49:0/3250592473 shutdown_connections 2024-03-18T23:11:50.304 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:50.301+0000 7fa7cc3ab740 1 --2- 172.21.15.49:0/3250592473 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x5563d82365e0 0x5563d82af2d0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:50.304 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:50.301+0000 7fa7cc3ab740 1 --2- 172.21.15.49:0/3250592473 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5563d822d810 0x5563d82ab820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:50.304 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:50.301+0000 7fa7a6ff5640 0 client.15753 ms_handle_reset on v2:172.21.15.139:6838/4280128209 2024-03-18T23:11:50.304 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:50.301+0000 7fa7cc3ab740 1 --2- 172.21.15.49:0/3250592473 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5563d822cef0 0x5563d823a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:50.304 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:50.301+0000 7fa7cc3ab740 1 --2- 172.21.15.49:0/3250592473 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5563d8227810 0x5563d823a200 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:50.304 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:50.301+0000 7fa7cc3ab740 1 -- 172.21.15.49:0/3250592473 >> 172.21.15.49:0/3250592473 conn(0x5563d8226e40 msgr2=0x5563d822dea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:11:50.304 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:50.302+0000 7fa7cc3ab740 1 -- 172.21.15.49:0/3250592473 shutdown_connections 2024-03-18T23:11:50.304 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:50.302+0000 7fa7cc3ab740 1 -- 172.21.15.49:0/3250592473 wait complete. 2024-03-18T23:11:50.304 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:50.302+0000 7fa7cc3ab740 20 client.15753 trim_cache size 0 max 16384 2024-03-18T23:11:50.304 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:{ 2024-03-18T23:11:50.305 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "message": "", 2024-03-18T23:11:50.305 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "return_code": 0 2024-03-18T23:11:50.305 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:} 2024-03-18T23:11:50.305 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:50.302+0000 7fa7c96e8640 1 -- 172.21.15.49:0/766891538 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7fa7ac0a2a10 msgr2=0x7fa7ac0a4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:11:50.305 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:50.302+0000 7fa7c96e8640 1 --2- 172.21.15.49:0/766891538 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7fa7ac0a2a10 0x7fa7ac0a4ed0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fa7bc0ee190 tx=0x7fa7bc0dd040 comp rx=0 tx=0).stop 2024-03-18T23:11:50.306 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:50.303+0000 7fa7c96e8640 1 -- 172.21.15.49:0/766891538 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa7c41056c0 msgr2=0x7fa7c41a3f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:11:50.306 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:50.303+0000 7fa7c96e8640 1 --2- 172.21.15.49:0/766891538 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa7c41056c0 0x7fa7c41a3f60 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fa7bc037040 tx=0x7fa7bc032ea0 comp rx=0 tx=0).stop 2024-03-18T23:11:50.306 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:50.303+0000 7fa7c96e8640 1 -- 172.21.15.49:0/766891538 shutdown_connections 2024-03-18T23:11:50.306 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:50.303+0000 7fa7c96e8640 1 --2- 172.21.15.49:0/766891538 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7fa7ac0a2a10 0x7fa7ac0a4ed0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:50.306 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:50.303+0000 7fa7c96e8640 1 --2- 172.21.15.49:0/766891538 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa7c41056c0 0x7fa7c41a3f60 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:50.306 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:50.303+0000 7fa7c96e8640 1 --2- 172.21.15.49:0/766891538 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa7c4101330 0x7fa7c4199420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:50.306 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:50.303+0000 7fa7c96e8640 1 --2- 172.21.15.49:0/766891538 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa7c4100970 0x7fa7c4198ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:50.306 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:50.303+0000 7fa7c96e8640 1 -- 172.21.15.49:0/766891538 >> 172.21.15.49:0/766891538 conn(0x7fa7c4077570 msgr2=0x7fa7c40ff5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:11:50.306 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:50.303+0000 7fa7c96e8640 1 -- 172.21.15.49:0/766891538 shutdown_connections 2024-03-18T23:11:50.306 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:11:50.303+0000 7fa7c96e8640 1 -- 172.21.15.49:0/766891538 wait complete. 2024-03-18T23:11:50.766 INFO:tasks.workunit.client.0.smithi049.stdout: 1 394035 11.88 MB/sec execute 52 sec latency 61.233 ms 2024-03-18T23:11:51.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:11:51 smithi175 ceph-mon[67352]: pgmap v111: 97 pgs: 97 active+clean; 50 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.1 MiB/s wr, 360 op/s 2024-03-18T23:11:51.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:11:51 smithi139 ceph-mon[67282]: pgmap v111: 97 pgs: 97 active+clean; 50 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.1 MiB/s wr, 360 op/s 2024-03-18T23:11:51.629 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:11:51 smithi049 ceph-mon[62991]: pgmap v111: 97 pgs: 97 active+clean; 50 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.1 MiB/s wr, 360 op/s 2024-03-18T23:11:51.766 INFO:tasks.workunit.client.0.smithi049.stdout: 1 396209 11.79 MB/sec execute 53 sec latency 25.489 ms 2024-03-18T23:11:52.766 INFO:tasks.workunit.client.0.smithi049.stdout: 1 398031 11.80 MB/sec execute 54 sec latency 48.774 ms 2024-03-18T23:11:53.432 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:11:53 smithi049 ceph-mon[62991]: pgmap v112: 97 pgs: 97 active+clean; 43 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 4.4 MiB/s wr, 553 op/s 2024-03-18T23:11:53.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:11:53 smithi175 ceph-mon[67352]: pgmap v112: 97 pgs: 97 active+clean; 43 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 4.4 MiB/s wr, 553 op/s 2024-03-18T23:11:53.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:11:53 smithi139 ceph-mon[67282]: pgmap v112: 97 pgs: 97 active+clean; 43 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 4.4 MiB/s wr, 553 op/s 2024-03-18T23:11:53.766 INFO:tasks.workunit.client.0.smithi049.stdout: 1 400516 11.82 MB/sec execute 55 sec latency 116.056 ms 2024-03-18T23:11:54.766 INFO:tasks.workunit.client.0.smithi049.stdout: 1 402293 11.81 MB/sec execute 56 sec latency 17.572 ms 2024-03-18T23:11:55.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:11:55 smithi175 ceph-mon[67352]: pgmap v113: 97 pgs: 97 active+clean; 43 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.9 MiB/s wr, 326 op/s 2024-03-18T23:11:55.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:11:55 smithi139 ceph-mon[67282]: pgmap v113: 97 pgs: 97 active+clean; 43 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.9 MiB/s wr, 326 op/s 2024-03-18T23:11:55.630 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:11:55 smithi049 ceph-mon[62991]: pgmap v113: 97 pgs: 97 active+clean; 43 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.9 MiB/s wr, 326 op/s 2024-03-18T23:11:55.766 INFO:tasks.workunit.client.0.smithi049.stdout: 1 404792 11.85 MB/sec execute 57 sec latency 12.281 ms 2024-03-18T23:11:56.766 INFO:tasks.workunit.client.0.smithi049.stdout: 1 407405 11.86 MB/sec execute 58 sec latency 18.871 ms 2024-03-18T23:11:57.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:11:57 smithi175 ceph-mon[67352]: pgmap v114: 97 pgs: 97 active+clean; 43 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.9 MiB/s wr, 326 op/s 2024-03-18T23:11:57.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:11:57 smithi139 ceph-mon[67282]: pgmap v114: 97 pgs: 97 active+clean; 43 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.9 MiB/s wr, 326 op/s 2024-03-18T23:11:57.629 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:11:57 smithi049 ceph-mon[62991]: pgmap v114: 97 pgs: 97 active+clean; 43 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.9 MiB/s wr, 326 op/s 2024-03-18T23:11:57.766 INFO:tasks.workunit.client.0.smithi049.stdout: 1 409359 11.85 MB/sec execute 59 sec latency 15.761 ms 2024-03-18T23:11:58.766 INFO:tasks.workunit.client.0.smithi049.stdout: 1 411746 11.89 MB/sec execute 60 sec latency 77.272 ms 2024-03-18T23:11:59.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:11:59 smithi175 ceph-mon[67352]: pgmap v115: 97 pgs: 97 active+clean; 49 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 4.5 MiB/s wr, 435 op/s 2024-03-18T23:11:59.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:11:59 smithi139 ceph-mon[67282]: pgmap v115: 97 pgs: 97 active+clean; 49 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 4.5 MiB/s wr, 435 op/s 2024-03-18T23:11:59.629 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:11:59 smithi049 ceph-mon[62991]: pgmap v115: 97 pgs: 97 active+clean; 49 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 4.5 MiB/s wr, 435 op/s 2024-03-18T23:11:59.766 INFO:tasks.workunit.client.0.smithi049.stdout: 1 413944 11.81 MB/sec execute 61 sec latency 35.381 ms 2024-03-18T23:12:00.766 INFO:tasks.workunit.client.0.smithi049.stdout: 1 415400 11.82 MB/sec execute 62 sec latency 27.879 ms 2024-03-18T23:12:01.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:12:01 smithi175 ceph-mon[67352]: pgmap v116: 97 pgs: 97 active+clean; 49 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.1 MiB/s wr, 325 op/s 2024-03-18T23:12:01.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:12:01 smithi139 ceph-mon[67282]: pgmap v116: 97 pgs: 97 active+clean; 49 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.1 MiB/s wr, 325 op/s 2024-03-18T23:12:01.629 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:12:01 smithi049 ceph-mon[62991]: pgmap v116: 97 pgs: 97 active+clean; 49 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.1 MiB/s wr, 325 op/s 2024-03-18T23:12:01.766 INFO:tasks.workunit.client.0.smithi049.stdout: 1 417814 11.80 MB/sec execute 63 sec latency 25.637 ms 2024-03-18T23:12:02.766 INFO:tasks.workunit.client.0.smithi049.stdout: 1 419562 11.76 MB/sec execute 64 sec latency 16.662 ms 2024-03-18T23:12:03.402 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:03.398+0000 7f5f3673b640 1 -- 172.21.15.49:0/3416944317 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5f30107800 msgr2=0x7f5f30107be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:12:03.402 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:03.398+0000 7f5f3673b640 1 --2- 172.21.15.49:0/3416944317 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5f30107800 0x7f5f30107be0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f5f20009930 tx=0x7f5f2002bc30 comp rx=0 tx=0).stop 2024-03-18T23:12:03.402 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:03.398+0000 7f5f3673b640 1 -- 172.21.15.49:0/3416944317 shutdown_connections 2024-03-18T23:12:03.402 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:03.398+0000 7f5f3673b640 1 --2- 172.21.15.49:0/3416944317 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5f301021b0 0x7f5f3010e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:03.402 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:03.398+0000 7f5f3673b640 1 --2- 172.21.15.49:0/3416944317 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5f301017f0 0x7f5f30101c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:03.403 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:03.398+0000 7f5f3673b640 1 --2- 172.21.15.49:0/3416944317 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5f30107800 0x7f5f30107be0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:03.403 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:03.398+0000 7f5f3673b640 1 -- 172.21.15.49:0/3416944317 >> 172.21.15.49:0/3416944317 conn(0x7f5f300fd300 msgr2=0x7f5f300ff720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:12:03.403 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:03.398+0000 7f5f3673b640 1 -- 172.21.15.49:0/3416944317 shutdown_connections 2024-03-18T23:12:03.403 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:03.398+0000 7f5f3673b640 1 -- 172.21.15.49:0/3416944317 wait complete. 2024-03-18T23:12:03.403 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:03.398+0000 7f5f3673b640 1 Processor -- start 2024-03-18T23:12:03.403 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:03.398+0000 7f5f3673b640 1 -- start start 2024-03-18T23:12:03.404 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:03.399+0000 7f5f3673b640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5f301017f0 0x7f5f30073b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:12:03.404 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:03.399+0000 7f5f3673b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5f301021b0 0x7f5f30074060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:12:03.404 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:03.399+0000 7f5f3673b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5f30107800 0x7f5f300745a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:12:03.404 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:03.399+0000 7f5f3673b640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f5f30111c10 con 0x7f5f30107800 2024-03-18T23:12:03.404 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:03.399+0000 7f5f3673b640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f5f30111a90 con 0x7f5f301021b0 2024-03-18T23:12:03.404 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:03.399+0000 7f5f3673b640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f5f30111d90 con 0x7f5f301017f0 2024-03-18T23:12:03.404 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:03.399+0000 7f5f34cd8640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5f30107800 0x7f5f300745a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:12:03.404 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:03.399+0000 7f5f2ffff640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5f301017f0 0x7f5f30073b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:12:03.405 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:03.399+0000 7f5f2f7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5f301021b0 0x7f5f30074060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:12:03.405 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:03.399+0000 7f5f2ffff640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5f301017f0 0x7f5f30073b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.49:43354/0 (socket says 172.21.15.49:43354) 2024-03-18T23:12:03.405 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:03.399+0000 7f5f2ffff640 1 -- 172.21.15.49:0/360332389 learned_addr learned my addr 172.21.15.49:0/360332389 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:12:03.405 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:03.399+0000 7f5f2ffff640 1 -- 172.21.15.49:0/360332389 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5f301021b0 msgr2=0x7f5f30074060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:12:03.405 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:03.399+0000 7f5f2ffff640 1 --2- 172.21.15.49:0/360332389 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5f301021b0 0x7f5f30074060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:03.405 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:03.399+0000 7f5f2ffff640 1 -- 172.21.15.49:0/360332389 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5f30107800 msgr2=0x7f5f300745a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:12:03.405 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:03.399+0000 7f5f2ffff640 1 --2- 172.21.15.49:0/360332389 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5f30107800 0x7f5f300745a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:03.405 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:03.399+0000 7f5f2ffff640 1 -- 172.21.15.49:0/360332389 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f301a6530 con 0x7f5f301017f0 2024-03-18T23:12:03.406 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:03.399+0000 7f5f34cd8640 1 --2- 172.21.15.49:0/360332389 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5f30107800 0x7f5f300745a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:12:03.406 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:03.400+0000 7f5f2ffff640 1 --2- 172.21.15.49:0/360332389 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5f301017f0 0x7f5f30073b20 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f5f20009230 tx=0x7f5f2002cb30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:12:03.406 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:03.400+0000 7f5f2d7fa640 1 -- 172.21.15.49:0/360332389 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f20041070 con 0x7f5f301017f0 2024-03-18T23:12:03.406 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:03.400+0000 7f5f3673b640 1 -- 172.21.15.49:0/360332389 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f301a67c0 con 0x7f5f301017f0 2024-03-18T23:12:03.406 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:03.400+0000 7f5f2d7fa640 1 -- 172.21.15.49:0/360332389 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5f20004450 con 0x7f5f301017f0 2024-03-18T23:12:03.406 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:03.400+0000 7f5f2d7fa640 1 -- 172.21.15.49:0/360332389 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f20003e50 con 0x7f5f301017f0 2024-03-18T23:12:03.406 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:03.400+0000 7f5f3673b640 1 -- 172.21.15.49:0/360332389 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f301a6d00 con 0x7f5f301017f0 2024-03-18T23:12:03.416 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:03.402+0000 7f5f2d7fa640 1 -- 172.21.15.49:0/360332389 <== mon.1 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 25) ==== 137421+0+0 (secure 0 0 0) 0x7f5f20044070 con 0x7f5f301017f0 2024-03-18T23:12:03.416 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:03.402+0000 7f5f39408740 1 Processor -- start 2024-03-18T23:12:03.416 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:03.403+0000 7f5f39408740 1 -- start start 2024-03-18T23:12:03.417 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:03.403+0000 7f5f39408740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556285074ef0 0x5562850752d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:12:03.417 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:03.403+0000 7f5f39408740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556285075810 0x55628507e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:12:03.417 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:03.403+0000 7f5f39408740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55628507e5e0 0x5562850809d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:12:03.417 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:03.403+0000 7f5f39408740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x556284fd8be0 con 0x55628507e5e0 2024-03-18T23:12:03.417 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:03.403+0000 7f5f39408740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x556284f81960 con 0x556285075810 2024-03-18T23:12:03.417 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:03.403+0000 7f5f39408740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x556284fd4c80 con 0x556285074ef0 2024-03-18T23:12:03.417 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:03.403+0000 7f5f2f7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55628507e5e0 0x5562850809d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:12:03.417 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:03.403+0000 7f5f34cd8640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556285074ef0 0x5562850752d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:12:03.417 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:03.403+0000 7f5f2f7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55628507e5e0 0x5562850809d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:37474/0 (socket says 172.21.15.49:37474) 2024-03-18T23:12:03.417 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:03.403+0000 7f5f2f7fe640 1 -- 172.21.15.49:0/4213677648 learned_addr learned my addr 172.21.15.49:0/4213677648 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:12:03.417 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:03.403+0000 7f5f2ffff640 1 --2- 172.21.15.49:0/4213677648 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556285075810 0x55628507e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:12:03.418 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:03.403+0000 7f5f2d7fa640 1 --2- 172.21.15.49:0/360332389 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f5f180a2bb0 0x7f5f180a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:12:03.418 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:03.403+0000 7f5f34cd8640 1 --2- 172.21.15.49:0/360332389 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f5f180a2bb0 0x7f5f180a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:12:03.418 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:03.403+0000 7f5f2f7fe640 1 -- 172.21.15.49:0/4213677648 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556285074ef0 msgr2=0x5562850752d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:12:03.418 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:03.403+0000 7f5f2f7fe640 1 --2- 172.21.15.49:0/4213677648 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556285074ef0 0x5562850752d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:03.418 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:03.403+0000 7f5f2f7fe640 1 -- 172.21.15.49:0/4213677648 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556285075810 msgr2=0x55628507e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:12:03.418 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:03.403+0000 7f5f2f7fe640 1 --2- 172.21.15.49:0/4213677648 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556285075810 0x55628507e0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:03.418 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:03.403+0000 7f5f2f7fe640 1 -- 172.21.15.49:0/4213677648 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556284f78d20 con 0x55628507e5e0 2024-03-18T23:12:03.418 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:03.403+0000 7f5f2d7fa640 1 -- 172.21.15.49:0/360332389 <== mon.1 v2:172.21.15.175:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8469+0+0 (secure 0 0 0) 0x7f5f200f0610 con 0x7f5f301017f0 2024-03-18T23:12:03.418 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:03.403+0000 7f5f2f7fe640 1 --2- 172.21.15.49:0/4213677648 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55628507e5e0 0x5562850809d0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f5f24009ca0 tx=0x7f5f24011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55547816e3dfd9c6 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:12:03.418 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:03.404+0000 7f5f34cd8640 1 --2- 172.21.15.49:0/360332389 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f5f180a2bb0 0x7f5f180a5070 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f5f280094d0 tx=0x7f5f28004ab0 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:12:03.418 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:03.404+0000 7f5f167fc640 1 -- 172.21.15.49:0/4213677648 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f2401e070 con 0x55628507e5e0 2024-03-18T23:12:03.418 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:03.404+0000 7f5f167fc640 1 -- 172.21.15.49:0/4213677648 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5f24019b10 con 0x55628507e5e0 2024-03-18T23:12:03.419 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:03.404+0000 7f5f167fc640 1 -- 172.21.15.49:0/4213677648 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f2400ea80 con 0x55628507e5e0 2024-03-18T23:12:03.419 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:03.404+0000 7f5f39408740 1 -- 172.21.15.49:0/4213677648 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55628507e5e0 msgr2=0x5562850809d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:12:03.419 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:03.404+0000 7f5f39408740 1 --2- 172.21.15.49:0/4213677648 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55628507e5e0 0x5562850809d0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f5f24009ca0 tx=0x7f5f24011810 comp rx=0 tx=0).stop 2024-03-18T23:12:03.419 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:03.404+0000 7f5f39408740 1 -- 172.21.15.49:0/4213677648 shutdown_connections 2024-03-18T23:12:03.419 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:03.404+0000 7f5f39408740 1 --2- 172.21.15.49:0/4213677648 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55628507e5e0 0x5562850809d0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:03.419 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:03.404+0000 7f5f39408740 1 --2- 172.21.15.49:0/4213677648 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556285075810 0x55628507e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:03.419 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:03.404+0000 7f5f39408740 1 --2- 172.21.15.49:0/4213677648 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556285074ef0 0x5562850752d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:03.419 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:03.404+0000 7f5f39408740 1 -- 172.21.15.49:0/4213677648 >> 172.21.15.49:0/4213677648 conn(0x55628506f810 msgr2=0x55628506fbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:12:03.419 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:03.404+0000 7f5f39408740 1 -- 172.21.15.49:0/4213677648 shutdown_connections 2024-03-18T23:12:03.419 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:03.404+0000 7f5f39408740 1 -- 172.21.15.49:0/4213677648 wait complete. 2024-03-18T23:12:03.419 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:03.405+0000 7f5f39408740 1 -- start start 2024-03-18T23:12:03.419 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:03.405+0000 7f5f39408740 1 Processor -- start 2024-03-18T23:12:03.419 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:03.405+0000 7f5f39408740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556285074ef0 0x55628507df80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:12:03.419 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:03.405+0000 7f5f39408740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556285075810 0x55628507b310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:12:03.419 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:03.405+0000 7f5f39408740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55628507e5e0 0x55628507b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:12:03.420 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:03.405+0000 7f5f39408740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x556285081210 con 0x556285075810 2024-03-18T23:12:03.420 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:03.405+0000 7f5f39408740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x556284f94db0 con 0x556285074ef0 2024-03-18T23:12:03.420 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:03.405+0000 7f5f39408740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x556285081390 con 0x55628507e5e0 2024-03-18T23:12:03.420 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:03.406+0000 7f5f2f7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556285075810 0x55628507b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:12:03.420 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:03.406+0000 7f5f39408740 10 client.0 _ll_register_callbacks cb 0x556284f99e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T23:12:03.420 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:03.406+0000 7f5f2f7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556285075810 0x55628507b310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:37484/0 (socket says 172.21.15.49:37484) 2024-03-18T23:12:03.420 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:03.406+0000 7f5f2ffff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556285074ef0 0x55628507df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:12:03.420 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:03.406+0000 7f5f2f7fe640 1 -- 172.21.15.49:0/1762286328 learned_addr learned my addr 172.21.15.49:0/1762286328 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:12:03.420 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:03.406+0000 7f5f34cd8640 1 --2- 172.21.15.49:0/1762286328 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55628507e5e0 0x55628507b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:12:03.420 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:03.406+0000 7f5f2f7fe640 1 -- 172.21.15.49:0/1762286328 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55628507e5e0 msgr2=0x55628507b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:12:03.420 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:03.406+0000 7f5f2f7fe640 1 --2- 172.21.15.49:0/1762286328 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55628507e5e0 0x55628507b8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:03.421 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:03.406+0000 7f5f2f7fe640 1 -- 172.21.15.49:0/1762286328 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556285074ef0 msgr2=0x55628507df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:12:03.421 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:03.406+0000 7f5f2f7fe640 1 --2- 172.21.15.49:0/1762286328 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556285074ef0 0x55628507df80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:03.421 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:03.406+0000 7f5f2f7fe640 1 -- 172.21.15.49:0/1762286328 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556284f93c60 con 0x556285075810 2024-03-18T23:12:03.421 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:03.406+0000 7f5f2f7fe640 1 --2- 172.21.15.49:0/1762286328 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556285075810 0x55628507b310 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f5f24009c70 tx=0x7f5f24009660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30527a462c69df19 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:12:03.421 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:03.406+0000 7f5f0f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-18T23:12:03.421 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:03.406+0000 7f5f0f7fe640 1 -- 172.21.15.49:0/1762286328 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f24027030 con 0x556285075810 2024-03-18T23:12:03.421 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:03.406+0000 7f5f0f7fe640 1 -- 172.21.15.49:0/1762286328 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8469+0+0 (secure 0 0 0) 0x7f5f24005260 con 0x556285075810 2024-03-18T23:12:03.421 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:03.406+0000 7f5f39408740 1 -- 172.21.15.49:0/1762286328 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5f24019b10 con 0x556285075810 2024-03-18T23:12:03.421 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:03.406+0000 7f5f0f7fe640 1 -- 172.21.15.49:0/1762286328 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f5f24031050 con 0x556285075810 2024-03-18T23:12:03.421 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:03.406+0000 7f5f39408740 10 client.15765 fetch_fsmap learned FSMap version 32 2024-03-18T23:12:03.422 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:03.407+0000 7f5f39408740 1 -- 172.21.15.49:0/1762286328 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x556284ced210 con 0x556285075810 2024-03-18T23:12:03.422 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:03.407+0000 7f5f0f7fe640 1 -- 172.21.15.49:0/1762286328 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5f24019b10 con 0x556285075810 2024-03-18T23:12:03.422 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:03.407+0000 7f5f0f7fe640 1 -- 172.21.15.49:0/1762286328 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f5f24004730 con 0x556285075810 2024-03-18T23:12:03.422 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:03.407+0000 7f5f39408740 10 client.15765 fetch_fsmap finished waiting for FSMap version 32 2024-03-18T23:12:03.422 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:03.407+0000 7f5f39408740 10 client.15765 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T23:12:03.422 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:03.407+0000 7f5f39408740 20 client.15765 populate_metadata read hostname 'smithi049' 2024-03-18T23:12:03.422 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:03.407+0000 7f5f39408740 1 --2- 172.21.15.49:0/1762286328 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x55628508e410 0x556285090800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:12:03.422 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:03.407+0000 7f5f39408740 4 client.15765 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T23:12:03.422 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:03.407+0000 7f5f39408740 1 -- 172.21.15.49:0/1762286328 --> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556284fe6120 con 0x55628508e410 2024-03-18T23:12:03.422 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:03.407+0000 7f5f2ffff640 1 --2- 172.21.15.49:0/1762286328 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x55628508e410 0x556285090800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:12:03.422 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:03.407+0000 7f5f0f7fe640 10 client.15765 ms_handle_connect on v2:172.21.15.139:6838/4280128209 2024-03-18T23:12:03.422 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:03.407+0000 7f5f2ffff640 1 --2- 172.21.15.49:0/1762286328 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x55628508e410 0x556285090800 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fdedea05a58a8e57 server_cookie=9b3ee56f8f98cb84 in_seq=0 out_seq=0 2024-03-18T23:12:03.422 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:03.408+0000 7f5f0f7fe640 1 -- 172.21.15.49:0/1762286328 <== mds.0 v2:172.21.15.139:6838/4280128209 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x556284fe6120 con 0x55628508e410 2024-03-18T23:12:03.423 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:03.408+0000 7f5f0f7fe640 10 client.15765 handle_command_reply: tid=0 2024-03-18T23:12:03.423 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:03.409+0000 7f5f39408740 1 client.15765 shutdown 2024-03-18T23:12:03.423 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:03.409+0000 7f5f39408740 1 -- 172.21.15.49:0/1762286328 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556285075810 msgr2=0x55628507b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:12:03.423 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:03.409+0000 7f5f39408740 1 --2- 172.21.15.49:0/1762286328 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556285075810 0x55628507b310 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f5f24009c70 tx=0x7f5f24009660 comp rx=0 tx=0).stop 2024-03-18T23:12:03.423 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:03.409+0000 7f5f39408740 1 -- 172.21.15.49:0/1762286328 shutdown_connections 2024-03-18T23:12:03.423 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:03.409+0000 7f5f39408740 1 --2- 172.21.15.49:0/1762286328 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x55628508e410 0x556285090800 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:03.423 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:03.409+0000 7f5f39408740 1 --2- 172.21.15.49:0/1762286328 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55628507e5e0 0x55628507b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:03.423 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:03.409+0000 7f5f0f7fe640 0 client.15765 ms_handle_reset on v2:172.21.15.139:6838/4280128209 2024-03-18T23:12:03.423 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:03.409+0000 7f5f39408740 1 --2- 172.21.15.49:0/1762286328 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556285075810 0x55628507b310 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:03.423 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:03.409+0000 7f5f39408740 1 --2- 172.21.15.49:0/1762286328 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556285074ef0 0x55628507df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:03.423 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:03.409+0000 7f5f39408740 1 -- 172.21.15.49:0/1762286328 >> 172.21.15.49:0/1762286328 conn(0x55628506f810 msgr2=0x55628506eab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:12:03.424 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:03.409+0000 7f5f39408740 1 -- 172.21.15.49:0/1762286328 shutdown_connections 2024-03-18T23:12:03.424 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:03.409+0000 7f5f39408740 1 -- 172.21.15.49:0/1762286328 wait complete. 2024-03-18T23:12:03.424 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:03.409+0000 7f5f39408740 20 client.15765 trim_cache size 0 max 16384 2024-03-18T23:12:03.444 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:03.443+0000 7f5f39408740 1 Processor -- start 2024-03-18T23:12:03.448 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:03.447+0000 7f5f39408740 1 -- start start 2024-03-18T23:12:03.448 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:03.447+0000 7f5f39408740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556285074ef0 0x556285089fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:12:03.448 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:03.447+0000 7f5f39408740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556285075810 0x556285095fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:12:03.448 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:03.447+0000 7f5f39408740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55628507e5e0 0x55628507da40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:12:03.449 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:03.447+0000 7f5f39408740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f5f301af4d0 con 0x55628507e5e0 2024-03-18T23:12:03.449 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:03.447+0000 7f5f39408740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f5f301b1010 con 0x556285074ef0 2024-03-18T23:12:03.449 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:03.447+0000 7f5f39408740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f5f301afba0 con 0x556285075810 2024-03-18T23:12:03.449 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:03.448+0000 7f5f2f7fe640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556285075810 0x556285095fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:12:03.449 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:03.448+0000 7f5f2f7fe640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556285075810 0x556285095fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.49:43390/0 (socket says 172.21.15.49:43390) 2024-03-18T23:12:03.449 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:03.448+0000 7f5f2f7fe640 1 -- 172.21.15.49:0/814121265 learned_addr learned my addr 172.21.15.49:0/814121265 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:12:03.450 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:03.449+0000 7f5f2f7fe640 1 -- 172.21.15.49:0/814121265 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556285074ef0 msgr2=0x556285089fb0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-18T23:12:03.450 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:03.449+0000 7f5f2f7fe640 1 --2- 172.21.15.49:0/814121265 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556285074ef0 0x556285089fb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:03.450 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:03.449+0000 7f5f2f7fe640 1 -- 172.21.15.49:0/814121265 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55628507e5e0 msgr2=0x55628507da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:12:03.450 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:03.449+0000 7f5f2f7fe640 1 --2- 172.21.15.49:0/814121265 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55628507e5e0 0x55628507da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:03.450 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:03.449+0000 7f5f2f7fe640 1 -- 172.21.15.49:0/814121265 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55628506af30 con 0x556285075810 2024-03-18T23:12:03.451 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:03.449+0000 7f5f2f7fe640 1 --2- 172.21.15.49:0/814121265 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556285075810 0x556285095fe0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f5f24009710 tx=0x7f5f24012f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a8ce9ea806ec8ee8 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:12:03.451 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:03.450+0000 7f5f15ffb640 1 -- 172.21.15.49:0/814121265 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f24027040 con 0x556285075810 2024-03-18T23:12:03.451 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:03.450+0000 7f5f15ffb640 1 -- 172.21.15.49:0/814121265 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5f2401f040 con 0x556285075810 2024-03-18T23:12:03.451 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:03.450+0000 7f5f15ffb640 1 -- 172.21.15.49:0/814121265 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f2401e070 con 0x556285075810 2024-03-18T23:12:03.452 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:03.451+0000 7f5f39408740 1 -- 172.21.15.49:0/814121265 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556285075810 msgr2=0x556285095fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:12:03.452 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:03.451+0000 7f5f39408740 1 --2- 172.21.15.49:0/814121265 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556285075810 0x556285095fe0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f5f24009710 tx=0x7f5f24012f90 comp rx=0 tx=0).stop 2024-03-18T23:12:03.454 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:03.452+0000 7f5f39408740 1 -- 172.21.15.49:0/814121265 shutdown_connections 2024-03-18T23:12:03.454 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:03.452+0000 7f5f39408740 1 --2- 172.21.15.49:0/814121265 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55628507e5e0 0x55628507da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:03.454 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:03.452+0000 7f5f39408740 1 --2- 172.21.15.49:0/814121265 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556285075810 0x556285095fe0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:03.454 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:03.452+0000 7f5f39408740 1 --2- 172.21.15.49:0/814121265 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556285074ef0 0x556285089fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:03.454 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:03.452+0000 7f5f39408740 1 -- 172.21.15.49:0/814121265 >> 172.21.15.49:0/814121265 conn(0x55628506f810 msgr2=0x55628506eab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:12:03.454 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:03.453+0000 7f5f39408740 1 -- 172.21.15.49:0/814121265 shutdown_connections 2024-03-18T23:12:03.454 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:03.453+0000 7f5f39408740 1 -- 172.21.15.49:0/814121265 wait complete. 2024-03-18T23:12:03.454 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:03.453+0000 7f5f39408740 1 -- start start 2024-03-18T23:12:03.455 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:03.453+0000 7f5f39408740 1 Processor -- start 2024-03-18T23:12:03.455 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:03.454+0000 7f5f39408740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556285074ef0 0x556285082110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:12:03.456 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:03.454+0000 7f5f39408740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556285075810 0x556285082650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:12:03.456 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:03.454+0000 7f5f39408740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55628507e5e0 0x556285082b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:12:03.456 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:03.454+0000 7f5f39408740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f5f301b0940 con 0x556285075810 2024-03-18T23:12:03.456 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:03.454+0000 7f5f39408740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f5f301b0270 con 0x55628507e5e0 2024-03-18T23:12:03.457 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:03.454+0000 7f5f39408740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f5f301ad4b0 con 0x556285074ef0 2024-03-18T23:12:03.457 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:03.454+0000 7f5f39408740 10 client.0 _ll_register_callbacks cb 0x7f5f2401c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T23:12:03.457 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:03.454+0000 7f5f34cd8640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55628507e5e0 0x556285082b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:12:03.457 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:03.454+0000 7f5f34cd8640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55628507e5e0 0x556285082b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.49:55376/0 (socket says 172.21.15.49:55376) 2024-03-18T23:12:03.457 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:03.454+0000 7f5f34cd8640 1 -- 172.21.15.49:0/1982736957 learned_addr learned my addr 172.21.15.49:0/1982736957 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:12:03.458 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:03.454+0000 7f5f2ffff640 1 --2- 172.21.15.49:0/1982736957 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556285074ef0 0x556285082110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:12:03.458 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:03.454+0000 7f5f2f7fe640 1 --2- 172.21.15.49:0/1982736957 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556285075810 0x556285082650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:12:03.459 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:03.455+0000 7f5f34cd8640 1 -- 172.21.15.49:0/1982736957 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556285074ef0 msgr2=0x556285082110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:12:03.459 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:03.455+0000 7f5f34cd8640 1 --2- 172.21.15.49:0/1982736957 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556285074ef0 0x556285082110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:03.460 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:03.455+0000 7f5f34cd8640 1 -- 172.21.15.49:0/1982736957 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556285075810 msgr2=0x556285082650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:12:03.460 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:03.455+0000 7f5f34cd8640 1 --2- 172.21.15.49:0/1982736957 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556285075810 0x556285082650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:03.460 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:03.455+0000 7f5f34cd8640 1 -- 172.21.15.49:0/1982736957 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556284db9620 con 0x55628507e5e0 2024-03-18T23:12:03.460 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:03.456+0000 7f5f34cd8640 1 --2- 172.21.15.49:0/1982736957 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55628507e5e0 0x556285082b90 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f5f28013f60 tx=0x7f5f28013610 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ed711e228522232a server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:12:03.460 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:03.456+0000 7f5f0f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-03-18T23:12:03.460 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:03.456+0000 7f5f0f7fe640 1 -- 172.21.15.49:0/1982736957 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f28004b20 con 0x55628507e5e0 2024-03-18T23:12:03.460 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:03.456+0000 7f5f0f7fe640 1 -- 172.21.15.49:0/1982736957 <== mon.2 v2:172.21.15.139:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8469+0+0 (secure 0 0 0) 0x7f5f28004cc0 con 0x55628507e5e0 2024-03-18T23:12:03.460 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:03.456+0000 7f5f39408740 1 -- 172.21.15.49:0/1982736957 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5f2401f040 con 0x55628507e5e0 2024-03-18T23:12:03.460 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:03.456+0000 7f5f0f7fe640 1 -- 172.21.15.49:0/1982736957 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f5f28032050 con 0x55628507e5e0 2024-03-18T23:12:03.460 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:03.456+0000 7f5f39408740 10 client.25586 fetch_fsmap learned FSMap version 32 2024-03-18T23:12:03.460 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:03.456+0000 7f5f39408740 1 -- 172.21.15.49:0/1982736957 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5562850f49e0 con 0x55628507e5e0 2024-03-18T23:12:03.460 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:03.457+0000 7f5f0f7fe640 1 -- 172.21.15.49:0/1982736957 <== mon.2 v2:172.21.15.139:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5f2401f040 con 0x55628507e5e0 2024-03-18T23:12:03.460 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:03.457+0000 7f5f0f7fe640 1 -- 172.21.15.49:0/1982736957 <== mon.2 v2:172.21.15.139:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f5f2801f040 con 0x55628507e5e0 2024-03-18T23:12:03.461 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:03.457+0000 7f5f39408740 10 client.25586 fetch_fsmap finished waiting for FSMap version 32 2024-03-18T23:12:03.461 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:03.457+0000 7f5f39408740 10 client.25586 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T23:12:03.461 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:03.457+0000 7f5f39408740 20 client.25586 populate_metadata read hostname 'smithi049' 2024-03-18T23:12:03.461 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:03.457+0000 7f5f39408740 1 --2- 172.21.15.49:0/1982736957 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x5562850f5120 0x5562850f7540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:12:03.461 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:03.457+0000 7f5f39408740 4 client.25586 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T23:12:03.461 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:03.457+0000 7f5f39408740 1 -- 172.21.15.49:0/1982736957 --> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556284fe0470 con 0x5562850f5120 2024-03-18T23:12:03.461 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:03.457+0000 7f5f2ffff640 1 --2- 172.21.15.49:0/1982736957 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x5562850f5120 0x5562850f7540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:12:03.461 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:03.457+0000 7f5f0f7fe640 10 client.25586 ms_handle_connect on v2:172.21.15.139:6838/4280128209 2024-03-18T23:12:03.461 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:03.457+0000 7f5f2ffff640 1 --2- 172.21.15.49:0/1982736957 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x5562850f5120 0x5562850f7540 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f10b48547349ef50 server_cookie=d4e413773fe731e8 in_seq=0 out_seq=0 2024-03-18T23:12:03.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:12:03 smithi175 ceph-mon[67352]: pgmap v117: 97 pgs: 97 active+clean; 55 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 4.6 MiB/s wr, 457 op/s 2024-03-18T23:12:03.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:12:03 smithi139 ceph-mon[67282]: pgmap v117: 97 pgs: 97 active+clean; 55 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 4.6 MiB/s wr, 457 op/s 2024-03-18T23:12:03.630 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:12:03 smithi049 ceph-mon[62991]: pgmap v117: 97 pgs: 97 active+clean; 55 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 4.6 MiB/s wr, 457 op/s 2024-03-18T23:12:03.721 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:03.720+0000 7f5f0f7fe640 1 -- 172.21.15.49:0/1982736957 <== mds.0 v2:172.21.15.139:6838/4280128209 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556284fe0470 con 0x5562850f5120 2024-03-18T23:12:03.721 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:03.720+0000 7f5f0f7fe640 10 client.25586 handle_command_reply: tid=0 2024-03-18T23:12:03.722 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:03.720+0000 7f5f39408740 1 client.25586 shutdown 2024-03-18T23:12:03.722 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:03.720+0000 7f5f39408740 1 -- 172.21.15.49:0/1982736957 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55628507e5e0 msgr2=0x556285082b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:12:03.722 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:03.720+0000 7f5f39408740 1 --2- 172.21.15.49:0/1982736957 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55628507e5e0 0x556285082b90 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f5f28013f60 tx=0x7f5f28013610 comp rx=0 tx=0).stop 2024-03-18T23:12:03.722 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:03.720+0000 7f5f39408740 1 -- 172.21.15.49:0/1982736957 shutdown_connections 2024-03-18T23:12:03.722 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:03.720+0000 7f5f39408740 1 --2- 172.21.15.49:0/1982736957 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x5562850f5120 0x5562850f7540 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:03.722 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:03.720+0000 7f5f39408740 1 --2- 172.21.15.49:0/1982736957 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55628507e5e0 0x556285082b90 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:03.722 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:03.720+0000 7f5f39408740 1 --2- 172.21.15.49:0/1982736957 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556285075810 0x556285082650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:03.722 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:03.720+0000 7f5f39408740 1 --2- 172.21.15.49:0/1982736957 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556285074ef0 0x556285082110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:03.722 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:03.720+0000 7f5f39408740 1 -- 172.21.15.49:0/1982736957 >> 172.21.15.49:0/1982736957 conn(0x55628506f810 msgr2=0x556285075df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:12:03.722 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:03.720+0000 7f5f0f7fe640 0 client.25586 ms_handle_reset on v2:172.21.15.139:6838/4280128209 2024-03-18T23:12:03.722 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:03.720+0000 7f5f39408740 1 -- 172.21.15.49:0/1982736957 shutdown_connections 2024-03-18T23:12:03.722 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:03.720+0000 7f5f39408740 1 -- 172.21.15.49:0/1982736957 wait complete. 2024-03-18T23:12:03.723 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:03.720+0000 7f5f39408740 20 client.25586 trim_cache size 0 max 16384 2024-03-18T23:12:03.723 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:{ 2024-03-18T23:12:03.723 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "message": "", 2024-03-18T23:12:03.723 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "return_code": 0 2024-03-18T23:12:03.723 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:} 2024-03-18T23:12:03.724 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:03.721+0000 7f5f3673b640 1 -- 172.21.15.49:0/360332389 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f5f180a2bb0 msgr2=0x7f5f180a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:12:03.724 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:03.721+0000 7f5f3673b640 1 --2- 172.21.15.49:0/360332389 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f5f180a2bb0 0x7f5f180a5070 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f5f280094d0 tx=0x7f5f28004ab0 comp rx=0 tx=0).stop 2024-03-18T23:12:03.724 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:03.721+0000 7f5f3673b640 1 -- 172.21.15.49:0/360332389 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5f301017f0 msgr2=0x7f5f30073b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:12:03.724 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:03.721+0000 7f5f3673b640 1 --2- 172.21.15.49:0/360332389 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5f301017f0 0x7f5f30073b20 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f5f20009230 tx=0x7f5f2002cb30 comp rx=0 tx=0).stop 2024-03-18T23:12:03.724 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:03.721+0000 7f5f3673b640 1 -- 172.21.15.49:0/360332389 shutdown_connections 2024-03-18T23:12:03.724 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:03.721+0000 7f5f3673b640 1 --2- 172.21.15.49:0/360332389 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f5f180a2bb0 0x7f5f180a5070 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:03.724 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:03.721+0000 7f5f3673b640 1 --2- 172.21.15.49:0/360332389 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5f30107800 0x7f5f300745a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:03.724 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:03.721+0000 7f5f3673b640 1 --2- 172.21.15.49:0/360332389 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5f301021b0 0x7f5f30074060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:03.724 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:03.721+0000 7f5f3673b640 1 --2- 172.21.15.49:0/360332389 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5f301017f0 0x7f5f30073b20 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:03.725 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:03.721+0000 7f5f3673b640 1 -- 172.21.15.49:0/360332389 >> 172.21.15.49:0/360332389 conn(0x7f5f300fd300 msgr2=0x7f5f300fd880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:12:03.725 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:03.721+0000 7f5f3673b640 1 -- 172.21.15.49:0/360332389 shutdown_connections 2024-03-18T23:12:03.725 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:03.722+0000 7f5f3673b640 1 -- 172.21.15.49:0/360332389 wait complete. 2024-03-18T23:12:03.766 INFO:tasks.workunit.client.0.smithi049.stdout: 1 421914 11.79 MB/sec execute 65 sec latency 82.971 ms 2024-03-18T23:12:04.767 INFO:tasks.workunit.client.0.smithi049.stdout: 1 423585 11.76 MB/sec execute 66 sec latency 24.487 ms 2024-03-18T23:12:05.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:12:05 smithi175 ceph-mon[67352]: pgmap v118: 97 pgs: 97 active+clean; 55 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.1 MiB/s wr, 240 op/s 2024-03-18T23:12:05.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:12:05 smithi139 ceph-mon[67282]: pgmap v118: 97 pgs: 97 active+clean; 55 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.1 MiB/s wr, 240 op/s 2024-03-18T23:12:05.630 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:12:05 smithi049 ceph-mon[62991]: pgmap v118: 97 pgs: 97 active+clean; 55 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.1 MiB/s wr, 240 op/s 2024-03-18T23:12:05.766 INFO:tasks.workunit.client.0.smithi049.stdout: 1 426061 11.79 MB/sec execute 67 sec latency 23.294 ms 2024-03-18T23:12:06.767 INFO:tasks.workunit.client.0.smithi049.stdout: 1 427058 11.69 MB/sec execute 68 sec latency 444.123 ms 2024-03-18T23:12:07.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:12:07 smithi175 ceph-mon[67352]: pgmap v119: 97 pgs: 97 active+clean; 55 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.1 MiB/s wr, 240 op/s 2024-03-18T23:12:07.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:12:07 smithi139 ceph-mon[67282]: pgmap v119: 97 pgs: 97 active+clean; 55 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.1 MiB/s wr, 240 op/s 2024-03-18T23:12:07.630 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:12:07 smithi049 ceph-mon[62991]: pgmap v119: 97 pgs: 97 active+clean; 55 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.1 MiB/s wr, 240 op/s 2024-03-18T23:12:07.767 INFO:tasks.workunit.client.0.smithi049.stdout: 1 427058 11.52 MB/sec execute 69 sec latency 1444.241 ms 2024-03-18T23:12:08.767 INFO:tasks.workunit.client.0.smithi049.stdout: 1 427058 11.36 MB/sec execute 70 sec latency 2444.336 ms 2024-03-18T23:12:09.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:12:09 smithi175 ceph-mon[67352]: pgmap v120: 97 pgs: 97 active+clean; 46 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 4.7 MiB/s wr, 448 op/s 2024-03-18T23:12:09.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:12:09 smithi139 ceph-mon[67282]: pgmap v120: 97 pgs: 97 active+clean; 46 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 4.7 MiB/s wr, 448 op/s 2024-03-18T23:12:09.630 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:12:09 smithi049 ceph-mon[62991]: pgmap v120: 97 pgs: 97 active+clean; 46 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 4.7 MiB/s wr, 448 op/s 2024-03-18T23:12:09.767 INFO:tasks.workunit.client.0.smithi049.stdout: 1 427058 11.20 MB/sec execute 71 sec latency 3444.431 ms 2024-03-18T23:12:10.767 INFO:tasks.workunit.client.0.smithi049.stdout: 1 427058 11.04 MB/sec execute 72 sec latency 4444.565 ms 2024-03-18T23:12:11.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:12:11 smithi175 ceph-mon[67352]: pgmap v121: 97 pgs: 97 active+clean; 46 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.1 MiB/s wr, 339 op/s 2024-03-18T23:12:11.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:12:11 smithi139 ceph-mon[67282]: pgmap v121: 97 pgs: 97 active+clean; 46 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.1 MiB/s wr, 339 op/s 2024-03-18T23:12:11.630 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:12:11 smithi049 ceph-mon[62991]: pgmap v121: 97 pgs: 97 active+clean; 46 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.1 MiB/s wr, 339 op/s 2024-03-18T23:12:11.767 INFO:tasks.workunit.client.0.smithi049.stdout: 1 427058 10.89 MB/sec execute 73 sec latency 5444.682 ms 2024-03-18T23:12:12.768 INFO:tasks.workunit.client.0.smithi049.stdout: 1 427058 10.74 MB/sec execute 74 sec latency 6444.788 ms 2024-03-18T23:12:13.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:12:13 smithi175 ceph-mon[67352]: pgmap v122: 97 pgs: 97 active+clean; 46 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 4.0 MiB/s wr, 373 op/s 2024-03-18T23:12:13.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:12:13 smithi139 ceph-mon[67282]: pgmap v122: 97 pgs: 97 active+clean; 46 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 4.0 MiB/s wr, 373 op/s 2024-03-18T23:12:13.630 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:12:13 smithi049 ceph-mon[62991]: pgmap v122: 97 pgs: 97 active+clean; 46 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 4.0 MiB/s wr, 373 op/s 2024-03-18T23:12:13.767 INFO:tasks.workunit.client.0.smithi049.stdout: 1 427058 10.60 MB/sec execute 75 sec latency 7444.858 ms 2024-03-18T23:12:14.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:12:14 smithi139 ceph-mon[67282]: pgmap v123: 97 pgs: 97 active+clean; 46 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.5 MiB/s wr, 241 op/s 2024-03-18T23:12:14.630 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:12:14 smithi049 ceph-mon[62991]: pgmap v123: 97 pgs: 97 active+clean; 46 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.5 MiB/s wr, 241 op/s 2024-03-18T23:12:14.711 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:12:14 smithi175 ceph-mon[67352]: pgmap v123: 97 pgs: 97 active+clean; 46 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.5 MiB/s wr, 241 op/s 2024-03-18T23:12:14.767 INFO:tasks.workunit.client.0.smithi049.stdout: 1 427058 10.46 MB/sec execute 76 sec latency 8444.975 ms 2024-03-18T23:12:15.768 INFO:tasks.workunit.client.0.smithi049.stdout: 1 427058 10.32 MB/sec execute 77 sec latency 9445.050 ms 2024-03-18T23:12:16.711 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:12:16 smithi175 ceph-mon[67352]: pgmap v124: 97 pgs: 97 active+clean; 46 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.5 MiB/s wr, 241 op/s 2024-03-18T23:12:16.767 INFO:tasks.workunit.client.0.smithi049.stdout: 1 428270 10.23 MB/sec execute 78 sec latency 9786.476 ms 2024-03-18T23:12:16.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:12:16 smithi139 ceph-mon[67282]: pgmap v124: 97 pgs: 97 active+clean; 46 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.5 MiB/s wr, 241 op/s 2024-03-18T23:12:16.822 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:16.820+0000 7f22d7bf3640 1 -- 172.21.15.49:0/4124014083 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f22d0069840 msgr2=0x7f22d010e6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:12:16.823 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:16.820+0000 7f22d7bf3640 1 --2- 172.21.15.49:0/4124014083 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f22d0069840 0x7f22d010e6b0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f22c8009930 tx=0x7f22c802bc30 comp rx=0 tx=0).stop 2024-03-18T23:12:16.823 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:16.820+0000 7f22d7bf3640 1 -- 172.21.15.49:0/4124014083 shutdown_connections 2024-03-18T23:12:16.823 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:16.820+0000 7f22d7bf3640 1 --2- 172.21.15.49:0/4124014083 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f22d0069840 0x7f22d010e6b0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:16.823 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:16.821+0000 7f22d7bf3640 1 --2- 172.21.15.49:0/4124014083 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f22d0068ea0 0x7f22d0069300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:16.823 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:16.821+0000 7f22d7bf3640 1 --2- 172.21.15.49:0/4124014083 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f22d01078f0 0x7f22d0107cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:16.823 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:16.821+0000 7f22d7bf3640 1 -- 172.21.15.49:0/4124014083 >> 172.21.15.49:0/4124014083 conn(0x7f22d0077570 msgr2=0x7f22d0077980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:12:16.823 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:16.821+0000 7f22d7bf3640 1 -- 172.21.15.49:0/4124014083 shutdown_connections 2024-03-18T23:12:16.823 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:16.821+0000 7f22d7bf3640 1 -- 172.21.15.49:0/4124014083 wait complete. 2024-03-18T23:12:16.823 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:16.821+0000 7f22d7bf3640 1 Processor -- start 2024-03-18T23:12:16.823 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:16.821+0000 7f22d7bf3640 1 -- start start 2024-03-18T23:12:16.823 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:16.821+0000 7f22d7bf3640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f22d0068ea0 0x7f22d0100a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:12:16.823 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:16.821+0000 7f22d7bf3640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f22d0069840 0x7f22d0100f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:12:16.824 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:16.822+0000 7f22d7bf3640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f22d01078f0 0x7f22d01014c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:12:16.824 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:16.822+0000 7f22d7bf3640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f22d0111be0 con 0x7f22d0069840 2024-03-18T23:12:16.824 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:16.822+0000 7f22d7bf3640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f22d0111a60 con 0x7f22d0068ea0 2024-03-18T23:12:16.824 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:16.822+0000 7f22d7bf3640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f22d0111d60 con 0x7f22d01078f0 2024-03-18T23:12:16.824 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:16.822+0000 7f22d518e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f22d0069840 0x7f22d0100f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:12:16.824 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:16.822+0000 7f22d518e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f22d0069840 0x7f22d0100f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:56474/0 (socket says 172.21.15.49:56474) 2024-03-18T23:12:16.824 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:16.822+0000 7f22d598f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f22d0068ea0 0x7f22d0100a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:12:16.824 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:16.822+0000 7f22d518e640 1 -- 172.21.15.49:0/3029796327 learned_addr learned my addr 172.21.15.49:0/3029796327 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:12:16.824 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:16.822+0000 7f22d6190640 1 --2- 172.21.15.49:0/3029796327 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f22d01078f0 0x7f22d01014c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:12:16.824 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:16.822+0000 7f22d518e640 1 -- 172.21.15.49:0/3029796327 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f22d01078f0 msgr2=0x7f22d01014c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:12:16.824 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:16.822+0000 7f22d518e640 1 --2- 172.21.15.49:0/3029796327 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f22d01078f0 0x7f22d01014c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:16.824 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:16.822+0000 7f22d518e640 1 -- 172.21.15.49:0/3029796327 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f22d0068ea0 msgr2=0x7f22d0100a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:12:16.825 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:16.822+0000 7f22d518e640 1 --2- 172.21.15.49:0/3029796327 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f22d0068ea0 0x7f22d0100a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:16.825 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:16.822+0000 7f22d518e640 1 -- 172.21.15.49:0/3029796327 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22d01a6480 con 0x7f22d0069840 2024-03-18T23:12:16.825 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:16.822+0000 7f22d6190640 1 --2- 172.21.15.49:0/3029796327 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f22d01078f0 0x7f22d01014c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:12:16.825 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:16.822+0000 7f22d598f640 1 --2- 172.21.15.49:0/3029796327 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f22d0068ea0 0x7f22d0100a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:12:16.825 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:16.822+0000 7f22d518e640 1 --2- 172.21.15.49:0/3029796327 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f22d0069840 0x7f22d0100f80 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f22c400e9f0 tx=0x7f22c400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:12:16.825 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:16.822+0000 7f22ceffd640 1 -- 172.21.15.49:0/3029796327 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22c400cd80 con 0x7f22d0069840 2024-03-18T23:12:16.825 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:16.822+0000 7f22d7bf3640 1 -- 172.21.15.49:0/3029796327 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22d01a6770 con 0x7f22d0069840 2024-03-18T23:12:16.825 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:16.822+0000 7f22ceffd640 1 -- 172.21.15.49:0/3029796327 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f22c4005500 con 0x7f22d0069840 2024-03-18T23:12:16.825 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:16.822+0000 7f22ceffd640 1 -- 172.21.15.49:0/3029796327 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22c4010690 con 0x7f22d0069840 2024-03-18T23:12:16.825 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:16.823+0000 7f22d7bf3640 1 -- 172.21.15.49:0/3029796327 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22d01a6c70 con 0x7f22d0069840 2024-03-18T23:12:16.831 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:16.825+0000 7f22ceffd640 1 -- 172.21.15.49:0/3029796327 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 25) ==== 137421+0+0 (secure 0 0 0) 0x7f22c40046c0 con 0x7f22d0069840 2024-03-18T23:12:16.831 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:16.825+0000 7f22da7d6740 1 Processor -- start 2024-03-18T23:12:16.831 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:16.826+0000 7f22da7d6740 1 -- start start 2024-03-18T23:12:16.831 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:16.826+0000 7f22ceffd640 1 --2- 172.21.15.49:0/3029796327 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f22b80a2a10 0x7f22b80a4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:12:16.831 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:16.826+0000 7f22da7d6740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5619fca08ef0 0x5619fca092d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:12:16.831 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:16.826+0000 7f22da7d6740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5619fca09810 0x5619fca120a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:12:16.831 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:16.826+0000 7f22da7d6740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5619fca125e0 0x5619fca149d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:12:16.831 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:16.826+0000 7f22da7d6740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5619fc96cbe0 con 0x5619fca09810 2024-03-18T23:12:16.831 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:16.826+0000 7f22da7d6740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x5619fc915960 con 0x5619fca125e0 2024-03-18T23:12:16.832 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:16.826+0000 7f22da7d6740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5619fc968c80 con 0x5619fca08ef0 2024-03-18T23:12:16.832 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:16.826+0000 7f22ceffd640 1 -- 172.21.15.49:0/3029796327 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8469+0+0 (secure 0 0 0) 0x7f22c40cf230 con 0x7f22d0069840 2024-03-18T23:12:16.832 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:16.826+0000 7f22d6190640 1 --2- 172.21.15.49:0/3029796327 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f22b80a2a10 0x7f22b80a4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:12:16.832 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:16.826+0000 7f22d598f640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5619fca08ef0 0x5619fca092d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:12:16.832 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:16.826+0000 7f22d598f640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5619fca08ef0 0x5619fca092d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.49:47306/0 (socket says 172.21.15.49:47306) 2024-03-18T23:12:16.832 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:16.826+0000 7f22d598f640 1 -- 172.21.15.49:0/941199085 learned_addr learned my addr 172.21.15.49:0/941199085 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:12:16.832 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:16.826+0000 7f22d518e640 1 --2- 172.21.15.49:0/941199085 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5619fca09810 0x5619fca120a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:12:16.832 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:16.826+0000 7f22d6190640 1 --2- 172.21.15.49:0/941199085 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5619fca125e0 0x5619fca149d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:12:16.832 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:16.826+0000 7f22d598f640 1 -- 172.21.15.49:0/941199085 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5619fca125e0 msgr2=0x5619fca149d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:12:16.832 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:16.826+0000 7f22d598f640 1 --2- 172.21.15.49:0/941199085 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5619fca125e0 0x5619fca149d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:16.832 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:16.826+0000 7f22d598f640 1 -- 172.21.15.49:0/941199085 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5619fca09810 msgr2=0x5619fca120a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:12:16.832 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:16.826+0000 7f22d598f640 1 --2- 172.21.15.49:0/941199085 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5619fca09810 0x5619fca120a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:16.832 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:16.826+0000 7f22d598f640 1 -- 172.21.15.49:0/941199085 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5619fc90cd20 con 0x5619fca08ef0 2024-03-18T23:12:16.832 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:16.827+0000 7f22d6190640 1 --2- 172.21.15.49:0/3029796327 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f22b80a2a10 0x7f22b80a4ed0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f22c802efd0 tx=0x7f22c8005d50 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:12:16.833 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:16.827+0000 7f22d598f640 1 --2- 172.21.15.49:0/941199085 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5619fca08ef0 0x5619fca092d0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f22c0005f40 tx=0x7f22c0011bf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d21415df81031915 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:12:16.833 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:16.827+0000 7f22b7fff640 1 -- 172.21.15.49:0/941199085 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22c0011220 con 0x5619fca08ef0 2024-03-18T23:12:16.833 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:16.827+0000 7f22b7fff640 1 -- 172.21.15.49:0/941199085 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f22c000f9a0 con 0x5619fca08ef0 2024-03-18T23:12:16.833 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:16.827+0000 7f22b7fff640 1 -- 172.21.15.49:0/941199085 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22c00045b0 con 0x5619fca08ef0 2024-03-18T23:12:16.833 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:16.827+0000 7f22da7d6740 1 -- 172.21.15.49:0/941199085 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5619fca08ef0 msgr2=0x5619fca092d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:12:16.833 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:16.827+0000 7f22da7d6740 1 --2- 172.21.15.49:0/941199085 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5619fca08ef0 0x5619fca092d0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f22c0005f40 tx=0x7f22c0011bf0 comp rx=0 tx=0).stop 2024-03-18T23:12:16.833 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:16.828+0000 7f22da7d6740 1 -- 172.21.15.49:0/941199085 shutdown_connections 2024-03-18T23:12:16.833 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:16.828+0000 7f22da7d6740 1 --2- 172.21.15.49:0/941199085 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5619fca125e0 0x5619fca149d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:16.833 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:16.828+0000 7f22da7d6740 1 --2- 172.21.15.49:0/941199085 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5619fca09810 0x5619fca120a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:16.833 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:16.828+0000 7f22da7d6740 1 --2- 172.21.15.49:0/941199085 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5619fca08ef0 0x5619fca092d0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:16.833 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:16.828+0000 7f22da7d6740 1 -- 172.21.15.49:0/941199085 >> 172.21.15.49:0/941199085 conn(0x5619fca03810 msgr2=0x5619fca03bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:12:16.833 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:16.828+0000 7f22da7d6740 1 -- 172.21.15.49:0/941199085 shutdown_connections 2024-03-18T23:12:16.833 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:16.828+0000 7f22da7d6740 1 -- 172.21.15.49:0/941199085 wait complete. 2024-03-18T23:12:16.833 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:16.828+0000 7f22da7d6740 1 -- start start 2024-03-18T23:12:16.834 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:16.828+0000 7f22da7d6740 1 Processor -- start 2024-03-18T23:12:16.834 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:16.829+0000 7f22da7d6740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5619fca08ef0 0x5619fca11f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:12:16.834 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:16.829+0000 7f22da7d6740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5619fca09810 0x5619fca0f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:12:16.834 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:16.829+0000 7f22da7d6740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5619fca125e0 0x5619fca0f840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:12:16.834 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:16.829+0000 7f22da7d6740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5619fca15210 con 0x5619fca125e0 2024-03-18T23:12:16.834 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:16.829+0000 7f22da7d6740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x5619fc928db0 con 0x5619fca09810 2024-03-18T23:12:16.834 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:16.829+0000 7f22da7d6740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5619fca15390 con 0x5619fca08ef0 2024-03-18T23:12:16.834 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:16.829+0000 7f22da7d6740 10 client.0 _ll_register_callbacks cb 0x5619fc92de70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T23:12:16.834 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:16.829+0000 7f22d6190640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5619fca125e0 0x5619fca0f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:12:16.834 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:16.829+0000 7f22d598f640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5619fca08ef0 0x5619fca11f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:12:16.834 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:16.829+0000 7f22d518e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5619fca09810 0x5619fca0f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:12:16.834 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:16.829+0000 7f22d6190640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5619fca125e0 0x5619fca0f840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:56496/0 (socket says 172.21.15.49:56496) 2024-03-18T23:12:16.834 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:16.829+0000 7f22d6190640 1 -- 172.21.15.49:0/2960956637 learned_addr learned my addr 172.21.15.49:0/2960956637 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:12:16.835 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:16.829+0000 7f22d6190640 1 -- 172.21.15.49:0/2960956637 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5619fca08ef0 msgr2=0x5619fca11f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:12:16.835 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:16.829+0000 7f22d6190640 1 --2- 172.21.15.49:0/2960956637 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5619fca08ef0 0x5619fca11f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:16.835 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:16.829+0000 7f22d6190640 1 -- 172.21.15.49:0/2960956637 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5619fca09810 msgr2=0x5619fca0f2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:12:16.835 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:16.829+0000 7f22d6190640 1 --2- 172.21.15.49:0/2960956637 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5619fca09810 0x5619fca0f2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:16.835 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:16.829+0000 7f22d6190640 1 -- 172.21.15.49:0/2960956637 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5619fc927c60 con 0x5619fca125e0 2024-03-18T23:12:16.835 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:16.829+0000 7f22d6190640 1 --2- 172.21.15.49:0/2960956637 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5619fca125e0 0x5619fca0f840 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f22c8009930 tx=0x7f22c802bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c4bcb96571831dd server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:12:16.835 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:16.829+0000 7f22b57fa640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-18T23:12:16.835 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:16.829+0000 7f22b57fa640 1 -- 172.21.15.49:0/2960956637 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22c8052060 con 0x5619fca125e0 2024-03-18T23:12:16.835 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:16.830+0000 7f22b57fa640 1 -- 172.21.15.49:0/2960956637 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8469+0+0 (secure 0 0 0) 0x7f22c8005300 con 0x5619fca125e0 2024-03-18T23:12:16.835 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:16.830+0000 7f22da7d6740 1 -- 172.21.15.49:0/2960956637 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f22c000f9a0 con 0x5619fca125e0 2024-03-18T23:12:16.835 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:16.830+0000 7f22b57fa640 1 -- 172.21.15.49:0/2960956637 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f22c8005500 con 0x5619fca125e0 2024-03-18T23:12:16.835 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:16.830+0000 7f22da7d6740 10 client.15780 fetch_fsmap learned FSMap version 32 2024-03-18T23:12:16.835 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:16.830+0000 7f22da7d6740 1 -- 172.21.15.49:0/2960956637 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5619fc681210 con 0x5619fca125e0 2024-03-18T23:12:16.835 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:16.830+0000 7f22b57fa640 1 -- 172.21.15.49:0/2960956637 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f22c000f9a0 con 0x5619fca125e0 2024-03-18T23:12:16.835 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:16.830+0000 7f22b57fa640 1 -- 172.21.15.49:0/2960956637 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f22c804d600 con 0x5619fca125e0 2024-03-18T23:12:16.836 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:16.830+0000 7f22da7d6740 10 client.15780 fetch_fsmap finished waiting for FSMap version 32 2024-03-18T23:12:16.836 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:16.830+0000 7f22da7d6740 10 client.15780 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T23:12:16.836 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:16.830+0000 7f22da7d6740 20 client.15780 populate_metadata read hostname 'smithi049' 2024-03-18T23:12:16.836 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:16.830+0000 7f22da7d6740 1 --2- 172.21.15.49:0/2960956637 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x5619fca223e0 0x5619fca247d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:12:16.836 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:16.830+0000 7f22da7d6740 4 client.15780 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T23:12:16.836 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:16.830+0000 7f22da7d6740 1 -- 172.21.15.49:0/2960956637 --> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5619fc97a120 con 0x5619fca223e0 2024-03-18T23:12:16.836 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:16.830+0000 7f22d598f640 1 --2- 172.21.15.49:0/2960956637 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x5619fca223e0 0x5619fca247d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:12:16.836 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:16.831+0000 7f22d598f640 1 --2- 172.21.15.49:0/2960956637 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x5619fca223e0 0x5619fca247d0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a9b0544cc81e3edc server_cookie=314836b532df252b in_seq=0 out_seq=0 2024-03-18T23:12:16.836 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:16.831+0000 7f22b57fa640 10 client.15780 ms_handle_connect on v2:172.21.15.139:6838/4280128209 2024-03-18T23:12:16.837 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:16.832+0000 7f22b57fa640 1 -- 172.21.15.49:0/2960956637 <== mds.0 v2:172.21.15.139:6838/4280128209 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5619fc97a120 con 0x5619fca223e0 2024-03-18T23:12:16.837 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:16.832+0000 7f22b57fa640 10 client.15780 handle_command_reply: tid=0 2024-03-18T23:12:16.837 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:16.832+0000 7f22da7d6740 1 client.15780 shutdown 2024-03-18T23:12:16.837 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:16.832+0000 7f22da7d6740 1 -- 172.21.15.49:0/2960956637 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5619fca125e0 msgr2=0x5619fca0f840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:12:16.837 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:16.832+0000 7f22da7d6740 1 --2- 172.21.15.49:0/2960956637 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5619fca125e0 0x5619fca0f840 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f22c8009930 tx=0x7f22c802bc30 comp rx=0 tx=0).stop 2024-03-18T23:12:16.837 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:16.832+0000 7f22da7d6740 1 -- 172.21.15.49:0/2960956637 shutdown_connections 2024-03-18T23:12:16.838 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:16.832+0000 7f22da7d6740 1 --2- 172.21.15.49:0/2960956637 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x5619fca223e0 0x5619fca247d0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:16.838 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:16.832+0000 7f22da7d6740 1 --2- 172.21.15.49:0/2960956637 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5619fca125e0 0x5619fca0f840 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:16.838 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:16.832+0000 7f22da7d6740 1 --2- 172.21.15.49:0/2960956637 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5619fca09810 0x5619fca0f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:16.838 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:16.832+0000 7f22b57fa640 0 client.15780 ms_handle_reset on v2:172.21.15.139:6838/4280128209 2024-03-18T23:12:16.838 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:16.832+0000 7f22da7d6740 1 --2- 172.21.15.49:0/2960956637 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5619fca08ef0 0x5619fca11f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:16.838 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:16.832+0000 7f22da7d6740 1 -- 172.21.15.49:0/2960956637 >> 172.21.15.49:0/2960956637 conn(0x5619fca03810 msgr2=0x5619fca02ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:12:16.838 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:16.832+0000 7f22da7d6740 1 -- 172.21.15.49:0/2960956637 shutdown_connections 2024-03-18T23:12:16.838 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:16.832+0000 7f22da7d6740 1 -- 172.21.15.49:0/2960956637 wait complete. 2024-03-18T23:12:16.838 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:16.832+0000 7f22da7d6740 20 client.15780 trim_cache size 0 max 16384 2024-03-18T23:12:16.849 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:16.848+0000 7f22da7d6740 1 Processor -- start 2024-03-18T23:12:16.849 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:16.848+0000 7f22da7d6740 1 -- start start 2024-03-18T23:12:16.849 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:16.848+0000 7f22da7d6740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5619fca08ef0 0x5619fca1dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:12:16.850 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:16.848+0000 7f22da7d6740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5619fca09810 0x5619fca2a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:12:16.850 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:16.848+0000 7f22da7d6740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5619fca125e0 0x5619fca11a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:12:16.850 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:16.848+0000 7f22da7d6740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f22d01af640 con 0x5619fca08ef0 2024-03-18T23:12:16.850 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:16.848+0000 7f22da7d6740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f22d01b1180 con 0x5619fca09810 2024-03-18T23:12:16.850 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:16.848+0000 7f22da7d6740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f22d01afd10 con 0x5619fca125e0 2024-03-18T23:12:16.850 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:16.848+0000 7f22d598f640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5619fca08ef0 0x5619fca1dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:12:16.850 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:16.848+0000 7f22d598f640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5619fca08ef0 0x5619fca1dfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:56508/0 (socket says 172.21.15.49:56508) 2024-03-18T23:12:16.850 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:16.848+0000 7f22d598f640 1 -- 172.21.15.49:0/2517441638 learned_addr learned my addr 172.21.15.49:0/2517441638 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:12:16.850 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:16.848+0000 7f22d518e640 1 --2- 172.21.15.49:0/2517441638 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5619fca09810 0x5619fca2a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:12:16.850 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:16.848+0000 7f22d6190640 1 --2- 172.21.15.49:0/2517441638 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5619fca125e0 0x5619fca11a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:12:16.850 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:16.849+0000 7f22d598f640 1 -- 172.21.15.49:0/2517441638 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5619fca125e0 msgr2=0x5619fca11a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:12:16.850 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:16.849+0000 7f22d598f640 1 --2- 172.21.15.49:0/2517441638 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5619fca125e0 0x5619fca11a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:16.850 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:16.849+0000 7f22d598f640 1 -- 172.21.15.49:0/2517441638 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5619fca09810 msgr2=0x5619fca2a0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:12:16.850 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:16.849+0000 7f22d598f640 1 --2- 172.21.15.49:0/2517441638 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5619fca09810 0x5619fca2a0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:16.851 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:16.849+0000 7f22d598f640 1 -- 172.21.15.49:0/2517441638 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5619fca0fbb0 con 0x5619fca08ef0 2024-03-18T23:12:16.851 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:16.849+0000 7f22d518e640 1 --2- 172.21.15.49:0/2517441638 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5619fca09810 0x5619fca2a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:12:16.851 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:16.849+0000 7f22d598f640 1 --2- 172.21.15.49:0/2517441638 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5619fca08ef0 0x5619fca1dfb0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f22c000fc70 tx=0x7f22c0004190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6699149ffdcf87bc server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:12:16.851 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:16.849+0000 7f22b77fe640 1 -- 172.21.15.49:0/2517441638 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22c001e040 con 0x5619fca08ef0 2024-03-18T23:12:16.851 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:16.849+0000 7f22b77fe640 1 -- 172.21.15.49:0/2517441638 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f22c0003970 con 0x5619fca08ef0 2024-03-18T23:12:16.851 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:16.849+0000 7f22b77fe640 1 -- 172.21.15.49:0/2517441638 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22c00054e0 con 0x5619fca08ef0 2024-03-18T23:12:16.854 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:16.850+0000 7f22da7d6740 1 -- 172.21.15.49:0/2517441638 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5619fca08ef0 msgr2=0x5619fca1dfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:12:16.855 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:16.850+0000 7f22da7d6740 1 --2- 172.21.15.49:0/2517441638 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5619fca08ef0 0x5619fca1dfb0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f22c000fc70 tx=0x7f22c0004190 comp rx=0 tx=0).stop 2024-03-18T23:12:16.855 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:16.850+0000 7f22da7d6740 1 -- 172.21.15.49:0/2517441638 shutdown_connections 2024-03-18T23:12:16.855 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:16.850+0000 7f22da7d6740 1 --2- 172.21.15.49:0/2517441638 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5619fca125e0 0x5619fca11a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:16.855 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:16.850+0000 7f22da7d6740 1 --2- 172.21.15.49:0/2517441638 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5619fca09810 0x5619fca2a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:16.855 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:16.850+0000 7f22da7d6740 1 --2- 172.21.15.49:0/2517441638 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5619fca08ef0 0x5619fca1dfb0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:16.855 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:16.850+0000 7f22da7d6740 1 -- 172.21.15.49:0/2517441638 >> 172.21.15.49:0/2517441638 conn(0x5619fca03810 msgr2=0x5619fca02ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:12:16.855 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:16.850+0000 7f22da7d6740 1 -- 172.21.15.49:0/2517441638 shutdown_connections 2024-03-18T23:12:16.855 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:16.850+0000 7f22da7d6740 1 -- 172.21.15.49:0/2517441638 wait complete. 2024-03-18T23:12:16.855 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:16.850+0000 7f22da7d6740 1 -- start start 2024-03-18T23:12:16.855 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:16.850+0000 7f22da7d6740 1 Processor -- start 2024-03-18T23:12:16.855 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:16.851+0000 7f22da7d6740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5619fca08ef0 0x5619fca1a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:12:16.855 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:16.851+0000 7f22da7d6740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5619fca09810 0x5619fca0bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:12:16.855 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:16.851+0000 7f22da7d6740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5619fca125e0 0x5619fca0c430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:12:16.855 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:16.851+0000 7f22da7d6740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f22d01b0ab0 con 0x5619fca08ef0 2024-03-18T23:12:16.855 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:16.851+0000 7f22da7d6740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f22d01b03e0 con 0x5619fca125e0 2024-03-18T23:12:16.856 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:16.851+0000 7f22da7d6740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f22d01ad620 con 0x5619fca09810 2024-03-18T23:12:16.856 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:16.851+0000 7f22da7d6740 10 client.0 _ll_register_callbacks cb 0x7f22c00093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T23:12:16.856 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:16.851+0000 7f22d598f640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5619fca08ef0 0x5619fca1a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:12:16.856 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:16.851+0000 7f22d598f640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5619fca08ef0 0x5619fca1a9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:56520/0 (socket says 172.21.15.49:56520) 2024-03-18T23:12:16.856 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:16.851+0000 7f22d598f640 1 -- 172.21.15.49:0/3600307145 learned_addr learned my addr 172.21.15.49:0/3600307145 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:12:16.856 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:16.851+0000 7f22d518e640 1 --2- 172.21.15.49:0/3600307145 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5619fca09810 0x5619fca0bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:12:16.856 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:16.851+0000 7f22d6190640 1 --2- 172.21.15.49:0/3600307145 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5619fca125e0 0x5619fca0c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:12:16.856 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:16.851+0000 7f22d598f640 1 -- 172.21.15.49:0/3600307145 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5619fca09810 msgr2=0x5619fca0bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:12:16.856 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:16.851+0000 7f22d598f640 1 --2- 172.21.15.49:0/3600307145 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5619fca09810 0x5619fca0bcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:16.856 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:16.851+0000 7f22d598f640 1 -- 172.21.15.49:0/3600307145 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5619fca125e0 msgr2=0x5619fca0c430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:12:16.856 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:16.851+0000 7f22d598f640 1 --2- 172.21.15.49:0/3600307145 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5619fca125e0 0x5619fca0c430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:16.856 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:16.851+0000 7f22d598f640 1 -- 172.21.15.49:0/3600307145 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5619fc74d620 con 0x5619fca08ef0 2024-03-18T23:12:16.856 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:16.851+0000 7f22d6190640 1 --2- 172.21.15.49:0/3600307145 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5619fca125e0 0x5619fca0c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T23:12:16.856 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:16.851+0000 7f22d598f640 1 --2- 172.21.15.49:0/3600307145 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5619fca08ef0 0x5619fca1a9d0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f22c0006010 tx=0x7f22c000fe10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1dd879e2672c77d server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:12:16.856 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:16.851+0000 7f22b57fa640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-18T23:12:16.856 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:16.851+0000 7f22b57fa640 1 -- 172.21.15.49:0/3600307145 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22c0037020 con 0x5619fca08ef0 2024-03-18T23:12:16.857 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:16.851+0000 7f22b57fa640 1 -- 172.21.15.49:0/3600307145 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8469+0+0 (secure 0 0 0) 0x7f22c000f850 con 0x5619fca08ef0 2024-03-18T23:12:16.857 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:16.851+0000 7f22da7d6740 1 -- 172.21.15.49:0/3600307145 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f22c0003970 con 0x5619fca08ef0 2024-03-18T23:12:16.857 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:16.852+0000 7f22b57fa640 1 -- 172.21.15.49:0/3600307145 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f22c0003b20 con 0x5619fca08ef0 2024-03-18T23:12:16.857 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:16.852+0000 7f22da7d6740 10 client.15786 fetch_fsmap learned FSMap version 32 2024-03-18T23:12:16.857 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:16.852+0000 7f22da7d6740 1 -- 172.21.15.49:0/3600307145 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5619fca886d0 con 0x5619fca08ef0 2024-03-18T23:12:16.857 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:16.852+0000 7f22b57fa640 1 -- 172.21.15.49:0/3600307145 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f22c0003970 con 0x5619fca08ef0 2024-03-18T23:12:16.857 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:16.852+0000 7f22b57fa640 1 -- 172.21.15.49:0/3600307145 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f22c0018070 con 0x5619fca08ef0 2024-03-18T23:12:16.857 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:16.852+0000 7f22da7d6740 10 client.15786 fetch_fsmap finished waiting for FSMap version 32 2024-03-18T23:12:16.857 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:16.852+0000 7f22da7d6740 10 client.15786 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T23:12:16.857 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:16.852+0000 7f22da7d6740 20 client.15786 populate_metadata read hostname 'smithi049' 2024-03-18T23:12:16.857 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:16.852+0000 7f22da7d6740 1 --2- 172.21.15.49:0/3600307145 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x5619fca88b90 0x5619fca8af80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:12:16.857 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:16.852+0000 7f22da7d6740 4 client.15786 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T23:12:16.857 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:16.852+0000 7f22da7d6740 1 -- 172.21.15.49:0/3600307145 --> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5619fc974470 con 0x5619fca88b90 2024-03-18T23:12:16.857 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:16.852+0000 7f22d518e640 1 --2- 172.21.15.49:0/3600307145 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x5619fca88b90 0x5619fca8af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:12:16.857 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:16.853+0000 7f22d518e640 1 --2- 172.21.15.49:0/3600307145 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x5619fca88b90 0x5619fca8af80 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e3914cd52257c25 server_cookie=8c4f89154225ba89 in_seq=0 out_seq=0 2024-03-18T23:12:16.857 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:16.853+0000 7f22b57fa640 10 client.15786 ms_handle_connect on v2:172.21.15.139:6838/4280128209 2024-03-18T23:12:16.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:12:16 smithi049 ceph-mon[62991]: pgmap v124: 97 pgs: 97 active+clean; 46 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.5 MiB/s wr, 241 op/s 2024-03-18T23:12:16.957 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:16.957+0000 7f22b57fa640 1 -- 172.21.15.49:0/3600307145 <== mds.0 v2:172.21.15.139:6838/4280128209 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5619fc974470 con 0x5619fca88b90 2024-03-18T23:12:16.957 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:16.957+0000 7f22b57fa640 10 client.15786 handle_command_reply: tid=0 2024-03-18T23:12:16.957 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:16.957+0000 7f22da7d6740 1 client.15786 shutdown 2024-03-18T23:12:16.958 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:16.957+0000 7f22da7d6740 1 -- 172.21.15.49:0/3600307145 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5619fca08ef0 msgr2=0x5619fca1a9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:12:16.958 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:16.957+0000 7f22da7d6740 1 --2- 172.21.15.49:0/3600307145 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5619fca08ef0 0x5619fca1a9d0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f22c0006010 tx=0x7f22c000fe10 comp rx=0 tx=0).stop 2024-03-18T23:12:16.958 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:16.957+0000 7f22da7d6740 1 -- 172.21.15.49:0/3600307145 shutdown_connections 2024-03-18T23:12:16.958 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:16.957+0000 7f22da7d6740 1 --2- 172.21.15.49:0/3600307145 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x5619fca88b90 0x5619fca8af80 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:16.958 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:16.957+0000 7f22da7d6740 1 --2- 172.21.15.49:0/3600307145 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5619fca125e0 0x5619fca0c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:16.958 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:16.957+0000 7f22da7d6740 1 --2- 172.21.15.49:0/3600307145 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5619fca09810 0x5619fca0bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:16.958 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:16.957+0000 7f22da7d6740 1 --2- 172.21.15.49:0/3600307145 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5619fca08ef0 0x5619fca1a9d0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:16.958 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:16.957+0000 7f22b57fa640 0 client.15786 ms_handle_reset on v2:172.21.15.139:6838/4280128209 2024-03-18T23:12:16.958 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:16.957+0000 7f22da7d6740 1 -- 172.21.15.49:0/3600307145 >> 172.21.15.49:0/3600307145 conn(0x5619fca03810 msgr2=0x5619fca09d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:12:16.958 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:16.957+0000 7f22da7d6740 1 -- 172.21.15.49:0/3600307145 shutdown_connections 2024-03-18T23:12:16.958 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:16.957+0000 7f22da7d6740 1 -- 172.21.15.49:0/3600307145 wait complete. 2024-03-18T23:12:16.958 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:16.957+0000 7f22da7d6740 20 client.15786 trim_cache size 0 max 16384 2024-03-18T23:12:16.959 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:{ 2024-03-18T23:12:16.959 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "message": "", 2024-03-18T23:12:16.959 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "return_code": 0 2024-03-18T23:12:16.959 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:} 2024-03-18T23:12:16.960 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:16.958+0000 7f22d7bf3640 1 -- 172.21.15.49:0/3029796327 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f22b80a2a10 msgr2=0x7f22b80a4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:12:16.960 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:16.958+0000 7f22d7bf3640 1 --2- 172.21.15.49:0/3029796327 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f22b80a2a10 0x7f22b80a4ed0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f22c802efd0 tx=0x7f22c8005d50 comp rx=0 tx=0).stop 2024-03-18T23:12:16.960 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:16.958+0000 7f22d7bf3640 1 -- 172.21.15.49:0/3029796327 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f22d0069840 msgr2=0x7f22d0100f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:12:16.960 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:16.958+0000 7f22d7bf3640 1 --2- 172.21.15.49:0/3029796327 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f22d0069840 0x7f22d0100f80 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f22c400e9f0 tx=0x7f22c400edf0 comp rx=0 tx=0).stop 2024-03-18T23:12:16.960 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:16.958+0000 7f22d7bf3640 1 -- 172.21.15.49:0/3029796327 shutdown_connections 2024-03-18T23:12:16.960 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:16.958+0000 7f22d7bf3640 1 --2- 172.21.15.49:0/3029796327 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f22b80a2a10 0x7f22b80a4ed0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:16.960 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:16.958+0000 7f22d7bf3640 1 --2- 172.21.15.49:0/3029796327 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f22d01078f0 0x7f22d01014c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:16.960 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:16.958+0000 7f22d7bf3640 1 --2- 172.21.15.49:0/3029796327 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f22d0069840 0x7f22d0100f80 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:16.960 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:16.958+0000 7f22d7bf3640 1 --2- 172.21.15.49:0/3029796327 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f22d0068ea0 0x7f22d0100a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:16.960 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:16.958+0000 7f22d7bf3640 1 -- 172.21.15.49:0/3029796327 >> 172.21.15.49:0/3029796327 conn(0x7f22d0077570 msgr2=0x7f22d00fe980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:12:16.961 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:16.958+0000 7f22d7bf3640 1 -- 172.21.15.49:0/3029796327 shutdown_connections 2024-03-18T23:12:16.961 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:16.958+0000 7f22d7bf3640 1 -- 172.21.15.49:0/3029796327 wait complete. 2024-03-18T23:12:17.768 INFO:tasks.workunit.client.0.smithi049.stdout: 1 429979 10.25 MB/sec execute 79 sec latency 51.844 ms 2024-03-18T23:12:18.711 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:12:18 smithi175 ceph-mon[67352]: pgmap v125: 97 pgs: 97 active+clean; 52 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.3 MiB/s wr, 274 op/s 2024-03-18T23:12:18.768 INFO:tasks.workunit.client.0.smithi049.stdout: 1 432194 10.27 MB/sec execute 80 sec latency 21.316 ms 2024-03-18T23:12:18.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:12:18 smithi139 ceph-mon[67282]: pgmap v125: 97 pgs: 97 active+clean; 52 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.3 MiB/s wr, 274 op/s 2024-03-18T23:12:18.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:12:18 smithi049 ceph-mon[62991]: pgmap v125: 97 pgs: 97 active+clean; 52 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.3 MiB/s wr, 274 op/s 2024-03-18T23:12:19.768 INFO:tasks.workunit.client.0.smithi049.stdout: 1 433550 10.23 MB/sec execute 81 sec latency 62.138 ms 2024-03-18T23:12:20.711 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:12:20 smithi175 ceph-mon[67352]: pgmap v126: 97 pgs: 97 active+clean; 52 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 364 KiB/s rd, 1.7 MiB/s wr, 67 op/s 2024-03-18T23:12:20.768 INFO:tasks.workunit.client.0.smithi049.stdout: 1 435665 10.24 MB/sec execute 82 sec latency 22.176 ms 2024-03-18T23:12:20.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:12:20 smithi139 ceph-mon[67282]: pgmap v126: 97 pgs: 97 active+clean; 52 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 364 KiB/s rd, 1.7 MiB/s wr, 67 op/s 2024-03-18T23:12:20.879 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:12:20 smithi049 ceph-mon[62991]: pgmap v126: 97 pgs: 97 active+clean; 52 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 364 KiB/s rd, 1.7 MiB/s wr, 67 op/s 2024-03-18T23:12:21.768 INFO:tasks.workunit.client.0.smithi049.stdout: 1 437477 10.23 MB/sec execute 83 sec latency 26.094 ms 2024-03-18T23:12:22.711 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:12:22 smithi175 ceph-mon[67352]: pgmap v127: 97 pgs: 97 active+clean; 47 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.3 MiB/s wr, 224 op/s 2024-03-18T23:12:22.768 INFO:tasks.workunit.client.0.smithi049.stdout: 1 439819 10.29 MB/sec execute 84 sec latency 25.552 ms 2024-03-18T23:12:22.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:12:22 smithi139 ceph-mon[67282]: pgmap v127: 97 pgs: 97 active+clean; 47 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.3 MiB/s wr, 224 op/s 2024-03-18T23:12:22.879 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:12:22 smithi049 ceph-mon[62991]: pgmap v127: 97 pgs: 97 active+clean; 47 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.3 MiB/s wr, 224 op/s 2024-03-18T23:12:23.768 INFO:tasks.workunit.client.0.smithi049.stdout: 1 441385 10.25 MB/sec execute 85 sec latency 26.360 ms 2024-03-18T23:12:24.711 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:12:24 smithi175 ceph-mon[67352]: pgmap v128: 97 pgs: 97 active+clean; 47 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.4 MiB/s wr, 190 op/s 2024-03-18T23:12:24.768 INFO:tasks.workunit.client.0.smithi049.stdout: 1 443401 10.28 MB/sec execute 86 sec latency 115.501 ms 2024-03-18T23:12:24.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:12:24 smithi139 ceph-mon[67282]: pgmap v128: 97 pgs: 97 active+clean; 47 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.4 MiB/s wr, 190 op/s 2024-03-18T23:12:24.879 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:12:24 smithi049 ceph-mon[62991]: pgmap v128: 97 pgs: 97 active+clean; 47 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.4 MiB/s wr, 190 op/s 2024-03-18T23:12:25.711 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:12:25 smithi175 ceph-mon[67352]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:12:25.711 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:12:25 smithi175 ceph-mon[67352]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:12:25.712 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:12:25 smithi175 ceph-mon[67352]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:12:25.712 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:12:25 smithi175 ceph-mon[67352]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' 2024-03-18T23:12:25.768 INFO:tasks.workunit.client.0.smithi049.stdout: 1 444857 10.24 MB/sec execute 87 sec latency 19.808 ms 2024-03-18T23:12:25.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:12:25 smithi139 ceph-mon[67282]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:12:25.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:12:25 smithi139 ceph-mon[67282]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:12:25.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:12:25 smithi139 ceph-mon[67282]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:12:25.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:12:25 smithi139 ceph-mon[67282]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' 2024-03-18T23:12:25.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:12:25 smithi049 ceph-mon[62991]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:12:25.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:12:25 smithi049 ceph-mon[62991]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:12:25.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:12:25 smithi049 ceph-mon[62991]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:12:25.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:12:25 smithi049 ceph-mon[62991]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' 2024-03-18T23:12:26.711 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:12:26 smithi175 ceph-mon[67352]: pgmap v129: 97 pgs: 97 active+clean; 47 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.4 MiB/s wr, 190 op/s 2024-03-18T23:12:26.768 INFO:tasks.workunit.client.0.smithi049.stdout: 1 447093 10.27 MB/sec execute 88 sec latency 27.160 ms 2024-03-18T23:12:26.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:12:26 smithi139 ceph-mon[67282]: pgmap v129: 97 pgs: 97 active+clean; 47 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.4 MiB/s wr, 190 op/s 2024-03-18T23:12:26.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:12:26 smithi049 ceph-mon[62991]: pgmap v129: 97 pgs: 97 active+clean; 47 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.4 MiB/s wr, 190 op/s 2024-03-18T23:12:27.768 INFO:tasks.workunit.client.0.smithi049.stdout: 1 448595 10.22 MB/sec execute 89 sec latency 27.916 ms 2024-03-18T23:12:28.711 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:12:28 smithi175 ceph-mon[67352]: pgmap v130: 97 pgs: 97 active+clean; 52 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.7 MiB/s wr, 282 op/s 2024-03-18T23:12:28.768 INFO:tasks.workunit.client.0.smithi049.stdout: 1 450813 10.26 MB/sec execute 90 sec latency 18.237 ms 2024-03-18T23:12:28.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:12:28 smithi139 ceph-mon[67282]: pgmap v130: 97 pgs: 97 active+clean; 52 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.7 MiB/s wr, 282 op/s 2024-03-18T23:12:28.879 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:12:28 smithi049 ceph-mon[62991]: pgmap v130: 97 pgs: 97 active+clean; 52 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.7 MiB/s wr, 282 op/s 2024-03-18T23:12:29.768 INFO:tasks.workunit.client.0.smithi049.stdout: 1 452163 10.21 MB/sec execute 91 sec latency 88.510 ms 2024-03-18T23:12:30.050 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:30.049+0000 7fee7c5b2640 1 -- 172.21.15.49:0/2258351040 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fee741056c0 msgr2=0x7fee74105aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:12:30.050 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:30.049+0000 7fee7c5b2640 1 --2- 172.21.15.49:0/2258351040 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fee741056c0 0x7fee74105aa0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fee70009900 tx=0x7fee7002bc30 comp rx=0 tx=0).stop 2024-03-18T23:12:30.051 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:30.049+0000 7fee7c5b2640 1 -- 172.21.15.49:0/2258351040 shutdown_connections 2024-03-18T23:12:30.051 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:30.049+0000 7fee7c5b2640 1 --2- 172.21.15.49:0/2258351040 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fee74101330 0x7fee7410d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:30.051 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:30.049+0000 7fee7c5b2640 1 --2- 172.21.15.49:0/2258351040 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fee74100970 0x7fee74100df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:30.051 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:30.049+0000 7fee7c5b2640 1 --2- 172.21.15.49:0/2258351040 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fee741056c0 0x7fee74105aa0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:30.051 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:30.049+0000 7fee7c5b2640 1 -- 172.21.15.49:0/2258351040 >> 172.21.15.49:0/2258351040 conn(0x7fee74077570 msgr2=0x7fee74077980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:12:30.051 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:30.049+0000 7fee7c5b2640 1 -- 172.21.15.49:0/2258351040 shutdown_connections 2024-03-18T23:12:30.051 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:30.049+0000 7fee7c5b2640 1 -- 172.21.15.49:0/2258351040 wait complete. 2024-03-18T23:12:30.051 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:30.050+0000 7fee7c5b2640 1 Processor -- start 2024-03-18T23:12:30.051 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:30.050+0000 7fee7c5b2640 1 -- start start 2024-03-18T23:12:30.053 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:30.050+0000 7fee7c5b2640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fee74100970 0x7fee74102390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:12:30.053 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:30.050+0000 7fee7c5b2640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fee74101330 0x7fee741028d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:12:30.053 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:30.050+0000 7fee7c5b2640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fee741056c0 0x7fee74102e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:12:30.053 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:30.050+0000 7fee7c5b2640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fee74110be0 con 0x7fee741056c0 2024-03-18T23:12:30.053 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:30.050+0000 7fee7c5b2640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fee74110a60 con 0x7fee74101330 2024-03-18T23:12:30.053 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:30.050+0000 7fee7c5b2640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fee74110d60 con 0x7fee74100970 2024-03-18T23:12:30.053 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:30.050+0000 7fee7ab4f640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fee741056c0 0x7fee74102e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:12:30.053 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:30.050+0000 7fee7ab4f640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fee741056c0 0x7fee74102e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49452/0 (socket says 172.21.15.49:49452) 2024-03-18T23:12:30.053 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:30.050+0000 7fee7ab4f640 1 -- 172.21.15.49:0/4073840543 learned_addr learned my addr 172.21.15.49:0/4073840543 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:12:30.053 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:30.050+0000 7fee7a34e640 1 --2- 172.21.15.49:0/4073840543 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fee74100970 0x7fee74102390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:12:30.054 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:30.050+0000 7fee79b4d640 1 --2- 172.21.15.49:0/4073840543 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fee74101330 0x7fee741028d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:12:30.054 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:30.050+0000 7fee7ab4f640 1 -- 172.21.15.49:0/4073840543 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fee74100970 msgr2=0x7fee74102390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:12:30.054 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:30.050+0000 7fee7ab4f640 1 --2- 172.21.15.49:0/4073840543 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fee74100970 0x7fee74102390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:30.054 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:30.050+0000 7fee7ab4f640 1 -- 172.21.15.49:0/4073840543 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fee74101330 msgr2=0x7fee741028d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:12:30.054 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:30.050+0000 7fee7ab4f640 1 --2- 172.21.15.49:0/4073840543 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fee74101330 0x7fee741028d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:30.054 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:30.050+0000 7fee7ab4f640 1 -- 172.21.15.49:0/4073840543 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee741a65e0 con 0x7fee741056c0 2024-03-18T23:12:30.054 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:30.051+0000 7fee7ab4f640 1 --2- 172.21.15.49:0/4073840543 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fee741056c0 0x7fee74102e10 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fee6c00ea60 tx=0x7fee6c00ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:12:30.054 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:30.051+0000 7fee6b7fe640 1 -- 172.21.15.49:0/4073840543 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee6c00ce40 con 0x7fee741056c0 2024-03-18T23:12:30.054 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:30.051+0000 7fee7c5b2640 1 -- 172.21.15.49:0/4073840543 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee741a68d0 con 0x7fee741056c0 2024-03-18T23:12:30.054 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:30.051+0000 7fee6b7fe640 1 -- 172.21.15.49:0/4073840543 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fee6c005020 con 0x7fee741056c0 2024-03-18T23:12:30.054 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:30.051+0000 7fee6b7fe640 1 -- 172.21.15.49:0/4073840543 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee6c010690 con 0x7fee741056c0 2024-03-18T23:12:30.054 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:30.051+0000 7fee7c5b2640 1 -- 172.21.15.49:0/4073840543 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee741a6e70 con 0x7fee741056c0 2024-03-18T23:12:30.060 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:30.053+0000 7fee6b7fe640 1 -- 172.21.15.49:0/4073840543 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 25) ==== 137421+0+0 (secure 0 0 0) 0x7fee6c0051d0 con 0x7fee741056c0 2024-03-18T23:12:30.060 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:30.053+0000 7fee7f19f740 1 Processor -- start 2024-03-18T23:12:30.060 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:30.054+0000 7fee7f19f740 1 -- start start 2024-03-18T23:12:30.060 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:30.054+0000 7fee7f19f740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c0b6941ef0 0x55c0b69422d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:12:30.060 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:30.054+0000 7fee7f19f740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c0b6942810 0x55c0b69538b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:12:30.060 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:30.054+0000 7fee6b7fe640 1 --2- 172.21.15.49:0/4073840543 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7fee600a2a10 0x7fee600a4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:12:30.060 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:30.054+0000 7fee7f19f740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c0b6953df0 0x55c0b69561e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:12:30.060 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:30.054+0000 7fee7f19f740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55c0b68a5be0 con 0x55c0b6953df0 2024-03-18T23:12:30.060 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:30.054+0000 7fee7f19f740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55c0b684e960 con 0x55c0b6941ef0 2024-03-18T23:12:30.061 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:30.054+0000 7fee7f19f740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55c0b68a1c80 con 0x55c0b6942810 2024-03-18T23:12:30.061 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:30.054+0000 7fee7ab4f640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c0b6953df0 0x55c0b69561e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:12:30.061 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:30.054+0000 7fee79b4d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c0b6941ef0 0x55c0b69422d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:12:30.061 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:30.054+0000 7fee79b4d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c0b6941ef0 0x55c0b69422d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.49:54882/0 (socket says 172.21.15.49:54882) 2024-03-18T23:12:30.061 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:30.054+0000 7fee79b4d640 1 -- 172.21.15.49:0/1693251920 learned_addr learned my addr 172.21.15.49:0/1693251920 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:12:30.061 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:30.054+0000 7fee7a34e640 1 --2- 172.21.15.49:0/1693251920 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c0b6942810 0x55c0b69538b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:12:30.061 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:30.054+0000 7fee79b4d640 1 --2- 172.21.15.49:0/4073840543 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7fee600a2a10 0x7fee600a4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:12:30.061 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:30.054+0000 7fee6b7fe640 1 -- 172.21.15.49:0/4073840543 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8469+0+0 (secure 0 0 0) 0x7fee6c0cf3d0 con 0x7fee741056c0 2024-03-18T23:12:30.061 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:30.054+0000 7fee7ab4f640 1 -- 172.21.15.49:0/1693251920 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c0b6942810 msgr2=0x55c0b69538b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:12:30.061 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:30.054+0000 7fee7ab4f640 1 --2- 172.21.15.49:0/1693251920 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c0b6942810 0x55c0b69538b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:30.061 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:30.054+0000 7fee7ab4f640 1 -- 172.21.15.49:0/1693251920 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c0b6941ef0 msgr2=0x55c0b69422d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:12:30.061 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:30.054+0000 7fee7ab4f640 1 --2- 172.21.15.49:0/1693251920 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c0b6941ef0 0x55c0b69422d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:30.061 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:30.054+0000 7fee7ab4f640 1 -- 172.21.15.49:0/1693251920 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c0b6845d20 con 0x55c0b6953df0 2024-03-18T23:12:30.061 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:30.054+0000 7fee7a34e640 1 --2- 172.21.15.49:0/1693251920 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c0b6942810 0x55c0b69538b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:12:30.062 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:30.055+0000 7fee79b4d640 1 --2- 172.21.15.49:0/4073840543 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7fee600a2a10 0x7fee600a4ed0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fee64009fd0 tx=0x7fee64005f90 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:12:30.062 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:30.055+0000 7fee7ab4f640 1 --2- 172.21.15.49:0/1693251920 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c0b6953df0 0x55c0b69561e0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fee6c0ccb50 tx=0x7fee6c01bb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25636c552f2d7938 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:12:30.062 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:30.055+0000 7fee68bfe640 1 -- 172.21.15.49:0/1693251920 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee6c0ae640 con 0x55c0b6953df0 2024-03-18T23:12:30.062 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:30.055+0000 7fee68bfe640 1 -- 172.21.15.49:0/1693251920 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fee6c09c160 con 0x55c0b6953df0 2024-03-18T23:12:30.062 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:30.055+0000 7fee68bfe640 1 -- 172.21.15.49:0/1693251920 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee6c0ae460 con 0x55c0b6953df0 2024-03-18T23:12:30.062 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:30.056+0000 7fee7f19f740 1 -- 172.21.15.49:0/1693251920 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c0b6953df0 msgr2=0x55c0b69561e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:12:30.062 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:30.056+0000 7fee7f19f740 1 --2- 172.21.15.49:0/1693251920 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c0b6953df0 0x55c0b69561e0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fee6c0ccb50 tx=0x7fee6c01bb30 comp rx=0 tx=0).stop 2024-03-18T23:12:30.062 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:30.056+0000 7fee7f19f740 1 -- 172.21.15.49:0/1693251920 shutdown_connections 2024-03-18T23:12:30.062 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:30.056+0000 7fee7f19f740 1 --2- 172.21.15.49:0/1693251920 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c0b6953df0 0x55c0b69561e0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:30.062 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:30.056+0000 7fee7f19f740 1 --2- 172.21.15.49:0/1693251920 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c0b6942810 0x55c0b69538b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:30.062 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:30.056+0000 7fee7f19f740 1 --2- 172.21.15.49:0/1693251920 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c0b6941ef0 0x55c0b69422d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:30.062 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:30.056+0000 7fee7f19f740 1 -- 172.21.15.49:0/1693251920 >> 172.21.15.49:0/1693251920 conn(0x55c0b693c810 msgr2=0x55c0b693cbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:12:30.062 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:30.056+0000 7fee7f19f740 1 -- 172.21.15.49:0/1693251920 shutdown_connections 2024-03-18T23:12:30.063 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:30.056+0000 7fee7f19f740 1 -- 172.21.15.49:0/1693251920 wait complete. 2024-03-18T23:12:30.063 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:30.056+0000 7fee7f19f740 1 -- start start 2024-03-18T23:12:30.063 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:30.056+0000 7fee7f19f740 1 Processor -- start 2024-03-18T23:12:30.063 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:30.056+0000 7fee7f19f740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c0b6941ef0 0x55c0b6953700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:12:30.063 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:30.056+0000 7fee7f19f740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c0b6942810 0x55c0b6950a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:12:30.063 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:30.056+0000 7fee7f19f740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c0b6953df0 0x55c0b6950fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:12:30.063 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:30.056+0000 7fee7f19f740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55c0b69569c0 con 0x55c0b6942810 2024-03-18T23:12:30.063 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:30.056+0000 7fee7f19f740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55c0b6861db0 con 0x55c0b6953df0 2024-03-18T23:12:30.063 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:30.056+0000 7fee7f19f740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55c0b6956b40 con 0x55c0b6941ef0 2024-03-18T23:12:30.063 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:30.057+0000 7fee79b4d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c0b6942810 0x55c0b6950a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:12:30.063 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:30.057+0000 7fee7f19f740 10 client.0 _ll_register_callbacks cb 0x55c0b6866e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T23:12:30.063 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:30.057+0000 7fee79b4d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c0b6942810 0x55c0b6950a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49468/0 (socket says 172.21.15.49:49468) 2024-03-18T23:12:30.063 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:30.057+0000 7fee79b4d640 1 -- 172.21.15.49:0/2651882018 learned_addr learned my addr 172.21.15.49:0/2651882018 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:12:30.063 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:30.057+0000 7fee7a34e640 1 --2- 172.21.15.49:0/2651882018 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c0b6941ef0 0x55c0b6953700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:12:30.063 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:30.057+0000 7fee7ab4f640 1 --2- 172.21.15.49:0/2651882018 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c0b6953df0 0x55c0b6950fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:12:30.064 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:30.057+0000 7fee79b4d640 1 -- 172.21.15.49:0/2651882018 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c0b6941ef0 msgr2=0x55c0b6953700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:12:30.064 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:30.057+0000 7fee79b4d640 1 --2- 172.21.15.49:0/2651882018 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c0b6941ef0 0x55c0b6953700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:30.064 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:30.057+0000 7fee79b4d640 1 -- 172.21.15.49:0/2651882018 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c0b6953df0 msgr2=0x55c0b6950fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:12:30.064 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:30.057+0000 7fee79b4d640 1 --2- 172.21.15.49:0/2651882018 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c0b6953df0 0x55c0b6950fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:30.064 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:30.057+0000 7fee79b4d640 1 -- 172.21.15.49:0/2651882018 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c0b6860c60 con 0x55c0b6942810 2024-03-18T23:12:30.064 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:30.057+0000 7fee79b4d640 1 --2- 172.21.15.49:0/2651882018 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c0b6942810 0x55c0b6950a90 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fee64004570 tx=0x7fee64004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2c490018285cd9e server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:12:30.064 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:30.057+0000 7fee5dffb640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-18T23:12:30.064 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:30.057+0000 7fee5dffb640 1 -- 172.21.15.49:0/2651882018 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee64025070 con 0x55c0b6942810 2024-03-18T23:12:30.064 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:30.057+0000 7fee5dffb640 1 -- 172.21.15.49:0/2651882018 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8469+0+0 (secure 0 0 0) 0x7fee64003d70 con 0x55c0b6942810 2024-03-18T23:12:30.064 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:30.057+0000 7fee7f19f740 1 -- 172.21.15.49:0/2651882018 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fee6c09c160 con 0x55c0b6942810 2024-03-18T23:12:30.064 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:30.057+0000 7fee5dffb640 1 -- 172.21.15.49:0/2651882018 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fee64009a60 con 0x55c0b6942810 2024-03-18T23:12:30.064 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:30.057+0000 7fee7f19f740 10 client.15798 fetch_fsmap learned FSMap version 32 2024-03-18T23:12:30.064 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:30.057+0000 7fee7f19f740 1 -- 172.21.15.49:0/2651882018 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55c0b65ba210 con 0x55c0b6942810 2024-03-18T23:12:30.064 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:30.058+0000 7fee5dffb640 1 -- 172.21.15.49:0/2651882018 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fee6c09c160 con 0x55c0b6942810 2024-03-18T23:12:30.065 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:30.058+0000 7fee5dffb640 1 -- 172.21.15.49:0/2651882018 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7fee64004bb0 con 0x55c0b6942810 2024-03-18T23:12:30.065 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:30.058+0000 7fee7f19f740 10 client.15798 fetch_fsmap finished waiting for FSMap version 32 2024-03-18T23:12:30.065 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:30.058+0000 7fee7f19f740 10 client.15798 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T23:12:30.065 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:30.058+0000 7fee7f19f740 20 client.15798 populate_metadata read hostname 'smithi049' 2024-03-18T23:12:30.065 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:30.058+0000 7fee7f19f740 1 --2- 172.21.15.49:0/2651882018 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x55c0b6963ba0 0x55c0b6965f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:12:30.065 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:30.058+0000 7fee7f19f740 4 client.15798 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T23:12:30.065 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:30.058+0000 7fee7f19f740 1 -- 172.21.15.49:0/2651882018 --> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c0b68b3120 con 0x55c0b6963ba0 2024-03-18T23:12:30.065 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:30.058+0000 7fee7a34e640 1 --2- 172.21.15.49:0/2651882018 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x55c0b6963ba0 0x55c0b6965f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:12:30.065 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:30.058+0000 7fee7a34e640 1 --2- 172.21.15.49:0/2651882018 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x55c0b6963ba0 0x55c0b6965f90 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ae77068a56c383af server_cookie=7a1029c30e0ca0d1 in_seq=0 out_seq=0 2024-03-18T23:12:30.065 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:30.058+0000 7fee5dffb640 10 client.15798 ms_handle_connect on v2:172.21.15.139:6838/4280128209 2024-03-18T23:12:30.066 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:30.060+0000 7fee5dffb640 1 -- 172.21.15.49:0/2651882018 <== mds.0 v2:172.21.15.139:6838/4280128209 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55c0b68b3120 con 0x55c0b6963ba0 2024-03-18T23:12:30.066 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:30.060+0000 7fee5dffb640 10 client.15798 handle_command_reply: tid=0 2024-03-18T23:12:30.066 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:30.060+0000 7fee7f19f740 1 client.15798 shutdown 2024-03-18T23:12:30.066 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:30.060+0000 7fee7f19f740 1 -- 172.21.15.49:0/2651882018 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c0b6942810 msgr2=0x55c0b6950a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:12:30.066 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:30.060+0000 7fee7f19f740 1 --2- 172.21.15.49:0/2651882018 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c0b6942810 0x55c0b6950a90 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fee64004570 tx=0x7fee64004730 comp rx=0 tx=0).stop 2024-03-18T23:12:30.067 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:30.060+0000 7fee7f19f740 1 -- 172.21.15.49:0/2651882018 shutdown_connections 2024-03-18T23:12:30.067 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:30.060+0000 7fee7f19f740 1 --2- 172.21.15.49:0/2651882018 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x55c0b6963ba0 0x55c0b6965f90 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:30.067 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:30.060+0000 7fee7f19f740 1 --2- 172.21.15.49:0/2651882018 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c0b6953df0 0x55c0b6950fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:30.067 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:30.060+0000 7fee7f19f740 1 --2- 172.21.15.49:0/2651882018 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c0b6942810 0x55c0b6950a90 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:30.067 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:30.060+0000 7fee5dffb640 0 client.15798 ms_handle_reset on v2:172.21.15.139:6838/4280128209 2024-03-18T23:12:30.067 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:30.060+0000 7fee7f19f740 1 --2- 172.21.15.49:0/2651882018 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c0b6941ef0 0x55c0b6953700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:30.067 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:30.060+0000 7fee7f19f740 1 -- 172.21.15.49:0/2651882018 >> 172.21.15.49:0/2651882018 conn(0x55c0b693c810 msgr2=0x55c0b693e560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:12:30.067 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:30.060+0000 7fee7f19f740 1 -- 172.21.15.49:0/2651882018 shutdown_connections 2024-03-18T23:12:30.067 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:30.060+0000 7fee7f19f740 1 -- 172.21.15.49:0/2651882018 wait complete. 2024-03-18T23:12:30.067 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:30.060+0000 7fee7f19f740 20 client.15798 trim_cache size 0 max 16384 2024-03-18T23:12:30.082 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:30.077+0000 7fee7f19f740 1 Processor -- start 2024-03-18T23:12:30.082 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:30.077+0000 7fee7f19f740 1 -- start start 2024-03-18T23:12:30.082 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:30.077+0000 7fee7f19f740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c0b6941ef0 0x55c0b695f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:12:30.082 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:30.077+0000 7fee7f19f740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c0b6942810 0x55c0b694bf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:12:30.082 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:30.077+0000 7fee7f19f740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c0b6953df0 0x55c0b696b880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:12:30.082 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:30.077+0000 7fee7f19f740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fee741ab2f0 con 0x55c0b6953df0 2024-03-18T23:12:30.082 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:30.077+0000 7fee7f19f740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fee741ace30 con 0x55c0b6942810 2024-03-18T23:12:30.082 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:30.077+0000 7fee7f19f740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fee741ab9c0 con 0x55c0b6941ef0 2024-03-18T23:12:30.082 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:30.077+0000 7fee7ab4f640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c0b6953df0 0x55c0b696b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:12:30.083 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:30.077+0000 7fee7a34e640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c0b6941ef0 0x55c0b695f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:12:30.083 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:30.077+0000 7fee7ab4f640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c0b6953df0 0x55c0b696b880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49476/0 (socket says 172.21.15.49:49476) 2024-03-18T23:12:30.083 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:30.077+0000 7fee7ab4f640 1 -- 172.21.15.49:0/788746990 learned_addr learned my addr 172.21.15.49:0/788746990 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:12:30.083 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:30.077+0000 7fee7a34e640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c0b6941ef0 0x55c0b695f760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.49:46458/0 (socket says 172.21.15.49:46458) 2024-03-18T23:12:30.083 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:30.077+0000 7fee79b4d640 1 --2- 172.21.15.49:0/788746990 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c0b6942810 0x55c0b694bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:12:30.083 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:30.077+0000 7fee7ab4f640 1 -- 172.21.15.49:0/788746990 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c0b6941ef0 msgr2=0x55c0b695f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:12:30.083 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:30.077+0000 7fee7ab4f640 1 --2- 172.21.15.49:0/788746990 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c0b6941ef0 0x55c0b695f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:30.083 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:30.077+0000 7fee7ab4f640 1 -- 172.21.15.49:0/788746990 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c0b6942810 msgr2=0x55c0b694bf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:12:30.083 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:30.077+0000 7fee7ab4f640 1 --2- 172.21.15.49:0/788746990 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c0b6942810 0x55c0b694bf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:30.083 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:30.077+0000 7fee7ab4f640 1 -- 172.21.15.49:0/788746990 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c0b6966300 con 0x55c0b6953df0 2024-03-18T23:12:30.083 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:30.077+0000 7fee7a34e640 1 --2- 172.21.15.49:0/788746990 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c0b6941ef0 0x55c0b695f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:12:30.083 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:30.077+0000 7fee7ab4f640 1 --2- 172.21.15.49:0/788746990 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c0b6953df0 0x55c0b696b880 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fee6c098430 tx=0x7fee6c09b6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f160c2fa49d2bf3 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:12:30.083 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:30.078+0000 7fee5ffff640 1 -- 172.21.15.49:0/788746990 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee6c09fbb0 con 0x55c0b6953df0 2024-03-18T23:12:30.083 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:30.078+0000 7fee5ffff640 1 -- 172.21.15.49:0/788746990 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fee6c0a4200 con 0x55c0b6953df0 2024-03-18T23:12:30.084 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:30.078+0000 7fee5ffff640 1 -- 172.21.15.49:0/788746990 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee6c098260 con 0x55c0b6953df0 2024-03-18T23:12:30.084 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:30.078+0000 7fee7f19f740 1 -- 172.21.15.49:0/788746990 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c0b6953df0 msgr2=0x55c0b696b880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:12:30.084 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:30.078+0000 7fee7f19f740 1 --2- 172.21.15.49:0/788746990 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c0b6953df0 0x55c0b696b880 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fee6c098430 tx=0x7fee6c09b6a0 comp rx=0 tx=0).stop 2024-03-18T23:12:30.084 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:30.078+0000 7fee7f19f740 1 -- 172.21.15.49:0/788746990 shutdown_connections 2024-03-18T23:12:30.084 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:30.078+0000 7fee7f19f740 1 --2- 172.21.15.49:0/788746990 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c0b6953df0 0x55c0b696b880 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:30.084 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:30.078+0000 7fee7f19f740 1 --2- 172.21.15.49:0/788746990 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c0b6942810 0x55c0b694bf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:30.084 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:30.078+0000 7fee7f19f740 1 --2- 172.21.15.49:0/788746990 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c0b6941ef0 0x55c0b695f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:30.084 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:30.078+0000 7fee7f19f740 1 -- 172.21.15.49:0/788746990 >> 172.21.15.49:0/788746990 conn(0x55c0b693c810 msgr2=0x55c0b694b850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:12:30.084 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:30.078+0000 7fee7f19f740 1 -- 172.21.15.49:0/788746990 shutdown_connections 2024-03-18T23:12:30.084 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:30.078+0000 7fee7f19f740 1 -- 172.21.15.49:0/788746990 wait complete. 2024-03-18T23:12:30.084 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:30.079+0000 7fee7f19f740 1 -- start start 2024-03-18T23:12:30.084 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:30.079+0000 7fee7f19f740 1 Processor -- start 2024-03-18T23:12:30.084 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:30.079+0000 7fee7f19f740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c0b6941ef0 0x55c0b694d710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:12:30.084 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:30.079+0000 7fee7f19f740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c0b6942810 0x55c0b694dc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:12:30.084 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:30.079+0000 7fee7f19f740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c0b6953df0 0x55c0b694e190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:12:30.085 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:30.079+0000 7fee7f19f740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fee741ac760 con 0x55c0b6942810 2024-03-18T23:12:30.085 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:30.079+0000 7fee7f19f740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fee741ac090 con 0x55c0b6953df0 2024-03-18T23:12:30.085 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:30.079+0000 7fee7f19f740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fee741a92f0 con 0x55c0b6941ef0 2024-03-18T23:12:30.085 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:30.079+0000 7fee7f19f740 10 client.0 _ll_register_callbacks cb 0x7fee64004360 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T23:12:30.085 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:30.079+0000 7fee79b4d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c0b6942810 0x55c0b694dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:12:30.085 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:30.079+0000 7fee79b4d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c0b6942810 0x55c0b694dc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49486/0 (socket says 172.21.15.49:49486) 2024-03-18T23:12:30.085 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:30.079+0000 7fee79b4d640 1 -- 172.21.15.49:0/2668241318 learned_addr learned my addr 172.21.15.49:0/2668241318 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:12:30.085 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:30.079+0000 7fee7a34e640 1 --2- 172.21.15.49:0/2668241318 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c0b6941ef0 0x55c0b694d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:12:30.085 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:30.079+0000 7fee7ab4f640 1 --2- 172.21.15.49:0/2668241318 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c0b6953df0 0x55c0b694e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:12:30.085 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:30.079+0000 7fee79b4d640 1 -- 172.21.15.49:0/2668241318 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c0b6941ef0 msgr2=0x55c0b694d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:12:30.085 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:30.079+0000 7fee79b4d640 1 --2- 172.21.15.49:0/2668241318 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c0b6941ef0 0x55c0b694d710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:30.085 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:30.079+0000 7fee79b4d640 1 -- 172.21.15.49:0/2668241318 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c0b6953df0 msgr2=0x55c0b694e190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:12:30.085 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:30.079+0000 7fee79b4d640 1 --2- 172.21.15.49:0/2668241318 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c0b6953df0 0x55c0b694e190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:30.085 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:30.079+0000 7fee79b4d640 1 -- 172.21.15.49:0/2668241318 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c0b6686620 con 0x55c0b6942810 2024-03-18T23:12:30.086 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:30.079+0000 7fee79b4d640 1 --2- 172.21.15.49:0/2668241318 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c0b6942810 0x55c0b694dc50 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fee64000c00 tx=0x7fee64022f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b43ae2ef3b28edcb server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:12:30.086 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:30.079+0000 7fee5dffb640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-18T23:12:30.086 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:30.080+0000 7fee5dffb640 1 -- 172.21.15.49:0/2668241318 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee64025070 con 0x55c0b6942810 2024-03-18T23:12:30.086 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:30.080+0000 7fee5dffb640 1 -- 172.21.15.49:0/2668241318 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8469+0+0 (secure 0 0 0) 0x7fee64003bc0 con 0x55c0b6942810 2024-03-18T23:12:30.086 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:30.080+0000 7fee7f19f740 1 -- 172.21.15.49:0/2668241318 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fee6c0a4200 con 0x55c0b6942810 2024-03-18T23:12:30.086 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:30.080+0000 7fee5dffb640 1 -- 172.21.15.49:0/2668241318 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fee64003dc0 con 0x55c0b6942810 2024-03-18T23:12:30.086 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:30.080+0000 7fee7f19f740 10 client.15804 fetch_fsmap learned FSMap version 32 2024-03-18T23:12:30.086 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:30.080+0000 7fee7f19f740 1 -- 172.21.15.49:0/2668241318 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55c0b69ca060 con 0x55c0b6942810 2024-03-18T23:12:30.086 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:30.080+0000 7fee5dffb640 1 -- 172.21.15.49:0/2668241318 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fee6c0a4200 con 0x55c0b6942810 2024-03-18T23:12:30.086 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:30.080+0000 7fee5dffb640 1 -- 172.21.15.49:0/2668241318 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7fee64005160 con 0x55c0b6942810 2024-03-18T23:12:30.086 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:30.080+0000 7fee7f19f740 10 client.15804 fetch_fsmap finished waiting for FSMap version 32 2024-03-18T23:12:30.086 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:30.080+0000 7fee7f19f740 10 client.15804 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T23:12:30.086 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:30.080+0000 7fee7f19f740 20 client.15804 populate_metadata read hostname 'smithi049' 2024-03-18T23:12:30.087 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:30.080+0000 7fee7f19f740 1 --2- 172.21.15.49:0/2668241318 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x55c0b69ca7a0 0x55c0b69ccbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:12:30.087 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:30.080+0000 7fee7f19f740 4 client.15804 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T23:12:30.087 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:30.080+0000 7fee7f19f740 1 -- 172.21.15.49:0/2668241318 --> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c0b68ad470 con 0x55c0b69ca7a0 2024-03-18T23:12:30.087 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:30.080+0000 7fee7a34e640 1 --2- 172.21.15.49:0/2668241318 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x55c0b69ca7a0 0x55c0b69ccbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:12:30.087 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:30.081+0000 7fee5dffb640 10 client.15804 ms_handle_connect on v2:172.21.15.139:6838/4280128209 2024-03-18T23:12:30.087 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:30.081+0000 7fee7a34e640 1 --2- 172.21.15.49:0/2668241318 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x55c0b69ca7a0 0x55c0b69ccbc0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dc37dd89be871a45 server_cookie=9c2de1a99fbfa4e3 in_seq=0 out_seq=0 2024-03-18T23:12:30.292 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:30.290+0000 7fee5dffb640 1 -- 172.21.15.49:0/2668241318 <== mds.0 v2:172.21.15.139:6838/4280128209 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c0b68ad470 con 0x55c0b69ca7a0 2024-03-18T23:12:30.292 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:30.290+0000 7fee5dffb640 10 client.15804 handle_command_reply: tid=0 2024-03-18T23:12:30.292 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:30.290+0000 7fee7f19f740 1 client.15804 shutdown 2024-03-18T23:12:30.292 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:30.290+0000 7fee7f19f740 1 -- 172.21.15.49:0/2668241318 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c0b6942810 msgr2=0x55c0b694dc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:12:30.292 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:30.290+0000 7fee7f19f740 1 --2- 172.21.15.49:0/2668241318 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c0b6942810 0x55c0b694dc50 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fee64000c00 tx=0x7fee64022f80 comp rx=0 tx=0).stop 2024-03-18T23:12:30.293 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:30.291+0000 7fee7f19f740 1 -- 172.21.15.49:0/2668241318 shutdown_connections 2024-03-18T23:12:30.293 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:30.291+0000 7fee7f19f740 1 --2- 172.21.15.49:0/2668241318 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x55c0b69ca7a0 0x55c0b69ccbc0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:30.293 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:30.291+0000 7fee7f19f740 1 --2- 172.21.15.49:0/2668241318 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c0b6953df0 0x55c0b694e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:30.293 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:30.291+0000 7fee7f19f740 1 --2- 172.21.15.49:0/2668241318 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c0b6942810 0x55c0b694dc50 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:30.293 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:30.291+0000 7fee7f19f740 1 --2- 172.21.15.49:0/2668241318 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c0b6941ef0 0x55c0b694d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:30.293 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:30.291+0000 7fee7f19f740 1 -- 172.21.15.49:0/2668241318 >> 172.21.15.49:0/2668241318 conn(0x55c0b693c810 msgr2=0x55c0b694c0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:12:30.293 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:30.291+0000 7fee5dffb640 0 client.15804 ms_handle_reset on v2:172.21.15.139:6838/4280128209 2024-03-18T23:12:30.293 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:30.291+0000 7fee7f19f740 1 -- 172.21.15.49:0/2668241318 shutdown_connections 2024-03-18T23:12:30.293 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:30.291+0000 7fee7f19f740 1 -- 172.21.15.49:0/2668241318 wait complete. 2024-03-18T23:12:30.293 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:30.291+0000 7fee7f19f740 20 client.15804 trim_cache size 0 max 16384 2024-03-18T23:12:30.294 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:{ 2024-03-18T23:12:30.294 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "message": "", 2024-03-18T23:12:30.294 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "return_code": 0 2024-03-18T23:12:30.294 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:} 2024-03-18T23:12:30.295 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:30.292+0000 7fee7c5b2640 1 -- 172.21.15.49:0/4073840543 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7fee600a2a10 msgr2=0x7fee600a4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:12:30.295 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:30.292+0000 7fee7c5b2640 1 --2- 172.21.15.49:0/4073840543 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7fee600a2a10 0x7fee600a4ed0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fee64009fd0 tx=0x7fee64005f90 comp rx=0 tx=0).stop 2024-03-18T23:12:30.295 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:30.292+0000 7fee7c5b2640 1 -- 172.21.15.49:0/4073840543 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fee741056c0 msgr2=0x7fee74102e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:12:30.295 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:30.292+0000 7fee7c5b2640 1 --2- 172.21.15.49:0/4073840543 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fee741056c0 0x7fee74102e10 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fee6c00ea60 tx=0x7fee6c00ef20 comp rx=0 tx=0).stop 2024-03-18T23:12:30.295 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:30.292+0000 7fee7c5b2640 1 -- 172.21.15.49:0/4073840543 shutdown_connections 2024-03-18T23:12:30.295 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:30.292+0000 7fee7c5b2640 1 --2- 172.21.15.49:0/4073840543 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7fee600a2a10 0x7fee600a4ed0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:30.295 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:30.292+0000 7fee7c5b2640 1 --2- 172.21.15.49:0/4073840543 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fee741056c0 0x7fee74102e10 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:30.296 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:30.292+0000 7fee7c5b2640 1 --2- 172.21.15.49:0/4073840543 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fee74101330 0x7fee741028d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:30.296 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:30.292+0000 7fee7c5b2640 1 --2- 172.21.15.49:0/4073840543 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fee74100970 0x7fee74102390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:30.296 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:30.292+0000 7fee7c5b2640 1 -- 172.21.15.49:0/4073840543 >> 172.21.15.49:0/4073840543 conn(0x7fee74077570 msgr2=0x7fee740690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:12:30.296 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:30.292+0000 7fee7c5b2640 1 -- 172.21.15.49:0/4073840543 shutdown_connections 2024-03-18T23:12:30.296 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:30.292+0000 7fee7c5b2640 1 -- 172.21.15.49:0/4073840543 wait complete. 2024-03-18T23:12:30.711 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:12:30 smithi175 ceph-mon[67352]: pgmap v131: 97 pgs: 97 active+clean; 52 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.9 MiB/s wr, 249 op/s 2024-03-18T23:12:30.768 INFO:tasks.workunit.client.0.smithi049.stdout: 1 454183 10.23 MB/sec execute 92 sec latency 111.085 ms 2024-03-18T23:12:30.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:12:30 smithi139 ceph-mon[67282]: pgmap v131: 97 pgs: 97 active+clean; 52 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.9 MiB/s wr, 249 op/s 2024-03-18T23:12:30.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:12:30 smithi049 ceph-mon[62991]: pgmap v131: 97 pgs: 97 active+clean; 52 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.9 MiB/s wr, 249 op/s 2024-03-18T23:12:31.768 INFO:tasks.workunit.client.0.smithi049.stdout: 1 455724 10.20 MB/sec execute 93 sec latency 21.956 ms 2024-03-18T23:12:32.711 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:12:32 smithi175 ceph-mon[67352]: pgmap v132: 97 pgs: 97 active+clean; 41 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 4.3 MiB/s wr, 432 op/s 2024-03-18T23:12:32.768 INFO:tasks.workunit.client.0.smithi049.stdout: 1 457965 10.23 MB/sec execute 94 sec latency 17.635 ms 2024-03-18T23:12:32.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:12:32 smithi139 ceph-mon[67282]: pgmap v132: 97 pgs: 97 active+clean; 41 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 4.3 MiB/s wr, 432 op/s 2024-03-18T23:12:32.879 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:12:32 smithi049 ceph-mon[62991]: pgmap v132: 97 pgs: 97 active+clean; 41 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 4.3 MiB/s wr, 432 op/s 2024-03-18T23:12:33.768 INFO:tasks.workunit.client.0.smithi049.stdout: 1 458866 10.32 MB/sec execute 95 sec latency 80.290 ms 2024-03-18T23:12:34.711 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:12:34 smithi175 ceph-mon[67352]: pgmap v133: 97 pgs: 97 active+clean; 41 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.8 MiB/s wr, 275 op/s 2024-03-18T23:12:34.768 INFO:tasks.workunit.client.0.smithi049.stdout: 1 460744 10.31 MB/sec execute 96 sec latency 149.792 ms 2024-03-18T23:12:34.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:12:34 smithi139 ceph-mon[67282]: pgmap v133: 97 pgs: 97 active+clean; 41 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.8 MiB/s wr, 275 op/s 2024-03-18T23:12:34.879 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:12:34 smithi049 ceph-mon[62991]: pgmap v133: 97 pgs: 97 active+clean; 41 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.8 MiB/s wr, 275 op/s 2024-03-18T23:12:35.768 INFO:tasks.workunit.client.0.smithi049.stdout: 1 462806 10.34 MB/sec execute 97 sec latency 16.544 ms 2024-03-18T23:12:36.711 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:12:36 smithi175 ceph-mon[67352]: pgmap v134: 97 pgs: 97 active+clean; 41 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.8 MiB/s wr, 275 op/s 2024-03-18T23:12:36.768 INFO:tasks.workunit.client.0.smithi049.stdout: 1 465088 10.36 MB/sec execute 98 sec latency 20.683 ms 2024-03-18T23:12:36.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:12:36 smithi139 ceph-mon[67282]: pgmap v134: 97 pgs: 97 active+clean; 41 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.8 MiB/s wr, 275 op/s 2024-03-18T23:12:36.879 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:12:36 smithi049 ceph-mon[62991]: pgmap v134: 97 pgs: 97 active+clean; 41 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.8 MiB/s wr, 275 op/s 2024-03-18T23:12:37.768 INFO:tasks.workunit.client.0.smithi049.stdout: 1 466856 10.37 MB/sec execute 99 sec latency 57.655 ms 2024-03-18T23:12:38.711 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:12:38 smithi175 ceph-mon[67352]: pgmap v135: 97 pgs: 97 active+clean; 46 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.0 MiB/s wr, 363 op/s 2024-03-18T23:12:38.769 INFO:tasks.workunit.client.0.smithi049.stdout: 1 469085 10.39 MB/sec execute 100 sec latency 36.731 ms 2024-03-18T23:12:38.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:12:38 smithi139 ceph-mon[67282]: pgmap v135: 97 pgs: 97 active+clean; 46 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.0 MiB/s wr, 363 op/s 2024-03-18T23:12:38.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:12:38 smithi049 ceph-mon[62991]: pgmap v135: 97 pgs: 97 active+clean; 46 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.0 MiB/s wr, 363 op/s 2024-03-18T23:12:39.769 INFO:tasks.workunit.client.0.smithi049.stdout: 1 470555 10.35 MB/sec execute 101 sec latency 23.990 ms 2024-03-18T23:12:40.711 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:12:40 smithi175 ceph-mon[67352]: pgmap v136: 97 pgs: 97 active+clean; 46 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.6 MiB/s wr, 270 op/s 2024-03-18T23:12:40.769 INFO:tasks.workunit.client.0.smithi049.stdout: 1 472930 10.39 MB/sec execute 102 sec latency 18.563 ms 2024-03-18T23:12:40.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:12:40 smithi139 ceph-mon[67282]: pgmap v136: 97 pgs: 97 active+clean; 46 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.6 MiB/s wr, 270 op/s 2024-03-18T23:12:40.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:12:40 smithi049 ceph-mon[62991]: pgmap v136: 97 pgs: 97 active+clean; 46 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.6 MiB/s wr, 270 op/s 2024-03-18T23:12:41.769 INFO:tasks.workunit.client.0.smithi049.stdout: 1 474928 10.36 MB/sec execute 103 sec latency 25.351 ms 2024-03-18T23:12:42.711 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:12:42 smithi175 ceph-mon[67352]: pgmap v137: 97 pgs: 97 active+clean; 51 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.6 MiB/s wr, 348 op/s 2024-03-18T23:12:42.769 INFO:tasks.workunit.client.0.smithi049.stdout: 1 477031 10.38 MB/sec execute 104 sec latency 27.989 ms 2024-03-18T23:12:42.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:12:42 smithi139 ceph-mon[67282]: pgmap v137: 97 pgs: 97 active+clean; 51 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.6 MiB/s wr, 348 op/s 2024-03-18T23:12:42.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:12:42 smithi049 ceph-mon[62991]: pgmap v137: 97 pgs: 97 active+clean; 51 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.6 MiB/s wr, 348 op/s 2024-03-18T23:12:43.389 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:43.387+0000 7f9372a73640 1 -- 172.21.15.49:0/3131850948 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f936c1006f0 msgr2=0x7f936c100b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:12:43.389 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:43.387+0000 7f9372a73640 1 --2- 172.21.15.49:0/3131850948 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f936c1006f0 0x7f936c100b70 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f9358009930 tx=0x7f935802bc30 comp rx=0 tx=0).stop 2024-03-18T23:12:43.390 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:43.387+0000 7f9372a73640 1 -- 172.21.15.49:0/3131850948 shutdown_connections 2024-03-18T23:12:43.390 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:43.387+0000 7f9372a73640 1 --2- 172.21.15.49:0/3131850948 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f936c1010b0 0x7f936c10d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:43.390 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:43.387+0000 7f9372a73640 1 --2- 172.21.15.49:0/3131850948 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f936c1006f0 0x7f936c100b70 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:43.390 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:43.387+0000 7f9372a73640 1 --2- 172.21.15.49:0/3131850948 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f936c106700 0x7f936c106ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:43.390 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:43.387+0000 7f9372a73640 1 -- 172.21.15.49:0/3131850948 >> 172.21.15.49:0/3131850948 conn(0x7f936c077850 msgr2=0x7f936c077c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:12:43.390 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:43.387+0000 7f9372a73640 1 -- 172.21.15.49:0/3131850948 shutdown_connections 2024-03-18T23:12:43.390 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:43.387+0000 7f9372a73640 1 -- 172.21.15.49:0/3131850948 wait complete. 2024-03-18T23:12:43.390 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:43.388+0000 7f9372a73640 1 Processor -- start 2024-03-18T23:12:43.390 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:43.388+0000 7f9372a73640 1 -- start start 2024-03-18T23:12:43.390 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:43.388+0000 7f9372a73640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f936c1006f0 0x7f936c196f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:12:43.391 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:43.388+0000 7f9372a73640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f936c1010b0 0x7f936c197490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:12:43.391 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:43.388+0000 7f9372a73640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f936c106700 0x7f936c19b810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:12:43.391 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:43.388+0000 7f9372a73640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f936c110b10 con 0x7f936c106700 2024-03-18T23:12:43.391 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:43.388+0000 7f9372a73640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f936c110990 con 0x7f936c1010b0 2024-03-18T23:12:43.391 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:43.388+0000 7f9372a73640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f936c110c90 con 0x7f936c1006f0 2024-03-18T23:12:43.391 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:43.388+0000 7f9371010640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f936c106700 0x7f936c19b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:12:43.391 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:43.388+0000 7f9371010640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f936c106700 0x7f936c19b810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:52692/0 (socket says 172.21.15.49:52692) 2024-03-18T23:12:43.391 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:43.388+0000 7f936bfff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f936c1010b0 0x7f936c197490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:12:43.391 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:43.388+0000 7f9371010640 1 -- 172.21.15.49:0/2831890358 learned_addr learned my addr 172.21.15.49:0/2831890358 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:12:43.391 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:43.388+0000 7f937080f640 1 --2- 172.21.15.49:0/2831890358 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f936c1006f0 0x7f936c196f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:12:43.391 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:43.388+0000 7f936bfff640 1 -- 172.21.15.49:0/2831890358 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f936c1006f0 msgr2=0x7f936c196f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:12:43.391 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:43.388+0000 7f936bfff640 1 --2- 172.21.15.49:0/2831890358 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f936c1006f0 0x7f936c196f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:43.391 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:43.388+0000 7f936bfff640 1 -- 172.21.15.49:0/2831890358 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f936c106700 msgr2=0x7f936c19b810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:12:43.391 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:43.388+0000 7f936bfff640 1 --2- 172.21.15.49:0/2831890358 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f936c106700 0x7f936c19b810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:43.392 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:43.388+0000 7f936bfff640 1 -- 172.21.15.49:0/2831890358 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f936c19bf40 con 0x7f936c1010b0 2024-03-18T23:12:43.392 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:43.388+0000 7f9371010640 1 --2- 172.21.15.49:0/2831890358 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f936c106700 0x7f936c19b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:12:43.392 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:43.389+0000 7f936bfff640 1 --2- 172.21.15.49:0/2831890358 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f936c1010b0 0x7f936c197490 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f9358009230 tx=0x7f935802cb20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:12:43.392 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:43.389+0000 7f9369ffb640 1 -- 172.21.15.49:0/2831890358 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9358041070 con 0x7f936c1010b0 2024-03-18T23:12:43.392 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:43.389+0000 7f9369ffb640 1 -- 172.21.15.49:0/2831890358 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9358004450 con 0x7f936c1010b0 2024-03-18T23:12:43.392 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:43.389+0000 7f9372a73640 1 -- 172.21.15.49:0/2831890358 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f936c19c1d0 con 0x7f936c1010b0 2024-03-18T23:12:43.392 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:43.389+0000 7f9369ffb640 1 -- 172.21.15.49:0/2831890358 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9358003d30 con 0x7f936c1010b0 2024-03-18T23:12:43.392 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:43.389+0000 7f9372a73640 1 -- 172.21.15.49:0/2831890358 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f936c06a020 con 0x7f936c1010b0 2024-03-18T23:12:43.397 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:43.392+0000 7f937565d740 1 Processor -- start 2024-03-18T23:12:43.397 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:43.392+0000 7f9369ffb640 1 -- 172.21.15.49:0/2831890358 <== mon.2 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 25) ==== 137421+0+0 (secure 0 0 0) 0x7f9358044070 con 0x7f936c1010b0 2024-03-18T23:12:43.397 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:43.392+0000 7f937565d740 1 -- start start 2024-03-18T23:12:43.398 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:43.393+0000 7f937565d740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5576f236bef0 0x5576f236c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:12:43.398 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:43.393+0000 7f937565d740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5576f236c810 0x5576f23750a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:12:43.398 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:43.393+0000 7f937565d740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5576f23755e0 0x5576f23779d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:12:43.398 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:43.393+0000 7f937565d740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5576f22cfbe0 con 0x5576f236bef0 2024-03-18T23:12:43.398 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:43.393+0000 7f937565d740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x5576f2278960 con 0x5576f236c810 2024-03-18T23:12:43.398 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:43.393+0000 7f937565d740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5576f22cbc80 con 0x5576f23755e0 2024-03-18T23:12:43.398 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:43.393+0000 7f9371010640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5576f236bef0 0x5576f236c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:12:43.398 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:43.393+0000 7f9371010640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5576f236bef0 0x5576f236c2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:52694/0 (socket says 172.21.15.49:52694) 2024-03-18T23:12:43.398 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:43.393+0000 7f9371010640 1 -- 172.21.15.49:0/2778896476 learned_addr learned my addr 172.21.15.49:0/2778896476 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:12:43.398 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:43.393+0000 7f936bfff640 1 --2- 172.21.15.49:0/2778896476 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5576f23755e0 0x5576f23779d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:12:43.398 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:43.393+0000 7f937080f640 1 --2- 172.21.15.49:0/2778896476 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5576f236c810 0x5576f23750a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:12:43.398 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:43.393+0000 7f9369ffb640 1 --2- 172.21.15.49:0/2831890358 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f93500a2a10 0x7f93500a4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:12:43.398 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:43.393+0000 7f9371010640 1 -- 172.21.15.49:0/2778896476 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5576f23755e0 msgr2=0x5576f23779d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:12:43.399 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:43.393+0000 7f9371010640 1 --2- 172.21.15.49:0/2778896476 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5576f23755e0 0x5576f23779d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:43.399 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:43.393+0000 7f9371010640 1 -- 172.21.15.49:0/2778896476 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5576f236c810 msgr2=0x5576f23750a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:12:43.399 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:43.393+0000 7f9371010640 1 --2- 172.21.15.49:0/2778896476 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5576f236c810 0x5576f23750a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:43.399 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:43.393+0000 7f9371010640 1 -- 172.21.15.49:0/2778896476 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5576f226fd20 con 0x5576f236bef0 2024-03-18T23:12:43.399 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:43.393+0000 7f936bfff640 1 --2- 172.21.15.49:0/2778896476 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5576f23755e0 0x5576f23779d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:12:43.399 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:43.393+0000 7f937080f640 1 --2- 172.21.15.49:0/2778896476 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5576f236c810 0x5576f23750a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:12:43.399 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:43.393+0000 7f9371010640 1 --2- 172.21.15.49:0/2831890358 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f93500a2a10 0x7f93500a4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:12:43.399 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:43.393+0000 7f9369ffb640 1 -- 172.21.15.49:0/2831890358 <== mon.2 v2:172.21.15.139:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8469+0+0 (secure 0 0 0) 0x7f93580f05c0 con 0x7f936c1010b0 2024-03-18T23:12:43.399 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:43.393+0000 7f9371010640 1 --2- 172.21.15.49:0/2778896476 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5576f236bef0 0x5576f236c2d0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f936400ea80 tx=0x7f936400ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c16d54f1dc157934 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:12:43.399 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:43.394+0000 7f9362ffd640 1 -- 172.21.15.49:0/2778896476 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f936401f040 con 0x5576f236bef0 2024-03-18T23:12:43.399 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:43.394+0000 7f9362ffd640 1 -- 172.21.15.49:0/2778896476 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f936401b8e0 con 0x5576f236bef0 2024-03-18T23:12:43.399 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:43.394+0000 7f9362ffd640 1 -- 172.21.15.49:0/2778896476 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9364025610 con 0x5576f236bef0 2024-03-18T23:12:43.399 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:43.394+0000 7f937565d740 1 -- 172.21.15.49:0/2778896476 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5576f236bef0 msgr2=0x5576f236c2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:12:43.400 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:43.394+0000 7f9371010640 1 --2- 172.21.15.49:0/2831890358 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f93500a2a10 0x7f93500a4ed0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f9364025000 tx=0x7f9364026000 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:12:43.400 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:43.394+0000 7f937565d740 1 --2- 172.21.15.49:0/2778896476 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5576f236bef0 0x5576f236c2d0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f936400ea80 tx=0x7f936400ef40 comp rx=0 tx=0).stop 2024-03-18T23:12:43.400 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:43.394+0000 7f937565d740 1 -- 172.21.15.49:0/2778896476 shutdown_connections 2024-03-18T23:12:43.400 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:43.394+0000 7f937565d740 1 --2- 172.21.15.49:0/2778896476 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5576f23755e0 0x5576f23779d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:43.400 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:43.394+0000 7f937565d740 1 --2- 172.21.15.49:0/2778896476 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5576f236c810 0x5576f23750a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:43.400 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:43.394+0000 7f937565d740 1 --2- 172.21.15.49:0/2778896476 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5576f236bef0 0x5576f236c2d0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:43.400 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:43.394+0000 7f937565d740 1 -- 172.21.15.49:0/2778896476 >> 172.21.15.49:0/2778896476 conn(0x5576f2366810 msgr2=0x5576f2366bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:12:43.400 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:43.394+0000 7f937565d740 1 -- 172.21.15.49:0/2778896476 shutdown_connections 2024-03-18T23:12:43.400 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:43.394+0000 7f937565d740 1 -- 172.21.15.49:0/2778896476 wait complete. 2024-03-18T23:12:43.400 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:43.395+0000 7f937565d740 1 -- start start 2024-03-18T23:12:43.400 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:43.395+0000 7f937565d740 1 Processor -- start 2024-03-18T23:12:43.400 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:43.395+0000 7f937565d740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5576f236bef0 0x5576f2374ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:12:43.400 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:43.395+0000 7f937565d740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5576f236c810 0x5576f2372260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:12:43.400 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:43.395+0000 7f937565d740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5576f23755e0 0x5576f23727a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:12:43.401 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:43.395+0000 7f937565d740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5576f2378210 con 0x5576f236bef0 2024-03-18T23:12:43.401 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:43.395+0000 7f937565d740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x5576f228bdb0 con 0x5576f23755e0 2024-03-18T23:12:43.401 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:43.395+0000 7f937565d740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5576f2378390 con 0x5576f236c810 2024-03-18T23:12:43.401 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:43.395+0000 7f937565d740 10 client.0 _ll_register_callbacks cb 0x5576f2290e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T23:12:43.401 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:43.395+0000 7f937080f640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5576f236bef0 0x5576f2374ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:12:43.401 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:43.395+0000 7f937080f640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5576f236bef0 0x5576f2374ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:52696/0 (socket says 172.21.15.49:52696) 2024-03-18T23:12:43.401 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:43.395+0000 7f937080f640 1 -- 172.21.15.49:0/3400350906 learned_addr learned my addr 172.21.15.49:0/3400350906 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:12:43.401 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:43.395+0000 7f936bfff640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5576f236c810 0x5576f2372260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:12:43.401 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:43.395+0000 7f9371010640 1 --2- 172.21.15.49:0/3400350906 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5576f23755e0 0x5576f23727a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:12:43.401 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:43.396+0000 7f937080f640 1 -- 172.21.15.49:0/3400350906 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5576f236c810 msgr2=0x5576f2372260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:12:43.401 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:43.396+0000 7f937080f640 1 --2- 172.21.15.49:0/3400350906 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5576f236c810 0x5576f2372260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:43.401 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:43.396+0000 7f937080f640 1 -- 172.21.15.49:0/3400350906 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5576f23755e0 msgr2=0x5576f23727a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:12:43.401 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:43.396+0000 7f937080f640 1 --2- 172.21.15.49:0/3400350906 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5576f23755e0 0x5576f23727a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:43.401 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:43.396+0000 7f937080f640 1 -- 172.21.15.49:0/3400350906 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5576f228ac60 con 0x5576f236bef0 2024-03-18T23:12:43.402 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:43.396+0000 7f936bfff640 1 --2- 172.21.15.49:0/3400350906 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5576f236c810 0x5576f2372260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:12:43.402 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:43.396+0000 7f937080f640 1 --2- 172.21.15.49:0/3400350906 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5576f236bef0 0x5576f2374ed0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f935c0126e0 tx=0x7f935c012ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5d78327586cee3a server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:12:43.402 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:43.396+0000 7f934ffff640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-18T23:12:43.402 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:43.396+0000 7f934ffff640 1 -- 172.21.15.49:0/3400350906 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f935c00f670 con 0x5576f236bef0 2024-03-18T23:12:43.402 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:43.396+0000 7f934ffff640 1 -- 172.21.15.49:0/3400350906 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8469+0+0 (secure 0 0 0) 0x7f935c00f810 con 0x5576f236bef0 2024-03-18T23:12:43.402 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:43.396+0000 7f937565d740 1 -- 172.21.15.49:0/3400350906 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f936401b8e0 con 0x5576f236bef0 2024-03-18T23:12:43.402 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:43.396+0000 7f934ffff640 1 -- 172.21.15.49:0/3400350906 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f935c00fa10 con 0x5576f236bef0 2024-03-18T23:12:43.402 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:43.396+0000 7f937565d740 10 client.15816 fetch_fsmap learned FSMap version 32 2024-03-18T23:12:43.402 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:43.396+0000 7f937565d740 1 -- 172.21.15.49:0/3400350906 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5576f1fe4210 con 0x5576f236bef0 2024-03-18T23:12:43.402 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:43.397+0000 7f934ffff640 1 -- 172.21.15.49:0/3400350906 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f936401b8e0 con 0x5576f236bef0 2024-03-18T23:12:43.402 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:43.397+0000 7f934ffff640 1 -- 172.21.15.49:0/3400350906 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f935c010430 con 0x5576f236bef0 2024-03-18T23:12:43.402 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:43.397+0000 7f937565d740 10 client.15816 fetch_fsmap finished waiting for FSMap version 32 2024-03-18T23:12:43.402 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:43.397+0000 7f937565d740 10 client.15816 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T23:12:43.403 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:43.397+0000 7f937565d740 20 client.15816 populate_metadata read hostname 'smithi049' 2024-03-18T23:12:43.403 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:43.397+0000 7f937565d740 1 --2- 172.21.15.49:0/3400350906 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x5576f2385430 0x5576f2387840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:12:43.403 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:43.397+0000 7f937565d740 4 client.15816 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T23:12:43.403 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:43.397+0000 7f937565d740 1 -- 172.21.15.49:0/3400350906 --> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5576f22dd120 con 0x5576f2385430 2024-03-18T23:12:43.403 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:43.397+0000 7f936bfff640 1 --2- 172.21.15.49:0/3400350906 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x5576f2385430 0x5576f2387840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:12:43.403 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:43.398+0000 7f936bfff640 1 --2- 172.21.15.49:0/3400350906 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x5576f2385430 0x5576f2387840 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=94b4707651b789d5 server_cookie=f70a29f11c6a4233 in_seq=0 out_seq=0 2024-03-18T23:12:43.403 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:43.398+0000 7f934ffff640 10 client.15816 ms_handle_connect on v2:172.21.15.139:6838/4280128209 2024-03-18T23:12:43.404 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:43.399+0000 7f934ffff640 1 -- 172.21.15.49:0/3400350906 <== mds.0 v2:172.21.15.139:6838/4280128209 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5576f22dd120 con 0x5576f2385430 2024-03-18T23:12:43.404 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:43.399+0000 7f934ffff640 10 client.15816 handle_command_reply: tid=0 2024-03-18T23:12:43.405 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:43.399+0000 7f937565d740 1 client.15816 shutdown 2024-03-18T23:12:43.405 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:43.400+0000 7f937565d740 1 -- 172.21.15.49:0/3400350906 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5576f236bef0 msgr2=0x5576f2374ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:12:43.405 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:43.400+0000 7f937565d740 1 --2- 172.21.15.49:0/3400350906 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5576f236bef0 0x5576f2374ed0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f935c0126e0 tx=0x7f935c012ae0 comp rx=0 tx=0).stop 2024-03-18T23:12:43.405 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:43.400+0000 7f937565d740 1 -- 172.21.15.49:0/3400350906 shutdown_connections 2024-03-18T23:12:43.405 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:43.400+0000 7f937565d740 1 --2- 172.21.15.49:0/3400350906 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x5576f2385430 0x5576f2387840 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:43.405 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:43.400+0000 7f937565d740 1 --2- 172.21.15.49:0/3400350906 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5576f23755e0 0x5576f23727a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:43.405 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:43.400+0000 7f937565d740 1 --2- 172.21.15.49:0/3400350906 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5576f236c810 0x5576f2372260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:43.405 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:43.400+0000 7f937565d740 1 --2- 172.21.15.49:0/3400350906 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5576f236bef0 0x5576f2374ed0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:43.405 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:43.400+0000 7f934ffff640 0 client.15816 ms_handle_reset on v2:172.21.15.139:6838/4280128209 2024-03-18T23:12:43.405 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:43.400+0000 7f937565d740 1 -- 172.21.15.49:0/3400350906 >> 172.21.15.49:0/3400350906 conn(0x5576f2366810 msgr2=0x5576f2365ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:12:43.405 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:43.400+0000 7f937565d740 1 -- 172.21.15.49:0/3400350906 shutdown_connections 2024-03-18T23:12:43.405 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:43.400+0000 7f937565d740 1 -- 172.21.15.49:0/3400350906 wait complete. 2024-03-18T23:12:43.405 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:43.400+0000 7f937565d740 20 client.15816 trim_cache size 0 max 16384 2024-03-18T23:12:43.416 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:43.415+0000 7f937565d740 1 Processor -- start 2024-03-18T23:12:43.416 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:43.415+0000 7f937565d740 1 -- start start 2024-03-18T23:12:43.416 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:43.415+0000 7f937565d740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5576f236bef0 0x5576f2380fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:12:43.416 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:43.415+0000 7f937565d740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5576f236c810 0x5576f238d150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:12:43.416 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:43.415+0000 7f937565d740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5576f23755e0 0x5576f2374990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:12:43.417 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:43.415+0000 7f937565d740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f936c1ab190 con 0x5576f236c810 2024-03-18T23:12:43.417 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:43.415+0000 7f937565d740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f936c1accd0 con 0x5576f236bef0 2024-03-18T23:12:43.417 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:43.415+0000 7f937565d740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f936c1ab860 con 0x5576f23755e0 2024-03-18T23:12:43.417 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:43.415+0000 7f936bfff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5576f236c810 0x5576f238d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:12:43.417 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:43.415+0000 7f936bfff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5576f236c810 0x5576f238d150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:52712/0 (socket says 172.21.15.49:52712) 2024-03-18T23:12:43.417 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:43.415+0000 7f936bfff640 1 -- 172.21.15.49:0/1724031653 learned_addr learned my addr 172.21.15.49:0/1724031653 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:12:43.417 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:43.415+0000 7f9371010640 1 --2- 172.21.15.49:0/1724031653 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5576f23755e0 0x5576f2374990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:12:43.417 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:43.416+0000 7f937080f640 1 --2- 172.21.15.49:0/1724031653 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5576f236bef0 0x5576f2380fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:12:43.417 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:43.416+0000 7f936bfff640 1 -- 172.21.15.49:0/1724031653 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5576f23755e0 msgr2=0x5576f2374990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:12:43.417 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:43.416+0000 7f936bfff640 1 --2- 172.21.15.49:0/1724031653 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5576f23755e0 0x5576f2374990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:43.417 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:43.416+0000 7f936bfff640 1 -- 172.21.15.49:0/1724031653 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5576f236bef0 msgr2=0x5576f2380fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:12:43.417 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:43.416+0000 7f936bfff640 1 --2- 172.21.15.49:0/1724031653 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5576f236bef0 0x5576f2380fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:43.417 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:43.416+0000 7f936bfff640 1 -- 172.21.15.49:0/1724031653 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5576f20b0620 con 0x5576f236c810 2024-03-18T23:12:43.418 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:43.416+0000 7f936bfff640 1 --2- 172.21.15.49:0/1724031653 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5576f236c810 0x5576f238d150 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f935802c170 tx=0x7f93580f6550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1113506dd1e59c8 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:12:43.418 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:43.416+0000 7f93627fc640 1 -- 172.21.15.49:0/1724031653 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f935804b040 con 0x5576f236c810 2024-03-18T23:12:43.418 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:43.416+0000 7f93627fc640 1 -- 172.21.15.49:0/1724031653 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f93580035f0 con 0x5576f236c810 2024-03-18T23:12:43.418 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:43.416+0000 7f93627fc640 1 -- 172.21.15.49:0/1724031653 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93580c5500 con 0x5576f236c810 2024-03-18T23:12:43.421 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:43.417+0000 7f937565d740 1 -- 172.21.15.49:0/1724031653 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5576f236c810 msgr2=0x5576f238d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:12:43.422 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:43.417+0000 7f937565d740 1 --2- 172.21.15.49:0/1724031653 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5576f236c810 0x5576f238d150 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f935802c170 tx=0x7f93580f6550 comp rx=0 tx=0).stop 2024-03-18T23:12:43.422 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:43.417+0000 7f937565d740 1 -- 172.21.15.49:0/1724031653 shutdown_connections 2024-03-18T23:12:43.422 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:43.417+0000 7f937565d740 1 --2- 172.21.15.49:0/1724031653 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5576f23755e0 0x5576f2374990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:43.422 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:43.417+0000 7f937565d740 1 --2- 172.21.15.49:0/1724031653 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5576f236c810 0x5576f238d150 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:43.422 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:43.417+0000 7f937565d740 1 --2- 172.21.15.49:0/1724031653 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5576f236bef0 0x5576f2380fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:43.422 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:43.417+0000 7f937565d740 1 -- 172.21.15.49:0/1724031653 >> 172.21.15.49:0/1724031653 conn(0x5576f2366810 msgr2=0x5576f2365ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:12:43.422 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:43.417+0000 7f937565d740 1 -- 172.21.15.49:0/1724031653 shutdown_connections 2024-03-18T23:12:43.422 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:43.417+0000 7f937565d740 1 -- 172.21.15.49:0/1724031653 wait complete. 2024-03-18T23:12:43.422 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:43.417+0000 7f937565d740 1 -- start start 2024-03-18T23:12:43.422 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:43.417+0000 7f937565d740 1 Processor -- start 2024-03-18T23:12:43.422 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:43.418+0000 7f937565d740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5576f236bef0 0x5576f23792c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:12:43.422 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:43.418+0000 7f937565d740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5576f236c810 0x5576f2379800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:12:43.422 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:43.418+0000 7f937565d740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5576f23755e0 0x5576f23ea780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:12:43.423 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:43.418+0000 7f937565d740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f936c1ac600 con 0x5576f236bef0 2024-03-18T23:12:43.423 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:43.418+0000 7f937565d740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f936c1abf30 con 0x5576f23755e0 2024-03-18T23:12:43.423 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:43.418+0000 7f937565d740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f936c1a9170 con 0x5576f236c810 2024-03-18T23:12:43.423 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:43.418+0000 7f937565d740 10 client.0 _ll_register_callbacks cb 0x7f935c009ea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T23:12:43.423 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:43.418+0000 7f937080f640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5576f236bef0 0x5576f23792c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:12:43.423 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:43.418+0000 7f937080f640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5576f236bef0 0x5576f23792c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:52724/0 (socket says 172.21.15.49:52724) 2024-03-18T23:12:43.423 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:43.418+0000 7f937080f640 1 -- 172.21.15.49:0/2851519372 learned_addr learned my addr 172.21.15.49:0/2851519372 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:12:43.423 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:43.418+0000 7f9371010640 1 --2- 172.21.15.49:0/2851519372 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5576f23755e0 0x5576f23ea780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:12:43.423 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:43.418+0000 7f936bfff640 1 --2- 172.21.15.49:0/2851519372 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5576f236c810 0x5576f2379800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:12:43.423 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:43.418+0000 7f937080f640 1 -- 172.21.15.49:0/2851519372 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5576f236c810 msgr2=0x5576f2379800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:12:43.423 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:43.418+0000 7f937080f640 1 --2- 172.21.15.49:0/2851519372 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5576f236c810 0x5576f2379800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:43.423 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:43.418+0000 7f937080f640 1 -- 172.21.15.49:0/2851519372 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5576f23755e0 msgr2=0x5576f23ea780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:12:43.423 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:43.418+0000 7f937080f640 1 --2- 172.21.15.49:0/2851519372 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5576f23755e0 0x5576f23ea780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:43.423 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:43.418+0000 7f937080f640 1 -- 172.21.15.49:0/2851519372 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5576f23eacc0 con 0x5576f236bef0 2024-03-18T23:12:43.423 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:43.418+0000 7f936bfff640 1 --2- 172.21.15.49:0/2851519372 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5576f236c810 0x5576f2379800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T23:12:43.424 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:43.418+0000 7f937080f640 1 --2- 172.21.15.49:0/2851519372 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5576f236bef0 0x5576f23792c0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f935c010a40 tx=0x7f935c00f8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eefe8106d19ba84f server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:12:43.424 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:43.418+0000 7f934ffff640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-18T23:12:43.424 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:43.419+0000 7f934ffff640 1 -- 172.21.15.49:0/2851519372 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f935c010640 con 0x5576f236bef0 2024-03-18T23:12:43.424 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:43.419+0000 7f934ffff640 1 -- 172.21.15.49:0/2851519372 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8469+0+0 (secure 0 0 0) 0x7f935c018070 con 0x5576f236bef0 2024-03-18T23:12:43.424 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:43.419+0000 7f937565d740 1 -- 172.21.15.49:0/2851519372 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5576f22d7470 con 0x5576f236bef0 2024-03-18T23:12:43.424 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:43.419+0000 7f934ffff640 1 -- 172.21.15.49:0/2851519372 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f935c035050 con 0x5576f236bef0 2024-03-18T23:12:43.424 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:43.419+0000 7f937565d740 10 client.15822 fetch_fsmap learned FSMap version 32 2024-03-18T23:12:43.424 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:43.419+0000 7f937565d740 1 -- 172.21.15.49:0/2851519372 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5576f23ebe70 con 0x5576f236bef0 2024-03-18T23:12:43.424 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:43.419+0000 7f934ffff640 1 -- 172.21.15.49:0/2851519372 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x5576f22d7470 con 0x5576f236bef0 2024-03-18T23:12:43.424 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:43.419+0000 7f934ffff640 1 -- 172.21.15.49:0/2851519372 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f935c00fbb0 con 0x5576f236bef0 2024-03-18T23:12:43.424 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:43.419+0000 7f937565d740 10 client.15822 fetch_fsmap finished waiting for FSMap version 32 2024-03-18T23:12:43.424 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:43.419+0000 7f937565d740 10 client.15822 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T23:12:43.424 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:43.419+0000 7f937565d740 20 client.15822 populate_metadata read hostname 'smithi049' 2024-03-18T23:12:43.424 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:43.419+0000 7f937565d740 1 --2- 172.21.15.49:0/2851519372 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x5576f23ec670 0x5576f23eea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:12:43.425 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:43.419+0000 7f937565d740 4 client.15822 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T23:12:43.425 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:43.419+0000 7f937565d740 1 -- 172.21.15.49:0/2851519372 --> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5576f2255770 con 0x5576f23ec670 2024-03-18T23:12:43.425 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:43.420+0000 7f936bfff640 1 --2- 172.21.15.49:0/2851519372 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x5576f23ec670 0x5576f23eea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:12:43.425 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:43.420+0000 7f936bfff640 1 --2- 172.21.15.49:0/2851519372 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x5576f23ec670 0x5576f23eea90 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ab6e8bc41f7c6fb1 server_cookie=7849d7060e61fd5d in_seq=0 out_seq=0 2024-03-18T23:12:43.425 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:43.420+0000 7f934ffff640 10 client.15822 ms_handle_connect on v2:172.21.15.139:6838/4280128209 2024-03-18T23:12:43.588 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:43.587+0000 7f934ffff640 1 -- 172.21.15.49:0/2851519372 <== mds.0 v2:172.21.15.139:6838/4280128209 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5576f2255770 con 0x5576f23ec670 2024-03-18T23:12:43.588 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:43.587+0000 7f934ffff640 10 client.15822 handle_command_reply: tid=0 2024-03-18T23:12:43.588 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:43.587+0000 7f937565d740 1 client.15822 shutdown 2024-03-18T23:12:43.588 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:43.587+0000 7f937565d740 1 -- 172.21.15.49:0/2851519372 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5576f236bef0 msgr2=0x5576f23792c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:12:43.588 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:43.587+0000 7f937565d740 1 --2- 172.21.15.49:0/2851519372 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5576f236bef0 0x5576f23792c0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f935c010a40 tx=0x7f935c00f8a0 comp rx=0 tx=0).stop 2024-03-18T23:12:43.588 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:43.587+0000 7f937565d740 1 -- 172.21.15.49:0/2851519372 shutdown_connections 2024-03-18T23:12:43.589 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:43.587+0000 7f937565d740 1 --2- 172.21.15.49:0/2851519372 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x5576f23ec670 0x5576f23eea90 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:43.589 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:43.587+0000 7f934ffff640 0 client.15822 ms_handle_reset on v2:172.21.15.139:6838/4280128209 2024-03-18T23:12:43.589 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:43.587+0000 7f937565d740 1 --2- 172.21.15.49:0/2851519372 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5576f23755e0 0x5576f23ea780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:43.589 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:43.587+0000 7f937565d740 1 --2- 172.21.15.49:0/2851519372 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5576f236c810 0x5576f2379800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:43.589 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:43.587+0000 7f937565d740 1 --2- 172.21.15.49:0/2851519372 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5576f236bef0 0x5576f23792c0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:43.589 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:43.587+0000 7f937565d740 1 -- 172.21.15.49:0/2851519372 >> 172.21.15.49:0/2851519372 conn(0x5576f2366810 msgr2=0x5576f2365220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:12:43.589 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:43.588+0000 7f937565d740 1 -- 172.21.15.49:0/2851519372 shutdown_connections 2024-03-18T23:12:43.589 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:43.588+0000 7f937565d740 1 -- 172.21.15.49:0/2851519372 wait complete. 2024-03-18T23:12:43.589 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:43.588+0000 7f937565d740 20 client.15822 trim_cache size 0 max 16384 2024-03-18T23:12:43.589 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:{ 2024-03-18T23:12:43.590 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "message": "", 2024-03-18T23:12:43.590 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "return_code": 0 2024-03-18T23:12:43.590 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:} 2024-03-18T23:12:43.591 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:43.588+0000 7f9372a73640 1 -- 172.21.15.49:0/2831890358 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f93500a2a10 msgr2=0x7f93500a4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:12:43.591 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:43.588+0000 7f9372a73640 1 --2- 172.21.15.49:0/2831890358 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f93500a2a10 0x7f93500a4ed0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f9364025000 tx=0x7f9364026000 comp rx=0 tx=0).stop 2024-03-18T23:12:43.591 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:43.589+0000 7f9372a73640 1 -- 172.21.15.49:0/2831890358 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f936c1010b0 msgr2=0x7f936c197490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:12:43.591 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:43.589+0000 7f9372a73640 1 --2- 172.21.15.49:0/2831890358 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f936c1010b0 0x7f936c197490 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f9358009230 tx=0x7f935802cb20 comp rx=0 tx=0).stop 2024-03-18T23:12:43.591 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:43.589+0000 7f9372a73640 1 -- 172.21.15.49:0/2831890358 shutdown_connections 2024-03-18T23:12:43.591 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:43.589+0000 7f9372a73640 1 --2- 172.21.15.49:0/2831890358 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f93500a2a10 0x7f93500a4ed0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:43.591 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:43.589+0000 7f9372a73640 1 --2- 172.21.15.49:0/2831890358 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f936c106700 0x7f936c19b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:43.591 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:43.589+0000 7f9372a73640 1 --2- 172.21.15.49:0/2831890358 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f936c1010b0 0x7f936c197490 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:43.591 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:43.589+0000 7f9372a73640 1 --2- 172.21.15.49:0/2831890358 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f936c1006f0 0x7f936c196f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:43.591 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:43.589+0000 7f9372a73640 1 -- 172.21.15.49:0/2831890358 >> 172.21.15.49:0/2831890358 conn(0x7f936c077850 msgr2=0x7f936c102dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:12:43.591 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:43.589+0000 7f9372a73640 1 -- 172.21.15.49:0/2831890358 shutdown_connections 2024-03-18T23:12:43.591 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:43.589+0000 7f9372a73640 1 -- 172.21.15.49:0/2831890358 wait complete. 2024-03-18T23:12:43.769 INFO:tasks.workunit.client.0.smithi049.stdout: 1 478904 10.38 MB/sec execute 105 sec latency 126.876 ms 2024-03-18T23:12:44.769 INFO:tasks.workunit.client.0.smithi049.stdout: 1 481053 10.41 MB/sec execute 106 sec latency 18.189 ms 2024-03-18T23:12:44.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:12:44 smithi139 ceph-mon[67282]: pgmap v138: 97 pgs: 97 active+clean; 51 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.1 MiB/s wr, 165 op/s 2024-03-18T23:12:44.879 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:12:44 smithi049 ceph-mon[62991]: pgmap v138: 97 pgs: 97 active+clean; 51 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.1 MiB/s wr, 165 op/s 2024-03-18T23:12:44.961 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:12:44 smithi175 ceph-mon[67352]: pgmap v138: 97 pgs: 97 active+clean; 51 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.1 MiB/s wr, 165 op/s 2024-03-18T23:12:45.769 INFO:tasks.workunit.client.0.smithi049.stdout: 1 483427 10.44 MB/sec execute 107 sec latency 19.985 ms 2024-03-18T23:12:46.769 INFO:tasks.workunit.client.0.smithi049.stdout: 1 484995 10.39 MB/sec execute 108 sec latency 20.702 ms 2024-03-18T23:12:46.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:12:46 smithi139 ceph-mon[67282]: pgmap v139: 97 pgs: 97 active+clean; 51 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.1 MiB/s wr, 165 op/s 2024-03-18T23:12:46.879 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:12:46 smithi049 ceph-mon[62991]: pgmap v139: 97 pgs: 97 active+clean; 51 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.1 MiB/s wr, 165 op/s 2024-03-18T23:12:46.961 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:12:46 smithi175 ceph-mon[67352]: pgmap v139: 97 pgs: 97 active+clean; 51 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.1 MiB/s wr, 165 op/s 2024-03-18T23:12:47.769 INFO:tasks.workunit.client.0.smithi049.stdout: 1 487267 10.42 MB/sec execute 109 sec latency 19.723 ms 2024-03-18T23:12:48.769 INFO:tasks.workunit.client.0.smithi049.stdout: 1 489097 10.39 MB/sec execute 110 sec latency 45.176 ms 2024-03-18T23:12:48.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:12:48 smithi139 ceph-mon[67282]: pgmap v140: 97 pgs: 97 active+clean; 46 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.8 MiB/s wr, 323 op/s 2024-03-18T23:12:48.879 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:12:48 smithi049 ceph-mon[62991]: pgmap v140: 97 pgs: 97 active+clean; 46 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.8 MiB/s wr, 323 op/s 2024-03-18T23:12:48.961 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:12:48 smithi175 ceph-mon[67352]: pgmap v140: 97 pgs: 97 active+clean; 46 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.8 MiB/s wr, 323 op/s 2024-03-18T23:12:49.769 INFO:tasks.workunit.client.0.smithi049.stdout: 1 490928 10.41 MB/sec execute 111 sec latency 58.334 ms 2024-03-18T23:12:50.769 INFO:tasks.workunit.client.0.smithi049.stdout: 1 492879 10.39 MB/sec execute 112 sec latency 37.397 ms 2024-03-18T23:12:50.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:12:50 smithi139 ceph-mon[67282]: pgmap v141: 97 pgs: 97 active+clean; 46 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 900 KiB/s rd, 1.7 MiB/s wr, 236 op/s 2024-03-18T23:12:50.879 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:12:50 smithi049 ceph-mon[62991]: pgmap v141: 97 pgs: 97 active+clean; 46 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 900 KiB/s rd, 1.7 MiB/s wr, 236 op/s 2024-03-18T23:12:50.962 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:12:50 smithi175 ceph-mon[67352]: pgmap v141: 97 pgs: 97 active+clean; 46 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 900 KiB/s rd, 1.7 MiB/s wr, 236 op/s 2024-03-18T23:12:51.769 INFO:tasks.workunit.client.0.smithi049.stdout: 1 494994 10.41 MB/sec execute 113 sec latency 22.689 ms 2024-03-18T23:12:52.769 INFO:tasks.workunit.client.0.smithi049.stdout: 1 497363 10.46 MB/sec execute 114 sec latency 19.059 ms 2024-03-18T23:12:52.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:12:52 smithi139 ceph-mon[67282]: pgmap v142: 97 pgs: 97 active+clean; 52 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.7 MiB/s wr, 312 op/s 2024-03-18T23:12:52.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:12:52 smithi049 ceph-mon[62991]: pgmap v142: 97 pgs: 97 active+clean; 52 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.7 MiB/s wr, 312 op/s 2024-03-18T23:12:52.961 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:12:52 smithi175 ceph-mon[67352]: pgmap v142: 97 pgs: 97 active+clean; 52 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.7 MiB/s wr, 312 op/s 2024-03-18T23:12:53.769 INFO:tasks.workunit.client.0.smithi049.stdout: 1 498959 10.42 MB/sec execute 115 sec latency 18.345 ms 2024-03-18T23:12:54.769 INFO:tasks.workunit.client.0.smithi049.stdout: 1 501022 10.44 MB/sec execute 116 sec latency 80.821 ms 2024-03-18T23:12:54.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:12:54 smithi139 ceph-mon[67282]: pgmap v143: 97 pgs: 97 active+clean; 52 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 822 KiB/s rd, 1.8 MiB/s wr, 235 op/s 2024-03-18T23:12:54.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:12:54 smithi049 ceph-mon[62991]: pgmap v143: 97 pgs: 97 active+clean; 52 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 822 KiB/s rd, 1.8 MiB/s wr, 235 op/s 2024-03-18T23:12:54.961 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:12:54 smithi175 ceph-mon[67352]: pgmap v143: 97 pgs: 97 active+clean; 52 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 822 KiB/s rd, 1.8 MiB/s wr, 235 op/s 2024-03-18T23:12:55.769 INFO:tasks.workunit.client.0.smithi049.stdout: 1 502550 10.41 MB/sec execute 117 sec latency 27.872 ms 2024-03-18T23:12:56.684 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:56.683+0000 7fd294052640 1 -- 172.21.15.49:0/880274312 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd28c068ea0 msgr2=0x7fd28c069300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:12:56.685 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:56.683+0000 7fd294052640 1 --2- 172.21.15.49:0/880274312 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd28c068ea0 0x7fd28c069300 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fd280009900 tx=0x7fd28002bc30 comp rx=0 tx=0).stop 2024-03-18T23:12:56.685 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:56.683+0000 7fd294052640 1 -- 172.21.15.49:0/880274312 shutdown_connections 2024-03-18T23:12:56.685 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:56.683+0000 7fd294052640 1 --2- 172.21.15.49:0/880274312 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd28c069840 0x7fd28c10e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:56.685 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:56.683+0000 7fd294052640 1 --2- 172.21.15.49:0/880274312 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd28c068ea0 0x7fd28c069300 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:56.685 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:56.683+0000 7fd294052640 1 --2- 172.21.15.49:0/880274312 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd28c107910 0x7fd28c107cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:56.685 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:56.683+0000 7fd294052640 1 -- 172.21.15.49:0/880274312 >> 172.21.15.49:0/880274312 conn(0x7fd28c077570 msgr2=0x7fd28c077980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:12:56.685 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:56.683+0000 7fd294052640 1 -- 172.21.15.49:0/880274312 shutdown_connections 2024-03-18T23:12:56.685 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:56.683+0000 7fd294052640 1 -- 172.21.15.49:0/880274312 wait complete. 2024-03-18T23:12:56.687 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:56.684+0000 7fd294052640 1 Processor -- start 2024-03-18T23:12:56.687 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:56.684+0000 7fd294052640 1 -- start start 2024-03-18T23:12:56.687 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:56.684+0000 7fd294052640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd28c068ea0 0x7fd28c100a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:12:56.688 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:56.684+0000 7fd294052640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd28c069840 0x7fd28c100f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:12:56.688 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:56.684+0000 7fd294052640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd28c107910 0x7fd28c1014c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:12:56.688 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:56.684+0000 7fd294052640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fd28c111c90 con 0x7fd28c069840 2024-03-18T23:12:56.688 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:56.685+0000 7fd294052640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fd28c111b10 con 0x7fd28c068ea0 2024-03-18T23:12:56.688 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:56.685+0000 7fd294052640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd28c111e10 con 0x7fd28c107910 2024-03-18T23:12:56.688 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:56.685+0000 7fd2915ed640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd28c069840 0x7fd28c100f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:12:56.688 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:56.685+0000 7fd291dee640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd28c068ea0 0x7fd28c100a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:12:56.688 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:56.685+0000 7fd2915ed640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd28c069840 0x7fd28c100f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:56076/0 (socket says 172.21.15.49:56076) 2024-03-18T23:12:56.688 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:56.685+0000 7fd2915ed640 1 -- 172.21.15.49:0/726543773 learned_addr learned my addr 172.21.15.49:0/726543773 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:12:56.688 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:56.685+0000 7fd2925ef640 1 --2- 172.21.15.49:0/726543773 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd28c107910 0x7fd28c1014c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:12:56.688 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:56.685+0000 7fd2915ed640 1 -- 172.21.15.49:0/726543773 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd28c107910 msgr2=0x7fd28c1014c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:12:56.688 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:56.685+0000 7fd2915ed640 1 --2- 172.21.15.49:0/726543773 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd28c107910 0x7fd28c1014c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:56.688 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:56.685+0000 7fd2915ed640 1 -- 172.21.15.49:0/726543773 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd28c068ea0 msgr2=0x7fd28c100a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:12:56.688 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:56.685+0000 7fd2915ed640 1 --2- 172.21.15.49:0/726543773 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd28c068ea0 0x7fd28c100a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:56.689 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:56.685+0000 7fd2915ed640 1 -- 172.21.15.49:0/726543773 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd28c1a6570 con 0x7fd28c069840 2024-03-18T23:12:56.689 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:56.685+0000 7fd291dee640 1 --2- 172.21.15.49:0/726543773 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd28c068ea0 0x7fd28c100a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:12:56.689 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:56.685+0000 7fd2915ed640 1 --2- 172.21.15.49:0/726543773 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd28c069840 0x7fd28c100f80 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fd280006010 tx=0x7fd28002cb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:12:56.689 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:56.685+0000 7fd27effd640 1 -- 172.21.15.49:0/726543773 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd280041070 con 0x7fd28c069840 2024-03-18T23:12:56.689 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:56.685+0000 7fd27effd640 1 -- 172.21.15.49:0/726543773 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd280004450 con 0x7fd28c069840 2024-03-18T23:12:56.689 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:56.685+0000 7fd27effd640 1 -- 172.21.15.49:0/726543773 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd280003e00 con 0x7fd28c069840 2024-03-18T23:12:56.689 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:56.685+0000 7fd294052640 1 -- 172.21.15.49:0/726543773 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd28c1a6800 con 0x7fd28c069840 2024-03-18T23:12:56.689 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:56.685+0000 7fd294052640 1 -- 172.21.15.49:0/726543773 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd28c1a6cf0 con 0x7fd28c069840 2024-03-18T23:12:56.697 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:56.688+0000 7fd27effd640 1 -- 172.21.15.49:0/726543773 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 25) ==== 137421+0+0 (secure 0 0 0) 0x7fd280044070 con 0x7fd28c069840 2024-03-18T23:12:56.698 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:56.688+0000 7fd296c3e740 1 Processor -- start 2024-03-18T23:12:56.698 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:56.688+0000 7fd296c3e740 1 -- start start 2024-03-18T23:12:56.698 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:56.688+0000 7fd296c3e740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56288f7c7ef0 0x56288f7c82d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:12:56.698 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:56.688+0000 7fd296c3e740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56288f7c8810 0x56288f7d10a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:12:56.698 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:56.688+0000 7fd296c3e740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56288f7d15e0 0x56288f7d39d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:12:56.698 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:56.688+0000 7fd296c3e740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x56288f72bbe0 con 0x56288f7d15e0 2024-03-18T23:12:56.698 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:56.688+0000 7fd296c3e740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x56288f6d4960 con 0x56288f7c7ef0 2024-03-18T23:12:56.698 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:56.688+0000 7fd296c3e740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x56288f727c80 con 0x56288f7c8810 2024-03-18T23:12:56.698 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:56.688+0000 7fd2915ed640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56288f7d15e0 0x56288f7d39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:12:56.698 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:56.688+0000 7fd27effd640 1 --2- 172.21.15.49:0/726543773 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7fd2780a2a10 0x7fd2780a4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:12:56.698 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:56.688+0000 7fd2925ef640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56288f7c7ef0 0x56288f7c82d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:12:56.698 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:56.688+0000 7fd291dee640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56288f7c8810 0x56288f7d10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:12:56.699 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:56.689+0000 7fd2915ed640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56288f7d15e0 0x56288f7d39d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:56082/0 (socket says 172.21.15.49:56082) 2024-03-18T23:12:56.699 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:56.689+0000 7fd2915ed640 1 -- 172.21.15.49:0/217341573 learned_addr learned my addr 172.21.15.49:0/217341573 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:12:56.699 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:56.689+0000 7fd291dee640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56288f7c8810 0x56288f7d10a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.49:37478/0 (socket says 172.21.15.49:37478) 2024-03-18T23:12:56.699 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:56.689+0000 7fd2925ef640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56288f7c7ef0 0x56288f7c82d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.49:39746/0 (socket says 172.21.15.49:39746) 2024-03-18T23:12:56.699 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:56.689+0000 7fd2925ef640 1 --2- 172.21.15.49:0/726543773 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7fd2780a2a10 0x7fd2780a4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:12:56.699 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:56.689+0000 7fd27effd640 1 -- 172.21.15.49:0/726543773 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8469+0+0 (secure 0 0 0) 0x7fd2800f07b0 con 0x7fd28c069840 2024-03-18T23:12:56.699 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:56.689+0000 7fd291dee640 1 -- 172.21.15.49:0/217341573 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56288f7c7ef0 msgr2=0x56288f7c82d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:12:56.699 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:56.689+0000 7fd291dee640 1 --2- 172.21.15.49:0/217341573 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56288f7c7ef0 0x56288f7c82d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:56.699 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:56.689+0000 7fd291dee640 1 -- 172.21.15.49:0/217341573 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56288f7d15e0 msgr2=0x56288f7d39d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:12:56.699 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:56.689+0000 7fd291dee640 1 --2- 172.21.15.49:0/217341573 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56288f7d15e0 0x56288f7d39d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:56.699 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:56.689+0000 7fd291dee640 1 -- 172.21.15.49:0/217341573 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56288f6cbd20 con 0x56288f7c8810 2024-03-18T23:12:56.699 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:56.689+0000 7fd2915ed640 1 --2- 172.21.15.49:0/217341573 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56288f7d15e0 0x56288f7d39d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:12:56.699 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:56.689+0000 7fd291dee640 1 --2- 172.21.15.49:0/217341573 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56288f7c8810 0x56288f7d10a0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fd288009fb0 tx=0x7fd2880118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3ac7629805170787 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:12:56.699 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:56.689+0000 7fd2925ef640 1 --2- 172.21.15.49:0/726543773 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7fd2780a2a10 0x7fd2780a4ed0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fd284009fd0 tx=0x7fd284004ab0 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:12:56.700 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:56.689+0000 7fd277fff640 1 -- 172.21.15.49:0/217341573 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd288004310 con 0x56288f7c8810 2024-03-18T23:12:56.700 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:56.689+0000 7fd277fff640 1 -- 172.21.15.49:0/217341573 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd2880044b0 con 0x56288f7c8810 2024-03-18T23:12:56.700 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:56.689+0000 7fd277fff640 1 -- 172.21.15.49:0/217341573 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2880035f0 con 0x56288f7c8810 2024-03-18T23:12:56.700 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:56.690+0000 7fd296c3e740 1 -- 172.21.15.49:0/217341573 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56288f7c8810 msgr2=0x56288f7d10a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:12:56.700 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:56.690+0000 7fd296c3e740 1 --2- 172.21.15.49:0/217341573 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56288f7c8810 0x56288f7d10a0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fd288009fb0 tx=0x7fd2880118d0 comp rx=0 tx=0).stop 2024-03-18T23:12:56.700 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:56.690+0000 7fd296c3e740 1 -- 172.21.15.49:0/217341573 shutdown_connections 2024-03-18T23:12:56.700 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:56.690+0000 7fd296c3e740 1 --2- 172.21.15.49:0/217341573 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56288f7d15e0 0x56288f7d39d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:56.700 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:56.690+0000 7fd296c3e740 1 --2- 172.21.15.49:0/217341573 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56288f7c8810 0x56288f7d10a0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:56.700 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:56.690+0000 7fd296c3e740 1 --2- 172.21.15.49:0/217341573 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56288f7c7ef0 0x56288f7c82d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:56.700 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:56.690+0000 7fd296c3e740 1 -- 172.21.15.49:0/217341573 >> 172.21.15.49:0/217341573 conn(0x56288f7c2810 msgr2=0x56288f7c2bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:12:56.700 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:56.690+0000 7fd296c3e740 1 -- 172.21.15.49:0/217341573 shutdown_connections 2024-03-18T23:12:56.700 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:56.690+0000 7fd296c3e740 1 -- 172.21.15.49:0/217341573 wait complete. 2024-03-18T23:12:56.700 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:56.691+0000 7fd296c3e740 1 -- start start 2024-03-18T23:12:56.700 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:56.691+0000 7fd296c3e740 1 Processor -- start 2024-03-18T23:12:56.701 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:56.691+0000 7fd296c3e740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56288f7c7ef0 0x56288f7d0f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:12:56.701 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:56.691+0000 7fd296c3e740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56288f7c8810 0x56288f7ce290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:12:56.701 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:56.691+0000 7fd296c3e740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56288f7d15e0 0x56288f7ce7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:12:56.701 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:56.691+0000 7fd296c3e740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x56288f7d4210 con 0x56288f7c7ef0 2024-03-18T23:12:56.701 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:56.691+0000 7fd296c3e740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x56288f6e7db0 con 0x56288f7c8810 2024-03-18T23:12:56.701 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:56.691+0000 7fd296c3e740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x56288f7d4390 con 0x56288f7d15e0 2024-03-18T23:12:56.701 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:56.691+0000 7fd296c3e740 10 client.0 _ll_register_callbacks cb 0x56288f6ece70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T23:12:56.701 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:56.691+0000 7fd2915ed640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56288f7c8810 0x56288f7ce290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:12:56.701 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:56.691+0000 7fd291dee640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56288f7c7ef0 0x56288f7d0f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:12:56.701 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:56.691+0000 7fd2915ed640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56288f7c8810 0x56288f7ce290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.49:39754/0 (socket says 172.21.15.49:39754) 2024-03-18T23:12:56.701 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:56.691+0000 7fd2915ed640 1 -- 172.21.15.49:0/8167051 learned_addr learned my addr 172.21.15.49:0/8167051 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:12:56.701 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:56.692+0000 7fd2925ef640 1 --2- 172.21.15.49:0/8167051 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56288f7d15e0 0x56288f7ce7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:12:56.701 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:56.692+0000 7fd2915ed640 1 -- 172.21.15.49:0/8167051 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56288f7d15e0 msgr2=0x56288f7ce7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:12:56.701 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:56.692+0000 7fd2915ed640 1 --2- 172.21.15.49:0/8167051 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56288f7d15e0 0x56288f7ce7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:56.702 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:56.692+0000 7fd2915ed640 1 -- 172.21.15.49:0/8167051 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56288f7c7ef0 msgr2=0x56288f7d0f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:12:56.702 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:56.692+0000 7fd2915ed640 1 --2- 172.21.15.49:0/8167051 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56288f7c7ef0 0x56288f7d0f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:56.702 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:56.692+0000 7fd2915ed640 1 -- 172.21.15.49:0/8167051 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56288f6e6c60 con 0x56288f7c8810 2024-03-18T23:12:56.702 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:56.692+0000 7fd2915ed640 1 --2- 172.21.15.49:0/8167051 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56288f7c8810 0x56288f7ce290 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fd280004e00 tx=0x7fd280005770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=db87d978f99139a6 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:12:56.702 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:56.692+0000 7fd2757fa640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-03-18T23:12:56.702 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:56.692+0000 7fd2757fa640 1 -- 172.21.15.49:0/8167051 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd28004b040 con 0x56288f7c8810 2024-03-18T23:12:56.702 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:56.692+0000 7fd2757fa640 1 -- 172.21.15.49:0/8167051 <== mon.2 v2:172.21.15.139:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8469+0+0 (secure 0 0 0) 0x7fd2800c5260 con 0x56288f7c8810 2024-03-18T23:12:56.702 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:56.692+0000 7fd296c3e740 1 -- 172.21.15.49:0/8167051 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd2880044b0 con 0x56288f7c8810 2024-03-18T23:12:56.702 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:56.693+0000 7fd2757fa640 1 -- 172.21.15.49:0/8167051 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fd2800bc960 con 0x56288f7c8810 2024-03-18T23:12:56.702 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:56.693+0000 7fd296c3e740 10 client.25652 fetch_fsmap learned FSMap version 32 2024-03-18T23:12:56.702 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:56.693+0000 7fd296c3e740 1 -- 172.21.15.49:0/8167051 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x56288f440210 con 0x56288f7c8810 2024-03-18T23:12:56.702 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:56.693+0000 7fd2757fa640 1 -- 172.21.15.49:0/8167051 <== mon.2 v2:172.21.15.139:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd2880044b0 con 0x56288f7c8810 2024-03-18T23:12:56.702 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:56.693+0000 7fd2757fa640 1 -- 172.21.15.49:0/8167051 <== mon.2 v2:172.21.15.139:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7fd2800bc450 con 0x56288f7c8810 2024-03-18T23:12:56.703 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:56.693+0000 7fd296c3e740 10 client.25652 fetch_fsmap finished waiting for FSMap version 32 2024-03-18T23:12:56.703 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:56.693+0000 7fd296c3e740 10 client.25652 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T23:12:56.703 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:56.693+0000 7fd296c3e740 20 client.25652 populate_metadata read hostname 'smithi049' 2024-03-18T23:12:56.703 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:56.693+0000 7fd296c3e740 1 --2- 172.21.15.49:0/8167051 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x56288f7e13d0 0x56288f7e37c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:12:56.703 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:56.693+0000 7fd296c3e740 4 client.25652 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T23:12:56.703 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:56.693+0000 7fd296c3e740 1 -- 172.21.15.49:0/8167051 --> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56288f739120 con 0x56288f7e13d0 2024-03-18T23:12:56.703 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:56.693+0000 7fd291dee640 1 --2- 172.21.15.49:0/8167051 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x56288f7e13d0 0x56288f7e37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:12:56.703 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:56.694+0000 7fd2757fa640 10 client.25652 ms_handle_connect on v2:172.21.15.139:6838/4280128209 2024-03-18T23:12:56.703 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:56.694+0000 7fd291dee640 1 --2- 172.21.15.49:0/8167051 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x56288f7e13d0 0x56288f7e37c0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e3014e93499968e server_cookie=550879796a9da771 in_seq=0 out_seq=0 2024-03-18T23:12:56.703 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:56.696+0000 7fd2757fa640 1 -- 172.21.15.49:0/8167051 <== mds.0 v2:172.21.15.139:6838/4280128209 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x56288f739120 con 0x56288f7e13d0 2024-03-18T23:12:56.703 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:56.696+0000 7fd2757fa640 10 client.25652 handle_command_reply: tid=0 2024-03-18T23:12:56.703 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:56.696+0000 7fd296c3e740 1 client.25652 shutdown 2024-03-18T23:12:56.703 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:56.696+0000 7fd296c3e740 1 -- 172.21.15.49:0/8167051 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56288f7c8810 msgr2=0x56288f7ce290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:12:56.703 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:56.696+0000 7fd296c3e740 1 --2- 172.21.15.49:0/8167051 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56288f7c8810 0x56288f7ce290 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fd280004e00 tx=0x7fd280005770 comp rx=0 tx=0).stop 2024-03-18T23:12:56.704 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:56.697+0000 7fd296c3e740 1 -- 172.21.15.49:0/8167051 shutdown_connections 2024-03-18T23:12:56.704 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:56.697+0000 7fd296c3e740 1 --2- 172.21.15.49:0/8167051 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x56288f7e13d0 0x56288f7e37c0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:56.704 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:56.697+0000 7fd296c3e740 1 --2- 172.21.15.49:0/8167051 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56288f7d15e0 0x56288f7ce7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:56.704 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:56.697+0000 7fd296c3e740 1 --2- 172.21.15.49:0/8167051 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56288f7c8810 0x56288f7ce290 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:56.704 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:56.697+0000 7fd2757fa640 0 client.25652 ms_handle_reset on v2:172.21.15.139:6838/4280128209 2024-03-18T23:12:56.704 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:56.697+0000 7fd296c3e740 1 --2- 172.21.15.49:0/8167051 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56288f7c7ef0 0x56288f7d0f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:56.704 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:56.697+0000 7fd296c3e740 1 -- 172.21.15.49:0/8167051 >> 172.21.15.49:0/8167051 conn(0x56288f7c2810 msgr2=0x56288f7c4560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:12:56.704 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:56.697+0000 7fd296c3e740 1 -- 172.21.15.49:0/8167051 shutdown_connections 2024-03-18T23:12:56.704 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:56.697+0000 7fd296c3e740 1 -- 172.21.15.49:0/8167051 wait complete. 2024-03-18T23:12:56.704 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:56.697+0000 7fd296c3e740 20 client.25652 trim_cache size 0 max 16384 2024-03-18T23:12:56.714 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:56.712+0000 7fd296c3e740 1 Processor -- start 2024-03-18T23:12:56.715 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:56.712+0000 7fd296c3e740 1 -- start start 2024-03-18T23:12:56.715 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:56.713+0000 7fd296c3e740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56288f7c7ef0 0x56288f7dcfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:12:56.715 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:56.713+0000 7fd296c3e740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56288f7c8810 0x56288f7c1930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:12:56.715 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:56.713+0000 7fd296c3e740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56288f7d15e0 0x56288f7e90d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:12:56.715 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:56.713+0000 7fd296c3e740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fd28c1af5f0 con 0x56288f7c7ef0 2024-03-18T23:12:56.715 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:56.713+0000 7fd296c3e740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fd28c1b1130 con 0x56288f7c8810 2024-03-18T23:12:56.715 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:56.713+0000 7fd296c3e740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd28c1afcc0 con 0x56288f7d15e0 2024-03-18T23:12:56.715 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:56.713+0000 7fd291dee640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56288f7c7ef0 0x56288f7dcfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:12:56.715 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:56.713+0000 7fd291dee640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56288f7c7ef0 0x56288f7dcfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:56090/0 (socket says 172.21.15.49:56090) 2024-03-18T23:12:56.715 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:56.713+0000 7fd291dee640 1 -- 172.21.15.49:0/3106620598 learned_addr learned my addr 172.21.15.49:0/3106620598 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:12:56.715 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:56.713+0000 7fd2915ed640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56288f7c8810 0x56288f7c1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:12:56.715 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:56.713+0000 7fd2925ef640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56288f7d15e0 0x56288f7e90d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:12:56.715 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:56.713+0000 7fd291dee640 1 -- 172.21.15.49:0/3106620598 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56288f7d15e0 msgr2=0x56288f7e90d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:12:56.715 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:56.713+0000 7fd291dee640 1 --2- 172.21.15.49:0/3106620598 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56288f7d15e0 0x56288f7e90d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:56.716 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:56.713+0000 7fd291dee640 1 -- 172.21.15.49:0/3106620598 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56288f7c8810 msgr2=0x56288f7c1930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:12:56.716 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:56.713+0000 7fd291dee640 1 --2- 172.21.15.49:0/3106620598 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56288f7c8810 0x56288f7c1930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:56.716 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:56.713+0000 7fd291dee640 1 -- 172.21.15.49:0/3106620598 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56288f7e3b30 con 0x56288f7c7ef0 2024-03-18T23:12:56.716 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:56.713+0000 7fd291dee640 1 --2- 172.21.15.49:0/3106620598 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56288f7c7ef0 0x56288f7dcfb0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fd288009d40 tx=0x7fd288004ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91f3747b44fd3f74 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:12:56.716 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:56.713+0000 7fd2777fe640 1 -- 172.21.15.49:0/3106620598 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd288033070 con 0x56288f7c7ef0 2024-03-18T23:12:56.716 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:56.714+0000 7fd2777fe640 1 -- 172.21.15.49:0/3106620598 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd288004940 con 0x56288f7c7ef0 2024-03-18T23:12:56.716 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:56.714+0000 7fd2777fe640 1 -- 172.21.15.49:0/3106620598 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd28800f6e0 con 0x56288f7c7ef0 2024-03-18T23:12:56.720 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:56.714+0000 7fd296c3e740 1 -- 172.21.15.49:0/3106620598 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56288f7c7ef0 msgr2=0x56288f7dcfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:12:56.720 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:56.714+0000 7fd296c3e740 1 --2- 172.21.15.49:0/3106620598 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56288f7c7ef0 0x56288f7dcfb0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fd288009d40 tx=0x7fd288004ba0 comp rx=0 tx=0).stop 2024-03-18T23:12:56.720 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:56.714+0000 7fd296c3e740 1 -- 172.21.15.49:0/3106620598 shutdown_connections 2024-03-18T23:12:56.720 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:56.714+0000 7fd296c3e740 1 --2- 172.21.15.49:0/3106620598 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56288f7d15e0 0x56288f7e90d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:56.720 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:56.714+0000 7fd296c3e740 1 --2- 172.21.15.49:0/3106620598 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56288f7c8810 0x56288f7c1930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:56.720 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:56.714+0000 7fd296c3e740 1 --2- 172.21.15.49:0/3106620598 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56288f7c7ef0 0x56288f7dcfb0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:56.720 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:56.714+0000 7fd296c3e740 1 -- 172.21.15.49:0/3106620598 >> 172.21.15.49:0/3106620598 conn(0x56288f7c2810 msgr2=0x56288f7c1220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:12:56.720 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:56.714+0000 7fd296c3e740 1 -- 172.21.15.49:0/3106620598 shutdown_connections 2024-03-18T23:12:56.720 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:56.714+0000 7fd296c3e740 1 -- 172.21.15.49:0/3106620598 wait complete. 2024-03-18T23:12:56.720 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:56.715+0000 7fd296c3e740 1 -- start start 2024-03-18T23:12:56.720 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:56.715+0000 7fd296c3e740 1 Processor -- start 2024-03-18T23:12:56.720 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:56.715+0000 7fd296c3e740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56288f7c7ef0 0x56288f7d4f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:12:56.721 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:56.715+0000 7fd296c3e740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56288f7c8810 0x56288f7d5440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:12:56.721 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:56.715+0000 7fd296c3e740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56288f7d15e0 0x56288f7d5980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:12:56.721 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:56.715+0000 7fd296c3e740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fd28c1b0a60 con 0x56288f7c7ef0 2024-03-18T23:12:56.721 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:56.715+0000 7fd296c3e740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fd28c1b0390 con 0x56288f7c8810 2024-03-18T23:12:56.721 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:56.715+0000 7fd296c3e740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd28c1ad5d0 con 0x56288f7d15e0 2024-03-18T23:12:56.721 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:56.715+0000 7fd296c3e740 10 client.0 _ll_register_callbacks cb 0x7fd280005670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T23:12:56.721 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:56.715+0000 7fd291dee640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56288f7c7ef0 0x56288f7d4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:12:56.721 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:56.715+0000 7fd291dee640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56288f7c7ef0 0x56288f7d4f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:56098/0 (socket says 172.21.15.49:56098) 2024-03-18T23:12:56.721 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:56.715+0000 7fd291dee640 1 -- 172.21.15.49:0/3484723508 learned_addr learned my addr 172.21.15.49:0/3484723508 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:12:56.721 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:56.715+0000 7fd2915ed640 1 --2- 172.21.15.49:0/3484723508 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56288f7c8810 0x56288f7d5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:12:56.721 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:56.715+0000 7fd2925ef640 1 --2- 172.21.15.49:0/3484723508 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56288f7d15e0 0x56288f7d5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:12:56.721 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:56.716+0000 7fd291dee640 1 -- 172.21.15.49:0/3484723508 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56288f7d15e0 msgr2=0x56288f7d5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:12:56.721 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:56.716+0000 7fd291dee640 1 --2- 172.21.15.49:0/3484723508 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56288f7d15e0 0x56288f7d5980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:56.721 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:56.716+0000 7fd291dee640 1 -- 172.21.15.49:0/3484723508 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56288f7c8810 msgr2=0x56288f7d5440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:12:56.722 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:56.716+0000 7fd291dee640 1 --2- 172.21.15.49:0/3484723508 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56288f7c8810 0x56288f7d5440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:56.722 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:56.716+0000 7fd291dee640 1 -- 172.21.15.49:0/3484723508 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56288f50c620 con 0x56288f7c7ef0 2024-03-18T23:12:56.722 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:56.716+0000 7fd2915ed640 1 --2- 172.21.15.49:0/3484723508 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56288f7c8810 0x56288f7d5440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:12:56.722 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:56.716+0000 7fd291dee640 1 --2- 172.21.15.49:0/3484723508 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56288f7c7ef0 0x56288f7d4f00 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fd288005de0 tx=0x7fd288011dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26d36155e3e98e56 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:12:56.722 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:56.716+0000 7fd2757fa640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-18T23:12:56.722 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:56.716+0000 7fd2757fa640 1 -- 172.21.15.49:0/3484723508 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd288033040 con 0x56288f7c7ef0 2024-03-18T23:12:56.722 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:56.716+0000 7fd2757fa640 1 -- 172.21.15.49:0/3484723508 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8469+0+0 (secure 0 0 0) 0x7fd28802e3e0 con 0x56288f7c7ef0 2024-03-18T23:12:56.722 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:56.716+0000 7fd296c3e740 1 -- 172.21.15.49:0/3484723508 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd288004940 con 0x56288f7c7ef0 2024-03-18T23:12:56.722 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:56.716+0000 7fd2757fa640 1 -- 172.21.15.49:0/3484723508 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fd28802e5e0 con 0x56288f7c7ef0 2024-03-18T23:12:56.722 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:56.716+0000 7fd296c3e740 10 client.15840 fetch_fsmap learned FSMap version 32 2024-03-18T23:12:56.722 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:56.716+0000 7fd296c3e740 1 -- 172.21.15.49:0/3484723508 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x56288f847850 con 0x56288f7c7ef0 2024-03-18T23:12:56.722 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:56.716+0000 7fd2757fa640 1 -- 172.21.15.49:0/3484723508 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd288004940 con 0x56288f7c7ef0 2024-03-18T23:12:56.722 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:56.717+0000 7fd2757fa640 1 -- 172.21.15.49:0/3484723508 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7fd288036070 con 0x56288f7c7ef0 2024-03-18T23:12:56.723 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:56.717+0000 7fd296c3e740 10 client.15840 fetch_fsmap finished waiting for FSMap version 32 2024-03-18T23:12:56.723 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:56.717+0000 7fd296c3e740 10 client.15840 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T23:12:56.723 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:56.717+0000 7fd296c3e740 20 client.15840 populate_metadata read hostname 'smithi049' 2024-03-18T23:12:56.723 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:56.717+0000 7fd296c3e740 1 --2- 172.21.15.49:0/3484723508 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x56288f847d20 0x56288f84a110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:12:56.723 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:56.717+0000 7fd296c3e740 4 client.15840 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T23:12:56.723 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:56.717+0000 7fd296c3e740 1 -- 172.21.15.49:0/3484723508 --> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56288f733470 con 0x56288f847d20 2024-03-18T23:12:56.723 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:56.717+0000 7fd2915ed640 1 --2- 172.21.15.49:0/3484723508 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x56288f847d20 0x56288f84a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:12:56.723 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:56.717+0000 7fd2757fa640 10 client.15840 ms_handle_connect on v2:172.21.15.139:6838/4280128209 2024-03-18T23:12:56.723 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:56.717+0000 7fd2915ed640 1 --2- 172.21.15.49:0/3484723508 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x56288f847d20 0x56288f84a110 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a4d0b51cf104e7a2 server_cookie=2d56d3bc89735bcb in_seq=0 out_seq=0 2024-03-18T23:12:56.769 INFO:tasks.workunit.client.0.smithi049.stdout: 1 504895 10.44 MB/sec execute 118 sec latency 40.388 ms 2024-03-18T23:12:56.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:12:56 smithi139 ceph-mon[67282]: pgmap v144: 97 pgs: 97 active+clean; 52 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 822 KiB/s rd, 1.8 MiB/s wr, 235 op/s 2024-03-18T23:12:56.879 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:12:56 smithi049 ceph-mon[62991]: pgmap v144: 97 pgs: 97 active+clean; 52 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 822 KiB/s rd, 1.8 MiB/s wr, 235 op/s 2024-03-18T23:12:56.896 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:56.895+0000 7fd2757fa640 1 -- 172.21.15.49:0/3484723508 <== mds.0 v2:172.21.15.139:6838/4280128209 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56288f733470 con 0x56288f847d20 2024-03-18T23:12:56.896 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:56.895+0000 7fd2757fa640 10 client.15840 handle_command_reply: tid=0 2024-03-18T23:12:56.896 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:56.895+0000 7fd296c3e740 1 client.15840 shutdown 2024-03-18T23:12:56.897 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:56.895+0000 7fd296c3e740 1 -- 172.21.15.49:0/3484723508 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56288f7c7ef0 msgr2=0x56288f7d4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:12:56.897 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:56.895+0000 7fd296c3e740 1 --2- 172.21.15.49:0/3484723508 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56288f7c7ef0 0x56288f7d4f00 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fd288005de0 tx=0x7fd288011dd0 comp rx=0 tx=0).stop 2024-03-18T23:12:56.897 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:56.895+0000 7fd296c3e740 1 -- 172.21.15.49:0/3484723508 shutdown_connections 2024-03-18T23:12:56.897 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:56.895+0000 7fd296c3e740 1 --2- 172.21.15.49:0/3484723508 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x56288f847d20 0x56288f84a110 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:56.897 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:56.895+0000 7fd296c3e740 1 --2- 172.21.15.49:0/3484723508 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56288f7d15e0 0x56288f7d5980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:56.897 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:56.895+0000 7fd296c3e740 1 --2- 172.21.15.49:0/3484723508 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56288f7c8810 0x56288f7d5440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:56.897 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:56.895+0000 7fd2757fa640 0 client.15840 ms_handle_reset on v2:172.21.15.139:6838/4280128209 2024-03-18T23:12:56.897 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:56.895+0000 7fd296c3e740 1 --2- 172.21.15.49:0/3484723508 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56288f7c7ef0 0x56288f7d4f00 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:56.897 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:56.895+0000 7fd296c3e740 1 -- 172.21.15.49:0/3484723508 >> 172.21.15.49:0/3484723508 conn(0x56288f7c2810 msgr2=0x56288f7c1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:12:56.897 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:56.895+0000 7fd296c3e740 1 -- 172.21.15.49:0/3484723508 shutdown_connections 2024-03-18T23:12:56.897 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:56.895+0000 7fd296c3e740 1 -- 172.21.15.49:0/3484723508 wait complete. 2024-03-18T23:12:56.897 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:56.895+0000 7fd296c3e740 20 client.15840 trim_cache size 0 max 16384 2024-03-18T23:12:56.898 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:{ 2024-03-18T23:12:56.898 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "message": "", 2024-03-18T23:12:56.898 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "return_code": 0 2024-03-18T23:12:56.898 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:} 2024-03-18T23:12:56.899 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:56.896+0000 7fd294052640 1 -- 172.21.15.49:0/726543773 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7fd2780a2a10 msgr2=0x7fd2780a4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:12:56.899 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:56.896+0000 7fd294052640 1 --2- 172.21.15.49:0/726543773 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7fd2780a2a10 0x7fd2780a4ed0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fd284009fd0 tx=0x7fd284004ab0 comp rx=0 tx=0).stop 2024-03-18T23:12:56.899 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:56.896+0000 7fd294052640 1 -- 172.21.15.49:0/726543773 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd28c069840 msgr2=0x7fd28c100f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:12:56.899 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:56.896+0000 7fd294052640 1 --2- 172.21.15.49:0/726543773 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd28c069840 0x7fd28c100f80 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fd280006010 tx=0x7fd28002cb30 comp rx=0 tx=0).stop 2024-03-18T23:12:56.899 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:56.896+0000 7fd294052640 1 -- 172.21.15.49:0/726543773 shutdown_connections 2024-03-18T23:12:56.899 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:56.896+0000 7fd294052640 1 --2- 172.21.15.49:0/726543773 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7fd2780a2a10 0x7fd2780a4ed0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:56.899 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:56.896+0000 7fd294052640 1 --2- 172.21.15.49:0/726543773 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd28c107910 0x7fd28c1014c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:56.899 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:56.896+0000 7fd294052640 1 --2- 172.21.15.49:0/726543773 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd28c069840 0x7fd28c100f80 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:56.899 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:56.896+0000 7fd294052640 1 --2- 172.21.15.49:0/726543773 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd28c068ea0 0x7fd28c100a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:56.899 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:56.896+0000 7fd294052640 1 -- 172.21.15.49:0/726543773 >> 172.21.15.49:0/726543773 conn(0x7fd28c077570 msgr2=0x7fd28c0fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:12:56.899 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:56.896+0000 7fd294052640 1 -- 172.21.15.49:0/726543773 shutdown_connections 2024-03-18T23:12:56.899 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:12:56.896+0000 7fd294052640 1 -- 172.21.15.49:0/726543773 wait complete. 2024-03-18T23:12:56.961 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:12:56 smithi175 ceph-mon[67352]: pgmap v144: 97 pgs: 97 active+clean; 52 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 822 KiB/s rd, 1.8 MiB/s wr, 235 op/s 2024-03-18T23:12:57.769 INFO:tasks.workunit.client.0.smithi049.stdout: 1 506470 10.40 MB/sec execute 119 sec latency 71.796 ms 2024-03-18T23:12:58.769 INFO:tasks.workunit.client.0.smithi049.stdout: 1 508729 10.43 MB/sec execute 120 sec latency 26.754 ms 2024-03-18T23:12:58.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:12:58 smithi139 ceph-mon[67282]: pgmap v145: 97 pgs: 97 active+clean; 57 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.1 MiB/s wr, 322 op/s 2024-03-18T23:12:58.879 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:12:58 smithi049 ceph-mon[62991]: pgmap v145: 97 pgs: 97 active+clean; 57 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.1 MiB/s wr, 322 op/s 2024-03-18T23:12:58.961 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:12:58 smithi175 ceph-mon[67352]: pgmap v145: 97 pgs: 97 active+clean; 57 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.1 MiB/s wr, 322 op/s 2024-03-18T23:12:59.769 INFO:tasks.workunit.client.0.smithi049.stdout: 1 510800 10.43 MB/sec execute 121 sec latency 44.069 ms 2024-03-18T23:13:00.770 INFO:tasks.workunit.client.0.smithi049.stdout: 1 511973 10.41 MB/sec execute 122 sec latency 79.403 ms 2024-03-18T23:13:00.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:13:00 smithi139 ceph-mon[67282]: pgmap v146: 97 pgs: 97 active+clean; 57 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 827 KiB/s rd, 2.4 MiB/s wr, 164 op/s 2024-03-18T23:13:00.879 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:13:00 smithi049 ceph-mon[62991]: pgmap v146: 97 pgs: 97 active+clean; 57 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 827 KiB/s rd, 2.4 MiB/s wr, 164 op/s 2024-03-18T23:13:00.961 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:13:00 smithi175 ceph-mon[67352]: pgmap v146: 97 pgs: 97 active+clean; 57 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 827 KiB/s rd, 2.4 MiB/s wr, 164 op/s 2024-03-18T23:13:01.770 INFO:tasks.workunit.client.0.smithi049.stdout: 1 513721 10.37 MB/sec execute 123 sec latency 19.903 ms 2024-03-18T23:13:02.770 INFO:tasks.workunit.client.0.smithi049.stdout: 1 515883 10.40 MB/sec execute 124 sec latency 18.147 ms 2024-03-18T23:13:02.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:13:02 smithi139 ceph-mon[67282]: pgmap v147: 97 pgs: 97 active+clean; 47 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.7 MiB/s wr, 390 op/s 2024-03-18T23:13:02.879 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:13:02 smithi049 ceph-mon[62991]: pgmap v147: 97 pgs: 97 active+clean; 47 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.7 MiB/s wr, 390 op/s 2024-03-18T23:13:02.961 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:13:02 smithi175 ceph-mon[67352]: pgmap v147: 97 pgs: 97 active+clean; 47 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.7 MiB/s wr, 390 op/s 2024-03-18T23:13:03.770 INFO:tasks.workunit.client.0.smithi049.stdout: 1 518296 10.42 MB/sec execute 125 sec latency 19.278 ms 2024-03-18T23:13:04.770 INFO:tasks.workunit.client.0.smithi049.stdout: 1 520083 10.42 MB/sec execute 126 sec latency 20.305 ms 2024-03-18T23:13:04.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:13:04 smithi139 ceph-mon[67282]: pgmap v148: 97 pgs: 97 active+clean; 47 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 908 KiB/s rd, 2.6 MiB/s wr, 314 op/s 2024-03-18T23:13:04.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:13:04 smithi049 ceph-mon[62991]: pgmap v148: 97 pgs: 97 active+clean; 47 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 908 KiB/s rd, 2.6 MiB/s wr, 314 op/s 2024-03-18T23:13:04.961 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:13:04 smithi175 ceph-mon[67352]: pgmap v148: 97 pgs: 97 active+clean; 47 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 908 KiB/s rd, 2.6 MiB/s wr, 314 op/s 2024-03-18T23:13:05.770 INFO:tasks.workunit.client.0.smithi049.stdout: 1 521592 10.40 MB/sec execute 127 sec latency 184.535 ms 2024-03-18T23:13:06.770 INFO:tasks.workunit.client.0.smithi049.stdout: 1 523158 10.38 MB/sec execute 128 sec latency 25.708 ms 2024-03-18T23:13:06.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:13:06 smithi139 ceph-mon[67282]: pgmap v149: 97 pgs: 97 active+clean; 47 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 908 KiB/s rd, 2.6 MiB/s wr, 314 op/s 2024-03-18T23:13:06.879 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:13:06 smithi049 ceph-mon[62991]: pgmap v149: 97 pgs: 97 active+clean; 47 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 908 KiB/s rd, 2.6 MiB/s wr, 314 op/s 2024-03-18T23:13:06.961 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:13:06 smithi175 ceph-mon[67352]: pgmap v149: 97 pgs: 97 active+clean; 47 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 908 KiB/s rd, 2.6 MiB/s wr, 314 op/s 2024-03-18T23:13:07.770 INFO:tasks.workunit.client.0.smithi049.stdout: 1 525388 10.39 MB/sec execute 129 sec latency 20.235 ms 2024-03-18T23:13:08.770 INFO:tasks.workunit.client.0.smithi049.stdout: 1 526971 10.37 MB/sec execute 130 sec latency 20.222 ms 2024-03-18T23:13:08.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:13:08 smithi139 ceph-mon[67282]: pgmap v150: 97 pgs: 97 active+clean; 72 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 5.4 MiB/s wr, 400 op/s 2024-03-18T23:13:08.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:13:08 smithi049 ceph-mon[62991]: pgmap v150: 97 pgs: 97 active+clean; 72 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 5.4 MiB/s wr, 400 op/s 2024-03-18T23:13:08.961 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:13:08 smithi175 ceph-mon[67352]: pgmap v150: 97 pgs: 97 active+clean; 72 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 5.4 MiB/s wr, 400 op/s 2024-03-18T23:13:09.770 INFO:tasks.workunit.client.0.smithi049.stdout: 1 529103 10.39 MB/sec execute 131 sec latency 62.589 ms 2024-03-18T23:13:10.010 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:10.009+0000 7f72fd70d640 1 -- 172.21.15.49:0/505208806 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f72f009b040 msgr2=0x7f72f00a2c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:13:10.010 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:10.009+0000 7f72fd70d640 1 --2- 172.21.15.49:0/505208806 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f72f009b040 0x7f72f00a2c10 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f72f804f850 tx=0x7f72f8097950 comp rx=0 tx=0).stop 2024-03-18T23:13:10.014 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:10.010+0000 7f72fd70d640 1 -- 172.21.15.49:0/505208806 shutdown_connections 2024-03-18T23:13:10.014 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:10.010+0000 7f72fd70d640 1 --2- 172.21.15.49:0/505208806 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f72f009b040 0x7f72f00a2c10 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:10.014 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:10.010+0000 7f72fd70d640 1 --2- 172.21.15.49:0/505208806 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f72f0096b50 0x7f72f009a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:10.014 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:10.010+0000 7f72fd70d640 1 --2- 172.21.15.49:0/505208806 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f72f00961a0 0x7f72f0096580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:10.014 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:10.010+0000 7f72fd70d640 1 -- 172.21.15.49:0/505208806 >> 172.21.15.49:0/505208806 conn(0x7f72f000b840 msgr2=0x7f72f000bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:13:10.014 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:10.012+0000 7f72fd70d640 1 -- 172.21.15.49:0/505208806 shutdown_connections 2024-03-18T23:13:10.014 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:10.012+0000 7f72fd70d640 1 -- 172.21.15.49:0/505208806 wait complete. 2024-03-18T23:13:10.014 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:10.013+0000 7f72fd70d640 1 Processor -- start 2024-03-18T23:13:10.014 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:10.013+0000 7f72fd70d640 1 -- start start 2024-03-18T23:13:10.014 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:10.013+0000 7f72fd70d640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f72f00961a0 0x7f72f00a2770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:13:10.016 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:10.013+0000 7f72fd70d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f72f009b040 0x7f72f009f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:13:10.017 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:10.014+0000 7f72fd70d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f72f009fd30 0x7f72f00a01e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:13:10.017 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:10.014+0000 7f72f77fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f72f009b040 0x7f72f009f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:13:10.017 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:10.014+0000 7f72f77fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f72f009b040 0x7f72f009f7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:44062/0 (socket says 172.21.15.49:44062) 2024-03-18T23:13:10.017 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:10.014+0000 7f72f77fe640 1 -- 172.21.15.49:0/3984379800 learned_addr learned my addr 172.21.15.49:0/3984379800 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:13:10.017 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:10.014+0000 7f72fd70d640 1 -- 172.21.15.49:0/3984379800 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f72f0008d40 con 0x7f72f009b040 2024-03-18T23:13:10.017 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:10.014+0000 7f72fd70d640 1 -- 172.21.15.49:0/3984379800 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f72f0008bc0 con 0x7f72f009fd30 2024-03-18T23:13:10.017 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:10.014+0000 7f72fd70d640 1 -- 172.21.15.49:0/3984379800 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f72f0008ec0 con 0x7f72f00961a0 2024-03-18T23:13:10.017 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:10.014+0000 7f72fcf0c640 1 --2- 172.21.15.49:0/3984379800 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f72f009fd30 0x7f72f00a01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:13:10.017 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:10.014+0000 7f72f7fff640 1 --2- 172.21.15.49:0/3984379800 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f72f00961a0 0x7f72f00a2770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:13:10.017 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:10.014+0000 7f72f77fe640 1 -- 172.21.15.49:0/3984379800 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f72f00961a0 msgr2=0x7f72f00a2770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:13:10.017 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:10.014+0000 7f72f77fe640 1 --2- 172.21.15.49:0/3984379800 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f72f00961a0 0x7f72f00a2770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:10.017 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:10.015+0000 7f72f77fe640 1 -- 172.21.15.49:0/3984379800 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f72f009fd30 msgr2=0x7f72f00a01e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:13:10.018 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:10.015+0000 7f72f77fe640 1 --2- 172.21.15.49:0/3984379800 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f72f009fd30 0x7f72f00a01e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:10.018 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:10.015+0000 7f72f77fe640 1 -- 172.21.15.49:0/3984379800 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72f013f2d0 con 0x7f72f009b040 2024-03-18T23:13:10.018 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:10.015+0000 7f72f7fff640 1 --2- 172.21.15.49:0/3984379800 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f72f00961a0 0x7f72f00a2770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T23:13:10.018 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:10.015+0000 7f72fcf0c640 1 --2- 172.21.15.49:0/3984379800 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f72f009fd30 0x7f72f00a01e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:13:10.018 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:10.015+0000 7f72f77fe640 1 --2- 172.21.15.49:0/3984379800 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f72f009b040 0x7f72f009f7f0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f72e400c970 tx=0x7f72e400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:13:10.018 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:10.015+0000 7f72f57fa640 1 -- 172.21.15.49:0/3984379800 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72e4007bb0 con 0x7f72f009b040 2024-03-18T23:13:10.018 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:10.015+0000 7f72fd70d640 1 -- 172.21.15.49:0/3984379800 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72f013f500 con 0x7f72f009b040 2024-03-18T23:13:10.018 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:10.015+0000 7f72fd70d640 1 -- 172.21.15.49:0/3984379800 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72f013fa70 con 0x7f72f009b040 2024-03-18T23:13:10.018 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:10.015+0000 7f72f57fa640 1 -- 172.21.15.49:0/3984379800 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f72e4007d50 con 0x7f72f009b040 2024-03-18T23:13:10.018 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:10.015+0000 7f72f57fa640 1 -- 172.21.15.49:0/3984379800 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72e400f450 con 0x7f72f009b040 2024-03-18T23:13:10.018 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:10.018+0000 7f7301630740 1 Processor -- start 2024-03-18T23:13:10.024 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:10.018+0000 7f72f57fa640 1 -- 172.21.15.49:0/3984379800 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 25) ==== 137421+0+0 (secure 0 0 0) 0x7f72e400f5f0 con 0x7f72f009b040 2024-03-18T23:13:10.024 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:10.018+0000 7f7301630740 1 -- start start 2024-03-18T23:13:10.025 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:10.018+0000 7f7301630740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560616bd60d0 0x560616bd64b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:13:10.025 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:10.018+0000 7f7301630740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560616bd69f0 0x560616bdf280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:13:10.025 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:10.018+0000 7f7301630740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560616bdf7c0 0x560616be1bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:13:10.025 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:10.018+0000 7f7301630740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x560616b39be0 con 0x560616bd69f0 2024-03-18T23:13:10.025 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:10.018+0000 7f7301630740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x560616ae2960 con 0x560616bdf7c0 2024-03-18T23:13:10.025 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:10.018+0000 7f7301630740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x560616b35c80 con 0x560616bd60d0 2024-03-18T23:13:10.025 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:10.018+0000 7f72f7fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560616bd69f0 0x560616bdf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:13:10.025 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:10.018+0000 7f72fcf0c640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560616bd60d0 0x560616bd64b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:13:10.025 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:10.018+0000 7f72f7fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560616bd69f0 0x560616bdf280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:44070/0 (socket says 172.21.15.49:44070) 2024-03-18T23:13:10.025 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:10.018+0000 7f72fcf0c640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560616bd60d0 0x560616bd64b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.49:44104/0 (socket says 172.21.15.49:44104) 2024-03-18T23:13:10.025 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:10.018+0000 7f72f7fff640 1 -- 172.21.15.49:0/1793322066 learned_addr learned my addr 172.21.15.49:0/1793322066 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:13:10.025 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:10.018+0000 7f72f77fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560616bdf7c0 0x560616be1bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:13:10.025 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:10.019+0000 7f72f7fff640 1 -- 172.21.15.49:0/1793322066 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560616bd60d0 msgr2=0x560616bd64b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:13:10.026 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:10.019+0000 7f72f7fff640 1 --2- 172.21.15.49:0/1793322066 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560616bd60d0 0x560616bd64b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:10.026 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:10.019+0000 7f72f7fff640 1 -- 172.21.15.49:0/1793322066 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560616bdf7c0 msgr2=0x560616be1bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:13:10.026 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:10.019+0000 7f72f7fff640 1 --2- 172.21.15.49:0/1793322066 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560616bdf7c0 0x560616be1bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:10.026 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:10.019+0000 7f72f7fff640 1 -- 172.21.15.49:0/1793322066 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560616ad9d20 con 0x560616bd69f0 2024-03-18T23:13:10.026 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:10.019+0000 7f72f57fa640 1 --2- 172.21.15.49:0/3984379800 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f72e80a2a30 0x7f72e80a4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:13:10.026 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:10.019+0000 7f72f7fff640 1 --2- 172.21.15.49:0/1793322066 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560616bd69f0 0x560616bdf280 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f72ec00bd70 tx=0x7f72ec00f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a74aed9db3682fe server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:13:10.026 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:10.019+0000 7f72fcf0c640 1 --2- 172.21.15.49:0/3984379800 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f72e80a2a30 0x7f72e80a4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:13:10.026 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:10.019+0000 7f72e27fc640 1 -- 172.21.15.49:0/1793322066 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72ec026070 con 0x560616bd69f0 2024-03-18T23:13:10.026 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:10.019+0000 7f72e27fc640 1 -- 172.21.15.49:0/1793322066 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f72ec02b940 con 0x560616bd69f0 2024-03-18T23:13:10.026 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:10.019+0000 7f72e27fc640 1 -- 172.21.15.49:0/1793322066 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72ec022ab0 con 0x560616bd69f0 2024-03-18T23:13:10.026 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:10.019+0000 7f72f57fa640 1 -- 172.21.15.49:0/3984379800 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8469+0+0 (secure 0 0 0) 0x7f72e40ce9e0 con 0x7f72f009b040 2024-03-18T23:13:10.026 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:10.020+0000 7f7301630740 1 -- 172.21.15.49:0/1793322066 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560616bd69f0 msgr2=0x560616bdf280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:13:10.026 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:10.020+0000 7f72fcf0c640 1 --2- 172.21.15.49:0/3984379800 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f72e80a2a30 0x7f72e80a4ef0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f72f8062de0 tx=0x7f72f8078040 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:13:10.026 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:10.020+0000 7f7301630740 1 --2- 172.21.15.49:0/1793322066 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560616bd69f0 0x560616bdf280 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f72ec00bd70 tx=0x7f72ec00f730 comp rx=0 tx=0).stop 2024-03-18T23:13:10.027 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:10.020+0000 7f7301630740 1 -- 172.21.15.49:0/1793322066 shutdown_connections 2024-03-18T23:13:10.027 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:10.020+0000 7f7301630740 1 --2- 172.21.15.49:0/1793322066 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560616bdf7c0 0x560616be1bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:10.027 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:10.020+0000 7f7301630740 1 --2- 172.21.15.49:0/1793322066 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560616bd69f0 0x560616bdf280 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:10.027 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:10.020+0000 7f7301630740 1 --2- 172.21.15.49:0/1793322066 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560616bd60d0 0x560616bd64b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:10.027 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:10.020+0000 7f7301630740 1 -- 172.21.15.49:0/1793322066 >> 172.21.15.49:0/1793322066 conn(0x560616bd09f0 msgr2=0x560616bd0dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:13:10.027 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:10.020+0000 7f7301630740 1 -- 172.21.15.49:0/1793322066 shutdown_connections 2024-03-18T23:13:10.027 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:10.020+0000 7f7301630740 1 -- 172.21.15.49:0/1793322066 wait complete. 2024-03-18T23:13:10.027 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:10.020+0000 7f7301630740 1 -- start start 2024-03-18T23:13:10.027 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:10.021+0000 7f7301630740 1 Processor -- start 2024-03-18T23:13:10.027 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:10.021+0000 7f7301630740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560616bd60d0 0x560616bdf110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:13:10.027 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:10.021+0000 7f7301630740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560616bd69f0 0x560616bdc4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:13:10.027 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:10.021+0000 7f7301630740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560616bdf7c0 0x560616bdca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:13:10.027 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:10.021+0000 7f7301630740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x560616bf3400 con 0x560616bd69f0 2024-03-18T23:13:10.027 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:10.021+0000 7f7301630740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x560616af5db0 con 0x560616bdf7c0 2024-03-18T23:13:10.027 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:10.021+0000 7f7301630740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x560616bf3580 con 0x560616bd60d0 2024-03-18T23:13:10.028 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:10.021+0000 7f7301630740 10 client.0 _ll_register_callbacks cb 0x560616afae70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T23:13:10.028 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:10.021+0000 7f72fcf0c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560616bdf7c0 0x560616bdca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:13:10.028 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:10.021+0000 7f72fcf0c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560616bdf7c0 0x560616bdca00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.49:45146/0 (socket says 172.21.15.49:45146) 2024-03-18T23:13:10.028 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:10.021+0000 7f72f77fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560616bd69f0 0x560616bdc4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:13:10.028 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:10.021+0000 7f72fcf0c640 1 -- 172.21.15.49:0/456504939 learned_addr learned my addr 172.21.15.49:0/456504939 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:13:10.028 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:10.021+0000 7f72f7fff640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560616bd60d0 0x560616bdf110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:13:10.028 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:10.022+0000 7f72fcf0c640 1 -- 172.21.15.49:0/456504939 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560616bd60d0 msgr2=0x560616bdf110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:13:10.028 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:10.022+0000 7f72fcf0c640 1 --2- 172.21.15.49:0/456504939 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560616bd60d0 0x560616bdf110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:10.028 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:10.022+0000 7f72fcf0c640 1 -- 172.21.15.49:0/456504939 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560616bd69f0 msgr2=0x560616bdc4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:13:10.028 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:10.022+0000 7f72fcf0c640 1 --2- 172.21.15.49:0/456504939 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560616bd69f0 0x560616bdc4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:10.028 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:10.022+0000 7f72fcf0c640 1 -- 172.21.15.49:0/456504939 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560616af4c60 con 0x560616bdf7c0 2024-03-18T23:13:10.028 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:10.022+0000 7f72fcf0c640 1 --2- 172.21.15.49:0/456504939 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560616bdf7c0 0x560616bdca00 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f72f8067500 tx=0x7f72f8064a60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=26d16c78e30baf72 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:13:10.028 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:10.022+0000 7f72d77fe640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-03-18T23:13:10.028 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:10.022+0000 7f72d77fe640 1 -- 172.21.15.49:0/456504939 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72f80a7660 con 0x560616bdf7c0 2024-03-18T23:13:10.029 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:10.022+0000 7f7301630740 1 -- 172.21.15.49:0/456504939 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f72ec02b940 con 0x560616bdf7c0 2024-03-18T23:13:10.029 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:10.022+0000 7f72d77fe640 1 -- 172.21.15.49:0/456504939 <== mon.2 v2:172.21.15.139:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8469+0+0 (secure 0 0 0) 0x7f72f80a2040 con 0x560616bdf7c0 2024-03-18T23:13:10.029 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:10.022+0000 7f72d77fe640 1 -- 172.21.15.49:0/456504939 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f72f80a7800 con 0x560616bdf7c0 2024-03-18T23:13:10.029 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:10.022+0000 7f7301630740 10 client.25667 fetch_fsmap learned FSMap version 32 2024-03-18T23:13:10.029 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:10.022+0000 7f7301630740 1 -- 172.21.15.49:0/456504939 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x56061684e210 con 0x560616bdf7c0 2024-03-18T23:13:10.029 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:10.023+0000 7f72d77fe640 1 -- 172.21.15.49:0/456504939 <== mon.2 v2:172.21.15.139:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f72ec02b940 con 0x560616bdf7c0 2024-03-18T23:13:10.029 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:10.023+0000 7f72d77fe640 1 -- 172.21.15.49:0/456504939 <== mon.2 v2:172.21.15.139:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f72f806fa30 con 0x560616bdf7c0 2024-03-18T23:13:10.029 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:10.023+0000 7f7301630740 10 client.25667 fetch_fsmap finished waiting for FSMap version 32 2024-03-18T23:13:10.029 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:10.023+0000 7f7301630740 10 client.25667 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T23:13:10.029 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:10.023+0000 7f7301630740 20 client.25667 populate_metadata read hostname 'smithi049' 2024-03-18T23:13:10.029 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:10.023+0000 7f7301630740 1 --2- 172.21.15.49:0/456504939 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x560616c00660 0x560616c02ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:13:10.029 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:10.023+0000 7f7301630740 4 client.25667 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T23:13:10.029 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:10.023+0000 7f7301630740 1 -- 172.21.15.49:0/456504939 --> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560616b47120 con 0x560616c00660 2024-03-18T23:13:10.029 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:10.023+0000 7f72f7fff640 1 --2- 172.21.15.49:0/456504939 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x560616c00660 0x560616c02ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:13:10.029 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:10.023+0000 7f72d77fe640 10 client.25667 ms_handle_connect on v2:172.21.15.139:6838/4280128209 2024-03-18T23:13:10.030 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:10.023+0000 7f72f7fff640 1 --2- 172.21.15.49:0/456504939 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x560616c00660 0x560616c02ab0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d7e98f8b6dfb4ec9 server_cookie=408b4ecbf1983220 in_seq=0 out_seq=0 2024-03-18T23:13:10.031 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:10.025+0000 7f72d77fe640 1 -- 172.21.15.49:0/456504939 <== mds.0 v2:172.21.15.139:6838/4280128209 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x560616b47120 con 0x560616c00660 2024-03-18T23:13:10.031 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:10.025+0000 7f72d77fe640 10 client.25667 handle_command_reply: tid=0 2024-03-18T23:13:10.031 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:10.025+0000 7f7301630740 1 client.25667 shutdown 2024-03-18T23:13:10.031 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:10.025+0000 7f7301630740 1 -- 172.21.15.49:0/456504939 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560616bdf7c0 msgr2=0x560616bdca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:13:10.031 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:10.025+0000 7f7301630740 1 --2- 172.21.15.49:0/456504939 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560616bdf7c0 0x560616bdca00 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f72f8067500 tx=0x7f72f8064a60 comp rx=0 tx=0).stop 2024-03-18T23:13:10.031 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:10.025+0000 7f7301630740 1 -- 172.21.15.49:0/456504939 shutdown_connections 2024-03-18T23:13:10.031 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:10.025+0000 7f7301630740 1 --2- 172.21.15.49:0/456504939 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x560616c00660 0x560616c02ab0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:10.031 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:10.025+0000 7f7301630740 1 --2- 172.21.15.49:0/456504939 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560616bdf7c0 0x560616bdca00 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:10.031 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:10.025+0000 7f72d77fe640 0 client.25667 ms_handle_reset on v2:172.21.15.139:6838/4280128209 2024-03-18T23:13:10.031 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:10.025+0000 7f7301630740 1 --2- 172.21.15.49:0/456504939 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560616bd69f0 0x560616bdc4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:10.031 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:10.025+0000 7f7301630740 1 --2- 172.21.15.49:0/456504939 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560616bd60d0 0x560616bdf110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:10.031 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:10.025+0000 7f7301630740 1 -- 172.21.15.49:0/456504939 >> 172.21.15.49:0/456504939 conn(0x560616bd09f0 msgr2=0x560616bd2740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:13:10.032 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:10.025+0000 7f7301630740 1 -- 172.21.15.49:0/456504939 shutdown_connections 2024-03-18T23:13:10.032 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:10.025+0000 7f7301630740 1 -- 172.21.15.49:0/456504939 wait complete. 2024-03-18T23:13:10.032 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:10.025+0000 7f7301630740 20 client.25667 trim_cache size 0 max 16384 2024-03-18T23:13:10.043 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:10.041+0000 7f7301630740 1 Processor -- start 2024-03-18T23:13:10.043 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:10.041+0000 7f7301630740 1 -- start start 2024-03-18T23:13:10.043 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:10.041+0000 7f7301630740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560616bd60d0 0x560616bcfab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:13:10.043 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:10.041+0000 7f7301630740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560616bd69f0 0x560616c083c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:13:10.043 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:10.041+0000 7f7301630740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560616bdf7c0 0x560616bf87c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:13:10.043 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:10.041+0000 7f7301630740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f72dc00e6d0 con 0x560616bd60d0 2024-03-18T23:13:10.043 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:10.041+0000 7f7301630740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f72dc010210 con 0x560616bdf7c0 2024-03-18T23:13:10.043 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:10.041+0000 7f7301630740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f72dc00eda0 con 0x560616bd69f0 2024-03-18T23:13:10.043 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:10.041+0000 7f72f7fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560616bd60d0 0x560616bcfab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:13:10.043 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:10.041+0000 7f72f7fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560616bd60d0 0x560616bcfab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:44086/0 (socket says 172.21.15.49:44086) 2024-03-18T23:13:10.043 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:10.041+0000 7f72f7fff640 1 -- 172.21.15.49:0/1586329810 learned_addr learned my addr 172.21.15.49:0/1586329810 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:13:10.044 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:10.041+0000 7f72f77fe640 1 --2- 172.21.15.49:0/1586329810 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560616bd69f0 0x560616c083c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:13:10.044 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:10.041+0000 7f72fcf0c640 1 --2- 172.21.15.49:0/1586329810 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560616bdf7c0 0x560616bf87c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:13:10.044 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:10.041+0000 7f72f7fff640 1 -- 172.21.15.49:0/1586329810 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560616bd69f0 msgr2=0x560616c083c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:13:10.044 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:10.041+0000 7f72f7fff640 1 --2- 172.21.15.49:0/1586329810 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560616bd69f0 0x560616c083c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:10.044 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:10.041+0000 7f72f7fff640 1 -- 172.21.15.49:0/1586329810 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560616bdf7c0 msgr2=0x560616bf87c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:13:10.044 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:10.041+0000 7f72f7fff640 1 --2- 172.21.15.49:0/1586329810 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560616bdf7c0 0x560616bf87c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:10.044 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:10.041+0000 7f72f7fff640 1 -- 172.21.15.49:0/1586329810 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56061691a620 con 0x560616bd60d0 2024-03-18T23:13:10.044 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:10.042+0000 7f72f7fff640 1 --2- 172.21.15.49:0/1586329810 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560616bd60d0 0x560616bcfab0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f72ec00bd70 tx=0x7f72ec022730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed62dd42b5d0c25b server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:13:10.044 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:10.042+0000 7f72e1ffb640 1 -- 172.21.15.49:0/1586329810 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72ec026040 con 0x560616bd60d0 2024-03-18T23:13:10.044 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:10.042+0000 7f72e1ffb640 1 -- 172.21.15.49:0/1586329810 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f72ec0055b0 con 0x560616bd60d0 2024-03-18T23:13:10.044 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:10.042+0000 7f72e1ffb640 1 -- 172.21.15.49:0/1586329810 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72ec022a00 con 0x560616bd60d0 2024-03-18T23:13:10.049 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:10.042+0000 7f7301630740 1 -- 172.21.15.49:0/1586329810 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560616bd60d0 msgr2=0x560616bcfab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:13:10.049 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:10.042+0000 7f7301630740 1 --2- 172.21.15.49:0/1586329810 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560616bd60d0 0x560616bcfab0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f72ec00bd70 tx=0x7f72ec022730 comp rx=0 tx=0).stop 2024-03-18T23:13:10.049 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:10.042+0000 7f7301630740 1 -- 172.21.15.49:0/1586329810 shutdown_connections 2024-03-18T23:13:10.049 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:10.042+0000 7f7301630740 1 --2- 172.21.15.49:0/1586329810 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560616bdf7c0 0x560616bf87c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:10.049 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:10.042+0000 7f7301630740 1 --2- 172.21.15.49:0/1586329810 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560616bd69f0 0x560616c083c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:10.049 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:10.042+0000 7f7301630740 1 --2- 172.21.15.49:0/1586329810 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560616bd60d0 0x560616bcfab0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:10.049 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:10.042+0000 7f7301630740 1 -- 172.21.15.49:0/1586329810 >> 172.21.15.49:0/1586329810 conn(0x560616bd09f0 msgr2=0x560616bd7080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:13:10.049 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:10.042+0000 7f7301630740 1 -- 172.21.15.49:0/1586329810 shutdown_connections 2024-03-18T23:13:10.049 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:10.042+0000 7f7301630740 1 -- 172.21.15.49:0/1586329810 wait complete. 2024-03-18T23:13:10.049 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:10.043+0000 7f7301630740 1 -- start start 2024-03-18T23:13:10.049 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:10.043+0000 7f7301630740 1 Processor -- start 2024-03-18T23:13:10.049 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:10.043+0000 7f7301630740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560616bd60d0 0x560616bf4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:13:10.050 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:10.043+0000 7f7301630740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560616bd69f0 0x560616bf5200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:13:10.050 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:10.043+0000 7f7301630740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560616bdf7c0 0x560616c5e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:13:10.050 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:10.043+0000 7f7301630740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f72dc00fb40 con 0x560616bdf7c0 2024-03-18T23:13:10.050 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:10.043+0000 7f7301630740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f72dc00f470 con 0x560616bd69f0 2024-03-18T23:13:10.050 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:10.043+0000 7f7301630740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f72dc00c6b0 con 0x560616bd60d0 2024-03-18T23:13:10.050 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:10.043+0000 7f7301630740 10 client.0 _ll_register_callbacks cb 0x7f72f80504b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T23:13:10.050 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:10.044+0000 7f72fcf0c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560616bdf7c0 0x560616c5e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:13:10.050 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:10.044+0000 7f72f7fff640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560616bd60d0 0x560616bf4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:13:10.050 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:10.044+0000 7f72f77fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560616bd69f0 0x560616bf5200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:13:10.050 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:10.044+0000 7f72f7fff640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560616bd60d0 0x560616bf4cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.49:44136/0 (socket says 172.21.15.49:44136) 2024-03-18T23:13:10.050 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:10.044+0000 7f72fcf0c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560616bdf7c0 0x560616c5e9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:44098/0 (socket says 172.21.15.49:44098) 2024-03-18T23:13:10.050 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:10.044+0000 7f72f7fff640 1 -- 172.21.15.49:0/1940033703 learned_addr learned my addr 172.21.15.49:0/1940033703 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:13:10.050 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:10.044+0000 7f72fcf0c640 1 -- 172.21.15.49:0/1940033703 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560616bd60d0 msgr2=0x560616bf4cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:13:10.051 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:10.044+0000 7f72fcf0c640 1 --2- 172.21.15.49:0/1940033703 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560616bd60d0 0x560616bf4cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:10.051 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:10.044+0000 7f72fcf0c640 1 -- 172.21.15.49:0/1940033703 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560616bd69f0 msgr2=0x560616bf5200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:13:10.051 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:10.044+0000 7f72fcf0c640 1 --2- 172.21.15.49:0/1940033703 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560616bd69f0 0x560616bf5200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:10.051 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:10.044+0000 7f72fcf0c640 1 -- 172.21.15.49:0/1940033703 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560616c5f1b0 con 0x560616bdf7c0 2024-03-18T23:13:10.051 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:10.044+0000 7f72f77fe640 1 --2- 172.21.15.49:0/1940033703 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560616bd69f0 0x560616bf5200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:13:10.051 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:10.044+0000 7f72f7fff640 1 --2- 172.21.15.49:0/1940033703 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560616bd60d0 0x560616bf4cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:13:10.051 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:10.044+0000 7f72fcf0c640 1 --2- 172.21.15.49:0/1940033703 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560616bdf7c0 0x560616c5e9b0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f72f8065040 tx=0x7f72f806fc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e6ca713a4e951b5 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:13:10.051 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:10.044+0000 7f72d77fe640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-18T23:13:10.051 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:10.044+0000 7f72d77fe640 1 -- 172.21.15.49:0/1940033703 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72f80a7660 con 0x560616bdf7c0 2024-03-18T23:13:10.051 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:10.044+0000 7f72d77fe640 1 -- 172.21.15.49:0/1940033703 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8469+0+0 (secure 0 0 0) 0x7f72f80a7bb0 con 0x560616bdf7c0 2024-03-18T23:13:10.051 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:10.044+0000 7f7301630740 1 -- 172.21.15.49:0/1940033703 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f72ec0055b0 con 0x560616bdf7c0 2024-03-18T23:13:10.051 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:10.044+0000 7f72d77fe640 1 -- 172.21.15.49:0/1940033703 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f72f80ca050 con 0x560616bdf7c0 2024-03-18T23:13:10.051 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:10.044+0000 7f7301630740 10 client.15858 fetch_fsmap learned FSMap version 32 2024-03-18T23:13:10.051 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:10.044+0000 7f7301630740 1 -- 172.21.15.49:0/1940033703 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x560616c67520 con 0x560616bdf7c0 2024-03-18T23:13:10.051 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:10.045+0000 7f72d77fe640 1 -- 172.21.15.49:0/1940033703 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f72ec0055b0 con 0x560616bdf7c0 2024-03-18T23:13:10.052 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:10.045+0000 7f72d77fe640 1 -- 172.21.15.49:0/1940033703 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f72f80c3950 con 0x560616bdf7c0 2024-03-18T23:13:10.052 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:10.045+0000 7f7301630740 10 client.15858 fetch_fsmap finished waiting for FSMap version 32 2024-03-18T23:13:10.052 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:10.045+0000 7f7301630740 10 client.15858 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T23:13:10.052 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:10.045+0000 7f7301630740 20 client.15858 populate_metadata read hostname 'smithi049' 2024-03-18T23:13:10.052 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:10.045+0000 7f7301630740 1 --2- 172.21.15.49:0/1940033703 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x560616c67be0 0x560616c69fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:13:10.052 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:10.045+0000 7f7301630740 4 client.15858 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T23:13:10.052 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:10.045+0000 7f7301630740 1 -- 172.21.15.49:0/1940033703 --> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560616b41470 con 0x560616c67be0 2024-03-18T23:13:10.052 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:10.045+0000 7f72f7fff640 1 --2- 172.21.15.49:0/1940033703 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x560616c67be0 0x560616c69fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:13:10.052 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:10.045+0000 7f72f7fff640 1 --2- 172.21.15.49:0/1940033703 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x560616c67be0 0x560616c69fd0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c46e477f367a01b0 server_cookie=64b3779bf21716a2 in_seq=0 out_seq=0 2024-03-18T23:13:10.052 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:10.045+0000 7f72d77fe640 10 client.15858 ms_handle_connect on v2:172.21.15.139:6838/4280128209 2024-03-18T23:13:10.319 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:10.318+0000 7f72d77fe640 1 -- 172.21.15.49:0/1940033703 <== mds.0 v2:172.21.15.139:6838/4280128209 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x560616b41470 con 0x560616c67be0 2024-03-18T23:13:10.319 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:10.318+0000 7f72d77fe640 10 client.15858 handle_command_reply: tid=0 2024-03-18T23:13:10.320 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:10.318+0000 7f7301630740 1 client.15858 shutdown 2024-03-18T23:13:10.320 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:10.318+0000 7f7301630740 1 -- 172.21.15.49:0/1940033703 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560616bdf7c0 msgr2=0x560616c5e9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:13:10.320 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:10.318+0000 7f7301630740 1 --2- 172.21.15.49:0/1940033703 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560616bdf7c0 0x560616c5e9b0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f72f8065040 tx=0x7f72f806fc00 comp rx=0 tx=0).stop 2024-03-18T23:13:10.320 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:10.318+0000 7f7301630740 1 -- 172.21.15.49:0/1940033703 shutdown_connections 2024-03-18T23:13:10.320 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:10.318+0000 7f7301630740 1 --2- 172.21.15.49:0/1940033703 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x560616c67be0 0x560616c69fd0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:10.320 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:10.318+0000 7f7301630740 1 --2- 172.21.15.49:0/1940033703 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560616bdf7c0 0x560616c5e9b0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:10.320 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:10.318+0000 7f72d77fe640 0 client.15858 ms_handle_reset on v2:172.21.15.139:6838/4280128209 2024-03-18T23:13:10.320 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:10.318+0000 7f7301630740 1 --2- 172.21.15.49:0/1940033703 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560616bd69f0 0x560616bf5200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:10.320 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:10.318+0000 7f7301630740 1 --2- 172.21.15.49:0/1940033703 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560616bd60d0 0x560616bf4cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:10.320 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:10.318+0000 7f7301630740 1 -- 172.21.15.49:0/1940033703 >> 172.21.15.49:0/1940033703 conn(0x560616bd09f0 msgr2=0x560616bcf930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:13:10.320 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:10.318+0000 7f7301630740 1 -- 172.21.15.49:0/1940033703 shutdown_connections 2024-03-18T23:13:10.320 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:10.318+0000 7f7301630740 1 -- 172.21.15.49:0/1940033703 wait complete. 2024-03-18T23:13:10.320 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:10.318+0000 7f7301630740 20 client.15858 trim_cache size 0 max 16384 2024-03-18T23:13:10.321 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:{ 2024-03-18T23:13:10.321 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "message": "", 2024-03-18T23:13:10.321 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "return_code": 0 2024-03-18T23:13:10.321 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:} 2024-03-18T23:13:10.321 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:10.319+0000 7f72fd70d640 1 -- 172.21.15.49:0/3984379800 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f72e80a2a30 msgr2=0x7f72e80a4ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:13:10.321 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:10.319+0000 7f72fd70d640 1 --2- 172.21.15.49:0/3984379800 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f72e80a2a30 0x7f72e80a4ef0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f72f8062de0 tx=0x7f72f8078040 comp rx=0 tx=0).stop 2024-03-18T23:13:10.322 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:10.319+0000 7f72fd70d640 1 -- 172.21.15.49:0/3984379800 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f72f009b040 msgr2=0x7f72f009f7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:13:10.322 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:10.319+0000 7f72fd70d640 1 --2- 172.21.15.49:0/3984379800 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f72f009b040 0x7f72f009f7f0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f72e400c970 tx=0x7f72e400ce30 comp rx=0 tx=0).stop 2024-03-18T23:13:10.322 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:10.319+0000 7f72fd70d640 1 -- 172.21.15.49:0/3984379800 shutdown_connections 2024-03-18T23:13:10.322 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:10.319+0000 7f72fd70d640 1 --2- 172.21.15.49:0/3984379800 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f72e80a2a30 0x7f72e80a4ef0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:10.322 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:10.319+0000 7f72fd70d640 1 --2- 172.21.15.49:0/3984379800 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f72f009fd30 0x7f72f00a01e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:10.322 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:10.319+0000 7f72fd70d640 1 --2- 172.21.15.49:0/3984379800 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f72f009b040 0x7f72f009f7f0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:10.322 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:10.319+0000 7f72fd70d640 1 --2- 172.21.15.49:0/3984379800 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f72f00961a0 0x7f72f00a2770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:10.322 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:10.319+0000 7f72fd70d640 1 -- 172.21.15.49:0/3984379800 >> 172.21.15.49:0/3984379800 conn(0x7f72f000b840 msgr2=0x7f72f0093da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:13:10.322 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:10.319+0000 7f72fd70d640 1 -- 172.21.15.49:0/3984379800 shutdown_connections 2024-03-18T23:13:10.322 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:10.319+0000 7f72fd70d640 1 -- 172.21.15.49:0/3984379800 wait complete. 2024-03-18T23:13:10.761 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:13:10 smithi175 ceph-mon[67352]: pgmap v151: 97 pgs: 97 active+clean; 72 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.1 MiB/s wr, 312 op/s 2024-03-18T23:13:10.770 INFO:tasks.workunit.client.0.smithi049.stdout: 1 530041 10.35 MB/sec execute 132 sec latency 81.222 ms 2024-03-18T23:13:10.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:13:10 smithi139 ceph-mon[67282]: pgmap v151: 97 pgs: 97 active+clean; 72 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.1 MiB/s wr, 312 op/s 2024-03-18T23:13:10.879 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:13:10 smithi049 ceph-mon[62991]: pgmap v151: 97 pgs: 97 active+clean; 72 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.1 MiB/s wr, 312 op/s 2024-03-18T23:13:11.770 INFO:tasks.workunit.client.0.smithi049.stdout: 1 531936 10.33 MB/sec execute 133 sec latency 39.770 ms 2024-03-18T23:13:11.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:13:11 smithi139 ceph-mon[67282]: evicting unresponsive client smithi049:x (15351), after 303.574 seconds 2024-03-18T23:13:11.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:13:11 smithi139 ceph-mon[67282]: Evicting (and blocklisting) client session 15351 (172.21.15.49:0/1463571219) 2024-03-18T23:13:11.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:13:11 smithi139 ceph-mon[67282]: from='mds.? [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209]' entity='mds.b' cmd=[{"prefix":"osd blocklist", "blocklistop":"add","addr":"172.21.15.49:0/1463571219"}]: dispatch 2024-03-18T23:13:11.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:13:11 smithi139 ceph-mon[67282]: from='mds.? ' entity='mds.b' cmd=[{"prefix":"osd blocklist", "blocklistop":"add","addr":"172.21.15.49:0/1463571219"}]: dispatch 2024-03-18T23:13:11.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:13:11 smithi049 ceph-mon[62991]: evicting unresponsive client smithi049:x (15351), after 303.574 seconds 2024-03-18T23:13:11.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:13:11 smithi049 ceph-mon[62991]: Evicting (and blocklisting) client session 15351 (172.21.15.49:0/1463571219) 2024-03-18T23:13:11.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:13:11 smithi049 ceph-mon[62991]: from='mds.? [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209]' entity='mds.b' cmd=[{"prefix":"osd blocklist", "blocklistop":"add","addr":"172.21.15.49:0/1463571219"}]: dispatch 2024-03-18T23:13:11.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:13:11 smithi049 ceph-mon[62991]: from='mds.? ' entity='mds.b' cmd=[{"prefix":"osd blocklist", "blocklistop":"add","addr":"172.21.15.49:0/1463571219"}]: dispatch 2024-03-18T23:13:11.961 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:13:11 smithi175 ceph-mon[67352]: evicting unresponsive client smithi049:x (15351), after 303.574 seconds 2024-03-18T23:13:11.961 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:13:11 smithi175 ceph-mon[67352]: Evicting (and blocklisting) client session 15351 (172.21.15.49:0/1463571219) 2024-03-18T23:13:11.961 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:13:11 smithi175 ceph-mon[67352]: from='mds.? [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209]' entity='mds.b' cmd=[{"prefix":"osd blocklist", "blocklistop":"add","addr":"172.21.15.49:0/1463571219"}]: dispatch 2024-03-18T23:13:11.961 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:13:11 smithi175 ceph-mon[67352]: from='mds.? ' entity='mds.b' cmd=[{"prefix":"osd blocklist", "blocklistop":"add","addr":"172.21.15.49:0/1463571219"}]: dispatch 2024-03-18T23:13:12.770 INFO:tasks.workunit.client.0.smithi049.stdout: 1 533867 10.34 MB/sec execute 134 sec latency 17.421 ms 2024-03-18T23:13:12.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:13:12 smithi139 ceph-mon[67282]: pgmap v152: 97 pgs: 97 active+clean; 64 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 5.7 MiB/s wr, 506 op/s 2024-03-18T23:13:12.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:13:12 smithi139 ceph-mon[67282]: from='mds.? ' entity='mds.b' cmd='[{"prefix":"osd blocklist", "blocklistop":"add","addr":"172.21.15.49:0/1463571219"}]': finished 2024-03-18T23:13:12.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:13:12 smithi139 ceph-mon[67282]: osdmap e91: 12 total, 12 up, 12 in 2024-03-18T23:13:12.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:13:12 smithi049 ceph-mon[62991]: pgmap v152: 97 pgs: 97 active+clean; 64 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 5.7 MiB/s wr, 506 op/s 2024-03-18T23:13:12.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:13:12 smithi049 ceph-mon[62991]: from='mds.? ' entity='mds.b' cmd='[{"prefix":"osd blocklist", "blocklistop":"add","addr":"172.21.15.49:0/1463571219"}]': finished 2024-03-18T23:13:12.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:13:12 smithi049 ceph-mon[62991]: osdmap e91: 12 total, 12 up, 12 in 2024-03-18T23:13:12.961 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:13:12 smithi175 ceph-mon[67352]: pgmap v152: 97 pgs: 97 active+clean; 64 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 5.7 MiB/s wr, 506 op/s 2024-03-18T23:13:12.961 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:13:12 smithi175 ceph-mon[67352]: from='mds.? ' entity='mds.b' cmd='[{"prefix":"osd blocklist", "blocklistop":"add","addr":"172.21.15.49:0/1463571219"}]': finished 2024-03-18T23:13:12.961 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:13:12 smithi175 ceph-mon[67352]: osdmap e91: 12 total, 12 up, 12 in 2024-03-18T23:13:13.770 INFO:tasks.workunit.client.0.smithi049.stdout: 1 536377 10.39 MB/sec execute 135 sec latency 17.589 ms 2024-03-18T23:13:14.770 INFO:tasks.workunit.client.0.smithi049.stdout: 1 537936 10.36 MB/sec execute 136 sec latency 29.208 ms 2024-03-18T23:13:14.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:13:14 smithi139 ceph-mon[67282]: pgmap v154: 97 pgs: 97 active+clean; 64 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 5.3 MiB/s wr, 336 op/s 2024-03-18T23:13:14.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:13:14 smithi049 ceph-mon[62991]: pgmap v154: 97 pgs: 97 active+clean; 64 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 5.3 MiB/s wr, 336 op/s 2024-03-18T23:13:14.961 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:13:14 smithi175 ceph-mon[67352]: pgmap v154: 97 pgs: 97 active+clean; 64 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 5.3 MiB/s wr, 336 op/s 2024-03-18T23:13:15.770 INFO:tasks.workunit.client.0.smithi049.stdout: 1 539770 10.35 MB/sec execute 137 sec latency 141.149 ms 2024-03-18T23:13:16.770 INFO:tasks.workunit.client.0.smithi049.stdout: 1 541404 10.35 MB/sec execute 138 sec latency 17.429 ms 2024-03-18T23:13:16.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:13:16 smithi139 ceph-mon[67282]: pgmap v155: 97 pgs: 97 active+clean; 64 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 5.3 MiB/s wr, 336 op/s 2024-03-18T23:13:16.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:13:16 smithi049 ceph-mon[62991]: pgmap v155: 97 pgs: 97 active+clean; 64 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 5.3 MiB/s wr, 336 op/s 2024-03-18T23:13:16.961 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:13:16 smithi175 ceph-mon[67352]: pgmap v155: 97 pgs: 97 active+clean; 64 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 5.3 MiB/s wr, 336 op/s 2024-03-18T23:13:17.770 INFO:tasks.workunit.client.0.smithi049.stdout: 1 543634 10.37 MB/sec execute 139 sec latency 20.077 ms 2024-03-18T23:13:18.771 INFO:tasks.workunit.client.0.smithi049.stdout: 1 545169 10.34 MB/sec execute 140 sec latency 26.693 ms 2024-03-18T23:13:18.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:13:18 smithi139 ceph-mon[67282]: pgmap v156: 97 pgs: 97 active+clean; 69 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.5 MiB/s wr, 346 op/s 2024-03-18T23:13:18.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:13:18 smithi049 ceph-mon[62991]: pgmap v156: 97 pgs: 97 active+clean; 69 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.5 MiB/s wr, 346 op/s 2024-03-18T23:13:18.961 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:13:18 smithi175 ceph-mon[67352]: pgmap v156: 97 pgs: 97 active+clean; 69 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.5 MiB/s wr, 346 op/s 2024-03-18T23:13:19.770 INFO:tasks.workunit.client.0.smithi049.stdout: 1 547441 10.36 MB/sec execute 141 sec latency 17.812 ms 2024-03-18T23:13:20.770 INFO:tasks.workunit.client.0.smithi049.stdout: 1 548916 10.33 MB/sec execute 142 sec latency 80.586 ms 2024-03-18T23:13:20.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:13:20 smithi139 ceph-mon[67282]: pgmap v157: 97 pgs: 97 active+clean; 69 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.5 MiB/s wr, 346 op/s 2024-03-18T23:13:20.879 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:13:20 smithi049 ceph-mon[62991]: pgmap v157: 97 pgs: 97 active+clean; 69 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.5 MiB/s wr, 346 op/s 2024-03-18T23:13:20.961 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:13:20 smithi175 ceph-mon[67352]: pgmap v157: 97 pgs: 97 active+clean; 69 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.5 MiB/s wr, 346 op/s 2024-03-18T23:13:21.770 INFO:tasks.workunit.client.0.smithi049.stdout: 1 551031 10.35 MB/sec execute 143 sec latency 22.230 ms 2024-03-18T23:13:22.771 INFO:tasks.workunit.client.0.smithi049.stdout: 1 552814 10.32 MB/sec execute 144 sec latency 20.852 ms 2024-03-18T23:13:22.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:13:22 smithi139 ceph-mon[67282]: pgmap v158: 97 pgs: 97 active+clean; 84 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.4 MiB/s wr, 216 op/s 2024-03-18T23:13:22.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:13:22 smithi049 ceph-mon[62991]: pgmap v158: 97 pgs: 97 active+clean; 84 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.4 MiB/s wr, 216 op/s 2024-03-18T23:13:22.961 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:13:22 smithi175 ceph-mon[67352]: pgmap v158: 97 pgs: 97 active+clean; 84 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.4 MiB/s wr, 216 op/s 2024-03-18T23:13:23.420 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:23.417+0000 7fc7d3f1b640 1 -- 172.21.15.49:0/3702403387 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc7cc1056c0 msgr2=0x7fc7cc105aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:13:23.420 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:23.417+0000 7fc7d3f1b640 1 --2- 172.21.15.49:0/3702403387 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc7cc1056c0 0x7fc7cc105aa0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fc7c8009900 tx=0x7fc7c802bc30 comp rx=0 tx=0).stop 2024-03-18T23:13:23.420 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:23.417+0000 7fc7d3f1b640 1 -- 172.21.15.49:0/3702403387 shutdown_connections 2024-03-18T23:13:23.420 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:23.417+0000 7fc7d3f1b640 1 --2- 172.21.15.49:0/3702403387 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc7cc101330 0x7fc7cc10d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:23.420 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:23.417+0000 7fc7d3f1b640 1 --2- 172.21.15.49:0/3702403387 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc7cc100970 0x7fc7cc100df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:23.420 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:23.417+0000 7fc7d3f1b640 1 --2- 172.21.15.49:0/3702403387 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc7cc1056c0 0x7fc7cc105aa0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:23.420 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:23.417+0000 7fc7d3f1b640 1 -- 172.21.15.49:0/3702403387 >> 172.21.15.49:0/3702403387 conn(0x7fc7cc077570 msgr2=0x7fc7cc077980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:13:23.420 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:23.417+0000 7fc7d3f1b640 1 -- 172.21.15.49:0/3702403387 shutdown_connections 2024-03-18T23:13:23.420 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:23.417+0000 7fc7d3f1b640 1 -- 172.21.15.49:0/3702403387 wait complete. 2024-03-18T23:13:23.420 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:23.418+0000 7fc7d3f1b640 1 Processor -- start 2024-03-18T23:13:23.420 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:23.418+0000 7fc7d3f1b640 1 -- start start 2024-03-18T23:13:23.421 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:23.418+0000 7fc7d3f1b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc7cc100970 0x7fc7cc198170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:13:23.421 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:23.418+0000 7fc7d3f1b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc7cc101330 0x7fc7cc1986b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:13:23.421 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:23.418+0000 7fc7d3f1b640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc7cc1056c0 0x7fc7cc19c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:13:23.421 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:23.418+0000 7fc7d3f1b640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fc7cc110c30 con 0x7fc7cc100970 2024-03-18T23:13:23.421 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:23.418+0000 7fc7d3f1b640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fc7cc110ab0 con 0x7fc7cc101330 2024-03-18T23:13:23.421 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:23.418+0000 7fc7d3f1b640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc7cc110db0 con 0x7fc7cc1056c0 2024-03-18T23:13:23.421 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:23.418+0000 7fc7d1cb7640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc7cc100970 0x7fc7cc198170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:13:23.421 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:23.418+0000 7fc7d1cb7640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc7cc100970 0x7fc7cc198170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:45392/0 (socket says 172.21.15.49:45392) 2024-03-18T23:13:23.421 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:23.418+0000 7fc7d1cb7640 1 -- 172.21.15.49:0/2378759070 learned_addr learned my addr 172.21.15.49:0/2378759070 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:13:23.421 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:23.418+0000 7fc7d14b6640 1 --2- 172.21.15.49:0/2378759070 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc7cc101330 0x7fc7cc1986b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:13:23.421 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:23.418+0000 7fc7d24b8640 1 --2- 172.21.15.49:0/2378759070 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc7cc1056c0 0x7fc7cc19c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:13:23.421 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:23.419+0000 7fc7d1cb7640 1 -- 172.21.15.49:0/2378759070 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc7cc1056c0 msgr2=0x7fc7cc19c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:13:23.421 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:23.419+0000 7fc7d1cb7640 1 --2- 172.21.15.49:0/2378759070 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc7cc1056c0 0x7fc7cc19c9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:23.421 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:23.419+0000 7fc7d1cb7640 1 -- 172.21.15.49:0/2378759070 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc7cc101330 msgr2=0x7fc7cc1986b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:13:23.422 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:23.419+0000 7fc7d1cb7640 1 --2- 172.21.15.49:0/2378759070 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc7cc101330 0x7fc7cc1986b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:23.422 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:23.419+0000 7fc7d1cb7640 1 -- 172.21.15.49:0/2378759070 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7cc19d110 con 0x7fc7cc100970 2024-03-18T23:13:23.422 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:23.419+0000 7fc7d1cb7640 1 --2- 172.21.15.49:0/2378759070 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc7cc100970 0x7fc7cc198170 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fc7c802efd0 tx=0x7fc7c80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:13:23.422 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:23.419+0000 7fc7c2ffd640 1 -- 172.21.15.49:0/2378759070 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7c8041070 con 0x7fc7cc100970 2024-03-18T23:13:23.422 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:23.419+0000 7fc7d3f1b640 1 -- 172.21.15.49:0/2378759070 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7cc19d3a0 con 0x7fc7cc100970 2024-03-18T23:13:23.422 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:23.419+0000 7fc7c2ffd640 1 -- 172.21.15.49:0/2378759070 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc7c802ccc0 con 0x7fc7cc100970 2024-03-18T23:13:23.422 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:23.419+0000 7fc7c2ffd640 1 -- 172.21.15.49:0/2378759070 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7c8003d40 con 0x7fc7cc100970 2024-03-18T23:13:23.422 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:23.419+0000 7fc7d3f1b640 1 -- 172.21.15.49:0/2378759070 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7cc1a4c90 con 0x7fc7cc100970 2024-03-18T23:13:23.429 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:23.421+0000 7fc7c2ffd640 1 -- 172.21.15.49:0/2378759070 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 25) ==== 137421+0+0 (secure 0 0 0) 0x7fc7c8044070 con 0x7fc7cc100970 2024-03-18T23:13:23.429 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:23.422+0000 7fc7d6be7740 1 Processor -- start 2024-03-18T23:13:23.429 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:23.422+0000 7fc7d6be7740 1 -- start start 2024-03-18T23:13:23.429 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:23.422+0000 7fc7d6be7740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56034bce6ef0 0x56034bce72d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:13:23.429 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:23.422+0000 7fc7d6be7740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56034bce7810 0x56034bcf00a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:13:23.429 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:23.422+0000 7fc7d6be7740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56034bcf05e0 0x56034bcf29d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:13:23.429 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:23.422+0000 7fc7c2ffd640 1 --2- 172.21.15.49:0/2378759070 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7fc7b80a2a10 0x7fc7b80a4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:13:23.429 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:23.422+0000 7fc7d6be7740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x56034bc4abe0 con 0x56034bcf05e0 2024-03-18T23:13:23.430 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:23.422+0000 7fc7d6be7740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x56034bbf3960 con 0x56034bce6ef0 2024-03-18T23:13:23.430 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:23.422+0000 7fc7d6be7740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x56034bc46c80 con 0x56034bce7810 2024-03-18T23:13:23.430 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:23.422+0000 7fc7d14b6640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56034bcf05e0 0x56034bcf29d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:13:23.430 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:23.422+0000 7fc7d14b6640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56034bcf05e0 0x56034bcf29d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:45400/0 (socket says 172.21.15.49:45400) 2024-03-18T23:13:23.430 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:23.422+0000 7fc7d24b8640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56034bce6ef0 0x56034bce72d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:13:23.430 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:23.422+0000 7fc7d1cb7640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56034bce7810 0x56034bcf00a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:13:23.430 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:23.422+0000 7fc7d14b6640 1 -- 172.21.15.49:0/883490123 learned_addr learned my addr 172.21.15.49:0/883490123 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:13:23.430 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:23.422+0000 7fc7d24b8640 1 --2- 172.21.15.49:0/2378759070 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7fc7b80a2a10 0x7fc7b80a4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:13:23.430 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:23.423+0000 7fc7c2ffd640 1 -- 172.21.15.49:0/2378759070 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8512+0+0 (secure 0 0 0) 0x7fc7c80f07c0 con 0x7fc7cc100970 2024-03-18T23:13:23.430 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:23.423+0000 7fc7d14b6640 1 -- 172.21.15.49:0/883490123 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56034bce7810 msgr2=0x56034bcf00a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:13:23.430 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:23.423+0000 7fc7d14b6640 1 --2- 172.21.15.49:0/883490123 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56034bce7810 0x56034bcf00a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:23.430 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:23.423+0000 7fc7d14b6640 1 -- 172.21.15.49:0/883490123 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56034bce6ef0 msgr2=0x56034bce72d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:13:23.430 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:23.423+0000 7fc7d14b6640 1 --2- 172.21.15.49:0/883490123 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56034bce6ef0 0x56034bce72d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:23.431 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:23.423+0000 7fc7d14b6640 1 -- 172.21.15.49:0/883490123 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56034bbead20 con 0x56034bcf05e0 2024-03-18T23:13:23.431 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:23.423+0000 7fc7d24b8640 1 --2- 172.21.15.49:0/883490123 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56034bce6ef0 0x56034bce72d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:13:23.431 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:23.423+0000 7fc7d14b6640 1 --2- 172.21.15.49:0/883490123 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56034bcf05e0 0x56034bcf29d0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fc7bc009ca0 tx=0x7fc7bc011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edd18d3af9dbca22 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:13:23.431 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:23.423+0000 7fc7d24b8640 1 --2- 172.21.15.49:0/2378759070 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7fc7b80a2a10 0x7fc7b80a4ed0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fc7c4009fd0 tx=0x7fc7c4009500 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:13:23.431 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:23.423+0000 7fc7b7fff640 1 -- 172.21.15.49:0/883490123 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7bc01e070 con 0x56034bcf05e0 2024-03-18T23:13:23.431 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:23.423+0000 7fc7b7fff640 1 -- 172.21.15.49:0/883490123 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc7bc019b10 con 0x56034bcf05e0 2024-03-18T23:13:23.431 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:23.423+0000 7fc7b7fff640 1 -- 172.21.15.49:0/883490123 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7bc00ea30 con 0x56034bcf05e0 2024-03-18T23:13:23.431 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:23.424+0000 7fc7d6be7740 1 -- 172.21.15.49:0/883490123 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56034bcf05e0 msgr2=0x56034bcf29d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:13:23.431 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:23.424+0000 7fc7d6be7740 1 --2- 172.21.15.49:0/883490123 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56034bcf05e0 0x56034bcf29d0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fc7bc009ca0 tx=0x7fc7bc011810 comp rx=0 tx=0).stop 2024-03-18T23:13:23.431 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:23.424+0000 7fc7d6be7740 1 -- 172.21.15.49:0/883490123 shutdown_connections 2024-03-18T23:13:23.431 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:23.424+0000 7fc7d6be7740 1 --2- 172.21.15.49:0/883490123 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56034bcf05e0 0x56034bcf29d0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:23.431 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:23.424+0000 7fc7d6be7740 1 --2- 172.21.15.49:0/883490123 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56034bce7810 0x56034bcf00a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:23.431 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:23.424+0000 7fc7d6be7740 1 --2- 172.21.15.49:0/883490123 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56034bce6ef0 0x56034bce72d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:23.431 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:23.424+0000 7fc7d6be7740 1 -- 172.21.15.49:0/883490123 >> 172.21.15.49:0/883490123 conn(0x56034bce1810 msgr2=0x56034bce1bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:13:23.431 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:23.424+0000 7fc7d6be7740 1 -- 172.21.15.49:0/883490123 shutdown_connections 2024-03-18T23:13:23.432 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:23.424+0000 7fc7d6be7740 1 -- 172.21.15.49:0/883490123 wait complete. 2024-03-18T23:13:23.432 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:23.424+0000 7fc7d6be7740 1 -- start start 2024-03-18T23:13:23.432 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:23.424+0000 7fc7d6be7740 1 Processor -- start 2024-03-18T23:13:23.432 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:23.425+0000 7fc7d6be7740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56034bce6ef0 0x56034bcefef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:13:23.432 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:23.425+0000 7fc7d6be7740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56034bce7810 0x56034bced2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:13:23.432 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:23.425+0000 7fc7d6be7740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56034bcf05e0 0x56034bced870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:13:23.432 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:23.425+0000 7fc7d6be7740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x56034bcf3210 con 0x56034bce7810 2024-03-18T23:13:23.432 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:23.425+0000 7fc7d6be7740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x56034bc06db0 con 0x56034bce6ef0 2024-03-18T23:13:23.432 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:23.425+0000 7fc7d6be7740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x56034bcf3390 con 0x56034bcf05e0 2024-03-18T23:13:23.432 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:23.425+0000 7fc7d6be7740 10 client.0 _ll_register_callbacks cb 0x56034bc0be70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T23:13:23.432 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:23.425+0000 7fc7d14b6640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56034bce7810 0x56034bced2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:13:23.432 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:23.425+0000 7fc7d14b6640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56034bce7810 0x56034bced2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:45408/0 (socket says 172.21.15.49:45408) 2024-03-18T23:13:23.432 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:23.425+0000 7fc7d1cb7640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56034bce6ef0 0x56034bcefef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:13:23.432 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:23.425+0000 7fc7d14b6640 1 -- 172.21.15.49:0/3150508890 learned_addr learned my addr 172.21.15.49:0/3150508890 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:13:23.433 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:23.425+0000 7fc7d24b8640 1 --2- 172.21.15.49:0/3150508890 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56034bcf05e0 0x56034bced870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:13:23.433 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:23.425+0000 7fc7d14b6640 1 -- 172.21.15.49:0/3150508890 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56034bcf05e0 msgr2=0x56034bced870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:13:23.433 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:23.425+0000 7fc7d14b6640 1 --2- 172.21.15.49:0/3150508890 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56034bcf05e0 0x56034bced870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:23.433 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:23.425+0000 7fc7d14b6640 1 -- 172.21.15.49:0/3150508890 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56034bce6ef0 msgr2=0x56034bcefef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:13:23.433 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:23.425+0000 7fc7d14b6640 1 --2- 172.21.15.49:0/3150508890 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56034bce6ef0 0x56034bcefef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:23.433 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:23.425+0000 7fc7d14b6640 1 -- 172.21.15.49:0/3150508890 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56034bc05c60 con 0x56034bce7810 2024-03-18T23:13:23.433 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:23.425+0000 7fc7d1cb7640 1 --2- 172.21.15.49:0/3150508890 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56034bce6ef0 0x56034bcefef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:13:23.433 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:23.425+0000 7fc7d14b6640 1 --2- 172.21.15.49:0/3150508890 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56034bce7810 0x56034bced2f0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fc7bc009c70 tx=0x7fc7bc019aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65580c31d9b28b01 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:13:23.433 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:23.425+0000 7fc7b57fa640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-18T23:13:23.433 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:23.426+0000 7fc7b57fa640 1 -- 172.21.15.49:0/3150508890 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7bc028020 con 0x56034bce7810 2024-03-18T23:13:23.433 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:23.426+0000 7fc7b57fa640 1 -- 172.21.15.49:0/3150508890 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8512+0+0 (secure 0 0 0) 0x7fc7bc005050 con 0x56034bce7810 2024-03-18T23:13:23.433 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:23.426+0000 7fc7d6be7740 1 -- 172.21.15.49:0/3150508890 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc7bc019b10 con 0x56034bce7810 2024-03-18T23:13:23.433 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:23.426+0000 7fc7b57fa640 1 -- 172.21.15.49:0/3150508890 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fc7bc033050 con 0x56034bce7810 2024-03-18T23:13:23.433 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:23.426+0000 7fc7d6be7740 10 client.15870 fetch_fsmap learned FSMap version 32 2024-03-18T23:13:23.433 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:23.426+0000 7fc7d6be7740 1 -- 172.21.15.49:0/3150508890 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x56034b95f210 con 0x56034bce7810 2024-03-18T23:13:23.434 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:23.426+0000 7fc7b57fa640 1 -- 172.21.15.49:0/3150508890 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc7bc019b10 con 0x56034bce7810 2024-03-18T23:13:23.434 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:23.426+0000 7fc7b57fa640 1 -- 172.21.15.49:0/3150508890 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7fc7bc0046c0 con 0x56034bce7810 2024-03-18T23:13:23.434 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:23.426+0000 7fc7d6be7740 10 client.15870 fetch_fsmap finished waiting for FSMap version 32 2024-03-18T23:13:23.434 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:23.426+0000 7fc7d6be7740 10 client.15870 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T23:13:23.434 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:23.426+0000 7fc7d6be7740 20 client.15870 populate_metadata read hostname 'smithi049' 2024-03-18T23:13:23.434 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:23.426+0000 7fc7d6be7740 1 --2- 172.21.15.49:0/3150508890 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x56034bd00720 0x56034bd02bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:13:23.434 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:23.426+0000 7fc7d6be7740 4 client.15870 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T23:13:23.434 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:23.426+0000 7fc7d6be7740 1 -- 172.21.15.49:0/3150508890 --> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56034bc58120 con 0x56034bd00720 2024-03-18T23:13:23.434 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:23.426+0000 7fc7d1cb7640 1 --2- 172.21.15.49:0/3150508890 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x56034bd00720 0x56034bd02bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:13:23.434 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:23.427+0000 7fc7d1cb7640 1 --2- 172.21.15.49:0/3150508890 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x56034bd00720 0x56034bd02bc0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6996a9aa1e60bdb1 server_cookie=eee37d9b770ca8a0 in_seq=0 out_seq=0 2024-03-18T23:13:23.434 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:23.427+0000 7fc7b57fa640 10 client.15870 ms_handle_connect on v2:172.21.15.139:6838/4280128209 2024-03-18T23:13:23.434 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:23.428+0000 7fc7b57fa640 1 -- 172.21.15.49:0/3150508890 <== mds.0 v2:172.21.15.139:6838/4280128209 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x56034bc58120 con 0x56034bd00720 2024-03-18T23:13:23.434 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:23.428+0000 7fc7b57fa640 10 client.15870 handle_command_reply: tid=0 2024-03-18T23:13:23.435 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:23.428+0000 7fc7d6be7740 1 client.15870 shutdown 2024-03-18T23:13:23.435 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:23.428+0000 7fc7d6be7740 1 -- 172.21.15.49:0/3150508890 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56034bce7810 msgr2=0x56034bced2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:13:23.435 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:23.428+0000 7fc7d6be7740 1 --2- 172.21.15.49:0/3150508890 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56034bce7810 0x56034bced2f0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fc7bc009c70 tx=0x7fc7bc019aa0 comp rx=0 tx=0).stop 2024-03-18T23:13:23.435 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:23.428+0000 7fc7d6be7740 1 -- 172.21.15.49:0/3150508890 shutdown_connections 2024-03-18T23:13:23.435 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:23.428+0000 7fc7d6be7740 1 --2- 172.21.15.49:0/3150508890 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x56034bd00720 0x56034bd02bc0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:23.435 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:23.428+0000 7fc7d6be7740 1 --2- 172.21.15.49:0/3150508890 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56034bcf05e0 0x56034bced870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:23.435 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:23.428+0000 7fc7d6be7740 1 --2- 172.21.15.49:0/3150508890 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56034bce7810 0x56034bced2f0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:23.435 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:23.428+0000 7fc7d6be7740 1 --2- 172.21.15.49:0/3150508890 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56034bce6ef0 0x56034bcefef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:23.435 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:23.428+0000 7fc7b57fa640 0 client.15870 ms_handle_reset on v2:172.21.15.139:6838/4280128209 2024-03-18T23:13:23.435 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:23.428+0000 7fc7d6be7740 1 -- 172.21.15.49:0/3150508890 >> 172.21.15.49:0/3150508890 conn(0x56034bce1810 msgr2=0x56034bce3560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:13:23.435 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:23.428+0000 7fc7d6be7740 1 -- 172.21.15.49:0/3150508890 shutdown_connections 2024-03-18T23:13:23.435 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:23.428+0000 7fc7d6be7740 1 -- 172.21.15.49:0/3150508890 wait complete. 2024-03-18T23:13:23.435 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:23.428+0000 7fc7d6be7740 20 client.15870 trim_cache size 0 max 16384 2024-03-18T23:13:23.447 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:23.443+0000 7fc7d6be7740 1 Processor -- start 2024-03-18T23:13:23.447 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:23.443+0000 7fc7d6be7740 1 -- start start 2024-03-18T23:13:23.447 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:23.443+0000 7fc7d6be7740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56034bce6ef0 0x56034bcfbfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:13:23.447 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:23.443+0000 7fc7d6be7740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56034bce7810 0x56034bd084d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:13:23.448 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:23.443+0000 7fc7d6be7740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56034bcf05e0 0x56034bcebbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:13:23.448 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:23.443+0000 7fc7d6be7740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fc7cc1ad470 con 0x56034bcf05e0 2024-03-18T23:13:23.448 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:23.443+0000 7fc7d6be7740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fc7cc1aefb0 con 0x56034bce7810 2024-03-18T23:13:23.448 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:23.443+0000 7fc7d6be7740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc7cc1adb40 con 0x56034bce6ef0 2024-03-18T23:13:23.448 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:23.443+0000 7fc7d24b8640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56034bcf05e0 0x56034bcebbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:13:23.448 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:23.443+0000 7fc7d1cb7640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56034bce6ef0 0x56034bcfbfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:13:23.448 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:23.443+0000 7fc7d24b8640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56034bcf05e0 0x56034bcebbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:45424/0 (socket says 172.21.15.49:45424) 2024-03-18T23:13:23.448 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:23.443+0000 7fc7d24b8640 1 -- 172.21.15.49:0/2996933377 learned_addr learned my addr 172.21.15.49:0/2996933377 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:13:23.448 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:23.443+0000 7fc7d14b6640 1 --2- 172.21.15.49:0/2996933377 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56034bce7810 0x56034bd084d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:13:23.448 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:23.444+0000 7fc7d24b8640 1 -- 172.21.15.49:0/2996933377 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56034bce6ef0 msgr2=0x56034bcfbfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:13:23.448 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:23.444+0000 7fc7d24b8640 1 --2- 172.21.15.49:0/2996933377 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56034bce6ef0 0x56034bcfbfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:23.448 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:23.444+0000 7fc7d24b8640 1 -- 172.21.15.49:0/2996933377 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56034bce7810 msgr2=0x56034bd084d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:13:23.448 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:23.444+0000 7fc7d24b8640 1 --2- 172.21.15.49:0/2996933377 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56034bce7810 0x56034bd084d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:23.448 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:23.444+0000 7fc7d24b8640 1 -- 172.21.15.49:0/2996933377 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56034ba2b620 con 0x56034bcf05e0 2024-03-18T23:13:23.449 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:23.444+0000 7fc7d24b8640 1 --2- 172.21.15.49:0/2996933377 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56034bcf05e0 0x56034bcebbc0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fc7c4009ef0 tx=0x7fc7c4013070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=619c0e4c126eb256 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:13:23.449 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:23.444+0000 7fc7b77fe640 1 -- 172.21.15.49:0/2996933377 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7c401ed90 con 0x56034bcf05e0 2024-03-18T23:13:23.449 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:23.444+0000 7fc7b77fe640 1 -- 172.21.15.49:0/2996933377 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc7c400fcc0 con 0x56034bcf05e0 2024-03-18T23:13:23.449 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:23.444+0000 7fc7b77fe640 1 -- 172.21.15.49:0/2996933377 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7c4005730 con 0x56034bcf05e0 2024-03-18T23:13:23.449 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:23.445+0000 7fc7d6be7740 1 -- 172.21.15.49:0/2996933377 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56034bcf05e0 msgr2=0x56034bcebbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:13:23.449 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:23.445+0000 7fc7d6be7740 1 --2- 172.21.15.49:0/2996933377 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56034bcf05e0 0x56034bcebbc0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fc7c4009ef0 tx=0x7fc7c4013070 comp rx=0 tx=0).stop 2024-03-18T23:13:23.449 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:23.445+0000 7fc7d6be7740 1 -- 172.21.15.49:0/2996933377 shutdown_connections 2024-03-18T23:13:23.449 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:23.445+0000 7fc7d6be7740 1 --2- 172.21.15.49:0/2996933377 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56034bcf05e0 0x56034bcebbc0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:23.449 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:23.445+0000 7fc7d6be7740 1 --2- 172.21.15.49:0/2996933377 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56034bce7810 0x56034bd084d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:23.449 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:23.445+0000 7fc7d6be7740 1 --2- 172.21.15.49:0/2996933377 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56034bce6ef0 0x56034bcfbfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:23.449 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:23.445+0000 7fc7d6be7740 1 -- 172.21.15.49:0/2996933377 >> 172.21.15.49:0/2996933377 conn(0x56034bce1810 msgr2=0x56034bcec830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:13:23.449 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:23.445+0000 7fc7d6be7740 1 -- 172.21.15.49:0/2996933377 shutdown_connections 2024-03-18T23:13:23.449 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:23.445+0000 7fc7d6be7740 1 -- 172.21.15.49:0/2996933377 wait complete. 2024-03-18T23:13:23.449 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:23.445+0000 7fc7d6be7740 1 -- start start 2024-03-18T23:13:23.449 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:23.445+0000 7fc7d6be7740 1 Processor -- start 2024-03-18T23:13:23.450 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:23.445+0000 7fc7d6be7740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56034bce6ef0 0x56034bcea570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:13:23.450 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:23.445+0000 7fc7d6be7740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56034bce7810 0x56034bceaab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:13:23.450 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:23.445+0000 7fc7d6be7740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56034bcf05e0 0x56034bceaff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:13:23.450 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:23.445+0000 7fc7d6be7740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fc7cc1ae8e0 con 0x56034bcf05e0 2024-03-18T23:13:23.450 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:23.445+0000 7fc7d6be7740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fc7cc1ae210 con 0x56034bce6ef0 2024-03-18T23:13:23.450 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:23.445+0000 7fc7d6be7740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc7cc1ab430 con 0x56034bce7810 2024-03-18T23:13:23.450 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:23.445+0000 7fc7d6be7740 10 client.0 _ll_register_callbacks cb 0x7fc7bc01c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T23:13:23.450 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:23.446+0000 7fc7d24b8640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56034bcf05e0 0x56034bceaff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:13:23.450 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:23.446+0000 7fc7d1cb7640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56034bce6ef0 0x56034bcea570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:13:23.450 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:23.446+0000 7fc7d24b8640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56034bcf05e0 0x56034bceaff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:45434/0 (socket says 172.21.15.49:45434) 2024-03-18T23:13:23.450 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:23.446+0000 7fc7d24b8640 1 -- 172.21.15.49:0/1365577833 learned_addr learned my addr 172.21.15.49:0/1365577833 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:13:23.450 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:23.446+0000 7fc7d14b6640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56034bce7810 0x56034bceaab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:13:23.451 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:23.446+0000 7fc7d24b8640 1 -- 172.21.15.49:0/1365577833 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56034bce7810 msgr2=0x56034bceaab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:13:23.451 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:23.446+0000 7fc7d24b8640 1 --2- 172.21.15.49:0/1365577833 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56034bce7810 0x56034bceaab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:23.451 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:23.446+0000 7fc7d24b8640 1 -- 172.21.15.49:0/1365577833 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56034bce6ef0 msgr2=0x56034bcea570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:13:23.451 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:23.446+0000 7fc7d24b8640 1 --2- 172.21.15.49:0/1365577833 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56034bce6ef0 0x56034bcea570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:23.451 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:23.446+0000 7fc7d24b8640 1 -- 172.21.15.49:0/1365577833 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56034bd075f0 con 0x56034bcf05e0 2024-03-18T23:13:23.451 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:23.446+0000 7fc7d24b8640 1 --2- 172.21.15.49:0/1365577833 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56034bcf05e0 0x56034bceaff0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fc7c4005270 tx=0x7fc7c4005950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e75063950035c77a server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:13:23.451 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:23.446+0000 7fc7b57fa640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-18T23:13:23.451 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:23.446+0000 7fc7b57fa640 1 -- 172.21.15.49:0/1365577833 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7c40059c0 con 0x56034bcf05e0 2024-03-18T23:13:23.451 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:23.446+0000 7fc7b57fa640 1 -- 172.21.15.49:0/1365577833 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8512+0+0 (secure 0 0 0) 0x7fc7c4004c50 con 0x56034bcf05e0 2024-03-18T23:13:23.451 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:23.446+0000 7fc7d6be7740 1 -- 172.21.15.49:0/1365577833 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc7c400fcc0 con 0x56034bcf05e0 2024-03-18T23:13:23.451 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:23.446+0000 7fc7b57fa640 1 -- 172.21.15.49:0/1365577833 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fc7c4042050 con 0x56034bcf05e0 2024-03-18T23:13:23.451 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:23.446+0000 7fc7d6be7740 10 client.15876 fetch_fsmap learned FSMap version 32 2024-03-18T23:13:23.451 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:23.446+0000 7fc7d6be7740 1 -- 172.21.15.49:0/1365577833 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x56034bd69f50 con 0x56034bcf05e0 2024-03-18T23:13:23.451 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:23.447+0000 7fc7b57fa640 1 -- 172.21.15.49:0/1365577833 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc7c400fcc0 con 0x56034bcf05e0 2024-03-18T23:13:23.451 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:23.447+0000 7fc7b57fa640 1 -- 172.21.15.49:0/1365577833 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7fc7c401d070 con 0x56034bcf05e0 2024-03-18T23:13:23.452 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:23.447+0000 7fc7d6be7740 10 client.15876 fetch_fsmap finished waiting for FSMap version 32 2024-03-18T23:13:23.452 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:23.447+0000 7fc7d6be7740 10 client.15876 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T23:13:23.452 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:23.447+0000 7fc7d6be7740 20 client.15876 populate_metadata read hostname 'smithi049' 2024-03-18T23:13:23.452 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:23.447+0000 7fc7d6be7740 1 --2- 172.21.15.49:0/1365577833 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x56034bd6a660 0x56034bd6ca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:13:23.452 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:23.447+0000 7fc7d6be7740 4 client.15876 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T23:13:23.452 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:23.447+0000 7fc7d6be7740 1 -- 172.21.15.49:0/1365577833 --> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56034bc52470 con 0x56034bd6a660 2024-03-18T23:13:23.452 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:23.447+0000 7fc7d1cb7640 1 --2- 172.21.15.49:0/1365577833 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x56034bd6a660 0x56034bd6ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:13:23.452 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:23.447+0000 7fc7d1cb7640 1 --2- 172.21.15.49:0/1365577833 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x56034bd6a660 0x56034bd6ca50 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a8047b20dc4ff26d server_cookie=b9d61381c3dff564 in_seq=0 out_seq=0 2024-03-18T23:13:23.452 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:23.447+0000 7fc7b57fa640 10 client.15876 ms_handle_connect on v2:172.21.15.139:6838/4280128209 2024-03-18T23:13:23.600 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:23.599+0000 7fc7b57fa640 1 -- 172.21.15.49:0/1365577833 <== mds.0 v2:172.21.15.139:6838/4280128209 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56034bc52470 con 0x56034bd6a660 2024-03-18T23:13:23.600 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:23.599+0000 7fc7b57fa640 10 client.15876 handle_command_reply: tid=0 2024-03-18T23:13:23.600 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:23.599+0000 7fc7d6be7740 1 client.15876 shutdown 2024-03-18T23:13:23.600 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:23.599+0000 7fc7d6be7740 1 -- 172.21.15.49:0/1365577833 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56034bcf05e0 msgr2=0x56034bceaff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:13:23.600 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:23.599+0000 7fc7d6be7740 1 --2- 172.21.15.49:0/1365577833 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56034bcf05e0 0x56034bceaff0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fc7c4005270 tx=0x7fc7c4005950 comp rx=0 tx=0).stop 2024-03-18T23:13:23.600 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:23.600+0000 7fc7d6be7740 1 -- 172.21.15.49:0/1365577833 shutdown_connections 2024-03-18T23:13:23.600 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:23.600+0000 7fc7d6be7740 1 --2- 172.21.15.49:0/1365577833 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x56034bd6a660 0x56034bd6ca50 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:23.600 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:23.600+0000 7fc7b57fa640 0 client.15876 ms_handle_reset on v2:172.21.15.139:6838/4280128209 2024-03-18T23:13:23.601 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:23.600+0000 7fc7d6be7740 1 --2- 172.21.15.49:0/1365577833 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56034bcf05e0 0x56034bceaff0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:23.601 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:23.600+0000 7fc7d6be7740 1 --2- 172.21.15.49:0/1365577833 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56034bce7810 0x56034bceaab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:23.601 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:23.600+0000 7fc7d6be7740 1 --2- 172.21.15.49:0/1365577833 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56034bce6ef0 0x56034bcea570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:23.601 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:23.600+0000 7fc7d6be7740 1 -- 172.21.15.49:0/1365577833 >> 172.21.15.49:0/1365577833 conn(0x56034bce1810 msgr2=0x56034bcebb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:13:23.601 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:23.600+0000 7fc7d6be7740 1 -- 172.21.15.49:0/1365577833 shutdown_connections 2024-03-18T23:13:23.601 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:23.600+0000 7fc7d6be7740 1 -- 172.21.15.49:0/1365577833 wait complete. 2024-03-18T23:13:23.601 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:23.600+0000 7fc7d6be7740 20 client.15876 trim_cache size 0 max 16384 2024-03-18T23:13:23.601 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:{ 2024-03-18T23:13:23.601 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "message": "", 2024-03-18T23:13:23.601 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "return_code": 0 2024-03-18T23:13:23.601 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:} 2024-03-18T23:13:23.604 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:23.604+0000 7fc7d3f1b640 1 -- 172.21.15.49:0/2378759070 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7fc7b80a2a10 msgr2=0x7fc7b80a4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:13:23.604 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:23.604+0000 7fc7d3f1b640 1 --2- 172.21.15.49:0/2378759070 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7fc7b80a2a10 0x7fc7b80a4ed0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fc7c4009fd0 tx=0x7fc7c4009500 comp rx=0 tx=0).stop 2024-03-18T23:13:23.604 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:23.604+0000 7fc7d3f1b640 1 -- 172.21.15.49:0/2378759070 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc7cc100970 msgr2=0x7fc7cc198170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:13:23.604 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:23.604+0000 7fc7d3f1b640 1 --2- 172.21.15.49:0/2378759070 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc7cc100970 0x7fc7cc198170 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fc7c802efd0 tx=0x7fc7c80042f0 comp rx=0 tx=0).stop 2024-03-18T23:13:23.604 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:23.604+0000 7fc7d3f1b640 1 -- 172.21.15.49:0/2378759070 shutdown_connections 2024-03-18T23:13:23.604 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:23.604+0000 7fc7d3f1b640 1 --2- 172.21.15.49:0/2378759070 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7fc7b80a2a10 0x7fc7b80a4ed0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:23.604 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:23.604+0000 7fc7d3f1b640 1 --2- 172.21.15.49:0/2378759070 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc7cc1056c0 0x7fc7cc19c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:23.604 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:23.604+0000 7fc7d3f1b640 1 --2- 172.21.15.49:0/2378759070 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc7cc101330 0x7fc7cc1986b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:23.604 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:23.604+0000 7fc7d3f1b640 1 --2- 172.21.15.49:0/2378759070 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc7cc100970 0x7fc7cc198170 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:23.604 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:23.604+0000 7fc7d3f1b640 1 -- 172.21.15.49:0/2378759070 >> 172.21.15.49:0/2378759070 conn(0x7fc7cc077570 msgr2=0x7fc7cc069120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:13:23.604 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:23.604+0000 7fc7d3f1b640 1 -- 172.21.15.49:0/2378759070 shutdown_connections 2024-03-18T23:13:23.604 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:23.604+0000 7fc7d3f1b640 1 -- 172.21.15.49:0/2378759070 wait complete. 2024-03-18T23:13:23.771 INFO:tasks.workunit.client.0.smithi049.stdout: 1 554620 10.34 MB/sec execute 145 sec latency 79.432 ms 2024-03-18T23:13:24.771 INFO:tasks.workunit.client.0.smithi049.stdout: 1 556569 10.32 MB/sec execute 146 sec latency 30.398 ms 2024-03-18T23:13:24.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:13:24 smithi139 ceph-mon[67282]: pgmap v159: 97 pgs: 97 active+clean; 84 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.8 MiB/s wr, 181 op/s 2024-03-18T23:13:24.879 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:13:24 smithi049 ceph-mon[62991]: pgmap v159: 97 pgs: 97 active+clean; 84 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.8 MiB/s wr, 181 op/s 2024-03-18T23:13:24.961 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:13:24 smithi175 ceph-mon[67352]: pgmap v159: 97 pgs: 97 active+clean; 84 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.8 MiB/s wr, 181 op/s 2024-03-18T23:13:25.771 INFO:tasks.workunit.client.0.smithi049.stdout: 1 558213 10.33 MB/sec execute 147 sec latency 97.487 ms 2024-03-18T23:13:25.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:13:25 smithi139 ceph-mon[67282]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:13:25.879 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:13:25 smithi049 ceph-mon[62991]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:13:25.961 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:13:25 smithi175 ceph-mon[67352]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:13:26.771 INFO:tasks.workunit.client.0.smithi049.stdout: 1 559998 10.31 MB/sec execute 148 sec latency 33.098 ms 2024-03-18T23:13:26.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:13:26 smithi139 ceph-mon[67282]: pgmap v160: 97 pgs: 97 active+clean; 84 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.8 MiB/s wr, 180 op/s 2024-03-18T23:13:26.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:13:26 smithi139 ceph-mon[67282]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:13:26.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:13:26 smithi139 ceph-mon[67282]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:13:26.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:13:26 smithi139 ceph-mon[67282]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' 2024-03-18T23:13:26.879 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:13:26 smithi049 ceph-mon[62991]: pgmap v160: 97 pgs: 97 active+clean; 84 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.8 MiB/s wr, 180 op/s 2024-03-18T23:13:26.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:13:26 smithi049 ceph-mon[62991]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:13:26.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:13:26 smithi049 ceph-mon[62991]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:13:26.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:13:26 smithi049 ceph-mon[62991]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' 2024-03-18T23:13:26.961 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:13:26 smithi175 ceph-mon[67352]: pgmap v160: 97 pgs: 97 active+clean; 84 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.8 MiB/s wr, 180 op/s 2024-03-18T23:13:26.961 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:13:26 smithi175 ceph-mon[67352]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:13:26.961 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:13:26 smithi175 ceph-mon[67352]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:13:26.961 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:13:26 smithi175 ceph-mon[67352]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' 2024-03-18T23:13:27.771 INFO:tasks.workunit.client.0.smithi049.stdout: 1 561873 10.32 MB/sec execute 149 sec latency 10.980 ms 2024-03-18T23:13:28.771 INFO:tasks.workunit.client.0.smithi049.stdout: 1 564013 10.33 MB/sec execute 150 sec latency 25.940 ms 2024-03-18T23:13:28.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:13:28 smithi139 ceph-mon[67282]: pgmap v161: 97 pgs: 97 active+clean; 79 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 4.2 MiB/s wr, 359 op/s 2024-03-18T23:13:28.879 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:13:28 smithi049 ceph-mon[62991]: pgmap v161: 97 pgs: 97 active+clean; 79 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 4.2 MiB/s wr, 359 op/s 2024-03-18T23:13:28.961 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:13:28 smithi175 ceph-mon[67352]: pgmap v161: 97 pgs: 97 active+clean; 79 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 4.2 MiB/s wr, 359 op/s 2024-03-18T23:13:29.771 INFO:tasks.workunit.client.0.smithi049.stdout: 1 565697 10.32 MB/sec execute 151 sec latency 25.512 ms 2024-03-18T23:13:30.771 INFO:tasks.workunit.client.0.smithi049.stdout: 1 567402 10.30 MB/sec execute 152 sec latency 75.725 ms 2024-03-18T23:13:30.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:13:30 smithi139 ceph-mon[67282]: pgmap v162: 97 pgs: 97 active+clean; 79 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.9 MiB/s wr, 264 op/s 2024-03-18T23:13:30.879 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:13:30 smithi049 ceph-mon[62991]: pgmap v162: 97 pgs: 97 active+clean; 79 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.9 MiB/s wr, 264 op/s 2024-03-18T23:13:30.961 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:13:30 smithi175 ceph-mon[67352]: pgmap v162: 97 pgs: 97 active+clean; 79 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.9 MiB/s wr, 264 op/s 2024-03-18T23:13:31.771 INFO:tasks.workunit.client.0.smithi049.stdout: 1 569157 10.30 MB/sec execute 153 sec latency 17.966 ms 2024-03-18T23:13:32.771 INFO:tasks.workunit.client.0.smithi049.stdout: 1 571352 10.32 MB/sec execute 154 sec latency 36.051 ms 2024-03-18T23:13:32.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:13:32 smithi139 ceph-mon[67282]: pgmap v163: 97 pgs: 97 active+clean; 82 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 4.0 MiB/s wr, 353 op/s 2024-03-18T23:13:32.879 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:13:32 smithi049 ceph-mon[62991]: pgmap v163: 97 pgs: 97 active+clean; 82 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 4.0 MiB/s wr, 353 op/s 2024-03-18T23:13:32.961 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:13:32 smithi175 ceph-mon[67352]: pgmap v163: 97 pgs: 97 active+clean; 82 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 4.0 MiB/s wr, 353 op/s 2024-03-18T23:13:33.771 INFO:tasks.workunit.client.0.smithi049.stdout: 1 573042 10.30 MB/sec execute 155 sec latency 21.806 ms 2024-03-18T23:13:34.771 INFO:tasks.workunit.client.0.smithi049.stdout: 1 574873 10.31 MB/sec execute 156 sec latency 122.468 ms 2024-03-18T23:13:34.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:13:34 smithi139 ceph-mon[67282]: pgmap v164: 97 pgs: 97 active+clean; 82 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.6 MiB/s wr, 267 op/s 2024-03-18T23:13:34.879 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:13:34 smithi049 ceph-mon[62991]: pgmap v164: 97 pgs: 97 active+clean; 82 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.6 MiB/s wr, 267 op/s 2024-03-18T23:13:34.961 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:13:34 smithi175 ceph-mon[67352]: pgmap v164: 97 pgs: 97 active+clean; 82 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.6 MiB/s wr, 267 op/s 2024-03-18T23:13:35.771 INFO:tasks.workunit.client.0.smithi049.stdout: 1 576360 10.29 MB/sec execute 157 sec latency 77.330 ms 2024-03-18T23:13:36.701 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:36.699+0000 7f4bd564d640 1 -- 172.21.15.49:0/2019487475 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4bd01077e0 msgr2=0x7f4bd0107bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:13:36.701 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:36.699+0000 7f4bd564d640 1 --2- 172.21.15.49:0/2019487475 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4bd01077e0 0x7f4bd0107bc0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f4bc0009930 tx=0x7f4bc002bc30 comp rx=0 tx=0).stop 2024-03-18T23:13:36.701 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:36.700+0000 7f4bd564d640 1 -- 172.21.15.49:0/2019487475 shutdown_connections 2024-03-18T23:13:36.701 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:36.700+0000 7f4bd564d640 1 --2- 172.21.15.49:0/2019487475 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4bd006a980 0x7f4bd010e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:36.701 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:36.700+0000 7f4bd564d640 1 --2- 172.21.15.49:0/2019487475 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4bd0069fc0 0x7f4bd006a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:36.701 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:36.700+0000 7f4bd564d640 1 --2- 172.21.15.49:0/2019487475 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4bd01077e0 0x7f4bd0107bc0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:36.701 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:36.700+0000 7f4bd564d640 1 -- 172.21.15.49:0/2019487475 >> 172.21.15.49:0/2019487475 conn(0x7f4bd0077700 msgr2=0x7f4bd0077b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:13:36.701 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:36.700+0000 7f4bd564d640 1 -- 172.21.15.49:0/2019487475 shutdown_connections 2024-03-18T23:13:36.701 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:36.700+0000 7f4bd564d640 1 -- 172.21.15.49:0/2019487475 wait complete. 2024-03-18T23:13:36.702 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:36.700+0000 7f4bd564d640 1 Processor -- start 2024-03-18T23:13:36.704 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:36.701+0000 7f4bd564d640 1 -- start start 2024-03-18T23:13:36.704 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:36.701+0000 7f4bd564d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4bd0069fc0 0x7f4bd0100ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:13:36.704 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:36.701+0000 7f4bd564d640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4bd006a980 0x7f4bd0101010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:13:36.704 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:36.701+0000 7f4bd564d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4bd01077e0 0x7f4bd0101550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:13:36.704 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:36.701+0000 7f4bd564d640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f4bd0111bf0 con 0x7f4bd0069fc0 2024-03-18T23:13:36.704 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:36.701+0000 7f4bd564d640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f4bd0111a70 con 0x7f4bd01077e0 2024-03-18T23:13:36.704 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:36.701+0000 7f4bd564d640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f4bd0111d70 con 0x7f4bd006a980 2024-03-18T23:13:36.704 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:36.701+0000 7f4bceffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4bd0069fc0 0x7f4bd0100ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:13:36.704 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:36.701+0000 7f4bce7fc640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4bd006a980 0x7f4bd0101010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:13:36.704 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:36.701+0000 7f4bceffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4bd0069fc0 0x7f4bd0100ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48042/0 (socket says 172.21.15.49:48042) 2024-03-18T23:13:36.704 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:36.701+0000 7f4bcf7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4bd01077e0 0x7f4bd0101550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:13:36.705 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:36.701+0000 7f4bce7fc640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4bd006a980 0x7f4bd0101010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.49:48748/0 (socket says 172.21.15.49:48748) 2024-03-18T23:13:36.705 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:36.701+0000 7f4bceffd640 1 -- 172.21.15.49:0/1513972690 learned_addr learned my addr 172.21.15.49:0/1513972690 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:13:36.705 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:36.701+0000 7f4bceffd640 1 -- 172.21.15.49:0/1513972690 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4bd006a980 msgr2=0x7f4bd0101010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:13:36.705 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:36.701+0000 7f4bceffd640 1 --2- 172.21.15.49:0/1513972690 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4bd006a980 0x7f4bd0101010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:36.705 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:36.701+0000 7f4bceffd640 1 -- 172.21.15.49:0/1513972690 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4bd01077e0 msgr2=0x7f4bd0101550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:13:36.705 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:36.701+0000 7f4bceffd640 1 --2- 172.21.15.49:0/1513972690 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4bd01077e0 0x7f4bd0101550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:36.705 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:36.701+0000 7f4bceffd640 1 -- 172.21.15.49:0/1513972690 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bd01a64f0 con 0x7f4bd0069fc0 2024-03-18T23:13:36.705 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:36.701+0000 7f4bce7fc640 1 --2- 172.21.15.49:0/1513972690 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4bd006a980 0x7f4bd0101010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:13:36.705 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:36.702+0000 7f4bceffd640 1 --2- 172.21.15.49:0/1513972690 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4bd0069fc0 0x7f4bd0100ad0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f4bc0009230 tx=0x7f4bc002cb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:13:36.705 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:36.702+0000 7f4bc7fff640 1 -- 172.21.15.49:0/1513972690 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bc0041070 con 0x7f4bd0069fc0 2024-03-18T23:13:36.705 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:36.702+0000 7f4bc7fff640 1 -- 172.21.15.49:0/1513972690 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4bc0004450 con 0x7f4bd0069fc0 2024-03-18T23:13:36.705 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:36.702+0000 7f4bd564d640 1 -- 172.21.15.49:0/1513972690 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4bd01a6780 con 0x7f4bd0069fc0 2024-03-18T23:13:36.706 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:36.702+0000 7f4bc7fff640 1 -- 172.21.15.49:0/1513972690 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bc0003e50 con 0x7f4bd0069fc0 2024-03-18T23:13:36.706 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:36.702+0000 7f4bd564d640 1 -- 172.21.15.49:0/1513972690 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4bd01a6cc0 con 0x7f4bd0069fc0 2024-03-18T23:13:36.712 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:36.704+0000 7f4bc7fff640 1 -- 172.21.15.49:0/1513972690 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 25) ==== 137421+0+0 (secure 0 0 0) 0x7f4bc0044070 con 0x7f4bd0069fc0 2024-03-18T23:13:36.712 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:36.704+0000 7f4bd8231740 1 Processor -- start 2024-03-18T23:13:36.712 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:36.705+0000 7f4bd8231740 1 -- start start 2024-03-18T23:13:36.712 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:36.705+0000 7f4bd8231740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fa3d6ccef0 0x55fa3d6cd2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:13:36.712 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:36.705+0000 7f4bc7fff640 1 --2- 172.21.15.49:0/1513972690 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f4bbc0a2c60 0x7f4bbc0a5120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:13:36.713 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:36.705+0000 7f4bd8231740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55fa3d6cd810 0x55fa3d6d60a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:13:36.713 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:36.705+0000 7f4bd8231740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fa3d6d65e0 0x55fa3d6d89d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:13:36.713 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:36.705+0000 7f4bd8231740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55fa3d630be0 con 0x55fa3d6d65e0 2024-03-18T23:13:36.713 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:36.705+0000 7f4bd8231740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55fa3d5d9960 con 0x55fa3d6cd810 2024-03-18T23:13:36.713 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:36.705+0000 7f4bd8231740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55fa3d62cc80 con 0x55fa3d6ccef0 2024-03-18T23:13:36.713 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:36.705+0000 7f4bcf7fe640 1 --2- 172.21.15.49:0/1513972690 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f4bbc0a2c60 0x7f4bbc0a5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:13:36.713 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:36.705+0000 7f4bce7fc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55fa3d6cd810 0x55fa3d6d60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:13:36.713 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:36.705+0000 7f4bceffd640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fa3d6ccef0 0x55fa3d6cd2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:13:36.713 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:36.705+0000 7f4bce7fc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55fa3d6cd810 0x55fa3d6d60a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.49:58196/0 (socket says 172.21.15.49:58196) 2024-03-18T23:13:36.713 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:36.705+0000 7f4bce7fc640 1 -- 172.21.15.49:0/2115136077 learned_addr learned my addr 172.21.15.49:0/2115136077 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:13:36.713 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:36.705+0000 7f4bcf7fe640 1 --2- 172.21.15.49:0/2115136077 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fa3d6d65e0 0x55fa3d6d89d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:13:36.713 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:36.705+0000 7f4bc7fff640 1 -- 172.21.15.49:0/1513972690 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8512+0+0 (secure 0 0 0) 0x7f4bc00f0530 con 0x7f4bd0069fc0 2024-03-18T23:13:36.714 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:36.705+0000 7f4bce7fc640 1 -- 172.21.15.49:0/2115136077 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fa3d6ccef0 msgr2=0x55fa3d6cd2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:13:36.714 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:36.705+0000 7f4bce7fc640 1 --2- 172.21.15.49:0/2115136077 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fa3d6ccef0 0x55fa3d6cd2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:36.714 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:36.705+0000 7f4bce7fc640 1 -- 172.21.15.49:0/2115136077 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fa3d6d65e0 msgr2=0x55fa3d6d89d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:13:36.714 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:36.705+0000 7f4bce7fc640 1 --2- 172.21.15.49:0/2115136077 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fa3d6d65e0 0x55fa3d6d89d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:36.714 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:36.705+0000 7f4bce7fc640 1 -- 172.21.15.49:0/2115136077 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fa3d5d0d20 con 0x55fa3d6cd810 2024-03-18T23:13:36.714 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:36.705+0000 7f4bceffd640 1 --2- 172.21.15.49:0/2115136077 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fa3d6ccef0 0x55fa3d6cd2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:13:36.714 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:36.706+0000 7f4bcf7fe640 1 --2- 172.21.15.49:0/1513972690 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f4bbc0a2c60 0x7f4bbc0a5120 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f4bc8005fd0 tx=0x7f4bc8009e90 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:13:36.714 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:36.706+0000 7f4bce7fc640 1 --2- 172.21.15.49:0/2115136077 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55fa3d6cd810 0x55fa3d6d60a0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f4bb80114f0 tx=0x7f4bb80119b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d15301287b997669 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:13:36.714 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:36.706+0000 7f4bc57fa640 1 -- 172.21.15.49:0/2115136077 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bb8011030 con 0x55fa3d6cd810 2024-03-18T23:13:36.714 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:36.706+0000 7f4bc57fa640 1 -- 172.21.15.49:0/2115136077 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4bb80111d0 con 0x55fa3d6cd810 2024-03-18T23:13:36.714 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:36.706+0000 7f4bc57fa640 1 -- 172.21.15.49:0/2115136077 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bb800f5c0 con 0x55fa3d6cd810 2024-03-18T23:13:36.714 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:36.707+0000 7f4bd8231740 1 -- 172.21.15.49:0/2115136077 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55fa3d6cd810 msgr2=0x55fa3d6d60a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:13:36.715 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:36.707+0000 7f4bd8231740 1 --2- 172.21.15.49:0/2115136077 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55fa3d6cd810 0x55fa3d6d60a0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f4bb80114f0 tx=0x7f4bb80119b0 comp rx=0 tx=0).stop 2024-03-18T23:13:36.715 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:36.707+0000 7f4bd8231740 1 -- 172.21.15.49:0/2115136077 shutdown_connections 2024-03-18T23:13:36.715 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:36.707+0000 7f4bd8231740 1 --2- 172.21.15.49:0/2115136077 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fa3d6d65e0 0x55fa3d6d89d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:36.715 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:36.707+0000 7f4bd8231740 1 --2- 172.21.15.49:0/2115136077 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55fa3d6cd810 0x55fa3d6d60a0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:36.715 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:36.707+0000 7f4bd8231740 1 --2- 172.21.15.49:0/2115136077 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fa3d6ccef0 0x55fa3d6cd2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:36.715 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:36.707+0000 7f4bd8231740 1 -- 172.21.15.49:0/2115136077 >> 172.21.15.49:0/2115136077 conn(0x55fa3d6c7810 msgr2=0x55fa3d6c7bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:13:36.715 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:36.707+0000 7f4bd8231740 1 -- 172.21.15.49:0/2115136077 shutdown_connections 2024-03-18T23:13:36.715 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:36.707+0000 7f4bd8231740 1 -- 172.21.15.49:0/2115136077 wait complete. 2024-03-18T23:13:36.715 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:36.707+0000 7f4bd8231740 1 -- start start 2024-03-18T23:13:36.715 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:36.707+0000 7f4bd8231740 1 Processor -- start 2024-03-18T23:13:36.715 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:36.708+0000 7f4bd8231740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55fa3d6ccef0 0x55fa3d6d5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:13:36.715 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:36.708+0000 7f4bd8231740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fa3d6cd810 0x55fa3d6d32d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:13:36.715 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:36.708+0000 7f4bd8231740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fa3d6d65e0 0x55fa3d6d3830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:13:36.715 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:36.708+0000 7f4bd8231740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55fa3d6d9210 con 0x55fa3d6d65e0 2024-03-18T23:13:36.716 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:36.708+0000 7f4bd8231740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55fa3d5ecdb0 con 0x55fa3d6ccef0 2024-03-18T23:13:36.716 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:36.708+0000 7f4bd8231740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55fa3d6d9390 con 0x55fa3d6cd810 2024-03-18T23:13:36.716 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:36.708+0000 7f4bce7fc640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fa3d6cd810 0x55fa3d6d32d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:13:36.716 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:36.708+0000 7f4bceffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55fa3d6ccef0 0x55fa3d6d5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:13:36.716 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:36.708+0000 7f4bce7fc640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fa3d6cd810 0x55fa3d6d32d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.49:48768/0 (socket says 172.21.15.49:48768) 2024-03-18T23:13:36.716 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:36.708+0000 7f4bce7fc640 1 -- 172.21.15.49:0/889684609 learned_addr learned my addr 172.21.15.49:0/889684609 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:13:36.716 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:36.708+0000 7f4bd8231740 10 client.0 _ll_register_callbacks cb 0x55fa3d5f1e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T23:13:36.716 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:36.708+0000 7f4bcf7fe640 1 --2- 172.21.15.49:0/889684609 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fa3d6d65e0 0x55fa3d6d3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:13:36.716 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:36.708+0000 7f4bce7fc640 1 -- 172.21.15.49:0/889684609 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55fa3d6ccef0 msgr2=0x55fa3d6d5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:13:36.716 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:36.708+0000 7f4bce7fc640 1 --2- 172.21.15.49:0/889684609 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55fa3d6ccef0 0x55fa3d6d5f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:36.716 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:36.708+0000 7f4bce7fc640 1 -- 172.21.15.49:0/889684609 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fa3d6d65e0 msgr2=0x55fa3d6d3830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:13:36.716 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:36.708+0000 7f4bce7fc640 1 --2- 172.21.15.49:0/889684609 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fa3d6d65e0 0x55fa3d6d3830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:36.716 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:36.708+0000 7f4bce7fc640 1 -- 172.21.15.49:0/889684609 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fa3d5ebc60 con 0x55fa3d6cd810 2024-03-18T23:13:36.717 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:36.708+0000 7f4bcf7fe640 1 --2- 172.21.15.49:0/889684609 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fa3d6d65e0 0x55fa3d6d3830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:13:36.717 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:36.708+0000 7f4bce7fc640 1 --2- 172.21.15.49:0/889684609 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fa3d6cd810 0x55fa3d6d32d0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f4bb8005890 tx=0x7f4bb80056f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a6c0087577d94ab3 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:13:36.717 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:36.708+0000 7f4bb67fc640 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-03-18T23:13:36.717 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:36.709+0000 7f4bb67fc640 1 -- 172.21.15.49:0/889684609 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bb8027070 con 0x55fa3d6cd810 2024-03-18T23:13:36.717 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:36.709+0000 7f4bb67fc640 1 -- 172.21.15.49:0/889684609 <== mon.1 v2:172.21.15.175:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8512+0+0 (secure 0 0 0) 0x7f4bb80046c0 con 0x55fa3d6cd810 2024-03-18T23:13:36.717 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:36.709+0000 7f4bd8231740 1 -- 172.21.15.49:0/889684609 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4bb80111d0 con 0x55fa3d6cd810 2024-03-18T23:13:36.717 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:36.709+0000 7f4bb67fc640 1 -- 172.21.15.49:0/889684609 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f4bb8033050 con 0x55fa3d6cd810 2024-03-18T23:13:36.717 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:36.709+0000 7f4bd8231740 10 client.25696 fetch_fsmap learned FSMap version 32 2024-03-18T23:13:36.717 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:36.709+0000 7f4bd8231740 1 -- 172.21.15.49:0/889684609 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55fa3d345210 con 0x55fa3d6cd810 2024-03-18T23:13:36.717 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:36.709+0000 7f4bb67fc640 1 -- 172.21.15.49:0/889684609 <== mon.1 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4bb80111d0 con 0x55fa3d6cd810 2024-03-18T23:13:36.717 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:36.709+0000 7f4bb67fc640 1 -- 172.21.15.49:0/889684609 <== mon.1 v2:172.21.15.175:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f4bb8003f80 con 0x55fa3d6cd810 2024-03-18T23:13:36.717 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:36.709+0000 7f4bd8231740 10 client.25696 fetch_fsmap finished waiting for FSMap version 32 2024-03-18T23:13:36.717 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:36.709+0000 7f4bd8231740 10 client.25696 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T23:13:36.718 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:36.709+0000 7f4bd8231740 20 client.25696 populate_metadata read hostname 'smithi049' 2024-03-18T23:13:36.718 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:36.709+0000 7f4bd8231740 1 --2- 172.21.15.49:0/889684609 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x55fa3d6e6380 0x55fa3d6e8770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:13:36.718 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:36.709+0000 7f4bd8231740 4 client.25696 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T23:13:36.718 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:36.709+0000 7f4bd8231740 1 -- 172.21.15.49:0/889684609 --> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fa3d63e120 con 0x55fa3d6e6380 2024-03-18T23:13:36.718 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:36.710+0000 7f4bceffd640 1 --2- 172.21.15.49:0/889684609 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x55fa3d6e6380 0x55fa3d6e8770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:13:36.718 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:36.710+0000 7f4bceffd640 1 --2- 172.21.15.49:0/889684609 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x55fa3d6e6380 0x55fa3d6e8770 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=39ae10203e92095c server_cookie=1e529ccb0a1dd615 in_seq=0 out_seq=0 2024-03-18T23:13:36.718 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:36.710+0000 7f4bb67fc640 10 client.25696 ms_handle_connect on v2:172.21.15.139:6838/4280128209 2024-03-18T23:13:36.718 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:36.711+0000 7f4bb67fc640 1 -- 172.21.15.49:0/889684609 <== mds.0 v2:172.21.15.139:6838/4280128209 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55fa3d63e120 con 0x55fa3d6e6380 2024-03-18T23:13:36.718 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:36.711+0000 7f4bb67fc640 10 client.25696 handle_command_reply: tid=0 2024-03-18T23:13:36.718 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:36.711+0000 7f4bd8231740 1 client.25696 shutdown 2024-03-18T23:13:36.718 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:36.711+0000 7f4bd8231740 1 -- 172.21.15.49:0/889684609 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fa3d6cd810 msgr2=0x55fa3d6d32d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:13:36.718 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:36.711+0000 7f4bd8231740 1 --2- 172.21.15.49:0/889684609 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fa3d6cd810 0x55fa3d6d32d0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f4bb8005890 tx=0x7f4bb80056f0 comp rx=0 tx=0).stop 2024-03-18T23:13:36.718 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:36.711+0000 7f4bd8231740 1 -- 172.21.15.49:0/889684609 shutdown_connections 2024-03-18T23:13:36.719 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:36.711+0000 7f4bd8231740 1 --2- 172.21.15.49:0/889684609 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x55fa3d6e6380 0x55fa3d6e8770 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:36.719 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:36.711+0000 7f4bd8231740 1 --2- 172.21.15.49:0/889684609 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fa3d6d65e0 0x55fa3d6d3830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:36.719 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:36.711+0000 7f4bd8231740 1 --2- 172.21.15.49:0/889684609 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fa3d6cd810 0x55fa3d6d32d0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:36.719 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:36.711+0000 7f4bb67fc640 0 client.25696 ms_handle_reset on v2:172.21.15.139:6838/4280128209 2024-03-18T23:13:36.719 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:36.711+0000 7f4bd8231740 1 --2- 172.21.15.49:0/889684609 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55fa3d6ccef0 0x55fa3d6d5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:36.719 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:36.711+0000 7f4bd8231740 1 -- 172.21.15.49:0/889684609 >> 172.21.15.49:0/889684609 conn(0x55fa3d6c7810 msgr2=0x55fa3d6c6ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:13:36.719 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:36.711+0000 7f4bd8231740 1 -- 172.21.15.49:0/889684609 shutdown_connections 2024-03-18T23:13:36.719 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:36.711+0000 7f4bd8231740 1 -- 172.21.15.49:0/889684609 wait complete. 2024-03-18T23:13:36.719 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:36.711+0000 7f4bd8231740 20 client.25696 trim_cache size 0 max 16384 2024-03-18T23:13:36.727 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:36.727+0000 7f4bd8231740 1 Processor -- start 2024-03-18T23:13:36.728 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:36.727+0000 7f4bd8231740 1 -- start start 2024-03-18T23:13:36.729 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:36.727+0000 7f4bd8231740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fa3d6ccef0 0x55fa3d6e1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:13:36.729 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:36.727+0000 7f4bd8231740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fa3d6cd810 0x55fa3d6edf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:13:36.729 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:36.727+0000 7f4bd8231740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55fa3d6d65e0 0x55fa3d6d5a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:13:36.729 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:36.727+0000 7f4bd8231740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f4bd01af640 con 0x55fa3d6cd810 2024-03-18T23:13:36.729 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:36.727+0000 7f4bd8231740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f4bd01b1180 con 0x55fa3d6d65e0 2024-03-18T23:13:36.730 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:36.727+0000 7f4bd8231740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f4bd01afd10 con 0x55fa3d6ccef0 2024-03-18T23:13:36.730 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:36.727+0000 7f4bceffd640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fa3d6ccef0 0x55fa3d6e1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:13:36.730 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:36.727+0000 7f4bceffd640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fa3d6ccef0 0x55fa3d6e1fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.49:48772/0 (socket says 172.21.15.49:48772) 2024-03-18T23:13:36.730 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:36.728+0000 7f4bceffd640 1 -- 172.21.15.49:0/2317894931 learned_addr learned my addr 172.21.15.49:0/2317894931 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:13:36.730 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:36.728+0000 7f4bce7fc640 1 --2- 172.21.15.49:0/2317894931 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fa3d6cd810 0x55fa3d6edf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:13:36.730 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:36.728+0000 7f4bcf7fe640 1 --2- 172.21.15.49:0/2317894931 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55fa3d6d65e0 0x55fa3d6d5a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:13:36.730 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:36.728+0000 7f4bceffd640 1 -- 172.21.15.49:0/2317894931 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55fa3d6d65e0 msgr2=0x55fa3d6d5a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:13:36.730 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:36.728+0000 7f4bceffd640 1 --2- 172.21.15.49:0/2317894931 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55fa3d6d65e0 0x55fa3d6d5a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:36.730 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:36.728+0000 7f4bceffd640 1 -- 172.21.15.49:0/2317894931 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fa3d6cd810 msgr2=0x55fa3d6edf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:13:36.730 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:36.728+0000 7f4bceffd640 1 --2- 172.21.15.49:0/2317894931 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fa3d6cd810 0x55fa3d6edf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:36.730 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:36.728+0000 7f4bceffd640 1 -- 172.21.15.49:0/2317894931 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fa3d411620 con 0x55fa3d6ccef0 2024-03-18T23:13:36.730 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:36.728+0000 7f4bcf7fe640 1 --2- 172.21.15.49:0/2317894931 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55fa3d6d65e0 0x55fa3d6d5a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T23:13:36.730 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:36.728+0000 7f4bceffd640 1 --2- 172.21.15.49:0/2317894931 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fa3d6ccef0 0x55fa3d6e1fb0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f4bc00b91c0 tx=0x7f4bc00e0040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=61234b6fa3cf889d server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:13:36.731 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:36.728+0000 7f4bc4ff9640 1 -- 172.21.15.49:0/2317894931 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bc004b040 con 0x55fa3d6ccef0 2024-03-18T23:13:36.731 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:36.728+0000 7f4bc4ff9640 1 -- 172.21.15.49:0/2317894931 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4bc0004c80 con 0x55fa3d6ccef0 2024-03-18T23:13:36.731 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:36.728+0000 7f4bc4ff9640 1 -- 172.21.15.49:0/2317894931 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bc00bb060 con 0x55fa3d6ccef0 2024-03-18T23:13:36.734 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:36.729+0000 7f4bd8231740 1 -- 172.21.15.49:0/2317894931 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fa3d6ccef0 msgr2=0x55fa3d6e1fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:13:36.734 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:36.729+0000 7f4bd8231740 1 --2- 172.21.15.49:0/2317894931 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fa3d6ccef0 0x55fa3d6e1fb0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f4bc00b91c0 tx=0x7f4bc00e0040 comp rx=0 tx=0).stop 2024-03-18T23:13:36.734 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:36.729+0000 7f4bd8231740 1 -- 172.21.15.49:0/2317894931 shutdown_connections 2024-03-18T23:13:36.734 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:36.729+0000 7f4bd8231740 1 --2- 172.21.15.49:0/2317894931 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55fa3d6d65e0 0x55fa3d6d5a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:36.734 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:36.729+0000 7f4bd8231740 1 --2- 172.21.15.49:0/2317894931 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fa3d6cd810 0x55fa3d6edf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:36.734 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:36.729+0000 7f4bd8231740 1 --2- 172.21.15.49:0/2317894931 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fa3d6ccef0 0x55fa3d6e1fb0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:36.734 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:36.729+0000 7f4bd8231740 1 -- 172.21.15.49:0/2317894931 >> 172.21.15.49:0/2317894931 conn(0x55fa3d6c7810 msgr2=0x55fa3d6c6ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:13:36.735 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:36.729+0000 7f4bd8231740 1 -- 172.21.15.49:0/2317894931 shutdown_connections 2024-03-18T23:13:36.735 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:36.729+0000 7f4bd8231740 1 -- 172.21.15.49:0/2317894931 wait complete. 2024-03-18T23:13:36.735 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:36.730+0000 7f4bd8231740 1 -- start start 2024-03-18T23:13:36.735 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:36.730+0000 7f4bd8231740 1 Processor -- start 2024-03-18T23:13:36.735 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:36.730+0000 7f4bd8231740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fa3d6ccef0 0x55fa3d6de940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:13:36.735 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:36.730+0000 7f4bd8231740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fa3d6cd810 0x55fa3d6d9cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:13:36.735 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:36.730+0000 7f4bd8231740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55fa3d6d65e0 0x55fa3d6da250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:13:36.735 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:36.730+0000 7f4bd8231740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f4bd01b0ab0 con 0x55fa3d6cd810 2024-03-18T23:13:36.735 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:36.730+0000 7f4bd8231740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f4bd01b03e0 con 0x55fa3d6d65e0 2024-03-18T23:13:36.735 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:36.730+0000 7f4bd8231740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f4bd01ad600 con 0x55fa3d6ccef0 2024-03-18T23:13:36.735 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:36.730+0000 7f4bd8231740 10 client.0 _ll_register_callbacks cb 0x7f4bc8011720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T23:13:36.735 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:36.730+0000 7f4bce7fc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fa3d6cd810 0x55fa3d6d9cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:13:36.735 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:36.730+0000 7f4bceffd640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fa3d6ccef0 0x55fa3d6de940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:13:36.735 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:36.730+0000 7f4bce7fc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fa3d6cd810 0x55fa3d6d9cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48064/0 (socket says 172.21.15.49:48064) 2024-03-18T23:13:36.736 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:36.730+0000 7f4bce7fc640 1 -- 172.21.15.49:0/4049452678 learned_addr learned my addr 172.21.15.49:0/4049452678 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:13:36.736 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:36.730+0000 7f4bcf7fe640 1 --2- 172.21.15.49:0/4049452678 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55fa3d6d65e0 0x55fa3d6da250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:13:36.736 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:36.730+0000 7f4bce7fc640 1 -- 172.21.15.49:0/4049452678 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fa3d6ccef0 msgr2=0x55fa3d6de940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:13:36.736 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:36.730+0000 7f4bce7fc640 1 --2- 172.21.15.49:0/4049452678 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fa3d6ccef0 0x55fa3d6de940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:36.736 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:36.730+0000 7f4bce7fc640 1 -- 172.21.15.49:0/4049452678 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55fa3d6d65e0 msgr2=0x55fa3d6da250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:13:36.736 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:36.730+0000 7f4bce7fc640 1 --2- 172.21.15.49:0/4049452678 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55fa3d6d65e0 0x55fa3d6da250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:36.736 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:36.730+0000 7f4bce7fc640 1 -- 172.21.15.49:0/4049452678 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fa3d6daa80 con 0x55fa3d6cd810 2024-03-18T23:13:36.736 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:36.730+0000 7f4bceffd640 1 --2- 172.21.15.49:0/4049452678 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fa3d6ccef0 0x55fa3d6de940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:13:36.736 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:36.731+0000 7f4bce7fc640 1 --2- 172.21.15.49:0/4049452678 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fa3d6cd810 0x55fa3d6d9cb0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f4bb8009380 tx=0x7f4bb800f7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=336ada1f8a81e3a5 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:13:36.736 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:36.731+0000 7f4bb67fc640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-18T23:13:36.736 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:36.731+0000 7f4bb67fc640 1 -- 172.21.15.49:0/4049452678 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bb8030070 con 0x55fa3d6cd810 2024-03-18T23:13:36.736 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:36.731+0000 7f4bb67fc640 1 -- 172.21.15.49:0/4049452678 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8512+0+0 (secure 0 0 0) 0x7f4bb8011240 con 0x55fa3d6cd810 2024-03-18T23:13:36.736 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:36.731+0000 7f4bd8231740 1 -- 172.21.15.49:0/4049452678 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4bc0004c80 con 0x55fa3d6cd810 2024-03-18T23:13:36.736 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:36.731+0000 7f4bb67fc640 1 -- 172.21.15.49:0/4049452678 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f4bb8027040 con 0x55fa3d6cd810 2024-03-18T23:13:36.737 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:36.731+0000 7f4bd8231740 10 client.15894 fetch_fsmap learned FSMap version 32 2024-03-18T23:13:36.737 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:36.731+0000 7f4bd8231740 1 -- 172.21.15.49:0/4049452678 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55fa3d74c680 con 0x55fa3d6cd810 2024-03-18T23:13:36.737 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:36.731+0000 7f4bb67fc640 1 -- 172.21.15.49:0/4049452678 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4bc0004c80 con 0x55fa3d6cd810 2024-03-18T23:13:36.737 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:36.731+0000 7f4bb67fc640 1 -- 172.21.15.49:0/4049452678 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f4bb8003e40 con 0x55fa3d6cd810 2024-03-18T23:13:36.737 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:36.731+0000 7f4bd8231740 10 client.15894 fetch_fsmap finished waiting for FSMap version 32 2024-03-18T23:13:36.737 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:36.731+0000 7f4bd8231740 10 client.15894 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T23:13:36.737 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:36.731+0000 7f4bd8231740 20 client.15894 populate_metadata read hostname 'smithi049' 2024-03-18T23:13:36.737 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:36.731+0000 7f4bd8231740 1 --2- 172.21.15.49:0/4049452678 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x55fa3d6e6380 0x55fa3d74edd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:13:36.737 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:36.731+0000 7f4bd8231740 4 client.15894 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T23:13:36.737 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:36.731+0000 7f4bd8231740 1 -- 172.21.15.49:0/4049452678 --> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55fa3d638470 con 0x55fa3d6e6380 2024-03-18T23:13:36.737 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:36.732+0000 7f4bceffd640 1 --2- 172.21.15.49:0/4049452678 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x55fa3d6e6380 0x55fa3d74edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:13:36.737 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:36.732+0000 7f4bb67fc640 10 client.15894 ms_handle_connect on v2:172.21.15.139:6838/4280128209 2024-03-18T23:13:36.737 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:36.732+0000 7f4bceffd640 1 --2- 172.21.15.49:0/4049452678 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x55fa3d6e6380 0x55fa3d74edd0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=729ec1e75f1f413c server_cookie=a21d718eeda1722b in_seq=0 out_seq=0 2024-03-18T23:13:36.771 INFO:tasks.workunit.client.0.smithi049.stdout: 1 578367 10.30 MB/sec execute 158 sec latency 32.849 ms 2024-03-18T23:13:36.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:13:36 smithi139 ceph-mon[67282]: pgmap v165: 97 pgs: 97 active+clean; 82 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.6 MiB/s wr, 267 op/s 2024-03-18T23:13:36.873 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:36.872+0000 7f4bb67fc640 1 -- 172.21.15.49:0/4049452678 <== mds.0 v2:172.21.15.139:6838/4280128209 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55fa3d638470 con 0x55fa3d6e6380 2024-03-18T23:13:36.873 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:36.872+0000 7f4bb67fc640 10 client.15894 handle_command_reply: tid=0 2024-03-18T23:13:36.873 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:36.872+0000 7f4bd8231740 1 client.15894 shutdown 2024-03-18T23:13:36.873 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:36.872+0000 7f4bd8231740 1 -- 172.21.15.49:0/4049452678 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fa3d6cd810 msgr2=0x55fa3d6d9cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:13:36.874 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:36.872+0000 7f4bd8231740 1 --2- 172.21.15.49:0/4049452678 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fa3d6cd810 0x55fa3d6d9cb0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f4bb8009380 tx=0x7f4bb800f7f0 comp rx=0 tx=0).stop 2024-03-18T23:13:36.874 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:36.872+0000 7f4bd8231740 1 -- 172.21.15.49:0/4049452678 shutdown_connections 2024-03-18T23:13:36.874 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:36.872+0000 7f4bd8231740 1 --2- 172.21.15.49:0/4049452678 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x55fa3d6e6380 0x55fa3d74edd0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:36.874 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:36.872+0000 7f4bd8231740 1 --2- 172.21.15.49:0/4049452678 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55fa3d6d65e0 0x55fa3d6da250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:36.874 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:36.872+0000 7f4bd8231740 1 --2- 172.21.15.49:0/4049452678 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fa3d6cd810 0x55fa3d6d9cb0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:36.874 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:36.872+0000 7f4bd8231740 1 --2- 172.21.15.49:0/4049452678 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fa3d6ccef0 0x55fa3d6de940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:36.874 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:36.872+0000 7f4bd8231740 1 -- 172.21.15.49:0/4049452678 >> 172.21.15.49:0/4049452678 conn(0x55fa3d6c7810 msgr2=0x55fa3d6c8c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:13:36.874 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:36.872+0000 7f4bb67fc640 0 client.15894 ms_handle_reset on v2:172.21.15.139:6838/4280128209 2024-03-18T23:13:36.874 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:36.872+0000 7f4bd8231740 1 -- 172.21.15.49:0/4049452678 shutdown_connections 2024-03-18T23:13:36.874 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:36.872+0000 7f4bd8231740 1 -- 172.21.15.49:0/4049452678 wait complete. 2024-03-18T23:13:36.874 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:36.872+0000 7f4bd8231740 20 client.15894 trim_cache size 0 max 16384 2024-03-18T23:13:36.874 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:{ 2024-03-18T23:13:36.875 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "message": "", 2024-03-18T23:13:36.875 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "return_code": 0 2024-03-18T23:13:36.875 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:} 2024-03-18T23:13:36.875 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:36.873+0000 7f4bd564d640 1 -- 172.21.15.49:0/1513972690 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f4bbc0a2c60 msgr2=0x7f4bbc0a5120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:13:36.875 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:36.873+0000 7f4bd564d640 1 --2- 172.21.15.49:0/1513972690 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f4bbc0a2c60 0x7f4bbc0a5120 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f4bc8005fd0 tx=0x7f4bc8009e90 comp rx=0 tx=0).stop 2024-03-18T23:13:36.875 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:36.873+0000 7f4bd564d640 1 -- 172.21.15.49:0/1513972690 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4bd0069fc0 msgr2=0x7f4bd0100ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:13:36.876 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:36.873+0000 7f4bd564d640 1 --2- 172.21.15.49:0/1513972690 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4bd0069fc0 0x7f4bd0100ad0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f4bc0009230 tx=0x7f4bc002cb30 comp rx=0 tx=0).stop 2024-03-18T23:13:36.876 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:36.873+0000 7f4bd564d640 1 -- 172.21.15.49:0/1513972690 shutdown_connections 2024-03-18T23:13:36.876 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:36.873+0000 7f4bd564d640 1 --2- 172.21.15.49:0/1513972690 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f4bbc0a2c60 0x7f4bbc0a5120 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:36.876 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:36.873+0000 7f4bd564d640 1 --2- 172.21.15.49:0/1513972690 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4bd01077e0 0x7f4bd0101550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:36.876 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:36.873+0000 7f4bd564d640 1 --2- 172.21.15.49:0/1513972690 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4bd006a980 0x7f4bd0101010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:36.876 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:36.873+0000 7f4bd564d640 1 --2- 172.21.15.49:0/1513972690 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4bd0069fc0 0x7f4bd0100ad0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:36.876 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:36.873+0000 7f4bd564d640 1 -- 172.21.15.49:0/1513972690 >> 172.21.15.49:0/1513972690 conn(0x7f4bd0077700 msgr2=0x7f4bd00fe570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:13:36.876 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:36.873+0000 7f4bd564d640 1 -- 172.21.15.49:0/1513972690 shutdown_connections 2024-03-18T23:13:36.876 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:36.874+0000 7f4bd564d640 1 -- 172.21.15.49:0/1513972690 wait complete. 2024-03-18T23:13:36.879 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:13:36 smithi049 ceph-mon[62991]: pgmap v165: 97 pgs: 97 active+clean; 82 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.6 MiB/s wr, 267 op/s 2024-03-18T23:13:36.961 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:13:36 smithi175 ceph-mon[67352]: pgmap v165: 97 pgs: 97 active+clean; 82 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.6 MiB/s wr, 267 op/s 2024-03-18T23:13:37.771 INFO:tasks.workunit.client.0.smithi049.stdout: 1 579852 10.28 MB/sec execute 159 sec latency 59.202 ms 2024-03-18T23:13:38.771 INFO:tasks.workunit.client.0.smithi049.stdout: 1 581947 10.29 MB/sec execute 160 sec latency 21.807 ms 2024-03-18T23:13:38.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:13:38 smithi139 ceph-mon[67282]: pgmap v166: 97 pgs: 97 active+clean; 89 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 3.8 MiB/s wr, 356 op/s 2024-03-18T23:13:38.879 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:13:38 smithi049 ceph-mon[62991]: pgmap v166: 97 pgs: 97 active+clean; 89 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 3.8 MiB/s wr, 356 op/s 2024-03-18T23:13:38.961 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:13:38 smithi175 ceph-mon[67352]: pgmap v166: 97 pgs: 97 active+clean; 89 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 3.8 MiB/s wr, 356 op/s 2024-03-18T23:13:39.771 INFO:tasks.workunit.client.0.smithi049.stdout: 1 583677 10.28 MB/sec execute 161 sec latency 21.501 ms 2024-03-18T23:13:40.772 INFO:tasks.workunit.client.0.smithi049.stdout: 1 585751 10.28 MB/sec execute 162 sec latency 41.707 ms 2024-03-18T23:13:40.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:13:40 smithi139 ceph-mon[67282]: pgmap v167: 97 pgs: 97 active+clean; 89 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.4 MiB/s wr, 177 op/s 2024-03-18T23:13:40.879 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:13:40 smithi049 ceph-mon[62991]: pgmap v167: 97 pgs: 97 active+clean; 89 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.4 MiB/s wr, 177 op/s 2024-03-18T23:13:40.961 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:13:40 smithi175 ceph-mon[67352]: pgmap v167: 97 pgs: 97 active+clean; 89 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.4 MiB/s wr, 177 op/s 2024-03-18T23:13:41.772 INFO:tasks.workunit.client.0.smithi049.stdout: 1 587309 10.27 MB/sec execute 163 sec latency 66.778 ms 2024-03-18T23:13:42.772 INFO:tasks.workunit.client.0.smithi049.stdout: 1 589706 10.30 MB/sec execute 164 sec latency 19.164 ms 2024-03-18T23:13:42.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:13:42 smithi139 ceph-mon[67282]: pgmap v168: 97 pgs: 97 active+clean; 80 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 3.8 MiB/s wr, 354 op/s 2024-03-18T23:13:42.879 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:13:42 smithi049 ceph-mon[62991]: pgmap v168: 97 pgs: 97 active+clean; 80 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 3.8 MiB/s wr, 354 op/s 2024-03-18T23:13:42.961 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:13:42 smithi175 ceph-mon[67352]: pgmap v168: 97 pgs: 97 active+clean; 80 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 3.8 MiB/s wr, 354 op/s 2024-03-18T23:13:43.772 INFO:tasks.workunit.client.0.smithi049.stdout: 1 591269 10.28 MB/sec execute 165 sec latency 26.515 ms 2024-03-18T23:13:44.772 INFO:tasks.workunit.client.0.smithi049.stdout: 1 593480 10.30 MB/sec execute 166 sec latency 22.688 ms 2024-03-18T23:13:44.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:13:44 smithi139 ceph-mon[67282]: pgmap v169: 97 pgs: 97 active+clean; 80 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.7 MiB/s wr, 265 op/s 2024-03-18T23:13:44.879 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:13:44 smithi049 ceph-mon[62991]: pgmap v169: 97 pgs: 97 active+clean; 80 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.7 MiB/s wr, 265 op/s 2024-03-18T23:13:44.961 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:13:44 smithi175 ceph-mon[67352]: pgmap v169: 97 pgs: 97 active+clean; 80 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.7 MiB/s wr, 265 op/s 2024-03-18T23:13:45.772 INFO:tasks.workunit.client.0.smithi049.stdout: 1 595079 10.27 MB/sec execute 167 sec latency 25.278 ms 2024-03-18T23:13:46.772 INFO:tasks.workunit.client.0.smithi049.stdout: 1 597100 10.29 MB/sec execute 168 sec latency 110.421 ms 2024-03-18T23:13:46.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:13:46 smithi139 ceph-mon[67282]: pgmap v170: 97 pgs: 97 active+clean; 80 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.7 MiB/s wr, 265 op/s 2024-03-18T23:13:46.879 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:13:46 smithi049 ceph-mon[62991]: pgmap v170: 97 pgs: 97 active+clean; 80 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.7 MiB/s wr, 265 op/s 2024-03-18T23:13:46.961 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:13:46 smithi175 ceph-mon[67352]: pgmap v170: 97 pgs: 97 active+clean; 80 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.7 MiB/s wr, 265 op/s 2024-03-18T23:13:47.772 INFO:tasks.workunit.client.0.smithi049.stdout: 1 598690 10.26 MB/sec execute 169 sec latency 21.859 ms 2024-03-18T23:13:48.772 INFO:tasks.workunit.client.0.smithi049.stdout: 1 600825 10.28 MB/sec execute 170 sec latency 19.906 ms 2024-03-18T23:13:48.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:13:48 smithi139 ceph-mon[67282]: pgmap v171: 97 pgs: 97 active+clean; 87 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 4.0 MiB/s wr, 357 op/s 2024-03-18T23:13:48.879 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:13:48 smithi049 ceph-mon[62991]: pgmap v171: 97 pgs: 97 active+clean; 87 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 4.0 MiB/s wr, 357 op/s 2024-03-18T23:13:48.961 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:13:48 smithi175 ceph-mon[67352]: pgmap v171: 97 pgs: 97 active+clean; 87 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 4.0 MiB/s wr, 357 op/s 2024-03-18T23:13:49.772 INFO:tasks.workunit.client.0.smithi049.stdout: 1 602695 10.27 MB/sec execute 171 sec latency 23.586 ms 2024-03-18T23:13:49.968 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:49.967+0000 7f0b28a4b640 1 -- 172.21.15.49:0/1223171083 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0b24100970 msgr2=0x7f0b24100dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:13:49.968 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:49.967+0000 7f0b28a4b640 1 --2- 172.21.15.49:0/1223171083 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0b24100970 0x7f0b24100dd0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f0b0c009900 tx=0x7f0b0c02bc30 comp rx=0 tx=0).stop 2024-03-18T23:13:49.968 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:49.967+0000 7f0b28a4b640 1 -- 172.21.15.49:0/1223171083 shutdown_connections 2024-03-18T23:13:49.969 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:49.967+0000 7f0b28a4b640 1 --2- 172.21.15.49:0/1223171083 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0b24101310 0x7f0b2410d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:49.969 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:49.967+0000 7f0b28a4b640 1 --2- 172.21.15.49:0/1223171083 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0b24100970 0x7f0b24100dd0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:49.969 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:49.967+0000 7f0b28a4b640 1 --2- 172.21.15.49:0/1223171083 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0b241056a0 0x7f0b24105a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:49.969 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:49.967+0000 7f0b28a4b640 1 -- 172.21.15.49:0/1223171083 >> 172.21.15.49:0/1223171083 conn(0x7f0b240774f0 msgr2=0x7f0b24077900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:13:49.969 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:49.967+0000 7f0b28a4b640 1 -- 172.21.15.49:0/1223171083 shutdown_connections 2024-03-18T23:13:49.969 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:49.967+0000 7f0b28a4b640 1 -- 172.21.15.49:0/1223171083 wait complete. 2024-03-18T23:13:49.971 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:49.968+0000 7f0b28a4b640 1 Processor -- start 2024-03-18T23:13:49.971 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:49.968+0000 7f0b28a4b640 1 -- start start 2024-03-18T23:13:49.971 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:49.968+0000 7f0b28a4b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0b24100970 0x7f0b240fe330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:13:49.971 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:49.968+0000 7f0b28a4b640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0b24101310 0x7f0b240fe870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:13:49.971 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:49.968+0000 7f0b28a4b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0b241056a0 0x7f0b240fedb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:13:49.971 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:49.968+0000 7f0b28a4b640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f0b24110bb0 con 0x7f0b241056a0 2024-03-18T23:13:49.971 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:49.968+0000 7f0b28a4b640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f0b24110a30 con 0x7f0b24100970 2024-03-18T23:13:49.971 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:49.968+0000 7f0b28a4b640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f0b24110d30 con 0x7f0b24101310 2024-03-18T23:13:49.971 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:49.968+0000 7f0b22d9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0b241056a0 0x7f0b240fedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:13:49.971 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:49.968+0000 7f0b22d9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0b241056a0 0x7f0b240fedb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:56588/0 (socket says 172.21.15.49:56588) 2024-03-18T23:13:49.971 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:49.968+0000 7f0b22d9d640 1 -- 172.21.15.49:0/1581043607 learned_addr learned my addr 172.21.15.49:0/1581043607 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:13:49.972 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:49.968+0000 7f0b2259c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0b24100970 0x7f0b240fe330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:13:49.972 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:49.969+0000 7f0b1bfff640 1 --2- 172.21.15.49:0/1581043607 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0b24101310 0x7f0b240fe870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:13:49.972 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:49.969+0000 7f0b22d9d640 1 -- 172.21.15.49:0/1581043607 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0b24101310 msgr2=0x7f0b240fe870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:13:49.972 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:49.969+0000 7f0b22d9d640 1 --2- 172.21.15.49:0/1581043607 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0b24101310 0x7f0b240fe870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:49.972 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:49.969+0000 7f0b22d9d640 1 -- 172.21.15.49:0/1581043607 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0b24100970 msgr2=0x7f0b240fe330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:13:49.972 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:49.969+0000 7f0b22d9d640 1 --2- 172.21.15.49:0/1581043607 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0b24100970 0x7f0b240fe330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:49.972 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:49.969+0000 7f0b22d9d640 1 -- 172.21.15.49:0/1581043607 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b241a6560 con 0x7f0b241056a0 2024-03-18T23:13:49.972 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:49.969+0000 7f0b22d9d640 1 --2- 172.21.15.49:0/1581043607 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0b241056a0 0x7f0b240fedb0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f0b1c00eb10 tx=0x7f0b1c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:13:49.972 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:49.969+0000 7f0b1b7fe640 1 -- 172.21.15.49:0/1581043607 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b1c004b20 con 0x7f0b241056a0 2024-03-18T23:13:49.972 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:49.969+0000 7f0b1b7fe640 1 -- 172.21.15.49:0/1581043607 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0b1c004cc0 con 0x7f0b241056a0 2024-03-18T23:13:49.972 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:49.969+0000 7f0b28a4b640 1 -- 172.21.15.49:0/1581043607 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b241a6850 con 0x7f0b241056a0 2024-03-18T23:13:49.972 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:49.969+0000 7f0b1b7fe640 1 -- 172.21.15.49:0/1581043607 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b1c010760 con 0x7f0b241056a0 2024-03-18T23:13:49.972 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:49.969+0000 7f0b28a4b640 1 -- 172.21.15.49:0/1581043607 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b241a6dc0 con 0x7f0b241056a0 2024-03-18T23:13:49.979 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:49.972+0000 7f0b1b7fe640 1 -- 172.21.15.49:0/1581043607 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 25) ==== 137421+0+0 (secure 0 0 0) 0x7f0b1c005020 con 0x7f0b241056a0 2024-03-18T23:13:49.979 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:49.972+0000 7f0b2b63a740 1 Processor -- start 2024-03-18T23:13:49.979 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:49.972+0000 7f0b2b63a740 1 -- start start 2024-03-18T23:13:49.979 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:49.972+0000 7f0b2b63a740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5611301abef0 0x5611301ac2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:13:49.979 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:49.972+0000 7f0b2b63a740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5611301ac810 0x5611301b50a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:13:49.979 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:49.972+0000 7f0b2b63a740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5611301b55e0 0x5611301b79d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:13:49.979 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:49.972+0000 7f0b2b63a740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x56113010fbe0 con 0x5611301b55e0 2024-03-18T23:13:49.979 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:49.972+0000 7f0b2b63a740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x5611300b8960 con 0x5611301ac810 2024-03-18T23:13:49.980 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:49.972+0000 7f0b2b63a740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x56113010bc80 con 0x5611301abef0 2024-03-18T23:13:49.980 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:49.972+0000 7f0b2259c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5611301ac810 0x5611301b50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:13:49.980 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:49.972+0000 7f0b1bfff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5611301b55e0 0x5611301b79d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:13:49.980 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:49.972+0000 7f0b2259c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5611301ac810 0x5611301b50a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.49:38328/0 (socket says 172.21.15.49:38328) 2024-03-18T23:13:49.980 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:49.972+0000 7f0b2259c640 1 -- 172.21.15.49:0/2317083732 learned_addr learned my addr 172.21.15.49:0/2317083732 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:13:49.980 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:49.972+0000 7f0b1bfff640 1 --2- 172.21.15.49:0/2317083732 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5611301abef0 0x5611301ac2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:13:49.980 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:49.972+0000 7f0b1b7fe640 1 --2- 172.21.15.49:0/1581043607 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f0b100a2a90 0x7f0b100a4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:13:49.980 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:49.973+0000 7f0b22d9d640 1 --2- 172.21.15.49:0/1581043607 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f0b100a2a90 0x7f0b100a4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:13:49.980 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:49.973+0000 7f0b2259c640 1 -- 172.21.15.49:0/2317083732 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5611301abef0 msgr2=0x5611301ac2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:13:49.980 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:49.973+0000 7f0b2259c640 1 --2- 172.21.15.49:0/2317083732 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5611301abef0 0x5611301ac2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:49.980 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:49.973+0000 7f0b2259c640 1 -- 172.21.15.49:0/2317083732 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5611301b55e0 msgr2=0x5611301b79d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:13:49.980 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:49.973+0000 7f0b2259c640 1 --2- 172.21.15.49:0/2317083732 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5611301b55e0 0x5611301b79d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:49.980 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:49.973+0000 7f0b2259c640 1 -- 172.21.15.49:0/2317083732 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5611300afd20 con 0x5611301ac810 2024-03-18T23:13:49.980 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:49.973+0000 7f0b1b7fe640 1 -- 172.21.15.49:0/1581043607 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8512+0+0 (secure 0 0 0) 0x7f0b1c0cf4b0 con 0x7f0b241056a0 2024-03-18T23:13:49.980 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:49.973+0000 7f0b2259c640 1 --2- 172.21.15.49:0/2317083732 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5611301ac810 0x5611301b50a0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f0b14009c70 tx=0x7f0b14011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2b86593a5481b94 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:13:49.980 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:49.973+0000 7f0b22d9d640 1 --2- 172.21.15.49:0/1581043607 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f0b100a2a90 0x7f0b100a4f50 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f0b1c0a3d90 tx=0x7f0b1c00efd0 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:13:49.981 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:49.973+0000 7f0b18ff9640 1 -- 172.21.15.49:0/2317083732 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b1401e070 con 0x5611301ac810 2024-03-18T23:13:49.981 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:49.973+0000 7f0b18ff9640 1 -- 172.21.15.49:0/2317083732 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0b14009e20 con 0x5611301ac810 2024-03-18T23:13:49.981 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:49.973+0000 7f0b18ff9640 1 -- 172.21.15.49:0/2317083732 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b14019630 con 0x5611301ac810 2024-03-18T23:13:49.981 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:49.974+0000 7f0b2b63a740 1 -- 172.21.15.49:0/2317083732 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5611301ac810 msgr2=0x5611301b50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:13:49.981 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:49.974+0000 7f0b2b63a740 1 --2- 172.21.15.49:0/2317083732 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5611301ac810 0x5611301b50a0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f0b14009c70 tx=0x7f0b14011810 comp rx=0 tx=0).stop 2024-03-18T23:13:49.981 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:49.974+0000 7f0b2b63a740 1 -- 172.21.15.49:0/2317083732 shutdown_connections 2024-03-18T23:13:49.981 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:49.974+0000 7f0b2b63a740 1 --2- 172.21.15.49:0/2317083732 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5611301b55e0 0x5611301b79d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:49.981 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:49.974+0000 7f0b2b63a740 1 --2- 172.21.15.49:0/2317083732 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5611301ac810 0x5611301b50a0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:49.981 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:49.974+0000 7f0b2b63a740 1 --2- 172.21.15.49:0/2317083732 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5611301abef0 0x5611301ac2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:49.981 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:49.974+0000 7f0b2b63a740 1 -- 172.21.15.49:0/2317083732 >> 172.21.15.49:0/2317083732 conn(0x5611301a6810 msgr2=0x5611301a6bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:13:49.981 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:49.974+0000 7f0b2b63a740 1 -- 172.21.15.49:0/2317083732 shutdown_connections 2024-03-18T23:13:49.981 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:49.974+0000 7f0b2b63a740 1 -- 172.21.15.49:0/2317083732 wait complete. 2024-03-18T23:13:49.981 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:49.974+0000 7f0b2b63a740 1 -- start start 2024-03-18T23:13:49.981 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:49.974+0000 7f0b2b63a740 1 Processor -- start 2024-03-18T23:13:49.981 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:49.975+0000 7f0b2b63a740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5611301abef0 0x5611301b4f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:13:49.981 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:49.975+0000 7f0b2b63a740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5611301ac810 0x5611301b2320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:13:49.982 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:49.975+0000 7f0b2b63a740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5611301b55e0 0x5611301b28c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:13:49.982 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:49.975+0000 7f0b2b63a740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5611301b8210 con 0x5611301ac810 2024-03-18T23:13:49.982 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:49.975+0000 7f0b2b63a740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x5611300cbdb0 con 0x5611301b55e0 2024-03-18T23:13:49.982 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:49.975+0000 7f0b2b63a740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5611301b8390 con 0x5611301abef0 2024-03-18T23:13:49.982 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:49.975+0000 7f0b2b63a740 10 client.0 _ll_register_callbacks cb 0x5611300d0e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T23:13:49.982 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:49.975+0000 7f0b2259c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5611301ac810 0x5611301b2320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:13:49.982 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:49.975+0000 7f0b2259c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5611301ac810 0x5611301b2320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:56606/0 (socket says 172.21.15.49:56606) 2024-03-18T23:13:49.982 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:49.975+0000 7f0b2259c640 1 -- 172.21.15.49:0/3348080510 learned_addr learned my addr 172.21.15.49:0/3348080510 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:13:49.982 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:49.975+0000 7f0b1bfff640 1 --2- 172.21.15.49:0/3348080510 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5611301abef0 0x5611301b4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:13:49.982 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:49.975+0000 7f0b1bfff640 1 --2- 172.21.15.49:0/3348080510 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5611301b55e0 0x5611301b28c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:13:49.982 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:49.975+0000 7f0b2259c640 1 -- 172.21.15.49:0/3348080510 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5611301abef0 msgr2=0x5611301b4f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:13:49.982 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:49.975+0000 7f0b2259c640 1 --2- 172.21.15.49:0/3348080510 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5611301abef0 0x5611301b4f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:49.982 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:49.975+0000 7f0b2259c640 1 -- 172.21.15.49:0/3348080510 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5611301b55e0 msgr2=0x5611301b28c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:13:49.982 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:49.975+0000 7f0b2259c640 1 --2- 172.21.15.49:0/3348080510 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5611301b55e0 0x5611301b28c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:49.983 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:49.975+0000 7f0b2259c640 1 -- 172.21.15.49:0/3348080510 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5611300cac60 con 0x5611301ac810 2024-03-18T23:13:49.983 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:49.975+0000 7f0b2259c640 1 --2- 172.21.15.49:0/3348080510 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5611301ac810 0x5611301b2320 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f0b14005bc0 tx=0x7f0b140098d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c3bf33556ca54e server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:13:49.983 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:49.975+0000 7f0b09ffb640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-18T23:13:49.983 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:49.975+0000 7f0b09ffb640 1 -- 172.21.15.49:0/3348080510 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b1401e040 con 0x5611301ac810 2024-03-18T23:13:49.983 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:49.976+0000 7f0b09ffb640 1 -- 172.21.15.49:0/3348080510 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8512+0+0 (secure 0 0 0) 0x7f0b140035f0 con 0x5611301ac810 2024-03-18T23:13:49.983 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:49.976+0000 7f0b2b63a740 1 -- 172.21.15.49:0/3348080510 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0b14009e20 con 0x5611301ac810 2024-03-18T23:13:49.983 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:49.976+0000 7f0b09ffb640 1 -- 172.21.15.49:0/3348080510 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f0b14031050 con 0x5611301ac810 2024-03-18T23:13:49.983 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:49.976+0000 7f0b2b63a740 10 client.15906 fetch_fsmap learned FSMap version 32 2024-03-18T23:13:49.983 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:49.976+0000 7f0b2b63a740 1 -- 172.21.15.49:0/3348080510 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x56112fe24210 con 0x5611301ac810 2024-03-18T23:13:49.983 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:49.976+0000 7f0b09ffb640 1 -- 172.21.15.49:0/3348080510 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0b14009e20 con 0x5611301ac810 2024-03-18T23:13:49.983 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:49.976+0000 7f0b09ffb640 1 -- 172.21.15.49:0/3348080510 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f0b14021030 con 0x5611301ac810 2024-03-18T23:13:49.983 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:49.976+0000 7f0b2b63a740 10 client.15906 fetch_fsmap finished waiting for FSMap version 32 2024-03-18T23:13:49.983 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:49.976+0000 7f0b2b63a740 10 client.15906 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T23:13:49.983 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:49.976+0000 7f0b2b63a740 20 client.15906 populate_metadata read hostname 'smithi049' 2024-03-18T23:13:49.984 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:49.976+0000 7f0b2b63a740 1 --2- 172.21.15.49:0/3348080510 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x5611301c53b0 0x5611301c77a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:13:49.984 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:49.976+0000 7f0b2b63a740 4 client.15906 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T23:13:49.984 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:49.976+0000 7f0b2b63a740 1 -- 172.21.15.49:0/3348080510 --> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56113011d120 con 0x5611301c53b0 2024-03-18T23:13:49.984 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:49.976+0000 7f0b1bfff640 1 --2- 172.21.15.49:0/3348080510 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x5611301c53b0 0x5611301c77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:13:49.984 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:49.977+0000 7f0b09ffb640 10 client.15906 ms_handle_connect on v2:172.21.15.139:6838/4280128209 2024-03-18T23:13:49.984 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:49.977+0000 7f0b1bfff640 1 --2- 172.21.15.49:0/3348080510 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x5611301c53b0 0x5611301c77a0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=99652e331de61e47 server_cookie=1c8c81eaa7192dde in_seq=0 out_seq=0 2024-03-18T23:13:49.984 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:49.978+0000 7f0b09ffb640 1 -- 172.21.15.49:0/3348080510 <== mds.0 v2:172.21.15.139:6838/4280128209 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x56113011d120 con 0x5611301c53b0 2024-03-18T23:13:49.984 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:49.978+0000 7f0b09ffb640 10 client.15906 handle_command_reply: tid=0 2024-03-18T23:13:49.984 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:49.978+0000 7f0b2b63a740 1 client.15906 shutdown 2024-03-18T23:13:49.984 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:49.978+0000 7f0b2b63a740 1 -- 172.21.15.49:0/3348080510 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5611301ac810 msgr2=0x5611301b2320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:13:49.984 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:49.978+0000 7f0b2b63a740 1 --2- 172.21.15.49:0/3348080510 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5611301ac810 0x5611301b2320 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f0b14005bc0 tx=0x7f0b140098d0 comp rx=0 tx=0).stop 2024-03-18T23:13:49.984 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:49.978+0000 7f0b2b63a740 1 -- 172.21.15.49:0/3348080510 shutdown_connections 2024-03-18T23:13:49.984 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:49.978+0000 7f0b2b63a740 1 --2- 172.21.15.49:0/3348080510 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x5611301c53b0 0x5611301c77a0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:49.985 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:49.978+0000 7f0b2b63a740 1 --2- 172.21.15.49:0/3348080510 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5611301b55e0 0x5611301b28c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:49.985 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:49.978+0000 7f0b09ffb640 0 client.15906 ms_handle_reset on v2:172.21.15.139:6838/4280128209 2024-03-18T23:13:49.985 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:49.978+0000 7f0b2b63a740 1 --2- 172.21.15.49:0/3348080510 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5611301ac810 0x5611301b2320 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:49.985 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:49.978+0000 7f0b2b63a740 1 --2- 172.21.15.49:0/3348080510 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5611301abef0 0x5611301b4f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:49.985 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:49.978+0000 7f0b2b63a740 1 -- 172.21.15.49:0/3348080510 >> 172.21.15.49:0/3348080510 conn(0x5611301a6810 msgr2=0x5611301a5ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:13:49.985 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:49.978+0000 7f0b2b63a740 1 -- 172.21.15.49:0/3348080510 shutdown_connections 2024-03-18T23:13:49.985 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:49.978+0000 7f0b2b63a740 1 -- 172.21.15.49:0/3348080510 wait complete. 2024-03-18T23:13:49.985 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:49.978+0000 7f0b2b63a740 20 client.15906 trim_cache size 0 max 16384 2024-03-18T23:13:49.998 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:49.993+0000 7f0b2b63a740 1 Processor -- start 2024-03-18T23:13:49.998 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:49.993+0000 7f0b2b63a740 1 -- start start 2024-03-18T23:13:49.998 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:49.993+0000 7f0b2b63a740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5611301abef0 0x5611301c0fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:13:49.999 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:49.993+0000 7f0b2b63a740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5611301ac810 0x5611301a1940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:13:49.999 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:49.993+0000 7f0b2b63a740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5611301b55e0 0x5611301ccec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:13:49.999 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:49.993+0000 7f0b2b63a740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f0b241af700 con 0x5611301ac810 2024-03-18T23:13:49.999 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:49.993+0000 7f0b2b63a740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f0b241b1240 con 0x5611301b55e0 2024-03-18T23:13:49.999 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:49.993+0000 7f0b2b63a740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f0b241afdd0 con 0x5611301abef0 2024-03-18T23:13:49.999 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:49.993+0000 7f0b2259c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5611301ac810 0x5611301a1940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:13:49.999 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:49.993+0000 7f0b2259c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5611301ac810 0x5611301a1940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:56616/0 (socket says 172.21.15.49:56616) 2024-03-18T23:13:49.999 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:49.993+0000 7f0b1bfff640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5611301abef0 0x5611301c0fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:13:49.999 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:49.993+0000 7f0b2259c640 1 -- 172.21.15.49:0/2586051171 learned_addr learned my addr 172.21.15.49:0/2586051171 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:13:49.999 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:49.993+0000 7f0b1bfff640 1 --2- 172.21.15.49:0/2586051171 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5611301b55e0 0x5611301ccec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:13:49.999 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:49.993+0000 7f0b2259c640 1 -- 172.21.15.49:0/2586051171 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5611301abef0 msgr2=0x5611301c0fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:13:49.999 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:49.993+0000 7f0b2259c640 1 --2- 172.21.15.49:0/2586051171 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5611301abef0 0x5611301c0fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:49.999 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:49.993+0000 7f0b2259c640 1 -- 172.21.15.49:0/2586051171 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5611301b55e0 msgr2=0x5611301ccec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:13:49.999 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:49.993+0000 7f0b2259c640 1 --2- 172.21.15.49:0/2586051171 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5611301b55e0 0x5611301ccec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:50.000 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:49.994+0000 7f0b2259c640 1 -- 172.21.15.49:0/2586051171 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56112fef0620 con 0x5611301ac810 2024-03-18T23:13:50.000 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:49.994+0000 7f0b1bfff640 1 --2- 172.21.15.49:0/2586051171 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5611301abef0 0x5611301c0fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:13:50.000 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:49.994+0000 7f0b2259c640 1 --2- 172.21.15.49:0/2586051171 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5611301ac810 0x5611301a1940 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f0b14009380 tx=0x7f0b14004a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90375c12042032ad server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:13:50.000 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:49.994+0000 7f0b0bfff640 1 -- 172.21.15.49:0/2586051171 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b14015070 con 0x5611301ac810 2024-03-18T23:13:50.000 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:49.994+0000 7f0b0bfff640 1 -- 172.21.15.49:0/2586051171 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0b14004e50 con 0x5611301ac810 2024-03-18T23:13:50.000 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:49.994+0000 7f0b0bfff640 1 -- 172.21.15.49:0/2586051171 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b1401e070 con 0x5611301ac810 2024-03-18T23:13:50.000 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:49.995+0000 7f0b2b63a740 1 -- 172.21.15.49:0/2586051171 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5611301ac810 msgr2=0x5611301a1940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:13:50.000 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:49.995+0000 7f0b2b63a740 1 --2- 172.21.15.49:0/2586051171 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5611301ac810 0x5611301a1940 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f0b14009380 tx=0x7f0b14004a10 comp rx=0 tx=0).stop 2024-03-18T23:13:50.000 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:49.995+0000 7f0b2b63a740 1 -- 172.21.15.49:0/2586051171 shutdown_connections 2024-03-18T23:13:50.000 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:49.995+0000 7f0b2b63a740 1 --2- 172.21.15.49:0/2586051171 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5611301b55e0 0x5611301ccec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:50.000 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:49.995+0000 7f0b2b63a740 1 --2- 172.21.15.49:0/2586051171 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5611301ac810 0x5611301a1940 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:50.000 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:49.995+0000 7f0b2b63a740 1 --2- 172.21.15.49:0/2586051171 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5611301abef0 0x5611301c0fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:50.000 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:49.995+0000 7f0b2b63a740 1 -- 172.21.15.49:0/2586051171 >> 172.21.15.49:0/2586051171 conn(0x5611301a6810 msgr2=0x5611301a5ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:13:50.000 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:49.995+0000 7f0b2b63a740 1 -- 172.21.15.49:0/2586051171 shutdown_connections 2024-03-18T23:13:50.001 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:49.995+0000 7f0b2b63a740 1 -- 172.21.15.49:0/2586051171 wait complete. 2024-03-18T23:13:50.001 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:49.995+0000 7f0b2b63a740 1 -- start start 2024-03-18T23:13:50.001 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:49.995+0000 7f0b2b63a740 1 Processor -- start 2024-03-18T23:13:50.001 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:49.995+0000 7f0b2b63a740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5611301abef0 0x5611301af140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:13:50.001 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:49.995+0000 7f0b2b63a740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5611301ac810 0x5611301af680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:13:50.001 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:49.995+0000 7f0b2b63a740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5611301b55e0 0x5611301afbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:13:50.001 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:49.995+0000 7f0b2b63a740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f0b241b0b70 con 0x5611301ac810 2024-03-18T23:13:50.001 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:49.995+0000 7f0b2b63a740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f0b241b04a0 con 0x5611301b55e0 2024-03-18T23:13:50.001 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:49.995+0000 7f0b2b63a740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f0b241ad6c0 con 0x5611301abef0 2024-03-18T23:13:50.001 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:49.995+0000 7f0b2b63a740 10 client.0 _ll_register_callbacks cb 0x7f0b1401c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T23:13:50.001 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:49.996+0000 7f0b2259c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5611301ac810 0x5611301af680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:13:50.001 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:49.996+0000 7f0b1bfff640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5611301abef0 0x5611301af140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:13:50.001 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:49.996+0000 7f0b1bfff640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5611301abef0 0x5611301af140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.49:50092/0 (socket says 172.21.15.49:50092) 2024-03-18T23:13:50.001 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:49.996+0000 7f0b1bfff640 1 -- 172.21.15.49:0/1498564471 learned_addr learned my addr 172.21.15.49:0/1498564471 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:13:50.001 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:49.996+0000 7f0b1bfff640 1 --2- 172.21.15.49:0/1498564471 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5611301b55e0 0x5611301afbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:13:50.002 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:49.996+0000 7f0b2259c640 1 -- 172.21.15.49:0/1498564471 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5611301abef0 msgr2=0x5611301af140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:13:50.002 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:49.996+0000 7f0b2259c640 1 --2- 172.21.15.49:0/1498564471 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5611301abef0 0x5611301af140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:50.002 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:49.996+0000 7f0b2259c640 1 -- 172.21.15.49:0/1498564471 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5611301b55e0 msgr2=0x5611301afbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:13:50.002 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:49.996+0000 7f0b2259c640 1 --2- 172.21.15.49:0/1498564471 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5611301b55e0 0x5611301afbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:50.002 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:49.996+0000 7f0b2259c640 1 -- 172.21.15.49:0/1498564471 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56113022a790 con 0x5611301ac810 2024-03-18T23:13:50.002 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:49.996+0000 7f0b2259c640 1 --2- 172.21.15.49:0/1498564471 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5611301ac810 0x5611301af680 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x56113019b290 tx=0x7f0b14009d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1294bd4f7c0ff8c server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:13:50.002 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:49.996+0000 7f0b09ffb640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-18T23:13:50.002 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:49.996+0000 7f0b09ffb640 1 -- 172.21.15.49:0/1498564471 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b14015070 con 0x5611301ac810 2024-03-18T23:13:50.002 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:49.996+0000 7f0b09ffb640 1 -- 172.21.15.49:0/1498564471 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8512+0+0 (secure 0 0 0) 0x7f0b14005250 con 0x5611301ac810 2024-03-18T23:13:50.002 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:49.996+0000 7f0b2b63a740 1 -- 172.21.15.49:0/1498564471 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0b14004e50 con 0x5611301ac810 2024-03-18T23:13:50.002 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:49.996+0000 7f0b09ffb640 1 -- 172.21.15.49:0/1498564471 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f0b1401e040 con 0x5611301ac810 2024-03-18T23:13:50.002 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:49.996+0000 7f0b2b63a740 10 client.15912 fetch_fsmap learned FSMap version 32 2024-03-18T23:13:50.002 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:49.996+0000 7f0b2b63a740 1 -- 172.21.15.49:0/1498564471 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x56113022ba80 con 0x5611301ac810 2024-03-18T23:13:50.002 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:49.997+0000 7f0b09ffb640 1 -- 172.21.15.49:0/1498564471 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0b14004e50 con 0x5611301ac810 2024-03-18T23:13:50.003 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:49.997+0000 7f0b09ffb640 1 -- 172.21.15.49:0/1498564471 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f0b14021070 con 0x5611301ac810 2024-03-18T23:13:50.003 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:49.997+0000 7f0b2b63a740 10 client.15912 fetch_fsmap finished waiting for FSMap version 32 2024-03-18T23:13:50.003 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:49.997+0000 7f0b2b63a740 10 client.15912 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T23:13:50.003 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:49.997+0000 7f0b2b63a740 20 client.15912 populate_metadata read hostname 'smithi049' 2024-03-18T23:13:50.003 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:49.997+0000 7f0b2b63a740 1 --2- 172.21.15.49:0/1498564471 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x56113022c190 0x56113022e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:13:50.003 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:49.997+0000 7f0b2b63a740 4 client.15912 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T23:13:50.003 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:49.997+0000 7f0b2b63a740 1 -- 172.21.15.49:0/1498564471 --> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561130117470 con 0x56113022c190 2024-03-18T23:13:50.003 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:49.997+0000 7f0b1bfff640 1 --2- 172.21.15.49:0/1498564471 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x56113022c190 0x56113022e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:13:50.003 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:49.997+0000 7f0b09ffb640 10 client.15912 ms_handle_connect on v2:172.21.15.139:6838/4280128209 2024-03-18T23:13:50.003 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:49.997+0000 7f0b1bfff640 1 --2- 172.21.15.49:0/1498564471 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x56113022c190 0x56113022e580 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=962cc0bfcf945d16 server_cookie=1f4f337e143effd6 in_seq=0 out_seq=0 2024-03-18T23:13:50.149 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:50.148+0000 7f0b09ffb640 1 -- 172.21.15.49:0/1498564471 <== mds.0 v2:172.21.15.139:6838/4280128209 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561130117470 con 0x56113022c190 2024-03-18T23:13:50.149 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:50.148+0000 7f0b09ffb640 10 client.15912 handle_command_reply: tid=0 2024-03-18T23:13:50.149 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:50.148+0000 7f0b2b63a740 1 client.15912 shutdown 2024-03-18T23:13:50.149 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:50.148+0000 7f0b2b63a740 1 -- 172.21.15.49:0/1498564471 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5611301ac810 msgr2=0x5611301af680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:13:50.149 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:50.148+0000 7f0b2b63a740 1 --2- 172.21.15.49:0/1498564471 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5611301ac810 0x5611301af680 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x56113019b290 tx=0x7f0b14009d60 comp rx=0 tx=0).stop 2024-03-18T23:13:50.149 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:50.148+0000 7f0b2b63a740 1 -- 172.21.15.49:0/1498564471 shutdown_connections 2024-03-18T23:13:50.149 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:50.148+0000 7f0b2b63a740 1 --2- 172.21.15.49:0/1498564471 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x56113022c190 0x56113022e580 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:50.149 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:50.148+0000 7f0b2b63a740 1 --2- 172.21.15.49:0/1498564471 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5611301b55e0 0x5611301afbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:50.150 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:50.148+0000 7f0b2b63a740 1 --2- 172.21.15.49:0/1498564471 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5611301ac810 0x5611301af680 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:50.150 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:50.148+0000 7f0b2b63a740 1 --2- 172.21.15.49:0/1498564471 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5611301abef0 0x5611301af140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:50.150 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:50.148+0000 7f0b2b63a740 1 -- 172.21.15.49:0/1498564471 >> 172.21.15.49:0/1498564471 conn(0x5611301a6810 msgr2=0x5611301a5ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:13:50.150 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:50.148+0000 7f0b09ffb640 0 client.15912 ms_handle_reset on v2:172.21.15.139:6838/4280128209 2024-03-18T23:13:50.150 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:50.148+0000 7f0b2b63a740 1 -- 172.21.15.49:0/1498564471 shutdown_connections 2024-03-18T23:13:50.150 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:50.148+0000 7f0b2b63a740 1 -- 172.21.15.49:0/1498564471 wait complete. 2024-03-18T23:13:50.150 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:50.148+0000 7f0b2b63a740 20 client.15912 trim_cache size 0 max 16384 2024-03-18T23:13:50.150 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:{ 2024-03-18T23:13:50.150 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "message": "", 2024-03-18T23:13:50.150 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "return_code": 0 2024-03-18T23:13:50.150 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:} 2024-03-18T23:13:50.151 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:50.149+0000 7f0b28a4b640 1 -- 172.21.15.49:0/1581043607 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f0b100a2a90 msgr2=0x7f0b100a4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:13:50.151 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:50.149+0000 7f0b28a4b640 1 --2- 172.21.15.49:0/1581043607 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f0b100a2a90 0x7f0b100a4f50 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f0b1c0a3d90 tx=0x7f0b1c00efd0 comp rx=0 tx=0).stop 2024-03-18T23:13:50.151 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:50.149+0000 7f0b28a4b640 1 -- 172.21.15.49:0/1581043607 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0b241056a0 msgr2=0x7f0b240fedb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:13:50.151 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:50.149+0000 7f0b28a4b640 1 --2- 172.21.15.49:0/1581043607 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0b241056a0 0x7f0b240fedb0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f0b1c00eb10 tx=0x7f0b1c00c2d0 comp rx=0 tx=0).stop 2024-03-18T23:13:50.151 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:50.149+0000 7f0b28a4b640 1 -- 172.21.15.49:0/1581043607 shutdown_connections 2024-03-18T23:13:50.151 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:50.149+0000 7f0b28a4b640 1 --2- 172.21.15.49:0/1581043607 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f0b100a2a90 0x7f0b100a4f50 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:50.151 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:50.149+0000 7f0b28a4b640 1 --2- 172.21.15.49:0/1581043607 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0b241056a0 0x7f0b240fedb0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:50.152 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:50.149+0000 7f0b28a4b640 1 --2- 172.21.15.49:0/1581043607 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0b24101310 0x7f0b240fe870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:50.152 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:50.149+0000 7f0b28a4b640 1 --2- 172.21.15.49:0/1581043607 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0b24100970 0x7f0b240fe330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:50.152 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:50.149+0000 7f0b28a4b640 1 -- 172.21.15.49:0/1581043607 >> 172.21.15.49:0/1581043607 conn(0x7f0b240774f0 msgr2=0x7f0b2406a200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:13:50.152 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:50.149+0000 7f0b28a4b640 1 -- 172.21.15.49:0/1581043607 shutdown_connections 2024-03-18T23:13:50.152 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:13:50.149+0000 7f0b28a4b640 1 -- 172.21.15.49:0/1581043607 wait complete. 2024-03-18T23:13:50.772 INFO:tasks.workunit.client.0.smithi049.stdout: 1 604374 10.28 MB/sec execute 172 sec latency 73.823 ms 2024-03-18T23:13:50.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:13:50 smithi139 ceph-mon[67282]: pgmap v172: 97 pgs: 97 active+clean; 87 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.8 MiB/s wr, 268 op/s 2024-03-18T23:13:50.879 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:13:50 smithi049 ceph-mon[62991]: pgmap v172: 97 pgs: 97 active+clean; 87 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.8 MiB/s wr, 268 op/s 2024-03-18T23:13:50.961 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:13:50 smithi175 ceph-mon[67352]: pgmap v172: 97 pgs: 97 active+clean; 87 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.8 MiB/s wr, 268 op/s 2024-03-18T23:13:51.772 INFO:tasks.workunit.client.0.smithi049.stdout: 1 605896 10.25 MB/sec execute 173 sec latency 84.597 ms 2024-03-18T23:13:52.772 INFO:tasks.workunit.client.0.smithi049.stdout: 1 608011 10.27 MB/sec execute 174 sec latency 17.617 ms 2024-03-18T23:13:52.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:13:52 smithi139 ceph-mon[67282]: pgmap v173: 97 pgs: 97 active+clean; 74 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.0 MiB/s wr, 446 op/s 2024-03-18T23:13:52.879 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:13:52 smithi049 ceph-mon[62991]: pgmap v173: 97 pgs: 97 active+clean; 74 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.0 MiB/s wr, 446 op/s 2024-03-18T23:13:52.961 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:13:52 smithi175 ceph-mon[67352]: pgmap v173: 97 pgs: 97 active+clean; 74 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.0 MiB/s wr, 446 op/s 2024-03-18T23:13:53.772 INFO:tasks.workunit.client.0.smithi049.stdout: 1 610114 10.27 MB/sec execute 175 sec latency 20.106 ms 2024-03-18T23:13:54.772 INFO:tasks.workunit.client.0.smithi049.stdout: 1 611920 10.27 MB/sec execute 176 sec latency 21.508 ms 2024-03-18T23:13:54.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:13:54 smithi139 ceph-mon[67282]: pgmap v174: 97 pgs: 97 active+clean; 74 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.6 MiB/s wr, 269 op/s 2024-03-18T23:13:54.879 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:13:54 smithi049 ceph-mon[62991]: pgmap v174: 97 pgs: 97 active+clean; 74 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.6 MiB/s wr, 269 op/s 2024-03-18T23:13:54.961 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:13:54 smithi175 ceph-mon[67352]: pgmap v174: 97 pgs: 97 active+clean; 74 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.6 MiB/s wr, 269 op/s 2024-03-18T23:13:55.772 INFO:tasks.workunit.client.0.smithi049.stdout: 1 614114 10.28 MB/sec execute 177 sec latency 19.795 ms 2024-03-18T23:13:56.772 INFO:tasks.workunit.client.0.smithi049.stdout: 1 615351 10.25 MB/sec execute 178 sec latency 37.933 ms 2024-03-18T23:13:56.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:13:56 smithi139 ceph-mon[67282]: pgmap v175: 97 pgs: 97 active+clean; 74 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.6 MiB/s wr, 269 op/s 2024-03-18T23:13:56.879 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:13:56 smithi049 ceph-mon[62991]: pgmap v175: 97 pgs: 97 active+clean; 74 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.6 MiB/s wr, 269 op/s 2024-03-18T23:13:56.961 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:13:56 smithi175 ceph-mon[67352]: pgmap v175: 97 pgs: 97 active+clean; 74 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.6 MiB/s wr, 269 op/s 2024-03-18T23:13:57.772 INFO:tasks.workunit.client.0.smithi049.stdout: 1 617446 10.26 MB/sec execute 179 sec latency 18.149 ms 2024-03-18T23:13:58.772 INFO:tasks.workunit.client.0.smithi049.stdout: 1 619221 10.25 MB/sec execute 180 sec latency 20.118 ms 2024-03-18T23:13:58.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:13:58 smithi139 ceph-mon[67282]: pgmap v176: 97 pgs: 97 active+clean; 82 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 4.0 MiB/s wr, 361 op/s 2024-03-18T23:13:58.879 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:13:58 smithi049 ceph-mon[62991]: pgmap v176: 97 pgs: 97 active+clean; 82 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 4.0 MiB/s wr, 361 op/s 2024-03-18T23:13:58.961 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:13:58 smithi175 ceph-mon[67352]: pgmap v176: 97 pgs: 97 active+clean; 82 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 4.0 MiB/s wr, 361 op/s 2024-03-18T23:13:59.772 INFO:tasks.workunit.client.0.smithi049.stdout: 1 621591 10.28 MB/sec execute 181 sec latency 19.482 ms 2024-03-18T23:14:00.772 INFO:tasks.workunit.client.0.smithi049.stdout: 1 622557 10.24 MB/sec execute 182 sec latency 30.477 ms 2024-03-18T23:14:01.379 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:14:01 smithi049 ceph-mon[62991]: pgmap v177: 97 pgs: 97 active+clean; 82 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.6 MiB/s wr, 270 op/s 2024-03-18T23:14:01.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:14:01 smithi175 ceph-mon[67352]: pgmap v177: 97 pgs: 97 active+clean; 82 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.6 MiB/s wr, 270 op/s 2024-03-18T23:14:01.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:14:01 smithi139 ceph-mon[67282]: pgmap v177: 97 pgs: 97 active+clean; 82 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.6 MiB/s wr, 270 op/s 2024-03-18T23:14:01.773 INFO:tasks.workunit.client.0.smithi049.stdout: 1 623911 10.22 MB/sec execute 183 sec latency 117.892 ms 2024-03-18T23:14:02.772 INFO:tasks.workunit.client.0.smithi049.stdout: 1 625809 10.23 MB/sec execute 184 sec latency 14.120 ms 2024-03-18T23:14:03.248 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:03.246+0000 7f7546f1b640 1 -- 172.21.15.49:0/1629825747 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f75401056c0 msgr2=0x7f7540105aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:14:03.248 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:03.246+0000 7f7546f1b640 1 --2- 172.21.15.49:0/1629825747 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f75401056c0 0x7f7540105aa0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f7534009900 tx=0x7f753402bc30 comp rx=0 tx=0).stop 2024-03-18T23:14:03.248 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:03.246+0000 7f7546f1b640 1 -- 172.21.15.49:0/1629825747 shutdown_connections 2024-03-18T23:14:03.248 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:03.246+0000 7f7546f1b640 1 --2- 172.21.15.49:0/1629825747 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7540101330 0x7f754010d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:03.248 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:03.246+0000 7f7546f1b640 1 --2- 172.21.15.49:0/1629825747 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7540100970 0x7f7540100df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:03.249 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:03.246+0000 7f7546f1b640 1 --2- 172.21.15.49:0/1629825747 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f75401056c0 0x7f7540105aa0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:03.249 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:03.246+0000 7f7546f1b640 1 -- 172.21.15.49:0/1629825747 >> 172.21.15.49:0/1629825747 conn(0x7f7540077570 msgr2=0x7f7540077980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:14:03.249 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:03.246+0000 7f7546f1b640 1 -- 172.21.15.49:0/1629825747 shutdown_connections 2024-03-18T23:14:03.249 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:03.246+0000 7f7546f1b640 1 -- 172.21.15.49:0/1629825747 wait complete. 2024-03-18T23:14:03.249 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:03.247+0000 7f7546f1b640 1 Processor -- start 2024-03-18T23:14:03.249 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:03.247+0000 7f7546f1b640 1 -- start start 2024-03-18T23:14:03.249 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:03.247+0000 7f7546f1b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7540100970 0x7f7540198ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:14:03.249 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:03.247+0000 7f7546f1b640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7540101330 0x7f7540199420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:14:03.249 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:03.247+0000 7f7546f1b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f75401056c0 0x7f75401a3f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:14:03.249 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:03.247+0000 7f7546f1b640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f7540110ba0 con 0x7f75401056c0 2024-03-18T23:14:03.249 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:03.247+0000 7f7546f1b640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f7540110a20 con 0x7f7540100970 2024-03-18T23:14:03.249 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:03.247+0000 7f7546f1b640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f7540110d20 con 0x7f7540101330 2024-03-18T23:14:03.250 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:03.247+0000 7f7544cb7640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7540100970 0x7f7540198ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:14:03.250 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:03.247+0000 7f75454b8640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f75401056c0 0x7f75401a3f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:14:03.250 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:03.247+0000 7f753ffff640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7540101330 0x7f7540199420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:14:03.250 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:03.247+0000 7f7544cb7640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7540100970 0x7f7540198ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.49:44414/0 (socket says 172.21.15.49:44414) 2024-03-18T23:14:03.250 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:03.247+0000 7f75454b8640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f75401056c0 0x7f75401a3f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:42904/0 (socket says 172.21.15.49:42904) 2024-03-18T23:14:03.250 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:03.247+0000 7f7544cb7640 1 -- 172.21.15.49:0/2373444457 learned_addr learned my addr 172.21.15.49:0/2373444457 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:14:03.250 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:03.248+0000 7f7544cb7640 1 -- 172.21.15.49:0/2373444457 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7540101330 msgr2=0x7f7540199420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:14:03.250 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:03.248+0000 7f7544cb7640 1 --2- 172.21.15.49:0/2373444457 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7540101330 0x7f7540199420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:03.250 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:03.248+0000 7f7544cb7640 1 -- 172.21.15.49:0/2373444457 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f75401056c0 msgr2=0x7f75401a3f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:14:03.250 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:03.248+0000 7f7544cb7640 1 --2- 172.21.15.49:0/2373444457 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f75401056c0 0x7f75401a3f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:03.250 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:03.248+0000 7f7544cb7640 1 -- 172.21.15.49:0/2373444457 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75401a44a0 con 0x7f7540100970 2024-03-18T23:14:03.250 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:03.248+0000 7f7544cb7640 1 --2- 172.21.15.49:0/2373444457 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7540100970 0x7f7540198ee0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f7534005940 tx=0x7f75340042f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:14:03.250 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:03.248+0000 7f753dffb640 1 -- 172.21.15.49:0/2373444457 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7534041070 con 0x7f7540100970 2024-03-18T23:14:03.251 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:03.248+0000 7f7546f1b640 1 -- 172.21.15.49:0/2373444457 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75401a4670 con 0x7f7540100970 2024-03-18T23:14:03.251 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:03.248+0000 7f753dffb640 1 -- 172.21.15.49:0/2373444457 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f753402ccc0 con 0x7f7540100970 2024-03-18T23:14:03.251 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:03.248+0000 7f753dffb640 1 -- 172.21.15.49:0/2373444457 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7534003d80 con 0x7f7540100970 2024-03-18T23:14:03.251 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:03.248+0000 7f7546f1b640 1 -- 172.21.15.49:0/2373444457 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75401a4ae0 con 0x7f7540100970 2024-03-18T23:14:03.256 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:03.251+0000 7f7549be2740 1 Processor -- start 2024-03-18T23:14:03.256 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:03.251+0000 7f753dffb640 1 -- 172.21.15.49:0/2373444457 <== mon.2 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 25) ==== 137421+0+0 (secure 0 0 0) 0x7f7534044070 con 0x7f7540100970 2024-03-18T23:14:03.256 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:03.251+0000 7f7549be2740 1 -- start start 2024-03-18T23:14:03.256 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:03.251+0000 7f7549be2740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ab6353def0 0x55ab6353e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:14:03.256 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:03.251+0000 7f7549be2740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ab6353e810 0x55ab635470a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:14:03.256 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:03.251+0000 7f7549be2740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ab635475e0 0x55ab635499d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:14:03.256 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:03.251+0000 7f7549be2740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55ab634a1be0 con 0x55ab6353def0 2024-03-18T23:14:03.256 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:03.251+0000 7f7549be2740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55ab6344a960 con 0x55ab6353e810 2024-03-18T23:14:03.256 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:03.251+0000 7f7549be2740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55ab6349dc80 con 0x55ab635475e0 2024-03-18T23:14:03.257 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:03.251+0000 7f75454b8640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ab6353def0 0x55ab6353e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:14:03.257 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:03.251+0000 7f75454b8640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ab6353def0 0x55ab6353e2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:42912/0 (socket says 172.21.15.49:42912) 2024-03-18T23:14:03.257 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:03.251+0000 7f75454b8640 1 -- 172.21.15.49:0/1538628136 learned_addr learned my addr 172.21.15.49:0/1538628136 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:14:03.257 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:03.251+0000 7f7544cb7640 1 --2- 172.21.15.49:0/1538628136 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ab6353e810 0x55ab635470a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:14:03.257 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:03.251+0000 7f753ffff640 1 --2- 172.21.15.49:0/1538628136 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ab635475e0 0x55ab635499d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:14:03.257 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:03.251+0000 7f753dffb640 1 --2- 172.21.15.49:0/2373444457 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f75300a2a10 0x7f75300a4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:14:03.257 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:03.251+0000 7f75454b8640 1 -- 172.21.15.49:0/1538628136 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ab635475e0 msgr2=0x55ab635499d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:14:03.257 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:03.251+0000 7f75454b8640 1 --2- 172.21.15.49:0/1538628136 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ab635475e0 0x55ab635499d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:03.257 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:03.251+0000 7f75454b8640 1 -- 172.21.15.49:0/1538628136 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ab6353e810 msgr2=0x55ab635470a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:14:03.257 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:03.251+0000 7f75454b8640 1 --2- 172.21.15.49:0/1538628136 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ab6353e810 0x55ab635470a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:03.257 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:03.251+0000 7f75454b8640 1 -- 172.21.15.49:0/1538628136 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ab63441d20 con 0x55ab6353def0 2024-03-18T23:14:03.257 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:03.252+0000 7f753dffb640 1 -- 172.21.15.49:0/2373444457 <== mon.2 v2:172.21.15.139:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8512+0+0 (secure 0 0 0) 0x7f75340f0bd0 con 0x7f7540100970 2024-03-18T23:14:03.257 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:03.252+0000 7f75454b8640 1 --2- 172.21.15.49:0/2373444457 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f75300a2a10 0x7f75300a4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:14:03.257 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:03.252+0000 7f75454b8640 1 --2- 172.21.15.49:0/1538628136 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ab6353def0 0x55ab6353e2d0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f7538009f40 tx=0x7f75380118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2fb36b5a9a2bad server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:14:03.258 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:03.252+0000 7f752affd640 1 -- 172.21.15.49:0/1538628136 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7538004250 con 0x55ab6353def0 2024-03-18T23:14:03.258 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:03.252+0000 7f752affd640 1 -- 172.21.15.49:0/1538628136 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f753800fe80 con 0x55ab6353def0 2024-03-18T23:14:03.258 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:03.252+0000 7f752affd640 1 -- 172.21.15.49:0/1538628136 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f753801bb20 con 0x55ab6353def0 2024-03-18T23:14:03.258 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:03.253+0000 7f7549be2740 1 -- 172.21.15.49:0/1538628136 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ab6353def0 msgr2=0x55ab6353e2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:14:03.258 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:03.253+0000 7f75454b8640 1 --2- 172.21.15.49:0/2373444457 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f75300a2a10 0x7f75300a4ed0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f7538009fd0 tx=0x7f7538021000 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:14:03.258 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:03.253+0000 7f7549be2740 1 --2- 172.21.15.49:0/1538628136 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ab6353def0 0x55ab6353e2d0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f7538009f40 tx=0x7f75380118d0 comp rx=0 tx=0).stop 2024-03-18T23:14:03.258 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:03.253+0000 7f7549be2740 1 -- 172.21.15.49:0/1538628136 shutdown_connections 2024-03-18T23:14:03.258 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:03.253+0000 7f7549be2740 1 --2- 172.21.15.49:0/1538628136 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ab635475e0 0x55ab635499d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:03.258 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:03.253+0000 7f7549be2740 1 --2- 172.21.15.49:0/1538628136 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ab6353e810 0x55ab635470a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:03.258 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:03.253+0000 7f7549be2740 1 --2- 172.21.15.49:0/1538628136 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ab6353def0 0x55ab6353e2d0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:03.258 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:03.253+0000 7f7549be2740 1 -- 172.21.15.49:0/1538628136 >> 172.21.15.49:0/1538628136 conn(0x55ab63538810 msgr2=0x55ab63538bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:14:03.258 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:03.253+0000 7f7549be2740 1 -- 172.21.15.49:0/1538628136 shutdown_connections 2024-03-18T23:14:03.258 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:03.253+0000 7f7549be2740 1 -- 172.21.15.49:0/1538628136 wait complete. 2024-03-18T23:14:03.258 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:03.253+0000 7f7549be2740 1 -- start start 2024-03-18T23:14:03.259 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:03.253+0000 7f7549be2740 1 Processor -- start 2024-03-18T23:14:03.259 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:03.254+0000 7f7549be2740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ab6353def0 0x55ab63546fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:14:03.259 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:03.254+0000 7f7549be2740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ab6353e810 0x55ab63544370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:14:03.259 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:03.254+0000 7f7549be2740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ab635475e0 0x55ab63544980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:14:03.259 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:03.254+0000 7f7549be2740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55ab6354a210 con 0x55ab6353e810 2024-03-18T23:14:03.259 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:03.254+0000 7f7549be2740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55ab6345ddb0 con 0x55ab6353def0 2024-03-18T23:14:03.259 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:03.254+0000 7f7549be2740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55ab6354a390 con 0x55ab635475e0 2024-03-18T23:14:03.259 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:03.254+0000 7f7549be2740 10 client.0 _ll_register_callbacks cb 0x55ab63462e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T23:14:03.259 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:03.254+0000 7f7544cb7640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ab6353def0 0x55ab63546fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:14:03.259 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:03.254+0000 7f75454b8640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ab635475e0 0x55ab63544980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:14:03.259 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:03.254+0000 7f7544cb7640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ab6353def0 0x55ab63546fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.49:44428/0 (socket says 172.21.15.49:44428) 2024-03-18T23:14:03.259 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:03.254+0000 7f7544cb7640 1 -- 172.21.15.49:0/1899600003 learned_addr learned my addr 172.21.15.49:0/1899600003 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:14:03.259 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:03.254+0000 7f753ffff640 1 --2- 172.21.15.49:0/1899600003 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ab6353e810 0x55ab63544370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:14:03.260 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:03.254+0000 7f7544cb7640 1 -- 172.21.15.49:0/1899600003 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ab635475e0 msgr2=0x55ab63544980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:14:03.260 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:03.254+0000 7f7544cb7640 1 --2- 172.21.15.49:0/1899600003 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ab635475e0 0x55ab63544980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:03.260 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:03.254+0000 7f7544cb7640 1 -- 172.21.15.49:0/1899600003 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ab6353e810 msgr2=0x55ab63544370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:14:03.260 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:03.254+0000 7f7544cb7640 1 --2- 172.21.15.49:0/1899600003 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ab6353e810 0x55ab63544370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:03.260 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:03.255+0000 7f7544cb7640 1 -- 172.21.15.49:0/1899600003 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ab6345cc60 con 0x55ab6353def0 2024-03-18T23:14:03.260 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:03.255+0000 7f7544cb7640 1 --2- 172.21.15.49:0/1899600003 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ab6353def0 0x55ab63546fe0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f7534000c00 tx=0x7f7534005b70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=15ff08f1dd4cc42b server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:14:03.260 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:03.255+0000 7f751ffff640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-03-18T23:14:03.260 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:03.255+0000 7f751ffff640 1 -- 172.21.15.49:0/1899600003 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f753404b070 con 0x55ab6353def0 2024-03-18T23:14:03.260 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:03.255+0000 7f751ffff640 1 -- 172.21.15.49:0/1899600003 <== mon.2 v2:172.21.15.139:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8512+0+0 (secure 0 0 0) 0x7f75340bd620 con 0x55ab6353def0 2024-03-18T23:14:03.260 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:03.255+0000 7f7549be2740 1 -- 172.21.15.49:0/1899600003 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55ab634af120 con 0x55ab6353def0 2024-03-18T23:14:03.260 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:03.255+0000 7f751ffff640 1 -- 172.21.15.49:0/1899600003 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f75340bd190 con 0x55ab6353def0 2024-03-18T23:14:03.260 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:03.255+0000 7f7549be2740 10 client.25739 fetch_fsmap learned FSMap version 32 2024-03-18T23:14:03.260 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:03.255+0000 7f7549be2740 1 -- 172.21.15.49:0/1899600003 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55ab631b6210 con 0x55ab6353def0 2024-03-18T23:14:03.260 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:03.256+0000 7f751ffff640 1 -- 172.21.15.49:0/1899600003 <== mon.2 v2:172.21.15.139:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55ab634af120 con 0x55ab6353def0 2024-03-18T23:14:03.260 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:03.256+0000 7f751ffff640 1 -- 172.21.15.49:0/1899600003 <== mon.2 v2:172.21.15.139:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f75340c5620 con 0x55ab6353def0 2024-03-18T23:14:03.261 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:03.256+0000 7f7549be2740 10 client.25739 fetch_fsmap finished waiting for FSMap version 32 2024-03-18T23:14:03.261 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:03.256+0000 7f7549be2740 10 client.25739 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T23:14:03.261 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:03.256+0000 7f7549be2740 20 client.25739 populate_metadata read hostname 'smithi049' 2024-03-18T23:14:03.261 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:03.256+0000 7f7549be2740 1 --2- 172.21.15.49:0/1899600003 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x55ab6355b8b0 0x55ab6355dd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:14:03.261 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:03.256+0000 7f7549be2740 4 client.25739 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T23:14:03.261 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:03.256+0000 7f7549be2740 1 -- 172.21.15.49:0/1899600003 --> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ab634a9470 con 0x55ab6355b8b0 2024-03-18T23:14:03.261 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:03.256+0000 7f753ffff640 1 --2- 172.21.15.49:0/1899600003 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x55ab6355b8b0 0x55ab6355dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:14:03.261 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:03.257+0000 7f751ffff640 10 client.25739 ms_handle_connect on v2:172.21.15.139:6838/4280128209 2024-03-18T23:14:03.261 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:03.257+0000 7f753ffff640 1 --2- 172.21.15.49:0/1899600003 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x55ab6355b8b0 0x55ab6355dd50 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=98f8dd4b112fc122 server_cookie=c82d4ec30c69eb30 in_seq=0 out_seq=0 2024-03-18T23:14:03.262 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:03.258+0000 7f751ffff640 1 -- 172.21.15.49:0/1899600003 <== mds.0 v2:172.21.15.139:6838/4280128209 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55ab634a9470 con 0x55ab6355b8b0 2024-03-18T23:14:03.262 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:03.258+0000 7f751ffff640 10 client.25739 handle_command_reply: tid=0 2024-03-18T23:14:03.262 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:03.258+0000 7f7549be2740 1 client.25739 shutdown 2024-03-18T23:14:03.262 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:03.258+0000 7f7549be2740 1 -- 172.21.15.49:0/1899600003 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ab6353def0 msgr2=0x55ab63546fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:14:03.263 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:03.258+0000 7f7549be2740 1 --2- 172.21.15.49:0/1899600003 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ab6353def0 0x55ab63546fe0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f7534000c00 tx=0x7f7534005b70 comp rx=0 tx=0).stop 2024-03-18T23:14:03.263 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:03.258+0000 7f7549be2740 1 -- 172.21.15.49:0/1899600003 shutdown_connections 2024-03-18T23:14:03.263 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:03.258+0000 7f7549be2740 1 --2- 172.21.15.49:0/1899600003 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x55ab6355b8b0 0x55ab6355dd50 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:03.263 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:03.258+0000 7f7549be2740 1 --2- 172.21.15.49:0/1899600003 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ab635475e0 0x55ab63544980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:03.263 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:03.258+0000 7f751ffff640 0 client.25739 ms_handle_reset on v2:172.21.15.139:6838/4280128209 2024-03-18T23:14:03.263 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:03.258+0000 7f7549be2740 1 --2- 172.21.15.49:0/1899600003 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ab6353e810 0x55ab63544370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:03.263 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:03.258+0000 7f7549be2740 1 --2- 172.21.15.49:0/1899600003 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ab6353def0 0x55ab63546fe0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:03.263 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:03.258+0000 7f7549be2740 1 -- 172.21.15.49:0/1899600003 >> 172.21.15.49:0/1899600003 conn(0x55ab63538810 msgr2=0x55ab63537ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:14:03.263 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:03.258+0000 7f7549be2740 1 -- 172.21.15.49:0/1899600003 shutdown_connections 2024-03-18T23:14:03.263 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:03.258+0000 7f7549be2740 1 -- 172.21.15.49:0/1899600003 wait complete. 2024-03-18T23:14:03.263 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:03.258+0000 7f7549be2740 20 client.25739 trim_cache size 0 max 16384 2024-03-18T23:14:03.274 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:03.273+0000 7f7549be2740 1 Processor -- start 2024-03-18T23:14:03.274 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:03.273+0000 7f7549be2740 1 -- start start 2024-03-18T23:14:03.274 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:03.273+0000 7f7549be2740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ab6353def0 0x55ab63552fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:14:03.274 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:03.273+0000 7f7549be2740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ab6353e810 0x55ab63563620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:14:03.274 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:03.273+0000 7f7549be2740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ab635475e0 0x55ab63546aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:14:03.274 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:03.273+0000 7f7549be2740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f75401ab2c0 con 0x55ab6353e810 2024-03-18T23:14:03.274 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:03.273+0000 7f7549be2740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f75401ace00 con 0x55ab635475e0 2024-03-18T23:14:03.274 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:03.273+0000 7f7549be2740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f75401ab990 con 0x55ab6353def0 2024-03-18T23:14:03.274 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:03.273+0000 7f753ffff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ab6353e810 0x55ab63563620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:14:03.275 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:03.274+0000 7f753ffff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ab6353e810 0x55ab63563620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:42928/0 (socket says 172.21.15.49:42928) 2024-03-18T23:14:03.275 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:03.274+0000 7f753ffff640 1 -- 172.21.15.49:0/1861757974 learned_addr learned my addr 172.21.15.49:0/1861757974 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:14:03.275 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:03.274+0000 7f75454b8640 1 --2- 172.21.15.49:0/1861757974 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ab635475e0 0x55ab63546aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:14:03.275 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:03.274+0000 7f7544cb7640 1 --2- 172.21.15.49:0/1861757974 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ab6353def0 0x55ab63552fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:14:03.275 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:03.274+0000 7f753ffff640 1 -- 172.21.15.49:0/1861757974 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ab6353def0 msgr2=0x55ab63552fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:14:03.275 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:03.274+0000 7f753ffff640 1 --2- 172.21.15.49:0/1861757974 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ab6353def0 0x55ab63552fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:03.275 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:03.274+0000 7f753ffff640 1 -- 172.21.15.49:0/1861757974 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ab635475e0 msgr2=0x55ab63546aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:14:03.275 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:03.274+0000 7f753ffff640 1 --2- 172.21.15.49:0/1861757974 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ab635475e0 0x55ab63546aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:03.275 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:03.274+0000 7f753ffff640 1 -- 172.21.15.49:0/1861757974 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ab63282620 con 0x55ab6353e810 2024-03-18T23:14:03.275 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:03.274+0000 7f753ffff640 1 --2- 172.21.15.49:0/1861757974 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ab6353e810 0x55ab63563620 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x55ab63539c30 tx=0x7f752c00b390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96375fd9c210a8a5 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:14:03.275 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:03.274+0000 7f752a7fc640 1 -- 172.21.15.49:0/1861757974 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f752c003d60 con 0x55ab6353e810 2024-03-18T23:14:03.275 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:03.274+0000 7f752a7fc640 1 -- 172.21.15.49:0/1861757974 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f752c005500 con 0x55ab6353e810 2024-03-18T23:14:03.275 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:03.274+0000 7f752a7fc640 1 -- 172.21.15.49:0/1861757974 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f752c004a00 con 0x55ab6353e810 2024-03-18T23:14:03.279 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:03.275+0000 7f7549be2740 1 -- 172.21.15.49:0/1861757974 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ab6353e810 msgr2=0x55ab63563620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:14:03.279 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:03.275+0000 7f7549be2740 1 --2- 172.21.15.49:0/1861757974 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ab6353e810 0x55ab63563620 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x55ab63539c30 tx=0x7f752c00b390 comp rx=0 tx=0).stop 2024-03-18T23:14:03.279 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:03.275+0000 7f7549be2740 1 -- 172.21.15.49:0/1861757974 shutdown_connections 2024-03-18T23:14:03.279 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:03.275+0000 7f7549be2740 1 --2- 172.21.15.49:0/1861757974 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ab635475e0 0x55ab63546aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:03.279 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:03.275+0000 7f7549be2740 1 --2- 172.21.15.49:0/1861757974 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ab6353e810 0x55ab63563620 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:03.279 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:03.275+0000 7f7549be2740 1 --2- 172.21.15.49:0/1861757974 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ab6353def0 0x55ab63552fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:03.279 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:03.275+0000 7f7549be2740 1 -- 172.21.15.49:0/1861757974 >> 172.21.15.49:0/1861757974 conn(0x55ab63538810 msgr2=0x55ab63537ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:14:03.279 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:03.275+0000 7f7549be2740 1 -- 172.21.15.49:0/1861757974 shutdown_connections 2024-03-18T23:14:03.279 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:03.275+0000 7f7549be2740 1 -- 172.21.15.49:0/1861757974 wait complete. 2024-03-18T23:14:03.279 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:03.275+0000 7f7549be2740 1 -- start start 2024-03-18T23:14:03.279 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:03.276+0000 7f7549be2740 1 Processor -- start 2024-03-18T23:14:03.280 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:03.276+0000 7f7549be2740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ab6353def0 0x55ab6354f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:14:03.280 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:03.276+0000 7f7549be2740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ab6353e810 0x55ab635409d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:14:03.280 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:03.276+0000 7f7549be2740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ab635475e0 0x55ab63540f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:14:03.280 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:03.276+0000 7f7549be2740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f75401ac730 con 0x55ab6353def0 2024-03-18T23:14:03.280 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:03.276+0000 7f7549be2740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f75401ac060 con 0x55ab635475e0 2024-03-18T23:14:03.280 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:03.276+0000 7f7549be2740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f75401a92a0 con 0x55ab6353e810 2024-03-18T23:14:03.280 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:03.276+0000 7f7549be2740 10 client.0 _ll_register_callbacks cb 0x7f7534003f20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T23:14:03.280 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:03.276+0000 7f7544cb7640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ab6353def0 0x55ab6354f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:14:03.280 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:03.276+0000 7f7544cb7640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ab6353def0 0x55ab6354f5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:42930/0 (socket says 172.21.15.49:42930) 2024-03-18T23:14:03.280 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:03.276+0000 7f7544cb7640 1 -- 172.21.15.49:0/557938515 learned_addr learned my addr 172.21.15.49:0/557938515 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:14:03.280 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:03.276+0000 7f75454b8640 1 --2- 172.21.15.49:0/557938515 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ab635475e0 0x55ab63540f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:14:03.280 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:03.276+0000 7f753ffff640 1 --2- 172.21.15.49:0/557938515 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ab6353e810 0x55ab635409d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:14:03.280 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:03.276+0000 7f7544cb7640 1 -- 172.21.15.49:0/557938515 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ab6353e810 msgr2=0x55ab635409d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:14:03.280 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:03.276+0000 7f7544cb7640 1 --2- 172.21.15.49:0/557938515 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ab6353e810 0x55ab635409d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:03.280 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:03.276+0000 7f7544cb7640 1 -- 172.21.15.49:0/557938515 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ab635475e0 msgr2=0x55ab63540f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:14:03.281 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:03.276+0000 7f7544cb7640 1 --2- 172.21.15.49:0/557938515 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ab635475e0 0x55ab63540f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:03.281 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:03.276+0000 7f7544cb7640 1 -- 172.21.15.49:0/557938515 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ab63541450 con 0x55ab6353def0 2024-03-18T23:14:03.281 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:03.277+0000 7f7544cb7640 1 --2- 172.21.15.49:0/557938515 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ab6353def0 0x55ab6354f5e0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f75340bd0c0 tx=0x7f7534009680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58848141e4738479 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:14:03.281 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:03.277+0000 7f751ffff640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-18T23:14:03.281 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:03.277+0000 7f751ffff640 1 -- 172.21.15.49:0/557938515 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f753404b040 con 0x55ab6353def0 2024-03-18T23:14:03.281 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:03.277+0000 7f751ffff640 1 -- 172.21.15.49:0/557938515 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8512+0+0 (secure 0 0 0) 0x7f75340ee4e0 con 0x55ab6353def0 2024-03-18T23:14:03.281 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:03.277+0000 7f7549be2740 1 -- 172.21.15.49:0/557938515 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f752c005500 con 0x55ab6353def0 2024-03-18T23:14:03.281 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:03.277+0000 7f751ffff640 1 -- 172.21.15.49:0/557938515 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f75340bcd20 con 0x55ab6353def0 2024-03-18T23:14:03.281 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:03.277+0000 7f7549be2740 10 client.15930 fetch_fsmap learned FSMap version 32 2024-03-18T23:14:03.281 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:03.277+0000 7f7549be2740 1 -- 172.21.15.49:0/557938515 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55ab635c1990 con 0x55ab6353def0 2024-03-18T23:14:03.281 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:03.277+0000 7f751ffff640 1 -- 172.21.15.49:0/557938515 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f752c005500 con 0x55ab6353def0 2024-03-18T23:14:03.281 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:03.277+0000 7f751ffff640 1 -- 172.21.15.49:0/557938515 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f75340bc810 con 0x55ab6353def0 2024-03-18T23:14:03.281 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:03.277+0000 7f7549be2740 10 client.15930 fetch_fsmap finished waiting for FSMap version 32 2024-03-18T23:14:03.281 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:03.277+0000 7f7549be2740 10 client.15930 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T23:14:03.282 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:03.277+0000 7f7549be2740 20 client.15930 populate_metadata read hostname 'smithi049' 2024-03-18T23:14:03.282 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:03.277+0000 7f7549be2740 1 --2- 172.21.15.49:0/557938515 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x55ab635c1f50 0x55ab635c4370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:14:03.282 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:03.277+0000 7f7549be2740 4 client.15930 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T23:14:03.282 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:03.277+0000 7f7549be2740 1 -- 172.21.15.49:0/557938515 --> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ab63427770 con 0x55ab635c1f50 2024-03-18T23:14:03.282 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:03.277+0000 7f753ffff640 1 --2- 172.21.15.49:0/557938515 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x55ab635c1f50 0x55ab635c4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:14:03.282 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:03.278+0000 7f753ffff640 1 --2- 172.21.15.49:0/557938515 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x55ab635c1f50 0x55ab635c4370 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9cfaac5926aa0af0 server_cookie=c78f61af8e72686b in_seq=0 out_seq=0 2024-03-18T23:14:03.282 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:03.278+0000 7f751ffff640 10 client.15930 ms_handle_connect on v2:172.21.15.139:6838/4280128209 2024-03-18T23:14:03.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:14:03 smithi049 ceph-mon[62991]: pgmap v178: 97 pgs: 97 active+clean; 88 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 4.1 MiB/s wr, 392 op/s 2024-03-18T23:14:03.443 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:03.442+0000 7f751ffff640 1 -- 172.21.15.49:0/557938515 <== mds.0 v2:172.21.15.139:6838/4280128209 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ab63427770 con 0x55ab635c1f50 2024-03-18T23:14:03.443 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:03.442+0000 7f751ffff640 10 client.15930 handle_command_reply: tid=0 2024-03-18T23:14:03.443 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:03.442+0000 7f7549be2740 1 client.15930 shutdown 2024-03-18T23:14:03.443 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:03.443+0000 7f7549be2740 1 -- 172.21.15.49:0/557938515 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ab6353def0 msgr2=0x55ab6354f5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:14:03.443 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:03.443+0000 7f7549be2740 1 --2- 172.21.15.49:0/557938515 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ab6353def0 0x55ab6354f5e0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f75340bd0c0 tx=0x7f7534009680 comp rx=0 tx=0).stop 2024-03-18T23:14:03.443 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:03.443+0000 7f7549be2740 1 -- 172.21.15.49:0/557938515 shutdown_connections 2024-03-18T23:14:03.443 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:03.443+0000 7f7549be2740 1 --2- 172.21.15.49:0/557938515 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x55ab635c1f50 0x55ab635c4370 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:03.444 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:03.443+0000 7f7549be2740 1 --2- 172.21.15.49:0/557938515 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ab635475e0 0x55ab63540f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:03.444 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:03.443+0000 7f7549be2740 1 --2- 172.21.15.49:0/557938515 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ab6353e810 0x55ab635409d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:03.444 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:03.443+0000 7f7549be2740 1 --2- 172.21.15.49:0/557938515 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ab6353def0 0x55ab6354f5e0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:03.444 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:03.443+0000 7f7549be2740 1 -- 172.21.15.49:0/557938515 >> 172.21.15.49:0/557938515 conn(0x55ab63538810 msgr2=0x55ab63539c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:14:03.444 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:03.443+0000 7f751ffff640 0 client.15930 ms_handle_reset on v2:172.21.15.139:6838/4280128209 2024-03-18T23:14:03.444 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:03.443+0000 7f7549be2740 1 -- 172.21.15.49:0/557938515 shutdown_connections 2024-03-18T23:14:03.444 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:03.443+0000 7f7549be2740 1 -- 172.21.15.49:0/557938515 wait complete. 2024-03-18T23:14:03.444 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:03.443+0000 7f7549be2740 20 client.15930 trim_cache size 0 max 16384 2024-03-18T23:14:03.444 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:{ 2024-03-18T23:14:03.444 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "message": "", 2024-03-18T23:14:03.444 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "return_code": 0 2024-03-18T23:14:03.445 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:} 2024-03-18T23:14:03.445 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:03.444+0000 7f7546f1b640 1 -- 172.21.15.49:0/2373444457 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f75300a2a10 msgr2=0x7f75300a4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:14:03.445 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:03.444+0000 7f7546f1b640 1 --2- 172.21.15.49:0/2373444457 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f75300a2a10 0x7f75300a4ed0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f7538009fd0 tx=0x7f7538021000 comp rx=0 tx=0).stop 2024-03-18T23:14:03.445 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:03.444+0000 7f7546f1b640 1 -- 172.21.15.49:0/2373444457 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7540100970 msgr2=0x7f7540198ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:14:03.445 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:03.444+0000 7f7546f1b640 1 --2- 172.21.15.49:0/2373444457 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7540100970 0x7f7540198ee0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f7534005940 tx=0x7f75340042f0 comp rx=0 tx=0).stop 2024-03-18T23:14:03.445 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:03.444+0000 7f7546f1b640 1 -- 172.21.15.49:0/2373444457 shutdown_connections 2024-03-18T23:14:03.445 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:03.444+0000 7f7546f1b640 1 --2- 172.21.15.49:0/2373444457 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f75300a2a10 0x7f75300a4ed0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:03.445 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:03.444+0000 7f7546f1b640 1 --2- 172.21.15.49:0/2373444457 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f75401056c0 0x7f75401a3f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:03.446 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:03.444+0000 7f7546f1b640 1 --2- 172.21.15.49:0/2373444457 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7540101330 0x7f7540199420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:03.446 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:03.444+0000 7f7546f1b640 1 --2- 172.21.15.49:0/2373444457 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7540100970 0x7f7540198ee0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:03.446 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:03.444+0000 7f7546f1b640 1 -- 172.21.15.49:0/2373444457 >> 172.21.15.49:0/2373444457 conn(0x7f7540077570 msgr2=0x7f75400fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:14:03.446 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:03.444+0000 7f7546f1b640 1 -- 172.21.15.49:0/2373444457 shutdown_connections 2024-03-18T23:14:03.446 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:03.445+0000 7f7546f1b640 1 -- 172.21.15.49:0/2373444457 wait complete. 2024-03-18T23:14:03.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:14:03 smithi175 ceph-mon[67352]: pgmap v178: 97 pgs: 97 active+clean; 88 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 4.1 MiB/s wr, 392 op/s 2024-03-18T23:14:03.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:14:03 smithi139 ceph-mon[67282]: pgmap v178: 97 pgs: 97 active+clean; 88 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 4.1 MiB/s wr, 392 op/s 2024-03-18T23:14:03.772 INFO:tasks.workunit.client.0.smithi049.stdout: 1 627472 10.21 MB/sec execute 185 sec latency 113.745 ms 2024-03-18T23:14:04.772 INFO:tasks.workunit.client.0.smithi049.stdout: 1 629293 10.22 MB/sec execute 186 sec latency 22.283 ms 2024-03-18T23:14:05.379 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:14:05 smithi049 ceph-mon[62991]: pgmap v179: 97 pgs: 97 active+clean; 88 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.8 MiB/s wr, 213 op/s 2024-03-18T23:14:05.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:14:05 smithi175 ceph-mon[67352]: pgmap v179: 97 pgs: 97 active+clean; 88 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.8 MiB/s wr, 213 op/s 2024-03-18T23:14:05.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:14:05 smithi139 ceph-mon[67282]: pgmap v179: 97 pgs: 97 active+clean; 88 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.8 MiB/s wr, 213 op/s 2024-03-18T23:14:05.772 INFO:tasks.workunit.client.0.smithi049.stdout: 1 631396 10.23 MB/sec execute 187 sec latency 19.710 ms 2024-03-18T23:14:06.773 INFO:tasks.workunit.client.0.smithi049.stdout: 1 632786 10.21 MB/sec execute 188 sec latency 70.964 ms 2024-03-18T23:14:07.379 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:14:07 smithi049 ceph-mon[62991]: pgmap v180: 97 pgs: 97 active+clean; 88 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.8 MiB/s wr, 213 op/s 2024-03-18T23:14:07.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:14:07 smithi175 ceph-mon[67352]: pgmap v180: 97 pgs: 97 active+clean; 88 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.8 MiB/s wr, 213 op/s 2024-03-18T23:14:07.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:14:07 smithi139 ceph-mon[67282]: pgmap v180: 97 pgs: 97 active+clean; 88 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.8 MiB/s wr, 213 op/s 2024-03-18T23:14:07.773 INFO:tasks.workunit.client.0.smithi049.stdout: 1 634762 10.20 MB/sec execute 189 sec latency 26.867 ms 2024-03-18T23:14:08.773 INFO:tasks.workunit.client.0.smithi049.stdout: 1 636623 10.21 MB/sec execute 190 sec latency 22.610 ms 2024-03-18T23:14:09.379 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:14:09 smithi049 ceph-mon[62991]: pgmap v181: 97 pgs: 97 active+clean; 77 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 4.1 MiB/s wr, 402 op/s 2024-03-18T23:14:09.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:14:09 smithi175 ceph-mon[67352]: pgmap v181: 97 pgs: 97 active+clean; 77 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 4.1 MiB/s wr, 402 op/s 2024-03-18T23:14:09.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:14:09 smithi139 ceph-mon[67282]: pgmap v181: 97 pgs: 97 active+clean; 77 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 4.1 MiB/s wr, 402 op/s 2024-03-18T23:14:09.773 INFO:tasks.workunit.client.0.smithi049.stdout: 1 638729 10.22 MB/sec execute 191 sec latency 18.103 ms 2024-03-18T23:14:10.773 INFO:tasks.workunit.client.0.smithi049.stdout: 1 640328 10.21 MB/sec execute 192 sec latency 30.403 ms 2024-03-18T23:14:11.379 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:14:11 smithi049 ceph-mon[62991]: pgmap v182: 97 pgs: 97 active+clean; 77 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.8 MiB/s wr, 310 op/s 2024-03-18T23:14:11.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:14:11 smithi175 ceph-mon[67352]: pgmap v182: 97 pgs: 97 active+clean; 77 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.8 MiB/s wr, 310 op/s 2024-03-18T23:14:11.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:14:11 smithi139 ceph-mon[67282]: pgmap v182: 97 pgs: 97 active+clean; 77 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.8 MiB/s wr, 310 op/s 2024-03-18T23:14:11.773 INFO:tasks.workunit.client.0.smithi049.stdout: 1 642020 10.20 MB/sec execute 193 sec latency 60.576 ms 2024-03-18T23:14:12.773 INFO:tasks.workunit.client.0.smithi049.stdout: 1 643591 10.20 MB/sec execute 194 sec latency 31.475 ms 2024-03-18T23:14:13.379 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:14:13 smithi049 ceph-mon[62991]: pgmap v183: 97 pgs: 97 active+clean; 80 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 3.8 MiB/s wr, 388 op/s 2024-03-18T23:14:13.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:14:13 smithi175 ceph-mon[67352]: pgmap v183: 97 pgs: 97 active+clean; 80 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 3.8 MiB/s wr, 388 op/s 2024-03-18T23:14:13.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:14:13 smithi139 ceph-mon[67282]: pgmap v183: 97 pgs: 97 active+clean; 80 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 3.8 MiB/s wr, 388 op/s 2024-03-18T23:14:13.773 INFO:tasks.workunit.client.0.smithi049.stdout: 1 645693 10.20 MB/sec execute 195 sec latency 21.503 ms 2024-03-18T23:14:14.773 INFO:tasks.workunit.client.0.smithi049.stdout: 1 647293 10.19 MB/sec execute 196 sec latency 32.392 ms 2024-03-18T23:14:15.379 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:14:15 smithi049 ceph-mon[62991]: pgmap v184: 97 pgs: 97 active+clean; 80 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.3 MiB/s wr, 265 op/s 2024-03-18T23:14:15.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:14:15 smithi175 ceph-mon[67352]: pgmap v184: 97 pgs: 97 active+clean; 80 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.3 MiB/s wr, 265 op/s 2024-03-18T23:14:15.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:14:15 smithi139 ceph-mon[67282]: pgmap v184: 97 pgs: 97 active+clean; 80 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.3 MiB/s wr, 265 op/s 2024-03-18T23:14:15.773 INFO:tasks.workunit.client.0.smithi049.stdout: 1 649648 10.21 MB/sec execute 197 sec latency 17.553 ms 2024-03-18T23:14:16.560 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:16.558+0000 7f8b3c85e640 1 -- 172.21.15.49:0/614938821 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8b30099bc0 msgr2=0x7f8b30099fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:14:16.560 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:16.558+0000 7f8b3c85e640 1 --2- 172.21.15.49:0/614938821 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8b30099bc0 0x7f8b30099fa0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f8b2c009930 tx=0x7f8b2c02bc50 comp rx=0 tx=0).stop 2024-03-18T23:14:16.560 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:16.558+0000 7f8b3c85e640 1 -- 172.21.15.49:0/614938821 shutdown_connections 2024-03-18T23:14:16.560 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:16.558+0000 7f8b3c85e640 1 --2- 172.21.15.49:0/614938821 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8b300957f0 0x7f8b300a1ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:16.560 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:16.558+0000 7f8b3c85e640 1 --2- 172.21.15.49:0/614938821 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8b30094e50 0x7f8b300952b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:16.560 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:16.558+0000 7f8b3c85e640 1 --2- 172.21.15.49:0/614938821 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8b30099bc0 0x7f8b30099fa0 secure :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f8b2c009930 tx=0x7f8b2c02bc50 comp rx=0 tx=0).stop 2024-03-18T23:14:16.560 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:16.558+0000 7f8b3c85e640 1 -- 172.21.15.49:0/614938821 >> 172.21.15.49:0/614938821 conn(0x7f8b3000a740 msgr2=0x7f8b3000ab50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:14:16.561 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:16.559+0000 7f8b3c85e640 1 -- 172.21.15.49:0/614938821 shutdown_connections 2024-03-18T23:14:16.561 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:16.559+0000 7f8b3c85e640 1 -- 172.21.15.49:0/614938821 wait complete. 2024-03-18T23:14:16.561 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:16.560+0000 7f8b3c85e640 1 Processor -- start 2024-03-18T23:14:16.563 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:16.561+0000 7f8b3c85e640 1 -- start start 2024-03-18T23:14:16.563 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:16.562+0000 7f8b3c85e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8b30094e50 0x7f8b300a0310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:14:16.563 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:16.562+0000 7f8b3c85e640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8b300957f0 0x7f8b300af610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:14:16.563 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:16.562+0000 7f8b3c85e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8b300a0850 0x7f8b30147af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:14:16.563 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:16.562+0000 7f8b3c85e640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f8b300a3570 con 0x7f8b300a0850 2024-03-18T23:14:16.563 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:16.562+0000 7f8b3c85e640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f8b300a33f0 con 0x7f8b30094e50 2024-03-18T23:14:16.563 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:16.562+0000 7f8b3c85e640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f8b300a36f0 con 0x7f8b300957f0 2024-03-18T23:14:16.564 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:16.563+0000 7f8b377fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8b30094e50 0x7f8b300a0310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:14:16.564 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:16.563+0000 7f8b377fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8b30094e50 0x7f8b300a0310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.49:43566/0 (socket says 172.21.15.49:43566) 2024-03-18T23:14:16.564 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:16.563+0000 7f8b377fe640 1 -- 172.21.15.49:0/3040591266 learned_addr learned my addr 172.21.15.49:0/3040591266 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:14:16.564 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:16.564+0000 7f8b37fff640 1 --2- 172.21.15.49:0/3040591266 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8b300a0850 0x7f8b30147af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:14:16.565 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:16.564+0000 7f8b377fe640 1 -- 172.21.15.49:0/3040591266 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8b300957f0 msgr2=0x7f8b300af610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:14:16.566 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:16.564+0000 7f8b377fe640 1 --2- 172.21.15.49:0/3040591266 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8b300957f0 0x7f8b300af610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:16.566 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:16.565+0000 7f8b377fe640 1 -- 172.21.15.49:0/3040591266 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8b300a0850 msgr2=0x7f8b30147af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:14:16.566 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:16.565+0000 7f8b377fe640 1 --2- 172.21.15.49:0/3040591266 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8b300a0850 0x7f8b30147af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:16.566 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:16.565+0000 7f8b377fe640 1 -- 172.21.15.49:0/3040591266 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b300a0dd0 con 0x7f8b30094e50 2024-03-18T23:14:16.567 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:16.566+0000 7f8b377fe640 1 --2- 172.21.15.49:0/3040591266 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8b30094e50 0x7f8b300a0310 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f8b2c009850 tx=0x7f8b2c02ce00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:14:16.568 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:16.567+0000 7f8b34ff9640 1 -- 172.21.15.49:0/3040591266 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b2c043020 con 0x7f8b30094e50 2024-03-18T23:14:16.568 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:16.567+0000 7f8b34ff9640 1 -- 172.21.15.49:0/3040591266 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8b2c0054e0 con 0x7f8b30094e50 2024-03-18T23:14:16.568 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:16.567+0000 7f8b34ff9640 1 -- 172.21.15.49:0/3040591266 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b2c003b90 con 0x7f8b30094e50 2024-03-18T23:14:16.568 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:16.567+0000 7f8b3c85e640 1 -- 172.21.15.49:0/3040591266 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b30148030 con 0x7f8b30094e50 2024-03-18T23:14:16.568 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:16.567+0000 7f8b3c85e640 1 -- 172.21.15.49:0/3040591266 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b30148440 con 0x7f8b30094e50 2024-03-18T23:14:16.571 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:16.571+0000 7f8b34ff9640 1 -- 172.21.15.49:0/3040591266 <== mon.2 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 25) ==== 137421+0+0 (secure 0 0 0) 0x7f8b2c046020 con 0x7f8b30094e50 2024-03-18T23:14:16.571 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:16.571+0000 7f8b40784740 1 Processor -- start 2024-03-18T23:14:16.572 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:16.571+0000 7f8b40784740 1 -- start start 2024-03-18T23:14:16.574 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:16.572+0000 7f8b40784740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56318080a0d0 0x56318080a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:14:16.574 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:16.572+0000 7f8b40784740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56318080a9f0 0x563180813280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:14:16.574 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:16.572+0000 7f8b40784740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5631808137c0 0x563180815bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:14:16.574 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:16.572+0000 7f8b40784740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x56318076dbe0 con 0x5631808137c0 2024-03-18T23:14:16.574 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:16.572+0000 7f8b40784740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x563180716960 con 0x56318080a0d0 2024-03-18T23:14:16.575 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:16.572+0000 7f8b40784740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x563180769c80 con 0x56318080a9f0 2024-03-18T23:14:16.575 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:16.572+0000 7f8b37fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56318080a0d0 0x56318080a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:14:16.575 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:16.572+0000 7f8b37fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56318080a0d0 0x56318080a4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.49:43578/0 (socket says 172.21.15.49:43578) 2024-03-18T23:14:16.575 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:16.572+0000 7f8b37fff640 1 -- 172.21.15.49:0/1634240119 learned_addr learned my addr 172.21.15.49:0/1634240119 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:14:16.575 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:16.572+0000 7f8b34ff9640 1 --2- 172.21.15.49:0/3040591266 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f8b280a2af0 0x7f8b280a4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:14:16.575 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:16.572+0000 7f8b377fe640 1 --2- 172.21.15.49:0/1634240119 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56318080a9f0 0x563180813280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:14:16.575 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:16.572+0000 7f8b36ffd640 1 --2- 172.21.15.49:0/1634240119 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5631808137c0 0x563180815bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:14:16.575 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:16.572+0000 7f8b37fff640 1 --2- 172.21.15.49:0/3040591266 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f8b280a2af0 0x7f8b280a4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:14:16.575 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:16.573+0000 7f8b37fff640 1 -- 172.21.15.49:0/1634240119 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56318080a9f0 msgr2=0x563180813280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:14:16.575 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:16.573+0000 7f8b37fff640 1 --2- 172.21.15.49:0/1634240119 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56318080a9f0 0x563180813280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:16.575 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:16.573+0000 7f8b37fff640 1 -- 172.21.15.49:0/1634240119 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5631808137c0 msgr2=0x563180815bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:14:16.576 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:16.573+0000 7f8b37fff640 1 --2- 172.21.15.49:0/1634240119 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5631808137c0 0x563180815bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:16.576 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:16.573+0000 7f8b37fff640 1 -- 172.21.15.49:0/1634240119 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56318070dd20 con 0x56318080a0d0 2024-03-18T23:14:16.576 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:16.573+0000 7f8b36ffd640 1 --2- 172.21.15.49:0/1634240119 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5631808137c0 0x563180815bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T23:14:16.576 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:16.573+0000 7f8b34ff9640 1 -- 172.21.15.49:0/3040591266 <== mon.2 v2:172.21.15.139:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8512+0+0 (secure 0 0 0) 0x7f8b2c0f0df0 con 0x7f8b30094e50 2024-03-18T23:14:16.576 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:16.573+0000 7f8b37fff640 1 --2- 172.21.15.49:0/1634240119 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56318080a0d0 0x56318080a4b0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f8b3804f850 tx=0x7f8b38082660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b0551bfb05ee499e server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:14:16.576 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:16.573+0000 7f8b37fff640 1 --2- 172.21.15.49:0/3040591266 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f8b280a2af0 0x7f8b280a4fb0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f8b38050ad0 tx=0x7f8b38088000 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:14:16.576 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:16.573+0000 7f8b21ffb640 1 -- 172.21.15.49:0/1634240119 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b38089070 con 0x56318080a0d0 2024-03-18T23:14:16.576 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:16.574+0000 7f8b21ffb640 1 -- 172.21.15.49:0/1634240119 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8b3807e040 con 0x56318080a0d0 2024-03-18T23:14:16.576 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:16.574+0000 7f8b21ffb640 1 -- 172.21.15.49:0/1634240119 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b3807d050 con 0x56318080a0d0 2024-03-18T23:14:16.579 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:16.574+0000 7f8b40784740 1 -- 172.21.15.49:0/1634240119 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56318080a0d0 msgr2=0x56318080a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:14:16.579 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:16.574+0000 7f8b40784740 1 --2- 172.21.15.49:0/1634240119 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56318080a0d0 0x56318080a4b0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f8b3804f850 tx=0x7f8b38082660 comp rx=0 tx=0).stop 2024-03-18T23:14:16.579 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:16.575+0000 7f8b40784740 1 -- 172.21.15.49:0/1634240119 shutdown_connections 2024-03-18T23:14:16.579 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:16.575+0000 7f8b40784740 1 --2- 172.21.15.49:0/1634240119 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5631808137c0 0x563180815bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:16.579 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:16.575+0000 7f8b40784740 1 --2- 172.21.15.49:0/1634240119 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56318080a9f0 0x563180813280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:16.579 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:16.575+0000 7f8b40784740 1 --2- 172.21.15.49:0/1634240119 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56318080a0d0 0x56318080a4b0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:16.580 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:16.575+0000 7f8b40784740 1 -- 172.21.15.49:0/1634240119 >> 172.21.15.49:0/1634240119 conn(0x5631808049f0 msgr2=0x563180804dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:14:16.580 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:16.575+0000 7f8b40784740 1 -- 172.21.15.49:0/1634240119 shutdown_connections 2024-03-18T23:14:16.580 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:16.575+0000 7f8b40784740 1 -- 172.21.15.49:0/1634240119 wait complete. 2024-03-18T23:14:16.580 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:16.575+0000 7f8b40784740 1 -- start start 2024-03-18T23:14:16.580 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:16.576+0000 7f8b40784740 1 Processor -- start 2024-03-18T23:14:16.580 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:16.576+0000 7f8b40784740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56318080a0d0 0x563180813100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:14:16.580 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:16.576+0000 7f8b40784740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56318080a9f0 0x563180810490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:14:16.580 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:16.576+0000 7f8b40784740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5631808137c0 0x5631808109d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:14:16.580 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:16.576+0000 7f8b40784740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x563180816390 con 0x56318080a0d0 2024-03-18T23:14:16.580 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:16.576+0000 7f8b40784740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x563180729db0 con 0x56318080a9f0 2024-03-18T23:14:16.580 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:16.576+0000 7f8b40784740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x563180816510 con 0x5631808137c0 2024-03-18T23:14:16.580 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:16.576+0000 7f8b40784740 10 client.0 _ll_register_callbacks cb 0x56318072ee70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T23:14:16.580 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:16.577+0000 7f8b377fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56318080a0d0 0x563180813100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:14:16.581 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:16.577+0000 7f8b377fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56318080a0d0 0x563180813100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53910/0 (socket says 172.21.15.49:53910) 2024-03-18T23:14:16.581 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:16.577+0000 7f8b377fe640 1 -- 172.21.15.49:0/750300517 learned_addr learned my addr 172.21.15.49:0/750300517 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:14:16.581 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:16.577+0000 7f8b37fff640 1 --2- 172.21.15.49:0/750300517 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5631808137c0 0x5631808109d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:14:16.581 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:16.577+0000 7f8b36ffd640 1 --2- 172.21.15.49:0/750300517 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56318080a9f0 0x563180810490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:14:16.581 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:16.577+0000 7f8b37fff640 1 -- 172.21.15.49:0/750300517 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56318080a9f0 msgr2=0x563180810490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:14:16.581 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:16.577+0000 7f8b37fff640 1 --2- 172.21.15.49:0/750300517 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56318080a9f0 0x563180810490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:16.581 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:16.577+0000 7f8b37fff640 1 -- 172.21.15.49:0/750300517 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56318080a0d0 msgr2=0x563180813100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:14:16.581 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:16.577+0000 7f8b37fff640 1 --2- 172.21.15.49:0/750300517 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56318080a0d0 0x563180813100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:16.581 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:16.577+0000 7f8b37fff640 1 -- 172.21.15.49:0/750300517 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563180728c60 con 0x5631808137c0 2024-03-18T23:14:16.581 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:16.577+0000 7f8b37fff640 1 --2- 172.21.15.49:0/750300517 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5631808137c0 0x5631808109d0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f8b38050330 tx=0x7f8b38068100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1f3d72d97988e111 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:14:16.581 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:16.577+0000 7f8b16ffd640 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-03-18T23:14:16.581 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:16.577+0000 7f8b16ffd640 1 -- 172.21.15.49:0/750300517 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b38089050 con 0x5631808137c0 2024-03-18T23:14:16.581 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:16.578+0000 7f8b16ffd640 1 -- 172.21.15.49:0/750300517 <== mon.1 v2:172.21.15.175:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8512+0+0 (secure 0 0 0) 0x7f8b38050470 con 0x5631808137c0 2024-03-18T23:14:16.582 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:16.578+0000 7f8b40784740 1 -- 172.21.15.49:0/750300517 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8b3807e040 con 0x5631808137c0 2024-03-18T23:14:16.582 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:16.578+0000 7f8b16ffd640 1 -- 172.21.15.49:0/750300517 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f8b38074a60 con 0x5631808137c0 2024-03-18T23:14:16.582 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:16.578+0000 7f8b40784740 10 client.25747 fetch_fsmap learned FSMap version 32 2024-03-18T23:14:16.582 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:16.578+0000 7f8b40784740 1 -- 172.21.15.49:0/750300517 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x563180482210 con 0x5631808137c0 2024-03-18T23:14:16.582 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:16.578+0000 7f8b16ffd640 1 -- 172.21.15.49:0/750300517 <== mon.1 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8b3807e040 con 0x5631808137c0 2024-03-18T23:14:16.582 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:16.578+0000 7f8b16ffd640 1 -- 172.21.15.49:0/750300517 <== mon.1 v2:172.21.15.175:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f8b38097750 con 0x5631808137c0 2024-03-18T23:14:16.582 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:16.578+0000 7f8b40784740 10 client.25747 fetch_fsmap finished waiting for FSMap version 32 2024-03-18T23:14:16.582 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:16.578+0000 7f8b40784740 10 client.25747 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T23:14:16.582 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:16.578+0000 7f8b40784740 20 client.25747 populate_metadata read hostname 'smithi049' 2024-03-18T23:14:16.582 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:16.578+0000 7f8b40784740 1 --2- 172.21.15.49:0/750300517 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x563180823580 0x563180825970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:14:16.582 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:16.578+0000 7f8b40784740 4 client.25747 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T23:14:16.582 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:16.578+0000 7f8b40784740 1 -- 172.21.15.49:0/750300517 --> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56318077b120 con 0x563180823580 2024-03-18T23:14:16.583 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:16.581+0000 7f8b377fe640 1 --2- 172.21.15.49:0/750300517 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x563180823580 0x563180825970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:14:16.583 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:16.581+0000 7f8b377fe640 1 --2- 172.21.15.49:0/750300517 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x563180823580 0x563180825970 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2bf69291624782f4 server_cookie=dd007dbaa27e7c3d in_seq=0 out_seq=0 2024-03-18T23:14:16.583 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:16.582+0000 7f8b16ffd640 10 client.25747 ms_handle_connect on v2:172.21.15.139:6838/4280128209 2024-03-18T23:14:16.585 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:16.584+0000 7f8b16ffd640 1 -- 172.21.15.49:0/750300517 <== mds.0 v2:172.21.15.139:6838/4280128209 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x56318077b120 con 0x563180823580 2024-03-18T23:14:16.585 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:16.584+0000 7f8b16ffd640 10 client.25747 handle_command_reply: tid=0 2024-03-18T23:14:16.585 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:16.584+0000 7f8b40784740 1 client.25747 shutdown 2024-03-18T23:14:16.585 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:16.584+0000 7f8b40784740 1 -- 172.21.15.49:0/750300517 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5631808137c0 msgr2=0x5631808109d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:14:16.585 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:16.584+0000 7f8b40784740 1 --2- 172.21.15.49:0/750300517 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5631808137c0 0x5631808109d0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f8b38050330 tx=0x7f8b38068100 comp rx=0 tx=0).stop 2024-03-18T23:14:16.586 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:16.585+0000 7f8b40784740 1 -- 172.21.15.49:0/750300517 shutdown_connections 2024-03-18T23:14:16.586 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:16.585+0000 7f8b40784740 1 --2- 172.21.15.49:0/750300517 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x563180823580 0x563180825970 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:16.586 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:16.585+0000 7f8b40784740 1 --2- 172.21.15.49:0/750300517 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5631808137c0 0x5631808109d0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:16.586 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:16.585+0000 7f8b40784740 1 --2- 172.21.15.49:0/750300517 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56318080a9f0 0x563180810490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:16.586 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:16.585+0000 7f8b40784740 1 --2- 172.21.15.49:0/750300517 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56318080a0d0 0x563180813100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:16.586 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:16.585+0000 7f8b40784740 1 -- 172.21.15.49:0/750300517 >> 172.21.15.49:0/750300517 conn(0x5631808049f0 msgr2=0x563180806740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:14:16.586 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:16.585+0000 7f8b16ffd640 0 client.25747 ms_handle_reset on v2:172.21.15.139:6838/4280128209 2024-03-18T23:14:16.586 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:16.585+0000 7f8b40784740 1 -- 172.21.15.49:0/750300517 shutdown_connections 2024-03-18T23:14:16.586 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:16.585+0000 7f8b40784740 1 -- 172.21.15.49:0/750300517 wait complete. 2024-03-18T23:14:16.586 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:16.585+0000 7f8b40784740 20 client.25747 trim_cache size 0 max 16384 2024-03-18T23:14:16.608 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:16.606+0000 7f8b40784740 1 Processor -- start 2024-03-18T23:14:16.608 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:16.606+0000 7f8b40784740 1 -- start start 2024-03-18T23:14:16.608 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:16.606+0000 7f8b40784740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56318080a0d0 0x56318081f130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:14:16.608 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:16.606+0000 7f8b40784740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56318080a9f0 0x56318082b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:14:16.608 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:16.606+0000 7f8b40784740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5631808137c0 0x563180812bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:14:16.608 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:16.606+0000 7f8b40784740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f8b30150cf0 con 0x56318080a9f0 2024-03-18T23:14:16.608 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:16.606+0000 7f8b40784740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f8b30152830 con 0x56318080a0d0 2024-03-18T23:14:16.608 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:16.606+0000 7f8b40784740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f8b301513c0 con 0x5631808137c0 2024-03-18T23:14:16.608 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:16.607+0000 7f8b36ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56318080a9f0 0x56318082b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:14:16.608 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:16.607+0000 7f8b36ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56318080a9f0 0x56318082b0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53918/0 (socket says 172.21.15.49:53918) 2024-03-18T23:14:16.609 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:16.607+0000 7f8b36ffd640 1 -- 172.21.15.49:0/670896441 learned_addr learned my addr 172.21.15.49:0/670896441 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:14:16.609 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:16.607+0000 7f8b377fe640 1 --2- 172.21.15.49:0/670896441 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56318080a0d0 0x56318081f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:14:16.609 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:16.607+0000 7f8b37fff640 1 --2- 172.21.15.49:0/670896441 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5631808137c0 0x563180812bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:14:16.609 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:16.607+0000 7f8b36ffd640 1 -- 172.21.15.49:0/670896441 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5631808137c0 msgr2=0x563180812bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:14:16.609 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:16.607+0000 7f8b36ffd640 1 --2- 172.21.15.49:0/670896441 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5631808137c0 0x563180812bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:16.609 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:16.607+0000 7f8b36ffd640 1 -- 172.21.15.49:0/670896441 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56318080a0d0 msgr2=0x56318081f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:14:16.609 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:16.607+0000 7f8b36ffd640 1 --2- 172.21.15.49:0/670896441 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56318080a0d0 0x56318081f130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:16.609 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:16.607+0000 7f8b36ffd640 1 -- 172.21.15.49:0/670896441 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56318054e620 con 0x56318080a9f0 2024-03-18T23:14:16.609 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:16.607+0000 7f8b37fff640 1 --2- 172.21.15.49:0/670896441 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5631808137c0 0x563180812bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T23:14:16.609 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:16.607+0000 7f8b36ffd640 1 --2- 172.21.15.49:0/670896441 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56318080a9f0 0x56318082b0e0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x563180805b80 tx=0x7f8b2400bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1d5060f69b07295 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:14:16.609 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:16.607+0000 7f8b217fa640 1 -- 172.21.15.49:0/670896441 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b2401c070 con 0x56318080a9f0 2024-03-18T23:14:16.609 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:16.607+0000 7f8b217fa640 1 -- 172.21.15.49:0/670896441 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8b240035f0 con 0x56318080a9f0 2024-03-18T23:14:16.609 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:16.607+0000 7f8b217fa640 1 -- 172.21.15.49:0/670896441 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b240176a0 con 0x56318080a9f0 2024-03-18T23:14:16.613 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:16.608+0000 7f8b40784740 1 -- 172.21.15.49:0/670896441 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56318080a9f0 msgr2=0x56318082b0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:14:16.613 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:16.608+0000 7f8b40784740 1 --2- 172.21.15.49:0/670896441 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56318080a9f0 0x56318082b0e0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x563180805b80 tx=0x7f8b2400bea0 comp rx=0 tx=0).stop 2024-03-18T23:14:16.613 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:16.608+0000 7f8b40784740 1 -- 172.21.15.49:0/670896441 shutdown_connections 2024-03-18T23:14:16.613 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:16.608+0000 7f8b40784740 1 --2- 172.21.15.49:0/670896441 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5631808137c0 0x563180812bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:16.613 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:16.608+0000 7f8b40784740 1 --2- 172.21.15.49:0/670896441 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56318080a9f0 0x56318082b0e0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:16.613 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:16.608+0000 7f8b40784740 1 --2- 172.21.15.49:0/670896441 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56318080a0d0 0x56318081f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:16.613 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:16.608+0000 7f8b40784740 1 -- 172.21.15.49:0/670896441 >> 172.21.15.49:0/670896441 conn(0x5631808049f0 msgr2=0x563180803ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:14:16.613 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:16.608+0000 7f8b40784740 1 -- 172.21.15.49:0/670896441 shutdown_connections 2024-03-18T23:14:16.613 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:16.608+0000 7f8b40784740 1 -- 172.21.15.49:0/670896441 wait complete. 2024-03-18T23:14:16.613 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:16.609+0000 7f8b40784740 1 -- start start 2024-03-18T23:14:16.614 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:16.609+0000 7f8b40784740 1 Processor -- start 2024-03-18T23:14:16.614 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:16.609+0000 7f8b40784740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56318080a0d0 0x56318081bab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:14:16.614 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:16.609+0000 7f8b40784740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56318080a9f0 0x563180816e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:14:16.614 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:16.609+0000 7f8b40784740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5631808137c0 0x5631808173c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:14:16.614 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:16.609+0000 7f8b40784740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f8b30152160 con 0x56318080a9f0 2024-03-18T23:14:16.614 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:16.609+0000 7f8b40784740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f8b30151a90 con 0x56318080a0d0 2024-03-18T23:14:16.614 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:16.609+0000 7f8b40784740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f8b3014ecb0 con 0x5631808137c0 2024-03-18T23:14:16.614 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:16.609+0000 7f8b40784740 10 client.0 _ll_register_callbacks cb 0x7f8b2c0b9120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T23:14:16.614 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:16.610+0000 7f8b36ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56318080a9f0 0x563180816e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:14:16.614 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:16.610+0000 7f8b36ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56318080a9f0 0x563180816e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53930/0 (socket says 172.21.15.49:53930) 2024-03-18T23:14:16.614 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:16.610+0000 7f8b36ffd640 1 -- 172.21.15.49:0/1371032836 learned_addr learned my addr 172.21.15.49:0/1371032836 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:14:16.614 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:16.610+0000 7f8b37fff640 1 --2- 172.21.15.49:0/1371032836 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5631808137c0 0x5631808173c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:14:16.614 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:16.610+0000 7f8b377fe640 1 --2- 172.21.15.49:0/1371032836 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56318080a0d0 0x56318081bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:14:16.614 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:16.610+0000 7f8b36ffd640 1 -- 172.21.15.49:0/1371032836 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5631808137c0 msgr2=0x5631808173c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:14:16.615 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:16.610+0000 7f8b36ffd640 1 --2- 172.21.15.49:0/1371032836 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5631808137c0 0x5631808173c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:16.615 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:16.610+0000 7f8b36ffd640 1 -- 172.21.15.49:0/1371032836 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56318080a0d0 msgr2=0x56318081bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:14:16.615 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:16.610+0000 7f8b36ffd640 1 --2- 172.21.15.49:0/1371032836 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56318080a0d0 0x56318081bab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:16.615 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:16.610+0000 7f8b36ffd640 1 -- 172.21.15.49:0/1371032836 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563180817bf0 con 0x56318080a9f0 2024-03-18T23:14:16.615 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:16.610+0000 7f8b36ffd640 1 --2- 172.21.15.49:0/1371032836 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56318080a9f0 0x563180816e20 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f8b2401a040 tx=0x7f8b24003a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fe798115bc6703 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:14:16.615 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:16.610+0000 7f8b16ffd640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-18T23:14:16.615 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:16.610+0000 7f8b16ffd640 1 -- 172.21.15.49:0/1371032836 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b24026070 con 0x56318080a9f0 2024-03-18T23:14:16.615 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:16.610+0000 7f8b16ffd640 1 -- 172.21.15.49:0/1371032836 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8512+0+0 (secure 0 0 0) 0x7f8b24004460 con 0x56318080a9f0 2024-03-18T23:14:16.615 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:16.610+0000 7f8b40784740 1 -- 172.21.15.49:0/1371032836 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8b240035f0 con 0x56318080a9f0 2024-03-18T23:14:16.615 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:16.610+0000 7f8b16ffd640 1 -- 172.21.15.49:0/1371032836 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f8b2402f050 con 0x56318080a9f0 2024-03-18T23:14:16.615 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:16.610+0000 7f8b40784740 10 client.15945 fetch_fsmap learned FSMap version 32 2024-03-18T23:14:16.615 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:16.610+0000 7f8b40784740 1 -- 172.21.15.49:0/1371032836 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5631808897f0 con 0x56318080a9f0 2024-03-18T23:14:16.615 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:16.611+0000 7f8b16ffd640 1 -- 172.21.15.49:0/1371032836 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8b240035f0 con 0x56318080a9f0 2024-03-18T23:14:16.615 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:16.611+0000 7f8b16ffd640 1 -- 172.21.15.49:0/1371032836 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f8b24029020 con 0x56318080a9f0 2024-03-18T23:14:16.616 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:16.611+0000 7f8b40784740 10 client.15945 fetch_fsmap finished waiting for FSMap version 32 2024-03-18T23:14:16.616 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:16.611+0000 7f8b40784740 10 client.15945 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T23:14:16.616 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:16.611+0000 7f8b40784740 20 client.15945 populate_metadata read hostname 'smithi049' 2024-03-18T23:14:16.616 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:16.611+0000 7f8b40784740 1 --2- 172.21.15.49:0/1371032836 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x563180889e60 0x56318088c280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:14:16.616 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:16.611+0000 7f8b40784740 4 client.15945 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T23:14:16.616 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:16.611+0000 7f8b40784740 1 -- 172.21.15.49:0/1371032836 --> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563180775470 con 0x563180889e60 2024-03-18T23:14:16.616 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:16.611+0000 7f8b377fe640 1 --2- 172.21.15.49:0/1371032836 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x563180889e60 0x56318088c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:14:16.616 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:16.612+0000 7f8b16ffd640 10 client.15945 ms_handle_connect on v2:172.21.15.139:6838/4280128209 2024-03-18T23:14:16.616 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:16.612+0000 7f8b377fe640 1 --2- 172.21.15.49:0/1371032836 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x563180889e60 0x56318088c280 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b92cfe810d405f6c server_cookie=b525172ed7988796 in_seq=0 out_seq=0 2024-03-18T23:14:16.773 INFO:tasks.workunit.client.0.smithi049.stdout: 1 650691 10.18 MB/sec execute 198 sec latency 163.877 ms 2024-03-18T23:14:16.870 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:16.869+0000 7f8b16ffd640 1 -- 172.21.15.49:0/1371032836 <== mds.0 v2:172.21.15.139:6838/4280128209 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x563180775470 con 0x563180889e60 2024-03-18T23:14:16.870 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:16.869+0000 7f8b16ffd640 10 client.15945 handle_command_reply: tid=0 2024-03-18T23:14:16.871 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:16.869+0000 7f8b40784740 1 client.15945 shutdown 2024-03-18T23:14:16.871 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:16.869+0000 7f8b40784740 1 -- 172.21.15.49:0/1371032836 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56318080a9f0 msgr2=0x563180816e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:14:16.871 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:16.869+0000 7f8b40784740 1 --2- 172.21.15.49:0/1371032836 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56318080a9f0 0x563180816e20 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f8b2401a040 tx=0x7f8b24003a30 comp rx=0 tx=0).stop 2024-03-18T23:14:16.871 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:16.869+0000 7f8b40784740 1 -- 172.21.15.49:0/1371032836 shutdown_connections 2024-03-18T23:14:16.871 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:16.869+0000 7f8b40784740 1 --2- 172.21.15.49:0/1371032836 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x563180889e60 0x56318088c280 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:16.871 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:16.869+0000 7f8b16ffd640 0 client.15945 ms_handle_reset on v2:172.21.15.139:6838/4280128209 2024-03-18T23:14:16.871 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:16.869+0000 7f8b40784740 1 --2- 172.21.15.49:0/1371032836 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5631808137c0 0x5631808173c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:16.871 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:16.869+0000 7f8b40784740 1 --2- 172.21.15.49:0/1371032836 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56318080a9f0 0x563180816e20 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:16.871 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:16.869+0000 7f8b40784740 1 --2- 172.21.15.49:0/1371032836 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56318080a0d0 0x56318081bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:16.871 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:16.869+0000 7f8b40784740 1 -- 172.21.15.49:0/1371032836 >> 172.21.15.49:0/1371032836 conn(0x5631808049f0 msgr2=0x563180805e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:14:16.871 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:16.869+0000 7f8b40784740 1 -- 172.21.15.49:0/1371032836 shutdown_connections 2024-03-18T23:14:16.871 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:16.869+0000 7f8b40784740 1 -- 172.21.15.49:0/1371032836 wait complete. 2024-03-18T23:14:16.871 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:16.870+0000 7f8b40784740 20 client.15945 trim_cache size 0 max 16384 2024-03-18T23:14:16.872 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:{ 2024-03-18T23:14:16.872 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "message": "", 2024-03-18T23:14:16.872 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "return_code": 0 2024-03-18T23:14:16.872 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:} 2024-03-18T23:14:16.873 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:16.870+0000 7f8b3c85e640 1 -- 172.21.15.49:0/3040591266 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f8b280a2af0 msgr2=0x7f8b280a4fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:14:16.873 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:16.870+0000 7f8b3c85e640 1 --2- 172.21.15.49:0/3040591266 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f8b280a2af0 0x7f8b280a4fb0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f8b38050ad0 tx=0x7f8b38088000 comp rx=0 tx=0).stop 2024-03-18T23:14:16.873 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:16.870+0000 7f8b3c85e640 1 -- 172.21.15.49:0/3040591266 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8b30094e50 msgr2=0x7f8b300a0310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:14:16.873 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:16.870+0000 7f8b3c85e640 1 --2- 172.21.15.49:0/3040591266 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8b30094e50 0x7f8b300a0310 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f8b2c009850 tx=0x7f8b2c02ce00 comp rx=0 tx=0).stop 2024-03-18T23:14:16.873 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:16.870+0000 7f8b3c85e640 1 -- 172.21.15.49:0/3040591266 shutdown_connections 2024-03-18T23:14:16.873 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:16.870+0000 7f8b3c85e640 1 --2- 172.21.15.49:0/3040591266 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f8b280a2af0 0x7f8b280a4fb0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:16.873 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:16.870+0000 7f8b3c85e640 1 --2- 172.21.15.49:0/3040591266 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8b300a0850 0x7f8b30147af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:16.873 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:16.870+0000 7f8b3c85e640 1 --2- 172.21.15.49:0/3040591266 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8b300957f0 0x7f8b300af610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:16.873 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:16.870+0000 7f8b3c85e640 1 --2- 172.21.15.49:0/3040591266 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8b30094e50 0x7f8b300a0310 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:16.873 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:16.870+0000 7f8b3c85e640 1 -- 172.21.15.49:0/3040591266 >> 172.21.15.49:0/3040591266 conn(0x7f8b3000a740 msgr2=0x7f8b300a4d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:14:16.874 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:16.871+0000 7f8b3c85e640 1 -- 172.21.15.49:0/3040591266 shutdown_connections 2024-03-18T23:14:16.874 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:16.871+0000 7f8b3c85e640 1 -- 172.21.15.49:0/3040591266 wait complete. 2024-03-18T23:14:17.367 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:14:17 smithi139 ceph-mon[67282]: pgmap v185: 97 pgs: 97 active+clean; 80 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.3 MiB/s wr, 265 op/s 2024-03-18T23:14:17.379 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:14:17 smithi049 ceph-mon[62991]: pgmap v185: 97 pgs: 97 active+clean; 80 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.3 MiB/s wr, 265 op/s 2024-03-18T23:14:17.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:14:17 smithi175 ceph-mon[67352]: pgmap v185: 97 pgs: 97 active+clean; 80 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.3 MiB/s wr, 265 op/s 2024-03-18T23:14:17.773 INFO:tasks.workunit.client.0.smithi049.stdout: 1 652370 10.17 MB/sec execute 199 sec latency 86.149 ms 2024-03-18T23:14:18.773 INFO:tasks.workunit.client.0.smithi049.stdout: 1 654221 10.18 MB/sec execute 200 sec latency 17.429 ms 2024-03-18T23:14:19.379 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:14:19 smithi049 ceph-mon[62991]: pgmap v186: 97 pgs: 97 active+clean; 86 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.7 MiB/s wr, 354 op/s 2024-03-18T23:14:19.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:14:19 smithi175 ceph-mon[67352]: pgmap v186: 97 pgs: 97 active+clean; 86 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.7 MiB/s wr, 354 op/s 2024-03-18T23:14:19.564 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:14:19 smithi139 ceph-mon[67282]: pgmap v186: 97 pgs: 97 active+clean; 86 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.7 MiB/s wr, 354 op/s 2024-03-18T23:14:19.773 INFO:tasks.workunit.client.0.smithi049.stdout: 1 656418 10.19 MB/sec execute 201 sec latency 19.591 ms 2024-03-18T23:14:20.773 INFO:tasks.workunit.client.0.smithi049.stdout: 1 658125 10.18 MB/sec execute 202 sec latency 16.345 ms 2024-03-18T23:14:21.379 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:14:21 smithi049 ceph-mon[62991]: pgmap v187: 97 pgs: 97 active+clean; 86 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.4 MiB/s wr, 165 op/s 2024-03-18T23:14:21.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:14:21 smithi175 ceph-mon[67352]: pgmap v187: 97 pgs: 97 active+clean; 86 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.4 MiB/s wr, 165 op/s 2024-03-18T23:14:21.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:14:21 smithi139 ceph-mon[67282]: pgmap v187: 97 pgs: 97 active+clean; 86 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.4 MiB/s wr, 165 op/s 2024-03-18T23:14:21.773 INFO:tasks.workunit.client.0.smithi049.stdout: 1 660156 10.18 MB/sec execute 203 sec latency 43.334 ms 2024-03-18T23:14:22.773 INFO:tasks.workunit.client.0.smithi049.stdout: 1 661747 10.17 MB/sec execute 204 sec latency 25.985 ms 2024-03-18T23:14:23.379 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:14:23 smithi049 ceph-mon[62991]: pgmap v188: 97 pgs: 97 active+clean; 76 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.8 MiB/s wr, 344 op/s 2024-03-18T23:14:23.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:14:23 smithi175 ceph-mon[67352]: pgmap v188: 97 pgs: 97 active+clean; 76 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.8 MiB/s wr, 344 op/s 2024-03-18T23:14:23.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:14:23 smithi139 ceph-mon[67282]: pgmap v188: 97 pgs: 97 active+clean; 76 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.8 MiB/s wr, 344 op/s 2024-03-18T23:14:23.773 INFO:tasks.workunit.client.0.smithi049.stdout: 1 664049 10.20 MB/sec execute 205 sec latency 20.893 ms 2024-03-18T23:14:24.773 INFO:tasks.workunit.client.0.smithi049.stdout: 1 665508 10.18 MB/sec execute 206 sec latency 18.303 ms 2024-03-18T23:14:25.379 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:14:25 smithi049 ceph-mon[62991]: pgmap v189: 97 pgs: 97 active+clean; 76 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.9 MiB/s wr, 267 op/s 2024-03-18T23:14:25.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:14:25 smithi175 ceph-mon[67352]: pgmap v189: 97 pgs: 97 active+clean; 76 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.9 MiB/s wr, 267 op/s 2024-03-18T23:14:25.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:14:25 smithi139 ceph-mon[67282]: pgmap v189: 97 pgs: 97 active+clean; 76 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.9 MiB/s wr, 267 op/s 2024-03-18T23:14:25.773 INFO:tasks.workunit.client.0.smithi049.stdout: 1 667630 10.19 MB/sec execute 207 sec latency 18.747 ms 2024-03-18T23:14:26.773 INFO:tasks.workunit.client.0.smithi049.stdout: 1 669115 10.18 MB/sec execute 208 sec latency 20.987 ms 2024-03-18T23:14:27.379 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:14:27 smithi049 ceph-mon[62991]: pgmap v190: 97 pgs: 97 active+clean; 76 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.9 MiB/s wr, 267 op/s 2024-03-18T23:14:27.379 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:14:27 smithi049 ceph-mon[62991]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:14:27.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:14:27 smithi175 ceph-mon[67352]: pgmap v190: 97 pgs: 97 active+clean; 76 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.9 MiB/s wr, 267 op/s 2024-03-18T23:14:27.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:14:27 smithi175 ceph-mon[67352]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:14:27.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:14:27 smithi139 ceph-mon[67282]: pgmap v190: 97 pgs: 97 active+clean; 76 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.9 MiB/s wr, 267 op/s 2024-03-18T23:14:27.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:14:27 smithi139 ceph-mon[67282]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:14:27.773 INFO:tasks.workunit.client.0.smithi049.stdout: 1 670893 10.17 MB/sec execute 209 sec latency 129.185 ms 2024-03-18T23:14:28.379 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:14:28 smithi049 ceph-mon[62991]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:14:28.379 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:14:28 smithi049 ceph-mon[62991]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:14:28.379 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:14:28 smithi049 ceph-mon[62991]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' 2024-03-18T23:14:28.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:14:28 smithi175 ceph-mon[67352]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:14:28.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:14:28 smithi175 ceph-mon[67352]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:14:28.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:14:28 smithi175 ceph-mon[67352]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' 2024-03-18T23:14:28.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:14:28 smithi139 ceph-mon[67282]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:14:28.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:14:28 smithi139 ceph-mon[67282]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:14:28.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:14:28 smithi139 ceph-mon[67282]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' 2024-03-18T23:14:28.773 INFO:tasks.workunit.client.0.smithi049.stdout: 1 672589 10.17 MB/sec execute 210 sec latency 22.221 ms 2024-03-18T23:14:29.379 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:14:29 smithi049 ceph-mon[62991]: pgmap v191: 97 pgs: 97 active+clean; 83 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 4.1 MiB/s wr, 354 op/s 2024-03-18T23:14:29.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:14:29 smithi175 ceph-mon[67352]: pgmap v191: 97 pgs: 97 active+clean; 83 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 4.1 MiB/s wr, 354 op/s 2024-03-18T23:14:29.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:14:29 smithi139 ceph-mon[67282]: pgmap v191: 97 pgs: 97 active+clean; 83 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 4.1 MiB/s wr, 354 op/s 2024-03-18T23:14:29.773 INFO:tasks.workunit.client.0.smithi049.stdout: 1 674874 10.18 MB/sec execute 211 sec latency 19.429 ms 2024-03-18T23:14:29.969 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:29.967+0000 7f0c96ec0640 1 -- 172.21.15.49:0/1028896745 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0c90101330 msgr2=0x7f0c9010d550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:14:29.969 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:29.967+0000 7f0c96ec0640 1 --2- 172.21.15.49:0/1028896745 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0c90101330 0x7f0c9010d550 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f0c88009ca0 tx=0x7f0c8802c0f0 comp rx=0 tx=0).stop 2024-03-18T23:14:29.969 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:29.967+0000 7f0c96ec0640 1 -- 172.21.15.49:0/1028896745 shutdown_connections 2024-03-18T23:14:29.969 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:29.967+0000 7f0c96ec0640 1 --2- 172.21.15.49:0/1028896745 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0c90101330 0x7f0c9010d550 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:29.969 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:29.967+0000 7f0c96ec0640 1 --2- 172.21.15.49:0/1028896745 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0c90100970 0x7f0c90100df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:29.970 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:29.967+0000 7f0c96ec0640 1 --2- 172.21.15.49:0/1028896745 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0c901056c0 0x7f0c90105aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:29.970 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:29.967+0000 7f0c96ec0640 1 -- 172.21.15.49:0/1028896745 >> 172.21.15.49:0/1028896745 conn(0x7f0c90077570 msgr2=0x7f0c90077980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:14:29.970 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:29.968+0000 7f0c96ec0640 1 -- 172.21.15.49:0/1028896745 shutdown_connections 2024-03-18T23:14:29.970 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:29.968+0000 7f0c96ec0640 1 -- 172.21.15.49:0/1028896745 wait complete. 2024-03-18T23:14:29.970 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:29.968+0000 7f0c96ec0640 1 Processor -- start 2024-03-18T23:14:29.970 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:29.968+0000 7f0c96ec0640 1 -- start start 2024-03-18T23:14:29.970 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:29.968+0000 7f0c96ec0640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0c90100970 0x7f0c90196fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:14:29.970 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:29.968+0000 7f0c96ec0640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0c90101330 0x7f0c90197500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:14:29.970 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:29.968+0000 7f0c96ec0640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0c901056c0 0x7f0c9019b880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:14:29.970 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:29.968+0000 7f0c96ec0640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f0c90110be0 con 0x7f0c90100970 2024-03-18T23:14:29.970 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:29.968+0000 7f0c96ec0640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f0c90110a60 con 0x7f0c90101330 2024-03-18T23:14:29.970 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:29.968+0000 7f0c96ec0640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f0c90110d60 con 0x7f0c901056c0 2024-03-18T23:14:29.970 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:29.968+0000 7f0c94c5c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0c90100970 0x7f0c90196fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:14:29.971 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:29.968+0000 7f0c94c5c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0c90100970 0x7f0c90196fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49272/0 (socket says 172.21.15.49:49272) 2024-03-18T23:14:29.971 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:29.968+0000 7f0c94c5c640 1 -- 172.21.15.49:0/4203230697 learned_addr learned my addr 172.21.15.49:0/4203230697 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:14:29.971 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:29.968+0000 7f0c87fff640 1 --2- 172.21.15.49:0/4203230697 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0c90101330 0x7f0c90197500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:14:29.971 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:29.969+0000 7f0c9545d640 1 --2- 172.21.15.49:0/4203230697 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0c901056c0 0x7f0c9019b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:14:29.971 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:29.969+0000 7f0c94c5c640 1 -- 172.21.15.49:0/4203230697 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0c901056c0 msgr2=0x7f0c9019b880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:14:29.971 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:29.969+0000 7f0c94c5c640 1 --2- 172.21.15.49:0/4203230697 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0c901056c0 0x7f0c9019b880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:29.971 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:29.969+0000 7f0c94c5c640 1 -- 172.21.15.49:0/4203230697 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0c90101330 msgr2=0x7f0c90197500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:14:29.971 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:29.969+0000 7f0c94c5c640 1 --2- 172.21.15.49:0/4203230697 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0c90101330 0x7f0c90197500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:29.971 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:29.969+0000 7f0c94c5c640 1 -- 172.21.15.49:0/4203230697 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c9019bfb0 con 0x7f0c90100970 2024-03-18T23:14:29.971 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:29.969+0000 7f0c94c5c640 1 --2- 172.21.15.49:0/4203230697 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0c90100970 0x7f0c90196fc0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f0c8c00e9f0 tx=0x7f0c8c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:14:29.971 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:29.969+0000 7f0c85ffb640 1 -- 172.21.15.49:0/4203230697 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c8c00cd80 con 0x7f0c90100970 2024-03-18T23:14:29.971 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:29.969+0000 7f0c85ffb640 1 -- 172.21.15.49:0/4203230697 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0c8c010910 con 0x7f0c90100970 2024-03-18T23:14:29.971 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:29.969+0000 7f0c96ec0640 1 -- 172.21.15.49:0/4203230697 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c9019c2a0 con 0x7f0c90100970 2024-03-18T23:14:29.971 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:29.969+0000 7f0c85ffb640 1 -- 172.21.15.49:0/4203230697 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c8c018990 con 0x7f0c90100970 2024-03-18T23:14:29.972 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:29.969+0000 7f0c96ec0640 1 -- 172.21.15.49:0/4203230697 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c901a3b30 con 0x7f0c90100970 2024-03-18T23:14:29.977 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:29.972+0000 7f0c99b84740 1 Processor -- start 2024-03-18T23:14:29.977 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:29.972+0000 7f0c85ffb640 1 -- 172.21.15.49:0/4203230697 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 25) ==== 137421+0+0 (secure 0 0 0) 0x7f0c8c010ac0 con 0x7f0c90100970 2024-03-18T23:14:29.977 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:29.972+0000 7f0c99b84740 1 -- start start 2024-03-18T23:14:29.977 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:29.972+0000 7f0c99b84740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556650116ef0 0x5566501172d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:14:29.977 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:29.972+0000 7f0c99b84740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556650117810 0x5566501200a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:14:29.977 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:29.972+0000 7f0c99b84740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5566501205e0 0x5566501229d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:14:29.977 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:29.972+0000 7f0c99b84740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55665007abe0 con 0x5566501205e0 2024-03-18T23:14:29.978 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:29.972+0000 7f0c99b84740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x556650023960 con 0x556650117810 2024-03-18T23:14:29.978 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:29.972+0000 7f0c99b84740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x556650076c80 con 0x556650116ef0 2024-03-18T23:14:29.978 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:29.973+0000 7f0c9545d640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556650116ef0 0x5566501172d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:14:29.978 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:29.973+0000 7f0c94c5c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556650117810 0x5566501200a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:14:29.978 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:29.973+0000 7f0c9545d640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556650116ef0 0x5566501172d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.49:49960/0 (socket says 172.21.15.49:49960) 2024-03-18T23:14:29.978 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:29.973+0000 7f0c9545d640 1 -- 172.21.15.49:0/1418574126 learned_addr learned my addr 172.21.15.49:0/1418574126 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:14:29.978 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:29.973+0000 7f0c87fff640 1 --2- 172.21.15.49:0/1418574126 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5566501205e0 0x5566501229d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:14:29.978 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:29.973+0000 7f0c85ffb640 1 --2- 172.21.15.49:0/4203230697 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f0c780a2a10 0x7f0c780a4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:14:29.978 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:29.973+0000 7f0c9545d640 1 -- 172.21.15.49:0/1418574126 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556650117810 msgr2=0x5566501200a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:14:29.978 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:29.973+0000 7f0c9545d640 1 --2- 172.21.15.49:0/1418574126 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556650117810 0x5566501200a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:29.978 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:29.973+0000 7f0c9545d640 1 -- 172.21.15.49:0/1418574126 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5566501205e0 msgr2=0x5566501229d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:14:29.978 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:29.973+0000 7f0c9545d640 1 --2- 172.21.15.49:0/1418574126 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5566501205e0 0x5566501229d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:29.978 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:29.973+0000 7f0c9545d640 1 -- 172.21.15.49:0/1418574126 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55665001ad20 con 0x556650116ef0 2024-03-18T23:14:29.979 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:29.973+0000 7f0c87fff640 1 --2- 172.21.15.49:0/1418574126 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5566501205e0 0x5566501229d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:14:29.979 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:29.973+0000 7f0c85ffb640 1 -- 172.21.15.49:0/4203230697 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8512+0+0 (secure 0 0 0) 0x7f0c8c0cf110 con 0x7f0c90100970 2024-03-18T23:14:29.979 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:29.973+0000 7f0c9545d640 1 --2- 172.21.15.49:0/4203230697 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f0c780a2a10 0x7f0c780a4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:14:29.979 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:29.973+0000 7f0c9545d640 1 --2- 172.21.15.49:0/1418574126 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556650116ef0 0x5566501172d0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f0c8802c700 tx=0x7f0c880098c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7ff67dd89fa8373b server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:14:29.979 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:29.973+0000 7f0c76ffd640 1 -- 172.21.15.49:0/1418574126 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c88039040 con 0x556650116ef0 2024-03-18T23:14:29.979 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:29.974+0000 7f0c9545d640 1 --2- 172.21.15.49:0/4203230697 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f0c780a2a10 0x7f0c780a4ed0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f0c8804e000 tx=0x7f0c8804b000 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:14:29.979 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:29.974+0000 7f0c76ffd640 1 -- 172.21.15.49:0/1418574126 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0c88003c30 con 0x556650116ef0 2024-03-18T23:14:29.979 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:29.974+0000 7f0c76ffd640 1 -- 172.21.15.49:0/1418574126 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c88042730 con 0x556650116ef0 2024-03-18T23:14:29.979 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:29.974+0000 7f0c99b84740 1 -- 172.21.15.49:0/1418574126 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556650116ef0 msgr2=0x5566501172d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:14:29.979 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:29.974+0000 7f0c99b84740 1 --2- 172.21.15.49:0/1418574126 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556650116ef0 0x5566501172d0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f0c8802c700 tx=0x7f0c880098c0 comp rx=0 tx=0).stop 2024-03-18T23:14:29.979 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:29.974+0000 7f0c99b84740 1 -- 172.21.15.49:0/1418574126 shutdown_connections 2024-03-18T23:14:29.979 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:29.974+0000 7f0c99b84740 1 --2- 172.21.15.49:0/1418574126 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5566501205e0 0x5566501229d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:29.979 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:29.974+0000 7f0c99b84740 1 --2- 172.21.15.49:0/1418574126 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556650117810 0x5566501200a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:29.979 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:29.974+0000 7f0c99b84740 1 --2- 172.21.15.49:0/1418574126 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556650116ef0 0x5566501172d0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:29.979 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:29.974+0000 7f0c99b84740 1 -- 172.21.15.49:0/1418574126 >> 172.21.15.49:0/1418574126 conn(0x556650111810 msgr2=0x556650111bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:14:29.980 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:29.974+0000 7f0c99b84740 1 -- 172.21.15.49:0/1418574126 shutdown_connections 2024-03-18T23:14:29.980 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:29.974+0000 7f0c99b84740 1 -- 172.21.15.49:0/1418574126 wait complete. 2024-03-18T23:14:29.980 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:29.975+0000 7f0c99b84740 1 -- start start 2024-03-18T23:14:29.980 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:29.975+0000 7f0c99b84740 1 Processor -- start 2024-03-18T23:14:29.980 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:29.975+0000 7f0c99b84740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556650116ef0 0x55665011ff40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:14:29.980 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:29.975+0000 7f0c99b84740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556650117810 0x55665011d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:14:29.980 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:29.975+0000 7f0c99b84740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5566501205e0 0x55665011d830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:14:29.980 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:29.975+0000 7f0c99b84740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x556650123210 con 0x556650117810 2024-03-18T23:14:29.980 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:29.975+0000 7f0c99b84740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x556650036db0 con 0x5566501205e0 2024-03-18T23:14:29.980 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:29.975+0000 7f0c99b84740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x556650123390 con 0x556650116ef0 2024-03-18T23:14:29.980 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:29.975+0000 7f0c99b84740 10 client.0 _ll_register_callbacks cb 0x55665003be70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T23:14:29.980 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:29.975+0000 7f0c87fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556650117810 0x55665011d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:14:29.980 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:29.975+0000 7f0c87fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556650117810 0x55665011d2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49284/0 (socket says 172.21.15.49:49284) 2024-03-18T23:14:29.980 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:29.975+0000 7f0c9545d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5566501205e0 0x55665011d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:14:29.981 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:29.975+0000 7f0c94c5c640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556650116ef0 0x55665011ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:14:29.981 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:29.975+0000 7f0c87fff640 1 -- 172.21.15.49:0/2051476430 learned_addr learned my addr 172.21.15.49:0/2051476430 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:14:29.981 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:29.976+0000 7f0c87fff640 1 -- 172.21.15.49:0/2051476430 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556650116ef0 msgr2=0x55665011ff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:14:29.981 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:29.976+0000 7f0c87fff640 1 --2- 172.21.15.49:0/2051476430 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556650116ef0 0x55665011ff40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:29.981 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:29.976+0000 7f0c87fff640 1 -- 172.21.15.49:0/2051476430 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5566501205e0 msgr2=0x55665011d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:14:29.981 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:29.976+0000 7f0c87fff640 1 --2- 172.21.15.49:0/2051476430 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5566501205e0 0x55665011d830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:29.981 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:29.976+0000 7f0c87fff640 1 -- 172.21.15.49:0/2051476430 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556650035c60 con 0x556650117810 2024-03-18T23:14:29.981 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:29.976+0000 7f0c9545d640 1 --2- 172.21.15.49:0/2051476430 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5566501205e0 0x55665011d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:14:29.981 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:29.976+0000 7f0c87fff640 1 --2- 172.21.15.49:0/2051476430 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556650117810 0x55665011d2d0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f0c80012700 tx=0x7f0c80012bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55bc45802001096 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:14:29.981 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:29.976+0000 7f0c6ffff640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-18T23:14:29.981 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:29.976+0000 7f0c6ffff640 1 -- 172.21.15.49:0/2051476430 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c8000f920 con 0x556650117810 2024-03-18T23:14:29.981 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:29.976+0000 7f0c6ffff640 1 -- 172.21.15.49:0/2051476430 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8512+0+0 (secure 0 0 0) 0x7f0c8000fac0 con 0x556650117810 2024-03-18T23:14:29.981 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:29.976+0000 7f0c99b84740 1 -- 172.21.15.49:0/2051476430 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0c88003c30 con 0x556650117810 2024-03-18T23:14:29.981 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:29.976+0000 7f0c6ffff640 1 -- 172.21.15.49:0/2051476430 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f0c8000fcc0 con 0x556650117810 2024-03-18T23:14:29.982 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:29.976+0000 7f0c99b84740 10 client.15957 fetch_fsmap learned FSMap version 32 2024-03-18T23:14:29.982 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:29.976+0000 7f0c99b84740 1 -- 172.21.15.49:0/2051476430 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55664fd8f210 con 0x556650117810 2024-03-18T23:14:29.982 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:29.976+0000 7f0c6ffff640 1 -- 172.21.15.49:0/2051476430 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0c88003c30 con 0x556650117810 2024-03-18T23:14:29.982 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:29.977+0000 7f0c6ffff640 1 -- 172.21.15.49:0/2051476430 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f0c8001c420 con 0x556650117810 2024-03-18T23:14:29.982 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:29.977+0000 7f0c99b84740 10 client.15957 fetch_fsmap finished waiting for FSMap version 32 2024-03-18T23:14:29.982 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:29.977+0000 7f0c99b84740 10 client.15957 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T23:14:29.982 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:29.977+0000 7f0c99b84740 20 client.15957 populate_metadata read hostname 'smithi049' 2024-03-18T23:14:29.982 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:29.977+0000 7f0c99b84740 1 --2- 172.21.15.49:0/2051476430 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x556650130570 0x556650132960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:14:29.982 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:29.977+0000 7f0c99b84740 4 client.15957 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T23:14:29.982 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:29.977+0000 7f0c99b84740 1 -- 172.21.15.49:0/2051476430 --> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556650088120 con 0x556650130570 2024-03-18T23:14:29.982 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:29.977+0000 7f0c94c5c640 1 --2- 172.21.15.49:0/2051476430 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x556650130570 0x556650132960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:14:29.982 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:29.977+0000 7f0c94c5c640 1 --2- 172.21.15.49:0/2051476430 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x556650130570 0x556650132960 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=965d62e4c09c2afa server_cookie=b65baf412131b408 in_seq=0 out_seq=0 2024-03-18T23:14:29.982 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:29.977+0000 7f0c6ffff640 10 client.15957 ms_handle_connect on v2:172.21.15.139:6838/4280128209 2024-03-18T23:14:29.983 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:29.978+0000 7f0c6ffff640 1 -- 172.21.15.49:0/2051476430 <== mds.0 v2:172.21.15.139:6838/4280128209 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x556650088120 con 0x556650130570 2024-03-18T23:14:29.984 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:29.978+0000 7f0c6ffff640 10 client.15957 handle_command_reply: tid=0 2024-03-18T23:14:29.984 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:29.978+0000 7f0c99b84740 1 client.15957 shutdown 2024-03-18T23:14:29.984 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:29.978+0000 7f0c99b84740 1 -- 172.21.15.49:0/2051476430 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556650117810 msgr2=0x55665011d2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:14:29.984 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:29.978+0000 7f0c99b84740 1 --2- 172.21.15.49:0/2051476430 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556650117810 0x55665011d2d0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f0c80012700 tx=0x7f0c80012bc0 comp rx=0 tx=0).stop 2024-03-18T23:14:29.984 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:29.978+0000 7f0c99b84740 1 -- 172.21.15.49:0/2051476430 shutdown_connections 2024-03-18T23:14:29.984 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:29.978+0000 7f0c99b84740 1 --2- 172.21.15.49:0/2051476430 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x556650130570 0x556650132960 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:29.984 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:29.978+0000 7f0c99b84740 1 --2- 172.21.15.49:0/2051476430 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5566501205e0 0x55665011d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:29.984 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:29.978+0000 7f0c99b84740 1 --2- 172.21.15.49:0/2051476430 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556650117810 0x55665011d2d0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:29.984 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:29.978+0000 7f0c99b84740 1 --2- 172.21.15.49:0/2051476430 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556650116ef0 0x55665011ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:29.984 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:29.978+0000 7f0c6ffff640 0 client.15957 ms_handle_reset on v2:172.21.15.139:6838/4280128209 2024-03-18T23:14:29.984 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:29.978+0000 7f0c99b84740 1 -- 172.21.15.49:0/2051476430 >> 172.21.15.49:0/2051476430 conn(0x556650111810 msgr2=0x556650110ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:14:29.984 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:29.978+0000 7f0c99b84740 1 -- 172.21.15.49:0/2051476430 shutdown_connections 2024-03-18T23:14:29.984 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:29.979+0000 7f0c99b84740 1 -- 172.21.15.49:0/2051476430 wait complete. 2024-03-18T23:14:29.984 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:29.979+0000 7f0c99b84740 20 client.15957 trim_cache size 0 max 16384 2024-03-18T23:14:29.994 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:29.994+0000 7f0c99b84740 1 Processor -- start 2024-03-18T23:14:29.994 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:29.994+0000 7f0c99b84740 1 -- start start 2024-03-18T23:14:29.994 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:29.994+0000 7f0c99b84740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556650116ef0 0x55665012bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:14:29.995 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:29.994+0000 7f0c99b84740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556650117810 0x556650138250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:14:29.995 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:29.994+0000 7f0c99b84740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5566501205e0 0x55665011fa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:14:29.995 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:29.994+0000 7f0c99b84740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f0c901ab200 con 0x5566501205e0 2024-03-18T23:14:29.995 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:29.994+0000 7f0c99b84740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f0c901acd40 con 0x556650117810 2024-03-18T23:14:29.995 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:29.994+0000 7f0c99b84740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f0c901ab8d0 con 0x556650116ef0 2024-03-18T23:14:29.995 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:29.994+0000 7f0c94c5c640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556650116ef0 0x55665012bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:14:29.995 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:29.994+0000 7f0c9545d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5566501205e0 0x55665011fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:14:29.995 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:29.994+0000 7f0c87fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556650117810 0x556650138250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:14:29.995 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:29.994+0000 7f0c9545d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5566501205e0 0x55665011fa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49286/0 (socket says 172.21.15.49:49286) 2024-03-18T23:14:29.995 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:29.994+0000 7f0c94c5c640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556650116ef0 0x55665012bfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.49:49974/0 (socket says 172.21.15.49:49974) 2024-03-18T23:14:29.995 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:29.994+0000 7f0c9545d640 1 -- 172.21.15.49:0/664889528 learned_addr learned my addr 172.21.15.49:0/664889528 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:14:29.995 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:29.994+0000 7f0c9545d640 1 -- 172.21.15.49:0/664889528 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556650116ef0 msgr2=0x55665012bfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:14:29.995 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:29.994+0000 7f0c9545d640 1 --2- 172.21.15.49:0/664889528 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556650116ef0 0x55665012bfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:29.995 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:29.994+0000 7f0c9545d640 1 -- 172.21.15.49:0/664889528 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556650117810 msgr2=0x556650138250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:14:29.996 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:29.995+0000 7f0c9545d640 1 --2- 172.21.15.49:0/664889528 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556650117810 0x556650138250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:29.996 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:29.995+0000 7f0c9545d640 1 -- 172.21.15.49:0/664889528 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55664fe5b620 con 0x5566501205e0 2024-03-18T23:14:29.996 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:29.995+0000 7f0c87fff640 1 --2- 172.21.15.49:0/664889528 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556650117810 0x556650138250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:14:29.996 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:29.995+0000 7f0c9545d640 1 --2- 172.21.15.49:0/664889528 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5566501205e0 0x55665011fa00 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f0c88005e30 tx=0x7f0c880048c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e576b4c812f97a server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:14:29.996 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:29.995+0000 7f0c767fc640 1 -- 172.21.15.49:0/664889528 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c88032c90 con 0x5566501205e0 2024-03-18T23:14:29.996 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:29.995+0000 7f0c767fc640 1 -- 172.21.15.49:0/664889528 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0c88004520 con 0x5566501205e0 2024-03-18T23:14:29.996 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:29.995+0000 7f0c767fc640 1 -- 172.21.15.49:0/664889528 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c88042800 con 0x5566501205e0 2024-03-18T23:14:29.999 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:29.996+0000 7f0c99b84740 1 -- 172.21.15.49:0/664889528 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5566501205e0 msgr2=0x55665011fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:14:29.999 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:29.996+0000 7f0c99b84740 1 --2- 172.21.15.49:0/664889528 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5566501205e0 0x55665011fa00 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f0c88005e30 tx=0x7f0c880048c0 comp rx=0 tx=0).stop 2024-03-18T23:14:29.999 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:29.996+0000 7f0c99b84740 1 -- 172.21.15.49:0/664889528 shutdown_connections 2024-03-18T23:14:29.999 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:29.996+0000 7f0c99b84740 1 --2- 172.21.15.49:0/664889528 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5566501205e0 0x55665011fa00 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:29.999 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:29.996+0000 7f0c99b84740 1 --2- 172.21.15.49:0/664889528 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556650117810 0x556650138250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:30.000 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:29.996+0000 7f0c99b84740 1 --2- 172.21.15.49:0/664889528 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556650116ef0 0x55665012bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:30.000 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:29.996+0000 7f0c99b84740 1 -- 172.21.15.49:0/664889528 >> 172.21.15.49:0/664889528 conn(0x556650111810 msgr2=0x556650110ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:14:30.000 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:29.996+0000 7f0c99b84740 1 -- 172.21.15.49:0/664889528 shutdown_connections 2024-03-18T23:14:30.000 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:29.996+0000 7f0c99b84740 1 -- 172.21.15.49:0/664889528 wait complete. 2024-03-18T23:14:30.000 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:29.996+0000 7f0c99b84740 1 -- start start 2024-03-18T23:14:30.000 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:29.996+0000 7f0c99b84740 1 Processor -- start 2024-03-18T23:14:30.000 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:29.997+0000 7f0c99b84740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556650116ef0 0x5566501287d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:14:30.000 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:29.997+0000 7f0c99b84740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556650117810 0x556650123bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:14:30.000 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:29.997+0000 7f0c99b84740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5566501205e0 0x556650124100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:14:30.000 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:29.997+0000 7f0c99b84740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f0c901ac670 con 0x5566501205e0 2024-03-18T23:14:30.000 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:29.997+0000 7f0c99b84740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f0c901abfa0 con 0x556650116ef0 2024-03-18T23:14:30.000 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:29.997+0000 7f0c99b84740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f0c901a91c0 con 0x556650117810 2024-03-18T23:14:30.000 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:29.997+0000 7f0c99b84740 10 client.0 _ll_register_callbacks cb 0x7f0c80009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T23:14:30.001 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:29.997+0000 7f0c94c5c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556650116ef0 0x5566501287d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:14:30.001 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:29.997+0000 7f0c9545d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5566501205e0 0x556650124100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:14:30.001 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:29.997+0000 7f0c94c5c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556650116ef0 0x5566501287d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.49:35238/0 (socket says 172.21.15.49:35238) 2024-03-18T23:14:30.001 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:29.997+0000 7f0c94c5c640 1 -- 172.21.15.49:0/3638833165 learned_addr learned my addr 172.21.15.49:0/3638833165 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:14:30.001 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:29.997+0000 7f0c87fff640 1 --2- 172.21.15.49:0/3638833165 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556650117810 0x556650123bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:14:30.001 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:29.997+0000 7f0c94c5c640 1 -- 172.21.15.49:0/3638833165 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556650117810 msgr2=0x556650123bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:14:30.001 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:29.997+0000 7f0c94c5c640 1 --2- 172.21.15.49:0/3638833165 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556650117810 0x556650123bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:30.001 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:29.997+0000 7f0c94c5c640 1 -- 172.21.15.49:0/3638833165 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5566501205e0 msgr2=0x556650124100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:14:30.001 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:29.997+0000 7f0c94c5c640 1 --2- 172.21.15.49:0/3638833165 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5566501205e0 0x556650124100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:30.001 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:29.997+0000 7f0c94c5c640 1 -- 172.21.15.49:0/3638833165 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5566501247f0 con 0x556650116ef0 2024-03-18T23:14:30.001 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:29.997+0000 7f0c94c5c640 1 --2- 172.21.15.49:0/3638833165 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556650116ef0 0x5566501287d0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f0c8c09b7c0 tx=0x7f0c8c086040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f45211309e889644 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:14:30.001 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:29.997+0000 7f0c6ffff640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-03-18T23:14:30.001 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:29.998+0000 7f0c6ffff640 1 -- 172.21.15.49:0/3638833165 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c8c0aa9c0 con 0x556650116ef0 2024-03-18T23:14:30.001 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:29.998+0000 7f0c6ffff640 1 -- 172.21.15.49:0/3638833165 <== mon.2 v2:172.21.15.139:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8512+0+0 (secure 0 0 0) 0x7f0c8c0860b0 con 0x556650116ef0 2024-03-18T23:14:30.001 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:29.998+0000 7f0c99b84740 1 -- 172.21.15.49:0/3638833165 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0c88004520 con 0x556650116ef0 2024-03-18T23:14:30.002 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:29.998+0000 7f0c6ffff640 1 -- 172.21.15.49:0/3638833165 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f0c8c097ad0 con 0x556650116ef0 2024-03-18T23:14:30.002 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:29.998+0000 7f0c99b84740 10 client.25781 fetch_fsmap learned FSMap version 32 2024-03-18T23:14:30.002 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:29.998+0000 7f0c99b84740 1 -- 172.21.15.49:0/3638833165 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55665019efe0 con 0x556650116ef0 2024-03-18T23:14:30.002 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:29.998+0000 7f0c6ffff640 1 -- 172.21.15.49:0/3638833165 <== mon.2 v2:172.21.15.139:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0c88004520 con 0x556650116ef0 2024-03-18T23:14:30.002 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:29.998+0000 7f0c6ffff640 1 -- 172.21.15.49:0/3638833165 <== mon.2 v2:172.21.15.139:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f0c8c0ccaf0 con 0x556650116ef0 2024-03-18T23:14:30.002 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:29.998+0000 7f0c99b84740 10 client.25781 fetch_fsmap finished waiting for FSMap version 32 2024-03-18T23:14:30.002 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:29.998+0000 7f0c99b84740 10 client.25781 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T23:14:30.002 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:29.998+0000 7f0c99b84740 20 client.25781 populate_metadata read hostname 'smithi049' 2024-03-18T23:14:30.002 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:29.998+0000 7f0c99b84740 1 --2- 172.21.15.49:0/3638833165 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x55665019f700 0x5566501a1af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:14:30.002 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:29.998+0000 7f0c99b84740 4 client.25781 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T23:14:30.002 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:29.998+0000 7f0c99b84740 1 -- 172.21.15.49:0/3638833165 --> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556650082470 con 0x55665019f700 2024-03-18T23:14:30.002 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:29.998+0000 7f0c87fff640 1 --2- 172.21.15.49:0/3638833165 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x55665019f700 0x5566501a1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:14:30.002 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:29.999+0000 7f0c6ffff640 10 client.25781 ms_handle_connect on v2:172.21.15.139:6838/4280128209 2024-03-18T23:14:30.003 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:29.999+0000 7f0c87fff640 1 --2- 172.21.15.49:0/3638833165 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x55665019f700 0x5566501a1af0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=deb5f8bf7ee75aca server_cookie=a1975e41aaff7660 in_seq=0 out_seq=0 2024-03-18T23:14:30.148 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:30.148+0000 7f0c6ffff640 1 -- 172.21.15.49:0/3638833165 <== mds.0 v2:172.21.15.139:6838/4280128209 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556650082470 con 0x55665019f700 2024-03-18T23:14:30.149 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:30.148+0000 7f0c6ffff640 10 client.25781 handle_command_reply: tid=0 2024-03-18T23:14:30.149 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:30.148+0000 7f0c99b84740 1 client.25781 shutdown 2024-03-18T23:14:30.149 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:30.148+0000 7f0c99b84740 1 -- 172.21.15.49:0/3638833165 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556650116ef0 msgr2=0x5566501287d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:14:30.149 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:30.148+0000 7f0c99b84740 1 --2- 172.21.15.49:0/3638833165 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556650116ef0 0x5566501287d0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f0c8c09b7c0 tx=0x7f0c8c086040 comp rx=0 tx=0).stop 2024-03-18T23:14:30.149 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:30.148+0000 7f0c99b84740 1 -- 172.21.15.49:0/3638833165 shutdown_connections 2024-03-18T23:14:30.149 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:30.148+0000 7f0c99b84740 1 --2- 172.21.15.49:0/3638833165 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x55665019f700 0x5566501a1af0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:30.149 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:30.148+0000 7f0c6ffff640 0 client.25781 ms_handle_reset on v2:172.21.15.139:6838/4280128209 2024-03-18T23:14:30.149 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:30.148+0000 7f0c99b84740 1 --2- 172.21.15.49:0/3638833165 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5566501205e0 0x556650124100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:30.149 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:30.148+0000 7f0c99b84740 1 --2- 172.21.15.49:0/3638833165 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556650117810 0x556650123bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:30.149 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:30.148+0000 7f0c99b84740 1 --2- 172.21.15.49:0/3638833165 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556650116ef0 0x5566501287d0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:30.149 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:30.148+0000 7f0c99b84740 1 -- 172.21.15.49:0/3638833165 >> 172.21.15.49:0/3638833165 conn(0x556650111810 msgr2=0x556650117f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:14:30.149 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:30.148+0000 7f0c99b84740 1 -- 172.21.15.49:0/3638833165 shutdown_connections 2024-03-18T23:14:30.149 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:30.148+0000 7f0c99b84740 1 -- 172.21.15.49:0/3638833165 wait complete. 2024-03-18T23:14:30.150 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:30.149+0000 7f0c99b84740 20 client.25781 trim_cache size 0 max 16384 2024-03-18T23:14:30.150 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:{ 2024-03-18T23:14:30.150 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "message": "", 2024-03-18T23:14:30.150 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "return_code": 0 2024-03-18T23:14:30.150 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:} 2024-03-18T23:14:30.151 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:30.149+0000 7f0c96ec0640 1 -- 172.21.15.49:0/4203230697 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f0c780a2a10 msgr2=0x7f0c780a4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:14:30.151 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:30.149+0000 7f0c96ec0640 1 --2- 172.21.15.49:0/4203230697 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f0c780a2a10 0x7f0c780a4ed0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f0c8804e000 tx=0x7f0c8804b000 comp rx=0 tx=0).stop 2024-03-18T23:14:30.151 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:30.149+0000 7f0c96ec0640 1 -- 172.21.15.49:0/4203230697 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0c90100970 msgr2=0x7f0c90196fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:14:30.151 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:30.149+0000 7f0c96ec0640 1 --2- 172.21.15.49:0/4203230697 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0c90100970 0x7f0c90196fc0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f0c8c00e9f0 tx=0x7f0c8c00eeb0 comp rx=0 tx=0).stop 2024-03-18T23:14:30.151 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:30.149+0000 7f0c96ec0640 1 -- 172.21.15.49:0/4203230697 shutdown_connections 2024-03-18T23:14:30.151 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:30.149+0000 7f0c96ec0640 1 --2- 172.21.15.49:0/4203230697 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f0c780a2a10 0x7f0c780a4ed0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:30.151 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:30.149+0000 7f0c96ec0640 1 --2- 172.21.15.49:0/4203230697 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0c901056c0 0x7f0c9019b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:30.151 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:30.149+0000 7f0c96ec0640 1 --2- 172.21.15.49:0/4203230697 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0c90101330 0x7f0c90197500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:30.151 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:30.149+0000 7f0c96ec0640 1 --2- 172.21.15.49:0/4203230697 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0c90100970 0x7f0c90196fc0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:30.152 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:30.150+0000 7f0c96ec0640 1 -- 172.21.15.49:0/4203230697 >> 172.21.15.49:0/4203230697 conn(0x7f0c90077570 msgr2=0x7f0c900ff5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:14:30.152 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:30.150+0000 7f0c96ec0640 1 -- 172.21.15.49:0/4203230697 shutdown_connections 2024-03-18T23:14:30.152 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:30.150+0000 7f0c96ec0640 1 -- 172.21.15.49:0/4203230697 wait complete. 2024-03-18T23:14:30.774 INFO:tasks.workunit.client.0.smithi049.stdout: 1 676201 10.16 MB/sec execute 212 sec latency 93.174 ms 2024-03-18T23:14:31.379 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:14:31 smithi049 ceph-mon[62991]: pgmap v192: 97 pgs: 97 active+clean; 83 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.7 MiB/s wr, 266 op/s 2024-03-18T23:14:31.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:14:31 smithi175 ceph-mon[67352]: pgmap v192: 97 pgs: 97 active+clean; 83 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.7 MiB/s wr, 266 op/s 2024-03-18T23:14:31.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:14:31 smithi139 ceph-mon[67282]: pgmap v192: 97 pgs: 97 active+clean; 83 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.7 MiB/s wr, 266 op/s 2024-03-18T23:14:31.773 INFO:tasks.workunit.client.0.smithi049.stdout: 1 678520 10.18 MB/sec execute 213 sec latency 18.879 ms 2024-03-18T23:14:32.774 INFO:tasks.workunit.client.0.smithi049.stdout: 1 679805 10.16 MB/sec execute 214 sec latency 52.797 ms 2024-03-18T23:14:33.379 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:14:33 smithi049 ceph-mon[62991]: pgmap v193: 97 pgs: 97 active+clean; 73 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 4.1 MiB/s wr, 448 op/s 2024-03-18T23:14:33.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:14:33 smithi175 ceph-mon[67352]: pgmap v193: 97 pgs: 97 active+clean; 73 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 4.1 MiB/s wr, 448 op/s 2024-03-18T23:14:33.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:14:33 smithi139 ceph-mon[67282]: pgmap v193: 97 pgs: 97 active+clean; 73 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 4.1 MiB/s wr, 448 op/s 2024-03-18T23:14:33.774 INFO:tasks.workunit.client.0.smithi049.stdout: 1 681973 10.17 MB/sec execute 215 sec latency 17.461 ms 2024-03-18T23:14:34.774 INFO:tasks.workunit.client.0.smithi049.stdout: 1 683438 10.15 MB/sec execute 216 sec latency 25.502 ms 2024-03-18T23:14:35.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:14:35 smithi049 ceph-mon[62991]: pgmap v194: 97 pgs: 97 active+clean; 73 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.7 MiB/s wr, 269 op/s 2024-03-18T23:14:35.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:14:35 smithi175 ceph-mon[67352]: pgmap v194: 97 pgs: 97 active+clean; 73 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.7 MiB/s wr, 269 op/s 2024-03-18T23:14:35.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:14:35 smithi139 ceph-mon[67282]: pgmap v194: 97 pgs: 97 active+clean; 73 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.7 MiB/s wr, 269 op/s 2024-03-18T23:14:35.774 INFO:tasks.workunit.client.0.smithi049.stdout: 1 685623 10.17 MB/sec execute 217 sec latency 17.851 ms 2024-03-18T23:14:36.774 INFO:tasks.workunit.client.0.smithi049.stdout: 1 686962 10.15 MB/sec execute 218 sec latency 28.441 ms 2024-03-18T23:14:37.379 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:14:37 smithi049 ceph-mon[62991]: pgmap v195: 97 pgs: 97 active+clean; 73 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.7 MiB/s wr, 269 op/s 2024-03-18T23:14:37.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:14:37 smithi175 ceph-mon[67352]: pgmap v195: 97 pgs: 97 active+clean; 73 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.7 MiB/s wr, 269 op/s 2024-03-18T23:14:37.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:14:37 smithi139 ceph-mon[67282]: pgmap v195: 97 pgs: 97 active+clean; 73 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.7 MiB/s wr, 269 op/s 2024-03-18T23:14:37.774 INFO:tasks.workunit.client.0.smithi049.stdout: 1 689055 10.16 MB/sec execute 219 sec latency 134.632 ms 2024-03-18T23:14:38.774 INFO:tasks.workunit.client.0.smithi049.stdout: 1 690588 10.14 MB/sec execute 220 sec latency 23.467 ms 2024-03-18T23:14:39.379 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:14:39 smithi049 ceph-mon[62991]: pgmap v196: 97 pgs: 97 active+clean; 79 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 3.7 MiB/s wr, 355 op/s 2024-03-18T23:14:39.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:14:39 smithi175 ceph-mon[67352]: pgmap v196: 97 pgs: 97 active+clean; 79 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 3.7 MiB/s wr, 355 op/s 2024-03-18T23:14:39.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:14:39 smithi139 ceph-mon[67282]: pgmap v196: 97 pgs: 97 active+clean; 79 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 3.7 MiB/s wr, 355 op/s 2024-03-18T23:14:39.774 INFO:tasks.workunit.client.0.smithi049.stdout: 1 692732 10.16 MB/sec execute 221 sec latency 20.587 ms 2024-03-18T23:14:40.774 INFO:tasks.workunit.client.0.smithi049.stdout: 1 694183 10.14 MB/sec execute 222 sec latency 29.834 ms 2024-03-18T23:14:41.379 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:14:41 smithi049 ceph-mon[62991]: pgmap v197: 97 pgs: 97 active+clean; 79 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.5 MiB/s wr, 268 op/s 2024-03-18T23:14:41.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:14:41 smithi175 ceph-mon[67352]: pgmap v197: 97 pgs: 97 active+clean; 79 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.5 MiB/s wr, 268 op/s 2024-03-18T23:14:41.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:14:41 smithi139 ceph-mon[67282]: pgmap v197: 97 pgs: 97 active+clean; 79 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.5 MiB/s wr, 268 op/s 2024-03-18T23:14:41.774 INFO:tasks.workunit.client.0.smithi049.stdout: 1 696352 10.15 MB/sec execute 223 sec latency 18.016 ms 2024-03-18T23:14:42.774 INFO:tasks.workunit.client.0.smithi049.stdout: 1 697602 10.13 MB/sec execute 224 sec latency 64.043 ms 2024-03-18T23:14:43.247 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:43.244+0000 7f7e4a377640 1 -- 172.21.15.49:0/1356289290 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7e44068ea0 msgr2=0x7f7e44069300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:14:43.247 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:43.244+0000 7f7e4a377640 1 --2- 172.21.15.49:0/1356289290 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7e44068ea0 0x7f7e44069300 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f7e38009930 tx=0x7f7e3802bc30 comp rx=0 tx=0).stop 2024-03-18T23:14:43.247 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:43.244+0000 7f7e4a377640 1 -- 172.21.15.49:0/1356289290 shutdown_connections 2024-03-18T23:14:43.248 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:43.244+0000 7f7e4a377640 1 --2- 172.21.15.49:0/1356289290 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7e44069840 0x7f7e4410e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:43.248 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:43.244+0000 7f7e4a377640 1 --2- 172.21.15.49:0/1356289290 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7e44068ea0 0x7f7e44069300 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:43.248 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:43.244+0000 7f7e4a377640 1 --2- 172.21.15.49:0/1356289290 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7e441078f0 0x7f7e44107cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:43.248 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:43.244+0000 7f7e4a377640 1 -- 172.21.15.49:0/1356289290 >> 172.21.15.49:0/1356289290 conn(0x7f7e44077570 msgr2=0x7f7e44077980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:14:43.248 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:43.244+0000 7f7e4a377640 1 -- 172.21.15.49:0/1356289290 shutdown_connections 2024-03-18T23:14:43.248 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:43.244+0000 7f7e4a377640 1 -- 172.21.15.49:0/1356289290 wait complete. 2024-03-18T23:14:43.248 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:43.245+0000 7f7e4a377640 1 Processor -- start 2024-03-18T23:14:43.248 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:43.245+0000 7f7e4a377640 1 -- start start 2024-03-18T23:14:43.248 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:43.245+0000 7f7e4a377640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7e44068ea0 0x7f7e441980a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:14:43.248 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:43.245+0000 7f7e4a377640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7e44069840 0x7f7e441985e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:14:43.248 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:43.245+0000 7f7e4a377640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7e441078f0 0x7f7e4419c960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:14:43.248 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:43.245+0000 7f7e4a377640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f7e44111be0 con 0x7f7e44068ea0 2024-03-18T23:14:43.249 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:43.245+0000 7f7e4a377640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f7e44111a60 con 0x7f7e441078f0 2024-03-18T23:14:43.249 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:43.245+0000 7f7e4a377640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f7e44111d60 con 0x7f7e44069840 2024-03-18T23:14:43.249 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:43.245+0000 7f7e43fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7e44068ea0 0x7f7e441980a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:14:43.249 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:43.245+0000 7f7e43fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7e44068ea0 0x7f7e441980a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:59868/0 (socket says 172.21.15.49:59868) 2024-03-18T23:14:43.249 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:43.245+0000 7f7e43fff640 1 -- 172.21.15.49:0/3588199509 learned_addr learned my addr 172.21.15.49:0/3588199509 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:14:43.249 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:43.245+0000 7f7e48914640 1 --2- 172.21.15.49:0/3588199509 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7e441078f0 0x7f7e4419c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:14:43.249 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:43.245+0000 7f7e437fe640 1 --2- 172.21.15.49:0/3588199509 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7e44069840 0x7f7e441985e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:14:43.249 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:43.246+0000 7f7e43fff640 1 -- 172.21.15.49:0/3588199509 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7e44069840 msgr2=0x7f7e441985e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:14:43.249 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:43.246+0000 7f7e43fff640 1 --2- 172.21.15.49:0/3588199509 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7e44069840 0x7f7e441985e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:43.249 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:43.246+0000 7f7e43fff640 1 -- 172.21.15.49:0/3588199509 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7e441078f0 msgr2=0x7f7e4419c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:14:43.249 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:43.246+0000 7f7e43fff640 1 --2- 172.21.15.49:0/3588199509 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7e441078f0 0x7f7e4419c960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:43.250 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:43.246+0000 7f7e43fff640 1 -- 172.21.15.49:0/3588199509 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e4419d090 con 0x7f7e44068ea0 2024-03-18T23:14:43.250 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:43.246+0000 7f7e43fff640 1 --2- 172.21.15.49:0/3588199509 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7e44068ea0 0x7f7e441980a0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f7e3400e9f0 tx=0x7f7e3400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:14:43.250 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:43.246+0000 7f7e417fa640 1 -- 172.21.15.49:0/3588199509 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e3400cd80 con 0x7f7e44068ea0 2024-03-18T23:14:43.250 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:43.246+0000 7f7e417fa640 1 -- 172.21.15.49:0/3588199509 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7e34010910 con 0x7f7e44068ea0 2024-03-18T23:14:43.250 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:43.246+0000 7f7e4a377640 1 -- 172.21.15.49:0/3588199509 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e4419d380 con 0x7f7e44068ea0 2024-03-18T23:14:43.250 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:43.246+0000 7f7e417fa640 1 -- 172.21.15.49:0/3588199509 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e34018990 con 0x7f7e44068ea0 2024-03-18T23:14:43.250 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:43.246+0000 7f7e4a377640 1 -- 172.21.15.49:0/3588199509 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e441a4c10 con 0x7f7e44068ea0 2024-03-18T23:14:43.258 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:43.248+0000 7f7e417fa640 1 -- 172.21.15.49:0/3588199509 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 25) ==== 137421+0+0 (secure 0 0 0) 0x7f7e34010ac0 con 0x7f7e44068ea0 2024-03-18T23:14:43.258 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:43.249+0000 7f7e4cf65740 1 Processor -- start 2024-03-18T23:14:43.258 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:43.249+0000 7f7e4cf65740 1 -- start start 2024-03-18T23:14:43.258 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:43.249+0000 7f7e417fa640 1 --2- 172.21.15.49:0/3588199509 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f7e2c0a2a10 0x7f7e2c0a4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:14:43.258 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:43.249+0000 7f7e4cf65740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e549a47ef0 0x55e549a482d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:14:43.258 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:43.249+0000 7f7e4cf65740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e549a48810 0x55e549a510a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:14:43.258 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:43.249+0000 7f7e48914640 1 --2- 172.21.15.49:0/3588199509 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f7e2c0a2a10 0x7f7e2c0a4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:14:43.259 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:43.249+0000 7f7e417fa640 1 -- 172.21.15.49:0/3588199509 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8512+0+0 (secure 0 0 0) 0x7f7e340cf110 con 0x7f7e44068ea0 2024-03-18T23:14:43.259 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:43.249+0000 7f7e4cf65740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e549a515e0 0x55e549a539d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:14:43.259 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:43.249+0000 7f7e437fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e549a48810 0x55e549a510a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:14:43.259 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:43.249+0000 7f7e43fff640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e549a47ef0 0x55e549a482d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:14:43.259 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:43.249+0000 7f7e437fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e549a48810 0x55e549a510a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:59878/0 (socket says 172.21.15.49:59878) 2024-03-18T23:14:43.259 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:43.249+0000 7f7e4cf65740 1 -- 172.21.15.49:0/153708435 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55e5499abbe0 con 0x55e549a48810 2024-03-18T23:14:43.259 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:43.249+0000 7f7e437fe640 1 -- 172.21.15.49:0/153708435 learned_addr learned my addr 172.21.15.49:0/153708435 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:14:43.259 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:43.249+0000 7f7e4cf65740 1 -- 172.21.15.49:0/153708435 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55e549954960 con 0x55e549a515e0 2024-03-18T23:14:43.259 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:43.249+0000 7f7e4cf65740 1 -- 172.21.15.49:0/153708435 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55e5499a7c80 con 0x55e549a47ef0 2024-03-18T23:14:43.259 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:43.249+0000 7f7e48914640 1 --2- 172.21.15.49:0/153708435 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e549a515e0 0x55e549a539d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:14:43.259 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:43.250+0000 7f7e437fe640 1 -- 172.21.15.49:0/153708435 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e549a47ef0 msgr2=0x55e549a482d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:14:43.259 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:43.250+0000 7f7e437fe640 1 --2- 172.21.15.49:0/153708435 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e549a47ef0 0x55e549a482d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:43.259 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:43.250+0000 7f7e437fe640 1 -- 172.21.15.49:0/153708435 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e549a515e0 msgr2=0x55e549a539d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:14:43.260 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:43.250+0000 7f7e437fe640 1 --2- 172.21.15.49:0/153708435 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e549a515e0 0x55e549a539d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:43.260 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:43.250+0000 7f7e437fe640 1 -- 172.21.15.49:0/153708435 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e54994bd20 con 0x55e549a48810 2024-03-18T23:14:43.260 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:43.250+0000 7f7e43fff640 1 --2- 172.21.15.49:0/153708435 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e549a47ef0 0x55e549a482d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:14:43.260 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:43.250+0000 7f7e48914640 1 --2- 172.21.15.49:0/3588199509 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f7e2c0a2a10 0x7f7e2c0a4ed0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f7e3c009fd0 tx=0x7f7e3c011bc0 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:14:43.260 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:43.250+0000 7f7e437fe640 1 --2- 172.21.15.49:0/153708435 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e549a48810 0x55e549a510a0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f7e38009900 tx=0x7f7e38005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b87ea842a4b9b6ce server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:14:43.260 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:43.250+0000 7f7e2a7fc640 1 -- 172.21.15.49:0/153708435 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e38042070 con 0x55e549a48810 2024-03-18T23:14:43.260 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:43.250+0000 7f7e2a7fc640 1 -- 172.21.15.49:0/153708435 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7e3802c8b0 con 0x55e549a48810 2024-03-18T23:14:43.260 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:43.250+0000 7f7e2a7fc640 1 -- 172.21.15.49:0/153708435 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e38004ba0 con 0x55e549a48810 2024-03-18T23:14:43.260 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:43.251+0000 7f7e4cf65740 1 -- 172.21.15.49:0/153708435 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e549a48810 msgr2=0x55e549a510a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:14:43.260 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:43.251+0000 7f7e4cf65740 1 --2- 172.21.15.49:0/153708435 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e549a48810 0x55e549a510a0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f7e38009900 tx=0x7f7e38005d50 comp rx=0 tx=0).stop 2024-03-18T23:14:43.260 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:43.251+0000 7f7e4cf65740 1 -- 172.21.15.49:0/153708435 shutdown_connections 2024-03-18T23:14:43.260 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:43.251+0000 7f7e4cf65740 1 --2- 172.21.15.49:0/153708435 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e549a515e0 0x55e549a539d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:43.260 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:43.251+0000 7f7e4cf65740 1 --2- 172.21.15.49:0/153708435 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e549a48810 0x55e549a510a0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:43.260 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:43.251+0000 7f7e4cf65740 1 --2- 172.21.15.49:0/153708435 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e549a47ef0 0x55e549a482d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:43.261 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:43.251+0000 7f7e4cf65740 1 -- 172.21.15.49:0/153708435 >> 172.21.15.49:0/153708435 conn(0x55e549a42810 msgr2=0x55e549a42bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:14:43.261 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:43.251+0000 7f7e4cf65740 1 -- 172.21.15.49:0/153708435 shutdown_connections 2024-03-18T23:14:43.261 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:43.251+0000 7f7e4cf65740 1 -- 172.21.15.49:0/153708435 wait complete. 2024-03-18T23:14:43.261 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:43.251+0000 7f7e4cf65740 1 -- start start 2024-03-18T23:14:43.261 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:43.251+0000 7f7e4cf65740 1 Processor -- start 2024-03-18T23:14:43.261 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:43.252+0000 7f7e4cf65740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e549a47ef0 0x55e549a50f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:14:43.261 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:43.252+0000 7f7e4cf65740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e549a48810 0x55e549a4e300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:14:43.261 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:43.252+0000 7f7e4cf65740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e549a515e0 0x55e549a4e880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:14:43.261 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:43.252+0000 7f7e4cf65740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55e549a585c0 con 0x55e549a48810 2024-03-18T23:14:43.261 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:43.252+0000 7f7e4cf65740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55e549967db0 con 0x55e549a515e0 2024-03-18T23:14:43.261 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:43.252+0000 7f7e4cf65740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55e549a58740 con 0x55e549a47ef0 2024-03-18T23:14:43.261 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:43.252+0000 7f7e4cf65740 10 client.0 _ll_register_callbacks cb 0x55e54996ce70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T23:14:43.261 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:43.252+0000 7f7e437fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e549a48810 0x55e549a4e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:14:43.262 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:43.252+0000 7f7e43fff640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e549a47ef0 0x55e549a50f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:14:43.262 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:43.252+0000 7f7e437fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e549a48810 0x55e549a4e300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:59886/0 (socket says 172.21.15.49:59886) 2024-03-18T23:14:43.262 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:43.252+0000 7f7e437fe640 1 -- 172.21.15.49:0/4238692084 learned_addr learned my addr 172.21.15.49:0/4238692084 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:14:43.262 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:43.252+0000 7f7e48914640 1 --2- 172.21.15.49:0/4238692084 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e549a515e0 0x55e549a4e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:14:43.262 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:43.252+0000 7f7e437fe640 1 -- 172.21.15.49:0/4238692084 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e549a47ef0 msgr2=0x55e549a50f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:14:43.262 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:43.252+0000 7f7e437fe640 1 --2- 172.21.15.49:0/4238692084 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e549a47ef0 0x55e549a50f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:43.262 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:43.252+0000 7f7e437fe640 1 -- 172.21.15.49:0/4238692084 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e549a515e0 msgr2=0x55e549a4e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:14:43.262 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:43.252+0000 7f7e437fe640 1 --2- 172.21.15.49:0/4238692084 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e549a515e0 0x55e549a4e880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:43.262 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:43.252+0000 7f7e437fe640 1 -- 172.21.15.49:0/4238692084 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e549966c60 con 0x55e549a48810 2024-03-18T23:14:43.262 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:43.252+0000 7f7e43fff640 1 --2- 172.21.15.49:0/4238692084 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e549a47ef0 0x55e549a50f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:14:43.262 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:43.252+0000 7f7e437fe640 1 --2- 172.21.15.49:0/4238692084 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e549a48810 0x55e549a4e300 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f7e3802c170 tx=0x7f7e38004b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc4f8388df4e9313 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:14:43.262 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:43.252+0000 7f7e237fe640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-18T23:14:43.263 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:43.252+0000 7f7e237fe640 1 -- 172.21.15.49:0/4238692084 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e3804c070 con 0x55e549a48810 2024-03-18T23:14:43.263 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:43.253+0000 7f7e237fe640 1 -- 172.21.15.49:0/4238692084 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8512+0+0 (secure 0 0 0) 0x7f7e38005120 con 0x55e549a48810 2024-03-18T23:14:43.263 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:43.253+0000 7f7e4cf65740 1 -- 172.21.15.49:0/4238692084 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7e3802c8b0 con 0x55e549a48810 2024-03-18T23:14:43.263 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:43.253+0000 7f7e237fe640 1 -- 172.21.15.49:0/4238692084 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f7e38058050 con 0x55e549a48810 2024-03-18T23:14:43.263 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:43.253+0000 7f7e4cf65740 10 client.15975 fetch_fsmap learned FSMap version 32 2024-03-18T23:14:43.263 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:43.253+0000 7f7e4cf65740 1 -- 172.21.15.49:0/4238692084 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55e5496c0210 con 0x55e549a48810 2024-03-18T23:14:43.263 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:43.253+0000 7f7e237fe640 1 -- 172.21.15.49:0/4238692084 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7e3802c8b0 con 0x55e549a48810 2024-03-18T23:14:43.263 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:43.253+0000 7f7e237fe640 1 -- 172.21.15.49:0/4238692084 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f7e38047ba0 con 0x55e549a48810 2024-03-18T23:14:43.263 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:43.253+0000 7f7e4cf65740 10 client.15975 fetch_fsmap finished waiting for FSMap version 32 2024-03-18T23:14:43.263 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:43.253+0000 7f7e4cf65740 10 client.15975 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T23:14:43.263 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:43.253+0000 7f7e4cf65740 20 client.15975 populate_metadata read hostname 'smithi049' 2024-03-18T23:14:43.263 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:43.253+0000 7f7e4cf65740 1 --2- 172.21.15.49:0/4238692084 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x55e549a65a70 0x55e549a67ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:14:43.264 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:43.253+0000 7f7e4cf65740 4 client.15975 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T23:14:43.264 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:43.253+0000 7f7e4cf65740 1 -- 172.21.15.49:0/4238692084 --> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e5499b9120 con 0x55e549a65a70 2024-03-18T23:14:43.264 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:43.253+0000 7f7e43fff640 1 --2- 172.21.15.49:0/4238692084 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x55e549a65a70 0x55e549a67ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:14:43.264 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:43.254+0000 7f7e43fff640 1 --2- 172.21.15.49:0/4238692084 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x55e549a65a70 0x55e549a67ed0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7a866bed34178114 server_cookie=9a6d8237ad4cb623 in_seq=0 out_seq=0 2024-03-18T23:14:43.264 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:43.254+0000 7f7e237fe640 10 client.15975 ms_handle_connect on v2:172.21.15.139:6838/4280128209 2024-03-18T23:14:43.264 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:43.255+0000 7f7e237fe640 1 -- 172.21.15.49:0/4238692084 <== mds.0 v2:172.21.15.139:6838/4280128209 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55e5499b9120 con 0x55e549a65a70 2024-03-18T23:14:43.264 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:43.255+0000 7f7e237fe640 10 client.15975 handle_command_reply: tid=0 2024-03-18T23:14:43.264 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:43.255+0000 7f7e4cf65740 1 client.15975 shutdown 2024-03-18T23:14:43.264 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:43.255+0000 7f7e4cf65740 1 -- 172.21.15.49:0/4238692084 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e549a48810 msgr2=0x55e549a4e300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:14:43.264 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:43.255+0000 7f7e4cf65740 1 --2- 172.21.15.49:0/4238692084 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e549a48810 0x55e549a4e300 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f7e3802c170 tx=0x7f7e38004b50 comp rx=0 tx=0).stop 2024-03-18T23:14:43.264 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:43.255+0000 7f7e4cf65740 1 -- 172.21.15.49:0/4238692084 shutdown_connections 2024-03-18T23:14:43.264 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:43.255+0000 7f7e4cf65740 1 --2- 172.21.15.49:0/4238692084 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x55e549a65a70 0x55e549a67ed0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:43.264 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:43.255+0000 7f7e4cf65740 1 --2- 172.21.15.49:0/4238692084 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e549a515e0 0x55e549a4e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:43.264 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:43.255+0000 7f7e4cf65740 1 --2- 172.21.15.49:0/4238692084 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e549a48810 0x55e549a4e300 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:43.265 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:43.255+0000 7f7e237fe640 0 client.15975 ms_handle_reset on v2:172.21.15.139:6838/4280128209 2024-03-18T23:14:43.265 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:43.255+0000 7f7e4cf65740 1 --2- 172.21.15.49:0/4238692084 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e549a47ef0 0x55e549a50f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:43.265 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:43.255+0000 7f7e4cf65740 1 -- 172.21.15.49:0/4238692084 >> 172.21.15.49:0/4238692084 conn(0x55e549a42810 msgr2=0x55e549a41ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:14:43.265 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:43.255+0000 7f7e4cf65740 1 -- 172.21.15.49:0/4238692084 shutdown_connections 2024-03-18T23:14:43.265 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:43.255+0000 7f7e4cf65740 1 -- 172.21.15.49:0/4238692084 wait complete. 2024-03-18T23:14:43.265 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:43.255+0000 7f7e4cf65740 20 client.15975 trim_cache size 0 max 16384 2024-03-18T23:14:43.277 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:43.271+0000 7f7e4cf65740 1 Processor -- start 2024-03-18T23:14:43.277 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:43.271+0000 7f7e4cf65740 1 -- start start 2024-03-18T23:14:43.277 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:43.271+0000 7f7e4cf65740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e549a47ef0 0x55e549a61360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:14:43.277 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:43.271+0000 7f7e4cf65740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e549a48810 0x55e549a4d460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:14:43.277 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:43.271+0000 7f7e4cf65740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e549a515e0 0x55e549a41930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:14:43.277 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:43.271+0000 7f7e4cf65740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f7e441ab1d0 con 0x55e549a47ef0 2024-03-18T23:14:43.277 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:43.271+0000 7f7e4cf65740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f7e441acd10 con 0x55e549a48810 2024-03-18T23:14:43.277 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:43.271+0000 7f7e4cf65740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f7e441ab8a0 con 0x55e549a515e0 2024-03-18T23:14:43.277 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:43.272+0000 7f7e43fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e549a47ef0 0x55e549a61360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:14:43.277 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:43.272+0000 7f7e43fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e549a47ef0 0x55e549a61360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:59890/0 (socket says 172.21.15.49:59890) 2024-03-18T23:14:43.277 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:43.272+0000 7f7e43fff640 1 -- 172.21.15.49:0/3852447272 learned_addr learned my addr 172.21.15.49:0/3852447272 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:14:43.278 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:43.272+0000 7f7e48914640 1 --2- 172.21.15.49:0/3852447272 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e549a515e0 0x55e549a41930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:14:43.278 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:43.272+0000 7f7e437fe640 1 --2- 172.21.15.49:0/3852447272 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e549a48810 0x55e549a4d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:14:43.278 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:43.272+0000 7f7e43fff640 1 -- 172.21.15.49:0/3852447272 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e549a515e0 msgr2=0x55e549a41930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:14:43.278 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:43.272+0000 7f7e43fff640 1 --2- 172.21.15.49:0/3852447272 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e549a515e0 0x55e549a41930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:43.278 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:43.272+0000 7f7e43fff640 1 -- 172.21.15.49:0/3852447272 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e549a48810 msgr2=0x55e549a4d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:14:43.278 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:43.272+0000 7f7e43fff640 1 --2- 172.21.15.49:0/3852447272 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e549a48810 0x55e549a4d460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:43.278 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:43.272+0000 7f7e43fff640 1 -- 172.21.15.49:0/3852447272 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e549a48640 con 0x55e549a47ef0 2024-03-18T23:14:43.278 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:43.272+0000 7f7e43fff640 1 --2- 172.21.15.49:0/3852447272 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e549a47ef0 0x55e549a61360 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f7e34009250 tx=0x7f7e34097430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc0816ad1df476c8 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:14:43.278 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:43.272+0000 7f7e29ffb640 1 -- 172.21.15.49:0/3852447272 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e340aa9c0 con 0x55e549a47ef0 2024-03-18T23:14:43.278 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:43.272+0000 7f7e29ffb640 1 -- 172.21.15.49:0/3852447272 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7e340860b0 con 0x55e549a47ef0 2024-03-18T23:14:43.278 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:43.272+0000 7f7e29ffb640 1 -- 172.21.15.49:0/3852447272 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e340a3cf0 con 0x55e549a47ef0 2024-03-18T23:14:43.279 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:43.273+0000 7f7e4cf65740 1 -- 172.21.15.49:0/3852447272 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e549a47ef0 msgr2=0x55e549a61360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:14:43.279 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:43.273+0000 7f7e4cf65740 1 --2- 172.21.15.49:0/3852447272 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e549a47ef0 0x55e549a61360 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f7e34009250 tx=0x7f7e34097430 comp rx=0 tx=0).stop 2024-03-18T23:14:43.279 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:43.273+0000 7f7e4cf65740 1 -- 172.21.15.49:0/3852447272 shutdown_connections 2024-03-18T23:14:43.279 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:43.273+0000 7f7e4cf65740 1 --2- 172.21.15.49:0/3852447272 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e549a515e0 0x55e549a41930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:43.279 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:43.273+0000 7f7e4cf65740 1 --2- 172.21.15.49:0/3852447272 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e549a48810 0x55e549a4d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:43.279 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:43.273+0000 7f7e4cf65740 1 --2- 172.21.15.49:0/3852447272 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e549a47ef0 0x55e549a61360 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:43.279 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:43.273+0000 7f7e4cf65740 1 -- 172.21.15.49:0/3852447272 >> 172.21.15.49:0/3852447272 conn(0x55e549a42810 msgr2=0x55e549a41220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:14:43.279 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:43.273+0000 7f7e4cf65740 1 -- 172.21.15.49:0/3852447272 shutdown_connections 2024-03-18T23:14:43.279 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:43.273+0000 7f7e4cf65740 1 -- 172.21.15.49:0/3852447272 wait complete. 2024-03-18T23:14:43.279 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:43.273+0000 7f7e4cf65740 1 -- start start 2024-03-18T23:14:43.279 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:43.274+0000 7f7e4cf65740 1 Processor -- start 2024-03-18T23:14:43.279 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:43.274+0000 7f7e4cf65740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e549a47ef0 0x55e549a58b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:14:43.279 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:43.274+0000 7f7e4cf65740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e549a48810 0x55e549a590a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:14:43.280 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:43.274+0000 7f7e4cf65740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e549a515e0 0x55e549ac3e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:14:43.280 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:43.274+0000 7f7e4cf65740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f7e441ac640 con 0x55e549a47ef0 2024-03-18T23:14:43.280 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:43.274+0000 7f7e4cf65740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f7e441abf70 con 0x55e549a515e0 2024-03-18T23:14:43.280 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:43.274+0000 7f7e4cf65740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f7e441a9190 con 0x55e549a48810 2024-03-18T23:14:43.280 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:43.274+0000 7f7e4cf65740 10 client.0 _ll_register_callbacks cb 0x7f7e34009700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T23:14:43.280 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:43.274+0000 7f7e43fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e549a47ef0 0x55e549a58b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:14:43.280 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:43.274+0000 7f7e43fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e549a47ef0 0x55e549a58b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:59892/0 (socket says 172.21.15.49:59892) 2024-03-18T23:14:43.280 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:43.274+0000 7f7e43fff640 1 -- 172.21.15.49:0/4149197913 learned_addr learned my addr 172.21.15.49:0/4149197913 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:14:43.280 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:43.274+0000 7f7e48914640 1 --2- 172.21.15.49:0/4149197913 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e549a515e0 0x55e549ac3e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:14:43.280 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:43.274+0000 7f7e437fe640 1 --2- 172.21.15.49:0/4149197913 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e549a48810 0x55e549a590a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:14:43.280 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:43.274+0000 7f7e43fff640 1 -- 172.21.15.49:0/4149197913 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e549a48810 msgr2=0x55e549a590a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:14:43.280 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:43.274+0000 7f7e43fff640 1 --2- 172.21.15.49:0/4149197913 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e549a48810 0x55e549a590a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:43.281 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:43.274+0000 7f7e43fff640 1 -- 172.21.15.49:0/4149197913 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e549a515e0 msgr2=0x55e549ac3e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:14:43.281 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:43.274+0000 7f7e43fff640 1 --2- 172.21.15.49:0/4149197913 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e549a515e0 0x55e549ac3e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:43.281 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:43.274+0000 7f7e43fff640 1 -- 172.21.15.49:0/4149197913 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e54978c620 con 0x55e549a47ef0 2024-03-18T23:14:43.281 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:43.274+0000 7f7e43fff640 1 --2- 172.21.15.49:0/4149197913 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e549a47ef0 0x55e549a58b60 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f7e34051630 tx=0x7f7e34005cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f67e011d3b023a01 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:14:43.281 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:43.274+0000 7f7e237fe640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-18T23:14:43.281 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:43.274+0000 7f7e237fe640 1 -- 172.21.15.49:0/4149197913 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e340a5060 con 0x55e549a47ef0 2024-03-18T23:14:43.281 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:43.275+0000 7f7e237fe640 1 -- 172.21.15.49:0/4149197913 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8512+0+0 (secure 0 0 0) 0x7f7e340a3a90 con 0x55e549a47ef0 2024-03-18T23:14:43.281 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:43.275+0000 7f7e4cf65740 1 -- 172.21.15.49:0/4149197913 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7e340860b0 con 0x55e549a47ef0 2024-03-18T23:14:43.281 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:43.275+0000 7f7e237fe640 1 -- 172.21.15.49:0/4149197913 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f7e340b3070 con 0x55e549a47ef0 2024-03-18T23:14:43.281 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:43.275+0000 7f7e4cf65740 10 client.15981 fetch_fsmap learned FSMap version 32 2024-03-18T23:14:43.281 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:43.275+0000 7f7e4cf65740 1 -- 172.21.15.49:0/4149197913 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55e549acb5c0 con 0x55e549a47ef0 2024-03-18T23:14:43.281 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:43.275+0000 7f7e237fe640 1 -- 172.21.15.49:0/4149197913 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7e340860b0 con 0x55e549a47ef0 2024-03-18T23:14:43.281 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:43.275+0000 7f7e237fe640 1 -- 172.21.15.49:0/4149197913 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f7e34018b30 con 0x55e549a47ef0 2024-03-18T23:14:43.282 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:43.275+0000 7f7e4cf65740 10 client.15981 fetch_fsmap finished waiting for FSMap version 32 2024-03-18T23:14:43.282 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:43.275+0000 7f7e4cf65740 10 client.15981 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T23:14:43.282 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:43.275+0000 7f7e4cf65740 20 client.15981 populate_metadata read hostname 'smithi049' 2024-03-18T23:14:43.282 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:43.275+0000 7f7e4cf65740 1 --2- 172.21.15.49:0/4149197913 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x55e549acbd90 0x55e549ace1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:14:43.282 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:43.275+0000 7f7e4cf65740 4 client.15981 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T23:14:43.282 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:43.275+0000 7f7e4cf65740 1 -- 172.21.15.49:0/4149197913 --> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e5499b3470 con 0x55e549acbd90 2024-03-18T23:14:43.282 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:43.275+0000 7f7e437fe640 1 --2- 172.21.15.49:0/4149197913 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x55e549acbd90 0x55e549ace1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:14:43.282 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:43.276+0000 7f7e437fe640 1 --2- 172.21.15.49:0/4149197913 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x55e549acbd90 0x55e549ace1b0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ab687cd58418133a server_cookie=978b99313dc103cd in_seq=0 out_seq=0 2024-03-18T23:14:43.282 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:43.276+0000 7f7e237fe640 10 client.15981 ms_handle_connect on v2:172.21.15.139:6838/4280128209 2024-03-18T23:14:43.379 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:14:43 smithi049 ceph-mon[62991]: pgmap v198: 97 pgs: 97 active+clean; 85 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 3.9 MiB/s wr, 356 op/s 2024-03-18T23:14:43.441 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:43.440+0000 7f7e237fe640 1 -- 172.21.15.49:0/4149197913 <== mds.0 v2:172.21.15.139:6838/4280128209 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e5499b3470 con 0x55e549acbd90 2024-03-18T23:14:43.441 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:43.440+0000 7f7e237fe640 10 client.15981 handle_command_reply: tid=0 2024-03-18T23:14:43.441 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:43.440+0000 7f7e4cf65740 1 client.15981 shutdown 2024-03-18T23:14:43.442 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:43.440+0000 7f7e4cf65740 1 -- 172.21.15.49:0/4149197913 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e549a47ef0 msgr2=0x55e549a58b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:14:43.442 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:43.440+0000 7f7e4cf65740 1 --2- 172.21.15.49:0/4149197913 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e549a47ef0 0x55e549a58b60 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f7e34051630 tx=0x7f7e34005cc0 comp rx=0 tx=0).stop 2024-03-18T23:14:43.442 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:43.440+0000 7f7e4cf65740 1 -- 172.21.15.49:0/4149197913 shutdown_connections 2024-03-18T23:14:43.442 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:43.440+0000 7f7e4cf65740 1 --2- 172.21.15.49:0/4149197913 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x55e549acbd90 0x55e549ace1b0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:43.442 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:43.440+0000 7f7e4cf65740 1 --2- 172.21.15.49:0/4149197913 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e549a515e0 0x55e549ac3e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:43.442 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:43.440+0000 7f7e4cf65740 1 --2- 172.21.15.49:0/4149197913 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e549a48810 0x55e549a590a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:43.442 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:43.440+0000 7f7e4cf65740 1 --2- 172.21.15.49:0/4149197913 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e549a47ef0 0x55e549a58b60 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:43.442 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:43.440+0000 7f7e4cf65740 1 -- 172.21.15.49:0/4149197913 >> 172.21.15.49:0/4149197913 conn(0x55e549a42810 msgr2=0x55e549a41ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:14:43.442 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:43.440+0000 7f7e237fe640 0 client.15981 ms_handle_reset on v2:172.21.15.139:6838/4280128209 2024-03-18T23:14:43.442 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:43.440+0000 7f7e4cf65740 1 -- 172.21.15.49:0/4149197913 shutdown_connections 2024-03-18T23:14:43.442 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:43.440+0000 7f7e4cf65740 1 -- 172.21.15.49:0/4149197913 wait complete. 2024-03-18T23:14:43.442 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:43.441+0000 7f7e4cf65740 20 client.15981 trim_cache size 0 max 16384 2024-03-18T23:14:43.443 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:{ 2024-03-18T23:14:43.443 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "message": "", 2024-03-18T23:14:43.443 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "return_code": 0 2024-03-18T23:14:43.443 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:} 2024-03-18T23:14:43.444 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:43.441+0000 7f7e4a377640 1 -- 172.21.15.49:0/3588199509 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f7e2c0a2a10 msgr2=0x7f7e2c0a4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:14:43.444 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:43.441+0000 7f7e4a377640 1 --2- 172.21.15.49:0/3588199509 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f7e2c0a2a10 0x7f7e2c0a4ed0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f7e3c009fd0 tx=0x7f7e3c011bc0 comp rx=0 tx=0).stop 2024-03-18T23:14:43.444 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:43.441+0000 7f7e4a377640 1 -- 172.21.15.49:0/3588199509 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7e44068ea0 msgr2=0x7f7e441980a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:14:43.444 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:43.441+0000 7f7e4a377640 1 --2- 172.21.15.49:0/3588199509 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7e44068ea0 0x7f7e441980a0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f7e3400e9f0 tx=0x7f7e3400eeb0 comp rx=0 tx=0).stop 2024-03-18T23:14:43.444 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:43.441+0000 7f7e4a377640 1 -- 172.21.15.49:0/3588199509 shutdown_connections 2024-03-18T23:14:43.444 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:43.441+0000 7f7e4a377640 1 --2- 172.21.15.49:0/3588199509 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f7e2c0a2a10 0x7f7e2c0a4ed0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:43.444 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:43.441+0000 7f7e4a377640 1 --2- 172.21.15.49:0/3588199509 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7e441078f0 0x7f7e4419c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:43.444 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:43.441+0000 7f7e4a377640 1 --2- 172.21.15.49:0/3588199509 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7e44069840 0x7f7e441985e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:43.444 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:43.441+0000 7f7e4a377640 1 --2- 172.21.15.49:0/3588199509 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7e44068ea0 0x7f7e441980a0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:43.444 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:43.441+0000 7f7e4a377640 1 -- 172.21.15.49:0/3588199509 >> 172.21.15.49:0/3588199509 conn(0x7f7e44077570 msgr2=0x7f7e440feac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:14:43.445 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:43.441+0000 7f7e4a377640 1 -- 172.21.15.49:0/3588199509 shutdown_connections 2024-03-18T23:14:43.445 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:43.441+0000 7f7e4a377640 1 -- 172.21.15.49:0/3588199509 wait complete. 2024-03-18T23:14:43.460 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:14:43 smithi175 ceph-mon[67352]: pgmap v198: 97 pgs: 97 active+clean; 85 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 3.9 MiB/s wr, 356 op/s 2024-03-18T23:14:43.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:14:43 smithi139 ceph-mon[67282]: pgmap v198: 97 pgs: 97 active+clean; 85 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 3.9 MiB/s wr, 356 op/s 2024-03-18T23:14:43.774 INFO:tasks.workunit.client.0.smithi049.stdout: 1 699671 10.14 MB/sec execute 225 sec latency 57.653 ms 2024-03-18T23:14:44.774 INFO:tasks.workunit.client.0.smithi049.stdout: 1 701142 10.13 MB/sec execute 226 sec latency 22.099 ms 2024-03-18T23:14:45.379 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:14:45 smithi049 ceph-mon[62991]: pgmap v199: 97 pgs: 97 active+clean; 85 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.5 MiB/s wr, 174 op/s 2024-03-18T23:14:45.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:14:45 smithi175 ceph-mon[67352]: pgmap v199: 97 pgs: 97 active+clean; 85 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.5 MiB/s wr, 174 op/s 2024-03-18T23:14:45.564 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:14:45 smithi139 ceph-mon[67282]: pgmap v199: 97 pgs: 97 active+clean; 85 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.5 MiB/s wr, 174 op/s 2024-03-18T23:14:45.774 INFO:tasks.workunit.client.0.smithi049.stdout: 1 703363 10.14 MB/sec execute 227 sec latency 19.263 ms 2024-03-18T23:14:46.774 INFO:tasks.workunit.client.0.smithi049.stdout: 1 704822 10.12 MB/sec execute 228 sec latency 26.042 ms 2024-03-18T23:14:47.366 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:14:47 smithi139 ceph-mon[67282]: pgmap v200: 97 pgs: 97 active+clean; 85 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.5 MiB/s wr, 174 op/s 2024-03-18T23:14:47.379 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:14:47 smithi049 ceph-mon[62991]: pgmap v200: 97 pgs: 97 active+clean; 85 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.5 MiB/s wr, 174 op/s 2024-03-18T23:14:47.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:14:47 smithi175 ceph-mon[67352]: pgmap v200: 97 pgs: 97 active+clean; 85 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.5 MiB/s wr, 174 op/s 2024-03-18T23:14:47.774 INFO:tasks.workunit.client.0.smithi049.stdout: 1 706897 10.14 MB/sec execute 229 sec latency 117.153 ms 2024-03-18T23:14:48.775 INFO:tasks.workunit.client.0.smithi049.stdout: 1 708377 10.12 MB/sec execute 230 sec latency 17.863 ms 2024-03-18T23:14:49.379 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:14:49 smithi049 ceph-mon[62991]: pgmap v201: 97 pgs: 97 active+clean; 75 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 3.9 MiB/s wr, 348 op/s 2024-03-18T23:14:49.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:14:49 smithi175 ceph-mon[67352]: pgmap v201: 97 pgs: 97 active+clean; 75 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 3.9 MiB/s wr, 348 op/s 2024-03-18T23:14:49.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:14:49 smithi139 ceph-mon[67282]: pgmap v201: 97 pgs: 97 active+clean; 75 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 3.9 MiB/s wr, 348 op/s 2024-03-18T23:14:49.774 INFO:tasks.workunit.client.0.smithi049.stdout: 1 710615 10.13 MB/sec execute 231 sec latency 26.453 ms 2024-03-18T23:14:50.774 INFO:tasks.workunit.client.0.smithi049.stdout: 1 712315 10.11 MB/sec execute 232 sec latency 22.927 ms 2024-03-18T23:14:51.379 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:14:51 smithi049 ceph-mon[62991]: pgmap v202: 97 pgs: 97 active+clean; 75 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.8 MiB/s wr, 262 op/s 2024-03-18T23:14:51.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:14:51 smithi175 ceph-mon[67352]: pgmap v202: 97 pgs: 97 active+clean; 75 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.8 MiB/s wr, 262 op/s 2024-03-18T23:14:51.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:14:51 smithi139 ceph-mon[67282]: pgmap v202: 97 pgs: 97 active+clean; 75 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.8 MiB/s wr, 262 op/s 2024-03-18T23:14:51.774 INFO:tasks.workunit.client.0.smithi049.stdout: 1 714357 10.13 MB/sec execute 233 sec latency 22.063 ms 2024-03-18T23:14:52.774 INFO:tasks.workunit.client.0.smithi049.stdout: 1 715747 10.11 MB/sec execute 234 sec latency 98.510 ms 2024-03-18T23:14:53.379 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:14:53 smithi049 ceph-mon[62991]: pgmap v203: 97 pgs: 97 active+clean; 82 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 4.1 MiB/s wr, 353 op/s 2024-03-18T23:14:53.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:14:53 smithi175 ceph-mon[67352]: pgmap v203: 97 pgs: 97 active+clean; 82 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 4.1 MiB/s wr, 353 op/s 2024-03-18T23:14:53.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:14:53 smithi139 ceph-mon[67282]: pgmap v203: 97 pgs: 97 active+clean; 82 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 4.1 MiB/s wr, 353 op/s 2024-03-18T23:14:53.775 INFO:tasks.workunit.client.0.smithi049.stdout: 1 717896 10.12 MB/sec execute 235 sec latency 15.514 ms 2024-03-18T23:14:54.775 INFO:tasks.workunit.client.0.smithi049.stdout: 1 719768 10.11 MB/sec execute 236 sec latency 19.057 ms 2024-03-18T23:14:55.379 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:14:55 smithi049 ceph-mon[62991]: pgmap v204: 97 pgs: 97 active+clean; 82 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.7 MiB/s wr, 264 op/s 2024-03-18T23:14:55.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:14:55 smithi175 ceph-mon[67352]: pgmap v204: 97 pgs: 97 active+clean; 82 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.7 MiB/s wr, 264 op/s 2024-03-18T23:14:55.564 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:14:55 smithi139 ceph-mon[67282]: pgmap v204: 97 pgs: 97 active+clean; 82 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.7 MiB/s wr, 264 op/s 2024-03-18T23:14:55.775 INFO:tasks.workunit.client.0.smithi049.stdout: 1 721589 10.12 MB/sec execute 237 sec latency 41.844 ms 2024-03-18T23:14:56.537 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:56.534+0000 7f65c4bd3640 1 -- 172.21.15.49:0/238341548 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f65c0101330 msgr2=0x7f65c010d550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:14:56.537 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:56.534+0000 7f65c4bd3640 1 --2- 172.21.15.49:0/238341548 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f65c0101330 0x7f65c010d550 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f65b8009ca0 tx=0x7f65b802c0f0 comp rx=0 tx=0).stop 2024-03-18T23:14:56.537 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:56.534+0000 7f65c4bd3640 1 -- 172.21.15.49:0/238341548 shutdown_connections 2024-03-18T23:14:56.537 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:56.534+0000 7f65c4bd3640 1 --2- 172.21.15.49:0/238341548 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f65c0101330 0x7f65c010d550 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:56.537 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:56.534+0000 7f65c4bd3640 1 --2- 172.21.15.49:0/238341548 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f65c0100970 0x7f65c0100df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:56.537 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:56.534+0000 7f65c4bd3640 1 --2- 172.21.15.49:0/238341548 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f65c01056c0 0x7f65c0105aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:56.538 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:56.534+0000 7f65c4bd3640 1 -- 172.21.15.49:0/238341548 >> 172.21.15.49:0/238341548 conn(0x7f65c0077570 msgr2=0x7f65c0077980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:14:56.538 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:56.534+0000 7f65c4bd3640 1 -- 172.21.15.49:0/238341548 shutdown_connections 2024-03-18T23:14:56.538 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:56.534+0000 7f65c4bd3640 1 -- 172.21.15.49:0/238341548 wait complete. 2024-03-18T23:14:56.538 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:56.535+0000 7f65c4bd3640 1 Processor -- start 2024-03-18T23:14:56.538 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:56.535+0000 7f65c4bd3640 1 -- start start 2024-03-18T23:14:56.538 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:56.535+0000 7f65c4bd3640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f65c0100970 0x7f65c0198ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:14:56.538 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:56.535+0000 7f65c4bd3640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f65c0101330 0x7f65c01993e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:14:56.538 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:56.535+0000 7f65c4bd3640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f65c01056c0 0x7f65c01a3f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:14:56.538 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:56.535+0000 7f65c4bd3640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f65c0110ba0 con 0x7f65c01056c0 2024-03-18T23:14:56.538 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:56.535+0000 7f65c4bd3640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f65c0110a20 con 0x7f65c0100970 2024-03-18T23:14:56.538 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:56.535+0000 7f65c4bd3640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f65c0110d20 con 0x7f65c0101330 2024-03-18T23:14:56.538 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:56.535+0000 7f65bed9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f65c01056c0 0x7f65c01a3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:14:56.538 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:56.535+0000 7f65bed9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f65c01056c0 0x7f65c01a3f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:43816/0 (socket says 172.21.15.49:43816) 2024-03-18T23:14:56.539 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:56.535+0000 7f65be59c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f65c0100970 0x7f65c0198ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:14:56.539 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:56.535+0000 7f65bed9d640 1 -- 172.21.15.49:0/887764718 learned_addr learned my addr 172.21.15.49:0/887764718 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:14:56.539 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:56.535+0000 7f65b7fff640 1 --2- 172.21.15.49:0/887764718 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f65c0101330 0x7f65c01993e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:14:56.539 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:56.536+0000 7f65bed9d640 1 -- 172.21.15.49:0/887764718 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f65c0101330 msgr2=0x7f65c01993e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:14:56.539 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:56.536+0000 7f65bed9d640 1 --2- 172.21.15.49:0/887764718 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f65c0101330 0x7f65c01993e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:56.539 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:56.536+0000 7f65bed9d640 1 -- 172.21.15.49:0/887764718 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f65c0100970 msgr2=0x7f65c0198ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:14:56.539 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:56.536+0000 7f65bed9d640 1 --2- 172.21.15.49:0/887764718 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f65c0100970 0x7f65c0198ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:56.539 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:56.536+0000 7f65bed9d640 1 -- 172.21.15.49:0/887764718 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65c01a4460 con 0x7f65c01056c0 2024-03-18T23:14:56.539 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:56.536+0000 7f65be59c640 1 --2- 172.21.15.49:0/887764718 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f65c0100970 0x7f65c0198ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:14:56.539 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:56.536+0000 7f65bed9d640 1 --2- 172.21.15.49:0/887764718 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f65c01056c0 0x7f65c01a3f20 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f65b802c700 tx=0x7f65b8032f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:14:56.539 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:56.536+0000 7f65b77fe640 1 -- 172.21.15.49:0/887764718 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65b8042020 con 0x7f65c01056c0 2024-03-18T23:14:56.539 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:56.536+0000 7f65b77fe640 1 -- 172.21.15.49:0/887764718 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f65b8034640 con 0x7f65c01056c0 2024-03-18T23:14:56.539 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:56.536+0000 7f65c4bd3640 1 -- 172.21.15.49:0/887764718 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65c01a4630 con 0x7f65c01056c0 2024-03-18T23:14:56.539 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:56.536+0000 7f65b77fe640 1 -- 172.21.15.49:0/887764718 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65b8039040 con 0x7f65c01056c0 2024-03-18T23:14:56.539 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:56.536+0000 7f65c4bd3640 1 -- 172.21.15.49:0/887764718 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65c01a4a30 con 0x7f65c01056c0 2024-03-18T23:14:56.545 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:56.539+0000 7f65b77fe640 1 -- 172.21.15.49:0/887764718 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 25) ==== 137421+0+0 (secure 0 0 0) 0x7f65b8045020 con 0x7f65c01056c0 2024-03-18T23:14:56.545 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:56.539+0000 7f65c77b7740 1 Processor -- start 2024-03-18T23:14:56.545 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:56.539+0000 7f65c77b7740 1 -- start start 2024-03-18T23:14:56.545 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:56.539+0000 7f65c77b7740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562ed5f01ef0 0x562ed5f022d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:14:56.545 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:56.539+0000 7f65c77b7740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562ed5f02810 0x562ed5f0b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:14:56.545 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:56.539+0000 7f65c77b7740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562ed5f0b5e0 0x562ed5f0d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:14:56.545 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:56.539+0000 7f65c77b7740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x562ed5e65be0 con 0x562ed5f02810 2024-03-18T23:14:56.545 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:56.539+0000 7f65c77b7740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x562ed5e0e960 con 0x562ed5f01ef0 2024-03-18T23:14:56.545 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:56.539+0000 7f65c77b7740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x562ed5e61c80 con 0x562ed5f0b5e0 2024-03-18T23:14:56.546 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:56.539+0000 7f65b77fe640 1 --2- 172.21.15.49:0/887764718 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f65ac0a2a10 0x7f65ac0a4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:14:56.546 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:56.539+0000 7f65be59c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562ed5f02810 0x562ed5f0b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:14:56.546 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:56.539+0000 7f65b7fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562ed5f01ef0 0x562ed5f022d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:14:56.546 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:56.540+0000 7f65be59c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562ed5f02810 0x562ed5f0b0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:43830/0 (socket says 172.21.15.49:43830) 2024-03-18T23:14:56.546 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:56.540+0000 7f65b7fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562ed5f01ef0 0x562ed5f022d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.49:53448/0 (socket says 172.21.15.49:53448) 2024-03-18T23:14:56.546 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:56.540+0000 7f65be59c640 1 -- 172.21.15.49:0/2235932145 learned_addr learned my addr 172.21.15.49:0/2235932145 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:14:56.546 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:56.540+0000 7f65b77fe640 1 -- 172.21.15.49:0/887764718 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8512+0+0 (secure 0 0 0) 0x7f65b80f0bf0 con 0x7f65c01056c0 2024-03-18T23:14:56.546 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:56.540+0000 7f65b7fff640 1 --2- 172.21.15.49:0/2235932145 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562ed5f0b5e0 0x562ed5f0d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:14:56.546 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:56.540+0000 7f65bed9d640 1 --2- 172.21.15.49:0/887764718 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f65ac0a2a10 0x7f65ac0a4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:14:56.546 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:56.540+0000 7f65be59c640 1 -- 172.21.15.49:0/2235932145 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562ed5f0b5e0 msgr2=0x562ed5f0d9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:14:56.546 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:56.540+0000 7f65be59c640 1 --2- 172.21.15.49:0/2235932145 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562ed5f0b5e0 0x562ed5f0d9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:56.546 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:56.540+0000 7f65be59c640 1 -- 172.21.15.49:0/2235932145 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562ed5f01ef0 msgr2=0x562ed5f022d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:14:56.546 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:56.540+0000 7f65be59c640 1 --2- 172.21.15.49:0/2235932145 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562ed5f01ef0 0x562ed5f022d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:56.546 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:56.540+0000 7f65be59c640 1 -- 172.21.15.49:0/2235932145 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562ed5e05d20 con 0x562ed5f02810 2024-03-18T23:14:56.547 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:56.540+0000 7f65b7fff640 1 --2- 172.21.15.49:0/2235932145 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562ed5f01ef0 0x562ed5f022d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:14:56.547 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:56.540+0000 7f65be59c640 1 --2- 172.21.15.49:0/2235932145 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562ed5f02810 0x562ed5f0b0a0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f65b00114f0 tx=0x7f65b00119b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e5c1e8daa4548e8 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:14:56.547 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:56.540+0000 7f65bed9d640 1 --2- 172.21.15.49:0/887764718 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f65ac0a2a10 0x7f65ac0a4ed0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f65b8009430 tx=0x7f65b802c0f0 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:14:56.547 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:56.540+0000 7f65b4ff9640 1 -- 172.21.15.49:0/2235932145 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65b001e070 con 0x562ed5f02810 2024-03-18T23:14:56.547 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:56.540+0000 7f65b4ff9640 1 -- 172.21.15.49:0/2235932145 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f65b0011250 con 0x562ed5f02810 2024-03-18T23:14:56.547 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:56.540+0000 7f65b4ff9640 1 -- 172.21.15.49:0/2235932145 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65b0019990 con 0x562ed5f02810 2024-03-18T23:14:56.547 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:56.541+0000 7f65c77b7740 1 -- 172.21.15.49:0/2235932145 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562ed5f02810 msgr2=0x562ed5f0b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:14:56.547 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:56.541+0000 7f65c77b7740 1 --2- 172.21.15.49:0/2235932145 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562ed5f02810 0x562ed5f0b0a0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f65b00114f0 tx=0x7f65b00119b0 comp rx=0 tx=0).stop 2024-03-18T23:14:56.547 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:56.541+0000 7f65c77b7740 1 -- 172.21.15.49:0/2235932145 shutdown_connections 2024-03-18T23:14:56.547 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:56.541+0000 7f65c77b7740 1 --2- 172.21.15.49:0/2235932145 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562ed5f0b5e0 0x562ed5f0d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:56.547 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:56.541+0000 7f65c77b7740 1 --2- 172.21.15.49:0/2235932145 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562ed5f02810 0x562ed5f0b0a0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:56.547 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:56.541+0000 7f65c77b7740 1 --2- 172.21.15.49:0/2235932145 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562ed5f01ef0 0x562ed5f022d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:56.547 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:56.541+0000 7f65c77b7740 1 -- 172.21.15.49:0/2235932145 >> 172.21.15.49:0/2235932145 conn(0x562ed5efc810 msgr2=0x562ed5efcbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:14:56.548 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:56.541+0000 7f65c77b7740 1 -- 172.21.15.49:0/2235932145 shutdown_connections 2024-03-18T23:14:56.548 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:56.541+0000 7f65c77b7740 1 -- 172.21.15.49:0/2235932145 wait complete. 2024-03-18T23:14:56.548 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:56.541+0000 7f65c77b7740 1 -- start start 2024-03-18T23:14:56.548 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:56.542+0000 7f65c77b7740 1 Processor -- start 2024-03-18T23:14:56.548 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:56.542+0000 7f65c77b7740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562ed5f01ef0 0x562ed5f0b020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:14:56.548 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:56.542+0000 7f65c77b7740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562ed5f02810 0x562ed5f083b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:14:56.548 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:56.542+0000 7f65c77b7740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562ed5f0b5e0 0x562ed5f08af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:14:56.548 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:56.542+0000 7f65c77b7740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x562ed5f0e210 con 0x562ed5f01ef0 2024-03-18T23:14:56.548 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:56.542+0000 7f65c77b7740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x562ed5e21db0 con 0x562ed5f0b5e0 2024-03-18T23:14:56.548 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:56.542+0000 7f65c77b7740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x562ed5f0e390 con 0x562ed5f02810 2024-03-18T23:14:56.548 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:56.542+0000 7f65c77b7740 10 client.0 _ll_register_callbacks cb 0x562ed5e26e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T23:14:56.548 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:56.542+0000 7f65b7fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562ed5f01ef0 0x562ed5f0b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:14:56.548 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:56.542+0000 7f65b7fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562ed5f01ef0 0x562ed5f0b020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:43834/0 (socket says 172.21.15.49:43834) 2024-03-18T23:14:56.549 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:56.542+0000 7f65b7fff640 1 -- 172.21.15.49:0/2829838223 learned_addr learned my addr 172.21.15.49:0/2829838223 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:14:56.549 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:56.542+0000 7f65b7fff640 1 -- 172.21.15.49:0/2829838223 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562ed5f02810 msgr2=0x562ed5f083b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:14:56.549 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:56.542+0000 7f65b7fff640 1 --2- 172.21.15.49:0/2829838223 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562ed5f02810 0x562ed5f083b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:56.549 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:56.542+0000 7f65b7fff640 1 -- 172.21.15.49:0/2829838223 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562ed5f0b5e0 msgr2=0x562ed5f08af0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-18T23:14:56.549 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:56.542+0000 7f65b7fff640 1 --2- 172.21.15.49:0/2829838223 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562ed5f0b5e0 0x562ed5f08af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:56.549 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:56.542+0000 7f65b7fff640 1 -- 172.21.15.49:0/2829838223 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562ed5e20c60 con 0x562ed5f01ef0 2024-03-18T23:14:56.549 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:56.542+0000 7f65b7fff640 1 --2- 172.21.15.49:0/2829838223 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562ed5f01ef0 0x562ed5f0b020 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f65a8004540 tx=0x7f65a8009ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e678bb2466ad8498 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:14:56.549 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:56.542+0000 7f65a5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-18T23:14:56.549 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:56.542+0000 7f65a5ffb640 1 -- 172.21.15.49:0/2829838223 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65a8005220 con 0x562ed5f01ef0 2024-03-18T23:14:56.549 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:56.542+0000 7f65a5ffb640 1 -- 172.21.15.49:0/2829838223 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8512+0+0 (secure 0 0 0) 0x7f65a80053c0 con 0x562ed5f01ef0 2024-03-18T23:14:56.549 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:56.543+0000 7f65c77b7740 1 -- 172.21.15.49:0/2829838223 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f65b0011250 con 0x562ed5f01ef0 2024-03-18T23:14:56.549 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:56.543+0000 7f65a5ffb640 1 -- 172.21.15.49:0/2829838223 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f65a80055c0 con 0x562ed5f01ef0 2024-03-18T23:14:56.549 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:56.543+0000 7f65c77b7740 10 client.15993 fetch_fsmap learned FSMap version 32 2024-03-18T23:14:56.549 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:56.543+0000 7f65c77b7740 1 -- 172.21.15.49:0/2829838223 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x562ed5b7a210 con 0x562ed5f01ef0 2024-03-18T23:14:56.549 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:56.543+0000 7f65a5ffb640 1 -- 172.21.15.49:0/2829838223 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f65b0011250 con 0x562ed5f01ef0 2024-03-18T23:14:56.550 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:56.543+0000 7f65a5ffb640 1 -- 172.21.15.49:0/2829838223 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f65a801f450 con 0x562ed5f01ef0 2024-03-18T23:14:56.550 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:56.543+0000 7f65c77b7740 10 client.15993 fetch_fsmap finished waiting for FSMap version 32 2024-03-18T23:14:56.550 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:56.543+0000 7f65c77b7740 10 client.15993 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T23:14:56.550 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:56.543+0000 7f65c77b7740 20 client.15993 populate_metadata read hostname 'smithi049' 2024-03-18T23:14:56.550 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:56.543+0000 7f65c77b7740 1 --2- 172.21.15.49:0/2829838223 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x562ed5f1b550 0x562ed5f1d980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:14:56.550 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:56.543+0000 7f65c77b7740 4 client.15993 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T23:14:56.550 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:56.543+0000 7f65c77b7740 1 -- 172.21.15.49:0/2829838223 --> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562ed5e73120 con 0x562ed5f1b550 2024-03-18T23:14:56.550 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:56.543+0000 7f65be59c640 1 --2- 172.21.15.49:0/2829838223 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x562ed5f1b550 0x562ed5f1d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:14:56.550 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:56.544+0000 7f65be59c640 1 --2- 172.21.15.49:0/2829838223 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x562ed5f1b550 0x562ed5f1d980 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=839b558587ac5ba1 server_cookie=17a9a30e2579e79e in_seq=0 out_seq=0 2024-03-18T23:14:56.550 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:56.544+0000 7f65a5ffb640 10 client.15993 ms_handle_connect on v2:172.21.15.139:6838/4280128209 2024-03-18T23:14:56.551 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:56.545+0000 7f65a5ffb640 1 -- 172.21.15.49:0/2829838223 <== mds.0 v2:172.21.15.139:6838/4280128209 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x562ed5e73120 con 0x562ed5f1b550 2024-03-18T23:14:56.551 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:56.545+0000 7f65a5ffb640 10 client.15993 handle_command_reply: tid=0 2024-03-18T23:14:56.551 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:56.545+0000 7f65c77b7740 1 client.15993 shutdown 2024-03-18T23:14:56.552 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:56.545+0000 7f65c77b7740 1 -- 172.21.15.49:0/2829838223 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562ed5f01ef0 msgr2=0x562ed5f0b020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:14:56.552 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:56.545+0000 7f65c77b7740 1 --2- 172.21.15.49:0/2829838223 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562ed5f01ef0 0x562ed5f0b020 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f65a8004540 tx=0x7f65a8009ac0 comp rx=0 tx=0).stop 2024-03-18T23:14:56.552 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:56.545+0000 7f65c77b7740 1 -- 172.21.15.49:0/2829838223 shutdown_connections 2024-03-18T23:14:56.552 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:56.545+0000 7f65c77b7740 1 --2- 172.21.15.49:0/2829838223 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x562ed5f1b550 0x562ed5f1d980 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:56.552 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:56.545+0000 7f65c77b7740 1 --2- 172.21.15.49:0/2829838223 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562ed5f0b5e0 0x562ed5f08af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:56.552 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:56.545+0000 7f65c77b7740 1 --2- 172.21.15.49:0/2829838223 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562ed5f02810 0x562ed5f083b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:56.552 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:56.545+0000 7f65c77b7740 1 --2- 172.21.15.49:0/2829838223 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562ed5f01ef0 0x562ed5f0b020 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:56.552 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:56.545+0000 7f65a5ffb640 0 client.15993 ms_handle_reset on v2:172.21.15.139:6838/4280128209 2024-03-18T23:14:56.552 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:56.545+0000 7f65c77b7740 1 -- 172.21.15.49:0/2829838223 >> 172.21.15.49:0/2829838223 conn(0x562ed5efc810 msgr2=0x562ed5efbab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:14:56.552 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:56.545+0000 7f65c77b7740 1 -- 172.21.15.49:0/2829838223 shutdown_connections 2024-03-18T23:14:56.552 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:56.545+0000 7f65c77b7740 1 -- 172.21.15.49:0/2829838223 wait complete. 2024-03-18T23:14:56.552 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:56.545+0000 7f65c77b7740 20 client.15993 trim_cache size 0 max 16384 2024-03-18T23:14:56.562 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:56.560+0000 7f65c77b7740 1 Processor -- start 2024-03-18T23:14:56.562 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:56.560+0000 7f65c77b7740 1 -- start start 2024-03-18T23:14:56.562 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:56.560+0000 7f65c77b7740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562ed5f01ef0 0x562ed5f16fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:14:56.562 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:56.560+0000 7f65c77b7740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562ed5f02810 0x562ed5f23240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:14:56.562 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:56.560+0000 7f65c77b7740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562ed5f0b5e0 0x562ed5f0aae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:14:56.562 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:56.560+0000 7f65c77b7740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f65c01ad430 con 0x562ed5f02810 2024-03-18T23:14:56.562 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:56.560+0000 7f65c77b7740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f65c01aef70 con 0x562ed5f0b5e0 2024-03-18T23:14:56.562 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:56.560+0000 7f65c77b7740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f65c01adb00 con 0x562ed5f01ef0 2024-03-18T23:14:56.562 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:56.560+0000 7f65be59c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562ed5f02810 0x562ed5f23240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:14:56.562 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:56.561+0000 7f65be59c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562ed5f02810 0x562ed5f23240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:43848/0 (socket says 172.21.15.49:43848) 2024-03-18T23:14:56.562 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:56.561+0000 7f65be59c640 1 -- 172.21.15.49:0/1537820256 learned_addr learned my addr 172.21.15.49:0/1537820256 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:14:56.562 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:56.561+0000 7f65be59c640 1 -- 172.21.15.49:0/1537820256 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562ed5f01ef0 msgr2=0x562ed5f16fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:14:56.563 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:56.561+0000 7f65be59c640 1 --2- 172.21.15.49:0/1537820256 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562ed5f01ef0 0x562ed5f16fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:56.563 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:56.561+0000 7f65be59c640 1 -- 172.21.15.49:0/1537820256 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562ed5f0b5e0 msgr2=0x562ed5f0aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:14:56.563 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:56.561+0000 7f65be59c640 1 --2- 172.21.15.49:0/1537820256 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562ed5f0b5e0 0x562ed5f0aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:56.563 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:56.561+0000 7f65be59c640 1 -- 172.21.15.49:0/1537820256 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562ed5f08e60 con 0x562ed5f02810 2024-03-18T23:14:56.563 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:56.561+0000 7f65be59c640 1 --2- 172.21.15.49:0/1537820256 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562ed5f02810 0x562ed5f23240 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f65b0009730 tx=0x7f65b00194d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad058a8e5983b75c server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:14:56.563 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:56.561+0000 7f65a7fff640 1 -- 172.21.15.49:0/1537820256 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65b001e040 con 0x562ed5f02810 2024-03-18T23:14:56.563 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:56.561+0000 7f65a7fff640 1 -- 172.21.15.49:0/1537820256 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f65b00055a0 con 0x562ed5f02810 2024-03-18T23:14:56.563 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:56.561+0000 7f65a7fff640 1 -- 172.21.15.49:0/1537820256 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65b0019630 con 0x562ed5f02810 2024-03-18T23:14:56.567 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:56.562+0000 7f65c77b7740 1 -- 172.21.15.49:0/1537820256 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562ed5f02810 msgr2=0x562ed5f23240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:14:56.567 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:56.562+0000 7f65c77b7740 1 --2- 172.21.15.49:0/1537820256 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562ed5f02810 0x562ed5f23240 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f65b0009730 tx=0x7f65b00194d0 comp rx=0 tx=0).stop 2024-03-18T23:14:56.567 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:56.562+0000 7f65c77b7740 1 -- 172.21.15.49:0/1537820256 shutdown_connections 2024-03-18T23:14:56.567 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:56.562+0000 7f65c77b7740 1 --2- 172.21.15.49:0/1537820256 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562ed5f0b5e0 0x562ed5f0aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:56.567 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:56.562+0000 7f65c77b7740 1 --2- 172.21.15.49:0/1537820256 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562ed5f02810 0x562ed5f23240 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:56.567 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:56.562+0000 7f65c77b7740 1 --2- 172.21.15.49:0/1537820256 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562ed5f01ef0 0x562ed5f16fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:56.567 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:56.562+0000 7f65c77b7740 1 -- 172.21.15.49:0/1537820256 >> 172.21.15.49:0/1537820256 conn(0x562ed5efc810 msgr2=0x562ed5efbab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:14:56.567 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:56.562+0000 7f65c77b7740 1 -- 172.21.15.49:0/1537820256 shutdown_connections 2024-03-18T23:14:56.567 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:56.562+0000 7f65c77b7740 1 -- 172.21.15.49:0/1537820256 wait complete. 2024-03-18T23:14:56.567 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:56.562+0000 7f65c77b7740 1 -- start start 2024-03-18T23:14:56.567 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:56.562+0000 7f65c77b7740 1 Processor -- start 2024-03-18T23:14:56.567 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:56.562+0000 7f65c77b7740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562ed5f01ef0 0x562ed5f0f240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:14:56.568 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:56.562+0000 7f65c77b7740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562ed5f02810 0x562ed5f0f780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:14:56.568 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:56.563+0000 7f65c77b7740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562ed5f0b5e0 0x562ed5f84c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:14:56.568 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:56.563+0000 7f65c77b7740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f65c01ae8a0 con 0x562ed5f0b5e0 2024-03-18T23:14:56.568 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:56.563+0000 7f65c77b7740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f65c01ae1d0 con 0x562ed5f01ef0 2024-03-18T23:14:56.568 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:56.563+0000 7f65c77b7740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f65c01ab3f0 con 0x562ed5f02810 2024-03-18T23:14:56.568 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:56.563+0000 7f65c77b7740 10 client.0 _ll_register_callbacks cb 0x7f65a8009ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T23:14:56.568 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:56.563+0000 7f65b7fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562ed5f0b5e0 0x562ed5f84c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:14:56.568 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:56.563+0000 7f65b7fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562ed5f0b5e0 0x562ed5f84c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:43858/0 (socket says 172.21.15.49:43858) 2024-03-18T23:14:56.568 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:56.563+0000 7f65be59c640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562ed5f02810 0x562ed5f0f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:14:56.568 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:56.563+0000 7f65b7fff640 1 -- 172.21.15.49:0/932982803 learned_addr learned my addr 172.21.15.49:0/932982803 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:14:56.568 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:56.563+0000 7f65b7fff640 1 --2- 172.21.15.49:0/932982803 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562ed5f01ef0 0x562ed5f0f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:14:56.568 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:56.563+0000 7f65b7fff640 1 -- 172.21.15.49:0/932982803 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562ed5f02810 msgr2=0x562ed5f0f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:14:56.568 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:56.563+0000 7f65b7fff640 1 --2- 172.21.15.49:0/932982803 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562ed5f02810 0x562ed5f0f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:56.569 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:56.563+0000 7f65b7fff640 1 -- 172.21.15.49:0/932982803 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562ed5f01ef0 msgr2=0x562ed5f0f240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:14:56.569 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:56.563+0000 7f65b7fff640 1 --2- 172.21.15.49:0/932982803 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562ed5f01ef0 0x562ed5f0f240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:56.569 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:56.563+0000 7f65b7fff640 1 -- 172.21.15.49:0/932982803 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562ed5c46620 con 0x562ed5f0b5e0 2024-03-18T23:14:56.569 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:56.563+0000 7f65be59c640 1 --2- 172.21.15.49:0/932982803 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562ed5f02810 0x562ed5f0f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:14:56.569 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:56.563+0000 7f65b7fff640 1 --2- 172.21.15.49:0/932982803 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562ed5f0b5e0 0x562ed5f84c80 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f65a8009730 tx=0x7f65a8009ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d62ae08fd964ffd server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:14:56.569 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:56.563+0000 7f65a5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-18T23:14:56.569 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:56.563+0000 7f65a5ffb640 1 -- 172.21.15.49:0/932982803 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65a802c030 con 0x562ed5f0b5e0 2024-03-18T23:14:56.569 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:56.563+0000 7f65a5ffb640 1 -- 172.21.15.49:0/932982803 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8512+0+0 (secure 0 0 0) 0x7f65a8011cf0 con 0x562ed5f0b5e0 2024-03-18T23:14:56.569 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:56.563+0000 7f65c77b7740 1 -- 172.21.15.49:0/932982803 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f65b00055a0 con 0x562ed5f0b5e0 2024-03-18T23:14:56.569 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:56.564+0000 7f65a5ffb640 1 -- 172.21.15.49:0/932982803 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f65a801f9f0 con 0x562ed5f0b5e0 2024-03-18T23:14:56.569 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:56.564+0000 7f65c77b7740 10 client.15999 fetch_fsmap learned FSMap version 32 2024-03-18T23:14:56.569 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:56.564+0000 7f65c77b7740 1 -- 172.21.15.49:0/932982803 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x562ed5f86100 con 0x562ed5f0b5e0 2024-03-18T23:14:56.569 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:56.564+0000 7f65a5ffb640 1 -- 172.21.15.49:0/932982803 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f65b00055a0 con 0x562ed5f0b5e0 2024-03-18T23:14:56.569 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:56.564+0000 7f65a5ffb640 1 -- 172.21.15.49:0/932982803 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f65a8020510 con 0x562ed5f0b5e0 2024-03-18T23:14:56.569 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:56.564+0000 7f65c77b7740 10 client.15999 fetch_fsmap finished waiting for FSMap version 32 2024-03-18T23:14:56.570 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:56.564+0000 7f65c77b7740 10 client.15999 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T23:14:56.570 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:56.564+0000 7f65c77b7740 20 client.15999 populate_metadata read hostname 'smithi049' 2024-03-18T23:14:56.570 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:56.564+0000 7f65c77b7740 1 --2- 172.21.15.49:0/932982803 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x562ed5f86820 0x562ed5f88c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:14:56.570 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:56.564+0000 7f65c77b7740 4 client.15999 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T23:14:56.570 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:56.564+0000 7f65c77b7740 1 -- 172.21.15.49:0/932982803 --> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562ed5e6d470 con 0x562ed5f86820 2024-03-18T23:14:56.570 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:56.564+0000 7f65be59c640 1 --2- 172.21.15.49:0/932982803 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x562ed5f86820 0x562ed5f88c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:14:56.570 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:56.565+0000 7f65be59c640 1 --2- 172.21.15.49:0/932982803 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x562ed5f86820 0x562ed5f88c10 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=21e7d3ceee668b44 server_cookie=462ed618c119bb2b in_seq=0 out_seq=0 2024-03-18T23:14:56.570 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:56.565+0000 7f65a5ffb640 10 client.15999 ms_handle_connect on v2:172.21.15.139:6838/4280128209 2024-03-18T23:14:56.716 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:56.714+0000 7f65a5ffb640 1 -- 172.21.15.49:0/932982803 <== mds.0 v2:172.21.15.139:6838/4280128209 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x562ed5e6d470 con 0x562ed5f86820 2024-03-18T23:14:56.716 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:56.714+0000 7f65a5ffb640 10 client.15999 handle_command_reply: tid=0 2024-03-18T23:14:56.716 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:56.714+0000 7f65c77b7740 1 client.15999 shutdown 2024-03-18T23:14:56.716 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:56.714+0000 7f65c77b7740 1 -- 172.21.15.49:0/932982803 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562ed5f0b5e0 msgr2=0x562ed5f84c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:14:56.716 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:56.715+0000 7f65c77b7740 1 --2- 172.21.15.49:0/932982803 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562ed5f0b5e0 0x562ed5f84c80 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f65a8009730 tx=0x7f65a8009ac0 comp rx=0 tx=0).stop 2024-03-18T23:14:56.716 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:56.715+0000 7f65c77b7740 1 -- 172.21.15.49:0/932982803 shutdown_connections 2024-03-18T23:14:56.716 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:56.715+0000 7f65c77b7740 1 --2- 172.21.15.49:0/932982803 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x562ed5f86820 0x562ed5f88c10 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:56.716 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:56.715+0000 7f65c77b7740 1 --2- 172.21.15.49:0/932982803 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562ed5f0b5e0 0x562ed5f84c80 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:56.716 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:56.715+0000 7f65a5ffb640 0 client.15999 ms_handle_reset on v2:172.21.15.139:6838/4280128209 2024-03-18T23:14:56.716 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:56.715+0000 7f65c77b7740 1 --2- 172.21.15.49:0/932982803 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562ed5f02810 0x562ed5f0f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:56.717 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:56.715+0000 7f65c77b7740 1 --2- 172.21.15.49:0/932982803 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562ed5f01ef0 0x562ed5f0f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:56.717 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:56.715+0000 7f65c77b7740 1 -- 172.21.15.49:0/932982803 >> 172.21.15.49:0/932982803 conn(0x562ed5efc810 msgr2=0x562ed5f02ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:14:56.717 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:56.715+0000 7f65c77b7740 1 -- 172.21.15.49:0/932982803 shutdown_connections 2024-03-18T23:14:56.717 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:56.715+0000 7f65c77b7740 1 -- 172.21.15.49:0/932982803 wait complete. 2024-03-18T23:14:56.717 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:56.715+0000 7f65c77b7740 20 client.15999 trim_cache size 0 max 16384 2024-03-18T23:14:56.717 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:{ 2024-03-18T23:14:56.717 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "message": "", 2024-03-18T23:14:56.717 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "return_code": 0 2024-03-18T23:14:56.717 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:} 2024-03-18T23:14:56.718 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:56.716+0000 7f65c4bd3640 1 -- 172.21.15.49:0/887764718 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f65ac0a2a10 msgr2=0x7f65ac0a4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:14:56.718 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:56.716+0000 7f65c4bd3640 1 --2- 172.21.15.49:0/887764718 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f65ac0a2a10 0x7f65ac0a4ed0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f65b8009430 tx=0x7f65b802c0f0 comp rx=0 tx=0).stop 2024-03-18T23:14:56.718 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:56.716+0000 7f65c4bd3640 1 -- 172.21.15.49:0/887764718 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f65c01056c0 msgr2=0x7f65c01a3f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:14:56.719 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:56.716+0000 7f65c4bd3640 1 --2- 172.21.15.49:0/887764718 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f65c01056c0 0x7f65c01a3f20 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f65b802c700 tx=0x7f65b8032f20 comp rx=0 tx=0).stop 2024-03-18T23:14:56.719 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:56.716+0000 7f65c4bd3640 1 -- 172.21.15.49:0/887764718 shutdown_connections 2024-03-18T23:14:56.719 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:56.716+0000 7f65c4bd3640 1 --2- 172.21.15.49:0/887764718 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f65ac0a2a10 0x7f65ac0a4ed0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:56.719 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:56.716+0000 7f65c4bd3640 1 --2- 172.21.15.49:0/887764718 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f65c01056c0 0x7f65c01a3f20 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:56.719 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:56.716+0000 7f65c4bd3640 1 --2- 172.21.15.49:0/887764718 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f65c0101330 0x7f65c01993e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:56.719 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:56.716+0000 7f65c4bd3640 1 --2- 172.21.15.49:0/887764718 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f65c0100970 0x7f65c0198ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:56.719 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:56.716+0000 7f65c4bd3640 1 -- 172.21.15.49:0/887764718 >> 172.21.15.49:0/887764718 conn(0x7f65c0077570 msgr2=0x7f65c00fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:14:56.719 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:56.716+0000 7f65c4bd3640 1 -- 172.21.15.49:0/887764718 shutdown_connections 2024-03-18T23:14:56.719 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:14:56.716+0000 7f65c4bd3640 1 -- 172.21.15.49:0/887764718 wait complete. 2024-03-18T23:14:56.775 INFO:tasks.workunit.client.0.smithi049.stdout: 1 723311 10.11 MB/sec execute 238 sec latency 82.423 ms 2024-03-18T23:14:57.379 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:14:57 smithi049 ceph-mon[62991]: pgmap v205: 97 pgs: 97 active+clean; 82 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.7 MiB/s wr, 264 op/s 2024-03-18T23:14:57.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:14:57 smithi175 ceph-mon[67352]: pgmap v205: 97 pgs: 97 active+clean; 82 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.7 MiB/s wr, 264 op/s 2024-03-18T23:14:57.564 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:14:57 smithi139 ceph-mon[67282]: pgmap v205: 97 pgs: 97 active+clean; 82 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.7 MiB/s wr, 264 op/s 2024-03-18T23:14:57.775 INFO:tasks.workunit.client.0.smithi049.stdout: 1 724908 10.11 MB/sec execute 239 sec latency 47.214 ms 2024-03-18T23:14:58.775 INFO:tasks.workunit.client.0.smithi049.stdout: 1 726764 10.10 MB/sec execute 240 sec latency 20.746 ms 2024-03-18T23:14:59.379 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:14:59 smithi049 ceph-mon[62991]: pgmap v206: 97 pgs: 97 active+clean; 88 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 4.1 MiB/s wr, 357 op/s 2024-03-18T23:14:59.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:14:59 smithi175 ceph-mon[67352]: pgmap v206: 97 pgs: 97 active+clean; 88 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 4.1 MiB/s wr, 357 op/s 2024-03-18T23:14:59.564 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:14:59 smithi139 ceph-mon[67282]: pgmap v206: 97 pgs: 97 active+clean; 88 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 4.1 MiB/s wr, 357 op/s 2024-03-18T23:14:59.775 INFO:tasks.workunit.client.0.smithi049.stdout: 1 728552 10.11 MB/sec execute 241 sec latency 16.823 ms 2024-03-18T23:15:00.775 INFO:tasks.workunit.client.0.smithi049.stdout: 1 730219 10.09 MB/sec execute 242 sec latency 26.582 ms 2024-03-18T23:15:01.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:15:01 smithi175 ceph-mon[67352]: pgmap v207: 97 pgs: 97 active+clean; 88 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 2.7 MiB/s wr, 183 op/s 2024-03-18T23:15:01.564 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:15:01 smithi139 ceph-mon[67282]: pgmap v207: 97 pgs: 97 active+clean; 88 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 2.7 MiB/s wr, 183 op/s 2024-03-18T23:15:01.629 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:15:01 smithi049 ceph-mon[62991]: pgmap v207: 97 pgs: 97 active+clean; 88 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 2.7 MiB/s wr, 183 op/s 2024-03-18T23:15:01.775 INFO:tasks.workunit.client.0.smithi049.stdout: 1 730866 10.08 MB/sec execute 243 sec latency 43.148 ms 2024-03-18T23:15:02.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:15:02 smithi175 ceph-mon[67352]: osdmap e92: 12 total, 12 up, 12 in 2024-03-18T23:15:02.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:15:02 smithi139 ceph-mon[67282]: osdmap e92: 12 total, 12 up, 12 in 2024-03-18T23:15:02.629 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:15:02 smithi049 ceph-mon[62991]: osdmap e92: 12 total, 12 up, 12 in 2024-03-18T23:15:02.775 INFO:tasks.workunit.client.0.smithi049.stdout: 1 731524 10.06 MB/sec execute 244 sec latency 91.212 ms 2024-03-18T23:15:03.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:15:03 smithi175 ceph-mon[67352]: pgmap v209: 97 pgs: 23 active+clean+snaptrim, 33 active+clean, 41 active+clean+snaptrim_wait; 73 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.3 MiB/s wr, 353 op/s 2024-03-18T23:15:03.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:15:03 smithi139 ceph-mon[67282]: pgmap v209: 97 pgs: 23 active+clean+snaptrim, 33 active+clean, 41 active+clean+snaptrim_wait; 73 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.3 MiB/s wr, 353 op/s 2024-03-18T23:15:03.629 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:15:03 smithi049 ceph-mon[62991]: pgmap v209: 97 pgs: 23 active+clean+snaptrim, 33 active+clean, 41 active+clean+snaptrim_wait; 73 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.3 MiB/s wr, 353 op/s 2024-03-18T23:15:03.775 INFO:tasks.workunit.client.0.smithi049.stdout: 1 732398 10.03 MB/sec execute 245 sec latency 61.286 ms 2024-03-18T23:15:04.775 INFO:tasks.workunit.client.0.smithi049.stdout: 1 734511 10.03 MB/sec execute 246 sec latency 18.199 ms 2024-03-18T23:15:05.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:15:05 smithi175 ceph-mon[67352]: pgmap v210: 97 pgs: 23 active+clean+snaptrim, 33 active+clean, 41 active+clean+snaptrim_wait; 73 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.3 MiB/s wr, 353 op/s 2024-03-18T23:15:05.564 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:15:05 smithi139 ceph-mon[67282]: pgmap v210: 97 pgs: 23 active+clean+snaptrim, 33 active+clean, 41 active+clean+snaptrim_wait; 73 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.3 MiB/s wr, 353 op/s 2024-03-18T23:15:05.629 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:15:05 smithi049 ceph-mon[62991]: pgmap v210: 97 pgs: 23 active+clean+snaptrim, 33 active+clean, 41 active+clean+snaptrim_wait; 73 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.3 MiB/s wr, 353 op/s 2024-03-18T23:15:05.775 INFO:tasks.workunit.client.0.smithi049.stdout: 1 736224 10.03 MB/sec execute 247 sec latency 23.062 ms 2024-03-18T23:15:06.775 INFO:tasks.workunit.client.0.smithi049.stdout: 1 738413 10.04 MB/sec execute 248 sec latency 22.017 ms 2024-03-18T23:15:07.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:15:07 smithi175 ceph-mon[67352]: pgmap v211: 97 pgs: 23 active+clean+snaptrim, 33 active+clean, 41 active+clean+snaptrim_wait; 73 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.3 MiB/s wr, 353 op/s 2024-03-18T23:15:07.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:15:07 smithi139 ceph-mon[67282]: pgmap v211: 97 pgs: 23 active+clean+snaptrim, 33 active+clean, 41 active+clean+snaptrim_wait; 73 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.3 MiB/s wr, 353 op/s 2024-03-18T23:15:07.629 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:15:07 smithi049 ceph-mon[62991]: pgmap v211: 97 pgs: 23 active+clean+snaptrim, 33 active+clean, 41 active+clean+snaptrim_wait; 73 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.3 MiB/s wr, 353 op/s 2024-03-18T23:15:07.775 INFO:tasks.workunit.client.0.smithi049.stdout: 1 739807 10.02 MB/sec execute 249 sec latency 32.286 ms 2024-03-18T23:15:08.775 INFO:tasks.workunit.client.0.smithi049.stdout: 1 741576 10.03 MB/sec execute 250 sec latency 59.813 ms 2024-03-18T23:15:09.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:15:09 smithi175 ceph-mon[67352]: pgmap v212: 97 pgs: 97 active+clean; 77 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.1 MiB/s wr, 330 op/s 2024-03-18T23:15:09.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:15:09 smithi175 ceph-mon[67352]: osdmap e93: 12 total, 12 up, 12 in 2024-03-18T23:15:09.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:15:09 smithi139 ceph-mon[67282]: pgmap v212: 97 pgs: 97 active+clean; 77 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.1 MiB/s wr, 330 op/s 2024-03-18T23:15:09.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:15:09 smithi139 ceph-mon[67282]: osdmap e93: 12 total, 12 up, 12 in 2024-03-18T23:15:09.629 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:15:09 smithi049 ceph-mon[62991]: pgmap v212: 97 pgs: 97 active+clean; 77 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.1 MiB/s wr, 330 op/s 2024-03-18T23:15:09.629 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:15:09 smithi049 ceph-mon[62991]: osdmap e93: 12 total, 12 up, 12 in 2024-03-18T23:15:09.775 INFO:tasks.workunit.client.0.smithi049.stdout: 1 743232 10.02 MB/sec execute 251 sec latency 26.188 ms 2024-03-18T23:15:09.818 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:09.819+0000 7f49c11cd640 1 -- 172.21.15.49:0/3366657974 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f49bc101330 msgr2=0x7f49bc10d550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:15:09.818 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:09.819+0000 7f49c11cd640 1 --2- 172.21.15.49:0/3366657974 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f49bc101330 0x7f49bc10d550 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f49b4009ca0 tx=0x7f49b402c0f0 comp rx=0 tx=0).stop 2024-03-18T23:15:09.819 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:09.819+0000 7f49c11cd640 1 -- 172.21.15.49:0/3366657974 shutdown_connections 2024-03-18T23:15:09.819 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:09.819+0000 7f49c11cd640 1 --2- 172.21.15.49:0/3366657974 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f49bc101330 0x7f49bc10d550 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:09.819 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:09.819+0000 7f49c11cd640 1 --2- 172.21.15.49:0/3366657974 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f49bc100970 0x7f49bc100df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:09.819 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:09.819+0000 7f49c11cd640 1 --2- 172.21.15.49:0/3366657974 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f49bc1056c0 0x7f49bc105aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:09.819 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:09.819+0000 7f49c11cd640 1 -- 172.21.15.49:0/3366657974 >> 172.21.15.49:0/3366657974 conn(0x7f49bc077570 msgr2=0x7f49bc077980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:15:09.819 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:09.819+0000 7f49c11cd640 1 -- 172.21.15.49:0/3366657974 shutdown_connections 2024-03-18T23:15:09.819 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:09.819+0000 7f49c11cd640 1 -- 172.21.15.49:0/3366657974 wait complete. 2024-03-18T23:15:09.821 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:09.820+0000 7f49c11cd640 1 Processor -- start 2024-03-18T23:15:09.821 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:09.820+0000 7f49c11cd640 1 -- start start 2024-03-18T23:15:09.821 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:09.820+0000 7f49c11cd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f49bc100970 0x7f49bc102390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:15:09.821 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:09.820+0000 7f49c11cd640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f49bc101330 0x7f49bc1028d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:15:09.821 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:09.820+0000 7f49c11cd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f49bc1056c0 0x7f49bc102e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:15:09.821 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:09.820+0000 7f49c11cd640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f49bc110be0 con 0x7f49bc100970 2024-03-18T23:15:09.821 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:09.820+0000 7f49c11cd640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f49bc110a60 con 0x7f49bc1056c0 2024-03-18T23:15:09.822 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:09.820+0000 7f49c11cd640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f49bc110d60 con 0x7f49bc101330 2024-03-18T23:15:09.822 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:09.820+0000 7f49bad9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f49bc100970 0x7f49bc102390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:15:09.822 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:09.820+0000 7f49ba59c640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f49bc101330 0x7f49bc1028d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:15:09.822 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:09.820+0000 7f49bad9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f49bc100970 0x7f49bc102390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53142/0 (socket says 172.21.15.49:53142) 2024-03-18T23:15:09.822 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:09.820+0000 7f49bad9d640 1 -- 172.21.15.49:0/1307161653 learned_addr learned my addr 172.21.15.49:0/1307161653 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:15:09.822 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:09.820+0000 7f49bb59e640 1 --2- 172.21.15.49:0/1307161653 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f49bc1056c0 0x7f49bc102e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:15:09.822 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:09.820+0000 7f49bad9d640 1 -- 172.21.15.49:0/1307161653 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f49bc101330 msgr2=0x7f49bc1028d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:15:09.822 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:09.820+0000 7f49bad9d640 1 --2- 172.21.15.49:0/1307161653 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f49bc101330 0x7f49bc1028d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:09.822 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:09.820+0000 7f49bad9d640 1 -- 172.21.15.49:0/1307161653 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f49bc1056c0 msgr2=0x7f49bc102e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:15:09.822 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:09.820+0000 7f49bad9d640 1 --2- 172.21.15.49:0/1307161653 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f49bc1056c0 0x7f49bc102e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:09.822 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:09.820+0000 7f49bad9d640 1 -- 172.21.15.49:0/1307161653 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49bc1a6570 con 0x7f49bc100970 2024-03-18T23:15:09.822 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:09.820+0000 7f49ba59c640 1 --2- 172.21.15.49:0/1307161653 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f49bc101330 0x7f49bc1028d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:15:09.822 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:09.821+0000 7f49bad9d640 1 --2- 172.21.15.49:0/1307161653 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f49bc100970 0x7f49bc102390 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f49a400e9f0 tx=0x7f49a400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:15:09.823 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:09.821+0000 7f49b3fff640 1 -- 172.21.15.49:0/1307161653 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49a400cd80 con 0x7f49bc100970 2024-03-18T23:15:09.823 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:09.821+0000 7f49c11cd640 1 -- 172.21.15.49:0/1307161653 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49bc1a6860 con 0x7f49bc100970 2024-03-18T23:15:09.823 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:09.821+0000 7f49b3fff640 1 -- 172.21.15.49:0/1307161653 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f49a4005500 con 0x7f49bc100970 2024-03-18T23:15:09.823 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:09.821+0000 7f49b3fff640 1 -- 172.21.15.49:0/1307161653 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49a4010690 con 0x7f49bc100970 2024-03-18T23:15:09.823 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:09.821+0000 7f49c11cd640 1 -- 172.21.15.49:0/1307161653 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49bc1a6db0 con 0x7f49bc100970 2024-03-18T23:15:09.829 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:09.823+0000 7f49b3fff640 1 -- 172.21.15.49:0/1307161653 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 25) ==== 137421+0+0 (secure 0 0 0) 0x7f49a40046c0 con 0x7f49bc100970 2024-03-18T23:15:09.829 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:09.823+0000 7f49c3db0740 1 Processor -- start 2024-03-18T23:15:09.829 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:09.823+0000 7f49c3db0740 1 -- start start 2024-03-18T23:15:09.830 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:09.824+0000 7f49c3db0740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564bef795ef0 0x564bef7962d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:15:09.830 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:09.824+0000 7f49c3db0740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564bef796810 0x564bef79f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:15:09.830 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:09.824+0000 7f49c3db0740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564bef79f5e0 0x564bef7a19d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:15:09.830 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:09.824+0000 7f49c3db0740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x564bef6f9be0 con 0x564bef79f5e0 2024-03-18T23:15:09.830 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:09.824+0000 7f49c3db0740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x564bef6a2960 con 0x564bef795ef0 2024-03-18T23:15:09.830 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:09.824+0000 7f49c3db0740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x564bef6f5c80 con 0x564bef796810 2024-03-18T23:15:09.830 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:09.824+0000 7f49ba59c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564bef79f5e0 0x564bef7a19d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:15:09.830 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:09.824+0000 7f49bb59e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564bef795ef0 0x564bef7962d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:15:09.830 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:09.824+0000 7f49b3fff640 1 --2- 172.21.15.49:0/1307161653 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f49a80a2a10 0x7f49a80a4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:15:09.830 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:09.824+0000 7f49ba59c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564bef79f5e0 0x564bef7a19d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53158/0 (socket says 172.21.15.49:53158) 2024-03-18T23:15:09.830 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:09.824+0000 7f49ba59c640 1 -- 172.21.15.49:0/4066009805 learned_addr learned my addr 172.21.15.49:0/4066009805 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:15:09.830 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:09.824+0000 7f49bad9d640 1 --2- 172.21.15.49:0/4066009805 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564bef796810 0x564bef79f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:15:09.830 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:09.824+0000 7f49bb59e640 1 --2- 172.21.15.49:0/1307161653 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f49a80a2a10 0x7f49a80a4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:15:09.831 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:09.824+0000 7f49b3fff640 1 -- 172.21.15.49:0/1307161653 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7f49a40cf260 con 0x7f49bc100970 2024-03-18T23:15:09.831 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:09.824+0000 7f49ba59c640 1 -- 172.21.15.49:0/4066009805 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564bef796810 msgr2=0x564bef79f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:15:09.831 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:09.824+0000 7f49ba59c640 1 --2- 172.21.15.49:0/4066009805 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564bef796810 0x564bef79f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:09.831 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:09.824+0000 7f49ba59c640 1 -- 172.21.15.49:0/4066009805 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564bef795ef0 msgr2=0x564bef7962d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:15:09.831 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:09.824+0000 7f49ba59c640 1 --2- 172.21.15.49:0/4066009805 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564bef795ef0 0x564bef7962d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:09.831 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:09.824+0000 7f49ba59c640 1 -- 172.21.15.49:0/4066009805 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564bef699d20 con 0x564bef79f5e0 2024-03-18T23:15:09.831 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:09.824+0000 7f49bb59e640 1 --2- 172.21.15.49:0/4066009805 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564bef795ef0 0x564bef7962d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:15:09.831 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:09.825+0000 7f49ba59c640 1 --2- 172.21.15.49:0/4066009805 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564bef79f5e0 0x564bef7a19d0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f49ac0114f0 tx=0x7f49ac0119b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bb5a19bf435f65b server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:15:09.831 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:09.825+0000 7f49bb59e640 1 --2- 172.21.15.49:0/1307161653 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f49a80a2a10 0x7f49a80a4ed0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f49b4004af0 tx=0x7f49b402c0f0 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:15:09.831 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:09.825+0000 7f49b17fa640 1 -- 172.21.15.49:0/4066009805 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49ac01e070 con 0x564bef79f5e0 2024-03-18T23:15:09.831 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:09.825+0000 7f49b17fa640 1 -- 172.21.15.49:0/4066009805 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f49ac0035f0 con 0x564bef79f5e0 2024-03-18T23:15:09.831 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:09.825+0000 7f49b17fa640 1 -- 172.21.15.49:0/4066009805 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49ac00f820 con 0x564bef79f5e0 2024-03-18T23:15:09.831 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:09.826+0000 7f49c3db0740 1 -- 172.21.15.49:0/4066009805 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564bef79f5e0 msgr2=0x564bef7a19d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:15:09.831 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:09.826+0000 7f49c3db0740 1 --2- 172.21.15.49:0/4066009805 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564bef79f5e0 0x564bef7a19d0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f49ac0114f0 tx=0x7f49ac0119b0 comp rx=0 tx=0).stop 2024-03-18T23:15:09.831 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:09.826+0000 7f49c3db0740 1 -- 172.21.15.49:0/4066009805 shutdown_connections 2024-03-18T23:15:09.832 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:09.826+0000 7f49c3db0740 1 --2- 172.21.15.49:0/4066009805 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564bef79f5e0 0x564bef7a19d0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:09.832 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:09.826+0000 7f49c3db0740 1 --2- 172.21.15.49:0/4066009805 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564bef796810 0x564bef79f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:09.832 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:09.826+0000 7f49c3db0740 1 --2- 172.21.15.49:0/4066009805 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564bef795ef0 0x564bef7962d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:09.832 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:09.826+0000 7f49c3db0740 1 -- 172.21.15.49:0/4066009805 >> 172.21.15.49:0/4066009805 conn(0x564bef790810 msgr2=0x564bef790bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:15:09.832 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:09.826+0000 7f49c3db0740 1 -- 172.21.15.49:0/4066009805 shutdown_connections 2024-03-18T23:15:09.832 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:09.826+0000 7f49c3db0740 1 -- 172.21.15.49:0/4066009805 wait complete. 2024-03-18T23:15:09.832 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:09.826+0000 7f49c3db0740 1 -- start start 2024-03-18T23:15:09.832 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:09.826+0000 7f49c3db0740 1 Processor -- start 2024-03-18T23:15:09.832 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:09.826+0000 7f49c3db0740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564bef795ef0 0x564bef79efd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:15:09.832 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:09.826+0000 7f49c3db0740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564bef796810 0x564bef79c360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:15:09.832 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:09.826+0000 7f49c3db0740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564bef79f5e0 0x564bef79c970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:15:09.832 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:09.826+0000 7f49c3db0740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x564bef7a2210 con 0x564bef795ef0 2024-03-18T23:15:09.832 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:09.826+0000 7f49c3db0740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x564bef6b5db0 con 0x564bef79f5e0 2024-03-18T23:15:09.833 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:09.826+0000 7f49c3db0740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x564bef7a2390 con 0x564bef796810 2024-03-18T23:15:09.833 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:09.826+0000 7f49c3db0740 10 client.0 _ll_register_callbacks cb 0x564bef6bae70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T23:15:09.833 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:09.826+0000 7f49bad9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564bef795ef0 0x564bef79efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:15:09.833 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:09.827+0000 7f49bad9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564bef795ef0 0x564bef79efd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53166/0 (socket says 172.21.15.49:53166) 2024-03-18T23:15:09.833 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:09.827+0000 7f49ba59c640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564bef796810 0x564bef79c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:15:09.833 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:09.827+0000 7f49bad9d640 1 -- 172.21.15.49:0/2944784669 learned_addr learned my addr 172.21.15.49:0/2944784669 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:15:09.833 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:09.827+0000 7f49bb59e640 1 --2- 172.21.15.49:0/2944784669 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564bef79f5e0 0x564bef79c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:15:09.833 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:09.827+0000 7f49bad9d640 1 -- 172.21.15.49:0/2944784669 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564bef796810 msgr2=0x564bef79c360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:15:09.833 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:09.827+0000 7f49bad9d640 1 --2- 172.21.15.49:0/2944784669 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564bef796810 0x564bef79c360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:09.833 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:09.827+0000 7f49bad9d640 1 -- 172.21.15.49:0/2944784669 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564bef79f5e0 msgr2=0x564bef79c970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:15:09.833 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:09.827+0000 7f49bad9d640 1 --2- 172.21.15.49:0/2944784669 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564bef79f5e0 0x564bef79c970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:09.833 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:09.827+0000 7f49bad9d640 1 -- 172.21.15.49:0/2944784669 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564bef6b4c60 con 0x564bef795ef0 2024-03-18T23:15:09.833 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:09.827+0000 7f49bad9d640 1 --2- 172.21.15.49:0/2944784669 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564bef795ef0 0x564bef79efd0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f49a409b830 tx=0x7f49a409b670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3bbcfb78f97b718 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:15:09.833 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:09.827+0000 7f49a27fc640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-18T23:15:09.833 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:09.827+0000 7f49a27fc640 1 -- 172.21.15.49:0/2944784669 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49a409bfa0 con 0x564bef795ef0 2024-03-18T23:15:09.834 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:09.827+0000 7f49a27fc640 1 -- 172.21.15.49:0/2944784669 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7f49a409bb00 con 0x564bef795ef0 2024-03-18T23:15:09.834 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:09.827+0000 7f49c3db0740 1 -- 172.21.15.49:0/2944784669 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f49ac0035f0 con 0x564bef795ef0 2024-03-18T23:15:09.834 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:09.827+0000 7f49a27fc640 1 -- 172.21.15.49:0/2944784669 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f49a409b8a0 con 0x564bef795ef0 2024-03-18T23:15:09.834 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:09.827+0000 7f49c3db0740 10 client.16011 fetch_fsmap learned FSMap version 32 2024-03-18T23:15:09.834 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:09.827+0000 7f49c3db0740 1 -- 172.21.15.49:0/2944784669 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x564bef40e210 con 0x564bef795ef0 2024-03-18T23:15:09.834 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:09.827+0000 7f49a27fc640 1 -- 172.21.15.49:0/2944784669 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f49ac0035f0 con 0x564bef795ef0 2024-03-18T23:15:09.834 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:09.828+0000 7f49a27fc640 1 -- 172.21.15.49:0/2944784669 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f49a40a3bb0 con 0x564bef795ef0 2024-03-18T23:15:09.834 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:09.828+0000 7f49c3db0740 10 client.16011 fetch_fsmap finished waiting for FSMap version 32 2024-03-18T23:15:09.834 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:09.828+0000 7f49c3db0740 10 client.16011 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T23:15:09.834 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:09.828+0000 7f49c3db0740 20 client.16011 populate_metadata read hostname 'smithi049' 2024-03-18T23:15:09.834 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:09.828+0000 7f49c3db0740 1 --2- 172.21.15.49:0/2944784669 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x564bef7af500 0x564bef7b18f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:15:09.834 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:09.828+0000 7f49c3db0740 4 client.16011 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T23:15:09.834 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:09.828+0000 7f49c3db0740 1 -- 172.21.15.49:0/2944784669 --> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564bef707120 con 0x564bef7af500 2024-03-18T23:15:09.835 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:09.828+0000 7f49ba59c640 1 --2- 172.21.15.49:0/2944784669 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x564bef7af500 0x564bef7b18f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:15:09.835 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:09.828+0000 7f49a27fc640 10 client.16011 ms_handle_connect on v2:172.21.15.139:6838/4280128209 2024-03-18T23:15:09.835 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:09.828+0000 7f49ba59c640 1 --2- 172.21.15.49:0/2944784669 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x564bef7af500 0x564bef7b18f0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2f5d299ddd3a958e server_cookie=a786a0c1d1303576 in_seq=0 out_seq=0 2024-03-18T23:15:09.835 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:09.829+0000 7f49a27fc640 1 -- 172.21.15.49:0/2944784669 <== mds.0 v2:172.21.15.139:6838/4280128209 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x564bef707120 con 0x564bef7af500 2024-03-18T23:15:09.835 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:09.829+0000 7f49a27fc640 10 client.16011 handle_command_reply: tid=0 2024-03-18T23:15:09.835 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:09.829+0000 7f49c3db0740 1 client.16011 shutdown 2024-03-18T23:15:09.835 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:09.829+0000 7f49c3db0740 1 -- 172.21.15.49:0/2944784669 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564bef795ef0 msgr2=0x564bef79efd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:15:09.835 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:09.829+0000 7f49c3db0740 1 --2- 172.21.15.49:0/2944784669 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564bef795ef0 0x564bef79efd0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f49a409b830 tx=0x7f49a409b670 comp rx=0 tx=0).stop 2024-03-18T23:15:09.835 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:09.829+0000 7f49c3db0740 1 -- 172.21.15.49:0/2944784669 shutdown_connections 2024-03-18T23:15:09.835 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:09.829+0000 7f49c3db0740 1 --2- 172.21.15.49:0/2944784669 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x564bef7af500 0x564bef7b18f0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:09.835 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:09.829+0000 7f49c3db0740 1 --2- 172.21.15.49:0/2944784669 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564bef79f5e0 0x564bef79c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:09.835 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:09.829+0000 7f49c3db0740 1 --2- 172.21.15.49:0/2944784669 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564bef796810 0x564bef79c360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:09.835 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:09.829+0000 7f49a27fc640 0 client.16011 ms_handle_reset on v2:172.21.15.139:6838/4280128209 2024-03-18T23:15:09.836 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:09.829+0000 7f49c3db0740 1 --2- 172.21.15.49:0/2944784669 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564bef795ef0 0x564bef79efd0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:09.836 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:09.829+0000 7f49c3db0740 1 -- 172.21.15.49:0/2944784669 >> 172.21.15.49:0/2944784669 conn(0x564bef790810 msgr2=0x564bef78fab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:15:09.836 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:09.829+0000 7f49c3db0740 1 -- 172.21.15.49:0/2944784669 shutdown_connections 2024-03-18T23:15:09.836 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:09.829+0000 7f49c3db0740 1 -- 172.21.15.49:0/2944784669 wait complete. 2024-03-18T23:15:09.836 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:09.830+0000 7f49c3db0740 20 client.16011 trim_cache size 0 max 16384 2024-03-18T23:15:09.846 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:09.845+0000 7f49c3db0740 1 Processor -- start 2024-03-18T23:15:09.846 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:09.845+0000 7f49c3db0740 1 -- start start 2024-03-18T23:15:09.846 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:09.845+0000 7f49c3db0740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564bef795ef0 0x564bef7aafb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:15:09.846 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:09.845+0000 7f49c3db0740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564bef796810 0x564bef7b70c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:15:09.846 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:09.845+0000 7f49c3db0740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564bef79f5e0 0x564bef79ea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:15:09.846 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:09.845+0000 7f49c3db0740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f49bc1af640 con 0x564bef79f5e0 2024-03-18T23:15:09.846 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:09.845+0000 7f49c3db0740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f49bc1b1180 con 0x564bef796810 2024-03-18T23:15:09.846 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:09.845+0000 7f49c3db0740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f49bc1afd10 con 0x564bef795ef0 2024-03-18T23:15:09.846 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:09.845+0000 7f49bb59e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564bef79f5e0 0x564bef79ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:15:09.846 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:09.846+0000 7f49bad9d640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564bef795ef0 0x564bef7aafb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:15:09.847 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:09.846+0000 7f49bb59e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564bef79f5e0 0x564bef79ea90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53172/0 (socket says 172.21.15.49:53172) 2024-03-18T23:15:09.847 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:09.846+0000 7f49bb59e640 1 -- 172.21.15.49:0/2798344569 learned_addr learned my addr 172.21.15.49:0/2798344569 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:15:09.847 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:09.846+0000 7f49ba59c640 1 --2- 172.21.15.49:0/2798344569 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564bef796810 0x564bef7b70c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:15:09.847 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:09.846+0000 7f49bb59e640 1 -- 172.21.15.49:0/2798344569 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564bef795ef0 msgr2=0x564bef7aafb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:15:09.847 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:09.846+0000 7f49bb59e640 1 --2- 172.21.15.49:0/2798344569 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564bef795ef0 0x564bef7aafb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:09.847 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:09.846+0000 7f49bb59e640 1 -- 172.21.15.49:0/2798344569 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564bef796810 msgr2=0x564bef7b70c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:15:09.847 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:09.846+0000 7f49bb59e640 1 --2- 172.21.15.49:0/2798344569 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564bef796810 0x564bef7b70c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:09.847 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:09.846+0000 7f49bb59e640 1 -- 172.21.15.49:0/2798344569 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564bef7b1c60 con 0x564bef79f5e0 2024-03-18T23:15:09.847 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:09.846+0000 7f49bb59e640 1 --2- 172.21.15.49:0/2798344569 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564bef79f5e0 0x564bef79ea90 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f49b4009ca0 tx=0x7f49b4009360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a97daa0e4c5b0d17 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:15:09.847 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:09.846+0000 7f49b0ff9640 1 -- 172.21.15.49:0/2798344569 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49b4039040 con 0x564bef79f5e0 2024-03-18T23:15:09.847 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:09.846+0000 7f49b0ff9640 1 -- 172.21.15.49:0/2798344569 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f49b4047070 con 0x564bef79f5e0 2024-03-18T23:15:09.847 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:09.846+0000 7f49b0ff9640 1 -- 172.21.15.49:0/2798344569 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49b40347a0 con 0x564bef79f5e0 2024-03-18T23:15:09.851 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:09.847+0000 7f49c3db0740 1 -- 172.21.15.49:0/2798344569 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564bef79f5e0 msgr2=0x564bef79ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:15:09.851 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:09.847+0000 7f49c3db0740 1 --2- 172.21.15.49:0/2798344569 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564bef79f5e0 0x564bef79ea90 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f49b4009ca0 tx=0x7f49b4009360 comp rx=0 tx=0).stop 2024-03-18T23:15:09.851 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:09.847+0000 7f49c3db0740 1 -- 172.21.15.49:0/2798344569 shutdown_connections 2024-03-18T23:15:09.851 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:09.847+0000 7f49c3db0740 1 --2- 172.21.15.49:0/2798344569 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564bef79f5e0 0x564bef79ea90 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:09.851 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:09.847+0000 7f49c3db0740 1 --2- 172.21.15.49:0/2798344569 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564bef796810 0x564bef7b70c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:09.851 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:09.847+0000 7f49c3db0740 1 --2- 172.21.15.49:0/2798344569 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564bef795ef0 0x564bef7aafb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:09.851 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:09.847+0000 7f49c3db0740 1 -- 172.21.15.49:0/2798344569 >> 172.21.15.49:0/2798344569 conn(0x564bef790810 msgr2=0x564bef78fab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:15:09.851 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:09.847+0000 7f49c3db0740 1 -- 172.21.15.49:0/2798344569 shutdown_connections 2024-03-18T23:15:09.851 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:09.847+0000 7f49c3db0740 1 -- 172.21.15.49:0/2798344569 wait complete. 2024-03-18T23:15:09.852 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:09.847+0000 7f49c3db0740 1 -- start start 2024-03-18T23:15:09.852 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:09.847+0000 7f49c3db0740 1 Processor -- start 2024-03-18T23:15:09.852 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:09.848+0000 7f49c3db0740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564bef795ef0 0x564bef7a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:15:09.852 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:09.848+0000 7f49c3db0740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564bef796810 0x564bef7a3c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:15:09.852 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:09.848+0000 7f49c3db0740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564bef79f5e0 0x564bef8146f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:15:09.852 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:09.848+0000 7f49c3db0740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f49bc1b0ab0 con 0x564bef79f5e0 2024-03-18T23:15:09.852 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:09.848+0000 7f49c3db0740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f49bc1b03e0 con 0x564bef795ef0 2024-03-18T23:15:09.852 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:09.848+0000 7f49c3db0740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f49bc1ad640 con 0x564bef796810 2024-03-18T23:15:09.852 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:09.848+0000 7f49c3db0740 10 client.0 _ll_register_callbacks cb 0x7f49a40974d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T23:15:09.852 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:09.848+0000 7f49bb59e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564bef79f5e0 0x564bef8146f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:15:09.852 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:09.848+0000 7f49bad9d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564bef795ef0 0x564bef7a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:15:09.852 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:09.848+0000 7f49bb59e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564bef79f5e0 0x564bef8146f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53178/0 (socket says 172.21.15.49:53178) 2024-03-18T23:15:09.852 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:09.848+0000 7f49bb59e640 1 -- 172.21.15.49:0/1346738143 learned_addr learned my addr 172.21.15.49:0/1346738143 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:15:09.852 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:09.848+0000 7f49ba59c640 1 --2- 172.21.15.49:0/1346738143 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564bef796810 0x564bef7a3c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:15:09.853 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:09.848+0000 7f49bb59e640 1 -- 172.21.15.49:0/1346738143 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564bef796810 msgr2=0x564bef7a3c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:15:09.853 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:09.848+0000 7f49bb59e640 1 --2- 172.21.15.49:0/1346738143 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564bef796810 0x564bef7a3c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:09.853 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:09.848+0000 7f49bb59e640 1 -- 172.21.15.49:0/1346738143 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564bef795ef0 msgr2=0x564bef7a36c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:15:09.853 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:09.848+0000 7f49bb59e640 1 --2- 172.21.15.49:0/1346738143 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564bef795ef0 0x564bef7a36c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:09.853 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:09.848+0000 7f49bb59e640 1 -- 172.21.15.49:0/1346738143 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564bef4da620 con 0x564bef79f5e0 2024-03-18T23:15:09.853 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:09.848+0000 7f49bad9d640 1 --2- 172.21.15.49:0/1346738143 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564bef795ef0 0x564bef7a36c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:15:09.853 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:09.848+0000 7f49bb59e640 1 --2- 172.21.15.49:0/1346738143 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564bef79f5e0 0x564bef8146f0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f49b4009890 tx=0x7f49b4005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4688c2d8df3e6b07 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:15:09.853 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:09.848+0000 7f49a27fc640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-18T23:15:09.853 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:09.849+0000 7f49a27fc640 1 -- 172.21.15.49:0/1346738143 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49b405c070 con 0x564bef79f5e0 2024-03-18T23:15:09.853 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:09.849+0000 7f49a27fc640 1 -- 172.21.15.49:0/1346738143 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7f49b4035d00 con 0x564bef79f5e0 2024-03-18T23:15:09.853 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:09.849+0000 7f49c3db0740 1 -- 172.21.15.49:0/1346738143 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f49b4047070 con 0x564bef79f5e0 2024-03-18T23:15:09.853 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:09.849+0000 7f49a27fc640 1 -- 172.21.15.49:0/1346738143 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f49b4068050 con 0x564bef79f5e0 2024-03-18T23:15:09.853 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:09.849+0000 7f49c3db0740 10 client.16017 fetch_fsmap learned FSMap version 32 2024-03-18T23:15:09.853 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:09.849+0000 7f49c3db0740 1 -- 172.21.15.49:0/1346738143 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x564bef815e00 con 0x564bef79f5e0 2024-03-18T23:15:09.854 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:09.849+0000 7f49a27fc640 1 -- 172.21.15.49:0/1346738143 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f49b4047070 con 0x564bef79f5e0 2024-03-18T23:15:09.854 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:09.849+0000 7f49a27fc640 1 -- 172.21.15.49:0/1346738143 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f49b404c070 con 0x564bef79f5e0 2024-03-18T23:15:09.854 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:09.849+0000 7f49c3db0740 10 client.16017 fetch_fsmap finished waiting for FSMap version 32 2024-03-18T23:15:09.854 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:09.849+0000 7f49c3db0740 10 client.16017 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T23:15:09.854 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:09.849+0000 7f49c3db0740 20 client.16017 populate_metadata read hostname 'smithi049' 2024-03-18T23:15:09.854 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:09.849+0000 7f49c3db0740 1 --2- 172.21.15.49:0/1346738143 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x564bef816520 0x564bef818940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:15:09.854 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:09.849+0000 7f49c3db0740 4 client.16017 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T23:15:09.854 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:09.849+0000 7f49c3db0740 1 -- 172.21.15.49:0/1346738143 --> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564bef701470 con 0x564bef816520 2024-03-18T23:15:09.854 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:09.849+0000 7f49bad9d640 1 --2- 172.21.15.49:0/1346738143 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x564bef816520 0x564bef818940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:15:09.854 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:09.850+0000 7f49a27fc640 10 client.16017 ms_handle_connect on v2:172.21.15.139:6838/4280128209 2024-03-18T23:15:09.854 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:09.850+0000 7f49bad9d640 1 --2- 172.21.15.49:0/1346738143 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x564bef816520 0x564bef818940 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2ec882112084ff33 server_cookie=30e6d6afc296d456 in_seq=0 out_seq=0 2024-03-18T23:15:10.034 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:10.034+0000 7f49a27fc640 1 -- 172.21.15.49:0/1346738143 <== mds.0 v2:172.21.15.139:6838/4280128209 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x564bef701470 con 0x564bef816520 2024-03-18T23:15:10.034 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:10.034+0000 7f49a27fc640 10 client.16017 handle_command_reply: tid=0 2024-03-18T23:15:10.034 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:10.034+0000 7f49c3db0740 1 client.16017 shutdown 2024-03-18T23:15:10.034 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:10.034+0000 7f49c3db0740 1 -- 172.21.15.49:0/1346738143 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564bef79f5e0 msgr2=0x564bef8146f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:15:10.034 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:10.034+0000 7f49c3db0740 1 --2- 172.21.15.49:0/1346738143 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564bef79f5e0 0x564bef8146f0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f49b4009890 tx=0x7f49b4005ef0 comp rx=0 tx=0).stop 2024-03-18T23:15:10.034 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:10.034+0000 7f49c3db0740 1 -- 172.21.15.49:0/1346738143 shutdown_connections 2024-03-18T23:15:10.034 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:10.034+0000 7f49c3db0740 1 --2- 172.21.15.49:0/1346738143 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x564bef816520 0x564bef818940 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:10.034 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:10.034+0000 7f49c3db0740 1 --2- 172.21.15.49:0/1346738143 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564bef79f5e0 0x564bef8146f0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:10.035 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:10.034+0000 7f49c3db0740 1 --2- 172.21.15.49:0/1346738143 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564bef796810 0x564bef7a3c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:10.035 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:10.034+0000 7f49a27fc640 0 client.16017 ms_handle_reset on v2:172.21.15.139:6838/4280128209 2024-03-18T23:15:10.035 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:10.034+0000 7f49c3db0740 1 --2- 172.21.15.49:0/1346738143 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564bef795ef0 0x564bef7a36c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:10.035 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:10.034+0000 7f49c3db0740 1 -- 172.21.15.49:0/1346738143 >> 172.21.15.49:0/1346738143 conn(0x564bef790810 msgr2=0x564bef791c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:15:10.035 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:10.034+0000 7f49c3db0740 1 -- 172.21.15.49:0/1346738143 shutdown_connections 2024-03-18T23:15:10.035 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:10.034+0000 7f49c3db0740 1 -- 172.21.15.49:0/1346738143 wait complete. 2024-03-18T23:15:10.035 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:10.034+0000 7f49c3db0740 20 client.16017 trim_cache size 0 max 16384 2024-03-18T23:15:10.035 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:{ 2024-03-18T23:15:10.035 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "message": "", 2024-03-18T23:15:10.035 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "return_code": 0 2024-03-18T23:15:10.035 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:} 2024-03-18T23:15:10.036 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:10.035+0000 7f49c11cd640 1 -- 172.21.15.49:0/1307161653 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f49a80a2a10 msgr2=0x7f49a80a4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:15:10.036 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:10.035+0000 7f49c11cd640 1 --2- 172.21.15.49:0/1307161653 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f49a80a2a10 0x7f49a80a4ed0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f49b4004af0 tx=0x7f49b402c0f0 comp rx=0 tx=0).stop 2024-03-18T23:15:10.037 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:10.035+0000 7f49c11cd640 1 -- 172.21.15.49:0/1307161653 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f49bc100970 msgr2=0x7f49bc102390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:15:10.037 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:10.035+0000 7f49c11cd640 1 --2- 172.21.15.49:0/1307161653 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f49bc100970 0x7f49bc102390 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f49a400e9f0 tx=0x7f49a400edf0 comp rx=0 tx=0).stop 2024-03-18T23:15:10.037 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:10.035+0000 7f49c11cd640 1 -- 172.21.15.49:0/1307161653 shutdown_connections 2024-03-18T23:15:10.037 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:10.035+0000 7f49c11cd640 1 --2- 172.21.15.49:0/1307161653 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f49a80a2a10 0x7f49a80a4ed0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:10.037 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:10.035+0000 7f49c11cd640 1 --2- 172.21.15.49:0/1307161653 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f49bc1056c0 0x7f49bc102e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:10.037 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:10.035+0000 7f49c11cd640 1 --2- 172.21.15.49:0/1307161653 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f49bc101330 0x7f49bc1028d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:10.037 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:10.035+0000 7f49c11cd640 1 --2- 172.21.15.49:0/1307161653 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f49bc100970 0x7f49bc102390 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:10.037 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:10.035+0000 7f49c11cd640 1 -- 172.21.15.49:0/1307161653 >> 172.21.15.49:0/1307161653 conn(0x7f49bc077570 msgr2=0x7f49bc0690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:15:10.037 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:10.035+0000 7f49c11cd640 1 -- 172.21.15.49:0/1307161653 shutdown_connections 2024-03-18T23:15:10.037 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:10.035+0000 7f49c11cd640 1 -- 172.21.15.49:0/1307161653 wait complete. 2024-03-18T23:15:10.775 INFO:tasks.workunit.client.0.smithi049.stdout: 1 745138 10.02 MB/sec execute 252 sec latency 137.624 ms 2024-03-18T23:15:11.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:15:11 smithi175 ceph-mon[67352]: pgmap v214: 97 pgs: 97 active+clean; 77 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.8 MiB/s wr, 398 op/s 2024-03-18T23:15:11.564 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:15:11 smithi139 ceph-mon[67282]: pgmap v214: 97 pgs: 97 active+clean; 77 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.8 MiB/s wr, 398 op/s 2024-03-18T23:15:11.629 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:15:11 smithi049 ceph-mon[62991]: pgmap v214: 97 pgs: 97 active+clean; 77 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.8 MiB/s wr, 398 op/s 2024-03-18T23:15:11.775 INFO:tasks.workunit.client.0.smithi049.stdout: 1 746860 10.02 MB/sec execute 253 sec latency 19.594 ms 2024-03-18T23:15:12.775 INFO:tasks.workunit.client.0.smithi049.stdout: 1 749185 10.04 MB/sec execute 254 sec latency 21.953 ms 2024-03-18T23:15:13.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:15:13 smithi175 ceph-mon[67352]: pgmap v215: 97 pgs: 97 active+clean; 68 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.3 MiB/s wr, 277 op/s 2024-03-18T23:15:13.564 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:15:13 smithi139 ceph-mon[67282]: pgmap v215: 97 pgs: 97 active+clean; 68 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.3 MiB/s wr, 277 op/s 2024-03-18T23:15:13.629 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:15:13 smithi049 ceph-mon[62991]: pgmap v215: 97 pgs: 97 active+clean; 68 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.3 MiB/s wr, 277 op/s 2024-03-18T23:15:13.775 INFO:tasks.workunit.client.0.smithi049.stdout: 1 750440 10.01 MB/sec execute 255 sec latency 61.185 ms 2024-03-18T23:15:14.775 INFO:tasks.workunit.client.0.smithi049.stdout: 1 752771 10.03 MB/sec execute 256 sec latency 20.267 ms 2024-03-18T23:15:15.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:15:15 smithi175 ceph-mon[67352]: pgmap v216: 97 pgs: 97 active+clean; 68 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.3 MiB/s wr, 277 op/s 2024-03-18T23:15:15.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:15:15 smithi139 ceph-mon[67282]: pgmap v216: 97 pgs: 97 active+clean; 68 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.3 MiB/s wr, 277 op/s 2024-03-18T23:15:15.629 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:15:15 smithi049 ceph-mon[62991]: pgmap v216: 97 pgs: 97 active+clean; 68 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.3 MiB/s wr, 277 op/s 2024-03-18T23:15:15.775 INFO:tasks.workunit.client.0.smithi049.stdout: 1 754298 10.02 MB/sec execute 257 sec latency 27.768 ms 2024-03-18T23:15:16.775 INFO:tasks.workunit.client.0.smithi049.stdout: 1 756553 10.04 MB/sec execute 258 sec latency 39.617 ms 2024-03-18T23:15:17.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:15:17 smithi175 ceph-mon[67352]: pgmap v217: 97 pgs: 97 active+clean; 68 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.3 MiB/s wr, 277 op/s 2024-03-18T23:15:17.564 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:15:17 smithi139 ceph-mon[67282]: pgmap v217: 97 pgs: 97 active+clean; 68 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.3 MiB/s wr, 277 op/s 2024-03-18T23:15:17.629 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:15:17 smithi049 ceph-mon[62991]: pgmap v217: 97 pgs: 97 active+clean; 68 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.3 MiB/s wr, 277 op/s 2024-03-18T23:15:17.775 INFO:tasks.workunit.client.0.smithi049.stdout: 1 757879 10.02 MB/sec execute 259 sec latency 93.811 ms 2024-03-18T23:15:18.775 INFO:tasks.workunit.client.0.smithi049.stdout: 1 759953 10.03 MB/sec execute 260 sec latency 103.571 ms 2024-03-18T23:15:19.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:15:19 smithi175 ceph-mon[67352]: pgmap v218: 97 pgs: 97 active+clean; 75 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.4 MiB/s wr, 295 op/s 2024-03-18T23:15:19.564 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:15:19 smithi139 ceph-mon[67282]: pgmap v218: 97 pgs: 97 active+clean; 75 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.4 MiB/s wr, 295 op/s 2024-03-18T23:15:19.629 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:15:19 smithi049 ceph-mon[62991]: pgmap v218: 97 pgs: 97 active+clean; 75 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.4 MiB/s wr, 295 op/s 2024-03-18T23:15:19.775 INFO:tasks.workunit.client.0.smithi049.stdout: 1 761385 10.01 MB/sec execute 261 sec latency 24.338 ms 2024-03-18T23:15:20.775 INFO:tasks.workunit.client.0.smithi049.stdout: 1 763619 10.03 MB/sec execute 262 sec latency 21.032 ms 2024-03-18T23:15:21.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:15:21 smithi175 ceph-mon[67352]: pgmap v219: 97 pgs: 97 active+clean; 75 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.2 MiB/s wr, 285 op/s 2024-03-18T23:15:21.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:15:21 smithi139 ceph-mon[67282]: pgmap v219: 97 pgs: 97 active+clean; 75 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.2 MiB/s wr, 285 op/s 2024-03-18T23:15:21.629 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:15:21 smithi049 ceph-mon[62991]: pgmap v219: 97 pgs: 97 active+clean; 75 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.2 MiB/s wr, 285 op/s 2024-03-18T23:15:21.776 INFO:tasks.workunit.client.0.smithi049.stdout: 1 765058 10.01 MB/sec execute 263 sec latency 23.037 ms 2024-03-18T23:15:22.776 INFO:tasks.workunit.client.0.smithi049.stdout: 1 767233 10.02 MB/sec execute 264 sec latency 22.521 ms 2024-03-18T23:15:23.154 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:23.151+0000 7fc5d4097640 1 -- 172.21.15.49:0/2931020977 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc5cc069840 msgr2=0x7fc5cc10e6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:15:23.154 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:23.151+0000 7fc5d4097640 1 --2- 172.21.15.49:0/2931020977 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc5cc069840 0x7fc5cc10e6b0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fc5c4009cd0 tx=0x7fc5c402c0f0 comp rx=0 tx=0).stop 2024-03-18T23:15:23.154 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:23.152+0000 7fc5d4097640 1 -- 172.21.15.49:0/2931020977 shutdown_connections 2024-03-18T23:15:23.154 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:23.152+0000 7fc5d4097640 1 --2- 172.21.15.49:0/2931020977 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc5cc069840 0x7fc5cc10e6b0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:23.154 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:23.152+0000 7fc5d4097640 1 --2- 172.21.15.49:0/2931020977 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc5cc068ea0 0x7fc5cc069300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:23.154 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:23.152+0000 7fc5d4097640 1 --2- 172.21.15.49:0/2931020977 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc5cc1078f0 0x7fc5cc107cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:23.154 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:23.152+0000 7fc5d4097640 1 -- 172.21.15.49:0/2931020977 >> 172.21.15.49:0/2931020977 conn(0x7fc5cc077570 msgr2=0x7fc5cc077980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:15:23.154 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:23.152+0000 7fc5d4097640 1 -- 172.21.15.49:0/2931020977 shutdown_connections 2024-03-18T23:15:23.154 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:23.152+0000 7fc5d4097640 1 -- 172.21.15.49:0/2931020977 wait complete. 2024-03-18T23:15:23.155 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:23.152+0000 7fc5d4097640 1 Processor -- start 2024-03-18T23:15:23.155 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:23.152+0000 7fc5d4097640 1 -- start start 2024-03-18T23:15:23.155 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:23.152+0000 7fc5d4097640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc5cc068ea0 0x7fc5cc197fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:15:23.155 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:23.152+0000 7fc5d4097640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc5cc069840 0x7fc5cc198500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:15:23.155 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:23.152+0000 7fc5d4097640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc5cc1078f0 0x7fc5cc19c830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:15:23.155 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:23.152+0000 7fc5d4097640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fc5cc111be0 con 0x7fc5cc1078f0 2024-03-18T23:15:23.155 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:23.152+0000 7fc5d4097640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fc5cc111a60 con 0x7fc5cc069840 2024-03-18T23:15:23.155 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:23.152+0000 7fc5d4097640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc5cc111d60 con 0x7fc5cc068ea0 2024-03-18T23:15:23.155 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:23.153+0000 7fc5d2634640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc5cc1078f0 0x7fc5cc19c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:15:23.155 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:23.153+0000 7fc5d1e33640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc5cc068ea0 0x7fc5cc197fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:15:23.155 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:23.153+0000 7fc5d1e33640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc5cc068ea0 0x7fc5cc197fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.49:56100/0 (socket says 172.21.15.49:56100) 2024-03-18T23:15:23.155 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:23.153+0000 7fc5d2634640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc5cc1078f0 0x7fc5cc19c830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:37844/0 (socket says 172.21.15.49:37844) 2024-03-18T23:15:23.155 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:23.153+0000 7fc5d1632640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc5cc069840 0x7fc5cc198500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:15:23.155 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:23.153+0000 7fc5d1e33640 1 -- 172.21.15.49:0/2952419652 learned_addr learned my addr 172.21.15.49:0/2952419652 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:15:23.155 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:23.153+0000 7fc5d2634640 1 -- 172.21.15.49:0/2952419652 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc5cc068ea0 msgr2=0x7fc5cc197fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:15:23.156 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:23.153+0000 7fc5d2634640 1 --2- 172.21.15.49:0/2952419652 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc5cc068ea0 0x7fc5cc197fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:23.156 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:23.153+0000 7fc5d2634640 1 -- 172.21.15.49:0/2952419652 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc5cc069840 msgr2=0x7fc5cc198500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:15:23.156 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:23.153+0000 7fc5d2634640 1 --2- 172.21.15.49:0/2952419652 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc5cc069840 0x7fc5cc198500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:23.156 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:23.153+0000 7fc5d2634640 1 -- 172.21.15.49:0/2952419652 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5cc19cf60 con 0x7fc5cc1078f0 2024-03-18T23:15:23.156 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:23.153+0000 7fc5d2634640 1 --2- 172.21.15.49:0/2952419652 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc5cc1078f0 0x7fc5cc19c830 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fc5c402c630 tx=0x7fc5c4032f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:15:23.156 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:23.153+0000 7fc5c2ffd640 1 -- 172.21.15.49:0/2952419652 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5c4042020 con 0x7fc5cc1078f0 2024-03-18T23:15:23.156 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:23.153+0000 7fc5d4097640 1 -- 172.21.15.49:0/2952419652 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5cc19d1f0 con 0x7fc5cc1078f0 2024-03-18T23:15:23.156 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:23.153+0000 7fc5d4097640 1 -- 172.21.15.49:0/2952419652 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5cc1a4ae0 con 0x7fc5cc1078f0 2024-03-18T23:15:23.156 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:23.153+0000 7fc5c2ffd640 1 -- 172.21.15.49:0/2952419652 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc5c4034640 con 0x7fc5cc1078f0 2024-03-18T23:15:23.156 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:23.153+0000 7fc5c2ffd640 1 -- 172.21.15.49:0/2952419652 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5c4039040 con 0x7fc5cc1078f0 2024-03-18T23:15:23.161 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:23.155+0000 7fc5c2ffd640 1 -- 172.21.15.49:0/2952419652 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 25) ==== 137421+0+0 (secure 0 0 0) 0x7fc5c4045020 con 0x7fc5cc1078f0 2024-03-18T23:15:23.161 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:23.156+0000 7fc5d6922740 1 Processor -- start 2024-03-18T23:15:23.161 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:23.156+0000 7fc5d6922740 1 -- start start 2024-03-18T23:15:23.161 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:23.156+0000 7fc5c2ffd640 1 --2- 172.21.15.49:0/2952419652 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7fc5b80a2a10 0x7fc5b80a4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:15:23.161 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:23.156+0000 7fc5d6922740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5608c643aa10 0x5608c643adf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:15:23.161 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:23.156+0000 7fc5d6922740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5608c643b330 0x5608c64484b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:15:23.161 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:23.156+0000 7fc5d6922740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5608c64489f0 0x5608c644ade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:15:23.161 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:23.156+0000 7fc5d6922740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5608c639ebe0 con 0x5608c643b330 2024-03-18T23:15:23.161 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:23.156+0000 7fc5d6922740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x5608c6347960 con 0x5608c64489f0 2024-03-18T23:15:23.161 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:23.156+0000 7fc5d6922740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5608c639ac80 con 0x5608c643aa10 2024-03-18T23:15:23.161 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:23.156+0000 7fc5d1632640 1 --2- 172.21.15.49:0/2952419652 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7fc5b80a2a10 0x7fc5b80a4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:15:23.161 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:23.156+0000 7fc5c2ffd640 1 -- 172.21.15.49:0/2952419652 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7fc5c40f04a0 con 0x7fc5cc1078f0 2024-03-18T23:15:23.162 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:23.156+0000 7fc5d1e33640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5608c643aa10 0x5608c643adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:15:23.162 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:23.157+0000 7fc5d1632640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5608c643b330 0x5608c64484b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:15:23.162 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:23.157+0000 7fc5d1e33640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5608c643aa10 0x5608c643adf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.49:56110/0 (socket says 172.21.15.49:56110) 2024-03-18T23:15:23.162 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:23.157+0000 7fc5d1e33640 1 -- 172.21.15.49:0/2608639842 learned_addr learned my addr 172.21.15.49:0/2608639842 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:15:23.162 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:23.157+0000 7fc5d2634640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5608c64489f0 0x5608c644ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:15:23.162 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:23.157+0000 7fc5d1e33640 1 -- 172.21.15.49:0/2608639842 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5608c64489f0 msgr2=0x5608c644ade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:15:23.162 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:23.157+0000 7fc5d1e33640 1 --2- 172.21.15.49:0/2608639842 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5608c64489f0 0x5608c644ade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:23.162 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:23.157+0000 7fc5d1e33640 1 -- 172.21.15.49:0/2608639842 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5608c643b330 msgr2=0x5608c64484b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:15:23.162 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:23.157+0000 7fc5d1e33640 1 --2- 172.21.15.49:0/2608639842 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5608c643b330 0x5608c64484b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:23.162 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:23.157+0000 7fc5d1e33640 1 -- 172.21.15.49:0/2608639842 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5608c633ed20 con 0x5608c643aa10 2024-03-18T23:15:23.162 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:23.157+0000 7fc5d1632640 1 --2- 172.21.15.49:0/2952419652 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7fc5b80a2a10 0x7fc5b80a4ed0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fc5bc009fd0 tx=0x7fc5bc009b80 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:15:23.162 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:23.157+0000 7fc5d1e33640 1 --2- 172.21.15.49:0/2608639842 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5608c643aa10 0x5608c643adf0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fc5c8009dc0 tx=0x7fc5c8011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=da97b4c2515d386f server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:15:23.162 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:23.157+0000 7fc5b7fff640 1 -- 172.21.15.49:0/2608639842 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5c8005220 con 0x5608c643aa10 2024-03-18T23:15:23.163 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:23.157+0000 7fc5b7fff640 1 -- 172.21.15.49:0/2608639842 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc5c8004c80 con 0x5608c643aa10 2024-03-18T23:15:23.163 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:23.157+0000 7fc5b7fff640 1 -- 172.21.15.49:0/2608639842 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5c80198b0 con 0x5608c643aa10 2024-03-18T23:15:23.163 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:23.158+0000 7fc5d6922740 1 -- 172.21.15.49:0/2608639842 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5608c643aa10 msgr2=0x5608c643adf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:15:23.163 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:23.158+0000 7fc5d6922740 1 --2- 172.21.15.49:0/2608639842 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5608c643aa10 0x5608c643adf0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fc5c8009dc0 tx=0x7fc5c8011810 comp rx=0 tx=0).stop 2024-03-18T23:15:23.163 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:23.158+0000 7fc5d6922740 1 -- 172.21.15.49:0/2608639842 shutdown_connections 2024-03-18T23:15:23.163 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:23.158+0000 7fc5d6922740 1 --2- 172.21.15.49:0/2608639842 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5608c64489f0 0x5608c644ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:23.163 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:23.158+0000 7fc5d6922740 1 --2- 172.21.15.49:0/2608639842 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5608c643b330 0x5608c64484b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:23.163 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:23.158+0000 7fc5d6922740 1 --2- 172.21.15.49:0/2608639842 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5608c643aa10 0x5608c643adf0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:23.163 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:23.158+0000 7fc5d6922740 1 -- 172.21.15.49:0/2608639842 >> 172.21.15.49:0/2608639842 conn(0x5608c6435810 msgr2=0x5608c6435bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:15:23.163 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:23.158+0000 7fc5d6922740 1 -- 172.21.15.49:0/2608639842 shutdown_connections 2024-03-18T23:15:23.163 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:23.158+0000 7fc5d6922740 1 -- 172.21.15.49:0/2608639842 wait complete. 2024-03-18T23:15:23.163 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:23.158+0000 7fc5d6922740 1 -- start start 2024-03-18T23:15:23.163 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:23.158+0000 7fc5d6922740 1 Processor -- start 2024-03-18T23:15:23.163 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:23.159+0000 7fc5d6922740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5608c643aa10 0x5608c64482e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:15:23.163 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:23.159+0000 7fc5d6922740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5608c643b330 0x5608c6445670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:15:23.164 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:23.159+0000 7fc5d6922740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5608c64489f0 0x5608c6445bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:15:23.164 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:23.159+0000 7fc5d6922740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5608c644b5b0 con 0x5608c643b330 2024-03-18T23:15:23.164 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:23.159+0000 7fc5d6922740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x5608c635adb0 con 0x5608c643aa10 2024-03-18T23:15:23.164 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:23.159+0000 7fc5d6922740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5608c644b730 con 0x5608c64489f0 2024-03-18T23:15:23.164 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:23.159+0000 7fc5d6922740 10 client.0 _ll_register_callbacks cb 0x5608c635fe70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T23:15:23.164 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:23.159+0000 7fc5d1632640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5608c643b330 0x5608c6445670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:15:23.164 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:23.159+0000 7fc5d1e33640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5608c643aa10 0x5608c64482e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:15:23.164 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:23.159+0000 7fc5d1632640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5608c643b330 0x5608c6445670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:37860/0 (socket says 172.21.15.49:37860) 2024-03-18T23:15:23.164 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:23.159+0000 7fc5d1632640 1 -- 172.21.15.49:0/82451798 learned_addr learned my addr 172.21.15.49:0/82451798 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:15:23.164 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:23.159+0000 7fc5d2634640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5608c64489f0 0x5608c6445bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:15:23.164 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:23.159+0000 7fc5d1632640 1 -- 172.21.15.49:0/82451798 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5608c64489f0 msgr2=0x5608c6445bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:15:23.164 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:23.159+0000 7fc5d1632640 1 --2- 172.21.15.49:0/82451798 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5608c64489f0 0x5608c6445bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:23.164 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:23.159+0000 7fc5d1632640 1 -- 172.21.15.49:0/82451798 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5608c643aa10 msgr2=0x5608c64482e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:15:23.164 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:23.159+0000 7fc5d1632640 1 --2- 172.21.15.49:0/82451798 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5608c643aa10 0x5608c64482e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:23.165 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:23.159+0000 7fc5d1632640 1 -- 172.21.15.49:0/82451798 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5608c6359c60 con 0x5608c643b330 2024-03-18T23:15:23.165 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:23.160+0000 7fc5d1632640 1 --2- 172.21.15.49:0/82451798 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5608c643b330 0x5608c6445670 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fc5bc011420 tx=0x7fc5bc01d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15ffa6f22e54b1d1 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:15:23.165 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:23.160+0000 7fc5b57fa640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-18T23:15:23.165 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:23.160+0000 7fc5b57fa640 1 -- 172.21.15.49:0/82451798 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5bc0044c0 con 0x5608c643b330 2024-03-18T23:15:23.165 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:23.160+0000 7fc5b57fa640 1 -- 172.21.15.49:0/82451798 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7fc5bc003800 con 0x5608c643b330 2024-03-18T23:15:23.165 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:23.160+0000 7fc5d6922740 1 -- 172.21.15.49:0/82451798 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc5c8004c80 con 0x5608c643b330 2024-03-18T23:15:23.165 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:23.160+0000 7fc5b57fa640 1 -- 172.21.15.49:0/82451798 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fc5bc003a00 con 0x5608c643b330 2024-03-18T23:15:23.165 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:23.160+0000 7fc5d6922740 10 client.16029 fetch_fsmap learned FSMap version 32 2024-03-18T23:15:23.165 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:23.160+0000 7fc5d6922740 1 -- 172.21.15.49:0/82451798 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5608c60b3210 con 0x5608c643b330 2024-03-18T23:15:23.167 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:23.163+0000 7fc5b57fa640 1 -- 172.21.15.49:0/82451798 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc5c8004c80 con 0x5608c643b330 2024-03-18T23:15:23.167 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:23.163+0000 7fc5b57fa640 1 -- 172.21.15.49:0/82451798 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7fc5bc010430 con 0x5608c643b330 2024-03-18T23:15:23.167 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:23.163+0000 7fc5d6922740 10 client.16029 fetch_fsmap finished waiting for FSMap version 32 2024-03-18T23:15:23.167 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:23.163+0000 7fc5d6922740 10 client.16029 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T23:15:23.167 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:23.163+0000 7fc5d6922740 20 client.16029 populate_metadata read hostname 'smithi049' 2024-03-18T23:15:23.167 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:23.163+0000 7fc5d6922740 1 --2- 172.21.15.49:0/82451798 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x5608c6458910 0x5608c645ad60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:15:23.167 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:23.163+0000 7fc5d6922740 4 client.16029 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T23:15:23.167 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:23.163+0000 7fc5d6922740 1 -- 172.21.15.49:0/82451798 --> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5608c63ac120 con 0x5608c6458910 2024-03-18T23:15:23.168 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:23.163+0000 7fc5d1e33640 1 --2- 172.21.15.49:0/82451798 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x5608c6458910 0x5608c645ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:15:23.168 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:23.164+0000 7fc5b57fa640 10 client.16029 ms_handle_connect on v2:172.21.15.139:6838/4280128209 2024-03-18T23:15:23.168 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:23.164+0000 7fc5d1e33640 1 --2- 172.21.15.49:0/82451798 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x5608c6458910 0x5608c645ad60 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=47baf35f66958848 server_cookie=89653519816fbe59 in_seq=0 out_seq=0 2024-03-18T23:15:23.168 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:23.165+0000 7fc5b57fa640 1 -- 172.21.15.49:0/82451798 <== mds.0 v2:172.21.15.139:6838/4280128209 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5608c63ac120 con 0x5608c6458910 2024-03-18T23:15:23.168 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:23.165+0000 7fc5b57fa640 10 client.16029 handle_command_reply: tid=0 2024-03-18T23:15:23.168 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:23.165+0000 7fc5d6922740 1 client.16029 shutdown 2024-03-18T23:15:23.168 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:23.165+0000 7fc5d6922740 1 -- 172.21.15.49:0/82451798 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5608c643b330 msgr2=0x5608c6445670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:15:23.168 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:23.165+0000 7fc5d6922740 1 --2- 172.21.15.49:0/82451798 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5608c643b330 0x5608c6445670 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fc5bc011420 tx=0x7fc5bc01d000 comp rx=0 tx=0).stop 2024-03-18T23:15:23.168 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:23.165+0000 7fc5d6922740 1 -- 172.21.15.49:0/82451798 shutdown_connections 2024-03-18T23:15:23.168 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:23.165+0000 7fc5d6922740 1 --2- 172.21.15.49:0/82451798 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x5608c6458910 0x5608c645ad60 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:23.168 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:23.165+0000 7fc5d6922740 1 --2- 172.21.15.49:0/82451798 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5608c64489f0 0x5608c6445bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:23.168 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:23.165+0000 7fc5d6922740 1 --2- 172.21.15.49:0/82451798 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5608c643b330 0x5608c6445670 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:23.168 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:23.165+0000 7fc5d6922740 1 --2- 172.21.15.49:0/82451798 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5608c643aa10 0x5608c64482e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:23.168 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:23.165+0000 7fc5b57fa640 0 client.16029 ms_handle_reset on v2:172.21.15.139:6838/4280128209 2024-03-18T23:15:23.168 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:23.165+0000 7fc5d6922740 1 -- 172.21.15.49:0/82451798 >> 172.21.15.49:0/82451798 conn(0x5608c6435810 msgr2=0x5608c6437560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:15:23.169 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:23.165+0000 7fc5d6922740 1 -- 172.21.15.49:0/82451798 shutdown_connections 2024-03-18T23:15:23.169 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:23.165+0000 7fc5d6922740 1 -- 172.21.15.49:0/82451798 wait complete. 2024-03-18T23:15:23.169 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:23.165+0000 7fc5d6922740 20 client.16029 trim_cache size 0 max 16384 2024-03-18T23:15:23.183 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:23.182+0000 7fc5d6922740 1 Processor -- start 2024-03-18T23:15:23.183 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:23.183+0000 7fc5d6922740 1 -- start start 2024-03-18T23:15:23.184 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:23.183+0000 7fc5d6922740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5608c643aa10 0x5608c64543f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:15:23.184 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:23.183+0000 7fc5d6922740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5608c643b330 0x5608c6460670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:15:23.184 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:23.183+0000 7fc5d6922740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5608c64489f0 0x5608c6447da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:15:23.184 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:23.183+0000 7fc5d6922740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fc5b000e7b0 con 0x5608c643b330 2024-03-18T23:15:23.184 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:23.183+0000 7fc5d6922740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fc5b00102f0 con 0x5608c64489f0 2024-03-18T23:15:23.184 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:23.183+0000 7fc5d6922740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc5b000ee80 con 0x5608c643aa10 2024-03-18T23:15:23.184 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:23.183+0000 7fc5d1632640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5608c643b330 0x5608c6460670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:15:23.184 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:23.183+0000 7fc5d1e33640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5608c643aa10 0x5608c64543f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:15:23.184 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:23.183+0000 7fc5d1e33640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5608c643aa10 0x5608c64543f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.49:56128/0 (socket says 172.21.15.49:56128) 2024-03-18T23:15:23.184 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:23.183+0000 7fc5d1e33640 1 -- 172.21.15.49:0/3372581434 learned_addr learned my addr 172.21.15.49:0/3372581434 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:15:23.184 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:23.183+0000 7fc5d2634640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5608c64489f0 0x5608c6447da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:15:23.184 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:23.183+0000 7fc5d1e33640 1 -- 172.21.15.49:0/3372581434 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5608c64489f0 msgr2=0x5608c6447da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:15:23.184 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:23.183+0000 7fc5d1e33640 1 --2- 172.21.15.49:0/3372581434 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5608c64489f0 0x5608c6447da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:23.184 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:23.183+0000 7fc5d1e33640 1 -- 172.21.15.49:0/3372581434 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5608c643b330 msgr2=0x5608c6460670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:15:23.184 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:23.183+0000 7fc5d1e33640 1 --2- 172.21.15.49:0/3372581434 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5608c643b330 0x5608c6460670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:23.185 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:23.183+0000 7fc5d1e33640 1 -- 172.21.15.49:0/3372581434 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5608c617f620 con 0x5608c643aa10 2024-03-18T23:15:23.185 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:23.183+0000 7fc5d1632640 1 --2- 172.21.15.49:0/3372581434 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5608c643b330 0x5608c6460670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:15:23.185 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:23.183+0000 7fc5d1e33640 1 --2- 172.21.15.49:0/3372581434 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5608c643aa10 0x5608c64543f0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fc5c8011d10 tx=0x7fc5c8009700 comp rx=0 tx=0).ready entity=mon.1 client_cookie=25f55dd287f08095 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:15:23.188 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:23.184+0000 7fc5b77fe640 1 -- 172.21.15.49:0/3372581434 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5c802bc40 con 0x5608c643aa10 2024-03-18T23:15:23.188 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:23.184+0000 7fc5b77fe640 1 -- 172.21.15.49:0/3372581434 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc5c8004500 con 0x5608c643aa10 2024-03-18T23:15:23.188 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:23.184+0000 7fc5b77fe640 1 -- 172.21.15.49:0/3372581434 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5c802b7b0 con 0x5608c643aa10 2024-03-18T23:15:23.188 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:23.184+0000 7fc5d6922740 1 -- 172.21.15.49:0/3372581434 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5608c643aa10 msgr2=0x5608c64543f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:15:23.189 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:23.184+0000 7fc5d6922740 1 --2- 172.21.15.49:0/3372581434 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5608c643aa10 0x5608c64543f0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fc5c8011d10 tx=0x7fc5c8009700 comp rx=0 tx=0).stop 2024-03-18T23:15:23.189 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:23.184+0000 7fc5d6922740 1 -- 172.21.15.49:0/3372581434 shutdown_connections 2024-03-18T23:15:23.189 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:23.184+0000 7fc5d6922740 1 --2- 172.21.15.49:0/3372581434 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5608c64489f0 0x5608c6447da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:23.189 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:23.184+0000 7fc5d6922740 1 --2- 172.21.15.49:0/3372581434 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5608c643b330 0x5608c6460670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:23.189 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:23.184+0000 7fc5d6922740 1 --2- 172.21.15.49:0/3372581434 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5608c643aa10 0x5608c64543f0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:23.189 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:23.184+0000 7fc5d6922740 1 -- 172.21.15.49:0/3372581434 >> 172.21.15.49:0/3372581434 conn(0x5608c6435810 msgr2=0x5608c6440ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:15:23.189 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:23.184+0000 7fc5d6922740 1 -- 172.21.15.49:0/3372581434 shutdown_connections 2024-03-18T23:15:23.189 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:23.184+0000 7fc5d6922740 1 -- 172.21.15.49:0/3372581434 wait complete. 2024-03-18T23:15:23.189 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:23.185+0000 7fc5d6922740 1 -- start start 2024-03-18T23:15:23.189 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:23.185+0000 7fc5d6922740 1 Processor -- start 2024-03-18T23:15:23.189 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:23.185+0000 7fc5d6922740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5608c643aa10 0x5608c64425b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:15:23.189 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:23.185+0000 7fc5d6922740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5608c643b330 0x5608c6442af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:15:23.189 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:23.185+0000 7fc5d6922740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5608c64489f0 0x5608c64bdca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:15:23.189 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:23.185+0000 7fc5d6922740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fc5b000fc20 con 0x5608c64489f0 2024-03-18T23:15:23.189 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:23.185+0000 7fc5d6922740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fc5b000f550 con 0x5608c643b330 2024-03-18T23:15:23.190 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:23.185+0000 7fc5d6922740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc5b000c7b0 con 0x5608c643aa10 2024-03-18T23:15:23.190 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:23.185+0000 7fc5d6922740 10 client.0 _ll_register_callbacks cb 0x7fc5bc009d10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T23:15:23.190 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:23.185+0000 7fc5d1e33640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5608c643aa10 0x5608c64425b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:15:23.190 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:23.185+0000 7fc5d2634640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5608c64489f0 0x5608c64bdca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:15:23.190 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:23.185+0000 7fc5d1632640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5608c643b330 0x5608c6442af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:15:23.190 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:23.185+0000 7fc5d1e33640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5608c643aa10 0x5608c64425b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.49:56136/0 (socket says 172.21.15.49:56136) 2024-03-18T23:15:23.190 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:23.185+0000 7fc5d1e33640 1 -- 172.21.15.49:0/3768616158 learned_addr learned my addr 172.21.15.49:0/3768616158 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:15:23.190 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:23.186+0000 7fc5d1632640 1 -- 172.21.15.49:0/3768616158 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5608c643aa10 msgr2=0x5608c64425b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:15:23.190 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:23.186+0000 7fc5d1632640 1 --2- 172.21.15.49:0/3768616158 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5608c643aa10 0x5608c64425b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:23.190 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:23.186+0000 7fc5d1632640 1 -- 172.21.15.49:0/3768616158 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5608c64489f0 msgr2=0x5608c64bdca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:15:23.190 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:23.186+0000 7fc5d1632640 1 --2- 172.21.15.49:0/3768616158 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5608c64489f0 0x5608c64bdca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:23.190 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:23.186+0000 7fc5d1632640 1 -- 172.21.15.49:0/3768616158 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5608c64433f0 con 0x5608c643b330 2024-03-18T23:15:23.190 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:23.186+0000 7fc5d1e33640 1 --2- 172.21.15.49:0/3768616158 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5608c643aa10 0x5608c64425b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:15:23.191 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:23.186+0000 7fc5d2634640 1 --2- 172.21.15.49:0/3768616158 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5608c64489f0 0x5608c64bdca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:15:23.191 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:23.186+0000 7fc5d1632640 1 --2- 172.21.15.49:0/3768616158 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5608c643b330 0x5608c6442af0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fc5bc0108a0 tx=0x7fc5bc011f70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bf3d285593547c37 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:15:23.191 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:23.186+0000 7fc5b57fa640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-03-18T23:15:23.191 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:23.186+0000 7fc5b57fa640 1 -- 172.21.15.49:0/3768616158 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5bc010e80 con 0x5608c643b330 2024-03-18T23:15:23.191 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:23.186+0000 7fc5b57fa640 1 -- 172.21.15.49:0/3768616158 <== mon.2 v2:172.21.15.139:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7fc5bc003800 con 0x5608c643b330 2024-03-18T23:15:23.191 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:23.186+0000 7fc5d6922740 1 -- 172.21.15.49:0/3768616158 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5608c63a6470 con 0x5608c643b330 2024-03-18T23:15:23.191 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:23.186+0000 7fc5b57fa640 1 -- 172.21.15.49:0/3768616158 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fc5bc009840 con 0x5608c643b330 2024-03-18T23:15:23.191 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:23.186+0000 7fc5d6922740 10 client.25847 fetch_fsmap learned FSMap version 32 2024-03-18T23:15:23.191 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:23.186+0000 7fc5d6922740 1 -- 172.21.15.49:0/3768616158 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5608c64bf320 con 0x5608c643b330 2024-03-18T23:15:23.191 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:23.186+0000 7fc5b57fa640 1 -- 172.21.15.49:0/3768616158 <== mon.2 v2:172.21.15.139:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x5608c63a6470 con 0x5608c643b330 2024-03-18T23:15:23.191 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:23.187+0000 7fc5b57fa640 1 -- 172.21.15.49:0/3768616158 <== mon.2 v2:172.21.15.139:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7fc5bc01f940 con 0x5608c643b330 2024-03-18T23:15:23.191 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:23.187+0000 7fc5d6922740 10 client.25847 fetch_fsmap finished waiting for FSMap version 32 2024-03-18T23:15:23.191 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:23.187+0000 7fc5d6922740 10 client.25847 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T23:15:23.191 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:23.187+0000 7fc5d6922740 20 client.25847 populate_metadata read hostname 'smithi049' 2024-03-18T23:15:23.191 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:23.187+0000 7fc5d6922740 1 --2- 172.21.15.49:0/3768616158 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x5608c64bfa60 0x5608c64c1e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:15:23.192 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:23.187+0000 7fc5d6922740 4 client.25847 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T23:15:23.192 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:23.187+0000 7fc5d6922740 1 -- 172.21.15.49:0/3768616158 --> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5608c6324770 con 0x5608c64bfa60 2024-03-18T23:15:23.192 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:23.187+0000 7fc5d1e33640 1 --2- 172.21.15.49:0/3768616158 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x5608c64bfa60 0x5608c64c1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:15:23.192 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:23.187+0000 7fc5b57fa640 10 client.25847 ms_handle_connect on v2:172.21.15.139:6838/4280128209 2024-03-18T23:15:23.192 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:23.187+0000 7fc5d1e33640 1 --2- 172.21.15.49:0/3768616158 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x5608c64bfa60 0x5608c64c1e80 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=deb5ce4ce9227e31 server_cookie=760591a2f724436c in_seq=0 out_seq=0 2024-03-18T23:15:23.396 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:23.395+0000 7fc5b57fa640 1 -- 172.21.15.49:0/3768616158 <== mds.0 v2:172.21.15.139:6838/4280128209 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5608c6324770 con 0x5608c64bfa60 2024-03-18T23:15:23.396 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:23.395+0000 7fc5b57fa640 10 client.25847 handle_command_reply: tid=0 2024-03-18T23:15:23.396 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:23.395+0000 7fc5d6922740 1 client.25847 shutdown 2024-03-18T23:15:23.396 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:23.395+0000 7fc5d6922740 1 -- 172.21.15.49:0/3768616158 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5608c643b330 msgr2=0x5608c6442af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:15:23.396 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:23.395+0000 7fc5d6922740 1 --2- 172.21.15.49:0/3768616158 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5608c643b330 0x5608c6442af0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fc5bc0108a0 tx=0x7fc5bc011f70 comp rx=0 tx=0).stop 2024-03-18T23:15:23.396 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:23.395+0000 7fc5d6922740 1 -- 172.21.15.49:0/3768616158 shutdown_connections 2024-03-18T23:15:23.397 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:23.395+0000 7fc5d6922740 1 --2- 172.21.15.49:0/3768616158 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x5608c64bfa60 0x5608c64c1e80 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:23.397 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:23.396+0000 7fc5d6922740 1 --2- 172.21.15.49:0/3768616158 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5608c64489f0 0x5608c64bdca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:23.397 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:23.396+0000 7fc5d6922740 1 --2- 172.21.15.49:0/3768616158 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5608c643b330 0x5608c6442af0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:23.397 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:23.396+0000 7fc5b57fa640 0 client.25847 ms_handle_reset on v2:172.21.15.139:6838/4280128209 2024-03-18T23:15:23.397 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:23.396+0000 7fc5d6922740 1 --2- 172.21.15.49:0/3768616158 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5608c643aa10 0x5608c64425b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:23.397 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:23.396+0000 7fc5d6922740 1 -- 172.21.15.49:0/3768616158 >> 172.21.15.49:0/3768616158 conn(0x5608c6435810 msgr2=0x5608c6434220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:15:23.397 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:23.396+0000 7fc5d6922740 1 -- 172.21.15.49:0/3768616158 shutdown_connections 2024-03-18T23:15:23.397 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:23.396+0000 7fc5d6922740 1 -- 172.21.15.49:0/3768616158 wait complete. 2024-03-18T23:15:23.397 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:23.396+0000 7fc5d6922740 20 client.25847 trim_cache size 0 max 16384 2024-03-18T23:15:23.397 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:{ 2024-03-18T23:15:23.398 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "message": "", 2024-03-18T23:15:23.398 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "return_code": 0 2024-03-18T23:15:23.398 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:} 2024-03-18T23:15:23.398 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:23.397+0000 7fc5d4097640 1 -- 172.21.15.49:0/2952419652 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7fc5b80a2a10 msgr2=0x7fc5b80a4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:15:23.398 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:23.397+0000 7fc5d4097640 1 --2- 172.21.15.49:0/2952419652 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7fc5b80a2a10 0x7fc5b80a4ed0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fc5bc009fd0 tx=0x7fc5bc009b80 comp rx=0 tx=0).stop 2024-03-18T23:15:23.398 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:23.397+0000 7fc5d4097640 1 -- 172.21.15.49:0/2952419652 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc5cc1078f0 msgr2=0x7fc5cc19c830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:15:23.398 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:23.397+0000 7fc5d4097640 1 --2- 172.21.15.49:0/2952419652 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc5cc1078f0 0x7fc5cc19c830 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fc5c402c630 tx=0x7fc5c4032f20 comp rx=0 tx=0).stop 2024-03-18T23:15:23.399 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:23.397+0000 7fc5d4097640 1 -- 172.21.15.49:0/2952419652 shutdown_connections 2024-03-18T23:15:23.399 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:23.397+0000 7fc5d4097640 1 --2- 172.21.15.49:0/2952419652 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7fc5b80a2a10 0x7fc5b80a4ed0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:23.399 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:23.397+0000 7fc5d4097640 1 --2- 172.21.15.49:0/2952419652 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc5cc1078f0 0x7fc5cc19c830 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:23.399 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:23.397+0000 7fc5d4097640 1 --2- 172.21.15.49:0/2952419652 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc5cc069840 0x7fc5cc198500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:23.399 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:23.397+0000 7fc5d4097640 1 --2- 172.21.15.49:0/2952419652 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc5cc068ea0 0x7fc5cc197fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:23.399 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:23.397+0000 7fc5d4097640 1 -- 172.21.15.49:0/2952419652 >> 172.21.15.49:0/2952419652 conn(0x7fc5cc077570 msgr2=0x7fc5cc0fe9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:15:23.399 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:23.397+0000 7fc5d4097640 1 -- 172.21.15.49:0/2952419652 shutdown_connections 2024-03-18T23:15:23.399 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:23.397+0000 7fc5d4097640 1 -- 172.21.15.49:0/2952419652 wait complete. 2024-03-18T23:15:23.462 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:15:23 smithi175 ceph-mon[67352]: pgmap v220: 97 pgs: 97 active+clean; 82 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 4.3 MiB/s wr, 337 op/s 2024-03-18T23:15:23.564 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:15:23 smithi139 ceph-mon[67282]: pgmap v220: 97 pgs: 97 active+clean; 82 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 4.3 MiB/s wr, 337 op/s 2024-03-18T23:15:23.629 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:15:23 smithi049 ceph-mon[62991]: pgmap v220: 97 pgs: 97 active+clean; 82 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 4.3 MiB/s wr, 337 op/s 2024-03-18T23:15:23.776 INFO:tasks.workunit.client.0.smithi049.stdout: 1 768463 10.01 MB/sec execute 265 sec latency 83.193 ms 2024-03-18T23:15:24.776 INFO:tasks.workunit.client.0.smithi049.stdout: 1 770639 10.02 MB/sec execute 266 sec latency 37.553 ms 2024-03-18T23:15:25.431 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:15:25 smithi049 ceph-mon[62991]: pgmap v221: 97 pgs: 97 active+clean; 82 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.8 MiB/s wr, 179 op/s 2024-03-18T23:15:25.460 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:15:25 smithi175 ceph-mon[67352]: pgmap v221: 97 pgs: 97 active+clean; 82 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.8 MiB/s wr, 179 op/s 2024-03-18T23:15:25.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:15:25 smithi139 ceph-mon[67282]: pgmap v221: 97 pgs: 97 active+clean; 82 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.8 MiB/s wr, 179 op/s 2024-03-18T23:15:25.776 INFO:tasks.workunit.client.0.smithi049.stdout: 1 772158 10.00 MB/sec execute 267 sec latency 32.980 ms 2024-03-18T23:15:26.776 INFO:tasks.workunit.client.0.smithi049.stdout: 1 774348 10.02 MB/sec execute 268 sec latency 34.042 ms 2024-03-18T23:15:27.460 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:15:27 smithi175 ceph-mon[67352]: pgmap v222: 97 pgs: 97 active+clean; 82 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.8 MiB/s wr, 179 op/s 2024-03-18T23:15:27.564 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:15:27 smithi139 ceph-mon[67282]: pgmap v222: 97 pgs: 97 active+clean; 82 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.8 MiB/s wr, 179 op/s 2024-03-18T23:15:27.629 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:15:27 smithi049 ceph-mon[62991]: pgmap v222: 97 pgs: 97 active+clean; 82 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.8 MiB/s wr, 179 op/s 2024-03-18T23:15:27.776 INFO:tasks.workunit.client.0.smithi049.stdout: 1 775947 10.00 MB/sec execute 269 sec latency 21.876 ms 2024-03-18T23:15:28.460 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:15:28 smithi175 ceph-mon[67352]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:15:28.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:15:28 smithi139 ceph-mon[67282]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:15:28.629 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:15:28 smithi049 ceph-mon[62991]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:15:28.776 INFO:tasks.workunit.client.0.smithi049.stdout: 1 777869 10.01 MB/sec execute 270 sec latency 68.202 ms 2024-03-18T23:15:29.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:15:29 smithi175 ceph-mon[67352]: pgmap v223: 97 pgs: 97 active+clean; 72 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.2 MiB/s wr, 357 op/s 2024-03-18T23:15:29.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:15:29 smithi175 ceph-mon[67352]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:15:29.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:15:29 smithi175 ceph-mon[67352]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:15:29.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:15:29 smithi175 ceph-mon[67352]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' 2024-03-18T23:15:29.564 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:15:29 smithi139 ceph-mon[67282]: pgmap v223: 97 pgs: 97 active+clean; 72 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.2 MiB/s wr, 357 op/s 2024-03-18T23:15:29.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:15:29 smithi139 ceph-mon[67282]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:15:29.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:15:29 smithi139 ceph-mon[67282]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:15:29.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:15:29 smithi139 ceph-mon[67282]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' 2024-03-18T23:15:29.629 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:15:29 smithi049 ceph-mon[62991]: pgmap v223: 97 pgs: 97 active+clean; 72 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.2 MiB/s wr, 357 op/s 2024-03-18T23:15:29.629 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:15:29 smithi049 ceph-mon[62991]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:15:29.629 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:15:29 smithi049 ceph-mon[62991]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:15:29.629 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:15:29 smithi049 ceph-mon[62991]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' 2024-03-18T23:15:29.776 INFO:tasks.workunit.client.0.smithi049.stdout: 1 779285 10.00 MB/sec execute 271 sec latency 25.116 ms 2024-03-18T23:15:30.776 INFO:tasks.workunit.client.0.smithi049.stdout: 1 781502 10.01 MB/sec execute 272 sec latency 23.177 ms 2024-03-18T23:15:31.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:15:31 smithi175 ceph-mon[67352]: pgmap v224: 97 pgs: 97 active+clean; 72 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.9 MiB/s wr, 268 op/s 2024-03-18T23:15:31.564 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:15:31 smithi139 ceph-mon[67282]: pgmap v224: 97 pgs: 97 active+clean; 72 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.9 MiB/s wr, 268 op/s 2024-03-18T23:15:31.629 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:15:31 smithi049 ceph-mon[62991]: pgmap v224: 97 pgs: 97 active+clean; 72 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.9 MiB/s wr, 268 op/s 2024-03-18T23:15:31.776 INFO:tasks.workunit.client.0.smithi049.stdout: 1 783033 10.00 MB/sec execute 273 sec latency 21.037 ms 2024-03-18T23:15:32.776 INFO:tasks.workunit.client.0.smithi049.stdout: 1 785253 10.01 MB/sec execute 274 sec latency 21.513 ms 2024-03-18T23:15:33.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:15:33 smithi175 ceph-mon[67352]: pgmap v225: 97 pgs: 97 active+clean; 78 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 4.2 MiB/s wr, 358 op/s 2024-03-18T23:15:33.564 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:15:33 smithi139 ceph-mon[67282]: pgmap v225: 97 pgs: 97 active+clean; 78 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 4.2 MiB/s wr, 358 op/s 2024-03-18T23:15:33.629 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:15:33 smithi049 ceph-mon[62991]: pgmap v225: 97 pgs: 97 active+clean; 78 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 4.2 MiB/s wr, 358 op/s 2024-03-18T23:15:33.776 INFO:tasks.workunit.client.0.smithi049.stdout: 1 787017 10.00 MB/sec execute 275 sec latency 96.131 ms 2024-03-18T23:15:34.776 INFO:tasks.workunit.client.0.smithi049.stdout: 1 788894 10.01 MB/sec execute 276 sec latency 18.110 ms 2024-03-18T23:15:35.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:15:35 smithi175 ceph-mon[67352]: pgmap v226: 97 pgs: 97 active+clean; 78 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.7 MiB/s wr, 267 op/s 2024-03-18T23:15:35.564 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:15:35 smithi139 ceph-mon[67282]: pgmap v226: 97 pgs: 97 active+clean; 78 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.7 MiB/s wr, 267 op/s 2024-03-18T23:15:35.629 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:15:35 smithi049 ceph-mon[62991]: pgmap v226: 97 pgs: 97 active+clean; 78 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.7 MiB/s wr, 267 op/s 2024-03-18T23:15:35.776 INFO:tasks.workunit.client.0.smithi049.stdout: 1 790811 10.00 MB/sec execute 277 sec latency 24.437 ms 2024-03-18T23:15:36.493 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:36.490+0000 7fc4cf95b640 1 -- 172.21.15.49:0/2479544072 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc4c8068ea0 msgr2=0x7fc4c8069320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:15:36.493 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:36.490+0000 7fc4cf95b640 1 --2- 172.21.15.49:0/2479544072 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc4c8068ea0 0x7fc4c8069320 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fc4b8009930 tx=0x7fc4b802bc30 comp rx=0 tx=0).stop 2024-03-18T23:15:36.493 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:36.490+0000 7fc4cf95b640 1 -- 172.21.15.49:0/2479544072 shutdown_connections 2024-03-18T23:15:36.494 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:36.491+0000 7fc4cf95b640 1 --2- 172.21.15.49:0/2479544072 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc4c8069860 0x7fc4c810e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:36.494 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:36.491+0000 7fc4cf95b640 1 --2- 172.21.15.49:0/2479544072 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc4c8068ea0 0x7fc4c8069320 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:36.494 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:36.491+0000 7fc4cf95b640 1 --2- 172.21.15.49:0/2479544072 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc4c81077a0 0x7fc4c8107b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:36.494 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:36.491+0000 7fc4cf95b640 1 -- 172.21.15.49:0/2479544072 >> 172.21.15.49:0/2479544072 conn(0x7fc4c80774b0 msgr2=0x7fc4c80778c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:15:36.494 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:36.491+0000 7fc4cf95b640 1 -- 172.21.15.49:0/2479544072 shutdown_connections 2024-03-18T23:15:36.494 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:36.491+0000 7fc4cf95b640 1 -- 172.21.15.49:0/2479544072 wait complete. 2024-03-18T23:15:36.495 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:36.491+0000 7fc4cf95b640 1 Processor -- start 2024-03-18T23:15:36.495 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:36.491+0000 7fc4cf95b640 1 -- start start 2024-03-18T23:15:36.495 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:36.491+0000 7fc4cf95b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc4c8068ea0 0x7fc4c8100a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:15:36.495 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:36.491+0000 7fc4cf95b640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc4c8069860 0x7fc4c8100f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:15:36.495 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:36.491+0000 7fc4cf95b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc4c81077a0 0x7fc4c81014c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:15:36.496 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:36.491+0000 7fc4cf95b640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fc4c8111bb0 con 0x7fc4c81077a0 2024-03-18T23:15:36.496 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:36.491+0000 7fc4cf95b640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fc4c8111a30 con 0x7fc4c8068ea0 2024-03-18T23:15:36.496 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:36.491+0000 7fc4cf95b640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc4c8111d30 con 0x7fc4c8069860 2024-03-18T23:15:36.496 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:36.492+0000 7fc4cdef8640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc4c81077a0 0x7fc4c81014c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:15:36.496 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:36.492+0000 7fc4ccef6640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc4c8069860 0x7fc4c8100f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:15:36.496 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:36.492+0000 7fc4cdef8640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc4c81077a0 0x7fc4c81014c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:59174/0 (socket says 172.21.15.49:59174) 2024-03-18T23:15:36.497 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:36.492+0000 7fc4cd6f7640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc4c8068ea0 0x7fc4c8100a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:15:36.497 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:36.492+0000 7fc4cdef8640 1 -- 172.21.15.49:0/3955386042 learned_addr learned my addr 172.21.15.49:0/3955386042 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:15:36.497 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:36.492+0000 7fc4cdef8640 1 -- 172.21.15.49:0/3955386042 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc4c8069860 msgr2=0x7fc4c8100f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:15:36.497 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:36.492+0000 7fc4cdef8640 1 --2- 172.21.15.49:0/3955386042 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc4c8069860 0x7fc4c8100f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:36.497 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:36.492+0000 7fc4cdef8640 1 -- 172.21.15.49:0/3955386042 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc4c8068ea0 msgr2=0x7fc4c8100a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:15:36.497 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:36.492+0000 7fc4cdef8640 1 --2- 172.21.15.49:0/3955386042 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc4c8068ea0 0x7fc4c8100a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:36.498 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:36.492+0000 7fc4cdef8640 1 -- 172.21.15.49:0/3955386042 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4c81a6510 con 0x7fc4c81077a0 2024-03-18T23:15:36.498 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:36.492+0000 7fc4cdef8640 1 --2- 172.21.15.49:0/3955386042 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc4c81077a0 0x7fc4c81014c0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fc4c0009fd0 tx=0x7fc4c000b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:15:36.498 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:36.492+0000 7fc4be7fc640 1 -- 172.21.15.49:0/3955386042 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4c0017070 con 0x7fc4c81077a0 2024-03-18T23:15:36.498 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:36.492+0000 7fc4be7fc640 1 -- 172.21.15.49:0/3955386042 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc4c0003e50 con 0x7fc4c81077a0 2024-03-18T23:15:36.498 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:36.492+0000 7fc4cf95b640 1 -- 172.21.15.49:0/3955386042 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4c81a6800 con 0x7fc4c81077a0 2024-03-18T23:15:36.499 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:36.492+0000 7fc4be7fc640 1 -- 172.21.15.49:0/3955386042 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4c00106c0 con 0x7fc4c81077a0 2024-03-18T23:15:36.499 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:36.492+0000 7fc4cf95b640 1 -- 172.21.15.49:0/3955386042 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4c81a6cd0 con 0x7fc4c81077a0 2024-03-18T23:15:36.507 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:36.494+0000 7fc4be7fc640 1 -- 172.21.15.49:0/3955386042 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 25) ==== 137421+0+0 (secure 0 0 0) 0x7fc4c0003970 con 0x7fc4c81077a0 2024-03-18T23:15:36.507 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:36.495+0000 7fc4be7fc640 1 --2- 172.21.15.49:0/3955386042 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7fc4b40a2a10 0x7fc4b40a4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:15:36.507 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:36.495+0000 7fc4cd6f7640 1 --2- 172.21.15.49:0/3955386042 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7fc4b40a2a10 0x7fc4b40a4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:15:36.507 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:36.495+0000 7fc4be7fc640 1 -- 172.21.15.49:0/3955386042 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7fc4c00ceb90 con 0x7fc4c81077a0 2024-03-18T23:15:36.507 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:36.495+0000 7fc4d2625740 1 Processor -- start 2024-03-18T23:15:36.507 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:36.495+0000 7fc4d2625740 1 -- start start 2024-03-18T23:15:36.507 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:36.496+0000 7fc4cd6f7640 1 --2- 172.21.15.49:0/3955386042 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7fc4b40a2a10 0x7fc4b40a4ed0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fc4c4009b50 tx=0x7fc4c4005f90 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:15:36.507 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:36.496+0000 7fc4d2625740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556d409b9ef0 0x556d409ba2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:15:36.507 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:36.496+0000 7fc4d2625740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556d409ba810 0x556d409c30a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:15:36.507 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:36.496+0000 7fc4d2625740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556d409c35e0 0x556d409c59d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:15:36.508 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:36.496+0000 7fc4d2625740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x556d4091dbe0 con 0x556d409b9ef0 2024-03-18T23:15:36.508 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:36.496+0000 7fc4d2625740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x556d408c6960 con 0x556d409ba810 2024-03-18T23:15:36.508 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:36.496+0000 7fc4d2625740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x556d40919c80 con 0x556d409c35e0 2024-03-18T23:15:36.508 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:36.496+0000 7fc4cd6f7640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556d409b9ef0 0x556d409ba2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:15:36.508 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:36.496+0000 7fc4cd6f7640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556d409b9ef0 0x556d409ba2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:59182/0 (socket says 172.21.15.49:59182) 2024-03-18T23:15:36.508 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:36.496+0000 7fc4cd6f7640 1 -- 172.21.15.49:0/1164462762 learned_addr learned my addr 172.21.15.49:0/1164462762 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:15:36.508 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:36.496+0000 7fc4ccef6640 1 --2- 172.21.15.49:0/1164462762 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556d409ba810 0x556d409c30a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:15:36.508 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:36.496+0000 7fc4cdef8640 1 --2- 172.21.15.49:0/1164462762 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556d409c35e0 0x556d409c59d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:15:36.508 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:36.496+0000 7fc4cd6f7640 1 -- 172.21.15.49:0/1164462762 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556d409c35e0 msgr2=0x556d409c59d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:15:36.508 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:36.496+0000 7fc4cd6f7640 1 --2- 172.21.15.49:0/1164462762 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556d409c35e0 0x556d409c59d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:36.508 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:36.496+0000 7fc4cd6f7640 1 -- 172.21.15.49:0/1164462762 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556d409ba810 msgr2=0x556d409c30a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:15:36.508 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:36.496+0000 7fc4cd6f7640 1 --2- 172.21.15.49:0/1164462762 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556d409ba810 0x556d409c30a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:36.508 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:36.496+0000 7fc4cd6f7640 1 -- 172.21.15.49:0/1164462762 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556d408bdd20 con 0x556d409b9ef0 2024-03-18T23:15:36.508 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:36.496+0000 7fc4cd6f7640 1 --2- 172.21.15.49:0/1164462762 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556d409b9ef0 0x556d409ba2d0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fc4c4005bc0 tx=0x7fc4c4014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f85a961025249dc server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:15:36.509 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:36.496+0000 7fc4b37fe640 1 -- 172.21.15.49:0/1164462762 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4c401e420 con 0x556d409b9ef0 2024-03-18T23:15:36.509 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:36.496+0000 7fc4b37fe640 1 -- 172.21.15.49:0/1164462762 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc4c401eaa0 con 0x556d409b9ef0 2024-03-18T23:15:36.509 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:36.497+0000 7fc4b37fe640 1 -- 172.21.15.49:0/1164462762 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4c401db70 con 0x556d409b9ef0 2024-03-18T23:15:36.509 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:36.497+0000 7fc4d2625740 1 -- 172.21.15.49:0/1164462762 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556d409b9ef0 msgr2=0x556d409ba2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:15:36.509 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:36.497+0000 7fc4d2625740 1 --2- 172.21.15.49:0/1164462762 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556d409b9ef0 0x556d409ba2d0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fc4c4005bc0 tx=0x7fc4c4014000 comp rx=0 tx=0).stop 2024-03-18T23:15:36.509 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:36.497+0000 7fc4d2625740 1 -- 172.21.15.49:0/1164462762 shutdown_connections 2024-03-18T23:15:36.509 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:36.497+0000 7fc4d2625740 1 --2- 172.21.15.49:0/1164462762 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556d409c35e0 0x556d409c59d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:36.509 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:36.497+0000 7fc4d2625740 1 --2- 172.21.15.49:0/1164462762 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556d409ba810 0x556d409c30a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:36.509 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:36.497+0000 7fc4d2625740 1 --2- 172.21.15.49:0/1164462762 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556d409b9ef0 0x556d409ba2d0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:36.509 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:36.497+0000 7fc4d2625740 1 -- 172.21.15.49:0/1164462762 >> 172.21.15.49:0/1164462762 conn(0x556d409b4810 msgr2=0x556d409b4bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:15:36.509 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:36.497+0000 7fc4d2625740 1 -- 172.21.15.49:0/1164462762 shutdown_connections 2024-03-18T23:15:36.509 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:36.497+0000 7fc4d2625740 1 -- 172.21.15.49:0/1164462762 wait complete. 2024-03-18T23:15:36.509 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:36.498+0000 7fc4d2625740 1 -- start start 2024-03-18T23:15:36.509 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:36.498+0000 7fc4d2625740 1 Processor -- start 2024-03-18T23:15:36.509 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:36.498+0000 7fc4d2625740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556d409b9ef0 0x556d409c2fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:15:36.510 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:36.498+0000 7fc4d2625740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556d409ba810 0x556d409c0330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:15:36.510 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:36.498+0000 7fc4d2625740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556d409c35e0 0x556d409c0940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:15:36.510 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:36.498+0000 7fc4d2625740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x556d409c6210 con 0x556d409ba810 2024-03-18T23:15:36.510 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:36.498+0000 7fc4d2625740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x556d408d9db0 con 0x556d409b9ef0 2024-03-18T23:15:36.510 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:36.498+0000 7fc4d2625740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x556d409c6390 con 0x556d409c35e0 2024-03-18T23:15:36.510 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:36.498+0000 7fc4d2625740 10 client.0 _ll_register_callbacks cb 0x556d408dee70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T23:15:36.510 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:36.498+0000 7fc4ccef6640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556d409ba810 0x556d409c0330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:15:36.510 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:36.498+0000 7fc4ccef6640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556d409ba810 0x556d409c0330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:59198/0 (socket says 172.21.15.49:59198) 2024-03-18T23:15:36.510 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:36.498+0000 7fc4cd6f7640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556d409b9ef0 0x556d409c2fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:15:36.510 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:36.498+0000 7fc4ccef6640 1 -- 172.21.15.49:0/1534484073 learned_addr learned my addr 172.21.15.49:0/1534484073 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:15:36.510 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:36.498+0000 7fc4cdef8640 1 --2- 172.21.15.49:0/1534484073 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556d409c35e0 0x556d409c0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:15:36.510 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:36.498+0000 7fc4ccef6640 1 -- 172.21.15.49:0/1534484073 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556d409c35e0 msgr2=0x556d409c0940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:15:36.510 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:36.498+0000 7fc4ccef6640 1 --2- 172.21.15.49:0/1534484073 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556d409c35e0 0x556d409c0940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:36.511 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:36.498+0000 7fc4ccef6640 1 -- 172.21.15.49:0/1534484073 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556d409b9ef0 msgr2=0x556d409c2fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:15:36.511 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:36.498+0000 7fc4ccef6640 1 --2- 172.21.15.49:0/1534484073 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556d409b9ef0 0x556d409c2fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:36.511 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:36.498+0000 7fc4ccef6640 1 -- 172.21.15.49:0/1534484073 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556d408d8c60 con 0x556d409ba810 2024-03-18T23:15:36.511 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:36.499+0000 7fc4ccef6640 1 --2- 172.21.15.49:0/1534484073 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556d409ba810 0x556d409c0330 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fc4b802cfd0 tx=0x7fc4b802ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4accd2e86f7e4e server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:15:36.511 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:36.499+0000 7fc4b0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-18T23:15:36.511 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:36.499+0000 7fc4b0ff9640 1 -- 172.21.15.49:0/1534484073 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4b8004240 con 0x556d409ba810 2024-03-18T23:15:36.511 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:36.499+0000 7fc4b0ff9640 1 -- 172.21.15.49:0/1534484073 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7fc4b80043e0 con 0x556d409ba810 2024-03-18T23:15:36.511 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:36.499+0000 7fc4d2625740 1 -- 172.21.15.49:0/1534484073 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc4c401eaa0 con 0x556d409ba810 2024-03-18T23:15:36.511 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:36.499+0000 7fc4b0ff9640 1 -- 172.21.15.49:0/1534484073 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fc4b80046c0 con 0x556d409ba810 2024-03-18T23:15:36.511 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:36.499+0000 7fc4d2625740 10 client.16047 fetch_fsmap learned FSMap version 32 2024-03-18T23:15:36.511 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:36.499+0000 7fc4d2625740 1 -- 172.21.15.49:0/1534484073 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x556d40632210 con 0x556d409ba810 2024-03-18T23:15:36.512 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:36.499+0000 7fc4b0ff9640 1 -- 172.21.15.49:0/1534484073 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc4c401eaa0 con 0x556d409ba810 2024-03-18T23:15:36.512 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:36.499+0000 7fc4b0ff9640 1 -- 172.21.15.49:0/1534484073 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7fc4b80035f0 con 0x556d409ba810 2024-03-18T23:15:36.512 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:36.499+0000 7fc4d2625740 10 client.16047 fetch_fsmap finished waiting for FSMap version 32 2024-03-18T23:15:36.512 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:36.499+0000 7fc4d2625740 10 client.16047 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T23:15:36.512 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:36.499+0000 7fc4d2625740 20 client.16047 populate_metadata read hostname 'smithi049' 2024-03-18T23:15:36.512 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:36.499+0000 7fc4d2625740 1 --2- 172.21.15.49:0/1534484073 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x556d409d3490 0x556d409d58a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:15:36.512 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:36.499+0000 7fc4d2625740 4 client.16047 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T23:15:36.512 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:36.499+0000 7fc4d2625740 1 -- 172.21.15.49:0/1534484073 --> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556d4092b120 con 0x556d409d3490 2024-03-18T23:15:36.512 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:36.500+0000 7fc4cd6f7640 1 --2- 172.21.15.49:0/1534484073 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x556d409d3490 0x556d409d58a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:15:36.512 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:36.500+0000 7fc4cd6f7640 1 --2- 172.21.15.49:0/1534484073 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x556d409d3490 0x556d409d58a0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=db36de94f1c6076b server_cookie=f0fafdb6186790d in_seq=0 out_seq=0 2024-03-18T23:15:36.512 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:36.500+0000 7fc4b0ff9640 10 client.16047 ms_handle_connect on v2:172.21.15.139:6838/4280128209 2024-03-18T23:15:36.512 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:36.502+0000 7fc4b0ff9640 1 -- 172.21.15.49:0/1534484073 <== mds.0 v2:172.21.15.139:6838/4280128209 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x556d4092b120 con 0x556d409d3490 2024-03-18T23:15:36.512 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:36.502+0000 7fc4b0ff9640 10 client.16047 handle_command_reply: tid=0 2024-03-18T23:15:36.512 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:36.502+0000 7fc4d2625740 1 client.16047 shutdown 2024-03-18T23:15:36.513 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:36.502+0000 7fc4d2625740 1 -- 172.21.15.49:0/1534484073 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556d409ba810 msgr2=0x556d409c0330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:15:36.513 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:36.502+0000 7fc4d2625740 1 --2- 172.21.15.49:0/1534484073 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556d409ba810 0x556d409c0330 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fc4b802cfd0 tx=0x7fc4b802ce30 comp rx=0 tx=0).stop 2024-03-18T23:15:36.513 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:36.502+0000 7fc4d2625740 1 -- 172.21.15.49:0/1534484073 shutdown_connections 2024-03-18T23:15:36.513 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:36.502+0000 7fc4d2625740 1 --2- 172.21.15.49:0/1534484073 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x556d409d3490 0x556d409d58a0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:36.513 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:36.502+0000 7fc4d2625740 1 --2- 172.21.15.49:0/1534484073 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556d409c35e0 0x556d409c0940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:36.513 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:36.502+0000 7fc4d2625740 1 --2- 172.21.15.49:0/1534484073 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556d409ba810 0x556d409c0330 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:36.513 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:36.502+0000 7fc4b0ff9640 0 client.16047 ms_handle_reset on v2:172.21.15.139:6838/4280128209 2024-03-18T23:15:36.513 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:36.502+0000 7fc4d2625740 1 --2- 172.21.15.49:0/1534484073 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556d409b9ef0 0x556d409c2fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:36.513 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:36.502+0000 7fc4d2625740 1 -- 172.21.15.49:0/1534484073 >> 172.21.15.49:0/1534484073 conn(0x556d409b4810 msgr2=0x556d409b3ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:15:36.513 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:36.502+0000 7fc4d2625740 1 -- 172.21.15.49:0/1534484073 shutdown_connections 2024-03-18T23:15:36.513 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:36.502+0000 7fc4d2625740 1 -- 172.21.15.49:0/1534484073 wait complete. 2024-03-18T23:15:36.513 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:36.503+0000 7fc4d2625740 20 client.16047 trim_cache size 0 max 16384 2024-03-18T23:15:36.522 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:36.517+0000 7fc4d2625740 1 Processor -- start 2024-03-18T23:15:36.522 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:36.517+0000 7fc4d2625740 1 -- start start 2024-03-18T23:15:36.522 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:36.517+0000 7fc4d2625740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556d409b9ef0 0x556d409cefb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:15:36.522 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:36.517+0000 7fc4d2625740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556d409ba810 0x556d409db1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:15:36.522 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:36.517+0000 7fc4d2625740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556d409c35e0 0x556d409c2a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:15:36.522 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:36.517+0000 7fc4d2625740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fc4c81af670 con 0x556d409b9ef0 2024-03-18T23:15:36.522 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:36.517+0000 7fc4d2625740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fc4c81b11b0 con 0x556d409ba810 2024-03-18T23:15:36.522 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:36.517+0000 7fc4d2625740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc4c81afd40 con 0x556d409c35e0 2024-03-18T23:15:36.522 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:36.518+0000 7fc4cd6f7640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556d409b9ef0 0x556d409cefb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:15:36.523 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:36.518+0000 7fc4cd6f7640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556d409b9ef0 0x556d409cefb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:59208/0 (socket says 172.21.15.49:59208) 2024-03-18T23:15:36.523 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:36.518+0000 7fc4cd6f7640 1 -- 172.21.15.49:0/4018159322 learned_addr learned my addr 172.21.15.49:0/4018159322 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:15:36.523 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:36.518+0000 7fc4cdef8640 1 --2- 172.21.15.49:0/4018159322 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556d409c35e0 0x556d409c2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:15:36.523 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:36.518+0000 7fc4ccef6640 1 --2- 172.21.15.49:0/4018159322 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556d409ba810 0x556d409db1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:15:36.523 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:36.518+0000 7fc4cd6f7640 1 -- 172.21.15.49:0/4018159322 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556d409c35e0 msgr2=0x556d409c2a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:15:36.523 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:36.518+0000 7fc4cd6f7640 1 --2- 172.21.15.49:0/4018159322 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556d409c35e0 0x556d409c2a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:36.523 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:36.518+0000 7fc4cd6f7640 1 -- 172.21.15.49:0/4018159322 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556d409ba810 msgr2=0x556d409db1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:15:36.523 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:36.518+0000 7fc4cd6f7640 1 --2- 172.21.15.49:0/4018159322 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556d409ba810 0x556d409db1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:36.523 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:36.518+0000 7fc4cd6f7640 1 -- 172.21.15.49:0/4018159322 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556d409c0cb0 con 0x556d409b9ef0 2024-03-18T23:15:36.523 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:36.518+0000 7fc4cd6f7640 1 --2- 172.21.15.49:0/4018159322 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556d409b9ef0 0x556d409cefb0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fc4c40042e0 tx=0x7fc4c4004510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c08ca42da0b813f server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:15:36.523 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:36.518+0000 7fc4b2ffd640 1 -- 172.21.15.49:0/4018159322 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4c4039070 con 0x556d409b9ef0 2024-03-18T23:15:36.523 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:36.518+0000 7fc4b2ffd640 1 -- 172.21.15.49:0/4018159322 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc4c401e420 con 0x556d409b9ef0 2024-03-18T23:15:36.524 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:36.518+0000 7fc4b2ffd640 1 -- 172.21.15.49:0/4018159322 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4c401da20 con 0x556d409b9ef0 2024-03-18T23:15:36.524 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:36.519+0000 7fc4d2625740 1 -- 172.21.15.49:0/4018159322 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556d409b9ef0 msgr2=0x556d409cefb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:15:36.524 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:36.519+0000 7fc4d2625740 1 --2- 172.21.15.49:0/4018159322 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556d409b9ef0 0x556d409cefb0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fc4c40042e0 tx=0x7fc4c4004510 comp rx=0 tx=0).stop 2024-03-18T23:15:36.524 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:36.519+0000 7fc4d2625740 1 -- 172.21.15.49:0/4018159322 shutdown_connections 2024-03-18T23:15:36.524 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:36.519+0000 7fc4d2625740 1 --2- 172.21.15.49:0/4018159322 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556d409c35e0 0x556d409c2a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:36.524 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:36.519+0000 7fc4d2625740 1 --2- 172.21.15.49:0/4018159322 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556d409ba810 0x556d409db1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:36.524 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:36.519+0000 7fc4d2625740 1 --2- 172.21.15.49:0/4018159322 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556d409b9ef0 0x556d409cefb0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:36.524 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:36.519+0000 7fc4d2625740 1 -- 172.21.15.49:0/4018159322 >> 172.21.15.49:0/4018159322 conn(0x556d409b4810 msgr2=0x556d409b3ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:15:36.524 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:36.519+0000 7fc4d2625740 1 -- 172.21.15.49:0/4018159322 shutdown_connections 2024-03-18T23:15:36.524 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:36.519+0000 7fc4d2625740 1 -- 172.21.15.49:0/4018159322 wait complete. 2024-03-18T23:15:36.524 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:36.519+0000 7fc4d2625740 1 -- start start 2024-03-18T23:15:36.524 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:36.519+0000 7fc4d2625740 1 Processor -- start 2024-03-18T23:15:36.524 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:36.520+0000 7fc4d2625740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556d409b9ef0 0x556d409c7270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:15:36.524 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:36.520+0000 7fc4d2625740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556d409ba810 0x556d409c77b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:15:36.525 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:36.520+0000 7fc4d2625740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556d409c35e0 0x556d40a387e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:15:36.525 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:36.520+0000 7fc4d2625740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fc4c81b0ae0 con 0x556d409c35e0 2024-03-18T23:15:36.525 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:36.520+0000 7fc4d2625740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fc4c81b0410 con 0x556d409ba810 2024-03-18T23:15:36.525 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:36.520+0000 7fc4d2625740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc4c81ad670 con 0x556d409b9ef0 2024-03-18T23:15:36.525 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:36.520+0000 7fc4d2625740 10 client.0 _ll_register_callbacks cb 0x7fc4b8009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T23:15:36.525 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:36.520+0000 7fc4cdef8640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556d409c35e0 0x556d40a387e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:15:36.525 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:36.520+0000 7fc4cdef8640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556d409c35e0 0x556d40a387e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:59216/0 (socket says 172.21.15.49:59216) 2024-03-18T23:15:36.525 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:36.520+0000 7fc4cdef8640 1 -- 172.21.15.49:0/3478012164 learned_addr learned my addr 172.21.15.49:0/3478012164 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:15:36.525 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:36.520+0000 7fc4cd6f7640 1 --2- 172.21.15.49:0/3478012164 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556d409b9ef0 0x556d409c7270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:15:36.526 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:36.520+0000 7fc4ccef6640 1 --2- 172.21.15.49:0/3478012164 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556d409ba810 0x556d409c77b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:15:36.526 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:36.520+0000 7fc4cdef8640 1 -- 172.21.15.49:0/3478012164 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556d409b9ef0 msgr2=0x556d409c7270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:15:36.526 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:36.520+0000 7fc4cdef8640 1 --2- 172.21.15.49:0/3478012164 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556d409b9ef0 0x556d409c7270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:36.526 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:36.520+0000 7fc4cdef8640 1 -- 172.21.15.49:0/3478012164 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556d409ba810 msgr2=0x556d409c77b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:15:36.526 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:36.520+0000 7fc4cdef8640 1 --2- 172.21.15.49:0/3478012164 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556d409ba810 0x556d409c77b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:36.526 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:36.520+0000 7fc4cdef8640 1 -- 172.21.15.49:0/3478012164 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556d406fe620 con 0x556d409c35e0 2024-03-18T23:15:36.526 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:36.520+0000 7fc4cdef8640 1 --2- 172.21.15.49:0/3478012164 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556d409c35e0 0x556d40a387e0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fc4c0097c50 tx=0x7fc4c0009940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0f0a561ea340914 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:15:36.526 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:36.520+0000 7fc4b0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-18T23:15:36.526 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:36.520+0000 7fc4b0ff9640 1 -- 172.21.15.49:0/3478012164 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4c0020040 con 0x556d409c35e0 2024-03-18T23:15:36.526 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:36.520+0000 7fc4b0ff9640 1 -- 172.21.15.49:0/3478012164 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7fc4c00040b0 con 0x556d409c35e0 2024-03-18T23:15:36.526 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:36.520+0000 7fc4d2625740 1 -- 172.21.15.49:0/3478012164 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc4c401e420 con 0x556d409c35e0 2024-03-18T23:15:36.526 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:36.520+0000 7fc4b0ff9640 1 -- 172.21.15.49:0/3478012164 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fc4c0003c60 con 0x556d409c35e0 2024-03-18T23:15:36.527 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:36.520+0000 7fc4d2625740 10 client.16053 fetch_fsmap learned FSMap version 32 2024-03-18T23:15:36.527 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:36.520+0000 7fc4d2625740 1 -- 172.21.15.49:0/3478012164 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x556d40a39c90 con 0x556d409c35e0 2024-03-18T23:15:36.527 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:36.521+0000 7fc4b0ff9640 1 -- 172.21.15.49:0/3478012164 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc4c401e420 con 0x556d409c35e0 2024-03-18T23:15:36.527 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:36.521+0000 7fc4b0ff9640 1 -- 172.21.15.49:0/3478012164 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7fc4c009a6d0 con 0x556d409c35e0 2024-03-18T23:15:36.527 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:36.521+0000 7fc4d2625740 10 client.16053 fetch_fsmap finished waiting for FSMap version 32 2024-03-18T23:15:36.527 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:36.521+0000 7fc4d2625740 10 client.16053 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T23:15:36.527 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:36.521+0000 7fc4d2625740 20 client.16053 populate_metadata read hostname 'smithi049' 2024-03-18T23:15:36.527 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:36.521+0000 7fc4d2625740 1 --2- 172.21.15.49:0/3478012164 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x556d40a3a400 0x556d40a3c820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:15:36.527 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:36.521+0000 7fc4d2625740 4 client.16053 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T23:15:36.527 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:36.521+0000 7fc4d2625740 1 -- 172.21.15.49:0/3478012164 --> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556d40925470 con 0x556d40a3a400 2024-03-18T23:15:36.527 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:36.521+0000 7fc4cd6f7640 1 --2- 172.21.15.49:0/3478012164 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x556d40a3a400 0x556d40a3c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:15:36.527 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:36.521+0000 7fc4b0ff9640 10 client.16053 ms_handle_connect on v2:172.21.15.139:6838/4280128209 2024-03-18T23:15:36.527 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:36.521+0000 7fc4cd6f7640 1 --2- 172.21.15.49:0/3478012164 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x556d40a3a400 0x556d40a3c820 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ddd9105e34228ade server_cookie=fdcfc0b62daeeebd in_seq=0 out_seq=0 2024-03-18T23:15:36.700 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:36.699+0000 7fc4b0ff9640 1 -- 172.21.15.49:0/3478012164 <== mds.0 v2:172.21.15.139:6838/4280128209 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556d40925470 con 0x556d40a3a400 2024-03-18T23:15:36.700 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:36.699+0000 7fc4b0ff9640 10 client.16053 handle_command_reply: tid=0 2024-03-18T23:15:36.700 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:36.699+0000 7fc4d2625740 1 client.16053 shutdown 2024-03-18T23:15:36.700 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:36.699+0000 7fc4d2625740 1 -- 172.21.15.49:0/3478012164 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556d409c35e0 msgr2=0x556d40a387e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:15:36.700 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:36.699+0000 7fc4d2625740 1 --2- 172.21.15.49:0/3478012164 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556d409c35e0 0x556d40a387e0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fc4c0097c50 tx=0x7fc4c0009940 comp rx=0 tx=0).stop 2024-03-18T23:15:36.701 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:36.699+0000 7fc4d2625740 1 -- 172.21.15.49:0/3478012164 shutdown_connections 2024-03-18T23:15:36.701 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:36.699+0000 7fc4d2625740 1 --2- 172.21.15.49:0/3478012164 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x556d40a3a400 0x556d40a3c820 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:36.701 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:36.699+0000 7fc4d2625740 1 --2- 172.21.15.49:0/3478012164 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556d409c35e0 0x556d40a387e0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:36.701 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:36.699+0000 7fc4b0ff9640 0 client.16053 ms_handle_reset on v2:172.21.15.139:6838/4280128209 2024-03-18T23:15:36.701 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:36.699+0000 7fc4d2625740 1 --2- 172.21.15.49:0/3478012164 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556d409ba810 0x556d409c77b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:36.701 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:36.699+0000 7fc4d2625740 1 --2- 172.21.15.49:0/3478012164 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556d409b9ef0 0x556d409c7270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:36.701 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:36.699+0000 7fc4d2625740 1 -- 172.21.15.49:0/3478012164 >> 172.21.15.49:0/3478012164 conn(0x556d409b4810 msgr2=0x556d409b5a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:15:36.701 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:36.700+0000 7fc4d2625740 1 -- 172.21.15.49:0/3478012164 shutdown_connections 2024-03-18T23:15:36.701 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:36.700+0000 7fc4d2625740 1 -- 172.21.15.49:0/3478012164 wait complete. 2024-03-18T23:15:36.701 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:36.700+0000 7fc4d2625740 20 client.16053 trim_cache size 0 max 16384 2024-03-18T23:15:36.701 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:{ 2024-03-18T23:15:36.702 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "message": "", 2024-03-18T23:15:36.702 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "return_code": 0 2024-03-18T23:15:36.702 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:} 2024-03-18T23:15:36.703 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:36.700+0000 7fc4cf95b640 1 -- 172.21.15.49:0/3955386042 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7fc4b40a2a10 msgr2=0x7fc4b40a4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:15:36.703 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:36.700+0000 7fc4cf95b640 1 --2- 172.21.15.49:0/3955386042 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7fc4b40a2a10 0x7fc4b40a4ed0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fc4c4009b50 tx=0x7fc4c4005f90 comp rx=0 tx=0).stop 2024-03-18T23:15:36.703 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:36.700+0000 7fc4cf95b640 1 -- 172.21.15.49:0/3955386042 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc4c81077a0 msgr2=0x7fc4c81014c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:15:36.703 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:36.700+0000 7fc4cf95b640 1 --2- 172.21.15.49:0/3955386042 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc4c81077a0 0x7fc4c81014c0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fc4c0009fd0 tx=0x7fc4c000b260 comp rx=0 tx=0).stop 2024-03-18T23:15:36.703 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:36.701+0000 7fc4cf95b640 1 -- 172.21.15.49:0/3955386042 shutdown_connections 2024-03-18T23:15:36.703 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:36.701+0000 7fc4cf95b640 1 --2- 172.21.15.49:0/3955386042 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7fc4b40a2a10 0x7fc4b40a4ed0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:36.703 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:36.701+0000 7fc4cf95b640 1 --2- 172.21.15.49:0/3955386042 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc4c81077a0 0x7fc4c81014c0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:36.703 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:36.701+0000 7fc4cf95b640 1 --2- 172.21.15.49:0/3955386042 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc4c8069860 0x7fc4c8100f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:36.704 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:36.701+0000 7fc4cf95b640 1 --2- 172.21.15.49:0/3955386042 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc4c8068ea0 0x7fc4c8100a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:36.704 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:36.701+0000 7fc4cf95b640 1 -- 172.21.15.49:0/3955386042 >> 172.21.15.49:0/3955386042 conn(0x7fc4c80774b0 msgr2=0x7fc4c80fea20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:15:36.704 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:36.701+0000 7fc4cf95b640 1 -- 172.21.15.49:0/3955386042 shutdown_connections 2024-03-18T23:15:36.704 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:36.701+0000 7fc4cf95b640 1 -- 172.21.15.49:0/3955386042 wait complete. 2024-03-18T23:15:36.776 INFO:tasks.workunit.client.0.smithi049.stdout: 1 792366 10.00 MB/sec execute 278 sec latency 95.293 ms 2024-03-18T23:15:37.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:15:37 smithi175 ceph-mon[67352]: pgmap v227: 97 pgs: 97 active+clean; 78 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.7 MiB/s wr, 267 op/s 2024-03-18T23:15:37.564 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:15:37 smithi139 ceph-mon[67282]: pgmap v227: 97 pgs: 97 active+clean; 78 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.7 MiB/s wr, 267 op/s 2024-03-18T23:15:37.629 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:15:37 smithi049 ceph-mon[62991]: pgmap v227: 97 pgs: 97 active+clean; 78 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.7 MiB/s wr, 267 op/s 2024-03-18T23:15:37.776 INFO:tasks.workunit.client.0.smithi049.stdout: 1 794209 9.99 MB/sec execute 279 sec latency 22.424 ms 2024-03-18T23:15:38.777 INFO:tasks.workunit.client.0.smithi049.stdout: 1 795869 10.00 MB/sec execute 280 sec latency 42.621 ms 2024-03-18T23:15:39.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:15:39 smithi175 ceph-mon[67352]: pgmap v228: 97 pgs: 97 active+clean; 85 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 3.8 MiB/s wr, 348 op/s 2024-03-18T23:15:39.564 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:15:39 smithi139 ceph-mon[67282]: pgmap v228: 97 pgs: 97 active+clean; 85 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 3.8 MiB/s wr, 348 op/s 2024-03-18T23:15:39.629 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:15:39 smithi049 ceph-mon[62991]: pgmap v228: 97 pgs: 97 active+clean; 85 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 3.8 MiB/s wr, 348 op/s 2024-03-18T23:15:39.777 INFO:tasks.workunit.client.0.smithi049.stdout: 1 797723 9.99 MB/sec execute 281 sec latency 48.362 ms 2024-03-18T23:15:40.776 INFO:tasks.workunit.client.0.smithi049.stdout: 1 799586 10.00 MB/sec execute 282 sec latency 22.165 ms 2024-03-18T23:15:41.460 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:15:41 smithi175 ceph-mon[67352]: pgmap v229: 97 pgs: 97 active+clean; 85 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.4 MiB/s wr, 170 op/s 2024-03-18T23:15:41.564 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:15:41 smithi139 ceph-mon[67282]: pgmap v229: 97 pgs: 97 active+clean; 85 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.4 MiB/s wr, 170 op/s 2024-03-18T23:15:41.629 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:15:41 smithi049 ceph-mon[62991]: pgmap v229: 97 pgs: 97 active+clean; 85 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.4 MiB/s wr, 170 op/s 2024-03-18T23:15:41.777 INFO:tasks.workunit.client.0.smithi049.stdout: 1 801621 10.00 MB/sec execute 283 sec latency 21.579 ms 2024-03-18T23:15:42.777 INFO:tasks.workunit.client.0.smithi049.stdout: 1 803234 9.99 MB/sec execute 284 sec latency 19.230 ms 2024-03-18T23:15:43.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:15:43 smithi175 ceph-mon[67352]: pgmap v230: 97 pgs: 97 active+clean; 75 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 3.8 MiB/s wr, 359 op/s 2024-03-18T23:15:43.564 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:15:43 smithi139 ceph-mon[67282]: pgmap v230: 97 pgs: 97 active+clean; 75 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 3.8 MiB/s wr, 359 op/s 2024-03-18T23:15:43.629 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:15:43 smithi049 ceph-mon[62991]: pgmap v230: 97 pgs: 97 active+clean; 75 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 3.8 MiB/s wr, 359 op/s 2024-03-18T23:15:43.777 INFO:tasks.workunit.client.0.smithi049.stdout: 1 804865 9.98 MB/sec execute 285 sec latency 74.625 ms 2024-03-18T23:15:44.777 INFO:tasks.workunit.client.0.smithi049.stdout: 1 806574 9.99 MB/sec execute 286 sec latency 15.516 ms 2024-03-18T23:15:45.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:15:45 smithi175 ceph-mon[67352]: pgmap v231: 97 pgs: 97 active+clean; 75 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.5 MiB/s wr, 269 op/s 2024-03-18T23:15:45.564 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:15:45 smithi139 ceph-mon[67282]: pgmap v231: 97 pgs: 97 active+clean; 75 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.5 MiB/s wr, 269 op/s 2024-03-18T23:15:45.629 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:15:45 smithi049 ceph-mon[62991]: pgmap v231: 97 pgs: 97 active+clean; 75 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.5 MiB/s wr, 269 op/s 2024-03-18T23:15:45.777 INFO:tasks.workunit.client.0.smithi049.stdout: 1 807622 9.97 MB/sec execute 287 sec latency 25.336 ms 2024-03-18T23:15:46.777 INFO:tasks.workunit.client.0.smithi049.stdout: 1 809499 9.98 MB/sec execute 288 sec latency 22.933 ms 2024-03-18T23:15:47.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:15:47 smithi175 ceph-mon[67352]: pgmap v232: 97 pgs: 97 active+clean; 75 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.5 MiB/s wr, 269 op/s 2024-03-18T23:15:47.564 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:15:47 smithi139 ceph-mon[67282]: pgmap v232: 97 pgs: 97 active+clean; 75 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.5 MiB/s wr, 269 op/s 2024-03-18T23:15:47.629 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:15:47 smithi049 ceph-mon[62991]: pgmap v232: 97 pgs: 97 active+clean; 75 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.5 MiB/s wr, 269 op/s 2024-03-18T23:15:47.777 INFO:tasks.workunit.client.0.smithi049.stdout: 1 811008 9.97 MB/sec execute 289 sec latency 18.700 ms 2024-03-18T23:15:48.777 INFO:tasks.workunit.client.0.smithi049.stdout: 1 813118 9.98 MB/sec execute 290 sec latency 65.086 ms 2024-03-18T23:15:49.460 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:15:49 smithi175 ceph-mon[67352]: pgmap v233: 97 pgs: 97 active+clean; 80 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 3.8 MiB/s wr, 349 op/s 2024-03-18T23:15:49.564 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:15:49 smithi139 ceph-mon[67282]: pgmap v233: 97 pgs: 97 active+clean; 80 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 3.8 MiB/s wr, 349 op/s 2024-03-18T23:15:49.629 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:15:49 smithi049 ceph-mon[62991]: pgmap v233: 97 pgs: 97 active+clean; 80 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 3.8 MiB/s wr, 349 op/s 2024-03-18T23:15:49.777 INFO:tasks.workunit.client.0.smithi049.stdout: 1 814494 9.97 MB/sec execute 291 sec latency 110.136 ms 2024-03-18T23:15:49.796 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:49.794+0000 7fd5a66ff640 1 -- 172.21.15.49:0/7144737 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd5a01056c0 msgr2=0x7fd5a0105aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:15:49.796 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:49.794+0000 7fd5a66ff640 1 --2- 172.21.15.49:0/7144737 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd5a01056c0 0x7fd5a0105aa0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fd590009900 tx=0x7fd59002bc30 comp rx=0 tx=0).stop 2024-03-18T23:15:49.796 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:49.794+0000 7fd5a66ff640 1 -- 172.21.15.49:0/7144737 shutdown_connections 2024-03-18T23:15:49.796 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:49.794+0000 7fd5a66ff640 1 --2- 172.21.15.49:0/7144737 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd5a0101330 0x7fd5a010d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:49.796 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:49.794+0000 7fd5a66ff640 1 --2- 172.21.15.49:0/7144737 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd5a0100970 0x7fd5a0100df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:49.796 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:49.794+0000 7fd5a66ff640 1 --2- 172.21.15.49:0/7144737 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd5a01056c0 0x7fd5a0105aa0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:49.796 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:49.795+0000 7fd5a66ff640 1 -- 172.21.15.49:0/7144737 >> 172.21.15.49:0/7144737 conn(0x7fd5a0077570 msgr2=0x7fd5a0077980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:15:49.797 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:49.795+0000 7fd5a66ff640 1 -- 172.21.15.49:0/7144737 shutdown_connections 2024-03-18T23:15:49.797 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:49.795+0000 7fd5a66ff640 1 -- 172.21.15.49:0/7144737 wait complete. 2024-03-18T23:15:49.797 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:49.795+0000 7fd5a66ff640 1 Processor -- start 2024-03-18T23:15:49.797 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:49.795+0000 7fd5a66ff640 1 -- start start 2024-03-18T23:15:49.797 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:49.795+0000 7fd5a66ff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd5a0100970 0x7fd5a01980c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:15:49.797 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:49.795+0000 7fd5a66ff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd5a0101330 0x7fd5a0198600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:15:49.797 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:49.795+0000 7fd5a66ff640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd5a01056c0 0x7fd5a019c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:15:49.797 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:49.795+0000 7fd5a66ff640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fd5a0111d00 con 0x7fd5a0101330 2024-03-18T23:15:49.797 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:49.795+0000 7fd5a66ff640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fd5a0111b80 con 0x7fd5a0100970 2024-03-18T23:15:49.797 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:49.795+0000 7fd5a66ff640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd5a0111e80 con 0x7fd5a01056c0 2024-03-18T23:15:49.797 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:49.796+0000 7fd59f7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd5a0101330 0x7fd5a0198600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:15:49.797 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:49.796+0000 7fd59ffff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd5a0100970 0x7fd5a01980c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:15:49.798 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:49.796+0000 7fd59f7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd5a0101330 0x7fd5a0198600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:60636/0 (socket says 172.21.15.49:60636) 2024-03-18T23:15:49.798 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:49.796+0000 7fd59f7fe640 1 -- 172.21.15.49:0/3274605839 learned_addr learned my addr 172.21.15.49:0/3274605839 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:15:49.798 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:49.796+0000 7fd5a4c9c640 1 --2- 172.21.15.49:0/3274605839 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd5a01056c0 0x7fd5a019c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:15:49.799 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:49.796+0000 7fd59f7fe640 1 -- 172.21.15.49:0/3274605839 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd5a01056c0 msgr2=0x7fd5a019c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:15:49.799 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:49.796+0000 7fd59f7fe640 1 --2- 172.21.15.49:0/3274605839 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd5a01056c0 0x7fd5a019c980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:49.799 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:49.796+0000 7fd59f7fe640 1 -- 172.21.15.49:0/3274605839 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd5a0100970 msgr2=0x7fd5a01980c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:15:49.799 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:49.796+0000 7fd59f7fe640 1 --2- 172.21.15.49:0/3274605839 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd5a0100970 0x7fd5a01980c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:49.800 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:49.796+0000 7fd59f7fe640 1 -- 172.21.15.49:0/3274605839 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5a019d0b0 con 0x7fd5a0101330 2024-03-18T23:15:49.800 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:49.796+0000 7fd59f7fe640 1 --2- 172.21.15.49:0/3274605839 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd5a0101330 0x7fd5a0198600 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fd59400e9f0 tx=0x7fd59400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:15:49.800 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:49.796+0000 7fd59d7fa640 1 -- 172.21.15.49:0/3274605839 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd59400cd80 con 0x7fd5a0101330 2024-03-18T23:15:49.800 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:49.796+0000 7fd59d7fa640 1 -- 172.21.15.49:0/3274605839 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd594010910 con 0x7fd5a0101330 2024-03-18T23:15:49.800 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:49.796+0000 7fd59d7fa640 1 -- 172.21.15.49:0/3274605839 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd594018990 con 0x7fd5a0101330 2024-03-18T23:15:49.800 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:49.796+0000 7fd5a66ff640 1 -- 172.21.15.49:0/3274605839 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5a019d3a0 con 0x7fd5a0101330 2024-03-18T23:15:49.801 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:49.796+0000 7fd5a66ff640 1 -- 172.21.15.49:0/3274605839 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5a01a4c30 con 0x7fd5a0101330 2024-03-18T23:15:49.809 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:49.799+0000 7fd59d7fa640 1 -- 172.21.15.49:0/3274605839 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 25) ==== 137421+0+0 (secure 0 0 0) 0x7fd594010ac0 con 0x7fd5a0101330 2024-03-18T23:15:49.809 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:49.799+0000 7fd5a93c9740 1 Processor -- start 2024-03-18T23:15:49.809 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:49.799+0000 7fd5a93c9740 1 -- start start 2024-03-18T23:15:49.809 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:49.799+0000 7fd59d7fa640 1 --2- 172.21.15.49:0/3274605839 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7fd5880a2a10 0x7fd5880a4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:15:49.809 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:49.799+0000 7fd5a93c9740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560843cf8a10 0x560843cf8df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:15:49.810 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:49.799+0000 7fd5a93c9740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560843cf9330 0x560843d064b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:15:49.810 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:49.799+0000 7fd5a93c9740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560843d069f0 0x560843d08de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:15:49.810 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:49.799+0000 7fd5a93c9740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x560843c5cbe0 con 0x560843cf8a10 2024-03-18T23:15:49.810 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:49.799+0000 7fd5a93c9740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x560843c05960 con 0x560843d069f0 2024-03-18T23:15:49.810 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:49.799+0000 7fd5a93c9740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x560843c58c80 con 0x560843cf9330 2024-03-18T23:15:49.810 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:49.800+0000 7fd59d7fa640 1 -- 172.21.15.49:0/3274605839 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7fd5940cf120 con 0x7fd5a0101330 2024-03-18T23:15:49.810 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:49.800+0000 7fd5a4c9c640 1 --2- 172.21.15.49:0/3274605839 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7fd5880a2a10 0x7fd5880a4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:15:49.810 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:49.800+0000 7fd59ffff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560843cf8a10 0x560843cf8df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:15:49.810 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:49.800+0000 7fd59ffff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560843cf8a10 0x560843cf8df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:60638/0 (socket says 172.21.15.49:60638) 2024-03-18T23:15:49.810 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:49.800+0000 7fd59ffff640 1 -- 172.21.15.49:0/660380400 learned_addr learned my addr 172.21.15.49:0/660380400 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:15:49.810 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:49.800+0000 7fd5a4c9c640 1 --2- 172.21.15.49:0/660380400 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560843d069f0 0x560843d08de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:15:49.810 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:49.800+0000 7fd59f7fe640 1 --2- 172.21.15.49:0/660380400 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560843cf9330 0x560843d064b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:15:49.810 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:49.800+0000 7fd59ffff640 1 -- 172.21.15.49:0/660380400 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560843cf9330 msgr2=0x560843d064b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:15:49.811 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:49.800+0000 7fd59ffff640 1 --2- 172.21.15.49:0/660380400 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560843cf9330 0x560843d064b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:49.811 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:49.800+0000 7fd59ffff640 1 -- 172.21.15.49:0/660380400 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560843d069f0 msgr2=0x560843d08de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:15:49.811 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:49.800+0000 7fd59ffff640 1 --2- 172.21.15.49:0/660380400 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560843d069f0 0x560843d08de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:49.811 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:49.800+0000 7fd59ffff640 1 -- 172.21.15.49:0/660380400 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560843bfcd20 con 0x560843cf8a10 2024-03-18T23:15:49.811 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:49.800+0000 7fd59ffff640 1 --2- 172.21.15.49:0/660380400 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560843cf8a10 0x560843cf8df0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fd590009900 tx=0x7fd590005be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c37151bf7ba1ebf server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:15:49.811 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:49.800+0000 7fd5a4c9c640 1 --2- 172.21.15.49:0/3274605839 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7fd5880a2a10 0x7fd5880a4ed0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fd598005fd0 tx=0x7fd598009df0 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:15:49.811 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:49.801+0000 7fd5867fc640 1 -- 172.21.15.49:0/660380400 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd590042070 con 0x560843cf8a10 2024-03-18T23:15:49.811 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:49.801+0000 7fd5867fc640 1 -- 172.21.15.49:0/660380400 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd59002c8b0 con 0x560843cf8a10 2024-03-18T23:15:49.811 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:49.801+0000 7fd5867fc640 1 -- 172.21.15.49:0/660380400 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd590004ec0 con 0x560843cf8a10 2024-03-18T23:15:49.811 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:49.801+0000 7fd5a93c9740 1 -- 172.21.15.49:0/660380400 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560843cf8a10 msgr2=0x560843cf8df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:15:49.811 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:49.801+0000 7fd5a93c9740 1 --2- 172.21.15.49:0/660380400 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560843cf8a10 0x560843cf8df0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fd590009900 tx=0x7fd590005be0 comp rx=0 tx=0).stop 2024-03-18T23:15:49.811 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:49.801+0000 7fd5a93c9740 1 -- 172.21.15.49:0/660380400 shutdown_connections 2024-03-18T23:15:49.811 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:49.802+0000 7fd5a93c9740 1 --2- 172.21.15.49:0/660380400 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560843d069f0 0x560843d08de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:49.812 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:49.802+0000 7fd5a93c9740 1 --2- 172.21.15.49:0/660380400 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560843cf9330 0x560843d064b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:49.812 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:49.802+0000 7fd5a93c9740 1 --2- 172.21.15.49:0/660380400 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560843cf8a10 0x560843cf8df0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:49.812 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:49.802+0000 7fd5a93c9740 1 -- 172.21.15.49:0/660380400 >> 172.21.15.49:0/660380400 conn(0x560843cf3810 msgr2=0x560843cf3bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:15:49.812 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:49.802+0000 7fd5a93c9740 1 -- 172.21.15.49:0/660380400 shutdown_connections 2024-03-18T23:15:49.812 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:49.802+0000 7fd5a93c9740 1 -- 172.21.15.49:0/660380400 wait complete. 2024-03-18T23:15:49.812 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:49.802+0000 7fd5a93c9740 1 -- start start 2024-03-18T23:15:49.812 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:49.802+0000 7fd5a93c9740 1 Processor -- start 2024-03-18T23:15:49.812 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:49.802+0000 7fd5a93c9740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560843cf8a10 0x560843d06360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:15:49.812 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:49.802+0000 7fd5a93c9740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560843cf9330 0x560843d036f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:15:49.812 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:49.802+0000 7fd5a93c9740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560843d069f0 0x560843d03c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:15:49.812 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:49.802+0000 7fd5a93c9740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x560843d09550 con 0x560843cf8a10 2024-03-18T23:15:49.813 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:49.802+0000 7fd5a93c9740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x560843c18db0 con 0x560843cf9330 2024-03-18T23:15:49.813 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:49.802+0000 7fd5a93c9740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x560843d096d0 con 0x560843d069f0 2024-03-18T23:15:49.813 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:49.802+0000 7fd5a93c9740 10 client.0 _ll_register_callbacks cb 0x560843c1de70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T23:15:49.813 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:49.803+0000 7fd59ffff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560843cf8a10 0x560843d06360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:15:49.813 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:49.803+0000 7fd59f7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560843cf9330 0x560843d036f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:15:49.813 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:49.803+0000 7fd59f7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560843cf9330 0x560843d036f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.49:60146/0 (socket says 172.21.15.49:60146) 2024-03-18T23:15:49.813 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:49.803+0000 7fd5a4c9c640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560843d069f0 0x560843d03c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:15:49.813 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:49.803+0000 7fd59ffff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560843cf8a10 0x560843d06360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:60640/0 (socket says 172.21.15.49:60640) 2024-03-18T23:15:49.813 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:49.803+0000 7fd59f7fe640 1 -- 172.21.15.49:0/863447934 learned_addr learned my addr 172.21.15.49:0/863447934 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:15:49.813 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:49.803+0000 7fd59ffff640 1 -- 172.21.15.49:0/863447934 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560843d069f0 msgr2=0x560843d03c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:15:49.813 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:49.803+0000 7fd59ffff640 1 --2- 172.21.15.49:0/863447934 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560843d069f0 0x560843d03c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:49.813 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:49.803+0000 7fd59ffff640 1 -- 172.21.15.49:0/863447934 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560843cf9330 msgr2=0x560843d036f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:15:49.813 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:49.803+0000 7fd59ffff640 1 --2- 172.21.15.49:0/863447934 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560843cf9330 0x560843d036f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:49.814 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:49.803+0000 7fd59ffff640 1 -- 172.21.15.49:0/863447934 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560843c17c60 con 0x560843cf8a10 2024-03-18T23:15:49.814 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:49.803+0000 7fd59ffff640 1 --2- 172.21.15.49:0/863447934 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560843cf8a10 0x560843d06360 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fd590004650 tx=0x7fd59002bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e75f9b674e2c603d server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:15:49.814 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:49.803+0000 7fd57f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-18T23:15:49.814 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:49.803+0000 7fd57f7fe640 1 -- 172.21.15.49:0/863447934 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd590042040 con 0x560843cf8a10 2024-03-18T23:15:49.814 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:49.803+0000 7fd5a93c9740 1 -- 172.21.15.49:0/863447934 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd59002c8b0 con 0x560843cf8a10 2024-03-18T23:15:49.814 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:49.803+0000 7fd57f7fe640 1 -- 172.21.15.49:0/863447934 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7fd590004fd0 con 0x560843cf8a10 2024-03-18T23:15:49.814 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:49.803+0000 7fd57f7fe640 1 -- 172.21.15.49:0/863447934 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fd59002cd30 con 0x560843cf8a10 2024-03-18T23:15:49.814 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:49.803+0000 7fd5a93c9740 10 client.16065 fetch_fsmap learned FSMap version 32 2024-03-18T23:15:49.814 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:49.803+0000 7fd5a93c9740 1 -- 172.21.15.49:0/863447934 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x560843971210 con 0x560843cf8a10 2024-03-18T23:15:49.814 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:49.804+0000 7fd57f7fe640 1 -- 172.21.15.49:0/863447934 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd59002c8b0 con 0x560843cf8a10 2024-03-18T23:15:49.814 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:49.804+0000 7fd57f7fe640 1 -- 172.21.15.49:0/863447934 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7fd590034440 con 0x560843cf8a10 2024-03-18T23:15:49.814 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:49.804+0000 7fd5a93c9740 10 client.16065 fetch_fsmap finished waiting for FSMap version 32 2024-03-18T23:15:49.814 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:49.804+0000 7fd5a93c9740 10 client.16065 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T23:15:49.814 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:49.804+0000 7fd5a93c9740 20 client.16065 populate_metadata read hostname 'smithi049' 2024-03-18T23:15:49.815 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:49.804+0000 7fd5a93c9740 1 --2- 172.21.15.49:0/863447934 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x560843d16680 0x560843d18a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:15:49.815 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:49.804+0000 7fd5a93c9740 4 client.16065 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T23:15:49.815 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:49.804+0000 7fd5a93c9740 1 -- 172.21.15.49:0/863447934 --> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560843c6a120 con 0x560843d16680 2024-03-18T23:15:49.815 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:49.804+0000 7fd59f7fe640 1 --2- 172.21.15.49:0/863447934 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x560843d16680 0x560843d18a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:15:49.815 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:49.804+0000 7fd59f7fe640 1 --2- 172.21.15.49:0/863447934 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x560843d16680 0x560843d18a70 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a7ee45a77889e14 server_cookie=c4b5ad61d44d72 in_seq=0 out_seq=0 2024-03-18T23:15:49.815 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:49.804+0000 7fd57f7fe640 10 client.16065 ms_handle_connect on v2:172.21.15.139:6838/4280128209 2024-03-18T23:15:49.815 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:49.806+0000 7fd57f7fe640 1 -- 172.21.15.49:0/863447934 <== mds.0 v2:172.21.15.139:6838/4280128209 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x560843c6a120 con 0x560843d16680 2024-03-18T23:15:49.815 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:49.806+0000 7fd57f7fe640 10 client.16065 handle_command_reply: tid=0 2024-03-18T23:15:49.815 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:49.806+0000 7fd5a93c9740 1 client.16065 shutdown 2024-03-18T23:15:49.815 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:49.806+0000 7fd5a93c9740 1 -- 172.21.15.49:0/863447934 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560843cf8a10 msgr2=0x560843d06360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:15:49.815 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:49.806+0000 7fd5a93c9740 1 --2- 172.21.15.49:0/863447934 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560843cf8a10 0x560843d06360 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fd590004650 tx=0x7fd59002bc30 comp rx=0 tx=0).stop 2024-03-18T23:15:49.815 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:49.806+0000 7fd5a93c9740 1 -- 172.21.15.49:0/863447934 shutdown_connections 2024-03-18T23:15:49.815 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:49.806+0000 7fd5a93c9740 1 --2- 172.21.15.49:0/863447934 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x560843d16680 0x560843d18a70 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:49.815 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:49.806+0000 7fd5a93c9740 1 --2- 172.21.15.49:0/863447934 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560843d069f0 0x560843d03c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:49.816 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:49.806+0000 7fd57f7fe640 0 client.16065 ms_handle_reset on v2:172.21.15.139:6838/4280128209 2024-03-18T23:15:49.816 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:49.806+0000 7fd5a93c9740 1 --2- 172.21.15.49:0/863447934 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560843cf9330 0x560843d036f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:49.816 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:49.806+0000 7fd5a93c9740 1 --2- 172.21.15.49:0/863447934 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560843cf8a10 0x560843d06360 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:49.816 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:49.807+0000 7fd5a93c9740 1 -- 172.21.15.49:0/863447934 >> 172.21.15.49:0/863447934 conn(0x560843cf3810 msgr2=0x560843cf5560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:15:49.816 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:49.807+0000 7fd5a93c9740 1 -- 172.21.15.49:0/863447934 shutdown_connections 2024-03-18T23:15:49.816 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:49.807+0000 7fd5a93c9740 1 -- 172.21.15.49:0/863447934 wait complete. 2024-03-18T23:15:49.816 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:49.807+0000 7fd5a93c9740 20 client.16065 trim_cache size 0 max 16384 2024-03-18T23:15:49.825 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:49.821+0000 7fd5a93c9740 1 Processor -- start 2024-03-18T23:15:49.825 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:49.821+0000 7fd5a93c9740 1 -- start start 2024-03-18T23:15:49.825 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:49.821+0000 7fd5a93c9740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560843cf8a10 0x560843d12330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:15:49.825 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:49.822+0000 7fd5a93c9740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560843cf9330 0x560843cf3bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:15:49.825 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:49.822+0000 7fd5a93c9740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560843d069f0 0x560843d1e120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:15:49.825 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:49.822+0000 7fd5a93c9740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fd5a01ab140 con 0x560843cf8a10 2024-03-18T23:15:49.825 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:49.822+0000 7fd5a93c9740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fd5a01acc80 con 0x560843cf9330 2024-03-18T23:15:49.826 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:49.822+0000 7fd5a93c9740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd5a01ab810 con 0x560843d069f0 2024-03-18T23:15:49.826 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:49.822+0000 7fd59ffff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560843cf8a10 0x560843d12330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:15:49.826 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:49.822+0000 7fd59ffff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560843cf8a10 0x560843d12330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:60646/0 (socket says 172.21.15.49:60646) 2024-03-18T23:15:49.826 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:49.822+0000 7fd59ffff640 1 -- 172.21.15.49:0/3464106876 learned_addr learned my addr 172.21.15.49:0/3464106876 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:15:49.826 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:49.822+0000 7fd59f7fe640 1 --2- 172.21.15.49:0/3464106876 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560843cf9330 0x560843cf3bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:15:49.826 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:49.822+0000 7fd5a4c9c640 1 --2- 172.21.15.49:0/3464106876 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560843d069f0 0x560843d1e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:15:49.826 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:49.822+0000 7fd59ffff640 1 -- 172.21.15.49:0/3464106876 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560843d069f0 msgr2=0x560843d1e120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:15:49.826 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:49.822+0000 7fd59ffff640 1 --2- 172.21.15.49:0/3464106876 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560843d069f0 0x560843d1e120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:49.826 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:49.822+0000 7fd59ffff640 1 -- 172.21.15.49:0/3464106876 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560843cf9330 msgr2=0x560843cf3bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:15:49.826 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:49.822+0000 7fd59ffff640 1 --2- 172.21.15.49:0/3464106876 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560843cf9330 0x560843cf3bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:49.826 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:49.822+0000 7fd59ffff640 1 -- 172.21.15.49:0/3464106876 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560843a3d620 con 0x560843cf8a10 2024-03-18T23:15:49.826 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:49.822+0000 7fd5a4c9c640 1 --2- 172.21.15.49:0/3464106876 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560843d069f0 0x560843d1e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T23:15:49.826 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:49.822+0000 7fd59f7fe640 1 --2- 172.21.15.49:0/3464106876 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560843cf9330 0x560843cf3bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:15:49.827 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:49.822+0000 7fd59ffff640 1 --2- 172.21.15.49:0/3464106876 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560843cf8a10 0x560843d12330 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fd590004620 tx=0x7fd590009800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2401fb2b6c434b7 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:15:49.827 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:49.822+0000 7fd585ffb640 1 -- 172.21.15.49:0/3464106876 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd59005c070 con 0x560843cf8a10 2024-03-18T23:15:49.827 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:49.822+0000 7fd585ffb640 1 -- 172.21.15.49:0/3464106876 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd59002c8b0 con 0x560843cf8a10 2024-03-18T23:15:49.827 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:49.822+0000 7fd585ffb640 1 -- 172.21.15.49:0/3464106876 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd590042040 con 0x560843cf8a10 2024-03-18T23:15:49.827 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:49.823+0000 7fd5a93c9740 1 -- 172.21.15.49:0/3464106876 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560843cf8a10 msgr2=0x560843d12330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:15:49.827 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:49.823+0000 7fd5a93c9740 1 --2- 172.21.15.49:0/3464106876 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560843cf8a10 0x560843d12330 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fd590004620 tx=0x7fd590009800 comp rx=0 tx=0).stop 2024-03-18T23:15:49.827 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:49.823+0000 7fd5a93c9740 1 -- 172.21.15.49:0/3464106876 shutdown_connections 2024-03-18T23:15:49.827 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:49.823+0000 7fd5a93c9740 1 --2- 172.21.15.49:0/3464106876 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560843d069f0 0x560843d1e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:49.827 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:49.823+0000 7fd5a93c9740 1 --2- 172.21.15.49:0/3464106876 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560843cf9330 0x560843cf3bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:49.827 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:49.823+0000 7fd5a93c9740 1 --2- 172.21.15.49:0/3464106876 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560843cf8a10 0x560843d12330 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:49.827 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:49.823+0000 7fd5a93c9740 1 -- 172.21.15.49:0/3464106876 >> 172.21.15.49:0/3464106876 conn(0x560843cf3810 msgr2=0x560843cf2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:15:49.827 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:49.823+0000 7fd5a93c9740 1 -- 172.21.15.49:0/3464106876 shutdown_connections 2024-03-18T23:15:49.827 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:49.823+0000 7fd5a93c9740 1 -- 172.21.15.49:0/3464106876 wait complete. 2024-03-18T23:15:49.827 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:49.823+0000 7fd5a93c9740 1 -- start start 2024-03-18T23:15:49.827 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:49.823+0000 7fd5a93c9740 1 Processor -- start 2024-03-18T23:15:49.828 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:49.824+0000 7fd5a93c9740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560843cf8a10 0x560843d0a990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:15:49.828 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:49.824+0000 7fd5a93c9740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560843cf9330 0x560843d0aed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:15:49.828 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:49.824+0000 7fd5a93c9740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560843d069f0 0x560843d7b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:15:49.828 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:49.824+0000 7fd5a93c9740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fd5a01ac5b0 con 0x560843d069f0 2024-03-18T23:15:49.828 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:49.824+0000 7fd5a93c9740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fd5a01abee0 con 0x560843cf8a10 2024-03-18T23:15:49.828 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:49.824+0000 7fd5a93c9740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd5a01a9140 con 0x560843cf9330 2024-03-18T23:15:49.828 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:49.824+0000 7fd5a93c9740 10 client.0 _ll_register_callbacks cb 0x7fd5980137a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T23:15:49.828 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:49.824+0000 7fd5a4c9c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560843d069f0 0x560843d7b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:15:49.828 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:49.824+0000 7fd5a4c9c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560843d069f0 0x560843d7b750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:60652/0 (socket says 172.21.15.49:60652) 2024-03-18T23:15:49.828 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:49.824+0000 7fd5a4c9c640 1 -- 172.21.15.49:0/3965009606 learned_addr learned my addr 172.21.15.49:0/3965009606 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:15:49.828 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:49.824+0000 7fd59f7fe640 1 --2- 172.21.15.49:0/3965009606 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560843cf9330 0x560843d0aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:15:49.828 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:49.824+0000 7fd59ffff640 1 --2- 172.21.15.49:0/3965009606 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560843cf8a10 0x560843d0a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:15:49.828 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:49.824+0000 7fd5a4c9c640 1 -- 172.21.15.49:0/3965009606 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560843cf9330 msgr2=0x560843d0aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:15:49.829 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:49.824+0000 7fd5a4c9c640 1 --2- 172.21.15.49:0/3965009606 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560843cf9330 0x560843d0aed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:49.829 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:49.824+0000 7fd5a4c9c640 1 -- 172.21.15.49:0/3965009606 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560843cf8a10 msgr2=0x560843d0a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:15:49.829 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:49.824+0000 7fd5a4c9c640 1 --2- 172.21.15.49:0/3965009606 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560843cf8a10 0x560843d0a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:49.829 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:49.824+0000 7fd5a4c9c640 1 -- 172.21.15.49:0/3965009606 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560843d7bc90 con 0x560843d069f0 2024-03-18T23:15:49.829 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:49.824+0000 7fd5a4c9c640 1 --2- 172.21.15.49:0/3965009606 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560843d069f0 0x560843d7b750 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fd598013320 tx=0x7fd598004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ee4508fc33d33c server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:15:49.829 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:49.824+0000 7fd57f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-18T23:15:49.829 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:49.824+0000 7fd57f7fe640 1 -- 172.21.15.49:0/3965009606 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd598004540 con 0x560843d069f0 2024-03-18T23:15:49.829 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:49.824+0000 7fd57f7fe640 1 -- 172.21.15.49:0/3965009606 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7fd598009040 con 0x560843d069f0 2024-03-18T23:15:49.829 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:49.824+0000 7fd5a93c9740 1 -- 172.21.15.49:0/3965009606 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd59002c8b0 con 0x560843d069f0 2024-03-18T23:15:49.829 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:49.824+0000 7fd57f7fe640 1 -- 172.21.15.49:0/3965009606 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fd5980046e0 con 0x560843d069f0 2024-03-18T23:15:49.829 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:49.824+0000 7fd5a93c9740 10 client.16071 fetch_fsmap learned FSMap version 32 2024-03-18T23:15:49.829 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:49.825+0000 7fd5a93c9740 1 -- 172.21.15.49:0/3965009606 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x560843d7cfa0 con 0x560843d069f0 2024-03-18T23:15:49.829 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:49.825+0000 7fd57f7fe640 1 -- 172.21.15.49:0/3965009606 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd59002c8b0 con 0x560843d069f0 2024-03-18T23:15:49.829 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:49.825+0000 7fd57f7fe640 1 -- 172.21.15.49:0/3965009606 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7fd598020050 con 0x560843d069f0 2024-03-18T23:15:49.830 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:49.825+0000 7fd5a93c9740 10 client.16071 fetch_fsmap finished waiting for FSMap version 32 2024-03-18T23:15:49.830 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:49.825+0000 7fd5a93c9740 10 client.16071 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T23:15:49.830 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:49.825+0000 7fd5a93c9740 20 client.16071 populate_metadata read hostname 'smithi049' 2024-03-18T23:15:49.830 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:49.825+0000 7fd5a93c9740 1 --2- 172.21.15.49:0/3965009606 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x560843d16680 0x560843d7f710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:15:49.830 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:49.825+0000 7fd5a93c9740 4 client.16071 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T23:15:49.830 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:49.825+0000 7fd5a93c9740 1 -- 172.21.15.49:0/3965009606 --> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560843c64470 con 0x560843d16680 2024-03-18T23:15:49.830 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:49.825+0000 7fd59ffff640 1 --2- 172.21.15.49:0/3965009606 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x560843d16680 0x560843d7f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:15:49.830 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:49.825+0000 7fd57f7fe640 10 client.16071 ms_handle_connect on v2:172.21.15.139:6838/4280128209 2024-03-18T23:15:49.830 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:49.825+0000 7fd59ffff640 1 --2- 172.21.15.49:0/3965009606 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x560843d16680 0x560843d7f710 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b4e7e6d2f62978bf server_cookie=69b498b5f87fa8 in_seq=0 out_seq=0 2024-03-18T23:15:50.031 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:50.030+0000 7fd57f7fe640 1 -- 172.21.15.49:0/3965009606 <== mds.0 v2:172.21.15.139:6838/4280128209 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x560843c64470 con 0x560843d16680 2024-03-18T23:15:50.031 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:50.030+0000 7fd57f7fe640 10 client.16071 handle_command_reply: tid=0 2024-03-18T23:15:50.031 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:50.030+0000 7fd5a93c9740 1 client.16071 shutdown 2024-03-18T23:15:50.031 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:50.030+0000 7fd5a93c9740 1 -- 172.21.15.49:0/3965009606 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560843d069f0 msgr2=0x560843d7b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:15:50.031 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:50.030+0000 7fd5a93c9740 1 --2- 172.21.15.49:0/3965009606 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560843d069f0 0x560843d7b750 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fd598013320 tx=0x7fd598004ab0 comp rx=0 tx=0).stop 2024-03-18T23:15:50.031 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:50.030+0000 7fd5a93c9740 1 -- 172.21.15.49:0/3965009606 shutdown_connections 2024-03-18T23:15:50.031 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:50.030+0000 7fd5a93c9740 1 --2- 172.21.15.49:0/3965009606 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x560843d16680 0x560843d7f710 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:50.031 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:50.031+0000 7fd5a93c9740 1 --2- 172.21.15.49:0/3965009606 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560843d069f0 0x560843d7b750 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:50.031 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:50.031+0000 7fd5a93c9740 1 --2- 172.21.15.49:0/3965009606 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560843cf9330 0x560843d0aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:50.031 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:50.031+0000 7fd5a93c9740 1 --2- 172.21.15.49:0/3965009606 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560843cf8a10 0x560843d0a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:50.032 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:50.031+0000 7fd5a93c9740 1 -- 172.21.15.49:0/3965009606 >> 172.21.15.49:0/3965009606 conn(0x560843cf3810 msgr2=0x560843cf54f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:15:50.032 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:50.031+0000 7fd57f7fe640 0 client.16071 ms_handle_reset on v2:172.21.15.139:6838/4280128209 2024-03-18T23:15:50.032 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:50.031+0000 7fd5a93c9740 1 -- 172.21.15.49:0/3965009606 shutdown_connections 2024-03-18T23:15:50.032 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:50.031+0000 7fd5a93c9740 1 -- 172.21.15.49:0/3965009606 wait complete. 2024-03-18T23:15:50.032 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:50.031+0000 7fd5a93c9740 20 client.16071 trim_cache size 0 max 16384 2024-03-18T23:15:50.032 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:{ 2024-03-18T23:15:50.032 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "message": "", 2024-03-18T23:15:50.032 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "return_code": 0 2024-03-18T23:15:50.033 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:} 2024-03-18T23:15:50.033 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:50.031+0000 7fd5a66ff640 1 -- 172.21.15.49:0/3274605839 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7fd5880a2a10 msgr2=0x7fd5880a4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:15:50.034 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:50.031+0000 7fd5a66ff640 1 --2- 172.21.15.49:0/3274605839 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7fd5880a2a10 0x7fd5880a4ed0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fd598005fd0 tx=0x7fd598009df0 comp rx=0 tx=0).stop 2024-03-18T23:15:50.034 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:50.032+0000 7fd5a66ff640 1 -- 172.21.15.49:0/3274605839 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd5a0101330 msgr2=0x7fd5a0198600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:15:50.034 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:50.032+0000 7fd5a66ff640 1 --2- 172.21.15.49:0/3274605839 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd5a0101330 0x7fd5a0198600 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fd59400e9f0 tx=0x7fd59400eeb0 comp rx=0 tx=0).stop 2024-03-18T23:15:50.034 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:50.032+0000 7fd5a66ff640 1 -- 172.21.15.49:0/3274605839 shutdown_connections 2024-03-18T23:15:50.034 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:50.032+0000 7fd5a66ff640 1 --2- 172.21.15.49:0/3274605839 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7fd5880a2a10 0x7fd5880a4ed0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:50.034 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:50.032+0000 7fd5a66ff640 1 --2- 172.21.15.49:0/3274605839 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd5a01056c0 0x7fd5a019c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:50.034 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:50.032+0000 7fd5a66ff640 1 --2- 172.21.15.49:0/3274605839 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd5a0101330 0x7fd5a0198600 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:50.034 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:50.032+0000 7fd5a66ff640 1 --2- 172.21.15.49:0/3274605839 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd5a0100970 0x7fd5a01980c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:50.034 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:50.032+0000 7fd5a66ff640 1 -- 172.21.15.49:0/3274605839 >> 172.21.15.49:0/3274605839 conn(0x7fd5a0077570 msgr2=0x7fd5a00fe980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:15:50.034 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:50.032+0000 7fd5a66ff640 1 -- 172.21.15.49:0/3274605839 shutdown_connections 2024-03-18T23:15:50.034 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:15:50.032+0000 7fd5a66ff640 1 -- 172.21.15.49:0/3274605839 wait complete. 2024-03-18T23:15:50.777 INFO:tasks.workunit.client.0.smithi049.stdout: 1 816066 9.96 MB/sec execute 292 sec latency 154.749 ms 2024-03-18T23:15:51.460 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:15:51 smithi175 ceph-mon[67352]: pgmap v234: 97 pgs: 97 active+clean; 80 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.6 MiB/s wr, 268 op/s 2024-03-18T23:15:51.564 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:15:51 smithi139 ceph-mon[67282]: pgmap v234: 97 pgs: 97 active+clean; 80 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.6 MiB/s wr, 268 op/s 2024-03-18T23:15:51.629 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:15:51 smithi049 ceph-mon[62991]: pgmap v234: 97 pgs: 97 active+clean; 80 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.6 MiB/s wr, 268 op/s 2024-03-18T23:15:51.777 INFO:tasks.workunit.client.0.smithi049.stdout: 1 817873 9.95 MB/sec execute 293 sec latency 26.743 ms 2024-03-18T23:15:52.777 INFO:tasks.workunit.client.0.smithi049.stdout: 1 819979 9.96 MB/sec execute 294 sec latency 49.879 ms 2024-03-18T23:15:53.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:15:53 smithi175 ceph-mon[67352]: pgmap v235: 97 pgs: 97 active+clean; 70 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 4.1 MiB/s wr, 448 op/s 2024-03-18T23:15:53.564 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:15:53 smithi139 ceph-mon[67282]: pgmap v235: 97 pgs: 97 active+clean; 70 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 4.1 MiB/s wr, 448 op/s 2024-03-18T23:15:53.629 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:15:53 smithi049 ceph-mon[62991]: pgmap v235: 97 pgs: 97 active+clean; 70 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 4.1 MiB/s wr, 448 op/s 2024-03-18T23:15:53.777 INFO:tasks.workunit.client.0.smithi049.stdout: 1 821641 9.96 MB/sec execute 295 sec latency 27.718 ms 2024-03-18T23:15:54.777 INFO:tasks.workunit.client.0.smithi049.stdout: 1 823663 9.97 MB/sec execute 296 sec latency 118.214 ms 2024-03-18T23:15:55.460 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:15:55 smithi175 ceph-mon[67352]: pgmap v236: 97 pgs: 97 active+clean; 70 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.7 MiB/s wr, 260 op/s 2024-03-18T23:15:55.564 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:15:55 smithi139 ceph-mon[67282]: pgmap v236: 97 pgs: 97 active+clean; 70 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.7 MiB/s wr, 260 op/s 2024-03-18T23:15:55.629 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:15:55 smithi049 ceph-mon[62991]: pgmap v236: 97 pgs: 97 active+clean; 70 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.7 MiB/s wr, 260 op/s 2024-03-18T23:15:55.777 INFO:tasks.workunit.client.0.smithi049.stdout: 1 825094 9.96 MB/sec execute 297 sec latency 23.435 ms 2024-03-18T23:15:56.777 INFO:tasks.workunit.client.0.smithi049.stdout: 1 827225 9.97 MB/sec execute 298 sec latency 28.572 ms 2024-03-18T23:15:57.460 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:15:57 smithi175 ceph-mon[67352]: pgmap v237: 97 pgs: 97 active+clean; 70 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.7 MiB/s wr, 260 op/s 2024-03-18T23:15:57.564 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:15:57 smithi139 ceph-mon[67282]: pgmap v237: 97 pgs: 97 active+clean; 70 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.7 MiB/s wr, 260 op/s 2024-03-18T23:15:57.629 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:15:57 smithi049 ceph-mon[62991]: pgmap v237: 97 pgs: 97 active+clean; 70 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.7 MiB/s wr, 260 op/s 2024-03-18T23:15:57.777 INFO:tasks.workunit.client.0.smithi049.stdout: 1 828662 9.96 MB/sec execute 299 sec latency 25.736 ms 2024-03-18T23:15:58.778 INFO:tasks.workunit.client.0.smithi049.stdout: 1 830835 9.97 MB/sec execute 300 sec latency 27.530 ms 2024-03-18T23:15:59.460 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:15:59 smithi175 ceph-mon[67352]: pgmap v238: 97 pgs: 97 active+clean; 77 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 4.0 MiB/s wr, 349 op/s 2024-03-18T23:15:59.564 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:15:59 smithi139 ceph-mon[67282]: pgmap v238: 97 pgs: 97 active+clean; 77 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 4.0 MiB/s wr, 349 op/s 2024-03-18T23:15:59.629 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:15:59 smithi049 ceph-mon[62991]: pgmap v238: 97 pgs: 97 active+clean; 77 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 4.0 MiB/s wr, 349 op/s 2024-03-18T23:15:59.777 INFO:tasks.workunit.client.0.smithi049.stdout: 1 832007 9.94 MB/sec execute 301 sec latency 97.826 ms 2024-03-18T23:16:00.777 INFO:tasks.workunit.client.0.smithi049.stdout: 1 833065 9.93 MB/sec execute 302 sec latency 34.442 ms 2024-03-18T23:16:01.460 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:16:01 smithi175 ceph-mon[67352]: pgmap v239: 97 pgs: 97 active+clean; 77 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 2.7 MiB/s wr, 269 op/s 2024-03-18T23:16:01.564 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:16:01 smithi139 ceph-mon[67282]: pgmap v239: 97 pgs: 97 active+clean; 77 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 2.7 MiB/s wr, 269 op/s 2024-03-18T23:16:01.629 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:16:01 smithi049 ceph-mon[62991]: pgmap v239: 97 pgs: 97 active+clean; 77 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 2.7 MiB/s wr, 269 op/s 2024-03-18T23:16:01.777 INFO:tasks.workunit.client.0.smithi049.stdout: 1 834280 9.93 MB/sec execute 303 sec latency 219.442 ms 2024-03-18T23:16:02.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:16:02 smithi139 ceph-mon[67282]: osdmap e94: 12 total, 12 up, 12 in 2024-03-18T23:16:02.629 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:16:02 smithi049 ceph-mon[62991]: osdmap e94: 12 total, 12 up, 12 in 2024-03-18T23:16:02.710 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:16:02 smithi175 ceph-mon[67352]: osdmap e94: 12 total, 12 up, 12 in 2024-03-18T23:16:02.777 INFO:tasks.workunit.client.0.smithi049.stdout: 1 834640 9.90 MB/sec execute 304 sec latency 79.689 ms 2024-03-18T23:16:03.168 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:03.167+0000 7f4226e19640 1 -- 172.21.15.49:0/3099103217 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f42201013c0 msgr2=0x7f422010d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:16:03.169 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:03.167+0000 7f4226e19640 1 --2- 172.21.15.49:0/3099103217 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f42201013c0 0x7f422010d4e0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f4218009ca0 tx=0x7f421802c0f0 comp rx=0 tx=0).stop 2024-03-18T23:16:03.169 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:03.167+0000 7f4226e19640 1 -- 172.21.15.49:0/3099103217 shutdown_connections 2024-03-18T23:16:03.169 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:03.167+0000 7f4226e19640 1 --2- 172.21.15.49:0/3099103217 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f42201013c0 0x7f422010d4e0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:03.169 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:03.167+0000 7f4226e19640 1 --2- 172.21.15.49:0/3099103217 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4220100a00 0x7f4220100e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:03.169 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:03.167+0000 7f4226e19640 1 --2- 172.21.15.49:0/3099103217 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4220106780 0x7f4220106b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:03.169 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:03.167+0000 7f4226e19640 1 -- 172.21.15.49:0/3099103217 >> 172.21.15.49:0/3099103217 conn(0x7f4220077700 msgr2=0x7f4220077b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:16:03.169 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:03.168+0000 7f4226e19640 1 -- 172.21.15.49:0/3099103217 shutdown_connections 2024-03-18T23:16:03.169 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:03.168+0000 7f4226e19640 1 -- 172.21.15.49:0/3099103217 wait complete. 2024-03-18T23:16:03.169 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:03.168+0000 7f4226e19640 1 Processor -- start 2024-03-18T23:16:03.169 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:03.168+0000 7f4226e19640 1 -- start start 2024-03-18T23:16:03.171 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:03.169+0000 7f4226e19640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4220100a00 0x7f4220197f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:16:03.171 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:03.169+0000 7f4226e19640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f42201013c0 0x7f42201984a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:16:03.171 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:03.169+0000 7f4226e19640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4220106780 0x7f422019c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:16:03.172 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:03.169+0000 7f4226e19640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f4220111be0 con 0x7f42201013c0 2024-03-18T23:16:03.172 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:03.169+0000 7f4226e19640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f4220111a60 con 0x7f4220100a00 2024-03-18T23:16:03.172 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:03.169+0000 7f4226e19640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f4220111d60 con 0x7f4220106780 2024-03-18T23:16:03.172 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:03.169+0000 7f4224bb5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4220100a00 0x7f4220197f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:16:03.172 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:03.169+0000 7f4224bb5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4220100a00 0x7f4220197f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.49:34626/0 (socket says 172.21.15.49:34626) 2024-03-18T23:16:03.172 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:03.169+0000 7f4224bb5640 1 -- 172.21.15.49:0/1138451974 learned_addr learned my addr 172.21.15.49:0/1138451974 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:16:03.172 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:03.169+0000 7f42253b6640 1 --2- 172.21.15.49:0/1138451974 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4220106780 0x7f422019c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:16:03.172 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:03.169+0000 7f421ffff640 1 --2- 172.21.15.49:0/1138451974 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f42201013c0 0x7f42201984a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:16:03.172 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:03.169+0000 7f4224bb5640 1 -- 172.21.15.49:0/1138451974 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4220106780 msgr2=0x7f422019c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:16:03.172 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:03.169+0000 7f4224bb5640 1 --2- 172.21.15.49:0/1138451974 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4220106780 0x7f422019c7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:03.172 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:03.169+0000 7f4224bb5640 1 -- 172.21.15.49:0/1138451974 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f42201013c0 msgr2=0x7f42201984a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:16:03.172 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:03.169+0000 7f4224bb5640 1 --2- 172.21.15.49:0/1138451974 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f42201013c0 0x7f42201984a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:03.172 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:03.169+0000 7f4224bb5640 1 -- 172.21.15.49:0/1138451974 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f422019cf00 con 0x7f4220100a00 2024-03-18T23:16:03.173 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:03.169+0000 7f421ffff640 1 --2- 172.21.15.49:0/1138451974 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f42201013c0 0x7f42201984a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T23:16:03.173 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:03.170+0000 7f4224bb5640 1 --2- 172.21.15.49:0/1138451974 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4220100a00 0x7f4220197f60 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f421400e9f0 tx=0x7f421400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:16:03.173 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:03.170+0000 7f421dffb640 1 -- 172.21.15.49:0/1138451974 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f421400cd80 con 0x7f4220100a00 2024-03-18T23:16:03.173 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:03.170+0000 7f4226e19640 1 -- 172.21.15.49:0/1138451974 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f422019d1f0 con 0x7f4220100a00 2024-03-18T23:16:03.173 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:03.170+0000 7f4226e19640 1 -- 172.21.15.49:0/1138451974 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f422010dea0 con 0x7f4220100a00 2024-03-18T23:16:03.173 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:03.170+0000 7f421dffb640 1 -- 172.21.15.49:0/1138451974 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4214010910 con 0x7f4220100a00 2024-03-18T23:16:03.173 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:03.170+0000 7f421dffb640 1 -- 172.21.15.49:0/1138451974 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4214018980 con 0x7f4220100a00 2024-03-18T23:16:03.174 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:03.173+0000 7f421dffb640 1 -- 172.21.15.49:0/1138451974 <== mon.2 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 25) ==== 137421+0+0 (secure 0 0 0) 0x7f4214010ac0 con 0x7f4220100a00 2024-03-18T23:16:03.174 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:03.174+0000 7f4229a0a740 1 Processor -- start 2024-03-18T23:16:03.174 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:03.174+0000 7f4229a0a740 1 -- start start 2024-03-18T23:16:03.176 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:03.175+0000 7f4229a0a740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5615ed12def0 0x5615ed12e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:16:03.176 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:03.175+0000 7f4229a0a740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5615ed12e810 0x5615ed1370a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:16:03.176 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:03.175+0000 7f4229a0a740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5615ed1375e0 0x5615ed1399d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:16:03.176 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:03.175+0000 7f4229a0a740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5615ed091be0 con 0x5615ed12e810 2024-03-18T23:16:03.176 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:03.175+0000 7f4229a0a740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x5615ed03a960 con 0x5615ed1375e0 2024-03-18T23:16:03.176 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:03.175+0000 7f4229a0a740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5615ed08dc80 con 0x5615ed12def0 2024-03-18T23:16:03.177 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:03.175+0000 7f42253b6640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5615ed12def0 0x5615ed12e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:16:03.177 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:03.175+0000 7f42253b6640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5615ed12def0 0x5615ed12e2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.49:55732/0 (socket says 172.21.15.49:55732) 2024-03-18T23:16:03.177 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:03.175+0000 7f42253b6640 1 -- 172.21.15.49:0/74583022 learned_addr learned my addr 172.21.15.49:0/74583022 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:16:03.177 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:03.175+0000 7f421ffff640 1 --2- 172.21.15.49:0/74583022 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5615ed1375e0 0x5615ed1399d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:16:03.177 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:03.175+0000 7f4224bb5640 1 --2- 172.21.15.49:0/74583022 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5615ed12e810 0x5615ed1370a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:16:03.177 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:03.175+0000 7f421dffb640 1 --2- 172.21.15.49:0/1138451974 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f42100a2a90 0x7f42100a4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:16:03.177 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:03.175+0000 7f42253b6640 1 -- 172.21.15.49:0/74583022 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5615ed1375e0 msgr2=0x5615ed1399d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:16:03.177 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:03.175+0000 7f42253b6640 1 --2- 172.21.15.49:0/74583022 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5615ed1375e0 0x5615ed1399d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:03.177 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:03.175+0000 7f42253b6640 1 -- 172.21.15.49:0/74583022 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5615ed12e810 msgr2=0x5615ed1370a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:16:03.177 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:03.175+0000 7f42253b6640 1 --2- 172.21.15.49:0/74583022 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5615ed12e810 0x5615ed1370a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:03.177 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:03.175+0000 7f42253b6640 1 -- 172.21.15.49:0/74583022 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5615ed031d20 con 0x5615ed12def0 2024-03-18T23:16:03.177 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:03.175+0000 7f421ffff640 1 --2- 172.21.15.49:0/74583022 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5615ed1375e0 0x5615ed1399d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:16:03.177 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:03.175+0000 7f421dffb640 1 -- 172.21.15.49:0/1138451974 <== mon.2 v2:172.21.15.139:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8600+0+0 (secure 0 0 0) 0x7f42140cf340 con 0x7f4220100a00 2024-03-18T23:16:03.178 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:03.176+0000 7f42253b6640 1 --2- 172.21.15.49:0/74583022 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5615ed12def0 0x5615ed12e2d0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f4218009ca0 tx=0x7f4218009900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a9c31d74ea7c1e25 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:16:03.178 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:03.176+0000 7f42253b6640 1 --2- 172.21.15.49:0/1138451974 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f42100a2a90 0x7f42100a4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:16:03.178 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:03.176+0000 7f420affd640 1 -- 172.21.15.49:0/74583022 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4218043070 con 0x5615ed12def0 2024-03-18T23:16:03.178 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:03.176+0000 7f420affd640 1 -- 172.21.15.49:0/74583022 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4218003e40 con 0x5615ed12def0 2024-03-18T23:16:03.179 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:03.177+0000 7f4229a0a740 1 -- 172.21.15.49:0/74583022 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5615ed12def0 msgr2=0x5615ed12e2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:16:03.180 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:03.177+0000 7f4229a0a740 1 --2- 172.21.15.49:0/74583022 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5615ed12def0 0x5615ed12e2d0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f4218009ca0 tx=0x7f4218009900 comp rx=0 tx=0).stop 2024-03-18T23:16:03.180 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:03.177+0000 7f42253b6640 1 --2- 172.21.15.49:0/1138451974 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f42100a2a90 0x7f42100a4f50 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f4218045000 tx=0x7f4218041000 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:16:03.180 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:03.177+0000 7f4229a0a740 1 -- 172.21.15.49:0/74583022 shutdown_connections 2024-03-18T23:16:03.180 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:03.177+0000 7f4229a0a740 1 --2- 172.21.15.49:0/74583022 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5615ed1375e0 0x5615ed1399d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:03.180 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:03.177+0000 7f4229a0a740 1 --2- 172.21.15.49:0/74583022 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5615ed12e810 0x5615ed1370a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:03.180 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:03.177+0000 7f4229a0a740 1 --2- 172.21.15.49:0/74583022 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5615ed12def0 0x5615ed12e2d0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:03.180 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:03.177+0000 7f4229a0a740 1 -- 172.21.15.49:0/74583022 >> 172.21.15.49:0/74583022 conn(0x5615ed128810 msgr2=0x5615ed128bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:16:03.180 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:03.177+0000 7f4229a0a740 1 -- 172.21.15.49:0/74583022 shutdown_connections 2024-03-18T23:16:03.180 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:03.178+0000 7f4229a0a740 1 -- 172.21.15.49:0/74583022 wait complete. 2024-03-18T23:16:03.180 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:03.178+0000 7f4229a0a740 1 -- start start 2024-03-18T23:16:03.180 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:03.178+0000 7f4229a0a740 1 Processor -- start 2024-03-18T23:16:03.180 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:03.179+0000 7f4229a0a740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5615ed12def0 0x5615ed136ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:16:03.180 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:03.179+0000 7f4229a0a740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5615ed12e810 0x5615ed134260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:16:03.181 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:03.179+0000 7f4229a0a740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5615ed1375e0 0x5615ed1347a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:16:03.181 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:03.179+0000 7f4229a0a740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5615ed13a1b0 con 0x5615ed12def0 2024-03-18T23:16:03.181 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:03.179+0000 7f4229a0a740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x5615ed04ddb0 con 0x5615ed1375e0 2024-03-18T23:16:03.181 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:03.179+0000 7f4229a0a740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5615ed13a330 con 0x5615ed12e810 2024-03-18T23:16:03.181 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:03.179+0000 7f4229a0a740 10 client.0 _ll_register_callbacks cb 0x5615ed052e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T23:16:03.181 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:03.179+0000 7f42253b6640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5615ed1375e0 0x5615ed1347a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:16:03.181 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:03.179+0000 7f4224bb5640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5615ed12def0 0x5615ed136ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:16:03.181 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:03.179+0000 7f42253b6640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5615ed1375e0 0x5615ed1347a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.49:34644/0 (socket says 172.21.15.49:34644) 2024-03-18T23:16:03.181 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:03.179+0000 7f421ffff640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5615ed12e810 0x5615ed134260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:16:03.181 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:03.179+0000 7f42253b6640 1 -- 172.21.15.49:0/3367557925 learned_addr learned my addr 172.21.15.49:0/3367557925 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:16:03.181 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:03.179+0000 7f4224bb5640 1 -- 172.21.15.49:0/3367557925 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5615ed12e810 msgr2=0x5615ed134260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:16:03.181 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:03.179+0000 7f4224bb5640 1 --2- 172.21.15.49:0/3367557925 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5615ed12e810 0x5615ed134260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:03.181 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:03.179+0000 7f4224bb5640 1 -- 172.21.15.49:0/3367557925 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5615ed1375e0 msgr2=0x5615ed1347a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:16:03.181 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:03.179+0000 7f4224bb5640 1 --2- 172.21.15.49:0/3367557925 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5615ed1375e0 0x5615ed1347a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:03.182 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:03.179+0000 7f4224bb5640 1 -- 172.21.15.49:0/3367557925 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5615ed04cc60 con 0x5615ed12def0 2024-03-18T23:16:03.182 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:03.179+0000 7f42253b6640 1 --2- 172.21.15.49:0/3367557925 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5615ed1375e0 0x5615ed1347a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:16:03.182 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:03.180+0000 7f4224bb5640 1 --2- 172.21.15.49:0/3367557925 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5615ed12def0 0x5615ed136ed0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f42140cc9e0 tx=0x7f4214010e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d58370697a87e67f server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:16:03.182 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:03.180+0000 7f41fffff640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-18T23:16:03.183 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:03.180+0000 7f41fffff640 1 -- 172.21.15.49:0/3367557925 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f421409b430 con 0x5615ed12def0 2024-03-18T23:16:03.184 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:03.180+0000 7f41fffff640 1 -- 172.21.15.49:0/3367557925 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8600+0+0 (secure 0 0 0) 0x7f42140860b0 con 0x5615ed12def0 2024-03-18T23:16:03.184 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:03.180+0000 7f4229a0a740 1 -- 172.21.15.49:0/3367557925 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5615ed09f120 con 0x5615ed12def0 2024-03-18T23:16:03.184 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:03.180+0000 7f41fffff640 1 -- 172.21.15.49:0/3367557925 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f421409b250 con 0x5615ed12def0 2024-03-18T23:16:03.184 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:03.180+0000 7f4229a0a740 10 client.16083 fetch_fsmap learned FSMap version 32 2024-03-18T23:16:03.184 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:03.180+0000 7f4229a0a740 1 -- 172.21.15.49:0/3367557925 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5615ecda6210 con 0x5615ed12def0 2024-03-18T23:16:03.184 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:03.181+0000 7f41fffff640 1 -- 172.21.15.49:0/3367557925 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x5615ed09f120 con 0x5615ed12def0 2024-03-18T23:16:03.184 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:03.181+0000 7f41fffff640 1 -- 172.21.15.49:0/3367557925 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f421409ad40 con 0x5615ed12def0 2024-03-18T23:16:03.184 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:03.181+0000 7f4229a0a740 10 client.16083 fetch_fsmap finished waiting for FSMap version 32 2024-03-18T23:16:03.184 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:03.181+0000 7f4229a0a740 10 client.16083 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T23:16:03.184 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:03.181+0000 7f4229a0a740 20 client.16083 populate_metadata read hostname 'smithi049' 2024-03-18T23:16:03.184 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:03.181+0000 7f4229a0a740 1 --2- 172.21.15.49:0/3367557925 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x5615ed147400 0x5615ed1497f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:16:03.184 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:03.181+0000 7f4229a0a740 4 client.16083 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T23:16:03.184 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:03.181+0000 7f4229a0a740 1 -- 172.21.15.49:0/3367557925 --> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5615ed099470 con 0x5615ed147400 2024-03-18T23:16:03.185 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:03.182+0000 7f421ffff640 1 --2- 172.21.15.49:0/3367557925 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x5615ed147400 0x5615ed1497f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:16:03.185 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:03.182+0000 7f421ffff640 1 --2- 172.21.15.49:0/3367557925 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x5615ed147400 0x5615ed1497f0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6734fbda72334ec5 server_cookie=989fc0cdddd98da5 in_seq=0 out_seq=0 2024-03-18T23:16:03.185 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:03.182+0000 7f41fffff640 10 client.16083 ms_handle_connect on v2:172.21.15.139:6838/4280128209 2024-03-18T23:16:03.189 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:03.188+0000 7f41fffff640 1 -- 172.21.15.49:0/3367557925 <== mds.0 v2:172.21.15.139:6838/4280128209 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5615ed099470 con 0x5615ed147400 2024-03-18T23:16:03.189 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:03.188+0000 7f41fffff640 10 client.16083 handle_command_reply: tid=0 2024-03-18T23:16:03.189 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:03.188+0000 7f4229a0a740 1 client.16083 shutdown 2024-03-18T23:16:03.189 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:03.188+0000 7f4229a0a740 1 -- 172.21.15.49:0/3367557925 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5615ed12def0 msgr2=0x5615ed136ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:16:03.189 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:03.188+0000 7f4229a0a740 1 --2- 172.21.15.49:0/3367557925 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5615ed12def0 0x5615ed136ed0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f42140cc9e0 tx=0x7f4214010e30 comp rx=0 tx=0).stop 2024-03-18T23:16:03.189 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:03.188+0000 7f4229a0a740 1 -- 172.21.15.49:0/3367557925 shutdown_connections 2024-03-18T23:16:03.189 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:03.188+0000 7f4229a0a740 1 --2- 172.21.15.49:0/3367557925 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x5615ed147400 0x5615ed1497f0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:03.189 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:03.188+0000 7f4229a0a740 1 --2- 172.21.15.49:0/3367557925 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5615ed1375e0 0x5615ed1347a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:03.190 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:03.188+0000 7f41fffff640 0 client.16083 ms_handle_reset on v2:172.21.15.139:6838/4280128209 2024-03-18T23:16:03.190 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:03.188+0000 7f4229a0a740 1 --2- 172.21.15.49:0/3367557925 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5615ed12e810 0x5615ed134260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:03.190 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:03.188+0000 7f4229a0a740 1 --2- 172.21.15.49:0/3367557925 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5615ed12def0 0x5615ed136ed0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:03.190 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:03.188+0000 7f4229a0a740 1 -- 172.21.15.49:0/3367557925 >> 172.21.15.49:0/3367557925 conn(0x5615ed128810 msgr2=0x5615ed127ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:16:03.190 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:03.189+0000 7f4229a0a740 1 -- 172.21.15.49:0/3367557925 shutdown_connections 2024-03-18T23:16:03.190 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:03.189+0000 7f4229a0a740 1 -- 172.21.15.49:0/3367557925 wait complete. 2024-03-18T23:16:03.190 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:03.189+0000 7f4229a0a740 20 client.16083 trim_cache size 0 max 16384 2024-03-18T23:16:03.216 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:03.214+0000 7f4229a0a740 1 Processor -- start 2024-03-18T23:16:03.216 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:03.214+0000 7f4229a0a740 1 -- start start 2024-03-18T23:16:03.216 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:03.215+0000 7f4229a0a740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5615ed12def0 0x5615ed12a370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:16:03.216 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:03.215+0000 7f4229a0a740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5615ed12e810 0x5615ed14f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:16:03.216 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:03.215+0000 7f4229a0a740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5615ed1375e0 0x5615ed13f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:16:03.216 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:03.215+0000 7f4229a0a740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f42201af690 con 0x5615ed12e810 2024-03-18T23:16:03.216 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:03.215+0000 7f4229a0a740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f42201b11d0 con 0x5615ed12def0 2024-03-18T23:16:03.216 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:03.215+0000 7f4229a0a740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f42201afd60 con 0x5615ed1375e0 2024-03-18T23:16:03.216 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:03.215+0000 7f4224bb5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5615ed12def0 0x5615ed12a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:16:03.216 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:03.215+0000 7f4224bb5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5615ed12def0 0x5615ed12a370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.49:34652/0 (socket says 172.21.15.49:34652) 2024-03-18T23:16:03.216 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:03.215+0000 7f4224bb5640 1 -- 172.21.15.49:0/1804696697 learned_addr learned my addr 172.21.15.49:0/1804696697 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:16:03.217 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:03.215+0000 7f421ffff640 1 --2- 172.21.15.49:0/1804696697 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5615ed12e810 0x5615ed14f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:16:03.217 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:03.215+0000 7f42253b6640 1 --2- 172.21.15.49:0/1804696697 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5615ed1375e0 0x5615ed13f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:16:03.217 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:03.215+0000 7f421ffff640 1 -- 172.21.15.49:0/1804696697 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5615ed1375e0 msgr2=0x5615ed13f5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:16:03.217 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:03.215+0000 7f421ffff640 1 --2- 172.21.15.49:0/1804696697 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5615ed1375e0 0x5615ed13f5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:03.217 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:03.215+0000 7f421ffff640 1 -- 172.21.15.49:0/1804696697 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5615ed12def0 msgr2=0x5615ed12a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:16:03.217 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:03.215+0000 7f421ffff640 1 --2- 172.21.15.49:0/1804696697 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5615ed12def0 0x5615ed12a370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:03.217 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:03.215+0000 7f421ffff640 1 -- 172.21.15.49:0/1804696697 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5615ed12e640 con 0x5615ed12e810 2024-03-18T23:16:03.217 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:03.216+0000 7f421ffff640 1 --2- 172.21.15.49:0/1804696697 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5615ed12e810 0x5615ed14f0b0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x5615ed14a050 tx=0x7f420c005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9a6149c8691cc8a server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:16:03.218 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:03.217+0000 7f420a7fc640 1 -- 172.21.15.49:0/1804696697 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f420c021420 con 0x5615ed12e810 2024-03-18T23:16:03.218 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:03.217+0000 7f420a7fc640 1 -- 172.21.15.49:0/1804696697 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f420c005500 con 0x5615ed12e810 2024-03-18T23:16:03.218 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:03.217+0000 7f420a7fc640 1 -- 172.21.15.49:0/1804696697 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f420c011500 con 0x5615ed12e810 2024-03-18T23:16:03.219 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:03.218+0000 7f4229a0a740 1 -- 172.21.15.49:0/1804696697 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5615ed12e810 msgr2=0x5615ed14f0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:16:03.219 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:03.218+0000 7f4229a0a740 1 --2- 172.21.15.49:0/1804696697 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5615ed12e810 0x5615ed14f0b0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x5615ed14a050 tx=0x7f420c005f90 comp rx=0 tx=0).stop 2024-03-18T23:16:03.219 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:03.218+0000 7f4229a0a740 1 -- 172.21.15.49:0/1804696697 shutdown_connections 2024-03-18T23:16:03.219 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:03.218+0000 7f4229a0a740 1 --2- 172.21.15.49:0/1804696697 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5615ed1375e0 0x5615ed13f5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:03.219 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:03.218+0000 7f4229a0a740 1 --2- 172.21.15.49:0/1804696697 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5615ed12e810 0x5615ed14f0b0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:03.219 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:03.218+0000 7f4229a0a740 1 --2- 172.21.15.49:0/1804696697 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5615ed12def0 0x5615ed12a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:03.219 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:03.218+0000 7f4229a0a740 1 -- 172.21.15.49:0/1804696697 >> 172.21.15.49:0/1804696697 conn(0x5615ed128810 msgr2=0x5615ed129c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:16:03.219 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:03.218+0000 7f4229a0a740 1 -- 172.21.15.49:0/1804696697 shutdown_connections 2024-03-18T23:16:03.219 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:03.218+0000 7f4229a0a740 1 -- 172.21.15.49:0/1804696697 wait complete. 2024-03-18T23:16:03.220 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:03.219+0000 7f4229a0a740 1 -- start start 2024-03-18T23:16:03.220 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:03.219+0000 7f4229a0a740 1 Processor -- start 2024-03-18T23:16:03.220 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:03.219+0000 7f4229a0a740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5615ed12def0 0x5615ed13f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:16:03.220 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:03.220+0000 7f4229a0a740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5615ed12e810 0x5615ed130b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:16:03.220 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:03.220+0000 7f4224bb5640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5615ed12def0 0x5615ed13f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:16:03.220 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:03.220+0000 7f4229a0a740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5615ed1375e0 0x5615ed131050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:16:03.220 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:03.220+0000 7f4224bb5640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5615ed12def0 0x5615ed13f830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.49:55754/0 (socket says 172.21.15.49:55754) 2024-03-18T23:16:03.220 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:03.220+0000 7f421ffff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5615ed12e810 0x5615ed130b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:16:03.221 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:03.220+0000 7f4229a0a740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f42201b0b00 con 0x5615ed12e810 2024-03-18T23:16:03.221 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:03.220+0000 7f421ffff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5615ed12e810 0x5615ed130b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58464/0 (socket says 172.21.15.49:58464) 2024-03-18T23:16:03.221 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:03.220+0000 7f4229a0a740 1 -- 172.21.15.49:0/3580639773 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f42201b0430 con 0x5615ed1375e0 2024-03-18T23:16:03.221 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:03.220+0000 7f421ffff640 1 -- 172.21.15.49:0/3580639773 learned_addr learned my addr 172.21.15.49:0/3580639773 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:16:03.221 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:03.220+0000 7f4229a0a740 1 -- 172.21.15.49:0/3580639773 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f42201ad6b0 con 0x5615ed12def0 2024-03-18T23:16:03.221 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:03.220+0000 7f4229a0a740 10 client.0 _ll_register_callbacks cb 0x7f421400cf20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T23:16:03.223 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:03.220+0000 7f42253b6640 1 --2- 172.21.15.49:0/3580639773 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5615ed1375e0 0x5615ed131050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:16:03.223 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:03.220+0000 7f4224bb5640 1 -- 172.21.15.49:0/3580639773 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5615ed1375e0 msgr2=0x5615ed131050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:16:03.223 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:03.220+0000 7f4224bb5640 1 --2- 172.21.15.49:0/3580639773 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5615ed1375e0 0x5615ed131050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:03.223 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:03.220+0000 7f4224bb5640 1 -- 172.21.15.49:0/3580639773 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5615ed12e810 msgr2=0x5615ed130b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:16:03.223 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:03.220+0000 7f4224bb5640 1 --2- 172.21.15.49:0/3580639773 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5615ed12e810 0x5615ed130b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:03.223 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:03.220+0000 7f4224bb5640 1 -- 172.21.15.49:0/3580639773 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5615ece72620 con 0x5615ed12def0 2024-03-18T23:16:03.223 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:03.221+0000 7f4224bb5640 1 --2- 172.21.15.49:0/3580639773 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5615ed12def0 0x5615ed13f830 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f421400e9c0 tx=0x7f4214018420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f2ad5fc05e20fa35 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:16:03.223 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:03.221+0000 7f41fffff640 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-03-18T23:16:03.224 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:03.221+0000 7f41fffff640 1 -- 172.21.15.49:0/3580639773 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f421409b430 con 0x5615ed12def0 2024-03-18T23:16:03.224 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:03.221+0000 7f41fffff640 1 -- 172.21.15.49:0/3580639773 <== mon.1 v2:172.21.15.175:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8600+0+0 (secure 0 0 0) 0x7f4214018b20 con 0x5615ed12def0 2024-03-18T23:16:03.224 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:03.221+0000 7f4229a0a740 1 -- 172.21.15.49:0/3580639773 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f420c005500 con 0x5615ed12def0 2024-03-18T23:16:03.224 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:03.222+0000 7f41fffff640 1 -- 172.21.15.49:0/3580639773 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f4214097b90 con 0x5615ed12def0 2024-03-18T23:16:03.224 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:03.222+0000 7f4229a0a740 10 client.25891 fetch_fsmap learned FSMap version 32 2024-03-18T23:16:03.224 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:03.222+0000 7f4229a0a740 1 -- 172.21.15.49:0/3580639773 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5615ed1ad4a0 con 0x5615ed12def0 2024-03-18T23:16:03.224 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:03.222+0000 7f41fffff640 1 -- 172.21.15.49:0/3580639773 <== mon.1 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f42140862c0 con 0x5615ed12def0 2024-03-18T23:16:03.224 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:03.223+0000 7f41fffff640 1 -- 172.21.15.49:0/3580639773 <== mon.1 v2:172.21.15.175:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f421409a7b0 con 0x5615ed12def0 2024-03-18T23:16:03.224 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:03.223+0000 7f4229a0a740 10 client.25891 fetch_fsmap finished waiting for FSMap version 32 2024-03-18T23:16:03.224 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:03.223+0000 7f4229a0a740 10 client.25891 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T23:16:03.224 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:03.223+0000 7f4229a0a740 20 client.25891 populate_metadata read hostname 'smithi049' 2024-03-18T23:16:03.224 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:03.223+0000 7f4229a0a740 1 --2- 172.21.15.49:0/3580639773 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x5615ed147400 0x5615ed1afad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:16:03.224 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:03.223+0000 7f4229a0a740 4 client.25891 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T23:16:03.225 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:03.223+0000 7f4229a0a740 1 -- 172.21.15.49:0/3580639773 --> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f420c005500 con 0x5615ed147400 2024-03-18T23:16:03.225 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:03.223+0000 7f421ffff640 1 --2- 172.21.15.49:0/3580639773 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x5615ed147400 0x5615ed1afad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:16:03.225 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:03.224+0000 7f421ffff640 1 --2- 172.21.15.49:0/3580639773 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x5615ed147400 0x5615ed1afad0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5e46ea358a5abcea server_cookie=535f5347612d0549 in_seq=0 out_seq=0 2024-03-18T23:16:03.225 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:03.224+0000 7f41fffff640 10 client.25891 ms_handle_connect on v2:172.21.15.139:6838/4280128209 2024-03-18T23:16:03.549 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:03.548+0000 7f41fffff640 1 -- 172.21.15.49:0/3580639773 <== mds.0 v2:172.21.15.139:6838/4280128209 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7f420c005500 con 0x5615ed147400 2024-03-18T23:16:03.549 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:03.548+0000 7f41fffff640 10 client.25891 handle_command_reply: tid=0 2024-03-18T23:16:03.549 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:03.548+0000 7f4229a0a740 1 client.25891 shutdown 2024-03-18T23:16:03.549 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:03.548+0000 7f4229a0a740 1 -- 172.21.15.49:0/3580639773 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5615ed12def0 msgr2=0x5615ed13f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:16:03.549 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:03.548+0000 7f4229a0a740 1 --2- 172.21.15.49:0/3580639773 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5615ed12def0 0x5615ed13f830 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f421400e9c0 tx=0x7f4214018420 comp rx=0 tx=0).stop 2024-03-18T23:16:03.549 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:03.548+0000 7f4229a0a740 1 -- 172.21.15.49:0/3580639773 shutdown_connections 2024-03-18T23:16:03.549 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:03.548+0000 7f4229a0a740 1 --2- 172.21.15.49:0/3580639773 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x5615ed147400 0x5615ed1afad0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:03.549 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:03.548+0000 7f4229a0a740 1 --2- 172.21.15.49:0/3580639773 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5615ed1375e0 0x5615ed131050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:03.550 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:03.548+0000 7f4229a0a740 1 --2- 172.21.15.49:0/3580639773 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5615ed12e810 0x5615ed130b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:03.550 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:03.548+0000 7f41fffff640 0 client.25891 ms_handle_reset on v2:172.21.15.139:6838/4280128209 2024-03-18T23:16:03.550 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:03.548+0000 7f4229a0a740 1 --2- 172.21.15.49:0/3580639773 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5615ed12def0 0x5615ed13f830 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:03.550 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:03.548+0000 7f4229a0a740 1 -- 172.21.15.49:0/3580639773 >> 172.21.15.49:0/3580639773 conn(0x5615ed128810 msgr2=0x5615ed129c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:16:03.550 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:03.548+0000 7f4229a0a740 1 -- 172.21.15.49:0/3580639773 shutdown_connections 2024-03-18T23:16:03.550 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:03.548+0000 7f4229a0a740 1 -- 172.21.15.49:0/3580639773 wait complete. 2024-03-18T23:16:03.550 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:03.548+0000 7f4229a0a740 20 client.25891 trim_cache size 0 max 16384 2024-03-18T23:16:03.550 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:{ 2024-03-18T23:16:03.550 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "message": "", 2024-03-18T23:16:03.550 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "return_code": 0 2024-03-18T23:16:03.551 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:} 2024-03-18T23:16:03.551 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:03.549+0000 7f4226e19640 1 -- 172.21.15.49:0/1138451974 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f42100a2a90 msgr2=0x7f42100a4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:16:03.551 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:03.549+0000 7f4226e19640 1 --2- 172.21.15.49:0/1138451974 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f42100a2a90 0x7f42100a4f50 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f4218045000 tx=0x7f4218041000 comp rx=0 tx=0).stop 2024-03-18T23:16:03.551 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:03.549+0000 7f4226e19640 1 -- 172.21.15.49:0/1138451974 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4220100a00 msgr2=0x7f4220197f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:16:03.552 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:03.549+0000 7f4226e19640 1 --2- 172.21.15.49:0/1138451974 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4220100a00 0x7f4220197f60 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f421400e9f0 tx=0x7f421400eeb0 comp rx=0 tx=0).stop 2024-03-18T23:16:03.552 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:03.549+0000 7f4226e19640 1 -- 172.21.15.49:0/1138451974 shutdown_connections 2024-03-18T23:16:03.552 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:03.549+0000 7f4226e19640 1 --2- 172.21.15.49:0/1138451974 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f42100a2a90 0x7f42100a4f50 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:03.552 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:03.549+0000 7f4226e19640 1 --2- 172.21.15.49:0/1138451974 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4220106780 0x7f422019c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:03.552 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:03.549+0000 7f4226e19640 1 --2- 172.21.15.49:0/1138451974 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f42201013c0 0x7f42201984a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:03.552 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:03.549+0000 7f4226e19640 1 --2- 172.21.15.49:0/1138451974 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4220100a00 0x7f4220197f60 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:03.552 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:03.549+0000 7f4226e19640 1 -- 172.21.15.49:0/1138451974 >> 172.21.15.49:0/1138451974 conn(0x7f4220077700 msgr2=0x7f42200ff3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:16:03.552 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:03.549+0000 7f4226e19640 1 -- 172.21.15.49:0/1138451974 shutdown_connections 2024-03-18T23:16:03.552 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:03.549+0000 7f4226e19640 1 -- 172.21.15.49:0/1138451974 wait complete. 2024-03-18T23:16:03.564 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:16:03 smithi139 ceph-mon[67282]: pgmap v241: 97 pgs: 21 active+clean+snaptrim, 39 active+clean+snaptrim_wait, 37 active+clean; 82 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.2 MiB/s wr, 244 op/s 2024-03-18T23:16:03.629 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:16:03 smithi049 ceph-mon[62991]: pgmap v241: 97 pgs: 21 active+clean+snaptrim, 39 active+clean+snaptrim_wait, 37 active+clean; 82 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.2 MiB/s wr, 244 op/s 2024-03-18T23:16:03.710 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:16:03 smithi175 ceph-mon[67352]: pgmap v241: 97 pgs: 21 active+clean+snaptrim, 39 active+clean+snaptrim_wait, 37 active+clean; 82 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.2 MiB/s wr, 244 op/s 2024-03-18T23:16:03.778 INFO:tasks.workunit.client.0.smithi049.stdout: 1 835488 9.88 MB/sec execute 305 sec latency 96.204 ms 2024-03-18T23:16:04.564 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:16:04 smithi139 ceph-mon[67282]: pgmap v242: 97 pgs: 23 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 33 active+clean; 82 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.2 MiB/s wr, 246 op/s 2024-03-18T23:16:04.629 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:16:04 smithi049 ceph-mon[62991]: pgmap v242: 97 pgs: 23 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 33 active+clean; 82 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.2 MiB/s wr, 246 op/s 2024-03-18T23:16:04.710 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:16:04 smithi175 ceph-mon[67352]: pgmap v242: 97 pgs: 23 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 33 active+clean; 82 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.2 MiB/s wr, 246 op/s 2024-03-18T23:16:04.778 INFO:tasks.workunit.client.0.smithi049.stdout: 1 837289 9.88 MB/sec execute 306 sec latency 125.100 ms 2024-03-18T23:16:05.778 INFO:tasks.workunit.client.0.smithi049.stdout: 1 838896 9.87 MB/sec execute 307 sec latency 24.254 ms 2024-03-18T23:16:06.778 INFO:tasks.workunit.client.0.smithi049.stdout: 1 840900 9.88 MB/sec execute 308 sec latency 20.553 ms 2024-03-18T23:16:06.814 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:16:06 smithi139 ceph-mon[67282]: pgmap v243: 97 pgs: 23 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 33 active+clean; 82 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.2 MiB/s wr, 246 op/s 2024-03-18T23:16:06.879 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:16:06 smithi049 ceph-mon[62991]: pgmap v243: 97 pgs: 23 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 33 active+clean; 82 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.2 MiB/s wr, 246 op/s 2024-03-18T23:16:06.961 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:16:06 smithi175 ceph-mon[67352]: pgmap v243: 97 pgs: 23 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 33 active+clean; 82 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.2 MiB/s wr, 246 op/s 2024-03-18T23:16:07.778 INFO:tasks.workunit.client.0.smithi049.stdout: 1 842375 9.87 MB/sec execute 309 sec latency 21.379 ms 2024-03-18T23:16:08.778 INFO:tasks.workunit.client.0.smithi049.stdout: 1 844387 9.87 MB/sec execute 310 sec latency 27.816 ms 2024-03-18T23:16:08.814 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:16:08 smithi139 ceph-mon[67282]: pgmap v244: 97 pgs: 97 active+clean; 69 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.2 MiB/s wr, 313 op/s 2024-03-18T23:16:08.879 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:16:08 smithi049 ceph-mon[62991]: pgmap v244: 97 pgs: 97 active+clean; 69 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.2 MiB/s wr, 313 op/s 2024-03-18T23:16:08.960 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:16:08 smithi175 ceph-mon[67352]: pgmap v244: 97 pgs: 97 active+clean; 69 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.2 MiB/s wr, 313 op/s 2024-03-18T23:16:09.778 INFO:tasks.workunit.client.0.smithi049.stdout: 1 845393 9.86 MB/sec execute 311 sec latency 105.251 ms 2024-03-18T23:16:10.379 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:16:10 smithi049 ceph-mon[62991]: osdmap e95: 12 total, 12 up, 12 in 2024-03-18T23:16:10.460 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:16:10 smithi175 ceph-mon[67352]: osdmap e95: 12 total, 12 up, 12 in 2024-03-18T23:16:10.564 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:16:10 smithi139 ceph-mon[67282]: osdmap e95: 12 total, 12 up, 12 in 2024-03-18T23:16:10.778 INFO:tasks.workunit.client.0.smithi049.stdout: 1 846846 9.85 MB/sec execute 312 sec latency 35.349 ms 2024-03-18T23:16:11.361 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:16:11 smithi175 ceph-mon[67352]: pgmap v246: 97 pgs: 97 active+clean; 69 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.9 MiB/s wr, 381 op/s 2024-03-18T23:16:11.379 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:16:11 smithi049 ceph-mon[62991]: pgmap v246: 97 pgs: 97 active+clean; 69 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.9 MiB/s wr, 381 op/s 2024-03-18T23:16:11.564 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:16:11 smithi139 ceph-mon[67282]: pgmap v246: 97 pgs: 97 active+clean; 69 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.9 MiB/s wr, 381 op/s 2024-03-18T23:16:11.778 INFO:tasks.workunit.client.0.smithi049.stdout: 1 848682 9.86 MB/sec execute 313 sec latency 22.900 ms 2024-03-18T23:16:12.778 INFO:tasks.workunit.client.0.smithi049.stdout: 1 850069 9.85 MB/sec execute 314 sec latency 20.413 ms 2024-03-18T23:16:13.379 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:16:13 smithi049 ceph-mon[62991]: pgmap v247: 97 pgs: 97 active+clean; 75 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.0 MiB/s wr, 271 op/s 2024-03-18T23:16:13.460 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:16:13 smithi175 ceph-mon[67352]: pgmap v247: 97 pgs: 97 active+clean; 75 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.0 MiB/s wr, 271 op/s 2024-03-18T23:16:13.564 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:16:13 smithi139 ceph-mon[67282]: pgmap v247: 97 pgs: 97 active+clean; 75 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.0 MiB/s wr, 271 op/s 2024-03-18T23:16:13.778 INFO:tasks.workunit.client.0.smithi049.stdout: 1 852116 9.85 MB/sec execute 315 sec latency 28.887 ms 2024-03-18T23:16:14.778 INFO:tasks.workunit.client.0.smithi049.stdout: 1 852777 9.83 MB/sec execute 316 sec latency 48.947 ms 2024-03-18T23:16:15.379 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:16:15 smithi049 ceph-mon[62991]: pgmap v248: 97 pgs: 97 active+clean; 76 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.0 MiB/s wr, 271 op/s 2024-03-18T23:16:15.460 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:16:15 smithi175 ceph-mon[67352]: pgmap v248: 97 pgs: 97 active+clean; 76 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.0 MiB/s wr, 271 op/s 2024-03-18T23:16:15.564 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:16:15 smithi139 ceph-mon[67282]: pgmap v248: 97 pgs: 97 active+clean; 76 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.0 MiB/s wr, 271 op/s 2024-03-18T23:16:15.778 INFO:tasks.workunit.client.0.smithi049.stdout: 1 854519 9.82 MB/sec execute 317 sec latency 56.377 ms 2024-03-18T23:16:16.643 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:16.641+0000 7f8996044640 1 -- 172.21.15.49:0/3810519637 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f89901017d0 msgr2=0x7f8990101c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:16:16.643 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:16.641+0000 7f8996044640 1 --2- 172.21.15.49:0/3810519637 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f89901017d0 0x7f8990101c50 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f8984009930 tx=0x7f898402bc30 comp rx=0 tx=0).stop 2024-03-18T23:16:16.643 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:16.641+0000 7f8996044640 1 -- 172.21.15.49:0/3810519637 shutdown_connections 2024-03-18T23:16:16.643 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:16.641+0000 7f8996044640 1 --2- 172.21.15.49:0/3810519637 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8990102190 0x7f899010e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:16.643 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:16.641+0000 7f8996044640 1 --2- 172.21.15.49:0/3810519637 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f89901017d0 0x7f8990101c50 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:16.643 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:16.641+0000 7f8996044640 1 --2- 172.21.15.49:0/3810519637 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f89901077e0 0x7f8990107bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:16.644 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:16.641+0000 7f8996044640 1 -- 172.21.15.49:0/3810519637 >> 172.21.15.49:0/3810519637 conn(0x7f89900fd2e0 msgr2=0x7f89900ff700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:16:16.644 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:16.641+0000 7f8996044640 1 -- 172.21.15.49:0/3810519637 shutdown_connections 2024-03-18T23:16:16.644 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:16.641+0000 7f8996044640 1 -- 172.21.15.49:0/3810519637 wait complete. 2024-03-18T23:16:16.644 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:16.642+0000 7f8996044640 1 Processor -- start 2024-03-18T23:16:16.644 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:16.642+0000 7f8996044640 1 -- start start 2024-03-18T23:16:16.644 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:16.642+0000 7f8996044640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f89901017d0 0x7f899019a260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:16:16.644 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:16.642+0000 7f8996044640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8990102190 0x7f899019a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:16:16.644 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:16.642+0000 7f8996044640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f89901077e0 0x7f899019eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:16:16.644 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:16.642+0000 7f8996044640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f8990075710 con 0x7f89901017d0 2024-03-18T23:16:16.644 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:16.642+0000 7f8996044640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f8990075590 con 0x7f8990102190 2024-03-18T23:16:16.644 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:16.642+0000 7f8996044640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f8990075890 con 0x7f89901077e0 2024-03-18T23:16:16.644 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:16.642+0000 7f898f7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f89901017d0 0x7f899019a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:16:16.645 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:16.642+0000 7f898f7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f89901017d0 0x7f899019a260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:39138/0 (socket says 172.21.15.49:39138) 2024-03-18T23:16:16.645 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:16.642+0000 7f898f7fe640 1 -- 172.21.15.49:0/2197873861 learned_addr learned my addr 172.21.15.49:0/2197873861 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:16:16.645 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:16.642+0000 7f898effd640 1 --2- 172.21.15.49:0/2197873861 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8990102190 0x7f899019a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:16:16.645 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:16.642+0000 7f898ffff640 1 --2- 172.21.15.49:0/2197873861 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f89901077e0 0x7f899019eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:16:16.645 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:16.643+0000 7f898f7fe640 1 -- 172.21.15.49:0/2197873861 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f89901077e0 msgr2=0x7f899019eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:16:16.645 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:16.643+0000 7f898f7fe640 1 --2- 172.21.15.49:0/2197873861 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f89901077e0 0x7f899019eb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:16.645 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:16.643+0000 7f898f7fe640 1 -- 172.21.15.49:0/2197873861 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8990102190 msgr2=0x7f899019a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:16:16.645 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:16.643+0000 7f898f7fe640 1 --2- 172.21.15.49:0/2197873861 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8990102190 0x7f899019a7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:16.645 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:16.643+0000 7f898f7fe640 1 -- 172.21.15.49:0/2197873861 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f899019f250 con 0x7f89901017d0 2024-03-18T23:16:16.645 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:16.643+0000 7f898f7fe640 1 --2- 172.21.15.49:0/2197873861 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f89901017d0 0x7f899019a260 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f898000e9f0 tx=0x7f898000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:16:16.645 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:16.643+0000 7f898cff9640 1 -- 172.21.15.49:0/2197873861 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f898000cd80 con 0x7f89901017d0 2024-03-18T23:16:16.645 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:16.643+0000 7f898cff9640 1 -- 172.21.15.49:0/2197873861 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8980010910 con 0x7f89901017d0 2024-03-18T23:16:16.645 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:16.643+0000 7f898cff9640 1 -- 172.21.15.49:0/2197873861 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8980018990 con 0x7f89901017d0 2024-03-18T23:16:16.646 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:16.643+0000 7f8996044640 1 -- 172.21.15.49:0/2197873861 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f899019f540 con 0x7f89901017d0 2024-03-18T23:16:16.646 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:16.643+0000 7f8996044640 1 -- 172.21.15.49:0/2197873861 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89901a6dd0 con 0x7f89901017d0 2024-03-18T23:16:16.654 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:16.645+0000 7f898cff9640 1 -- 172.21.15.49:0/2197873861 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 25) ==== 137421+0+0 (secure 0 0 0) 0x7f8980010ac0 con 0x7f89901017d0 2024-03-18T23:16:16.654 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:16.645+0000 7f8998c2b740 1 Processor -- start 2024-03-18T23:16:16.654 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:16.646+0000 7f8998c2b740 1 -- start start 2024-03-18T23:16:16.654 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:16.646+0000 7f898cff9640 1 --2- 172.21.15.49:0/2197873861 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f89780a2a90 0x7f89780a4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:16:16.655 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:16.646+0000 7f8998c2b740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e6937d3ef0 0x55e6937d42d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:16:16.655 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:16.646+0000 7f8998c2b740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e6937d4810 0x55e6937dd0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:16:16.655 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:16.646+0000 7f8998c2b740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e6937dd5e0 0x55e6937df9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:16:16.655 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:16.646+0000 7f8998c2b740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55e693737be0 con 0x55e6937dd5e0 2024-03-18T23:16:16.655 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:16.646+0000 7f8998c2b740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55e6936e0960 con 0x55e6937d4810 2024-03-18T23:16:16.655 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:16.646+0000 7f8998c2b740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55e693733c80 con 0x55e6937d3ef0 2024-03-18T23:16:16.655 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:16.646+0000 7f898ffff640 1 --2- 172.21.15.49:0/2197873861 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f89780a2a90 0x7f89780a4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:16:16.655 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:16.646+0000 7f898cff9640 1 -- 172.21.15.49:0/2197873861 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8556+0+0 (secure 0 0 0) 0x7f89800cf090 con 0x7f89901017d0 2024-03-18T23:16:16.656 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:16.646+0000 7f898f7fe640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e6937d3ef0 0x55e6937d42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:16:16.656 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:16.646+0000 7f898effd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e6937d4810 0x55e6937dd0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:16:16.656 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:16.646+0000 7f898f7fe640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e6937d3ef0 0x55e6937d42d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.49:43222/0 (socket says 172.21.15.49:43222) 2024-03-18T23:16:16.656 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:16.646+0000 7f898f7fe640 1 -- 172.21.15.49:0/1906313483 learned_addr learned my addr 172.21.15.49:0/1906313483 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:16:16.656 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:16.646+0000 7f898effd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e6937d4810 0x55e6937dd0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.49:48888/0 (socket says 172.21.15.49:48888) 2024-03-18T23:16:16.656 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:16.646+0000 7f898ffff640 1 --2- 172.21.15.49:0/1906313483 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e6937dd5e0 0x55e6937df9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:16:16.656 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:16.646+0000 7f898f7fe640 1 -- 172.21.15.49:0/1906313483 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e6937d4810 msgr2=0x55e6937dd0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:16:16.656 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:16.646+0000 7f898f7fe640 1 --2- 172.21.15.49:0/1906313483 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e6937d4810 0x55e6937dd0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:16.656 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:16.646+0000 7f898f7fe640 1 -- 172.21.15.49:0/1906313483 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e6937dd5e0 msgr2=0x55e6937df9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:16:16.656 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:16.646+0000 7f898f7fe640 1 --2- 172.21.15.49:0/1906313483 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e6937dd5e0 0x55e6937df9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:16.656 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:16.646+0000 7f898f7fe640 1 -- 172.21.15.49:0/1906313483 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e6936d7d20 con 0x55e6937d3ef0 2024-03-18T23:16:16.656 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:16.646+0000 7f898ffff640 1 --2- 172.21.15.49:0/2197873861 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f89780a2a90 0x7f89780a4f50 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f8988005fd0 tx=0x7f8988009f10 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:16:16.656 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:16.647+0000 7f898f7fe640 1 --2- 172.21.15.49:0/1906313483 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e6937d3ef0 0x55e6937d42d0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f89800cc820 tx=0x7f8980097b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=89659746b3cc3a3 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:16:16.656 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:16.647+0000 7f8975ffb640 1 -- 172.21.15.49:0/1906313483 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f898009f820 con 0x55e6937d3ef0 2024-03-18T23:16:16.657 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:16.647+0000 7f8975ffb640 1 -- 172.21.15.49:0/1906313483 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f89800a3e70 con 0x55e6937d3ef0 2024-03-18T23:16:16.657 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:16.647+0000 7f8975ffb640 1 -- 172.21.15.49:0/1906313483 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89800bbcd0 con 0x55e6937d3ef0 2024-03-18T23:16:16.657 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:16.647+0000 7f8998c2b740 1 -- 172.21.15.49:0/1906313483 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e6937d3ef0 msgr2=0x55e6937d42d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:16:16.657 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:16.647+0000 7f8998c2b740 1 --2- 172.21.15.49:0/1906313483 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e6937d3ef0 0x55e6937d42d0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f89800cc820 tx=0x7f8980097b50 comp rx=0 tx=0).stop 2024-03-18T23:16:16.657 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:16.647+0000 7f8998c2b740 1 -- 172.21.15.49:0/1906313483 shutdown_connections 2024-03-18T23:16:16.657 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:16.647+0000 7f8998c2b740 1 --2- 172.21.15.49:0/1906313483 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e6937dd5e0 0x55e6937df9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:16.657 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:16.647+0000 7f8998c2b740 1 --2- 172.21.15.49:0/1906313483 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e6937d4810 0x55e6937dd0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:16.657 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:16.647+0000 7f8998c2b740 1 --2- 172.21.15.49:0/1906313483 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e6937d3ef0 0x55e6937d42d0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:16.657 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:16.647+0000 7f8998c2b740 1 -- 172.21.15.49:0/1906313483 >> 172.21.15.49:0/1906313483 conn(0x55e6937ce810 msgr2=0x55e6937cebf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:16:16.657 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:16.647+0000 7f8998c2b740 1 -- 172.21.15.49:0/1906313483 shutdown_connections 2024-03-18T23:16:16.657 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:16.648+0000 7f8998c2b740 1 -- 172.21.15.49:0/1906313483 wait complete. 2024-03-18T23:16:16.658 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:16.648+0000 7f8998c2b740 1 -- start start 2024-03-18T23:16:16.658 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:16.648+0000 7f8998c2b740 1 Processor -- start 2024-03-18T23:16:16.658 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:16.648+0000 7f8998c2b740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e6937d3ef0 0x55e6937dcfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:16:16.658 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:16.648+0000 7f8998c2b740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e6937d4810 0x55e6937da340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:16:16.658 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:16.648+0000 7f8998c2b740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e6937dd5e0 0x55e6937da950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:16:16.658 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:16.648+0000 7f8998c2b740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55e6937e0150 con 0x55e6937d4810 2024-03-18T23:16:16.658 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:16.648+0000 7f8998c2b740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55e6936f3db0 con 0x55e6937d3ef0 2024-03-18T23:16:16.658 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:16.648+0000 7f8998c2b740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55e6937e02d0 con 0x55e6937dd5e0 2024-03-18T23:16:16.658 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:16.648+0000 7f8998c2b740 10 client.0 _ll_register_callbacks cb 0x55e6936f8e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T23:16:16.658 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:16.648+0000 7f898effd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e6937d4810 0x55e6937da340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:16:16.658 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:16.648+0000 7f898f7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e6937d3ef0 0x55e6937dcfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:16:16.658 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:16.648+0000 7f898effd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e6937d4810 0x55e6937da340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:39158/0 (socket says 172.21.15.49:39158) 2024-03-18T23:16:16.659 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:16.648+0000 7f898effd640 1 -- 172.21.15.49:0/2561671324 learned_addr learned my addr 172.21.15.49:0/2561671324 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:16:16.659 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:16.648+0000 7f898ffff640 1 --2- 172.21.15.49:0/2561671324 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e6937dd5e0 0x55e6937da950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:16:16.659 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:16.649+0000 7f898effd640 1 -- 172.21.15.49:0/2561671324 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e6937dd5e0 msgr2=0x55e6937da950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:16:16.659 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:16.649+0000 7f898effd640 1 --2- 172.21.15.49:0/2561671324 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e6937dd5e0 0x55e6937da950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:16.659 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:16.649+0000 7f898effd640 1 -- 172.21.15.49:0/2561671324 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e6937d3ef0 msgr2=0x55e6937dcfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:16:16.659 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:16.649+0000 7f898effd640 1 --2- 172.21.15.49:0/2561671324 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e6937d3ef0 0x55e6937dcfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:16.659 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:16.649+0000 7f898effd640 1 -- 172.21.15.49:0/2561671324 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e6936f2c60 con 0x55e6937d4810 2024-03-18T23:16:16.659 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:16.649+0000 7f898effd640 1 --2- 172.21.15.49:0/2561671324 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e6937d4810 0x55e6937da340 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f898402cfd0 tx=0x7f898402cda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c816bbbbf8a6d613 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:16:16.659 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:16.649+0000 7f896effd640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-18T23:16:16.659 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:16.649+0000 7f896effd640 1 -- 172.21.15.49:0/2561671324 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8984004240 con 0x55e6937d4810 2024-03-18T23:16:16.659 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:16.649+0000 7f8998c2b740 1 -- 172.21.15.49:0/2561671324 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f89800a3e70 con 0x55e6937d4810 2024-03-18T23:16:16.659 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:16.649+0000 7f896effd640 1 -- 172.21.15.49:0/2561671324 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8556+0+0 (secure 0 0 0) 0x7f89840043e0 con 0x55e6937d4810 2024-03-18T23:16:16.659 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:16.649+0000 7f896effd640 1 -- 172.21.15.49:0/2561671324 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f89840046c0 con 0x55e6937d4810 2024-03-18T23:16:16.659 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:16.649+0000 7f8998c2b740 10 client.16101 fetch_fsmap learned FSMap version 32 2024-03-18T23:16:16.660 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:16.649+0000 7f8998c2b740 1 -- 172.21.15.49:0/2561671324 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55e69344c210 con 0x55e6937d4810 2024-03-18T23:16:16.660 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:16.649+0000 7f896effd640 1 -- 172.21.15.49:0/2561671324 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f89800a3e70 con 0x55e6937d4810 2024-03-18T23:16:16.660 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:16.650+0000 7f896effd640 1 -- 172.21.15.49:0/2561671324 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f89840035f0 con 0x55e6937d4810 2024-03-18T23:16:16.660 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:16.650+0000 7f8998c2b740 10 client.16101 fetch_fsmap finished waiting for FSMap version 32 2024-03-18T23:16:16.660 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:16.650+0000 7f8998c2b740 10 client.16101 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T23:16:16.660 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:16.650+0000 7f8998c2b740 20 client.16101 populate_metadata read hostname 'smithi049' 2024-03-18T23:16:16.660 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:16.650+0000 7f8998c2b740 1 --2- 172.21.15.49:0/2561671324 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x55e6937ed3d0 0x55e6937ef7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:16:16.660 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:16.650+0000 7f8998c2b740 4 client.16101 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T23:16:16.660 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:16.650+0000 7f8998c2b740 1 -- 172.21.15.49:0/2561671324 --> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e693745120 con 0x55e6937ed3d0 2024-03-18T23:16:16.660 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:16.650+0000 7f898f7fe640 1 --2- 172.21.15.49:0/2561671324 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x55e6937ed3d0 0x55e6937ef7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:16:16.660 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:16.650+0000 7f896effd640 10 client.16101 ms_handle_connect on v2:172.21.15.139:6838/4280128209 2024-03-18T23:16:16.660 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:16.650+0000 7f898f7fe640 1 --2- 172.21.15.49:0/2561671324 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x55e6937ed3d0 0x55e6937ef7e0 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=689984454b113e7b server_cookie=504c9ca0ea4d0281 in_seq=0 out_seq=0 2024-03-18T23:16:16.660 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:16.652+0000 7f896effd640 1 -- 172.21.15.49:0/2561671324 <== mds.0 v2:172.21.15.139:6838/4280128209 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55e693745120 con 0x55e6937ed3d0 2024-03-18T23:16:16.660 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:16.652+0000 7f896effd640 10 client.16101 handle_command_reply: tid=0 2024-03-18T23:16:16.661 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:16.652+0000 7f8998c2b740 1 client.16101 shutdown 2024-03-18T23:16:16.661 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:16.653+0000 7f8998c2b740 1 -- 172.21.15.49:0/2561671324 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e6937d4810 msgr2=0x55e6937da340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:16:16.661 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:16.653+0000 7f8998c2b740 1 --2- 172.21.15.49:0/2561671324 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e6937d4810 0x55e6937da340 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f898402cfd0 tx=0x7f898402cda0 comp rx=0 tx=0).stop 2024-03-18T23:16:16.661 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:16.653+0000 7f8998c2b740 1 -- 172.21.15.49:0/2561671324 shutdown_connections 2024-03-18T23:16:16.661 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:16.653+0000 7f8998c2b740 1 --2- 172.21.15.49:0/2561671324 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x55e6937ed3d0 0x55e6937ef7e0 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:16.661 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:16.653+0000 7f8998c2b740 1 --2- 172.21.15.49:0/2561671324 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e6937dd5e0 0x55e6937da950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:16.661 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:16.653+0000 7f8998c2b740 1 --2- 172.21.15.49:0/2561671324 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e6937d4810 0x55e6937da340 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:16.661 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:16.653+0000 7f896effd640 0 client.16101 ms_handle_reset on v2:172.21.15.139:6838/4280128209 2024-03-18T23:16:16.661 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:16.653+0000 7f8998c2b740 1 --2- 172.21.15.49:0/2561671324 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e6937d3ef0 0x55e6937dcfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:16.661 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:16.653+0000 7f8998c2b740 1 -- 172.21.15.49:0/2561671324 >> 172.21.15.49:0/2561671324 conn(0x55e6937ce810 msgr2=0x55e6937cdab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:16:16.661 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:16.653+0000 7f8998c2b740 1 -- 172.21.15.49:0/2561671324 shutdown_connections 2024-03-18T23:16:16.661 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:16.653+0000 7f8998c2b740 1 -- 172.21.15.49:0/2561671324 wait complete. 2024-03-18T23:16:16.661 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:16.653+0000 7f8998c2b740 20 client.16101 trim_cache size 0 max 16384 2024-03-18T23:16:16.672 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:16.668+0000 7f8998c2b740 1 Processor -- start 2024-03-18T23:16:16.672 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:16.668+0000 7f8998c2b740 1 -- start start 2024-03-18T23:16:16.672 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:16.668+0000 7f8998c2b740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e6937d3ef0 0x55e6937e8ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:16:16.672 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:16.668+0000 7f8998c2b740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e6937d4810 0x55e6937f50f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:16:16.672 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:16.668+0000 7f8998c2b740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e6937dd5e0 0x55e6937dca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:16:16.672 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:16.668+0000 7f8998c2b740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f89901af780 con 0x55e6937d3ef0 2024-03-18T23:16:16.672 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:16.668+0000 7f8998c2b740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f89901b12c0 con 0x55e6937dd5e0 2024-03-18T23:16:16.672 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:16.668+0000 7f8998c2b740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f89901afe50 con 0x55e6937d4810 2024-03-18T23:16:16.672 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:16.668+0000 7f898f7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e6937d3ef0 0x55e6937e8ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:16:16.672 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:16.668+0000 7f898f7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e6937d3ef0 0x55e6937e8ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:39166/0 (socket says 172.21.15.49:39166) 2024-03-18T23:16:16.672 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:16.668+0000 7f898effd640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e6937d4810 0x55e6937f50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:16:16.672 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:16.668+0000 7f898f7fe640 1 -- 172.21.15.49:0/3950425240 learned_addr learned my addr 172.21.15.49:0/3950425240 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:16:16.673 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:16.668+0000 7f898ffff640 1 --2- 172.21.15.49:0/3950425240 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e6937dd5e0 0x55e6937dca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:16:16.673 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:16.668+0000 7f898effd640 1 -- 172.21.15.49:0/3950425240 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e6937dd5e0 msgr2=0x55e6937dca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:16:16.673 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:16.668+0000 7f898effd640 1 --2- 172.21.15.49:0/3950425240 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e6937dd5e0 0x55e6937dca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:16.673 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:16.668+0000 7f898effd640 1 -- 172.21.15.49:0/3950425240 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e6937d3ef0 msgr2=0x55e6937e8ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:16:16.673 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:16.668+0000 7f898effd640 1 --2- 172.21.15.49:0/3950425240 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e6937d3ef0 0x55e6937e8ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:16.673 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:16.668+0000 7f898effd640 1 -- 172.21.15.49:0/3950425240 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e693518620 con 0x55e6937d4810 2024-03-18T23:16:16.673 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:16.668+0000 7f898effd640 1 --2- 172.21.15.49:0/3950425240 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e6937d4810 0x55e6937f50f0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f8984003c00 tx=0x7f898402bc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e4a260e82e8a77b8 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:16:16.673 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:16.669+0000 7f89757fa640 1 -- 172.21.15.49:0/3950425240 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f898405d070 con 0x55e6937d4810 2024-03-18T23:16:16.673 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:16.669+0000 7f89757fa640 1 -- 172.21.15.49:0/3950425240 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8984004d40 con 0x55e6937d4810 2024-03-18T23:16:16.673 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:16.669+0000 7f89757fa640 1 -- 172.21.15.49:0/3950425240 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89840052d0 con 0x55e6937d4810 2024-03-18T23:16:16.673 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:16.669+0000 7f8998c2b740 1 -- 172.21.15.49:0/3950425240 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e6937d4810 msgr2=0x55e6937f50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:16:16.674 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:16.669+0000 7f8998c2b740 1 --2- 172.21.15.49:0/3950425240 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e6937d4810 0x55e6937f50f0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f8984003c00 tx=0x7f898402bc30 comp rx=0 tx=0).stop 2024-03-18T23:16:16.674 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:16.669+0000 7f8998c2b740 1 -- 172.21.15.49:0/3950425240 shutdown_connections 2024-03-18T23:16:16.674 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:16.669+0000 7f8998c2b740 1 --2- 172.21.15.49:0/3950425240 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e6937dd5e0 0x55e6937dca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:16.674 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:16.669+0000 7f8998c2b740 1 --2- 172.21.15.49:0/3950425240 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e6937d4810 0x55e6937f50f0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:16.674 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:16.669+0000 7f8998c2b740 1 --2- 172.21.15.49:0/3950425240 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e6937d3ef0 0x55e6937e8ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:16.674 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:16.669+0000 7f8998c2b740 1 -- 172.21.15.49:0/3950425240 >> 172.21.15.49:0/3950425240 conn(0x55e6937ce810 msgr2=0x55e6937cebf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:16:16.674 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:16.669+0000 7f8998c2b740 1 -- 172.21.15.49:0/3950425240 shutdown_connections 2024-03-18T23:16:16.674 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:16.669+0000 7f8998c2b740 1 -- 172.21.15.49:0/3950425240 wait complete. 2024-03-18T23:16:16.674 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:16.670+0000 7f8998c2b740 1 -- start start 2024-03-18T23:16:16.674 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:16.670+0000 7f8998c2b740 1 Processor -- start 2024-03-18T23:16:16.674 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:16.670+0000 7f8998c2b740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e6937d3ef0 0x55e6937e1120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:16:16.675 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:16.670+0000 7f8998c2b740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e6937d4810 0x55e6937e1660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:16:16.675 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:16.670+0000 7f8998c2b740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e6937dd5e0 0x55e693852720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:16:16.675 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:16.670+0000 7f8998c2b740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f89901b0bf0 con 0x55e6937d4810 2024-03-18T23:16:16.675 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:16.670+0000 7f8998c2b740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f89901b0520 con 0x55e6937dd5e0 2024-03-18T23:16:16.675 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:16.670+0000 7f8998c2b740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f89901ad780 con 0x55e6937d3ef0 2024-03-18T23:16:16.675 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:16.670+0000 7f8998c2b740 10 client.0 _ll_register_callbacks cb 0x7f8984009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T23:16:16.675 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:16.670+0000 7f898effd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e6937d4810 0x55e6937e1660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:16:16.675 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:16.670+0000 7f898effd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e6937d4810 0x55e6937e1660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:39176/0 (socket says 172.21.15.49:39176) 2024-03-18T23:16:16.675 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:16.670+0000 7f898effd640 1 -- 172.21.15.49:0/2599158736 learned_addr learned my addr 172.21.15.49:0/2599158736 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:16:16.675 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:16.670+0000 7f898f7fe640 1 --2- 172.21.15.49:0/2599158736 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e6937d3ef0 0x55e6937e1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:16:16.676 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:16.670+0000 7f898ffff640 1 --2- 172.21.15.49:0/2599158736 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e6937dd5e0 0x55e693852720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:16:16.676 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:16.671+0000 7f898effd640 1 -- 172.21.15.49:0/2599158736 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e6937d3ef0 msgr2=0x55e6937e1120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:16:16.676 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:16.671+0000 7f898effd640 1 --2- 172.21.15.49:0/2599158736 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e6937d3ef0 0x55e6937e1120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:16.676 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:16.671+0000 7f898effd640 1 -- 172.21.15.49:0/2599158736 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e6937dd5e0 msgr2=0x55e693852720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:16:16.676 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:16.671+0000 7f898effd640 1 --2- 172.21.15.49:0/2599158736 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e6937dd5e0 0x55e693852720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:16.676 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:16.671+0000 7f898effd640 1 -- 172.21.15.49:0/2599158736 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e6937e1ef0 con 0x55e6937d4810 2024-03-18T23:16:16.677 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:16.671+0000 7f898f7fe640 1 --2- 172.21.15.49:0/2599158736 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e6937d3ef0 0x55e6937e1120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:16:16.677 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:16.671+0000 7f898effd640 1 --2- 172.21.15.49:0/2599158736 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e6937d4810 0x55e6937e1660 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f898402c820 tx=0x7f8984009300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1706bc261a3a2e41 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:16:16.677 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:16.671+0000 7f896effd640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-18T23:16:16.677 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:16.671+0000 7f896effd640 1 -- 172.21.15.49:0/2599158736 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f898405d070 con 0x55e6937d4810 2024-03-18T23:16:16.677 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:16.671+0000 7f8998c2b740 1 -- 172.21.15.49:0/2599158736 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8984004d40 con 0x55e6937d4810 2024-03-18T23:16:16.677 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:16.671+0000 7f896effd640 1 -- 172.21.15.49:0/2599158736 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8556+0+0 (secure 0 0 0) 0x7f8984004860 con 0x55e6937d4810 2024-03-18T23:16:16.677 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:16.671+0000 7f896effd640 1 -- 172.21.15.49:0/2599158736 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f8984069050 con 0x55e6937d4810 2024-03-18T23:16:16.677 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:16.671+0000 7f8998c2b740 10 client.16107 fetch_fsmap learned FSMap version 32 2024-03-18T23:16:16.677 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:16.671+0000 7f8998c2b740 1 -- 172.21.15.49:0/2599158736 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55e693853d40 con 0x55e6937d4810 2024-03-18T23:16:16.677 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:16.671+0000 7f896effd640 1 -- 172.21.15.49:0/2599158736 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8984004d40 con 0x55e6937d4810 2024-03-18T23:16:16.677 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:16.672+0000 7f896effd640 1 -- 172.21.15.49:0/2599158736 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f8984003f20 con 0x55e6937d4810 2024-03-18T23:16:16.677 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:16.672+0000 7f8998c2b740 10 client.16107 fetch_fsmap finished waiting for FSMap version 32 2024-03-18T23:16:16.677 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:16.672+0000 7f8998c2b740 10 client.16107 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T23:16:16.677 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:16.672+0000 7f8998c2b740 20 client.16107 populate_metadata read hostname 'smithi049' 2024-03-18T23:16:16.678 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:16.672+0000 7f8998c2b740 1 --2- 172.21.15.49:0/2599158736 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x55e6938543a0 0x55e6938567c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:16:16.678 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:16.672+0000 7f8998c2b740 4 client.16107 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T23:16:16.678 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:16.672+0000 7f8998c2b740 1 -- 172.21.15.49:0/2599158736 --> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e69373f470 con 0x55e6938543a0 2024-03-18T23:16:16.678 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:16.672+0000 7f898f7fe640 1 --2- 172.21.15.49:0/2599158736 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x55e6938543a0 0x55e6938567c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:16:16.678 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:16.672+0000 7f898f7fe640 1 --2- 172.21.15.49:0/2599158736 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x55e6938543a0 0x55e6938567c0 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6b631e244314d8cd server_cookie=9f5ff997ae795cf6 in_seq=0 out_seq=0 2024-03-18T23:16:16.678 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:16.672+0000 7f896effd640 10 client.16107 ms_handle_connect on v2:172.21.15.139:6838/4280128209 2024-03-18T23:16:16.778 INFO:tasks.workunit.client.0.smithi049.stdout: 1 855813 9.82 MB/sec execute 318 sec latency 77.235 ms 2024-03-18T23:16:16.826 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:16.826+0000 7f896effd640 1 -- 172.21.15.49:0/2599158736 <== mds.0 v2:172.21.15.139:6838/4280128209 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e69373f470 con 0x55e6938543a0 2024-03-18T23:16:16.826 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:16.826+0000 7f896effd640 10 client.16107 handle_command_reply: tid=0 2024-03-18T23:16:16.827 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:16.826+0000 7f8998c2b740 1 client.16107 shutdown 2024-03-18T23:16:16.827 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:16.826+0000 7f8998c2b740 1 -- 172.21.15.49:0/2599158736 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e6937d4810 msgr2=0x55e6937e1660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:16:16.827 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:16.826+0000 7f8998c2b740 1 --2- 172.21.15.49:0/2599158736 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e6937d4810 0x55e6937e1660 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f898402c820 tx=0x7f8984009300 comp rx=0 tx=0).stop 2024-03-18T23:16:16.827 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:16.826+0000 7f8998c2b740 1 -- 172.21.15.49:0/2599158736 shutdown_connections 2024-03-18T23:16:16.827 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:16.826+0000 7f8998c2b740 1 --2- 172.21.15.49:0/2599158736 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x55e6938543a0 0x55e6938567c0 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:16.827 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:16.826+0000 7f8998c2b740 1 --2- 172.21.15.49:0/2599158736 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e6937dd5e0 0x55e693852720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:16.827 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:16.826+0000 7f896effd640 0 client.16107 ms_handle_reset on v2:172.21.15.139:6838/4280128209 2024-03-18T23:16:16.827 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:16.826+0000 7f8998c2b740 1 --2- 172.21.15.49:0/2599158736 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e6937d4810 0x55e6937e1660 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:16.827 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:16.826+0000 7f8998c2b740 1 --2- 172.21.15.49:0/2599158736 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e6937d3ef0 0x55e6937e1120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:16.827 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:16.826+0000 7f8998c2b740 1 -- 172.21.15.49:0/2599158736 >> 172.21.15.49:0/2599158736 conn(0x55e6937ce810 msgr2=0x55e6937cfc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:16:16.827 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:16.827+0000 7f8998c2b740 1 -- 172.21.15.49:0/2599158736 shutdown_connections 2024-03-18T23:16:16.827 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:16.827+0000 7f8998c2b740 1 -- 172.21.15.49:0/2599158736 wait complete. 2024-03-18T23:16:16.827 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:16.827+0000 7f8998c2b740 20 client.16107 trim_cache size 0 max 16384 2024-03-18T23:16:16.828 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:{ 2024-03-18T23:16:16.828 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "message": "", 2024-03-18T23:16:16.828 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "return_code": 0 2024-03-18T23:16:16.828 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:} 2024-03-18T23:16:16.829 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:16.827+0000 7f8996044640 1 -- 172.21.15.49:0/2197873861 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f89780a2a90 msgr2=0x7f89780a4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:16:16.829 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:16.827+0000 7f8996044640 1 --2- 172.21.15.49:0/2197873861 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f89780a2a90 0x7f89780a4f50 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f8988005fd0 tx=0x7f8988009f10 comp rx=0 tx=0).stop 2024-03-18T23:16:16.829 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:16.827+0000 7f8996044640 1 -- 172.21.15.49:0/2197873861 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f89901017d0 msgr2=0x7f899019a260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:16:16.829 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:16.827+0000 7f8996044640 1 --2- 172.21.15.49:0/2197873861 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f89901017d0 0x7f899019a260 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f898000e9f0 tx=0x7f898000eeb0 comp rx=0 tx=0).stop 2024-03-18T23:16:16.829 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:16.827+0000 7f8996044640 1 -- 172.21.15.49:0/2197873861 shutdown_connections 2024-03-18T23:16:16.829 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:16.827+0000 7f8996044640 1 --2- 172.21.15.49:0/2197873861 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f89780a2a90 0x7f89780a4f50 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:16.829 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:16.827+0000 7f8996044640 1 --2- 172.21.15.49:0/2197873861 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f89901077e0 0x7f899019eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:16.829 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:16.827+0000 7f8996044640 1 --2- 172.21.15.49:0/2197873861 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8990102190 0x7f899019a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:16.829 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:16.827+0000 7f8996044640 1 --2- 172.21.15.49:0/2197873861 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f89901017d0 0x7f899019a260 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:16.829 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:16.827+0000 7f8996044640 1 -- 172.21.15.49:0/2197873861 >> 172.21.15.49:0/2197873861 conn(0x7f89900fd2e0 msgr2=0x7f89900fd800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:16:16.829 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:16.827+0000 7f8996044640 1 -- 172.21.15.49:0/2197873861 shutdown_connections 2024-03-18T23:16:16.830 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:16.828+0000 7f8996044640 1 -- 172.21.15.49:0/2197873861 wait complete. 2024-03-18T23:16:17.365 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:16:17 smithi139 ceph-mon[67282]: pgmap v249: 97 pgs: 97 active+clean; 76 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.0 MiB/s wr, 271 op/s 2024-03-18T23:16:17.379 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:16:17 smithi049 ceph-mon[62991]: pgmap v249: 97 pgs: 97 active+clean; 76 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.0 MiB/s wr, 271 op/s 2024-03-18T23:16:17.460 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:16:17 smithi175 ceph-mon[67352]: pgmap v249: 97 pgs: 97 active+clean; 76 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.0 MiB/s wr, 271 op/s 2024-03-18T23:16:17.778 INFO:tasks.workunit.client.0.smithi049.stdout: 1 857162 9.81 MB/sec execute 319 sec latency 92.244 ms 2024-03-18T23:16:18.778 INFO:tasks.workunit.client.0.smithi049.stdout: 1 859209 9.82 MB/sec execute 320 sec latency 26.125 ms 2024-03-18T23:16:19.379 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:16:19 smithi049 ceph-mon[62991]: pgmap v250: 97 pgs: 97 active+clean; 81 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.0 MiB/s wr, 190 op/s 2024-03-18T23:16:19.460 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:16:19 smithi175 ceph-mon[67352]: pgmap v250: 97 pgs: 97 active+clean; 81 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.0 MiB/s wr, 190 op/s 2024-03-18T23:16:19.564 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:16:19 smithi139 ceph-mon[67282]: pgmap v250: 97 pgs: 97 active+clean; 81 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.0 MiB/s wr, 190 op/s 2024-03-18T23:16:19.778 INFO:tasks.workunit.client.0.smithi049.stdout: 1 859924 9.80 MB/sec execute 321 sec latency 92.973 ms 2024-03-18T23:16:20.778 INFO:tasks.workunit.client.0.smithi049.stdout: 1 861959 9.80 MB/sec execute 322 sec latency 18.392 ms 2024-03-18T23:16:21.379 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:16:21 smithi049 ceph-mon[62991]: pgmap v251: 97 pgs: 97 active+clean; 81 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.9 MiB/s wr, 184 op/s 2024-03-18T23:16:21.460 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:16:21 smithi175 ceph-mon[67352]: pgmap v251: 97 pgs: 97 active+clean; 81 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.9 MiB/s wr, 184 op/s 2024-03-18T23:16:21.564 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:16:21 smithi139 ceph-mon[67282]: pgmap v251: 97 pgs: 97 active+clean; 81 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.9 MiB/s wr, 184 op/s 2024-03-18T23:16:21.778 INFO:tasks.workunit.client.0.smithi049.stdout: 1 863127 9.79 MB/sec execute 323 sec latency 32.477 ms 2024-03-18T23:16:22.778 INFO:tasks.workunit.client.0.smithi049.stdout: 1 864695 9.78 MB/sec execute 324 sec latency 26.884 ms 2024-03-18T23:16:23.379 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:16:23 smithi049 ceph-mon[62991]: pgmap v252: 97 pgs: 97 active+clean; 70 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.3 MiB/s wr, 272 op/s 2024-03-18T23:16:23.460 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:16:23 smithi175 ceph-mon[67352]: pgmap v252: 97 pgs: 97 active+clean; 70 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.3 MiB/s wr, 272 op/s 2024-03-18T23:16:23.564 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:16:23 smithi139 ceph-mon[67282]: pgmap v252: 97 pgs: 97 active+clean; 70 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.3 MiB/s wr, 272 op/s 2024-03-18T23:16:23.779 INFO:tasks.workunit.client.0.smithi049.stdout: 1 866430 9.79 MB/sec execute 325 sec latency 18.356 ms 2024-03-18T23:16:24.779 INFO:tasks.workunit.client.0.smithi049.stdout: 1 867703 9.78 MB/sec execute 326 sec latency 80.394 ms 2024-03-18T23:16:25.379 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:16:25 smithi049 ceph-mon[62991]: pgmap v253: 97 pgs: 97 active+clean; 73 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.4 MiB/s wr, 226 op/s 2024-03-18T23:16:25.460 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:16:25 smithi175 ceph-mon[67352]: pgmap v253: 97 pgs: 97 active+clean; 73 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.4 MiB/s wr, 226 op/s 2024-03-18T23:16:25.564 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:16:25 smithi139 ceph-mon[67282]: pgmap v253: 97 pgs: 97 active+clean; 73 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.4 MiB/s wr, 226 op/s 2024-03-18T23:16:25.779 INFO:tasks.workunit.client.0.smithi049.stdout: 1 869818 9.79 MB/sec execute 327 sec latency 32.216 ms 2024-03-18T23:16:26.779 INFO:tasks.workunit.client.0.smithi049.stdout: 1 870575 9.76 MB/sec execute 328 sec latency 24.549 ms 2024-03-18T23:16:27.379 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:16:27 smithi049 ceph-mon[62991]: pgmap v254: 97 pgs: 97 active+clean; 73 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.4 MiB/s wr, 224 op/s 2024-03-18T23:16:27.460 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:16:27 smithi175 ceph-mon[67352]: pgmap v254: 97 pgs: 97 active+clean; 73 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.4 MiB/s wr, 224 op/s 2024-03-18T23:16:27.564 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:16:27 smithi139 ceph-mon[67282]: pgmap v254: 97 pgs: 97 active+clean; 73 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.4 MiB/s wr, 224 op/s 2024-03-18T23:16:27.779 INFO:tasks.workunit.client.0.smithi049.stdout: 1 872733 9.77 MB/sec execute 329 sec latency 29.331 ms 2024-03-18T23:16:28.779 INFO:tasks.workunit.client.0.smithi049.stdout: 1 873791 9.76 MB/sec execute 330 sec latency 18.958 ms 2024-03-18T23:16:29.379 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:16:29 smithi049 ceph-mon[62991]: pgmap v255: 97 pgs: 97 active+clean; 78 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.6 MiB/s wr, 305 op/s 2024-03-18T23:16:29.379 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:16:29 smithi049 ceph-mon[62991]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:16:29.460 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:16:29 smithi175 ceph-mon[67352]: pgmap v255: 97 pgs: 97 active+clean; 78 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.6 MiB/s wr, 305 op/s 2024-03-18T23:16:29.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:16:29 smithi175 ceph-mon[67352]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:16:29.564 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:16:29 smithi139 ceph-mon[67282]: pgmap v255: 97 pgs: 97 active+clean; 78 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.6 MiB/s wr, 305 op/s 2024-03-18T23:16:29.564 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:16:29 smithi139 ceph-mon[67282]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:16:29.779 INFO:tasks.workunit.client.0.smithi049.stdout: 1 875088 9.75 MB/sec execute 331 sec latency 69.057 ms 2024-03-18T23:16:29.922 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:29.919+0000 7f0c41d3b640 1 -- 172.21.15.49:0/488765833 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0c3c100970 msgr2=0x7f0c3c100df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:16:29.922 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:29.919+0000 7f0c41d3b640 1 --2- 172.21.15.49:0/488765833 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0c3c100970 0x7f0c3c100df0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f0c30009900 tx=0x7f0c3002bc30 comp rx=0 tx=0).stop 2024-03-18T23:16:29.922 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:29.919+0000 7f0c41d3b640 1 -- 172.21.15.49:0/488765833 shutdown_connections 2024-03-18T23:16:29.922 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:29.919+0000 7f0c41d3b640 1 --2- 172.21.15.49:0/488765833 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0c3c101330 0x7f0c3c10d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:29.922 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:29.919+0000 7f0c41d3b640 1 --2- 172.21.15.49:0/488765833 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0c3c100970 0x7f0c3c100df0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:29.922 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:29.919+0000 7f0c41d3b640 1 --2- 172.21.15.49:0/488765833 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0c3c1056c0 0x7f0c3c105aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:29.922 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:29.919+0000 7f0c41d3b640 1 -- 172.21.15.49:0/488765833 >> 172.21.15.49:0/488765833 conn(0x7f0c3c077570 msgr2=0x7f0c3c077980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:16:29.922 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:29.919+0000 7f0c41d3b640 1 -- 172.21.15.49:0/488765833 shutdown_connections 2024-03-18T23:16:29.923 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:29.919+0000 7f0c41d3b640 1 -- 172.21.15.49:0/488765833 wait complete. 2024-03-18T23:16:29.923 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:29.919+0000 7f0c41d3b640 1 Processor -- start 2024-03-18T23:16:29.923 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:29.920+0000 7f0c41d3b640 1 -- start start 2024-03-18T23:16:29.923 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:29.920+0000 7f0c41d3b640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0c3c100970 0x7f0c3c102390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:16:29.923 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:29.920+0000 7f0c41d3b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0c3c101330 0x7f0c3c1028d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:16:29.923 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:29.920+0000 7f0c41d3b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0c3c1056c0 0x7f0c3c102e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:16:29.923 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:29.920+0000 7f0c41d3b640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f0c3c110be0 con 0x7f0c3c1056c0 2024-03-18T23:16:29.923 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:29.920+0000 7f0c41d3b640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f0c3c110a60 con 0x7f0c3c101330 2024-03-18T23:16:29.923 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:29.920+0000 7f0c41d3b640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f0c3c110d60 con 0x7f0c3c100970 2024-03-18T23:16:29.923 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:29.920+0000 7f0c3bfff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0c3c1056c0 0x7f0c3c102e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:16:29.924 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:29.920+0000 7f0c3b7fe640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0c3c100970 0x7f0c3c102390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:16:29.924 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:29.920+0000 7f0c3bfff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0c3c1056c0 0x7f0c3c102e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53460/0 (socket says 172.21.15.49:53460) 2024-03-18T23:16:29.924 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:29.920+0000 7f0c3b7fe640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0c3c100970 0x7f0c3c102390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.49:46016/0 (socket says 172.21.15.49:46016) 2024-03-18T23:16:29.924 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:29.920+0000 7f0c3affd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0c3c101330 0x7f0c3c1028d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:16:29.924 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:29.920+0000 7f0c3bfff640 1 -- 172.21.15.49:0/842381024 learned_addr learned my addr 172.21.15.49:0/842381024 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:16:29.924 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:29.920+0000 7f0c3bfff640 1 -- 172.21.15.49:0/842381024 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0c3c100970 msgr2=0x7f0c3c102390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:16:29.924 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:29.920+0000 7f0c3bfff640 1 --2- 172.21.15.49:0/842381024 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0c3c100970 0x7f0c3c102390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:29.924 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:29.920+0000 7f0c3bfff640 1 -- 172.21.15.49:0/842381024 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0c3c101330 msgr2=0x7f0c3c1028d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:16:29.924 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:29.920+0000 7f0c3bfff640 1 --2- 172.21.15.49:0/842381024 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0c3c101330 0x7f0c3c1028d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:29.924 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:29.920+0000 7f0c3bfff640 1 -- 172.21.15.49:0/842381024 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c3c1a6640 con 0x7f0c3c1056c0 2024-03-18T23:16:29.924 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:29.921+0000 7f0c3bfff640 1 --2- 172.21.15.49:0/842381024 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0c3c1056c0 0x7f0c3c102e10 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f0c3400ea60 tx=0x7f0c3400ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:16:29.925 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:29.921+0000 7f0c38ff9640 1 -- 172.21.15.49:0/842381024 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c3400ce40 con 0x7f0c3c1056c0 2024-03-18T23:16:29.925 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:29.921+0000 7f0c38ff9640 1 -- 172.21.15.49:0/842381024 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0c34005020 con 0x7f0c3c1056c0 2024-03-18T23:16:29.925 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:29.921+0000 7f0c41d3b640 1 -- 172.21.15.49:0/842381024 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c3c1a6930 con 0x7f0c3c1056c0 2024-03-18T23:16:29.925 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:29.921+0000 7f0c38ff9640 1 -- 172.21.15.49:0/842381024 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c34010690 con 0x7f0c3c1056c0 2024-03-18T23:16:29.925 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:29.921+0000 7f0c41d3b640 1 -- 172.21.15.49:0/842381024 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c3c1a6e00 con 0x7f0c3c1056c0 2024-03-18T23:16:29.931 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:29.923+0000 7f0c38ff9640 1 -- 172.21.15.49:0/842381024 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 25) ==== 137421+0+0 (secure 0 0 0) 0x7f0c34005520 con 0x7f0c3c1056c0 2024-03-18T23:16:29.932 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:29.923+0000 7f0c449fe740 1 Processor -- start 2024-03-18T23:16:29.932 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:29.923+0000 7f0c449fe740 1 -- start start 2024-03-18T23:16:29.932 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:29.924+0000 7f0c449fe740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55947bb76ef0 0x55947bb772d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:16:29.932 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:29.924+0000 7f0c449fe740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55947bb77810 0x55947bb800a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:16:29.932 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:29.924+0000 7f0c449fe740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55947bb805e0 0x55947bb829d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:16:29.932 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:29.924+0000 7f0c449fe740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55947badabe0 con 0x55947bb77810 2024-03-18T23:16:29.932 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:29.924+0000 7f0c449fe740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55947ba83960 con 0x55947bb76ef0 2024-03-18T23:16:29.932 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:29.924+0000 7f0c449fe740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55947bad6c80 con 0x55947bb805e0 2024-03-18T23:16:29.932 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:29.924+0000 7f0c38ff9640 1 --2- 172.21.15.49:0/842381024 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f0c240a2a10 0x7f0c240a4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:16:29.932 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:29.924+0000 7f0c3affd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55947bb76ef0 0x55947bb772d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:16:29.933 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:29.924+0000 7f0c3b7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55947bb77810 0x55947bb800a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:16:29.933 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:29.924+0000 7f0c3affd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55947bb76ef0 0x55947bb772d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.49:40604/0 (socket says 172.21.15.49:40604) 2024-03-18T23:16:29.933 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:29.924+0000 7f0c3b7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55947bb77810 0x55947bb800a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53468/0 (socket says 172.21.15.49:53468) 2024-03-18T23:16:29.933 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:29.924+0000 7f0c3affd640 1 -- 172.21.15.49:0/350383811 learned_addr learned my addr 172.21.15.49:0/350383811 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:16:29.933 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:29.924+0000 7f0c3affd640 1 --2- 172.21.15.49:0/350383811 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55947bb805e0 0x55947bb829d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:16:29.933 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:29.924+0000 7f0c3bfff640 1 --2- 172.21.15.49:0/842381024 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f0c240a2a10 0x7f0c240a4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:16:29.933 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:29.924+0000 7f0c38ff9640 1 -- 172.21.15.49:0/842381024 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8556+0+0 (secure 0 0 0) 0x7f0c340cf370 con 0x7f0c3c1056c0 2024-03-18T23:16:29.933 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:29.924+0000 7f0c3b7fe640 1 -- 172.21.15.49:0/350383811 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55947bb805e0 msgr2=0x55947bb829d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:16:29.933 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:29.924+0000 7f0c3b7fe640 1 --2- 172.21.15.49:0/350383811 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55947bb805e0 0x55947bb829d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:29.934 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:29.924+0000 7f0c3b7fe640 1 -- 172.21.15.49:0/350383811 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55947bb76ef0 msgr2=0x55947bb772d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:16:29.934 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:29.924+0000 7f0c3b7fe640 1 --2- 172.21.15.49:0/350383811 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55947bb76ef0 0x55947bb772d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:29.934 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:29.924+0000 7f0c3b7fe640 1 -- 172.21.15.49:0/350383811 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55947ba7ad20 con 0x55947bb77810 2024-03-18T23:16:29.934 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:29.924+0000 7f0c3affd640 1 --2- 172.21.15.49:0/350383811 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55947bb76ef0 0x55947bb772d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:16:29.934 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:29.924+0000 7f0c3b7fe640 1 --2- 172.21.15.49:0/350383811 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55947bb77810 0x55947bb800a0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f0c2c009c00 tx=0x7f0c2c011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de97d1958825cc2d server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:16:29.934 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:29.924+0000 7f0c3bfff640 1 --2- 172.21.15.49:0/842381024 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f0c240a2a10 0x7f0c240a4ed0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f0c34097e10 tx=0x7f0c3400ef90 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:16:29.934 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:29.925+0000 7f0c21ffb640 1 -- 172.21.15.49:0/350383811 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c2c004050 con 0x55947bb77810 2024-03-18T23:16:29.934 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:29.925+0000 7f0c21ffb640 1 -- 172.21.15.49:0/350383811 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0c2c005500 con 0x55947bb77810 2024-03-18T23:16:29.934 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:29.925+0000 7f0c21ffb640 1 -- 172.21.15.49:0/350383811 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c2c004d40 con 0x55947bb77810 2024-03-18T23:16:29.934 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:29.925+0000 7f0c449fe740 1 -- 172.21.15.49:0/350383811 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55947bb77810 msgr2=0x55947bb800a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:16:29.935 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:29.925+0000 7f0c449fe740 1 --2- 172.21.15.49:0/350383811 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55947bb77810 0x55947bb800a0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f0c2c009c00 tx=0x7f0c2c011810 comp rx=0 tx=0).stop 2024-03-18T23:16:29.935 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:29.925+0000 7f0c449fe740 1 -- 172.21.15.49:0/350383811 shutdown_connections 2024-03-18T23:16:29.935 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:29.925+0000 7f0c449fe740 1 --2- 172.21.15.49:0/350383811 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55947bb805e0 0x55947bb829d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:29.935 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:29.925+0000 7f0c449fe740 1 --2- 172.21.15.49:0/350383811 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55947bb77810 0x55947bb800a0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:29.935 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:29.925+0000 7f0c449fe740 1 --2- 172.21.15.49:0/350383811 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55947bb76ef0 0x55947bb772d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:29.935 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:29.925+0000 7f0c449fe740 1 -- 172.21.15.49:0/350383811 >> 172.21.15.49:0/350383811 conn(0x55947bb71810 msgr2=0x55947bb71bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:16:29.935 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:29.925+0000 7f0c449fe740 1 -- 172.21.15.49:0/350383811 shutdown_connections 2024-03-18T23:16:29.935 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:29.925+0000 7f0c449fe740 1 -- 172.21.15.49:0/350383811 wait complete. 2024-03-18T23:16:29.935 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:29.926+0000 7f0c449fe740 1 -- start start 2024-03-18T23:16:29.935 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:29.926+0000 7f0c449fe740 1 Processor -- start 2024-03-18T23:16:29.935 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:29.926+0000 7f0c449fe740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55947bb76ef0 0x55947bb7ff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:16:29.935 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:29.926+0000 7f0c449fe740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55947bb77810 0x55947bb7d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:16:29.935 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:29.926+0000 7f0c449fe740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55947bb805e0 0x55947bb7d840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:16:29.935 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:29.926+0000 7f0c449fe740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55947bb83210 con 0x55947bb77810 2024-03-18T23:16:29.936 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:29.926+0000 7f0c449fe740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55947ba96db0 con 0x55947bb76ef0 2024-03-18T23:16:29.936 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:29.926+0000 7f0c449fe740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55947bb83390 con 0x55947bb805e0 2024-03-18T23:16:29.936 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:29.926+0000 7f0c449fe740 10 client.0 _ll_register_callbacks cb 0x55947ba9be70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T23:16:29.936 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:29.926+0000 7f0c3b7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55947bb77810 0x55947bb7d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:16:29.936 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:29.926+0000 7f0c3b7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55947bb77810 0x55947bb7d2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53484/0 (socket says 172.21.15.49:53484) 2024-03-18T23:16:29.936 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:29.926+0000 7f0c3affd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55947bb76ef0 0x55947bb7ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:16:29.936 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:29.926+0000 7f0c3b7fe640 1 -- 172.21.15.49:0/1914198169 learned_addr learned my addr 172.21.15.49:0/1914198169 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:16:29.936 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:29.926+0000 7f0c3affd640 1 --2- 172.21.15.49:0/1914198169 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55947bb805e0 0x55947bb7d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:16:29.936 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:29.927+0000 7f0c3b7fe640 1 -- 172.21.15.49:0/1914198169 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55947bb805e0 msgr2=0x55947bb7d840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:16:29.936 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:29.927+0000 7f0c3b7fe640 1 --2- 172.21.15.49:0/1914198169 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55947bb805e0 0x55947bb7d840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:29.936 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:29.927+0000 7f0c3b7fe640 1 -- 172.21.15.49:0/1914198169 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55947bb76ef0 msgr2=0x55947bb7ff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:16:29.936 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:29.927+0000 7f0c3b7fe640 1 --2- 172.21.15.49:0/1914198169 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55947bb76ef0 0x55947bb7ff50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:29.936 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:29.927+0000 7f0c3b7fe640 1 -- 172.21.15.49:0/1914198169 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55947ba95c60 con 0x55947bb77810 2024-03-18T23:16:29.937 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:29.927+0000 7f0c3b7fe640 1 --2- 172.21.15.49:0/1914198169 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55947bb77810 0x55947bb7d2e0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f0c2c004880 tx=0x7f0c2c005730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19c3b89b9a0b8d2d server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:16:29.937 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:29.927+0000 7f0c1affd640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-18T23:16:29.937 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:29.927+0000 7f0c1affd640 1 -- 172.21.15.49:0/1914198169 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c2c029070 con 0x55947bb77810 2024-03-18T23:16:29.937 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:29.927+0000 7f0c1affd640 1 -- 172.21.15.49:0/1914198169 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8556+0+0 (secure 0 0 0) 0x7f0c2c003a00 con 0x55947bb77810 2024-03-18T23:16:29.937 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:29.927+0000 7f0c449fe740 1 -- 172.21.15.49:0/1914198169 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0c2c005500 con 0x55947bb77810 2024-03-18T23:16:29.937 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:29.927+0000 7f0c1affd640 1 -- 172.21.15.49:0/1914198169 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f0c2c035050 con 0x55947bb77810 2024-03-18T23:16:29.937 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:29.927+0000 7f0c449fe740 10 client.16119 fetch_fsmap learned FSMap version 32 2024-03-18T23:16:29.937 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:29.927+0000 7f0c449fe740 1 -- 172.21.15.49:0/1914198169 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55947b7ef210 con 0x55947bb77810 2024-03-18T23:16:29.937 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:29.928+0000 7f0c1affd640 1 -- 172.21.15.49:0/1914198169 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0c2c005500 con 0x55947bb77810 2024-03-18T23:16:29.937 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:29.928+0000 7f0c1affd640 1 -- 172.21.15.49:0/1914198169 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f0c2c005190 con 0x55947bb77810 2024-03-18T23:16:29.937 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:29.928+0000 7f0c449fe740 10 client.16119 fetch_fsmap finished waiting for FSMap version 32 2024-03-18T23:16:29.937 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:29.928+0000 7f0c449fe740 10 client.16119 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T23:16:29.938 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:29.928+0000 7f0c449fe740 20 client.16119 populate_metadata read hostname 'smithi049' 2024-03-18T23:16:29.938 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:29.928+0000 7f0c449fe740 1 --2- 172.21.15.49:0/1914198169 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x55947bb90360 0x55947bb92750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:16:29.938 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:29.928+0000 7f0c449fe740 4 client.16119 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T23:16:29.938 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:29.928+0000 7f0c449fe740 1 -- 172.21.15.49:0/1914198169 --> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55947bae8120 con 0x55947bb90360 2024-03-18T23:16:29.938 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:29.928+0000 7f0c3affd640 1 --2- 172.21.15.49:0/1914198169 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x55947bb90360 0x55947bb92750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:16:29.938 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:29.928+0000 7f0c3affd640 1 --2- 172.21.15.49:0/1914198169 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x55947bb90360 0x55947bb92750 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=18ecf6f876e47fb server_cookie=b3d74d47a8cab2d4 in_seq=0 out_seq=0 2024-03-18T23:16:29.938 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:29.928+0000 7f0c1affd640 10 client.16119 ms_handle_connect on v2:172.21.15.139:6838/4280128209 2024-03-18T23:16:29.938 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:29.929+0000 7f0c1affd640 1 -- 172.21.15.49:0/1914198169 <== mds.0 v2:172.21.15.139:6838/4280128209 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55947bae8120 con 0x55947bb90360 2024-03-18T23:16:29.938 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:29.929+0000 7f0c1affd640 10 client.16119 handle_command_reply: tid=0 2024-03-18T23:16:29.938 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:29.929+0000 7f0c449fe740 1 client.16119 shutdown 2024-03-18T23:16:29.938 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:29.929+0000 7f0c449fe740 1 -- 172.21.15.49:0/1914198169 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55947bb77810 msgr2=0x55947bb7d2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:16:29.939 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:29.929+0000 7f0c449fe740 1 --2- 172.21.15.49:0/1914198169 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55947bb77810 0x55947bb7d2e0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f0c2c004880 tx=0x7f0c2c005730 comp rx=0 tx=0).stop 2024-03-18T23:16:29.939 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:29.929+0000 7f0c449fe740 1 -- 172.21.15.49:0/1914198169 shutdown_connections 2024-03-18T23:16:29.939 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:29.929+0000 7f0c449fe740 1 --2- 172.21.15.49:0/1914198169 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x55947bb90360 0x55947bb92750 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:29.939 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:29.929+0000 7f0c449fe740 1 --2- 172.21.15.49:0/1914198169 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55947bb805e0 0x55947bb7d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:29.939 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:29.929+0000 7f0c449fe740 1 --2- 172.21.15.49:0/1914198169 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55947bb77810 0x55947bb7d2e0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:29.939 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:29.929+0000 7f0c449fe740 1 --2- 172.21.15.49:0/1914198169 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55947bb76ef0 0x55947bb7ff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:29.939 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:29.929+0000 7f0c1affd640 0 client.16119 ms_handle_reset on v2:172.21.15.139:6838/4280128209 2024-03-18T23:16:29.939 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:29.929+0000 7f0c449fe740 1 -- 172.21.15.49:0/1914198169 >> 172.21.15.49:0/1914198169 conn(0x55947bb71810 msgr2=0x55947bb70ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:16:29.939 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:29.929+0000 7f0c449fe740 1 -- 172.21.15.49:0/1914198169 shutdown_connections 2024-03-18T23:16:29.939 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:29.929+0000 7f0c449fe740 1 -- 172.21.15.49:0/1914198169 wait complete. 2024-03-18T23:16:29.939 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:29.929+0000 7f0c449fe740 20 client.16119 trim_cache size 0 max 16384 2024-03-18T23:16:29.947 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:29.945+0000 7f0c449fe740 1 Processor -- start 2024-03-18T23:16:29.947 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:29.945+0000 7f0c449fe740 1 -- start start 2024-03-18T23:16:29.947 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:29.945+0000 7f0c449fe740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55947bb76ef0 0x55947bb8bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:16:29.947 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:29.945+0000 7f0c449fe740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55947bb77810 0x55947bb97f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:16:29.947 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:29.945+0000 7f0c449fe740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55947bb805e0 0x55947bb7fa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:16:29.947 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:29.945+0000 7f0c449fe740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f0c3c1af690 con 0x55947bb805e0 2024-03-18T23:16:29.948 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:29.945+0000 7f0c449fe740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f0c3c1b11d0 con 0x55947bb77810 2024-03-18T23:16:29.948 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:29.945+0000 7f0c449fe740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f0c3c1afd60 con 0x55947bb76ef0 2024-03-18T23:16:29.948 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:29.945+0000 7f0c3affd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55947bb805e0 0x55947bb7fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:16:29.948 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:29.945+0000 7f0c3b7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55947bb77810 0x55947bb97f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:16:29.948 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:29.945+0000 7f0c3affd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55947bb805e0 0x55947bb7fa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53492/0 (socket says 172.21.15.49:53492) 2024-03-18T23:16:29.948 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:29.945+0000 7f0c3affd640 1 -- 172.21.15.49:0/1548715859 learned_addr learned my addr 172.21.15.49:0/1548715859 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:16:29.948 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:29.945+0000 7f0c3affd640 1 --2- 172.21.15.49:0/1548715859 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55947bb76ef0 0x55947bb8bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:16:29.948 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:29.946+0000 7f0c3affd640 1 -- 172.21.15.49:0/1548715859 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55947bb76ef0 msgr2=0x55947bb8bfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:16:29.948 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:29.946+0000 7f0c3affd640 1 --2- 172.21.15.49:0/1548715859 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55947bb76ef0 0x55947bb8bfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:29.948 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:29.946+0000 7f0c3affd640 1 -- 172.21.15.49:0/1548715859 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55947bb77810 msgr2=0x55947bb97f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:16:29.948 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:29.946+0000 7f0c3affd640 1 --2- 172.21.15.49:0/1548715859 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55947bb77810 0x55947bb97f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:29.948 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:29.946+0000 7f0c3affd640 1 -- 172.21.15.49:0/1548715859 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55947b8bb620 con 0x55947bb805e0 2024-03-18T23:16:29.948 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:29.946+0000 7f0c3b7fe640 1 --2- 172.21.15.49:0/1548715859 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55947bb77810 0x55947bb97f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:16:29.949 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:29.946+0000 7f0c3affd640 1 --2- 172.21.15.49:0/1548715859 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55947bb805e0 0x55947bb7fa10 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f0c30000c00 tx=0x7f0c3002bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61aab70217009901 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:16:29.949 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:29.946+0000 7f0c217fa640 1 -- 172.21.15.49:0/1548715859 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c30050630 con 0x55947bb805e0 2024-03-18T23:16:29.949 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:29.946+0000 7f0c217fa640 1 -- 172.21.15.49:0/1548715859 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0c300507d0 con 0x55947bb805e0 2024-03-18T23:16:29.949 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:29.946+0000 7f0c217fa640 1 -- 172.21.15.49:0/1548715859 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c30005120 con 0x55947bb805e0 2024-03-18T23:16:29.949 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:29.947+0000 7f0c449fe740 1 -- 172.21.15.49:0/1548715859 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55947bb805e0 msgr2=0x55947bb7fa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:16:29.949 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:29.947+0000 7f0c449fe740 1 --2- 172.21.15.49:0/1548715859 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55947bb805e0 0x55947bb7fa10 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f0c30000c00 tx=0x7f0c3002bc30 comp rx=0 tx=0).stop 2024-03-18T23:16:29.949 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:29.947+0000 7f0c449fe740 1 -- 172.21.15.49:0/1548715859 shutdown_connections 2024-03-18T23:16:29.949 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:29.947+0000 7f0c449fe740 1 --2- 172.21.15.49:0/1548715859 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55947bb805e0 0x55947bb7fa10 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:29.949 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:29.947+0000 7f0c449fe740 1 --2- 172.21.15.49:0/1548715859 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55947bb77810 0x55947bb97f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:29.949 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:29.947+0000 7f0c449fe740 1 --2- 172.21.15.49:0/1548715859 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55947bb76ef0 0x55947bb8bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:29.950 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:29.947+0000 7f0c449fe740 1 -- 172.21.15.49:0/1548715859 >> 172.21.15.49:0/1548715859 conn(0x55947bb71810 msgr2=0x55947bb70ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:16:29.950 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:29.947+0000 7f0c449fe740 1 -- 172.21.15.49:0/1548715859 shutdown_connections 2024-03-18T23:16:29.950 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:29.947+0000 7f0c449fe740 1 -- 172.21.15.49:0/1548715859 wait complete. 2024-03-18T23:16:29.950 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:29.947+0000 7f0c449fe740 1 -- start start 2024-03-18T23:16:29.950 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:29.947+0000 7f0c449fe740 1 Processor -- start 2024-03-18T23:16:29.954 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:29.947+0000 7f0c449fe740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55947bb76ef0 0x55947bb885b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:16:29.954 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:29.947+0000 7f0c449fe740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55947bb77810 0x55947bb83b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:16:29.954 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:29.948+0000 7f0c449fe740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55947bb805e0 0x55947bb840a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:16:29.954 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:29.948+0000 7f0c449fe740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f0c3c1b0b00 con 0x55947bb805e0 2024-03-18T23:16:29.954 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:29.948+0000 7f0c449fe740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f0c3c1b0430 con 0x55947bb76ef0 2024-03-18T23:16:29.954 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:29.948+0000 7f0c449fe740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f0c3c1ad690 con 0x55947bb77810 2024-03-18T23:16:29.954 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:29.948+0000 7f0c449fe740 10 client.0 _ll_register_callbacks cb 0x7f0c2c005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T23:16:29.954 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:29.948+0000 7f0c3affd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55947bb805e0 0x55947bb840a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:16:29.954 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:29.948+0000 7f0c3affd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55947bb805e0 0x55947bb840a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53494/0 (socket says 172.21.15.49:53494) 2024-03-18T23:16:29.954 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:29.948+0000 7f0c3affd640 1 -- 172.21.15.49:0/3586963669 learned_addr learned my addr 172.21.15.49:0/3586963669 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:16:29.955 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:29.948+0000 7f0c3b7fe640 1 --2- 172.21.15.49:0/3586963669 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55947bb77810 0x55947bb83b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:16:29.955 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:29.948+0000 7f0c3affd640 1 --2- 172.21.15.49:0/3586963669 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55947bb76ef0 0x55947bb885b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:16:29.955 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:29.948+0000 7f0c3b7fe640 1 -- 172.21.15.49:0/3586963669 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55947bb76ef0 msgr2=0x55947bb885b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:16:29.955 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:29.948+0000 7f0c3b7fe640 1 --2- 172.21.15.49:0/3586963669 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55947bb76ef0 0x55947bb885b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:29.955 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:29.948+0000 7f0c3b7fe640 1 -- 172.21.15.49:0/3586963669 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55947bb805e0 msgr2=0x55947bb840a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:16:29.955 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:29.948+0000 7f0c3b7fe640 1 --2- 172.21.15.49:0/3586963669 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55947bb805e0 0x55947bb840a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:29.955 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:29.948+0000 7f0c3b7fe640 1 -- 172.21.15.49:0/3586963669 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55947bb84610 con 0x55947bb77810 2024-03-18T23:16:29.955 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:29.948+0000 7f0c3affd640 1 --2- 172.21.15.49:0/3586963669 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55947bb805e0 0x55947bb840a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:16:29.955 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:29.948+0000 7f0c3b7fe640 1 --2- 172.21.15.49:0/3586963669 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55947bb77810 0x55947bb83b60 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f0c2c0057c0 tx=0x7f0c2c005f10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dda701ec0845c611 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:16:29.956 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:29.948+0000 7f0c1affd640 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-03-18T23:16:29.956 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:29.948+0000 7f0c1affd640 1 -- 172.21.15.49:0/3586963669 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c2c029050 con 0x55947bb77810 2024-03-18T23:16:29.956 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:29.948+0000 7f0c449fe740 1 -- 172.21.15.49:0/3586963669 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0c300507d0 con 0x55947bb77810 2024-03-18T23:16:29.956 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:29.948+0000 7f0c1affd640 1 -- 172.21.15.49:0/3586963669 <== mon.1 v2:172.21.15.175:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8556+0+0 (secure 0 0 0) 0x7f0c2c0043a0 con 0x55947bb77810 2024-03-18T23:16:29.956 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:29.949+0000 7f0c1affd640 1 -- 172.21.15.49:0/3586963669 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f0c2c0045a0 con 0x55947bb77810 2024-03-18T23:16:29.956 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:29.949+0000 7f0c449fe740 10 client.25927 fetch_fsmap learned FSMap version 32 2024-03-18T23:16:29.956 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:29.949+0000 7f0c449fe740 1 -- 172.21.15.49:0/3586963669 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55947bbf62f0 con 0x55947bb77810 2024-03-18T23:16:29.956 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:29.949+0000 7f0c1affd640 1 -- 172.21.15.49:0/3586963669 <== mon.1 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0c300507d0 con 0x55947bb77810 2024-03-18T23:16:29.956 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:29.949+0000 7f0c1affd640 1 -- 172.21.15.49:0/3586963669 <== mon.1 v2:172.21.15.175:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f0c2c003c30 con 0x55947bb77810 2024-03-18T23:16:29.956 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:29.949+0000 7f0c449fe740 10 client.25927 fetch_fsmap finished waiting for FSMap version 32 2024-03-18T23:16:29.956 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:29.949+0000 7f0c449fe740 10 client.25927 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T23:16:29.956 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:29.949+0000 7f0c449fe740 20 client.25927 populate_metadata read hostname 'smithi049' 2024-03-18T23:16:29.956 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:29.949+0000 7f0c449fe740 1 --2- 172.21.15.49:0/3586963669 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x55947bbf6950 0x55947bbf8d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:16:29.956 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:29.949+0000 7f0c449fe740 4 client.25927 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T23:16:29.956 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:29.949+0000 7f0c449fe740 1 -- 172.21.15.49:0/3586963669 --> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55947bae2470 con 0x55947bbf6950 2024-03-18T23:16:29.957 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:29.949+0000 7f0c3affd640 1 --2- 172.21.15.49:0/3586963669 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x55947bbf6950 0x55947bbf8d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:16:29.957 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:29.949+0000 7f0c3affd640 1 --2- 172.21.15.49:0/3586963669 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x55947bbf6950 0x55947bbf8d70 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a5e57ac7622c09e1 server_cookie=324b49a29072275c in_seq=0 out_seq=0 2024-03-18T23:16:29.957 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:29.949+0000 7f0c1affd640 10 client.25927 ms_handle_connect on v2:172.21.15.139:6838/4280128209 2024-03-18T23:16:30.134 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:30.132+0000 7f0c1affd640 1 -- 172.21.15.49:0/3586963669 <== mds.0 v2:172.21.15.139:6838/4280128209 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55947bae2470 con 0x55947bbf6950 2024-03-18T23:16:30.134 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:30.132+0000 7f0c1affd640 10 client.25927 handle_command_reply: tid=0 2024-03-18T23:16:30.134 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:30.132+0000 7f0c449fe740 1 client.25927 shutdown 2024-03-18T23:16:30.134 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:30.133+0000 7f0c449fe740 1 -- 172.21.15.49:0/3586963669 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55947bb77810 msgr2=0x55947bb83b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:16:30.134 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:30.133+0000 7f0c449fe740 1 --2- 172.21.15.49:0/3586963669 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55947bb77810 0x55947bb83b60 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f0c2c0057c0 tx=0x7f0c2c005f10 comp rx=0 tx=0).stop 2024-03-18T23:16:30.134 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:30.133+0000 7f0c449fe740 1 -- 172.21.15.49:0/3586963669 shutdown_connections 2024-03-18T23:16:30.134 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:30.133+0000 7f0c449fe740 1 --2- 172.21.15.49:0/3586963669 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x55947bbf6950 0x55947bbf8d70 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:30.134 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:30.133+0000 7f0c449fe740 1 --2- 172.21.15.49:0/3586963669 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55947bb805e0 0x55947bb840a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:30.134 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:30.133+0000 7f0c449fe740 1 --2- 172.21.15.49:0/3586963669 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55947bb77810 0x55947bb83b60 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:30.135 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:30.133+0000 7f0c449fe740 1 --2- 172.21.15.49:0/3586963669 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55947bb76ef0 0x55947bb885b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:30.135 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:30.133+0000 7f0c449fe740 1 -- 172.21.15.49:0/3586963669 >> 172.21.15.49:0/3586963669 conn(0x55947bb71810 msgr2=0x55947bb72c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:16:30.135 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:30.133+0000 7f0c1affd640 0 client.25927 ms_handle_reset on v2:172.21.15.139:6838/4280128209 2024-03-18T23:16:30.135 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:30.133+0000 7f0c449fe740 1 -- 172.21.15.49:0/3586963669 shutdown_connections 2024-03-18T23:16:30.135 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:30.133+0000 7f0c449fe740 1 -- 172.21.15.49:0/3586963669 wait complete. 2024-03-18T23:16:30.135 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:30.133+0000 7f0c449fe740 20 client.25927 trim_cache size 0 max 16384 2024-03-18T23:16:30.135 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:{ 2024-03-18T23:16:30.135 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "message": "", 2024-03-18T23:16:30.136 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "return_code": 0 2024-03-18T23:16:30.136 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:} 2024-03-18T23:16:30.136 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:30.134+0000 7f0c41d3b640 1 -- 172.21.15.49:0/842381024 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f0c240a2a10 msgr2=0x7f0c240a4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:16:30.137 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:30.134+0000 7f0c41d3b640 1 --2- 172.21.15.49:0/842381024 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f0c240a2a10 0x7f0c240a4ed0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f0c34097e10 tx=0x7f0c3400ef90 comp rx=0 tx=0).stop 2024-03-18T23:16:30.137 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:30.134+0000 7f0c41d3b640 1 -- 172.21.15.49:0/842381024 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0c3c1056c0 msgr2=0x7f0c3c102e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:16:30.137 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:30.134+0000 7f0c41d3b640 1 --2- 172.21.15.49:0/842381024 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0c3c1056c0 0x7f0c3c102e10 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f0c3400ea60 tx=0x7f0c3400ef20 comp rx=0 tx=0).stop 2024-03-18T23:16:30.137 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:30.134+0000 7f0c41d3b640 1 -- 172.21.15.49:0/842381024 shutdown_connections 2024-03-18T23:16:30.137 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:30.134+0000 7f0c41d3b640 1 --2- 172.21.15.49:0/842381024 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f0c240a2a10 0x7f0c240a4ed0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:30.137 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:30.134+0000 7f0c41d3b640 1 --2- 172.21.15.49:0/842381024 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0c3c1056c0 0x7f0c3c102e10 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:30.138 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:30.134+0000 7f0c41d3b640 1 --2- 172.21.15.49:0/842381024 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0c3c101330 0x7f0c3c1028d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:30.138 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:30.134+0000 7f0c41d3b640 1 --2- 172.21.15.49:0/842381024 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0c3c100970 0x7f0c3c102390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:30.138 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:30.134+0000 7f0c41d3b640 1 -- 172.21.15.49:0/842381024 >> 172.21.15.49:0/842381024 conn(0x7f0c3c077570 msgr2=0x7f0c3c069240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:16:30.138 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:30.134+0000 7f0c41d3b640 1 -- 172.21.15.49:0/842381024 shutdown_connections 2024-03-18T23:16:30.138 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:30.134+0000 7f0c41d3b640 1 -- 172.21.15.49:0/842381024 wait complete. 2024-03-18T23:16:30.379 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:16:30 smithi049 ceph-mon[62991]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:16:30.379 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:16:30 smithi049 ceph-mon[62991]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:16:30.379 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:16:30 smithi049 ceph-mon[62991]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' 2024-03-18T23:16:30.460 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:16:30 smithi175 ceph-mon[67352]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:16:30.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:16:30 smithi175 ceph-mon[67352]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:16:30.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:16:30 smithi175 ceph-mon[67352]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' 2024-03-18T23:16:30.564 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:16:30 smithi139 ceph-mon[67282]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:16:30.564 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:16:30 smithi139 ceph-mon[67282]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:16:30.564 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:16:30 smithi139 ceph-mon[67282]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' 2024-03-18T23:16:30.779 INFO:tasks.workunit.client.0.smithi049.stdout: 1 876941 9.75 MB/sec execute 332 sec latency 129.463 ms 2024-03-18T23:16:31.379 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:16:31 smithi049 ceph-mon[62991]: pgmap v256: 97 pgs: 97 active+clean; 78 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.3 MiB/s wr, 228 op/s 2024-03-18T23:16:31.528 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:16:31 smithi175 ceph-mon[67352]: pgmap v256: 97 pgs: 97 active+clean; 78 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.3 MiB/s wr, 228 op/s 2024-03-18T23:16:31.564 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:16:31 smithi139 ceph-mon[67282]: pgmap v256: 97 pgs: 97 active+clean; 78 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.3 MiB/s wr, 228 op/s 2024-03-18T23:16:31.779 INFO:tasks.workunit.client.0.smithi049.stdout: 1 877872 9.73 MB/sec execute 333 sec latency 28.463 ms 2024-03-18T23:16:32.779 INFO:tasks.workunit.client.0.smithi049.stdout: 1 880114 9.74 MB/sec execute 334 sec latency 18.851 ms 2024-03-18T23:16:33.379 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:16:33 smithi049 ceph-mon[62991]: pgmap v257: 97 pgs: 97 active+clean; 71 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.5 MiB/s wr, 377 op/s 2024-03-18T23:16:33.460 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:16:33 smithi175 ceph-mon[67352]: pgmap v257: 97 pgs: 97 active+clean; 71 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.5 MiB/s wr, 377 op/s 2024-03-18T23:16:33.564 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:16:33 smithi139 ceph-mon[67282]: pgmap v257: 97 pgs: 97 active+clean; 71 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.5 MiB/s wr, 377 op/s 2024-03-18T23:16:33.779 INFO:tasks.workunit.client.0.smithi049.stdout: 1 881095 9.73 MB/sec execute 335 sec latency 29.810 ms 2024-03-18T23:16:34.779 INFO:tasks.workunit.client.0.smithi049.stdout: 1 882894 9.72 MB/sec execute 336 sec latency 96.640 ms 2024-03-18T23:16:35.379 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:16:35 smithi049 ceph-mon[62991]: pgmap v258: 97 pgs: 97 active+clean; 67 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.9 MiB/s wr, 272 op/s 2024-03-18T23:16:35.460 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:16:35 smithi175 ceph-mon[67352]: pgmap v258: 97 pgs: 97 active+clean; 67 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.9 MiB/s wr, 272 op/s 2024-03-18T23:16:35.564 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:16:35 smithi139 ceph-mon[67282]: pgmap v258: 97 pgs: 97 active+clean; 67 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.9 MiB/s wr, 272 op/s 2024-03-18T23:16:35.779 INFO:tasks.workunit.client.0.smithi049.stdout: 1 884177 9.72 MB/sec execute 337 sec latency 38.938 ms 2024-03-18T23:16:36.779 INFO:tasks.workunit.client.0.smithi049.stdout: 1 885576 9.71 MB/sec execute 338 sec latency 27.441 ms 2024-03-18T23:16:37.379 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:16:37 smithi049 ceph-mon[62991]: pgmap v259: 97 pgs: 97 active+clean; 67 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.5 MiB/s wr, 238 op/s 2024-03-18T23:16:37.460 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:16:37 smithi175 ceph-mon[67352]: pgmap v259: 97 pgs: 97 active+clean; 67 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.5 MiB/s wr, 238 op/s 2024-03-18T23:16:37.564 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:16:37 smithi139 ceph-mon[67282]: pgmap v259: 97 pgs: 97 active+clean; 67 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.5 MiB/s wr, 238 op/s 2024-03-18T23:16:37.779 INFO:tasks.workunit.client.0.smithi049.stdout: 1 887637 9.72 MB/sec execute 339 sec latency 19.073 ms 2024-03-18T23:16:38.779 INFO:tasks.workunit.client.0.smithi049.stdout: 1 888828 9.70 MB/sec execute 340 sec latency 24.264 ms 2024-03-18T23:16:39.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:16:39 smithi049 ceph-mon[62991]: pgmap v260: 97 pgs: 97 active+clean; 73 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 3.5 MiB/s wr, 312 op/s 2024-03-18T23:16:39.460 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:16:39 smithi175 ceph-mon[67352]: pgmap v260: 97 pgs: 97 active+clean; 73 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 3.5 MiB/s wr, 312 op/s 2024-03-18T23:16:39.564 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:16:39 smithi139 ceph-mon[67282]: pgmap v260: 97 pgs: 97 active+clean; 73 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 3.5 MiB/s wr, 312 op/s 2024-03-18T23:16:39.779 INFO:tasks.workunit.client.0.smithi049.stdout: 1 891074 9.72 MB/sec execute 341 sec latency 53.902 ms 2024-03-18T23:16:40.779 INFO:tasks.workunit.client.0.smithi049.stdout: 1 891703 9.70 MB/sec execute 342 sec latency 134.828 ms 2024-03-18T23:16:41.379 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:16:41 smithi049 ceph-mon[62991]: pgmap v261: 97 pgs: 97 active+clean; 73 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.3 MiB/s wr, 231 op/s 2024-03-18T23:16:41.460 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:16:41 smithi175 ceph-mon[67352]: pgmap v261: 97 pgs: 97 active+clean; 73 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.3 MiB/s wr, 231 op/s 2024-03-18T23:16:41.564 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:16:41 smithi139 ceph-mon[67282]: pgmap v261: 97 pgs: 97 active+clean; 73 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.3 MiB/s wr, 231 op/s 2024-03-18T23:16:41.779 INFO:tasks.workunit.client.0.smithi049.stdout: 1 893511 9.69 MB/sec execute 343 sec latency 26.577 ms 2024-03-18T23:16:42.779 INFO:tasks.workunit.client.0.smithi049.stdout: 1 895024 9.70 MB/sec execute 344 sec latency 16.824 ms 2024-03-18T23:16:43.240 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:43.238+0000 7f634f2f8640 1 -- 172.21.15.49:0/3922542941 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f634810e110 msgr2=0x7f6348110540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:16:43.240 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:43.238+0000 7f634f2f8640 1 --2- 172.21.15.49:0/3922542941 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f634810e110 0x7f6348110540 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f6340009ca0 tx=0x7f634002c0f0 comp rx=0 tx=0).stop 2024-03-18T23:16:43.241 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:43.238+0000 7f634f2f8640 1 -- 172.21.15.49:0/3922542941 shutdown_connections 2024-03-18T23:16:43.241 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:43.238+0000 7f634f2f8640 1 --2- 172.21.15.49:0/3922542941 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f634810e110 0x7f6348110540 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:43.241 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:43.238+0000 7f634f2f8640 1 --2- 172.21.15.49:0/3922542941 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6348069470 0x7f63480698f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:43.241 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:43.238+0000 7f634f2f8640 1 --2- 172.21.15.49:0/3922542941 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6348102770 0x7f6348068ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:43.241 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:43.238+0000 7f634f2f8640 1 -- 172.21.15.49:0/3922542941 >> 172.21.15.49:0/3922542941 conn(0x7f63480774b0 msgr2=0x7f63480778c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:16:43.241 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:43.238+0000 7f634f2f8640 1 -- 172.21.15.49:0/3922542941 shutdown_connections 2024-03-18T23:16:43.241 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:43.238+0000 7f634f2f8640 1 -- 172.21.15.49:0/3922542941 wait complete. 2024-03-18T23:16:43.241 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:43.239+0000 7f634f2f8640 1 Processor -- start 2024-03-18T23:16:43.241 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:43.239+0000 7f634f2f8640 1 -- start start 2024-03-18T23:16:43.241 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:43.239+0000 7f634f2f8640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6348069470 0x7f634819a210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:16:43.241 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:43.239+0000 7f634f2f8640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6348102770 0x7f634819a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:16:43.241 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:43.239+0000 7f634f2f8640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f634810e110 0x7f634819ead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:16:43.241 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:43.239+0000 7f634f2f8640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f6348113e70 con 0x7f6348069470 2024-03-18T23:16:43.242 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:43.239+0000 7f634f2f8640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f6348113cf0 con 0x7f634810e110 2024-03-18T23:16:43.242 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:43.239+0000 7f634f2f8640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f6348113ff0 con 0x7f6348102770 2024-03-18T23:16:43.242 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:43.239+0000 7f634d094640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6348069470 0x7f634819a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:16:43.242 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:43.239+0000 7f634d094640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6348069470 0x7f634819a210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:46524/0 (socket says 172.21.15.49:46524) 2024-03-18T23:16:43.242 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:43.239+0000 7f634d094640 1 -- 172.21.15.49:0/910701383 learned_addr learned my addr 172.21.15.49:0/910701383 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:16:43.242 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:43.239+0000 7f634d895640 1 --2- 172.21.15.49:0/910701383 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f634810e110 0x7f634819ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:16:43.242 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:43.239+0000 7f634c893640 1 --2- 172.21.15.49:0/910701383 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6348102770 0x7f634819a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:16:43.242 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:43.239+0000 7f634d094640 1 -- 172.21.15.49:0/910701383 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6348102770 msgr2=0x7f634819a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:16:43.242 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:43.239+0000 7f634d094640 1 --2- 172.21.15.49:0/910701383 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6348102770 0x7f634819a750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:43.242 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:43.239+0000 7f634d094640 1 -- 172.21.15.49:0/910701383 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f634810e110 msgr2=0x7f634819ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:16:43.242 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:43.239+0000 7f634d094640 1 --2- 172.21.15.49:0/910701383 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f634810e110 0x7f634819ead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:43.242 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:43.239+0000 7f634d094640 1 -- 172.21.15.49:0/910701383 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f634819f200 con 0x7f6348069470 2024-03-18T23:16:43.243 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:43.240+0000 7f634d094640 1 --2- 172.21.15.49:0/910701383 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6348069470 0x7f634819a210 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f634400ea60 tx=0x7f634400ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:16:43.243 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:43.240+0000 7f633e7fc640 1 -- 172.21.15.49:0/910701383 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f634400cde0 con 0x7f6348069470 2024-03-18T23:16:43.243 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:43.240+0000 7f634f2f8640 1 -- 172.21.15.49:0/910701383 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f634819f4f0 con 0x7f6348069470 2024-03-18T23:16:43.243 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:43.240+0000 7f633e7fc640 1 -- 172.21.15.49:0/910701383 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f63440035f0 con 0x7f6348069470 2024-03-18T23:16:43.243 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:43.240+0000 7f633e7fc640 1 -- 172.21.15.49:0/910701383 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6344010670 con 0x7f6348069470 2024-03-18T23:16:43.243 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:43.240+0000 7f634f2f8640 1 -- 172.21.15.49:0/910701383 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63481a6d80 con 0x7f6348069470 2024-03-18T23:16:43.249 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:43.242+0000 7f633e7fc640 1 -- 172.21.15.49:0/910701383 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 25) ==== 137421+0+0 (secure 0 0 0) 0x7f63440037a0 con 0x7f6348069470 2024-03-18T23:16:43.249 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:43.243+0000 7f6351fbf740 1 Processor -- start 2024-03-18T23:16:43.249 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:43.243+0000 7f633e7fc640 1 --2- 172.21.15.49:0/910701383 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f63340a2a90 0x7f63340a4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:16:43.249 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:43.243+0000 7f6351fbf740 1 -- start start 2024-03-18T23:16:43.249 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:43.243+0000 7f634d895640 1 --2- 172.21.15.49:0/910701383 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f63340a2a90 0x7f63340a4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:16:43.249 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:43.243+0000 7f6351fbf740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ee6a567ef0 0x55ee6a5682d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:16:43.250 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:43.243+0000 7f6351fbf740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ee6a568810 0x55ee6a5710a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:16:43.250 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:43.243+0000 7f6351fbf740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ee6a5715e0 0x55ee6a5739d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:16:43.250 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:43.243+0000 7f6351fbf740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55ee6a4cbbe0 con 0x55ee6a567ef0 2024-03-18T23:16:43.250 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:43.243+0000 7f633e7fc640 1 -- 172.21.15.49:0/910701383 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8556+0+0 (secure 0 0 0) 0x7f63440cf090 con 0x7f6348069470 2024-03-18T23:16:43.250 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:43.243+0000 7f6351fbf740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55ee6a474960 con 0x55ee6a568810 2024-03-18T23:16:43.250 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:43.243+0000 7f6351fbf740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55ee6a4c7c80 con 0x55ee6a5715e0 2024-03-18T23:16:43.250 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:43.243+0000 7f634c893640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ee6a568810 0x55ee6a5710a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:16:43.250 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:43.243+0000 7f634d094640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ee6a567ef0 0x55ee6a5682d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:16:43.250 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:43.243+0000 7f634d895640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ee6a5715e0 0x55ee6a5739d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:16:43.250 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:43.243+0000 7f634c893640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ee6a568810 0x55ee6a5710a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.49:33772/0 (socket says 172.21.15.49:33772) 2024-03-18T23:16:43.250 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:43.243+0000 7f634d094640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ee6a567ef0 0x55ee6a5682d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:46538/0 (socket says 172.21.15.49:46538) 2024-03-18T23:16:43.250 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:43.243+0000 7f634c893640 1 -- 172.21.15.49:0/1502616612 learned_addr learned my addr 172.21.15.49:0/1502616612 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:16:43.251 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:43.243+0000 7f634d094640 1 -- 172.21.15.49:0/1502616612 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ee6a5715e0 msgr2=0x55ee6a5739d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:16:43.251 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:43.243+0000 7f634d094640 1 --2- 172.21.15.49:0/1502616612 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ee6a5715e0 0x55ee6a5739d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:43.251 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:43.243+0000 7f634d094640 1 -- 172.21.15.49:0/1502616612 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ee6a568810 msgr2=0x55ee6a5710a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:16:43.251 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:43.243+0000 7f634d094640 1 --2- 172.21.15.49:0/1502616612 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ee6a568810 0x55ee6a5710a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:43.251 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:43.243+0000 7f634d094640 1 -- 172.21.15.49:0/1502616612 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ee6a46bd20 con 0x55ee6a567ef0 2024-03-18T23:16:43.251 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:43.243+0000 7f634c893640 1 --2- 172.21.15.49:0/1502616612 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ee6a568810 0x55ee6a5710a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:16:43.251 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:43.243+0000 7f634d895640 1 --2- 172.21.15.49:0/910701383 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f63340a2a90 0x7f63340a4f50 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f6340005e30 tx=0x7f634002ce70 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:16:43.251 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:43.244+0000 7f634d094640 1 --2- 172.21.15.49:0/1502616612 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ee6a567ef0 0x55ee6a5682d0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f63440cc7c0 tx=0x7f634409b2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2ae672b6537d98c server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:16:43.251 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:43.244+0000 7f63337fe640 1 -- 172.21.15.49:0/1502616612 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f634409b6d0 con 0x55ee6a567ef0 2024-03-18T23:16:43.251 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:43.244+0000 7f63337fe640 1 -- 172.21.15.49:0/1502616612 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f634409f820 con 0x55ee6a567ef0 2024-03-18T23:16:43.251 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:43.244+0000 7f63337fe640 1 -- 172.21.15.49:0/1502616612 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6344097d40 con 0x55ee6a567ef0 2024-03-18T23:16:43.251 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:43.245+0000 7f6351fbf740 1 -- 172.21.15.49:0/1502616612 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ee6a567ef0 msgr2=0x55ee6a5682d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:16:43.252 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:43.245+0000 7f6351fbf740 1 --2- 172.21.15.49:0/1502616612 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ee6a567ef0 0x55ee6a5682d0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f63440cc7c0 tx=0x7f634409b2d0 comp rx=0 tx=0).stop 2024-03-18T23:16:43.252 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:43.245+0000 7f6351fbf740 1 -- 172.21.15.49:0/1502616612 shutdown_connections 2024-03-18T23:16:43.252 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:43.245+0000 7f6351fbf740 1 --2- 172.21.15.49:0/1502616612 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ee6a5715e0 0x55ee6a5739d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:43.252 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:43.245+0000 7f6351fbf740 1 --2- 172.21.15.49:0/1502616612 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ee6a568810 0x55ee6a5710a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:43.252 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:43.245+0000 7f6351fbf740 1 --2- 172.21.15.49:0/1502616612 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ee6a567ef0 0x55ee6a5682d0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:43.252 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:43.245+0000 7f6351fbf740 1 -- 172.21.15.49:0/1502616612 >> 172.21.15.49:0/1502616612 conn(0x55ee6a562810 msgr2=0x55ee6a562bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:16:43.252 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:43.245+0000 7f6351fbf740 1 -- 172.21.15.49:0/1502616612 shutdown_connections 2024-03-18T23:16:43.252 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:43.245+0000 7f6351fbf740 1 -- 172.21.15.49:0/1502616612 wait complete. 2024-03-18T23:16:43.252 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:43.245+0000 7f6351fbf740 1 -- start start 2024-03-18T23:16:43.252 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:43.245+0000 7f6351fbf740 1 Processor -- start 2024-03-18T23:16:43.252 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:43.246+0000 7f6351fbf740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ee6a567ef0 0x55ee6a570fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:16:43.253 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:43.246+0000 7f6351fbf740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ee6a568810 0x55ee6a56e360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:16:43.253 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:43.246+0000 7f634c893640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ee6a568810 0x55ee6a56e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:16:43.253 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:43.246+0000 7f634d094640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ee6a567ef0 0x55ee6a570fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:16:43.253 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:43.246+0000 7f634c893640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ee6a568810 0x55ee6a56e360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:46544/0 (socket says 172.21.15.49:46544) 2024-03-18T23:16:43.253 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:43.246+0000 7f634d094640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ee6a567ef0 0x55ee6a570fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.49:33788/0 (socket says 172.21.15.49:33788) 2024-03-18T23:16:43.253 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:43.246+0000 7f6351fbf740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ee6a5715e0 0x55ee6a56e970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:16:43.253 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:43.246+0000 7f634c893640 1 -- 172.21.15.49:0/2102500877 learned_addr learned my addr 172.21.15.49:0/2102500877 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:16:43.253 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:43.246+0000 7f6351fbf740 1 -- 172.21.15.49:0/2102500877 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55ee6a574210 con 0x55ee6a568810 2024-03-18T23:16:43.253 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:43.246+0000 7f6351fbf740 1 -- 172.21.15.49:0/2102500877 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55ee6a487db0 con 0x55ee6a567ef0 2024-03-18T23:16:43.253 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:43.246+0000 7f6351fbf740 1 -- 172.21.15.49:0/2102500877 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55ee6a574390 con 0x55ee6a5715e0 2024-03-18T23:16:43.253 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:43.246+0000 7f634d895640 1 --2- 172.21.15.49:0/2102500877 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ee6a5715e0 0x55ee6a56e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:16:43.253 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:43.246+0000 7f6351fbf740 10 client.0 _ll_register_callbacks cb 0x55ee6a48ce70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T23:16:43.254 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:43.246+0000 7f634c893640 1 -- 172.21.15.49:0/2102500877 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ee6a5715e0 msgr2=0x55ee6a56e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:16:43.254 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:43.247+0000 7f634c893640 1 --2- 172.21.15.49:0/2102500877 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ee6a5715e0 0x55ee6a56e970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:43.254 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:43.247+0000 7f634c893640 1 -- 172.21.15.49:0/2102500877 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ee6a567ef0 msgr2=0x55ee6a570fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:16:43.254 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:43.247+0000 7f634c893640 1 --2- 172.21.15.49:0/2102500877 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ee6a567ef0 0x55ee6a570fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:43.254 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:43.247+0000 7f634c893640 1 -- 172.21.15.49:0/2102500877 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ee6a486c60 con 0x55ee6a568810 2024-03-18T23:16:43.254 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:43.247+0000 7f634d895640 1 --2- 172.21.15.49:0/2102500877 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ee6a5715e0 0x55ee6a56e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T23:16:43.254 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:43.247+0000 7f634d094640 1 --2- 172.21.15.49:0/2102500877 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ee6a567ef0 0x55ee6a570fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:16:43.254 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:43.247+0000 7f634c893640 1 --2- 172.21.15.49:0/2102500877 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ee6a568810 0x55ee6a56e360 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f6338012740 tx=0x7f6338012c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bea6ce4277c67779 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:16:43.254 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:43.247+0000 7f6330ff9640 10 client.16137 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-18T23:16:43.254 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:43.247+0000 7f6351fbf740 1 -- 172.21.15.49:0/2102500877 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f634409f820 con 0x55ee6a568810 2024-03-18T23:16:43.254 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:43.247+0000 7f6330ff9640 1 -- 172.21.15.49:0/2102500877 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6338004030 con 0x55ee6a568810 2024-03-18T23:16:43.255 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:43.247+0000 7f6330ff9640 1 -- 172.21.15.49:0/2102500877 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8556+0+0 (secure 0 0 0) 0x7f63380041d0 con 0x55ee6a568810 2024-03-18T23:16:43.255 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:43.247+0000 7f6330ff9640 1 -- 172.21.15.49:0/2102500877 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f6338004430 con 0x55ee6a568810 2024-03-18T23:16:43.255 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:43.247+0000 7f6351fbf740 10 client.16137 fetch_fsmap learned FSMap version 32 2024-03-18T23:16:43.255 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:43.247+0000 7f6351fbf740 1 -- 172.21.15.49:0/2102500877 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({fsmap=32}) -- 0x55ee6a1e0210 con 0x55ee6a568810 2024-03-18T23:16:43.255 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:43.247+0000 7f6330ff9640 1 -- 172.21.15.49:0/2102500877 <== mon.0 v2:172.21.15.49:3300/0 4 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f633800f920 con 0x55ee6a568810 2024-03-18T23:16:43.255 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:43.247+0000 7f6351fbf740 10 client.16137 fetch_fsmap finished waiting for FSMap version 32 2024-03-18T23:16:43.255 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:43.247+0000 7f6351fbf740 10 client.16137 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T23:16:43.255 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:43.248+0000 7f6351fbf740 20 client.16137 populate_metadata read hostname 'smithi049' 2024-03-18T23:16:43.255 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:43.248+0000 7f6351fbf740 1 --2- 172.21.15.49:0/2102500877 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x55ee6a5813f0 0x55ee6a583820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:16:43.255 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:43.248+0000 7f6351fbf740 4 client.16137 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T23:16:43.255 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:43.248+0000 7f6351fbf740 1 -- 172.21.15.49:0/2102500877 --> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f634409f820 con 0x55ee6a5813f0 2024-03-18T23:16:43.255 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:43.248+0000 7f634d094640 1 --2- 172.21.15.49:0/2102500877 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x55ee6a5813f0 0x55ee6a583820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:16:43.256 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:43.248+0000 7f634d094640 1 --2- 172.21.15.49:0/2102500877 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x55ee6a5813f0 0x55ee6a583820 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a05bc83ed2028f3e server_cookie=79ee09758da7a1c7 in_seq=0 out_seq=0 2024-03-18T23:16:43.256 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:43.248+0000 7f6330ff9640 10 client.16137 ms_handle_connect on v2:172.21.15.139:6838/4280128209 2024-03-18T23:16:43.257 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:43.250+0000 7f6330ff9640 1 -- 172.21.15.49:0/2102500877 <== mds.0 v2:172.21.15.139:6838/4280128209 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x7f634409f820 con 0x55ee6a5813f0 2024-03-18T23:16:43.257 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:43.250+0000 7f6330ff9640 10 client.16137 handle_command_reply: tid=0 2024-03-18T23:16:43.257 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:43.250+0000 7f6351fbf740 1 client.16137 shutdown 2024-03-18T23:16:43.257 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:43.250+0000 7f6351fbf740 1 -- 172.21.15.49:0/2102500877 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ee6a568810 msgr2=0x55ee6a56e360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:16:43.258 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:43.250+0000 7f6351fbf740 1 --2- 172.21.15.49:0/2102500877 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ee6a568810 0x55ee6a56e360 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f6338012740 tx=0x7f6338012c00 comp rx=0 tx=0).stop 2024-03-18T23:16:43.258 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:43.250+0000 7f6351fbf740 1 -- 172.21.15.49:0/2102500877 shutdown_connections 2024-03-18T23:16:43.258 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:43.250+0000 7f6351fbf740 1 --2- 172.21.15.49:0/2102500877 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x55ee6a5813f0 0x55ee6a583820 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:43.258 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:43.250+0000 7f6351fbf740 1 --2- 172.21.15.49:0/2102500877 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ee6a5715e0 0x55ee6a56e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:43.258 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:43.250+0000 7f6330ff9640 0 client.16137 ms_handle_reset on v2:172.21.15.139:6838/4280128209 2024-03-18T23:16:43.258 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:43.250+0000 7f6351fbf740 1 --2- 172.21.15.49:0/2102500877 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ee6a568810 0x55ee6a56e360 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:43.258 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:43.250+0000 7f6351fbf740 1 --2- 172.21.15.49:0/2102500877 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ee6a567ef0 0x55ee6a570fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:43.258 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:43.250+0000 7f6351fbf740 1 -- 172.21.15.49:0/2102500877 >> 172.21.15.49:0/2102500877 conn(0x55ee6a562810 msgr2=0x55ee6a561ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:16:43.258 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:43.250+0000 7f6351fbf740 1 -- 172.21.15.49:0/2102500877 shutdown_connections 2024-03-18T23:16:43.258 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:43.250+0000 7f6351fbf740 1 -- 172.21.15.49:0/2102500877 wait complete. 2024-03-18T23:16:43.258 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:43.250+0000 7f6351fbf740 20 client.16137 trim_cache size 0 max 16384 2024-03-18T23:16:43.267 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:43.266+0000 7f6351fbf740 1 Processor -- start 2024-03-18T23:16:43.267 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:43.266+0000 7f6351fbf740 1 -- start start 2024-03-18T23:16:43.268 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:43.266+0000 7f6351fbf740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ee6a567ef0 0x55ee6a56e320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:16:43.268 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:43.266+0000 7f6351fbf740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ee6a568810 0x55ee6a57cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:16:43.268 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:43.266+0000 7f6351fbf740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ee6a5715e0 0x55ee6a562bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:16:43.268 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:43.266+0000 7f6351fbf740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f63481af5e0 con 0x55ee6a568810 2024-03-18T23:16:43.268 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:43.266+0000 7f6351fbf740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f63481b1120 con 0x55ee6a567ef0 2024-03-18T23:16:43.268 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:43.266+0000 7f6351fbf740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f63481afcb0 con 0x55ee6a5715e0 2024-03-18T23:16:43.268 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:43.266+0000 7f634c893640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ee6a568810 0x55ee6a57cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:16:43.268 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:43.266+0000 7f634c893640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ee6a568810 0x55ee6a57cfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:46546/0 (socket says 172.21.15.49:46546) 2024-03-18T23:16:43.268 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:43.266+0000 7f634d094640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ee6a567ef0 0x55ee6a56e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:16:43.268 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:43.266+0000 7f634c893640 1 -- 172.21.15.49:0/2297193931 learned_addr learned my addr 172.21.15.49:0/2297193931 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:16:43.268 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:43.266+0000 7f634d895640 1 --2- 172.21.15.49:0/2297193931 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ee6a5715e0 0x55ee6a562bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:16:43.268 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:43.266+0000 7f634c893640 1 -- 172.21.15.49:0/2297193931 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ee6a5715e0 msgr2=0x55ee6a562bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:16:43.268 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:43.266+0000 7f634c893640 1 --2- 172.21.15.49:0/2297193931 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ee6a5715e0 0x55ee6a562bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:43.268 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:43.266+0000 7f634c893640 1 -- 172.21.15.49:0/2297193931 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ee6a567ef0 msgr2=0x55ee6a56e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:16:43.268 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:43.266+0000 7f634c893640 1 --2- 172.21.15.49:0/2297193931 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ee6a567ef0 0x55ee6a56e320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:43.269 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:43.266+0000 7f634c893640 1 -- 172.21.15.49:0/2297193931 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ee6a2ac620 con 0x55ee6a568810 2024-03-18T23:16:43.269 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:43.267+0000 7f634c893640 1 --2- 172.21.15.49:0/2297193931 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ee6a568810 0x55ee6a57cfb0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f63380098e0 tx=0x7f6338014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc3edc5494edfddb server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:16:43.269 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:43.267+0000 7f6332ffd640 1 -- 172.21.15.49:0/2297193931 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63380047a0 con 0x55ee6a568810 2024-03-18T23:16:43.269 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:43.267+0000 7f6332ffd640 1 -- 172.21.15.49:0/2297193931 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6338012e50 con 0x55ee6a568810 2024-03-18T23:16:43.269 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:43.267+0000 7f6332ffd640 1 -- 172.21.15.49:0/2297193931 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63380056e0 con 0x55ee6a568810 2024-03-18T23:16:43.273 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:43.268+0000 7f6351fbf740 1 -- 172.21.15.49:0/2297193931 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ee6a568810 msgr2=0x55ee6a57cfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:16:43.273 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:43.268+0000 7f6351fbf740 1 --2- 172.21.15.49:0/2297193931 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ee6a568810 0x55ee6a57cfb0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f63380098e0 tx=0x7f6338014000 comp rx=0 tx=0).stop 2024-03-18T23:16:43.273 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:43.268+0000 7f6351fbf740 1 -- 172.21.15.49:0/2297193931 shutdown_connections 2024-03-18T23:16:43.273 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:43.268+0000 7f6351fbf740 1 --2- 172.21.15.49:0/2297193931 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ee6a5715e0 0x55ee6a562bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:43.273 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:43.268+0000 7f6351fbf740 1 --2- 172.21.15.49:0/2297193931 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ee6a568810 0x55ee6a57cfb0 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:43.273 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:43.268+0000 7f6351fbf740 1 --2- 172.21.15.49:0/2297193931 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ee6a567ef0 0x55ee6a56e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:43.273 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:43.268+0000 7f6351fbf740 1 -- 172.21.15.49:0/2297193931 >> 172.21.15.49:0/2297193931 conn(0x55ee6a562810 msgr2=0x55ee6a561ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:16:43.273 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:43.268+0000 7f6351fbf740 1 -- 172.21.15.49:0/2297193931 shutdown_connections 2024-03-18T23:16:43.273 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:43.268+0000 7f6351fbf740 1 -- 172.21.15.49:0/2297193931 wait complete. 2024-03-18T23:16:43.273 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:43.268+0000 7f6351fbf740 1 -- start start 2024-03-18T23:16:43.273 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:43.268+0000 7f6351fbf740 1 Processor -- start 2024-03-18T23:16:43.273 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:43.269+0000 7f6351fbf740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ee6a567ef0 0x55ee6a5799b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:16:43.274 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:43.269+0000 7f6351fbf740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ee6a568810 0x55ee6a56acc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:16:43.274 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:43.269+0000 7f6351fbf740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ee6a5715e0 0x55ee6a56b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:16:43.274 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:43.269+0000 7f6351fbf740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f63481b0a50 con 0x55ee6a567ef0 2024-03-18T23:16:43.274 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:43.269+0000 7f6351fbf740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f63481b0380 con 0x55ee6a5715e0 2024-03-18T23:16:43.274 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:43.269+0000 7f6351fbf740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f63481ad5e0 con 0x55ee6a568810 2024-03-18T23:16:43.274 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:43.269+0000 7f6351fbf740 10 client.0 _ll_register_callbacks cb 0x7f6338009910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T23:16:43.274 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:43.269+0000 7f634d094640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ee6a567ef0 0x55ee6a5799b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:16:43.274 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:43.269+0000 7f634d094640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ee6a567ef0 0x55ee6a5799b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:46558/0 (socket says 172.21.15.49:46558) 2024-03-18T23:16:43.274 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:43.269+0000 7f634d094640 1 -- 172.21.15.49:0/1812686654 learned_addr learned my addr 172.21.15.49:0/1812686654 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:16:43.274 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:43.269+0000 7f634d895640 1 --2- 172.21.15.49:0/1812686654 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ee6a5715e0 0x55ee6a56b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:16:43.274 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:43.269+0000 7f634c893640 1 --2- 172.21.15.49:0/1812686654 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ee6a568810 0x55ee6a56acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:16:43.274 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:43.269+0000 7f634d094640 1 -- 172.21.15.49:0/1812686654 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ee6a568810 msgr2=0x55ee6a56acc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:16:43.274 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:43.269+0000 7f634d094640 1 --2- 172.21.15.49:0/1812686654 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ee6a568810 0x55ee6a56acc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:43.275 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:43.269+0000 7f634d094640 1 -- 172.21.15.49:0/1812686654 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ee6a5715e0 msgr2=0x55ee6a56b2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:16:43.275 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:43.269+0000 7f634d094640 1 --2- 172.21.15.49:0/1812686654 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ee6a5715e0 0x55ee6a56b2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:43.275 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:43.269+0000 7f634d094640 1 -- 172.21.15.49:0/1812686654 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ee6a56bad0 con 0x55ee6a567ef0 2024-03-18T23:16:43.275 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:43.269+0000 7f634d094640 1 --2- 172.21.15.49:0/1812686654 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ee6a567ef0 0x55ee6a5799b0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f63440cc870 tx=0x7f634409b340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9704a7ff097c3fe server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:16:43.275 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:43.269+0000 7f6330ff9640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-18T23:16:43.275 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:43.269+0000 7f6330ff9640 1 -- 172.21.15.49:0/1812686654 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6344003c50 con 0x55ee6a567ef0 2024-03-18T23:16:43.275 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:43.269+0000 7f6330ff9640 1 -- 172.21.15.49:0/1812686654 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8556+0+0 (secure 0 0 0) 0x7f63440cca40 con 0x55ee6a567ef0 2024-03-18T23:16:43.275 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:43.270+0000 7f6351fbf740 1 -- 172.21.15.49:0/1812686654 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6338012e50 con 0x55ee6a567ef0 2024-03-18T23:16:43.275 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:43.270+0000 7f6330ff9640 1 -- 172.21.15.49:0/1812686654 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f63440a3e70 con 0x55ee6a567ef0 2024-03-18T23:16:43.275 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:43.270+0000 7f6351fbf740 10 client.16143 fetch_fsmap learned FSMap version 32 2024-03-18T23:16:43.275 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:43.270+0000 7f6351fbf740 1 -- 172.21.15.49:0/1812686654 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55ee6a5e77e0 con 0x55ee6a567ef0 2024-03-18T23:16:43.275 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:43.270+0000 7f6330ff9640 1 -- 172.21.15.49:0/1812686654 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6338012e50 con 0x55ee6a567ef0 2024-03-18T23:16:43.275 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:43.270+0000 7f6330ff9640 1 -- 172.21.15.49:0/1812686654 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f634409ac20 con 0x55ee6a567ef0 2024-03-18T23:16:43.275 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:43.270+0000 7f6351fbf740 10 client.16143 fetch_fsmap finished waiting for FSMap version 32 2024-03-18T23:16:43.276 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:43.270+0000 7f6351fbf740 10 client.16143 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T23:16:43.276 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:43.270+0000 7f6351fbf740 20 client.16143 populate_metadata read hostname 'smithi049' 2024-03-18T23:16:43.276 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:43.270+0000 7f6351fbf740 1 --2- 172.21.15.49:0/1812686654 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x55ee6a5813f0 0x55ee6a5e9ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:16:43.276 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:43.270+0000 7f6351fbf740 4 client.16143 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T23:16:43.276 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:43.270+0000 7f6351fbf740 1 -- 172.21.15.49:0/1812686654 --> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ee6a4d9120 con 0x55ee6a5813f0 2024-03-18T23:16:43.276 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:43.270+0000 7f634c893640 1 --2- 172.21.15.49:0/1812686654 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x55ee6a5813f0 0x55ee6a5e9ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:16:43.276 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:43.271+0000 7f6330ff9640 10 client.16143 ms_handle_connect on v2:172.21.15.139:6838/4280128209 2024-03-18T23:16:43.276 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:43.271+0000 7f634c893640 1 --2- 172.21.15.49:0/1812686654 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x55ee6a5813f0 0x55ee6a5e9ad0 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bd85d47d446e745f server_cookie=ea0c24c35e566d39 in_seq=0 out_seq=0 2024-03-18T23:16:43.410 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:43.409+0000 7f6330ff9640 1 -- 172.21.15.49:0/1812686654 <== mds.0 v2:172.21.15.139:6838/4280128209 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ee6a4d9120 con 0x55ee6a5813f0 2024-03-18T23:16:43.411 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:43.410+0000 7f6330ff9640 10 client.16143 handle_command_reply: tid=0 2024-03-18T23:16:43.411 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:43.410+0000 7f6351fbf740 1 client.16143 shutdown 2024-03-18T23:16:43.411 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:43.410+0000 7f6351fbf740 1 -- 172.21.15.49:0/1812686654 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ee6a567ef0 msgr2=0x55ee6a5799b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:16:43.411 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:43.410+0000 7f6351fbf740 1 --2- 172.21.15.49:0/1812686654 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ee6a567ef0 0x55ee6a5799b0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f63440cc870 tx=0x7f634409b340 comp rx=0 tx=0).stop 2024-03-18T23:16:43.411 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:43.410+0000 7f6351fbf740 1 -- 172.21.15.49:0/1812686654 shutdown_connections 2024-03-18T23:16:43.411 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:43.410+0000 7f6351fbf740 1 --2- 172.21.15.49:0/1812686654 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x55ee6a5813f0 0x55ee6a5e9ad0 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:43.411 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:43.410+0000 7f6351fbf740 1 --2- 172.21.15.49:0/1812686654 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ee6a5715e0 0x55ee6a56b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:43.411 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:43.410+0000 7f6351fbf740 1 --2- 172.21.15.49:0/1812686654 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ee6a568810 0x55ee6a56acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:43.411 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:43.410+0000 7f6351fbf740 1 --2- 172.21.15.49:0/1812686654 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ee6a567ef0 0x55ee6a5799b0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:43.411 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:43.410+0000 7f6351fbf740 1 -- 172.21.15.49:0/1812686654 >> 172.21.15.49:0/1812686654 conn(0x55ee6a562810 msgr2=0x55ee6a561ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:16:43.411 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:43.410+0000 7f6330ff9640 0 client.16143 ms_handle_reset on v2:172.21.15.139:6838/4280128209 2024-03-18T23:16:43.411 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:43.410+0000 7f6351fbf740 1 -- 172.21.15.49:0/1812686654 shutdown_connections 2024-03-18T23:16:43.412 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:43.410+0000 7f6351fbf740 1 -- 172.21.15.49:0/1812686654 wait complete. 2024-03-18T23:16:43.412 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:43.410+0000 7f6351fbf740 20 client.16143 trim_cache size 0 max 16384 2024-03-18T23:16:43.412 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:{ 2024-03-18T23:16:43.412 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "message": "", 2024-03-18T23:16:43.412 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "return_code": 0 2024-03-18T23:16:43.412 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:} 2024-03-18T23:16:43.413 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:43.411+0000 7f634f2f8640 1 -- 172.21.15.49:0/910701383 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f63340a2a90 msgr2=0x7f63340a4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:16:43.413 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:43.411+0000 7f634f2f8640 1 --2- 172.21.15.49:0/910701383 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f63340a2a90 0x7f63340a4f50 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f6340005e30 tx=0x7f634002ce70 comp rx=0 tx=0).stop 2024-03-18T23:16:43.413 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:43.411+0000 7f634f2f8640 1 -- 172.21.15.49:0/910701383 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6348069470 msgr2=0x7f634819a210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:16:43.413 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:43.411+0000 7f634f2f8640 1 --2- 172.21.15.49:0/910701383 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6348069470 0x7f634819a210 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f634400ea60 tx=0x7f634400ef20 comp rx=0 tx=0).stop 2024-03-18T23:16:43.413 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:43.411+0000 7f634f2f8640 1 -- 172.21.15.49:0/910701383 shutdown_connections 2024-03-18T23:16:43.413 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:43.411+0000 7f634f2f8640 1 --2- 172.21.15.49:0/910701383 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f63340a2a90 0x7f63340a4f50 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:43.413 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:43.411+0000 7f634f2f8640 1 --2- 172.21.15.49:0/910701383 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f634810e110 0x7f634819ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:43.413 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:43.411+0000 7f634f2f8640 1 --2- 172.21.15.49:0/910701383 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6348102770 0x7f634819a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:43.413 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:43.411+0000 7f634f2f8640 1 --2- 172.21.15.49:0/910701383 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6348069470 0x7f634819a210 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:43.413 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:43.411+0000 7f634f2f8640 1 -- 172.21.15.49:0/910701383 >> 172.21.15.49:0/910701383 conn(0x7f63480774b0 msgr2=0x7f63480ff960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:16:43.413 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:43.412+0000 7f634f2f8640 1 -- 172.21.15.49:0/910701383 shutdown_connections 2024-03-18T23:16:43.414 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:43.412+0000 7f634f2f8640 1 -- 172.21.15.49:0/910701383 wait complete. 2024-03-18T23:16:43.460 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:16:43 smithi175 ceph-mon[67352]: pgmap v262: 97 pgs: 97 active+clean; 79 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 3.5 MiB/s wr, 308 op/s 2024-03-18T23:16:43.564 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:16:43 smithi139 ceph-mon[67282]: pgmap v262: 97 pgs: 97 active+clean; 79 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 3.5 MiB/s wr, 308 op/s 2024-03-18T23:16:43.629 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:16:43 smithi049 ceph-mon[62991]: pgmap v262: 97 pgs: 97 active+clean; 79 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 3.5 MiB/s wr, 308 op/s 2024-03-18T23:16:43.779 INFO:tasks.workunit.client.0.smithi049.stdout: 1 896316 9.68 MB/sec execute 345 sec latency 48.685 ms 2024-03-18T23:16:44.779 INFO:tasks.workunit.client.0.smithi049.stdout: 1 898308 9.69 MB/sec execute 346 sec latency 16.563 ms 2024-03-18T23:16:45.460 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:16:45 smithi175 ceph-mon[67352]: pgmap v263: 97 pgs: 97 active+clean; 79 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.4 MiB/s wr, 161 op/s 2024-03-18T23:16:45.564 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:16:45 smithi139 ceph-mon[67282]: pgmap v263: 97 pgs: 97 active+clean; 79 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.4 MiB/s wr, 161 op/s 2024-03-18T23:16:45.629 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:16:45 smithi049 ceph-mon[62991]: pgmap v263: 97 pgs: 97 active+clean; 79 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.4 MiB/s wr, 161 op/s 2024-03-18T23:16:45.779 INFO:tasks.workunit.client.0.smithi049.stdout: 1 899303 9.67 MB/sec execute 347 sec latency 90.896 ms 2024-03-18T23:16:46.779 INFO:tasks.workunit.client.0.smithi049.stdout: 1 901569 9.68 MB/sec execute 348 sec latency 36.474 ms 2024-03-18T23:16:47.460 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:16:47 smithi175 ceph-mon[67352]: pgmap v264: 97 pgs: 97 active+clean; 79 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.3 MiB/s wr, 152 op/s 2024-03-18T23:16:47.564 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:16:47 smithi139 ceph-mon[67282]: pgmap v264: 97 pgs: 97 active+clean; 79 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.3 MiB/s wr, 152 op/s 2024-03-18T23:16:47.629 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:16:47 smithi049 ceph-mon[62991]: pgmap v264: 97 pgs: 97 active+clean; 79 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.3 MiB/s wr, 152 op/s 2024-03-18T23:16:47.780 INFO:tasks.workunit.client.0.smithi049.stdout: 1 902470 9.67 MB/sec execute 349 sec latency 25.586 ms 2024-03-18T23:16:48.780 INFO:tasks.workunit.client.0.smithi049.stdout: 1 904349 9.66 MB/sec execute 350 sec latency 29.113 ms 2024-03-18T23:16:49.460 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:16:49 smithi175 ceph-mon[67352]: pgmap v265: 97 pgs: 97 active+clean; 72 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.4 MiB/s wr, 288 op/s 2024-03-18T23:16:49.564 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:16:49 smithi139 ceph-mon[67282]: pgmap v265: 97 pgs: 97 active+clean; 72 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.4 MiB/s wr, 288 op/s 2024-03-18T23:16:49.629 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:16:49 smithi049 ceph-mon[62991]: pgmap v265: 97 pgs: 97 active+clean; 72 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.4 MiB/s wr, 288 op/s 2024-03-18T23:16:49.780 INFO:tasks.workunit.client.0.smithi049.stdout: 1 905619 9.66 MB/sec execute 351 sec latency 18.699 ms 2024-03-18T23:16:50.780 INFO:tasks.workunit.client.0.smithi049.stdout: 1 906982 9.65 MB/sec execute 352 sec latency 63.911 ms 2024-03-18T23:16:51.460 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:16:51 smithi175 ceph-mon[67352]: pgmap v266: 97 pgs: 97 active+clean; 72 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.3 MiB/s wr, 214 op/s 2024-03-18T23:16:51.564 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:16:51 smithi139 ceph-mon[67282]: pgmap v266: 97 pgs: 97 active+clean; 72 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.3 MiB/s wr, 214 op/s 2024-03-18T23:16:51.629 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:16:51 smithi049 ceph-mon[62991]: pgmap v266: 97 pgs: 97 active+clean; 72 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.3 MiB/s wr, 214 op/s 2024-03-18T23:16:51.780 INFO:tasks.workunit.client.0.smithi049.stdout: 1 908932 9.66 MB/sec execute 353 sec latency 28.193 ms 2024-03-18T23:16:52.780 INFO:tasks.workunit.client.0.smithi049.stdout: 1 910101 9.64 MB/sec execute 354 sec latency 23.837 ms 2024-03-18T23:16:53.460 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:16:53 smithi175 ceph-mon[67352]: pgmap v267: 97 pgs: 97 active+clean; 77 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.4 MiB/s wr, 288 op/s 2024-03-18T23:16:53.564 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:16:53 smithi139 ceph-mon[67282]: pgmap v267: 97 pgs: 97 active+clean; 77 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.4 MiB/s wr, 288 op/s 2024-03-18T23:16:53.629 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:16:53 smithi049 ceph-mon[62991]: pgmap v267: 97 pgs: 97 active+clean; 77 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.4 MiB/s wr, 288 op/s 2024-03-18T23:16:53.780 INFO:tasks.workunit.client.0.smithi049.stdout: 1 912040 9.65 MB/sec execute 355 sec latency 26.592 ms 2024-03-18T23:16:54.780 INFO:tasks.workunit.client.0.smithi049.stdout: 1 913001 9.64 MB/sec execute 356 sec latency 16.625 ms 2024-03-18T23:16:55.431 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:16:55 smithi049 ceph-mon[62991]: pgmap v268: 97 pgs: 97 active+clean; 77 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.2 MiB/s wr, 212 op/s 2024-03-18T23:16:55.460 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:16:55 smithi175 ceph-mon[67352]: pgmap v268: 97 pgs: 97 active+clean; 77 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.2 MiB/s wr, 212 op/s 2024-03-18T23:16:55.564 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:16:55 smithi139 ceph-mon[67282]: pgmap v268: 97 pgs: 97 active+clean; 77 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.2 MiB/s wr, 212 op/s 2024-03-18T23:16:55.781 INFO:tasks.workunit.client.0.smithi049.stdout: 1 914780 9.63 MB/sec execute 357 sec latency 98.617 ms 2024-03-18T23:16:56.506 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:56.504+0000 7f5398513640 1 -- 172.21.15.49:0/1233424 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f53901021b0 msgr2=0x7f539010e6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:16:56.506 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:56.504+0000 7f5398513640 1 --2- 172.21.15.49:0/1233424 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f53901021b0 0x7f539010e6c0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f5388009cd0 tx=0x7f538802c0f0 comp rx=0 tx=0).stop 2024-03-18T23:16:56.506 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:56.504+0000 7f5398513640 1 -- 172.21.15.49:0/1233424 shutdown_connections 2024-03-18T23:16:56.506 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:56.504+0000 7f5398513640 1 --2- 172.21.15.49:0/1233424 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f53901021b0 0x7f539010e6c0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:56.506 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:56.504+0000 7f5398513640 1 --2- 172.21.15.49:0/1233424 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f53901017f0 0x7f5390101c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:56.506 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:56.504+0000 7f5398513640 1 --2- 172.21.15.49:0/1233424 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5390107800 0x7f5390107be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:56.506 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:56.504+0000 7f5398513640 1 -- 172.21.15.49:0/1233424 >> 172.21.15.49:0/1233424 conn(0x7f53900fd300 msgr2=0x7f53900ff720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:16:56.506 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:56.504+0000 7f5398513640 1 -- 172.21.15.49:0/1233424 shutdown_connections 2024-03-18T23:16:56.506 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:56.504+0000 7f5398513640 1 -- 172.21.15.49:0/1233424 wait complete. 2024-03-18T23:16:56.507 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:56.505+0000 7f5398513640 1 Processor -- start 2024-03-18T23:16:56.507 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:56.505+0000 7f5398513640 1 -- start start 2024-03-18T23:16:56.507 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:56.505+0000 7f5398513640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f53901017f0 0x7f539019a210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:16:56.507 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:56.505+0000 7f5398513640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f53901021b0 0x7f539019a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:16:56.507 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:56.505+0000 7f5398513640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5390107800 0x7f539019ead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:16:56.507 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:56.505+0000 7f5398513640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f5390111c10 con 0x7f53901017f0 2024-03-18T23:16:56.507 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:56.505+0000 7f5398513640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f5390111a90 con 0x7f53901021b0 2024-03-18T23:16:56.507 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:56.505+0000 7f5398513640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f5390111d90 con 0x7f5390107800 2024-03-18T23:16:56.507 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:56.505+0000 7f53962af640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f53901017f0 0x7f539019a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:16:56.507 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:56.505+0000 7f53962af640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f53901017f0 0x7f539019a210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:36604/0 (socket says 172.21.15.49:36604) 2024-03-18T23:16:56.507 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:56.505+0000 7f53962af640 1 -- 172.21.15.49:0/2015360123 learned_addr learned my addr 172.21.15.49:0/2015360123 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:16:56.507 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:56.505+0000 7f5395aae640 1 --2- 172.21.15.49:0/2015360123 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f53901021b0 0x7f539019a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:16:56.507 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:56.505+0000 7f5396ab0640 1 --2- 172.21.15.49:0/2015360123 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5390107800 0x7f539019ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:16:56.508 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:56.505+0000 7f5395aae640 1 -- 172.21.15.49:0/2015360123 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5390107800 msgr2=0x7f539019ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:16:56.508 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:56.506+0000 7f5395aae640 1 --2- 172.21.15.49:0/2015360123 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5390107800 0x7f539019ead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:56.508 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:56.506+0000 7f5395aae640 1 -- 172.21.15.49:0/2015360123 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f53901017f0 msgr2=0x7f539019a210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:16:56.508 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:56.506+0000 7f5395aae640 1 --2- 172.21.15.49:0/2015360123 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f53901017f0 0x7f539019a210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:56.508 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:56.506+0000 7f5395aae640 1 -- 172.21.15.49:0/2015360123 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f539019f200 con 0x7f53901021b0 2024-03-18T23:16:56.508 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:56.506+0000 7f53962af640 1 --2- 172.21.15.49:0/2015360123 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f53901017f0 0x7f539019a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:16:56.508 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:56.506+0000 7f5395aae640 1 --2- 172.21.15.49:0/2015360123 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f53901021b0 0x7f539019a750 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f538000e9f0 tx=0x7f538000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:16:56.508 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:56.506+0000 7f53877fe640 1 -- 172.21.15.49:0/2015360123 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f538000ccc0 con 0x7f53901021b0 2024-03-18T23:16:56.508 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:56.506+0000 7f5398513640 1 -- 172.21.15.49:0/2015360123 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f539019f4f0 con 0x7f53901021b0 2024-03-18T23:16:56.508 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:56.506+0000 7f53877fe640 1 -- 172.21.15.49:0/2015360123 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5380010910 con 0x7f53901021b0 2024-03-18T23:16:56.508 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:56.506+0000 7f53877fe640 1 -- 172.21.15.49:0/2015360123 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5380018960 con 0x7f53901021b0 2024-03-18T23:16:56.508 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:56.506+0000 7f5398513640 1 -- 172.21.15.49:0/2015360123 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53901a6d80 con 0x7f53901021b0 2024-03-18T23:16:56.516 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:56.508+0000 7f53877fe640 1 -- 172.21.15.49:0/2015360123 <== mon.2 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 25) ==== 137421+0+0 (secure 0 0 0) 0x7f5380010ac0 con 0x7f53901021b0 2024-03-18T23:16:56.516 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:56.508+0000 7f539b1d0740 1 Processor -- start 2024-03-18T23:16:56.516 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:56.509+0000 7f539b1d0740 1 -- start start 2024-03-18T23:16:56.516 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:56.509+0000 7f539b1d0740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56162d8e3ef0 0x56162d8e42d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:16:56.516 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:56.509+0000 7f539b1d0740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56162d8e4810 0x56162d8ed0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:16:56.516 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:56.509+0000 7f539b1d0740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56162d8ed5e0 0x56162d8ef9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:16:56.516 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:56.509+0000 7f539b1d0740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x56162d847be0 con 0x56162d8ed5e0 2024-03-18T23:16:56.517 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:56.509+0000 7f539b1d0740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x56162d7f0960 con 0x56162d8e4810 2024-03-18T23:16:56.517 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:56.509+0000 7f539b1d0740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x56162d843c80 con 0x56162d8e3ef0 2024-03-18T23:16:56.517 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:56.509+0000 7f53962af640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56162d8e4810 0x56162d8ed0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:16:56.517 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:56.509+0000 7f5396ab0640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56162d8e3ef0 0x56162d8e42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:16:56.517 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:56.509+0000 7f5395aae640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56162d8ed5e0 0x56162d8ef9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:16:56.517 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:56.509+0000 7f53962af640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56162d8e4810 0x56162d8ed0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.49:41882/0 (socket says 172.21.15.49:41882) 2024-03-18T23:16:56.517 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:56.509+0000 7f5396ab0640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56162d8e3ef0 0x56162d8e42d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.49:47328/0 (socket says 172.21.15.49:47328) 2024-03-18T23:16:56.517 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:56.509+0000 7f53962af640 1 -- 172.21.15.49:0/1903175334 learned_addr learned my addr 172.21.15.49:0/1903175334 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:16:56.517 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:56.509+0000 7f53877fe640 1 --2- 172.21.15.49:0/2015360123 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f537c0a2a10 0x7f537c0a4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:16:56.517 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:56.509+0000 7f5396ab0640 1 --2- 172.21.15.49:0/2015360123 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f537c0a2a10 0x7f537c0a4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:16:56.517 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:56.509+0000 7f53962af640 1 -- 172.21.15.49:0/1903175334 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56162d8e3ef0 msgr2=0x56162d8e42d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:16:56.517 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:56.509+0000 7f53962af640 1 --2- 172.21.15.49:0/1903175334 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56162d8e3ef0 0x56162d8e42d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:56.517 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:56.509+0000 7f53962af640 1 -- 172.21.15.49:0/1903175334 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56162d8ed5e0 msgr2=0x56162d8ef9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:16:56.518 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:56.509+0000 7f53962af640 1 --2- 172.21.15.49:0/1903175334 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56162d8ed5e0 0x56162d8ef9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:56.518 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:56.510+0000 7f53962af640 1 -- 172.21.15.49:0/1903175334 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56162d7e7d20 con 0x56162d8e4810 2024-03-18T23:16:56.518 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:56.510+0000 7f53877fe640 1 -- 172.21.15.49:0/2015360123 <== mon.2 v2:172.21.15.139:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8556+0+0 (secure 0 0 0) 0x7f53800cf240 con 0x7f53901021b0 2024-03-18T23:16:56.518 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:56.510+0000 7f53962af640 1 --2- 172.21.15.49:0/1903175334 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56162d8e4810 0x56162d8ed0a0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f538c0115f0 tx=0x7f538c011ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4cf5f4409b16f1af server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:16:56.518 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:56.510+0000 7f5396ab0640 1 --2- 172.21.15.49:0/2015360123 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f537c0a2a10 0x7f537c0a4ed0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f5388005cc0 tx=0x7f538802c0f0 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:16:56.518 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:56.510+0000 7f5384bfe640 1 -- 172.21.15.49:0/1903175334 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f538c01e070 con 0x56162d8e4810 2024-03-18T23:16:56.518 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:56.510+0000 7f5384bfe640 1 -- 172.21.15.49:0/1903175334 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f538c00fdd0 con 0x56162d8e4810 2024-03-18T23:16:56.518 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:56.510+0000 7f5384bfe640 1 -- 172.21.15.49:0/1903175334 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f538c00e3b0 con 0x56162d8e4810 2024-03-18T23:16:56.518 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:56.511+0000 7f539b1d0740 1 -- 172.21.15.49:0/1903175334 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56162d8e4810 msgr2=0x56162d8ed0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:16:56.518 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:56.511+0000 7f539b1d0740 1 --2- 172.21.15.49:0/1903175334 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56162d8e4810 0x56162d8ed0a0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f538c0115f0 tx=0x7f538c011ab0 comp rx=0 tx=0).stop 2024-03-18T23:16:56.519 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:56.511+0000 7f539b1d0740 1 -- 172.21.15.49:0/1903175334 shutdown_connections 2024-03-18T23:16:56.519 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:56.511+0000 7f539b1d0740 1 --2- 172.21.15.49:0/1903175334 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56162d8ed5e0 0x56162d8ef9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:56.519 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:56.511+0000 7f539b1d0740 1 --2- 172.21.15.49:0/1903175334 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56162d8e4810 0x56162d8ed0a0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:56.519 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:56.511+0000 7f539b1d0740 1 --2- 172.21.15.49:0/1903175334 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56162d8e3ef0 0x56162d8e42d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:56.519 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:56.511+0000 7f539b1d0740 1 -- 172.21.15.49:0/1903175334 >> 172.21.15.49:0/1903175334 conn(0x56162d8de810 msgr2=0x56162d8debf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:16:56.519 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:56.511+0000 7f539b1d0740 1 -- 172.21.15.49:0/1903175334 shutdown_connections 2024-03-18T23:16:56.519 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:56.511+0000 7f539b1d0740 1 -- 172.21.15.49:0/1903175334 wait complete. 2024-03-18T23:16:56.519 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:56.511+0000 7f539b1d0740 1 -- start start 2024-03-18T23:16:56.519 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:56.511+0000 7f539b1d0740 1 Processor -- start 2024-03-18T23:16:56.519 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:56.511+0000 7f539b1d0740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56162d8e3ef0 0x56162d8ecfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:16:56.519 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:56.511+0000 7f539b1d0740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56162d8e4810 0x56162d8ea330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:16:56.519 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:56.511+0000 7f539b1d0740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56162d8ed5e0 0x56162d8ea940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:16:56.520 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:56.511+0000 7f539b1d0740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x56162d8f0210 con 0x56162d8e4810 2024-03-18T23:16:56.520 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:56.512+0000 7f539b1d0740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x56162d803db0 con 0x56162d8ed5e0 2024-03-18T23:16:56.520 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:56.512+0000 7f539b1d0740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x56162d8f0390 con 0x56162d8e3ef0 2024-03-18T23:16:56.520 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:56.512+0000 7f539b1d0740 10 client.0 _ll_register_callbacks cb 0x56162d808e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T23:16:56.520 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:56.512+0000 7f53962af640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56162d8e3ef0 0x56162d8ecfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:16:56.520 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:56.512+0000 7f5395aae640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56162d8e4810 0x56162d8ea330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:16:56.520 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:56.512+0000 7f53962af640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56162d8e3ef0 0x56162d8ecfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.49:47342/0 (socket says 172.21.15.49:47342) 2024-03-18T23:16:56.520 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:56.512+0000 7f53962af640 1 -- 172.21.15.49:0/1947935273 learned_addr learned my addr 172.21.15.49:0/1947935273 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:16:56.520 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:56.512+0000 7f5396ab0640 1 --2- 172.21.15.49:0/1947935273 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56162d8ed5e0 0x56162d8ea940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:16:56.520 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:56.512+0000 7f53962af640 1 -- 172.21.15.49:0/1947935273 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56162d8ed5e0 msgr2=0x56162d8ea940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:16:56.520 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:56.512+0000 7f53962af640 1 --2- 172.21.15.49:0/1947935273 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56162d8ed5e0 0x56162d8ea940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:56.520 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:56.512+0000 7f53962af640 1 -- 172.21.15.49:0/1947935273 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56162d8e4810 msgr2=0x56162d8ea330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:16:56.520 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:56.512+0000 7f53962af640 1 --2- 172.21.15.49:0/1947935273 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56162d8e4810 0x56162d8ea330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:56.521 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:56.512+0000 7f53962af640 1 -- 172.21.15.49:0/1947935273 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56162d802c60 con 0x56162d8e3ef0 2024-03-18T23:16:56.521 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:56.512+0000 7f5395aae640 1 --2- 172.21.15.49:0/1947935273 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56162d8e4810 0x56162d8ea330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:16:56.521 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:56.512+0000 7f53962af640 1 --2- 172.21.15.49:0/1947935273 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56162d8e3ef0 0x56162d8ecfa0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f538c011fd0 tx=0x7f538c019660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e9d4c8c67a435ede server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:16:56.521 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:56.512+0000 7f5379ffb640 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-03-18T23:16:56.521 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:56.512+0000 7f5379ffb640 1 -- 172.21.15.49:0/1947935273 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f538c01e050 con 0x56162d8e3ef0 2024-03-18T23:16:56.521 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:56.512+0000 7f539b1d0740 1 -- 172.21.15.49:0/1947935273 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f538c00fdd0 con 0x56162d8e3ef0 2024-03-18T23:16:56.521 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:56.512+0000 7f5379ffb640 1 -- 172.21.15.49:0/1947935273 <== mon.1 v2:172.21.15.175:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8556+0+0 (secure 0 0 0) 0x7f538c00fad0 con 0x56162d8e3ef0 2024-03-18T23:16:56.521 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:56.513+0000 7f5379ffb640 1 -- 172.21.15.49:0/1947935273 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f538c033050 con 0x56162d8e3ef0 2024-03-18T23:16:56.521 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:56.513+0000 7f539b1d0740 10 client.25954 fetch_fsmap learned FSMap version 32 2024-03-18T23:16:56.521 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:56.513+0000 7f539b1d0740 1 -- 172.21.15.49:0/1947935273 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x56162d55c210 con 0x56162d8e3ef0 2024-03-18T23:16:56.521 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:56.513+0000 7f5379ffb640 1 -- 172.21.15.49:0/1947935273 <== mon.1 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f538c00fdd0 con 0x56162d8e3ef0 2024-03-18T23:16:56.521 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:56.513+0000 7f5379ffb640 1 -- 172.21.15.49:0/1947935273 <== mon.1 v2:172.21.15.175:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f538c004990 con 0x56162d8e3ef0 2024-03-18T23:16:56.521 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:56.513+0000 7f539b1d0740 10 client.25954 fetch_fsmap finished waiting for FSMap version 32 2024-03-18T23:16:56.522 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:56.513+0000 7f539b1d0740 10 client.25954 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T23:16:56.522 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:56.513+0000 7f539b1d0740 20 client.25954 populate_metadata read hostname 'smithi049' 2024-03-18T23:16:56.522 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:56.513+0000 7f539b1d0740 1 --2- 172.21.15.49:0/1947935273 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x56162d8fd430 0x56162d8ff820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:16:56.522 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:56.513+0000 7f539b1d0740 4 client.25954 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T23:16:56.522 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:56.513+0000 7f539b1d0740 1 -- 172.21.15.49:0/1947935273 --> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56162d855120 con 0x56162d8fd430 2024-03-18T23:16:56.522 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:56.513+0000 7f5395aae640 1 --2- 172.21.15.49:0/1947935273 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x56162d8fd430 0x56162d8ff820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:16:56.522 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:56.514+0000 7f5379ffb640 10 client.25954 ms_handle_connect on v2:172.21.15.139:6838/4280128209 2024-03-18T23:16:56.522 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:56.514+0000 7f5395aae640 1 --2- 172.21.15.49:0/1947935273 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x56162d8fd430 0x56162d8ff820 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1fd6d346a236c9d6 server_cookie=298a2f61b8314497 in_seq=0 out_seq=0 2024-03-18T23:16:56.522 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:56.515+0000 7f5379ffb640 1 -- 172.21.15.49:0/1947935273 <== mds.0 v2:172.21.15.139:6838/4280128209 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x56162d855120 con 0x56162d8fd430 2024-03-18T23:16:56.522 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:56.515+0000 7f5379ffb640 10 client.25954 handle_command_reply: tid=0 2024-03-18T23:16:56.522 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:56.515+0000 7f539b1d0740 1 client.25954 shutdown 2024-03-18T23:16:56.523 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:56.515+0000 7f539b1d0740 1 -- 172.21.15.49:0/1947935273 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56162d8e3ef0 msgr2=0x56162d8ecfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:16:56.523 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:56.515+0000 7f539b1d0740 1 --2- 172.21.15.49:0/1947935273 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56162d8e3ef0 0x56162d8ecfa0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f538c011fd0 tx=0x7f538c019660 comp rx=0 tx=0).stop 2024-03-18T23:16:56.523 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:56.516+0000 7f539b1d0740 1 -- 172.21.15.49:0/1947935273 shutdown_connections 2024-03-18T23:16:56.523 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:56.516+0000 7f539b1d0740 1 --2- 172.21.15.49:0/1947935273 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x56162d8fd430 0x56162d8ff820 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:56.523 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:56.516+0000 7f539b1d0740 1 --2- 172.21.15.49:0/1947935273 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56162d8ed5e0 0x56162d8ea940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:56.523 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:56.516+0000 7f5379ffb640 0 client.25954 ms_handle_reset on v2:172.21.15.139:6838/4280128209 2024-03-18T23:16:56.523 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:56.516+0000 7f539b1d0740 1 --2- 172.21.15.49:0/1947935273 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56162d8e4810 0x56162d8ea330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:56.523 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:56.516+0000 7f539b1d0740 1 --2- 172.21.15.49:0/1947935273 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56162d8e3ef0 0x56162d8ecfa0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:56.523 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:56.516+0000 7f539b1d0740 1 -- 172.21.15.49:0/1947935273 >> 172.21.15.49:0/1947935273 conn(0x56162d8de810 msgr2=0x56162d8e0560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:16:56.523 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:56.516+0000 7f539b1d0740 1 -- 172.21.15.49:0/1947935273 shutdown_connections 2024-03-18T23:16:56.523 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:56.516+0000 7f539b1d0740 1 -- 172.21.15.49:0/1947935273 wait complete. 2024-03-18T23:16:56.523 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:56.516+0000 7f539b1d0740 20 client.25954 trim_cache size 0 max 16384 2024-03-18T23:16:56.534 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:56.531+0000 7f539b1d0740 1 Processor -- start 2024-03-18T23:16:56.535 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:56.531+0000 7f539b1d0740 1 -- start start 2024-03-18T23:16:56.535 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:56.531+0000 7f539b1d0740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56162d8e3ef0 0x56162d8f8fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:16:56.535 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:56.531+0000 7f539b1d0740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56162d8e4810 0x56162d904f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:16:56.535 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:56.531+0000 7f539b1d0740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56162d8ed5e0 0x56162d8eca60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:16:56.535 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:56.531+0000 7f539b1d0740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f53901af6d0 con 0x56162d8ed5e0 2024-03-18T23:16:56.535 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:56.531+0000 7f539b1d0740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f53901b1210 con 0x56162d8e3ef0 2024-03-18T23:16:56.535 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:56.531+0000 7f539b1d0740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f53901afda0 con 0x56162d8e4810 2024-03-18T23:16:56.535 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:56.531+0000 7f5396ab0640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56162d8ed5e0 0x56162d8eca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:16:56.535 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:56.531+0000 7f5396ab0640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56162d8ed5e0 0x56162d8eca60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:36628/0 (socket says 172.21.15.49:36628) 2024-03-18T23:16:56.535 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:56.531+0000 7f5396ab0640 1 -- 172.21.15.49:0/555288732 learned_addr learned my addr 172.21.15.49:0/555288732 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:16:56.535 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:56.531+0000 7f53962af640 1 --2- 172.21.15.49:0/555288732 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56162d8e3ef0 0x56162d8f8fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:16:56.535 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:56.531+0000 7f5395aae640 1 --2- 172.21.15.49:0/555288732 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56162d8e4810 0x56162d904f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:16:56.535 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:56.531+0000 7f5396ab0640 1 -- 172.21.15.49:0/555288732 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56162d8e4810 msgr2=0x56162d904f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:16:56.535 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:56.531+0000 7f5396ab0640 1 --2- 172.21.15.49:0/555288732 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56162d8e4810 0x56162d904f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:56.536 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:56.531+0000 7f5396ab0640 1 -- 172.21.15.49:0/555288732 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56162d8e3ef0 msgr2=0x56162d8f8fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:16:56.536 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:56.531+0000 7f5396ab0640 1 --2- 172.21.15.49:0/555288732 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56162d8e3ef0 0x56162d8f8fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:56.536 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:56.531+0000 7f5396ab0640 1 -- 172.21.15.49:0/555288732 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56162d628620 con 0x56162d8ed5e0 2024-03-18T23:16:56.536 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:56.531+0000 7f5395aae640 1 --2- 172.21.15.49:0/555288732 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56162d8e4810 0x56162d904f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:16:56.536 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:56.531+0000 7f5396ab0640 1 --2- 172.21.15.49:0/555288732 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56162d8ed5e0 0x56162d8eca60 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f5388009cd0 tx=0x7f5388005f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dab8ed4cdae6701a server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:16:56.536 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:56.532+0000 7f537bfff640 1 -- 172.21.15.49:0/555288732 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5388039040 con 0x56162d8ed5e0 2024-03-18T23:16:56.536 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:56.532+0000 7f537bfff640 1 -- 172.21.15.49:0/555288732 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5388054070 con 0x56162d8ed5e0 2024-03-18T23:16:56.536 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:56.532+0000 7f537bfff640 1 -- 172.21.15.49:0/555288732 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f538804f450 con 0x56162d8ed5e0 2024-03-18T23:16:56.536 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:56.532+0000 7f539b1d0740 1 -- 172.21.15.49:0/555288732 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56162d8ed5e0 msgr2=0x56162d8eca60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:16:56.536 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:56.532+0000 7f539b1d0740 1 --2- 172.21.15.49:0/555288732 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56162d8ed5e0 0x56162d8eca60 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f5388009cd0 tx=0x7f5388005f70 comp rx=0 tx=0).stop 2024-03-18T23:16:56.536 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:56.532+0000 7f539b1d0740 1 -- 172.21.15.49:0/555288732 shutdown_connections 2024-03-18T23:16:56.536 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:56.532+0000 7f539b1d0740 1 --2- 172.21.15.49:0/555288732 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56162d8ed5e0 0x56162d8eca60 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:56.536 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:56.532+0000 7f539b1d0740 1 --2- 172.21.15.49:0/555288732 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56162d8e4810 0x56162d904f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:56.536 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:56.532+0000 7f539b1d0740 1 --2- 172.21.15.49:0/555288732 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56162d8e3ef0 0x56162d8f8fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:56.537 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:56.532+0000 7f539b1d0740 1 -- 172.21.15.49:0/555288732 >> 172.21.15.49:0/555288732 conn(0x56162d8de810 msgr2=0x56162d8ddab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:16:56.537 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:56.532+0000 7f539b1d0740 1 -- 172.21.15.49:0/555288732 shutdown_connections 2024-03-18T23:16:56.537 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:56.532+0000 7f539b1d0740 1 -- 172.21.15.49:0/555288732 wait complete. 2024-03-18T23:16:56.537 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:56.533+0000 7f539b1d0740 1 -- start start 2024-03-18T23:16:56.537 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:56.533+0000 7f539b1d0740 1 Processor -- start 2024-03-18T23:16:56.537 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:56.533+0000 7f539b1d0740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56162d8e3ef0 0x56162d8f57e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:16:56.537 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:56.533+0000 7f539b1d0740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56162d8e4810 0x56162d8f0b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:16:56.537 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:56.533+0000 7f539b1d0740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56162d8ed5e0 0x56162d8f10c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:16:56.537 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:56.533+0000 7f539b1d0740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f53901b0b40 con 0x56162d8e3ef0 2024-03-18T23:16:56.537 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:56.533+0000 7f539b1d0740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f53901b0470 con 0x56162d8ed5e0 2024-03-18T23:16:56.537 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:56.533+0000 7f539b1d0740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f53901ad6d0 con 0x56162d8e4810 2024-03-18T23:16:56.537 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:56.533+0000 7f539b1d0740 10 client.0 _ll_register_callbacks cb 0x7f538c01c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T23:16:56.537 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:56.533+0000 7f53962af640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56162d8e3ef0 0x56162d8f57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:16:56.537 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:56.533+0000 7f53962af640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56162d8e3ef0 0x56162d8f57e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:36642/0 (socket says 172.21.15.49:36642) 2024-03-18T23:16:56.537 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:56.533+0000 7f53962af640 1 -- 172.21.15.49:0/2120243873 learned_addr learned my addr 172.21.15.49:0/2120243873 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:16:56.538 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:56.533+0000 7f5395aae640 1 --2- 172.21.15.49:0/2120243873 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56162d8e4810 0x56162d8f0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:16:56.538 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:56.533+0000 7f5396ab0640 1 --2- 172.21.15.49:0/2120243873 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56162d8ed5e0 0x56162d8f10c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:16:56.538 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:56.534+0000 7f5395aae640 1 -- 172.21.15.49:0/2120243873 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56162d8ed5e0 msgr2=0x56162d8f10c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:16:56.538 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:56.534+0000 7f5395aae640 1 --2- 172.21.15.49:0/2120243873 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56162d8ed5e0 0x56162d8f10c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:56.538 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:56.534+0000 7f5395aae640 1 -- 172.21.15.49:0/2120243873 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56162d8e3ef0 msgr2=0x56162d8f57e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:16:56.538 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:56.534+0000 7f5395aae640 1 --2- 172.21.15.49:0/2120243873 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56162d8e3ef0 0x56162d8f57e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:56.538 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:56.534+0000 7f5395aae640 1 -- 172.21.15.49:0/2120243873 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56162d8f18c0 con 0x56162d8e4810 2024-03-18T23:16:56.538 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:56.534+0000 7f5396ab0640 1 --2- 172.21.15.49:0/2120243873 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56162d8ed5e0 0x56162d8f10c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:16:56.538 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:56.534+0000 7f53962af640 1 --2- 172.21.15.49:0/2120243873 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56162d8e3ef0 0x56162d8f57e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:16:56.538 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:56.534+0000 7f5395aae640 1 --2- 172.21.15.49:0/2120243873 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56162d8e4810 0x56162d8f0b80 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f53800cc970 tx=0x7f5380086040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=98dac231a09e20ca server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:16:56.538 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:56.534+0000 7f5379ffb640 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-03-18T23:16:56.538 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:56.534+0000 7f5379ffb640 1 -- 172.21.15.49:0/2120243873 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f538009b930 con 0x56162d8e4810 2024-03-18T23:16:56.539 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:56.534+0000 7f5379ffb640 1 -- 172.21.15.49:0/2120243873 <== mon.1 v2:172.21.15.175:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8556+0+0 (secure 0 0 0) 0x7f53800a3b90 con 0x56162d8e4810 2024-03-18T23:16:56.539 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:56.534+0000 7f539b1d0740 1 -- 172.21.15.49:0/2120243873 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5388054070 con 0x56162d8e4810 2024-03-18T23:16:56.539 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:56.534+0000 7f5379ffb640 1 -- 172.21.15.49:0/2120243873 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f538009f9d0 con 0x56162d8e4810 2024-03-18T23:16:56.539 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:56.534+0000 7f539b1d0740 10 client.25960 fetch_fsmap learned FSMap version 32 2024-03-18T23:16:56.539 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:56.534+0000 7f539b1d0740 1 -- 172.21.15.49:0/2120243873 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x56162d9635a0 con 0x56162d8e4810 2024-03-18T23:16:56.539 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:56.534+0000 7f5379ffb640 1 -- 172.21.15.49:0/2120243873 <== mon.1 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5388054070 con 0x56162d8e4810 2024-03-18T23:16:56.539 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:56.535+0000 7f5379ffb640 1 -- 172.21.15.49:0/2120243873 <== mon.1 v2:172.21.15.175:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f53800d2b60 con 0x56162d8e4810 2024-03-18T23:16:56.539 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:56.535+0000 7f539b1d0740 10 client.25960 fetch_fsmap finished waiting for FSMap version 32 2024-03-18T23:16:56.539 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:56.535+0000 7f539b1d0740 10 client.25960 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T23:16:56.539 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:56.535+0000 7f539b1d0740 20 client.25960 populate_metadata read hostname 'smithi049' 2024-03-18T23:16:56.540 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:56.535+0000 7f539b1d0740 1 --2- 172.21.15.49:0/2120243873 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x56162d963cd0 0x56162d9660f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:16:56.540 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:56.535+0000 7f539b1d0740 4 client.25960 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T23:16:56.540 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:56.535+0000 7f539b1d0740 1 -- 172.21.15.49:0/2120243873 --> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56162d84f470 con 0x56162d963cd0 2024-03-18T23:16:56.540 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:56.535+0000 7f53962af640 1 --2- 172.21.15.49:0/2120243873 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x56162d963cd0 0x56162d9660f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:16:56.540 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:56.535+0000 7f53962af640 1 --2- 172.21.15.49:0/2120243873 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x56162d963cd0 0x56162d9660f0 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=42e66453bbf105db server_cookie=f207851dcf4e9f12 in_seq=0 out_seq=0 2024-03-18T23:16:56.540 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:56.535+0000 7f5379ffb640 10 client.25960 ms_handle_connect on v2:172.21.15.139:6838/4280128209 2024-03-18T23:16:56.688 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:56.688+0000 7f5379ffb640 1 -- 172.21.15.49:0/2120243873 <== mds.0 v2:172.21.15.139:6838/4280128209 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56162d84f470 con 0x56162d963cd0 2024-03-18T23:16:56.689 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:56.688+0000 7f5379ffb640 10 client.25960 handle_command_reply: tid=0 2024-03-18T23:16:56.689 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:56.689+0000 7f539b1d0740 1 client.25960 shutdown 2024-03-18T23:16:56.689 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:56.689+0000 7f539b1d0740 1 -- 172.21.15.49:0/2120243873 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56162d8e4810 msgr2=0x56162d8f0b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:16:56.689 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:56.689+0000 7f539b1d0740 1 --2- 172.21.15.49:0/2120243873 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56162d8e4810 0x56162d8f0b80 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f53800cc970 tx=0x7f5380086040 comp rx=0 tx=0).stop 2024-03-18T23:16:56.689 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:56.689+0000 7f539b1d0740 1 -- 172.21.15.49:0/2120243873 shutdown_connections 2024-03-18T23:16:56.689 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:56.689+0000 7f539b1d0740 1 --2- 172.21.15.49:0/2120243873 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x56162d963cd0 0x56162d9660f0 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:56.689 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:56.689+0000 7f5379ffb640 0 client.25960 ms_handle_reset on v2:172.21.15.139:6838/4280128209 2024-03-18T23:16:56.689 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:56.689+0000 7f539b1d0740 1 --2- 172.21.15.49:0/2120243873 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56162d8ed5e0 0x56162d8f10c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:56.689 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:56.689+0000 7f539b1d0740 1 --2- 172.21.15.49:0/2120243873 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56162d8e4810 0x56162d8f0b80 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:56.689 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:56.689+0000 7f539b1d0740 1 --2- 172.21.15.49:0/2120243873 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56162d8e3ef0 0x56162d8f57e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:56.689 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:56.689+0000 7f539b1d0740 1 -- 172.21.15.49:0/2120243873 >> 172.21.15.49:0/2120243873 conn(0x56162d8de810 msgr2=0x56162d8df810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:16:56.689 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:56.689+0000 7f539b1d0740 1 -- 172.21.15.49:0/2120243873 shutdown_connections 2024-03-18T23:16:56.690 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:56.689+0000 7f539b1d0740 1 -- 172.21.15.49:0/2120243873 wait complete. 2024-03-18T23:16:56.690 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:56.689+0000 7f539b1d0740 20 client.25960 trim_cache size 0 max 16384 2024-03-18T23:16:56.690 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:{ 2024-03-18T23:16:56.690 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "message": "", 2024-03-18T23:16:56.690 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "return_code": 0 2024-03-18T23:16:56.690 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:} 2024-03-18T23:16:56.691 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:56.690+0000 7f5398513640 1 -- 172.21.15.49:0/2015360123 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f537c0a2a10 msgr2=0x7f537c0a4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:16:56.691 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:56.690+0000 7f5398513640 1 --2- 172.21.15.49:0/2015360123 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f537c0a2a10 0x7f537c0a4ed0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f5388005cc0 tx=0x7f538802c0f0 comp rx=0 tx=0).stop 2024-03-18T23:16:56.691 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:56.690+0000 7f5398513640 1 -- 172.21.15.49:0/2015360123 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f53901021b0 msgr2=0x7f539019a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:16:56.691 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:56.690+0000 7f5398513640 1 --2- 172.21.15.49:0/2015360123 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f53901021b0 0x7f539019a750 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f538000e9f0 tx=0x7f538000eeb0 comp rx=0 tx=0).stop 2024-03-18T23:16:56.691 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:56.690+0000 7f5398513640 1 -- 172.21.15.49:0/2015360123 shutdown_connections 2024-03-18T23:16:56.691 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:56.690+0000 7f5398513640 1 --2- 172.21.15.49:0/2015360123 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f537c0a2a10 0x7f537c0a4ed0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:56.691 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:56.690+0000 7f5398513640 1 --2- 172.21.15.49:0/2015360123 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5390107800 0x7f539019ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:56.691 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:56.690+0000 7f5398513640 1 --2- 172.21.15.49:0/2015360123 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f53901021b0 0x7f539019a750 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:56.691 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:56.690+0000 7f5398513640 1 --2- 172.21.15.49:0/2015360123 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f53901017f0 0x7f539019a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:56.692 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:56.690+0000 7f5398513640 1 -- 172.21.15.49:0/2015360123 >> 172.21.15.49:0/2015360123 conn(0x7f53900fd300 msgr2=0x7f539010cfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:16:56.692 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:56.690+0000 7f5398513640 1 -- 172.21.15.49:0/2015360123 shutdown_connections 2024-03-18T23:16:56.692 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:16:56.690+0000 7f5398513640 1 -- 172.21.15.49:0/2015360123 wait complete. 2024-03-18T23:16:56.780 INFO:tasks.workunit.client.0.smithi049.stdout: 1 916065 9.63 MB/sec execute 358 sec latency 81.482 ms 2024-03-18T23:16:57.460 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:16:57 smithi175 ceph-mon[67352]: pgmap v269: 97 pgs: 97 active+clean; 77 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.2 MiB/s wr, 210 op/s 2024-03-18T23:16:57.564 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:16:57 smithi139 ceph-mon[67282]: pgmap v269: 97 pgs: 97 active+clean; 77 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.2 MiB/s wr, 210 op/s 2024-03-18T23:16:57.629 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:16:57 smithi049 ceph-mon[62991]: pgmap v269: 97 pgs: 97 active+clean; 77 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.2 MiB/s wr, 210 op/s 2024-03-18T23:16:57.780 INFO:tasks.workunit.client.0.smithi049.stdout: 1 916935 9.64 MB/sec execute 359 sec latency 107.127 ms 2024-03-18T23:16:58.780 INFO:tasks.workunit.client.0.smithi049.stdout: 1 918386 9.66 MB/sec execute 360 sec latency 18.735 ms 2024-03-18T23:16:59.460 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:16:59 smithi175 ceph-mon[67352]: pgmap v270: 97 pgs: 97 active+clean; 82 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.2 MiB/s wr, 288 op/s 2024-03-18T23:16:59.564 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:16:59 smithi139 ceph-mon[67282]: pgmap v270: 97 pgs: 97 active+clean; 82 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.2 MiB/s wr, 288 op/s 2024-03-18T23:16:59.629 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:16:59 smithi049 ceph-mon[62991]: pgmap v270: 97 pgs: 97 active+clean; 82 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.2 MiB/s wr, 288 op/s 2024-03-18T23:16:59.780 INFO:tasks.workunit.client.0.smithi049.stdout: 1 920566 9.67 MB/sec execute 361 sec latency 17.797 ms 2024-03-18T23:17:00.780 INFO:tasks.workunit.client.0.smithi049.stdout: 1 921856 9.66 MB/sec execute 362 sec latency 80.882 ms 2024-03-18T23:17:01.460 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:17:01 smithi175 ceph-mon[67352]: pgmap v271: 97 pgs: 97 active+clean; 82 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.2 MiB/s wr, 152 op/s 2024-03-18T23:17:01.564 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:17:01 smithi139 ceph-mon[67282]: pgmap v271: 97 pgs: 97 active+clean; 82 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.2 MiB/s wr, 152 op/s 2024-03-18T23:17:01.629 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:17:01 smithi049 ceph-mon[62991]: pgmap v271: 97 pgs: 97 active+clean; 82 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.2 MiB/s wr, 152 op/s 2024-03-18T23:17:01.780 INFO:tasks.workunit.client.0.smithi049.stdout: 1 922531 9.63 MB/sec execute 363 sec latency 204.476 ms 2024-03-18T23:17:02.460 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:17:02 smithi175 ceph-mon[67352]: osdmap e96: 12 total, 12 up, 12 in 2024-03-18T23:17:02.564 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:17:02 smithi139 ceph-mon[67282]: osdmap e96: 12 total, 12 up, 12 in 2024-03-18T23:17:02.629 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:17:02 smithi049 ceph-mon[62991]: osdmap e96: 12 total, 12 up, 12 in 2024-03-18T23:17:02.780 INFO:tasks.workunit.client.0.smithi049.stdout: 1 922536 9.61 MB/sec execute 364 sec latency 226.510 ms 2024-03-18T23:17:03.460 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:17:03 smithi175 ceph-mon[67352]: pgmap v273: 97 pgs: 34 active+clean+snaptrim_wait, 17 active+clean+snaptrim, 46 active+clean; 72 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.5 MiB/s wr, 252 op/s 2024-03-18T23:17:03.564 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:17:03 smithi139 ceph-mon[67282]: pgmap v273: 97 pgs: 34 active+clean+snaptrim_wait, 17 active+clean+snaptrim, 46 active+clean; 72 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.5 MiB/s wr, 252 op/s 2024-03-18T23:17:03.629 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:17:03 smithi049 ceph-mon[62991]: pgmap v273: 97 pgs: 34 active+clean+snaptrim_wait, 17 active+clean+snaptrim, 46 active+clean; 72 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.5 MiB/s wr, 252 op/s 2024-03-18T23:17:03.780 INFO:tasks.workunit.client.0.smithi049.stdout: 1 923852 9.61 MB/sec execute 365 sec latency 128.503 ms 2024-03-18T23:17:04.780 INFO:tasks.workunit.client.0.smithi049.stdout: 1 925246 9.60 MB/sec execute 366 sec latency 23.376 ms 2024-03-18T23:17:05.460 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:17:05 smithi175 ceph-mon[67352]: pgmap v274: 97 pgs: 41 active+clean+snaptrim_wait, 23 active+clean+snaptrim, 33 active+clean; 72 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.7 MiB/s wr, 289 op/s 2024-03-18T23:17:05.564 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:17:05 smithi139 ceph-mon[67282]: pgmap v274: 97 pgs: 41 active+clean+snaptrim_wait, 23 active+clean+snaptrim, 33 active+clean; 72 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.7 MiB/s wr, 289 op/s 2024-03-18T23:17:05.629 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:17:05 smithi049 ceph-mon[62991]: pgmap v274: 97 pgs: 41 active+clean+snaptrim_wait, 23 active+clean+snaptrim, 33 active+clean; 72 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.7 MiB/s wr, 289 op/s 2024-03-18T23:17:05.780 INFO:tasks.workunit.client.0.smithi049.stdout: 1 927299 9.61 MB/sec execute 367 sec latency 60.685 ms 2024-03-18T23:17:06.780 INFO:tasks.workunit.client.0.smithi049.stdout: 1 928478 9.59 MB/sec execute 368 sec latency 18.864 ms 2024-03-18T23:17:07.460 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:17:07 smithi175 ceph-mon[67352]: pgmap v275: 97 pgs: 41 active+clean+snaptrim_wait, 23 active+clean+snaptrim, 33 active+clean; 72 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.7 MiB/s wr, 289 op/s 2024-03-18T23:17:07.564 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:17:07 smithi139 ceph-mon[67282]: pgmap v275: 97 pgs: 41 active+clean+snaptrim_wait, 23 active+clean+snaptrim, 33 active+clean; 72 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.7 MiB/s wr, 289 op/s 2024-03-18T23:17:07.629 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:17:07 smithi049 ceph-mon[62991]: pgmap v275: 97 pgs: 41 active+clean+snaptrim_wait, 23 active+clean+snaptrim, 33 active+clean; 72 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.7 MiB/s wr, 289 op/s 2024-03-18T23:17:07.781 INFO:tasks.workunit.client.0.smithi049.stdout: 1 930547 9.60 MB/sec execute 369 sec latency 54.191 ms 2024-03-18T23:17:08.781 INFO:tasks.workunit.client.0.smithi049.stdout: 1 931706 9.59 MB/sec execute 370 sec latency 147.201 ms 2024-03-18T23:17:09.460 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:17:09 smithi175 ceph-mon[67352]: pgmap v276: 97 pgs: 97 active+clean; 75 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.8 MiB/s wr, 283 op/s 2024-03-18T23:17:09.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:17:09 smithi175 ceph-mon[67352]: osdmap e97: 12 total, 12 up, 12 in 2024-03-18T23:17:09.564 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:17:09 smithi139 ceph-mon[67282]: pgmap v276: 97 pgs: 97 active+clean; 75 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.8 MiB/s wr, 283 op/s 2024-03-18T23:17:09.564 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:17:09 smithi139 ceph-mon[67282]: osdmap e97: 12 total, 12 up, 12 in 2024-03-18T23:17:09.629 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:17:09 smithi049 ceph-mon[62991]: pgmap v276: 97 pgs: 97 active+clean; 75 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.8 MiB/s wr, 283 op/s 2024-03-18T23:17:09.629 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:17:09 smithi049 ceph-mon[62991]: osdmap e97: 12 total, 12 up, 12 in 2024-03-18T23:17:09.781 INFO:tasks.workunit.client.0.smithi049.stdout: 1 933858 9.60 MB/sec execute 371 sec latency 34.757 ms 2024-03-18T23:17:09.783 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:09.783+0000 7f62fa71d640 1 -- 172.21.15.49:0/1472292682 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f62f4068ea0 msgr2=0x7f62f4069300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:17:09.783 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:09.783+0000 7f62fa71d640 1 --2- 172.21.15.49:0/1472292682 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f62f4068ea0 0x7f62f4069300 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f62e8009930 tx=0x7f62e802bc30 comp rx=0 tx=0).stop 2024-03-18T23:17:09.783 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:09.783+0000 7f62fa71d640 1 -- 172.21.15.49:0/1472292682 shutdown_connections 2024-03-18T23:17:09.783 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:09.783+0000 7f62fa71d640 1 --2- 172.21.15.49:0/1472292682 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f62f4069840 0x7f62f410e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:09.784 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:09.783+0000 7f62fa71d640 1 --2- 172.21.15.49:0/1472292682 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f62f4068ea0 0x7f62f4069300 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:09.784 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:09.783+0000 7f62fa71d640 1 --2- 172.21.15.49:0/1472292682 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f62f41078f0 0x7f62f4107cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:09.784 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:09.783+0000 7f62fa71d640 1 -- 172.21.15.49:0/1472292682 >> 172.21.15.49:0/1472292682 conn(0x7f62f4077570 msgr2=0x7f62f4077980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:17:09.784 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:09.783+0000 7f62fa71d640 1 -- 172.21.15.49:0/1472292682 shutdown_connections 2024-03-18T23:17:09.784 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:09.783+0000 7f62fa71d640 1 -- 172.21.15.49:0/1472292682 wait complete. 2024-03-18T23:17:09.784 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:09.783+0000 7f62fa71d640 1 Processor -- start 2024-03-18T23:17:09.784 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:09.783+0000 7f62fa71d640 1 -- start start 2024-03-18T23:17:09.786 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:09.784+0000 7f62fa71d640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f62f4068ea0 0x7f62f4198060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:17:09.786 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:09.784+0000 7f62fa71d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f62f4069840 0x7f62f41985a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:17:09.786 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:09.784+0000 7f62fa71d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f62f41078f0 0x7f62f419c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:17:09.786 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:09.784+0000 7f62fa71d640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f62f4111be0 con 0x7f62f41078f0 2024-03-18T23:17:09.786 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:09.784+0000 7f62fa71d640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f62f4111a60 con 0x7f62f4069840 2024-03-18T23:17:09.786 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:09.784+0000 7f62fa71d640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f62f4111d60 con 0x7f62f4068ea0 2024-03-18T23:17:09.786 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:09.784+0000 7f62f37fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f62f4069840 0x7f62f41985a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:17:09.786 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:09.784+0000 7f62f8cba640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f62f41078f0 0x7f62f419c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:17:09.786 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:09.784+0000 7f62f37fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f62f4069840 0x7f62f41985a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.49:58832/0 (socket says 172.21.15.49:58832) 2024-03-18T23:17:09.786 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:09.784+0000 7f62f37fe640 1 -- 172.21.15.49:0/29892479 learned_addr learned my addr 172.21.15.49:0/29892479 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:17:09.786 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:09.784+0000 7f62f8cba640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f62f41078f0 0x7f62f419c8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:55478/0 (socket says 172.21.15.49:55478) 2024-03-18T23:17:09.786 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:09.784+0000 7f62f3fff640 1 --2- 172.21.15.49:0/29892479 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f62f4068ea0 0x7f62f4198060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:17:09.787 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:09.784+0000 7f62f37fe640 1 -- 172.21.15.49:0/29892479 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f62f4068ea0 msgr2=0x7f62f4198060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:17:09.787 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:09.784+0000 7f62f37fe640 1 --2- 172.21.15.49:0/29892479 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f62f4068ea0 0x7f62f4198060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:09.787 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:09.784+0000 7f62f37fe640 1 -- 172.21.15.49:0/29892479 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f62f41078f0 msgr2=0x7f62f419c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:17:09.787 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:09.784+0000 7f62f37fe640 1 --2- 172.21.15.49:0/29892479 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f62f41078f0 0x7f62f419c8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:09.787 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:09.784+0000 7f62f37fe640 1 -- 172.21.15.49:0/29892479 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62f419d000 con 0x7f62f4069840 2024-03-18T23:17:09.787 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:09.785+0000 7f62f37fe640 1 --2- 172.21.15.49:0/29892479 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f62f4069840 0x7f62f41985a0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f62e8009230 tx=0x7f62e8004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:17:09.787 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:09.785+0000 7f62f17fa640 1 -- 172.21.15.49:0/29892479 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62e8041070 con 0x7f62f4069840 2024-03-18T23:17:09.787 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:09.785+0000 7f62f17fa640 1 -- 172.21.15.49:0/29892479 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f62e80054e0 con 0x7f62f4069840 2024-03-18T23:17:09.787 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:09.785+0000 7f62fa71d640 1 -- 172.21.15.49:0/29892479 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62f419d290 con 0x7f62f4069840 2024-03-18T23:17:09.787 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:09.785+0000 7f62f17fa640 1 -- 172.21.15.49:0/29892479 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62e8003c90 con 0x7f62f4069840 2024-03-18T23:17:09.787 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:09.785+0000 7f62fa71d640 1 -- 172.21.15.49:0/29892479 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62f41a4b80 con 0x7f62f4069840 2024-03-18T23:17:09.794 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:09.787+0000 7f62f17fa640 1 -- 172.21.15.49:0/29892479 <== mon.2 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 25) ==== 137421+0+0 (secure 0 0 0) 0x7f62e8044070 con 0x7f62f4069840 2024-03-18T23:17:09.794 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:09.787+0000 7f62fd3df740 1 Processor -- start 2024-03-18T23:17:09.794 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:09.787+0000 7f62fd3df740 1 -- start start 2024-03-18T23:17:09.794 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:09.787+0000 7f62fd3df740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e2ff48def0 0x55e2ff48e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:17:09.795 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:09.787+0000 7f62fd3df740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e2ff48e810 0x55e2ff4970a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:17:09.795 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:09.787+0000 7f62fd3df740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e2ff4975e0 0x55e2ff4999d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:17:09.795 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:09.787+0000 7f62fd3df740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55e2ff3f1be0 con 0x55e2ff48def0 2024-03-18T23:17:09.795 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:09.787+0000 7f62fd3df740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55e2ff39a960 con 0x55e2ff4975e0 2024-03-18T23:17:09.795 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:09.787+0000 7f62fd3df740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55e2ff3edc80 con 0x55e2ff48e810 2024-03-18T23:17:09.795 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:09.788+0000 7f62f8cba640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e2ff48def0 0x55e2ff48e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:17:09.795 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:09.788+0000 7f62f3fff640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e2ff48e810 0x55e2ff4970a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:17:09.795 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:09.788+0000 7f62f8cba640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e2ff48def0 0x55e2ff48e2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:55492/0 (socket says 172.21.15.49:55492) 2024-03-18T23:17:09.795 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:09.788+0000 7f62f3fff640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e2ff48e810 0x55e2ff4970a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.49:43792/0 (socket says 172.21.15.49:43792) 2024-03-18T23:17:09.795 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:09.788+0000 7f62f8cba640 1 -- 172.21.15.49:0/186447958 learned_addr learned my addr 172.21.15.49:0/186447958 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:17:09.795 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:09.788+0000 7f62f37fe640 1 --2- 172.21.15.49:0/186447958 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e2ff4975e0 0x55e2ff4999d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:17:09.795 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:09.788+0000 7f62f17fa640 1 --2- 172.21.15.49:0/29892479 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f62dc0a2a90 0x7f62dc0a4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:17:09.795 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:09.788+0000 7f62f8cba640 1 -- 172.21.15.49:0/186447958 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e2ff48e810 msgr2=0x55e2ff4970a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:17:09.796 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:09.788+0000 7f62f8cba640 1 --2- 172.21.15.49:0/186447958 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e2ff48e810 0x55e2ff4970a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:09.796 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:09.788+0000 7f62f8cba640 1 -- 172.21.15.49:0/186447958 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e2ff4975e0 msgr2=0x55e2ff4999d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:17:09.796 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:09.788+0000 7f62f8cba640 1 --2- 172.21.15.49:0/186447958 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e2ff4975e0 0x55e2ff4999d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:09.796 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:09.788+0000 7f62f8cba640 1 -- 172.21.15.49:0/186447958 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e2ff391d20 con 0x55e2ff48def0 2024-03-18T23:17:09.796 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:09.788+0000 7f62f17fa640 1 -- 172.21.15.49:0/29892479 <== mon.2 v2:172.21.15.139:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8556+0+0 (secure 0 0 0) 0x7f62e80f06a0 con 0x7f62f4069840 2024-03-18T23:17:09.796 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:09.788+0000 7f62f8cba640 1 --2- 172.21.15.49:0/29892479 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f62dc0a2a90 0x7f62dc0a4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:17:09.796 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:09.788+0000 7f62f8cba640 1 --2- 172.21.15.49:0/186447958 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e2ff48def0 0x55e2ff48e2d0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f62ec011560 tx=0x7f62ec011a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85372b113cfe7a1a server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:17:09.796 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:09.788+0000 7f62da7fc640 1 -- 172.21.15.49:0/186447958 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62ec017050 con 0x55e2ff48def0 2024-03-18T23:17:09.796 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:09.788+0000 7f62da7fc640 1 -- 172.21.15.49:0/186447958 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f62ec00fd20 con 0x55e2ff48def0 2024-03-18T23:17:09.796 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:09.788+0000 7f62da7fc640 1 -- 172.21.15.49:0/186447958 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62ec01bb80 con 0x55e2ff48def0 2024-03-18T23:17:09.796 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:09.788+0000 7f62f8cba640 1 --2- 172.21.15.49:0/29892479 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f62dc0a2a90 0x7f62dc0a4f50 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f62ec022000 tx=0x7f62ec023000 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:17:09.796 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:09.789+0000 7f62fd3df740 1 -- 172.21.15.49:0/186447958 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e2ff48def0 msgr2=0x55e2ff48e2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:17:09.796 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:09.789+0000 7f62fd3df740 1 --2- 172.21.15.49:0/186447958 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e2ff48def0 0x55e2ff48e2d0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f62ec011560 tx=0x7f62ec011a20 comp rx=0 tx=0).stop 2024-03-18T23:17:09.796 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:09.789+0000 7f62fd3df740 1 -- 172.21.15.49:0/186447958 shutdown_connections 2024-03-18T23:17:09.796 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:09.789+0000 7f62fd3df740 1 --2- 172.21.15.49:0/186447958 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e2ff4975e0 0x55e2ff4999d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:09.796 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:09.789+0000 7f62fd3df740 1 --2- 172.21.15.49:0/186447958 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e2ff48e810 0x55e2ff4970a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:09.797 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:09.789+0000 7f62fd3df740 1 --2- 172.21.15.49:0/186447958 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e2ff48def0 0x55e2ff48e2d0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:09.797 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:09.789+0000 7f62fd3df740 1 -- 172.21.15.49:0/186447958 >> 172.21.15.49:0/186447958 conn(0x55e2ff488810 msgr2=0x55e2ff488bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:17:09.797 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:09.789+0000 7f62fd3df740 1 -- 172.21.15.49:0/186447958 shutdown_connections 2024-03-18T23:17:09.797 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:09.789+0000 7f62fd3df740 1 -- 172.21.15.49:0/186447958 wait complete. 2024-03-18T23:17:09.797 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:09.789+0000 7f62fd3df740 1 -- start start 2024-03-18T23:17:09.797 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:09.790+0000 7f62fd3df740 1 Processor -- start 2024-03-18T23:17:09.797 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:09.790+0000 7f62fd3df740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e2ff48def0 0x55e2ff4a1360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:17:09.797 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:09.790+0000 7f62fd3df740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e2ff48e810 0x55e2ff4a18a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:17:09.797 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:09.790+0000 7f62fd3df740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e2ff4975e0 0x55e2ff49b770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:17:09.797 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:09.790+0000 7f62fd3df740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55e2ff49a210 con 0x55e2ff48def0 2024-03-18T23:17:09.797 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:09.790+0000 7f62fd3df740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55e2ff3addb0 con 0x55e2ff48e810 2024-03-18T23:17:09.797 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:09.790+0000 7f62fd3df740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55e2ff49a390 con 0x55e2ff4975e0 2024-03-18T23:17:09.797 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:09.790+0000 7f62f3fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e2ff48def0 0x55e2ff4a1360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:17:09.797 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:09.790+0000 7f62fd3df740 10 client.0 _ll_register_callbacks cb 0x55e2ff3b2e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T23:17:09.797 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:09.790+0000 7f62f3fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e2ff48def0 0x55e2ff4a1360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:55508/0 (socket says 172.21.15.49:55508) 2024-03-18T23:17:09.798 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:09.790+0000 7f62f3fff640 1 -- 172.21.15.49:0/3386040822 learned_addr learned my addr 172.21.15.49:0/3386040822 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:17:09.798 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:09.790+0000 7f62f37fe640 1 --2- 172.21.15.49:0/3386040822 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e2ff48e810 0x55e2ff4a18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:17:09.798 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:09.790+0000 7f62f8cba640 1 --2- 172.21.15.49:0/3386040822 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e2ff4975e0 0x55e2ff49b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:17:09.798 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:09.790+0000 7f62f3fff640 1 -- 172.21.15.49:0/3386040822 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e2ff4975e0 msgr2=0x55e2ff49b770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:17:09.798 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:09.790+0000 7f62f3fff640 1 --2- 172.21.15.49:0/3386040822 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e2ff4975e0 0x55e2ff49b770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:09.798 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:09.790+0000 7f62f3fff640 1 -- 172.21.15.49:0/3386040822 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e2ff48e810 msgr2=0x55e2ff4a18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:17:09.798 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:09.790+0000 7f62f3fff640 1 --2- 172.21.15.49:0/3386040822 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e2ff48e810 0x55e2ff4a18a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:09.798 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:09.790+0000 7f62f3fff640 1 -- 172.21.15.49:0/3386040822 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e2ff3acc60 con 0x55e2ff48def0 2024-03-18T23:17:09.798 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:09.791+0000 7f62f3fff640 1 --2- 172.21.15.49:0/3386040822 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e2ff48def0 0x55e2ff4a1360 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f62e4012740 tx=0x7f62e4012c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c56f6666939d6823 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:17:09.798 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:09.791+0000 7f62d37fe640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-18T23:17:09.798 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:09.791+0000 7f62d37fe640 1 -- 172.21.15.49:0/3386040822 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62e400f920 con 0x55e2ff48def0 2024-03-18T23:17:09.798 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:09.791+0000 7f62d37fe640 1 -- 172.21.15.49:0/3386040822 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8556+0+0 (secure 0 0 0) 0x7f62e400fac0 con 0x55e2ff48def0 2024-03-18T23:17:09.798 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:09.791+0000 7f62fd3df740 1 -- 172.21.15.49:0/3386040822 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f62ec00fd20 con 0x55e2ff48def0 2024-03-18T23:17:09.799 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:09.791+0000 7f62d37fe640 1 -- 172.21.15.49:0/3386040822 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f62e400fd00 con 0x55e2ff48def0 2024-03-18T23:17:09.799 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:09.791+0000 7f62fd3df740 10 client.16173 fetch_fsmap learned FSMap version 32 2024-03-18T23:17:09.799 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:09.791+0000 7f62fd3df740 1 -- 172.21.15.49:0/3386040822 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55e2ff106210 con 0x55e2ff48def0 2024-03-18T23:17:09.799 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:09.791+0000 7f62d37fe640 1 -- 172.21.15.49:0/3386040822 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f62ec00fd20 con 0x55e2ff48def0 2024-03-18T23:17:09.799 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:09.791+0000 7f62d37fe640 1 -- 172.21.15.49:0/3386040822 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f62e4024420 con 0x55e2ff48def0 2024-03-18T23:17:09.799 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:09.791+0000 7f62fd3df740 10 client.16173 fetch_fsmap finished waiting for FSMap version 32 2024-03-18T23:17:09.799 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:09.791+0000 7f62fd3df740 10 client.16173 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T23:17:09.799 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:09.791+0000 7f62fd3df740 20 client.16173 populate_metadata read hostname 'smithi049' 2024-03-18T23:17:09.799 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:09.791+0000 7f62fd3df740 1 --2- 172.21.15.49:0/3386040822 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x55e2ff4ab980 0x55e2ff4addb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:17:09.799 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:09.791+0000 7f62fd3df740 4 client.16173 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T23:17:09.799 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:09.791+0000 7f62fd3df740 1 -- 172.21.15.49:0/3386040822 --> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e2ff3ff120 con 0x55e2ff4ab980 2024-03-18T23:17:09.799 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:09.792+0000 7f62f37fe640 1 --2- 172.21.15.49:0/3386040822 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x55e2ff4ab980 0x55e2ff4addb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:17:09.799 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:09.792+0000 7f62d37fe640 10 client.16173 ms_handle_connect on v2:172.21.15.139:6838/4280128209 2024-03-18T23:17:09.799 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:09.792+0000 7f62f37fe640 1 --2- 172.21.15.49:0/3386040822 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x55e2ff4ab980 0x55e2ff4addb0 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=33ffbe71e6621b97 server_cookie=e915268132d4e582 in_seq=0 out_seq=0 2024-03-18T23:17:09.800 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:09.793+0000 7f62d37fe640 1 -- 172.21.15.49:0/3386040822 <== mds.0 v2:172.21.15.139:6838/4280128209 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55e2ff3ff120 con 0x55e2ff4ab980 2024-03-18T23:17:09.800 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:09.793+0000 7f62d37fe640 10 client.16173 handle_command_reply: tid=0 2024-03-18T23:17:09.800 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:09.793+0000 7f62fd3df740 1 client.16173 shutdown 2024-03-18T23:17:09.800 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:09.793+0000 7f62fd3df740 1 -- 172.21.15.49:0/3386040822 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e2ff48def0 msgr2=0x55e2ff4a1360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:17:09.800 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:09.793+0000 7f62fd3df740 1 --2- 172.21.15.49:0/3386040822 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e2ff48def0 0x55e2ff4a1360 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f62e4012740 tx=0x7f62e4012c00 comp rx=0 tx=0).stop 2024-03-18T23:17:09.800 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:09.793+0000 7f62fd3df740 1 -- 172.21.15.49:0/3386040822 shutdown_connections 2024-03-18T23:17:09.800 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:09.793+0000 7f62fd3df740 1 --2- 172.21.15.49:0/3386040822 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x55e2ff4ab980 0x55e2ff4addb0 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:09.800 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:09.793+0000 7f62fd3df740 1 --2- 172.21.15.49:0/3386040822 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e2ff4975e0 0x55e2ff49b770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:09.800 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:09.793+0000 7f62fd3df740 1 --2- 172.21.15.49:0/3386040822 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e2ff48e810 0x55e2ff4a18a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:09.800 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:09.793+0000 7f62fd3df740 1 --2- 172.21.15.49:0/3386040822 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e2ff48def0 0x55e2ff4a1360 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:09.800 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:09.793+0000 7f62fd3df740 1 -- 172.21.15.49:0/3386040822 >> 172.21.15.49:0/3386040822 conn(0x55e2ff488810 msgr2=0x55e2ff487ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:17:09.800 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:09.793+0000 7f62d37fe640 0 client.16173 ms_handle_reset on v2:172.21.15.139:6838/4280128209 2024-03-18T23:17:09.800 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:09.794+0000 7f62fd3df740 1 -- 172.21.15.49:0/3386040822 shutdown_connections 2024-03-18T23:17:09.800 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:09.794+0000 7f62fd3df740 1 -- 172.21.15.49:0/3386040822 wait complete. 2024-03-18T23:17:09.801 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:09.794+0000 7f62fd3df740 20 client.16173 trim_cache size 0 max 16384 2024-03-18T23:17:09.811 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:09.809+0000 7f62fd3df740 1 Processor -- start 2024-03-18T23:17:09.811 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:09.809+0000 7f62fd3df740 1 -- start start 2024-03-18T23:17:09.811 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:09.809+0000 7f62fd3df740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e2ff48def0 0x55e2ff4b36c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:17:09.811 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:09.809+0000 7f62fd3df740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e2ff48e810 0x55e2ff4959d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:17:09.811 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:09.809+0000 7f62fd3df740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e2ff4975e0 0x55e2ff495f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:17:09.811 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:09.809+0000 7f62fd3df740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f62f41ab230 con 0x55e2ff48def0 2024-03-18T23:17:09.811 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:09.809+0000 7f62fd3df740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f62f41acd70 con 0x55e2ff4975e0 2024-03-18T23:17:09.811 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:09.809+0000 7f62fd3df740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f62f41ab900 con 0x55e2ff48e810 2024-03-18T23:17:09.811 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:09.809+0000 7f62f3fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e2ff48def0 0x55e2ff4b36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:17:09.812 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:09.809+0000 7f62f3fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e2ff48def0 0x55e2ff4b36c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:55510/0 (socket says 172.21.15.49:55510) 2024-03-18T23:17:09.812 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:09.809+0000 7f62f3fff640 1 -- 172.21.15.49:0/441482456 learned_addr learned my addr 172.21.15.49:0/441482456 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:17:09.812 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:09.809+0000 7f62f37fe640 1 --2- 172.21.15.49:0/441482456 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e2ff48e810 0x55e2ff4959d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:17:09.812 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:09.809+0000 7f62f8cba640 1 --2- 172.21.15.49:0/441482456 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e2ff4975e0 0x55e2ff495f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:17:09.812 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:09.809+0000 7f62f3fff640 1 -- 172.21.15.49:0/441482456 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e2ff48e810 msgr2=0x55e2ff4959d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:17:09.812 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:09.809+0000 7f62f3fff640 1 --2- 172.21.15.49:0/441482456 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e2ff48e810 0x55e2ff4959d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:09.812 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:09.809+0000 7f62f3fff640 1 -- 172.21.15.49:0/441482456 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e2ff4975e0 msgr2=0x55e2ff495f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:17:09.812 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:09.809+0000 7f62f3fff640 1 --2- 172.21.15.49:0/441482456 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e2ff4975e0 0x55e2ff495f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:09.812 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:09.809+0000 7f62f3fff640 1 -- 172.21.15.49:0/441482456 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e2ff48e640 con 0x55e2ff48def0 2024-03-18T23:17:09.812 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:09.809+0000 7f62f8cba640 1 --2- 172.21.15.49:0/441482456 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e2ff4975e0 0x55e2ff495f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T23:17:09.812 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:09.810+0000 7f62f3fff640 1 --2- 172.21.15.49:0/441482456 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e2ff48def0 0x55e2ff4b36c0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f62e4024890 tx=0x7f62e4009620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d05307aab4efdcf6 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:17:09.812 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:09.810+0000 7f62d9ffb640 1 -- 172.21.15.49:0/441482456 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62e400fea0 con 0x55e2ff48def0 2024-03-18T23:17:09.812 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:09.810+0000 7f62d9ffb640 1 -- 172.21.15.49:0/441482456 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f62e4012e50 con 0x55e2ff48def0 2024-03-18T23:17:09.812 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:09.810+0000 7f62d9ffb640 1 -- 172.21.15.49:0/441482456 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62e4024680 con 0x55e2ff48def0 2024-03-18T23:17:09.813 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:09.810+0000 7f62fd3df740 1 -- 172.21.15.49:0/441482456 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e2ff48def0 msgr2=0x55e2ff4b36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:17:09.813 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:09.810+0000 7f62fd3df740 1 --2- 172.21.15.49:0/441482456 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e2ff48def0 0x55e2ff4b36c0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f62e4024890 tx=0x7f62e4009620 comp rx=0 tx=0).stop 2024-03-18T23:17:09.813 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:09.810+0000 7f62fd3df740 1 -- 172.21.15.49:0/441482456 shutdown_connections 2024-03-18T23:17:09.813 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:09.810+0000 7f62fd3df740 1 --2- 172.21.15.49:0/441482456 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e2ff4975e0 0x55e2ff495f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:09.813 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:09.810+0000 7f62fd3df740 1 --2- 172.21.15.49:0/441482456 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e2ff48e810 0x55e2ff4959d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:09.813 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:09.810+0000 7f62fd3df740 1 --2- 172.21.15.49:0/441482456 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e2ff48def0 0x55e2ff4b36c0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:09.813 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:09.810+0000 7f62fd3df740 1 -- 172.21.15.49:0/441482456 >> 172.21.15.49:0/441482456 conn(0x55e2ff488810 msgr2=0x55e2ff487ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:17:09.813 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:09.810+0000 7f62fd3df740 1 -- 172.21.15.49:0/441482456 shutdown_connections 2024-03-18T23:17:09.813 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:09.811+0000 7f62fd3df740 1 -- 172.21.15.49:0/441482456 wait complete. 2024-03-18T23:17:09.813 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:09.811+0000 7f62fd3df740 1 -- start start 2024-03-18T23:17:09.813 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:09.811+0000 7f62fd3df740 1 Processor -- start 2024-03-18T23:17:09.816 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:09.811+0000 7f62fd3df740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e2ff48def0 0x55e2ff4a7970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:17:09.816 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:09.811+0000 7f62fd3df740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e2ff48e810 0x55e2ff4a5d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:17:09.816 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:09.811+0000 7f62fd3df740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e2ff4975e0 0x55e2ff4a6250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:17:09.816 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:09.811+0000 7f62fd3df740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f62f41ac6a0 con 0x55e2ff48e810 2024-03-18T23:17:09.816 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:09.811+0000 7f62fd3df740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f62f41abfd0 con 0x55e2ff4975e0 2024-03-18T23:17:09.816 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:09.811+0000 7f62fd3df740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f62f41a9210 con 0x55e2ff48def0 2024-03-18T23:17:09.816 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:09.811+0000 7f62fd3df740 10 client.0 _ll_register_callbacks cb 0x7f62e4009910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T23:17:09.816 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:09.811+0000 7f62f37fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e2ff48e810 0x55e2ff4a5d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:17:09.816 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:09.811+0000 7f62f37fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e2ff48e810 0x55e2ff4a5d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:55520/0 (socket says 172.21.15.49:55520) 2024-03-18T23:17:09.817 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:09.811+0000 7f62f3fff640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e2ff48def0 0x55e2ff4a7970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:17:09.817 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:09.811+0000 7f62f37fe640 1 -- 172.21.15.49:0/1355723542 learned_addr learned my addr 172.21.15.49:0/1355723542 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:17:09.817 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:09.811+0000 7f62f8cba640 1 --2- 172.21.15.49:0/1355723542 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e2ff4975e0 0x55e2ff4a6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:17:09.817 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:09.812+0000 7f62f37fe640 1 -- 172.21.15.49:0/1355723542 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e2ff48def0 msgr2=0x55e2ff4a7970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:17:09.817 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:09.812+0000 7f62f37fe640 1 --2- 172.21.15.49:0/1355723542 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e2ff48def0 0x55e2ff4a7970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:09.817 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:09.812+0000 7f62f37fe640 1 -- 172.21.15.49:0/1355723542 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e2ff4975e0 msgr2=0x55e2ff4a6250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:17:09.817 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:09.812+0000 7f62f37fe640 1 --2- 172.21.15.49:0/1355723542 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e2ff4975e0 0x55e2ff4a6250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:09.817 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:09.812+0000 7f62f37fe640 1 -- 172.21.15.49:0/1355723542 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e2ff1d2620 con 0x55e2ff48e810 2024-03-18T23:17:09.817 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:09.812+0000 7f62f3fff640 1 --2- 172.21.15.49:0/1355723542 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e2ff48def0 0x55e2ff4a7970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:17:09.817 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:09.812+0000 7f62f37fe640 1 --2- 172.21.15.49:0/1355723542 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e2ff48e810 0x55e2ff4a5d10 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f62e8006010 tx=0x7f62e80f2dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2dc07121338548c4 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:17:09.817 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:09.812+0000 7f62d37fe640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-18T23:17:09.817 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:09.812+0000 7f62d37fe640 1 -- 172.21.15.49:0/1355723542 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62e804b040 con 0x55e2ff48e810 2024-03-18T23:17:09.817 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:09.812+0000 7f62d37fe640 1 -- 172.21.15.49:0/1355723542 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8556+0+0 (secure 0 0 0) 0x7f62e802cab0 con 0x55e2ff48e810 2024-03-18T23:17:09.817 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:09.812+0000 7f62fd3df740 1 -- 172.21.15.49:0/1355723542 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f62e4012e50 con 0x55e2ff48e810 2024-03-18T23:17:09.818 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:09.812+0000 7f62d37fe640 1 -- 172.21.15.49:0/1355723542 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f62e80bc800 con 0x55e2ff48e810 2024-03-18T23:17:09.818 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:09.812+0000 7f62fd3df740 10 client.16179 fetch_fsmap learned FSMap version 32 2024-03-18T23:17:09.818 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:09.812+0000 7f62fd3df740 1 -- 172.21.15.49:0/1355723542 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55e2ff512860 con 0x55e2ff48e810 2024-03-18T23:17:09.818 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:09.812+0000 7f62d37fe640 1 -- 172.21.15.49:0/1355723542 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f62e4012e50 con 0x55e2ff48e810 2024-03-18T23:17:09.818 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:09.813+0000 7f62d37fe640 1 -- 172.21.15.49:0/1355723542 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f62e80ee050 con 0x55e2ff48e810 2024-03-18T23:17:09.818 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:09.813+0000 7f62fd3df740 10 client.16179 fetch_fsmap finished waiting for FSMap version 32 2024-03-18T23:17:09.818 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:09.813+0000 7f62fd3df740 10 client.16179 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T23:17:09.818 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:09.813+0000 7f62fd3df740 20 client.16179 populate_metadata read hostname 'smithi049' 2024-03-18T23:17:09.818 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:09.813+0000 7f62fd3df740 1 --2- 172.21.15.49:0/1355723542 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x55e2ff512fe0 0x55e2ff515400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:17:09.818 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:09.813+0000 7f62fd3df740 4 client.16179 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T23:17:09.818 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:09.813+0000 7f62fd3df740 1 -- 172.21.15.49:0/1355723542 --> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e2ff3f9470 con 0x55e2ff512fe0 2024-03-18T23:17:09.818 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:09.813+0000 7f62f3fff640 1 --2- 172.21.15.49:0/1355723542 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x55e2ff512fe0 0x55e2ff515400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:17:09.818 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:09.813+0000 7f62f3fff640 1 --2- 172.21.15.49:0/1355723542 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x55e2ff512fe0 0x55e2ff515400 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c323c5bbb689735a server_cookie=f313d9ddec3d5fa5 in_seq=0 out_seq=0 2024-03-18T23:17:09.818 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:09.813+0000 7f62d37fe640 10 client.16179 ms_handle_connect on v2:172.21.15.139:6838/4280128209 2024-03-18T23:17:09.980 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:09.979+0000 7f62d37fe640 1 -- 172.21.15.49:0/1355723542 <== mds.0 v2:172.21.15.139:6838/4280128209 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e2ff3f9470 con 0x55e2ff512fe0 2024-03-18T23:17:09.981 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:09.979+0000 7f62d37fe640 10 client.16179 handle_command_reply: tid=0 2024-03-18T23:17:09.981 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:09.979+0000 7f62fd3df740 1 client.16179 shutdown 2024-03-18T23:17:09.981 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:09.980+0000 7f62fd3df740 1 -- 172.21.15.49:0/1355723542 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e2ff48e810 msgr2=0x55e2ff4a5d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:17:09.981 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:09.980+0000 7f62fd3df740 1 --2- 172.21.15.49:0/1355723542 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e2ff48e810 0x55e2ff4a5d10 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f62e8006010 tx=0x7f62e80f2dc0 comp rx=0 tx=0).stop 2024-03-18T23:17:09.981 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:09.980+0000 7f62fd3df740 1 -- 172.21.15.49:0/1355723542 shutdown_connections 2024-03-18T23:17:09.981 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:09.980+0000 7f62fd3df740 1 --2- 172.21.15.49:0/1355723542 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x55e2ff512fe0 0x55e2ff515400 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:09.981 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:09.980+0000 7f62fd3df740 1 --2- 172.21.15.49:0/1355723542 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e2ff4975e0 0x55e2ff4a6250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:09.981 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:09.980+0000 7f62fd3df740 1 --2- 172.21.15.49:0/1355723542 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e2ff48e810 0x55e2ff4a5d10 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:09.981 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:09.980+0000 7f62fd3df740 1 --2- 172.21.15.49:0/1355723542 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e2ff48def0 0x55e2ff4a7970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:09.981 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:09.980+0000 7f62d37fe640 0 client.16179 ms_handle_reset on v2:172.21.15.139:6838/4280128209 2024-03-18T23:17:09.981 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:09.980+0000 7f62fd3df740 1 -- 172.21.15.49:0/1355723542 >> 172.21.15.49:0/1355723542 conn(0x55e2ff488810 msgr2=0x55e2ff493450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:17:09.981 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:09.980+0000 7f62fd3df740 1 -- 172.21.15.49:0/1355723542 shutdown_connections 2024-03-18T23:17:09.982 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:09.980+0000 7f62fd3df740 1 -- 172.21.15.49:0/1355723542 wait complete. 2024-03-18T23:17:09.982 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:09.980+0000 7f62fd3df740 20 client.16179 trim_cache size 0 max 16384 2024-03-18T23:17:09.982 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:{ 2024-03-18T23:17:09.982 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "message": "", 2024-03-18T23:17:09.982 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "return_code": 0 2024-03-18T23:17:09.982 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:} 2024-03-18T23:17:09.983 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:09.980+0000 7f62fa71d640 1 -- 172.21.15.49:0/29892479 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f62dc0a2a90 msgr2=0x7f62dc0a4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:17:09.983 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:09.980+0000 7f62fa71d640 1 --2- 172.21.15.49:0/29892479 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f62dc0a2a90 0x7f62dc0a4f50 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f62ec022000 tx=0x7f62ec023000 comp rx=0 tx=0).stop 2024-03-18T23:17:09.983 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:09.980+0000 7f62fa71d640 1 -- 172.21.15.49:0/29892479 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f62f4069840 msgr2=0x7f62f41985a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:17:09.983 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:09.980+0000 7f62fa71d640 1 --2- 172.21.15.49:0/29892479 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f62f4069840 0x7f62f41985a0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f62e8009230 tx=0x7f62e8004270 comp rx=0 tx=0).stop 2024-03-18T23:17:09.983 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:09.981+0000 7f62fa71d640 1 -- 172.21.15.49:0/29892479 shutdown_connections 2024-03-18T23:17:09.983 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:09.981+0000 7f62fa71d640 1 --2- 172.21.15.49:0/29892479 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f62dc0a2a90 0x7f62dc0a4f50 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:09.983 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:09.981+0000 7f62fa71d640 1 --2- 172.21.15.49:0/29892479 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f62f41078f0 0x7f62f419c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:09.984 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:09.981+0000 7f62fa71d640 1 --2- 172.21.15.49:0/29892479 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f62f4069840 0x7f62f41985a0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:09.984 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:09.981+0000 7f62fa71d640 1 --2- 172.21.15.49:0/29892479 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f62f4068ea0 0x7f62f4198060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:09.984 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:09.981+0000 7f62fa71d640 1 -- 172.21.15.49:0/29892479 >> 172.21.15.49:0/29892479 conn(0x7f62f4077570 msgr2=0x7f62f40fe9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:17:09.984 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:09.981+0000 7f62fa71d640 1 -- 172.21.15.49:0/29892479 shutdown_connections 2024-03-18T23:17:09.984 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:09.981+0000 7f62fa71d640 1 -- 172.21.15.49:0/29892479 wait complete. 2024-03-18T23:17:10.781 INFO:tasks.workunit.client.0.smithi049.stdout: 1 934899 9.59 MB/sec execute 372 sec latency 89.951 ms 2024-03-18T23:17:11.460 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:17:11 smithi175 ceph-mon[67352]: pgmap v278: 97 pgs: 97 active+clean; 75 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.4 MiB/s wr, 341 op/s 2024-03-18T23:17:11.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:17:11 smithi139 ceph-mon[67282]: pgmap v278: 97 pgs: 97 active+clean; 75 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.4 MiB/s wr, 341 op/s 2024-03-18T23:17:11.629 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:17:11 smithi049 ceph-mon[62991]: pgmap v278: 97 pgs: 97 active+clean; 75 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.4 MiB/s wr, 341 op/s 2024-03-18T23:17:11.781 INFO:tasks.workunit.client.0.smithi049.stdout: 1 936673 9.58 MB/sec execute 373 sec latency 44.253 ms 2024-03-18T23:17:12.781 INFO:tasks.workunit.client.0.smithi049.stdout: 1 938565 9.59 MB/sec execute 374 sec latency 16.154 ms 2024-03-18T23:17:13.460 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:17:13 smithi175 ceph-mon[67352]: pgmap v279: 97 pgs: 97 active+clean; 71 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 792 KiB/s rd, 2.8 MiB/s wr, 267 op/s 2024-03-18T23:17:13.564 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:17:13 smithi139 ceph-mon[67282]: pgmap v279: 97 pgs: 97 active+clean; 71 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 792 KiB/s rd, 2.8 MiB/s wr, 267 op/s 2024-03-18T23:17:13.629 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:17:13 smithi049 ceph-mon[62991]: pgmap v279: 97 pgs: 97 active+clean; 71 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 792 KiB/s rd, 2.8 MiB/s wr, 267 op/s 2024-03-18T23:17:13.781 INFO:tasks.workunit.client.0.smithi049.stdout: 1 940499 9.58 MB/sec execute 375 sec latency 18.340 ms 2024-03-18T23:17:14.781 INFO:tasks.workunit.client.0.smithi049.stdout: 1 942119 9.59 MB/sec execute 376 sec latency 33.999 ms 2024-03-18T23:17:15.460 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:17:15 smithi175 ceph-mon[67352]: pgmap v280: 97 pgs: 97 active+clean; 69 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.9 MiB/s wr, 277 op/s 2024-03-18T23:17:15.564 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:17:15 smithi139 ceph-mon[67282]: pgmap v280: 97 pgs: 97 active+clean; 69 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.9 MiB/s wr, 277 op/s 2024-03-18T23:17:15.629 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:17:15 smithi049 ceph-mon[62991]: pgmap v280: 97 pgs: 97 active+clean; 69 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.9 MiB/s wr, 277 op/s 2024-03-18T23:17:15.781 INFO:tasks.workunit.client.0.smithi049.stdout: 1 943909 9.58 MB/sec execute 377 sec latency 45.243 ms 2024-03-18T23:17:16.781 INFO:tasks.workunit.client.0.smithi049.stdout: 1 945545 9.59 MB/sec execute 378 sec latency 76.053 ms 2024-03-18T23:17:17.460 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:17:17 smithi175 ceph-mon[67352]: pgmap v281: 97 pgs: 97 active+clean; 69 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.9 MiB/s wr, 277 op/s 2024-03-18T23:17:17.564 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:17:17 smithi139 ceph-mon[67282]: pgmap v281: 97 pgs: 97 active+clean; 69 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.9 MiB/s wr, 277 op/s 2024-03-18T23:17:17.629 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:17:17 smithi049 ceph-mon[62991]: pgmap v281: 97 pgs: 97 active+clean; 69 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.9 MiB/s wr, 277 op/s 2024-03-18T23:17:17.781 INFO:tasks.workunit.client.0.smithi049.stdout: 1 947483 9.58 MB/sec execute 379 sec latency 21.352 ms 2024-03-18T23:17:18.781 INFO:tasks.workunit.client.0.smithi049.stdout: 1 949218 9.59 MB/sec execute 380 sec latency 18.170 ms 2024-03-18T23:17:19.460 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:17:19 smithi175 ceph-mon[67352]: pgmap v282: 97 pgs: 97 active+clean; 74 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.9 MiB/s wr, 278 op/s 2024-03-18T23:17:19.564 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:17:19 smithi139 ceph-mon[67282]: pgmap v282: 97 pgs: 97 active+clean; 74 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.9 MiB/s wr, 278 op/s 2024-03-18T23:17:19.629 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:17:19 smithi049 ceph-mon[62991]: pgmap v282: 97 pgs: 97 active+clean; 74 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.9 MiB/s wr, 278 op/s 2024-03-18T23:17:19.781 INFO:tasks.workunit.client.0.smithi049.stdout: 1 951107 9.58 MB/sec execute 381 sec latency 21.386 ms 2024-03-18T23:17:20.781 INFO:tasks.workunit.client.0.smithi049.stdout: 1 952797 9.59 MB/sec execute 382 sec latency 15.821 ms 2024-03-18T23:17:21.460 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:17:21 smithi175 ceph-mon[67352]: pgmap v283: 97 pgs: 97 active+clean; 74 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.8 MiB/s wr, 269 op/s 2024-03-18T23:17:21.564 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:17:21 smithi139 ceph-mon[67282]: pgmap v283: 97 pgs: 97 active+clean; 74 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.8 MiB/s wr, 269 op/s 2024-03-18T23:17:21.629 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:17:21 smithi049 ceph-mon[62991]: pgmap v283: 97 pgs: 97 active+clean; 74 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.8 MiB/s wr, 269 op/s 2024-03-18T23:17:21.781 INFO:tasks.workunit.client.0.smithi049.stdout: 1 954111 9.58 MB/sec execute 383 sec latency 72.868 ms 2024-03-18T23:17:22.781 INFO:tasks.workunit.client.0.smithi049.stdout: 1 956166 9.59 MB/sec execute 384 sec latency 17.152 ms 2024-03-18T23:17:23.072 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:23.072+0000 7fc069d90640 1 -- 172.21.15.49:0/3226998141 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc064069840 msgr2=0x7fc06410e670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:17:23.072 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:23.072+0000 7fc069d90640 1 --2- 172.21.15.49:0/3226998141 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc064069840 0x7fc06410e670 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fc054009be0 tx=0x7fc05402bef0 comp rx=0 tx=0).stop 2024-03-18T23:17:23.072 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:23.072+0000 7fc069d90640 1 -- 172.21.15.49:0/3226998141 shutdown_connections 2024-03-18T23:17:23.073 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:23.072+0000 7fc069d90640 1 --2- 172.21.15.49:0/3226998141 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc064069840 0x7fc06410e670 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:23.073 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:23.072+0000 7fc069d90640 1 --2- 172.21.15.49:0/3226998141 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc064068ea0 0x7fc064069300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:23.073 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:23.072+0000 7fc069d90640 1 --2- 172.21.15.49:0/3226998141 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc064107890 0x7fc064107c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:23.073 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:23.072+0000 7fc069d90640 1 -- 172.21.15.49:0/3226998141 >> 172.21.15.49:0/3226998141 conn(0x7fc064077570 msgr2=0x7fc064077980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:17:23.073 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:23.072+0000 7fc069d90640 1 -- 172.21.15.49:0/3226998141 shutdown_connections 2024-03-18T23:17:23.073 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:23.072+0000 7fc069d90640 1 -- 172.21.15.49:0/3226998141 wait complete. 2024-03-18T23:17:23.073 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:23.072+0000 7fc069d90640 1 Processor -- start 2024-03-18T23:17:23.073 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:23.072+0000 7fc069d90640 1 -- start start 2024-03-18T23:17:23.075 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:23.073+0000 7fc069d90640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc064068ea0 0x7fc064100a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:17:23.075 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:23.073+0000 7fc069d90640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc064069840 0x7fc064100f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:17:23.075 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:23.073+0000 7fc069d90640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc064107890 0x7fc0641014c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:17:23.075 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:23.073+0000 7fc069d90640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fc064111bb0 con 0x7fc064068ea0 2024-03-18T23:17:23.075 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:23.073+0000 7fc069d90640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fc064111a30 con 0x7fc064107890 2024-03-18T23:17:23.075 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:23.073+0000 7fc069d90640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc064111d30 con 0x7fc064069840 2024-03-18T23:17:23.075 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:23.073+0000 7fc0637fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc064068ea0 0x7fc064100a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:17:23.075 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:23.073+0000 7fc0637fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc064068ea0 0x7fc064100a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:45380/0 (socket says 172.21.15.49:45380) 2024-03-18T23:17:23.075 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:23.073+0000 7fc0637fe640 1 -- 172.21.15.49:0/1688485432 learned_addr learned my addr 172.21.15.49:0/1688485432 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:17:23.076 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:23.073+0000 7fc062ffd640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc064069840 0x7fc064100f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:17:23.076 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:23.073+0000 7fc063fff640 1 --2- 172.21.15.49:0/1688485432 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc064107890 0x7fc0641014c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:17:23.076 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:23.073+0000 7fc0637fe640 1 -- 172.21.15.49:0/1688485432 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc064069840 msgr2=0x7fc064100f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:17:23.076 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:23.073+0000 7fc0637fe640 1 --2- 172.21.15.49:0/1688485432 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc064069840 0x7fc064100f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:23.076 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:23.073+0000 7fc0637fe640 1 -- 172.21.15.49:0/1688485432 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc064107890 msgr2=0x7fc0641014c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:17:23.076 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:23.073+0000 7fc0637fe640 1 --2- 172.21.15.49:0/1688485432 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc064107890 0x7fc0641014c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:23.076 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:23.073+0000 7fc0637fe640 1 -- 172.21.15.49:0/1688485432 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0641a6510 con 0x7fc064068ea0 2024-03-18T23:17:23.076 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:23.073+0000 7fc0637fe640 1 --2- 172.21.15.49:0/1688485432 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc064068ea0 0x7fc064100a40 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fc05c00e9c0 tx=0x7fc05c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:17:23.076 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:23.074+0000 7fc060ff9640 1 -- 172.21.15.49:0/1688485432 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc05c00cde0 con 0x7fc064068ea0 2024-03-18T23:17:23.076 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:23.074+0000 7fc060ff9640 1 -- 172.21.15.49:0/1688485432 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc05c0035f0 con 0x7fc064068ea0 2024-03-18T23:17:23.076 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:23.074+0000 7fc069d90640 1 -- 172.21.15.49:0/1688485432 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0641a6800 con 0x7fc064068ea0 2024-03-18T23:17:23.076 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:23.074+0000 7fc060ff9640 1 -- 172.21.15.49:0/1688485432 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc05c010690 con 0x7fc064068ea0 2024-03-18T23:17:23.076 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:23.074+0000 7fc069d90640 1 -- 172.21.15.49:0/1688485432 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0641a6d50 con 0x7fc064068ea0 2024-03-18T23:17:23.082 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:23.076+0000 7fc060ff9640 1 -- 172.21.15.49:0/1688485432 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 25) ==== 137421+0+0 (secure 0 0 0) 0x7fc05c0037a0 con 0x7fc064068ea0 2024-03-18T23:17:23.082 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:23.076+0000 7fc06c977740 1 Processor -- start 2024-03-18T23:17:23.082 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:23.076+0000 7fc06c977740 1 -- start start 2024-03-18T23:17:23.082 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:23.076+0000 7fc06c977740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559dc2573ef0 0x559dc25742d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:17:23.082 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:23.076+0000 7fc06c977740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559dc2574810 0x559dc257d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:17:23.082 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:23.076+0000 7fc06c977740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559dc257d5e0 0x559dc257f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:17:23.082 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:23.076+0000 7fc060ff9640 1 --2- 172.21.15.49:0/1688485432 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7fc0500a2a10 0x7fc0500a4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:17:23.082 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:23.076+0000 7fc06c977740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x559dc24d7be0 con 0x559dc257d5e0 2024-03-18T23:17:23.083 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:23.076+0000 7fc06c977740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x559dc2480960 con 0x559dc2574810 2024-03-18T23:17:23.083 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:23.076+0000 7fc06c977740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x559dc24d3c80 con 0x559dc2573ef0 2024-03-18T23:17:23.083 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:23.077+0000 7fc063fff640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559dc2573ef0 0x559dc25742d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:17:23.083 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:23.077+0000 7fc0637fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559dc2574810 0x559dc257d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:17:23.083 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:23.077+0000 7fc062ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559dc257d5e0 0x559dc257f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:17:23.083 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:23.077+0000 7fc063fff640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559dc2573ef0 0x559dc25742d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.49:35858/0 (socket says 172.21.15.49:35858) 2024-03-18T23:17:23.083 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:23.077+0000 7fc063fff640 1 -- 172.21.15.49:0/3054920561 learned_addr learned my addr 172.21.15.49:0/3054920561 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:17:23.083 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:23.077+0000 7fc063fff640 1 --2- 172.21.15.49:0/1688485432 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7fc0500a2a10 0x7fc0500a4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:17:23.083 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:23.077+0000 7fc060ff9640 1 -- 172.21.15.49:0/1688485432 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8556+0+0 (secure 0 0 0) 0x7fc05c0cf120 con 0x7fc064068ea0 2024-03-18T23:17:23.083 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:23.077+0000 7fc062ffd640 1 -- 172.21.15.49:0/3054920561 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559dc2573ef0 msgr2=0x559dc25742d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:17:23.083 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:23.077+0000 7fc062ffd640 1 --2- 172.21.15.49:0/3054920561 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559dc2573ef0 0x559dc25742d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:23.083 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:23.077+0000 7fc062ffd640 1 -- 172.21.15.49:0/3054920561 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559dc2574810 msgr2=0x559dc257d0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:17:23.083 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:23.077+0000 7fc062ffd640 1 --2- 172.21.15.49:0/3054920561 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559dc2574810 0x559dc257d0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:23.083 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:23.077+0000 7fc062ffd640 1 -- 172.21.15.49:0/3054920561 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559dc2477d20 con 0x559dc257d5e0 2024-03-18T23:17:23.084 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:23.077+0000 7fc063fff640 1 --2- 172.21.15.49:0/3054920561 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559dc2573ef0 0x559dc25742d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:17:23.084 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:23.077+0000 7fc062ffd640 1 --2- 172.21.15.49:0/3054920561 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559dc257d5e0 0x559dc257f9d0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fc04c009bf0 tx=0x7fc04c011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e76dd69a1aaa623 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:17:23.084 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:23.077+0000 7fc063fff640 1 --2- 172.21.15.49:0/1688485432 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7fc0500a2a10 0x7fc0500a4ed0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fc054036040 tx=0x7fc05402bef0 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:17:23.084 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:23.077+0000 7fc059ffb640 1 -- 172.21.15.49:0/3054920561 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc04c01e070 con 0x559dc257d5e0 2024-03-18T23:17:23.084 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:23.077+0000 7fc059ffb640 1 -- 172.21.15.49:0/3054920561 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc04c009da0 con 0x559dc257d5e0 2024-03-18T23:17:23.084 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:23.077+0000 7fc059ffb640 1 -- 172.21.15.49:0/3054920561 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc04c018740 con 0x559dc257d5e0 2024-03-18T23:17:23.084 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:23.078+0000 7fc06c977740 1 -- 172.21.15.49:0/3054920561 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559dc257d5e0 msgr2=0x559dc257f9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:17:23.084 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:23.078+0000 7fc06c977740 1 --2- 172.21.15.49:0/3054920561 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559dc257d5e0 0x559dc257f9d0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fc04c009bf0 tx=0x7fc04c011810 comp rx=0 tx=0).stop 2024-03-18T23:17:23.084 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:23.078+0000 7fc06c977740 1 -- 172.21.15.49:0/3054920561 shutdown_connections 2024-03-18T23:17:23.084 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:23.078+0000 7fc06c977740 1 --2- 172.21.15.49:0/3054920561 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559dc257d5e0 0x559dc257f9d0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:23.084 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:23.078+0000 7fc06c977740 1 --2- 172.21.15.49:0/3054920561 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559dc2574810 0x559dc257d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:23.084 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:23.078+0000 7fc06c977740 1 --2- 172.21.15.49:0/3054920561 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559dc2573ef0 0x559dc25742d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:23.084 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:23.078+0000 7fc06c977740 1 -- 172.21.15.49:0/3054920561 >> 172.21.15.49:0/3054920561 conn(0x559dc256e810 msgr2=0x559dc256ebf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:17:23.084 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:23.078+0000 7fc06c977740 1 -- 172.21.15.49:0/3054920561 shutdown_connections 2024-03-18T23:17:23.084 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:23.078+0000 7fc06c977740 1 -- 172.21.15.49:0/3054920561 wait complete. 2024-03-18T23:17:23.085 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:23.078+0000 7fc06c977740 1 -- start start 2024-03-18T23:17:23.085 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:23.079+0000 7fc06c977740 1 Processor -- start 2024-03-18T23:17:23.085 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:23.079+0000 7fc06c977740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559dc2573ef0 0x559dc257cf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:17:23.085 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:23.079+0000 7fc06c977740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559dc2574810 0x559dc257a310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:17:23.085 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:23.079+0000 7fc06c977740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559dc257d5e0 0x559dc257a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:17:23.085 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:23.079+0000 7fc06c977740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x559dc2580210 con 0x559dc2573ef0 2024-03-18T23:17:23.085 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:23.079+0000 7fc06c977740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x559dc2493db0 con 0x559dc2574810 2024-03-18T23:17:23.085 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:23.079+0000 7fc06c977740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x559dc2580390 con 0x559dc257d5e0 2024-03-18T23:17:23.085 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:23.079+0000 7fc06c977740 10 client.0 _ll_register_callbacks cb 0x559dc2498e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T23:17:23.085 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:23.079+0000 7fc0637fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559dc2573ef0 0x559dc257cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:17:23.085 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:23.079+0000 7fc0637fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559dc2573ef0 0x559dc257cf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:45398/0 (socket says 172.21.15.49:45398) 2024-03-18T23:17:23.085 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:23.079+0000 7fc0637fe640 1 -- 172.21.15.49:0/421255004 learned_addr learned my addr 172.21.15.49:0/421255004 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:17:23.085 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:23.079+0000 7fc062ffd640 1 --2- 172.21.15.49:0/421255004 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559dc2574810 0x559dc257a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:17:23.085 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:23.079+0000 7fc063fff640 1 --2- 172.21.15.49:0/421255004 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559dc257d5e0 0x559dc257a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:17:23.086 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:23.079+0000 7fc0637fe640 1 -- 172.21.15.49:0/421255004 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559dc257d5e0 msgr2=0x559dc257a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:17:23.086 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:23.079+0000 7fc0637fe640 1 --2- 172.21.15.49:0/421255004 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559dc257d5e0 0x559dc257a8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:23.086 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:23.079+0000 7fc0637fe640 1 -- 172.21.15.49:0/421255004 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559dc2574810 msgr2=0x559dc257a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:17:23.086 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:23.079+0000 7fc0637fe640 1 --2- 172.21.15.49:0/421255004 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559dc2574810 0x559dc257a310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:23.086 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:23.079+0000 7fc0637fe640 1 -- 172.21.15.49:0/421255004 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559dc2492c60 con 0x559dc2573ef0 2024-03-18T23:17:23.086 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:23.079+0000 7fc062ffd640 1 --2- 172.21.15.49:0/421255004 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559dc2574810 0x559dc257a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:17:23.086 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:23.080+0000 7fc0637fe640 1 --2- 172.21.15.49:0/421255004 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559dc2573ef0 0x559dc257cf80 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fc05c0cc8b0 tx=0x7fc05c0664d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef52debc95f5b37 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:17:23.086 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:23.080+0000 7fc042ffd640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-18T23:17:23.086 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:23.080+0000 7fc042ffd640 1 -- 172.21.15.49:0/421255004 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc05c09be60 con 0x559dc2573ef0 2024-03-18T23:17:23.086 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:23.080+0000 7fc042ffd640 1 -- 172.21.15.49:0/421255004 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8556+0+0 (secure 0 0 0) 0x7fc05c097dd0 con 0x559dc2573ef0 2024-03-18T23:17:23.086 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:23.080+0000 7fc06c977740 1 -- 172.21.15.49:0/421255004 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc04c009da0 con 0x559dc2573ef0 2024-03-18T23:17:23.086 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:23.080+0000 7fc042ffd640 1 -- 172.21.15.49:0/421255004 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fc05c09b760 con 0x559dc2573ef0 2024-03-18T23:17:23.086 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:23.080+0000 7fc06c977740 10 client.16191 fetch_fsmap learned FSMap version 32 2024-03-18T23:17:23.086 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:23.080+0000 7fc06c977740 1 -- 172.21.15.49:0/421255004 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x559dc21ec210 con 0x559dc2573ef0 2024-03-18T23:17:23.087 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:23.080+0000 7fc042ffd640 1 -- 172.21.15.49:0/421255004 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc04c009da0 con 0x559dc2573ef0 2024-03-18T23:17:23.087 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:23.080+0000 7fc042ffd640 1 -- 172.21.15.49:0/421255004 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7fc05c0a3a70 con 0x559dc2573ef0 2024-03-18T23:17:23.087 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:23.080+0000 7fc06c977740 10 client.16191 fetch_fsmap finished waiting for FSMap version 32 2024-03-18T23:17:23.087 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:23.080+0000 7fc06c977740 10 client.16191 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T23:17:23.087 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:23.080+0000 7fc06c977740 20 client.16191 populate_metadata read hostname 'smithi049' 2024-03-18T23:17:23.087 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:23.080+0000 7fc06c977740 1 --2- 172.21.15.49:0/421255004 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x559dc258d470 0x559dc258f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:17:23.087 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:23.080+0000 7fc06c977740 4 client.16191 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T23:17:23.087 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:23.080+0000 7fc06c977740 1 -- 172.21.15.49:0/421255004 --> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559dc24e5120 con 0x559dc258d470 2024-03-18T23:17:23.087 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:23.081+0000 7fc062ffd640 1 --2- 172.21.15.49:0/421255004 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x559dc258d470 0x559dc258f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:17:23.087 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:23.081+0000 7fc062ffd640 1 --2- 172.21.15.49:0/421255004 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x559dc258d470 0x559dc258f8a0 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=17561f47d0eb153 server_cookie=6b84bde0818a7aed in_seq=0 out_seq=0 2024-03-18T23:17:23.087 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:23.081+0000 7fc042ffd640 10 client.16191 ms_handle_connect on v2:172.21.15.139:6838/4280128209 2024-03-18T23:17:23.087 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:23.082+0000 7fc042ffd640 1 -- 172.21.15.49:0/421255004 <== mds.0 v2:172.21.15.139:6838/4280128209 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x559dc24e5120 con 0x559dc258d470 2024-03-18T23:17:23.087 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:23.082+0000 7fc042ffd640 10 client.16191 handle_command_reply: tid=0 2024-03-18T23:17:23.087 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:23.082+0000 7fc06c977740 1 client.16191 shutdown 2024-03-18T23:17:23.088 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:23.082+0000 7fc06c977740 1 -- 172.21.15.49:0/421255004 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559dc2573ef0 msgr2=0x559dc257cf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:17:23.088 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:23.082+0000 7fc06c977740 1 --2- 172.21.15.49:0/421255004 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559dc2573ef0 0x559dc257cf80 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fc05c0cc8b0 tx=0x7fc05c0664d0 comp rx=0 tx=0).stop 2024-03-18T23:17:23.088 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:23.082+0000 7fc06c977740 1 -- 172.21.15.49:0/421255004 shutdown_connections 2024-03-18T23:17:23.088 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:23.082+0000 7fc06c977740 1 --2- 172.21.15.49:0/421255004 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x559dc258d470 0x559dc258f8a0 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:23.089 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:23.082+0000 7fc06c977740 1 --2- 172.21.15.49:0/421255004 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559dc257d5e0 0x559dc257a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:23.089 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:23.082+0000 7fc06c977740 1 --2- 172.21.15.49:0/421255004 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559dc2574810 0x559dc257a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:23.089 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:23.082+0000 7fc06c977740 1 --2- 172.21.15.49:0/421255004 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559dc2573ef0 0x559dc257cf80 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:23.089 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:23.082+0000 7fc042ffd640 0 client.16191 ms_handle_reset on v2:172.21.15.139:6838/4280128209 2024-03-18T23:17:23.089 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:23.082+0000 7fc06c977740 1 -- 172.21.15.49:0/421255004 >> 172.21.15.49:0/421255004 conn(0x559dc256e810 msgr2=0x559dc256dab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:17:23.089 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:23.082+0000 7fc06c977740 1 -- 172.21.15.49:0/421255004 shutdown_connections 2024-03-18T23:17:23.089 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:23.082+0000 7fc06c977740 1 -- 172.21.15.49:0/421255004 wait complete. 2024-03-18T23:17:23.089 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:23.083+0000 7fc06c977740 20 client.16191 trim_cache size 0 max 16384 2024-03-18T23:17:23.100 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:23.098+0000 7fc06c977740 1 Processor -- start 2024-03-18T23:17:23.100 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:23.098+0000 7fc06c977740 1 -- start start 2024-03-18T23:17:23.100 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:23.098+0000 7fc06c977740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559dc2573ef0 0x559dc256dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:17:23.100 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:23.098+0000 7fc06c977740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559dc2574810 0x559dc25951d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:17:23.100 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:23.098+0000 7fc06c977740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559dc257d5e0 0x559dc25855d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:17:23.100 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:23.098+0000 7fc06c977740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fc0641af780 con 0x559dc2573ef0 2024-03-18T23:17:23.100 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:23.098+0000 7fc06c977740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fc0641b12c0 con 0x559dc257d5e0 2024-03-18T23:17:23.100 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:23.098+0000 7fc06c977740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc0641afe50 con 0x559dc2574810 2024-03-18T23:17:23.100 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:23.098+0000 7fc0637fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559dc2573ef0 0x559dc256dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:17:23.101 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:23.098+0000 7fc0637fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559dc2573ef0 0x559dc256dcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:45404/0 (socket says 172.21.15.49:45404) 2024-03-18T23:17:23.101 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:23.098+0000 7fc0637fe640 1 -- 172.21.15.49:0/2723191140 learned_addr learned my addr 172.21.15.49:0/2723191140 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:17:23.101 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:23.098+0000 7fc063fff640 1 --2- 172.21.15.49:0/2723191140 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559dc257d5e0 0x559dc25855d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:17:23.101 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:23.098+0000 7fc062ffd640 1 --2- 172.21.15.49:0/2723191140 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559dc2574810 0x559dc25951d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:17:23.101 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:23.098+0000 7fc0637fe640 1 -- 172.21.15.49:0/2723191140 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559dc2574810 msgr2=0x559dc25951d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:17:23.101 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:23.098+0000 7fc0637fe640 1 --2- 172.21.15.49:0/2723191140 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559dc2574810 0x559dc25951d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:23.101 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:23.098+0000 7fc0637fe640 1 -- 172.21.15.49:0/2723191140 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559dc257d5e0 msgr2=0x559dc25855d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:17:23.101 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:23.098+0000 7fc0637fe640 1 --2- 172.21.15.49:0/2723191140 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559dc257d5e0 0x559dc25855d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:23.101 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:23.099+0000 7fc0637fe640 1 -- 172.21.15.49:0/2723191140 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559dc2569f30 con 0x559dc2573ef0 2024-03-18T23:17:23.101 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:23.099+0000 7fc0637fe640 1 --2- 172.21.15.49:0/2723191140 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559dc2573ef0 0x559dc256dcf0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fc05c0cc900 tx=0x7fc05c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d78b4dd69725278 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:17:23.101 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:23.099+0000 7fc0597fa640 1 -- 172.21.15.49:0/2723191140 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc05c09b2a0 con 0x559dc2573ef0 2024-03-18T23:17:23.101 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:23.099+0000 7fc0597fa640 1 -- 172.21.15.49:0/2723191140 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc05c060e50 con 0x559dc2573ef0 2024-03-18T23:17:23.101 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:23.099+0000 7fc0597fa640 1 -- 172.21.15.49:0/2723191140 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc05c0a3f00 con 0x559dc2573ef0 2024-03-18T23:17:23.101 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:23.099+0000 7fc06c977740 1 -- 172.21.15.49:0/2723191140 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559dc2573ef0 msgr2=0x559dc256dcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:17:23.102 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:23.099+0000 7fc06c977740 1 --2- 172.21.15.49:0/2723191140 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559dc2573ef0 0x559dc256dcf0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fc05c0cc900 tx=0x7fc05c000bc0 comp rx=0 tx=0).stop 2024-03-18T23:17:23.102 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:23.100+0000 7fc06c977740 1 -- 172.21.15.49:0/2723191140 shutdown_connections 2024-03-18T23:17:23.102 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:23.100+0000 7fc06c977740 1 --2- 172.21.15.49:0/2723191140 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559dc257d5e0 0x559dc25855d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:23.102 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:23.100+0000 7fc06c977740 1 --2- 172.21.15.49:0/2723191140 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559dc2574810 0x559dc25951d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:23.102 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:23.100+0000 7fc06c977740 1 --2- 172.21.15.49:0/2723191140 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559dc2573ef0 0x559dc256dcf0 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:23.102 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:23.100+0000 7fc06c977740 1 -- 172.21.15.49:0/2723191140 >> 172.21.15.49:0/2723191140 conn(0x559dc256e810 msgr2=0x559dc25704f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:17:23.102 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:23.100+0000 7fc06c977740 1 -- 172.21.15.49:0/2723191140 shutdown_connections 2024-03-18T23:17:23.102 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:23.100+0000 7fc06c977740 1 -- 172.21.15.49:0/2723191140 wait complete. 2024-03-18T23:17:23.102 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:23.100+0000 7fc06c977740 1 -- start start 2024-03-18T23:17:23.102 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:23.100+0000 7fc06c977740 1 Processor -- start 2024-03-18T23:17:23.105 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:23.100+0000 7fc06c977740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559dc2573ef0 0x559dc25854d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:17:23.105 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:23.100+0000 7fc06c977740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559dc2574810 0x559dc2585a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:17:23.105 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:23.100+0000 7fc06c977740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559dc257d5e0 0x559dc2581af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:17:23.105 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:23.100+0000 7fc06c977740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fc0641b0bf0 con 0x559dc257d5e0 2024-03-18T23:17:23.105 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:23.100+0000 7fc06c977740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fc0641b0520 con 0x559dc2573ef0 2024-03-18T23:17:23.105 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:23.100+0000 7fc06c977740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc0641ad780 con 0x559dc2574810 2024-03-18T23:17:23.105 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:23.100+0000 7fc06c977740 10 client.0 _ll_register_callbacks cb 0x7fc05c097af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T23:17:23.105 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:23.100+0000 7fc063fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559dc257d5e0 0x559dc2581af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:17:23.105 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:23.101+0000 7fc062ffd640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559dc2574810 0x559dc2585a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:17:23.105 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:23.101+0000 7fc063fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559dc257d5e0 0x559dc2581af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:45412/0 (socket says 172.21.15.49:45412) 2024-03-18T23:17:23.105 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:23.101+0000 7fc063fff640 1 -- 172.21.15.49:0/802590481 learned_addr learned my addr 172.21.15.49:0/802590481 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:17:23.105 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:23.101+0000 7fc0637fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559dc2573ef0 0x559dc25854d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:17:23.105 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:23.101+0000 7fc063fff640 1 -- 172.21.15.49:0/802590481 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559dc2574810 msgr2=0x559dc2585a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:17:23.106 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:23.101+0000 7fc063fff640 1 --2- 172.21.15.49:0/802590481 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559dc2574810 0x559dc2585a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:23.106 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:23.101+0000 7fc063fff640 1 -- 172.21.15.49:0/802590481 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559dc2573ef0 msgr2=0x559dc25854d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:17:23.106 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:23.101+0000 7fc063fff640 1 --2- 172.21.15.49:0/802590481 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559dc2573ef0 0x559dc25854d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:23.106 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:23.101+0000 7fc063fff640 1 -- 172.21.15.49:0/802590481 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559dc22b8620 con 0x559dc257d5e0 2024-03-18T23:17:23.106 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:23.101+0000 7fc062ffd640 1 --2- 172.21.15.49:0/802590481 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559dc2574810 0x559dc2585a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:17:23.106 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:23.101+0000 7fc063fff640 1 --2- 172.21.15.49:0/802590481 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559dc257d5e0 0x559dc2581af0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fc054009be0 tx=0x7fc054009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c5442af6e7bc600 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:17:23.106 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:23.101+0000 7fc042ffd640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-18T23:17:23.106 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:23.101+0000 7fc042ffd640 1 -- 172.21.15.49:0/802590481 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc054004250 con 0x559dc257d5e0 2024-03-18T23:17:23.106 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:23.101+0000 7fc042ffd640 1 -- 172.21.15.49:0/802590481 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8556+0+0 (secure 0 0 0) 0x7fc0540043f0 con 0x559dc257d5e0 2024-03-18T23:17:23.106 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:23.101+0000 7fc06c977740 1 -- 172.21.15.49:0/802590481 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc05c060e50 con 0x559dc257d5e0 2024-03-18T23:17:23.106 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:23.101+0000 7fc042ffd640 1 -- 172.21.15.49:0/802590481 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fc0540045f0 con 0x559dc257d5e0 2024-03-18T23:17:23.106 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:23.101+0000 7fc06c977740 10 client.16197 fetch_fsmap learned FSMap version 32 2024-03-18T23:17:23.106 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:23.101+0000 7fc06c977740 1 -- 172.21.15.49:0/802590481 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x559dc25f42a0 con 0x559dc257d5e0 2024-03-18T23:17:23.106 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:23.102+0000 7fc042ffd640 1 -- 172.21.15.49:0/802590481 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc05c060e50 con 0x559dc257d5e0 2024-03-18T23:17:23.106 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:23.102+0000 7fc042ffd640 1 -- 172.21.15.49:0/802590481 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7fc05404e040 con 0x559dc257d5e0 2024-03-18T23:17:23.107 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:23.102+0000 7fc06c977740 10 client.16197 fetch_fsmap finished waiting for FSMap version 32 2024-03-18T23:17:23.107 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:23.102+0000 7fc06c977740 10 client.16197 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T23:17:23.107 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:23.102+0000 7fc06c977740 20 client.16197 populate_metadata read hostname 'smithi049' 2024-03-18T23:17:23.107 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:23.102+0000 7fc06c977740 1 --2- 172.21.15.49:0/802590481 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x559dc25f4a50 0x559dc25f6e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:17:23.107 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:23.102+0000 7fc06c977740 4 client.16197 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T23:17:23.107 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:23.102+0000 7fc06c977740 1 -- 172.21.15.49:0/802590481 --> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559dc24df470 con 0x559dc25f4a50 2024-03-18T23:17:23.107 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:23.102+0000 7fc0637fe640 1 --2- 172.21.15.49:0/802590481 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x559dc25f4a50 0x559dc25f6e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:17:23.107 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:23.102+0000 7fc042ffd640 10 client.16197 ms_handle_connect on v2:172.21.15.139:6838/4280128209 2024-03-18T23:17:23.107 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:23.102+0000 7fc0637fe640 1 --2- 172.21.15.49:0/802590481 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x559dc25f4a50 0x559dc25f6e70 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ed7e9dbe7155da50 server_cookie=19c50cdf488c734 in_seq=0 out_seq=0 2024-03-18T23:17:23.301 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:23.300+0000 7fc042ffd640 1 -- 172.21.15.49:0/802590481 <== mds.0 v2:172.21.15.139:6838/4280128209 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x559dc24df470 con 0x559dc25f4a50 2024-03-18T23:17:23.301 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:23.300+0000 7fc042ffd640 10 client.16197 handle_command_reply: tid=0 2024-03-18T23:17:23.301 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:23.300+0000 7fc06c977740 1 client.16197 shutdown 2024-03-18T23:17:23.301 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:23.300+0000 7fc06c977740 1 -- 172.21.15.49:0/802590481 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559dc257d5e0 msgr2=0x559dc2581af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:17:23.301 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:23.300+0000 7fc06c977740 1 --2- 172.21.15.49:0/802590481 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559dc257d5e0 0x559dc2581af0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fc054009be0 tx=0x7fc054009840 comp rx=0 tx=0).stop 2024-03-18T23:17:23.301 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:23.300+0000 7fc06c977740 1 -- 172.21.15.49:0/802590481 shutdown_connections 2024-03-18T23:17:23.302 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:23.300+0000 7fc06c977740 1 --2- 172.21.15.49:0/802590481 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x559dc25f4a50 0x559dc25f6e70 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:23.302 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:23.300+0000 7fc06c977740 1 --2- 172.21.15.49:0/802590481 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559dc257d5e0 0x559dc2581af0 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:23.302 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:23.300+0000 7fc06c977740 1 --2- 172.21.15.49:0/802590481 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559dc2574810 0x559dc2585a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:23.302 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:23.300+0000 7fc06c977740 1 --2- 172.21.15.49:0/802590481 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559dc2573ef0 0x559dc25854d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:23.302 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:23.300+0000 7fc06c977740 1 -- 172.21.15.49:0/802590481 >> 172.21.15.49:0/802590481 conn(0x559dc256e810 msgr2=0x559dc2574d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:17:23.302 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:23.300+0000 7fc042ffd640 0 client.16197 ms_handle_reset on v2:172.21.15.139:6838/4280128209 2024-03-18T23:17:23.302 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:23.301+0000 7fc06c977740 1 -- 172.21.15.49:0/802590481 shutdown_connections 2024-03-18T23:17:23.302 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:23.301+0000 7fc06c977740 1 -- 172.21.15.49:0/802590481 wait complete. 2024-03-18T23:17:23.302 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:23.301+0000 7fc06c977740 20 client.16197 trim_cache size 0 max 16384 2024-03-18T23:17:23.302 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:{ 2024-03-18T23:17:23.302 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "message": "", 2024-03-18T23:17:23.303 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "return_code": 0 2024-03-18T23:17:23.303 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:} 2024-03-18T23:17:23.303 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:23.301+0000 7fc069d90640 1 -- 172.21.15.49:0/1688485432 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7fc0500a2a10 msgr2=0x7fc0500a4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:17:23.303 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:23.301+0000 7fc069d90640 1 --2- 172.21.15.49:0/1688485432 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7fc0500a2a10 0x7fc0500a4ed0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fc054036040 tx=0x7fc05402bef0 comp rx=0 tx=0).stop 2024-03-18T23:17:23.304 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:23.301+0000 7fc069d90640 1 -- 172.21.15.49:0/1688485432 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc064068ea0 msgr2=0x7fc064100a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:17:23.304 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:23.301+0000 7fc069d90640 1 --2- 172.21.15.49:0/1688485432 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc064068ea0 0x7fc064100a40 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fc05c00e9c0 tx=0x7fc05c00ee80 comp rx=0 tx=0).stop 2024-03-18T23:17:23.304 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:23.301+0000 7fc069d90640 1 -- 172.21.15.49:0/1688485432 shutdown_connections 2024-03-18T23:17:23.304 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:23.301+0000 7fc069d90640 1 --2- 172.21.15.49:0/1688485432 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7fc0500a2a10 0x7fc0500a4ed0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:23.304 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:23.301+0000 7fc069d90640 1 --2- 172.21.15.49:0/1688485432 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc064107890 0x7fc0641014c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:23.304 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:23.301+0000 7fc069d90640 1 --2- 172.21.15.49:0/1688485432 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc064069840 0x7fc064100f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:23.304 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:23.301+0000 7fc069d90640 1 --2- 172.21.15.49:0/1688485432 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc064068ea0 0x7fc064100a40 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:23.304 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:23.301+0000 7fc069d90640 1 -- 172.21.15.49:0/1688485432 >> 172.21.15.49:0/1688485432 conn(0x7fc064077570 msgr2=0x7fc0640ff460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:17:23.304 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:23.302+0000 7fc069d90640 1 -- 172.21.15.49:0/1688485432 shutdown_connections 2024-03-18T23:17:23.304 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:23.302+0000 7fc069d90640 1 -- 172.21.15.49:0/1688485432 wait complete. 2024-03-18T23:17:23.460 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:17:23 smithi175 ceph-mon[67352]: pgmap v284: 97 pgs: 97 active+clean; 77 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.2 MiB/s wr, 283 op/s 2024-03-18T23:17:23.564 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:17:23 smithi139 ceph-mon[67282]: pgmap v284: 97 pgs: 97 active+clean; 77 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.2 MiB/s wr, 283 op/s 2024-03-18T23:17:23.629 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:17:23 smithi049 ceph-mon[62991]: pgmap v284: 97 pgs: 97 active+clean; 77 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.2 MiB/s wr, 283 op/s 2024-03-18T23:17:23.782 INFO:tasks.workunit.client.0.smithi049.stdout: 1 957520 9.58 MB/sec execute 385 sec latency 135.315 ms 2024-03-18T23:17:24.782 INFO:tasks.workunit.client.0.smithi049.stdout: 1 959531 9.59 MB/sec execute 386 sec latency 33.137 ms 2024-03-18T23:17:25.460 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:17:25 smithi175 ceph-mon[67352]: pgmap v285: 97 pgs: 97 active+clean; 80 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.6 MiB/s wr, 193 op/s 2024-03-18T23:17:25.564 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:17:25 smithi139 ceph-mon[67282]: pgmap v285: 97 pgs: 97 active+clean; 80 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.6 MiB/s wr, 193 op/s 2024-03-18T23:17:25.629 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:17:25 smithi049 ceph-mon[62991]: pgmap v285: 97 pgs: 97 active+clean; 80 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.6 MiB/s wr, 193 op/s 2024-03-18T23:17:25.782 INFO:tasks.workunit.client.0.smithi049.stdout: 1 960977 9.58 MB/sec execute 387 sec latency 25.606 ms 2024-03-18T23:17:26.782 INFO:tasks.workunit.client.0.smithi049.stdout: 1 962954 9.58 MB/sec execute 388 sec latency 106.760 ms 2024-03-18T23:17:27.460 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:17:27 smithi175 ceph-mon[67352]: pgmap v286: 97 pgs: 97 active+clean; 80 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 904 KiB/s rd, 2.3 MiB/s wr, 152 op/s 2024-03-18T23:17:27.564 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:17:27 smithi139 ceph-mon[67282]: pgmap v286: 97 pgs: 97 active+clean; 80 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 904 KiB/s rd, 2.3 MiB/s wr, 152 op/s 2024-03-18T23:17:27.629 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:17:27 smithi049 ceph-mon[62991]: pgmap v286: 97 pgs: 97 active+clean; 80 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 904 KiB/s rd, 2.3 MiB/s wr, 152 op/s 2024-03-18T23:17:27.782 INFO:tasks.workunit.client.0.smithi049.stdout: 1 964444 9.58 MB/sec execute 389 sec latency 21.208 ms 2024-03-18T23:17:28.782 INFO:tasks.workunit.client.0.smithi049.stdout: 1 966587 9.59 MB/sec execute 390 sec latency 20.411 ms 2024-03-18T23:17:29.460 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:17:29 smithi175 ceph-mon[67352]: pgmap v287: 97 pgs: 97 active+clean; 70 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.6 MiB/s wr, 314 op/s 2024-03-18T23:17:29.564 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:17:29 smithi139 ceph-mon[67282]: pgmap v287: 97 pgs: 97 active+clean; 70 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.6 MiB/s wr, 314 op/s 2024-03-18T23:17:29.629 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:17:29 smithi049 ceph-mon[62991]: pgmap v287: 97 pgs: 97 active+clean; 70 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.6 MiB/s wr, 314 op/s 2024-03-18T23:17:29.782 INFO:tasks.workunit.client.0.smithi049.stdout: 1 968037 9.58 MB/sec execute 391 sec latency 26.539 ms 2024-03-18T23:17:30.564 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:17:30 smithi139 ceph-mon[67282]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:17:30.629 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:17:30 smithi049 ceph-mon[62991]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:17:30.710 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:17:30 smithi175 ceph-mon[67352]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:17:30.782 INFO:tasks.workunit.client.0.smithi049.stdout: 1 970062 9.58 MB/sec execute 392 sec latency 21.125 ms 2024-03-18T23:17:31.564 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:17:31 smithi139 ceph-mon[67282]: pgmap v288: 97 pgs: 97 active+clean; 70 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1004 KiB/s rd, 2.4 MiB/s wr, 239 op/s 2024-03-18T23:17:31.564 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:17:31 smithi139 ceph-mon[67282]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:17:31.564 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:17:31 smithi139 ceph-mon[67282]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:17:31.564 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:17:31 smithi139 ceph-mon[67282]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' 2024-03-18T23:17:31.629 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:17:31 smithi049 ceph-mon[62991]: pgmap v288: 97 pgs: 97 active+clean; 70 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1004 KiB/s rd, 2.4 MiB/s wr, 239 op/s 2024-03-18T23:17:31.629 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:17:31 smithi049 ceph-mon[62991]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:17:31.629 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:17:31 smithi049 ceph-mon[62991]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:17:31.629 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:17:31 smithi049 ceph-mon[62991]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' 2024-03-18T23:17:31.710 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:17:31 smithi175 ceph-mon[67352]: pgmap v288: 97 pgs: 97 active+clean; 70 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1004 KiB/s rd, 2.4 MiB/s wr, 239 op/s 2024-03-18T23:17:31.710 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:17:31 smithi175 ceph-mon[67352]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:17:31.710 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:17:31 smithi175 ceph-mon[67352]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:17:31.711 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:17:31 smithi175 ceph-mon[67352]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' 2024-03-18T23:17:31.782 INFO:tasks.workunit.client.0.smithi049.stdout: 1 971349 9.58 MB/sec execute 393 sec latency 76.467 ms 2024-03-18T23:17:32.782 INFO:tasks.workunit.client.0.smithi049.stdout: 1 973461 9.58 MB/sec execute 394 sec latency 22.154 ms 2024-03-18T23:17:33.564 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:17:33 smithi139 ceph-mon[67282]: pgmap v289: 97 pgs: 97 active+clean; 93 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.4 MiB/s wr, 275 op/s 2024-03-18T23:17:33.629 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:17:33 smithi049 ceph-mon[62991]: pgmap v289: 97 pgs: 97 active+clean; 93 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.4 MiB/s wr, 275 op/s 2024-03-18T23:17:33.710 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:17:33 smithi175 ceph-mon[67352]: pgmap v289: 97 pgs: 97 active+clean; 93 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.4 MiB/s wr, 275 op/s 2024-03-18T23:17:33.782 INFO:tasks.workunit.client.0.smithi049.stdout: 1 974259 9.56 MB/sec execute 395 sec latency 39.085 ms 2024-03-18T23:17:34.782 INFO:tasks.workunit.client.0.smithi049.stdout: 1 976395 9.57 MB/sec execute 396 sec latency 21.981 ms 2024-03-18T23:17:35.564 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:17:35 smithi139 ceph-mon[67282]: pgmap v290: 97 pgs: 97 active+clean; 100 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.3 MiB/s wr, 254 op/s 2024-03-18T23:17:35.629 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:17:35 smithi049 ceph-mon[62991]: pgmap v290: 97 pgs: 97 active+clean; 100 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.3 MiB/s wr, 254 op/s 2024-03-18T23:17:35.710 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:17:35 smithi175 ceph-mon[67352]: pgmap v290: 97 pgs: 97 active+clean; 100 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.3 MiB/s wr, 254 op/s 2024-03-18T23:17:35.782 INFO:tasks.workunit.client.0.smithi049.stdout: 1 977462 9.56 MB/sec execute 397 sec latency 19.309 ms 2024-03-18T23:17:36.401 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:36.400+0000 7f31a193b640 1 -- 172.21.15.49:0/642458658 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f319c1056c0 msgr2=0x7f319c105aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:17:36.401 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:36.400+0000 7f31a193b640 1 --2- 172.21.15.49:0/642458658 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f319c1056c0 0x7f319c105aa0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f318c009900 tx=0x7f318c02bc30 comp rx=0 tx=0).stop 2024-03-18T23:17:36.401 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:36.400+0000 7f31a193b640 1 -- 172.21.15.49:0/642458658 shutdown_connections 2024-03-18T23:17:36.401 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:36.400+0000 7f31a193b640 1 --2- 172.21.15.49:0/642458658 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f319c101330 0x7f319c10d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:36.401 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:36.400+0000 7f31a193b640 1 --2- 172.21.15.49:0/642458658 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f319c100970 0x7f319c100df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:36.402 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:36.400+0000 7f31a193b640 1 --2- 172.21.15.49:0/642458658 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f319c1056c0 0x7f319c105aa0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:36.402 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:36.400+0000 7f31a193b640 1 -- 172.21.15.49:0/642458658 >> 172.21.15.49:0/642458658 conn(0x7f319c077570 msgr2=0x7f319c077980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:17:36.402 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:36.400+0000 7f31a193b640 1 -- 172.21.15.49:0/642458658 shutdown_connections 2024-03-18T23:17:36.402 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:36.400+0000 7f31a193b640 1 -- 172.21.15.49:0/642458658 wait complete. 2024-03-18T23:17:36.402 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:36.401+0000 7f31a193b640 1 Processor -- start 2024-03-18T23:17:36.402 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:36.401+0000 7f31a193b640 1 -- start start 2024-03-18T23:17:36.402 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:36.401+0000 7f31a193b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f319c100970 0x7f319c198100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:17:36.402 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:36.401+0000 7f31a193b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f319c101330 0x7f319c198640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:17:36.402 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:36.401+0000 7f31a193b640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f319c1056c0 0x7f319c19c970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:17:36.402 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:36.401+0000 7f31a193b640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f319c110ba0 con 0x7f319c100970 2024-03-18T23:17:36.402 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:36.401+0000 7f31a193b640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f319c110a20 con 0x7f319c101330 2024-03-18T23:17:36.402 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:36.401+0000 7f31a193b640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f319c110d20 con 0x7f319c1056c0 2024-03-18T23:17:36.403 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:36.401+0000 7f319affd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f319c100970 0x7f319c198100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:17:36.403 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:36.401+0000 7f319affd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f319c100970 0x7f319c198100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:34380/0 (socket says 172.21.15.49:34380) 2024-03-18T23:17:36.403 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:36.401+0000 7f319affd640 1 -- 172.21.15.49:0/1640394090 learned_addr learned my addr 172.21.15.49:0/1640394090 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:17:36.403 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:36.401+0000 7f319b7fe640 1 --2- 172.21.15.49:0/1640394090 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f319c1056c0 0x7f319c19c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:17:36.403 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:36.401+0000 7f3193fff640 1 --2- 172.21.15.49:0/1640394090 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f319c101330 0x7f319c198640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:17:36.403 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:36.401+0000 7f319affd640 1 -- 172.21.15.49:0/1640394090 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f319c1056c0 msgr2=0x7f319c19c970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:17:36.403 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:36.401+0000 7f319affd640 1 --2- 172.21.15.49:0/1640394090 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f319c1056c0 0x7f319c19c970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:36.403 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:36.401+0000 7f319affd640 1 -- 172.21.15.49:0/1640394090 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f319c101330 msgr2=0x7f319c198640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:17:36.403 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:36.401+0000 7f319affd640 1 --2- 172.21.15.49:0/1640394090 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f319c101330 0x7f319c198640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:36.403 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:36.401+0000 7f319affd640 1 -- 172.21.15.49:0/1640394090 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f319c19d0a0 con 0x7f319c100970 2024-03-18T23:17:36.403 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:36.401+0000 7f319affd640 1 --2- 172.21.15.49:0/1640394090 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f319c100970 0x7f319c198100 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f318c005940 tx=0x7f318c0043b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:17:36.403 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:36.402+0000 7f31a0939640 1 -- 172.21.15.49:0/1640394090 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f318c041070 con 0x7f319c100970 2024-03-18T23:17:36.403 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:36.402+0000 7f31a0939640 1 -- 172.21.15.49:0/1640394090 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f318c02ccc0 con 0x7f319c100970 2024-03-18T23:17:36.404 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:36.402+0000 7f31a193b640 1 -- 172.21.15.49:0/1640394090 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f319c19d330 con 0x7f319c100970 2024-03-18T23:17:36.404 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:36.402+0000 7f31a0939640 1 -- 172.21.15.49:0/1640394090 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f318c003e20 con 0x7f319c100970 2024-03-18T23:17:36.404 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:36.402+0000 7f31a193b640 1 -- 172.21.15.49:0/1640394090 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f319c1a4c20 con 0x7f319c100970 2024-03-18T23:17:36.409 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:36.404+0000 7f31a0939640 1 -- 172.21.15.49:0/1640394090 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 25) ==== 137421+0+0 (secure 0 0 0) 0x7f318c044070 con 0x7f319c100970 2024-03-18T23:17:36.409 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:36.404+0000 7f31a4601740 1 Processor -- start 2024-03-18T23:17:36.409 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:36.404+0000 7f31a4601740 1 -- start start 2024-03-18T23:17:36.409 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:36.405+0000 7f31a0939640 1 --2- 172.21.15.49:0/1640394090 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f31880a2a90 0x7f31880a4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:17:36.409 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:36.405+0000 7f31a4601740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556f35e15ef0 0x556f35e162d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:17:36.409 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:36.405+0000 7f31a4601740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556f35e16810 0x556f35e1f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:17:36.410 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:36.405+0000 7f31a4601740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556f35e1f5e0 0x556f35e219d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:17:36.410 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:36.405+0000 7f31a4601740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x556f35d79be0 con 0x556f35e1f5e0 2024-03-18T23:17:36.410 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:36.405+0000 7f31a4601740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x556f35d22960 con 0x556f35e15ef0 2024-03-18T23:17:36.410 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:36.405+0000 7f31a4601740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x556f35d75c80 con 0x556f35e16810 2024-03-18T23:17:36.410 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:36.405+0000 7f319b7fe640 1 --2- 172.21.15.49:0/1640394090 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f31880a2a90 0x7f31880a4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:17:36.410 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:36.405+0000 7f319affd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556f35e15ef0 0x556f35e162d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:17:36.410 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:36.405+0000 7f3193fff640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556f35e16810 0x556f35e1f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:17:36.410 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:36.405+0000 7f31a0939640 1 -- 172.21.15.49:0/1640394090 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8556+0+0 (secure 0 0 0) 0x7f318c0f0860 con 0x7f319c100970 2024-03-18T23:17:36.410 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:36.405+0000 7f3193fff640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556f35e16810 0x556f35e1f0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.49:54858/0 (socket says 172.21.15.49:54858) 2024-03-18T23:17:36.410 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:36.405+0000 7f319affd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556f35e15ef0 0x556f35e162d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.49:42980/0 (socket says 172.21.15.49:42980) 2024-03-18T23:17:36.410 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:36.405+0000 7f3193fff640 1 -- 172.21.15.49:0/836008537 learned_addr learned my addr 172.21.15.49:0/836008537 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:17:36.410 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:36.405+0000 7f319b7fe640 1 --2- 172.21.15.49:0/836008537 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556f35e1f5e0 0x556f35e219d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:17:36.410 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:36.405+0000 7f3193fff640 1 -- 172.21.15.49:0/836008537 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556f35e15ef0 msgr2=0x556f35e162d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:17:36.410 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:36.405+0000 7f3193fff640 1 --2- 172.21.15.49:0/836008537 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556f35e15ef0 0x556f35e162d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:36.411 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:36.405+0000 7f3193fff640 1 -- 172.21.15.49:0/836008537 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556f35e1f5e0 msgr2=0x556f35e219d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:17:36.411 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:36.405+0000 7f3193fff640 1 --2- 172.21.15.49:0/836008537 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556f35e1f5e0 0x556f35e219d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:36.411 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:36.405+0000 7f3193fff640 1 -- 172.21.15.49:0/836008537 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556f35d19d20 con 0x556f35e16810 2024-03-18T23:17:36.411 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:36.405+0000 7f319b7fe640 1 --2- 172.21.15.49:0/1640394090 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f31880a2a90 0x7f31880a4f50 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f3194009800 tx=0x7f3194009e10 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:17:36.411 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:36.405+0000 7f3193fff640 1 --2- 172.21.15.49:0/836008537 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556f35e16810 0x556f35e1f0a0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f318400bcb0 tx=0x7f31840107a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=df5ce3ce196ffacc server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:17:36.411 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:36.406+0000 7f3191ffb640 1 -- 172.21.15.49:0/836008537 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3184004030 con 0x556f35e16810 2024-03-18T23:17:36.411 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:36.406+0000 7f3191ffb640 1 -- 172.21.15.49:0/836008537 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f31840041d0 con 0x556f35e16810 2024-03-18T23:17:36.411 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:36.406+0000 7f3191ffb640 1 -- 172.21.15.49:0/836008537 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31840116a0 con 0x556f35e16810 2024-03-18T23:17:36.411 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:36.406+0000 7f31a4601740 1 -- 172.21.15.49:0/836008537 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556f35e16810 msgr2=0x556f35e1f0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:17:36.411 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:36.406+0000 7f31a4601740 1 --2- 172.21.15.49:0/836008537 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556f35e16810 0x556f35e1f0a0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f318400bcb0 tx=0x7f31840107a0 comp rx=0 tx=0).stop 2024-03-18T23:17:36.411 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:36.406+0000 7f31a4601740 1 -- 172.21.15.49:0/836008537 shutdown_connections 2024-03-18T23:17:36.411 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:36.406+0000 7f31a4601740 1 --2- 172.21.15.49:0/836008537 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556f35e1f5e0 0x556f35e219d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:36.411 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:36.406+0000 7f31a4601740 1 --2- 172.21.15.49:0/836008537 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556f35e16810 0x556f35e1f0a0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:36.411 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:36.406+0000 7f31a4601740 1 --2- 172.21.15.49:0/836008537 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556f35e15ef0 0x556f35e162d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:36.411 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:36.406+0000 7f31a4601740 1 -- 172.21.15.49:0/836008537 >> 172.21.15.49:0/836008537 conn(0x556f35e10810 msgr2=0x556f35e10bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:17:36.411 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:36.406+0000 7f31a4601740 1 -- 172.21.15.49:0/836008537 shutdown_connections 2024-03-18T23:17:36.412 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:36.406+0000 7f31a4601740 1 -- 172.21.15.49:0/836008537 wait complete. 2024-03-18T23:17:36.412 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:36.407+0000 7f31a4601740 1 -- start start 2024-03-18T23:17:36.412 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:36.407+0000 7f31a4601740 1 Processor -- start 2024-03-18T23:17:36.412 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:36.407+0000 7f31a4601740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556f35e15ef0 0x556f35e1ef80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:17:36.412 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:36.407+0000 7f31a4601740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556f35e16810 0x556f35e1c310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:17:36.412 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:36.407+0000 7f31a4601740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556f35e1f5e0 0x556f35e1c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:17:36.412 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:36.407+0000 7f31a4601740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x556f35e22210 con 0x556f35e15ef0 2024-03-18T23:17:36.412 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:36.407+0000 7f31a4601740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x556f35d35db0 con 0x556f35e1f5e0 2024-03-18T23:17:36.412 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:36.407+0000 7f31a4601740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x556f35e22390 con 0x556f35e16810 2024-03-18T23:17:36.412 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:36.407+0000 7f31a4601740 10 client.0 _ll_register_callbacks cb 0x556f35d3ae70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T23:17:36.412 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:36.407+0000 7f319affd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556f35e15ef0 0x556f35e1ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:17:36.412 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:36.407+0000 7f319affd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556f35e15ef0 0x556f35e1ef80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:34398/0 (socket says 172.21.15.49:34398) 2024-03-18T23:17:36.412 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:36.407+0000 7f319affd640 1 -- 172.21.15.49:0/3009977797 learned_addr learned my addr 172.21.15.49:0/3009977797 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:17:36.412 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:36.407+0000 7f3193fff640 1 --2- 172.21.15.49:0/3009977797 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556f35e16810 0x556f35e1c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:17:36.412 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:36.407+0000 7f319b7fe640 1 --2- 172.21.15.49:0/3009977797 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556f35e1f5e0 0x556f35e1c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:17:36.412 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:36.407+0000 7f319affd640 1 -- 172.21.15.49:0/3009977797 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556f35e16810 msgr2=0x556f35e1c310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:17:36.413 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:36.407+0000 7f319affd640 1 --2- 172.21.15.49:0/3009977797 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556f35e16810 0x556f35e1c310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:36.413 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:36.407+0000 7f319affd640 1 -- 172.21.15.49:0/3009977797 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556f35e1f5e0 msgr2=0x556f35e1c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:17:36.413 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:36.407+0000 7f319affd640 1 --2- 172.21.15.49:0/3009977797 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556f35e1f5e0 0x556f35e1c8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:36.413 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:36.408+0000 7f319affd640 1 -- 172.21.15.49:0/3009977797 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556f35d34c60 con 0x556f35e15ef0 2024-03-18T23:17:36.413 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:36.408+0000 7f319affd640 1 --2- 172.21.15.49:0/3009977797 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556f35e15ef0 0x556f35e1ef80 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f318c0bcd40 tx=0x7f318c0bcc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d03abf52c0e62d13 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:17:36.413 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:36.408+0000 7f317affd640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-18T23:17:36.413 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:36.408+0000 7f317affd640 1 -- 172.21.15.49:0/3009977797 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f318c04b040 con 0x556f35e15ef0 2024-03-18T23:17:36.413 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:36.408+0000 7f317affd640 1 -- 172.21.15.49:0/3009977797 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8556+0+0 (secure 0 0 0) 0x7f318c0b96b0 con 0x556f35e15ef0 2024-03-18T23:17:36.413 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:36.408+0000 7f31a4601740 1 -- 172.21.15.49:0/3009977797 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f31840041d0 con 0x556f35e15ef0 2024-03-18T23:17:36.413 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:36.408+0000 7f317affd640 1 -- 172.21.15.49:0/3009977797 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f318c0c10e0 con 0x556f35e15ef0 2024-03-18T23:17:36.413 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:36.408+0000 7f31a4601740 10 client.16209 fetch_fsmap learned FSMap version 32 2024-03-18T23:17:36.413 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:36.408+0000 7f31a4601740 1 -- 172.21.15.49:0/3009977797 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x556f35a8e210 con 0x556f35e15ef0 2024-03-18T23:17:36.413 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:36.408+0000 7f317affd640 1 -- 172.21.15.49:0/3009977797 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f31840041d0 con 0x556f35e15ef0 2024-03-18T23:17:36.413 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:36.408+0000 7f317affd640 1 -- 172.21.15.49:0/3009977797 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f318c0e2150 con 0x556f35e15ef0 2024-03-18T23:17:36.413 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:36.408+0000 7f31a4601740 10 client.16209 fetch_fsmap finished waiting for FSMap version 32 2024-03-18T23:17:36.413 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:36.408+0000 7f31a4601740 10 client.16209 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T23:17:36.414 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:36.408+0000 7f31a4601740 20 client.16209 populate_metadata read hostname 'smithi049' 2024-03-18T23:17:36.414 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:36.408+0000 7f31a4601740 1 --2- 172.21.15.49:0/3009977797 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x556f35e2f450 0x556f35e318a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:17:36.414 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:36.408+0000 7f31a4601740 4 client.16209 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T23:17:36.414 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:36.408+0000 7f31a4601740 1 -- 172.21.15.49:0/3009977797 --> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556f35d87120 con 0x556f35e2f450 2024-03-18T23:17:36.414 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:36.409+0000 7f3193fff640 1 --2- 172.21.15.49:0/3009977797 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x556f35e2f450 0x556f35e318a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:17:36.414 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:36.409+0000 7f3193fff640 1 --2- 172.21.15.49:0/3009977797 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x556f35e2f450 0x556f35e318a0 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=447160324d00b141 server_cookie=a84472dbaa923fbd in_seq=0 out_seq=0 2024-03-18T23:17:36.414 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:36.409+0000 7f317affd640 10 client.16209 ms_handle_connect on v2:172.21.15.139:6838/4280128209 2024-03-18T23:17:36.415 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:36.411+0000 7f317affd640 1 -- 172.21.15.49:0/3009977797 <== mds.0 v2:172.21.15.139:6838/4280128209 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x556f35d87120 con 0x556f35e2f450 2024-03-18T23:17:36.415 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:36.411+0000 7f317affd640 10 client.16209 handle_command_reply: tid=0 2024-03-18T23:17:36.415 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:36.411+0000 7f31a4601740 1 client.16209 shutdown 2024-03-18T23:17:36.415 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:36.411+0000 7f31a4601740 1 -- 172.21.15.49:0/3009977797 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556f35e15ef0 msgr2=0x556f35e1ef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:17:36.415 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:36.411+0000 7f31a4601740 1 --2- 172.21.15.49:0/3009977797 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556f35e15ef0 0x556f35e1ef80 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f318c0bcd40 tx=0x7f318c0bcc40 comp rx=0 tx=0).stop 2024-03-18T23:17:36.415 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:36.411+0000 7f31a4601740 1 -- 172.21.15.49:0/3009977797 shutdown_connections 2024-03-18T23:17:36.415 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:36.411+0000 7f31a4601740 1 --2- 172.21.15.49:0/3009977797 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x556f35e2f450 0x556f35e318a0 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:36.415 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:36.411+0000 7f31a4601740 1 --2- 172.21.15.49:0/3009977797 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556f35e1f5e0 0x556f35e1c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:36.415 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:36.411+0000 7f31a4601740 1 --2- 172.21.15.49:0/3009977797 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556f35e16810 0x556f35e1c310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:36.415 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:36.411+0000 7f31a4601740 1 --2- 172.21.15.49:0/3009977797 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556f35e15ef0 0x556f35e1ef80 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:36.416 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:36.411+0000 7f317affd640 0 client.16209 ms_handle_reset on v2:172.21.15.139:6838/4280128209 2024-03-18T23:17:36.416 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:36.411+0000 7f31a4601740 1 -- 172.21.15.49:0/3009977797 >> 172.21.15.49:0/3009977797 conn(0x556f35e10810 msgr2=0x556f35e0fab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:17:36.416 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:36.411+0000 7f31a4601740 1 -- 172.21.15.49:0/3009977797 shutdown_connections 2024-03-18T23:17:36.416 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:36.411+0000 7f31a4601740 1 -- 172.21.15.49:0/3009977797 wait complete. 2024-03-18T23:17:36.416 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:36.411+0000 7f31a4601740 20 client.16209 trim_cache size 0 max 16384 2024-03-18T23:17:36.426 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:36.426+0000 7f31a4601740 1 Processor -- start 2024-03-18T23:17:36.426 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:36.426+0000 7f31a4601740 1 -- start start 2024-03-18T23:17:36.426 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:36.426+0000 7f31a4601740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556f35e15ef0 0x556f35e0fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:17:36.426 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:36.426+0000 7f31a4601740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556f35e16810 0x556f35e37150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:17:36.426 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:36.426+0000 7f31a4601740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556f35e1f5e0 0x556f35e275d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:17:36.426 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:36.426+0000 7f31a4601740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f319c1ad450 con 0x556f35e15ef0 2024-03-18T23:17:36.427 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:36.426+0000 7f31a4601740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f319c1aef90 con 0x556f35e1f5e0 2024-03-18T23:17:36.427 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:36.426+0000 7f31a4601740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f319c1adb20 con 0x556f35e16810 2024-03-18T23:17:36.427 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:36.426+0000 7f319affd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556f35e15ef0 0x556f35e0fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:17:36.427 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:36.426+0000 7f319affd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556f35e15ef0 0x556f35e0fcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:34414/0 (socket says 172.21.15.49:34414) 2024-03-18T23:17:36.427 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:36.426+0000 7f319affd640 1 -- 172.21.15.49:0/3941202345 learned_addr learned my addr 172.21.15.49:0/3941202345 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:17:36.427 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:36.426+0000 7f3193fff640 1 --2- 172.21.15.49:0/3941202345 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556f35e16810 0x556f35e37150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:17:36.427 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:36.426+0000 7f319b7fe640 1 --2- 172.21.15.49:0/3941202345 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556f35e1f5e0 0x556f35e275d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:17:36.427 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:36.427+0000 7f319affd640 1 -- 172.21.15.49:0/3941202345 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556f35e16810 msgr2=0x556f35e37150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:17:36.427 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:36.427+0000 7f319affd640 1 --2- 172.21.15.49:0/3941202345 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556f35e16810 0x556f35e37150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:36.427 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:36.427+0000 7f319affd640 1 -- 172.21.15.49:0/3941202345 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556f35e1f5e0 msgr2=0x556f35e275d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:17:36.427 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:36.427+0000 7f319affd640 1 --2- 172.21.15.49:0/3941202345 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556f35e1f5e0 0x556f35e275d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:36.427 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:36.427+0000 7f319affd640 1 -- 172.21.15.49:0/3941202345 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556f35b5a620 con 0x556f35e15ef0 2024-03-18T23:17:36.427 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:36.427+0000 7f319affd640 1 --2- 172.21.15.49:0/3941202345 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556f35e15ef0 0x556f35e0fcf0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f318c006010 tx=0x7f318c0bb550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe7e6d3e427666c0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:17:36.427 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:36.427+0000 7f31917fa640 1 -- 172.21.15.49:0/3941202345 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f318c04b040 con 0x556f35e15ef0 2024-03-18T23:17:36.427 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:36.427+0000 7f31917fa640 1 -- 172.21.15.49:0/3941202345 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f318c0c5730 con 0x556f35e15ef0 2024-03-18T23:17:36.428 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:36.427+0000 7f31917fa640 1 -- 172.21.15.49:0/3941202345 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f318c0bc490 con 0x556f35e15ef0 2024-03-18T23:17:36.431 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:36.428+0000 7f31a4601740 1 -- 172.21.15.49:0/3941202345 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556f35e15ef0 msgr2=0x556f35e0fcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:17:36.431 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:36.428+0000 7f31a4601740 1 --2- 172.21.15.49:0/3941202345 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556f35e15ef0 0x556f35e0fcf0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f318c006010 tx=0x7f318c0bb550 comp rx=0 tx=0).stop 2024-03-18T23:17:36.431 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:36.428+0000 7f31a4601740 1 -- 172.21.15.49:0/3941202345 shutdown_connections 2024-03-18T23:17:36.431 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:36.428+0000 7f31a4601740 1 --2- 172.21.15.49:0/3941202345 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556f35e1f5e0 0x556f35e275d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:36.431 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:36.428+0000 7f31a4601740 1 --2- 172.21.15.49:0/3941202345 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556f35e16810 0x556f35e37150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:36.431 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:36.428+0000 7f31a4601740 1 --2- 172.21.15.49:0/3941202345 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556f35e15ef0 0x556f35e0fcf0 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:36.431 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:36.428+0000 7f31a4601740 1 -- 172.21.15.49:0/3941202345 >> 172.21.15.49:0/3941202345 conn(0x556f35e10810 msgr2=0x556f35e124f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:17:36.431 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:36.428+0000 7f31a4601740 1 -- 172.21.15.49:0/3941202345 shutdown_connections 2024-03-18T23:17:36.431 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:36.428+0000 7f31a4601740 1 -- 172.21.15.49:0/3941202345 wait complete. 2024-03-18T23:17:36.431 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:36.428+0000 7f31a4601740 1 -- start start 2024-03-18T23:17:36.432 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:36.428+0000 7f31a4601740 1 Processor -- start 2024-03-18T23:17:36.432 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:36.428+0000 7f31a4601740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556f35e15ef0 0x556f35e235e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:17:36.432 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:36.428+0000 7f31a4601740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556f35e16810 0x556f35e23b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:17:36.432 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:36.428+0000 7f31a4601740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556f35e1f5e0 0x556f35e94780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:17:36.432 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:36.428+0000 7f31a4601740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f319c1ae8c0 con 0x556f35e1f5e0 2024-03-18T23:17:36.432 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:36.429+0000 7f31a4601740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f319c1ae1f0 con 0x556f35e15ef0 2024-03-18T23:17:36.432 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:36.429+0000 7f31a4601740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f319c1ab450 con 0x556f35e16810 2024-03-18T23:17:36.432 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:36.429+0000 7f31a4601740 10 client.0 _ll_register_callbacks cb 0x7f318c0b93b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T23:17:36.432 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:36.429+0000 7f319b7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556f35e1f5e0 0x556f35e94780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:17:36.432 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:36.429+0000 7f319b7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556f35e1f5e0 0x556f35e94780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:34418/0 (socket says 172.21.15.49:34418) 2024-03-18T23:17:36.432 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:36.429+0000 7f319b7fe640 1 -- 172.21.15.49:0/3521573287 learned_addr learned my addr 172.21.15.49:0/3521573287 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:17:36.432 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:36.429+0000 7f319affd640 1 --2- 172.21.15.49:0/3521573287 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556f35e15ef0 0x556f35e235e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:17:36.432 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:36.429+0000 7f3193fff640 1 --2- 172.21.15.49:0/3521573287 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556f35e16810 0x556f35e23b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:17:36.432 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:36.429+0000 7f319b7fe640 1 -- 172.21.15.49:0/3521573287 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556f35e16810 msgr2=0x556f35e23b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:17:36.432 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:36.429+0000 7f319b7fe640 1 --2- 172.21.15.49:0/3521573287 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556f35e16810 0x556f35e23b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:36.433 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:36.429+0000 7f319b7fe640 1 -- 172.21.15.49:0/3521573287 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556f35e15ef0 msgr2=0x556f35e235e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:17:36.433 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:36.429+0000 7f319b7fe640 1 --2- 172.21.15.49:0/3521573287 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556f35e15ef0 0x556f35e235e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:36.433 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:36.429+0000 7f319b7fe640 1 -- 172.21.15.49:0/3521573287 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556f35e94cc0 con 0x556f35e1f5e0 2024-03-18T23:17:36.433 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:36.429+0000 7f319b7fe640 1 --2- 172.21.15.49:0/3521573287 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556f35e1f5e0 0x556f35e94780 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f3194005fd0 tx=0x7f319401c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=575941f56c8f11a9 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:17:36.433 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:36.429+0000 7f317affd640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-18T23:17:36.433 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:36.429+0000 7f317affd640 1 -- 172.21.15.49:0/3521573287 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3194003e00 con 0x556f35e1f5e0 2024-03-18T23:17:36.433 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:36.429+0000 7f31a4601740 1 -- 172.21.15.49:0/3521573287 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f318c0c5730 con 0x556f35e1f5e0 2024-03-18T23:17:36.433 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:36.429+0000 7f317affd640 1 -- 172.21.15.49:0/3521573287 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8556+0+0 (secure 0 0 0) 0x7f3194004740 con 0x556f35e1f5e0 2024-03-18T23:17:36.433 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:36.429+0000 7f317affd640 1 -- 172.21.15.49:0/3521573287 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f3194004b20 con 0x556f35e1f5e0 2024-03-18T23:17:36.433 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:36.429+0000 7f31a4601740 10 client.16215 fetch_fsmap learned FSMap version 32 2024-03-18T23:17:36.433 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:36.429+0000 7f31a4601740 1 -- 172.21.15.49:0/3521573287 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x556f35e95fa0 con 0x556f35e1f5e0 2024-03-18T23:17:36.433 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:36.430+0000 7f317affd640 1 -- 172.21.15.49:0/3521573287 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f318c0c5730 con 0x556f35e1f5e0 2024-03-18T23:17:36.433 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:36.430+0000 7f317affd640 1 -- 172.21.15.49:0/3521573287 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f319401f070 con 0x556f35e1f5e0 2024-03-18T23:17:36.433 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:36.430+0000 7f31a4601740 10 client.16215 fetch_fsmap finished waiting for FSMap version 32 2024-03-18T23:17:36.433 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:36.430+0000 7f31a4601740 10 client.16215 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T23:17:36.433 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:36.430+0000 7f31a4601740 20 client.16215 populate_metadata read hostname 'smithi049' 2024-03-18T23:17:36.434 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:36.430+0000 7f31a4601740 1 --2- 172.21.15.49:0/3521573287 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x556f35e96710 0x556f35e98b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:17:36.434 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:36.430+0000 7f31a4601740 4 client.16215 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T23:17:36.434 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:36.430+0000 7f31a4601740 1 -- 172.21.15.49:0/3521573287 --> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556f35d81470 con 0x556f35e96710 2024-03-18T23:17:36.434 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:36.430+0000 7f319affd640 1 --2- 172.21.15.49:0/3521573287 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x556f35e96710 0x556f35e98b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:17:36.434 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:36.430+0000 7f317affd640 10 client.16215 ms_handle_connect on v2:172.21.15.139:6838/4280128209 2024-03-18T23:17:36.434 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:36.430+0000 7f319affd640 1 --2- 172.21.15.49:0/3521573287 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x556f35e96710 0x556f35e98b00 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b3228debf69f8f28 server_cookie=34c0e96977352e68 in_seq=0 out_seq=0 2024-03-18T23:17:36.591 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:36.591+0000 7f317affd640 1 -- 172.21.15.49:0/3521573287 <== mds.0 v2:172.21.15.139:6838/4280128209 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556f35d81470 con 0x556f35e96710 2024-03-18T23:17:36.591 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:36.591+0000 7f317affd640 10 client.16215 handle_command_reply: tid=0 2024-03-18T23:17:36.591 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:36.591+0000 7f31a4601740 1 client.16215 shutdown 2024-03-18T23:17:36.591 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:36.591+0000 7f31a4601740 1 -- 172.21.15.49:0/3521573287 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556f35e1f5e0 msgr2=0x556f35e94780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:17:36.592 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:36.591+0000 7f31a4601740 1 --2- 172.21.15.49:0/3521573287 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556f35e1f5e0 0x556f35e94780 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f3194005fd0 tx=0x7f319401c000 comp rx=0 tx=0).stop 2024-03-18T23:17:36.592 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:36.591+0000 7f31a4601740 1 -- 172.21.15.49:0/3521573287 shutdown_connections 2024-03-18T23:17:36.592 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:36.591+0000 7f31a4601740 1 --2- 172.21.15.49:0/3521573287 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x556f35e96710 0x556f35e98b00 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:36.592 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:36.591+0000 7f31a4601740 1 --2- 172.21.15.49:0/3521573287 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556f35e1f5e0 0x556f35e94780 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:36.592 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:36.591+0000 7f317affd640 0 client.16215 ms_handle_reset on v2:172.21.15.139:6838/4280128209 2024-03-18T23:17:36.592 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:36.591+0000 7f31a4601740 1 --2- 172.21.15.49:0/3521573287 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556f35e16810 0x556f35e23b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:36.592 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:36.591+0000 7f31a4601740 1 --2- 172.21.15.49:0/3521573287 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556f35e15ef0 0x556f35e235e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:36.592 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:36.591+0000 7f31a4601740 1 -- 172.21.15.49:0/3521573287 >> 172.21.15.49:0/3521573287 conn(0x556f35e10810 msgr2=0x556f35e16e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:17:36.592 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:36.592+0000 7f31a4601740 1 -- 172.21.15.49:0/3521573287 shutdown_connections 2024-03-18T23:17:36.592 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:36.592+0000 7f31a4601740 1 -- 172.21.15.49:0/3521573287 wait complete. 2024-03-18T23:17:36.592 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:36.592+0000 7f31a4601740 20 client.16215 trim_cache size 0 max 16384 2024-03-18T23:17:36.592 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:{ 2024-03-18T23:17:36.592 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "message": "", 2024-03-18T23:17:36.593 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "return_code": 0 2024-03-18T23:17:36.593 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:} 2024-03-18T23:17:36.593 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:36.592+0000 7f31a193b640 1 -- 172.21.15.49:0/1640394090 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f31880a2a90 msgr2=0x7f31880a4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:17:36.593 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:36.592+0000 7f31a193b640 1 --2- 172.21.15.49:0/1640394090 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f31880a2a90 0x7f31880a4f50 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f3194009800 tx=0x7f3194009e10 comp rx=0 tx=0).stop 2024-03-18T23:17:36.593 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:36.592+0000 7f31a193b640 1 -- 172.21.15.49:0/1640394090 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f319c100970 msgr2=0x7f319c198100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:17:36.593 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:36.592+0000 7f31a193b640 1 --2- 172.21.15.49:0/1640394090 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f319c100970 0x7f319c198100 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f318c005940 tx=0x7f318c0043b0 comp rx=0 tx=0).stop 2024-03-18T23:17:36.594 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:36.592+0000 7f31a193b640 1 -- 172.21.15.49:0/1640394090 shutdown_connections 2024-03-18T23:17:36.594 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:36.592+0000 7f31a193b640 1 --2- 172.21.15.49:0/1640394090 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f31880a2a90 0x7f31880a4f50 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:36.594 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:36.592+0000 7f31a193b640 1 --2- 172.21.15.49:0/1640394090 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f319c1056c0 0x7f319c19c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:36.594 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:36.592+0000 7f31a193b640 1 --2- 172.21.15.49:0/1640394090 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f319c101330 0x7f319c198640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:36.594 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:36.592+0000 7f31a193b640 1 --2- 172.21.15.49:0/1640394090 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f319c100970 0x7f319c198100 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:36.594 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:36.592+0000 7f31a193b640 1 -- 172.21.15.49:0/1640394090 >> 172.21.15.49:0/1640394090 conn(0x7f319c077570 msgr2=0x7f319c069120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:17:36.594 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:36.592+0000 7f31a193b640 1 -- 172.21.15.49:0/1640394090 shutdown_connections 2024-03-18T23:17:36.594 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:36.592+0000 7f31a193b640 1 -- 172.21.15.49:0/1640394090 wait complete. 2024-03-18T23:17:36.782 INFO:tasks.workunit.client.0.smithi049.stdout: 1 978712 9.55 MB/sec execute 398 sec latency 104.898 ms 2024-03-18T23:17:37.564 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:17:37 smithi139 ceph-mon[67282]: pgmap v291: 97 pgs: 97 active+clean; 100 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 983 KiB/s rd, 3.8 MiB/s wr, 227 op/s 2024-03-18T23:17:37.629 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:17:37 smithi049 ceph-mon[62991]: pgmap v291: 97 pgs: 97 active+clean; 100 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 983 KiB/s rd, 3.8 MiB/s wr, 227 op/s 2024-03-18T23:17:37.710 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:17:37 smithi175 ceph-mon[67352]: pgmap v291: 97 pgs: 97 active+clean; 100 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 983 KiB/s rd, 3.8 MiB/s wr, 227 op/s 2024-03-18T23:17:37.782 INFO:tasks.workunit.client.0.smithi049.stdout: 1 980667 9.56 MB/sec execute 399 sec latency 21.573 ms 2024-03-18T23:17:38.782 INFO:tasks.workunit.client.0.smithi049.stdout: 1 981942 9.55 MB/sec execute 400 sec latency 25.467 ms 2024-03-18T23:17:39.564 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:17:39 smithi139 ceph-mon[67282]: pgmap v292: 97 pgs: 97 active+clean; 106 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.9 MiB/s wr, 301 op/s 2024-03-18T23:17:39.629 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:17:39 smithi049 ceph-mon[62991]: pgmap v292: 97 pgs: 97 active+clean; 106 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.9 MiB/s wr, 301 op/s 2024-03-18T23:17:39.710 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:17:39 smithi175 ceph-mon[67352]: pgmap v292: 97 pgs: 97 active+clean; 106 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.9 MiB/s wr, 301 op/s 2024-03-18T23:17:39.782 INFO:tasks.workunit.client.0.smithi049.stdout: 1 984087 9.56 MB/sec execute 401 sec latency 23.941 ms 2024-03-18T23:17:40.782 INFO:tasks.workunit.client.0.smithi049.stdout: 1 984841 9.54 MB/sec execute 402 sec latency 27.871 ms 2024-03-18T23:17:41.511 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:17:41 smithi175 ceph-mon[67352]: pgmap v293: 97 pgs: 97 active+clean; 106 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.7 MiB/s wr, 140 op/s 2024-03-18T23:17:41.564 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:17:41 smithi139 ceph-mon[67282]: pgmap v293: 97 pgs: 97 active+clean; 106 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.7 MiB/s wr, 140 op/s 2024-03-18T23:17:41.629 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:17:41 smithi049 ceph-mon[62991]: pgmap v293: 97 pgs: 97 active+clean; 106 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.7 MiB/s wr, 140 op/s 2024-03-18T23:17:41.782 INFO:tasks.workunit.client.0.smithi049.stdout: 1 986515 9.53 MB/sec execute 403 sec latency 117.210 ms 2024-03-18T23:17:42.564 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:17:42 smithi139 ceph-mon[67282]: pgmap v294: 97 pgs: 97 active+clean; 105 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.7 MiB/s wr, 255 op/s 2024-03-18T23:17:42.629 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:17:42 smithi049 ceph-mon[62991]: pgmap v294: 97 pgs: 97 active+clean; 105 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.7 MiB/s wr, 255 op/s 2024-03-18T23:17:42.710 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:17:42 smithi175 ceph-mon[67352]: pgmap v294: 97 pgs: 97 active+clean; 105 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.7 MiB/s wr, 255 op/s 2024-03-18T23:17:42.782 INFO:tasks.workunit.client.0.smithi049.stdout: 1 987853 9.54 MB/sec execute 404 sec latency 37.323 ms 2024-03-18T23:17:43.782 INFO:tasks.workunit.client.0.smithi049.stdout: 1 989181 9.53 MB/sec execute 405 sec latency 28.147 ms 2024-03-18T23:17:44.783 INFO:tasks.workunit.client.0.smithi049.stdout: 1 991241 9.53 MB/sec execute 406 sec latency 18.667 ms 2024-03-18T23:17:44.814 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:17:44 smithi139 ceph-mon[67282]: pgmap v295: 97 pgs: 97 active+clean; 96 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.8 MiB/s wr, 243 op/s 2024-03-18T23:17:44.879 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:17:44 smithi049 ceph-mon[62991]: pgmap v295: 97 pgs: 97 active+clean; 96 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.8 MiB/s wr, 243 op/s 2024-03-18T23:17:44.960 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:17:44 smithi175 ceph-mon[67352]: pgmap v295: 97 pgs: 97 active+clean; 96 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.8 MiB/s wr, 243 op/s 2024-03-18T23:17:45.783 INFO:tasks.workunit.client.0.smithi049.stdout: 1 992211 9.52 MB/sec execute 407 sec latency 23.797 ms 2024-03-18T23:17:46.783 INFO:tasks.workunit.client.0.smithi049.stdout: 1 993802 9.51 MB/sec execute 408 sec latency 118.243 ms 2024-03-18T23:17:46.814 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:17:46 smithi139 ceph-mon[67282]: pgmap v296: 97 pgs: 97 active+clean; 96 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.2 MiB/s wr, 213 op/s 2024-03-18T23:17:46.879 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:17:46 smithi049 ceph-mon[62991]: pgmap v296: 97 pgs: 97 active+clean; 96 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.2 MiB/s wr, 213 op/s 2024-03-18T23:17:46.960 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:17:46 smithi175 ceph-mon[67352]: pgmap v296: 97 pgs: 97 active+clean; 96 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.2 MiB/s wr, 213 op/s 2024-03-18T23:17:47.783 INFO:tasks.workunit.client.0.smithi049.stdout: 1 994942 9.51 MB/sec execute 409 sec latency 118.295 ms 2024-03-18T23:17:48.783 INFO:tasks.workunit.client.0.smithi049.stdout: 1 996368 9.51 MB/sec execute 410 sec latency 23.987 ms 2024-03-18T23:17:48.814 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:17:48 smithi139 ceph-mon[67282]: pgmap v297: 97 pgs: 97 active+clean; 97 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.0 MiB/s wr, 282 op/s 2024-03-18T23:17:48.879 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:17:48 smithi049 ceph-mon[62991]: pgmap v297: 97 pgs: 97 active+clean; 97 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.0 MiB/s wr, 282 op/s 2024-03-18T23:17:48.960 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:17:48 smithi175 ceph-mon[67352]: pgmap v297: 97 pgs: 97 active+clean; 97 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.0 MiB/s wr, 282 op/s 2024-03-18T23:17:49.684 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:49.682+0000 7f327813b640 1 -- 172.21.15.49:0/2251764286 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3270100970 msgr2=0x7f3270100df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:17:49.684 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:49.682+0000 7f327813b640 1 --2- 172.21.15.49:0/2251764286 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3270100970 0x7f3270100df0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f3260009900 tx=0x7f326002bc30 comp rx=0 tx=0).stop 2024-03-18T23:17:49.684 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:49.682+0000 7f327813b640 1 -- 172.21.15.49:0/2251764286 shutdown_connections 2024-03-18T23:17:49.684 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:49.682+0000 7f327813b640 1 --2- 172.21.15.49:0/2251764286 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3270101330 0x7f327010d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:49.684 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:49.682+0000 7f327813b640 1 --2- 172.21.15.49:0/2251764286 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3270100970 0x7f3270100df0 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:49.685 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:49.682+0000 7f327813b640 1 --2- 172.21.15.49:0/2251764286 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f32701056c0 0x7f3270105aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:49.685 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:49.682+0000 7f327813b640 1 -- 172.21.15.49:0/2251764286 >> 172.21.15.49:0/2251764286 conn(0x7f3270077570 msgr2=0x7f3270077980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:17:49.685 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:49.682+0000 7f327813b640 1 -- 172.21.15.49:0/2251764286 shutdown_connections 2024-03-18T23:17:49.685 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:49.682+0000 7f327813b640 1 -- 172.21.15.49:0/2251764286 wait complete. 2024-03-18T23:17:49.685 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:49.682+0000 7f327813b640 1 Processor -- start 2024-03-18T23:17:49.685 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:49.683+0000 7f327813b640 1 -- start start 2024-03-18T23:17:49.685 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:49.683+0000 7f327813b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3270100970 0x7f3270102390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:17:49.685 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:49.683+0000 7f327813b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3270101330 0x7f32701028d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:17:49.685 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:49.683+0000 7f327813b640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f32701056c0 0x7f3270102e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:17:49.685 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:49.683+0000 7f327813b640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f3270110be0 con 0x7f3270101330 2024-03-18T23:17:49.685 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:49.683+0000 7f327813b640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f3270110a60 con 0x7f3270100970 2024-03-18T23:17:49.685 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:49.683+0000 7f327813b640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f3270110d60 con 0x7f32701056c0 2024-03-18T23:17:49.685 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:49.683+0000 7f3275ed7640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3270100970 0x7f3270102390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:17:49.685 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:49.683+0000 7f32756d6640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3270101330 0x7f32701028d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:17:49.686 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:49.683+0000 7f3275ed7640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3270100970 0x7f3270102390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.49:40208/0 (socket says 172.21.15.49:40208) 2024-03-18T23:17:49.686 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:49.683+0000 7f3275ed7640 1 -- 172.21.15.49:0/1065622445 learned_addr learned my addr 172.21.15.49:0/1065622445 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:17:49.686 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:49.683+0000 7f32756d6640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3270101330 0x7f32701028d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:40072/0 (socket says 172.21.15.49:40072) 2024-03-18T23:17:49.686 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:49.683+0000 7f32766d8640 1 --2- 172.21.15.49:0/1065622445 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f32701056c0 0x7f3270102e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:17:49.686 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:49.683+0000 7f3275ed7640 1 -- 172.21.15.49:0/1065622445 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f32701056c0 msgr2=0x7f3270102e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:17:49.686 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:49.683+0000 7f3275ed7640 1 --2- 172.21.15.49:0/1065622445 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f32701056c0 0x7f3270102e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:49.686 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:49.683+0000 7f3275ed7640 1 -- 172.21.15.49:0/1065622445 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3270101330 msgr2=0x7f32701028d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:17:49.686 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:49.683+0000 7f3275ed7640 1 --2- 172.21.15.49:0/1065622445 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3270101330 0x7f32701028d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:49.686 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:49.683+0000 7f3275ed7640 1 -- 172.21.15.49:0/1065622445 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32701a6640 con 0x7f3270100970 2024-03-18T23:17:49.686 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:49.683+0000 7f32756d6640 1 --2- 172.21.15.49:0/1065622445 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3270101330 0x7f32701028d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:17:49.686 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:49.683+0000 7f3275ed7640 1 --2- 172.21.15.49:0/1065622445 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3270100970 0x7f3270102390 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f326c00e9f0 tx=0x7f326c00edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:17:49.686 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:49.684+0000 7f3266ffd640 1 -- 172.21.15.49:0/1065622445 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f326c00cd80 con 0x7f3270100970 2024-03-18T23:17:49.686 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:49.684+0000 7f3266ffd640 1 -- 172.21.15.49:0/1065622445 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f326c005500 con 0x7f3270100970 2024-03-18T23:17:49.687 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:49.684+0000 7f327813b640 1 -- 172.21.15.49:0/1065622445 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32701a6930 con 0x7f3270100970 2024-03-18T23:17:49.687 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:49.684+0000 7f3266ffd640 1 -- 172.21.15.49:0/1065622445 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f326c010690 con 0x7f3270100970 2024-03-18T23:17:49.687 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:49.684+0000 7f327813b640 1 -- 172.21.15.49:0/1065622445 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32701a6e00 con 0x7f3270100970 2024-03-18T23:17:49.694 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:49.686+0000 7f327ae08740 1 Processor -- start 2024-03-18T23:17:49.694 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:49.686+0000 7f327ae08740 1 -- start start 2024-03-18T23:17:49.694 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:49.686+0000 7f3266ffd640 1 -- 172.21.15.49:0/1065622445 <== mon.2 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 25) ==== 137421+0+0 (secure 0 0 0) 0x7f326c0046c0 con 0x7f3270100970 2024-03-18T23:17:49.694 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:49.686+0000 7f327ae08740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558b289e7ef0 0x558b289e82d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:17:49.694 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:49.686+0000 7f327ae08740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558b289e8810 0x558b289f10a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:17:49.694 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:49.686+0000 7f327ae08740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558b289f15e0 0x558b289f39d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:17:49.694 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:49.686+0000 7f327ae08740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x558b2894bbe0 con 0x558b289e7ef0 2024-03-18T23:17:49.694 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:49.686+0000 7f327ae08740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x558b288f4960 con 0x558b289f15e0 2024-03-18T23:17:49.694 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:49.686+0000 7f327ae08740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x558b28947c80 con 0x558b289e8810 2024-03-18T23:17:49.694 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:49.687+0000 7f32766d8640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558b289e7ef0 0x558b289e82d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:17:49.694 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:49.687+0000 7f32766d8640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558b289e7ef0 0x558b289e82d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:40076/0 (socket says 172.21.15.49:40076) 2024-03-18T23:17:49.694 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:49.687+0000 7f32766d8640 1 -- 172.21.15.49:0/2794842348 learned_addr learned my addr 172.21.15.49:0/2794842348 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:17:49.695 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:49.687+0000 7f3275ed7640 1 --2- 172.21.15.49:0/2794842348 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558b289e8810 0x558b289f10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:17:49.695 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:49.687+0000 7f32756d6640 1 --2- 172.21.15.49:0/2794842348 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558b289f15e0 0x558b289f39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:17:49.695 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:49.687+0000 7f32766d8640 1 -- 172.21.15.49:0/2794842348 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558b289e8810 msgr2=0x558b289f10a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:17:49.695 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:49.687+0000 7f32766d8640 1 --2- 172.21.15.49:0/2794842348 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558b289e8810 0x558b289f10a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:49.695 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:49.687+0000 7f32766d8640 1 -- 172.21.15.49:0/2794842348 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558b289f15e0 msgr2=0x558b289f39d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:17:49.695 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:49.687+0000 7f32766d8640 1 --2- 172.21.15.49:0/2794842348 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558b289f15e0 0x558b289f39d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:49.695 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:49.687+0000 7f32766d8640 1 -- 172.21.15.49:0/2794842348 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558b288ebd20 con 0x558b289e7ef0 2024-03-18T23:17:49.695 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:49.687+0000 7f32766d8640 1 --2- 172.21.15.49:0/2794842348 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558b289e7ef0 0x558b289e82d0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f3268009fc0 tx=0x7f32680118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e58fef2002fc3cdc server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:17:49.695 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:49.687+0000 7f325bfff640 1 -- 172.21.15.49:0/2794842348 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3268004d20 con 0x558b289e7ef0 2024-03-18T23:17:49.695 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:49.687+0000 7f3266ffd640 1 --2- 172.21.15.49:0/1065622445 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f325c0a2a10 0x7f325c0a4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:17:49.695 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:49.687+0000 7f325bfff640 1 -- 172.21.15.49:0/2794842348 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3268004050 con 0x558b289e7ef0 2024-03-18T23:17:49.695 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:49.687+0000 7f325bfff640 1 -- 172.21.15.49:0/2794842348 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3268005240 con 0x558b289e7ef0 2024-03-18T23:17:49.695 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:49.687+0000 7f3275ed7640 1 --2- 172.21.15.49:0/1065622445 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f325c0a2a10 0x7f325c0a4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:17:49.695 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:49.687+0000 7f3266ffd640 1 -- 172.21.15.49:0/1065622445 <== mon.2 v2:172.21.15.139:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8556+0+0 (secure 0 0 0) 0x7f326c0cf260 con 0x7f3270100970 2024-03-18T23:17:49.695 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:49.688+0000 7f327ae08740 1 -- 172.21.15.49:0/2794842348 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558b289e7ef0 msgr2=0x558b289e82d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:17:49.696 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:49.688+0000 7f3275ed7640 1 --2- 172.21.15.49:0/1065622445 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f325c0a2a10 0x7f325c0a4ed0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f326c0cca40 tx=0x7f326c0d2db0 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:17:49.696 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:49.688+0000 7f327ae08740 1 --2- 172.21.15.49:0/2794842348 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558b289e7ef0 0x558b289e82d0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f3268009fc0 tx=0x7f32680118d0 comp rx=0 tx=0).stop 2024-03-18T23:17:49.696 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:49.688+0000 7f327ae08740 1 -- 172.21.15.49:0/2794842348 shutdown_connections 2024-03-18T23:17:49.696 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:49.688+0000 7f327ae08740 1 --2- 172.21.15.49:0/2794842348 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558b289f15e0 0x558b289f39d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:49.696 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:49.688+0000 7f327ae08740 1 --2- 172.21.15.49:0/2794842348 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558b289e8810 0x558b289f10a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:49.696 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:49.688+0000 7f327ae08740 1 --2- 172.21.15.49:0/2794842348 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558b289e7ef0 0x558b289e82d0 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:49.696 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:49.688+0000 7f327ae08740 1 -- 172.21.15.49:0/2794842348 >> 172.21.15.49:0/2794842348 conn(0x558b289e2810 msgr2=0x558b289e2bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:17:49.696 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:49.688+0000 7f327ae08740 1 -- 172.21.15.49:0/2794842348 shutdown_connections 2024-03-18T23:17:49.696 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:49.688+0000 7f327ae08740 1 -- 172.21.15.49:0/2794842348 wait complete. 2024-03-18T23:17:49.696 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:49.688+0000 7f327ae08740 1 -- start start 2024-03-18T23:17:49.696 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:49.688+0000 7f327ae08740 1 Processor -- start 2024-03-18T23:17:49.696 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:49.689+0000 7f327ae08740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558b289e7ef0 0x558b289f0fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:17:49.696 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:49.689+0000 7f327ae08740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558b289e8810 0x558b289ee370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:17:49.696 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:49.689+0000 7f327ae08740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558b289f15e0 0x558b289ee980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:17:49.696 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:49.689+0000 7f327ae08740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x558b289f4210 con 0x558b289f15e0 2024-03-18T23:17:49.697 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:49.689+0000 7f327ae08740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x558b28907db0 con 0x558b289e7ef0 2024-03-18T23:17:49.697 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:49.689+0000 7f327ae08740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x558b289f4390 con 0x558b289e8810 2024-03-18T23:17:49.697 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:49.689+0000 7f327ae08740 10 client.0 _ll_register_callbacks cb 0x558b2890ce70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T23:17:49.697 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:49.689+0000 7f32766d8640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558b289f15e0 0x558b289ee980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:17:49.697 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:49.689+0000 7f32766d8640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558b289f15e0 0x558b289ee980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:40082/0 (socket says 172.21.15.49:40082) 2024-03-18T23:17:49.697 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:49.689+0000 7f32766d8640 1 -- 172.21.15.49:0/1180894923 learned_addr learned my addr 172.21.15.49:0/1180894923 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:17:49.697 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:49.689+0000 7f32756d6640 1 --2- 172.21.15.49:0/1180894923 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558b289e8810 0x558b289ee370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:17:49.697 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:49.689+0000 7f32766d8640 1 --2- 172.21.15.49:0/1180894923 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558b289e7ef0 0x558b289f0fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:17:49.697 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:49.689+0000 7f32756d6640 1 -- 172.21.15.49:0/1180894923 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558b289e7ef0 msgr2=0x558b289f0fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:17:49.697 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:49.689+0000 7f32756d6640 1 --2- 172.21.15.49:0/1180894923 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558b289e7ef0 0x558b289f0fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:49.697 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:49.689+0000 7f32756d6640 1 -- 172.21.15.49:0/1180894923 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558b289f15e0 msgr2=0x558b289ee980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:17:49.697 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:49.689+0000 7f32756d6640 1 --2- 172.21.15.49:0/1180894923 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558b289f15e0 0x558b289ee980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:49.697 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:49.689+0000 7f32756d6640 1 -- 172.21.15.49:0/1180894923 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558b28906c60 con 0x558b289e8810 2024-03-18T23:17:49.697 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:49.689+0000 7f32766d8640 1 --2- 172.21.15.49:0/1180894923 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558b289f15e0 0x558b289ee980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:17:49.697 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:49.690+0000 7f32756d6640 1 --2- 172.21.15.49:0/1180894923 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558b289e8810 0x558b289ee370 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f3260000c00 tx=0x7f3260004270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bd63d68855a6ac6a server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:17:49.698 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:49.690+0000 7f32597fa640 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-03-18T23:17:49.698 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:49.690+0000 7f32597fa640 1 -- 172.21.15.49:0/1180894923 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f326002cab0 con 0x558b289e8810 2024-03-18T23:17:49.698 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:49.690+0000 7f327ae08740 1 -- 172.21.15.49:0/1180894923 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3268004050 con 0x558b289e8810 2024-03-18T23:17:49.698 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:49.690+0000 7f32597fa640 1 -- 172.21.15.49:0/1180894923 <== mon.1 v2:172.21.15.175:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8556+0+0 (secure 0 0 0) 0x7f326002cc50 con 0x558b289e8810 2024-03-18T23:17:49.698 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:49.690+0000 7f32597fa640 1 -- 172.21.15.49:0/1180894923 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f3260004f20 con 0x558b289e8810 2024-03-18T23:17:49.698 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:49.690+0000 7f327ae08740 10 client.26026 fetch_fsmap learned FSMap version 32 2024-03-18T23:17:49.698 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:49.690+0000 7f327ae08740 1 -- 172.21.15.49:0/1180894923 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x558b28660210 con 0x558b289e8810 2024-03-18T23:17:49.698 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:49.690+0000 7f32597fa640 1 -- 172.21.15.49:0/1180894923 <== mon.1 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3268004050 con 0x558b289e8810 2024-03-18T23:17:49.698 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:49.690+0000 7f32597fa640 1 -- 172.21.15.49:0/1180894923 <== mon.1 v2:172.21.15.175:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f3260046070 con 0x558b289e8810 2024-03-18T23:17:49.699 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:49.690+0000 7f327ae08740 10 client.26026 fetch_fsmap finished waiting for FSMap version 32 2024-03-18T23:17:49.699 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:49.690+0000 7f327ae08740 10 client.26026 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T23:17:49.699 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:49.691+0000 7f327ae08740 20 client.26026 populate_metadata read hostname 'smithi049' 2024-03-18T23:17:49.699 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:49.691+0000 7f327ae08740 1 --2- 172.21.15.49:0/1180894923 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x558b28a01500 0x558b28a038f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:17:49.699 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:49.691+0000 7f327ae08740 4 client.26026 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T23:17:49.699 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:49.691+0000 7f327ae08740 1 -- 172.21.15.49:0/1180894923 --> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558b28959120 con 0x558b28a01500 2024-03-18T23:17:49.699 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:49.691+0000 7f32766d8640 1 --2- 172.21.15.49:0/1180894923 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x558b28a01500 0x558b28a038f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:17:49.699 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:49.691+0000 7f32766d8640 1 --2- 172.21.15.49:0/1180894923 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x558b28a01500 0x558b28a038f0 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1636e23901a0e7ea server_cookie=c7574b5acace749f in_seq=0 out_seq=0 2024-03-18T23:17:49.699 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:49.691+0000 7f32597fa640 10 client.26026 ms_handle_connect on v2:172.21.15.139:6838/4280128209 2024-03-18T23:17:49.699 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:49.693+0000 7f32597fa640 1 -- 172.21.15.49:0/1180894923 <== mds.0 v2:172.21.15.139:6838/4280128209 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x558b28959120 con 0x558b28a01500 2024-03-18T23:17:49.699 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:49.693+0000 7f32597fa640 10 client.26026 handle_command_reply: tid=0 2024-03-18T23:17:49.699 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:49.693+0000 7f327ae08740 1 client.26026 shutdown 2024-03-18T23:17:49.699 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:49.693+0000 7f327ae08740 1 -- 172.21.15.49:0/1180894923 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558b289e8810 msgr2=0x558b289ee370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:17:49.699 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:49.693+0000 7f327ae08740 1 --2- 172.21.15.49:0/1180894923 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558b289e8810 0x558b289ee370 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f3260000c00 tx=0x7f3260004270 comp rx=0 tx=0).stop 2024-03-18T23:17:49.700 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:49.693+0000 7f327ae08740 1 -- 172.21.15.49:0/1180894923 shutdown_connections 2024-03-18T23:17:49.700 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:49.693+0000 7f327ae08740 1 --2- 172.21.15.49:0/1180894923 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x558b28a01500 0x558b28a038f0 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:49.700 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:49.693+0000 7f327ae08740 1 --2- 172.21.15.49:0/1180894923 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558b289f15e0 0x558b289ee980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:49.700 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:49.693+0000 7f327ae08740 1 --2- 172.21.15.49:0/1180894923 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558b289e8810 0x558b289ee370 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:49.700 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:49.693+0000 7f32597fa640 0 client.26026 ms_handle_reset on v2:172.21.15.139:6838/4280128209 2024-03-18T23:17:49.700 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:49.693+0000 7f327ae08740 1 --2- 172.21.15.49:0/1180894923 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558b289e7ef0 0x558b289f0fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:49.700 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:49.693+0000 7f327ae08740 1 -- 172.21.15.49:0/1180894923 >> 172.21.15.49:0/1180894923 conn(0x558b289e2810 msgr2=0x558b289e1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:17:49.700 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:49.693+0000 7f327ae08740 1 -- 172.21.15.49:0/1180894923 shutdown_connections 2024-03-18T23:17:49.700 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:49.693+0000 7f327ae08740 1 -- 172.21.15.49:0/1180894923 wait complete. 2024-03-18T23:17:49.700 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:49.693+0000 7f327ae08740 20 client.26026 trim_cache size 0 max 16384 2024-03-18T23:17:49.713 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:49.709+0000 7f327ae08740 1 Processor -- start 2024-03-18T23:17:49.713 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:49.709+0000 7f327ae08740 1 -- start start 2024-03-18T23:17:49.713 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:49.709+0000 7f327ae08740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558b289e7ef0 0x558b289fcfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:17:49.713 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:49.709+0000 7f327ae08740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558b289e8810 0x558b28a091e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:17:49.713 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:49.709+0000 7f327ae08740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558b289f15e0 0x558b289f0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:17:49.714 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:49.709+0000 7f327ae08740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f32701af700 con 0x558b289e8810 2024-03-18T23:17:49.714 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:49.709+0000 7f327ae08740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f32701b1240 con 0x558b289e7ef0 2024-03-18T23:17:49.714 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:49.709+0000 7f327ae08740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f32701afdd0 con 0x558b289f15e0 2024-03-18T23:17:49.714 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:49.709+0000 7f32756d6640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558b289e8810 0x558b28a091e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:17:49.714 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:49.709+0000 7f32756d6640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558b289e8810 0x558b28a091e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:40084/0 (socket says 172.21.15.49:40084) 2024-03-18T23:17:49.714 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:49.709+0000 7f32756d6640 1 -- 172.21.15.49:0/415365101 learned_addr learned my addr 172.21.15.49:0/415365101 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:17:49.714 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:49.709+0000 7f32766d8640 1 --2- 172.21.15.49:0/415365101 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558b289e7ef0 0x558b289fcfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:17:49.714 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:49.709+0000 7f32766d8640 1 --2- 172.21.15.49:0/415365101 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558b289f15e0 0x558b289f0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:17:49.714 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:49.709+0000 7f32756d6640 1 -- 172.21.15.49:0/415365101 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558b289f15e0 msgr2=0x558b289f0aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:17:49.714 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:49.709+0000 7f32756d6640 1 --2- 172.21.15.49:0/415365101 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558b289f15e0 0x558b289f0aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:49.714 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:49.709+0000 7f32756d6640 1 -- 172.21.15.49:0/415365101 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558b289e7ef0 msgr2=0x558b289fcfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:17:49.714 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:49.709+0000 7f32756d6640 1 --2- 172.21.15.49:0/415365101 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558b289e7ef0 0x558b289fcfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:49.715 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:49.709+0000 7f32756d6640 1 -- 172.21.15.49:0/415365101 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558b289e8640 con 0x558b289e8810 2024-03-18T23:17:49.715 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:49.710+0000 7f32756d6640 1 --2- 172.21.15.49:0/415365101 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558b289e8810 0x558b28a091e0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f3260001e90 tx=0x7f3260009880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49763e14bdf3c160 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:17:49.715 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:49.710+0000 7f325b7fe640 1 -- 172.21.15.49:0/415365101 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32600035f0 con 0x558b289e8810 2024-03-18T23:17:49.715 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:49.710+0000 7f325b7fe640 1 -- 172.21.15.49:0/415365101 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3260003790 con 0x558b289e8810 2024-03-18T23:17:49.715 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:49.710+0000 7f325b7fe640 1 -- 172.21.15.49:0/415365101 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32600053b0 con 0x558b289e8810 2024-03-18T23:17:49.715 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:49.710+0000 7f327ae08740 1 -- 172.21.15.49:0/415365101 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558b289e8810 msgr2=0x558b28a091e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:17:49.715 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:49.710+0000 7f327ae08740 1 --2- 172.21.15.49:0/415365101 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558b289e8810 0x558b28a091e0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f3260001e90 tx=0x7f3260009880 comp rx=0 tx=0).stop 2024-03-18T23:17:49.715 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:49.710+0000 7f327ae08740 1 -- 172.21.15.49:0/415365101 shutdown_connections 2024-03-18T23:17:49.715 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:49.710+0000 7f327ae08740 1 --2- 172.21.15.49:0/415365101 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558b289f15e0 0x558b289f0aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:49.715 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:49.710+0000 7f327ae08740 1 --2- 172.21.15.49:0/415365101 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558b289e8810 0x558b28a091e0 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:49.715 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:49.710+0000 7f327ae08740 1 --2- 172.21.15.49:0/415365101 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558b289e7ef0 0x558b289fcfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:49.715 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:49.710+0000 7f327ae08740 1 -- 172.21.15.49:0/415365101 >> 172.21.15.49:0/415365101 conn(0x558b289e2810 msgr2=0x558b289e1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:17:49.715 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:49.710+0000 7f327ae08740 1 -- 172.21.15.49:0/415365101 shutdown_connections 2024-03-18T23:17:49.715 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:49.710+0000 7f327ae08740 1 -- 172.21.15.49:0/415365101 wait complete. 2024-03-18T23:17:49.716 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:49.711+0000 7f327ae08740 1 -- start start 2024-03-18T23:17:49.716 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:49.711+0000 7f327ae08740 1 Processor -- start 2024-03-18T23:17:49.716 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:49.711+0000 7f327ae08740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558b289e7ef0 0x558b289f5160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:17:49.716 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:49.711+0000 7f327ae08740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558b289e8810 0x558b289f56a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:17:49.716 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:49.711+0000 7f327ae08740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558b289f15e0 0x558b289f5be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:17:49.716 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:49.711+0000 7f327ae08740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f32701b0b70 con 0x558b289e8810 2024-03-18T23:17:49.716 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:49.711+0000 7f327ae08740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f32701b04a0 con 0x558b289e7ef0 2024-03-18T23:17:49.716 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:49.711+0000 7f327ae08740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f32701ad700 con 0x558b289f15e0 2024-03-18T23:17:49.716 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:49.711+0000 7f327ae08740 10 client.0 _ll_register_callbacks cb 0x7f3260009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T23:17:49.716 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:49.711+0000 7f32756d6640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558b289e8810 0x558b289f56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:17:49.716 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:49.711+0000 7f32756d6640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558b289e8810 0x558b289f56a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:40096/0 (socket says 172.21.15.49:40096) 2024-03-18T23:17:49.717 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:49.711+0000 7f32756d6640 1 -- 172.21.15.49:0/106352576 learned_addr learned my addr 172.21.15.49:0/106352576 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:17:49.717 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:49.711+0000 7f32766d8640 1 --2- 172.21.15.49:0/106352576 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558b289e7ef0 0x558b289f5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:17:49.717 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:49.711+0000 7f32766d8640 1 --2- 172.21.15.49:0/106352576 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558b289f15e0 0x558b289f5be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:17:49.717 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:49.712+0000 7f32756d6640 1 -- 172.21.15.49:0/106352576 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558b289f15e0 msgr2=0x558b289f5be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:17:49.717 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:49.712+0000 7f32756d6640 1 --2- 172.21.15.49:0/106352576 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558b289f15e0 0x558b289f5be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:49.717 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:49.712+0000 7f32756d6640 1 -- 172.21.15.49:0/106352576 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558b289e7ef0 msgr2=0x558b289f5160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:17:49.717 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:49.712+0000 7f32756d6640 1 --2- 172.21.15.49:0/106352576 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558b289e7ef0 0x558b289f5160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:49.717 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:49.712+0000 7f32756d6640 1 -- 172.21.15.49:0/106352576 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558b2872c620 con 0x558b289e8810 2024-03-18T23:17:49.717 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:49.712+0000 7f32766d8640 1 --2- 172.21.15.49:0/106352576 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558b289f15e0 0x558b289f5be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T23:17:49.717 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:49.712+0000 7f32756d6640 1 --2- 172.21.15.49:0/106352576 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558b289e8810 0x558b289f56a0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f3260006010 tx=0x7f326002cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fdc774166c255f3 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:17:49.718 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:49.712+0000 7f32597fa640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-18T23:17:49.718 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:49.712+0000 7f32597fa640 1 -- 172.21.15.49:0/106352576 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32600035f0 con 0x558b289e8810 2024-03-18T23:17:49.718 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:49.712+0000 7f32597fa640 1 -- 172.21.15.49:0/106352576 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8556+0+0 (secure 0 0 0) 0x7f3260005540 con 0x558b289e8810 2024-03-18T23:17:49.718 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:49.712+0000 7f327ae08740 1 -- 172.21.15.49:0/106352576 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3260003790 con 0x558b289e8810 2024-03-18T23:17:49.718 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:49.712+0000 7f32597fa640 1 -- 172.21.15.49:0/106352576 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f326002ccd0 con 0x558b289e8810 2024-03-18T23:17:49.718 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:49.712+0000 7f327ae08740 10 client.16233 fetch_fsmap learned FSMap version 32 2024-03-18T23:17:49.718 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:49.712+0000 7f327ae08740 1 -- 172.21.15.49:0/106352576 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x558b28a67bd0 con 0x558b289e8810 2024-03-18T23:17:49.718 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:49.712+0000 7f32597fa640 1 -- 172.21.15.49:0/106352576 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3260003790 con 0x558b289e8810 2024-03-18T23:17:49.718 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:49.712+0000 7f32597fa640 1 -- 172.21.15.49:0/106352576 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f3260046040 con 0x558b289e8810 2024-03-18T23:17:49.718 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:49.712+0000 7f327ae08740 10 client.16233 fetch_fsmap finished waiting for FSMap version 32 2024-03-18T23:17:49.718 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:49.713+0000 7f327ae08740 10 client.16233 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T23:17:49.718 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:49.713+0000 7f327ae08740 20 client.16233 populate_metadata read hostname 'smithi049' 2024-03-18T23:17:49.718 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:49.713+0000 7f327ae08740 1 --2- 172.21.15.49:0/106352576 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x558b28a68220 0x558b28a6a610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:17:49.719 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:49.713+0000 7f327ae08740 4 client.16233 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T23:17:49.719 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:49.713+0000 7f327ae08740 1 -- 172.21.15.49:0/106352576 --> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558b28953470 con 0x558b28a68220 2024-03-18T23:17:49.719 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:49.713+0000 7f32766d8640 1 --2- 172.21.15.49:0/106352576 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x558b28a68220 0x558b28a6a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:17:49.719 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:49.713+0000 7f32766d8640 1 --2- 172.21.15.49:0/106352576 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x558b28a68220 0x558b28a6a610 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1de64a857cb5421 server_cookie=278712780b7f6a88 in_seq=0 out_seq=0 2024-03-18T23:17:49.719 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:49.713+0000 7f32597fa640 10 client.16233 ms_handle_connect on v2:172.21.15.139:6838/4280128209 2024-03-18T23:17:49.784 INFO:tasks.workunit.client.0.smithi049.stdout: 1 998381 9.51 MB/sec execute 411 sec latency 35.504 ms 2024-03-18T23:17:49.917 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:49.915+0000 7f32597fa640 1 -- 172.21.15.49:0/106352576 <== mds.0 v2:172.21.15.139:6838/4280128209 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x558b28953470 con 0x558b28a68220 2024-03-18T23:17:49.917 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:49.915+0000 7f32597fa640 10 client.16233 handle_command_reply: tid=0 2024-03-18T23:17:49.917 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:49.915+0000 7f327ae08740 1 client.16233 shutdown 2024-03-18T23:17:49.917 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:49.916+0000 7f327ae08740 1 -- 172.21.15.49:0/106352576 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558b289e8810 msgr2=0x558b289f56a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:17:49.917 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:49.916+0000 7f327ae08740 1 --2- 172.21.15.49:0/106352576 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558b289e8810 0x558b289f56a0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f3260006010 tx=0x7f326002cae0 comp rx=0 tx=0).stop 2024-03-18T23:17:49.917 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:49.916+0000 7f327ae08740 1 -- 172.21.15.49:0/106352576 shutdown_connections 2024-03-18T23:17:49.917 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:49.916+0000 7f327ae08740 1 --2- 172.21.15.49:0/106352576 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x558b28a68220 0x558b28a6a610 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:49.917 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:49.916+0000 7f327ae08740 1 --2- 172.21.15.49:0/106352576 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558b289f15e0 0x558b289f5be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:49.918 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:49.916+0000 7f32597fa640 0 client.16233 ms_handle_reset on v2:172.21.15.139:6838/4280128209 2024-03-18T23:17:49.918 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:49.916+0000 7f327ae08740 1 --2- 172.21.15.49:0/106352576 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558b289e8810 0x558b289f56a0 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:49.918 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:49.916+0000 7f327ae08740 1 --2- 172.21.15.49:0/106352576 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558b289e7ef0 0x558b289f5160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:49.918 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:49.916+0000 7f327ae08740 1 -- 172.21.15.49:0/106352576 >> 172.21.15.49:0/106352576 conn(0x558b289e2810 msgr2=0x558b289e8d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:17:49.918 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:49.917+0000 7f327ae08740 1 -- 172.21.15.49:0/106352576 shutdown_connections 2024-03-18T23:17:49.918 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:49.917+0000 7f327ae08740 1 -- 172.21.15.49:0/106352576 wait complete. 2024-03-18T23:17:49.918 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:49.917+0000 7f327ae08740 20 client.16233 trim_cache size 0 max 16384 2024-03-18T23:17:49.918 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:{ 2024-03-18T23:17:49.918 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "message": "", 2024-03-18T23:17:49.918 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "return_code": 0 2024-03-18T23:17:49.918 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:} 2024-03-18T23:17:49.919 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:49.917+0000 7f327813b640 1 -- 172.21.15.49:0/1065622445 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f325c0a2a10 msgr2=0x7f325c0a4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:17:49.919 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:49.917+0000 7f327813b640 1 --2- 172.21.15.49:0/1065622445 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f325c0a2a10 0x7f325c0a4ed0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f326c0cca40 tx=0x7f326c0d2db0 comp rx=0 tx=0).stop 2024-03-18T23:17:49.919 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:49.918+0000 7f327813b640 1 -- 172.21.15.49:0/1065622445 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3270100970 msgr2=0x7f3270102390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:17:49.919 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:49.918+0000 7f327813b640 1 --2- 172.21.15.49:0/1065622445 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3270100970 0x7f3270102390 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f326c00e9f0 tx=0x7f326c00edf0 comp rx=0 tx=0).stop 2024-03-18T23:17:49.919 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:49.918+0000 7f327813b640 1 -- 172.21.15.49:0/1065622445 shutdown_connections 2024-03-18T23:17:49.920 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:49.918+0000 7f327813b640 1 --2- 172.21.15.49:0/1065622445 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f325c0a2a10 0x7f325c0a4ed0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:49.920 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:49.918+0000 7f327813b640 1 --2- 172.21.15.49:0/1065622445 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f32701056c0 0x7f3270102e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:49.920 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:49.918+0000 7f327813b640 1 --2- 172.21.15.49:0/1065622445 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3270101330 0x7f32701028d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:49.920 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:49.918+0000 7f327813b640 1 --2- 172.21.15.49:0/1065622445 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3270100970 0x7f3270102390 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:49.920 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:49.918+0000 7f327813b640 1 -- 172.21.15.49:0/1065622445 >> 172.21.15.49:0/1065622445 conn(0x7f3270077570 msgr2=0x7f3270069240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:17:49.920 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:49.918+0000 7f327813b640 1 -- 172.21.15.49:0/1065622445 shutdown_connections 2024-03-18T23:17:49.920 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:17:49.918+0000 7f327813b640 1 -- 172.21.15.49:0/1065622445 wait complete. 2024-03-18T23:17:50.784 INFO:tasks.workunit.client.0.smithi049.stdout: 1 999292 9.49 MB/sec execute 412 sec latency 110.425 ms 2024-03-18T23:17:50.814 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:17:50 smithi139 ceph-mon[67282]: pgmap v298: 97 pgs: 97 active+clean; 97 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 1.9 MiB/s wr, 208 op/s 2024-03-18T23:17:50.878 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:17:50 smithi049 ceph-mon[62991]: pgmap v298: 97 pgs: 97 active+clean; 97 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 1.9 MiB/s wr, 208 op/s 2024-03-18T23:17:50.960 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:17:50 smithi175 ceph-mon[67352]: pgmap v298: 97 pgs: 97 active+clean; 97 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 1.9 MiB/s wr, 208 op/s 2024-03-18T23:17:51.784 INFO:tasks.workunit.client.0.smithi049.stdout: 1 1001295 9.50 MB/sec execute 413 sec latency 42.194 ms 2024-03-18T23:17:52.784 INFO:tasks.workunit.client.0.smithi049.stdout: 1 1002429 9.49 MB/sec execute 414 sec latency 35.049 ms 2024-03-18T23:17:52.814 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:17:52 smithi139 ceph-mon[67282]: pgmap v299: 97 pgs: 97 active+clean; 90 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.0 MiB/s wr, 337 op/s 2024-03-18T23:17:52.879 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:17:52 smithi049 ceph-mon[62991]: pgmap v299: 97 pgs: 97 active+clean; 90 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.0 MiB/s wr, 337 op/s 2024-03-18T23:17:52.960 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:17:52 smithi175 ceph-mon[67352]: pgmap v299: 97 pgs: 97 active+clean; 90 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.0 MiB/s wr, 337 op/s 2024-03-18T23:17:53.784 INFO:tasks.workunit.client.0.smithi049.stdout: 1 1004234 9.49 MB/sec execute 415 sec latency 58.344 ms 2024-03-18T23:17:54.784 INFO:tasks.workunit.client.0.smithi049.stdout: 1 1005655 9.49 MB/sec execute 416 sec latency 17.552 ms 2024-03-18T23:17:54.814 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:17:54 smithi139 ceph-mon[67282]: pgmap v300: 97 pgs: 97 active+clean; 90 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.1 MiB/s wr, 253 op/s 2024-03-18T23:17:54.879 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:17:54 smithi049 ceph-mon[62991]: pgmap v300: 97 pgs: 97 active+clean; 90 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.1 MiB/s wr, 253 op/s 2024-03-18T23:17:54.960 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:17:54 smithi175 ceph-mon[67352]: pgmap v300: 97 pgs: 97 active+clean; 90 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.1 MiB/s wr, 253 op/s 2024-03-18T23:17:55.784 INFO:tasks.workunit.client.0.smithi049.stdout: 1 1007118 9.48 MB/sec execute 417 sec latency 18.995 ms 2024-03-18T23:17:56.784 INFO:tasks.workunit.client.0.smithi049.stdout: 1 1009035 9.49 MB/sec execute 418 sec latency 99.623 ms 2024-03-18T23:17:56.814 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:17:56 smithi139 ceph-mon[67282]: pgmap v301: 97 pgs: 97 active+clean; 90 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.0 MiB/s wr, 229 op/s 2024-03-18T23:17:56.879 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:17:56 smithi049 ceph-mon[62991]: pgmap v301: 97 pgs: 97 active+clean; 90 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.0 MiB/s wr, 229 op/s 2024-03-18T23:17:56.960 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:17:56 smithi175 ceph-mon[67352]: pgmap v301: 97 pgs: 97 active+clean; 90 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.0 MiB/s wr, 229 op/s 2024-03-18T23:17:57.784 INFO:tasks.workunit.client.0.smithi049.stdout: 1 1009870 9.47 MB/sec execute 419 sec latency 116.668 ms 2024-03-18T23:17:58.784 INFO:tasks.workunit.client.0.smithi049.stdout: 1 1011835 9.48 MB/sec execute 420 sec latency 28.860 ms 2024-03-18T23:17:58.814 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:17:58 smithi139 ceph-mon[67282]: pgmap v302: 97 pgs: 97 active+clean; 95 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.1 MiB/s wr, 307 op/s 2024-03-18T23:17:58.879 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:17:58 smithi049 ceph-mon[62991]: pgmap v302: 97 pgs: 97 active+clean; 95 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.1 MiB/s wr, 307 op/s 2024-03-18T23:17:58.960 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:17:58 smithi175 ceph-mon[67352]: pgmap v302: 97 pgs: 97 active+clean; 95 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.1 MiB/s wr, 307 op/s 2024-03-18T23:17:59.784 INFO:tasks.workunit.client.0.smithi049.stdout: 1 1012846 9.47 MB/sec execute 421 sec latency 52.623 ms 2024-03-18T23:18:00.784 INFO:tasks.workunit.client.0.smithi049.stdout: 1 1014209 9.46 MB/sec execute 422 sec latency 25.217 ms 2024-03-18T23:18:01.379 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:18:01 smithi049 ceph-mon[62991]: pgmap v303: 97 pgs: 97 active+clean; 95 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.4 MiB/s wr, 239 op/s 2024-03-18T23:18:01.460 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:18:01 smithi175 ceph-mon[67352]: pgmap v303: 97 pgs: 97 active+clean; 95 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.4 MiB/s wr, 239 op/s 2024-03-18T23:18:01.564 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:18:01 smithi139 ceph-mon[67282]: pgmap v303: 97 pgs: 97 active+clean; 95 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.4 MiB/s wr, 239 op/s 2024-03-18T23:18:01.784 INFO:tasks.workunit.client.0.smithi049.stdout: 1 1014789 9.44 MB/sec execute 423 sec latency 370.701 ms 2024-03-18T23:18:02.379 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:18:02 smithi049 ceph-mon[62991]: osdmap e98: 12 total, 12 up, 12 in 2024-03-18T23:18:02.460 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:18:02 smithi175 ceph-mon[67352]: osdmap e98: 12 total, 12 up, 12 in 2024-03-18T23:18:02.564 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:18:02 smithi139 ceph-mon[67282]: osdmap e98: 12 total, 12 up, 12 in 2024-03-18T23:18:02.784 INFO:tasks.workunit.client.0.smithi049.stdout: 1 1015080 9.42 MB/sec execute 424 sec latency 178.366 ms 2024-03-18T23:18:03.043 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:03.044+0000 7f09dda4b640 1 -- 172.21.15.49:0/3097252046 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f09d8108a10 msgr2=0x7f09d810ca50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:18:03.043 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:03.044+0000 7f09dda4b640 1 --2- 172.21.15.49:0/3097252046 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f09d8108a10 0x7f09d810ca50 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f09c0009860 tx=0x7f09c002bb70 comp rx=0 tx=0).stop 2024-03-18T23:18:03.043 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:03.044+0000 7f09dda4b640 1 -- 172.21.15.49:0/3097252046 shutdown_connections 2024-03-18T23:18:03.044 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:03.044+0000 7f09dda4b640 1 --2- 172.21.15.49:0/3097252046 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f09d810d130 0x7f09d8114d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:03.044 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:03.044+0000 7f09dda4b640 1 --2- 172.21.15.49:0/3097252046 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f09d8108a10 0x7f09d810ca50 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:03.044 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:03.044+0000 7f09dda4b640 1 --2- 172.21.15.49:0/3097252046 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f09d8108060 0x7f09d8108440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:03.044 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:03.044+0000 7f09dda4b640 1 -- 172.21.15.49:0/3097252046 >> 172.21.15.49:0/3097252046 conn(0x7f09d8075d90 msgr2=0x7f09d80781b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:18:03.044 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:03.044+0000 7f09dda4b640 1 -- 172.21.15.49:0/3097252046 shutdown_connections 2024-03-18T23:18:03.044 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:03.044+0000 7f09dda4b640 1 -- 172.21.15.49:0/3097252046 wait complete. 2024-03-18T23:18:03.045 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:03.045+0000 7f09dda4b640 1 Processor -- start 2024-03-18T23:18:03.045 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:03.045+0000 7f09dda4b640 1 -- start start 2024-03-18T23:18:03.045 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:03.045+0000 7f09dda4b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f09d8108060 0x7f09d819e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:18:03.045 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:03.045+0000 7f09dda4b640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f09d810d130 0x7f09d819ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:18:03.045 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:03.045+0000 7f09dda4b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f09d81a2ef0 0x7f09d81a33a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:18:03.045 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:03.045+0000 7f09dda4b640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f09d81181a0 con 0x7f09d8108060 2024-03-18T23:18:03.045 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:03.045+0000 7f09dda4b640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f09d8118020 con 0x7f09d81a2ef0 2024-03-18T23:18:03.045 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:03.045+0000 7f09dda4b640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f09d8118320 con 0x7f09d810d130 2024-03-18T23:18:03.046 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:03.045+0000 7f09d77fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f09d81a2ef0 0x7f09d81a33a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:18:03.046 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:03.045+0000 7f09d77fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f09d81a2ef0 0x7f09d81a33a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.49:57108/0 (socket says 172.21.15.49:57108) 2024-03-18T23:18:03.046 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:03.045+0000 7f09d77fe640 1 -- 172.21.15.49:0/2393659551 learned_addr learned my addr 172.21.15.49:0/2393659551 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:18:03.046 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:03.045+0000 7f09cffff640 1 --2- 172.21.15.49:0/2393659551 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f09d810d130 0x7f09d819ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:18:03.046 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:03.046+0000 7f09d6ffd640 1 --2- 172.21.15.49:0/2393659551 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f09d8108060 0x7f09d819e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:18:03.046 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:03.046+0000 7f09cffff640 1 -- 172.21.15.49:0/2393659551 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f09d81a2ef0 msgr2=0x7f09d81a33a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:18:03.046 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:03.046+0000 7f09cffff640 1 --2- 172.21.15.49:0/2393659551 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f09d81a2ef0 0x7f09d81a33a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:03.046 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:03.046+0000 7f09cffff640 1 -- 172.21.15.49:0/2393659551 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f09d8108060 msgr2=0x7f09d819e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:18:03.046 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:03.046+0000 7f09cffff640 1 --2- 172.21.15.49:0/2393659551 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f09d8108060 0x7f09d819e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:03.046 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:03.046+0000 7f09cffff640 1 -- 172.21.15.49:0/2393659551 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09d81a39e0 con 0x7f09d810d130 2024-03-18T23:18:03.046 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:03.046+0000 7f09d6ffd640 1 --2- 172.21.15.49:0/2393659551 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f09d8108060 0x7f09d819e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T23:18:03.047 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:03.047+0000 7f09cffff640 1 --2- 172.21.15.49:0/2393659551 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f09d810d130 0x7f09d819ebc0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f09c0037040 tx=0x7f09c00045b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:18:03.047 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:03.047+0000 7f09dca49640 1 -- 172.21.15.49:0/2393659551 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09c0041020 con 0x7f09d810d130 2024-03-18T23:18:03.047 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:03.047+0000 7f09dda4b640 1 -- 172.21.15.49:0/2393659551 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09d81a3c70 con 0x7f09d810d130 2024-03-18T23:18:03.047 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:03.047+0000 7f09dda4b640 1 -- 172.21.15.49:0/2393659551 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09d81ab490 con 0x7f09d810d130 2024-03-18T23:18:03.047 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:03.047+0000 7f09dca49640 1 -- 172.21.15.49:0/2393659551 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f09c002c9f0 con 0x7f09d810d130 2024-03-18T23:18:03.047 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:03.047+0000 7f09dca49640 1 -- 172.21.15.49:0/2393659551 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09c0039040 con 0x7f09d810d130 2024-03-18T23:18:03.050 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:03.051+0000 7f09e0636740 1 Processor -- start 2024-03-18T23:18:03.051 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:03.052+0000 7f09dca49640 1 -- 172.21.15.49:0/2393659551 <== mon.1 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 25) ==== 137421+0+0 (secure 0 0 0) 0x7f09c0044020 con 0x7f09d810d130 2024-03-18T23:18:03.051 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:03.052+0000 7f09e0636740 1 -- start start 2024-03-18T23:18:03.052 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:03.053+0000 7f09e0636740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e0805fcef0 0x55e0805fd2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:18:03.052 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:03.053+0000 7f09e0636740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e0805fd810 0x55e0806060a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:18:03.052 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:03.053+0000 7f09e0636740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e0806065e0 0x55e0806089d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:18:03.052 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:03.053+0000 7f09e0636740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55e080560be0 con 0x55e0806065e0 2024-03-18T23:18:03.052 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:03.053+0000 7f09e0636740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55e080509960 con 0x55e0805fcef0 2024-03-18T23:18:03.052 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:03.053+0000 7f09e0636740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55e08055cc80 con 0x55e0805fd810 2024-03-18T23:18:03.052 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:03.053+0000 7f09dca49640 1 --2- 172.21.15.49:0/2393659551 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f09c40a2a90 0x7f09c40a4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:18:03.052 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:03.053+0000 7f09d77fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e0805fcef0 0x55e0805fd2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:18:03.052 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:03.053+0000 7f09d77fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e0805fcef0 0x55e0805fd2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.49:57116/0 (socket says 172.21.15.49:57116) 2024-03-18T23:18:03.052 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:03.053+0000 7f09d77fe640 1 -- 172.21.15.49:0/3072024421 learned_addr learned my addr 172.21.15.49:0/3072024421 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:18:03.053 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:03.053+0000 7f09d77fe640 1 --2- 172.21.15.49:0/2393659551 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f09c40a2a90 0x7f09c40a4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:18:03.053 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:03.053+0000 7f09cffff640 1 --2- 172.21.15.49:0/3072024421 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e0806065e0 0x55e0806089d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:18:03.053 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:03.053+0000 7f09dca49640 1 -- 172.21.15.49:0/2393659551 <== mon.1 v2:172.21.15.175:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8600+0+0 (secure 0 0 0) 0x7f09c00efde0 con 0x7f09d810d130 2024-03-18T23:18:03.053 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:03.053+0000 7f09d6ffd640 1 --2- 172.21.15.49:0/3072024421 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e0805fd810 0x55e0806060a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:18:03.054 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:03.053+0000 7f09d77fe640 1 -- 172.21.15.49:0/3072024421 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e0805fd810 msgr2=0x55e0806060a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:18:03.054 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:03.054+0000 7f09d77fe640 1 --2- 172.21.15.49:0/3072024421 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e0805fd810 0x55e0806060a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:03.054 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:03.054+0000 7f09d77fe640 1 -- 172.21.15.49:0/3072024421 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e0806065e0 msgr2=0x55e0806089d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:18:03.054 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:03.054+0000 7f09d77fe640 1 --2- 172.21.15.49:0/3072024421 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e0806065e0 0x55e0806089d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:03.054 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:03.054+0000 7f09d77fe640 1 -- 172.21.15.49:0/3072024421 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e080500d20 con 0x55e0805fcef0 2024-03-18T23:18:03.054 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:03.054+0000 7f09cffff640 1 --2- 172.21.15.49:0/3072024421 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e0806065e0 0x55e0806089d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T23:18:03.054 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:03.054+0000 7f09d77fe640 1 --2- 172.21.15.49:0/3072024421 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e0805fcef0 0x55e0805fd2d0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f09d0013eb0 tx=0x7f09d00117b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c57686ae3b315355 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:18:03.054 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:03.054+0000 7f09d77fe640 1 --2- 172.21.15.49:0/2393659551 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f09c40a2a90 0x7f09c40a4f50 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f09d0011cb0 tx=0x7f09d0008000 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:18:03.054 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:03.054+0000 7f09cdffb640 1 -- 172.21.15.49:0/3072024421 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09d0024070 con 0x55e0805fcef0 2024-03-18T23:18:03.054 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:03.054+0000 7f09cdffb640 1 -- 172.21.15.49:0/3072024421 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f09d00139c0 con 0x55e0805fcef0 2024-03-18T23:18:03.054 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:03.054+0000 7f09cdffb640 1 -- 172.21.15.49:0/3072024421 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09d0005020 con 0x55e0805fcef0 2024-03-18T23:18:03.056 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:03.055+0000 7f09e0636740 1 -- 172.21.15.49:0/3072024421 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e0805fcef0 msgr2=0x55e0805fd2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:18:03.056 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:03.055+0000 7f09e0636740 1 --2- 172.21.15.49:0/3072024421 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e0805fcef0 0x55e0805fd2d0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f09d0013eb0 tx=0x7f09d00117b0 comp rx=0 tx=0).stop 2024-03-18T23:18:03.056 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:03.055+0000 7f09e0636740 1 -- 172.21.15.49:0/3072024421 shutdown_connections 2024-03-18T23:18:03.057 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:03.055+0000 7f09e0636740 1 --2- 172.21.15.49:0/3072024421 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e0806065e0 0x55e0806089d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:03.057 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:03.055+0000 7f09e0636740 1 --2- 172.21.15.49:0/3072024421 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e0805fd810 0x55e0806060a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:03.057 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:03.055+0000 7f09e0636740 1 --2- 172.21.15.49:0/3072024421 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e0805fcef0 0x55e0805fd2d0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:03.057 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:03.055+0000 7f09e0636740 1 -- 172.21.15.49:0/3072024421 >> 172.21.15.49:0/3072024421 conn(0x55e0805f7810 msgr2=0x55e0805f7bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:18:03.057 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:03.056+0000 7f09e0636740 1 -- 172.21.15.49:0/3072024421 shutdown_connections 2024-03-18T23:18:03.057 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:03.056+0000 7f09e0636740 1 -- 172.21.15.49:0/3072024421 wait complete. 2024-03-18T23:18:03.057 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:03.056+0000 7f09e0636740 1 -- start start 2024-03-18T23:18:03.057 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:03.056+0000 7f09e0636740 1 Processor -- start 2024-03-18T23:18:03.057 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:03.057+0000 7f09e0636740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e0805fcef0 0x55e080605fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:18:03.057 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:03.057+0000 7f09e0636740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e0805fd810 0x55e080603370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:18:03.057 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:03.057+0000 7f09e0636740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e0806065e0 0x55e080603980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:18:03.057 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:03.057+0000 7f09e0636740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55e0806091b0 con 0x55e0806065e0 2024-03-18T23:18:03.057 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:03.057+0000 7f09e0636740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55e08051cdb0 con 0x55e0805fd810 2024-03-18T23:18:03.057 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:03.057+0000 7f09e0636740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55e080609330 con 0x55e0805fcef0 2024-03-18T23:18:03.058 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:03.057+0000 7f09e0636740 10 client.0 _ll_register_callbacks cb 0x55e080521e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T23:18:03.058 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:03.057+0000 7f09d6ffd640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e0805fcef0 0x55e080605fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:18:03.058 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:03.057+0000 7f09cffff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e0805fd810 0x55e080603370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:18:03.058 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:03.057+0000 7f09d77fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e0806065e0 0x55e080603980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:18:03.058 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:03.057+0000 7f09cffff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e0805fd810 0x55e080603370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.49:57128/0 (socket says 172.21.15.49:57128) 2024-03-18T23:18:03.058 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:03.057+0000 7f09cffff640 1 -- 172.21.15.49:0/2906515040 learned_addr learned my addr 172.21.15.49:0/2906515040 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:18:03.058 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:03.057+0000 7f09d77fe640 1 -- 172.21.15.49:0/2906515040 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e0805fcef0 msgr2=0x55e080605fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:18:03.058 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:03.057+0000 7f09d77fe640 1 --2- 172.21.15.49:0/2906515040 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e0805fcef0 0x55e080605fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:03.058 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:03.057+0000 7f09d77fe640 1 -- 172.21.15.49:0/2906515040 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e0805fd810 msgr2=0x55e080603370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:18:03.058 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:03.057+0000 7f09d77fe640 1 --2- 172.21.15.49:0/2906515040 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e0805fd810 0x55e080603370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:03.058 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:03.057+0000 7f09d77fe640 1 -- 172.21.15.49:0/2906515040 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e08051bc60 con 0x55e0806065e0 2024-03-18T23:18:03.058 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:03.057+0000 7f09cffff640 1 --2- 172.21.15.49:0/2906515040 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e0805fd810 0x55e080603370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:18:03.058 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:03.058+0000 7f09d77fe640 1 --2- 172.21.15.49:0/2906515040 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e0806065e0 0x55e080603980 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f09d00134b0 tx=0x7f09d0009ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=233172c7fa082d36 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:18:03.058 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:03.058+0000 7f09b6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-18T23:18:03.060 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:03.058+0000 7f09b6ffd640 1 -- 172.21.15.49:0/2906515040 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09d002c020 con 0x55e0806065e0 2024-03-18T23:18:03.060 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:03.058+0000 7f09b6ffd640 1 -- 172.21.15.49:0/2906515040 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8600+0+0 (secure 0 0 0) 0x7f09d0004ca0 con 0x55e0806065e0 2024-03-18T23:18:03.060 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:03.058+0000 7f09e0636740 1 -- 172.21.15.49:0/2906515040 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f09d00139c0 con 0x55e0806065e0 2024-03-18T23:18:03.060 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:03.058+0000 7f09b6ffd640 1 -- 172.21.15.49:0/2906515040 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f09d0037050 con 0x55e0806065e0 2024-03-18T23:18:03.060 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:03.058+0000 7f09e0636740 10 client.16242 fetch_fsmap learned FSMap version 32 2024-03-18T23:18:03.060 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:03.058+0000 7f09e0636740 1 -- 172.21.15.49:0/2906515040 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55e080275210 con 0x55e0806065e0 2024-03-18T23:18:03.060 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:03.058+0000 7f09b6ffd640 1 -- 172.21.15.49:0/2906515040 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f09d00139c0 con 0x55e0806065e0 2024-03-18T23:18:03.060 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:03.059+0000 7f09b6ffd640 1 -- 172.21.15.49:0/2906515040 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f09d00334b0 con 0x55e0806065e0 2024-03-18T23:18:03.060 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:03.059+0000 7f09e0636740 10 client.16242 fetch_fsmap finished waiting for FSMap version 32 2024-03-18T23:18:03.060 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:03.059+0000 7f09e0636740 10 client.16242 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T23:18:03.060 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:03.059+0000 7f09e0636740 20 client.16242 populate_metadata read hostname 'smithi049' 2024-03-18T23:18:03.060 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:03.059+0000 7f09e0636740 1 --2- 172.21.15.49:0/2906515040 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x55e080616490 0x55e080618880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:18:03.060 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:03.059+0000 7f09e0636740 4 client.16242 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T23:18:03.060 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:03.059+0000 7f09e0636740 1 -- 172.21.15.49:0/2906515040 --> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e08056e120 con 0x55e080616490 2024-03-18T23:18:03.060 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:03.060+0000 7f09d6ffd640 1 --2- 172.21.15.49:0/2906515040 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x55e080616490 0x55e080618880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:18:03.061 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:03.062+0000 7f09b6ffd640 10 client.16242 ms_handle_connect on v2:172.21.15.139:6838/4280128209 2024-03-18T23:18:03.061 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:03.062+0000 7f09d6ffd640 1 --2- 172.21.15.49:0/2906515040 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x55e080616490 0x55e080618880 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bc0a78b3b6cdce1b server_cookie=7444fae56855b659 in_seq=0 out_seq=0 2024-03-18T23:18:03.065 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:03.066+0000 7f09b6ffd640 1 -- 172.21.15.49:0/2906515040 <== mds.0 v2:172.21.15.139:6838/4280128209 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55e08056e120 con 0x55e080616490 2024-03-18T23:18:03.065 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:03.066+0000 7f09b6ffd640 10 client.16242 handle_command_reply: tid=0 2024-03-18T23:18:03.065 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:03.066+0000 7f09e0636740 1 client.16242 shutdown 2024-03-18T23:18:03.066 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:03.067+0000 7f09e0636740 1 -- 172.21.15.49:0/2906515040 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e0806065e0 msgr2=0x55e080603980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:18:03.066 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:03.067+0000 7f09e0636740 1 --2- 172.21.15.49:0/2906515040 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e0806065e0 0x55e080603980 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f09d00134b0 tx=0x7f09d0009ea0 comp rx=0 tx=0).stop 2024-03-18T23:18:03.067 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:03.068+0000 7f09e0636740 1 -- 172.21.15.49:0/2906515040 shutdown_connections 2024-03-18T23:18:03.067 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:03.068+0000 7f09e0636740 1 --2- 172.21.15.49:0/2906515040 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x55e080616490 0x55e080618880 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:03.067 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:03.068+0000 7f09e0636740 1 --2- 172.21.15.49:0/2906515040 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e0806065e0 0x55e080603980 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:03.067 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:03.068+0000 7f09e0636740 1 --2- 172.21.15.49:0/2906515040 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e0805fd810 0x55e080603370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:03.067 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:03.068+0000 7f09e0636740 1 --2- 172.21.15.49:0/2906515040 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e0805fcef0 0x55e080605fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:03.067 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:03.068+0000 7f09e0636740 1 -- 172.21.15.49:0/2906515040 >> 172.21.15.49:0/2906515040 conn(0x55e0805f7810 msgr2=0x55e0805f6ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:18:03.067 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:03.068+0000 7f09b6ffd640 0 client.16242 ms_handle_reset on v2:172.21.15.139:6838/4280128209 2024-03-18T23:18:03.068 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:03.068+0000 7f09e0636740 1 -- 172.21.15.49:0/2906515040 shutdown_connections 2024-03-18T23:18:03.068 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:03.068+0000 7f09e0636740 1 -- 172.21.15.49:0/2906515040 wait complete. 2024-03-18T23:18:03.068 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:03.068+0000 7f09e0636740 20 client.16242 trim_cache size 0 max 16384 2024-03-18T23:18:03.090 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:03.091+0000 7f09e0636740 1 Processor -- start 2024-03-18T23:18:03.091 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:03.091+0000 7f09e0636740 1 -- start start 2024-03-18T23:18:03.091 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:03.091+0000 7f09e0636740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e0805fcef0 0x55e0805f6cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:18:03.091 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:03.091+0000 7f09e0636740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e0805fd810 0x55e080611f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:18:03.091 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:03.091+0000 7f09e0636740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e0806065e0 0x55e08061df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:18:03.091 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:03.091+0000 7f09e0636740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f09b800e7f0 con 0x55e0805fcef0 2024-03-18T23:18:03.091 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:03.091+0000 7f09e0636740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f09b8010330 con 0x55e0806065e0 2024-03-18T23:18:03.091 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:03.091+0000 7f09e0636740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f09b800eec0 con 0x55e0805fd810 2024-03-18T23:18:03.091 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:03.092+0000 7f09d6ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e0805fcef0 0x55e0805f6cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:18:03.092 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:03.092+0000 7f09d77fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e0806065e0 0x55e08061df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:18:03.092 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:03.092+0000 7f09cffff640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e0805fd810 0x55e080611f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:18:03.092 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:03.092+0000 7f09d77fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e0806065e0 0x55e08061df10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.49:57136/0 (socket says 172.21.15.49:57136) 2024-03-18T23:18:03.092 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:03.092+0000 7f09d6ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e0805fcef0 0x55e0805f6cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:35876/0 (socket says 172.21.15.49:35876) 2024-03-18T23:18:03.092 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:03.092+0000 7f09d77fe640 1 -- 172.21.15.49:0/2093093546 learned_addr learned my addr 172.21.15.49:0/2093093546 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:18:03.092 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:03.092+0000 7f09cffff640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e0805fd810 0x55e080611f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.49:41680/0 (socket says 172.21.15.49:41680) 2024-03-18T23:18:03.094 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:03.092+0000 7f09d6ffd640 1 -- 172.21.15.49:0/2093093546 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e0805fd810 msgr2=0x55e080611f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:18:03.094 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:03.092+0000 7f09d6ffd640 1 --2- 172.21.15.49:0/2093093546 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e0805fd810 0x55e080611f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:03.094 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:03.092+0000 7f09d6ffd640 1 -- 172.21.15.49:0/2093093546 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e0806065e0 msgr2=0x55e08061df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:18:03.094 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:03.092+0000 7f09d6ffd640 1 --2- 172.21.15.49:0/2093093546 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e0806065e0 0x55e08061df10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:03.094 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:03.092+0000 7f09d6ffd640 1 -- 172.21.15.49:0/2093093546 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e080341620 con 0x55e0805fcef0 2024-03-18T23:18:03.094 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:03.093+0000 7f09d6ffd640 1 --2- 172.21.15.49:0/2093093546 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e0805fcef0 0x55e0805f6cf0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f09c8000c00 tx=0x7f09c8009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70775c0a969bb624 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:18:03.094 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:03.093+0000 7f09cd7fa640 1 -- 172.21.15.49:0/2093093546 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09c801b530 con 0x55e0805fcef0 2024-03-18T23:18:03.094 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:03.093+0000 7f09cd7fa640 1 -- 172.21.15.49:0/2093093546 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f09c8009e40 con 0x55e0805fcef0 2024-03-18T23:18:03.094 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:03.093+0000 7f09cd7fa640 1 -- 172.21.15.49:0/2093093546 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09c8004da0 con 0x55e0805fcef0 2024-03-18T23:18:03.094 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:03.094+0000 7f09e0636740 1 -- 172.21.15.49:0/2093093546 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e0805fcef0 msgr2=0x55e0805f6cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:18:03.095 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:03.094+0000 7f09e0636740 1 --2- 172.21.15.49:0/2093093546 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e0805fcef0 0x55e0805f6cf0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f09c8000c00 tx=0x7f09c8009260 comp rx=0 tx=0).stop 2024-03-18T23:18:03.095 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:03.094+0000 7f09e0636740 1 -- 172.21.15.49:0/2093093546 shutdown_connections 2024-03-18T23:18:03.095 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:03.094+0000 7f09e0636740 1 --2- 172.21.15.49:0/2093093546 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e0806065e0 0x55e08061df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:03.095 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:03.094+0000 7f09e0636740 1 --2- 172.21.15.49:0/2093093546 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e0805fd810 0x55e080611f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:03.095 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:03.094+0000 7f09e0636740 1 --2- 172.21.15.49:0/2093093546 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e0805fcef0 0x55e0805f6cf0 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:03.095 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:03.094+0000 7f09e0636740 1 -- 172.21.15.49:0/2093093546 >> 172.21.15.49:0/2093093546 conn(0x55e0805f7810 msgr2=0x55e0805fe8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:18:03.095 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:03.094+0000 7f09e0636740 1 -- 172.21.15.49:0/2093093546 shutdown_connections 2024-03-18T23:18:03.095 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:03.094+0000 7f09e0636740 1 -- 172.21.15.49:0/2093093546 wait complete. 2024-03-18T23:18:03.095 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:03.095+0000 7f09e0636740 1 -- start start 2024-03-18T23:18:03.095 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:03.095+0000 7f09e0636740 1 Processor -- start 2024-03-18T23:18:03.097 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:03.095+0000 7f09e0636740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e0805fcef0 0x55e08060e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:18:03.097 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:03.095+0000 7f09e0636740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e0805fd810 0x55e08060e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:18:03.097 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:03.095+0000 7f09e0636740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e0806065e0 0x55e080609b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:18:03.097 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:03.096+0000 7f09d6ffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e0805fcef0 0x55e08060e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:18:03.097 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:03.096+0000 7f09cffff640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e0805fd810 0x55e08060e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:18:03.098 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:03.096+0000 7f09d6ffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e0805fcef0 0x55e08060e4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.49:57150/0 (socket says 172.21.15.49:57150) 2024-03-18T23:18:03.098 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:03.096+0000 7f09d6ffd640 1 -- 172.21.15.49:0/1451790585 learned_addr learned my addr 172.21.15.49:0/1451790585 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:18:03.098 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:03.096+0000 7f09e0636740 1 -- 172.21.15.49:0/1451790585 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f09b800fc60 con 0x55e0806065e0 2024-03-18T23:18:03.098 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:03.096+0000 7f09e0636740 1 -- 172.21.15.49:0/1451790585 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f09b800f590 con 0x55e0805fcef0 2024-03-18T23:18:03.098 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:03.096+0000 7f09e0636740 1 -- 172.21.15.49:0/1451790585 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f09b800c7f0 con 0x55e0805fd810 2024-03-18T23:18:03.098 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:03.096+0000 7f09d77fe640 1 --2- 172.21.15.49:0/1451790585 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e0806065e0 0x55e080609b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:18:03.098 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:03.096+0000 7f09e0636740 10 client.0 _ll_register_callbacks cb 0x7f09d0013cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T23:18:03.098 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:03.096+0000 7f09cffff640 1 -- 172.21.15.49:0/1451790585 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e0805fcef0 msgr2=0x55e08060e4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:18:03.098 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:03.096+0000 7f09cffff640 1 --2- 172.21.15.49:0/1451790585 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e0805fcef0 0x55e08060e4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:03.098 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:03.096+0000 7f09cffff640 1 -- 172.21.15.49:0/1451790585 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e0806065e0 msgr2=0x55e080609b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:18:03.098 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:03.096+0000 7f09cffff640 1 --2- 172.21.15.49:0/1451790585 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e0806065e0 0x55e080609b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:03.098 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:03.096+0000 7f09cffff640 1 -- 172.21.15.49:0/1451790585 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e08060a350 con 0x55e0805fd810 2024-03-18T23:18:03.098 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:03.096+0000 7f09d77fe640 1 --2- 172.21.15.49:0/1451790585 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e0806065e0 0x55e080609b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:18:03.098 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:03.096+0000 7f09cffff640 1 --2- 172.21.15.49:0/1451790585 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e0805fd810 0x55e08060e9e0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f09c00ed5f0 tx=0x7f09c0032530 comp rx=0 tx=0).ready entity=mon.1 client_cookie=45d774f3f121f8c5 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:18:03.098 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:03.096+0000 7f09b6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-03-18T23:18:03.099 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:03.097+0000 7f09b6ffd640 1 -- 172.21.15.49:0/1451790585 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09c00bcba0 con 0x55e0805fd810 2024-03-18T23:18:03.099 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:03.097+0000 7f09b6ffd640 1 -- 172.21.15.49:0/1451790585 <== mon.1 v2:172.21.15.175:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8600+0+0 (secure 0 0 0) 0x7f09c00bc700 con 0x55e0805fd810 2024-03-18T23:18:03.099 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:03.097+0000 7f09e0636740 1 -- 172.21.15.49:0/1451790585 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55e080568470 con 0x55e0805fd810 2024-03-18T23:18:03.099 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:03.097+0000 7f09b6ffd640 1 -- 172.21.15.49:0/1451790585 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f09c00bc4a0 con 0x55e0805fd810 2024-03-18T23:18:03.099 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:03.097+0000 7f09e0636740 10 client.26050 fetch_fsmap learned FSMap version 32 2024-03-18T23:18:03.099 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:03.097+0000 7f09e0636740 1 -- 172.21.15.49:0/1451790585 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55e08067c240 con 0x55e0805fd810 2024-03-18T23:18:03.099 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:03.098+0000 7f09b6ffd640 1 -- 172.21.15.49:0/1451790585 <== mon.1 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f09c00c05f0 con 0x55e0805fd810 2024-03-18T23:18:03.099 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:03.098+0000 7f09b6ffd640 1 -- 172.21.15.49:0/1451790585 <== mon.1 v2:172.21.15.175:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f09c003c030 con 0x55e0805fd810 2024-03-18T23:18:03.099 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:03.098+0000 7f09e0636740 10 client.26050 fetch_fsmap finished waiting for FSMap version 32 2024-03-18T23:18:03.099 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:03.098+0000 7f09e0636740 10 client.26050 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T23:18:03.099 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:03.098+0000 7f09e0636740 20 client.26050 populate_metadata read hostname 'smithi049' 2024-03-18T23:18:03.099 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:03.098+0000 7f09e0636740 1 --2- 172.21.15.49:0/1451790585 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x55e08067ca10 0x55e08067ee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:18:03.099 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:03.098+0000 7f09e0636740 4 client.26050 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T23:18:03.099 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:03.098+0000 7f09e0636740 1 -- 172.21.15.49:0/1451790585 --> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e080568470 con 0x55e08067ca10 2024-03-18T23:18:03.099 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:03.098+0000 7f09d6ffd640 1 --2- 172.21.15.49:0/1451790585 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x55e08067ca10 0x55e08067ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:18:03.100 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:03.099+0000 7f09d6ffd640 1 --2- 172.21.15.49:0/1451790585 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x55e08067ca10 0x55e08067ee30 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9dc9f82e7663f656 server_cookie=9abf227289e2f6df in_seq=0 out_seq=0 2024-03-18T23:18:03.100 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:03.099+0000 7f09b6ffd640 10 client.26050 ms_handle_connect on v2:172.21.15.139:6838/4280128209 2024-03-18T23:18:03.319 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:03.320+0000 7f09b6ffd640 1 -- 172.21.15.49:0/1451790585 <== mds.0 v2:172.21.15.139:6838/4280128209 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e080568470 con 0x55e08067ca10 2024-03-18T23:18:03.320 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:03.320+0000 7f09b6ffd640 10 client.26050 handle_command_reply: tid=0 2024-03-18T23:18:03.320 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:03.320+0000 7f09e0636740 1 client.26050 shutdown 2024-03-18T23:18:03.320 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:03.320+0000 7f09e0636740 1 -- 172.21.15.49:0/1451790585 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e0805fd810 msgr2=0x55e08060e9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:18:03.320 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:03.320+0000 7f09e0636740 1 --2- 172.21.15.49:0/1451790585 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e0805fd810 0x55e08060e9e0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f09c00ed5f0 tx=0x7f09c0032530 comp rx=0 tx=0).stop 2024-03-18T23:18:03.320 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:03.320+0000 7f09e0636740 1 -- 172.21.15.49:0/1451790585 shutdown_connections 2024-03-18T23:18:03.320 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:03.320+0000 7f09e0636740 1 --2- 172.21.15.49:0/1451790585 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x55e08067ca10 0x55e08067ee30 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:03.320 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:03.320+0000 7f09e0636740 1 --2- 172.21.15.49:0/1451790585 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e0806065e0 0x55e080609b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:03.320 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:03.320+0000 7f09b6ffd640 0 client.26050 ms_handle_reset on v2:172.21.15.139:6838/4280128209 2024-03-18T23:18:03.320 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:03.320+0000 7f09e0636740 1 --2- 172.21.15.49:0/1451790585 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e0805fd810 0x55e08060e9e0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:03.320 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:03.320+0000 7f09e0636740 1 --2- 172.21.15.49:0/1451790585 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e0805fcef0 0x55e08060e4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:03.320 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:03.320+0000 7f09e0636740 1 -- 172.21.15.49:0/1451790585 >> 172.21.15.49:0/1451790585 conn(0x55e0805f7810 msgr2=0x55e0805fde20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:18:03.320 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:03.320+0000 7f09e0636740 1 -- 172.21.15.49:0/1451790585 shutdown_connections 2024-03-18T23:18:03.320 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:03.320+0000 7f09e0636740 1 -- 172.21.15.49:0/1451790585 wait complete. 2024-03-18T23:18:03.321 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:03.320+0000 7f09e0636740 20 client.26050 trim_cache size 0 max 16384 2024-03-18T23:18:03.321 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:{ 2024-03-18T23:18:03.321 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "message": "", 2024-03-18T23:18:03.321 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "return_code": 0 2024-03-18T23:18:03.321 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:} 2024-03-18T23:18:03.322 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:03.321+0000 7f09dda4b640 1 -- 172.21.15.49:0/2393659551 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f09c40a2a90 msgr2=0x7f09c40a4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:18:03.322 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:03.321+0000 7f09dda4b640 1 --2- 172.21.15.49:0/2393659551 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f09c40a2a90 0x7f09c40a4f50 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f09d0011cb0 tx=0x7f09d0008000 comp rx=0 tx=0).stop 2024-03-18T23:18:03.322 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:03.321+0000 7f09dda4b640 1 -- 172.21.15.49:0/2393659551 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f09d810d130 msgr2=0x7f09d819ebc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:18:03.322 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:03.321+0000 7f09dda4b640 1 --2- 172.21.15.49:0/2393659551 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f09d810d130 0x7f09d819ebc0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f09c0037040 tx=0x7f09c00045b0 comp rx=0 tx=0).stop 2024-03-18T23:18:03.322 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:03.321+0000 7f09dda4b640 1 -- 172.21.15.49:0/2393659551 shutdown_connections 2024-03-18T23:18:03.322 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:03.321+0000 7f09dda4b640 1 --2- 172.21.15.49:0/2393659551 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f09c40a2a90 0x7f09c40a4f50 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:03.322 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:03.321+0000 7f09dda4b640 1 --2- 172.21.15.49:0/2393659551 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f09d81a2ef0 0x7f09d81a33a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:03.322 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:03.321+0000 7f09dda4b640 1 --2- 172.21.15.49:0/2393659551 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f09d810d130 0x7f09d819ebc0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:03.322 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:03.321+0000 7f09dda4b640 1 --2- 172.21.15.49:0/2393659551 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f09d8108060 0x7f09d819e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:03.322 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:03.321+0000 7f09dda4b640 1 -- 172.21.15.49:0/2393659551 >> 172.21.15.49:0/2393659551 conn(0x7f09d8075d90 msgr2=0x7f09d8076930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:18:03.322 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:03.321+0000 7f09dda4b640 1 -- 172.21.15.49:0/2393659551 shutdown_connections 2024-03-18T23:18:03.323 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:03.321+0000 7f09dda4b640 1 -- 172.21.15.49:0/2393659551 wait complete. 2024-03-18T23:18:03.379 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:18:03 smithi049 ceph-mon[62991]: pgmap v305: 97 pgs: 32 active+clean+snaptrim_wait, 15 active+clean+snaptrim, 50 active+clean; 96 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.6 MiB/s wr, 216 op/s 2024-03-18T23:18:03.460 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:18:03 smithi175 ceph-mon[67352]: pgmap v305: 97 pgs: 32 active+clean+snaptrim_wait, 15 active+clean+snaptrim, 50 active+clean; 96 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.6 MiB/s wr, 216 op/s 2024-03-18T23:18:03.564 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:18:03 smithi139 ceph-mon[67282]: pgmap v305: 97 pgs: 32 active+clean+snaptrim_wait, 15 active+clean+snaptrim, 50 active+clean; 96 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.6 MiB/s wr, 216 op/s 2024-03-18T23:18:03.784 INFO:tasks.workunit.client.0.smithi049.stdout: 1 1016112 9.42 MB/sec execute 425 sec latency 105.399 ms 2024-03-18T23:18:04.784 INFO:tasks.workunit.client.0.smithi049.stdout: 1 1017108 9.41 MB/sec execute 426 sec latency 17.256 ms 2024-03-18T23:18:05.379 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:18:05 smithi049 ceph-mon[62991]: pgmap v306: 97 pgs: 41 active+clean+snaptrim_wait, 23 active+clean+snaptrim, 33 active+clean; 97 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.9 MiB/s wr, 210 op/s 2024-03-18T23:18:05.460 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:18:05 smithi175 ceph-mon[67352]: pgmap v306: 97 pgs: 41 active+clean+snaptrim_wait, 23 active+clean+snaptrim, 33 active+clean; 97 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.9 MiB/s wr, 210 op/s 2024-03-18T23:18:05.564 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:18:05 smithi139 ceph-mon[67282]: pgmap v306: 97 pgs: 41 active+clean+snaptrim_wait, 23 active+clean+snaptrim, 33 active+clean; 97 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.9 MiB/s wr, 210 op/s 2024-03-18T23:18:05.784 INFO:tasks.workunit.client.0.smithi049.stdout: 1 1019216 9.41 MB/sec execute 427 sec latency 23.684 ms 2024-03-18T23:18:06.784 INFO:tasks.workunit.client.0.smithi049.stdout: 1 1020161 9.41 MB/sec execute 428 sec latency 33.206 ms 2024-03-18T23:18:07.379 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:18:07 smithi049 ceph-mon[62991]: pgmap v307: 97 pgs: 41 active+clean+snaptrim_wait, 23 active+clean+snaptrim, 33 active+clean; 97 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.9 MiB/s wr, 210 op/s 2024-03-18T23:18:07.460 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:18:07 smithi175 ceph-mon[67352]: pgmap v307: 97 pgs: 41 active+clean+snaptrim_wait, 23 active+clean+snaptrim, 33 active+clean; 97 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.9 MiB/s wr, 210 op/s 2024-03-18T23:18:07.564 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:18:07 smithi139 ceph-mon[67282]: pgmap v307: 97 pgs: 41 active+clean+snaptrim_wait, 23 active+clean+snaptrim, 33 active+clean; 97 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.9 MiB/s wr, 210 op/s 2024-03-18T23:18:07.785 INFO:tasks.workunit.client.0.smithi049.stdout: 1 1021459 9.40 MB/sec execute 429 sec latency 68.894 ms 2024-03-18T23:18:08.784 INFO:tasks.workunit.client.0.smithi049.stdout: 1 1023251 9.40 MB/sec execute 430 sec latency 19.868 ms 2024-03-18T23:18:09.460 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:18:09 smithi175 ceph-mon[67352]: pgmap v308: 97 pgs: 97 active+clean; 68 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.8 MiB/s wr, 266 op/s 2024-03-18T23:18:09.460 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:18:09 smithi175 ceph-mon[67352]: osdmap e99: 12 total, 12 up, 12 in 2024-03-18T23:18:09.564 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:18:09 smithi139 ceph-mon[67282]: pgmap v308: 97 pgs: 97 active+clean; 68 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.8 MiB/s wr, 266 op/s 2024-03-18T23:18:09.564 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:18:09 smithi139 ceph-mon[67282]: osdmap e99: 12 total, 12 up, 12 in 2024-03-18T23:18:09.629 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:18:09 smithi049 ceph-mon[62991]: pgmap v308: 97 pgs: 97 active+clean; 68 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.8 MiB/s wr, 266 op/s 2024-03-18T23:18:09.629 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:18:09 smithi049 ceph-mon[62991]: osdmap e99: 12 total, 12 up, 12 in 2024-03-18T23:18:09.785 INFO:tasks.workunit.client.0.smithi049.stdout: 1 1024317 9.39 MB/sec execute 431 sec latency 51.176 ms 2024-03-18T23:18:10.785 INFO:tasks.workunit.client.0.smithi049.stdout: 1 1026512 9.40 MB/sec execute 432 sec latency 20.576 ms 2024-03-18T23:18:11.460 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:18:11 smithi175 ceph-mon[67352]: pgmap v310: 97 pgs: 97 active+clean; 68 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.3 MiB/s wr, 319 op/s 2024-03-18T23:18:11.564 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:18:11 smithi139 ceph-mon[67282]: pgmap v310: 97 pgs: 97 active+clean; 68 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.3 MiB/s wr, 319 op/s 2024-03-18T23:18:11.629 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:18:11 smithi049 ceph-mon[62991]: pgmap v310: 97 pgs: 97 active+clean; 68 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.3 MiB/s wr, 319 op/s 2024-03-18T23:18:11.785 INFO:tasks.workunit.client.0.smithi049.stdout: 1 1027308 9.39 MB/sec execute 433 sec latency 25.482 ms 2024-03-18T23:18:12.785 INFO:tasks.workunit.client.0.smithi049.stdout: 1 1028829 9.38 MB/sec execute 434 sec latency 80.345 ms 2024-03-18T23:18:13.460 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:18:13 smithi175 ceph-mon[67352]: pgmap v311: 97 pgs: 97 active+clean; 74 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.8 MiB/s wr, 257 op/s 2024-03-18T23:18:13.564 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:18:13 smithi139 ceph-mon[67282]: pgmap v311: 97 pgs: 97 active+clean; 74 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.8 MiB/s wr, 257 op/s 2024-03-18T23:18:13.629 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:18:13 smithi049 ceph-mon[62991]: pgmap v311: 97 pgs: 97 active+clean; 74 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.8 MiB/s wr, 257 op/s 2024-03-18T23:18:13.785 INFO:tasks.workunit.client.0.smithi049.stdout: 1 1030375 9.38 MB/sec execute 435 sec latency 18.159 ms 2024-03-18T23:18:14.785 INFO:tasks.workunit.client.0.smithi049.stdout: 1 1031724 9.38 MB/sec execute 436 sec latency 25.573 ms 2024-03-18T23:18:15.460 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:18:15 smithi175 ceph-mon[67352]: pgmap v312: 97 pgs: 97 active+clean; 73 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.5 MiB/s wr, 237 op/s 2024-03-18T23:18:15.564 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:18:15 smithi139 ceph-mon[67282]: pgmap v312: 97 pgs: 97 active+clean; 73 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.5 MiB/s wr, 237 op/s 2024-03-18T23:18:15.629 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:18:15 smithi049 ceph-mon[62991]: pgmap v312: 97 pgs: 97 active+clean; 73 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.5 MiB/s wr, 237 op/s 2024-03-18T23:18:15.785 INFO:tasks.workunit.client.0.smithi049.stdout: 1 1033783 9.38 MB/sec execute 437 sec latency 19.455 ms 2024-03-18T23:18:16.424 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:16.424+0000 7f97c9ec6640 1 -- 172.21.15.49:0/3564252468 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f97c41056c0 msgr2=0x7f97c4105aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:18:16.425 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:16.424+0000 7f97c9ec6640 1 --2- 172.21.15.49:0/3564252468 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f97c41056c0 0x7f97c4105aa0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f97b4009900 tx=0x7f97b402bc30 comp rx=0 tx=0).stop 2024-03-18T23:18:16.425 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:16.424+0000 7f97c9ec6640 1 -- 172.21.15.49:0/3564252468 shutdown_connections 2024-03-18T23:18:16.425 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:16.424+0000 7f97c9ec6640 1 --2- 172.21.15.49:0/3564252468 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f97c4101330 0x7f97c410d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:16.425 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:16.424+0000 7f97c9ec6640 1 --2- 172.21.15.49:0/3564252468 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f97c4100970 0x7f97c4100df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:16.425 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:16.424+0000 7f97c9ec6640 1 --2- 172.21.15.49:0/3564252468 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f97c41056c0 0x7f97c4105aa0 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:16.425 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:16.424+0000 7f97c9ec6640 1 -- 172.21.15.49:0/3564252468 >> 172.21.15.49:0/3564252468 conn(0x7f97c4077570 msgr2=0x7f97c4077980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:18:16.425 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:16.424+0000 7f97c9ec6640 1 -- 172.21.15.49:0/3564252468 shutdown_connections 2024-03-18T23:18:16.425 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:16.424+0000 7f97c9ec6640 1 -- 172.21.15.49:0/3564252468 wait complete. 2024-03-18T23:18:16.425 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:16.425+0000 7f97c9ec6640 1 Processor -- start 2024-03-18T23:18:16.427 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:16.425+0000 7f97c9ec6640 1 -- start start 2024-03-18T23:18:16.428 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:16.425+0000 7f97c9ec6640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f97c4100970 0x7f97c4198120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:18:16.428 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:16.425+0000 7f97c9ec6640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f97c4101330 0x7f97c4198660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:18:16.428 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:16.425+0000 7f97c9ec6640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f97c41056c0 0x7f97c419c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:18:16.428 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:16.425+0000 7f97c9ec6640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f97c4111d00 con 0x7f97c41056c0 2024-03-18T23:18:16.428 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:16.425+0000 7f97c9ec6640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f97c4111b80 con 0x7f97c4100970 2024-03-18T23:18:16.428 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:16.425+0000 7f97c9ec6640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f97c4111e80 con 0x7f97c4101330 2024-03-18T23:18:16.428 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:16.425+0000 7f97c3fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f97c41056c0 0x7f97c419c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:18:16.428 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:16.425+0000 7f97c3fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f97c41056c0 0x7f97c419c9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49182/0 (socket says 172.21.15.49:49182) 2024-03-18T23:18:16.428 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:16.425+0000 7f97c3fff640 1 -- 172.21.15.49:0/3055310418 learned_addr learned my addr 172.21.15.49:0/3055310418 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:18:16.428 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:16.425+0000 7f97c37fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f97c4100970 0x7f97c4198120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:18:16.428 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:16.425+0000 7f97c2ffd640 1 --2- 172.21.15.49:0/3055310418 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f97c4101330 0x7f97c4198660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:18:16.428 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:16.426+0000 7f97c3fff640 1 -- 172.21.15.49:0/3055310418 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f97c4101330 msgr2=0x7f97c4198660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:18:16.429 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:16.426+0000 7f97c3fff640 1 --2- 172.21.15.49:0/3055310418 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f97c4101330 0x7f97c4198660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:16.429 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:16.426+0000 7f97c3fff640 1 -- 172.21.15.49:0/3055310418 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f97c4100970 msgr2=0x7f97c4198120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:18:16.429 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:16.426+0000 7f97c3fff640 1 --2- 172.21.15.49:0/3055310418 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f97c4100970 0x7f97c4198120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:16.429 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:16.426+0000 7f97c3fff640 1 -- 172.21.15.49:0/3055310418 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97c419d110 con 0x7f97c41056c0 2024-03-18T23:18:16.429 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:16.426+0000 7f97c2ffd640 1 --2- 172.21.15.49:0/3055310418 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f97c4101330 0x7f97c4198660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:18:16.429 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:16.426+0000 7f97c3fff640 1 --2- 172.21.15.49:0/3055310418 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f97c41056c0 0x7f97c419c9e0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f97bc00ea60 tx=0x7f97bc00ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:18:16.429 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:16.426+0000 7f97c0ff9640 1 -- 172.21.15.49:0/3055310418 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97bc00ce40 con 0x7f97c41056c0 2024-03-18T23:18:16.429 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:16.426+0000 7f97c9ec6640 1 -- 172.21.15.49:0/3055310418 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97c419d400 con 0x7f97c41056c0 2024-03-18T23:18:16.429 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:16.426+0000 7f97c0ff9640 1 -- 172.21.15.49:0/3055310418 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f97bc005020 con 0x7f97c41056c0 2024-03-18T23:18:16.429 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:16.426+0000 7f97c0ff9640 1 -- 172.21.15.49:0/3055310418 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97bc010690 con 0x7f97c41056c0 2024-03-18T23:18:16.429 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:16.426+0000 7f97c9ec6640 1 -- 172.21.15.49:0/3055310418 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97c41a4c90 con 0x7f97c41056c0 2024-03-18T23:18:16.437 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:16.429+0000 7f97c0ff9640 1 -- 172.21.15.49:0/3055310418 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 25) ==== 137421+0+0 (secure 0 0 0) 0x7f97bc005520 con 0x7f97c41056c0 2024-03-18T23:18:16.437 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:16.429+0000 7f97ccb8c740 1 Processor -- start 2024-03-18T23:18:16.437 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:16.429+0000 7f97ccb8c740 1 -- start start 2024-03-18T23:18:16.437 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:16.429+0000 7f97ccb8c740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c1e871fef0 0x55c1e87202d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:18:16.437 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:16.429+0000 7f97ccb8c740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c1e8720810 0x55c1e87290a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:18:16.437 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:16.429+0000 7f97ccb8c740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c1e87295e0 0x55c1e872b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:18:16.437 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:16.429+0000 7f97ccb8c740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55c1e8683be0 con 0x55c1e871fef0 2024-03-18T23:18:16.437 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:16.429+0000 7f97ccb8c740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55c1e862c960 con 0x55c1e87295e0 2024-03-18T23:18:16.437 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:16.429+0000 7f97ccb8c740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55c1e867fc80 con 0x55c1e8720810 2024-03-18T23:18:16.438 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:16.429+0000 7f97c0ff9640 1 --2- 172.21.15.49:0/3055310418 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f97ac0a2a10 0x7f97ac0a4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:18:16.438 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:16.429+0000 7f97c2ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c1e871fef0 0x55c1e87202d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:18:16.438 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:16.429+0000 7f97c2ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c1e871fef0 0x55c1e87202d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49186/0 (socket says 172.21.15.49:49186) 2024-03-18T23:18:16.438 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:16.430+0000 7f97c2ffd640 1 -- 172.21.15.49:0/367536471 learned_addr learned my addr 172.21.15.49:0/367536471 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:18:16.438 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:16.430+0000 7f97c37fe640 1 --2- 172.21.15.49:0/367536471 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c1e8720810 0x55c1e87290a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:18:16.438 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:16.430+0000 7f97c2ffd640 1 --2- 172.21.15.49:0/367536471 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c1e87295e0 0x55c1e872b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:18:16.438 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:16.430+0000 7f97c3fff640 1 --2- 172.21.15.49:0/3055310418 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f97ac0a2a10 0x7f97ac0a4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:18:16.438 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:16.430+0000 7f97c0ff9640 1 -- 172.21.15.49:0/3055310418 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8556+0+0 (secure 0 0 0) 0x7f97bc0cf2c0 con 0x7f97c41056c0 2024-03-18T23:18:16.438 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:16.430+0000 7f97c2ffd640 1 -- 172.21.15.49:0/367536471 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c1e8720810 msgr2=0x55c1e87290a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:18:16.438 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:16.430+0000 7f97c2ffd640 1 --2- 172.21.15.49:0/367536471 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c1e8720810 0x55c1e87290a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:16.438 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:16.430+0000 7f97c2ffd640 1 -- 172.21.15.49:0/367536471 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c1e87295e0 msgr2=0x55c1e872b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:18:16.438 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:16.430+0000 7f97c2ffd640 1 --2- 172.21.15.49:0/367536471 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c1e87295e0 0x55c1e872b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:16.439 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:16.430+0000 7f97c2ffd640 1 -- 172.21.15.49:0/367536471 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c1e8623d20 con 0x55c1e871fef0 2024-03-18T23:18:16.439 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:16.430+0000 7f97c3fff640 1 --2- 172.21.15.49:0/3055310418 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f97ac0a2a10 0x7f97ac0a4ed0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f97bc0ccaa0 tx=0x7f97bc086040 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:18:16.439 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:16.430+0000 7f97c2ffd640 1 --2- 172.21.15.49:0/367536471 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c1e871fef0 0x55c1e87202d0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f97b8009710 tx=0x7f97b8009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4d17acdf668b6ce server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:18:16.439 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:16.430+0000 7f97a9ffb640 1 -- 172.21.15.49:0/367536471 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97b8013cd0 con 0x55c1e871fef0 2024-03-18T23:18:16.439 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:16.430+0000 7f97a9ffb640 1 -- 172.21.15.49:0/367536471 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f97b8005500 con 0x55c1e871fef0 2024-03-18T23:18:16.439 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:16.430+0000 7f97a9ffb640 1 -- 172.21.15.49:0/367536471 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97b8004a00 con 0x55c1e871fef0 2024-03-18T23:18:16.439 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:16.431+0000 7f97ccb8c740 1 -- 172.21.15.49:0/367536471 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c1e871fef0 msgr2=0x55c1e87202d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:18:16.439 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:16.431+0000 7f97ccb8c740 1 --2- 172.21.15.49:0/367536471 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c1e871fef0 0x55c1e87202d0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f97b8009710 tx=0x7f97b8009b30 comp rx=0 tx=0).stop 2024-03-18T23:18:16.439 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:16.431+0000 7f97ccb8c740 1 -- 172.21.15.49:0/367536471 shutdown_connections 2024-03-18T23:18:16.439 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:16.431+0000 7f97ccb8c740 1 --2- 172.21.15.49:0/367536471 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c1e87295e0 0x55c1e872b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:16.439 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:16.431+0000 7f97ccb8c740 1 --2- 172.21.15.49:0/367536471 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c1e8720810 0x55c1e87290a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:16.439 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:16.431+0000 7f97ccb8c740 1 --2- 172.21.15.49:0/367536471 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c1e871fef0 0x55c1e87202d0 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:16.439 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:16.431+0000 7f97ccb8c740 1 -- 172.21.15.49:0/367536471 >> 172.21.15.49:0/367536471 conn(0x55c1e871a810 msgr2=0x55c1e871abf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:18:16.440 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:16.431+0000 7f97ccb8c740 1 -- 172.21.15.49:0/367536471 shutdown_connections 2024-03-18T23:18:16.440 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:16.431+0000 7f97ccb8c740 1 -- 172.21.15.49:0/367536471 wait complete. 2024-03-18T23:18:16.440 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:16.431+0000 7f97ccb8c740 1 -- start start 2024-03-18T23:18:16.440 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:16.432+0000 7f97ccb8c740 1 Processor -- start 2024-03-18T23:18:16.440 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:16.432+0000 7f97ccb8c740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c1e871fef0 0x55c1e8728f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:18:16.440 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:16.432+0000 7f97ccb8c740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c1e8720810 0x55c1e8726300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:18:16.440 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:16.432+0000 7f97ccb8c740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c1e87295e0 0x55c1e8726880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:18:16.440 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:16.432+0000 7f97ccb8c740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55c1e872c1b0 con 0x55c1e87295e0 2024-03-18T23:18:16.440 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:16.432+0000 7f97ccb8c740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55c1e863fdb0 con 0x55c1e871fef0 2024-03-18T23:18:16.440 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:16.432+0000 7f97ccb8c740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55c1e872c330 con 0x55c1e8720810 2024-03-18T23:18:16.440 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:16.432+0000 7f97ccb8c740 10 client.0 _ll_register_callbacks cb 0x55c1e8644e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T23:18:16.440 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:16.432+0000 7f97c2ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c1e87295e0 0x55c1e8726880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:18:16.440 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:16.432+0000 7f97c37fe640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c1e8720810 0x55c1e8726300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:18:16.441 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:16.432+0000 7f97c37fe640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c1e8720810 0x55c1e8726300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.49:41118/0 (socket says 172.21.15.49:41118) 2024-03-18T23:18:16.441 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:16.432+0000 7f97c2ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c1e87295e0 0x55c1e8726880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49188/0 (socket says 172.21.15.49:49188) 2024-03-18T23:18:16.441 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:16.432+0000 7f97c37fe640 1 -- 172.21.15.49:0/2168756573 learned_addr learned my addr 172.21.15.49:0/2168756573 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:18:16.441 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:16.432+0000 7f97c2ffd640 1 --2- 172.21.15.49:0/2168756573 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c1e871fef0 0x55c1e8728f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:18:16.441 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:16.432+0000 7f97c37fe640 1 -- 172.21.15.49:0/2168756573 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c1e871fef0 msgr2=0x55c1e8728f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:18:16.441 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:16.432+0000 7f97c37fe640 1 --2- 172.21.15.49:0/2168756573 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c1e871fef0 0x55c1e8728f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:16.441 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:16.432+0000 7f97c37fe640 1 -- 172.21.15.49:0/2168756573 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c1e87295e0 msgr2=0x55c1e8726880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:18:16.441 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:16.432+0000 7f97c37fe640 1 --2- 172.21.15.49:0/2168756573 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c1e87295e0 0x55c1e8726880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:16.441 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:16.432+0000 7f97c37fe640 1 -- 172.21.15.49:0/2168756573 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c1e863ec60 con 0x55c1e8720810 2024-03-18T23:18:16.441 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:16.432+0000 7f97c2ffd640 1 --2- 172.21.15.49:0/2168756573 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c1e87295e0 0x55c1e8726880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:18:16.441 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:16.433+0000 7f97c37fe640 1 --2- 172.21.15.49:0/2168756573 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c1e8720810 0x55c1e8726300 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f97b4009900 tx=0x7f97b402c7e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5546543b236fe81f server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:18:16.441 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:16.433+0000 7f97a2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-03-18T23:18:16.442 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:16.433+0000 7f97a2ffd640 1 -- 172.21.15.49:0/2168756573 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97b402ccc0 con 0x55c1e8720810 2024-03-18T23:18:16.442 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:16.433+0000 7f97ccb8c740 1 -- 172.21.15.49:0/2168756573 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f97b8005500 con 0x55c1e8720810 2024-03-18T23:18:16.442 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:16.433+0000 7f97a2ffd640 1 -- 172.21.15.49:0/2168756573 <== mon.1 v2:172.21.15.175:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8556+0+0 (secure 0 0 0) 0x7f97b40046c0 con 0x55c1e8720810 2024-03-18T23:18:16.442 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:16.433+0000 7f97a2ffd640 1 -- 172.21.15.49:0/2168756573 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f97b40048c0 con 0x55c1e8720810 2024-03-18T23:18:16.442 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:16.433+0000 7f97ccb8c740 10 client.26062 fetch_fsmap learned FSMap version 32 2024-03-18T23:18:16.442 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:16.433+0000 7f97ccb8c740 1 -- 172.21.15.49:0/2168756573 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55c1e8398210 con 0x55c1e8720810 2024-03-18T23:18:16.442 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:16.433+0000 7f97a2ffd640 1 -- 172.21.15.49:0/2168756573 <== mon.1 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f97b8005500 con 0x55c1e8720810 2024-03-18T23:18:16.442 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:16.434+0000 7f97a2ffd640 1 -- 172.21.15.49:0/2168756573 <== mon.1 v2:172.21.15.175:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f97b4046070 con 0x55c1e8720810 2024-03-18T23:18:16.442 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:16.434+0000 7f97ccb8c740 10 client.26062 fetch_fsmap finished waiting for FSMap version 32 2024-03-18T23:18:16.442 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:16.434+0000 7f97ccb8c740 10 client.26062 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T23:18:16.442 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:16.434+0000 7f97ccb8c740 20 client.26062 populate_metadata read hostname 'smithi049' 2024-03-18T23:18:16.442 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:16.434+0000 7f97ccb8c740 1 --2- 172.21.15.49:0/2168756573 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x55c1e8739440 0x55c1e873b860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:18:16.443 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:16.434+0000 7f97ccb8c740 4 client.26062 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T23:18:16.443 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:16.434+0000 7f97ccb8c740 1 -- 172.21.15.49:0/2168756573 --> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c1e8691120 con 0x55c1e8739440 2024-03-18T23:18:16.443 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:16.434+0000 7f97c2ffd640 1 --2- 172.21.15.49:0/2168756573 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x55c1e8739440 0x55c1e873b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:18:16.443 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:16.434+0000 7f97a2ffd640 10 client.26062 ms_handle_connect on v2:172.21.15.139:6838/4280128209 2024-03-18T23:18:16.443 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:16.434+0000 7f97c2ffd640 1 --2- 172.21.15.49:0/2168756573 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x55c1e8739440 0x55c1e873b860 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=64deea733a6a4df4 server_cookie=df835558d0a7b32 in_seq=0 out_seq=0 2024-03-18T23:18:16.443 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:16.435+0000 7f97a2ffd640 1 -- 172.21.15.49:0/2168756573 <== mds.0 v2:172.21.15.139:6838/4280128209 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55c1e8691120 con 0x55c1e8739440 2024-03-18T23:18:16.443 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:16.435+0000 7f97a2ffd640 10 client.26062 handle_command_reply: tid=0 2024-03-18T23:18:16.443 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:16.435+0000 7f97ccb8c740 1 client.26062 shutdown 2024-03-18T23:18:16.443 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:16.435+0000 7f97ccb8c740 1 -- 172.21.15.49:0/2168756573 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c1e8720810 msgr2=0x55c1e8726300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:18:16.443 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:16.435+0000 7f97ccb8c740 1 --2- 172.21.15.49:0/2168756573 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c1e8720810 0x55c1e8726300 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f97b4009900 tx=0x7f97b402c7e0 comp rx=0 tx=0).stop 2024-03-18T23:18:16.443 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:16.435+0000 7f97ccb8c740 1 -- 172.21.15.49:0/2168756573 shutdown_connections 2024-03-18T23:18:16.443 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:16.435+0000 7f97ccb8c740 1 --2- 172.21.15.49:0/2168756573 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x55c1e8739440 0x55c1e873b860 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:16.444 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:16.435+0000 7f97ccb8c740 1 --2- 172.21.15.49:0/2168756573 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c1e87295e0 0x55c1e8726880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:16.444 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:16.435+0000 7f97ccb8c740 1 --2- 172.21.15.49:0/2168756573 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c1e8720810 0x55c1e8726300 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:16.444 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:16.435+0000 7f97a2ffd640 0 client.26062 ms_handle_reset on v2:172.21.15.139:6838/4280128209 2024-03-18T23:18:16.444 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:16.435+0000 7f97ccb8c740 1 --2- 172.21.15.49:0/2168756573 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c1e871fef0 0x55c1e8728f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:16.444 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:16.435+0000 7f97ccb8c740 1 -- 172.21.15.49:0/2168756573 >> 172.21.15.49:0/2168756573 conn(0x55c1e871a810 msgr2=0x55c1e8719ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:18:16.444 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:16.435+0000 7f97ccb8c740 1 -- 172.21.15.49:0/2168756573 shutdown_connections 2024-03-18T23:18:16.444 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:16.435+0000 7f97ccb8c740 1 -- 172.21.15.49:0/2168756573 wait complete. 2024-03-18T23:18:16.444 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:16.436+0000 7f97ccb8c740 20 client.26062 trim_cache size 0 max 16384 2024-03-18T23:18:16.457 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:16.451+0000 7f97ccb8c740 1 Processor -- start 2024-03-18T23:18:16.457 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:16.451+0000 7f97ccb8c740 1 -- start start 2024-03-18T23:18:16.457 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:16.451+0000 7f97ccb8c740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c1e871fef0 0x55c1e8734f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:18:16.457 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:16.451+0000 7f97ccb8c740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c1e8720810 0x55c1e8741150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:18:16.457 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:16.451+0000 7f97ccb8c740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c1e87295e0 0x55c1e8721040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:18:16.457 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:16.451+0000 7f97ccb8c740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f97c41ad450 con 0x55c1e8720810 2024-03-18T23:18:16.457 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:16.451+0000 7f97ccb8c740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f97c41aef90 con 0x55c1e871fef0 2024-03-18T23:18:16.457 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:16.451+0000 7f97ccb8c740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f97c41adb20 con 0x55c1e87295e0 2024-03-18T23:18:16.457 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:16.451+0000 7f97c37fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c1e8720810 0x55c1e8741150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:18:16.457 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:16.451+0000 7f97c37fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c1e8720810 0x55c1e8741150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49198/0 (socket says 172.21.15.49:49198) 2024-03-18T23:18:16.457 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:16.452+0000 7f97c37fe640 1 -- 172.21.15.49:0/3372900460 learned_addr learned my addr 172.21.15.49:0/3372900460 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:18:16.458 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:16.452+0000 7f97c2ffd640 1 --2- 172.21.15.49:0/3372900460 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c1e871fef0 0x55c1e8734f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:18:16.458 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:16.452+0000 7f97c2ffd640 1 --2- 172.21.15.49:0/3372900460 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c1e87295e0 0x55c1e8721040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:18:16.458 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:16.452+0000 7f97c37fe640 1 -- 172.21.15.49:0/3372900460 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c1e87295e0 msgr2=0x55c1e8721040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:18:16.458 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:16.452+0000 7f97c37fe640 1 --2- 172.21.15.49:0/3372900460 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c1e87295e0 0x55c1e8721040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:16.458 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:16.452+0000 7f97c37fe640 1 -- 172.21.15.49:0/3372900460 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c1e871fef0 msgr2=0x55c1e8734f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:18:16.458 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:16.452+0000 7f97c37fe640 1 --2- 172.21.15.49:0/3372900460 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c1e871fef0 0x55c1e8734f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:16.458 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:16.452+0000 7f97c37fe640 1 -- 172.21.15.49:0/3372900460 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c1e8720640 con 0x55c1e8720810 2024-03-18T23:18:16.458 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:16.452+0000 7f97c37fe640 1 --2- 172.21.15.49:0/3372900460 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c1e8720810 0x55c1e8741150 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f97b4001eb0 tx=0x7f97b402c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67329097349cc2bf server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:18:16.458 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:16.452+0000 7f97a97fa640 1 -- 172.21.15.49:0/3372900460 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97b40043e0 con 0x55c1e8720810 2024-03-18T23:18:16.458 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:16.452+0000 7f97a97fa640 1 -- 172.21.15.49:0/3372900460 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f97b4003dd0 con 0x55c1e8720810 2024-03-18T23:18:16.458 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:16.452+0000 7f97a97fa640 1 -- 172.21.15.49:0/3372900460 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97b40348a0 con 0x55c1e8720810 2024-03-18T23:18:16.458 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:16.453+0000 7f97ccb8c740 1 -- 172.21.15.49:0/3372900460 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c1e8720810 msgr2=0x55c1e8741150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:18:16.458 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:16.453+0000 7f97ccb8c740 1 --2- 172.21.15.49:0/3372900460 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c1e8720810 0x55c1e8741150 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f97b4001eb0 tx=0x7f97b402c330 comp rx=0 tx=0).stop 2024-03-18T23:18:16.459 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:16.453+0000 7f97ccb8c740 1 -- 172.21.15.49:0/3372900460 shutdown_connections 2024-03-18T23:18:16.459 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:16.453+0000 7f97ccb8c740 1 --2- 172.21.15.49:0/3372900460 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c1e87295e0 0x55c1e8721040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:16.459 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:16.453+0000 7f97ccb8c740 1 --2- 172.21.15.49:0/3372900460 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c1e8720810 0x55c1e8741150 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:16.459 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:16.453+0000 7f97ccb8c740 1 --2- 172.21.15.49:0/3372900460 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c1e871fef0 0x55c1e8734f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:16.459 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:16.453+0000 7f97ccb8c740 1 -- 172.21.15.49:0/3372900460 >> 172.21.15.49:0/3372900460 conn(0x55c1e871a810 msgr2=0x55c1e87260d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:18:16.459 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:16.453+0000 7f97ccb8c740 1 -- 172.21.15.49:0/3372900460 shutdown_connections 2024-03-18T23:18:16.459 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:16.453+0000 7f97ccb8c740 1 -- 172.21.15.49:0/3372900460 wait complete. 2024-03-18T23:18:16.459 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:16.453+0000 7f97ccb8c740 1 -- start start 2024-03-18T23:18:16.459 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:16.453+0000 7f97ccb8c740 1 Processor -- start 2024-03-18T23:18:16.459 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:16.454+0000 7f97ccb8c740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c1e871fef0 0x55c1e872d070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:18:16.459 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:16.454+0000 7f97ccb8c740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c1e8720810 0x55c1e872d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:18:16.459 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:16.454+0000 7f97ccb8c740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c1e87295e0 0x55c1e872daf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:18:16.460 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:16.454+0000 7f97ccb8c740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f97c41ae8c0 con 0x55c1e87295e0 2024-03-18T23:18:16.460 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:16.454+0000 7f97ccb8c740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f97c41ae1f0 con 0x55c1e8720810 2024-03-18T23:18:16.460 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:16.454+0000 7f97ccb8c740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f97c41ab450 con 0x55c1e871fef0 2024-03-18T23:18:16.460 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:16.454+0000 7f97ccb8c740 10 client.0 _ll_register_callbacks cb 0x7f97b4009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T23:18:16.460 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:16.454+0000 7f97c2ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c1e87295e0 0x55c1e872daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:18:16.460 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:16.454+0000 7f97c37fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c1e8720810 0x55c1e872d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:18:16.460 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:16.454+0000 7f97c2ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c1e87295e0 0x55c1e872daf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49210/0 (socket says 172.21.15.49:49210) 2024-03-18T23:18:16.460 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:16.454+0000 7f97c2ffd640 1 -- 172.21.15.49:0/545509307 learned_addr learned my addr 172.21.15.49:0/545509307 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:18:16.460 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:16.454+0000 7f97c37fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c1e8720810 0x55c1e872d5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.49:56296/0 (socket says 172.21.15.49:56296) 2024-03-18T23:18:16.460 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:16.454+0000 7f97c2ffd640 1 --2- 172.21.15.49:0/545509307 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c1e871fef0 0x55c1e872d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:18:16.460 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:16.454+0000 7f97c2ffd640 1 -- 172.21.15.49:0/545509307 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c1e871fef0 msgr2=0x55c1e872d070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:18:16.460 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:16.454+0000 7f97c2ffd640 1 --2- 172.21.15.49:0/545509307 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c1e871fef0 0x55c1e872d070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:16.461 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:16.454+0000 7f97c2ffd640 1 -- 172.21.15.49:0/545509307 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c1e8720810 msgr2=0x55c1e872d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:18:16.461 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:16.454+0000 7f97c2ffd640 1 --2- 172.21.15.49:0/545509307 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c1e8720810 0x55c1e872d5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:16.461 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:16.454+0000 7f97c2ffd640 1 -- 172.21.15.49:0/545509307 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c1e8464620 con 0x55c1e87295e0 2024-03-18T23:18:16.461 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:16.454+0000 7f97c37fe640 1 --2- 172.21.15.49:0/545509307 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c1e8720810 0x55c1e872d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:18:16.461 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:16.454+0000 7f97c2ffd640 1 --2- 172.21.15.49:0/545509307 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c1e87295e0 0x55c1e872daf0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f97b800fd60 tx=0x7f97b8015040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=111a473164073c01 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:18:16.461 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:16.454+0000 7f97a2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-18T23:18:16.461 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:16.455+0000 7f97a2ffd640 1 -- 172.21.15.49:0/545509307 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97b80051c0 con 0x55c1e87295e0 2024-03-18T23:18:16.461 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:16.455+0000 7f97a2ffd640 1 -- 172.21.15.49:0/545509307 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8556+0+0 (secure 0 0 0) 0x7f97b801c040 con 0x55c1e87295e0 2024-03-18T23:18:16.461 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:16.455+0000 7f97ccb8c740 1 -- 172.21.15.49:0/545509307 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f97b4003dd0 con 0x55c1e87295e0 2024-03-18T23:18:16.461 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:16.455+0000 7f97a2ffd640 1 -- 172.21.15.49:0/545509307 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f97b803e050 con 0x55c1e87295e0 2024-03-18T23:18:16.461 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:16.455+0000 7f97ccb8c740 10 client.16266 fetch_fsmap learned FSMap version 32 2024-03-18T23:18:16.461 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:16.455+0000 7f97ccb8c740 1 -- 172.21.15.49:0/545509307 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55c1e879faa0 con 0x55c1e87295e0 2024-03-18T23:18:16.461 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:16.455+0000 7f97a2ffd640 1 -- 172.21.15.49:0/545509307 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f97b4003dd0 con 0x55c1e87295e0 2024-03-18T23:18:16.462 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:16.455+0000 7f97a2ffd640 1 -- 172.21.15.49:0/545509307 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f97b8003790 con 0x55c1e87295e0 2024-03-18T23:18:16.462 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:16.455+0000 7f97ccb8c740 10 client.16266 fetch_fsmap finished waiting for FSMap version 32 2024-03-18T23:18:16.462 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:16.455+0000 7f97ccb8c740 10 client.16266 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T23:18:16.462 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:16.455+0000 7f97ccb8c740 20 client.16266 populate_metadata read hostname 'smithi049' 2024-03-18T23:18:16.462 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:16.455+0000 7f97ccb8c740 1 --2- 172.21.15.49:0/545509307 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x55c1e87a0080 0x55c1e87a2470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:18:16.462 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:16.455+0000 7f97ccb8c740 4 client.16266 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T23:18:16.462 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:16.455+0000 7f97ccb8c740 1 -- 172.21.15.49:0/545509307 --> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c1e868b470 con 0x55c1e87a0080 2024-03-18T23:18:16.462 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:16.455+0000 7f97c37fe640 1 --2- 172.21.15.49:0/545509307 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x55c1e87a0080 0x55c1e87a2470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:18:16.462 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:16.456+0000 7f97a2ffd640 10 client.16266 ms_handle_connect on v2:172.21.15.139:6838/4280128209 2024-03-18T23:18:16.462 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:16.456+0000 7f97c37fe640 1 --2- 172.21.15.49:0/545509307 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x55c1e87a0080 0x55c1e87a2470 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5b9eff76d8b121fa server_cookie=8643319bb1d13eb6 in_seq=0 out_seq=0 2024-03-18T23:18:16.621 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:16.620+0000 7f97a2ffd640 1 -- 172.21.15.49:0/545509307 <== mds.0 v2:172.21.15.139:6838/4280128209 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c1e868b470 con 0x55c1e87a0080 2024-03-18T23:18:16.621 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:16.620+0000 7f97a2ffd640 10 client.16266 handle_command_reply: tid=0 2024-03-18T23:18:16.621 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:16.620+0000 7f97ccb8c740 1 client.16266 shutdown 2024-03-18T23:18:16.621 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:16.620+0000 7f97ccb8c740 1 -- 172.21.15.49:0/545509307 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c1e87295e0 msgr2=0x55c1e872daf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:18:16.622 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:16.620+0000 7f97ccb8c740 1 --2- 172.21.15.49:0/545509307 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c1e87295e0 0x55c1e872daf0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f97b800fd60 tx=0x7f97b8015040 comp rx=0 tx=0).stop 2024-03-18T23:18:16.622 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:16.620+0000 7f97ccb8c740 1 -- 172.21.15.49:0/545509307 shutdown_connections 2024-03-18T23:18:16.622 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:16.620+0000 7f97ccb8c740 1 --2- 172.21.15.49:0/545509307 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x55c1e87a0080 0x55c1e87a2470 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:16.622 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:16.620+0000 7f97ccb8c740 1 --2- 172.21.15.49:0/545509307 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c1e87295e0 0x55c1e872daf0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:16.622 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:16.620+0000 7f97ccb8c740 1 --2- 172.21.15.49:0/545509307 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c1e8720810 0x55c1e872d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:16.622 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:16.620+0000 7f97a2ffd640 0 client.16266 ms_handle_reset on v2:172.21.15.139:6838/4280128209 2024-03-18T23:18:16.622 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:16.620+0000 7f97ccb8c740 1 --2- 172.21.15.49:0/545509307 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c1e871fef0 0x55c1e872d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:16.622 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:16.620+0000 7f97ccb8c740 1 -- 172.21.15.49:0/545509307 >> 172.21.15.49:0/545509307 conn(0x55c1e871a810 msgr2=0x55c1e8719220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:18:16.622 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:16.621+0000 7f97ccb8c740 1 -- 172.21.15.49:0/545509307 shutdown_connections 2024-03-18T23:18:16.622 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:16.621+0000 7f97ccb8c740 1 -- 172.21.15.49:0/545509307 wait complete. 2024-03-18T23:18:16.622 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:16.621+0000 7f97ccb8c740 20 client.16266 trim_cache size 0 max 16384 2024-03-18T23:18:16.623 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:{ 2024-03-18T23:18:16.623 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "message": "", 2024-03-18T23:18:16.623 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "return_code": 0 2024-03-18T23:18:16.623 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:} 2024-03-18T23:18:16.624 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:16.621+0000 7f97c9ec6640 1 -- 172.21.15.49:0/3055310418 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f97ac0a2a10 msgr2=0x7f97ac0a4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:18:16.624 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:16.621+0000 7f97c9ec6640 1 --2- 172.21.15.49:0/3055310418 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f97ac0a2a10 0x7f97ac0a4ed0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f97bc0ccaa0 tx=0x7f97bc086040 comp rx=0 tx=0).stop 2024-03-18T23:18:16.624 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:16.621+0000 7f97c9ec6640 1 -- 172.21.15.49:0/3055310418 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f97c41056c0 msgr2=0x7f97c419c9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:18:16.624 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:16.621+0000 7f97c9ec6640 1 --2- 172.21.15.49:0/3055310418 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f97c41056c0 0x7f97c419c9e0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f97bc00ea60 tx=0x7f97bc00ef20 comp rx=0 tx=0).stop 2024-03-18T23:18:16.624 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:16.621+0000 7f97c9ec6640 1 -- 172.21.15.49:0/3055310418 shutdown_connections 2024-03-18T23:18:16.624 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:16.621+0000 7f97c9ec6640 1 --2- 172.21.15.49:0/3055310418 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f97ac0a2a10 0x7f97ac0a4ed0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:16.625 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:16.621+0000 7f97c9ec6640 1 --2- 172.21.15.49:0/3055310418 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f97c41056c0 0x7f97c419c9e0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:16.625 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:16.622+0000 7f97c9ec6640 1 --2- 172.21.15.49:0/3055310418 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f97c4101330 0x7f97c4198660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:16.625 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:16.622+0000 7f97c9ec6640 1 --2- 172.21.15.49:0/3055310418 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f97c4100970 0x7f97c4198120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:16.625 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:16.622+0000 7f97c9ec6640 1 -- 172.21.15.49:0/3055310418 >> 172.21.15.49:0/3055310418 conn(0x7f97c4077570 msgr2=0x7f97c40fe980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:18:16.625 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:16.622+0000 7f97c9ec6640 1 -- 172.21.15.49:0/3055310418 shutdown_connections 2024-03-18T23:18:16.625 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:16.622+0000 7f97c9ec6640 1 -- 172.21.15.49:0/3055310418 wait complete. 2024-03-18T23:18:16.785 INFO:tasks.workunit.client.0.smithi049.stdout: 1 1034444 9.37 MB/sec execute 438 sec latency 81.410 ms 2024-03-18T23:18:17.462 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:18:17 smithi175 ceph-mon[67352]: pgmap v313: 97 pgs: 97 active+clean; 73 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.5 MiB/s wr, 237 op/s 2024-03-18T23:18:17.564 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:18:17 smithi139 ceph-mon[67282]: pgmap v313: 97 pgs: 97 active+clean; 73 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.5 MiB/s wr, 237 op/s 2024-03-18T23:18:17.629 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:18:17 smithi049 ceph-mon[62991]: pgmap v313: 97 pgs: 97 active+clean; 73 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.5 MiB/s wr, 237 op/s 2024-03-18T23:18:17.785 INFO:tasks.workunit.client.0.smithi049.stdout: 1 1036168 9.36 MB/sec execute 439 sec latency 72.771 ms 2024-03-18T23:18:18.785 INFO:tasks.workunit.client.0.smithi049.stdout: 1 1037544 9.37 MB/sec execute 440 sec latency 17.656 ms 2024-03-18T23:18:19.460 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:18:19 smithi175 ceph-mon[67352]: pgmap v314: 97 pgs: 97 active+clean; 79 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.8 MiB/s wr, 183 op/s 2024-03-18T23:18:19.564 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:18:19 smithi139 ceph-mon[67282]: pgmap v314: 97 pgs: 97 active+clean; 79 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.8 MiB/s wr, 183 op/s 2024-03-18T23:18:19.629 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:18:19 smithi049 ceph-mon[62991]: pgmap v314: 97 pgs: 97 active+clean; 79 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.8 MiB/s wr, 183 op/s 2024-03-18T23:18:19.785 INFO:tasks.workunit.client.0.smithi049.stdout: 1 1038989 9.36 MB/sec execute 441 sec latency 23.193 ms 2024-03-18T23:18:20.785 INFO:tasks.workunit.client.0.smithi049.stdout: 1 1040954 9.36 MB/sec execute 442 sec latency 20.724 ms 2024-03-18T23:18:21.460 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:18:21 smithi175 ceph-mon[67352]: pgmap v315: 97 pgs: 97 active+clean; 79 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.7 MiB/s wr, 176 op/s 2024-03-18T23:18:21.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:18:21 smithi139 ceph-mon[67282]: pgmap v315: 97 pgs: 97 active+clean; 79 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.7 MiB/s wr, 176 op/s 2024-03-18T23:18:21.629 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:18:21 smithi049 ceph-mon[62991]: pgmap v315: 97 pgs: 97 active+clean; 79 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.7 MiB/s wr, 176 op/s 2024-03-18T23:18:21.785 INFO:tasks.workunit.client.0.smithi049.stdout: 1 1042128 9.35 MB/sec execute 443 sec latency 19.313 ms 2024-03-18T23:18:22.785 INFO:tasks.workunit.client.0.smithi049.stdout: 1 1044040 9.36 MB/sec execute 444 sec latency 118.489 ms 2024-03-18T23:18:23.460 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:18:23 smithi175 ceph-mon[67352]: pgmap v316: 97 pgs: 97 active+clean; 75 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.2 MiB/s wr, 255 op/s 2024-03-18T23:18:23.564 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:18:23 smithi139 ceph-mon[67282]: pgmap v316: 97 pgs: 97 active+clean; 75 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.2 MiB/s wr, 255 op/s 2024-03-18T23:18:23.629 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:18:23 smithi049 ceph-mon[62991]: pgmap v316: 97 pgs: 97 active+clean; 75 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.2 MiB/s wr, 255 op/s 2024-03-18T23:18:23.785 INFO:tasks.workunit.client.0.smithi049.stdout: 1 1044942 9.35 MB/sec execute 445 sec latency 31.912 ms 2024-03-18T23:18:24.785 INFO:tasks.workunit.client.0.smithi049.stdout: 1 1046556 9.34 MB/sec execute 446 sec latency 90.991 ms 2024-03-18T23:18:25.460 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:18:25 smithi175 ceph-mon[67352]: pgmap v317: 97 pgs: 97 active+clean; 72 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.5 MiB/s wr, 226 op/s 2024-03-18T23:18:25.564 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:18:25 smithi139 ceph-mon[67282]: pgmap v317: 97 pgs: 97 active+clean; 72 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.5 MiB/s wr, 226 op/s 2024-03-18T23:18:25.629 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:18:25 smithi049 ceph-mon[62991]: pgmap v317: 97 pgs: 97 active+clean; 72 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.5 MiB/s wr, 226 op/s 2024-03-18T23:18:25.785 INFO:tasks.workunit.client.0.smithi049.stdout: 1 1048120 9.35 MB/sec execute 447 sec latency 18.128 ms 2024-03-18T23:18:26.785 INFO:tasks.workunit.client.0.smithi049.stdout: 1 1049573 9.34 MB/sec execute 448 sec latency 25.934 ms 2024-03-18T23:18:27.431 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:18:27 smithi049 ceph-mon[62991]: pgmap v318: 97 pgs: 97 active+clean; 72 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.4 MiB/s wr, 217 op/s 2024-03-18T23:18:27.460 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:18:27 smithi175 ceph-mon[67352]: pgmap v318: 97 pgs: 97 active+clean; 72 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.4 MiB/s wr, 217 op/s 2024-03-18T23:18:27.564 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:18:27 smithi139 ceph-mon[67282]: pgmap v318: 97 pgs: 97 active+clean; 72 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.4 MiB/s wr, 217 op/s 2024-03-18T23:18:27.785 INFO:tasks.workunit.client.0.smithi049.stdout: 1 1051516 9.35 MB/sec execute 449 sec latency 39.763 ms 2024-03-18T23:18:28.785 INFO:tasks.workunit.client.0.smithi049.stdout: 1 1052392 9.33 MB/sec execute 450 sec latency 24.008 ms 2024-03-18T23:18:29.460 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:18:29 smithi175 ceph-mon[67352]: pgmap v319: 97 pgs: 97 active+clean; 77 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.5 MiB/s wr, 288 op/s 2024-03-18T23:18:29.564 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:18:29 smithi139 ceph-mon[67282]: pgmap v319: 97 pgs: 97 active+clean; 77 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.5 MiB/s wr, 288 op/s 2024-03-18T23:18:29.629 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:18:29 smithi049 ceph-mon[62991]: pgmap v319: 97 pgs: 97 active+clean; 77 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.5 MiB/s wr, 288 op/s 2024-03-18T23:18:29.720 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:29.717+0000 7feac75e6640 1 -- 172.21.15.49:0/1618269279 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feac0106700 msgr2=0x7feac0106ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:18:29.720 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:29.717+0000 7feac75e6640 1 --2- 172.21.15.49:0/1618269279 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feac0106700 0x7feac0106ae0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7feabc009930 tx=0x7feabc02bc30 comp rx=0 tx=0).stop 2024-03-18T23:18:29.720 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:29.717+0000 7feac75e6640 1 -- 172.21.15.49:0/1618269279 shutdown_connections 2024-03-18T23:18:29.720 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:29.717+0000 7feac75e6640 1 --2- 172.21.15.49:0/1618269279 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feac01010b0 0x7feac010d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:29.720 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:29.717+0000 7feac75e6640 1 --2- 172.21.15.49:0/1618269279 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feac01006f0 0x7feac0100b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:29.720 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:29.717+0000 7feac75e6640 1 --2- 172.21.15.49:0/1618269279 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feac0106700 0x7feac0106ae0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:29.720 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:29.717+0000 7feac75e6640 1 -- 172.21.15.49:0/1618269279 >> 172.21.15.49:0/1618269279 conn(0x7feac0077850 msgr2=0x7feac0077c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:18:29.720 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:29.718+0000 7feac75e6640 1 -- 172.21.15.49:0/1618269279 shutdown_connections 2024-03-18T23:18:29.720 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:29.718+0000 7feac75e6640 1 -- 172.21.15.49:0/1618269279 wait complete. 2024-03-18T23:18:29.720 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:29.718+0000 7feac75e6640 1 Processor -- start 2024-03-18T23:18:29.720 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:29.718+0000 7feac75e6640 1 -- start start 2024-03-18T23:18:29.721 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:29.718+0000 7feac75e6640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feac01006f0 0x7feac019a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:18:29.721 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:29.718+0000 7feac75e6640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feac01010b0 0x7feac019a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:18:29.721 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:29.718+0000 7feac75e6640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feac0106700 0x7feac019ea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:18:29.721 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:29.718+0000 7feac75e6640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7feac0110b10 con 0x7feac0106700 2024-03-18T23:18:29.721 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:29.718+0000 7feac75e6640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7feac0110990 con 0x7feac01006f0 2024-03-18T23:18:29.721 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:29.718+0000 7feac75e6640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7feac0110c90 con 0x7feac01010b0 2024-03-18T23:18:29.721 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:29.719+0000 7feac5b83640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feac0106700 0x7feac019ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:18:29.721 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:29.719+0000 7feac4b81640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feac01010b0 0x7feac019a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:18:29.721 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:29.719+0000 7feac5382640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feac01006f0 0x7feac019a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:18:29.721 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:29.719+0000 7feac4b81640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feac01010b0 0x7feac019a6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.49:52638/0 (socket says 172.21.15.49:52638) 2024-03-18T23:18:29.721 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:29.719+0000 7feac4b81640 1 -- 172.21.15.49:0/1791303167 learned_addr learned my addr 172.21.15.49:0/1791303167 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:18:29.721 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:29.719+0000 7feac5b83640 1 -- 172.21.15.49:0/1791303167 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feac01010b0 msgr2=0x7feac019a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:18:29.722 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:29.719+0000 7feac5b83640 1 --2- 172.21.15.49:0/1791303167 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feac01010b0 0x7feac019a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:29.722 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:29.719+0000 7feac5b83640 1 -- 172.21.15.49:0/1791303167 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feac01006f0 msgr2=0x7feac019a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:18:29.722 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:29.719+0000 7feac5b83640 1 --2- 172.21.15.49:0/1791303167 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feac01006f0 0x7feac019a1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:29.722 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:29.719+0000 7feac5b83640 1 -- 172.21.15.49:0/1791303167 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feac019f1a0 con 0x7feac0106700 2024-03-18T23:18:29.722 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:29.719+0000 7feac4b81640 1 --2- 172.21.15.49:0/1791303167 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feac01010b0 0x7feac019a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:18:29.722 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:29.719+0000 7feac5b83640 1 --2- 172.21.15.49:0/1791303167 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feac0106700 0x7feac019ea70 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7feab8009fd0 tx=0x7feab800ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:18:29.722 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:29.719+0000 7feab67fc640 1 -- 172.21.15.49:0/1791303167 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feab800ce20 con 0x7feac0106700 2024-03-18T23:18:29.722 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:29.719+0000 7feac75e6640 1 -- 172.21.15.49:0/1791303167 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feac019f490 con 0x7feac0106700 2024-03-18T23:18:29.722 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:29.719+0000 7feab67fc640 1 -- 172.21.15.49:0/1791303167 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feab8005020 con 0x7feac0106700 2024-03-18T23:18:29.722 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:29.720+0000 7feab67fc640 1 -- 172.21.15.49:0/1791303167 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feab8010720 con 0x7feac0106700 2024-03-18T23:18:29.722 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:29.720+0000 7feac75e6640 1 -- 172.21.15.49:0/1791303167 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feac01a6d20 con 0x7feac0106700 2024-03-18T23:18:29.728 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:29.722+0000 7feab67fc640 1 -- 172.21.15.49:0/1791303167 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 25) ==== 137421+0+0 (secure 0 0 0) 0x7feab80051d0 con 0x7feac0106700 2024-03-18T23:18:29.728 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:29.722+0000 7feab67fc640 1 --2- 172.21.15.49:0/1791303167 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7feaac0a2900 0x7feaac0a4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:18:29.728 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:29.723+0000 7feac5382640 1 --2- 172.21.15.49:0/1791303167 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7feaac0a2900 0x7feaac0a4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:18:29.728 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:29.723+0000 7feab67fc640 1 -- 172.21.15.49:0/1791303167 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8556+0+0 (secure 0 0 0) 0x7feab80cf3c0 con 0x7feac0106700 2024-03-18T23:18:29.728 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:29.723+0000 7feaca1d3740 1 Processor -- start 2024-03-18T23:18:29.728 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:29.723+0000 7feaca1d3740 1 -- start start 2024-03-18T23:18:29.728 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:29.723+0000 7feac5382640 1 --2- 172.21.15.49:0/1791303167 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7feaac0a2900 0x7feaac0a4dc0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7feabc005ab0 tx=0x7feabc005fd0 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:18:29.729 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:29.723+0000 7feaca1d3740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559737148ef0 0x5597371492d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:18:29.729 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:29.723+0000 7feaca1d3740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559737149810 0x5597371520a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:18:29.729 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:29.723+0000 7feaca1d3740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5597371525e0 0x5597371549d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:18:29.729 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:29.723+0000 7feaca1d3740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5597370acbe0 con 0x559737148ef0 2024-03-18T23:18:29.729 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:29.723+0000 7feaca1d3740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x559737055960 con 0x559737149810 2024-03-18T23:18:29.729 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:29.723+0000 7feaca1d3740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5597370a8c80 con 0x5597371525e0 2024-03-18T23:18:29.729 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:29.723+0000 7feac5382640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559737148ef0 0x5597371492d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:18:29.729 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:29.723+0000 7feac5382640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559737148ef0 0x5597371492d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:44884/0 (socket says 172.21.15.49:44884) 2024-03-18T23:18:29.729 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:29.723+0000 7feac5382640 1 -- 172.21.15.49:0/3465080617 learned_addr learned my addr 172.21.15.49:0/3465080617 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:18:29.729 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:29.723+0000 7feac4b81640 1 --2- 172.21.15.49:0/3465080617 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559737149810 0x5597371520a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:18:29.729 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:29.724+0000 7feac5b83640 1 --2- 172.21.15.49:0/3465080617 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5597371525e0 0x5597371549d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:18:29.729 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:29.724+0000 7feac5382640 1 -- 172.21.15.49:0/3465080617 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5597371525e0 msgr2=0x5597371549d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:18:29.729 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:29.724+0000 7feac5382640 1 --2- 172.21.15.49:0/3465080617 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5597371525e0 0x5597371549d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:29.730 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:29.724+0000 7feac5382640 1 -- 172.21.15.49:0/3465080617 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559737149810 msgr2=0x5597371520a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:18:29.730 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:29.724+0000 7feac5382640 1 --2- 172.21.15.49:0/3465080617 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559737149810 0x5597371520a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:29.730 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:29.724+0000 7feac5382640 1 -- 172.21.15.49:0/3465080617 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55973704cd20 con 0x559737148ef0 2024-03-18T23:18:29.730 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:29.724+0000 7feac5382640 1 --2- 172.21.15.49:0/3465080617 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559737148ef0 0x5597371492d0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7feabc035000 tx=0x7feabc03f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32b982339740b4b7 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:18:29.730 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:29.724+0000 7feaab7fe640 1 -- 172.21.15.49:0/3465080617 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feabc004180 con 0x559737148ef0 2024-03-18T23:18:29.730 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:29.724+0000 7feaab7fe640 1 -- 172.21.15.49:0/3465080617 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feabc004ba0 con 0x559737148ef0 2024-03-18T23:18:29.730 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:29.724+0000 7feaab7fe640 1 -- 172.21.15.49:0/3465080617 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feabc0035f0 con 0x559737148ef0 2024-03-18T23:18:29.730 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:29.725+0000 7feaca1d3740 1 -- 172.21.15.49:0/3465080617 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559737148ef0 msgr2=0x5597371492d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:18:29.730 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:29.725+0000 7feaca1d3740 1 --2- 172.21.15.49:0/3465080617 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559737148ef0 0x5597371492d0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7feabc035000 tx=0x7feabc03f000 comp rx=0 tx=0).stop 2024-03-18T23:18:29.730 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:29.725+0000 7feaca1d3740 1 -- 172.21.15.49:0/3465080617 shutdown_connections 2024-03-18T23:18:29.730 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:29.725+0000 7feaca1d3740 1 --2- 172.21.15.49:0/3465080617 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5597371525e0 0x5597371549d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:29.730 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:29.725+0000 7feaca1d3740 1 --2- 172.21.15.49:0/3465080617 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559737149810 0x5597371520a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:29.730 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:29.725+0000 7feaca1d3740 1 --2- 172.21.15.49:0/3465080617 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559737148ef0 0x5597371492d0 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:29.731 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:29.725+0000 7feaca1d3740 1 -- 172.21.15.49:0/3465080617 >> 172.21.15.49:0/3465080617 conn(0x559737143810 msgr2=0x559737143bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:18:29.731 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:29.725+0000 7feaca1d3740 1 -- 172.21.15.49:0/3465080617 shutdown_connections 2024-03-18T23:18:29.731 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:29.725+0000 7feaca1d3740 1 -- 172.21.15.49:0/3465080617 wait complete. 2024-03-18T23:18:29.731 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:29.725+0000 7feaca1d3740 1 -- start start 2024-03-18T23:18:29.731 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:29.725+0000 7feaca1d3740 1 Processor -- start 2024-03-18T23:18:29.731 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:29.726+0000 7feaca1d3740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559737148ef0 0x559737151f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:18:29.731 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:29.726+0000 7feaca1d3740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559737149810 0x55973714f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:18:29.731 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:29.726+0000 7feaca1d3740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5597371525e0 0x55973714f870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:18:29.731 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:29.726+0000 7feaca1d3740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5597371551b0 con 0x5597371525e0 2024-03-18T23:18:29.731 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:29.726+0000 7feaca1d3740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x559737068db0 con 0x559737149810 2024-03-18T23:18:29.731 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:29.726+0000 7feaca1d3740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x559737155330 con 0x559737148ef0 2024-03-18T23:18:29.731 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:29.726+0000 7feac5b83640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5597371525e0 0x55973714f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:18:29.731 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:29.726+0000 7feaca1d3740 10 client.0 _ll_register_callbacks cb 0x55973706de70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T23:18:29.732 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:29.726+0000 7feac5382640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559737148ef0 0x559737151f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:18:29.732 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:29.726+0000 7feac5b83640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5597371525e0 0x55973714f870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:44894/0 (socket says 172.21.15.49:44894) 2024-03-18T23:18:29.732 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:29.726+0000 7feac5b83640 1 -- 172.21.15.49:0/2185340020 learned_addr learned my addr 172.21.15.49:0/2185340020 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:18:29.732 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:29.726+0000 7feac5382640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559737148ef0 0x559737151f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.49:52662/0 (socket says 172.21.15.49:52662) 2024-03-18T23:18:29.732 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:29.726+0000 7feac4b81640 1 --2- 172.21.15.49:0/2185340020 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559737149810 0x55973714f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:18:29.732 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:29.726+0000 7feac4b81640 1 -- 172.21.15.49:0/2185340020 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559737148ef0 msgr2=0x559737151f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:18:29.732 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:29.726+0000 7feac4b81640 1 --2- 172.21.15.49:0/2185340020 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559737148ef0 0x559737151f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:29.732 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:29.726+0000 7feac4b81640 1 -- 172.21.15.49:0/2185340020 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5597371525e0 msgr2=0x55973714f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:18:29.732 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:29.726+0000 7feac4b81640 1 --2- 172.21.15.49:0/2185340020 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5597371525e0 0x55973714f870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:29.732 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:29.726+0000 7feac4b81640 1 -- 172.21.15.49:0/2185340020 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559737067c60 con 0x559737149810 2024-03-18T23:18:29.732 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:29.726+0000 7feac5b83640 1 --2- 172.21.15.49:0/2185340020 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5597371525e0 0x55973714f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:18:29.732 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:29.727+0000 7feac4b81640 1 --2- 172.21.15.49:0/2185340020 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559737149810 0x55973714f2f0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x5597371489a0 tx=0x7feab000f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8c6efbdb62f87b8b server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:18:29.732 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:29.727+0000 7feaa8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-03-18T23:18:29.732 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:29.727+0000 7feaa8ff9640 1 -- 172.21.15.49:0/2185340020 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feab0004050 con 0x559737149810 2024-03-18T23:18:29.733 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:29.727+0000 7feaa8ff9640 1 -- 172.21.15.49:0/2185340020 <== mon.2 v2:172.21.15.139:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8556+0+0 (secure 0 0 0) 0x7feab00041f0 con 0x559737149810 2024-03-18T23:18:29.733 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:29.727+0000 7feaca1d3740 1 -- 172.21.15.49:0/2185340020 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feabc004ba0 con 0x559737149810 2024-03-18T23:18:29.733 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:29.727+0000 7feaa8ff9640 1 -- 172.21.15.49:0/2185340020 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7feab00043f0 con 0x559737149810 2024-03-18T23:18:29.733 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:29.727+0000 7feaca1d3740 10 client.26093 fetch_fsmap learned FSMap version 32 2024-03-18T23:18:29.733 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:29.727+0000 7feaca1d3740 1 -- 172.21.15.49:0/2185340020 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x559736dc1210 con 0x559737149810 2024-03-18T23:18:29.733 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:29.727+0000 7feaa8ff9640 1 -- 172.21.15.49:0/2185340020 <== mon.2 v2:172.21.15.139:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feabc004ba0 con 0x559737149810 2024-03-18T23:18:29.733 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:29.728+0000 7feaa8ff9640 1 -- 172.21.15.49:0/2185340020 <== mon.2 v2:172.21.15.139:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7feab00104e0 con 0x559737149810 2024-03-18T23:18:29.733 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:29.728+0000 7feaca1d3740 10 client.26093 fetch_fsmap finished waiting for FSMap version 32 2024-03-18T23:18:29.733 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:29.728+0000 7feaca1d3740 10 client.26093 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T23:18:29.733 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:29.728+0000 7feaca1d3740 20 client.26093 populate_metadata read hostname 'smithi049' 2024-03-18T23:18:29.733 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:29.728+0000 7feaca1d3740 1 --2- 172.21.15.49:0/2185340020 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x559737162450 0x559737164840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:18:29.733 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:29.728+0000 7feaca1d3740 4 client.26093 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T23:18:29.733 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:29.728+0000 7feaca1d3740 1 -- 172.21.15.49:0/2185340020 --> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5597370ba120 con 0x559737162450 2024-03-18T23:18:29.734 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:29.728+0000 7feac5382640 1 --2- 172.21.15.49:0/2185340020 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x559737162450 0x559737164840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:18:29.734 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:29.728+0000 7feaa8ff9640 10 client.26093 ms_handle_connect on v2:172.21.15.139:6838/4280128209 2024-03-18T23:18:29.734 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:29.728+0000 7feac5382640 1 --2- 172.21.15.49:0/2185340020 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x559737162450 0x559737164840 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7024b9e1b5bddaf5 server_cookie=b267f25843121b46 in_seq=0 out_seq=0 2024-03-18T23:18:29.735 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:29.730+0000 7feaa8ff9640 1 -- 172.21.15.49:0/2185340020 <== mds.0 v2:172.21.15.139:6838/4280128209 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5597370ba120 con 0x559737162450 2024-03-18T23:18:29.735 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:29.730+0000 7feaa8ff9640 10 client.26093 handle_command_reply: tid=0 2024-03-18T23:18:29.735 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:29.730+0000 7feaca1d3740 1 client.26093 shutdown 2024-03-18T23:18:29.735 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:29.730+0000 7feaca1d3740 1 -- 172.21.15.49:0/2185340020 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559737149810 msgr2=0x55973714f2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:18:29.735 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:29.730+0000 7feaca1d3740 1 --2- 172.21.15.49:0/2185340020 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559737149810 0x55973714f2f0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x5597371489a0 tx=0x7feab000f790 comp rx=0 tx=0).stop 2024-03-18T23:18:29.735 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:29.730+0000 7feaca1d3740 1 -- 172.21.15.49:0/2185340020 shutdown_connections 2024-03-18T23:18:29.735 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:29.730+0000 7feaca1d3740 1 --2- 172.21.15.49:0/2185340020 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x559737162450 0x559737164840 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:29.735 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:29.730+0000 7feaca1d3740 1 --2- 172.21.15.49:0/2185340020 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5597371525e0 0x55973714f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:29.736 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:29.730+0000 7feaca1d3740 1 --2- 172.21.15.49:0/2185340020 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559737149810 0x55973714f2f0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:29.736 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:29.730+0000 7feaca1d3740 1 --2- 172.21.15.49:0/2185340020 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559737148ef0 0x559737151f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:29.736 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:29.730+0000 7feaa8ff9640 0 client.26093 ms_handle_reset on v2:172.21.15.139:6838/4280128209 2024-03-18T23:18:29.736 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:29.730+0000 7feaca1d3740 1 -- 172.21.15.49:0/2185340020 >> 172.21.15.49:0/2185340020 conn(0x559737143810 msgr2=0x559737142ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:18:29.736 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:29.730+0000 7feaca1d3740 1 -- 172.21.15.49:0/2185340020 shutdown_connections 2024-03-18T23:18:29.736 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:29.730+0000 7feaca1d3740 1 -- 172.21.15.49:0/2185340020 wait complete. 2024-03-18T23:18:29.736 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:29.730+0000 7feaca1d3740 20 client.26093 trim_cache size 0 max 16384 2024-03-18T23:18:29.747 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:29.746+0000 7feaca1d3740 1 Processor -- start 2024-03-18T23:18:29.747 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:29.746+0000 7feaca1d3740 1 -- start start 2024-03-18T23:18:29.747 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:29.746+0000 7feaca1d3740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559737148ef0 0x55973714f290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:18:29.747 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:29.746+0000 7feaca1d3740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559737149810 0x55973715df50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:18:29.747 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:29.746+0000 7feaca1d3740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5597371525e0 0x55973716a130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:18:29.748 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:29.746+0000 7feaca1d3740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7feac01af410 con 0x5597371525e0 2024-03-18T23:18:29.748 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:29.746+0000 7feaca1d3740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7feac01b0f50 con 0x559737149810 2024-03-18T23:18:29.748 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:29.746+0000 7feaca1d3740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7feac01afae0 con 0x559737148ef0 2024-03-18T23:18:29.748 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:29.746+0000 7feac5b83640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5597371525e0 0x55973716a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:18:29.748 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:29.746+0000 7feac4b81640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559737149810 0x55973715df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:18:29.748 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:29.746+0000 7feac5382640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559737148ef0 0x55973714f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:18:29.748 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:29.746+0000 7feac5b83640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5597371525e0 0x55973716a130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:44902/0 (socket says 172.21.15.49:44902) 2024-03-18T23:18:29.748 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:29.746+0000 7feac5b83640 1 -- 172.21.15.49:0/1253530282 learned_addr learned my addr 172.21.15.49:0/1253530282 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:18:29.748 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:29.747+0000 7feac5382640 1 -- 172.21.15.49:0/1253530282 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559737149810 msgr2=0x55973715df50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:18:29.748 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:29.747+0000 7feac5382640 1 --2- 172.21.15.49:0/1253530282 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559737149810 0x55973715df50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:29.748 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:29.747+0000 7feac5382640 1 -- 172.21.15.49:0/1253530282 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5597371525e0 msgr2=0x55973716a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:18:29.748 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:29.747+0000 7feac5382640 1 --2- 172.21.15.49:0/1253530282 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5597371525e0 0x55973716a130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:29.748 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:29.747+0000 7feac5382640 1 -- 172.21.15.49:0/1253530282 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55973714fbe0 con 0x559737148ef0 2024-03-18T23:18:29.748 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:29.747+0000 7feac5382640 1 --2- 172.21.15.49:0/1253530282 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559737148ef0 0x55973714f290 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7feabc02cfd0 tx=0x7feabc042000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=46e36c2e5e1a1e9b server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:18:29.748 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:29.747+0000 7feaaaffd640 1 -- 172.21.15.49:0/1253530282 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feabc005530 con 0x559737148ef0 2024-03-18T23:18:29.749 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:29.747+0000 7feaaaffd640 1 -- 172.21.15.49:0/1253530282 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feabc05aaa0 con 0x559737148ef0 2024-03-18T23:18:29.749 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:29.747+0000 7feaaaffd640 1 -- 172.21.15.49:0/1253530282 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feabc05a810 con 0x559737148ef0 2024-03-18T23:18:29.755 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:29.748+0000 7feaca1d3740 1 -- 172.21.15.49:0/1253530282 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559737148ef0 msgr2=0x55973714f290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:18:29.755 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:29.748+0000 7feaca1d3740 1 --2- 172.21.15.49:0/1253530282 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559737148ef0 0x55973714f290 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7feabc02cfd0 tx=0x7feabc042000 comp rx=0 tx=0).stop 2024-03-18T23:18:29.755 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:29.748+0000 7feaca1d3740 1 -- 172.21.15.49:0/1253530282 shutdown_connections 2024-03-18T23:18:29.755 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:29.748+0000 7feaca1d3740 1 --2- 172.21.15.49:0/1253530282 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5597371525e0 0x55973716a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:29.755 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:29.748+0000 7feaca1d3740 1 --2- 172.21.15.49:0/1253530282 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559737149810 0x55973715df50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:29.755 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:29.748+0000 7feaca1d3740 1 --2- 172.21.15.49:0/1253530282 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559737148ef0 0x55973714f290 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:29.755 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:29.748+0000 7feaca1d3740 1 -- 172.21.15.49:0/1253530282 >> 172.21.15.49:0/1253530282 conn(0x559737143810 msgr2=0x55973714eb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:18:29.755 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:29.748+0000 7feaca1d3740 1 -- 172.21.15.49:0/1253530282 shutdown_connections 2024-03-18T23:18:29.755 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:29.748+0000 7feaca1d3740 1 -- 172.21.15.49:0/1253530282 wait complete. 2024-03-18T23:18:29.755 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:29.748+0000 7feaca1d3740 1 -- start start 2024-03-18T23:18:29.755 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:29.748+0000 7feaca1d3740 1 Processor -- start 2024-03-18T23:18:29.755 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:29.749+0000 7feaca1d3740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559737148ef0 0x55973714c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:18:29.755 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:29.749+0000 7feaca1d3740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559737149810 0x55973714c540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:18:29.755 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:29.749+0000 7feaca1d3740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5597371525e0 0x55973714ca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:18:29.756 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:29.749+0000 7feaca1d3740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7feac01b0880 con 0x559737148ef0 2024-03-18T23:18:29.756 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:29.749+0000 7feaca1d3740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7feac01b01b0 con 0x559737149810 2024-03-18T23:18:29.756 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:29.749+0000 7feaca1d3740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7feac01ad410 con 0x5597371525e0 2024-03-18T23:18:29.756 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:29.749+0000 7feaca1d3740 10 client.0 _ll_register_callbacks cb 0x7feab0009cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T23:18:29.756 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:29.749+0000 7feac5382640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559737148ef0 0x55973714c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:18:29.756 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:29.749+0000 7feac5382640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559737148ef0 0x55973714c000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:44912/0 (socket says 172.21.15.49:44912) 2024-03-18T23:18:29.756 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:29.749+0000 7feac5382640 1 -- 172.21.15.49:0/2888418974 learned_addr learned my addr 172.21.15.49:0/2888418974 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:18:29.756 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:29.749+0000 7feac4b81640 1 --2- 172.21.15.49:0/2888418974 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559737149810 0x55973714c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:18:29.756 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:29.749+0000 7feac5b83640 1 --2- 172.21.15.49:0/2888418974 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5597371525e0 0x55973714ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:18:29.756 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:29.749+0000 7feac4b81640 1 -- 172.21.15.49:0/2888418974 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5597371525e0 msgr2=0x55973714ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:18:29.756 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:29.749+0000 7feac4b81640 1 --2- 172.21.15.49:0/2888418974 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5597371525e0 0x55973714ca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:29.756 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:29.749+0000 7feac4b81640 1 -- 172.21.15.49:0/2888418974 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559737148ef0 msgr2=0x55973714c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:18:29.756 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:29.749+0000 7feac4b81640 1 --2- 172.21.15.49:0/2888418974 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559737148ef0 0x55973714c000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:29.756 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:29.749+0000 7feac4b81640 1 -- 172.21.15.49:0/2888418974 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559737164bb0 con 0x559737149810 2024-03-18T23:18:29.756 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:29.749+0000 7feac4b81640 1 --2- 172.21.15.49:0/2888418974 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559737149810 0x55973714c540 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7feab0010950 tx=0x7feab00123a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7f5c3c20036fcdc8 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:18:29.757 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:29.749+0000 7feaa8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-03-18T23:18:29.757 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:29.750+0000 7feaa8ff9640 1 -- 172.21.15.49:0/2888418974 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feab0029030 con 0x559737149810 2024-03-18T23:18:29.757 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:29.750+0000 7feaa8ff9640 1 -- 172.21.15.49:0/2888418974 <== mon.2 v2:172.21.15.139:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8556+0+0 (secure 0 0 0) 0x7feab0018040 con 0x559737149810 2024-03-18T23:18:29.757 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:29.750+0000 7feaca1d3740 1 -- 172.21.15.49:0/2888418974 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feabc05aaa0 con 0x559737149810 2024-03-18T23:18:29.757 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:29.750+0000 7feaa8ff9640 1 -- 172.21.15.49:0/2888418974 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7feab003b050 con 0x559737149810 2024-03-18T23:18:29.757 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:29.750+0000 7feaca1d3740 10 client.26099 fetch_fsmap learned FSMap version 32 2024-03-18T23:18:29.757 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:29.750+0000 7feaca1d3740 1 -- 172.21.15.49:0/2888418974 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x559736e8d620 con 0x559737149810 2024-03-18T23:18:29.757 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:29.750+0000 7feaa8ff9640 1 -- 172.21.15.49:0/2888418974 <== mon.2 v2:172.21.15.139:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feabc05aaa0 con 0x559737149810 2024-03-18T23:18:29.757 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:29.750+0000 7feaa8ff9640 1 -- 172.21.15.49:0/2888418974 <== mon.2 v2:172.21.15.139:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7feab0004620 con 0x559737149810 2024-03-18T23:18:29.757 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:29.750+0000 7feaca1d3740 10 client.26099 fetch_fsmap finished waiting for FSMap version 32 2024-03-18T23:18:29.757 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:29.750+0000 7feaca1d3740 10 client.26099 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T23:18:29.757 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:29.750+0000 7feaca1d3740 20 client.26099 populate_metadata read hostname 'smithi049' 2024-03-18T23:18:29.757 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:29.750+0000 7feaca1d3740 1 --2- 172.21.15.49:0/2888418974 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x5597371c8ef0 0x5597371cb310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:18:29.757 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:29.750+0000 7feaca1d3740 4 client.26099 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T23:18:29.757 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:29.750+0000 7feaca1d3740 1 -- 172.21.15.49:0/2888418974 --> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5597370b4470 con 0x5597371c8ef0 2024-03-18T23:18:29.757 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:29.750+0000 7feac5382640 1 --2- 172.21.15.49:0/2888418974 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x5597371c8ef0 0x5597371cb310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:18:29.758 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:29.751+0000 7feaa8ff9640 10 client.26099 ms_handle_connect on v2:172.21.15.139:6838/4280128209 2024-03-18T23:18:29.758 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:29.751+0000 7feac5382640 1 --2- 172.21.15.49:0/2888418974 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x5597371c8ef0 0x5597371cb310 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=411a8ec768be63e6 server_cookie=3bca36c9c27df505 in_seq=0 out_seq=0 2024-03-18T23:18:29.786 INFO:tasks.workunit.client.0.smithi049.stdout: 1 1054517 9.34 MB/sec execute 451 sec latency 25.425 ms 2024-03-18T23:18:29.924 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:29.922+0000 7feaa8ff9640 1 -- 172.21.15.49:0/2888418974 <== mds.0 v2:172.21.15.139:6838/4280128209 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5597370b4470 con 0x5597371c8ef0 2024-03-18T23:18:29.924 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:29.922+0000 7feaa8ff9640 10 client.26099 handle_command_reply: tid=0 2024-03-18T23:18:29.925 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:29.922+0000 7feaca1d3740 1 client.26099 shutdown 2024-03-18T23:18:29.925 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:29.922+0000 7feaca1d3740 1 -- 172.21.15.49:0/2888418974 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559737149810 msgr2=0x55973714c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:18:29.925 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:29.922+0000 7feaca1d3740 1 --2- 172.21.15.49:0/2888418974 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559737149810 0x55973714c540 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7feab0010950 tx=0x7feab00123a0 comp rx=0 tx=0).stop 2024-03-18T23:18:29.925 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:29.922+0000 7feaca1d3740 1 -- 172.21.15.49:0/2888418974 shutdown_connections 2024-03-18T23:18:29.925 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:29.922+0000 7feaca1d3740 1 --2- 172.21.15.49:0/2888418974 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x5597371c8ef0 0x5597371cb310 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:29.925 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:29.922+0000 7feaa8ff9640 0 client.26099 ms_handle_reset on v2:172.21.15.139:6838/4280128209 2024-03-18T23:18:29.925 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:29.922+0000 7feaca1d3740 1 --2- 172.21.15.49:0/2888418974 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5597371525e0 0x55973714ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:29.925 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:29.922+0000 7feaca1d3740 1 --2- 172.21.15.49:0/2888418974 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559737149810 0x55973714c540 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:29.925 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:29.922+0000 7feaca1d3740 1 --2- 172.21.15.49:0/2888418974 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559737148ef0 0x55973714c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:29.925 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:29.922+0000 7feaca1d3740 1 -- 172.21.15.49:0/2888418974 >> 172.21.15.49:0/2888418974 conn(0x559737143810 msgr2=0x559737149fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:18:29.925 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:29.922+0000 7feaca1d3740 1 -- 172.21.15.49:0/2888418974 shutdown_connections 2024-03-18T23:18:29.925 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:29.922+0000 7feaca1d3740 1 -- 172.21.15.49:0/2888418974 wait complete. 2024-03-18T23:18:29.925 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:29.923+0000 7feaca1d3740 20 client.26099 trim_cache size 0 max 16384 2024-03-18T23:18:29.926 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:{ 2024-03-18T23:18:29.926 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "message": "", 2024-03-18T23:18:29.926 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "return_code": 0 2024-03-18T23:18:29.926 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:} 2024-03-18T23:18:29.927 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:29.923+0000 7feac75e6640 1 -- 172.21.15.49:0/1791303167 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7feaac0a2900 msgr2=0x7feaac0a4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:18:29.927 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:29.923+0000 7feac75e6640 1 --2- 172.21.15.49:0/1791303167 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7feaac0a2900 0x7feaac0a4dc0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7feabc005ab0 tx=0x7feabc005fd0 comp rx=0 tx=0).stop 2024-03-18T23:18:29.927 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:29.923+0000 7feac75e6640 1 -- 172.21.15.49:0/1791303167 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feac0106700 msgr2=0x7feac019ea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:18:29.927 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:29.923+0000 7feac75e6640 1 --2- 172.21.15.49:0/1791303167 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feac0106700 0x7feac019ea70 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7feab8009fd0 tx=0x7feab800ef10 comp rx=0 tx=0).stop 2024-03-18T23:18:29.927 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:29.923+0000 7feac75e6640 1 -- 172.21.15.49:0/1791303167 shutdown_connections 2024-03-18T23:18:29.927 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:29.923+0000 7feac75e6640 1 --2- 172.21.15.49:0/1791303167 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7feaac0a2900 0x7feaac0a4dc0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:29.927 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:29.924+0000 7feac75e6640 1 --2- 172.21.15.49:0/1791303167 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feac0106700 0x7feac019ea70 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:29.927 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:29.924+0000 7feac75e6640 1 --2- 172.21.15.49:0/1791303167 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feac01010b0 0x7feac019a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:29.927 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:29.924+0000 7feac75e6640 1 --2- 172.21.15.49:0/1791303167 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feac01006f0 0x7feac019a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:29.927 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:29.924+0000 7feac75e6640 1 -- 172.21.15.49:0/1791303167 >> 172.21.15.49:0/1791303167 conn(0x7feac0077850 msgr2=0x7feac010c3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:18:29.927 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:29.924+0000 7feac75e6640 1 -- 172.21.15.49:0/1791303167 shutdown_connections 2024-03-18T23:18:29.928 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:29.924+0000 7feac75e6640 1 -- 172.21.15.49:0/1791303167 wait complete. 2024-03-18T23:18:30.785 INFO:tasks.workunit.client.0.smithi049.stdout: 1 1055441 9.33 MB/sec execute 452 sec latency 73.355 ms 2024-03-18T23:18:31.431 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:18:31 smithi049 ceph-mon[62991]: pgmap v320: 97 pgs: 97 active+clean; 77 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.2 MiB/s wr, 208 op/s 2024-03-18T23:18:31.431 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:18:31 smithi049 ceph-mon[62991]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:18:31.460 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:18:31 smithi175 ceph-mon[67352]: pgmap v320: 97 pgs: 97 active+clean; 77 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.2 MiB/s wr, 208 op/s 2024-03-18T23:18:31.460 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:18:31 smithi175 ceph-mon[67352]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:18:31.564 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:18:31 smithi139 ceph-mon[67282]: pgmap v320: 97 pgs: 97 active+clean; 77 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.2 MiB/s wr, 208 op/s 2024-03-18T23:18:31.564 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:18:31 smithi139 ceph-mon[67282]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:18:31.786 INFO:tasks.workunit.client.0.smithi049.stdout: 1 1056881 9.32 MB/sec execute 453 sec latency 32.210 ms 2024-03-18T23:18:32.786 INFO:tasks.workunit.client.0.smithi049.stdout: 1 1058671 9.33 MB/sec execute 454 sec latency 97.794 ms 2024-03-18T23:18:33.460 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:18:33 smithi175 ceph-mon[67352]: pgmap v321: 97 pgs: 97 active+clean; 70 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.2 MiB/s wr, 333 op/s 2024-03-18T23:18:33.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:18:33 smithi175 ceph-mon[67352]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' 2024-03-18T23:18:33.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:18:33 smithi175 ceph-mon[67352]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' 2024-03-18T23:18:33.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:18:33 smithi175 ceph-mon[67352]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' 2024-03-18T23:18:33.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:18:33 smithi175 ceph-mon[67352]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' 2024-03-18T23:18:33.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:18:33 smithi175 ceph-mon[67352]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' 2024-03-18T23:18:33.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:18:33 smithi175 ceph-mon[67352]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' 2024-03-18T23:18:33.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:18:33 smithi175 ceph-mon[67352]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:18:33.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:18:33 smithi175 ceph-mon[67352]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:18:33.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:18:33 smithi175 ceph-mon[67352]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:18:33.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:18:33 smithi175 ceph-mon[67352]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:18:33.564 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:18:33 smithi139 ceph-mon[67282]: pgmap v321: 97 pgs: 97 active+clean; 70 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.2 MiB/s wr, 333 op/s 2024-03-18T23:18:33.564 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:18:33 smithi139 ceph-mon[67282]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' 2024-03-18T23:18:33.564 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:18:33 smithi139 ceph-mon[67282]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' 2024-03-18T23:18:33.564 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:18:33 smithi139 ceph-mon[67282]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' 2024-03-18T23:18:33.564 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:18:33 smithi139 ceph-mon[67282]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' 2024-03-18T23:18:33.564 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:18:33 smithi139 ceph-mon[67282]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' 2024-03-18T23:18:33.564 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:18:33 smithi139 ceph-mon[67282]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' 2024-03-18T23:18:33.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:18:33 smithi139 ceph-mon[67282]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:18:33.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:18:33 smithi139 ceph-mon[67282]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:18:33.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:18:33 smithi139 ceph-mon[67282]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:18:33.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:18:33 smithi139 ceph-mon[67282]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:18:33.629 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:18:33 smithi049 ceph-mon[62991]: pgmap v321: 97 pgs: 97 active+clean; 70 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.2 MiB/s wr, 333 op/s 2024-03-18T23:18:33.629 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:18:33 smithi049 ceph-mon[62991]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' 2024-03-18T23:18:33.629 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:18:33 smithi049 ceph-mon[62991]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' 2024-03-18T23:18:33.629 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:18:33 smithi049 ceph-mon[62991]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' 2024-03-18T23:18:33.629 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:18:33 smithi049 ceph-mon[62991]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' 2024-03-18T23:18:33.629 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:18:33 smithi049 ceph-mon[62991]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' 2024-03-18T23:18:33.629 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:18:33 smithi049 ceph-mon[62991]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' 2024-03-18T23:18:33.629 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:18:33 smithi049 ceph-mon[62991]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:18:33.629 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:18:33 smithi049 ceph-mon[62991]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:18:33.629 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:18:33 smithi049 ceph-mon[62991]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:18:33.629 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:18:33 smithi049 ceph-mon[62991]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:18:33.786 INFO:tasks.workunit.client.0.smithi049.stdout: 1 1059473 9.31 MB/sec execute 455 sec latency 26.921 ms 2024-03-18T23:18:34.460 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:18:34 smithi175 ceph-mon[67352]: Adjusting osd_memory_target on smithi049 to 176.2M 2024-03-18T23:18:34.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:18:34 smithi175 ceph-mon[67352]: Unable to set osd_memory_target on smithi049 to 184762368: error parsing value: Value '184762368' is below minimum 939524096 2024-03-18T23:18:34.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:18:34 smithi175 ceph-mon[67352]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:18:34.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:18:34 smithi175 ceph-mon[67352]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:18:34.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:18:34 smithi175 ceph-mon[67352]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:18:34.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:18:34 smithi175 ceph-mon[67352]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:18:34.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:18:34 smithi175 ceph-mon[67352]: Adjusting osd_memory_target on smithi175 to 174.1M 2024-03-18T23:18:34.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:18:34 smithi175 ceph-mon[67352]: Unable to set osd_memory_target on smithi175 to 182618419: error parsing value: Value '182618419' is below minimum 939524096 2024-03-18T23:18:34.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:18:34 smithi175 ceph-mon[67352]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:18:34.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:18:34 smithi175 ceph-mon[67352]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:18:34.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:18:34 smithi175 ceph-mon[67352]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:18:34.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:18:34 smithi175 ceph-mon[67352]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:18:34.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:18:34 smithi175 ceph-mon[67352]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:18:34.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:18:34 smithi175 ceph-mon[67352]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:18:34.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:18:34 smithi175 ceph-mon[67352]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' 2024-03-18T23:18:34.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:18:34 smithi139 ceph-mon[67282]: Adjusting osd_memory_target on smithi049 to 176.2M 2024-03-18T23:18:34.566 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:18:34 smithi139 ceph-mon[67282]: Unable to set osd_memory_target on smithi049 to 184762368: error parsing value: Value '184762368' is below minimum 939524096 2024-03-18T23:18:34.566 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:18:34 smithi139 ceph-mon[67282]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:18:34.566 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:18:34 smithi139 ceph-mon[67282]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:18:34.566 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:18:34 smithi139 ceph-mon[67282]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:18:34.566 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:18:34 smithi139 ceph-mon[67282]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:18:34.566 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:18:34 smithi139 ceph-mon[67282]: Adjusting osd_memory_target on smithi175 to 174.1M 2024-03-18T23:18:34.566 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:18:34 smithi139 ceph-mon[67282]: Unable to set osd_memory_target on smithi175 to 182618419: error parsing value: Value '182618419' is below minimum 939524096 2024-03-18T23:18:34.566 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:18:34 smithi139 ceph-mon[67282]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:18:34.566 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:18:34 smithi139 ceph-mon[67282]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:18:34.566 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:18:34 smithi139 ceph-mon[67282]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:18:34.566 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:18:34 smithi139 ceph-mon[67282]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:18:34.566 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:18:34 smithi139 ceph-mon[67282]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:18:34.567 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:18:34 smithi139 ceph-mon[67282]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:18:34.567 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:18:34 smithi139 ceph-mon[67282]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' 2024-03-18T23:18:34.629 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:18:34 smithi049 ceph-mon[62991]: Adjusting osd_memory_target on smithi049 to 176.2M 2024-03-18T23:18:34.629 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:18:34 smithi049 ceph-mon[62991]: Unable to set osd_memory_target on smithi049 to 184762368: error parsing value: Value '184762368' is below minimum 939524096 2024-03-18T23:18:34.629 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:18:34 smithi049 ceph-mon[62991]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:18:34.629 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:18:34 smithi049 ceph-mon[62991]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:18:34.629 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:18:34 smithi049 ceph-mon[62991]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:18:34.629 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:18:34 smithi049 ceph-mon[62991]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:18:34.629 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:18:34 smithi049 ceph-mon[62991]: Adjusting osd_memory_target on smithi175 to 174.1M 2024-03-18T23:18:34.629 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:18:34 smithi049 ceph-mon[62991]: Unable to set osd_memory_target on smithi175 to 182618419: error parsing value: Value '182618419' is below minimum 939524096 2024-03-18T23:18:34.629 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:18:34 smithi049 ceph-mon[62991]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:18:34.629 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:18:34 smithi049 ceph-mon[62991]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:18:34.630 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:18:34 smithi049 ceph-mon[62991]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:18:34.630 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:18:34 smithi049 ceph-mon[62991]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:18:34.630 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:18:34 smithi049 ceph-mon[62991]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:18:34.630 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:18:34 smithi049 ceph-mon[62991]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:18:34.630 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:18:34 smithi049 ceph-mon[62991]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' 2024-03-18T23:18:34.786 INFO:tasks.workunit.client.0.smithi049.stdout: 1 1061538 9.32 MB/sec execute 456 sec latency 20.756 ms 2024-03-18T23:18:35.460 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:18:35 smithi175 ceph-mon[67352]: pgmap v322: 97 pgs: 97 active+clean; 66 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.5 MiB/s wr, 254 op/s 2024-03-18T23:18:35.460 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:18:35 smithi175 ceph-mon[67352]: Adjusting osd_memory_target on smithi139 to 174.1M 2024-03-18T23:18:35.460 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:18:35 smithi175 ceph-mon[67352]: Unable to set osd_memory_target on smithi139 to 182618419: error parsing value: Value '182618419' is below minimum 939524096 2024-03-18T23:18:35.564 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:18:35 smithi139 ceph-mon[67282]: pgmap v322: 97 pgs: 97 active+clean; 66 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.5 MiB/s wr, 254 op/s 2024-03-18T23:18:35.564 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:18:35 smithi139 ceph-mon[67282]: Adjusting osd_memory_target on smithi139 to 174.1M 2024-03-18T23:18:35.564 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:18:35 smithi139 ceph-mon[67282]: Unable to set osd_memory_target on smithi139 to 182618419: error parsing value: Value '182618419' is below minimum 939524096 2024-03-18T23:18:35.629 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:18:35 smithi049 ceph-mon[62991]: pgmap v322: 97 pgs: 97 active+clean; 66 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.5 MiB/s wr, 254 op/s 2024-03-18T23:18:35.629 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:18:35 smithi049 ceph-mon[62991]: Adjusting osd_memory_target on smithi139 to 174.1M 2024-03-18T23:18:35.629 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:18:35 smithi049 ceph-mon[62991]: Unable to set osd_memory_target on smithi139 to 182618419: error parsing value: Value '182618419' is below minimum 939524096 2024-03-18T23:18:35.786 INFO:tasks.workunit.client.0.smithi049.stdout: 1 1062614 9.31 MB/sec execute 457 sec latency 23.717 ms 2024-03-18T23:18:36.786 INFO:tasks.workunit.client.0.smithi049.stdout: 1 1064349 9.30 MB/sec execute 458 sec latency 28.913 ms 2024-03-18T23:18:37.460 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:18:37 smithi175 ceph-mon[67352]: pgmap v323: 97 pgs: 97 active+clean; 66 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.2 MiB/s wr, 219 op/s 2024-03-18T23:18:37.564 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:18:37 smithi139 ceph-mon[67282]: pgmap v323: 97 pgs: 97 active+clean; 66 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.2 MiB/s wr, 219 op/s 2024-03-18T23:18:37.629 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:18:37 smithi049 ceph-mon[62991]: pgmap v323: 97 pgs: 97 active+clean; 66 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.2 MiB/s wr, 219 op/s 2024-03-18T23:18:37.786 INFO:tasks.workunit.client.0.smithi049.stdout: 1 1065830 9.31 MB/sec execute 459 sec latency 98.484 ms 2024-03-18T23:18:38.786 INFO:tasks.workunit.client.0.smithi049.stdout: 1 1067158 9.30 MB/sec execute 460 sec latency 23.638 ms 2024-03-18T23:18:39.460 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:18:39 smithi175 ceph-mon[67352]: pgmap v324: 97 pgs: 97 active+clean; 72 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.1 MiB/s wr, 287 op/s 2024-03-18T23:18:39.564 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:18:39 smithi139 ceph-mon[67282]: pgmap v324: 97 pgs: 97 active+clean; 72 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.1 MiB/s wr, 287 op/s 2024-03-18T23:18:39.629 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:18:39 smithi049 ceph-mon[62991]: pgmap v324: 97 pgs: 97 active+clean; 72 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.1 MiB/s wr, 287 op/s 2024-03-18T23:18:39.786 INFO:tasks.workunit.client.0.smithi049.stdout: 1 1069180 9.31 MB/sec execute 461 sec latency 21.241 ms 2024-03-18T23:18:40.786 INFO:tasks.workunit.client.0.smithi049.stdout: 1 1069858 9.29 MB/sec execute 462 sec latency 17.144 ms 2024-03-18T23:18:41.460 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:18:41 smithi175 ceph-mon[67352]: pgmap v325: 97 pgs: 97 active+clean; 72 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.0 MiB/s wr, 216 op/s 2024-03-18T23:18:41.564 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:18:41 smithi139 ceph-mon[67282]: pgmap v325: 97 pgs: 97 active+clean; 72 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.0 MiB/s wr, 216 op/s 2024-03-18T23:18:41.629 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:18:41 smithi049 ceph-mon[62991]: pgmap v325: 97 pgs: 97 active+clean; 72 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.0 MiB/s wr, 216 op/s 2024-03-18T23:18:41.786 INFO:tasks.workunit.client.0.smithi049.stdout: 1 1071716 9.29 MB/sec execute 463 sec latency 26.016 ms 2024-03-18T23:18:42.786 INFO:tasks.workunit.client.0.smithi049.stdout: 1 1073093 9.29 MB/sec execute 464 sec latency 18.851 ms 2024-03-18T23:18:43.024 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:43.022+0000 7fd040a62640 1 -- 172.21.15.49:0/2160961040 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd03c107910 msgr2=0x7fd03c107cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:18:43.024 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:43.022+0000 7fd040a62640 1 --2- 172.21.15.49:0/2160961040 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd03c107910 0x7fd03c107cf0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fd02c009900 tx=0x7fd02c02bc30 comp rx=0 tx=0).stop 2024-03-18T23:18:43.024 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:43.022+0000 7fd040a62640 1 -- 172.21.15.49:0/2160961040 shutdown_connections 2024-03-18T23:18:43.024 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:43.022+0000 7fd040a62640 1 --2- 172.21.15.49:0/2160961040 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd03c069840 0x7fd03c10e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:43.024 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:43.022+0000 7fd040a62640 1 --2- 172.21.15.49:0/2160961040 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd03c068ea0 0x7fd03c069300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:43.024 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:43.022+0000 7fd040a62640 1 --2- 172.21.15.49:0/2160961040 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd03c107910 0x7fd03c107cf0 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:43.024 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:43.022+0000 7fd040a62640 1 -- 172.21.15.49:0/2160961040 >> 172.21.15.49:0/2160961040 conn(0x7fd03c077570 msgr2=0x7fd03c077980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:18:43.024 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:43.022+0000 7fd040a62640 1 -- 172.21.15.49:0/2160961040 shutdown_connections 2024-03-18T23:18:43.024 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:43.022+0000 7fd040a62640 1 -- 172.21.15.49:0/2160961040 wait complete. 2024-03-18T23:18:43.024 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:43.023+0000 7fd040a62640 1 Processor -- start 2024-03-18T23:18:43.024 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:43.023+0000 7fd040a62640 1 -- start start 2024-03-18T23:18:43.025 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:43.023+0000 7fd040a62640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd03c068ea0 0x7fd03c198270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:18:43.025 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:43.023+0000 7fd040a62640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd03c069840 0x7fd03c1987b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:18:43.025 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:43.023+0000 7fd040a62640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd03c107910 0x7fd03c19cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:18:43.025 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:43.023+0000 7fd040a62640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fd03c111c90 con 0x7fd03c107910 2024-03-18T23:18:43.025 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:43.023+0000 7fd040a62640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fd03c111b10 con 0x7fd03c068ea0 2024-03-18T23:18:43.025 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:43.023+0000 7fd040a62640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd03c111e10 con 0x7fd03c069840 2024-03-18T23:18:43.025 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:43.023+0000 7fd03ad9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd03c107910 0x7fd03c19cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:18:43.025 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:43.023+0000 7fd03ad9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd03c107910 0x7fd03c19cae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:37782/0 (socket says 172.21.15.49:37782) 2024-03-18T23:18:43.025 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:43.023+0000 7fd03a59c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd03c068ea0 0x7fd03c198270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:18:43.025 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:43.023+0000 7fd03ad9d640 1 -- 172.21.15.49:0/3102296855 learned_addr learned my addr 172.21.15.49:0/3102296855 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:18:43.025 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:43.023+0000 7fd033fff640 1 --2- 172.21.15.49:0/3102296855 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd03c069840 0x7fd03c1987b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:18:43.025 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:43.024+0000 7fd03ad9d640 1 -- 172.21.15.49:0/3102296855 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd03c069840 msgr2=0x7fd03c1987b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:18:43.025 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:43.024+0000 7fd03ad9d640 1 --2- 172.21.15.49:0/3102296855 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd03c069840 0x7fd03c1987b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:43.025 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:43.024+0000 7fd03ad9d640 1 -- 172.21.15.49:0/3102296855 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd03c068ea0 msgr2=0x7fd03c198270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:18:43.025 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:43.024+0000 7fd03ad9d640 1 --2- 172.21.15.49:0/3102296855 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd03c068ea0 0x7fd03c198270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:43.026 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:43.024+0000 7fd03ad9d640 1 -- 172.21.15.49:0/3102296855 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd03c19d210 con 0x7fd03c107910 2024-03-18T23:18:43.026 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:43.024+0000 7fd03ad9d640 1 --2- 172.21.15.49:0/3102296855 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd03c107910 0x7fd03c19cae0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fd03400ea60 tx=0x7fd03400ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:18:43.026 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:43.024+0000 7fd0337fe640 1 -- 172.21.15.49:0/3102296855 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd03400ce40 con 0x7fd03c107910 2024-03-18T23:18:43.026 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:43.024+0000 7fd0337fe640 1 -- 172.21.15.49:0/3102296855 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd034005020 con 0x7fd03c107910 2024-03-18T23:18:43.026 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:43.024+0000 7fd040a62640 1 -- 172.21.15.49:0/3102296855 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd03c19d500 con 0x7fd03c107910 2024-03-18T23:18:43.026 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:43.024+0000 7fd0337fe640 1 -- 172.21.15.49:0/3102296855 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd034010690 con 0x7fd03c107910 2024-03-18T23:18:43.026 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:43.024+0000 7fd040a62640 1 -- 172.21.15.49:0/3102296855 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd03c1a4d90 con 0x7fd03c107910 2024-03-18T23:18:43.029 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:43.027+0000 7fd0337fe640 1 -- 172.21.15.49:0/3102296855 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 25) ==== 137421+0+0 (secure 0 0 0) 0x7fd034005520 con 0x7fd03c107910 2024-03-18T23:18:43.029 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:43.027+0000 7fd043645740 1 Processor -- start 2024-03-18T23:18:43.029 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:43.028+0000 7fd043645740 1 -- start start 2024-03-18T23:18:43.029 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:43.028+0000 7fd043645740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559b03506a10 0x559b03506df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:18:43.029 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:43.028+0000 7fd043645740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559b03507330 0x559b035144b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:18:43.029 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:43.028+0000 7fd043645740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559b035149f0 0x559b03516de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:18:43.029 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:43.028+0000 7fd043645740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x559b0346abe0 con 0x559b03506a10 2024-03-18T23:18:43.029 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:43.028+0000 7fd043645740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x559b03413960 con 0x559b035149f0 2024-03-18T23:18:43.029 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:43.028+0000 7fd043645740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x559b03466c80 con 0x559b03507330 2024-03-18T23:18:43.029 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:43.028+0000 7fd033fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559b03506a10 0x559b03506df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:18:43.029 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:43.028+0000 7fd033fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559b03506a10 0x559b03506df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:37790/0 (socket says 172.21.15.49:37790) 2024-03-18T23:18:43.029 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:43.028+0000 7fd03a59c640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559b03507330 0x559b035144b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:18:43.029 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:43.028+0000 7fd033fff640 1 -- 172.21.15.49:0/2638868382 learned_addr learned my addr 172.21.15.49:0/2638868382 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:18:43.030 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:43.028+0000 7fd0337fe640 1 --2- 172.21.15.49:0/3102296855 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7fd0280a2a10 0x7fd0280a4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:18:43.030 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:43.028+0000 7fd033fff640 1 --2- 172.21.15.49:0/2638868382 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559b035149f0 0x559b03516de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:18:43.030 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:43.028+0000 7fd03ad9d640 1 --2- 172.21.15.49:0/3102296855 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7fd0280a2a10 0x7fd0280a4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:18:43.030 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:43.028+0000 7fd03a59c640 1 -- 172.21.15.49:0/2638868382 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559b035149f0 msgr2=0x559b03516de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:18:43.030 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:43.028+0000 7fd03a59c640 1 --2- 172.21.15.49:0/2638868382 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559b035149f0 0x559b03516de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:43.030 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:43.028+0000 7fd03a59c640 1 -- 172.21.15.49:0/2638868382 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559b03506a10 msgr2=0x559b03506df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:18:43.030 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:43.028+0000 7fd03a59c640 1 --2- 172.21.15.49:0/2638868382 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559b03506a10 0x559b03506df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:43.030 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:43.028+0000 7fd03a59c640 1 -- 172.21.15.49:0/2638868382 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559b0340ad20 con 0x559b03507330 2024-03-18T23:18:43.030 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:43.028+0000 7fd033fff640 1 --2- 172.21.15.49:0/2638868382 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559b03506a10 0x559b03506df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:18:43.030 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:43.028+0000 7fd0337fe640 1 -- 172.21.15.49:0/3102296855 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8556+0+0 (secure 0 0 0) 0x7fd0340cf360 con 0x7fd03c107910 2024-03-18T23:18:43.030 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:43.029+0000 7fd03a59c640 1 --2- 172.21.15.49:0/2638868382 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559b03507330 0x559b035144b0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fd02c000c00 tx=0x7fd02c005be0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a05a537312d2a9ea server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:18:43.030 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:43.029+0000 7fd03ad9d640 1 --2- 172.21.15.49:0/3102296855 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7fd0280a2a10 0x7fd0280a4ed0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fd03409b640 tx=0x7fd034080040 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:18:43.030 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:43.029+0000 7fd030ff9640 1 -- 172.21.15.49:0/2638868382 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd02c042070 con 0x559b03507330 2024-03-18T23:18:43.031 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:43.029+0000 7fd030ff9640 1 -- 172.21.15.49:0/2638868382 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd02c02ca80 con 0x559b03507330 2024-03-18T23:18:43.031 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:43.029+0000 7fd030ff9640 1 -- 172.21.15.49:0/2638868382 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd02c004f00 con 0x559b03507330 2024-03-18T23:18:43.031 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:43.030+0000 7fd043645740 1 -- 172.21.15.49:0/2638868382 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559b03507330 msgr2=0x559b035144b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:18:43.031 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:43.030+0000 7fd043645740 1 --2- 172.21.15.49:0/2638868382 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559b03507330 0x559b035144b0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fd02c000c00 tx=0x7fd02c005be0 comp rx=0 tx=0).stop 2024-03-18T23:18:43.031 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:43.030+0000 7fd043645740 1 -- 172.21.15.49:0/2638868382 shutdown_connections 2024-03-18T23:18:43.031 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:43.030+0000 7fd043645740 1 --2- 172.21.15.49:0/2638868382 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559b035149f0 0x559b03516de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:43.031 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:43.030+0000 7fd043645740 1 --2- 172.21.15.49:0/2638868382 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559b03507330 0x559b035144b0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:43.031 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:43.030+0000 7fd043645740 1 --2- 172.21.15.49:0/2638868382 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559b03506a10 0x559b03506df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:43.031 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:43.030+0000 7fd043645740 1 -- 172.21.15.49:0/2638868382 >> 172.21.15.49:0/2638868382 conn(0x559b03501810 msgr2=0x559b03501bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:18:43.031 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:43.030+0000 7fd043645740 1 -- 172.21.15.49:0/2638868382 shutdown_connections 2024-03-18T23:18:43.031 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:43.030+0000 7fd043645740 1 -- 172.21.15.49:0/2638868382 wait complete. 2024-03-18T23:18:43.035 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:43.030+0000 7fd043645740 1 -- start start 2024-03-18T23:18:43.035 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:43.030+0000 7fd043645740 1 Processor -- start 2024-03-18T23:18:43.035 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:43.031+0000 7fd043645740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559b03506a10 0x559b035143f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:18:43.035 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:43.031+0000 7fd043645740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559b03507330 0x559b03511780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:18:43.035 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:43.031+0000 7fd043645740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559b035149f0 0x559b03511d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:18:43.035 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:43.031+0000 7fd043645740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x559b03517550 con 0x559b035149f0 2024-03-18T23:18:43.035 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:43.031+0000 7fd043645740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x559b03426db0 con 0x559b03507330 2024-03-18T23:18:43.035 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:43.031+0000 7fd043645740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x559b035176d0 con 0x559b03506a10 2024-03-18T23:18:43.036 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:43.031+0000 7fd033fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559b035149f0 0x559b03511d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:18:43.036 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:43.031+0000 7fd043645740 10 client.0 _ll_register_callbacks cb 0x559b0342be70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T23:18:43.036 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:43.031+0000 7fd03a59c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559b03507330 0x559b03511780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:18:43.036 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:43.031+0000 7fd033fff640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559b03506a10 0x559b035143f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:18:43.036 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:43.031+0000 7fd03a59c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559b03507330 0x559b03511780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.49:37538/0 (socket says 172.21.15.49:37538) 2024-03-18T23:18:43.036 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:43.031+0000 7fd03a59c640 1 -- 172.21.15.49:0/3125324868 learned_addr learned my addr 172.21.15.49:0/3125324868 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:18:43.036 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:43.031+0000 7fd033fff640 1 --2- 172.21.15.49:0/3125324868 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559b03506a10 0x559b035143f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.49:41878/0 (socket says 172.21.15.49:41878) 2024-03-18T23:18:43.036 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:43.031+0000 7fd03a59c640 1 -- 172.21.15.49:0/3125324868 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559b03506a10 msgr2=0x559b035143f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:18:43.036 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:43.031+0000 7fd03a59c640 1 --2- 172.21.15.49:0/3125324868 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559b03506a10 0x559b035143f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:43.036 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:43.031+0000 7fd03a59c640 1 -- 172.21.15.49:0/3125324868 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559b035149f0 msgr2=0x559b03511d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:18:43.036 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:43.031+0000 7fd03a59c640 1 --2- 172.21.15.49:0/3125324868 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559b035149f0 0x559b03511d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:43.036 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:43.031+0000 7fd03a59c640 1 -- 172.21.15.49:0/3125324868 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559b03425c60 con 0x559b03507330 2024-03-18T23:18:43.036 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:43.031+0000 7fd03a59c640 1 --2- 172.21.15.49:0/3125324868 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559b03507330 0x559b03511780 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fd02c02f020 tx=0x7fd02c02c710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=56dc9d42003f9ac6 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:18:43.036 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:43.031+0000 7fd021ffb640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-03-18T23:18:43.037 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:43.032+0000 7fd021ffb640 1 -- 172.21.15.49:0/3125324868 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd02c042070 con 0x559b03507330 2024-03-18T23:18:43.037 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:43.032+0000 7fd043645740 1 -- 172.21.15.49:0/3125324868 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd02c02ca80 con 0x559b03507330 2024-03-18T23:18:43.037 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:43.032+0000 7fd021ffb640 1 -- 172.21.15.49:0/3125324868 <== mon.2 v2:172.21.15.139:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8556+0+0 (secure 0 0 0) 0x7fd02c004230 con 0x559b03507330 2024-03-18T23:18:43.037 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:43.032+0000 7fd021ffb640 1 -- 172.21.15.49:0/3125324868 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fd02c057050 con 0x559b03507330 2024-03-18T23:18:43.037 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:43.032+0000 7fd043645740 10 client.26111 fetch_fsmap learned FSMap version 32 2024-03-18T23:18:43.037 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:43.032+0000 7fd043645740 1 -- 172.21.15.49:0/3125324868 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x559b0317f210 con 0x559b03507330 2024-03-18T23:18:43.037 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:43.032+0000 7fd021ffb640 1 -- 172.21.15.49:0/3125324868 <== mon.2 v2:172.21.15.139:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd02c02ca80 con 0x559b03507330 2024-03-18T23:18:43.037 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:43.032+0000 7fd021ffb640 1 -- 172.21.15.49:0/3125324868 <== mon.2 v2:172.21.15.139:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7fd02c04e070 con 0x559b03507330 2024-03-18T23:18:43.037 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:43.032+0000 7fd043645740 10 client.26111 fetch_fsmap finished waiting for FSMap version 32 2024-03-18T23:18:43.037 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:43.032+0000 7fd043645740 10 client.26111 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T23:18:43.037 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:43.032+0000 7fd043645740 20 client.26111 populate_metadata read hostname 'smithi049' 2024-03-18T23:18:43.037 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:43.032+0000 7fd043645740 1 --2- 172.21.15.49:0/3125324868 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x559b035246e0 0x559b03526ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:18:43.037 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:43.032+0000 7fd043645740 4 client.26111 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T23:18:43.037 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:43.032+0000 7fd043645740 1 -- 172.21.15.49:0/3125324868 --> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559b03478120 con 0x559b035246e0 2024-03-18T23:18:43.037 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:43.032+0000 7fd033fff640 1 --2- 172.21.15.49:0/3125324868 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x559b035246e0 0x559b03526ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:18:43.038 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:43.033+0000 7fd021ffb640 10 client.26111 ms_handle_connect on v2:172.21.15.139:6838/4280128209 2024-03-18T23:18:43.038 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:43.033+0000 7fd033fff640 1 --2- 172.21.15.49:0/3125324868 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x559b035246e0 0x559b03526ad0 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1420001d314f7753 server_cookie=83834afeb3563b82 in_seq=0 out_seq=0 2024-03-18T23:18:43.038 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:43.034+0000 7fd021ffb640 1 -- 172.21.15.49:0/3125324868 <== mds.0 v2:172.21.15.139:6838/4280128209 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x559b03478120 con 0x559b035246e0 2024-03-18T23:18:43.038 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:43.034+0000 7fd021ffb640 10 client.26111 handle_command_reply: tid=0 2024-03-18T23:18:43.038 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:43.034+0000 7fd043645740 1 client.26111 shutdown 2024-03-18T23:18:43.038 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:43.035+0000 7fd043645740 1 -- 172.21.15.49:0/3125324868 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559b03507330 msgr2=0x559b03511780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:18:43.038 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:43.035+0000 7fd043645740 1 --2- 172.21.15.49:0/3125324868 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559b03507330 0x559b03511780 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fd02c02f020 tx=0x7fd02c02c710 comp rx=0 tx=0).stop 2024-03-18T23:18:43.038 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:43.035+0000 7fd043645740 1 -- 172.21.15.49:0/3125324868 shutdown_connections 2024-03-18T23:18:43.038 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:43.035+0000 7fd043645740 1 --2- 172.21.15.49:0/3125324868 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x559b035246e0 0x559b03526ad0 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:43.038 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:43.035+0000 7fd043645740 1 --2- 172.21.15.49:0/3125324868 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559b035149f0 0x559b03511d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:43.038 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:43.035+0000 7fd043645740 1 --2- 172.21.15.49:0/3125324868 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559b03507330 0x559b03511780 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:43.038 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:43.035+0000 7fd043645740 1 --2- 172.21.15.49:0/3125324868 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559b03506a10 0x559b035143f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:43.038 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:43.035+0000 7fd021ffb640 0 client.26111 ms_handle_reset on v2:172.21.15.139:6838/4280128209 2024-03-18T23:18:43.039 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:43.035+0000 7fd043645740 1 -- 172.21.15.49:0/3125324868 >> 172.21.15.49:0/3125324868 conn(0x559b03501810 msgr2=0x559b03503560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:18:43.039 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:43.035+0000 7fd043645740 1 -- 172.21.15.49:0/3125324868 shutdown_connections 2024-03-18T23:18:43.039 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:43.035+0000 7fd043645740 1 -- 172.21.15.49:0/3125324868 wait complete. 2024-03-18T23:18:43.039 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:43.035+0000 7fd043645740 20 client.26111 trim_cache size 0 max 16384 2024-03-18T23:18:43.052 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:43.051+0000 7fd043645740 1 Processor -- start 2024-03-18T23:18:43.052 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:43.051+0000 7fd043645740 1 -- start start 2024-03-18T23:18:43.052 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:43.051+0000 7fd043645740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559b03506a10 0x559b03520330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:18:43.052 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:43.051+0000 7fd043645740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559b03507330 0x559b03501bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:18:43.052 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:43.052+0000 7fd043645740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559b035149f0 0x559b0352c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:18:43.052 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:43.052+0000 7fd043645740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fd03c1ab220 con 0x559b035149f0 2024-03-18T23:18:43.052 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:43.052+0000 7fd043645740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fd03c1acd60 con 0x559b03506a10 2024-03-18T23:18:43.052 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:43.052+0000 7fd043645740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd03c1ab8f0 con 0x559b03507330 2024-03-18T23:18:43.052 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:43.052+0000 7fd033fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559b035149f0 0x559b0352c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:18:43.052 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:43.052+0000 7fd03a59c640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559b03507330 0x559b03501bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:18:43.052 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:43.052+0000 7fd033fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559b035149f0 0x559b0352c0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:37810/0 (socket says 172.21.15.49:37810) 2024-03-18T23:18:43.052 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:43.052+0000 7fd033fff640 1 -- 172.21.15.49:0/1202703452 learned_addr learned my addr 172.21.15.49:0/1202703452 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:18:43.053 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:43.052+0000 7fd033fff640 1 --2- 172.21.15.49:0/1202703452 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559b03506a10 0x559b03520330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:18:43.053 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:43.052+0000 7fd03a59c640 1 -- 172.21.15.49:0/1202703452 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559b03506a10 msgr2=0x559b03520330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:18:43.053 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:43.052+0000 7fd03a59c640 1 --2- 172.21.15.49:0/1202703452 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559b03506a10 0x559b03520330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:43.053 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:43.052+0000 7fd03a59c640 1 -- 172.21.15.49:0/1202703452 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559b035149f0 msgr2=0x559b0352c0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:18:43.053 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:43.052+0000 7fd03a59c640 1 --2- 172.21.15.49:0/1202703452 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559b035149f0 0x559b0352c0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:43.053 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:43.052+0000 7fd03a59c640 1 -- 172.21.15.49:0/1202703452 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559b03526e40 con 0x559b03507330 2024-03-18T23:18:43.053 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:43.052+0000 7fd033fff640 1 --2- 172.21.15.49:0/1202703452 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559b035149f0 0x559b0352c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:18:43.053 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:43.052+0000 7fd03a59c640 1 --2- 172.21.15.49:0/1202703452 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559b03507330 0x559b03501bf0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fd02c0036d0 tx=0x7fd02c005dd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ece3ae4e7c065351 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:18:43.053 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:43.053+0000 7fd023fff640 1 -- 172.21.15.49:0/1202703452 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd02c042040 con 0x559b03507330 2024-03-18T23:18:43.053 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:43.053+0000 7fd023fff640 1 -- 172.21.15.49:0/1202703452 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd02c005850 con 0x559b03507330 2024-03-18T23:18:43.053 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:43.053+0000 7fd023fff640 1 -- 172.21.15.49:0/1202703452 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd02c005020 con 0x559b03507330 2024-03-18T23:18:43.057 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:43.053+0000 7fd043645740 1 -- 172.21.15.49:0/1202703452 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559b03507330 msgr2=0x559b03501bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:18:43.057 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:43.053+0000 7fd043645740 1 --2- 172.21.15.49:0/1202703452 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559b03507330 0x559b03501bf0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fd02c0036d0 tx=0x7fd02c005dd0 comp rx=0 tx=0).stop 2024-03-18T23:18:43.057 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:43.053+0000 7fd043645740 1 -- 172.21.15.49:0/1202703452 shutdown_connections 2024-03-18T23:18:43.057 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:43.053+0000 7fd043645740 1 --2- 172.21.15.49:0/1202703452 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559b035149f0 0x559b0352c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:43.057 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:43.053+0000 7fd043645740 1 --2- 172.21.15.49:0/1202703452 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559b03507330 0x559b03501bf0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:43.057 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:43.053+0000 7fd043645740 1 --2- 172.21.15.49:0/1202703452 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559b03506a10 0x559b03520330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:43.057 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:43.053+0000 7fd043645740 1 -- 172.21.15.49:0/1202703452 >> 172.21.15.49:0/1202703452 conn(0x559b03501810 msgr2=0x559b03500ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:18:43.057 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:43.053+0000 7fd043645740 1 -- 172.21.15.49:0/1202703452 shutdown_connections 2024-03-18T23:18:43.057 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:43.053+0000 7fd043645740 1 -- 172.21.15.49:0/1202703452 wait complete. 2024-03-18T23:18:43.057 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:43.054+0000 7fd043645740 1 -- start start 2024-03-18T23:18:43.057 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:43.054+0000 7fd043645740 1 Processor -- start 2024-03-18T23:18:43.057 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:43.054+0000 7fd043645740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559b03506a10 0x559b0350e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:18:43.057 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:43.054+0000 7fd043645740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559b03507330 0x559b0350eac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:18:43.058 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:43.054+0000 7fd043645740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559b035149f0 0x559b0350f000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:18:43.058 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:43.054+0000 7fd043645740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fd03c1ac690 con 0x559b03506a10 2024-03-18T23:18:43.058 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:43.054+0000 7fd043645740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fd03c1abfc0 con 0x559b03507330 2024-03-18T23:18:43.058 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:43.054+0000 7fd043645740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd03c1a9220 con 0x559b035149f0 2024-03-18T23:18:43.058 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:43.054+0000 7fd043645740 10 client.0 _ll_register_callbacks cb 0x7fd024009850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T23:18:43.058 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:43.054+0000 7fd033fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559b03506a10 0x559b0350e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:18:43.058 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:43.054+0000 7fd03a59c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559b03507330 0x559b0350eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:18:43.058 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:43.054+0000 7fd033fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559b03506a10 0x559b0350e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:37826/0 (socket says 172.21.15.49:37826) 2024-03-18T23:18:43.058 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:43.054+0000 7fd033fff640 1 -- 172.21.15.49:0/180291047 learned_addr learned my addr 172.21.15.49:0/180291047 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:18:43.058 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:43.054+0000 7fd033fff640 1 --2- 172.21.15.49:0/180291047 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559b035149f0 0x559b0350f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:18:43.058 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:43.055+0000 7fd03a59c640 1 -- 172.21.15.49:0/180291047 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559b035149f0 msgr2=0x559b0350f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:18:43.058 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:43.055+0000 7fd03a59c640 1 --2- 172.21.15.49:0/180291047 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559b035149f0 0x559b0350f000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:43.058 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:43.055+0000 7fd03a59c640 1 -- 172.21.15.49:0/180291047 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559b03506a10 msgr2=0x559b0350e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:18:43.059 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:43.055+0000 7fd03a59c640 1 --2- 172.21.15.49:0/180291047 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559b03506a10 0x559b0350e580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:43.059 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:43.055+0000 7fd03a59c640 1 -- 172.21.15.49:0/180291047 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559b0324b620 con 0x559b03507330 2024-03-18T23:18:43.059 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:43.055+0000 7fd033fff640 1 --2- 172.21.15.49:0/180291047 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559b03506a10 0x559b0350e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:18:43.059 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:43.055+0000 7fd03a59c640 1 --2- 172.21.15.49:0/180291047 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559b03507330 0x559b0350eac0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x559b03503c20 tx=0x7fd02c0035f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=54fcc64aea8b1074 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:18:43.059 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:43.055+0000 7fd021ffb640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-03-18T23:18:43.059 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:43.055+0000 7fd021ffb640 1 -- 172.21.15.49:0/180291047 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd02c042050 con 0x559b03507330 2024-03-18T23:18:43.059 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:43.055+0000 7fd043645740 1 -- 172.21.15.49:0/180291047 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd02c005850 con 0x559b03507330 2024-03-18T23:18:43.059 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:43.055+0000 7fd021ffb640 1 -- 172.21.15.49:0/180291047 <== mon.2 v2:172.21.15.139:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8556+0+0 (secure 0 0 0) 0x7fd02c004fd0 con 0x559b03507330 2024-03-18T23:18:43.059 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:43.055+0000 7fd021ffb640 1 -- 172.21.15.49:0/180291047 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fd02c0051d0 con 0x559b03507330 2024-03-18T23:18:43.059 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:43.055+0000 7fd043645740 10 client.26117 fetch_fsmap learned FSMap version 32 2024-03-18T23:18:43.059 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:43.055+0000 7fd043645740 1 -- 172.21.15.49:0/180291047 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x559b0358aa70 con 0x559b03507330 2024-03-18T23:18:43.059 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:43.055+0000 7fd021ffb640 1 -- 172.21.15.49:0/180291047 <== mon.2 v2:172.21.15.139:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd02c005850 con 0x559b03507330 2024-03-18T23:18:43.059 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:43.055+0000 7fd021ffb640 1 -- 172.21.15.49:0/180291047 <== mon.2 v2:172.21.15.139:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7fd02c04e070 con 0x559b03507330 2024-03-18T23:18:43.059 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:43.056+0000 7fd043645740 10 client.26117 fetch_fsmap finished waiting for FSMap version 32 2024-03-18T23:18:43.059 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:43.056+0000 7fd043645740 10 client.26117 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T23:18:43.060 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:43.056+0000 7fd043645740 20 client.26117 populate_metadata read hostname 'smithi049' 2024-03-18T23:18:43.060 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:43.056+0000 7fd043645740 1 --2- 172.21.15.49:0/180291047 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x559b0358af40 0x559b0358d330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:18:43.060 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:43.056+0000 7fd043645740 4 client.26117 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T23:18:43.060 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:43.056+0000 7fd043645740 1 -- 172.21.15.49:0/180291047 --> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559b03472470 con 0x559b0358af40 2024-03-18T23:18:43.060 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:43.056+0000 7fd033fff640 1 --2- 172.21.15.49:0/180291047 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x559b0358af40 0x559b0358d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:18:43.060 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:43.056+0000 7fd033fff640 1 --2- 172.21.15.49:0/180291047 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x559b0358af40 0x559b0358d330 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cd9873b9714cc380 server_cookie=f27d1c399d9f5ffd in_seq=0 out_seq=0 2024-03-18T23:18:43.060 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:43.056+0000 7fd021ffb640 10 client.26117 ms_handle_connect on v2:172.21.15.139:6838/4280128209 2024-03-18T23:18:43.215 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:43.215+0000 7fd021ffb640 1 -- 172.21.15.49:0/180291047 <== mds.0 v2:172.21.15.139:6838/4280128209 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x559b03472470 con 0x559b0358af40 2024-03-18T23:18:43.215 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:43.215+0000 7fd021ffb640 10 client.26117 handle_command_reply: tid=0 2024-03-18T23:18:43.215 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:43.215+0000 7fd043645740 1 client.26117 shutdown 2024-03-18T23:18:43.215 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:43.215+0000 7fd043645740 1 -- 172.21.15.49:0/180291047 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559b03507330 msgr2=0x559b0350eac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:18:43.216 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:43.215+0000 7fd043645740 1 --2- 172.21.15.49:0/180291047 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559b03507330 0x559b0350eac0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x559b03503c20 tx=0x7fd02c0035f0 comp rx=0 tx=0).stop 2024-03-18T23:18:43.216 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:43.215+0000 7fd043645740 1 -- 172.21.15.49:0/180291047 shutdown_connections 2024-03-18T23:18:43.216 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:43.215+0000 7fd043645740 1 --2- 172.21.15.49:0/180291047 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x559b0358af40 0x559b0358d330 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:43.216 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:43.215+0000 7fd043645740 1 --2- 172.21.15.49:0/180291047 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559b035149f0 0x559b0350f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:43.216 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:43.215+0000 7fd043645740 1 --2- 172.21.15.49:0/180291047 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559b03507330 0x559b0350eac0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:43.216 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:43.215+0000 7fd043645740 1 --2- 172.21.15.49:0/180291047 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559b03506a10 0x559b0350e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:43.216 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:43.215+0000 7fd043645740 1 -- 172.21.15.49:0/180291047 >> 172.21.15.49:0/180291047 conn(0x559b03501810 msgr2=0x559b03500ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:18:43.216 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:43.215+0000 7fd021ffb640 0 client.26117 ms_handle_reset on v2:172.21.15.139:6838/4280128209 2024-03-18T23:18:43.216 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:43.216+0000 7fd043645740 1 -- 172.21.15.49:0/180291047 shutdown_connections 2024-03-18T23:18:43.216 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:43.216+0000 7fd043645740 1 -- 172.21.15.49:0/180291047 wait complete. 2024-03-18T23:18:43.216 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:43.216+0000 7fd043645740 20 client.26117 trim_cache size 0 max 16384 2024-03-18T23:18:43.217 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:{ 2024-03-18T23:18:43.217 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "message": "", 2024-03-18T23:18:43.217 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "return_code": 0 2024-03-18T23:18:43.217 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:} 2024-03-18T23:18:43.218 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:43.216+0000 7fd040a62640 1 -- 172.21.15.49:0/3102296855 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7fd0280a2a10 msgr2=0x7fd0280a4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:18:43.218 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:43.216+0000 7fd040a62640 1 --2- 172.21.15.49:0/3102296855 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7fd0280a2a10 0x7fd0280a4ed0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fd03409b640 tx=0x7fd034080040 comp rx=0 tx=0).stop 2024-03-18T23:18:43.218 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:43.216+0000 7fd040a62640 1 -- 172.21.15.49:0/3102296855 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd03c107910 msgr2=0x7fd03c19cae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:18:43.218 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:43.216+0000 7fd040a62640 1 --2- 172.21.15.49:0/3102296855 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd03c107910 0x7fd03c19cae0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fd03400ea60 tx=0x7fd03400ef20 comp rx=0 tx=0).stop 2024-03-18T23:18:43.218 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:43.216+0000 7fd040a62640 1 -- 172.21.15.49:0/3102296855 shutdown_connections 2024-03-18T23:18:43.218 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:43.216+0000 7fd040a62640 1 --2- 172.21.15.49:0/3102296855 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7fd0280a2a10 0x7fd0280a4ed0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:43.218 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:43.216+0000 7fd040a62640 1 --2- 172.21.15.49:0/3102296855 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd03c107910 0x7fd03c19cae0 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:43.218 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:43.216+0000 7fd040a62640 1 --2- 172.21.15.49:0/3102296855 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd03c069840 0x7fd03c1987b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:43.218 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:43.216+0000 7fd040a62640 1 --2- 172.21.15.49:0/3102296855 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd03c068ea0 0x7fd03c198270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:43.218 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:43.216+0000 7fd040a62640 1 -- 172.21.15.49:0/3102296855 >> 172.21.15.49:0/3102296855 conn(0x7fd03c077570 msgr2=0x7fd03c0fe9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:18:43.218 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:43.217+0000 7fd040a62640 1 -- 172.21.15.49:0/3102296855 shutdown_connections 2024-03-18T23:18:43.218 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:43.217+0000 7fd040a62640 1 -- 172.21.15.49:0/3102296855 wait complete. 2024-03-18T23:18:43.460 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:18:43 smithi175 ceph-mon[67352]: pgmap v326: 97 pgs: 97 active+clean; 77 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 2.9 MiB/s wr, 277 op/s 2024-03-18T23:18:43.564 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:18:43 smithi139 ceph-mon[67282]: pgmap v326: 97 pgs: 97 active+clean; 77 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 2.9 MiB/s wr, 277 op/s 2024-03-18T23:18:43.628 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:18:43 smithi049 ceph-mon[62991]: pgmap v326: 97 pgs: 97 active+clean; 77 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 2.9 MiB/s wr, 277 op/s 2024-03-18T23:18:43.786 INFO:tasks.workunit.client.0.smithi049.stdout: 1 1074236 9.29 MB/sec execute 465 sec latency 88.355 ms 2024-03-18T23:18:44.786 INFO:tasks.workunit.client.0.smithi049.stdout: 1 1076333 9.29 MB/sec execute 466 sec latency 21.101 ms 2024-03-18T23:18:45.460 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:18:45 smithi175 ceph-mon[67352]: pgmap v327: 97 pgs: 97 active+clean; 77 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.0 MiB/s wr, 165 op/s 2024-03-18T23:18:45.564 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:18:45 smithi139 ceph-mon[67282]: pgmap v327: 97 pgs: 97 active+clean; 77 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.0 MiB/s wr, 165 op/s 2024-03-18T23:18:45.628 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:18:45 smithi049 ceph-mon[62991]: pgmap v327: 97 pgs: 97 active+clean; 77 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.0 MiB/s wr, 165 op/s 2024-03-18T23:18:45.786 INFO:tasks.workunit.client.0.smithi049.stdout: 1 1077387 9.28 MB/sec execute 467 sec latency 29.642 ms 2024-03-18T23:18:46.786 INFO:tasks.workunit.client.0.smithi049.stdout: 1 1079678 9.29 MB/sec execute 468 sec latency 20.303 ms 2024-03-18T23:18:47.460 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:18:47 smithi175 ceph-mon[67352]: pgmap v328: 97 pgs: 97 active+clean; 77 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.0 MiB/s wr, 141 op/s 2024-03-18T23:18:47.564 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:18:47 smithi139 ceph-mon[67282]: pgmap v328: 97 pgs: 97 active+clean; 77 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.0 MiB/s wr, 141 op/s 2024-03-18T23:18:47.629 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:18:47 smithi049 ceph-mon[62991]: pgmap v328: 97 pgs: 97 active+clean; 77 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.0 MiB/s wr, 141 op/s 2024-03-18T23:18:47.786 INFO:tasks.workunit.client.0.smithi049.stdout: 1 1080538 9.28 MB/sec execute 469 sec latency 28.779 ms 2024-03-18T23:18:48.786 INFO:tasks.workunit.client.0.smithi049.stdout: 1 1082163 9.27 MB/sec execute 470 sec latency 78.903 ms 2024-03-18T23:18:49.460 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:18:49 smithi175 ceph-mon[67352]: pgmap v329: 97 pgs: 97 active+clean; 71 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 3.1 MiB/s wr, 291 op/s 2024-03-18T23:18:49.564 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:18:49 smithi139 ceph-mon[67282]: pgmap v329: 97 pgs: 97 active+clean; 71 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 3.1 MiB/s wr, 291 op/s 2024-03-18T23:18:49.629 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:18:49 smithi049 ceph-mon[62991]: pgmap v329: 97 pgs: 97 active+clean; 71 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 3.1 MiB/s wr, 291 op/s 2024-03-18T23:18:49.786 INFO:tasks.workunit.client.0.smithi049.stdout: 1 1083643 9.28 MB/sec execute 471 sec latency 16.770 ms 2024-03-18T23:18:50.786 INFO:tasks.workunit.client.0.smithi049.stdout: 1 1085088 9.27 MB/sec execute 472 sec latency 26.919 ms 2024-03-18T23:18:51.460 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:18:51 smithi175 ceph-mon[67352]: pgmap v330: 97 pgs: 97 active+clean; 71 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.2 MiB/s wr, 223 op/s 2024-03-18T23:18:51.564 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:18:51 smithi139 ceph-mon[67282]: pgmap v330: 97 pgs: 97 active+clean; 71 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.2 MiB/s wr, 223 op/s 2024-03-18T23:18:51.629 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:18:51 smithi049 ceph-mon[62991]: pgmap v330: 97 pgs: 97 active+clean; 71 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.2 MiB/s wr, 223 op/s 2024-03-18T23:18:51.786 INFO:tasks.workunit.client.0.smithi049.stdout: 1 1087070 9.28 MB/sec execute 473 sec latency 19.173 ms 2024-03-18T23:18:52.787 INFO:tasks.workunit.client.0.smithi049.stdout: 1 1088371 9.27 MB/sec execute 474 sec latency 26.854 ms 2024-03-18T23:18:53.460 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:18:53 smithi175 ceph-mon[67352]: pgmap v331: 97 pgs: 97 active+clean; 76 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 3.2 MiB/s wr, 293 op/s 2024-03-18T23:18:53.564 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:18:53 smithi139 ceph-mon[67282]: pgmap v331: 97 pgs: 97 active+clean; 76 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 3.2 MiB/s wr, 293 op/s 2024-03-18T23:18:53.628 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:18:53 smithi049 ceph-mon[62991]: pgmap v331: 97 pgs: 97 active+clean; 76 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 3.2 MiB/s wr, 293 op/s 2024-03-18T23:18:53.787 INFO:tasks.workunit.client.0.smithi049.stdout: 1 1090184 9.27 MB/sec execute 475 sec latency 97.800 ms 2024-03-18T23:18:54.787 INFO:tasks.workunit.client.0.smithi049.stdout: 1 1091130 9.26 MB/sec execute 476 sec latency 17.443 ms 2024-03-18T23:18:55.460 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:18:55 smithi175 ceph-mon[67352]: pgmap v332: 97 pgs: 97 active+clean; 76 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.5 MiB/s wr, 240 op/s 2024-03-18T23:18:55.564 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:18:55 smithi139 ceph-mon[67282]: pgmap v332: 97 pgs: 97 active+clean; 76 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.5 MiB/s wr, 240 op/s 2024-03-18T23:18:55.629 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:18:55 smithi049 ceph-mon[62991]: pgmap v332: 97 pgs: 97 active+clean; 76 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.5 MiB/s wr, 240 op/s 2024-03-18T23:18:55.787 INFO:tasks.workunit.client.0.smithi049.stdout: 1 1093009 9.26 MB/sec execute 477 sec latency 21.812 ms 2024-03-18T23:18:56.313 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:56.311+0000 7f28a5cb5640 1 -- 172.21.15.49:0/299544792 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f28a0100970 msgr2=0x7f28a0100df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:18:56.313 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:56.311+0000 7f28a5cb5640 1 --2- 172.21.15.49:0/299544792 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f28a0100970 0x7f28a0100df0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f2894009900 tx=0x7f289402bc30 comp rx=0 tx=0).stop 2024-03-18T23:18:56.313 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:56.311+0000 7f28a5cb5640 1 -- 172.21.15.49:0/299544792 shutdown_connections 2024-03-18T23:18:56.313 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:56.311+0000 7f28a5cb5640 1 --2- 172.21.15.49:0/299544792 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f28a0101330 0x7f28a010d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:56.313 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:56.311+0000 7f28a5cb5640 1 --2- 172.21.15.49:0/299544792 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f28a0100970 0x7f28a0100df0 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:56.314 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:56.311+0000 7f28a5cb5640 1 --2- 172.21.15.49:0/299544792 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f28a01056c0 0x7f28a0105aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:56.314 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:56.311+0000 7f28a5cb5640 1 -- 172.21.15.49:0/299544792 >> 172.21.15.49:0/299544792 conn(0x7f28a0077570 msgr2=0x7f28a0077980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:18:56.314 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:56.311+0000 7f28a5cb5640 1 -- 172.21.15.49:0/299544792 shutdown_connections 2024-03-18T23:18:56.314 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:56.311+0000 7f28a5cb5640 1 -- 172.21.15.49:0/299544792 wait complete. 2024-03-18T23:18:56.314 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:56.312+0000 7f28a5cb5640 1 Processor -- start 2024-03-18T23:18:56.314 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:56.312+0000 7f28a5cb5640 1 -- start start 2024-03-18T23:18:56.314 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:56.312+0000 7f28a5cb5640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f28a0100970 0x7f28a01980d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:18:56.314 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:56.312+0000 7f28a5cb5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f28a0101330 0x7f28a0198610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:18:56.314 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:56.312+0000 7f28a5cb5640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f28a01056c0 0x7f28a019c940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:18:56.314 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:56.312+0000 7f28a5cb5640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f28a0110be0 con 0x7f28a0100970 2024-03-18T23:18:56.314 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:56.312+0000 7f28a5cb5640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f28a0110a60 con 0x7f28a0101330 2024-03-18T23:18:56.314 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:56.312+0000 7f28a5cb5640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f28a0110d60 con 0x7f28a01056c0 2024-03-18T23:18:56.314 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:56.312+0000 7f289f7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f28a0100970 0x7f28a01980d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:18:56.315 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:56.312+0000 7f289f7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f28a0100970 0x7f28a01980d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:57206/0 (socket says 172.21.15.49:57206) 2024-03-18T23:18:56.315 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:56.312+0000 7f289f7fe640 1 -- 172.21.15.49:0/1057699488 learned_addr learned my addr 172.21.15.49:0/1057699488 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:18:56.315 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:56.312+0000 7f289effd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f28a0101330 0x7f28a0198610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:18:56.315 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:56.312+0000 7f289ffff640 1 --2- 172.21.15.49:0/1057699488 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f28a01056c0 0x7f28a019c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:18:56.315 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:56.313+0000 7f289f7fe640 1 -- 172.21.15.49:0/1057699488 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f28a01056c0 msgr2=0x7f28a019c940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:18:56.315 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:56.313+0000 7f289f7fe640 1 --2- 172.21.15.49:0/1057699488 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f28a01056c0 0x7f28a019c940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:56.315 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:56.313+0000 7f289f7fe640 1 -- 172.21.15.49:0/1057699488 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f28a0101330 msgr2=0x7f28a0198610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:18:56.315 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:56.313+0000 7f289f7fe640 1 --2- 172.21.15.49:0/1057699488 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f28a0101330 0x7f28a0198610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:56.315 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:56.313+0000 7f289f7fe640 1 -- 172.21.15.49:0/1057699488 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28a019d070 con 0x7f28a0100970 2024-03-18T23:18:56.315 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:56.313+0000 7f289f7fe640 1 --2- 172.21.15.49:0/1057699488 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f28a0100970 0x7f28a01980d0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f289000e9f0 tx=0x7f289000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:18:56.315 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:56.313+0000 7f289cff9640 1 -- 172.21.15.49:0/1057699488 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f289000cd80 con 0x7f28a0100970 2024-03-18T23:18:56.315 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:56.313+0000 7f289cff9640 1 -- 172.21.15.49:0/1057699488 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2890010910 con 0x7f28a0100970 2024-03-18T23:18:56.315 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:56.313+0000 7f289cff9640 1 -- 172.21.15.49:0/1057699488 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2890018990 con 0x7f28a0100970 2024-03-18T23:18:56.315 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:56.313+0000 7f28a5cb5640 1 -- 172.21.15.49:0/1057699488 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28a019d360 con 0x7f28a0100970 2024-03-18T23:18:56.316 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:56.313+0000 7f28a5cb5640 1 -- 172.21.15.49:0/1057699488 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28a01a4bf0 con 0x7f28a0100970 2024-03-18T23:18:56.322 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:56.315+0000 7f28a8976740 1 Processor -- start 2024-03-18T23:18:56.322 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:56.315+0000 7f289cff9640 1 -- 172.21.15.49:0/1057699488 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 25) ==== 137421+0+0 (secure 0 0 0) 0x7f2890010ac0 con 0x7f28a0100970 2024-03-18T23:18:56.322 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:56.316+0000 7f28a8976740 1 -- start start 2024-03-18T23:18:56.322 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:56.316+0000 7f28a8976740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5562caa5eef0 0x5562caa5f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:18:56.322 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:56.316+0000 7f28a8976740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5562caa5f810 0x5562caa680a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:18:56.323 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:56.316+0000 7f28a8976740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5562caa685e0 0x5562caa6a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:18:56.323 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:56.316+0000 7f28a8976740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5562ca9c2be0 con 0x5562caa5f810 2024-03-18T23:18:56.323 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:56.316+0000 7f28a8976740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x5562ca96b960 con 0x5562caa5eef0 2024-03-18T23:18:56.323 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:56.316+0000 7f28a8976740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5562ca9bec80 con 0x5562caa685e0 2024-03-18T23:18:56.323 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:56.316+0000 7f289f7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5562caa5f810 0x5562caa680a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:18:56.323 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:56.316+0000 7f289ffff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5562caa5eef0 0x5562caa5f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:18:56.323 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:56.316+0000 7f289ffff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5562caa5eef0 0x5562caa5f2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.49:43820/0 (socket says 172.21.15.49:43820) 2024-03-18T23:18:56.323 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:56.316+0000 7f289f7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5562caa5f810 0x5562caa680a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:57216/0 (socket says 172.21.15.49:57216) 2024-03-18T23:18:56.323 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:56.316+0000 7f289ffff640 1 -- 172.21.15.49:0/3303610324 learned_addr learned my addr 172.21.15.49:0/3303610324 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:18:56.323 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:56.316+0000 7f289effd640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5562caa685e0 0x5562caa6a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:18:56.323 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:56.316+0000 7f289ffff640 1 -- 172.21.15.49:0/3303610324 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5562caa685e0 msgr2=0x5562caa6a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:18:56.323 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:56.316+0000 7f289ffff640 1 --2- 172.21.15.49:0/3303610324 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5562caa685e0 0x5562caa6a9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:56.323 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:56.316+0000 7f289ffff640 1 -- 172.21.15.49:0/3303610324 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5562caa5f810 msgr2=0x5562caa680a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:18:56.324 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:56.316+0000 7f289ffff640 1 --2- 172.21.15.49:0/3303610324 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5562caa5f810 0x5562caa680a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:56.324 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:56.316+0000 7f289ffff640 1 -- 172.21.15.49:0/3303610324 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5562ca962d20 con 0x5562caa5eef0 2024-03-18T23:18:56.324 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:56.316+0000 7f289f7fe640 1 --2- 172.21.15.49:0/3303610324 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5562caa5f810 0x5562caa680a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:18:56.324 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:56.316+0000 7f289cff9640 1 --2- 172.21.15.49:0/1057699488 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f28880a2a90 0x7f28880a4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:18:56.324 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:56.317+0000 7f289f7fe640 1 --2- 172.21.15.49:0/1057699488 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f28880a2a90 0x7f28880a4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:18:56.324 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:56.317+0000 7f289ffff640 1 --2- 172.21.15.49:0/3303610324 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5562caa5eef0 0x5562caa5f2d0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f2898009fc0 tx=0x7f28980118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9ef60158d5cf5d53 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:18:56.324 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:56.317+0000 7f289cff9640 1 -- 172.21.15.49:0/1057699488 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8556+0+0 (secure 0 0 0) 0x7f28900cf120 con 0x7f28a0100970 2024-03-18T23:18:56.324 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:56.317+0000 7f2885ffb640 1 -- 172.21.15.49:0/3303610324 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2898004d20 con 0x5562caa5eef0 2024-03-18T23:18:56.324 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:56.317+0000 7f2885ffb640 1 -- 172.21.15.49:0/3303610324 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2898004050 con 0x5562caa5eef0 2024-03-18T23:18:56.324 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:56.317+0000 7f2885ffb640 1 -- 172.21.15.49:0/3303610324 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2898005260 con 0x5562caa5eef0 2024-03-18T23:18:56.324 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:56.317+0000 7f28a8976740 1 -- 172.21.15.49:0/3303610324 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5562caa5eef0 msgr2=0x5562caa5f2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:18:56.324 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:56.317+0000 7f289f7fe640 1 --2- 172.21.15.49:0/1057699488 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f28880a2a90 0x7f28880a4f50 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f28900cc900 tx=0x7f28900d2c70 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:18:56.324 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:56.317+0000 7f28a8976740 1 --2- 172.21.15.49:0/3303610324 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5562caa5eef0 0x5562caa5f2d0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f2898009fc0 tx=0x7f28980118d0 comp rx=0 tx=0).stop 2024-03-18T23:18:56.324 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:56.318+0000 7f28a8976740 1 -- 172.21.15.49:0/3303610324 shutdown_connections 2024-03-18T23:18:56.325 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:56.318+0000 7f28a8976740 1 --2- 172.21.15.49:0/3303610324 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5562caa685e0 0x5562caa6a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:56.325 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:56.318+0000 7f28a8976740 1 --2- 172.21.15.49:0/3303610324 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5562caa5f810 0x5562caa680a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:56.325 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:56.318+0000 7f28a8976740 1 --2- 172.21.15.49:0/3303610324 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5562caa5eef0 0x5562caa5f2d0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:56.325 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:56.318+0000 7f28a8976740 1 -- 172.21.15.49:0/3303610324 >> 172.21.15.49:0/3303610324 conn(0x5562caa59810 msgr2=0x5562caa59bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:18:56.325 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:56.318+0000 7f28a8976740 1 -- 172.21.15.49:0/3303610324 shutdown_connections 2024-03-18T23:18:56.325 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:56.318+0000 7f28a8976740 1 -- 172.21.15.49:0/3303610324 wait complete. 2024-03-18T23:18:56.325 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:56.318+0000 7f28a8976740 1 -- start start 2024-03-18T23:18:56.325 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:56.318+0000 7f28a8976740 1 Processor -- start 2024-03-18T23:18:56.325 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:56.318+0000 7f28a8976740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5562caa5eef0 0x5562caa67fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:18:56.325 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:56.318+0000 7f28a8976740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5562caa5f810 0x5562caa65340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:18:56.325 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:56.318+0000 7f28a8976740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5562caa685e0 0x5562caa65950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:18:56.325 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:56.318+0000 7f28a8976740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5562caa6b210 con 0x5562caa685e0 2024-03-18T23:18:56.325 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:56.318+0000 7f28a8976740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x5562ca97edb0 con 0x5562caa5eef0 2024-03-18T23:18:56.325 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:56.318+0000 7f28a8976740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5562caa6b390 con 0x5562caa5f810 2024-03-18T23:18:56.325 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:56.319+0000 7f28a8976740 10 client.0 _ll_register_callbacks cb 0x5562ca983e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T23:18:56.326 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:56.319+0000 7f289ffff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5562caa685e0 0x5562caa65950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:18:56.326 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:56.319+0000 7f289ffff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5562caa685e0 0x5562caa65950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:57224/0 (socket says 172.21.15.49:57224) 2024-03-18T23:18:56.326 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:56.319+0000 7f289ffff640 1 -- 172.21.15.49:0/1491177433 learned_addr learned my addr 172.21.15.49:0/1491177433 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:18:56.326 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:56.319+0000 7f289effd640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5562caa5f810 0x5562caa65340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:18:56.326 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:56.319+0000 7f289ffff640 1 --2- 172.21.15.49:0/1491177433 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5562caa5eef0 0x5562caa67fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:18:56.326 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:56.319+0000 7f289ffff640 1 -- 172.21.15.49:0/1491177433 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5562caa5f810 msgr2=0x5562caa65340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:18:56.326 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:56.319+0000 7f289ffff640 1 --2- 172.21.15.49:0/1491177433 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5562caa5f810 0x5562caa65340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:56.326 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:56.319+0000 7f289ffff640 1 -- 172.21.15.49:0/1491177433 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5562caa5eef0 msgr2=0x5562caa67fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:18:56.326 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:56.319+0000 7f289ffff640 1 --2- 172.21.15.49:0/1491177433 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5562caa5eef0 0x5562caa67fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:56.326 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:56.319+0000 7f289ffff640 1 -- 172.21.15.49:0/1491177433 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5562ca97dc60 con 0x5562caa685e0 2024-03-18T23:18:56.326 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:56.319+0000 7f289ffff640 1 --2- 172.21.15.49:0/1491177433 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5562caa685e0 0x5562caa65950 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f2898003ed0 tx=0x7f2898009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8248f7367f9aaa70 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:18:56.326 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:56.319+0000 7f287effd640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-18T23:18:56.326 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:56.319+0000 7f287effd640 1 -- 172.21.15.49:0/1491177433 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f289802d060 con 0x5562caa685e0 2024-03-18T23:18:56.327 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:56.319+0000 7f287effd640 1 -- 172.21.15.49:0/1491177433 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8556+0+0 (secure 0 0 0) 0x7f28980046d0 con 0x5562caa685e0 2024-03-18T23:18:56.327 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:56.319+0000 7f28a8976740 1 -- 172.21.15.49:0/1491177433 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2898004050 con 0x5562caa685e0 2024-03-18T23:18:56.327 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:56.319+0000 7f287effd640 1 -- 172.21.15.49:0/1491177433 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f289800f4e0 con 0x5562caa685e0 2024-03-18T23:18:56.327 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:56.319+0000 7f28a8976740 10 client.16314 fetch_fsmap learned FSMap version 32 2024-03-18T23:18:56.327 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:56.319+0000 7f28a8976740 1 -- 172.21.15.49:0/1491177433 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5562ca6d7210 con 0x5562caa685e0 2024-03-18T23:18:56.327 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:56.320+0000 7f287effd640 1 -- 172.21.15.49:0/1491177433 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2898004050 con 0x5562caa685e0 2024-03-18T23:18:56.327 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:56.320+0000 7f287effd640 1 -- 172.21.15.49:0/1491177433 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f2898025040 con 0x5562caa685e0 2024-03-18T23:18:56.327 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:56.320+0000 7f28a8976740 10 client.16314 fetch_fsmap finished waiting for FSMap version 32 2024-03-18T23:18:56.327 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:56.320+0000 7f28a8976740 10 client.16314 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T23:18:56.327 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:56.320+0000 7f28a8976740 20 client.16314 populate_metadata read hostname 'smithi049' 2024-03-18T23:18:56.327 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:56.320+0000 7f28a8976740 1 --2- 172.21.15.49:0/1491177433 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x5562caa783d0 0x5562caa7a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:18:56.327 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:56.320+0000 7f28a8976740 4 client.16314 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T23:18:56.327 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:56.320+0000 7f28a8976740 1 -- 172.21.15.49:0/1491177433 --> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5562ca9d0120 con 0x5562caa783d0 2024-03-18T23:18:56.327 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:56.320+0000 7f289effd640 1 --2- 172.21.15.49:0/1491177433 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x5562caa783d0 0x5562caa7a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:18:56.327 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:56.320+0000 7f287effd640 10 client.16314 ms_handle_connect on v2:172.21.15.139:6838/4280128209 2024-03-18T23:18:56.328 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:56.320+0000 7f289effd640 1 --2- 172.21.15.49:0/1491177433 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x5562caa783d0 0x5562caa7a7c0 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1fc3655e2b5b0b0d server_cookie=234a10e81a10428f in_seq=0 out_seq=0 2024-03-18T23:18:56.328 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:56.321+0000 7f287effd640 1 -- 172.21.15.49:0/1491177433 <== mds.0 v2:172.21.15.139:6838/4280128209 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5562ca9d0120 con 0x5562caa783d0 2024-03-18T23:18:56.328 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:56.321+0000 7f287effd640 10 client.16314 handle_command_reply: tid=0 2024-03-18T23:18:56.328 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:56.322+0000 7f28a8976740 1 client.16314 shutdown 2024-03-18T23:18:56.328 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:56.322+0000 7f28a8976740 1 -- 172.21.15.49:0/1491177433 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5562caa685e0 msgr2=0x5562caa65950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:18:56.328 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:56.322+0000 7f28a8976740 1 --2- 172.21.15.49:0/1491177433 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5562caa685e0 0x5562caa65950 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f2898003ed0 tx=0x7f2898009bd0 comp rx=0 tx=0).stop 2024-03-18T23:18:56.328 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:56.322+0000 7f28a8976740 1 -- 172.21.15.49:0/1491177433 shutdown_connections 2024-03-18T23:18:56.328 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:56.322+0000 7f28a8976740 1 --2- 172.21.15.49:0/1491177433 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x5562caa783d0 0x5562caa7a7c0 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:56.328 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:56.322+0000 7f28a8976740 1 --2- 172.21.15.49:0/1491177433 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5562caa685e0 0x5562caa65950 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:56.328 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:56.322+0000 7f28a8976740 1 --2- 172.21.15.49:0/1491177433 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5562caa5f810 0x5562caa65340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:56.328 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:56.322+0000 7f28a8976740 1 --2- 172.21.15.49:0/1491177433 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5562caa5eef0 0x5562caa67fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:56.328 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:56.322+0000 7f287effd640 0 client.16314 ms_handle_reset on v2:172.21.15.139:6838/4280128209 2024-03-18T23:18:56.328 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:56.322+0000 7f28a8976740 1 -- 172.21.15.49:0/1491177433 >> 172.21.15.49:0/1491177433 conn(0x5562caa59810 msgr2=0x5562caa58ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:18:56.328 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:56.322+0000 7f28a8976740 1 -- 172.21.15.49:0/1491177433 shutdown_connections 2024-03-18T23:18:56.329 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:56.322+0000 7f28a8976740 1 -- 172.21.15.49:0/1491177433 wait complete. 2024-03-18T23:18:56.329 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:56.322+0000 7f28a8976740 20 client.16314 trim_cache size 0 max 16384 2024-03-18T23:18:56.343 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:56.337+0000 7f28a8976740 1 Processor -- start 2024-03-18T23:18:56.343 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:56.337+0000 7f28a8976740 1 -- start start 2024-03-18T23:18:56.343 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:56.338+0000 7f28a8976740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5562caa5eef0 0x5562caa64c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:18:56.344 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:56.338+0000 7f28a8976740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5562caa5f810 0x5562caa7fe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:18:56.344 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:56.338+0000 7f28a8976740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5562caa685e0 0x5562caa67a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:18:56.344 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:56.338+0000 7f28a8976740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f28a01ad530 con 0x5562caa5f810 2024-03-18T23:18:56.344 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:56.338+0000 7f28a8976740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f28a01af070 con 0x5562caa5eef0 2024-03-18T23:18:56.344 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:56.338+0000 7f28a8976740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f28a01adc00 con 0x5562caa685e0 2024-03-18T23:18:56.344 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:56.338+0000 7f289effd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5562caa5f810 0x5562caa7fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:18:56.344 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:56.338+0000 7f289effd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5562caa5f810 0x5562caa7fe80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:57230/0 (socket says 172.21.15.49:57230) 2024-03-18T23:18:56.344 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:56.338+0000 7f289effd640 1 -- 172.21.15.49:0/907720547 learned_addr learned my addr 172.21.15.49:0/907720547 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:18:56.344 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:56.338+0000 7f289ffff640 1 --2- 172.21.15.49:0/907720547 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5562caa5eef0 0x5562caa64c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:18:56.344 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:56.338+0000 7f289ffff640 1 --2- 172.21.15.49:0/907720547 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5562caa685e0 0x5562caa67a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:18:56.345 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:56.338+0000 7f289effd640 1 -- 172.21.15.49:0/907720547 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5562caa685e0 msgr2=0x5562caa67a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:18:56.345 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:56.338+0000 7f289effd640 1 --2- 172.21.15.49:0/907720547 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5562caa685e0 0x5562caa67a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:56.345 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:56.338+0000 7f289effd640 1 -- 172.21.15.49:0/907720547 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5562caa5eef0 msgr2=0x5562caa64c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:18:56.345 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:56.338+0000 7f289effd640 1 --2- 172.21.15.49:0/907720547 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5562caa5eef0 0x5562caa64c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:56.345 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:56.338+0000 7f289effd640 1 -- 172.21.15.49:0/907720547 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5562caa5f640 con 0x5562caa5f810 2024-03-18T23:18:56.345 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:56.338+0000 7f289ffff640 1 --2- 172.21.15.49:0/907720547 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5562caa685e0 0x5562caa67a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T23:18:56.345 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:56.338+0000 7f289effd640 1 --2- 172.21.15.49:0/907720547 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5562caa5f810 0x5562caa7fe80 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f289402c170 tx=0x7f2894009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2889d7ca41609b server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:18:56.345 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:56.338+0000 7f28857fa640 1 -- 172.21.15.49:0/907720547 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2894004300 con 0x5562caa5f810 2024-03-18T23:18:56.345 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:56.338+0000 7f28857fa640 1 -- 172.21.15.49:0/907720547 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f289404f070 con 0x5562caa5f810 2024-03-18T23:18:56.345 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:56.338+0000 7f28857fa640 1 -- 172.21.15.49:0/907720547 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2894005120 con 0x5562caa5f810 2024-03-18T23:18:56.345 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:56.339+0000 7f28a8976740 1 -- 172.21.15.49:0/907720547 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5562caa5f810 msgr2=0x5562caa7fe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:18:56.345 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:56.339+0000 7f28a8976740 1 --2- 172.21.15.49:0/907720547 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5562caa5f810 0x5562caa7fe80 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f289402c170 tx=0x7f2894009260 comp rx=0 tx=0).stop 2024-03-18T23:18:56.345 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:56.339+0000 7f28a8976740 1 -- 172.21.15.49:0/907720547 shutdown_connections 2024-03-18T23:18:56.345 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:56.339+0000 7f28a8976740 1 --2- 172.21.15.49:0/907720547 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5562caa685e0 0x5562caa67a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:56.345 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:56.339+0000 7f28a8976740 1 --2- 172.21.15.49:0/907720547 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5562caa5f810 0x5562caa7fe80 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:56.346 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:56.339+0000 7f28a8976740 1 --2- 172.21.15.49:0/907720547 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5562caa5eef0 0x5562caa64c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:56.346 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:56.339+0000 7f28a8976740 1 -- 172.21.15.49:0/907720547 >> 172.21.15.49:0/907720547 conn(0x5562caa59810 msgr2=0x5562caa608d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:18:56.346 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:56.339+0000 7f28a8976740 1 -- 172.21.15.49:0/907720547 shutdown_connections 2024-03-18T23:18:56.346 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:56.339+0000 7f28a8976740 1 -- 172.21.15.49:0/907720547 wait complete. 2024-03-18T23:18:56.346 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:56.339+0000 7f28a8976740 1 -- start start 2024-03-18T23:18:56.346 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:56.340+0000 7f28a8976740 1 Processor -- start 2024-03-18T23:18:56.346 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:56.340+0000 7f28a8976740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5562caa5eef0 0x5562caa708d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:18:56.346 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:56.340+0000 7f28a8976740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5562caa5f810 0x5562caa61c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:18:56.346 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:56.340+0000 7f28a8976740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5562caa685e0 0x5562caa621f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:18:56.346 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:56.340+0000 7f28a8976740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f28a01ae9a0 con 0x5562caa685e0 2024-03-18T23:18:56.346 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:56.340+0000 7f28a8976740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f28a01ae2d0 con 0x5562caa5f810 2024-03-18T23:18:56.346 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:56.340+0000 7f28a8976740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f28a01ab530 con 0x5562caa5eef0 2024-03-18T23:18:56.346 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:56.340+0000 7f28a8976740 10 client.0 _ll_register_callbacks cb 0x7f28900d2b40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T23:18:56.347 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:56.340+0000 7f289ffff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5562caa685e0 0x5562caa621f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:18:56.347 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:56.340+0000 7f289ffff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5562caa685e0 0x5562caa621f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:57234/0 (socket says 172.21.15.49:57234) 2024-03-18T23:18:56.347 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:56.340+0000 7f289ffff640 1 -- 172.21.15.49:0/3585286627 learned_addr learned my addr 172.21.15.49:0/3585286627 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:18:56.347 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:56.340+0000 7f289effd640 1 --2- 172.21.15.49:0/3585286627 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5562caa5f810 0x5562caa61c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:18:56.347 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:56.340+0000 7f289ffff640 1 --2- 172.21.15.49:0/3585286627 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5562caa5eef0 0x5562caa708d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:18:56.347 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:56.340+0000 7f289ffff640 1 -- 172.21.15.49:0/3585286627 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5562caa5eef0 msgr2=0x5562caa708d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:18:56.347 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:56.340+0000 7f289ffff640 1 --2- 172.21.15.49:0/3585286627 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5562caa5eef0 0x5562caa708d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:56.347 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:56.340+0000 7f289ffff640 1 -- 172.21.15.49:0/3585286627 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5562caa5f810 msgr2=0x5562caa61c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:18:56.347 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:56.340+0000 7f289ffff640 1 --2- 172.21.15.49:0/3585286627 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5562caa5f810 0x5562caa61c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:56.347 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:56.340+0000 7f289ffff640 1 -- 172.21.15.49:0/3585286627 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5562ca7a3620 con 0x5562caa685e0 2024-03-18T23:18:56.347 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:56.340+0000 7f289ffff640 1 --2- 172.21.15.49:0/3585286627 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5562caa685e0 0x5562caa621f0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f2898009560 tx=0x7f2898005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8e1f447f1542f3c server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:18:56.348 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:56.340+0000 7f287effd640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-18T23:18:56.348 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:56.341+0000 7f287effd640 1 -- 172.21.15.49:0/3585286627 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f289802d070 con 0x5562caa685e0 2024-03-18T23:18:56.348 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:56.341+0000 7f287effd640 1 -- 172.21.15.49:0/3585286627 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8556+0+0 (secure 0 0 0) 0x7f28980044d0 con 0x5562caa685e0 2024-03-18T23:18:56.348 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:56.341+0000 7f28a8976740 1 -- 172.21.15.49:0/3585286627 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f289404f070 con 0x5562caa685e0 2024-03-18T23:18:56.348 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:56.341+0000 7f287effd640 1 -- 172.21.15.49:0/3585286627 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f2898019430 con 0x5562caa685e0 2024-03-18T23:18:56.348 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:56.341+0000 7f28a8976740 10 client.16320 fetch_fsmap learned FSMap version 32 2024-03-18T23:18:56.348 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:56.341+0000 7f28a8976740 1 -- 172.21.15.49:0/3585286627 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5562caade350 con 0x5562caa685e0 2024-03-18T23:18:56.348 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:56.341+0000 7f287effd640 1 -- 172.21.15.49:0/3585286627 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f289404f070 con 0x5562caa685e0 2024-03-18T23:18:56.348 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:56.341+0000 7f287effd640 1 -- 172.21.15.49:0/3585286627 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f2898019960 con 0x5562caa685e0 2024-03-18T23:18:56.348 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:56.341+0000 7f28a8976740 10 client.16320 fetch_fsmap finished waiting for FSMap version 32 2024-03-18T23:18:56.348 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:56.341+0000 7f28a8976740 10 client.16320 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T23:18:56.348 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:56.341+0000 7f28a8976740 20 client.16320 populate_metadata read hostname 'smithi049' 2024-03-18T23:18:56.348 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:56.341+0000 7f28a8976740 1 --2- 172.21.15.49:0/3585286627 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x5562caade9c0 0x5562caae0db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:18:56.348 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:56.341+0000 7f28a8976740 4 client.16320 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T23:18:56.348 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:56.341+0000 7f28a8976740 1 -- 172.21.15.49:0/3585286627 --> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5562ca9ca470 con 0x5562caade9c0 2024-03-18T23:18:56.349 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:56.341+0000 7f289effd640 1 --2- 172.21.15.49:0/3585286627 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x5562caade9c0 0x5562caae0db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:18:56.349 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:56.342+0000 7f287effd640 10 client.16320 ms_handle_connect on v2:172.21.15.139:6838/4280128209 2024-03-18T23:18:56.349 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:56.342+0000 7f289effd640 1 --2- 172.21.15.49:0/3585286627 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x5562caade9c0 0x5562caae0db0 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=40268110bf2e08ab server_cookie=5119333d5682a210 in_seq=0 out_seq=0 2024-03-18T23:18:56.525 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:56.524+0000 7f287effd640 1 -- 172.21.15.49:0/3585286627 <== mds.0 v2:172.21.15.139:6838/4280128209 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5562ca9ca470 con 0x5562caade9c0 2024-03-18T23:18:56.526 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:56.524+0000 7f287effd640 10 client.16320 handle_command_reply: tid=0 2024-03-18T23:18:56.526 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:56.524+0000 7f28a8976740 1 client.16320 shutdown 2024-03-18T23:18:56.526 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:56.525+0000 7f28a8976740 1 -- 172.21.15.49:0/3585286627 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5562caa685e0 msgr2=0x5562caa621f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:18:56.526 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:56.525+0000 7f28a8976740 1 --2- 172.21.15.49:0/3585286627 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5562caa685e0 0x5562caa621f0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f2898009560 tx=0x7f2898005bd0 comp rx=0 tx=0).stop 2024-03-18T23:18:56.526 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:56.525+0000 7f28a8976740 1 -- 172.21.15.49:0/3585286627 shutdown_connections 2024-03-18T23:18:56.526 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:56.525+0000 7f28a8976740 1 --2- 172.21.15.49:0/3585286627 >> [v2:172.21.15.139:6838/4280128209,v1:172.21.15.139:6839/4280128209] conn(0x5562caade9c0 0x5562caae0db0 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:56.526 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:56.525+0000 7f287effd640 0 client.16320 ms_handle_reset on v2:172.21.15.139:6838/4280128209 2024-03-18T23:18:56.526 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:56.525+0000 7f28a8976740 1 --2- 172.21.15.49:0/3585286627 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5562caa685e0 0x5562caa621f0 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:56.526 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:56.525+0000 7f28a8976740 1 --2- 172.21.15.49:0/3585286627 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5562caa5f810 0x5562caa61c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:56.526 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:56.525+0000 7f28a8976740 1 --2- 172.21.15.49:0/3585286627 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5562caa5eef0 0x5562caa708d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:56.526 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:56.525+0000 7f28a8976740 1 -- 172.21.15.49:0/3585286627 >> 172.21.15.49:0/3585286627 conn(0x5562caa59810 msgr2=0x5562caa5aaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:18:56.526 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:56.525+0000 7f28a8976740 1 -- 172.21.15.49:0/3585286627 shutdown_connections 2024-03-18T23:18:56.527 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:56.525+0000 7f28a8976740 1 -- 172.21.15.49:0/3585286627 wait complete. 2024-03-18T23:18:56.527 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:56.525+0000 7f28a8976740 20 client.16320 trim_cache size 0 max 16384 2024-03-18T23:18:56.527 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:{ 2024-03-18T23:18:56.527 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "message": "", 2024-03-18T23:18:56.527 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "return_code": 0 2024-03-18T23:18:56.527 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:} 2024-03-18T23:18:56.528 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:56.526+0000 7f28a5cb5640 1 -- 172.21.15.49:0/1057699488 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f28880a2a90 msgr2=0x7f28880a4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:18:56.528 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:56.526+0000 7f28a5cb5640 1 --2- 172.21.15.49:0/1057699488 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f28880a2a90 0x7f28880a4f50 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f28900cc900 tx=0x7f28900d2c70 comp rx=0 tx=0).stop 2024-03-18T23:18:56.528 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:56.526+0000 7f28a5cb5640 1 -- 172.21.15.49:0/1057699488 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f28a0100970 msgr2=0x7f28a01980d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:18:56.528 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:56.526+0000 7f28a5cb5640 1 --2- 172.21.15.49:0/1057699488 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f28a0100970 0x7f28a01980d0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f289000e9f0 tx=0x7f289000eeb0 comp rx=0 tx=0).stop 2024-03-18T23:18:56.528 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:56.526+0000 7f28a5cb5640 1 -- 172.21.15.49:0/1057699488 shutdown_connections 2024-03-18T23:18:56.529 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:56.526+0000 7f28a5cb5640 1 --2- 172.21.15.49:0/1057699488 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f28880a2a90 0x7f28880a4f50 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:56.529 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:56.527+0000 7f28a5cb5640 1 --2- 172.21.15.49:0/1057699488 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f28a01056c0 0x7f28a019c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:56.529 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:56.527+0000 7f28a5cb5640 1 --2- 172.21.15.49:0/1057699488 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f28a0101330 0x7f28a0198610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:56.529 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:56.527+0000 7f28a5cb5640 1 --2- 172.21.15.49:0/1057699488 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f28a0100970 0x7f28a01980d0 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:56.529 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:56.527+0000 7f28a5cb5640 1 -- 172.21.15.49:0/1057699488 >> 172.21.15.49:0/1057699488 conn(0x7f28a0077570 msgr2=0x7f28a0069730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:18:56.529 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:56.527+0000 7f28a5cb5640 1 -- 172.21.15.49:0/1057699488 shutdown_connections 2024-03-18T23:18:56.529 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2024-03-18T23:18:56.527+0000 7f28a5cb5640 1 -- 172.21.15.49:0/1057699488 wait complete. 2024-03-18T23:18:56.787 INFO:tasks.workunit.client.0.smithi049.stdout: 1 1094198 9.26 MB/sec execute 478 sec latency 129.750 ms 2024-03-18T23:18:57.431 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:18:57 smithi049 ceph-mon[62991]: pgmap v333: 97 pgs: 97 active+clean; 76 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.3 MiB/s wr, 228 op/s 2024-03-18T23:18:57.460 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:18:57 smithi175 ceph-mon[67352]: pgmap v333: 97 pgs: 97 active+clean; 76 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.3 MiB/s wr, 228 op/s 2024-03-18T23:18:57.564 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:18:57 smithi139 ceph-mon[67282]: pgmap v333: 97 pgs: 97 active+clean; 76 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.3 MiB/s wr, 228 op/s 2024-03-18T23:18:57.787 INFO:tasks.workunit.client.0.smithi049.stdout: 1 1095572 9.25 MB/sec execute 479 sec latency 23.678 ms 2024-03-18T23:18:58.787 INFO:tasks.workunit.client.0.smithi049.stdout: 1 1097379 9.25 MB/sec execute 480 sec latency 87.976 ms 2024-03-18T23:18:59.460 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:18:59 smithi175 ceph-mon[67352]: pgmap v334: 97 pgs: 97 active+clean; 77 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 3.5 MiB/s wr, 339 op/s 2024-03-18T23:18:59.564 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:18:59 smithi139 ceph-mon[67282]: pgmap v334: 97 pgs: 97 active+clean; 77 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 3.5 MiB/s wr, 339 op/s 2024-03-18T23:18:59.629 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:18:59 smithi049 ceph-mon[62991]: pgmap v334: 97 pgs: 97 active+clean; 77 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 3.5 MiB/s wr, 339 op/s 2024-03-18T23:18:59.787 INFO:tasks.workunit.client.0.smithi049.stdout: 1 1098159 9.24 MB/sec execute 481 sec latency 24.315 ms 2024-03-18T23:19:00.713 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-18T23:19:00.713 INFO:tasks.workunit.client.0.smithi049.stdout:[1099704] write failed on handle 11685 (Resource temporarily unavailable) 2024-03-18T23:19:00.713 INFO:tasks.workunit.client.0.smithi049.stdout:Child failed with status 1 2024-03-18T23:19:00.714 INFO:tasks.workunit:Stopping ['suites/dbench.sh'] on client.0... 2024-03-18T23:19:00.714 DEBUG:teuthology.orchestra.run.smithi049:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-03-18T23:19:01.017 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/github.com_batrick_ceph_e48b1f6c0608b2615e1890cc46286c3877d6e910/qa/tasks/workunit.py", line 145, in task _spawn_on_all_clients(ctx, refspec, all_tasks, config.get('env'), File "/home/teuthworker/src/github.com_batrick_ceph_e48b1f6c0608b2615e1890cc46286c3877d6e910/qa/tasks/workunit.py", line 295, in _spawn_on_all_clients p.spawn(_run_tests, ctx, refspec, role, [unit], env, File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/github.com_batrick_ceph_e48b1f6c0608b2615e1890cc46286c3877d6e910/qa/tasks/workunit.py", line 434, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test suites/dbench.sh) on smithi049 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=e48b1f6c0608b2615e1890cc46286c3877d6e910 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh' 2024-03-18T23:19:01.262 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=e291eef62e39418ab6f34355786da3bc Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/github.com_batrick_ceph_e48b1f6c0608b2615e1890cc46286c3877d6e910/qa/tasks/workunit.py", line 145, in task _spawn_on_all_clients(ctx, refspec, all_tasks, config.get('env'), File "/home/teuthworker/src/github.com_batrick_ceph_e48b1f6c0608b2615e1890cc46286c3877d6e910/qa/tasks/workunit.py", line 295, in _spawn_on_all_clients p.spawn(_run_tests, ctx, refspec, role, [unit], env, File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/github.com_batrick_ceph_e48b1f6c0608b2615e1890cc46286c3877d6e910/qa/tasks/workunit.py", line 434, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test suites/dbench.sh) on smithi049 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=e48b1f6c0608b2615e1890cc46286c3877d6e910 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh' 2024-03-18T23:19:01.265 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2024-03-18T23:19:01.275 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi049.front.sepia.ceph.com 2024-03-18T23:19:01.288 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:19:01 smithi049 ceph-mon[62991]: pgmap v335: 97 pgs: 97 active+clean; 77 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.4 MiB/s wr, 189 op/s 2024-03-18T23:19:01.289 DEBUG:teuthology.run_tasks:Unwinding manager full_sequential_finally 2024-03-18T23:19:01.315 INFO:teuthology.task.full_sequential_finally:In full_sequential_finally, running task exec... 2024-03-18T23:19:01.315 INFO:teuthology.task.exec:Executing custom commands... 2024-03-18T23:19:01.315 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi049.front.sepia.ceph.com 2024-03-18T23:19:01.316 DEBUG:teuthology.orchestra.run.smithi049:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > END_TIME' 2024-03-18T23:19:01.363 DEBUG:teuthology.orchestra.run.smithi049:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi' 2024-03-18T23:19:01.436 DEBUG:teuthology.orchestra.run.smithi049:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-03-18T23:19:01.460 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:19:01 smithi175 ceph-mon[67352]: pgmap v335: 97 pgs: 97 active+clean; 77 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.4 MiB/s wr, 189 op/s 2024-03-18T23:19:01.564 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:19:01 smithi139 ceph-mon[67282]: pgmap v335: 97 pgs: 97 active+clean; 77 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.4 MiB/s wr, 189 op/s 2024-03-18T23:19:01.589 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:01.587+0000 7f5087836640 1 -- 172.21.15.49:0/3788127505 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f508010be70 msgr2=0x7f508010e2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:19:01.589 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:01.587+0000 7f5087836640 1 --2- 172.21.15.49:0/3788127505 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f508010be70 0x7f508010e2e0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f5078009ca0 tx=0x7f507802c0f0 comp rx=0 tx=0).stop 2024-03-18T23:19:01.589 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:01.587+0000 7f5087836640 1 -- 172.21.15.49:0/3788127505 shutdown_connections 2024-03-18T23:19:01.590 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:01.587+0000 7f5087836640 1 --2- 172.21.15.49:0/3788127505 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f508010be70 0x7f508010e2e0 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:01.590 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:01.587+0000 7f5087836640 1 --2- 172.21.15.49:0/3788127505 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5080103320 0x7f508010b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:01.590 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:01.587+0000 7f5087836640 1 --2- 172.21.15.49:0/3788127505 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5080102a00 0x7f5080102de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:01.590 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:01.587+0000 7f5087836640 1 -- 172.21.15.49:0/3788127505 >> 172.21.15.49:0/3788127505 conn(0x7f5080077850 msgr2=0x7f5080077c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:19:01.590 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:01.587+0000 7f5087836640 1 -- 172.21.15.49:0/3788127505 shutdown_connections 2024-03-18T23:19:01.590 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:01.587+0000 7f5087836640 1 -- 172.21.15.49:0/3788127505 wait complete. 2024-03-18T23:19:01.590 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:01.588+0000 7f5087836640 1 Processor -- start 2024-03-18T23:19:01.590 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:01.588+0000 7f5087836640 1 -- start start 2024-03-18T23:19:01.591 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:01.588+0000 7f5087836640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5080102a00 0x7f5080197fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:19:01.591 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:01.588+0000 7f5087836640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5080103320 0x7f5080198500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:19:01.591 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:01.588+0000 7f5087836640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f508010be70 0x7f508019c880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:19:01.591 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:01.588+0000 7f5087836640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f5080111c40 con 0x7f5080102a00 2024-03-18T23:19:01.591 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:01.588+0000 7f5087836640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f5080111ac0 con 0x7f508010be70 2024-03-18T23:19:01.591 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:01.588+0000 7f5087836640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f5080111dc0 con 0x7f5080103320 2024-03-18T23:19:01.591 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:01.588+0000 7f50855d2640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5080102a00 0x7f5080197fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:19:01.591 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:01.588+0000 7f50855d2640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5080102a00 0x7f5080197fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:57264/0 (socket says 172.21.15.49:57264) 2024-03-18T23:19:01.591 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:01.588+0000 7f50855d2640 1 -- 172.21.15.49:0/1903851136 learned_addr learned my addr 172.21.15.49:0/1903851136 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:19:01.592 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:01.588+0000 7f5085dd3640 1 --2- 172.21.15.49:0/1903851136 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f508010be70 0x7f508019c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:19:01.592 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:01.588+0000 7f5084dd1640 1 --2- 172.21.15.49:0/1903851136 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5080103320 0x7f5080198500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:19:01.592 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:01.589+0000 7f50855d2640 1 -- 172.21.15.49:0/1903851136 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5080103320 msgr2=0x7f5080198500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:19:01.592 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:01.589+0000 7f50855d2640 1 --2- 172.21.15.49:0/1903851136 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5080103320 0x7f5080198500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:01.592 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:01.589+0000 7f50855d2640 1 -- 172.21.15.49:0/1903851136 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f508010be70 msgr2=0x7f508019c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:19:01.592 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:01.589+0000 7f50855d2640 1 --2- 172.21.15.49:0/1903851136 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f508010be70 0x7f508019c880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:01.592 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:01.589+0000 7f50855d2640 1 -- 172.21.15.49:0/1903851136 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f508019cfb0 con 0x7f5080102a00 2024-03-18T23:19:01.592 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:01.589+0000 7f50855d2640 1 --2- 172.21.15.49:0/1903851136 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5080102a00 0x7f5080197fc0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f507c00e9f0 tx=0x7f507c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:19:01.592 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:01.589+0000 7f50767fc640 1 -- 172.21.15.49:0/1903851136 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f507c00cd80 con 0x7f5080102a00 2024-03-18T23:19:01.593 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:01.589+0000 7f50767fc640 1 -- 172.21.15.49:0/1903851136 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f507c010910 con 0x7f5080102a00 2024-03-18T23:19:01.593 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:01.589+0000 7f5087836640 1 -- 172.21.15.49:0/1903851136 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f508019d2a0 con 0x7f5080102a00 2024-03-18T23:19:01.593 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:01.589+0000 7f50767fc640 1 -- 172.21.15.49:0/1903851136 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f507c018990 con 0x7f5080102a00 2024-03-18T23:19:01.593 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:01.589+0000 7f5087836640 1 -- 172.21.15.49:0/1903851136 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50801a4b30 con 0x7f5080102a00 2024-03-18T23:19:01.594 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:01.590+0000 7f5087836640 1 -- 172.21.15.49:0/1903851136 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f508006a8c0 con 0x7f5080102a00 2024-03-18T23:19:01.594 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:01.591+0000 7f50767fc640 1 -- 172.21.15.49:0/1903851136 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 25) ==== 137421+0+0 (secure 0 0 0) 0x7f507c010ac0 con 0x7f5080102a00 2024-03-18T23:19:01.594 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:01.592+0000 7f50767fc640 1 --2- 172.21.15.49:0/1903851136 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f506c0a2a10 0x7f506c0a4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:19:01.594 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:01.592+0000 7f5084dd1640 1 --2- 172.21.15.49:0/1903851136 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f506c0a2a10 0x7f506c0a4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:19:01.594 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:01.592+0000 7f50767fc640 1 -- 172.21.15.49:0/1903851136 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8556+0+0 (secure 0 0 0) 0x7f507c0cf120 con 0x7f5080102a00 2024-03-18T23:19:01.595 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:01.592+0000 7f5084dd1640 1 --2- 172.21.15.49:0/1903851136 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f506c0a2a10 0x7f506c0a4ed0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f5080100000 tx=0x7f5070005f90 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:19:01.596 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:01.596+0000 7f50767fc640 1 -- 172.21.15.49:0/1903851136 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f507c09b8b0 con 0x7f5080102a00 2024-03-18T23:19:01.802 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:01.802+0000 7f5087836640 1 -- 172.21.15.49:0/1903851136 --> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7f50800fde10 con 0x7f506c0a2a10 2024-03-18T23:19:01.810 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:01.810+0000 7f50767fc640 1 -- 172.21.15.49:0/1903851136 <== mgr.15366 v2:172.21.15.49:6800/1112476276 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+350 (secure 0 0 0) 0x7f50800fde10 con 0x7f506c0a2a10 2024-03-18T23:19:01.810 INFO:teuthology.orchestra.run.smithi049.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-03-18T00:00:00", "created": "2024-03-18T23:08:16", "first": "2024-03-18T23:09:00", "last": "2024-03-18T23:19:00", "last_pruned": "2024-03-18T23:19:00", "created_count": 11, "pruned_count": 5, "active": true} 2024-03-18T23:19:01.813 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:01.813+0000 7f5087836640 1 -- 172.21.15.49:0/1903851136 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f506c0a2a10 msgr2=0x7f506c0a4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:19:01.813 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:01.813+0000 7f5087836640 1 --2- 172.21.15.49:0/1903851136 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f506c0a2a10 0x7f506c0a4ed0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f5080100000 tx=0x7f5070005f90 comp rx=0 tx=0).stop 2024-03-18T23:19:01.813 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:01.813+0000 7f5087836640 1 -- 172.21.15.49:0/1903851136 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5080102a00 msgr2=0x7f5080197fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:19:01.813 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:01.813+0000 7f5087836640 1 --2- 172.21.15.49:0/1903851136 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5080102a00 0x7f5080197fc0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f507c00e9f0 tx=0x7f507c00eeb0 comp rx=0 tx=0).stop 2024-03-18T23:19:01.813 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:01.813+0000 7f5087836640 1 -- 172.21.15.49:0/1903851136 shutdown_connections 2024-03-18T23:19:01.813 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:01.813+0000 7f5087836640 1 --2- 172.21.15.49:0/1903851136 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f506c0a2a10 0x7f506c0a4ed0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:01.813 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:01.813+0000 7f5087836640 1 --2- 172.21.15.49:0/1903851136 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f508010be70 0x7f508019c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:01.814 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:01.813+0000 7f5087836640 1 --2- 172.21.15.49:0/1903851136 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5080103320 0x7f5080198500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:01.814 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:01.813+0000 7f5087836640 1 --2- 172.21.15.49:0/1903851136 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5080102a00 0x7f5080197fc0 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:01.814 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:01.813+0000 7f5087836640 1 -- 172.21.15.49:0/1903851136 >> 172.21.15.49:0/1903851136 conn(0x7f5080077850 msgr2=0x7f50800fd700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:19:01.814 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:01.813+0000 7f5087836640 1 -- 172.21.15.49:0/1903851136 shutdown_connections 2024-03-18T23:19:01.814 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:01.813+0000 7f5087836640 1 -- 172.21.15.49:0/1903851136 wait complete. 2024-03-18T23:19:01.824 DEBUG:teuthology.orchestra.run.smithi049:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-03-18T23:19:01.936 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:01.934+0000 7ff269dd4640 1 -- 172.21.15.49:0/292778931 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff264103320 msgr2=0x7ff26410b800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:19:01.937 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:01.934+0000 7ff269dd4640 1 --2- 172.21.15.49:0/292778931 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff264103320 0x7ff26410b800 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7ff254009900 tx=0x7ff25402bc30 comp rx=0 tx=0).stop 2024-03-18T23:19:01.937 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:01.934+0000 7ff269dd4640 1 -- 172.21.15.49:0/292778931 shutdown_connections 2024-03-18T23:19:01.937 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:01.934+0000 7ff269dd4640 1 --2- 172.21.15.49:0/292778931 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff26410be70 0x7ff26410e2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:01.937 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:01.934+0000 7ff269dd4640 1 --2- 172.21.15.49:0/292778931 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff264103320 0x7ff26410b800 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:01.937 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:01.934+0000 7ff269dd4640 1 --2- 172.21.15.49:0/292778931 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff264102a00 0x7ff264102de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:01.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:01.934+0000 7ff269dd4640 1 -- 172.21.15.49:0/292778931 >> 172.21.15.49:0/292778931 conn(0x7ff264077850 msgr2=0x7ff264077c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:19:01.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:01.934+0000 7ff269dd4640 1 -- 172.21.15.49:0/292778931 shutdown_connections 2024-03-18T23:19:01.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:01.934+0000 7ff269dd4640 1 -- 172.21.15.49:0/292778931 wait complete. 2024-03-18T23:19:01.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:01.934+0000 7ff269dd4640 1 Processor -- start 2024-03-18T23:19:01.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:01.935+0000 7ff269dd4640 1 -- start start 2024-03-18T23:19:01.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:01.935+0000 7ff269dd4640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff264102a00 0x7ff264198000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:19:01.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:01.935+0000 7ff269dd4640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff264103320 0x7ff264198540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:19:01.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:01.935+0000 7ff269dd4640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff26410be70 0x7ff26419c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:19:01.939 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:01.935+0000 7ff269dd4640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7ff264111c40 con 0x7ff264103320 2024-03-18T23:19:01.939 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:01.935+0000 7ff269dd4640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7ff264111ac0 con 0x7ff26410be70 2024-03-18T23:19:01.939 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:01.935+0000 7ff269dd4640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff264111dc0 con 0x7ff264102a00 2024-03-18T23:19:01.939 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:01.935+0000 7ff262ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff264103320 0x7ff264198540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:19:01.939 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:01.935+0000 7ff262ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff264103320 0x7ff264198540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:57282/0 (socket says 172.21.15.49:57282) 2024-03-18T23:19:01.939 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:01.935+0000 7ff262ffd640 1 -- 172.21.15.49:0/3724954442 learned_addr learned my addr 172.21.15.49:0/3724954442 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:19:01.939 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:01.935+0000 7ff2637fe640 1 --2- 172.21.15.49:0/3724954442 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff264102a00 0x7ff264198000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:19:01.940 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:01.935+0000 7ff263fff640 1 --2- 172.21.15.49:0/3724954442 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff26410be70 0x7ff26419c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:19:01.940 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:01.935+0000 7ff262ffd640 1 -- 172.21.15.49:0/3724954442 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff264102a00 msgr2=0x7ff264198000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:19:01.940 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:01.935+0000 7ff262ffd640 1 --2- 172.21.15.49:0/3724954442 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff264102a00 0x7ff264198000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:01.940 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:01.935+0000 7ff262ffd640 1 -- 172.21.15.49:0/3724954442 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff26410be70 msgr2=0x7ff26419c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:19:01.940 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:01.935+0000 7ff262ffd640 1 --2- 172.21.15.49:0/3724954442 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff26410be70 0x7ff26419c8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:01.940 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:01.935+0000 7ff262ffd640 1 -- 172.21.15.49:0/3724954442 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff26419cff0 con 0x7ff264103320 2024-03-18T23:19:01.940 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:01.935+0000 7ff262ffd640 1 --2- 172.21.15.49:0/3724954442 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff264103320 0x7ff264198540 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7ff254006010 tx=0x7ff25402cb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:19:01.940 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:01.935+0000 7ff260ff9640 1 -- 172.21.15.49:0/3724954442 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff254041070 con 0x7ff264103320 2024-03-18T23:19:01.941 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:01.935+0000 7ff260ff9640 1 -- 172.21.15.49:0/3724954442 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff254004450 con 0x7ff264103320 2024-03-18T23:19:01.941 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:01.935+0000 7ff269dd4640 1 -- 172.21.15.49:0/3724954442 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff26419d280 con 0x7ff264103320 2024-03-18T23:19:01.941 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:01.936+0000 7ff260ff9640 1 -- 172.21.15.49:0/3724954442 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff254003e00 con 0x7ff264103320 2024-03-18T23:19:01.941 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:01.936+0000 7ff269dd4640 1 -- 172.21.15.49:0/3724954442 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2641a4b70 con 0x7ff264103320 2024-03-18T23:19:01.941 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:01.937+0000 7ff269dd4640 1 -- 172.21.15.49:0/3724954442 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff26406a8c0 con 0x7ff264103320 2024-03-18T23:19:01.942 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:01.938+0000 7ff260ff9640 1 -- 172.21.15.49:0/3724954442 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 25) ==== 137421+0+0 (secure 0 0 0) 0x7ff254044070 con 0x7ff264103320 2024-03-18T23:19:01.942 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:01.938+0000 7ff260ff9640 1 --2- 172.21.15.49:0/3724954442 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7ff2500a2a90 0x7ff2500a4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:19:01.942 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:01.938+0000 7ff2637fe640 1 --2- 172.21.15.49:0/3724954442 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7ff2500a2a90 0x7ff2500a4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:19:01.943 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:01.939+0000 7ff260ff9640 1 -- 172.21.15.49:0/3724954442 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8556+0+0 (secure 0 0 0) 0x7ff2540f0960 con 0x7ff264103320 2024-03-18T23:19:01.943 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:01.939+0000 7ff2637fe640 1 --2- 172.21.15.49:0/3724954442 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7ff2500a2a90 0x7ff2500a4f50 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7ff24c009bd0 tx=0x7ff24c005f90 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:19:01.943 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:01.942+0000 7ff260ff9640 1 -- 172.21.15.49:0/3724954442 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7ff2540bd160 con 0x7ff264103320 2024-03-18T23:19:02.148 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:02.148+0000 7ff269dd4640 1 -- 172.21.15.49:0/3724954442 --> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7ff2640fddb0 con 0x7ff2500a2a90 2024-03-18T23:19:02.157 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:02.156+0000 7ff260ff9640 1 -- 172.21.15.49:0/3724954442 <== mgr.15366 v2:172.21.15.49:6800/1112476276 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7ff2640fddb0 con 0x7ff2500a2a90 2024-03-18T23:19:02.157 INFO:teuthology.orchestra.run.smithi049.stdout:/ 1m 6m3h 2024-03-18T23:19:02.159 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:02.159+0000 7ff269dd4640 1 -- 172.21.15.49:0/3724954442 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7ff2500a2a90 msgr2=0x7ff2500a4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:19:02.159 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:02.159+0000 7ff269dd4640 1 --2- 172.21.15.49:0/3724954442 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7ff2500a2a90 0x7ff2500a4f50 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7ff24c009bd0 tx=0x7ff24c005f90 comp rx=0 tx=0).stop 2024-03-18T23:19:02.159 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:02.159+0000 7ff269dd4640 1 -- 172.21.15.49:0/3724954442 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff264103320 msgr2=0x7ff264198540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:19:02.160 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:02.159+0000 7ff269dd4640 1 --2- 172.21.15.49:0/3724954442 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff264103320 0x7ff264198540 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7ff254006010 tx=0x7ff25402cb30 comp rx=0 tx=0).stop 2024-03-18T23:19:02.160 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:02.159+0000 7ff269dd4640 1 -- 172.21.15.49:0/3724954442 shutdown_connections 2024-03-18T23:19:02.160 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:02.159+0000 7ff269dd4640 1 --2- 172.21.15.49:0/3724954442 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7ff2500a2a90 0x7ff2500a4f50 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:02.160 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:02.159+0000 7ff269dd4640 1 --2- 172.21.15.49:0/3724954442 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff26410be70 0x7ff26419c8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:02.160 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:02.159+0000 7ff269dd4640 1 --2- 172.21.15.49:0/3724954442 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff264103320 0x7ff264198540 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:02.160 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:02.159+0000 7ff269dd4640 1 --2- 172.21.15.49:0/3724954442 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff264102a00 0x7ff264198000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:02.160 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:02.159+0000 7ff269dd4640 1 -- 172.21.15.49:0/3724954442 >> 172.21.15.49:0/3724954442 conn(0x7ff264077850 msgr2=0x7ff2640fd6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:19:02.160 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:02.159+0000 7ff269dd4640 1 -- 172.21.15.49:0/3724954442 shutdown_connections 2024-03-18T23:19:02.161 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:02.159+0000 7ff269dd4640 1 -- 172.21.15.49:0/3724954442 wait complete. 2024-03-18T23:19:02.170 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-03-18T23:19:02.181 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-18T23:19:02.291 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:02.291+0000 7f1bb29c6640 1 -- 172.21.15.49:0/989486860 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1bac1077f0 msgr2=0x7f1bac107bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:19:02.292 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:02.291+0000 7f1bb29c6640 1 --2- 172.21.15.49:0/989486860 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1bac1077f0 0x7f1bac107bd0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f1b9c009930 tx=0x7f1b9c02bc30 comp rx=0 tx=0).stop 2024-03-18T23:19:02.292 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:02.291+0000 7f1bb29c6640 1 -- 172.21.15.49:0/989486860 shutdown_connections 2024-03-18T23:19:02.292 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:02.291+0000 7f1bb29c6640 1 --2- 172.21.15.49:0/989486860 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1bac1021a0 0x7f1bac10e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:02.292 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:02.291+0000 7f1bb29c6640 1 --2- 172.21.15.49:0/989486860 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1bac1017e0 0x7f1bac101c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:02.292 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:02.291+0000 7f1bb29c6640 1 --2- 172.21.15.49:0/989486860 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1bac1077f0 0x7f1bac107bd0 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:02.292 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:02.291+0000 7f1bb29c6640 1 -- 172.21.15.49:0/989486860 >> 172.21.15.49:0/989486860 conn(0x7f1bac0fd290 msgr2=0x7f1bac0ff6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:19:02.292 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:02.291+0000 7f1bb29c6640 1 -- 172.21.15.49:0/989486860 shutdown_connections 2024-03-18T23:19:02.293 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:02.291+0000 7f1bb29c6640 1 -- 172.21.15.49:0/989486860 wait complete. 2024-03-18T23:19:02.293 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:02.292+0000 7f1bb29c6640 1 Processor -- start 2024-03-18T23:19:02.293 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:02.292+0000 7f1bb29c6640 1 -- start start 2024-03-18T23:19:02.295 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:02.292+0000 7f1bb29c6640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1bac1017e0 0x7f1bac198110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:19:02.295 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:02.292+0000 7f1bb29c6640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1bac1021a0 0x7f1bac198650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:19:02.295 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:02.292+0000 7f1bb29c6640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1bac1077f0 0x7f1bac19c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:19:02.295 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:02.292+0000 7f1bb29c6640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f1bac111c10 con 0x7f1bac1077f0 2024-03-18T23:19:02.295 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:02.292+0000 7f1bb29c6640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f1bac111a90 con 0x7f1bac1017e0 2024-03-18T23:19:02.295 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:02.292+0000 7f1bb29c6640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f1bac111d90 con 0x7f1bac1021a0 2024-03-18T23:19:02.296 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:02.292+0000 7f1bb0f63640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1bac1077f0 0x7f1bac19c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:19:02.296 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:02.292+0000 7f1bb0f63640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1bac1077f0 0x7f1bac19c9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:57294/0 (socket says 172.21.15.49:57294) 2024-03-18T23:19:02.296 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:02.292+0000 7f1bb0f63640 1 -- 172.21.15.49:0/4085472577 learned_addr learned my addr 172.21.15.49:0/4085472577 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:19:02.296 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:02.292+0000 7f1bab7fe640 1 --2- 172.21.15.49:0/4085472577 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1bac1021a0 0x7f1bac198650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:19:02.296 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:02.292+0000 7f1babfff640 1 --2- 172.21.15.49:0/4085472577 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1bac1017e0 0x7f1bac198110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:19:02.296 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:02.293+0000 7f1bb0f63640 1 -- 172.21.15.49:0/4085472577 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1bac1021a0 msgr2=0x7f1bac198650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:19:02.296 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:02.293+0000 7f1bb0f63640 1 --2- 172.21.15.49:0/4085472577 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1bac1021a0 0x7f1bac198650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:02.296 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:02.293+0000 7f1bb0f63640 1 -- 172.21.15.49:0/4085472577 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1bac1017e0 msgr2=0x7f1bac198110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:19:02.297 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:02.293+0000 7f1bb0f63640 1 --2- 172.21.15.49:0/4085472577 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1bac1017e0 0x7f1bac198110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:02.297 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:02.293+0000 7f1bb0f63640 1 -- 172.21.15.49:0/4085472577 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1bac19d100 con 0x7f1bac1077f0 2024-03-18T23:19:02.297 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:02.293+0000 7f1bb0f63640 1 --2- 172.21.15.49:0/4085472577 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1bac1077f0 0x7f1bac19c9d0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f1ba400eb10 tx=0x7f1ba400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:19:02.297 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:02.293+0000 7f1ba97fa640 1 -- 172.21.15.49:0/4085472577 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ba4004b20 con 0x7f1bac1077f0 2024-03-18T23:19:02.297 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:02.293+0000 7f1ba97fa640 1 -- 172.21.15.49:0/4085472577 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ba4004cc0 con 0x7f1bac1077f0 2024-03-18T23:19:02.297 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:02.293+0000 7f1bb29c6640 1 -- 172.21.15.49:0/4085472577 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1bac19d3f0 con 0x7f1bac1077f0 2024-03-18T23:19:02.297 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:02.293+0000 7f1ba97fa640 1 -- 172.21.15.49:0/4085472577 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ba4010760 con 0x7f1bac1077f0 2024-03-18T23:19:02.298 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:02.293+0000 7f1bb29c6640 1 -- 172.21.15.49:0/4085472577 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1bac1a4c80 con 0x7f1bac1077f0 2024-03-18T23:19:02.298 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:02.294+0000 7f1bb29c6640 1 -- 172.21.15.49:0/4085472577 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1bac068a60 con 0x7f1bac1077f0 2024-03-18T23:19:02.298 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:02.295+0000 7f1ba97fa640 1 -- 172.21.15.49:0/4085472577 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 25) ==== 137421+0+0 (secure 0 0 0) 0x7f1ba4005020 con 0x7f1bac1077f0 2024-03-18T23:19:02.298 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:02.296+0000 7f1ba97fa640 1 --2- 172.21.15.49:0/4085472577 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f1b940a2970 0x7f1b940a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:19:02.299 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:02.296+0000 7f1ba97fa640 1 -- 172.21.15.49:0/4085472577 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8556+0+0 (secure 0 0 0) 0x7f1ba40cf390 con 0x7f1bac1077f0 2024-03-18T23:19:02.299 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:02.296+0000 7f1babfff640 1 --2- 172.21.15.49:0/4085472577 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f1b940a2970 0x7f1b940a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:19:02.299 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:02.296+0000 7f1babfff640 1 --2- 172.21.15.49:0/4085472577 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f1b940a2970 0x7f1b940a4e30 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f1b9c009900 tx=0x7f1b9c02df90 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:19:02.300 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:02.300+0000 7f1ba97fa640 1 -- 172.21.15.49:0/4085472577 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f1ba409c0d0 con 0x7f1bac1077f0 2024-03-18T23:19:02.485 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:02.485+0000 7f1bb29c6640 1 -- 172.21.15.49:0/4085472577 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1bac1097c0 con 0x7f1bac1077f0 2024-03-18T23:19:02.486 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:02.487+0000 7f1ba97fa640 1 -- 172.21.15.49:0/4085472577 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v99) ==== 74+0+23201 (secure 0 0 0) 0x7f1ba409b9d0 con 0x7f1bac1077f0 2024-03-18T23:19:02.487 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-18T23:19:02.487 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":99,"fsid":"c63591ea-e57a-11ee-95c9-87774f69a715","created":"2024-03-18T22:57:58.734467+0000","modified":"2024-03-18T23:18:09.111053+0000","last_up_change":"2024-03-18T23:05:16.576310+0000","last_in_change":"2024-03-18T23:04:57.024067+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-18T23:01:28.363432+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-18T23:06:03.172963+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-18T23:06:03.973303+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"98","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":15,"snap_epoch":98,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e7fc9daa-4098-4ba5-8019-9aaef1aab3e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":1651162501},{"type":"v1","addr":"172.21.15.49:6803","nonce":1651162501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":1651162501},{"type":"v1","addr":"172.21.15.49:6805","nonce":1651162501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":1651162501},{"type":"v1","addr":"172.21.15.49:6809","nonce":1651162501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":1651162501},{"type":"v1","addr":"172.21.15.49:6807","nonce":1651162501}]},"public_addr":"172.21.15.49:6803/1651162501","cluster_addr":"172.21.15.49:6805/1651162501","heartbeat_back_addr":"172.21.15.49:6809/1651162501","heartbeat_front_addr":"172.21.15.49:6807/1651162501","state":["exists","up"]},{"osd":1,"uuid":"40ac929c-e90f-434c-a796-9aabd686f1fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":1766700359},{"type":"v1","addr":"172.21.15.139:6801","nonce":1766700359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":1766700359},{"type":"v1","addr":"172.21.15.139:6803","nonce":1766700359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":1766700359},{"type":"v1","addr":"172.21.15.139:6807","nonce":1766700359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":1766700359},{"type":"v1","addr":"172.21.15.139:6805","nonce":1766700359}]},"public_addr":"172.21.15.139:6801/1766700359","cluster_addr":"172.21.15.139:6803/1766700359","heartbeat_back_addr":"172.21.15.139:6807/1766700359","heartbeat_front_addr":"172.21.15.139:6805/1766700359","state":["exists","up"]},{"osd":2,"uuid":"8ae3b666-00b4-45ed-a5ee-4d0771edabb6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":3752671234},{"type":"v1","addr":"172.21.15.175:6801","nonce":3752671234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":3752671234},{"type":"v1","addr":"172.21.15.175:6803","nonce":3752671234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":3752671234},{"type":"v1","addr":"172.21.15.175:6807","nonce":3752671234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":3752671234},{"type":"v1","addr":"172.21.15.175:6805","nonce":3752671234}]},"public_addr":"172.21.15.175:6801/3752671234","cluster_addr":"172.21.15.175:6803/3752671234","heartbeat_back_addr":"172.21.15.175:6807/3752671234","heartbeat_front_addr":"172.21.15.175:6805/3752671234","state":["exists","up"]},{"osd":3,"uuid":"1a62e89d-45d4-4237-b813-7f96a59aca7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":956160536},{"type":"v1","addr":"172.21.15.49:6811","nonce":956160536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":956160536},{"type":"v1","addr":"172.21.15.49:6813","nonce":956160536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":956160536},{"type":"v1","addr":"172.21.15.49:6817","nonce":956160536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":956160536},{"type":"v1","addr":"172.21.15.49:6815","nonce":956160536}]},"public_addr":"172.21.15.49:6811/956160536","cluster_addr":"172.21.15.49:6813/956160536","heartbeat_back_addr":"172.21.15.49:6817/956160536","heartbeat_front_addr":"172.21.15.49:6815/956160536","state":["exists","up"]},{"osd":4,"uuid":"c715101c-0c3f-4b98-b142-d64a35e187df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":3129712668},{"type":"v1","addr":"172.21.15.139:6809","nonce":3129712668}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":3129712668},{"type":"v1","addr":"172.21.15.139:6811","nonce":3129712668}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":3129712668},{"type":"v1","addr":"172.21.15.139:6815","nonce":3129712668}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":3129712668},{"type":"v1","addr":"172.21.15.139:6813","nonce":3129712668}]},"public_addr":"172.21.15.139:6809/3129712668","cluster_addr":"172.21.15.139:6811/3129712668","heartbeat_back_addr":"172.21.15.139:6815/3129712668","heartbeat_front_addr":"172.21.15.139:6813/3129712668","state":["exists","up"]},{"osd":5,"uuid":"4b641f81-77eb-45fd-8464-511e7b3062b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":2774567215},{"type":"v1","addr":"172.21.15.175:6809","nonce":2774567215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":2774567215},{"type":"v1","addr":"172.21.15.175:6811","nonce":2774567215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":2774567215},{"type":"v1","addr":"172.21.15.175:6815","nonce":2774567215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":2774567215},{"type":"v1","addr":"172.21.15.175:6813","nonce":2774567215}]},"public_addr":"172.21.15.175:6809/2774567215","cluster_addr":"172.21.15.175:6811/2774567215","heartbeat_back_addr":"172.21.15.175:6815/2774567215","heartbeat_front_addr":"172.21.15.175:6813/2774567215","state":["exists","up"]},{"osd":6,"uuid":"653da8e0-1197-4b48-abed-470f2a9c4dce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":1472215461},{"type":"v1","addr":"172.21.15.49:6819","nonce":1472215461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":1472215461},{"type":"v1","addr":"172.21.15.49:6821","nonce":1472215461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":1472215461},{"type":"v1","addr":"172.21.15.49:6825","nonce":1472215461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":1472215461},{"type":"v1","addr":"172.21.15.49:6823","nonce":1472215461}]},"public_addr":"172.21.15.49:6819/1472215461","cluster_addr":"172.21.15.49:6821/1472215461","heartbeat_back_addr":"172.21.15.49:6825/1472215461","heartbeat_front_addr":"172.21.15.49:6823/1472215461","state":["exists","up"]},{"osd":7,"uuid":"ad157c9b-4b75-451c-8740-630f0975b7fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":2163986787},{"type":"v1","addr":"172.21.15.139:6817","nonce":2163986787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":2163986787},{"type":"v1","addr":"172.21.15.139:6819","nonce":2163986787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":2163986787},{"type":"v1","addr":"172.21.15.139:6823","nonce":2163986787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":2163986787},{"type":"v1","addr":"172.21.15.139:6821","nonce":2163986787}]},"public_addr":"172.21.15.139:6817/2163986787","cluster_addr":"172.21.15.139:6819/2163986787","heartbeat_back_addr":"172.21.15.139:6823/2163986787","heartbeat_front_addr":"172.21.15.139:6821/2163986787","state":["exists","up"]},{"osd":8,"uuid":"c8e75f15-df75-4d5d-9b92-a064bffb6c01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":1194026992},{"type":"v1","addr":"172.21.15.175:6817","nonce":1194026992}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1194026992},{"type":"v1","addr":"172.21.15.175:6819","nonce":1194026992}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":1194026992},{"type":"v1","addr":"172.21.15.175:6823","nonce":1194026992}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1194026992},{"type":"v1","addr":"172.21.15.175:6821","nonce":1194026992}]},"public_addr":"172.21.15.175:6817/1194026992","cluster_addr":"172.21.15.175:6819/1194026992","heartbeat_back_addr":"172.21.15.175:6823/1194026992","heartbeat_front_addr":"172.21.15.175:6821/1194026992","state":["exists","up"]},{"osd":9,"uuid":"edb5ffef-b6a2-4941-b5ab-40f982ad30d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":2742084884},{"type":"v1","addr":"172.21.15.49:6827","nonce":2742084884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":2742084884},{"type":"v1","addr":"172.21.15.49:6829","nonce":2742084884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":2742084884},{"type":"v1","addr":"172.21.15.49:6833","nonce":2742084884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":2742084884},{"type":"v1","addr":"172.21.15.49:6831","nonce":2742084884}]},"public_addr":"172.21.15.49:6827/2742084884","cluster_addr":"172.21.15.49:6829/2742084884","heartbeat_back_addr":"172.21.15.49:6833/2742084884","heartbeat_front_addr":"172.21.15.49:6831/2742084884","state":["exists","up"]},{"osd":10,"uuid":"3945e46d-766a-4c4a-8821-d43648dd1578","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":2404740201},{"type":"v1","addr":"172.21.15.139:6825","nonce":2404740201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":2404740201},{"type":"v1","addr":"172.21.15.139:6827","nonce":2404740201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":2404740201},{"type":"v1","addr":"172.21.15.139:6831","nonce":2404740201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":2404740201},{"type":"v1","addr":"172.21.15.139:6829","nonce":2404740201}]},"public_addr":"172.21.15.139:6825/2404740201","cluster_addr":"172.21.15.139:6827/2404740201","heartbeat_back_addr":"172.21.15.139:6831/2404740201","heartbeat_front_addr":"172.21.15.139:6829/2404740201","state":["exists","up"]},{"osd":11,"uuid":"9954ba15-d877-46cc-b24a-893f9168deb8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":3519503868},{"type":"v1","addr":"172.21.15.175:6825","nonce":3519503868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":3519503868},{"type":"v1","addr":"172.21.15.175:6827","nonce":3519503868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":3519503868},{"type":"v1","addr":"172.21.15.175:6831","nonce":3519503868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":3519503868},{"type":"v1","addr":"172.21.15.175:6829","nonce":3519503868}]},"public_addr":"172.21.15.175:6825/3519503868","cluster_addr":"172.21.15.175:6827/3519503868","heartbeat_back_addr":"172.21.15.175:6831/3519503868","heartbeat_front_addr":"172.21.15.175:6829/3519503868","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:00:38.367865+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:01:02.237687+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:01:25.609751+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:01:51.268572+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:02:16.514605+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:02:41.586523+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:03:07.029964+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:03:32.956250+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:03:57.547984+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:04:23.948972+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:04:49.667210+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:05:15.284693+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.6","mappings":[{"from":4,"to":6}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.49:0/1463571219":"2024-03-19T00:13:11.015881+0000","172.21.15.49:6800/2609458951":"2024-03-19T22:58:59.806253+0000","172.21.15.49:6801/1985190312":"2024-03-19T22:58:31.972474+0000","172.21.15.49:0/2176210039":"2024-03-19T22:58:31.972474+0000","172.21.15.49:0/1912446700":"2024-03-19T22:58:59.806253+0000","172.21.15.49:0/21283102":"2024-03-19T22:58:31.972474+0000","172.21.15.49:6801/2609458951":"2024-03-19T22:58:59.806253+0000","172.21.15.49:0/1036383534":"2024-03-19T22:58:59.806253+0000","172.21.15.49:0/563527416":"2024-03-19T22:58:59.806253+0000","172.21.15.49:6801/2083805211":"2024-03-19T23:08:15.312250+0000","172.21.15.49:0/1976299520":"2024-03-19T23:08:15.312250+0000","172.21.15.49:0/861926960":"2024-03-19T23:08:15.312250+0000","172.21.15.49:0/1292058922":"2024-03-19T22:58:31.972474+0000","172.21.15.49:0/94393705":"2024-03-19T23:08:15.312250+0000","172.21.15.49:6800/1985190312":"2024-03-19T22:58:31.972474+0000","172.21.15.49:6800/2083805211":"2024-03-19T23:08:15.312250+0000","172.21.15.49:0/3268662009":"2024-03-19T23:08:15.312250+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":5,"length":1},{"begin":15,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-18T23:19:02.489 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:02.489+0000 7f1bb29c6640 1 -- 172.21.15.49:0/4085472577 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f1b940a2970 msgr2=0x7f1b940a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:19:02.489 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:02.489+0000 7f1bb29c6640 1 --2- 172.21.15.49:0/4085472577 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f1b940a2970 0x7f1b940a4e30 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f1b9c009900 tx=0x7f1b9c02df90 comp rx=0 tx=0).stop 2024-03-18T23:19:02.489 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:02.489+0000 7f1bb29c6640 1 -- 172.21.15.49:0/4085472577 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1bac1077f0 msgr2=0x7f1bac19c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:19:02.489 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:02.489+0000 7f1bb29c6640 1 --2- 172.21.15.49:0/4085472577 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1bac1077f0 0x7f1bac19c9d0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f1ba400eb10 tx=0x7f1ba400c2d0 comp rx=0 tx=0).stop 2024-03-18T23:19:02.489 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:02.489+0000 7f1bb29c6640 1 -- 172.21.15.49:0/4085472577 shutdown_connections 2024-03-18T23:19:02.489 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:02.489+0000 7f1bb29c6640 1 --2- 172.21.15.49:0/4085472577 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f1b940a2970 0x7f1b940a4e30 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:02.490 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:02.489+0000 7f1bb29c6640 1 --2- 172.21.15.49:0/4085472577 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1bac1077f0 0x7f1bac19c9d0 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:02.490 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:02.489+0000 7f1bb29c6640 1 --2- 172.21.15.49:0/4085472577 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1bac1021a0 0x7f1bac198650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:02.490 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:02.489+0000 7f1bb29c6640 1 --2- 172.21.15.49:0/4085472577 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1bac1017e0 0x7f1bac198110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:02.490 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:02.489+0000 7f1bb29c6640 1 -- 172.21.15.49:0/4085472577 >> 172.21.15.49:0/4085472577 conn(0x7f1bac0fd290 msgr2=0x7f1bac0fdad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:19:02.490 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:02.489+0000 7f1bb29c6640 1 -- 172.21.15.49:0/4085472577 shutdown_connections 2024-03-18T23:19:02.490 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:02.489+0000 7f1bb29c6640 1 -- 172.21.15.49:0/4085472577 wait complete. 2024-03-18T23:19:02.501 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-18T23:01:28.363432+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-18T23:06:03.172963+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-18T23:06:03.973303+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '98', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 15, 'snap_epoch': 98, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-18T23:19:02.501 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-18T23:19:02.615 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:02.612+0000 7fb828cd1640 1 -- 172.21.15.49:0/682226266 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb824100690 msgr2=0x7fb824100b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:19:02.615 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:02.612+0000 7fb828cd1640 1 --2- 172.21.15.49:0/682226266 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb824100690 0x7fb824100b10 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fb80c009930 tx=0x7fb80c02bc30 comp rx=0 tx=0).stop 2024-03-18T23:19:02.615 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:02.613+0000 7fb828cd1640 1 -- 172.21.15.49:0/682226266 shutdown_connections 2024-03-18T23:19:02.615 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:02.613+0000 7fb828cd1640 1 --2- 172.21.15.49:0/682226266 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb824101050 0x7fb82410d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:02.615 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:02.613+0000 7fb828cd1640 1 --2- 172.21.15.49:0/682226266 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb824100690 0x7fb824100b10 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:02.615 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:02.613+0000 7fb828cd1640 1 --2- 172.21.15.49:0/682226266 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb8241066a0 0x7fb824106a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:02.616 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:02.613+0000 7fb828cd1640 1 -- 172.21.15.49:0/682226266 >> 172.21.15.49:0/682226266 conn(0x7fb824077820 msgr2=0x7fb824077c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:19:02.616 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:02.613+0000 7fb828cd1640 1 -- 172.21.15.49:0/682226266 shutdown_connections 2024-03-18T23:19:02.616 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:02.613+0000 7fb828cd1640 1 -- 172.21.15.49:0/682226266 wait complete. 2024-03-18T23:19:02.616 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:02.613+0000 7fb828cd1640 1 Processor -- start 2024-03-18T23:19:02.616 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:02.613+0000 7fb828cd1640 1 -- start start 2024-03-18T23:19:02.616 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:02.614+0000 7fb828cd1640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb824100690 0x7fb8241980d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:19:02.616 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:02.614+0000 7fb828cd1640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb824101050 0x7fb824198610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:19:02.616 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:02.614+0000 7fb828cd1640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb8241066a0 0x7fb82419c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:19:02.617 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:02.614+0000 7fb828cd1640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fb824111bb0 con 0x7fb8241066a0 2024-03-18T23:19:02.617 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:02.614+0000 7fb828cd1640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fb824111a30 con 0x7fb824101050 2024-03-18T23:19:02.617 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:02.614+0000 7fb828cd1640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb824111d30 con 0x7fb824100690 2024-03-18T23:19:02.617 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:02.614+0000 7fb822d9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb8241066a0 0x7fb82419c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:19:02.617 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:02.614+0000 7fb82259c640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb824100690 0x7fb8241980d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:19:02.617 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:02.614+0000 7fb822d9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb8241066a0 0x7fb82419c990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:57300/0 (socket says 172.21.15.49:57300) 2024-03-18T23:19:02.618 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:02.614+0000 7fb822d9d640 1 -- 172.21.15.49:0/2570032439 learned_addr learned my addr 172.21.15.49:0/2570032439 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:19:02.618 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:02.614+0000 7fb821d9b640 1 --2- 172.21.15.49:0/2570032439 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb824101050 0x7fb824198610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:19:02.618 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:02.614+0000 7fb822d9d640 1 -- 172.21.15.49:0/2570032439 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb824100690 msgr2=0x7fb8241980d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:19:02.618 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:02.614+0000 7fb822d9d640 1 --2- 172.21.15.49:0/2570032439 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb824100690 0x7fb8241980d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:02.618 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:02.614+0000 7fb822d9d640 1 -- 172.21.15.49:0/2570032439 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb824101050 msgr2=0x7fb824198610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:19:02.618 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:02.614+0000 7fb822d9d640 1 --2- 172.21.15.49:0/2570032439 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb824101050 0x7fb824198610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:02.618 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:02.614+0000 7fb822d9d640 1 -- 172.21.15.49:0/2570032439 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb82419d0c0 con 0x7fb8241066a0 2024-03-18T23:19:02.618 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:02.614+0000 7fb822d9d640 1 --2- 172.21.15.49:0/2570032439 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb8241066a0 0x7fb82419c990 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fb81400eb10 tx=0x7fb81400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:19:02.618 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:02.614+0000 7fb81b7fe640 1 -- 172.21.15.49:0/2570032439 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb814004b20 con 0x7fb8241066a0 2024-03-18T23:19:02.619 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:02.614+0000 7fb828cd1640 1 -- 172.21.15.49:0/2570032439 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb82419d3b0 con 0x7fb8241066a0 2024-03-18T23:19:02.619 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:02.614+0000 7fb81b7fe640 1 -- 172.21.15.49:0/2570032439 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb814004cc0 con 0x7fb8241066a0 2024-03-18T23:19:02.619 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:02.614+0000 7fb828cd1640 1 -- 172.21.15.49:0/2570032439 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8241a4c40 con 0x7fb8241066a0 2024-03-18T23:19:02.619 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:02.614+0000 7fb81b7fe640 1 -- 172.21.15.49:0/2570032439 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb814010760 con 0x7fb8241066a0 2024-03-18T23:19:02.620 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:02.616+0000 7fb828cd1640 1 -- 172.21.15.49:0/2570032439 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb824068610 con 0x7fb8241066a0 2024-03-18T23:19:02.620 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:02.617+0000 7fb81b7fe640 1 -- 172.21.15.49:0/2570032439 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 25) ==== 137421+0+0 (secure 0 0 0) 0x7fb814005000 con 0x7fb8241066a0 2024-03-18T23:19:02.620 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:02.617+0000 7fb81b7fe640 1 --2- 172.21.15.49:0/2570032439 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7fb8100a2970 0x7fb8100a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:19:02.620 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:02.617+0000 7fb82259c640 1 --2- 172.21.15.49:0/2570032439 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7fb8100a2970 0x7fb8100a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:19:02.620 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:02.617+0000 7fb81b7fe640 1 -- 172.21.15.49:0/2570032439 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8556+0+0 (secure 0 0 0) 0x7fb8140cfb40 con 0x7fb8241066a0 2024-03-18T23:19:02.621 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:02.618+0000 7fb82259c640 1 --2- 172.21.15.49:0/2570032439 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7fb8100a2970 0x7fb8100a4e30 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fb81c009b50 tx=0x7fb81c005f90 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:19:02.621 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:02.621+0000 7fb81b7fe640 1 -- 172.21.15.49:0/2570032439 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fb81409c880 con 0x7fb8241066a0 2024-03-18T23:19:02.804 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:02.803+0000 7fb828cd1640 1 -- 172.21.15.49:0/2570032439 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fb824108c60 con 0x7fb8241066a0 2024-03-18T23:19:02.804 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:02.804+0000 7fb81b7fe640 1 -- 172.21.15.49:0/2570032439 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v99) ==== 93+0+10 (secure 0 0 0) 0x7fb81409c180 con 0x7fb8241066a0 2024-03-18T23:19:02.804 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 1 2024-03-18T23:19:02.806 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:02.806+0000 7fb828cd1640 1 -- 172.21.15.49:0/2570032439 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7fb8100a2970 msgr2=0x7fb8100a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:19:02.806 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:02.806+0000 7fb828cd1640 1 --2- 172.21.15.49:0/2570032439 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7fb8100a2970 0x7fb8100a4e30 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fb81c009b50 tx=0x7fb81c005f90 comp rx=0 tx=0).stop 2024-03-18T23:19:02.806 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:02.806+0000 7fb828cd1640 1 -- 172.21.15.49:0/2570032439 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb8241066a0 msgr2=0x7fb82419c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:19:02.806 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:02.806+0000 7fb828cd1640 1 --2- 172.21.15.49:0/2570032439 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb8241066a0 0x7fb82419c990 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fb81400eb10 tx=0x7fb81400c2d0 comp rx=0 tx=0).stop 2024-03-18T23:19:02.806 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:02.806+0000 7fb828cd1640 1 -- 172.21.15.49:0/2570032439 shutdown_connections 2024-03-18T23:19:02.807 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:02.806+0000 7fb828cd1640 1 --2- 172.21.15.49:0/2570032439 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7fb8100a2970 0x7fb8100a4e30 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:02.807 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:02.806+0000 7fb828cd1640 1 --2- 172.21.15.49:0/2570032439 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb8241066a0 0x7fb82419c990 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:02.807 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:02.806+0000 7fb828cd1640 1 --2- 172.21.15.49:0/2570032439 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb824101050 0x7fb824198610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:02.807 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:02.806+0000 7fb828cd1640 1 --2- 172.21.15.49:0/2570032439 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb824100690 0x7fb8241980d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:02.807 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:02.806+0000 7fb828cd1640 1 -- 172.21.15.49:0/2570032439 >> 172.21.15.49:0/2570032439 conn(0x7fb824077820 msgr2=0x7fb824102ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:19:02.807 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:02.806+0000 7fb828cd1640 1 -- 172.21.15.49:0/2570032439 shutdown_connections 2024-03-18T23:19:02.807 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:02.806+0000 7fb828cd1640 1 -- 172.21.15.49:0/2570032439 wait complete. 2024-03-18T23:19:02.817 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-18T23:19:02.930 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:02.928+0000 7f33bd2c6640 1 -- 172.21.15.49:0/2009828007 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f33b8101100 msgr2=0x7f33b810d610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:19:02.930 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:02.928+0000 7f33bd2c6640 1 --2- 172.21.15.49:0/2009828007 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f33b8101100 0x7f33b810d610 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f33a8009cd0 tx=0x7f33a802c0f0 comp rx=0 tx=0).stop 2024-03-18T23:19:02.931 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:02.928+0000 7f33bd2c6640 1 -- 172.21.15.49:0/2009828007 shutdown_connections 2024-03-18T23:19:02.931 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:02.928+0000 7f33bd2c6640 1 --2- 172.21.15.49:0/2009828007 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f33b8101100 0x7f33b810d610 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:02.931 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:02.928+0000 7f33bd2c6640 1 --2- 172.21.15.49:0/2009828007 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f33b8100740 0x7f33b8100bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:02.931 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:02.928+0000 7f33bd2c6640 1 --2- 172.21.15.49:0/2009828007 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f33b8106750 0x7f33b8106b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:02.931 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:02.928+0000 7f33bd2c6640 1 -- 172.21.15.49:0/2009828007 >> 172.21.15.49:0/2009828007 conn(0x7f33b8077880 msgr2=0x7f33b8077c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:19:02.931 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:02.928+0000 7f33bd2c6640 1 -- 172.21.15.49:0/2009828007 shutdown_connections 2024-03-18T23:19:02.931 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:02.928+0000 7f33bd2c6640 1 -- 172.21.15.49:0/2009828007 wait complete. 2024-03-18T23:19:02.932 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:02.929+0000 7f33bd2c6640 1 Processor -- start 2024-03-18T23:19:02.932 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:02.929+0000 7f33bd2c6640 1 -- start start 2024-03-18T23:19:02.932 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:02.929+0000 7f33bd2c6640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f33b8100740 0x7f33b8198ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:19:02.932 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:02.929+0000 7f33bd2c6640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f33b8101100 0x7f33b81993e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:19:02.932 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:02.929+0000 7f33bd2c6640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f33b8106750 0x7f33b81a3f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:19:02.932 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:02.929+0000 7f33bd2c6640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f33b8110b60 con 0x7f33b8106750 2024-03-18T23:19:02.932 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:02.929+0000 7f33bd2c6640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f33b81109e0 con 0x7f33b8101100 2024-03-18T23:19:02.933 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:02.929+0000 7f33bd2c6640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f33b8110ce0 con 0x7f33b8100740 2024-03-18T23:19:02.933 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:02.929+0000 7f33b77fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f33b8106750 0x7f33b81a3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:19:02.933 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:02.929+0000 7f33b77fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f33b8106750 0x7f33b81a3f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:57326/0 (socket says 172.21.15.49:57326) 2024-03-18T23:19:02.933 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:02.929+0000 7f33b77fe640 1 -- 172.21.15.49:0/3793298975 learned_addr learned my addr 172.21.15.49:0/3793298975 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:19:02.933 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:02.929+0000 7f33b6ffd640 1 --2- 172.21.15.49:0/3793298975 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f33b8100740 0x7f33b8198ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:19:02.933 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:02.929+0000 7f33b67fc640 1 --2- 172.21.15.49:0/3793298975 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f33b8101100 0x7f33b81993e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:19:02.933 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:02.930+0000 7f33b77fe640 1 -- 172.21.15.49:0/3793298975 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f33b8100740 msgr2=0x7f33b8198ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:19:02.933 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:02.930+0000 7f33b77fe640 1 --2- 172.21.15.49:0/3793298975 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f33b8100740 0x7f33b8198ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:02.934 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:02.930+0000 7f33b77fe640 1 -- 172.21.15.49:0/3793298975 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f33b8101100 msgr2=0x7f33b81993e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:19:02.934 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:02.930+0000 7f33b77fe640 1 --2- 172.21.15.49:0/3793298975 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f33b8101100 0x7f33b81993e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:02.934 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:02.930+0000 7f33b77fe640 1 -- 172.21.15.49:0/3793298975 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33b81a4460 con 0x7f33b8106750 2024-03-18T23:19:02.934 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:02.930+0000 7f33b6ffd640 1 --2- 172.21.15.49:0/3793298975 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f33b8100740 0x7f33b8198ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:19:02.934 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:02.930+0000 7f33b77fe640 1 --2- 172.21.15.49:0/3793298975 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f33b8106750 0x7f33b81a3f20 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f33a80348f0 tx=0x7f33a8032c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:19:02.934 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:02.930+0000 7f33affff640 1 -- 172.21.15.49:0/3793298975 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33a8041020 con 0x7f33b8106750 2024-03-18T23:19:02.934 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:02.930+0000 7f33affff640 1 -- 172.21.15.49:0/3793298975 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f33a8032e80 con 0x7f33b8106750 2024-03-18T23:19:02.935 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:02.930+0000 7f33bd2c6640 1 -- 172.21.15.49:0/3793298975 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33b81a4630 con 0x7f33b8106750 2024-03-18T23:19:02.935 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:02.930+0000 7f33affff640 1 -- 172.21.15.49:0/3793298975 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33a8039040 con 0x7f33b8106750 2024-03-18T23:19:02.935 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:02.930+0000 7f33bd2c6640 1 -- 172.21.15.49:0/3793298975 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33b81a4af0 con 0x7f33b8106750 2024-03-18T23:19:02.936 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:02.931+0000 7f33bd2c6640 1 -- 172.21.15.49:0/3793298975 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f33b8068a60 con 0x7f33b8106750 2024-03-18T23:19:02.936 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:02.932+0000 7f33affff640 1 -- 172.21.15.49:0/3793298975 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 25) ==== 137421+0+0 (secure 0 0 0) 0x7f33a8044020 con 0x7f33b8106750 2024-03-18T23:19:02.936 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:02.933+0000 7f33affff640 1 --2- 172.21.15.49:0/3793298975 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f33a40a2970 0x7f33a40a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:19:02.936 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:02.933+0000 7f33b6ffd640 1 --2- 172.21.15.49:0/3793298975 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f33a40a2970 0x7f33a40a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:19:02.936 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:02.933+0000 7f33affff640 1 -- 172.21.15.49:0/3793298975 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8556+0+0 (secure 0 0 0) 0x7f33a80f0920 con 0x7f33b8106750 2024-03-18T23:19:02.936 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:02.933+0000 7f33b6ffd640 1 --2- 172.21.15.49:0/3793298975 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f33a40a2970 0x7f33a40a4e30 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f33b0009f40 tx=0x7f33b0005f90 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:19:02.937 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:02.937+0000 7f33affff640 1 -- 172.21.15.49:0/3793298975 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f33a804b020 con 0x7f33b8106750 2024-03-18T23:19:03.120 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:03.120+0000 7f33bd2c6640 1 -- 172.21.15.49:0/3793298975 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f33b8108cb0 con 0x7f33b8106750 2024-03-18T23:19:03.120 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:03.120+0000 7f33affff640 1 -- 172.21.15.49:0/3793298975 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v99) ==== 104+0+11 (secure 0 0 0) 0x7f33a80bd770 con 0x7f33b8106750 2024-03-18T23:19:03.121 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 32 2024-03-18T23:19:03.123 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:03.122+0000 7f33bd2c6640 1 -- 172.21.15.49:0/3793298975 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f33a40a2970 msgr2=0x7f33a40a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:19:03.123 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:03.122+0000 7f33bd2c6640 1 --2- 172.21.15.49:0/3793298975 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f33a40a2970 0x7f33a40a4e30 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f33b0009f40 tx=0x7f33b0005f90 comp rx=0 tx=0).stop 2024-03-18T23:19:03.123 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:03.122+0000 7f33bd2c6640 1 -- 172.21.15.49:0/3793298975 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f33b8106750 msgr2=0x7f33b81a3f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:19:03.123 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:03.122+0000 7f33bd2c6640 1 --2- 172.21.15.49:0/3793298975 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f33b8106750 0x7f33b81a3f20 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f33a80348f0 tx=0x7f33a8032c90 comp rx=0 tx=0).stop 2024-03-18T23:19:03.123 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:03.123+0000 7f33bd2c6640 1 -- 172.21.15.49:0/3793298975 shutdown_connections 2024-03-18T23:19:03.123 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:03.123+0000 7f33bd2c6640 1 --2- 172.21.15.49:0/3793298975 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f33a40a2970 0x7f33a40a4e30 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:03.124 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:03.123+0000 7f33bd2c6640 1 --2- 172.21.15.49:0/3793298975 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f33b8106750 0x7f33b81a3f20 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:03.124 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:03.123+0000 7f33bd2c6640 1 --2- 172.21.15.49:0/3793298975 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f33b8101100 0x7f33b81993e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:03.124 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:03.123+0000 7f33bd2c6640 1 --2- 172.21.15.49:0/3793298975 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f33b8100740 0x7f33b8198ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:03.124 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:03.123+0000 7f33bd2c6640 1 -- 172.21.15.49:0/3793298975 >> 172.21.15.49:0/3793298975 conn(0x7f33b8077880 msgr2=0x7f33b8103310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:19:03.124 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:03.123+0000 7f33bd2c6640 1 -- 172.21.15.49:0/3793298975 shutdown_connections 2024-03-18T23:19:03.124 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:03.123+0000 7f33bd2c6640 1 -- 172.21.15.49:0/3793298975 wait complete. 2024-03-18T23:19:03.134 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-18T23:19:03.249 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:03.247+0000 7f46806ea640 1 -- 172.21.15.49:0/2438782521 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4678102180 msgr2=0x7f467810e690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:19:03.249 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:03.247+0000 7f46806ea640 1 --2- 172.21.15.49:0/2438782521 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4678102180 0x7f467810e690 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f4670009cd0 tx=0x7f467002c0f0 comp rx=0 tx=0).stop 2024-03-18T23:19:03.249 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:03.247+0000 7f46806ea640 1 -- 172.21.15.49:0/2438782521 shutdown_connections 2024-03-18T23:19:03.249 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:03.247+0000 7f46806ea640 1 --2- 172.21.15.49:0/2438782521 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4678102180 0x7f467810e690 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:03.249 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:03.247+0000 7f46806ea640 1 --2- 172.21.15.49:0/2438782521 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f46781017c0 0x7f4678101c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:03.250 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:03.247+0000 7f46806ea640 1 --2- 172.21.15.49:0/2438782521 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f46781077d0 0x7f4678107bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:03.250 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:03.247+0000 7f46806ea640 1 -- 172.21.15.49:0/2438782521 >> 172.21.15.49:0/2438782521 conn(0x7f46780fd250 msgr2=0x7f46780ff670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:19:03.250 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:03.247+0000 7f46806ea640 1 -- 172.21.15.49:0/2438782521 shutdown_connections 2024-03-18T23:19:03.250 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:03.247+0000 7f46806ea640 1 -- 172.21.15.49:0/2438782521 wait complete. 2024-03-18T23:19:03.250 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:03.248+0000 7f46806ea640 1 Processor -- start 2024-03-18T23:19:03.250 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:03.248+0000 7f46806ea640 1 -- start start 2024-03-18T23:19:03.250 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:03.248+0000 7f46806ea640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f46781017c0 0x7f4678073b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:19:03.250 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:03.248+0000 7f46806ea640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4678102180 0x7f4678074060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:19:03.251 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:03.248+0000 7f46806ea640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f46781077d0 0x7f46780745a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:19:03.251 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:03.248+0000 7f46806ea640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f4678111be0 con 0x7f46781017c0 2024-03-18T23:19:03.251 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:03.248+0000 7f46806ea640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f4678111a60 con 0x7f46781077d0 2024-03-18T23:19:03.251 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:03.248+0000 7f46806ea640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f4678111d60 con 0x7f4678102180 2024-03-18T23:19:03.251 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:03.248+0000 7f467e486640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f46781017c0 0x7f4678073b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:19:03.251 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:03.248+0000 7f467e486640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f46781017c0 0x7f4678073b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:57344/0 (socket says 172.21.15.49:57344) 2024-03-18T23:19:03.251 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:03.248+0000 7f467e486640 1 -- 172.21.15.49:0/2217512547 learned_addr learned my addr 172.21.15.49:0/2217512547 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:19:03.251 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:03.248+0000 7f467dc85640 1 --2- 172.21.15.49:0/2217512547 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4678102180 0x7f4678074060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:19:03.251 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:03.248+0000 7f467ec87640 1 --2- 172.21.15.49:0/2217512547 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f46781077d0 0x7f46780745a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:19:03.252 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:03.248+0000 7f467e486640 1 -- 172.21.15.49:0/2217512547 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4678102180 msgr2=0x7f4678074060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:19:03.252 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:03.248+0000 7f467e486640 1 --2- 172.21.15.49:0/2217512547 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4678102180 0x7f4678074060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:03.252 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:03.248+0000 7f467e486640 1 -- 172.21.15.49:0/2217512547 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f46781077d0 msgr2=0x7f46780745a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:19:03.252 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:03.248+0000 7f467e486640 1 --2- 172.21.15.49:0/2217512547 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f46781077d0 0x7f46780745a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:03.252 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:03.248+0000 7f467e486640 1 -- 172.21.15.49:0/2217512547 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46781a65a0 con 0x7f46781017c0 2024-03-18T23:19:03.252 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:03.248+0000 7f467e486640 1 --2- 172.21.15.49:0/2217512547 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f46781017c0 0x7f4678073b20 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f467400e9f0 tx=0x7f467400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:19:03.252 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:03.249+0000 7f466b7fe640 1 -- 172.21.15.49:0/2217512547 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f467400cd80 con 0x7f46781017c0 2024-03-18T23:19:03.253 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:03.249+0000 7f466b7fe640 1 -- 172.21.15.49:0/2217512547 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4674005500 con 0x7f46781017c0 2024-03-18T23:19:03.253 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:03.249+0000 7f466b7fe640 1 -- 172.21.15.49:0/2217512547 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4674010690 con 0x7f46781017c0 2024-03-18T23:19:03.253 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:03.249+0000 7f46806ea640 1 -- 172.21.15.49:0/2217512547 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46781a6890 con 0x7f46781017c0 2024-03-18T23:19:03.253 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:03.249+0000 7f46806ea640 1 -- 172.21.15.49:0/2217512547 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46781a6de0 con 0x7f46781017c0 2024-03-18T23:19:03.254 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:03.250+0000 7f46806ea640 1 -- 172.21.15.49:0/2217512547 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4678068a60 con 0x7f46781017c0 2024-03-18T23:19:03.254 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:03.251+0000 7f466b7fe640 1 -- 172.21.15.49:0/2217512547 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 25) ==== 137421+0+0 (secure 0 0 0) 0x7f46740046c0 con 0x7f46781017c0 2024-03-18T23:19:03.254 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:03.251+0000 7f466b7fe640 1 --2- 172.21.15.49:0/2217512547 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f46640a29f0 0x7f46640a4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:19:03.255 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:03.251+0000 7f467dc85640 1 --2- 172.21.15.49:0/2217512547 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f46640a29f0 0x7f46640a4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:19:03.255 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:03.251+0000 7f466b7fe640 1 -- 172.21.15.49:0/2217512547 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8556+0+0 (secure 0 0 0) 0x7f46740cf100 con 0x7f46781017c0 2024-03-18T23:19:03.255 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:03.252+0000 7f467dc85640 1 --2- 172.21.15.49:0/2217512547 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f46640a29f0 0x7f46640a4eb0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f46780755b0 tx=0x7f466c005f90 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:19:03.267 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:03.266+0000 7f466b7fe640 1 -- 172.21.15.49:0/2217512547 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f467409bee0 con 0x7f46781017c0 2024-03-18T23:19:03.460 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:19:03 smithi175 ceph-mon[67352]: pgmap v336: 97 pgs: 97 active+clean; 67 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.0 MiB/s wr, 282 op/s 2024-03-18T23:19:03.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:19:03 smithi175 ceph-mon[67352]: from='client.16326 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T23:19:03.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:19:03 smithi175 ceph-mon[67352]: from='client.16332 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T23:19:03.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:19:03 smithi175 ceph-mon[67352]: from='client.? 172.21.15.49:0/4085472577' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T23:19:03.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:19:03 smithi175 ceph-mon[67352]: from='client.? 172.21.15.49:0/2570032439' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-18T23:19:03.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:19:03 smithi175 ceph-mon[67352]: from='client.? 172.21.15.49:0/3793298975' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-18T23:19:03.469 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:03.469+0000 7f46806ea640 1 -- 172.21.15.49:0/2217512547 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f4678109d30 con 0x7f46781017c0 2024-03-18T23:19:03.469 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:03.469+0000 7f466b7fe640 1 -- 172.21.15.49:0/2217512547 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v99) ==== 100+0+11 (secure 0 0 0) 0x7f467409b7e0 con 0x7f46781017c0 2024-03-18T23:19:03.469 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 64 2024-03-18T23:19:03.472 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:03.471+0000 7f46806ea640 1 -- 172.21.15.49:0/2217512547 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f46640a29f0 msgr2=0x7f46640a4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:19:03.472 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:03.471+0000 7f46806ea640 1 --2- 172.21.15.49:0/2217512547 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f46640a29f0 0x7f46640a4eb0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f46780755b0 tx=0x7f466c005f90 comp rx=0 tx=0).stop 2024-03-18T23:19:03.472 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:03.471+0000 7f46806ea640 1 -- 172.21.15.49:0/2217512547 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f46781017c0 msgr2=0x7f4678073b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:19:03.472 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:03.471+0000 7f46806ea640 1 --2- 172.21.15.49:0/2217512547 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f46781017c0 0x7f4678073b20 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f467400e9f0 tx=0x7f467400edf0 comp rx=0 tx=0).stop 2024-03-18T23:19:03.472 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:03.472+0000 7f46806ea640 1 -- 172.21.15.49:0/2217512547 shutdown_connections 2024-03-18T23:19:03.472 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:03.472+0000 7f46806ea640 1 --2- 172.21.15.49:0/2217512547 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f46640a29f0 0x7f46640a4eb0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:03.473 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:03.472+0000 7f46806ea640 1 --2- 172.21.15.49:0/2217512547 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f46781077d0 0x7f46780745a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:03.473 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:03.472+0000 7f46806ea640 1 --2- 172.21.15.49:0/2217512547 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4678102180 0x7f4678074060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:03.473 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:03.472+0000 7f46806ea640 1 --2- 172.21.15.49:0/2217512547 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f46781017c0 0x7f4678073b20 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:03.473 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:03.472+0000 7f46806ea640 1 -- 172.21.15.49:0/2217512547 >> 172.21.15.49:0/2217512547 conn(0x7f46780fd250 msgr2=0x7f46780fd850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:19:03.473 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:03.472+0000 7f46806ea640 1 -- 172.21.15.49:0/2217512547 shutdown_connections 2024-03-18T23:19:03.473 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:03.472+0000 7f46806ea640 1 -- 172.21.15.49:0/2217512547 wait complete. 2024-03-18T23:19:03.482 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:19:03 smithi049 ceph-mon[62991]: pgmap v336: 97 pgs: 97 active+clean; 67 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.0 MiB/s wr, 282 op/s 2024-03-18T23:19:03.482 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:19:03 smithi049 ceph-mon[62991]: from='client.16326 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T23:19:03.482 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:19:03 smithi049 ceph-mon[62991]: from='client.16332 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T23:19:03.482 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:19:03 smithi049 ceph-mon[62991]: from='client.? 172.21.15.49:0/4085472577' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T23:19:03.483 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:19:03 smithi049 ceph-mon[62991]: from='client.? 172.21.15.49:0/2570032439' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-18T23:19:03.483 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:19:03 smithi049 ceph-mon[62991]: from='client.? 172.21.15.49:0/3793298975' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-18T23:19:03.484 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-03-18T23:19:03.564 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:19:03 smithi139 ceph-mon[67282]: pgmap v336: 97 pgs: 97 active+clean; 67 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.0 MiB/s wr, 282 op/s 2024-03-18T23:19:03.564 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:19:03 smithi139 ceph-mon[67282]: from='client.16326 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T23:19:03.564 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:19:03 smithi139 ceph-mon[67282]: from='client.16332 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T23:19:03.564 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:19:03 smithi139 ceph-mon[67282]: from='client.? 172.21.15.49:0/4085472577' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T23:19:03.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:19:03 smithi139 ceph-mon[67282]: from='client.? 172.21.15.49:0/2570032439' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-18T23:19:03.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:19:03 smithi139 ceph-mon[67282]: from='client.? 172.21.15.49:0/3793298975' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-18T23:19:03.603 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:03.600+0000 7ff21ea7e640 1 -- 172.21.15.49:0/296988538 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff218100720 msgr2=0x7ff218100ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:19:03.603 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:03.600+0000 7ff21ea7e640 1 --2- 172.21.15.49:0/296988538 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff218100720 0x7ff218100ba0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7ff20c009930 tx=0x7ff20c02bc30 comp rx=0 tx=0).stop 2024-03-18T23:19:03.603 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:03.600+0000 7ff21ea7e640 1 -- 172.21.15.49:0/296988538 shutdown_connections 2024-03-18T23:19:03.604 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:03.600+0000 7ff21ea7e640 1 --2- 172.21.15.49:0/296988538 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff2181010e0 0x7ff21810d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:03.604 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:03.600+0000 7ff21ea7e640 1 --2- 172.21.15.49:0/296988538 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff218100720 0x7ff218100ba0 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:03.604 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:03.600+0000 7ff21ea7e640 1 --2- 172.21.15.49:0/296988538 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff218106730 0x7ff218106b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:03.604 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:03.600+0000 7ff21ea7e640 1 -- 172.21.15.49:0/296988538 >> 172.21.15.49:0/296988538 conn(0x7ff218077820 msgr2=0x7ff218077c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:19:03.604 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:03.600+0000 7ff21ea7e640 1 -- 172.21.15.49:0/296988538 shutdown_connections 2024-03-18T23:19:03.604 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:03.600+0000 7ff21ea7e640 1 -- 172.21.15.49:0/296988538 wait complete. 2024-03-18T23:19:03.604 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:03.600+0000 7ff21ea7e640 1 Processor -- start 2024-03-18T23:19:03.605 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:03.600+0000 7ff21ea7e640 1 -- start start 2024-03-18T23:19:03.605 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:03.600+0000 7ff21ea7e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff218100720 0x7ff218198fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:19:03.605 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:03.600+0000 7ff21ea7e640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff2181010e0 0x7ff218199520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:19:03.605 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:03.600+0000 7ff21ea7e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff218106730 0x7ff2181a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:19:03.606 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:03.601+0000 7ff21ea7e640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7ff218110b40 con 0x7ff218100720 2024-03-18T23:19:03.606 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:03.601+0000 7ff21ea7e640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7ff2181109c0 con 0x7ff218106730 2024-03-18T23:19:03.606 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:03.601+0000 7ff21ea7e640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff218110cc0 con 0x7ff2181010e0 2024-03-18T23:19:03.606 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:03.601+0000 7ff21c81a640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff218100720 0x7ff218198fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:19:03.607 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:03.601+0000 7ff21c81a640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff218100720 0x7ff218198fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:57360/0 (socket says 172.21.15.49:57360) 2024-03-18T23:19:03.607 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:03.601+0000 7ff21c81a640 1 -- 172.21.15.49:0/928810796 learned_addr learned my addr 172.21.15.49:0/928810796 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:19:03.607 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:03.601+0000 7ff21d01b640 1 --2- 172.21.15.49:0/928810796 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff218106730 0x7ff2181a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:19:03.607 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:03.601+0000 7ff217fff640 1 --2- 172.21.15.49:0/928810796 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff2181010e0 0x7ff218199520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:19:03.608 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:03.601+0000 7ff21c81a640 1 -- 172.21.15.49:0/928810796 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff2181010e0 msgr2=0x7ff218199520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:19:03.608 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:03.601+0000 7ff21c81a640 1 --2- 172.21.15.49:0/928810796 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff2181010e0 0x7ff218199520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:03.608 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:03.601+0000 7ff21c81a640 1 -- 172.21.15.49:0/928810796 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff218106730 msgr2=0x7ff2181a4060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:19:03.608 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:03.601+0000 7ff21c81a640 1 --2- 172.21.15.49:0/928810796 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff218106730 0x7ff2181a4060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:03.609 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:03.601+0000 7ff21c81a640 1 -- 172.21.15.49:0/928810796 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2181a45a0 con 0x7ff218100720 2024-03-18T23:19:03.609 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:03.601+0000 7ff21c81a640 1 --2- 172.21.15.49:0/928810796 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff218100720 0x7ff218198fe0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7ff21000ebc0 tx=0x7ff21000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:19:03.609 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:03.601+0000 7ff215ffb640 1 -- 172.21.15.49:0/928810796 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff210010630 con 0x7ff218100720 2024-03-18T23:19:03.609 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:03.601+0000 7ff215ffb640 1 -- 172.21.15.49:0/928810796 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff210010cb0 con 0x7ff218100720 2024-03-18T23:19:03.610 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:03.601+0000 7ff21ea7e640 1 -- 172.21.15.49:0/928810796 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2181a4770 con 0x7ff218100720 2024-03-18T23:19:03.610 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:03.601+0000 7ff215ffb640 1 -- 172.21.15.49:0/928810796 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff210018bd0 con 0x7ff218100720 2024-03-18T23:19:03.610 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:03.601+0000 7ff21ea7e640 1 -- 172.21.15.49:0/928810796 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2181a4c90 con 0x7ff218100720 2024-03-18T23:19:03.610 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:03.603+0000 7ff21ea7e640 1 -- 172.21.15.49:0/928810796 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff218068610 con 0x7ff218100720 2024-03-18T23:19:03.612 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:03.604+0000 7ff215ffb640 1 -- 172.21.15.49:0/928810796 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 25) ==== 137421+0+0 (secure 0 0 0) 0x7ff2100107d0 con 0x7ff218100720 2024-03-18T23:19:03.613 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:03.604+0000 7ff215ffb640 1 --2- 172.21.15.49:0/928810796 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7ff2000a2970 0x7ff2000a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:19:03.613 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:03.604+0000 7ff215ffb640 1 -- 172.21.15.49:0/928810796 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8556+0+0 (secure 0 0 0) 0x7ff2100cf420 con 0x7ff218100720 2024-03-18T23:19:03.613 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:03.604+0000 7ff217fff640 1 --2- 172.21.15.49:0/928810796 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7ff2000a2970 0x7ff2000a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:19:03.613 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:03.605+0000 7ff217fff640 1 --2- 172.21.15.49:0/928810796 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7ff2000a2970 0x7ff2000a4e30 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7ff20c005ab0 tx=0x7ff20c005fd0 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:19:03.614 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:03.609+0000 7ff215ffb640 1 -- 172.21.15.49:0/928810796 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7ff21009bb10 con 0x7ff218100720 2024-03-18T23:19:03.843 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:03.843+0000 7ff21ea7e640 1 -- 172.21.15.49:0/928810796 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7ff218108cb0 con 0x7ff218100720 2024-03-18T23:19:03.851 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:03.851+0000 7ff215ffb640 1 -- 172.21.15.49:0/928810796 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v25) ==== 81+0+740363 (secure 0 0 0) 0x7ff21009fbb0 con 0x7ff218100720 2024-03-18T23:19:03.864 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-18T23:19:03.864 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-03-18T23:19:03.864 INFO:teuthology.orchestra.run.smithi049.stdout: "epoch": 25, 2024-03-18T23:19:03.865 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:03.865 INFO:teuthology.orchestra.run.smithi049.stdout: "active_gid": 15366, 2024-03-18T23:19:03.865 INFO:teuthology.orchestra.run.smithi049.stdout: "active_name": "x", 2024-03-18T23:19:03.865 INFO:teuthology.orchestra.run.smithi049.stdout: "active_addrs": { 2024-03-18T23:19:03.865 INFO:teuthology.orchestra.run.smithi049.stdout: "addrvec": [ 2024-03-18T23:19:03.865 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-18T23:19:03.865 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v2", 2024-03-18T23:19:03.865 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.49:6800", 2024-03-18T23:19:03.866 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 1112476276 2024-03-18T23:19:03.866 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:03.866 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-18T23:19:03.866 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v1", 2024-03-18T23:19:03.866 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.49:6801", 2024-03-18T23:19:03.866 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 1112476276 2024-03-18T23:19:03.867 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:19:03.867 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-03-18T23:19:03.867 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:03.868 INFO:teuthology.orchestra.run.smithi049.stdout: "active_addr": "172.21.15.49:6801/1112476276", 2024-03-18T23:19:03.868 INFO:teuthology.orchestra.run.smithi049.stdout: "active_change": "2024-03-18T23:08:15.312472+0000", 2024-03-18T23:19:03.868 INFO:teuthology.orchestra.run.smithi049.stdout: "active_mgr_features": 4540701547738038271, 2024-03-18T23:19:03.868 INFO:teuthology.orchestra.run.smithi049.stdout: "available": true, 2024-03-18T23:19:03.868 INFO:teuthology.orchestra.run.smithi049.stdout: "standbys": [ 2024-03-18T23:19:03.868 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-18T23:19:03.868 INFO:teuthology.orchestra.run.smithi049.stdout: "gid": 25177, 2024-03-18T23:19:03.868 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "z", 2024-03-18T23:19:03.869 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr_features": 4540701547738038271, 2024-03-18T23:19:03.869 INFO:teuthology.orchestra.run.smithi049.stdout: "available_modules": [ 2024-03-18T23:19:03.869 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-18T23:19:03.869 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "alerts", 2024-03-18T23:19:03.869 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-18T23:19:03.869 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-18T23:19:03.869 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-18T23:19:03.869 INFO:teuthology.orchestra.run.smithi049.stdout: "interval": { 2024-03-18T23:19:03.870 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "interval", 2024-03-18T23:19:03.870 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-03-18T23:19:03.870 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:03.870 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:03.870 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "60", 2024-03-18T23:19:03.870 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:03.870 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:03.870 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:03.871 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "How frequently to reexamine health status", 2024-03-18T23:19:03.871 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:03.871 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:03.871 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:03.871 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:03.871 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-18T23:19:03.871 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-18T23:19:03.871 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:03.872 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:03.872 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:03.872 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:03.872 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:03.872 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:03.872 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-18T23:19:03.872 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-18T23:19:03.872 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-18T23:19:03.873 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-18T23:19:03.873 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-18T23:19:03.873 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-18T23:19:03.873 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-18T23:19:03.873 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-18T23:19:03.873 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:03.873 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:03.873 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:03.874 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:03.874 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:03.874 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-18T23:19:03.874 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-18T23:19:03.874 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:03.874 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:03.874 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:03.874 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:03.875 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:03.875 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:03.875 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:03.875 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:03.875 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:03.875 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:03.875 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:03.875 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:03.876 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-18T23:19:03.876 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-18T23:19:03.876 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:03.876 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:03.876 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:03.876 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-18T23:19:03.876 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:03.876 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:03.877 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-18T23:19:03.877 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-18T23:19:03.877 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-18T23:19:03.877 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-18T23:19:03.877 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-18T23:19:03.877 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-18T23:19:03.877 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-18T23:19:03.877 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-18T23:19:03.877 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:03.878 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:03.878 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:03.878 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:03.878 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:03.878 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-18T23:19:03.879 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-18T23:19:03.879 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:03.879 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:03.879 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:03.879 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:03.879 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:03.879 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:03.880 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:03.880 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:03.880 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:03.880 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:03.880 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:03.880 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:03.880 INFO:teuthology.orchestra.run.smithi049.stdout: "smtp_destination": { 2024-03-18T23:19:03.880 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "smtp_destination", 2024-03-18T23:19:03.880 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:03.881 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:03.881 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:03.881 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:03.881 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:03.881 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:03.881 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:03.881 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Email address to send alerts to", 2024-03-18T23:19:03.881 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:03.882 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:03.882 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:03.882 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:03.882 INFO:teuthology.orchestra.run.smithi049.stdout: "smtp_from_name": { 2024-03-18T23:19:03.882 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "smtp_from_name", 2024-03-18T23:19:03.882 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:03.882 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:03.883 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:03.883 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "Ceph", 2024-03-18T23:19:03.883 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:03.883 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:03.883 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:03.883 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Email From: name", 2024-03-18T23:19:03.883 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:03.883 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:03.883 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:03.884 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:03.884 INFO:teuthology.orchestra.run.smithi049.stdout: "smtp_host": { 2024-03-18T23:19:03.884 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "smtp_host", 2024-03-18T23:19:03.884 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:03.884 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:03.884 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:03.884 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:03.884 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:03.885 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:03.885 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:03.885 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "SMTP server", 2024-03-18T23:19:03.885 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:03.885 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:03.885 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:03.885 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:03.885 INFO:teuthology.orchestra.run.smithi049.stdout: "smtp_password": { 2024-03-18T23:19:03.885 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "smtp_password", 2024-03-18T23:19:03.886 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:03.886 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:03.886 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:03.886 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:03.886 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:03.886 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:03.886 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:03.887 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Password to authenticate with", 2024-03-18T23:19:03.887 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:03.887 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:03.887 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:03.887 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:03.887 INFO:teuthology.orchestra.run.smithi049.stdout: "smtp_port": { 2024-03-18T23:19:03.888 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "smtp_port", 2024-03-18T23:19:03.888 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-18T23:19:03.888 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:03.888 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:03.888 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "465", 2024-03-18T23:19:03.888 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:03.888 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:03.889 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:03.889 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "SMTP port", 2024-03-18T23:19:03.889 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:03.889 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:03.889 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:03.889 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:03.890 INFO:teuthology.orchestra.run.smithi049.stdout: "smtp_sender": { 2024-03-18T23:19:03.890 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "smtp_sender", 2024-03-18T23:19:03.890 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:03.890 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:03.890 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:03.890 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:03.891 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:03.891 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:03.891 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:03.891 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "SMTP envelope sender", 2024-03-18T23:19:03.891 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:03.891 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:03.891 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:03.891 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:03.892 INFO:teuthology.orchestra.run.smithi049.stdout: "smtp_ssl": { 2024-03-18T23:19:03.892 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "smtp_ssl", 2024-03-18T23:19:03.892 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:03.892 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:03.892 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:03.892 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-18T23:19:03.892 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:03.892 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:03.893 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:03.893 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Use SSL to connect to SMTP server", 2024-03-18T23:19:03.893 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:03.893 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:03.893 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:03.893 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:03.893 INFO:teuthology.orchestra.run.smithi049.stdout: "smtp_user": { 2024-03-18T23:19:03.893 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "smtp_user", 2024-03-18T23:19:03.893 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:03.894 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:03.894 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:03.894 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:03.894 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:03.894 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:03.894 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:03.894 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "User to authenticate as", 2024-03-18T23:19:03.895 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:03.895 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:03.895 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:03.895 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:19:03.895 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:19:03.895 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:03.896 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-18T23:19:03.896 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "balancer", 2024-03-18T23:19:03.896 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-18T23:19:03.896 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-18T23:19:03.896 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-18T23:19:03.896 INFO:teuthology.orchestra.run.smithi049.stdout: "active": { 2024-03-18T23:19:03.896 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "active", 2024-03-18T23:19:03.896 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:03.897 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:03.897 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:03.897 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-18T23:19:03.897 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:03.897 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:03.897 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:03.897 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "automatically balance PGs across cluster", 2024-03-18T23:19:03.897 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:03.898 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:03.898 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:03.898 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:03.898 INFO:teuthology.orchestra.run.smithi049.stdout: "begin_time": { 2024-03-18T23:19:03.898 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "begin_time", 2024-03-18T23:19:03.898 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:03.898 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:03.898 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:03.899 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "0000", 2024-03-18T23:19:03.899 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:03.899 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:03.899 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:03.899 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "beginning time of day to automatically balance", 2024-03-18T23:19:03.899 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-18T23:19:03.900 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:03.900 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:03.900 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:03.900 INFO:teuthology.orchestra.run.smithi049.stdout: "begin_weekday": { 2024-03-18T23:19:03.900 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "begin_weekday", 2024-03-18T23:19:03.900 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "uint", 2024-03-18T23:19:03.900 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:03.901 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:03.901 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "0", 2024-03-18T23:19:03.901 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "0", 2024-03-18T23:19:03.901 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "6", 2024-03-18T23:19:03.901 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:03.901 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-03-18T23:19:03.901 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-18T23:19:03.901 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:03.902 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:03.902 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:03.902 INFO:teuthology.orchestra.run.smithi049.stdout: "crush_compat_max_iterations": { 2024-03-18T23:19:03.902 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "crush_compat_max_iterations", 2024-03-18T23:19:03.902 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "uint", 2024-03-18T23:19:03.902 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:03.902 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:03.903 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "25", 2024-03-18T23:19:03.903 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "1", 2024-03-18T23:19:03.903 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "250", 2024-03-18T23:19:03.903 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:03.903 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-03-18T23:19:03.903 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:03.904 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:03.904 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:03.904 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:03.904 INFO:teuthology.orchestra.run.smithi049.stdout: "crush_compat_metrics": { 2024-03-18T23:19:03.904 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "crush_compat_metrics", 2024-03-18T23:19:03.904 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:03.904 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:03.904 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:03.904 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "pgs,objects,bytes", 2024-03-18T23:19:03.905 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:03.905 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:03.905 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:03.905 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-03-18T23:19:03.905 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-03-18T23:19:03.905 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:03.905 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:03.906 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:03.906 INFO:teuthology.orchestra.run.smithi049.stdout: "crush_compat_step": { 2024-03-18T23:19:03.906 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "crush_compat_step", 2024-03-18T23:19:03.906 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "float", 2024-03-18T23:19:03.906 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:03.906 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:03.906 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "0.5", 2024-03-18T23:19:03.906 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "0.001", 2024-03-18T23:19:03.907 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "0.999", 2024-03-18T23:19:03.907 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:03.907 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "aggressiveness of optimization", 2024-03-18T23:19:03.907 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-03-18T23:19:03.907 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:03.907 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:03.907 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:03.908 INFO:teuthology.orchestra.run.smithi049.stdout: "end_time": { 2024-03-18T23:19:03.908 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "end_time", 2024-03-18T23:19:03.908 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:03.908 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:03.908 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:03.908 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "2359", 2024-03-18T23:19:03.908 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:03.908 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:03.909 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:03.909 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "ending time of day to automatically balance", 2024-03-18T23:19:03.909 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-18T23:19:03.909 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:03.909 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:03.909 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:03.909 INFO:teuthology.orchestra.run.smithi049.stdout: "end_weekday": { 2024-03-18T23:19:03.909 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "end_weekday", 2024-03-18T23:19:03.910 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "uint", 2024-03-18T23:19:03.910 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:03.910 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:03.910 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "0", 2024-03-18T23:19:03.910 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "0", 2024-03-18T23:19:03.911 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "6", 2024-03-18T23:19:03.911 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:03.911 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-03-18T23:19:03.911 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-18T23:19:03.911 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:03.912 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:03.912 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:03.912 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-18T23:19:03.912 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-18T23:19:03.913 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:03.913 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:03.913 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:03.913 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:03.914 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:03.914 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:03.914 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-18T23:19:03.914 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-18T23:19:03.915 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-18T23:19:03.915 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-18T23:19:03.915 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-18T23:19:03.915 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-18T23:19:03.916 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-18T23:19:03.916 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-18T23:19:03.916 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:03.916 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:03.917 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:03.917 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:03.917 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:03.918 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-18T23:19:03.918 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-18T23:19:03.918 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:03.918 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:03.919 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:03.919 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:03.919 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:03.920 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:03.920 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:03.920 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:03.920 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:03.921 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:03.921 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:03.921 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:03.921 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-18T23:19:03.922 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-18T23:19:03.922 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:03.922 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:03.922 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:03.922 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-18T23:19:03.923 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:03.923 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:03.923 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-18T23:19:03.923 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-18T23:19:03.923 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-18T23:19:03.924 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-18T23:19:03.924 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-18T23:19:03.924 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-18T23:19:03.924 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-18T23:19:03.924 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-18T23:19:03.925 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:03.925 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:03.925 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:03.925 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:03.925 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:03.926 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-18T23:19:03.926 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-18T23:19:03.926 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:03.926 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:03.926 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:03.927 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:03.927 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:03.927 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:03.927 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:03.927 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:03.928 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:03.928 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:03.928 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:03.928 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:03.928 INFO:teuthology.orchestra.run.smithi049.stdout: "min_score": { 2024-03-18T23:19:03.929 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "min_score", 2024-03-18T23:19:03.929 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "float", 2024-03-18T23:19:03.929 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:03.929 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:03.929 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "0", 2024-03-18T23:19:03.930 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:03.930 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:03.930 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:03.930 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-03-18T23:19:03.930 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:03.931 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:03.931 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:03.931 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:03.931 INFO:teuthology.orchestra.run.smithi049.stdout: "mode": { 2024-03-18T23:19:03.931 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "mode", 2024-03-18T23:19:03.932 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:03.932 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:03.932 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:03.932 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "upmap", 2024-03-18T23:19:03.932 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:03.933 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:03.933 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-18T23:19:03.933 INFO:teuthology.orchestra.run.smithi049.stdout: "crush-compat", 2024-03-18T23:19:03.933 INFO:teuthology.orchestra.run.smithi049.stdout: "none", 2024-03-18T23:19:03.933 INFO:teuthology.orchestra.run.smithi049.stdout: "read", 2024-03-18T23:19:03.934 INFO:teuthology.orchestra.run.smithi049.stdout: "upmap", 2024-03-18T23:19:03.934 INFO:teuthology.orchestra.run.smithi049.stdout: "upmap-read" 2024-03-18T23:19:03.934 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-18T23:19:03.934 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Balancer mode", 2024-03-18T23:19:03.934 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:03.935 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:03.935 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:03.935 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:03.935 INFO:teuthology.orchestra.run.smithi049.stdout: "pool_ids": { 2024-03-18T23:19:03.935 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "pool_ids", 2024-03-18T23:19:03.936 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:03.936 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:03.936 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:03.936 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:03.936 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:03.936 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:03.936 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:03.936 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-03-18T23:19:03.937 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:03.937 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:03.937 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:03.937 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:03.937 INFO:teuthology.orchestra.run.smithi049.stdout: "sleep_interval": { 2024-03-18T23:19:03.937 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "sleep_interval", 2024-03-18T23:19:03.938 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-03-18T23:19:03.938 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:03.938 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:03.938 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "60", 2024-03-18T23:19:03.938 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:03.938 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:03.938 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:03.938 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-03-18T23:19:03.939 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:03.939 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:03.939 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:03.939 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:03.939 INFO:teuthology.orchestra.run.smithi049.stdout: "upmap_max_deviation": { 2024-03-18T23:19:03.939 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "upmap_max_deviation", 2024-03-18T23:19:03.939 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-18T23:19:03.939 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:03.940 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:03.940 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "5", 2024-03-18T23:19:03.940 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "1", 2024-03-18T23:19:03.940 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:03.940 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:03.940 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "deviation below which no optimization is attempted", 2024-03-18T23:19:03.940 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-03-18T23:19:03.940 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:03.941 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:03.941 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:03.941 INFO:teuthology.orchestra.run.smithi049.stdout: "upmap_max_optimizations": { 2024-03-18T23:19:03.941 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "upmap_max_optimizations", 2024-03-18T23:19:03.941 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "uint", 2024-03-18T23:19:03.941 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:03.941 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:03.941 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "10", 2024-03-18T23:19:03.942 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:03.942 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:03.942 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:03.942 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-03-18T23:19:03.942 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:03.942 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:03.943 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:03.943 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:19:03.943 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:19:03.943 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:03.943 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-18T23:19:03.943 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "cephadm", 2024-03-18T23:19:03.943 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-18T23:19:03.943 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-18T23:19:03.943 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-18T23:19:03.944 INFO:teuthology.orchestra.run.smithi049.stdout: "agent_down_multiplier": { 2024-03-18T23:19:03.944 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "agent_down_multiplier", 2024-03-18T23:19:03.944 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "float", 2024-03-18T23:19:03.944 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:03.944 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:03.944 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "3.0", 2024-03-18T23:19:03.944 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:03.944 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:03.945 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:03.945 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-03-18T23:19:03.945 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:03.945 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:03.945 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:03.945 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:03.945 INFO:teuthology.orchestra.run.smithi049.stdout: "agent_refresh_rate": { 2024-03-18T23:19:03.945 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "agent_refresh_rate", 2024-03-18T23:19:03.945 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-03-18T23:19:03.946 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:03.946 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:03.946 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "20", 2024-03-18T23:19:03.946 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:03.946 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:03.946 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:03.946 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-03-18T23:19:03.946 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:03.947 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:03.947 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:03.947 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:03.947 INFO:teuthology.orchestra.run.smithi049.stdout: "agent_starting_port": { 2024-03-18T23:19:03.947 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "agent_starting_port", 2024-03-18T23:19:03.947 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-18T23:19:03.947 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:03.947 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:03.948 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "4721", 2024-03-18T23:19:03.948 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:03.948 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:03.948 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:03.948 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-03-18T23:19:03.948 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:03.948 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:03.949 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:03.949 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:03.949 INFO:teuthology.orchestra.run.smithi049.stdout: "allow_ptrace": { 2024-03-18T23:19:03.949 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "allow_ptrace", 2024-03-18T23:19:03.949 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:03.949 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:03.949 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:03.949 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:03.949 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:03.950 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:03.950 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:03.950 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-03-18T23:19:03.950 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-03-18T23:19:03.950 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:03.950 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:03.950 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:03.950 INFO:teuthology.orchestra.run.smithi049.stdout: "autotune_interval": { 2024-03-18T23:19:03.951 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "autotune_interval", 2024-03-18T23:19:03.951 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-03-18T23:19:03.951 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:03.951 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:03.951 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "600", 2024-03-18T23:19:03.951 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:03.951 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:03.951 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:03.951 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "how frequently to autotune daemon memory", 2024-03-18T23:19:03.951 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:03.952 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:03.952 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:03.952 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:03.952 INFO:teuthology.orchestra.run.smithi049.stdout: "autotune_memory_target_ratio": { 2024-03-18T23:19:03.952 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "autotune_memory_target_ratio", 2024-03-18T23:19:03.952 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "float", 2024-03-18T23:19:03.952 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:03.953 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:03.953 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "0.7", 2024-03-18T23:19:03.953 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:03.953 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:03.953 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:03.953 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-03-18T23:19:03.953 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:03.954 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:03.954 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:03.954 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:03.954 INFO:teuthology.orchestra.run.smithi049.stdout: "cephadm_log_destination": { 2024-03-18T23:19:03.954 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "cephadm_log_destination", 2024-03-18T23:19:03.954 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:03.954 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:03.955 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:03.955 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:03.955 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:03.955 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:03.955 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-18T23:19:03.955 INFO:teuthology.orchestra.run.smithi049.stdout: "file", 2024-03-18T23:19:03.955 INFO:teuthology.orchestra.run.smithi049.stdout: "file,syslog", 2024-03-18T23:19:03.955 INFO:teuthology.orchestra.run.smithi049.stdout: "syslog" 2024-03-18T23:19:03.956 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-18T23:19:03.956 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-03-18T23:19:03.956 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:03.956 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:03.956 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:03.956 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:03.956 INFO:teuthology.orchestra.run.smithi049.stdout: "cgroups_split": { 2024-03-18T23:19:03.956 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "cgroups_split", 2024-03-18T23:19:03.956 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:03.957 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:03.957 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:03.957 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-18T23:19:03.957 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:03.957 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:03.957 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:03.957 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-03-18T23:19:03.957 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:03.957 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:03.958 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:03.958 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:03.958 INFO:teuthology.orchestra.run.smithi049.stdout: "config_checks_enabled": { 2024-03-18T23:19:03.958 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "config_checks_enabled", 2024-03-18T23:19:03.958 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:03.958 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:03.958 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:03.958 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:03.959 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:03.959 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:03.959 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:03.959 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-03-18T23:19:03.959 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:03.959 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:03.959 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:03.959 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:03.959 INFO:teuthology.orchestra.run.smithi049.stdout: "config_dashboard": { 2024-03-18T23:19:03.960 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "config_dashboard", 2024-03-18T23:19:03.960 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:03.960 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:03.960 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:03.960 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-18T23:19:03.960 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:03.960 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:03.960 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:03.960 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-03-18T23:19:03.961 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:03.961 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:03.961 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:03.961 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:03.961 INFO:teuthology.orchestra.run.smithi049.stdout: "container_image_alertmanager": { 2024-03-18T23:19:03.961 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_image_alertmanager", 2024-03-18T23:19:03.961 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:03.961 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:03.961 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:03.961 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-03-18T23:19:03.962 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:03.962 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:03.962 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:03.962 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Prometheus container image", 2024-03-18T23:19:03.962 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:03.962 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:03.962 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:03.963 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:03.963 INFO:teuthology.orchestra.run.smithi049.stdout: "container_image_base": { 2024-03-18T23:19:03.963 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_image_base", 2024-03-18T23:19:03.963 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:03.963 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:03.963 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:03.963 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "quay.io/ceph/ceph", 2024-03-18T23:19:03.963 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:03.963 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:03.964 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:03.964 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Container image name, without the tag", 2024-03-18T23:19:03.964 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:03.964 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:03.964 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:03.964 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:03.964 INFO:teuthology.orchestra.run.smithi049.stdout: "container_image_elasticsearch": { 2024-03-18T23:19:03.964 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_image_elasticsearch", 2024-03-18T23:19:03.964 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:03.965 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:03.965 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:03.965 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-03-18T23:19:03.965 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:03.965 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:03.965 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:03.965 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "elasticsearch container image", 2024-03-18T23:19:03.965 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:03.965 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:03.966 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:03.966 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:03.966 INFO:teuthology.orchestra.run.smithi049.stdout: "container_image_grafana": { 2024-03-18T23:19:03.966 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_image_grafana", 2024-03-18T23:19:03.966 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:03.966 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:03.966 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:03.966 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.12", 2024-03-18T23:19:03.967 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:03.967 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:03.967 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:03.967 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Prometheus container image", 2024-03-18T23:19:03.967 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:03.967 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:03.967 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:03.967 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:03.968 INFO:teuthology.orchestra.run.smithi049.stdout: "container_image_haproxy": { 2024-03-18T23:19:03.968 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_image_haproxy", 2024-03-18T23:19:03.968 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:03.968 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:03.968 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:03.968 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-03-18T23:19:03.968 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:03.969 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:03.969 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:03.969 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "HAproxy container image", 2024-03-18T23:19:03.969 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:03.969 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:03.969 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:03.969 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:03.969 INFO:teuthology.orchestra.run.smithi049.stdout: "container_image_jaeger_agent": { 2024-03-18T23:19:03.970 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_image_jaeger_agent", 2024-03-18T23:19:03.970 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:03.970 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:03.970 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:03.970 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-03-18T23:19:03.970 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:03.970 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:03.970 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:03.971 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Jaeger agent container image", 2024-03-18T23:19:03.971 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:03.971 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:03.971 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:03.971 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:03.971 INFO:teuthology.orchestra.run.smithi049.stdout: "container_image_jaeger_collector": { 2024-03-18T23:19:03.971 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_image_jaeger_collector", 2024-03-18T23:19:03.971 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:03.971 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:03.972 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:03.972 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-03-18T23:19:03.972 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:03.972 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:03.972 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:03.972 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Jaeger collector container image", 2024-03-18T23:19:03.972 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:03.973 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:03.973 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:03.973 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:03.973 INFO:teuthology.orchestra.run.smithi049.stdout: "container_image_jaeger_query": { 2024-03-18T23:19:03.973 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_image_jaeger_query", 2024-03-18T23:19:03.973 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:03.973 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:03.973 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:03.974 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-03-18T23:19:03.974 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:03.974 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:03.974 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:03.974 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Jaeger query container image", 2024-03-18T23:19:03.974 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:03.974 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:03.974 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:03.975 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:03.975 INFO:teuthology.orchestra.run.smithi049.stdout: "container_image_keepalived": { 2024-03-18T23:19:03.975 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_image_keepalived", 2024-03-18T23:19:03.975 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:03.975 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:03.975 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:03.975 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-03-18T23:19:03.975 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:03.976 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:03.976 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:03.976 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Keepalived container image", 2024-03-18T23:19:03.976 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:03.976 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:03.976 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:03.976 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:03.976 INFO:teuthology.orchestra.run.smithi049.stdout: "container_image_loki": { 2024-03-18T23:19:03.977 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_image_loki", 2024-03-18T23:19:03.977 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:03.977 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:03.977 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:03.977 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-03-18T23:19:03.977 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:03.977 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:03.978 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:03.978 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Loki container image", 2024-03-18T23:19:03.978 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:03.978 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:03.978 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:03.978 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:03.978 INFO:teuthology.orchestra.run.smithi049.stdout: "container_image_node_exporter": { 2024-03-18T23:19:03.978 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_image_node_exporter", 2024-03-18T23:19:03.979 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:03.979 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:03.979 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:03.979 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-03-18T23:19:03.979 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:03.979 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:03.979 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:03.979 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Prometheus container image", 2024-03-18T23:19:03.980 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:03.980 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:03.980 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:03.980 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:03.980 INFO:teuthology.orchestra.run.smithi049.stdout: "container_image_nvmeof": { 2024-03-18T23:19:03.980 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_image_nvmeof", 2024-03-18T23:19:03.980 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:03.980 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:03.981 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:03.981 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "quay.io/ceph/nvmeof:1.0.0", 2024-03-18T23:19:03.981 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:03.981 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:03.981 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:03.982 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Nvme-of container image", 2024-03-18T23:19:03.982 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:03.982 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:03.982 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:03.982 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:03.982 INFO:teuthology.orchestra.run.smithi049.stdout: "container_image_prometheus": { 2024-03-18T23:19:03.982 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_image_prometheus", 2024-03-18T23:19:03.983 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:03.983 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:03.983 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:03.983 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-03-18T23:19:03.983 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:03.983 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:03.983 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:03.984 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Prometheus container image", 2024-03-18T23:19:03.984 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:03.984 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:03.984 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:03.984 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:03.984 INFO:teuthology.orchestra.run.smithi049.stdout: "container_image_promtail": { 2024-03-18T23:19:03.984 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_image_promtail", 2024-03-18T23:19:03.984 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:03.984 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:03.985 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:03.985 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-03-18T23:19:03.985 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:03.985 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:03.985 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:03.985 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Promtail container image", 2024-03-18T23:19:03.985 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:03.985 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:03.985 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:03.986 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:03.986 INFO:teuthology.orchestra.run.smithi049.stdout: "container_image_snmp_gateway": { 2024-03-18T23:19:03.986 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_image_snmp_gateway", 2024-03-18T23:19:03.986 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:03.986 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:03.986 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:03.986 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-03-18T23:19:03.986 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:03.987 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:03.987 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:03.987 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "SNMP Gateway container image", 2024-03-18T23:19:03.987 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:03.987 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:03.987 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:03.987 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:03.987 INFO:teuthology.orchestra.run.smithi049.stdout: "container_init": { 2024-03-18T23:19:03.988 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_init", 2024-03-18T23:19:03.988 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:03.988 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:03.988 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:03.988 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-18T23:19:03.988 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:03.988 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:03.988 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:03.989 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Run podman/docker with `--init`", 2024-03-18T23:19:03.989 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:03.989 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:03.989 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:03.989 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:03.989 INFO:teuthology.orchestra.run.smithi049.stdout: "daemon_cache_timeout": { 2024-03-18T23:19:03.989 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "daemon_cache_timeout", 2024-03-18T23:19:03.989 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-03-18T23:19:03.990 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:03.990 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:03.990 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "600", 2024-03-18T23:19:03.990 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:03.990 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:03.990 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:03.990 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-03-18T23:19:03.990 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:03.991 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:03.991 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:03.991 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:03.991 INFO:teuthology.orchestra.run.smithi049.stdout: "default_cephadm_command_timeout": { 2024-03-18T23:19:03.991 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "default_cephadm_command_timeout", 2024-03-18T23:19:03.991 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-18T23:19:03.991 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:03.991 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:03.992 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "900", 2024-03-18T23:19:03.992 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:03.992 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:03.992 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:03.992 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-03-18T23:19:03.992 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:03.992 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:03.992 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:03.992 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:03.993 INFO:teuthology.orchestra.run.smithi049.stdout: "default_registry": { 2024-03-18T23:19:03.993 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "default_registry", 2024-03-18T23:19:03.993 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:03.993 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:03.993 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:03.993 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "docker.io", 2024-03-18T23:19:03.993 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:03.993 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:03.993 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:03.994 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-03-18T23:19:03.994 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:03.994 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:03.994 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:03.994 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:03.994 INFO:teuthology.orchestra.run.smithi049.stdout: "device_cache_timeout": { 2024-03-18T23:19:03.994 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "device_cache_timeout", 2024-03-18T23:19:03.995 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-03-18T23:19:03.995 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:03.995 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:03.995 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "1800", 2024-03-18T23:19:03.995 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:03.995 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:03.995 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:03.995 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "seconds to cache device inventory", 2024-03-18T23:19:03.995 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:03.995 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:03.996 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:03.996 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:03.996 INFO:teuthology.orchestra.run.smithi049.stdout: "device_enhanced_scan": { 2024-03-18T23:19:03.996 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "device_enhanced_scan", 2024-03-18T23:19:03.996 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:03.996 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:03.996 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:03.996 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:03.997 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:03.997 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:03.997 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:03.997 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Use libstoragemgmt during device scans", 2024-03-18T23:19:03.997 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:03.997 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:03.997 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:03.997 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:03.998 INFO:teuthology.orchestra.run.smithi049.stdout: "facts_cache_timeout": { 2024-03-18T23:19:03.998 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "facts_cache_timeout", 2024-03-18T23:19:03.998 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-03-18T23:19:03.998 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:03.998 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:03.998 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "60", 2024-03-18T23:19:03.999 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:03.999 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:03.999 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:03.999 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "seconds to cache host facts data", 2024-03-18T23:19:03.999 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:03.999 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.002 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.002 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.002 INFO:teuthology.orchestra.run.smithi049.stdout: "host_check_interval": { 2024-03-18T23:19:04.002 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "host_check_interval", 2024-03-18T23:19:04.002 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-03-18T23:19:04.002 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.003 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.003 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "600", 2024-03-18T23:19:04.003 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.003 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.003 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.003 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "how frequently to perform a host check", 2024-03-18T23:19:04.003 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.003 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.003 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.004 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.004 INFO:teuthology.orchestra.run.smithi049.stdout: "hw_monitoring": { 2024-03-18T23:19:04.004 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "hw_monitoring", 2024-03-18T23:19:04.004 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:04.004 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.004 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.004 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:04.004 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.005 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.005 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.005 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-03-18T23:19:04.005 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.005 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.005 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.005 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.005 INFO:teuthology.orchestra.run.smithi049.stdout: "inventory_list_all": { 2024-03-18T23:19:04.006 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "inventory_list_all", 2024-03-18T23:19:04.006 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:04.006 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.006 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.006 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:04.006 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.006 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.006 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.007 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-03-18T23:19:04.007 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.007 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.007 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.007 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.007 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-18T23:19:04.007 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-18T23:19:04.007 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:04.008 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.008 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:04.008 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:04.008 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.008 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.008 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-18T23:19:04.008 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-18T23:19:04.008 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-18T23:19:04.009 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-18T23:19:04.009 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-18T23:19:04.009 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-18T23:19:04.009 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-18T23:19:04.009 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-18T23:19:04.009 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.009 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.009 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.010 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.010 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.010 INFO:teuthology.orchestra.run.smithi049.stdout: "log_refresh_metadata": { 2024-03-18T23:19:04.010 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_refresh_metadata", 2024-03-18T23:19:04.010 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:04.010 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.010 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.010 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:04.011 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.011 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.011 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.011 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-03-18T23:19:04.011 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.011 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.011 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.011 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.012 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-18T23:19:04.012 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-18T23:19:04.012 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:04.012 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.012 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.012 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-18T23:19:04.012 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.012 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.013 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.013 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-03-18T23:19:04.013 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.013 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.013 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.013 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.013 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-18T23:19:04.013 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-18T23:19:04.014 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:04.014 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.014 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:04.014 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-18T23:19:04.014 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.014 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.014 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-18T23:19:04.015 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-18T23:19:04.015 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-18T23:19:04.015 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-18T23:19:04.015 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-18T23:19:04.015 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-18T23:19:04.015 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-18T23:19:04.015 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-18T23:19:04.016 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.016 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.016 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.016 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.016 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.016 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-18T23:19:04.016 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-18T23:19:04.016 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:04.017 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.017 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:04.017 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:04.017 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.017 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.017 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.017 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.017 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.018 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.018 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.018 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.018 INFO:teuthology.orchestra.run.smithi049.stdout: "manage_etc_ceph_ceph_conf": { 2024-03-18T23:19:04.018 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-03-18T23:19:04.018 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:04.018 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.019 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.019 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:04.019 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.019 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.019 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.019 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-03-18T23:19:04.019 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.019 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.020 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.020 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.020 INFO:teuthology.orchestra.run.smithi049.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-03-18T23:19:04.020 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-03-18T23:19:04.020 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:04.020 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.020 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.020 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "*", 2024-03-18T23:19:04.020 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.021 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.021 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.021 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-03-18T23:19:04.021 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.021 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.021 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.022 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.022 INFO:teuthology.orchestra.run.smithi049.stdout: "max_count_per_host": { 2024-03-18T23:19:04.022 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "max_count_per_host", 2024-03-18T23:19:04.022 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-18T23:19:04.022 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.023 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.023 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "10", 2024-03-18T23:19:04.023 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.023 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.023 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.023 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "max number of daemons per service per host", 2024-03-18T23:19:04.023 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.024 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.024 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.024 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.024 INFO:teuthology.orchestra.run.smithi049.stdout: "max_osd_draining_count": { 2024-03-18T23:19:04.024 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "max_osd_draining_count", 2024-03-18T23:19:04.024 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-18T23:19:04.024 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.024 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.024 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "10", 2024-03-18T23:19:04.025 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.025 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.025 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.025 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-03-18T23:19:04.025 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.025 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.025 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.025 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.025 INFO:teuthology.orchestra.run.smithi049.stdout: "migration_current": { 2024-03-18T23:19:04.026 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "migration_current", 2024-03-18T23:19:04.026 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-18T23:19:04.026 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.026 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.026 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:04.026 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.026 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.026 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.027 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "internal - do not modify", 2024-03-18T23:19:04.027 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.027 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.027 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.027 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.027 INFO:teuthology.orchestra.run.smithi049.stdout: "mode": { 2024-03-18T23:19:04.027 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "mode", 2024-03-18T23:19:04.027 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:04.027 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.028 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.028 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "root", 2024-03-18T23:19:04.028 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.028 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.028 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-18T23:19:04.028 INFO:teuthology.orchestra.run.smithi049.stdout: "cephadm-package", 2024-03-18T23:19:04.028 INFO:teuthology.orchestra.run.smithi049.stdout: "root" 2024-03-18T23:19:04.028 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-18T23:19:04.028 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "mode for remote execution of cephadm", 2024-03-18T23:19:04.029 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.029 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.029 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.029 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.029 INFO:teuthology.orchestra.run.smithi049.stdout: "oob_default_addr": { 2024-03-18T23:19:04.029 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "oob_default_addr", 2024-03-18T23:19:04.029 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:04.029 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.029 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.029 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "169.254.1.1", 2024-03-18T23:19:04.030 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.030 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.030 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.030 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Default address for RedFish API (oob management).", 2024-03-18T23:19:04.030 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.030 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.030 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.031 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.031 INFO:teuthology.orchestra.run.smithi049.stdout: "prometheus_alerts_path": { 2024-03-18T23:19:04.031 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "prometheus_alerts_path", 2024-03-18T23:19:04.031 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:04.031 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.031 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.031 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-03-18T23:19:04.031 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.031 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.032 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.032 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-03-18T23:19:04.032 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.032 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.032 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.032 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.032 INFO:teuthology.orchestra.run.smithi049.stdout: "registry_insecure": { 2024-03-18T23:19:04.032 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "registry_insecure", 2024-03-18T23:19:04.032 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:04.032 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.033 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.033 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:04.033 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.033 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.033 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.033 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-03-18T23:19:04.033 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.033 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.033 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.034 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.034 INFO:teuthology.orchestra.run.smithi049.stdout: "registry_password": { 2024-03-18T23:19:04.034 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "registry_password", 2024-03-18T23:19:04.034 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:04.034 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.034 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.034 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:04.035 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.035 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.035 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.035 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-03-18T23:19:04.035 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.035 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.035 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.035 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.035 INFO:teuthology.orchestra.run.smithi049.stdout: "registry_url": { 2024-03-18T23:19:04.036 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "registry_url", 2024-03-18T23:19:04.036 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:04.036 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.036 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.036 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:04.036 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.036 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.036 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.037 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-03-18T23:19:04.037 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.037 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.037 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.037 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.037 INFO:teuthology.orchestra.run.smithi049.stdout: "registry_username": { 2024-03-18T23:19:04.037 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "registry_username", 2024-03-18T23:19:04.037 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:04.038 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.038 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.038 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:04.038 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.038 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.038 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.038 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-03-18T23:19:04.038 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.039 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.039 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.039 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.039 INFO:teuthology.orchestra.run.smithi049.stdout: "secure_monitoring_stack": { 2024-03-18T23:19:04.039 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "secure_monitoring_stack", 2024-03-18T23:19:04.039 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:04.039 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.039 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.040 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:04.040 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.040 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.040 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.041 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-03-18T23:19:04.041 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.041 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.041 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.041 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.041 INFO:teuthology.orchestra.run.smithi049.stdout: "service_discovery_port": { 2024-03-18T23:19:04.041 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "service_discovery_port", 2024-03-18T23:19:04.042 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-18T23:19:04.042 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.042 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.042 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "8765", 2024-03-18T23:19:04.042 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.042 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.042 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.042 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "cephadm service discovery port", 2024-03-18T23:19:04.043 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.043 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.043 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.043 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.043 INFO:teuthology.orchestra.run.smithi049.stdout: "ssh_config_file": { 2024-03-18T23:19:04.043 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "ssh_config_file", 2024-03-18T23:19:04.043 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:04.043 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.044 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.044 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:04.044 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.044 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.044 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.044 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-03-18T23:19:04.044 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.044 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.044 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.044 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.045 INFO:teuthology.orchestra.run.smithi049.stdout: "use_agent": { 2024-03-18T23:19:04.045 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "use_agent", 2024-03-18T23:19:04.045 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:04.045 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.045 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.045 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:04.045 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.045 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.045 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.046 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-03-18T23:19:04.046 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.046 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.046 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.046 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.046 INFO:teuthology.orchestra.run.smithi049.stdout: "use_repo_digest": { 2024-03-18T23:19:04.046 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "use_repo_digest", 2024-03-18T23:19:04.046 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:04.047 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.047 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.047 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-18T23:19:04.047 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.047 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.047 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.047 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-03-18T23:19:04.047 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.048 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.048 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.048 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.048 INFO:teuthology.orchestra.run.smithi049.stdout: "warn_on_failed_host_check": { 2024-03-18T23:19:04.048 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "warn_on_failed_host_check", 2024-03-18T23:19:04.048 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:04.048 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.048 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.048 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-18T23:19:04.049 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.049 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.049 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.049 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "raise a health warning if the host check fails", 2024-03-18T23:19:04.049 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.049 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.049 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.050 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.050 INFO:teuthology.orchestra.run.smithi049.stdout: "warn_on_stray_daemons": { 2024-03-18T23:19:04.050 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "warn_on_stray_daemons", 2024-03-18T23:19:04.050 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:04.050 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.051 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.051 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-18T23:19:04.051 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.051 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.051 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.052 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-03-18T23:19:04.052 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.052 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.052 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.052 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.053 INFO:teuthology.orchestra.run.smithi049.stdout: "warn_on_stray_hosts": { 2024-03-18T23:19:04.053 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "warn_on_stray_hosts", 2024-03-18T23:19:04.053 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:04.053 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.053 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.054 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-18T23:19:04.054 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.054 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.054 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.054 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-03-18T23:19:04.055 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.055 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.055 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.055 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:19:04.055 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:19:04.056 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.056 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-18T23:19:04.056 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "crash", 2024-03-18T23:19:04.056 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-18T23:19:04.056 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-18T23:19:04.056 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-18T23:19:04.057 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-18T23:19:04.057 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-18T23:19:04.057 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:04.057 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.057 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:04.058 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:04.058 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.058 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.058 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-18T23:19:04.058 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-18T23:19:04.059 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-18T23:19:04.059 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-18T23:19:04.059 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-18T23:19:04.059 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-18T23:19:04.059 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-18T23:19:04.060 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-18T23:19:04.060 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.060 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.060 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.060 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.060 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.061 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-18T23:19:04.061 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-18T23:19:04.061 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:04.061 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.061 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:04.062 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:04.062 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.062 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.062 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.062 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.063 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.063 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.063 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.063 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.063 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-18T23:19:04.063 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-18T23:19:04.063 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:04.064 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.064 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:04.064 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-18T23:19:04.064 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.064 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.064 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-18T23:19:04.064 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-18T23:19:04.064 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-18T23:19:04.065 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-18T23:19:04.065 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-18T23:19:04.065 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-18T23:19:04.065 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-18T23:19:04.065 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-18T23:19:04.065 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.065 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.065 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.065 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.066 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.066 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-18T23:19:04.066 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-18T23:19:04.066 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:04.066 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.066 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:04.066 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:04.067 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.067 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.067 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.067 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.067 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.067 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.067 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.067 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.068 INFO:teuthology.orchestra.run.smithi049.stdout: "retain_interval": { 2024-03-18T23:19:04.068 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "retain_interval", 2024-03-18T23:19:04.068 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-03-18T23:19:04.068 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.068 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:04.068 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "31536000", 2024-03-18T23:19:04.068 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.068 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.068 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.069 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "how long to retain crashes before pruning them", 2024-03-18T23:19:04.069 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.069 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.069 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.069 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.069 INFO:teuthology.orchestra.run.smithi049.stdout: "warn_recent_interval": { 2024-03-18T23:19:04.069 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "warn_recent_interval", 2024-03-18T23:19:04.069 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-03-18T23:19:04.069 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.070 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:04.070 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "1209600", 2024-03-18T23:19:04.070 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.070 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.070 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.070 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "time interval in which to warn about recent crashes", 2024-03-18T23:19:04.071 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.071 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.071 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.071 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:19:04.071 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:19:04.071 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.071 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-18T23:19:04.072 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "dashboard", 2024-03-18T23:19:04.072 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-18T23:19:04.072 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-18T23:19:04.072 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-18T23:19:04.072 INFO:teuthology.orchestra.run.smithi049.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-03-18T23:19:04.072 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-03-18T23:19:04.072 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-18T23:19:04.073 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.073 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.073 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "10", 2024-03-18T23:19:04.073 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.073 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.073 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.073 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.073 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.074 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.074 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.074 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.074 INFO:teuthology.orchestra.run.smithi049.stdout: "ALERTMANAGER_API_HOST": { 2024-03-18T23:19:04.074 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "ALERTMANAGER_API_HOST", 2024-03-18T23:19:04.074 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:04.074 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.074 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.075 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:04.075 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.075 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.075 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.075 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.075 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.075 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.075 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.076 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.076 INFO:teuthology.orchestra.run.smithi049.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-03-18T23:19:04.076 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-03-18T23:19:04.076 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:04.076 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.077 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.077 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-18T23:19:04.077 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.077 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.077 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.077 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.077 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.077 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.078 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.078 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.078 INFO:teuthology.orchestra.run.smithi049.stdout: "AUDIT_API_ENABLED": { 2024-03-18T23:19:04.078 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "AUDIT_API_ENABLED", 2024-03-18T23:19:04.078 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:04.078 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.078 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.079 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:04.079 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.079 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.079 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.079 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.079 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.079 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.079 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.080 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.080 INFO:teuthology.orchestra.run.smithi049.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-03-18T23:19:04.080 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-03-18T23:19:04.080 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:04.080 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.080 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.080 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-18T23:19:04.080 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.081 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.081 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.081 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.081 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.081 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.081 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.082 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.082 INFO:teuthology.orchestra.run.smithi049.stdout: "ENABLE_BROWSABLE_API": { 2024-03-18T23:19:04.082 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "ENABLE_BROWSABLE_API", 2024-03-18T23:19:04.082 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:04.082 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.082 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.082 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-18T23:19:04.082 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.083 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.083 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.083 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.083 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.083 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.083 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.083 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.083 INFO:teuthology.orchestra.run.smithi049.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-03-18T23:19:04.083 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-03-18T23:19:04.084 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:04.084 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.084 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.084 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-18T23:19:04.084 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.084 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.084 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.084 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.085 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.085 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.085 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.085 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.085 INFO:teuthology.orchestra.run.smithi049.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-03-18T23:19:04.085 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-03-18T23:19:04.085 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:04.085 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.086 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.086 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-18T23:19:04.086 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.086 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.086 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.086 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.086 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.087 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.087 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.087 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.087 INFO:teuthology.orchestra.run.smithi049.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-03-18T23:19:04.087 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-03-18T23:19:04.087 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:04.087 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.087 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.088 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-18T23:19:04.088 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.088 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.088 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.088 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.088 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.089 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.089 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.089 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.089 INFO:teuthology.orchestra.run.smithi049.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-03-18T23:19:04.089 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-03-18T23:19:04.089 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:04.089 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.089 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.090 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-18T23:19:04.090 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.090 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.090 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.090 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.090 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.090 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.091 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.091 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.091 INFO:teuthology.orchestra.run.smithi049.stdout: "FEATURE_TOGGLE_NFS": { 2024-03-18T23:19:04.091 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-03-18T23:19:04.091 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:04.091 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.091 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.091 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-18T23:19:04.092 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.092 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.092 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.092 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.092 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.092 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.092 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.092 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.093 INFO:teuthology.orchestra.run.smithi049.stdout: "FEATURE_TOGGLE_RBD": { 2024-03-18T23:19:04.093 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-03-18T23:19:04.093 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:04.093 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.093 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.093 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-18T23:19:04.093 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.093 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.094 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.094 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.094 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.094 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.094 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.094 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.094 INFO:teuthology.orchestra.run.smithi049.stdout: "FEATURE_TOGGLE_RGW": { 2024-03-18T23:19:04.094 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-03-18T23:19:04.095 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:04.095 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.095 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.095 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-18T23:19:04.095 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.095 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.095 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.095 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.096 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.096 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.096 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.096 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.096 INFO:teuthology.orchestra.run.smithi049.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-03-18T23:19:04.096 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-03-18T23:19:04.096 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:04.096 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.097 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.097 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:04.097 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.097 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.097 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.097 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.097 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.097 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.098 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.098 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.098 INFO:teuthology.orchestra.run.smithi049.stdout: "GRAFANA_API_PASSWORD": { 2024-03-18T23:19:04.098 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "GRAFANA_API_PASSWORD", 2024-03-18T23:19:04.098 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:04.098 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.098 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.098 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "admin", 2024-03-18T23:19:04.099 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.099 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.099 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.099 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.099 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.100 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.100 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.100 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.100 INFO:teuthology.orchestra.run.smithi049.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-03-18T23:19:04.100 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-03-18T23:19:04.100 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:04.100 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.100 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.100 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-18T23:19:04.101 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.101 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.101 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.101 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.101 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.101 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.101 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.101 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.102 INFO:teuthology.orchestra.run.smithi049.stdout: "GRAFANA_API_URL": { 2024-03-18T23:19:04.102 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "GRAFANA_API_URL", 2024-03-18T23:19:04.102 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:04.102 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.102 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.102 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:04.102 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.102 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.103 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.103 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.103 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.103 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.103 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.103 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.103 INFO:teuthology.orchestra.run.smithi049.stdout: "GRAFANA_API_USERNAME": { 2024-03-18T23:19:04.103 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "GRAFANA_API_USERNAME", 2024-03-18T23:19:04.103 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:04.104 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.104 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.104 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "admin", 2024-03-18T23:19:04.104 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.104 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.104 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.104 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.105 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.105 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.105 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.105 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.105 INFO:teuthology.orchestra.run.smithi049.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-03-18T23:19:04.105 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-03-18T23:19:04.105 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:04.105 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.105 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.106 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:04.106 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.106 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.106 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.106 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.106 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.106 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.107 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.107 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.107 INFO:teuthology.orchestra.run.smithi049.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-03-18T23:19:04.107 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-03-18T23:19:04.107 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:04.107 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.107 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.108 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:04.108 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.108 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.108 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.108 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.108 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.109 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.109 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.109 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.109 INFO:teuthology.orchestra.run.smithi049.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-03-18T23:19:04.110 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-03-18T23:19:04.110 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:04.110 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.110 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.110 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-18T23:19:04.110 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.110 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.111 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.111 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.111 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.111 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.111 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.111 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.111 INFO:teuthology.orchestra.run.smithi049.stdout: "ISSUE_TRACKER_API_KEY": { 2024-03-18T23:19:04.111 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-03-18T23:19:04.112 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:04.112 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.112 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.112 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:04.112 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.112 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.112 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.113 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.113 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.113 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.113 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.113 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.113 INFO:teuthology.orchestra.run.smithi049.stdout: "MULTICLUSTER_CONFIG": { 2024-03-18T23:19:04.113 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "MULTICLUSTER_CONFIG", 2024-03-18T23:19:04.113 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:04.114 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.114 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.114 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "{}", 2024-03-18T23:19:04.114 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.114 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.114 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.114 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.115 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.115 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.115 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.115 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.115 INFO:teuthology.orchestra.run.smithi049.stdout: "PROMETHEUS_API_HOST": { 2024-03-18T23:19:04.115 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "PROMETHEUS_API_HOST", 2024-03-18T23:19:04.115 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:04.115 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.115 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.116 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:04.116 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.116 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.116 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.116 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.116 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.116 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.117 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.117 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.117 INFO:teuthology.orchestra.run.smithi049.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-03-18T23:19:04.117 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-03-18T23:19:04.117 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:04.117 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.117 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.117 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-18T23:19:04.118 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.118 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.118 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.118 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.118 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.118 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.118 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.119 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.119 INFO:teuthology.orchestra.run.smithi049.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-03-18T23:19:04.119 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-03-18T23:19:04.119 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:04.119 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.119 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.119 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:04.120 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.120 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.120 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.120 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.120 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.120 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.120 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.121 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.121 INFO:teuthology.orchestra.run.smithi049.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-03-18T23:19:04.121 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-03-18T23:19:04.121 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:04.121 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.121 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.121 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:04.121 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.122 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.122 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.122 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.122 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.122 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.122 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.122 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.122 INFO:teuthology.orchestra.run.smithi049.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-03-18T23:19:04.123 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-03-18T23:19:04.123 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:04.123 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.123 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.123 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-18T23:19:04.123 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.123 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.123 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.123 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.124 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.124 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.124 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.124 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.124 INFO:teuthology.orchestra.run.smithi049.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-03-18T23:19:04.124 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-03-18T23:19:04.124 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:04.124 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.125 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.125 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-18T23:19:04.125 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.125 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.125 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.125 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.125 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.126 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.126 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.126 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.126 INFO:teuthology.orchestra.run.smithi049.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-03-18T23:19:04.126 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-03-18T23:19:04.126 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:04.126 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.127 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.127 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:04.127 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.127 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.127 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.128 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.128 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.128 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.128 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.129 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.129 INFO:teuthology.orchestra.run.smithi049.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-03-18T23:19:04.129 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-03-18T23:19:04.129 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:04.130 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.130 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.130 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:04.130 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.131 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.131 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.131 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.131 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.132 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.132 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.132 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.133 INFO:teuthology.orchestra.run.smithi049.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-03-18T23:19:04.133 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-03-18T23:19:04.133 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:04.133 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.134 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.134 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:04.134 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.134 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.135 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.135 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.135 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.135 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.136 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.136 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.136 INFO:teuthology.orchestra.run.smithi049.stdout: "PWD_POLICY_ENABLED": { 2024-03-18T23:19:04.136 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "PWD_POLICY_ENABLED", 2024-03-18T23:19:04.137 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:04.137 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.137 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.137 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-18T23:19:04.138 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.138 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.138 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.139 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.139 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.139 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.139 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.140 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.140 INFO:teuthology.orchestra.run.smithi049.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-03-18T23:19:04.140 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-03-18T23:19:04.140 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:04.140 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.141 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.141 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-03-18T23:19:04.141 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.141 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.141 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.142 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.142 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.142 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.142 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.142 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.143 INFO:teuthology.orchestra.run.smithi049.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-03-18T23:19:04.143 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-03-18T23:19:04.143 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-18T23:19:04.143 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.144 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.144 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "10", 2024-03-18T23:19:04.144 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.144 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.145 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.145 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.145 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.145 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.145 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.145 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.145 INFO:teuthology.orchestra.run.smithi049.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-03-18T23:19:04.145 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-03-18T23:19:04.146 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-18T23:19:04.146 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.146 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.146 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "8", 2024-03-18T23:19:04.146 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.146 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.146 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.147 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.147 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.147 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.147 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.147 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.147 INFO:teuthology.orchestra.run.smithi049.stdout: "REST_REQUESTS_TIMEOUT": { 2024-03-18T23:19:04.147 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-03-18T23:19:04.147 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-18T23:19:04.148 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.148 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.148 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "45", 2024-03-18T23:19:04.148 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.148 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.148 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.148 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.148 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.148 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.149 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.149 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.149 INFO:teuthology.orchestra.run.smithi049.stdout: "RGW_API_ACCESS_KEY": { 2024-03-18T23:19:04.149 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "RGW_API_ACCESS_KEY", 2024-03-18T23:19:04.149 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:04.149 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.149 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.149 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:04.150 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.150 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.150 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.150 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.150 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.150 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.150 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.151 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.151 INFO:teuthology.orchestra.run.smithi049.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-03-18T23:19:04.151 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-03-18T23:19:04.151 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:04.151 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.151 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.151 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "admin", 2024-03-18T23:19:04.152 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.152 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.152 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.152 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.152 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.152 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.152 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.152 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.152 INFO:teuthology.orchestra.run.smithi049.stdout: "RGW_API_SECRET_KEY": { 2024-03-18T23:19:04.153 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "RGW_API_SECRET_KEY", 2024-03-18T23:19:04.153 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:04.153 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.153 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.153 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:04.153 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.153 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.154 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.154 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.154 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.154 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.154 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.154 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.154 INFO:teuthology.orchestra.run.smithi049.stdout: "RGW_API_SSL_VERIFY": { 2024-03-18T23:19:04.155 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "RGW_API_SSL_VERIFY", 2024-03-18T23:19:04.155 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:04.155 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.155 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.155 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-18T23:19:04.155 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.155 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.155 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.155 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.156 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.156 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.156 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.156 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.156 INFO:teuthology.orchestra.run.smithi049.stdout: "UNSAFE_TLS_v1_2": { 2024-03-18T23:19:04.156 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "UNSAFE_TLS_v1_2", 2024-03-18T23:19:04.156 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:04.156 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.156 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.157 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:04.157 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.157 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.157 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.157 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.157 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.157 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.157 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.157 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.158 INFO:teuthology.orchestra.run.smithi049.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-03-18T23:19:04.158 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-03-18T23:19:04.158 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-18T23:19:04.158 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.158 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.158 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "0", 2024-03-18T23:19:04.158 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.158 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.159 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.159 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.159 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.159 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.159 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.159 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.159 INFO:teuthology.orchestra.run.smithi049.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-03-18T23:19:04.159 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-03-18T23:19:04.160 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-18T23:19:04.160 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.160 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.160 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "10", 2024-03-18T23:19:04.160 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.160 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.160 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.160 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.160 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.161 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.161 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.161 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.161 INFO:teuthology.orchestra.run.smithi049.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-03-18T23:19:04.161 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-03-18T23:19:04.161 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-18T23:19:04.161 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.161 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.161 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "5", 2024-03-18T23:19:04.162 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.162 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.162 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.162 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.162 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.162 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.162 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.162 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.163 INFO:teuthology.orchestra.run.smithi049.stdout: "cross_origin_url": { 2024-03-18T23:19:04.163 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "cross_origin_url", 2024-03-18T23:19:04.163 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:04.163 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.163 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.163 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:04.163 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.163 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.163 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.164 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.164 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.164 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.164 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.164 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.164 INFO:teuthology.orchestra.run.smithi049.stdout: "crt_file": { 2024-03-18T23:19:04.164 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "crt_file", 2024-03-18T23:19:04.164 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:04.165 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.165 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.165 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:04.165 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.165 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.165 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.165 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.166 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.166 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.166 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.166 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.166 INFO:teuthology.orchestra.run.smithi049.stdout: "debug": { 2024-03-18T23:19:04.166 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "debug", 2024-03-18T23:19:04.166 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:04.166 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.167 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.167 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:04.167 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.167 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.167 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.167 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Enable/disable debug options", 2024-03-18T23:19:04.167 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.167 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.168 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.168 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.168 INFO:teuthology.orchestra.run.smithi049.stdout: "jwt_token_ttl": { 2024-03-18T23:19:04.168 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "jwt_token_ttl", 2024-03-18T23:19:04.168 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-18T23:19:04.168 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.168 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.168 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "28800", 2024-03-18T23:19:04.168 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.169 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.169 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.169 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.169 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.169 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.169 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.169 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.169 INFO:teuthology.orchestra.run.smithi049.stdout: "key_file": { 2024-03-18T23:19:04.170 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "key_file", 2024-03-18T23:19:04.170 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:04.170 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.170 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.170 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:04.170 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.171 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.171 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.171 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.171 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.171 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.171 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.171 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.171 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-18T23:19:04.171 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-18T23:19:04.172 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:04.172 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.172 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:04.172 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:04.172 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.172 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.172 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-18T23:19:04.172 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-18T23:19:04.173 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-18T23:19:04.173 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-18T23:19:04.173 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-18T23:19:04.173 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-18T23:19:04.173 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-18T23:19:04.173 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-18T23:19:04.173 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.174 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.174 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.174 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.174 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.174 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-18T23:19:04.175 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-18T23:19:04.175 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:04.175 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.175 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:04.175 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:04.175 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.175 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.175 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.175 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.176 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.176 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.176 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.176 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.176 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-18T23:19:04.177 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-18T23:19:04.177 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:04.177 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.177 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:04.177 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-18T23:19:04.177 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.178 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.178 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-18T23:19:04.178 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-18T23:19:04.178 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-18T23:19:04.178 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-18T23:19:04.179 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-18T23:19:04.179 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-18T23:19:04.179 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-18T23:19:04.179 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-18T23:19:04.179 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.179 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.179 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.179 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.180 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.180 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-18T23:19:04.180 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-18T23:19:04.180 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:04.180 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.180 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:04.180 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:04.181 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.181 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.181 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.181 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.182 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.182 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.182 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.182 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.182 INFO:teuthology.orchestra.run.smithi049.stdout: "motd": { 2024-03-18T23:19:04.182 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "motd", 2024-03-18T23:19:04.182 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:04.183 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.183 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.183 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:04.183 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.183 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.183 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.183 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "The message of the day", 2024-03-18T23:19:04.183 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.184 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.184 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.184 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.184 INFO:teuthology.orchestra.run.smithi049.stdout: "redirect_resolve_ip_addr": { 2024-03-18T23:19:04.184 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "redirect_resolve_ip_addr", 2024-03-18T23:19:04.184 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:04.184 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.184 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.185 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:04.185 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.185 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.185 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.185 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.185 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.185 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.185 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.186 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.186 INFO:teuthology.orchestra.run.smithi049.stdout: "server_addr": { 2024-03-18T23:19:04.186 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "server_addr", 2024-03-18T23:19:04.186 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:04.186 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.186 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.186 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "::", 2024-03-18T23:19:04.186 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.187 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.187 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.187 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.187 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.188 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.188 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.188 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.188 INFO:teuthology.orchestra.run.smithi049.stdout: "server_port": { 2024-03-18T23:19:04.188 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "server_port", 2024-03-18T23:19:04.188 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-18T23:19:04.188 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.189 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.189 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "8080", 2024-03-18T23:19:04.189 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.189 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.189 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.189 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.189 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.190 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.190 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.190 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.190 INFO:teuthology.orchestra.run.smithi049.stdout: "ssl": { 2024-03-18T23:19:04.190 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "ssl", 2024-03-18T23:19:04.190 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:04.190 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.190 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.191 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-18T23:19:04.191 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.191 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.191 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.191 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.191 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.192 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.192 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.192 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.192 INFO:teuthology.orchestra.run.smithi049.stdout: "ssl_server_port": { 2024-03-18T23:19:04.192 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "ssl_server_port", 2024-03-18T23:19:04.192 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-18T23:19:04.192 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.192 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.193 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "8443", 2024-03-18T23:19:04.193 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.193 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.193 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.193 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.193 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.193 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.193 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.193 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.194 INFO:teuthology.orchestra.run.smithi049.stdout: "standby_behaviour": { 2024-03-18T23:19:04.194 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "standby_behaviour", 2024-03-18T23:19:04.194 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:04.194 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.194 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.194 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "redirect", 2024-03-18T23:19:04.195 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.195 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.195 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-18T23:19:04.195 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-18T23:19:04.195 INFO:teuthology.orchestra.run.smithi049.stdout: "redirect" 2024-03-18T23:19:04.195 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-18T23:19:04.195 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.195 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.196 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.196 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.196 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.196 INFO:teuthology.orchestra.run.smithi049.stdout: "standby_error_status_code": { 2024-03-18T23:19:04.196 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "standby_error_status_code", 2024-03-18T23:19:04.196 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-18T23:19:04.196 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.196 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.197 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "500", 2024-03-18T23:19:04.197 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "400", 2024-03-18T23:19:04.197 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "599", 2024-03-18T23:19:04.197 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.197 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.197 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.197 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.197 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.198 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.198 INFO:teuthology.orchestra.run.smithi049.stdout: "url_prefix": { 2024-03-18T23:19:04.198 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "url_prefix", 2024-03-18T23:19:04.198 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:04.198 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.198 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.199 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:04.199 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.199 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.199 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.199 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.199 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.199 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.200 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.200 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:19:04.200 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:19:04.200 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.200 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-18T23:19:04.200 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "devicehealth", 2024-03-18T23:19:04.200 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-18T23:19:04.200 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-18T23:19:04.200 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-18T23:19:04.201 INFO:teuthology.orchestra.run.smithi049.stdout: "enable_monitoring": { 2024-03-18T23:19:04.201 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "enable_monitoring", 2024-03-18T23:19:04.201 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:04.201 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.201 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:04.201 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-18T23:19:04.201 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.201 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.201 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.202 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "monitor device health metrics", 2024-03-18T23:19:04.202 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.202 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.202 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.202 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.202 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-18T23:19:04.203 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-18T23:19:04.203 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:04.203 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.203 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:04.203 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:04.203 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.203 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.203 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-18T23:19:04.204 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-18T23:19:04.204 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-18T23:19:04.204 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-18T23:19:04.204 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-18T23:19:04.204 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-18T23:19:04.204 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-18T23:19:04.204 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-18T23:19:04.204 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.205 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.205 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.205 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.205 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.205 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-18T23:19:04.205 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-18T23:19:04.206 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:04.206 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.206 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:04.206 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:04.206 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.206 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.206 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.207 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.207 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.207 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.207 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.207 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.207 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-18T23:19:04.207 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-18T23:19:04.208 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:04.208 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.208 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:04.208 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-18T23:19:04.208 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.209 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.209 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-18T23:19:04.209 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-18T23:19:04.209 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-18T23:19:04.209 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-18T23:19:04.209 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-18T23:19:04.209 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-18T23:19:04.210 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-18T23:19:04.210 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-18T23:19:04.210 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.210 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.210 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.211 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.211 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.211 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-18T23:19:04.211 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-18T23:19:04.211 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:04.211 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.211 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:04.212 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:04.212 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.212 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.212 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.212 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.212 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.212 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.213 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.213 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.213 INFO:teuthology.orchestra.run.smithi049.stdout: "mark_out_threshold": { 2024-03-18T23:19:04.213 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "mark_out_threshold", 2024-03-18T23:19:04.213 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-03-18T23:19:04.214 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.214 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:04.214 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "2419200", 2024-03-18T23:19:04.214 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.214 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.214 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.214 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-03-18T23:19:04.215 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.215 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.215 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.215 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.215 INFO:teuthology.orchestra.run.smithi049.stdout: "pool_name": { 2024-03-18T23:19:04.215 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "pool_name", 2024-03-18T23:19:04.215 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:04.215 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.216 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:04.216 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "device_health_metrics", 2024-03-18T23:19:04.216 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.216 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.216 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.216 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "name of pool in which to store device health metrics", 2024-03-18T23:19:04.216 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.217 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.217 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.217 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.217 INFO:teuthology.orchestra.run.smithi049.stdout: "retention_period": { 2024-03-18T23:19:04.217 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "retention_period", 2024-03-18T23:19:04.217 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-03-18T23:19:04.217 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.217 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:04.218 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "15552000", 2024-03-18T23:19:04.218 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.218 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.218 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.218 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "how long to retain device health metrics", 2024-03-18T23:19:04.218 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.218 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.218 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.219 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.219 INFO:teuthology.orchestra.run.smithi049.stdout: "scrape_frequency": { 2024-03-18T23:19:04.219 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "scrape_frequency", 2024-03-18T23:19:04.219 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-03-18T23:19:04.219 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.219 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:04.219 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "86400", 2024-03-18T23:19:04.219 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.219 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.220 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.220 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "how frequently to scrape device health metrics", 2024-03-18T23:19:04.220 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.220 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.220 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.221 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.221 INFO:teuthology.orchestra.run.smithi049.stdout: "self_heal": { 2024-03-18T23:19:04.221 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "self_heal", 2024-03-18T23:19:04.221 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:04.221 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.221 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:04.221 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-18T23:19:04.221 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.221 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.222 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.222 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-03-18T23:19:04.222 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.222 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.223 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.223 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.223 INFO:teuthology.orchestra.run.smithi049.stdout: "sleep_interval": { 2024-03-18T23:19:04.223 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "sleep_interval", 2024-03-18T23:19:04.224 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-03-18T23:19:04.224 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.224 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:04.224 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "600", 2024-03-18T23:19:04.224 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.224 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.224 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.224 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "how frequently to wake up and check device health", 2024-03-18T23:19:04.225 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.225 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.225 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.225 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.225 INFO:teuthology.orchestra.run.smithi049.stdout: "warn_threshold": { 2024-03-18T23:19:04.225 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "warn_threshold", 2024-03-18T23:19:04.225 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-03-18T23:19:04.225 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.225 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:04.226 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "7257600", 2024-03-18T23:19:04.226 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.226 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.226 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.226 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-03-18T23:19:04.226 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.226 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.227 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.227 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:19:04.227 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:19:04.227 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.227 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-18T23:19:04.227 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "diskprediction_local", 2024-03-18T23:19:04.228 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-18T23:19:04.228 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-18T23:19:04.228 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-18T23:19:04.228 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-18T23:19:04.228 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-18T23:19:04.228 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:04.228 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.228 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:04.229 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:04.229 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.229 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.229 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-18T23:19:04.229 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-18T23:19:04.229 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-18T23:19:04.229 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-18T23:19:04.229 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-18T23:19:04.230 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-18T23:19:04.230 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-18T23:19:04.230 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-18T23:19:04.230 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.230 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.231 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.231 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.231 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.231 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-18T23:19:04.231 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-18T23:19:04.231 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:04.231 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.232 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:04.232 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:04.232 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.232 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.232 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.232 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.232 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.232 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.233 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.233 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.233 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-18T23:19:04.233 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-18T23:19:04.233 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:04.233 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.234 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:04.234 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-18T23:19:04.234 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.234 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.235 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-18T23:19:04.235 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-18T23:19:04.235 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-18T23:19:04.235 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-18T23:19:04.235 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-18T23:19:04.235 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-18T23:19:04.235 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-18T23:19:04.235 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-18T23:19:04.235 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.236 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.236 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.236 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.236 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.236 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-18T23:19:04.236 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-18T23:19:04.236 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:04.236 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.236 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:04.237 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:04.237 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.237 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.237 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.237 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.238 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.238 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.238 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.238 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.238 INFO:teuthology.orchestra.run.smithi049.stdout: "predict_interval": { 2024-03-18T23:19:04.238 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "predict_interval", 2024-03-18T23:19:04.238 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:04.239 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.239 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.239 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "86400", 2024-03-18T23:19:04.239 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.239 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.239 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.239 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.239 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.240 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.240 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.240 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.240 INFO:teuthology.orchestra.run.smithi049.stdout: "predictor_model": { 2024-03-18T23:19:04.240 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "predictor_model", 2024-03-18T23:19:04.240 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:04.240 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.240 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.240 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "prophetstor", 2024-03-18T23:19:04.241 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.241 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.241 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.241 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.241 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.241 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.242 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.242 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.242 INFO:teuthology.orchestra.run.smithi049.stdout: "sleep_interval": { 2024-03-18T23:19:04.242 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "sleep_interval", 2024-03-18T23:19:04.242 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:04.242 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.243 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.243 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "600", 2024-03-18T23:19:04.243 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.243 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.243 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.243 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.243 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.243 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.244 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.244 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:19:04.244 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:19:04.244 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.244 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-18T23:19:04.244 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "influx", 2024-03-18T23:19:04.244 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": false, 2024-03-18T23:19:04.244 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "influxdb python module not found", 2024-03-18T23:19:04.245 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-18T23:19:04.245 INFO:teuthology.orchestra.run.smithi049.stdout: "batch_size": { 2024-03-18T23:19:04.245 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "batch_size", 2024-03-18T23:19:04.245 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-18T23:19:04.245 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.245 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.246 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "5000", 2024-03-18T23:19:04.246 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.246 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.246 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.246 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-03-18T23:19:04.246 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.247 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.247 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.247 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.247 INFO:teuthology.orchestra.run.smithi049.stdout: "database": { 2024-03-18T23:19:04.248 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "database", 2024-03-18T23:19:04.248 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:04.248 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.248 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.249 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "ceph", 2024-03-18T23:19:04.249 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.249 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.249 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.249 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-03-18T23:19:04.249 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.249 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.250 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.250 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.250 INFO:teuthology.orchestra.run.smithi049.stdout: "hostname": { 2024-03-18T23:19:04.250 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "hostname", 2024-03-18T23:19:04.250 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:04.250 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.250 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.250 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:04.251 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.251 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.251 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.251 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "InfluxDB server hostname", 2024-03-18T23:19:04.251 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.251 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.251 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.252 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.252 INFO:teuthology.orchestra.run.smithi049.stdout: "interval": { 2024-03-18T23:19:04.252 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "interval", 2024-03-18T23:19:04.252 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-03-18T23:19:04.252 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.252 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.252 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "30", 2024-03-18T23:19:04.252 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "5", 2024-03-18T23:19:04.253 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.253 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.253 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-03-18T23:19:04.253 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.253 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.253 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.253 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.253 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-18T23:19:04.254 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-18T23:19:04.254 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:04.254 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.254 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:04.254 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:04.254 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.254 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.255 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-18T23:19:04.255 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-18T23:19:04.255 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-18T23:19:04.255 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-18T23:19:04.255 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-18T23:19:04.256 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-18T23:19:04.256 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-18T23:19:04.256 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-18T23:19:04.256 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.256 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.256 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.256 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.257 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.257 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-18T23:19:04.257 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-18T23:19:04.257 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:04.257 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.257 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:04.257 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:04.257 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.258 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.258 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.258 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.258 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.258 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.259 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.259 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.259 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-18T23:19:04.259 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-18T23:19:04.259 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:04.259 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.259 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:04.260 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-18T23:19:04.260 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.260 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.260 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-18T23:19:04.260 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-18T23:19:04.260 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-18T23:19:04.260 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-18T23:19:04.261 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-18T23:19:04.261 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-18T23:19:04.261 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-18T23:19:04.261 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-18T23:19:04.261 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.261 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.261 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.262 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.262 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.262 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-18T23:19:04.262 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-18T23:19:04.262 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:04.263 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.263 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:04.263 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:04.263 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.263 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.263 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.263 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.263 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.264 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.264 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.264 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.264 INFO:teuthology.orchestra.run.smithi049.stdout: "password": { 2024-03-18T23:19:04.264 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "password", 2024-03-18T23:19:04.264 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:04.264 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.265 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.265 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:04.265 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.265 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.265 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.265 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "password of InfluxDB server user", 2024-03-18T23:19:04.265 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.266 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.266 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.266 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.266 INFO:teuthology.orchestra.run.smithi049.stdout: "port": { 2024-03-18T23:19:04.266 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "port", 2024-03-18T23:19:04.266 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-18T23:19:04.266 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.267 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.267 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "8086", 2024-03-18T23:19:04.267 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.267 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.267 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.267 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "InfluxDB server port", 2024-03-18T23:19:04.267 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.267 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.268 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.268 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.268 INFO:teuthology.orchestra.run.smithi049.stdout: "ssl": { 2024-03-18T23:19:04.268 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "ssl", 2024-03-18T23:19:04.268 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:04.268 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.268 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.268 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "false", 2024-03-18T23:19:04.268 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.269 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.269 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.269 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-03-18T23:19:04.269 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.269 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.270 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.270 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.270 INFO:teuthology.orchestra.run.smithi049.stdout: "threads": { 2024-03-18T23:19:04.270 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "threads", 2024-03-18T23:19:04.270 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-18T23:19:04.270 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.270 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.270 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "5", 2024-03-18T23:19:04.271 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "1", 2024-03-18T23:19:04.271 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "32", 2024-03-18T23:19:04.271 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.271 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-03-18T23:19:04.271 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.271 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.271 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.271 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.272 INFO:teuthology.orchestra.run.smithi049.stdout: "username": { 2024-03-18T23:19:04.272 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "username", 2024-03-18T23:19:04.272 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:04.272 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.272 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.272 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:04.272 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.273 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.273 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.273 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "username of InfluxDB server user", 2024-03-18T23:19:04.273 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.273 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.273 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.273 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.273 INFO:teuthology.orchestra.run.smithi049.stdout: "verify_ssl": { 2024-03-18T23:19:04.274 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "verify_ssl", 2024-03-18T23:19:04.274 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:04.274 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.274 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.274 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "true", 2024-03-18T23:19:04.275 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.275 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.275 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.275 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-03-18T23:19:04.275 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.275 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.275 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.275 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:19:04.276 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:19:04.276 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.276 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-18T23:19:04.276 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "insights", 2024-03-18T23:19:04.276 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-18T23:19:04.276 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-18T23:19:04.276 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-18T23:19:04.276 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-18T23:19:04.277 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-18T23:19:04.277 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:04.277 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.277 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:04.277 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:04.277 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.277 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.277 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-18T23:19:04.277 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-18T23:19:04.278 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-18T23:19:04.278 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-18T23:19:04.278 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-18T23:19:04.278 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-18T23:19:04.278 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-18T23:19:04.278 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-18T23:19:04.278 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.279 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.279 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.279 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.279 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.279 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-18T23:19:04.279 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-18T23:19:04.280 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:04.280 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.280 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:04.280 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:04.280 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.280 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.280 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.280 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.280 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.281 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.281 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.281 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.281 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-18T23:19:04.281 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-18T23:19:04.281 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:04.281 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.282 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:04.282 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-18T23:19:04.282 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.282 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.282 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-18T23:19:04.282 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-18T23:19:04.282 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-18T23:19:04.283 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-18T23:19:04.283 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-18T23:19:04.283 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-18T23:19:04.283 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-18T23:19:04.283 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-18T23:19:04.283 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.284 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.284 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.284 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.284 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.284 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-18T23:19:04.284 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-18T23:19:04.284 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:04.285 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.285 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:04.285 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:04.285 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.285 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.285 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.285 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.285 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.285 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.286 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.286 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:19:04.286 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:19:04.286 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.286 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-18T23:19:04.286 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "iostat", 2024-03-18T23:19:04.286 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-18T23:19:04.287 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-18T23:19:04.287 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-18T23:19:04.287 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-18T23:19:04.287 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-18T23:19:04.287 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:04.287 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.287 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:04.287 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:04.287 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.288 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.288 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-18T23:19:04.288 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-18T23:19:04.288 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-18T23:19:04.288 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-18T23:19:04.288 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-18T23:19:04.288 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-18T23:19:04.289 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-18T23:19:04.289 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-18T23:19:04.289 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.289 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.289 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.289 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.289 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.289 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-18T23:19:04.290 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-18T23:19:04.290 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:04.290 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.290 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:04.290 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:04.290 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.291 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.291 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.291 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.291 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.291 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.291 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.291 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.291 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-18T23:19:04.292 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-18T23:19:04.292 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:04.292 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.292 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:04.292 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-18T23:19:04.292 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.292 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.292 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-18T23:19:04.292 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-18T23:19:04.293 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-18T23:19:04.293 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-18T23:19:04.293 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-18T23:19:04.293 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-18T23:19:04.293 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-18T23:19:04.293 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-18T23:19:04.294 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.294 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.294 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.294 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.294 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.294 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-18T23:19:04.294 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-18T23:19:04.295 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:04.295 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.295 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:04.295 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:04.295 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.295 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.295 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.295 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.296 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.296 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.296 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.296 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:19:04.296 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:19:04.296 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.296 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-18T23:19:04.296 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "k8sevents", 2024-03-18T23:19:04.297 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-18T23:19:04.297 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-18T23:19:04.297 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-18T23:19:04.297 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph_event_retention_days": { 2024-03-18T23:19:04.297 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "ceph_event_retention_days", 2024-03-18T23:19:04.297 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-18T23:19:04.297 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.297 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.298 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "7", 2024-03-18T23:19:04.298 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.298 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.298 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.298 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Days to hold ceph event information within local cache", 2024-03-18T23:19:04.298 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.299 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.299 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.299 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.299 INFO:teuthology.orchestra.run.smithi049.stdout: "config_check_secs": { 2024-03-18T23:19:04.299 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "config_check_secs", 2024-03-18T23:19:04.299 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-18T23:19:04.299 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.299 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.300 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "10", 2024-03-18T23:19:04.300 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "10", 2024-03-18T23:19:04.300 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.300 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.300 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-03-18T23:19:04.301 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.301 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.301 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.301 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.301 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-18T23:19:04.301 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-18T23:19:04.302 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:04.302 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.302 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:04.302 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:04.302 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.302 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.302 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-18T23:19:04.302 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-18T23:19:04.303 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-18T23:19:04.303 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-18T23:19:04.303 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-18T23:19:04.303 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-18T23:19:04.303 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-18T23:19:04.303 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-18T23:19:04.303 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.304 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.304 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.304 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.304 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.304 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-18T23:19:04.304 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-18T23:19:04.305 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:04.305 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.305 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:04.305 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:04.305 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.305 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.306 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.306 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.306 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.306 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.306 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.306 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.306 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-18T23:19:04.306 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-18T23:19:04.307 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:04.307 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.307 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:04.307 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-18T23:19:04.307 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.307 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.307 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-18T23:19:04.307 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-18T23:19:04.308 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-18T23:19:04.308 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-18T23:19:04.308 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-18T23:19:04.308 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-18T23:19:04.308 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-18T23:19:04.308 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-18T23:19:04.308 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.308 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.309 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.309 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.309 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.309 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-18T23:19:04.309 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-18T23:19:04.309 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:04.309 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.309 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:04.310 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:04.310 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.310 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.310 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.310 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.310 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.310 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.311 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.311 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:19:04.311 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:19:04.311 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.311 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-18T23:19:04.311 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "localpool", 2024-03-18T23:19:04.311 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-18T23:19:04.312 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-18T23:19:04.312 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-18T23:19:04.312 INFO:teuthology.orchestra.run.smithi049.stdout: "failure_domain": { 2024-03-18T23:19:04.312 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "failure_domain", 2024-03-18T23:19:04.312 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:04.312 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.312 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:04.312 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "host", 2024-03-18T23:19:04.313 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.313 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.313 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.313 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "failure domain for any created local pool", 2024-03-18T23:19:04.313 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-03-18T23:19:04.313 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.313 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.313 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.314 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-18T23:19:04.314 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-18T23:19:04.314 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:04.314 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.314 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:04.314 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:04.315 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.315 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.315 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-18T23:19:04.315 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-18T23:19:04.315 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-18T23:19:04.315 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-18T23:19:04.315 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-18T23:19:04.315 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-18T23:19:04.316 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-18T23:19:04.316 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-18T23:19:04.316 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.316 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.316 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.316 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.316 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.317 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-18T23:19:04.317 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-18T23:19:04.317 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:04.317 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.317 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:04.317 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:04.317 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.317 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.318 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.318 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.318 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.318 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.318 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.318 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.318 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-18T23:19:04.318 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-18T23:19:04.319 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:04.319 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.319 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:04.319 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-18T23:19:04.319 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.319 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.319 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-18T23:19:04.319 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-18T23:19:04.320 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-18T23:19:04.320 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-18T23:19:04.320 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-18T23:19:04.320 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-18T23:19:04.320 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-18T23:19:04.320 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-18T23:19:04.320 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.320 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.321 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.321 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.321 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.321 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-18T23:19:04.321 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-18T23:19:04.321 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:04.321 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.322 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:04.322 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:04.322 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.322 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.322 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.322 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.322 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.323 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.323 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.323 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.323 INFO:teuthology.orchestra.run.smithi049.stdout: "min_size": { 2024-03-18T23:19:04.323 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "min_size", 2024-03-18T23:19:04.323 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-18T23:19:04.323 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.323 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:04.324 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:04.324 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.324 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.324 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.324 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "default min_size for any created local pool", 2024-03-18T23:19:04.324 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-03-18T23:19:04.324 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.324 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.325 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.325 INFO:teuthology.orchestra.run.smithi049.stdout: "num_rep": { 2024-03-18T23:19:04.325 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "num_rep", 2024-03-18T23:19:04.325 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-18T23:19:04.325 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.325 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:04.325 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "3", 2024-03-18T23:19:04.326 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.326 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.326 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.326 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "default replica count for any created local pool", 2024-03-18T23:19:04.326 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.326 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.326 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.327 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.327 INFO:teuthology.orchestra.run.smithi049.stdout: "pg_num": { 2024-03-18T23:19:04.327 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "pg_num", 2024-03-18T23:19:04.327 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-18T23:19:04.327 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.327 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:04.327 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "128", 2024-03-18T23:19:04.327 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.327 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.328 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.328 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "default pg_num for any created local pool", 2024-03-18T23:19:04.328 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.328 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.328 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.328 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.328 INFO:teuthology.orchestra.run.smithi049.stdout: "prefix": { 2024-03-18T23:19:04.328 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "prefix", 2024-03-18T23:19:04.328 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:04.329 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.329 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:04.329 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:04.329 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.329 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.329 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.329 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "name prefix for any created local pool", 2024-03-18T23:19:04.329 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.329 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.329 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.330 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.330 INFO:teuthology.orchestra.run.smithi049.stdout: "subtree": { 2024-03-18T23:19:04.330 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "subtree", 2024-03-18T23:19:04.330 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:04.330 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.330 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:04.330 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "rack", 2024-03-18T23:19:04.330 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.331 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.331 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.331 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "CRUSH level for which to create a local pool", 2024-03-18T23:19:04.331 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-03-18T23:19:04.331 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.331 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.331 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:19:04.331 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:19:04.332 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.332 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-18T23:19:04.332 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "mds_autoscaler", 2024-03-18T23:19:04.332 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-18T23:19:04.332 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-18T23:19:04.332 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-18T23:19:04.332 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-18T23:19:04.332 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-18T23:19:04.333 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:04.333 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.333 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:04.333 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:04.333 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.333 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.334 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-18T23:19:04.334 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-18T23:19:04.334 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-18T23:19:04.334 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-18T23:19:04.334 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-18T23:19:04.334 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-18T23:19:04.334 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-18T23:19:04.335 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-18T23:19:04.335 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.335 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.335 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.335 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.335 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.336 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-18T23:19:04.336 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-18T23:19:04.336 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:04.336 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.336 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:04.336 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:04.336 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.336 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.337 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.337 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.337 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.337 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.337 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.337 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.337 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-18T23:19:04.337 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-18T23:19:04.338 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:04.338 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.338 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:04.338 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-18T23:19:04.338 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.338 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.338 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-18T23:19:04.338 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-18T23:19:04.339 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-18T23:19:04.339 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-18T23:19:04.339 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-18T23:19:04.339 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-18T23:19:04.339 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-18T23:19:04.339 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-18T23:19:04.339 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.339 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.339 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.340 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.340 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.340 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-18T23:19:04.340 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-18T23:19:04.340 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:04.340 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.340 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:04.341 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:04.341 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.341 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.341 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.341 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.342 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.342 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.342 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.342 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:19:04.343 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:19:04.343 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.343 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-18T23:19:04.343 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "mirroring", 2024-03-18T23:19:04.343 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-18T23:19:04.343 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-18T23:19:04.344 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-18T23:19:04.344 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-18T23:19:04.344 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-18T23:19:04.344 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:04.344 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.344 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:04.344 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:04.344 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.344 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.345 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-18T23:19:04.345 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-18T23:19:04.345 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-18T23:19:04.345 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-18T23:19:04.345 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-18T23:19:04.345 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-18T23:19:04.345 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-18T23:19:04.346 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-18T23:19:04.346 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.346 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.346 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.346 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.346 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.347 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-18T23:19:04.347 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-18T23:19:04.347 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:04.347 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.347 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:04.347 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:04.347 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.347 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.348 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.348 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.348 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.348 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.348 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.348 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.348 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-18T23:19:04.349 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-18T23:19:04.349 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:04.349 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.349 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:04.349 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-18T23:19:04.349 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.350 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.350 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-18T23:19:04.350 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-18T23:19:04.350 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-18T23:19:04.350 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-18T23:19:04.351 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-18T23:19:04.351 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-18T23:19:04.351 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-18T23:19:04.351 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-18T23:19:04.351 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.352 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.352 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.352 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.352 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.353 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-18T23:19:04.353 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-18T23:19:04.353 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:04.353 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.354 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:04.354 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:04.354 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.355 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.355 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.355 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.355 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.356 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.356 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.356 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:19:04.356 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:19:04.356 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.357 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-18T23:19:04.357 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "nfs", 2024-03-18T23:19:04.357 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-18T23:19:04.357 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-18T23:19:04.358 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-18T23:19:04.358 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-18T23:19:04.358 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-18T23:19:04.359 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:04.359 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.359 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:04.359 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:04.360 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.360 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.360 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-18T23:19:04.360 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-18T23:19:04.360 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-18T23:19:04.360 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-18T23:19:04.360 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-18T23:19:04.361 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-18T23:19:04.361 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-18T23:19:04.361 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-18T23:19:04.361 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.361 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.361 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.361 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.361 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.361 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-18T23:19:04.362 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-18T23:19:04.362 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:04.362 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.362 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:04.362 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:04.362 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.362 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.362 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.363 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.363 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.363 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.363 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.363 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.363 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-18T23:19:04.363 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-18T23:19:04.363 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:04.364 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.364 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:04.364 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-18T23:19:04.364 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.364 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.364 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-18T23:19:04.365 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-18T23:19:04.365 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-18T23:19:04.365 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-18T23:19:04.365 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-18T23:19:04.365 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-18T23:19:04.365 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-18T23:19:04.365 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-18T23:19:04.366 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.366 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.366 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.366 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.366 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.366 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-18T23:19:04.366 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-18T23:19:04.366 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:04.367 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.367 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:04.367 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:04.367 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.367 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.367 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.368 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.368 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.368 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.368 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.368 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:19:04.368 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:19:04.368 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.368 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-18T23:19:04.368 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "orchestrator", 2024-03-18T23:19:04.369 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-18T23:19:04.369 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-18T23:19:04.369 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-18T23:19:04.369 INFO:teuthology.orchestra.run.smithi049.stdout: "fail_fs": { 2024-03-18T23:19:04.369 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "fail_fs", 2024-03-18T23:19:04.369 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:04.369 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.369 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.369 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:04.369 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.370 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.370 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.370 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-03-18T23:19:04.370 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.370 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.370 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.370 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.370 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-18T23:19:04.371 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-18T23:19:04.371 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:04.371 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.371 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:04.371 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:04.371 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.371 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.371 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-18T23:19:04.372 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-18T23:19:04.372 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-18T23:19:04.372 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-18T23:19:04.372 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-18T23:19:04.372 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-18T23:19:04.372 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-18T23:19:04.372 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-18T23:19:04.372 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.373 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.373 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.373 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.373 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.373 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-18T23:19:04.373 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-18T23:19:04.373 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:04.373 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.374 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:04.374 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:04.374 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.374 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.374 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.374 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.374 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.374 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.375 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.375 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.375 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-18T23:19:04.375 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-18T23:19:04.375 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:04.375 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.376 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:04.376 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-18T23:19:04.376 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.376 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.376 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-18T23:19:04.376 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-18T23:19:04.376 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-18T23:19:04.376 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-18T23:19:04.377 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-18T23:19:04.377 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-18T23:19:04.377 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-18T23:19:04.377 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-18T23:19:04.377 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.377 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.378 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.378 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.378 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.378 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-18T23:19:04.378 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-18T23:19:04.379 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:04.379 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.379 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:04.379 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:04.379 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.380 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.380 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.380 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.380 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.380 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.380 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.381 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.381 INFO:teuthology.orchestra.run.smithi049.stdout: "orchestrator": { 2024-03-18T23:19:04.381 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "orchestrator", 2024-03-18T23:19:04.381 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:04.381 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.382 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:04.382 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:04.382 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.382 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.383 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-18T23:19:04.383 INFO:teuthology.orchestra.run.smithi049.stdout: "cephadm", 2024-03-18T23:19:04.383 INFO:teuthology.orchestra.run.smithi049.stdout: "rook", 2024-03-18T23:19:04.383 INFO:teuthology.orchestra.run.smithi049.stdout: "test_orchestrator" 2024-03-18T23:19:04.383 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-18T23:19:04.384 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Orchestrator backend", 2024-03-18T23:19:04.384 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.384 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.384 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.384 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:19:04.385 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:19:04.385 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.385 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-18T23:19:04.385 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "osd_perf_query", 2024-03-18T23:19:04.385 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-18T23:19:04.386 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-18T23:19:04.386 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-18T23:19:04.386 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-18T23:19:04.386 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-18T23:19:04.387 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:04.387 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.387 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:04.387 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:04.387 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.388 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.388 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-18T23:19:04.388 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-18T23:19:04.388 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-18T23:19:04.388 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-18T23:19:04.389 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-18T23:19:04.389 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-18T23:19:04.389 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-18T23:19:04.389 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-18T23:19:04.389 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.390 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.390 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.390 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.390 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.390 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-18T23:19:04.390 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-18T23:19:04.391 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:04.391 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.391 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:04.391 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:04.391 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.391 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.391 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.392 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.392 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.392 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.392 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.392 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.392 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-18T23:19:04.392 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-18T23:19:04.393 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:04.393 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.393 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:04.393 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-18T23:19:04.393 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.393 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.394 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-18T23:19:04.394 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-18T23:19:04.394 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-18T23:19:04.394 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-18T23:19:04.394 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-18T23:19:04.394 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-18T23:19:04.395 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-18T23:19:04.395 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-18T23:19:04.395 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.395 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.395 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.395 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.395 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.395 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-18T23:19:04.395 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-18T23:19:04.396 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:04.396 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.396 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:04.396 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:04.396 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.396 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.396 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.397 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.397 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.397 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.397 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.397 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:19:04.397 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:19:04.397 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.397 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-18T23:19:04.397 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "osd_support", 2024-03-18T23:19:04.398 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-18T23:19:04.398 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-18T23:19:04.398 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-18T23:19:04.398 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-18T23:19:04.398 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-18T23:19:04.398 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:04.398 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.398 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:04.399 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:04.399 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.399 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.399 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-18T23:19:04.399 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-18T23:19:04.399 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-18T23:19:04.399 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-18T23:19:04.399 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-18T23:19:04.400 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-18T23:19:04.400 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-18T23:19:04.400 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-18T23:19:04.400 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.400 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.400 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.400 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.400 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.401 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-18T23:19:04.401 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-18T23:19:04.401 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:04.401 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.401 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:04.401 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:04.401 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.402 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.402 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.402 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.402 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.402 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.402 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.402 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.403 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-18T23:19:04.403 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-18T23:19:04.403 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:04.403 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.403 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:04.403 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-18T23:19:04.403 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.403 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.404 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-18T23:19:04.404 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-18T23:19:04.404 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-18T23:19:04.404 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-18T23:19:04.404 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-18T23:19:04.404 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-18T23:19:04.404 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-18T23:19:04.405 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-18T23:19:04.405 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.405 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.405 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.405 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.405 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.406 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-18T23:19:04.406 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-18T23:19:04.406 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:04.406 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.406 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:04.406 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:04.406 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.406 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.407 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.407 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.407 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.407 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.407 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.407 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:19:04.407 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:19:04.407 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.407 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-18T23:19:04.408 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "pg_autoscaler", 2024-03-18T23:19:04.408 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-18T23:19:04.408 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-18T23:19:04.408 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-18T23:19:04.408 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-18T23:19:04.408 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-18T23:19:04.408 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:04.409 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.409 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:04.409 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:04.409 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.409 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.409 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-18T23:19:04.409 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-18T23:19:04.409 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-18T23:19:04.409 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-18T23:19:04.409 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-18T23:19:04.410 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-18T23:19:04.410 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-18T23:19:04.410 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-18T23:19:04.410 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.410 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.410 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.410 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.410 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.411 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-18T23:19:04.411 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-18T23:19:04.411 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:04.411 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.411 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:04.411 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:04.411 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.411 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.411 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.412 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.412 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.412 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.412 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.412 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.412 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-18T23:19:04.412 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-18T23:19:04.412 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:04.413 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.413 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:04.413 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-18T23:19:04.413 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.413 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.414 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-18T23:19:04.414 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-18T23:19:04.414 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-18T23:19:04.414 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-18T23:19:04.414 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-18T23:19:04.414 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-18T23:19:04.415 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-18T23:19:04.415 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-18T23:19:04.415 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.415 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.415 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.415 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.416 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.416 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-18T23:19:04.416 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-18T23:19:04.416 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:04.416 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.416 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:04.416 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:04.416 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.417 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.417 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.417 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.417 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.417 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.417 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.417 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.417 INFO:teuthology.orchestra.run.smithi049.stdout: "sleep_interval": { 2024-03-18T23:19:04.418 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "sleep_interval", 2024-03-18T23:19:04.418 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-03-18T23:19:04.418 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.418 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.418 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "60", 2024-03-18T23:19:04.419 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.419 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.419 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.419 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.419 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.419 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.419 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.419 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.420 INFO:teuthology.orchestra.run.smithi049.stdout: "threshold": { 2024-03-18T23:19:04.420 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "threshold", 2024-03-18T23:19:04.420 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "float", 2024-03-18T23:19:04.420 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.420 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.420 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "3.0", 2024-03-18T23:19:04.420 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "1.0", 2024-03-18T23:19:04.421 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.421 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.421 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "scaling threshold", 2024-03-18T23:19:04.421 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-03-18T23:19:04.421 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.421 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.421 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:19:04.421 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:19:04.422 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.422 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-18T23:19:04.422 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "progress", 2024-03-18T23:19:04.422 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-18T23:19:04.422 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-18T23:19:04.422 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-18T23:19:04.422 INFO:teuthology.orchestra.run.smithi049.stdout: "allow_pg_recovery_event": { 2024-03-18T23:19:04.422 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "allow_pg_recovery_event", 2024-03-18T23:19:04.423 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:04.423 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.423 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:04.423 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:04.423 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.423 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.423 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.423 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "allow the module to show pg recovery progress", 2024-03-18T23:19:04.424 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.424 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.424 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.424 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.424 INFO:teuthology.orchestra.run.smithi049.stdout: "enabled": { 2024-03-18T23:19:04.424 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "enabled", 2024-03-18T23:19:04.424 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:04.424 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.425 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.425 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-18T23:19:04.425 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.425 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.425 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.426 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.426 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.426 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.426 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.426 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.427 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-18T23:19:04.427 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-18T23:19:04.427 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:04.427 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.427 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:04.427 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:04.427 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.427 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.428 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-18T23:19:04.428 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-18T23:19:04.428 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-18T23:19:04.428 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-18T23:19:04.428 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-18T23:19:04.428 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-18T23:19:04.428 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-18T23:19:04.428 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-18T23:19:04.429 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.429 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.429 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.429 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.429 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.429 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-18T23:19:04.429 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-18T23:19:04.430 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:04.430 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.430 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:04.430 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:04.430 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.430 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.430 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.431 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.431 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.431 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.431 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.431 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.431 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-18T23:19:04.431 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-18T23:19:04.431 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:04.431 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.432 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:04.432 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-18T23:19:04.432 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.432 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.432 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-18T23:19:04.432 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-18T23:19:04.432 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-18T23:19:04.432 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-18T23:19:04.433 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-18T23:19:04.433 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-18T23:19:04.433 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-18T23:19:04.433 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-18T23:19:04.433 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.433 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.433 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.433 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.434 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.434 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-18T23:19:04.434 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-18T23:19:04.434 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:04.434 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.434 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:04.435 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:04.435 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.435 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.435 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.435 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.435 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.435 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.435 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.435 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.436 INFO:teuthology.orchestra.run.smithi049.stdout: "max_completed_events": { 2024-03-18T23:19:04.436 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "max_completed_events", 2024-03-18T23:19:04.436 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-18T23:19:04.436 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.436 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:04.437 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "50", 2024-03-18T23:19:04.437 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.437 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.437 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.437 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "number of past completed events to remember", 2024-03-18T23:19:04.437 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.438 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.438 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.438 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.438 INFO:teuthology.orchestra.run.smithi049.stdout: "sleep_interval": { 2024-03-18T23:19:04.438 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "sleep_interval", 2024-03-18T23:19:04.438 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-03-18T23:19:04.438 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.439 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:04.439 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "5", 2024-03-18T23:19:04.439 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.439 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.439 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.439 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "how long the module is going to sleep", 2024-03-18T23:19:04.440 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.440 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.440 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.440 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:19:04.440 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:19:04.440 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.440 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-18T23:19:04.441 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "prometheus", 2024-03-18T23:19:04.441 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-18T23:19:04.441 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-18T23:19:04.441 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-18T23:19:04.441 INFO:teuthology.orchestra.run.smithi049.stdout: "cache": { 2024-03-18T23:19:04.441 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "cache", 2024-03-18T23:19:04.441 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:04.441 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.442 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.442 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-18T23:19:04.442 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.442 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.442 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.442 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.442 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.442 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.443 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.443 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.443 INFO:teuthology.orchestra.run.smithi049.stdout: "exclude_perf_counters": { 2024-03-18T23:19:04.443 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "exclude_perf_counters", 2024-03-18T23:19:04.443 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:04.443 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.444 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:04.444 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-18T23:19:04.444 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.444 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.444 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.444 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-03-18T23:19:04.444 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-03-18T23:19:04.445 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.445 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.445 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.445 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-18T23:19:04.445 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-18T23:19:04.445 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:04.445 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.445 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:04.446 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:04.446 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.446 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.446 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-18T23:19:04.446 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-18T23:19:04.446 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-18T23:19:04.446 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-18T23:19:04.446 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-18T23:19:04.447 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-18T23:19:04.447 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-18T23:19:04.447 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-18T23:19:04.447 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.448 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.448 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.448 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.448 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.448 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-18T23:19:04.448 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-18T23:19:04.448 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:04.448 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.449 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:04.449 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:04.449 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.449 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.449 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.449 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.449 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.450 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.450 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.450 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.450 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-18T23:19:04.450 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-18T23:19:04.450 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:04.451 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.451 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:04.451 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-18T23:19:04.451 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.451 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.451 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-18T23:19:04.451 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-18T23:19:04.451 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-18T23:19:04.451 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-18T23:19:04.451 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-18T23:19:04.452 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-18T23:19:04.452 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-18T23:19:04.452 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-18T23:19:04.452 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.452 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.452 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.452 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.452 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.453 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-18T23:19:04.453 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-18T23:19:04.453 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:04.453 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.453 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:04.453 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:04.453 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.453 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.454 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.454 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.454 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.454 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.454 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.454 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.454 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_stats_pools": { 2024-03-18T23:19:04.454 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "rbd_stats_pools", 2024-03-18T23:19:04.455 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:04.455 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.455 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.455 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:04.455 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.455 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.456 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.456 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.456 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.456 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.456 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.456 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.457 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_stats_pools_refresh_interval": { 2024-03-18T23:19:04.457 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-03-18T23:19:04.457 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-18T23:19:04.457 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.457 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.457 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "300", 2024-03-18T23:19:04.457 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.458 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.458 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.458 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.458 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.458 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.458 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.458 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.458 INFO:teuthology.orchestra.run.smithi049.stdout: "scrape_interval": { 2024-03-18T23:19:04.459 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "scrape_interval", 2024-03-18T23:19:04.459 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "float", 2024-03-18T23:19:04.459 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.459 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.459 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "15.0", 2024-03-18T23:19:04.459 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.459 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.459 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.460 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.460 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.460 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.460 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.460 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.460 INFO:teuthology.orchestra.run.smithi049.stdout: "server_addr": { 2024-03-18T23:19:04.460 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "server_addr", 2024-03-18T23:19:04.461 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:04.461 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.461 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.461 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "::", 2024-03-18T23:19:04.461 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.461 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.461 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.461 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-03-18T23:19:04.461 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.462 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.462 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.462 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.462 INFO:teuthology.orchestra.run.smithi049.stdout: "server_port": { 2024-03-18T23:19:04.462 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "server_port", 2024-03-18T23:19:04.462 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-18T23:19:04.462 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.462 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:04.463 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "9283", 2024-03-18T23:19:04.463 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.463 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.463 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.463 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-03-18T23:19:04.463 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.463 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.463 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.464 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.464 INFO:teuthology.orchestra.run.smithi049.stdout: "stale_cache_strategy": { 2024-03-18T23:19:04.464 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "stale_cache_strategy", 2024-03-18T23:19:04.464 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:04.464 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.464 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.464 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "log", 2024-03-18T23:19:04.464 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.465 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.465 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.465 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.465 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.465 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.465 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.465 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.465 INFO:teuthology.orchestra.run.smithi049.stdout: "standby_behaviour": { 2024-03-18T23:19:04.466 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "standby_behaviour", 2024-03-18T23:19:04.466 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:04.466 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.466 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:04.466 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "default", 2024-03-18T23:19:04.466 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.466 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.467 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-18T23:19:04.467 INFO:teuthology.orchestra.run.smithi049.stdout: "default", 2024-03-18T23:19:04.467 INFO:teuthology.orchestra.run.smithi049.stdout: "error" 2024-03-18T23:19:04.467 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-18T23:19:04.467 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.467 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.467 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.467 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.468 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.468 INFO:teuthology.orchestra.run.smithi049.stdout: "standby_error_status_code": { 2024-03-18T23:19:04.468 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "standby_error_status_code", 2024-03-18T23:19:04.468 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-18T23:19:04.468 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.468 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:04.468 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "500", 2024-03-18T23:19:04.468 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "400", 2024-03-18T23:19:04.469 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "599", 2024-03-18T23:19:04.469 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.469 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.469 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.469 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.469 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.469 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:19:04.469 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:19:04.470 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.470 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-18T23:19:04.470 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "rbd_support", 2024-03-18T23:19:04.470 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-18T23:19:04.470 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-18T23:19:04.470 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-18T23:19:04.470 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-18T23:19:04.470 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-18T23:19:04.471 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:04.471 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.471 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:04.471 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:04.471 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.471 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.472 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-18T23:19:04.472 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-18T23:19:04.472 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-18T23:19:04.472 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-18T23:19:04.472 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-18T23:19:04.472 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-18T23:19:04.472 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-18T23:19:04.473 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-18T23:19:04.473 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.473 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.473 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.473 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.473 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.473 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-18T23:19:04.473 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-18T23:19:04.473 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:04.474 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.474 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:04.474 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:04.474 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.474 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.474 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.474 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.474 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.475 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.475 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.475 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.475 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-18T23:19:04.475 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-18T23:19:04.475 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:04.475 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.476 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:04.476 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-18T23:19:04.476 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.476 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.476 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-18T23:19:04.476 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-18T23:19:04.476 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-18T23:19:04.476 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-18T23:19:04.477 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-18T23:19:04.477 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-18T23:19:04.477 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-18T23:19:04.477 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-18T23:19:04.477 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.477 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.477 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.477 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.478 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.478 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-18T23:19:04.478 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-18T23:19:04.478 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:04.478 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.478 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:04.478 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:04.479 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.479 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.479 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.479 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.479 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.479 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.479 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.479 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.479 INFO:teuthology.orchestra.run.smithi049.stdout: "max_concurrent_snap_create": { 2024-03-18T23:19:04.479 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "max_concurrent_snap_create", 2024-03-18T23:19:04.480 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-18T23:19:04.480 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.480 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.480 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "10", 2024-03-18T23:19:04.480 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.480 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.480 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.480 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.481 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.481 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.481 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.481 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.481 INFO:teuthology.orchestra.run.smithi049.stdout: "mirror_snapshot_schedule": { 2024-03-18T23:19:04.481 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "mirror_snapshot_schedule", 2024-03-18T23:19:04.482 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:04.482 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.482 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.482 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:04.482 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.482 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.482 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.483 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.483 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.483 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.483 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.483 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.483 INFO:teuthology.orchestra.run.smithi049.stdout: "trash_purge_schedule": { 2024-03-18T23:19:04.483 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "trash_purge_schedule", 2024-03-18T23:19:04.483 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:04.483 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.484 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.484 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:04.484 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.484 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.484 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.484 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.484 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.484 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.485 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.485 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:19:04.485 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:19:04.485 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.485 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-18T23:19:04.485 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "restful", 2024-03-18T23:19:04.485 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-18T23:19:04.485 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-18T23:19:04.485 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-18T23:19:04.486 INFO:teuthology.orchestra.run.smithi049.stdout: "enable_auth": { 2024-03-18T23:19:04.486 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "enable_auth", 2024-03-18T23:19:04.486 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:04.486 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.486 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.486 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-18T23:19:04.486 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.486 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.487 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.487 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.487 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.487 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.487 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.487 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.487 INFO:teuthology.orchestra.run.smithi049.stdout: "key_file": { 2024-03-18T23:19:04.487 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "key_file", 2024-03-18T23:19:04.487 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:04.488 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.488 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.488 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:04.488 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.488 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.488 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.488 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.488 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.489 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.489 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.489 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.489 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-18T23:19:04.489 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-18T23:19:04.490 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:04.490 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.490 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:04.490 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:04.490 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.490 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.490 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-18T23:19:04.490 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-18T23:19:04.491 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-18T23:19:04.491 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-18T23:19:04.491 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-18T23:19:04.491 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-18T23:19:04.491 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-18T23:19:04.491 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-18T23:19:04.491 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.491 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.492 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.492 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.492 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.492 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-18T23:19:04.492 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-18T23:19:04.492 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:04.493 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.493 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:04.493 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:04.493 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.493 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.493 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.493 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.493 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.494 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.494 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.494 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.494 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-18T23:19:04.494 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-18T23:19:04.494 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:04.494 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.495 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:04.495 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-18T23:19:04.495 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.495 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.495 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-18T23:19:04.495 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-18T23:19:04.495 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-18T23:19:04.495 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-18T23:19:04.495 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-18T23:19:04.496 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-18T23:19:04.496 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-18T23:19:04.496 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-18T23:19:04.496 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.496 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.496 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.497 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.497 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.497 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-18T23:19:04.497 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-18T23:19:04.497 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:04.497 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.497 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:04.497 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:04.498 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.498 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.498 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.498 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.498 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.498 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.498 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.498 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.499 INFO:teuthology.orchestra.run.smithi049.stdout: "server_addr": { 2024-03-18T23:19:04.499 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "server_addr", 2024-03-18T23:19:04.499 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:04.499 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.499 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.499 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:04.499 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.500 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.500 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.500 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.500 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.500 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.500 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.500 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.500 INFO:teuthology.orchestra.run.smithi049.stdout: "server_port": { 2024-03-18T23:19:04.501 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "server_port", 2024-03-18T23:19:04.501 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:04.501 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.501 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.501 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:04.501 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.501 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.501 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.502 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.502 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.502 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.502 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.502 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:19:04.502 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:19:04.502 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.502 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-18T23:19:04.503 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "rgw", 2024-03-18T23:19:04.503 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-18T23:19:04.503 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-18T23:19:04.503 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-18T23:19:04.503 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-18T23:19:04.503 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-18T23:19:04.503 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:04.503 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.503 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:04.504 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:04.504 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.504 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.504 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-18T23:19:04.504 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-18T23:19:04.504 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-18T23:19:04.504 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-18T23:19:04.504 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-18T23:19:04.505 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-18T23:19:04.505 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-18T23:19:04.505 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-18T23:19:04.505 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.505 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.505 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.505 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.505 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.506 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-18T23:19:04.506 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-18T23:19:04.506 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:04.506 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.506 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:04.506 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:04.506 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.506 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.506 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.507 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.507 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.507 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.507 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.507 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.507 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-18T23:19:04.507 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-18T23:19:04.507 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:04.507 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.508 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:04.508 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-18T23:19:04.508 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.508 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.508 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-18T23:19:04.508 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-18T23:19:04.508 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-18T23:19:04.508 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-18T23:19:04.509 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-18T23:19:04.509 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-18T23:19:04.509 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-18T23:19:04.509 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-18T23:19:04.509 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.509 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.509 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.509 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.509 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.510 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-18T23:19:04.510 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-18T23:19:04.510 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:04.510 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.510 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:04.510 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:04.510 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.510 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.511 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.511 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.511 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.511 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.511 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.511 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:19:04.511 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:19:04.511 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.512 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-18T23:19:04.512 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "rook", 2024-03-18T23:19:04.512 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-18T23:19:04.512 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-18T23:19:04.512 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-18T23:19:04.512 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-18T23:19:04.512 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-18T23:19:04.512 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:04.512 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.513 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:04.513 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:04.513 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.513 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.513 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-18T23:19:04.513 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-18T23:19:04.513 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-18T23:19:04.513 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-18T23:19:04.514 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-18T23:19:04.514 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-18T23:19:04.514 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-18T23:19:04.514 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-18T23:19:04.514 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.514 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.514 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.515 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.515 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.515 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-18T23:19:04.515 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-18T23:19:04.515 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:04.515 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.515 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:04.515 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:04.516 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.516 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.516 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.516 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.516 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.516 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.516 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.516 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.516 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-18T23:19:04.517 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-18T23:19:04.517 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:04.517 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.517 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:04.517 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-18T23:19:04.517 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.517 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.517 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-18T23:19:04.518 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-18T23:19:04.518 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-18T23:19:04.518 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-18T23:19:04.518 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-18T23:19:04.518 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-18T23:19:04.518 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-18T23:19:04.518 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-18T23:19:04.518 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.519 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.519 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.519 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.519 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.519 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-18T23:19:04.519 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-18T23:19:04.519 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:04.519 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.520 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:04.520 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:04.520 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.520 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.520 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.520 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.520 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.520 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.520 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.521 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.521 INFO:teuthology.orchestra.run.smithi049.stdout: "storage_class": { 2024-03-18T23:19:04.521 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "storage_class", 2024-03-18T23:19:04.521 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:04.521 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.521 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.521 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "local", 2024-03-18T23:19:04.521 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.522 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.522 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.522 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-03-18T23:19:04.522 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.522 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.522 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.522 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:19:04.522 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:19:04.523 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.523 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-18T23:19:04.523 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "selftest", 2024-03-18T23:19:04.523 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-18T23:19:04.523 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-18T23:19:04.523 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-18T23:19:04.523 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-18T23:19:04.523 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-18T23:19:04.524 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:04.524 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.524 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:04.524 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:04.524 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.524 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.524 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-18T23:19:04.524 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-18T23:19:04.524 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-18T23:19:04.525 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-18T23:19:04.525 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-18T23:19:04.525 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-18T23:19:04.525 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-18T23:19:04.525 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-18T23:19:04.525 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.525 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.525 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.525 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.526 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.526 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-18T23:19:04.526 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-18T23:19:04.526 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:04.526 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.526 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:04.526 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:04.526 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.527 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.527 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.527 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.527 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.527 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.527 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.527 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.527 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-18T23:19:04.528 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-18T23:19:04.528 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:04.528 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.528 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:04.528 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-18T23:19:04.528 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.528 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.528 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-18T23:19:04.528 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-18T23:19:04.529 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-18T23:19:04.529 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-18T23:19:04.529 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-18T23:19:04.529 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-18T23:19:04.529 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-18T23:19:04.529 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-18T23:19:04.529 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.529 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.529 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.530 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.530 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.530 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-18T23:19:04.530 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-18T23:19:04.530 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:04.530 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.530 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:04.530 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:04.531 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.531 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.531 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.531 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.531 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.531 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.531 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.531 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.531 INFO:teuthology.orchestra.run.smithi049.stdout: "roption1": { 2024-03-18T23:19:04.532 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "roption1", 2024-03-18T23:19:04.532 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:04.532 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.532 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.532 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:04.532 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.532 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.532 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.532 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.533 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.533 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.533 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.533 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.533 INFO:teuthology.orchestra.run.smithi049.stdout: "roption2": { 2024-03-18T23:19:04.533 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "roption2", 2024-03-18T23:19:04.533 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:04.533 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.534 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.534 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "xyz", 2024-03-18T23:19:04.534 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.534 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.534 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.534 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.534 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.534 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.535 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.535 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.535 INFO:teuthology.orchestra.run.smithi049.stdout: "rwoption1": { 2024-03-18T23:19:04.535 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "rwoption1", 2024-03-18T23:19:04.535 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:04.535 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.535 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.535 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:04.536 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.536 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.536 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.536 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.536 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.536 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.536 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.536 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.537 INFO:teuthology.orchestra.run.smithi049.stdout: "rwoption2": { 2024-03-18T23:19:04.537 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "rwoption2", 2024-03-18T23:19:04.537 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-18T23:19:04.537 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.537 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.537 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:04.537 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.537 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.537 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.538 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.538 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.538 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.538 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.538 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.538 INFO:teuthology.orchestra.run.smithi049.stdout: "rwoption3": { 2024-03-18T23:19:04.539 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "rwoption3", 2024-03-18T23:19:04.539 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "float", 2024-03-18T23:19:04.539 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.539 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.539 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:04.539 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.539 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.539 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.539 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.540 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.540 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.540 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.540 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.540 INFO:teuthology.orchestra.run.smithi049.stdout: "rwoption4": { 2024-03-18T23:19:04.540 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "rwoption4", 2024-03-18T23:19:04.540 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:04.540 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.541 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.541 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:04.541 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.541 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.541 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.541 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.541 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.541 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.541 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.542 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.542 INFO:teuthology.orchestra.run.smithi049.stdout: "rwoption5": { 2024-03-18T23:19:04.542 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "rwoption5", 2024-03-18T23:19:04.542 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:04.542 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.542 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.542 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:04.542 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.543 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.543 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.543 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.543 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.543 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.543 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.543 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.543 INFO:teuthology.orchestra.run.smithi049.stdout: "rwoption6": { 2024-03-18T23:19:04.543 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "rwoption6", 2024-03-18T23:19:04.544 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:04.544 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.544 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.544 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-18T23:19:04.544 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.544 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.544 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.544 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.545 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.545 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.545 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.545 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.545 INFO:teuthology.orchestra.run.smithi049.stdout: "rwoption7": { 2024-03-18T23:19:04.545 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "rwoption7", 2024-03-18T23:19:04.545 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-18T23:19:04.545 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.545 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.546 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:04.546 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "1", 2024-03-18T23:19:04.546 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "42", 2024-03-18T23:19:04.546 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.546 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.546 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.546 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.546 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.547 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.547 INFO:teuthology.orchestra.run.smithi049.stdout: "testkey": { 2024-03-18T23:19:04.547 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "testkey", 2024-03-18T23:19:04.547 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:04.547 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.547 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.547 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:04.547 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.547 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.548 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.548 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.548 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.548 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.548 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.548 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.548 INFO:teuthology.orchestra.run.smithi049.stdout: "testlkey": { 2024-03-18T23:19:04.548 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "testlkey", 2024-03-18T23:19:04.548 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:04.549 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.549 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.549 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:04.549 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.549 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.549 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.549 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.549 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.550 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.550 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.550 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.550 INFO:teuthology.orchestra.run.smithi049.stdout: "testnewline": { 2024-03-18T23:19:04.550 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "testnewline", 2024-03-18T23:19:04.550 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:04.550 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.550 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.550 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:04.551 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.551 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.551 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.551 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.551 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.551 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.551 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.551 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:19:04.552 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:19:04.552 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.552 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-18T23:19:04.552 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "snap_schedule", 2024-03-18T23:19:04.552 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-18T23:19:04.552 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-18T23:19:04.552 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-18T23:19:04.552 INFO:teuthology.orchestra.run.smithi049.stdout: "allow_m_granularity": { 2024-03-18T23:19:04.552 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "allow_m_granularity", 2024-03-18T23:19:04.553 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:04.553 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.553 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:04.553 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:04.553 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.553 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.553 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.553 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "allow minute scheduled snapshots", 2024-03-18T23:19:04.553 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.554 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.554 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.554 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.554 INFO:teuthology.orchestra.run.smithi049.stdout: "dump_on_update": { 2024-03-18T23:19:04.554 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "dump_on_update", 2024-03-18T23:19:04.554 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:04.554 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.554 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:04.555 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:04.555 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.555 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.555 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.555 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "dump database to debug log on update", 2024-03-18T23:19:04.555 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.555 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.555 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.555 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.556 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-18T23:19:04.556 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-18T23:19:04.556 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:04.556 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.556 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:04.556 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:04.556 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.556 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.556 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-18T23:19:04.557 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-18T23:19:04.557 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-18T23:19:04.557 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-18T23:19:04.557 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-18T23:19:04.557 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-18T23:19:04.557 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-18T23:19:04.557 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-18T23:19:04.557 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.558 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.558 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.558 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.558 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.558 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-18T23:19:04.558 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-18T23:19:04.558 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:04.558 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.559 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:04.559 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:04.559 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.559 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.559 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.559 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.559 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.559 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.559 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.560 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.560 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-18T23:19:04.560 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-18T23:19:04.560 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:04.560 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.560 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:04.560 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-18T23:19:04.561 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.561 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.561 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-18T23:19:04.561 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-18T23:19:04.561 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-18T23:19:04.561 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-18T23:19:04.561 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-18T23:19:04.561 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-18T23:19:04.561 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-18T23:19:04.562 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-18T23:19:04.562 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.562 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.562 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.562 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.562 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.562 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-18T23:19:04.562 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-18T23:19:04.563 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:04.563 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.563 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:04.563 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:04.563 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.563 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.563 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.563 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.564 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.564 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.564 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.565 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:19:04.565 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:19:04.565 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.565 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-18T23:19:04.565 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "stats", 2024-03-18T23:19:04.565 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-18T23:19:04.565 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-18T23:19:04.565 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-18T23:19:04.566 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-18T23:19:04.566 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-18T23:19:04.566 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:04.566 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.566 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:04.566 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:04.566 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.566 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.566 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-18T23:19:04.567 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-18T23:19:04.567 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-18T23:19:04.567 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-18T23:19:04.567 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-18T23:19:04.567 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-18T23:19:04.567 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-18T23:19:04.567 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-18T23:19:04.567 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.567 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.568 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.568 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.568 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.568 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-18T23:19:04.568 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-18T23:19:04.568 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:04.568 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.568 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:04.569 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:04.569 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.569 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.569 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.569 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.569 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.569 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.569 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.569 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.570 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-18T23:19:04.570 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-18T23:19:04.570 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:04.570 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.570 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:04.570 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-18T23:19:04.570 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.570 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.571 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-18T23:19:04.571 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-18T23:19:04.571 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-18T23:19:04.571 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-18T23:19:04.571 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-18T23:19:04.571 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-18T23:19:04.571 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-18T23:19:04.571 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-18T23:19:04.572 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.572 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.572 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.572 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.572 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.572 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-18T23:19:04.572 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-18T23:19:04.572 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:04.572 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.573 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:04.573 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:04.573 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.573 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.573 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.573 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.573 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.573 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.574 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.574 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:19:04.574 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:19:04.574 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.574 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-18T23:19:04.574 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "status", 2024-03-18T23:19:04.574 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-18T23:19:04.574 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-18T23:19:04.575 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-18T23:19:04.575 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-18T23:19:04.575 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-18T23:19:04.575 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:04.575 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.575 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:04.575 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:04.575 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.576 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.576 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-18T23:19:04.576 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-18T23:19:04.576 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-18T23:19:04.576 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-18T23:19:04.576 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-18T23:19:04.576 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-18T23:19:04.576 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-18T23:19:04.576 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-18T23:19:04.577 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.577 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.577 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.577 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.577 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.577 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-18T23:19:04.577 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-18T23:19:04.577 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:04.577 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.578 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:04.578 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:04.578 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.578 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.578 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.578 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.578 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.578 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.579 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.579 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.579 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-18T23:19:04.579 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-18T23:19:04.579 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:04.579 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.579 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:04.579 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-18T23:19:04.579 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.580 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.580 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-18T23:19:04.580 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-18T23:19:04.580 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-18T23:19:04.580 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-18T23:19:04.580 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-18T23:19:04.580 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-18T23:19:04.580 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-18T23:19:04.580 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-18T23:19:04.581 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.581 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.581 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.581 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.581 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.581 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-18T23:19:04.581 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-18T23:19:04.581 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:04.582 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.582 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:04.582 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:04.582 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.582 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.582 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.582 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.582 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.582 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.583 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.583 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:19:04.583 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:19:04.583 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.583 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-18T23:19:04.583 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "telegraf", 2024-03-18T23:19:04.583 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-18T23:19:04.583 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-18T23:19:04.584 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-18T23:19:04.584 INFO:teuthology.orchestra.run.smithi049.stdout: "address": { 2024-03-18T23:19:04.584 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "address", 2024-03-18T23:19:04.584 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:04.584 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.584 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.584 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-03-18T23:19:04.584 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.584 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.585 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.585 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.585 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.585 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.585 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.585 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.585 INFO:teuthology.orchestra.run.smithi049.stdout: "interval": { 2024-03-18T23:19:04.586 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "interval", 2024-03-18T23:19:04.586 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-03-18T23:19:04.586 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.586 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.586 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "15", 2024-03-18T23:19:04.586 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.586 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.586 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.586 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.587 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.587 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.587 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.587 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.587 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-18T23:19:04.587 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-18T23:19:04.587 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:04.587 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.587 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:04.588 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:04.588 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.588 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.588 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-18T23:19:04.588 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-18T23:19:04.588 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-18T23:19:04.588 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-18T23:19:04.588 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-18T23:19:04.589 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-18T23:19:04.589 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-18T23:19:04.589 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-18T23:19:04.589 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.589 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.589 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.589 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.589 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.589 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-18T23:19:04.590 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-18T23:19:04.590 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:04.590 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.590 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:04.590 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:04.590 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.590 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.590 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.591 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.591 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.591 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.591 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.591 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.591 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-18T23:19:04.591 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-18T23:19:04.591 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:04.591 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.592 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:04.592 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-18T23:19:04.592 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.592 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.592 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-18T23:19:04.592 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-18T23:19:04.592 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-18T23:19:04.592 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-18T23:19:04.592 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-18T23:19:04.593 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-18T23:19:04.593 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-18T23:19:04.593 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-18T23:19:04.593 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.593 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.593 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.593 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.593 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.593 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-18T23:19:04.594 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-18T23:19:04.594 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:04.594 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.594 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:04.594 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:04.594 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.594 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.594 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.595 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.595 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.595 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.595 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.595 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:19:04.595 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:19:04.595 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.595 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-18T23:19:04.595 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "telemetry", 2024-03-18T23:19:04.596 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-18T23:19:04.596 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-18T23:19:04.596 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-18T23:19:04.596 INFO:teuthology.orchestra.run.smithi049.stdout: "channel_basic": { 2024-03-18T23:19:04.596 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "channel_basic", 2024-03-18T23:19:04.596 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:04.596 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.596 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.596 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-18T23:19:04.597 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.597 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.597 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.597 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Share basic cluster information (size, version)", 2024-03-18T23:19:04.597 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.597 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.597 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.597 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.597 INFO:teuthology.orchestra.run.smithi049.stdout: "channel_crash": { 2024-03-18T23:19:04.598 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "channel_crash", 2024-03-18T23:19:04.598 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:04.598 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.598 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.598 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-18T23:19:04.598 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.598 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.598 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.598 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-03-18T23:19:04.599 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.599 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.599 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.599 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.599 INFO:teuthology.orchestra.run.smithi049.stdout: "channel_device": { 2024-03-18T23:19:04.599 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "channel_device", 2024-03-18T23:19:04.599 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:04.599 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.599 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.600 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-18T23:19:04.600 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.600 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.600 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.600 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-03-18T23:19:04.600 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.600 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.600 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.600 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.601 INFO:teuthology.orchestra.run.smithi049.stdout: "channel_ident": { 2024-03-18T23:19:04.601 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "channel_ident", 2024-03-18T23:19:04.601 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:04.601 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.601 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.601 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:04.601 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.601 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.601 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.602 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-03-18T23:19:04.602 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.602 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.602 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.602 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.602 INFO:teuthology.orchestra.run.smithi049.stdout: "channel_perf": { 2024-03-18T23:19:04.602 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "channel_perf", 2024-03-18T23:19:04.602 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:04.603 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.603 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.603 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:04.603 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.603 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.603 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.603 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Share various performance metrics of a cluster", 2024-03-18T23:19:04.603 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.603 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.604 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.604 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.604 INFO:teuthology.orchestra.run.smithi049.stdout: "contact": { 2024-03-18T23:19:04.604 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "contact", 2024-03-18T23:19:04.604 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:04.604 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.604 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.604 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:04.605 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.605 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.605 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.605 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.605 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.605 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.605 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.605 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.606 INFO:teuthology.orchestra.run.smithi049.stdout: "description": { 2024-03-18T23:19:04.606 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "description", 2024-03-18T23:19:04.606 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:04.606 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.606 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.606 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:04.606 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.606 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.607 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.607 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.607 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.607 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.607 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.607 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.607 INFO:teuthology.orchestra.run.smithi049.stdout: "device_url": { 2024-03-18T23:19:04.607 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "device_url", 2024-03-18T23:19:04.608 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:04.608 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.608 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.608 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-03-18T23:19:04.608 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.608 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.608 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.608 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.608 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.609 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.609 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.609 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.609 INFO:teuthology.orchestra.run.smithi049.stdout: "enabled": { 2024-03-18T23:19:04.609 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "enabled", 2024-03-18T23:19:04.609 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:04.609 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.609 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.609 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:04.609 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.610 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.610 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.610 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.610 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.610 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.610 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.610 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.610 INFO:teuthology.orchestra.run.smithi049.stdout: "interval": { 2024-03-18T23:19:04.611 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "interval", 2024-03-18T23:19:04.611 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-18T23:19:04.611 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.611 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.611 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "24", 2024-03-18T23:19:04.611 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "8", 2024-03-18T23:19:04.611 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.611 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.612 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.612 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.612 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.612 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.612 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.612 INFO:teuthology.orchestra.run.smithi049.stdout: "last_opt_revision": { 2024-03-18T23:19:04.612 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "last_opt_revision", 2024-03-18T23:19:04.612 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-18T23:19:04.612 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.613 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.613 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "1", 2024-03-18T23:19:04.613 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.613 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.613 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.613 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.613 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.613 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.613 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.614 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.614 INFO:teuthology.orchestra.run.smithi049.stdout: "leaderboard": { 2024-03-18T23:19:04.614 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "leaderboard", 2024-03-18T23:19:04.614 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:04.614 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.614 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.614 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:04.614 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.615 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.615 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.615 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.615 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.615 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.615 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.615 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.615 INFO:teuthology.orchestra.run.smithi049.stdout: "leaderboard_description": { 2024-03-18T23:19:04.615 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "leaderboard_description", 2024-03-18T23:19:04.615 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:04.616 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.616 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.616 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:04.616 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.616 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.616 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.616 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.616 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.617 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.617 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.617 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.617 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-18T23:19:04.617 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-18T23:19:04.617 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:04.617 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.617 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:04.617 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:04.618 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.618 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.618 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-18T23:19:04.618 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-18T23:19:04.618 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-18T23:19:04.618 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-18T23:19:04.618 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-18T23:19:04.618 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-18T23:19:04.619 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-18T23:19:04.619 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-18T23:19:04.619 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.619 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.619 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.619 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.619 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.619 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-18T23:19:04.619 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-18T23:19:04.620 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:04.620 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.620 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:04.620 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:04.620 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.622 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.622 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.622 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.622 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.622 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.622 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.623 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.623 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-18T23:19:04.623 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-18T23:19:04.623 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:04.623 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.623 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:04.623 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-18T23:19:04.623 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.624 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.624 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-18T23:19:04.624 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-18T23:19:04.624 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-18T23:19:04.624 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-18T23:19:04.624 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-18T23:19:04.624 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-18T23:19:04.624 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-18T23:19:04.624 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-18T23:19:04.625 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.625 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.625 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.625 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.625 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.625 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-18T23:19:04.625 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-18T23:19:04.625 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:04.625 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.626 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:04.626 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:04.626 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.626 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.626 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.626 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.626 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.627 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.627 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.627 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.627 INFO:teuthology.orchestra.run.smithi049.stdout: "organization": { 2024-03-18T23:19:04.627 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "organization", 2024-03-18T23:19:04.627 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:04.627 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.628 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.628 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:04.628 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.628 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.628 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.628 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.628 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.628 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.628 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.629 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.631 INFO:teuthology.orchestra.run.smithi049.stdout: "proxy": { 2024-03-18T23:19:04.631 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "proxy", 2024-03-18T23:19:04.631 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:04.631 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.632 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.632 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:04.632 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.632 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.633 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.633 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.633 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.633 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.634 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.634 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.634 INFO:teuthology.orchestra.run.smithi049.stdout: "url": { 2024-03-18T23:19:04.634 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "url", 2024-03-18T23:19:04.635 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:04.635 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.635 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.635 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-03-18T23:19:04.636 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.636 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.637 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.637 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.637 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.637 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.638 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.638 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:19:04.638 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:19:04.638 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.639 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-18T23:19:04.639 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "test_orchestrator", 2024-03-18T23:19:04.639 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-18T23:19:04.639 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-18T23:19:04.640 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-18T23:19:04.640 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-18T23:19:04.640 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-18T23:19:04.640 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:04.641 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.641 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:04.641 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:04.642 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.642 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.642 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-18T23:19:04.642 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-18T23:19:04.643 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-18T23:19:04.643 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-18T23:19:04.643 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-18T23:19:04.643 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-18T23:19:04.644 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-18T23:19:04.644 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-18T23:19:04.644 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.644 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.645 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.645 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.645 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.646 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-18T23:19:04.646 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-18T23:19:04.646 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:04.646 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.647 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:04.647 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:04.647 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.647 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.648 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.648 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.648 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.648 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.648 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.649 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.649 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-18T23:19:04.649 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-18T23:19:04.649 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:04.650 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.650 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:04.650 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-18T23:19:04.651 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.651 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.651 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-18T23:19:04.651 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-18T23:19:04.651 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-18T23:19:04.652 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-18T23:19:04.652 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-18T23:19:04.652 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-18T23:19:04.652 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-18T23:19:04.652 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-18T23:19:04.652 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.653 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.653 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.653 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.653 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.653 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-18T23:19:04.653 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-18T23:19:04.654 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:04.654 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.654 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:04.654 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:04.655 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.655 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.655 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.655 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.655 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.656 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.656 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.656 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:19:04.656 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:19:04.656 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.656 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-18T23:19:04.656 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "volumes", 2024-03-18T23:19:04.657 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-18T23:19:04.657 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-18T23:19:04.657 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-18T23:19:04.657 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-18T23:19:04.657 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-18T23:19:04.657 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:04.657 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.657 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:04.658 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:04.658 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.658 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.658 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-18T23:19:04.658 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-18T23:19:04.658 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-18T23:19:04.658 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-18T23:19:04.658 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-18T23:19:04.659 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-18T23:19:04.659 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-18T23:19:04.659 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-18T23:19:04.659 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.659 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.659 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.659 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.659 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.659 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-18T23:19:04.660 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-18T23:19:04.660 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:04.660 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.660 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:04.660 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:04.660 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.660 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.660 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.661 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.661 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.661 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.661 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.661 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.661 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-18T23:19:04.661 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-18T23:19:04.661 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:04.661 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.662 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:04.662 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-18T23:19:04.662 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.662 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.662 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-18T23:19:04.662 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-18T23:19:04.663 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-18T23:19:04.663 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-18T23:19:04.663 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-18T23:19:04.663 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-18T23:19:04.663 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-18T23:19:04.663 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-18T23:19:04.664 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.664 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.664 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.664 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.664 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.664 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-18T23:19:04.664 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-18T23:19:04.664 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:04.664 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.665 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:04.665 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:04.665 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.665 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.665 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.665 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.665 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.665 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.666 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.666 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.666 INFO:teuthology.orchestra.run.smithi049.stdout: "max_concurrent_clones": { 2024-03-18T23:19:04.666 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "max_concurrent_clones", 2024-03-18T23:19:04.666 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-18T23:19:04.666 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.666 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.666 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "4", 2024-03-18T23:19:04.667 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.667 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.667 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.667 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Number of asynchronous cloner threads", 2024-03-18T23:19:04.667 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.667 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.667 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.667 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.668 INFO:teuthology.orchestra.run.smithi049.stdout: "periodic_async_work": { 2024-03-18T23:19:04.668 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "periodic_async_work", 2024-03-18T23:19:04.668 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:04.668 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.668 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.668 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:04.668 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.668 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.668 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.669 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Periodically check for async work", 2024-03-18T23:19:04.669 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.669 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.669 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.669 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.669 INFO:teuthology.orchestra.run.smithi049.stdout: "snapshot_clone_delay": { 2024-03-18T23:19:04.669 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "snapshot_clone_delay", 2024-03-18T23:19:04.669 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-18T23:19:04.670 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.670 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.670 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "0", 2024-03-18T23:19:04.670 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.670 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.670 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.670 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-03-18T23:19:04.671 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.671 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.671 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.671 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.671 INFO:teuthology.orchestra.run.smithi049.stdout: "snapshot_clone_no_wait": { 2024-03-18T23:19:04.671 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "snapshot_clone_no_wait", 2024-03-18T23:19:04.671 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:04.671 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.671 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.672 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-18T23:19:04.672 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.672 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.672 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.672 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-03-18T23:19:04.672 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.673 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.673 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.673 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:19:04.673 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:19:04.673 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.674 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-18T23:19:04.674 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "zabbix", 2024-03-18T23:19:04.674 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-18T23:19:04.674 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-18T23:19:04.675 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-18T23:19:04.675 INFO:teuthology.orchestra.run.smithi049.stdout: "discovery_interval": { 2024-03-18T23:19:04.675 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "discovery_interval", 2024-03-18T23:19:04.675 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "uint", 2024-03-18T23:19:04.675 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.676 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.676 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "100", 2024-03-18T23:19:04.676 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.676 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.676 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.676 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.676 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.676 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.677 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.677 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.677 INFO:teuthology.orchestra.run.smithi049.stdout: "identifier": { 2024-03-18T23:19:04.677 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "identifier", 2024-03-18T23:19:04.677 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:04.677 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.677 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.677 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:04.678 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.678 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.678 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.678 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.678 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.678 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.678 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.678 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.679 INFO:teuthology.orchestra.run.smithi049.stdout: "interval": { 2024-03-18T23:19:04.679 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "interval", 2024-03-18T23:19:04.679 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-03-18T23:19:04.679 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.679 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.679 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "60", 2024-03-18T23:19:04.679 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.679 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.680 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.680 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.680 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.680 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.680 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.680 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.680 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-18T23:19:04.680 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-18T23:19:04.680 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:04.681 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.681 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:04.681 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:04.681 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.681 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.681 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-18T23:19:04.681 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-18T23:19:04.681 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-18T23:19:04.681 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-18T23:19:04.682 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-18T23:19:04.682 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-18T23:19:04.682 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-18T23:19:04.682 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-18T23:19:04.682 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.682 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.682 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.682 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.683 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.683 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-18T23:19:04.683 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-18T23:19:04.683 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:04.683 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.683 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:04.683 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:04.683 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.683 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.684 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.684 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.684 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.684 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.684 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.684 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.684 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-18T23:19:04.684 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-18T23:19:04.684 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:04.685 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.685 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:04.685 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-18T23:19:04.685 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.685 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.685 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-18T23:19:04.685 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-18T23:19:04.685 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-18T23:19:04.685 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-18T23:19:04.686 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-18T23:19:04.686 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-18T23:19:04.686 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-18T23:19:04.686 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-18T23:19:04.686 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.686 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.686 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.686 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.686 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.687 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-18T23:19:04.687 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-18T23:19:04.687 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:04.687 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.687 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:04.687 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:04.687 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.687 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.688 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.688 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.688 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.688 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.688 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.688 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.688 INFO:teuthology.orchestra.run.smithi049.stdout: "zabbix_host": { 2024-03-18T23:19:04.688 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "zabbix_host", 2024-03-18T23:19:04.688 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:04.689 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.689 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.689 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:04.689 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.689 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.689 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.689 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.689 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.689 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.689 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.690 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.690 INFO:teuthology.orchestra.run.smithi049.stdout: "zabbix_port": { 2024-03-18T23:19:04.690 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "zabbix_port", 2024-03-18T23:19:04.690 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-18T23:19:04.690 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.690 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.690 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "10051", 2024-03-18T23:19:04.691 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.691 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.691 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.691 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.691 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.691 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.691 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.691 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.691 INFO:teuthology.orchestra.run.smithi049.stdout: "zabbix_sender": { 2024-03-18T23:19:04.692 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "zabbix_sender", 2024-03-18T23:19:04.692 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:04.692 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.692 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.692 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-03-18T23:19:04.692 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.692 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.692 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.692 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.693 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.693 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.693 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.693 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:19:04.693 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:19:04.693 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:19:04.693 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-03-18T23:19:04.693 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.694 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-18T23:19:04.694 INFO:teuthology.orchestra.run.smithi049.stdout: "gid": 25196, 2024-03-18T23:19:04.694 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "y", 2024-03-18T23:19:04.694 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr_features": 4540701547738038271, 2024-03-18T23:19:04.694 INFO:teuthology.orchestra.run.smithi049.stdout: "available_modules": [ 2024-03-18T23:19:04.694 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-18T23:19:04.694 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "alerts", 2024-03-18T23:19:04.694 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-18T23:19:04.695 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-18T23:19:04.695 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-18T23:19:04.695 INFO:teuthology.orchestra.run.smithi049.stdout: "interval": { 2024-03-18T23:19:04.695 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "interval", 2024-03-18T23:19:04.695 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-03-18T23:19:04.695 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.695 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:04.695 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "60", 2024-03-18T23:19:04.696 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.696 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.696 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.696 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "How frequently to reexamine health status", 2024-03-18T23:19:04.696 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.696 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.696 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.696 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.697 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-18T23:19:04.697 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-18T23:19:04.697 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:04.697 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.697 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:04.697 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:04.697 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.697 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.697 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-18T23:19:04.698 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-18T23:19:04.698 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-18T23:19:04.698 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-18T23:19:04.698 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-18T23:19:04.698 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-18T23:19:04.698 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-18T23:19:04.698 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-18T23:19:04.698 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.698 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.699 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.699 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.699 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.699 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-18T23:19:04.699 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-18T23:19:04.699 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:04.699 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.699 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:04.699 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:04.700 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.700 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.700 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.700 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.700 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.700 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.700 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.700 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.701 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-18T23:19:04.701 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-18T23:19:04.701 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:04.701 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.701 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:04.701 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-18T23:19:04.701 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.701 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.701 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-18T23:19:04.701 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-18T23:19:04.702 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-18T23:19:04.702 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-18T23:19:04.702 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-18T23:19:04.702 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-18T23:19:04.702 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-18T23:19:04.702 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-18T23:19:04.702 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.702 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.703 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.703 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.703 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.703 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-18T23:19:04.703 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-18T23:19:04.703 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:04.703 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.703 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:04.703 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:04.704 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.704 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.704 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.704 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.704 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.704 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.704 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.704 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.704 INFO:teuthology.orchestra.run.smithi049.stdout: "smtp_destination": { 2024-03-18T23:19:04.705 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "smtp_destination", 2024-03-18T23:19:04.705 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:04.705 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.705 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:04.705 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:04.705 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.705 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.705 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.705 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Email address to send alerts to", 2024-03-18T23:19:04.706 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.706 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.706 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.706 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.706 INFO:teuthology.orchestra.run.smithi049.stdout: "smtp_from_name": { 2024-03-18T23:19:04.706 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "smtp_from_name", 2024-03-18T23:19:04.706 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:04.706 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.706 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:04.707 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "Ceph", 2024-03-18T23:19:04.707 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.707 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.707 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.707 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Email From: name", 2024-03-18T23:19:04.707 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.707 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.707 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.708 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.708 INFO:teuthology.orchestra.run.smithi049.stdout: "smtp_host": { 2024-03-18T23:19:04.708 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "smtp_host", 2024-03-18T23:19:04.708 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:04.708 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.708 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:04.708 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:04.708 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.709 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.709 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.709 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "SMTP server", 2024-03-18T23:19:04.709 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.709 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.709 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.709 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.709 INFO:teuthology.orchestra.run.smithi049.stdout: "smtp_password": { 2024-03-18T23:19:04.709 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "smtp_password", 2024-03-18T23:19:04.710 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:04.710 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.710 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:04.710 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:04.710 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.710 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.710 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.710 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Password to authenticate with", 2024-03-18T23:19:04.711 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.711 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.711 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.711 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.711 INFO:teuthology.orchestra.run.smithi049.stdout: "smtp_port": { 2024-03-18T23:19:04.711 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "smtp_port", 2024-03-18T23:19:04.711 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-18T23:19:04.711 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.712 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:04.712 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "465", 2024-03-18T23:19:04.712 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.712 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.712 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.712 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "SMTP port", 2024-03-18T23:19:04.712 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.712 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.712 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.713 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.713 INFO:teuthology.orchestra.run.smithi049.stdout: "smtp_sender": { 2024-03-18T23:19:04.713 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "smtp_sender", 2024-03-18T23:19:04.713 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:04.713 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.713 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:04.713 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:04.713 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.713 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.714 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.714 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "SMTP envelope sender", 2024-03-18T23:19:04.714 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.714 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.714 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.714 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.714 INFO:teuthology.orchestra.run.smithi049.stdout: "smtp_ssl": { 2024-03-18T23:19:04.714 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "smtp_ssl", 2024-03-18T23:19:04.715 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:04.715 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.715 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:04.715 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-18T23:19:04.715 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.715 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.715 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.715 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Use SSL to connect to SMTP server", 2024-03-18T23:19:04.716 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.716 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.716 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.716 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.716 INFO:teuthology.orchestra.run.smithi049.stdout: "smtp_user": { 2024-03-18T23:19:04.716 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "smtp_user", 2024-03-18T23:19:04.716 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:04.716 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.717 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:04.717 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:04.717 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.717 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.717 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.717 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "User to authenticate as", 2024-03-18T23:19:04.717 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.717 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.718 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.718 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:19:04.718 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:19:04.718 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.718 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-18T23:19:04.718 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "balancer", 2024-03-18T23:19:04.718 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-18T23:19:04.718 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-18T23:19:04.719 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-18T23:19:04.719 INFO:teuthology.orchestra.run.smithi049.stdout: "active": { 2024-03-18T23:19:04.719 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "active", 2024-03-18T23:19:04.719 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:04.719 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.719 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:04.719 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-18T23:19:04.719 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.719 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.720 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.720 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "automatically balance PGs across cluster", 2024-03-18T23:19:04.720 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.720 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.720 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.720 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.720 INFO:teuthology.orchestra.run.smithi049.stdout: "begin_time": { 2024-03-18T23:19:04.720 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "begin_time", 2024-03-18T23:19:04.720 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:04.721 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.721 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:04.721 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "0000", 2024-03-18T23:19:04.721 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.721 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.721 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.721 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "beginning time of day to automatically balance", 2024-03-18T23:19:04.721 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-18T23:19:04.721 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.722 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.722 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.722 INFO:teuthology.orchestra.run.smithi049.stdout: "begin_weekday": { 2024-03-18T23:19:04.722 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "begin_weekday", 2024-03-18T23:19:04.722 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "uint", 2024-03-18T23:19:04.722 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.722 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:04.722 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "0", 2024-03-18T23:19:04.722 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "0", 2024-03-18T23:19:04.723 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "6", 2024-03-18T23:19:04.723 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.723 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-03-18T23:19:04.723 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-18T23:19:04.723 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.723 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.723 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.723 INFO:teuthology.orchestra.run.smithi049.stdout: "crush_compat_max_iterations": { 2024-03-18T23:19:04.724 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "crush_compat_max_iterations", 2024-03-18T23:19:04.724 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "uint", 2024-03-18T23:19:04.724 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.724 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:04.724 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "25", 2024-03-18T23:19:04.724 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "1", 2024-03-18T23:19:04.724 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "250", 2024-03-18T23:19:04.724 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.724 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-03-18T23:19:04.725 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.725 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.725 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.725 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.725 INFO:teuthology.orchestra.run.smithi049.stdout: "crush_compat_metrics": { 2024-03-18T23:19:04.725 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "crush_compat_metrics", 2024-03-18T23:19:04.725 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:04.725 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.725 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:04.726 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "pgs,objects,bytes", 2024-03-18T23:19:04.726 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.726 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.726 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.726 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-03-18T23:19:04.726 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-03-18T23:19:04.726 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.726 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.727 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.727 INFO:teuthology.orchestra.run.smithi049.stdout: "crush_compat_step": { 2024-03-18T23:19:04.727 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "crush_compat_step", 2024-03-18T23:19:04.727 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "float", 2024-03-18T23:19:04.727 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.727 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:04.727 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "0.5", 2024-03-18T23:19:04.727 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "0.001", 2024-03-18T23:19:04.728 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "0.999", 2024-03-18T23:19:04.728 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.728 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "aggressiveness of optimization", 2024-03-18T23:19:04.728 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-03-18T23:19:04.728 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.728 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.728 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.728 INFO:teuthology.orchestra.run.smithi049.stdout: "end_time": { 2024-03-18T23:19:04.728 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "end_time", 2024-03-18T23:19:04.729 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:04.729 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.729 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:04.729 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "2359", 2024-03-18T23:19:04.729 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.729 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.729 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.729 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "ending time of day to automatically balance", 2024-03-18T23:19:04.729 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-18T23:19:04.730 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.730 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.730 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.730 INFO:teuthology.orchestra.run.smithi049.stdout: "end_weekday": { 2024-03-18T23:19:04.730 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "end_weekday", 2024-03-18T23:19:04.730 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "uint", 2024-03-18T23:19:04.730 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.730 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:04.731 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "0", 2024-03-18T23:19:04.731 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "0", 2024-03-18T23:19:04.731 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "6", 2024-03-18T23:19:04.731 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.731 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-03-18T23:19:04.731 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-18T23:19:04.731 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.731 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.731 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.732 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-18T23:19:04.732 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-18T23:19:04.732 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:04.732 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.732 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:04.732 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:04.732 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.732 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.732 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-18T23:19:04.733 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-18T23:19:04.733 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-18T23:19:04.733 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-18T23:19:04.733 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-18T23:19:04.733 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-18T23:19:04.733 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-18T23:19:04.733 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-18T23:19:04.733 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.733 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.733 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.734 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.734 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.734 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-18T23:19:04.734 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-18T23:19:04.734 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:04.734 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.734 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:04.734 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:04.735 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.735 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.735 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.735 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.735 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.735 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.735 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.735 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.735 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-18T23:19:04.735 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-18T23:19:04.736 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:04.736 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.736 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:04.736 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-18T23:19:04.736 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.736 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.736 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-18T23:19:04.736 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-18T23:19:04.736 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-18T23:19:04.737 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-18T23:19:04.737 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-18T23:19:04.737 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-18T23:19:04.737 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-18T23:19:04.737 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-18T23:19:04.737 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.737 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.737 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.737 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.738 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.738 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-18T23:19:04.738 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-18T23:19:04.738 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:04.738 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.738 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:04.738 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:04.738 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.739 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.739 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.739 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.739 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.739 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.739 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.739 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.739 INFO:teuthology.orchestra.run.smithi049.stdout: "min_score": { 2024-03-18T23:19:04.739 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "min_score", 2024-03-18T23:19:04.739 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "float", 2024-03-18T23:19:04.740 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.740 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:04.740 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "0", 2024-03-18T23:19:04.740 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.740 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.740 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.740 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-03-18T23:19:04.740 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.740 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.741 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.741 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.741 INFO:teuthology.orchestra.run.smithi049.stdout: "mode": { 2024-03-18T23:19:04.741 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "mode", 2024-03-18T23:19:04.741 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:04.741 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.741 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:04.741 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "upmap", 2024-03-18T23:19:04.741 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.742 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.742 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-18T23:19:04.742 INFO:teuthology.orchestra.run.smithi049.stdout: "crush-compat", 2024-03-18T23:19:04.742 INFO:teuthology.orchestra.run.smithi049.stdout: "none", 2024-03-18T23:19:04.742 INFO:teuthology.orchestra.run.smithi049.stdout: "read", 2024-03-18T23:19:04.742 INFO:teuthology.orchestra.run.smithi049.stdout: "upmap", 2024-03-18T23:19:04.742 INFO:teuthology.orchestra.run.smithi049.stdout: "upmap-read" 2024-03-18T23:19:04.742 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-18T23:19:04.742 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Balancer mode", 2024-03-18T23:19:04.743 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.743 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.743 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.743 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.743 INFO:teuthology.orchestra.run.smithi049.stdout: "pool_ids": { 2024-03-18T23:19:04.743 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "pool_ids", 2024-03-18T23:19:04.743 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:04.743 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.743 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:04.743 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:04.744 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.744 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.744 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.744 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-03-18T23:19:04.744 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.744 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.744 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.744 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.744 INFO:teuthology.orchestra.run.smithi049.stdout: "sleep_interval": { 2024-03-18T23:19:04.745 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "sleep_interval", 2024-03-18T23:19:04.745 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-03-18T23:19:04.745 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.745 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:04.745 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "60", 2024-03-18T23:19:04.745 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.745 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.745 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.745 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-03-18T23:19:04.745 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.746 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.746 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.746 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.746 INFO:teuthology.orchestra.run.smithi049.stdout: "upmap_max_deviation": { 2024-03-18T23:19:04.746 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "upmap_max_deviation", 2024-03-18T23:19:04.746 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-18T23:19:04.746 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.746 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:04.747 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "5", 2024-03-18T23:19:04.747 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "1", 2024-03-18T23:19:04.747 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.747 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.747 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "deviation below which no optimization is attempted", 2024-03-18T23:19:04.747 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-03-18T23:19:04.747 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.747 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.747 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.748 INFO:teuthology.orchestra.run.smithi049.stdout: "upmap_max_optimizations": { 2024-03-18T23:19:04.748 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "upmap_max_optimizations", 2024-03-18T23:19:04.748 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "uint", 2024-03-18T23:19:04.748 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.748 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:04.748 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "10", 2024-03-18T23:19:04.748 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.748 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.749 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.749 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-03-18T23:19:04.749 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.749 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.749 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.749 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:19:04.749 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:19:04.749 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.749 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-18T23:19:04.750 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "cephadm", 2024-03-18T23:19:04.750 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-18T23:19:04.750 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-18T23:19:04.750 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-18T23:19:04.750 INFO:teuthology.orchestra.run.smithi049.stdout: "agent_down_multiplier": { 2024-03-18T23:19:04.750 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "agent_down_multiplier", 2024-03-18T23:19:04.750 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "float", 2024-03-18T23:19:04.750 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.750 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.751 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "3.0", 2024-03-18T23:19:04.751 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.751 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.751 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.751 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-03-18T23:19:04.751 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.751 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.751 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.751 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.752 INFO:teuthology.orchestra.run.smithi049.stdout: "agent_refresh_rate": { 2024-03-18T23:19:04.752 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "agent_refresh_rate", 2024-03-18T23:19:04.752 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-03-18T23:19:04.752 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.752 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.752 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "20", 2024-03-18T23:19:04.752 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.752 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.753 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.753 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-03-18T23:19:04.753 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.753 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.753 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.753 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.753 INFO:teuthology.orchestra.run.smithi049.stdout: "agent_starting_port": { 2024-03-18T23:19:04.753 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "agent_starting_port", 2024-03-18T23:19:04.753 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-18T23:19:04.754 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.754 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.754 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "4721", 2024-03-18T23:19:04.754 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.754 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.754 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.754 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-03-18T23:19:04.755 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.755 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.755 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.755 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.755 INFO:teuthology.orchestra.run.smithi049.stdout: "allow_ptrace": { 2024-03-18T23:19:04.755 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "allow_ptrace", 2024-03-18T23:19:04.755 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:04.755 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.755 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.756 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:04.756 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.756 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.756 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.756 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-03-18T23:19:04.756 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-03-18T23:19:04.756 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.756 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.757 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.757 INFO:teuthology.orchestra.run.smithi049.stdout: "autotune_interval": { 2024-03-18T23:19:04.757 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "autotune_interval", 2024-03-18T23:19:04.757 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-03-18T23:19:04.757 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.758 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.758 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "600", 2024-03-18T23:19:04.758 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.758 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.758 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.758 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "how frequently to autotune daemon memory", 2024-03-18T23:19:04.758 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.758 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.759 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.759 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.759 INFO:teuthology.orchestra.run.smithi049.stdout: "autotune_memory_target_ratio": { 2024-03-18T23:19:04.759 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "autotune_memory_target_ratio", 2024-03-18T23:19:04.759 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "float", 2024-03-18T23:19:04.759 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.759 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.759 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "0.7", 2024-03-18T23:19:04.759 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.760 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.760 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.760 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-03-18T23:19:04.760 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.760 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.760 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.760 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.760 INFO:teuthology.orchestra.run.smithi049.stdout: "cephadm_log_destination": { 2024-03-18T23:19:04.760 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "cephadm_log_destination", 2024-03-18T23:19:04.761 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:04.761 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.761 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.761 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:04.761 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.761 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.761 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-18T23:19:04.761 INFO:teuthology.orchestra.run.smithi049.stdout: "file", 2024-03-18T23:19:04.761 INFO:teuthology.orchestra.run.smithi049.stdout: "file,syslog", 2024-03-18T23:19:04.762 INFO:teuthology.orchestra.run.smithi049.stdout: "syslog" 2024-03-18T23:19:04.762 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-18T23:19:04.762 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-03-18T23:19:04.762 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.762 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.762 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.762 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.762 INFO:teuthology.orchestra.run.smithi049.stdout: "cgroups_split": { 2024-03-18T23:19:04.763 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "cgroups_split", 2024-03-18T23:19:04.763 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:04.763 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.763 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.763 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-18T23:19:04.763 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.763 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.763 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.763 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-03-18T23:19:04.764 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.764 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.764 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.764 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.764 INFO:teuthology.orchestra.run.smithi049.stdout: "config_checks_enabled": { 2024-03-18T23:19:04.764 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "config_checks_enabled", 2024-03-18T23:19:04.764 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:04.764 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.764 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.765 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:04.765 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.765 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.765 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.765 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-03-18T23:19:04.765 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.765 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.765 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.765 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.766 INFO:teuthology.orchestra.run.smithi049.stdout: "config_dashboard": { 2024-03-18T23:19:04.766 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "config_dashboard", 2024-03-18T23:19:04.766 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:04.766 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.766 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.766 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-18T23:19:04.766 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.766 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.767 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.767 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-03-18T23:19:04.767 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.767 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.767 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.767 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.767 INFO:teuthology.orchestra.run.smithi049.stdout: "container_image_alertmanager": { 2024-03-18T23:19:04.767 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_image_alertmanager", 2024-03-18T23:19:04.768 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:04.768 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.768 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.768 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-03-18T23:19:04.768 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.768 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.768 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.768 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Prometheus container image", 2024-03-18T23:19:04.769 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.769 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.769 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.769 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.770 INFO:teuthology.orchestra.run.smithi049.stdout: "container_image_base": { 2024-03-18T23:19:04.770 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_image_base", 2024-03-18T23:19:04.770 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:04.770 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.771 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:04.771 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "quay.io/ceph/ceph", 2024-03-18T23:19:04.771 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.771 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.771 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.771 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Container image name, without the tag", 2024-03-18T23:19:04.771 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.771 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.772 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.772 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.772 INFO:teuthology.orchestra.run.smithi049.stdout: "container_image_elasticsearch": { 2024-03-18T23:19:04.772 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_image_elasticsearch", 2024-03-18T23:19:04.772 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:04.772 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.772 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.772 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-03-18T23:19:04.773 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.773 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.773 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.773 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "elasticsearch container image", 2024-03-18T23:19:04.773 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.773 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.773 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.773 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.773 INFO:teuthology.orchestra.run.smithi049.stdout: "container_image_grafana": { 2024-03-18T23:19:04.774 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_image_grafana", 2024-03-18T23:19:04.774 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:04.774 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.774 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.774 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.12", 2024-03-18T23:19:04.774 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.774 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.774 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.775 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Prometheus container image", 2024-03-18T23:19:04.775 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.775 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.775 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.775 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.775 INFO:teuthology.orchestra.run.smithi049.stdout: "container_image_haproxy": { 2024-03-18T23:19:04.775 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_image_haproxy", 2024-03-18T23:19:04.775 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:04.775 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.776 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.776 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-03-18T23:19:04.776 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.776 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.776 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.776 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "HAproxy container image", 2024-03-18T23:19:04.776 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.776 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.777 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.777 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.777 INFO:teuthology.orchestra.run.smithi049.stdout: "container_image_jaeger_agent": { 2024-03-18T23:19:04.777 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_image_jaeger_agent", 2024-03-18T23:19:04.777 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:04.777 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.777 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.777 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-03-18T23:19:04.778 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.778 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.778 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.778 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Jaeger agent container image", 2024-03-18T23:19:04.778 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.778 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.778 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.779 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.779 INFO:teuthology.orchestra.run.smithi049.stdout: "container_image_jaeger_collector": { 2024-03-18T23:19:04.779 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_image_jaeger_collector", 2024-03-18T23:19:04.779 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:04.779 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.779 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.779 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-03-18T23:19:04.779 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.779 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.780 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.780 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Jaeger collector container image", 2024-03-18T23:19:04.780 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.780 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.780 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.780 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.780 INFO:teuthology.orchestra.run.smithi049.stdout: "container_image_jaeger_query": { 2024-03-18T23:19:04.780 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_image_jaeger_query", 2024-03-18T23:19:04.780 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:04.781 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.781 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.781 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-03-18T23:19:04.781 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.781 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.781 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.781 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Jaeger query container image", 2024-03-18T23:19:04.781 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.781 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.782 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.782 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.782 INFO:teuthology.orchestra.run.smithi049.stdout: "container_image_keepalived": { 2024-03-18T23:19:04.782 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_image_keepalived", 2024-03-18T23:19:04.782 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:04.782 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.782 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.782 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-03-18T23:19:04.782 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.783 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.783 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.783 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Keepalived container image", 2024-03-18T23:19:04.783 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.783 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.783 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.783 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.783 INFO:teuthology.orchestra.run.smithi049.stdout: "container_image_loki": { 2024-03-18T23:19:04.784 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_image_loki", 2024-03-18T23:19:04.784 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:04.784 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.784 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.784 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-03-18T23:19:04.784 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.784 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.784 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.784 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Loki container image", 2024-03-18T23:19:04.785 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.785 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.785 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.785 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.785 INFO:teuthology.orchestra.run.smithi049.stdout: "container_image_node_exporter": { 2024-03-18T23:19:04.785 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_image_node_exporter", 2024-03-18T23:19:04.785 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:04.785 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.786 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.786 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-03-18T23:19:04.786 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.786 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.786 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.786 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Prometheus container image", 2024-03-18T23:19:04.786 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.786 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.787 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.787 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.787 INFO:teuthology.orchestra.run.smithi049.stdout: "container_image_nvmeof": { 2024-03-18T23:19:04.787 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_image_nvmeof", 2024-03-18T23:19:04.787 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:04.787 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.787 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.787 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "quay.io/ceph/nvmeof:1.0.0", 2024-03-18T23:19:04.787 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.788 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.788 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.788 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Nvme-of container image", 2024-03-18T23:19:04.788 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.788 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.788 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.788 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.788 INFO:teuthology.orchestra.run.smithi049.stdout: "container_image_prometheus": { 2024-03-18T23:19:04.789 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_image_prometheus", 2024-03-18T23:19:04.789 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:04.789 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.789 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.789 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-03-18T23:19:04.789 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.789 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.789 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.789 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Prometheus container image", 2024-03-18T23:19:04.790 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.790 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.790 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.790 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.790 INFO:teuthology.orchestra.run.smithi049.stdout: "container_image_promtail": { 2024-03-18T23:19:04.790 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_image_promtail", 2024-03-18T23:19:04.790 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:04.790 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.791 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.791 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-03-18T23:19:04.791 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.791 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.791 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.791 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Promtail container image", 2024-03-18T23:19:04.791 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.791 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.792 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.792 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.792 INFO:teuthology.orchestra.run.smithi049.stdout: "container_image_snmp_gateway": { 2024-03-18T23:19:04.792 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_image_snmp_gateway", 2024-03-18T23:19:04.792 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:04.792 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.792 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.792 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-03-18T23:19:04.793 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.793 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.793 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.793 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "SNMP Gateway container image", 2024-03-18T23:19:04.793 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.793 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.793 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.793 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.793 INFO:teuthology.orchestra.run.smithi049.stdout: "container_init": { 2024-03-18T23:19:04.794 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_init", 2024-03-18T23:19:04.794 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:04.794 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.794 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.794 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-18T23:19:04.794 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.794 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.794 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.795 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Run podman/docker with `--init`", 2024-03-18T23:19:04.795 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.795 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.795 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.795 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.795 INFO:teuthology.orchestra.run.smithi049.stdout: "daemon_cache_timeout": { 2024-03-18T23:19:04.795 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "daemon_cache_timeout", 2024-03-18T23:19:04.795 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-03-18T23:19:04.796 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.796 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.796 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "600", 2024-03-18T23:19:04.796 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.796 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.796 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.796 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-03-18T23:19:04.796 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.796 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.796 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.797 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.797 INFO:teuthology.orchestra.run.smithi049.stdout: "default_cephadm_command_timeout": { 2024-03-18T23:19:04.797 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "default_cephadm_command_timeout", 2024-03-18T23:19:04.797 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-18T23:19:04.797 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.797 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.797 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "900", 2024-03-18T23:19:04.797 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.798 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.798 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.798 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-03-18T23:19:04.798 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.798 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.798 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.798 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.798 INFO:teuthology.orchestra.run.smithi049.stdout: "default_registry": { 2024-03-18T23:19:04.799 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "default_registry", 2024-03-18T23:19:04.799 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:04.799 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.799 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.799 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "docker.io", 2024-03-18T23:19:04.799 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.799 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.801 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.801 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-03-18T23:19:04.801 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.801 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.801 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.802 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.802 INFO:teuthology.orchestra.run.smithi049.stdout: "device_cache_timeout": { 2024-03-18T23:19:04.802 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "device_cache_timeout", 2024-03-18T23:19:04.802 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-03-18T23:19:04.802 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.802 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.802 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "1800", 2024-03-18T23:19:04.802 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.803 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.803 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.803 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "seconds to cache device inventory", 2024-03-18T23:19:04.803 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.803 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.803 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.803 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.803 INFO:teuthology.orchestra.run.smithi049.stdout: "device_enhanced_scan": { 2024-03-18T23:19:04.803 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "device_enhanced_scan", 2024-03-18T23:19:04.804 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:04.804 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.804 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.804 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:04.804 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.804 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.804 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.804 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Use libstoragemgmt during device scans", 2024-03-18T23:19:04.804 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.805 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.805 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.805 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.805 INFO:teuthology.orchestra.run.smithi049.stdout: "facts_cache_timeout": { 2024-03-18T23:19:04.805 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "facts_cache_timeout", 2024-03-18T23:19:04.805 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-03-18T23:19:04.805 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.805 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.805 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "60", 2024-03-18T23:19:04.806 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.806 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.806 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.806 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "seconds to cache host facts data", 2024-03-18T23:19:04.806 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.806 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.806 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.806 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.807 INFO:teuthology.orchestra.run.smithi049.stdout: "host_check_interval": { 2024-03-18T23:19:04.807 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "host_check_interval", 2024-03-18T23:19:04.807 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-03-18T23:19:04.807 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.807 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.807 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "600", 2024-03-18T23:19:04.807 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.807 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.807 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.808 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "how frequently to perform a host check", 2024-03-18T23:19:04.808 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.808 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.808 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.808 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.808 INFO:teuthology.orchestra.run.smithi049.stdout: "hw_monitoring": { 2024-03-18T23:19:04.808 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "hw_monitoring", 2024-03-18T23:19:04.808 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:04.809 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.809 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.809 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:04.809 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.809 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.809 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.809 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-03-18T23:19:04.809 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.809 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.810 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.810 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.810 INFO:teuthology.orchestra.run.smithi049.stdout: "inventory_list_all": { 2024-03-18T23:19:04.810 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "inventory_list_all", 2024-03-18T23:19:04.810 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:04.810 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.810 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.810 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:04.811 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.811 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.811 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.811 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-03-18T23:19:04.811 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.811 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.811 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.811 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.811 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-18T23:19:04.812 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-18T23:19:04.812 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:04.812 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.812 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:04.812 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:04.812 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.812 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.812 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-18T23:19:04.812 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-18T23:19:04.813 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-18T23:19:04.813 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-18T23:19:04.813 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-18T23:19:04.813 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-18T23:19:04.813 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-18T23:19:04.813 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-18T23:19:04.813 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.813 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.813 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.814 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.814 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.814 INFO:teuthology.orchestra.run.smithi049.stdout: "log_refresh_metadata": { 2024-03-18T23:19:04.814 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_refresh_metadata", 2024-03-18T23:19:04.814 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:04.814 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.814 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.814 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:04.815 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.815 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.815 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.815 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-03-18T23:19:04.815 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.815 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.815 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.815 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.815 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-18T23:19:04.816 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-18T23:19:04.816 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:04.816 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.816 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.816 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-18T23:19:04.816 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.816 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.816 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.817 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-03-18T23:19:04.817 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.817 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.817 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.817 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.817 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-18T23:19:04.817 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-18T23:19:04.817 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:04.817 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.818 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:04.818 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-18T23:19:04.818 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.818 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.818 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-18T23:19:04.818 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-18T23:19:04.818 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-18T23:19:04.818 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-18T23:19:04.819 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-18T23:19:04.819 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-18T23:19:04.819 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-18T23:19:04.819 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-18T23:19:04.819 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.819 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.819 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.819 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.819 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.820 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-18T23:19:04.820 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-18T23:19:04.820 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:04.820 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.820 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:04.820 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:04.820 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.820 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.821 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.821 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.821 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.821 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.821 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.821 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.821 INFO:teuthology.orchestra.run.smithi049.stdout: "manage_etc_ceph_ceph_conf": { 2024-03-18T23:19:04.821 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-03-18T23:19:04.821 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:04.822 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.822 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.822 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:04.822 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.822 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.822 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.822 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-03-18T23:19:04.822 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.823 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.823 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.823 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.823 INFO:teuthology.orchestra.run.smithi049.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-03-18T23:19:04.823 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-03-18T23:19:04.823 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:04.823 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.823 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.824 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "*", 2024-03-18T23:19:04.824 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.824 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.824 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.824 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-03-18T23:19:04.824 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.824 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.824 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.824 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.825 INFO:teuthology.orchestra.run.smithi049.stdout: "max_count_per_host": { 2024-03-18T23:19:04.825 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "max_count_per_host", 2024-03-18T23:19:04.825 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-18T23:19:04.825 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.825 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.825 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "10", 2024-03-18T23:19:04.825 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.825 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.825 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.826 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "max number of daemons per service per host", 2024-03-18T23:19:04.826 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.826 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.826 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.826 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.826 INFO:teuthology.orchestra.run.smithi049.stdout: "max_osd_draining_count": { 2024-03-18T23:19:04.826 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "max_osd_draining_count", 2024-03-18T23:19:04.826 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-18T23:19:04.827 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.827 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.827 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "10", 2024-03-18T23:19:04.827 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.827 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.827 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.827 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-03-18T23:19:04.827 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.828 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.828 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.828 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.828 INFO:teuthology.orchestra.run.smithi049.stdout: "migration_current": { 2024-03-18T23:19:04.828 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "migration_current", 2024-03-18T23:19:04.828 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-18T23:19:04.828 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.828 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.828 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:04.829 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.829 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.829 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.829 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "internal - do not modify", 2024-03-18T23:19:04.829 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.829 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.829 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.829 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.829 INFO:teuthology.orchestra.run.smithi049.stdout: "mode": { 2024-03-18T23:19:04.830 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "mode", 2024-03-18T23:19:04.830 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:04.830 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.830 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.830 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "root", 2024-03-18T23:19:04.830 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.830 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.830 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-18T23:19:04.831 INFO:teuthology.orchestra.run.smithi049.stdout: "cephadm-package", 2024-03-18T23:19:04.831 INFO:teuthology.orchestra.run.smithi049.stdout: "root" 2024-03-18T23:19:04.831 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-18T23:19:04.831 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "mode for remote execution of cephadm", 2024-03-18T23:19:04.831 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.831 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.831 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.831 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.832 INFO:teuthology.orchestra.run.smithi049.stdout: "oob_default_addr": { 2024-03-18T23:19:04.832 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "oob_default_addr", 2024-03-18T23:19:04.832 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:04.832 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.832 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.832 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "169.254.1.1", 2024-03-18T23:19:04.832 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.832 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.833 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.833 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Default address for RedFish API (oob management).", 2024-03-18T23:19:04.833 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.833 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.833 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.833 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.833 INFO:teuthology.orchestra.run.smithi049.stdout: "prometheus_alerts_path": { 2024-03-18T23:19:04.833 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "prometheus_alerts_path", 2024-03-18T23:19:04.833 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:04.834 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.834 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.834 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-03-18T23:19:04.834 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.834 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.834 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.834 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-03-18T23:19:04.834 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.835 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.835 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.835 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.835 INFO:teuthology.orchestra.run.smithi049.stdout: "registry_insecure": { 2024-03-18T23:19:04.835 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "registry_insecure", 2024-03-18T23:19:04.835 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:04.835 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.835 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.836 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:04.836 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.836 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.836 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.836 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-03-18T23:19:04.836 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.836 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.836 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.836 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.837 INFO:teuthology.orchestra.run.smithi049.stdout: "registry_password": { 2024-03-18T23:19:04.837 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "registry_password", 2024-03-18T23:19:04.837 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:04.837 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.837 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.837 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:04.837 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.837 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.838 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.838 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-03-18T23:19:04.838 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.838 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.838 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.838 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.838 INFO:teuthology.orchestra.run.smithi049.stdout: "registry_url": { 2024-03-18T23:19:04.838 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "registry_url", 2024-03-18T23:19:04.839 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:04.839 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.839 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.839 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:04.839 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.839 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.839 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.839 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-03-18T23:19:04.839 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.840 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.840 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.840 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.840 INFO:teuthology.orchestra.run.smithi049.stdout: "registry_username": { 2024-03-18T23:19:04.840 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "registry_username", 2024-03-18T23:19:04.840 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:04.840 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.840 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.841 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:04.841 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.841 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.841 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.841 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-03-18T23:19:04.841 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.841 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.841 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.841 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.842 INFO:teuthology.orchestra.run.smithi049.stdout: "secure_monitoring_stack": { 2024-03-18T23:19:04.842 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "secure_monitoring_stack", 2024-03-18T23:19:04.842 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:04.842 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.842 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.842 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:04.842 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.842 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.843 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.843 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-03-18T23:19:04.843 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.843 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.843 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.843 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.843 INFO:teuthology.orchestra.run.smithi049.stdout: "service_discovery_port": { 2024-03-18T23:19:04.843 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "service_discovery_port", 2024-03-18T23:19:04.843 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-18T23:19:04.844 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.844 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.844 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "8765", 2024-03-18T23:19:04.844 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.844 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.844 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.844 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "cephadm service discovery port", 2024-03-18T23:19:04.844 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.844 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.845 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.845 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.845 INFO:teuthology.orchestra.run.smithi049.stdout: "ssh_config_file": { 2024-03-18T23:19:04.845 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "ssh_config_file", 2024-03-18T23:19:04.845 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:04.845 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.845 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.845 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:04.846 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.846 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.846 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.846 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-03-18T23:19:04.846 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.846 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.846 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.846 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.846 INFO:teuthology.orchestra.run.smithi049.stdout: "use_agent": { 2024-03-18T23:19:04.847 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "use_agent", 2024-03-18T23:19:04.847 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:04.847 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.847 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.847 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:04.847 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.847 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.847 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.847 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-03-18T23:19:04.848 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.848 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.848 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.848 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.848 INFO:teuthology.orchestra.run.smithi049.stdout: "use_repo_digest": { 2024-03-18T23:19:04.848 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "use_repo_digest", 2024-03-18T23:19:04.848 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:04.848 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.848 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.849 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-18T23:19:04.849 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.849 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.849 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.849 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-03-18T23:19:04.849 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.849 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.849 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.849 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.850 INFO:teuthology.orchestra.run.smithi049.stdout: "warn_on_failed_host_check": { 2024-03-18T23:19:04.850 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "warn_on_failed_host_check", 2024-03-18T23:19:04.850 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:04.850 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.850 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.850 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-18T23:19:04.850 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.850 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.850 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.851 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "raise a health warning if the host check fails", 2024-03-18T23:19:04.851 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.851 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.851 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.851 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.851 INFO:teuthology.orchestra.run.smithi049.stdout: "warn_on_stray_daemons": { 2024-03-18T23:19:04.851 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "warn_on_stray_daemons", 2024-03-18T23:19:04.851 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:04.851 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.851 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.852 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-18T23:19:04.852 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.852 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.852 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.852 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-03-18T23:19:04.852 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.852 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.852 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.853 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.853 INFO:teuthology.orchestra.run.smithi049.stdout: "warn_on_stray_hosts": { 2024-03-18T23:19:04.853 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "warn_on_stray_hosts", 2024-03-18T23:19:04.853 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:04.853 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.853 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.853 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-18T23:19:04.853 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.853 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.854 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.854 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-03-18T23:19:04.854 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.854 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.854 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.854 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:19:04.854 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:19:04.854 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.855 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-18T23:19:04.855 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "crash", 2024-03-18T23:19:04.855 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-18T23:19:04.855 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-18T23:19:04.855 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-18T23:19:04.855 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-18T23:19:04.855 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-18T23:19:04.855 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:04.855 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.856 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:04.856 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:04.856 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.856 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.856 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-18T23:19:04.856 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-18T23:19:04.856 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-18T23:19:04.856 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-18T23:19:04.856 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-18T23:19:04.857 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-18T23:19:04.857 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-18T23:19:04.857 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-18T23:19:04.857 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.857 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.857 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.857 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.857 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.857 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-18T23:19:04.857 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-18T23:19:04.858 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:04.858 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.858 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:04.858 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:04.858 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.858 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.858 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.858 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.859 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.859 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.859 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.859 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.859 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-18T23:19:04.859 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-18T23:19:04.859 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:04.859 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.859 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:04.860 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-18T23:19:04.860 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.860 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.860 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-18T23:19:04.860 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-18T23:19:04.860 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-18T23:19:04.860 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-18T23:19:04.860 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-18T23:19:04.860 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-18T23:19:04.861 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-18T23:19:04.861 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-18T23:19:04.861 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.861 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.861 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.861 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.861 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.861 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-18T23:19:04.861 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-18T23:19:04.862 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:04.862 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.862 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:04.862 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:04.862 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.862 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.862 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.862 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.863 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.863 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.863 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.863 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.863 INFO:teuthology.orchestra.run.smithi049.stdout: "retain_interval": { 2024-03-18T23:19:04.863 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "retain_interval", 2024-03-18T23:19:04.863 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-03-18T23:19:04.863 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.864 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:04.864 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "31536000", 2024-03-18T23:19:04.864 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.864 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.864 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.864 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "how long to retain crashes before pruning them", 2024-03-18T23:19:04.864 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.864 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.864 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.865 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.865 INFO:teuthology.orchestra.run.smithi049.stdout: "warn_recent_interval": { 2024-03-18T23:19:04.865 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "warn_recent_interval", 2024-03-18T23:19:04.865 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-03-18T23:19:04.865 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.865 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:04.865 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "1209600", 2024-03-18T23:19:04.865 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.865 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.866 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.866 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "time interval in which to warn about recent crashes", 2024-03-18T23:19:04.866 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.866 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.866 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.866 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:19:04.866 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:19:04.866 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.867 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-18T23:19:04.867 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "dashboard", 2024-03-18T23:19:04.867 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-18T23:19:04.867 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-18T23:19:04.867 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-18T23:19:04.867 INFO:teuthology.orchestra.run.smithi049.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-03-18T23:19:04.867 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-03-18T23:19:04.867 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-18T23:19:04.867 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.868 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.868 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "10", 2024-03-18T23:19:04.868 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.868 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.868 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.868 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.868 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.868 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.868 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.869 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.869 INFO:teuthology.orchestra.run.smithi049.stdout: "ALERTMANAGER_API_HOST": { 2024-03-18T23:19:04.869 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "ALERTMANAGER_API_HOST", 2024-03-18T23:19:04.869 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:04.869 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.869 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.869 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:04.869 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.869 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.869 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.870 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.870 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.870 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.870 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.870 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.870 INFO:teuthology.orchestra.run.smithi049.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-03-18T23:19:04.870 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-03-18T23:19:04.870 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:04.871 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.871 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.871 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-18T23:19:04.871 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.871 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.871 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.871 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.871 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.871 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.872 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.872 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.872 INFO:teuthology.orchestra.run.smithi049.stdout: "AUDIT_API_ENABLED": { 2024-03-18T23:19:04.872 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "AUDIT_API_ENABLED", 2024-03-18T23:19:04.872 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:04.872 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.872 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.872 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:04.872 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.872 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.873 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.873 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.873 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.873 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.873 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.873 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.873 INFO:teuthology.orchestra.run.smithi049.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-03-18T23:19:04.873 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-03-18T23:19:04.874 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:04.874 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.874 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.874 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-18T23:19:04.874 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.874 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.874 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.874 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.875 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.875 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.875 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.875 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.875 INFO:teuthology.orchestra.run.smithi049.stdout: "ENABLE_BROWSABLE_API": { 2024-03-18T23:19:04.875 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "ENABLE_BROWSABLE_API", 2024-03-18T23:19:04.875 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:04.875 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.875 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.876 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-18T23:19:04.876 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.876 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.876 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.876 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.876 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.876 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.876 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.877 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.877 INFO:teuthology.orchestra.run.smithi049.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-03-18T23:19:04.877 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-03-18T23:19:04.877 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:04.877 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.877 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.877 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-18T23:19:04.877 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.877 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.878 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.878 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.878 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.878 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.878 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.878 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.878 INFO:teuthology.orchestra.run.smithi049.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-03-18T23:19:04.878 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-03-18T23:19:04.879 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:04.879 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.879 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.879 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-18T23:19:04.879 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.879 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.879 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.879 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.879 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.880 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.880 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.880 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.880 INFO:teuthology.orchestra.run.smithi049.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-03-18T23:19:04.880 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-03-18T23:19:04.880 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:04.880 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.880 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.880 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-18T23:19:04.881 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.881 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.881 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.881 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.881 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.881 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.881 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.881 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.881 INFO:teuthology.orchestra.run.smithi049.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-03-18T23:19:04.882 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-03-18T23:19:04.882 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:04.882 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.882 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.882 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-18T23:19:04.882 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.882 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.883 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.883 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.883 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.883 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.883 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.883 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.883 INFO:teuthology.orchestra.run.smithi049.stdout: "FEATURE_TOGGLE_NFS": { 2024-03-18T23:19:04.883 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-03-18T23:19:04.883 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:04.884 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.884 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.884 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-18T23:19:04.884 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.884 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.884 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.884 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.884 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.884 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.885 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.885 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.885 INFO:teuthology.orchestra.run.smithi049.stdout: "FEATURE_TOGGLE_RBD": { 2024-03-18T23:19:04.885 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-03-18T23:19:04.885 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:04.885 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.885 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.885 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-18T23:19:04.886 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.886 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.886 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.886 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.886 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.886 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.886 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.886 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.887 INFO:teuthology.orchestra.run.smithi049.stdout: "FEATURE_TOGGLE_RGW": { 2024-03-18T23:19:04.887 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-03-18T23:19:04.887 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:04.887 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.887 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.887 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-18T23:19:04.887 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.887 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.887 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.888 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.888 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.888 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.888 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.888 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.888 INFO:teuthology.orchestra.run.smithi049.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-03-18T23:19:04.888 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-03-18T23:19:04.888 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:04.888 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.889 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.889 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:04.889 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.889 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.889 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.889 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.889 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.889 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.890 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.890 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.890 INFO:teuthology.orchestra.run.smithi049.stdout: "GRAFANA_API_PASSWORD": { 2024-03-18T23:19:04.890 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "GRAFANA_API_PASSWORD", 2024-03-18T23:19:04.890 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:04.890 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.890 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.890 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "admin", 2024-03-18T23:19:04.890 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.891 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.891 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.891 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.891 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.891 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.891 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.891 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.891 INFO:teuthology.orchestra.run.smithi049.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-03-18T23:19:04.892 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-03-18T23:19:04.892 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:04.892 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.892 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.892 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-18T23:19:04.892 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.892 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.892 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.892 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.893 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.893 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.893 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.893 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.893 INFO:teuthology.orchestra.run.smithi049.stdout: "GRAFANA_API_URL": { 2024-03-18T23:19:04.893 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "GRAFANA_API_URL", 2024-03-18T23:19:04.893 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:04.893 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.894 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.894 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:04.894 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.894 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.894 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.894 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.894 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.894 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.895 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.895 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.895 INFO:teuthology.orchestra.run.smithi049.stdout: "GRAFANA_API_USERNAME": { 2024-03-18T23:19:04.895 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "GRAFANA_API_USERNAME", 2024-03-18T23:19:04.895 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:04.895 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.895 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.895 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "admin", 2024-03-18T23:19:04.896 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.896 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.896 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.896 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.896 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.896 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.896 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.896 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.896 INFO:teuthology.orchestra.run.smithi049.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-03-18T23:19:04.897 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-03-18T23:19:04.897 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:04.897 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.897 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.897 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:04.897 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.897 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.897 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.898 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.898 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.898 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.898 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.898 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.898 INFO:teuthology.orchestra.run.smithi049.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-03-18T23:19:04.898 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-03-18T23:19:04.898 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:04.899 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.899 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.899 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:04.899 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.899 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.899 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.899 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.899 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.899 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.900 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.900 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.900 INFO:teuthology.orchestra.run.smithi049.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-03-18T23:19:04.900 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-03-18T23:19:04.900 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:04.900 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.900 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.900 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-18T23:19:04.900 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.901 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.901 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.901 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.901 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.901 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.901 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.901 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.901 INFO:teuthology.orchestra.run.smithi049.stdout: "ISSUE_TRACKER_API_KEY": { 2024-03-18T23:19:04.901 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-03-18T23:19:04.902 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:04.902 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.902 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.902 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:04.902 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.902 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.902 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.902 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.902 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.903 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.903 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.903 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.903 INFO:teuthology.orchestra.run.smithi049.stdout: "MULTICLUSTER_CONFIG": { 2024-03-18T23:19:04.903 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "MULTICLUSTER_CONFIG", 2024-03-18T23:19:04.903 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:04.903 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.903 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.903 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "{}", 2024-03-18T23:19:04.904 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.904 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.904 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.904 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.904 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.904 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.904 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.904 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.905 INFO:teuthology.orchestra.run.smithi049.stdout: "PROMETHEUS_API_HOST": { 2024-03-18T23:19:04.905 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "PROMETHEUS_API_HOST", 2024-03-18T23:19:04.905 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:04.905 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.905 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.905 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:04.905 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.905 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.906 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.906 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.906 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.906 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.906 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.906 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.906 INFO:teuthology.orchestra.run.smithi049.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-03-18T23:19:04.906 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-03-18T23:19:04.906 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:04.907 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.907 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.907 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-18T23:19:04.907 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.907 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.907 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.907 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.907 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.908 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.908 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.908 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.908 INFO:teuthology.orchestra.run.smithi049.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-03-18T23:19:04.908 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-03-18T23:19:04.908 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:04.908 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.908 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.908 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:04.909 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.909 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.909 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.909 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.909 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.909 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.909 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.909 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.909 INFO:teuthology.orchestra.run.smithi049.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-03-18T23:19:04.910 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-03-18T23:19:04.910 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:04.910 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.910 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.910 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:04.910 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.910 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.910 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.911 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.911 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.911 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.911 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.911 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.911 INFO:teuthology.orchestra.run.smithi049.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-03-18T23:19:04.911 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-03-18T23:19:04.911 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:04.911 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.912 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.912 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-18T23:19:04.912 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.912 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.912 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.912 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.912 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.912 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.912 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.913 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.913 INFO:teuthology.orchestra.run.smithi049.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-03-18T23:19:04.913 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-03-18T23:19:04.913 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:04.913 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.913 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.913 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-18T23:19:04.913 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.914 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.914 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.914 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.914 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.914 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.914 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.914 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.914 INFO:teuthology.orchestra.run.smithi049.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-03-18T23:19:04.915 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-03-18T23:19:04.915 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:04.915 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.915 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.915 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:04.915 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.915 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.915 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.916 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.916 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.916 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.916 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.916 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.916 INFO:teuthology.orchestra.run.smithi049.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-03-18T23:19:04.916 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-03-18T23:19:04.916 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:04.916 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.917 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.917 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:04.917 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.917 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.917 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.917 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.917 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.917 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.917 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.918 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.918 INFO:teuthology.orchestra.run.smithi049.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-03-18T23:19:04.918 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-03-18T23:19:04.918 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:04.918 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.918 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.918 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:04.918 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.919 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.919 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.919 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.919 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.919 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.919 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.919 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.919 INFO:teuthology.orchestra.run.smithi049.stdout: "PWD_POLICY_ENABLED": { 2024-03-18T23:19:04.920 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "PWD_POLICY_ENABLED", 2024-03-18T23:19:04.920 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:04.920 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.920 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.920 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-18T23:19:04.920 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.920 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.920 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.920 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.921 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.921 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.921 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.921 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.921 INFO:teuthology.orchestra.run.smithi049.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-03-18T23:19:04.921 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-03-18T23:19:04.921 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:04.921 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.921 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.922 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-03-18T23:19:04.922 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.922 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.922 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.922 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.922 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.922 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.922 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.923 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.923 INFO:teuthology.orchestra.run.smithi049.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-03-18T23:19:04.923 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-03-18T23:19:04.923 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-18T23:19:04.923 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.923 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.923 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "10", 2024-03-18T23:19:04.923 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.923 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.924 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.924 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.924 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.924 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.924 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.924 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.924 INFO:teuthology.orchestra.run.smithi049.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-03-18T23:19:04.924 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-03-18T23:19:04.924 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-18T23:19:04.925 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.925 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.925 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "8", 2024-03-18T23:19:04.925 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.925 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.925 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.925 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.925 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.926 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.926 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.926 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.926 INFO:teuthology.orchestra.run.smithi049.stdout: "REST_REQUESTS_TIMEOUT": { 2024-03-18T23:19:04.926 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-03-18T23:19:04.926 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-18T23:19:04.926 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.926 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.927 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "45", 2024-03-18T23:19:04.927 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.927 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.927 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.927 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.927 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.927 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.927 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.927 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.928 INFO:teuthology.orchestra.run.smithi049.stdout: "RGW_API_ACCESS_KEY": { 2024-03-18T23:19:04.928 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "RGW_API_ACCESS_KEY", 2024-03-18T23:19:04.928 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:04.928 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.928 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.928 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:04.928 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.928 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.928 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.929 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.929 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.929 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.929 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.929 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.929 INFO:teuthology.orchestra.run.smithi049.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-03-18T23:19:04.929 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-03-18T23:19:04.929 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:04.930 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.930 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.930 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "admin", 2024-03-18T23:19:04.930 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.930 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.930 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.930 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.930 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.931 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.931 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.931 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.931 INFO:teuthology.orchestra.run.smithi049.stdout: "RGW_API_SECRET_KEY": { 2024-03-18T23:19:04.931 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "RGW_API_SECRET_KEY", 2024-03-18T23:19:04.931 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:04.931 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.931 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.931 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:04.931 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.932 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.932 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.932 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.932 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.932 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.932 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.932 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.932 INFO:teuthology.orchestra.run.smithi049.stdout: "RGW_API_SSL_VERIFY": { 2024-03-18T23:19:04.932 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "RGW_API_SSL_VERIFY", 2024-03-18T23:19:04.933 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:04.933 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.933 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.933 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-18T23:19:04.933 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.933 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.933 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.933 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.933 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.933 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.934 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.934 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.934 INFO:teuthology.orchestra.run.smithi049.stdout: "UNSAFE_TLS_v1_2": { 2024-03-18T23:19:04.934 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "UNSAFE_TLS_v1_2", 2024-03-18T23:19:04.934 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:04.934 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.934 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.934 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:04.935 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.935 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.935 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.935 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.935 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.935 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.935 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.935 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.935 INFO:teuthology.orchestra.run.smithi049.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-03-18T23:19:04.935 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-03-18T23:19:04.936 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-18T23:19:04.936 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.936 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.936 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "0", 2024-03-18T23:19:04.936 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.936 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.936 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.936 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.936 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.937 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.937 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.937 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.937 INFO:teuthology.orchestra.run.smithi049.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-03-18T23:19:04.937 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-03-18T23:19:04.937 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-18T23:19:04.937 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.937 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.937 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "10", 2024-03-18T23:19:04.938 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.938 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.938 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.938 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.938 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.938 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.938 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.938 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.938 INFO:teuthology.orchestra.run.smithi049.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-03-18T23:19:04.939 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-03-18T23:19:04.939 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-18T23:19:04.939 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.939 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.939 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "5", 2024-03-18T23:19:04.939 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.939 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.939 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.939 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.940 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.940 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.940 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.940 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.940 INFO:teuthology.orchestra.run.smithi049.stdout: "cross_origin_url": { 2024-03-18T23:19:04.940 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "cross_origin_url", 2024-03-18T23:19:04.940 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:04.940 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.940 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.940 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:04.941 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.941 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.941 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.941 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.941 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.941 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.941 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.941 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.941 INFO:teuthology.orchestra.run.smithi049.stdout: "crt_file": { 2024-03-18T23:19:04.941 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "crt_file", 2024-03-18T23:19:04.942 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:04.942 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.942 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.942 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:04.942 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.942 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.942 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.942 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.942 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.943 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.943 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.943 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.943 INFO:teuthology.orchestra.run.smithi049.stdout: "debug": { 2024-03-18T23:19:04.943 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "debug", 2024-03-18T23:19:04.943 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:04.943 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.943 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.943 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:04.943 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.944 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.944 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.944 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Enable/disable debug options", 2024-03-18T23:19:04.944 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.944 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.944 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.944 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.944 INFO:teuthology.orchestra.run.smithi049.stdout: "jwt_token_ttl": { 2024-03-18T23:19:04.944 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "jwt_token_ttl", 2024-03-18T23:19:04.945 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-18T23:19:04.945 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.945 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.945 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "28800", 2024-03-18T23:19:04.945 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.945 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.945 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.945 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.946 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.946 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.946 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.946 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.946 INFO:teuthology.orchestra.run.smithi049.stdout: "key_file": { 2024-03-18T23:19:04.946 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "key_file", 2024-03-18T23:19:04.946 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:04.946 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.947 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.947 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:04.947 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.947 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.947 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.947 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.947 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.947 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.947 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.948 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.948 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-18T23:19:04.948 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-18T23:19:04.948 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:04.948 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.948 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:04.948 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:04.948 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.948 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.949 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-18T23:19:04.949 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-18T23:19:04.949 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-18T23:19:04.949 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-18T23:19:04.949 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-18T23:19:04.949 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-18T23:19:04.949 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-18T23:19:04.949 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-18T23:19:04.949 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.949 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.950 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.950 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.950 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.950 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-18T23:19:04.950 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-18T23:19:04.950 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:04.950 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.950 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:04.951 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:04.951 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.951 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.951 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.951 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.951 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.951 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.951 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.951 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.951 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-18T23:19:04.952 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-18T23:19:04.952 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:04.952 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.952 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:04.952 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-18T23:19:04.952 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.952 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.952 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-18T23:19:04.952 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-18T23:19:04.953 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-18T23:19:04.953 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-18T23:19:04.953 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-18T23:19:04.953 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-18T23:19:04.953 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-18T23:19:04.953 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-18T23:19:04.953 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.953 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.954 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.954 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.954 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.954 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-18T23:19:04.954 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-18T23:19:04.954 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:04.954 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.954 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:04.955 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:04.955 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.955 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.955 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.955 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.955 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.955 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.955 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.955 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.956 INFO:teuthology.orchestra.run.smithi049.stdout: "motd": { 2024-03-18T23:19:04.956 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "motd", 2024-03-18T23:19:04.956 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:04.956 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.956 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.956 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:04.956 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.956 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.957 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.957 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "The message of the day", 2024-03-18T23:19:04.957 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.957 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.957 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.957 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.957 INFO:teuthology.orchestra.run.smithi049.stdout: "redirect_resolve_ip_addr": { 2024-03-18T23:19:04.957 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "redirect_resolve_ip_addr", 2024-03-18T23:19:04.957 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:04.958 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.958 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.958 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:04.958 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.958 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.958 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.958 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.958 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.959 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.959 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.959 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.959 INFO:teuthology.orchestra.run.smithi049.stdout: "server_addr": { 2024-03-18T23:19:04.959 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "server_addr", 2024-03-18T23:19:04.959 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:04.959 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.959 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.959 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "::", 2024-03-18T23:19:04.960 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.960 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.960 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.960 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.960 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.960 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.960 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.960 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.960 INFO:teuthology.orchestra.run.smithi049.stdout: "server_port": { 2024-03-18T23:19:04.961 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "server_port", 2024-03-18T23:19:04.961 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-18T23:19:04.961 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.961 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.961 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "8080", 2024-03-18T23:19:04.961 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.961 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.961 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.962 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.962 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.962 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.962 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.962 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.962 INFO:teuthology.orchestra.run.smithi049.stdout: "ssl": { 2024-03-18T23:19:04.962 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "ssl", 2024-03-18T23:19:04.962 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:04.963 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.963 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.963 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-18T23:19:04.963 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.963 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.963 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.963 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.963 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.963 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.964 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.964 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.964 INFO:teuthology.orchestra.run.smithi049.stdout: "ssl_server_port": { 2024-03-18T23:19:04.964 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "ssl_server_port", 2024-03-18T23:19:04.964 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-18T23:19:04.964 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.964 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.964 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "8443", 2024-03-18T23:19:04.965 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.965 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.965 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.965 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.965 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.965 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.965 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.965 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.965 INFO:teuthology.orchestra.run.smithi049.stdout: "standby_behaviour": { 2024-03-18T23:19:04.966 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "standby_behaviour", 2024-03-18T23:19:04.966 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:04.966 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.966 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.966 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "redirect", 2024-03-18T23:19:04.966 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.966 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.966 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-18T23:19:04.966 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-18T23:19:04.967 INFO:teuthology.orchestra.run.smithi049.stdout: "redirect" 2024-03-18T23:19:04.967 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-18T23:19:04.967 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.967 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.967 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.967 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.967 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.967 INFO:teuthology.orchestra.run.smithi049.stdout: "standby_error_status_code": { 2024-03-18T23:19:04.968 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "standby_error_status_code", 2024-03-18T23:19:04.968 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-18T23:19:04.968 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.968 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.968 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "500", 2024-03-18T23:19:04.968 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "400", 2024-03-18T23:19:04.968 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "599", 2024-03-18T23:19:04.968 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.968 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.969 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.969 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.969 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.969 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.969 INFO:teuthology.orchestra.run.smithi049.stdout: "url_prefix": { 2024-03-18T23:19:04.969 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "url_prefix", 2024-03-18T23:19:04.969 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:04.969 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.969 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:04.970 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:04.970 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.970 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.970 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.970 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.970 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.970 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.970 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.971 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:19:04.971 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:19:04.971 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.971 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-18T23:19:04.971 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "devicehealth", 2024-03-18T23:19:04.971 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-18T23:19:04.971 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-18T23:19:04.971 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-18T23:19:04.971 INFO:teuthology.orchestra.run.smithi049.stdout: "enable_monitoring": { 2024-03-18T23:19:04.972 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "enable_monitoring", 2024-03-18T23:19:04.972 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:04.972 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.972 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:04.972 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-18T23:19:04.972 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.972 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.972 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.973 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "monitor device health metrics", 2024-03-18T23:19:04.973 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.973 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.973 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.973 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.973 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-18T23:19:04.973 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-18T23:19:04.973 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:04.974 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.974 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:04.974 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:04.974 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.974 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.974 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-18T23:19:04.974 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-18T23:19:04.974 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-18T23:19:04.974 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-18T23:19:04.975 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-18T23:19:04.975 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-18T23:19:04.975 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-18T23:19:04.975 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-18T23:19:04.975 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.975 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.975 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.975 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.975 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.976 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-18T23:19:04.976 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-18T23:19:04.976 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:04.976 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.976 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:04.976 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:04.976 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.976 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.977 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.977 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.977 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.977 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.977 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.977 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.977 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-18T23:19:04.977 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-18T23:19:04.977 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:04.978 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.978 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:04.978 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-18T23:19:04.978 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.978 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.978 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-18T23:19:04.978 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-18T23:19:04.978 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-18T23:19:04.979 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-18T23:19:04.979 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-18T23:19:04.979 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-18T23:19:04.979 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-18T23:19:04.979 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-18T23:19:04.979 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.979 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.979 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.979 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.980 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.980 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-18T23:19:04.980 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-18T23:19:04.980 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:04.980 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.980 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:04.980 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:04.980 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.981 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.981 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.981 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:04.981 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.981 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.981 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.981 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.981 INFO:teuthology.orchestra.run.smithi049.stdout: "mark_out_threshold": { 2024-03-18T23:19:04.981 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "mark_out_threshold", 2024-03-18T23:19:04.982 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-03-18T23:19:04.982 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.982 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:04.982 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "2419200", 2024-03-18T23:19:04.982 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.982 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.982 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.982 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-03-18T23:19:04.983 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.983 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.983 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.983 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.983 INFO:teuthology.orchestra.run.smithi049.stdout: "pool_name": { 2024-03-18T23:19:04.983 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "pool_name", 2024-03-18T23:19:04.983 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:04.983 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.983 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:04.984 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "device_health_metrics", 2024-03-18T23:19:04.984 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.984 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.984 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.984 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "name of pool in which to store device health metrics", 2024-03-18T23:19:04.984 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.984 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.984 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.984 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.985 INFO:teuthology.orchestra.run.smithi049.stdout: "retention_period": { 2024-03-18T23:19:04.985 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "retention_period", 2024-03-18T23:19:04.985 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-03-18T23:19:04.985 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.985 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:04.985 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "15552000", 2024-03-18T23:19:04.985 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.985 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.985 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.986 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "how long to retain device health metrics", 2024-03-18T23:19:04.986 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.986 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.986 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.986 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.986 INFO:teuthology.orchestra.run.smithi049.stdout: "scrape_frequency": { 2024-03-18T23:19:04.986 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "scrape_frequency", 2024-03-18T23:19:04.986 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-03-18T23:19:04.987 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.987 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:04.987 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "86400", 2024-03-18T23:19:04.987 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.987 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.987 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.987 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "how frequently to scrape device health metrics", 2024-03-18T23:19:04.987 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.988 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.988 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.988 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.988 INFO:teuthology.orchestra.run.smithi049.stdout: "self_heal": { 2024-03-18T23:19:04.988 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "self_heal", 2024-03-18T23:19:04.988 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:04.988 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.988 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:04.988 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-18T23:19:04.989 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.989 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.989 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.989 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-03-18T23:19:04.989 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:04.989 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:04.989 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:04.990 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:04.990 INFO:teuthology.orchestra.run.smithi049.stdout: "sleep_interval": { 2024-03-18T23:19:04.990 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "sleep_interval", 2024-03-18T23:19:04.990 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-03-18T23:19:04.990 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:04.990 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:04.990 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "600", 2024-03-18T23:19:04.990 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:04.991 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:04.991 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:04.991 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "how frequently to wake up and check device health", 2024-03-18T23:19:05.088 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.089 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.089 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.089 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.089 INFO:teuthology.orchestra.run.smithi049.stdout: "warn_threshold": { 2024-03-18T23:19:05.089 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "warn_threshold", 2024-03-18T23:19:05.089 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-03-18T23:19:05.089 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.089 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.090 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "7257600", 2024-03-18T23:19:05.090 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.090 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.090 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.090 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-03-18T23:19:05.090 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.090 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.090 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.091 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:19:05.091 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:19:05.091 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.091 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-18T23:19:05.091 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "diskprediction_local", 2024-03-18T23:19:05.091 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-18T23:19:05.091 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-18T23:19:05.091 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-18T23:19:05.092 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-18T23:19:05.092 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-18T23:19:05.092 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:05.092 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.092 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.092 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:05.092 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.092 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.093 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-18T23:19:05.093 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-18T23:19:05.093 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-18T23:19:05.093 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-18T23:19:05.093 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-18T23:19:05.093 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-18T23:19:05.093 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-18T23:19:05.093 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-18T23:19:05.093 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.094 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.094 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.094 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.094 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.094 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-18T23:19:05.094 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-18T23:19:05.094 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:05.094 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.095 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.095 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:05.095 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.095 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.095 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.095 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.095 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.095 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.095 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.096 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.096 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-18T23:19:05.096 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-18T23:19:05.096 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:05.096 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.096 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.096 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-18T23:19:05.096 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.097 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.097 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-18T23:19:05.097 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-18T23:19:05.097 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-18T23:19:05.097 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-18T23:19:05.097 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-18T23:19:05.097 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-18T23:19:05.097 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-18T23:19:05.098 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-18T23:19:05.098 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.098 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.098 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.098 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.098 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.098 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-18T23:19:05.098 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-18T23:19:05.099 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:05.099 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.099 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.099 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:05.099 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.099 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.099 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.099 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.099 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.100 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.100 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.100 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.100 INFO:teuthology.orchestra.run.smithi049.stdout: "predict_interval": { 2024-03-18T23:19:05.100 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "predict_interval", 2024-03-18T23:19:05.100 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:05.100 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.100 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:05.100 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "86400", 2024-03-18T23:19:05.101 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.101 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.101 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.101 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.101 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.101 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.101 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.101 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.101 INFO:teuthology.orchestra.run.smithi049.stdout: "predictor_model": { 2024-03-18T23:19:05.102 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "predictor_model", 2024-03-18T23:19:05.102 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:05.102 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.102 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:05.102 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "prophetstor", 2024-03-18T23:19:05.102 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.102 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.102 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.103 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.103 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.103 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.103 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.103 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.103 INFO:teuthology.orchestra.run.smithi049.stdout: "sleep_interval": { 2024-03-18T23:19:05.103 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "sleep_interval", 2024-03-18T23:19:05.103 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:05.104 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.104 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:05.104 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "600", 2024-03-18T23:19:05.104 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.104 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.104 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.104 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.104 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.104 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.105 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.105 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:19:05.105 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:19:05.105 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.105 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-18T23:19:05.105 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "influx", 2024-03-18T23:19:05.105 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": false, 2024-03-18T23:19:05.105 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "influxdb python module not found", 2024-03-18T23:19:05.106 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-18T23:19:05.106 INFO:teuthology.orchestra.run.smithi049.stdout: "batch_size": { 2024-03-18T23:19:05.106 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "batch_size", 2024-03-18T23:19:05.106 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-18T23:19:05.106 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.106 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:05.106 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "5000", 2024-03-18T23:19:05.106 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.107 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.107 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.107 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-03-18T23:19:05.107 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.107 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.107 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.107 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.107 INFO:teuthology.orchestra.run.smithi049.stdout: "database": { 2024-03-18T23:19:05.107 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "database", 2024-03-18T23:19:05.108 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:05.108 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.108 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:05.108 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "ceph", 2024-03-18T23:19:05.108 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.108 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.108 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.108 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-03-18T23:19:05.109 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.109 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.109 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.109 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.109 INFO:teuthology.orchestra.run.smithi049.stdout: "hostname": { 2024-03-18T23:19:05.109 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "hostname", 2024-03-18T23:19:05.109 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:05.109 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.110 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:05.110 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:05.110 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.110 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.110 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.110 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "InfluxDB server hostname", 2024-03-18T23:19:05.110 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.110 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.111 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.111 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.111 INFO:teuthology.orchestra.run.smithi049.stdout: "interval": { 2024-03-18T23:19:05.111 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "interval", 2024-03-18T23:19:05.111 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-03-18T23:19:05.111 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.111 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:05.111 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "30", 2024-03-18T23:19:05.111 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "5", 2024-03-18T23:19:05.112 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.112 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.112 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-03-18T23:19:05.112 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.112 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.112 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.112 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.112 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-18T23:19:05.112 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-18T23:19:05.113 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:05.113 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.113 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.113 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:05.113 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.113 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.113 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-18T23:19:05.113 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-18T23:19:05.114 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-18T23:19:05.114 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-18T23:19:05.114 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-18T23:19:05.114 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-18T23:19:05.114 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-18T23:19:05.114 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-18T23:19:05.114 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.114 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.114 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.115 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.115 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.115 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-18T23:19:05.115 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-18T23:19:05.115 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:05.115 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.115 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.115 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:05.115 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.116 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.116 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.116 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.116 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.116 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.116 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.116 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.116 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-18T23:19:05.117 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-18T23:19:05.117 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:05.117 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.117 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.117 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-18T23:19:05.117 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.117 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.117 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-18T23:19:05.117 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-18T23:19:05.118 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-18T23:19:05.118 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-18T23:19:05.118 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-18T23:19:05.118 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-18T23:19:05.118 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-18T23:19:05.118 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-18T23:19:05.118 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.118 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.119 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.119 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.119 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.119 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-18T23:19:05.119 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-18T23:19:05.119 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:05.119 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.119 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.119 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:05.120 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.120 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.120 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.120 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.120 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.120 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.120 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.120 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.120 INFO:teuthology.orchestra.run.smithi049.stdout: "password": { 2024-03-18T23:19:05.121 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "password", 2024-03-18T23:19:05.121 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:05.121 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.121 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:05.121 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:05.121 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.121 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.121 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.121 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "password of InfluxDB server user", 2024-03-18T23:19:05.122 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.122 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.122 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.122 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.122 INFO:teuthology.orchestra.run.smithi049.stdout: "port": { 2024-03-18T23:19:05.122 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "port", 2024-03-18T23:19:05.122 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-18T23:19:05.122 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.123 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:05.123 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "8086", 2024-03-18T23:19:05.123 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.123 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.123 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.123 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "InfluxDB server port", 2024-03-18T23:19:05.123 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.123 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.123 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.124 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.124 INFO:teuthology.orchestra.run.smithi049.stdout: "ssl": { 2024-03-18T23:19:05.124 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "ssl", 2024-03-18T23:19:05.124 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:05.124 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.124 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:05.124 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "false", 2024-03-18T23:19:05.124 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.125 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.125 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.125 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-03-18T23:19:05.125 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.125 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.125 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.125 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.125 INFO:teuthology.orchestra.run.smithi049.stdout: "threads": { 2024-03-18T23:19:05.126 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "threads", 2024-03-18T23:19:05.126 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-18T23:19:05.126 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.126 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:05.126 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "5", 2024-03-18T23:19:05.126 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "1", 2024-03-18T23:19:05.126 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "32", 2024-03-18T23:19:05.126 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.126 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-03-18T23:19:05.127 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.127 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.127 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.127 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.127 INFO:teuthology.orchestra.run.smithi049.stdout: "username": { 2024-03-18T23:19:05.127 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "username", 2024-03-18T23:19:05.127 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:05.127 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.128 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:05.128 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:05.128 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.128 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.128 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.128 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "username of InfluxDB server user", 2024-03-18T23:19:05.128 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.128 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.128 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.128 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.129 INFO:teuthology.orchestra.run.smithi049.stdout: "verify_ssl": { 2024-03-18T23:19:05.129 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "verify_ssl", 2024-03-18T23:19:05.129 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:05.129 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.129 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:05.129 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "true", 2024-03-18T23:19:05.129 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.129 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.129 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.130 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-03-18T23:19:05.130 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.130 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.130 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.130 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:19:05.130 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:19:05.130 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.131 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-18T23:19:05.131 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "insights", 2024-03-18T23:19:05.131 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-18T23:19:05.131 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-18T23:19:05.131 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-18T23:19:05.131 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-18T23:19:05.131 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-18T23:19:05.131 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:05.131 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.132 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.132 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:05.132 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.132 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.132 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-18T23:19:05.132 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-18T23:19:05.132 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-18T23:19:05.132 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-18T23:19:05.132 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-18T23:19:05.132 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-18T23:19:05.133 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-18T23:19:05.133 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-18T23:19:05.133 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.133 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.133 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.133 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.133 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.133 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-18T23:19:05.133 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-18T23:19:05.134 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:05.134 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.134 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.134 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:05.134 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.134 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.134 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.134 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.135 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.135 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.135 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.135 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.135 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-18T23:19:05.135 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-18T23:19:05.135 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:05.135 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.135 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.136 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-18T23:19:05.136 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.136 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.136 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-18T23:19:05.136 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-18T23:19:05.136 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-18T23:19:05.136 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-18T23:19:05.136 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-18T23:19:05.136 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-18T23:19:05.137 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-18T23:19:05.137 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-18T23:19:05.137 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.137 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.137 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.137 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.137 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.137 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-18T23:19:05.138 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-18T23:19:05.138 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:05.138 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.138 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.138 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:05.138 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.138 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.138 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.138 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.139 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.139 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.139 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.139 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:19:05.139 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:19:05.139 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.139 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-18T23:19:05.139 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "iostat", 2024-03-18T23:19:05.139 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-18T23:19:05.140 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-18T23:19:05.140 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-18T23:19:05.140 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-18T23:19:05.140 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-18T23:19:05.140 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:05.140 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.140 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.140 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:05.140 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.141 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.141 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-18T23:19:05.141 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-18T23:19:05.141 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-18T23:19:05.141 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-18T23:19:05.141 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-18T23:19:05.141 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-18T23:19:05.141 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-18T23:19:05.142 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-18T23:19:05.142 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.142 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.142 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.142 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.142 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.142 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-18T23:19:05.142 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-18T23:19:05.143 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:05.143 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.143 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.143 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:05.143 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.143 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.143 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.143 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.143 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.144 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.144 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.144 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.144 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-18T23:19:05.144 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-18T23:19:05.144 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:05.144 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.144 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.145 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-18T23:19:05.145 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.145 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.145 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-18T23:19:05.145 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-18T23:19:05.145 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-18T23:19:05.145 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-18T23:19:05.145 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-18T23:19:05.146 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-18T23:19:05.146 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-18T23:19:05.146 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-18T23:19:05.146 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.146 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.146 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.146 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.146 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.147 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-18T23:19:05.147 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-18T23:19:05.147 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:05.147 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.147 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.147 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:05.147 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.147 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.147 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.148 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.148 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.148 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.148 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.148 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:19:05.148 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:19:05.148 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.148 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-18T23:19:05.148 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "k8sevents", 2024-03-18T23:19:05.149 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-18T23:19:05.149 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-18T23:19:05.149 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-18T23:19:05.149 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph_event_retention_days": { 2024-03-18T23:19:05.149 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "ceph_event_retention_days", 2024-03-18T23:19:05.149 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-18T23:19:05.149 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.149 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:05.149 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "7", 2024-03-18T23:19:05.149 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.150 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.150 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.150 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Days to hold ceph event information within local cache", 2024-03-18T23:19:05.150 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.150 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.150 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.150 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.150 INFO:teuthology.orchestra.run.smithi049.stdout: "config_check_secs": { 2024-03-18T23:19:05.151 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "config_check_secs", 2024-03-18T23:19:05.151 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-18T23:19:05.151 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.151 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:05.151 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "10", 2024-03-18T23:19:05.151 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "10", 2024-03-18T23:19:05.151 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.151 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.151 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-03-18T23:19:05.152 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.152 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.152 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.152 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.152 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-18T23:19:05.152 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-18T23:19:05.152 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:05.152 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.152 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.153 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:05.153 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.153 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.153 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-18T23:19:05.153 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-18T23:19:05.153 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-18T23:19:05.153 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-18T23:19:05.153 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-18T23:19:05.154 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-18T23:19:05.154 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-18T23:19:05.154 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-18T23:19:05.154 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.154 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.154 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.154 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.154 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.155 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-18T23:19:05.155 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-18T23:19:05.155 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:05.155 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.155 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.155 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:05.155 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.155 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.156 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.156 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.156 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.156 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.156 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.156 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.156 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-18T23:19:05.156 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-18T23:19:05.157 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:05.157 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.157 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.157 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-18T23:19:05.157 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.157 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.157 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-18T23:19:05.157 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-18T23:19:05.158 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-18T23:19:05.158 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-18T23:19:05.158 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-18T23:19:05.158 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-18T23:19:05.158 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-18T23:19:05.158 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-18T23:19:05.158 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.158 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.159 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.159 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.159 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.159 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-18T23:19:05.159 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-18T23:19:05.159 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:05.159 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.159 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.159 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:05.160 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.160 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.160 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.160 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.160 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.160 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.160 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.160 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:19:05.160 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:19:05.161 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.161 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-18T23:19:05.161 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "localpool", 2024-03-18T23:19:05.161 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-18T23:19:05.161 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-18T23:19:05.161 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-18T23:19:05.161 INFO:teuthology.orchestra.run.smithi049.stdout: "failure_domain": { 2024-03-18T23:19:05.161 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "failure_domain", 2024-03-18T23:19:05.162 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:05.162 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.162 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.162 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "host", 2024-03-18T23:19:05.162 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.162 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.162 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.162 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "failure domain for any created local pool", 2024-03-18T23:19:05.163 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-03-18T23:19:05.163 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.163 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.163 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.163 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-18T23:19:05.163 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-18T23:19:05.163 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:05.163 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.164 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.164 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:05.164 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.164 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.164 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-18T23:19:05.164 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-18T23:19:05.164 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-18T23:19:05.164 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-18T23:19:05.164 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-18T23:19:05.165 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-18T23:19:05.165 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-18T23:19:05.165 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-18T23:19:05.165 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.165 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.165 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.165 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.165 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.166 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-18T23:19:05.166 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-18T23:19:05.166 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:05.166 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.166 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.166 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:05.166 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.166 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.166 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.167 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.167 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.167 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.167 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.167 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.167 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-18T23:19:05.167 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-18T23:19:05.167 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:05.167 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.168 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.168 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-18T23:19:05.168 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.168 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.168 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-18T23:19:05.168 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-18T23:19:05.168 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-18T23:19:05.168 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-18T23:19:05.169 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-18T23:19:05.169 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-18T23:19:05.169 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-18T23:19:05.169 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-18T23:19:05.169 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.169 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.169 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.169 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.169 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.170 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-18T23:19:05.170 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-18T23:19:05.170 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:05.170 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.170 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.170 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:05.170 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.170 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.171 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.171 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.171 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.171 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.171 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.171 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.171 INFO:teuthology.orchestra.run.smithi049.stdout: "min_size": { 2024-03-18T23:19:05.171 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "min_size", 2024-03-18T23:19:05.171 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-18T23:19:05.172 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.172 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.172 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:05.172 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.172 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.172 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.172 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "default min_size for any created local pool", 2024-03-18T23:19:05.172 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-03-18T23:19:05.173 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.173 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.173 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.173 INFO:teuthology.orchestra.run.smithi049.stdout: "num_rep": { 2024-03-18T23:19:05.173 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "num_rep", 2024-03-18T23:19:05.173 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-18T23:19:05.173 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.173 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.173 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "3", 2024-03-18T23:19:05.174 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.174 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.174 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.174 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "default replica count for any created local pool", 2024-03-18T23:19:05.174 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.174 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.174 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.174 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.174 INFO:teuthology.orchestra.run.smithi049.stdout: "pg_num": { 2024-03-18T23:19:05.175 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "pg_num", 2024-03-18T23:19:05.175 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-18T23:19:05.175 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.175 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.175 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "128", 2024-03-18T23:19:05.175 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.175 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.175 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.176 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "default pg_num for any created local pool", 2024-03-18T23:19:05.176 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.176 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.176 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.176 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.176 INFO:teuthology.orchestra.run.smithi049.stdout: "prefix": { 2024-03-18T23:19:05.176 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "prefix", 2024-03-18T23:19:05.177 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:05.177 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.177 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.177 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:05.177 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.177 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.177 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.177 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "name prefix for any created local pool", 2024-03-18T23:19:05.178 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.178 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.178 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.178 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.178 INFO:teuthology.orchestra.run.smithi049.stdout: "subtree": { 2024-03-18T23:19:05.178 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "subtree", 2024-03-18T23:19:05.178 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:05.178 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.179 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.179 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "rack", 2024-03-18T23:19:05.179 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.179 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.179 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.179 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "CRUSH level for which to create a local pool", 2024-03-18T23:19:05.179 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-03-18T23:19:05.179 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.180 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.180 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:19:05.180 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:19:05.180 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.180 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-18T23:19:05.180 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "mds_autoscaler", 2024-03-18T23:19:05.180 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-18T23:19:05.180 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-18T23:19:05.180 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-18T23:19:05.181 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-18T23:19:05.181 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-18T23:19:05.181 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:05.181 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.181 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.181 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:05.181 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.181 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.181 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-18T23:19:05.182 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-18T23:19:05.182 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-18T23:19:05.182 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-18T23:19:05.182 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-18T23:19:05.182 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-18T23:19:05.182 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-18T23:19:05.182 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-18T23:19:05.182 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.183 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.183 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.183 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.183 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.183 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-18T23:19:05.183 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-18T23:19:05.183 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:05.183 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.184 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.184 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:05.184 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.184 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.184 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.184 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.184 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.184 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.184 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.185 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.185 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-18T23:19:05.185 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-18T23:19:05.185 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:05.185 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.185 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.185 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-18T23:19:05.185 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.186 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.186 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-18T23:19:05.186 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-18T23:19:05.186 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-18T23:19:05.186 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-18T23:19:05.186 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-18T23:19:05.186 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-18T23:19:05.186 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-18T23:19:05.187 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-18T23:19:05.187 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.187 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.187 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.187 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.187 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.187 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-18T23:19:05.187 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-18T23:19:05.188 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:05.188 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.188 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.188 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:05.188 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.188 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.188 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.188 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.188 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.189 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.189 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.189 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:19:05.189 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:19:05.189 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.189 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-18T23:19:05.189 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "mirroring", 2024-03-18T23:19:05.189 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-18T23:19:05.189 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-18T23:19:05.190 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-18T23:19:05.190 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-18T23:19:05.190 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-18T23:19:05.190 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:05.190 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.190 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.190 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:05.190 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.191 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.191 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-18T23:19:05.191 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-18T23:19:05.191 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-18T23:19:05.191 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-18T23:19:05.191 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-18T23:19:05.191 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-18T23:19:05.191 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-18T23:19:05.192 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-18T23:19:05.192 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.192 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.192 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.192 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.192 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.192 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-18T23:19:05.192 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-18T23:19:05.193 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:05.193 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.193 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.193 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:05.193 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.193 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.193 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.193 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.193 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.194 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.194 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.194 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.194 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-18T23:19:05.194 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-18T23:19:05.194 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:05.194 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.194 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.195 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-18T23:19:05.195 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.195 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.195 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-18T23:19:05.195 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-18T23:19:05.195 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-18T23:19:05.195 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-18T23:19:05.195 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-18T23:19:05.195 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-18T23:19:05.196 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-18T23:19:05.196 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-18T23:19:05.196 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.196 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.196 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.196 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.196 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.196 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-18T23:19:05.197 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-18T23:19:05.197 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:05.197 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.197 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.197 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:05.197 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.197 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.197 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.198 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.198 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.198 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.198 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.198 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:19:05.198 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:19:05.198 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.198 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-18T23:19:05.199 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "nfs", 2024-03-18T23:19:05.199 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-18T23:19:05.199 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-18T23:19:05.199 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-18T23:19:05.199 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-18T23:19:05.199 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-18T23:19:05.199 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:05.199 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.199 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.200 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:05.200 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.200 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.200 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-18T23:19:05.200 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-18T23:19:05.200 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-18T23:19:05.200 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-18T23:19:05.200 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-18T23:19:05.200 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-18T23:19:05.201 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-18T23:19:05.201 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-18T23:19:05.201 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.201 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.201 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.201 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.201 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.201 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-18T23:19:05.201 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-18T23:19:05.202 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:05.202 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.202 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.202 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:05.202 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.202 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.202 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.202 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.203 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.203 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.203 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.203 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.203 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-18T23:19:05.203 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-18T23:19:05.203 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:05.203 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.203 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.204 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-18T23:19:05.204 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.204 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.204 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-18T23:19:05.204 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-18T23:19:05.204 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-18T23:19:05.204 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-18T23:19:05.204 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-18T23:19:05.204 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-18T23:19:05.205 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-18T23:19:05.205 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-18T23:19:05.205 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.205 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.205 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.205 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.205 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.205 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-18T23:19:05.205 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-18T23:19:05.205 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:05.206 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.206 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.206 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:05.206 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.206 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.206 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.206 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.206 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.207 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.207 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.207 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:19:05.207 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:19:05.207 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.207 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-18T23:19:05.207 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "orchestrator", 2024-03-18T23:19:05.207 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-18T23:19:05.208 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-18T23:19:05.208 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-18T23:19:05.208 INFO:teuthology.orchestra.run.smithi049.stdout: "fail_fs": { 2024-03-18T23:19:05.208 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "fail_fs", 2024-03-18T23:19:05.208 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:05.208 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.208 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:05.208 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:05.208 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.209 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.209 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.209 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-03-18T23:19:05.209 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.209 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.209 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.209 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.209 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-18T23:19:05.209 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-18T23:19:05.210 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:05.210 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.210 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.210 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:05.210 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.210 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.210 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-18T23:19:05.210 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-18T23:19:05.211 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-18T23:19:05.211 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-18T23:19:05.211 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-18T23:19:05.211 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-18T23:19:05.211 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-18T23:19:05.211 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-18T23:19:05.211 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.211 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.211 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.212 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.212 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.212 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-18T23:19:05.212 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-18T23:19:05.212 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:05.212 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.212 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.212 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:05.212 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.213 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.213 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.213 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.213 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.213 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.213 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.213 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.213 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-18T23:19:05.214 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-18T23:19:05.214 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:05.214 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.214 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.214 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-18T23:19:05.214 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.214 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.214 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-18T23:19:05.214 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-18T23:19:05.215 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-18T23:19:05.215 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-18T23:19:05.215 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-18T23:19:05.215 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-18T23:19:05.215 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-18T23:19:05.215 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-18T23:19:05.215 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.215 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.216 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.216 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.216 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.216 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-18T23:19:05.216 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-18T23:19:05.216 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:05.216 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.216 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.216 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:05.217 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.217 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.217 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.217 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.217 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.217 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.217 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.217 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.217 INFO:teuthology.orchestra.run.smithi049.stdout: "orchestrator": { 2024-03-18T23:19:05.218 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "orchestrator", 2024-03-18T23:19:05.218 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:05.218 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.218 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.218 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:05.218 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.218 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.218 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-18T23:19:05.219 INFO:teuthology.orchestra.run.smithi049.stdout: "cephadm", 2024-03-18T23:19:05.219 INFO:teuthology.orchestra.run.smithi049.stdout: "rook", 2024-03-18T23:19:05.219 INFO:teuthology.orchestra.run.smithi049.stdout: "test_orchestrator" 2024-03-18T23:19:05.219 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-18T23:19:05.219 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Orchestrator backend", 2024-03-18T23:19:05.219 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.219 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.219 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.219 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:19:05.220 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:19:05.220 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.220 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-18T23:19:05.220 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "osd_perf_query", 2024-03-18T23:19:05.220 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-18T23:19:05.220 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-18T23:19:05.220 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-18T23:19:05.220 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-18T23:19:05.220 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-18T23:19:05.221 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:05.221 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.221 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.221 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:05.221 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.221 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.221 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-18T23:19:05.221 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-18T23:19:05.222 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-18T23:19:05.222 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-18T23:19:05.222 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-18T23:19:05.222 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-18T23:19:05.222 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-18T23:19:05.222 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-18T23:19:05.222 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.223 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.223 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.223 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.223 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.223 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-18T23:19:05.223 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-18T23:19:05.223 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:05.223 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.223 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.224 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:05.224 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.224 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.224 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.224 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.224 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.224 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.224 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.225 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.225 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-18T23:19:05.225 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-18T23:19:05.225 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:05.225 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.225 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.225 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-18T23:19:05.225 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.225 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.226 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-18T23:19:05.226 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-18T23:19:05.226 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-18T23:19:05.226 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-18T23:19:05.226 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-18T23:19:05.226 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-18T23:19:05.226 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-18T23:19:05.226 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-18T23:19:05.227 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.227 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.227 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.227 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.227 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.227 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-18T23:19:05.227 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-18T23:19:05.227 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:05.227 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.228 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.228 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:05.228 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.228 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.228 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.228 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.228 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.228 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.229 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.229 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:19:05.229 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:19:05.229 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.229 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-18T23:19:05.229 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "osd_support", 2024-03-18T23:19:05.229 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-18T23:19:05.229 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-18T23:19:05.229 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-18T23:19:05.230 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-18T23:19:05.230 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-18T23:19:05.230 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:05.230 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.230 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.230 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:05.230 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.230 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.231 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-18T23:19:05.231 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-18T23:19:05.231 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-18T23:19:05.231 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-18T23:19:05.231 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-18T23:19:05.231 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-18T23:19:05.231 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-18T23:19:05.231 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-18T23:19:05.231 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.232 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.232 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.232 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.232 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.232 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-18T23:19:05.232 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-18T23:19:05.232 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:05.232 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.233 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.233 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:05.233 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.233 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.233 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.233 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.233 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.233 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.233 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.234 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.234 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-18T23:19:05.234 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-18T23:19:05.234 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:05.234 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.234 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.234 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-18T23:19:05.234 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.235 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.235 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-18T23:19:05.235 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-18T23:19:05.235 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-18T23:19:05.235 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-18T23:19:05.235 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-18T23:19:05.235 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-18T23:19:05.235 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-18T23:19:05.235 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-18T23:19:05.236 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.236 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.236 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.236 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.236 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.236 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-18T23:19:05.236 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-18T23:19:05.236 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:05.236 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.237 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.237 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:05.237 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.237 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.237 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.237 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.237 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.237 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.238 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.238 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:19:05.238 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:19:05.238 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.238 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-18T23:19:05.238 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "pg_autoscaler", 2024-03-18T23:19:05.238 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-18T23:19:05.238 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-18T23:19:05.238 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-18T23:19:05.239 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-18T23:19:05.239 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-18T23:19:05.239 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:05.239 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.239 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.239 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:05.239 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.239 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.240 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-18T23:19:05.240 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-18T23:19:05.240 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-18T23:19:05.240 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-18T23:19:05.240 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-18T23:19:05.240 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-18T23:19:05.240 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-18T23:19:05.240 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-18T23:19:05.240 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.241 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.241 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.241 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.241 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.241 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-18T23:19:05.241 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-18T23:19:05.241 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:05.241 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.242 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.242 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:05.242 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.242 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.242 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.242 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.242 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.242 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.243 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.243 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.243 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-18T23:19:05.243 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-18T23:19:05.243 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:05.243 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.243 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.243 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-18T23:19:05.243 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.244 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.244 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-18T23:19:05.244 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-18T23:19:05.244 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-18T23:19:05.244 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-18T23:19:05.244 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-18T23:19:05.244 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-18T23:19:05.244 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-18T23:19:05.245 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-18T23:19:05.245 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.245 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.245 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.245 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.245 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.245 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-18T23:19:05.245 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-18T23:19:05.245 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:05.246 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.246 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.246 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:05.246 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.246 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.246 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.246 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.246 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.247 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.247 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.247 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.247 INFO:teuthology.orchestra.run.smithi049.stdout: "sleep_interval": { 2024-03-18T23:19:05.247 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "sleep_interval", 2024-03-18T23:19:05.247 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-03-18T23:19:05.247 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.247 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:05.247 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "60", 2024-03-18T23:19:05.248 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.248 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.248 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.248 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.248 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.248 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.248 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.248 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.248 INFO:teuthology.orchestra.run.smithi049.stdout: "threshold": { 2024-03-18T23:19:05.249 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "threshold", 2024-03-18T23:19:05.249 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "float", 2024-03-18T23:19:05.249 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.249 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:05.249 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "3.0", 2024-03-18T23:19:05.249 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "1.0", 2024-03-18T23:19:05.249 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.249 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.249 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "scaling threshold", 2024-03-18T23:19:05.250 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-03-18T23:19:05.250 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.250 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.250 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:19:05.250 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:19:05.250 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.250 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-18T23:19:05.250 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "progress", 2024-03-18T23:19:05.251 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-18T23:19:05.251 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-18T23:19:05.251 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-18T23:19:05.251 INFO:teuthology.orchestra.run.smithi049.stdout: "allow_pg_recovery_event": { 2024-03-18T23:19:05.251 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "allow_pg_recovery_event", 2024-03-18T23:19:05.251 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:05.251 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.251 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.251 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:05.252 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.252 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.252 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.252 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "allow the module to show pg recovery progress", 2024-03-18T23:19:05.252 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.252 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.252 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.252 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.253 INFO:teuthology.orchestra.run.smithi049.stdout: "enabled": { 2024-03-18T23:19:05.253 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "enabled", 2024-03-18T23:19:05.253 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:05.253 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.253 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:05.253 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-18T23:19:05.253 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.253 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.253 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.254 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.254 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.254 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.254 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.254 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.254 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-18T23:19:05.254 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-18T23:19:05.254 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:05.255 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.255 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.255 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:05.255 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.255 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.255 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-18T23:19:05.255 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-18T23:19:05.255 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-18T23:19:05.255 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-18T23:19:05.256 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-18T23:19:05.256 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-18T23:19:05.256 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-18T23:19:05.256 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-18T23:19:05.256 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.256 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.256 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.256 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.257 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.257 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-18T23:19:05.257 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-18T23:19:05.257 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:05.257 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.257 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.257 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:05.257 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.257 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.258 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.258 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.258 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.258 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.258 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.258 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.258 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-18T23:19:05.258 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-18T23:19:05.259 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:05.259 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.259 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.259 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-18T23:19:05.259 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.259 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.259 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-18T23:19:05.259 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-18T23:19:05.259 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-18T23:19:05.260 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-18T23:19:05.260 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-18T23:19:05.260 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-18T23:19:05.260 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-18T23:19:05.260 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-18T23:19:05.260 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.260 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.260 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.261 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.261 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.261 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-18T23:19:05.261 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-18T23:19:05.261 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:05.261 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.261 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.261 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:05.262 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.262 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.262 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.262 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.262 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.262 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.262 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.262 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.263 INFO:teuthology.orchestra.run.smithi049.stdout: "max_completed_events": { 2024-03-18T23:19:05.263 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "max_completed_events", 2024-03-18T23:19:05.263 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-18T23:19:05.263 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.263 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.263 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "50", 2024-03-18T23:19:05.263 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.263 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.263 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.264 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "number of past completed events to remember", 2024-03-18T23:19:05.264 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.264 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.264 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.264 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.264 INFO:teuthology.orchestra.run.smithi049.stdout: "sleep_interval": { 2024-03-18T23:19:05.264 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "sleep_interval", 2024-03-18T23:19:05.264 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-03-18T23:19:05.265 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.265 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.265 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "5", 2024-03-18T23:19:05.265 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.265 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.265 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.265 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "how long the module is going to sleep", 2024-03-18T23:19:05.265 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.265 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.266 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.266 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:19:05.266 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:19:05.266 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.266 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-18T23:19:05.266 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "prometheus", 2024-03-18T23:19:05.266 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-18T23:19:05.266 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-18T23:19:05.267 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-18T23:19:05.267 INFO:teuthology.orchestra.run.smithi049.stdout: "cache": { 2024-03-18T23:19:05.267 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "cache", 2024-03-18T23:19:05.267 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:05.267 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.267 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:05.267 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-18T23:19:05.267 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.268 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.268 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.268 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.268 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.268 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.268 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.268 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.268 INFO:teuthology.orchestra.run.smithi049.stdout: "exclude_perf_counters": { 2024-03-18T23:19:05.269 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "exclude_perf_counters", 2024-03-18T23:19:05.269 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:05.269 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.269 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.269 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-18T23:19:05.269 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.269 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.269 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.269 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-03-18T23:19:05.270 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-03-18T23:19:05.270 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.270 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.270 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.270 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-18T23:19:05.270 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-18T23:19:05.270 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:05.270 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.271 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.271 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:05.271 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.271 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.271 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-18T23:19:05.271 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-18T23:19:05.271 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-18T23:19:05.271 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-18T23:19:05.272 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-18T23:19:05.272 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-18T23:19:05.272 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-18T23:19:05.272 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-18T23:19:05.272 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.272 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.272 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.272 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.272 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.273 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-18T23:19:05.273 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-18T23:19:05.273 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:05.273 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.273 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.273 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:05.273 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.273 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.273 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.274 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.274 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.274 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.274 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.274 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.274 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-18T23:19:05.274 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-18T23:19:05.274 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:05.275 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.275 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.275 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-18T23:19:05.275 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.275 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.275 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-18T23:19:05.275 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-18T23:19:05.275 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-18T23:19:05.275 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-18T23:19:05.276 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-18T23:19:05.276 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-18T23:19:05.276 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-18T23:19:05.276 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-18T23:19:05.276 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.276 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.276 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.276 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.276 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.277 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-18T23:19:05.277 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-18T23:19:05.277 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:05.277 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.277 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.277 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:05.277 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.277 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.278 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.278 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.278 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.278 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.278 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.278 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.278 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_stats_pools": { 2024-03-18T23:19:05.278 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "rbd_stats_pools", 2024-03-18T23:19:05.278 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:05.279 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.279 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:05.279 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:05.279 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.279 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.279 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.279 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.279 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.279 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.280 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.280 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.280 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_stats_pools_refresh_interval": { 2024-03-18T23:19:05.280 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-03-18T23:19:05.280 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-18T23:19:05.280 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.280 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:05.280 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "300", 2024-03-18T23:19:05.280 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.281 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.281 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.281 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.281 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.281 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.281 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.281 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.281 INFO:teuthology.orchestra.run.smithi049.stdout: "scrape_interval": { 2024-03-18T23:19:05.282 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "scrape_interval", 2024-03-18T23:19:05.282 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "float", 2024-03-18T23:19:05.282 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.282 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:05.282 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "15.0", 2024-03-18T23:19:05.282 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.282 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.282 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.283 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.283 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.283 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.283 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.283 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.283 INFO:teuthology.orchestra.run.smithi049.stdout: "server_addr": { 2024-03-18T23:19:05.283 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "server_addr", 2024-03-18T23:19:05.283 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:05.283 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.283 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:05.284 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "::", 2024-03-18T23:19:05.284 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.284 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.284 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.284 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-03-18T23:19:05.284 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.284 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.284 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.285 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.285 INFO:teuthology.orchestra.run.smithi049.stdout: "server_port": { 2024-03-18T23:19:05.285 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "server_port", 2024-03-18T23:19:05.285 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-18T23:19:05.285 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.285 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.285 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "9283", 2024-03-18T23:19:05.285 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.285 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.286 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.286 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-03-18T23:19:05.286 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.286 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.286 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.286 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.286 INFO:teuthology.orchestra.run.smithi049.stdout: "stale_cache_strategy": { 2024-03-18T23:19:05.286 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "stale_cache_strategy", 2024-03-18T23:19:05.287 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:05.287 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.287 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:05.287 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "log", 2024-03-18T23:19:05.287 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.287 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.287 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.287 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.287 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.288 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.288 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.288 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.288 INFO:teuthology.orchestra.run.smithi049.stdout: "standby_behaviour": { 2024-03-18T23:19:05.288 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "standby_behaviour", 2024-03-18T23:19:05.288 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:05.288 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.288 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.289 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "default", 2024-03-18T23:19:05.289 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.289 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.289 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-18T23:19:05.289 INFO:teuthology.orchestra.run.smithi049.stdout: "default", 2024-03-18T23:19:05.289 INFO:teuthology.orchestra.run.smithi049.stdout: "error" 2024-03-18T23:19:05.289 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-18T23:19:05.290 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.290 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.290 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.290 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.290 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.290 INFO:teuthology.orchestra.run.smithi049.stdout: "standby_error_status_code": { 2024-03-18T23:19:05.290 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "standby_error_status_code", 2024-03-18T23:19:05.290 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-18T23:19:05.291 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.291 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.291 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "500", 2024-03-18T23:19:05.291 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "400", 2024-03-18T23:19:05.291 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "599", 2024-03-18T23:19:05.291 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.291 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.291 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.292 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.292 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.292 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:19:05.292 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:19:05.292 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.292 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-18T23:19:05.292 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "rbd_support", 2024-03-18T23:19:05.293 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-18T23:19:05.293 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-18T23:19:05.293 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-18T23:19:05.293 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-18T23:19:05.293 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-18T23:19:05.293 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:05.293 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.293 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.293 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:05.294 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.294 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.294 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-18T23:19:05.294 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-18T23:19:05.294 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-18T23:19:05.294 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-18T23:19:05.294 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-18T23:19:05.294 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-18T23:19:05.295 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-18T23:19:05.295 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-18T23:19:05.295 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.295 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.295 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.295 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.295 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.295 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-18T23:19:05.295 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-18T23:19:05.296 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:05.296 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.296 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.296 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:05.296 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.296 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.296 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.296 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.296 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.297 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.297 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.297 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.297 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-18T23:19:05.297 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-18T23:19:05.297 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:05.297 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.297 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.298 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-18T23:19:05.298 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.298 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.298 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-18T23:19:05.298 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-18T23:19:05.298 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-18T23:19:05.298 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-18T23:19:05.298 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-18T23:19:05.298 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-18T23:19:05.299 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-18T23:19:05.299 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-18T23:19:05.299 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.299 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.299 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.299 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.299 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.299 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-18T23:19:05.300 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-18T23:19:05.300 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:05.300 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.300 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.300 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:05.300 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.300 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.300 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.301 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.301 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.301 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.301 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.301 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.301 INFO:teuthology.orchestra.run.smithi049.stdout: "max_concurrent_snap_create": { 2024-03-18T23:19:05.301 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "max_concurrent_snap_create", 2024-03-18T23:19:05.301 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-18T23:19:05.301 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.302 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:05.302 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "10", 2024-03-18T23:19:05.302 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.302 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.302 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.302 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.302 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.302 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.303 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.303 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.303 INFO:teuthology.orchestra.run.smithi049.stdout: "mirror_snapshot_schedule": { 2024-03-18T23:19:05.303 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "mirror_snapshot_schedule", 2024-03-18T23:19:05.303 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:05.303 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.303 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:05.303 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:05.303 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.304 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.304 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.304 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.304 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.304 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.304 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.304 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.304 INFO:teuthology.orchestra.run.smithi049.stdout: "trash_purge_schedule": { 2024-03-18T23:19:05.305 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "trash_purge_schedule", 2024-03-18T23:19:05.305 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:05.305 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.305 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:05.305 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:05.305 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.305 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.305 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.305 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.306 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.306 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.306 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.306 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:19:05.306 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:19:05.306 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.306 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-18T23:19:05.306 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "restful", 2024-03-18T23:19:05.307 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-18T23:19:05.307 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-18T23:19:05.307 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-18T23:19:05.307 INFO:teuthology.orchestra.run.smithi049.stdout: "enable_auth": { 2024-03-18T23:19:05.307 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "enable_auth", 2024-03-18T23:19:05.307 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:05.307 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.307 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:05.307 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-18T23:19:05.308 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.308 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.308 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.308 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.308 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.308 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.308 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.308 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.309 INFO:teuthology.orchestra.run.smithi049.stdout: "key_file": { 2024-03-18T23:19:05.309 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "key_file", 2024-03-18T23:19:05.309 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:05.309 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.309 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:05.309 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:05.309 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.309 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.309 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.310 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.310 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.310 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.310 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.310 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.310 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-18T23:19:05.310 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-18T23:19:05.310 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:05.311 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.311 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.311 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:05.311 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.311 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.311 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-18T23:19:05.311 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-18T23:19:05.311 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-18T23:19:05.312 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-18T23:19:05.312 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-18T23:19:05.312 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-18T23:19:05.312 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-18T23:19:05.312 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-18T23:19:05.312 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.312 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.312 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.312 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.313 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.313 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-18T23:19:05.313 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-18T23:19:05.313 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:05.313 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.313 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.313 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:05.313 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.314 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.314 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.314 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.314 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.314 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.314 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.314 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.314 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-18T23:19:05.315 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-18T23:19:05.315 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:05.315 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.315 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.315 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-18T23:19:05.315 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.315 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.315 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-18T23:19:05.316 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-18T23:19:05.316 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-18T23:19:05.316 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-18T23:19:05.316 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-18T23:19:05.316 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-18T23:19:05.316 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-18T23:19:05.316 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-18T23:19:05.316 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.316 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.317 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.317 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.317 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.317 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-18T23:19:05.317 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-18T23:19:05.317 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:05.317 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.317 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.317 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:05.318 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.318 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.318 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.318 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.318 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.318 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.318 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.318 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.319 INFO:teuthology.orchestra.run.smithi049.stdout: "server_addr": { 2024-03-18T23:19:05.319 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "server_addr", 2024-03-18T23:19:05.319 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:05.319 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.319 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:05.319 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:05.319 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.319 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.319 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.320 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.320 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.320 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.320 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.320 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.320 INFO:teuthology.orchestra.run.smithi049.stdout: "server_port": { 2024-03-18T23:19:05.320 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "server_port", 2024-03-18T23:19:05.320 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:05.321 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.321 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:05.321 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:05.321 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.321 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.321 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.321 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.321 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.321 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.322 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.322 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:19:05.322 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:19:05.322 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.322 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-18T23:19:05.322 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "rgw", 2024-03-18T23:19:05.322 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-18T23:19:05.322 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-18T23:19:05.322 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-18T23:19:05.323 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-18T23:19:05.323 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-18T23:19:05.323 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:05.323 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.323 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.323 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:05.323 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.323 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.323 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-18T23:19:05.324 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-18T23:19:05.324 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-18T23:19:05.324 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-18T23:19:05.324 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-18T23:19:05.324 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-18T23:19:05.324 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-18T23:19:05.324 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-18T23:19:05.324 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.325 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.325 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.325 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.325 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.325 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-18T23:19:05.325 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-18T23:19:05.325 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:05.325 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.326 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.326 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:05.326 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.326 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.326 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.326 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.326 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.326 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.326 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.327 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.327 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-18T23:19:05.327 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-18T23:19:05.327 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:05.327 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.327 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.327 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-18T23:19:05.327 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.327 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.328 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-18T23:19:05.328 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-18T23:19:05.328 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-18T23:19:05.328 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-18T23:19:05.328 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-18T23:19:05.328 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-18T23:19:05.328 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-18T23:19:05.328 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-18T23:19:05.329 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.329 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.329 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.329 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.329 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.329 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-18T23:19:05.329 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-18T23:19:05.329 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:05.329 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.330 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.330 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:05.330 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.330 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.330 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.330 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.330 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.330 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.331 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.331 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:19:05.331 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:19:05.331 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.331 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-18T23:19:05.331 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "rook", 2024-03-18T23:19:05.331 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-18T23:19:05.331 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-18T23:19:05.331 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-18T23:19:05.332 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-18T23:19:05.332 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-18T23:19:05.332 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:05.332 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.332 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.332 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:05.332 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.332 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.333 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-18T23:19:05.333 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-18T23:19:05.333 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-18T23:19:05.333 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-18T23:19:05.333 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-18T23:19:05.333 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-18T23:19:05.333 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-18T23:19:05.333 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-18T23:19:05.333 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.334 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.334 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.334 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.334 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.334 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-18T23:19:05.334 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-18T23:19:05.334 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:05.334 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.335 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.335 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:05.335 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.335 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.335 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.335 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.335 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.335 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.335 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.336 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.336 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-18T23:19:05.336 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-18T23:19:05.336 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:05.336 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.336 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.336 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-18T23:19:05.336 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.337 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.337 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-18T23:19:05.337 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-18T23:19:05.337 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-18T23:19:05.337 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-18T23:19:05.337 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-18T23:19:05.337 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-18T23:19:05.337 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-18T23:19:05.337 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-18T23:19:05.338 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.338 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.338 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.338 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.338 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.338 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-18T23:19:05.338 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-18T23:19:05.338 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:05.339 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.339 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.339 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:05.339 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.339 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.339 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.339 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.339 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.339 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.340 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.340 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.340 INFO:teuthology.orchestra.run.smithi049.stdout: "storage_class": { 2024-03-18T23:19:05.340 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "storage_class", 2024-03-18T23:19:05.340 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:05.340 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.340 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:05.340 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "local", 2024-03-18T23:19:05.340 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.341 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.341 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.341 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-03-18T23:19:05.341 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.341 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.341 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.341 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:19:05.341 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:19:05.342 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.342 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-18T23:19:05.342 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "selftest", 2024-03-18T23:19:05.342 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-18T23:19:05.342 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-18T23:19:05.342 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-18T23:19:05.342 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-18T23:19:05.342 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-18T23:19:05.343 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:05.343 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.343 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.343 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:05.343 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.343 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.343 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-18T23:19:05.343 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-18T23:19:05.344 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-18T23:19:05.344 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-18T23:19:05.344 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-18T23:19:05.344 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-18T23:19:05.344 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-18T23:19:05.344 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-18T23:19:05.344 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.344 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.344 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.345 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.345 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.345 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-18T23:19:05.345 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-18T23:19:05.345 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:05.345 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.345 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.345 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:05.346 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.346 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.346 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.346 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.346 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.346 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.346 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.346 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.346 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-18T23:19:05.347 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-18T23:19:05.347 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:05.347 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.347 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.347 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-18T23:19:05.347 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.347 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.347 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-18T23:19:05.348 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-18T23:19:05.348 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-18T23:19:05.348 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-18T23:19:05.348 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-18T23:19:05.348 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-18T23:19:05.348 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-18T23:19:05.348 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-18T23:19:05.348 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.348 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.349 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.349 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.349 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.349 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-18T23:19:05.349 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-18T23:19:05.349 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:05.349 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.349 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.350 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:05.350 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.350 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.350 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.350 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.350 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.350 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.350 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.351 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.351 INFO:teuthology.orchestra.run.smithi049.stdout: "roption1": { 2024-03-18T23:19:05.351 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "roption1", 2024-03-18T23:19:05.351 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:05.351 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.351 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:05.351 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:05.351 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.351 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.352 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.352 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.352 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.352 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.352 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.352 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.352 INFO:teuthology.orchestra.run.smithi049.stdout: "roption2": { 2024-03-18T23:19:05.352 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "roption2", 2024-03-18T23:19:05.352 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:05.353 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.353 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:05.353 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "xyz", 2024-03-18T23:19:05.353 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.353 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.353 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.353 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.353 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.353 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.354 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.354 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.354 INFO:teuthology.orchestra.run.smithi049.stdout: "rwoption1": { 2024-03-18T23:19:05.354 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "rwoption1", 2024-03-18T23:19:05.354 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:05.354 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.354 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:05.354 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:05.355 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.355 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.355 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.355 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.355 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.355 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.355 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.355 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.355 INFO:teuthology.orchestra.run.smithi049.stdout: "rwoption2": { 2024-03-18T23:19:05.356 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "rwoption2", 2024-03-18T23:19:05.356 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-18T23:19:05.356 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.356 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:05.356 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:05.356 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.356 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.356 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.356 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.357 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.357 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.357 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.357 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.357 INFO:teuthology.orchestra.run.smithi049.stdout: "rwoption3": { 2024-03-18T23:19:05.357 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "rwoption3", 2024-03-18T23:19:05.357 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "float", 2024-03-18T23:19:05.357 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.358 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:05.358 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:05.358 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.358 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.358 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.358 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.358 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.358 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.359 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.359 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.359 INFO:teuthology.orchestra.run.smithi049.stdout: "rwoption4": { 2024-03-18T23:19:05.359 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "rwoption4", 2024-03-18T23:19:05.359 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:05.359 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.359 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:05.359 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:05.360 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.360 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.360 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.360 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.360 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.360 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.360 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.360 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.360 INFO:teuthology.orchestra.run.smithi049.stdout: "rwoption5": { 2024-03-18T23:19:05.361 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "rwoption5", 2024-03-18T23:19:05.361 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:05.361 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.361 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:05.361 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:05.361 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.361 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.361 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.362 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.362 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.362 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.362 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.362 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.362 INFO:teuthology.orchestra.run.smithi049.stdout: "rwoption6": { 2024-03-18T23:19:05.362 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "rwoption6", 2024-03-18T23:19:05.362 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:05.363 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.363 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:05.363 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-18T23:19:05.363 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.363 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.363 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.363 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.363 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.363 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.364 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.364 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.364 INFO:teuthology.orchestra.run.smithi049.stdout: "rwoption7": { 2024-03-18T23:19:05.364 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "rwoption7", 2024-03-18T23:19:05.364 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-18T23:19:05.364 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.364 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:05.364 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:05.365 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "1", 2024-03-18T23:19:05.365 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "42", 2024-03-18T23:19:05.365 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.365 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.365 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.365 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.365 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.365 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.365 INFO:teuthology.orchestra.run.smithi049.stdout: "testkey": { 2024-03-18T23:19:05.366 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "testkey", 2024-03-18T23:19:05.366 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:05.366 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.366 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:05.366 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:05.366 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.366 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.366 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.367 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.367 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.367 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.367 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.367 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.367 INFO:teuthology.orchestra.run.smithi049.stdout: "testlkey": { 2024-03-18T23:19:05.367 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "testlkey", 2024-03-18T23:19:05.367 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:05.368 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.368 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:05.368 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:05.368 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.368 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.368 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.368 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.368 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.368 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.369 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.369 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.369 INFO:teuthology.orchestra.run.smithi049.stdout: "testnewline": { 2024-03-18T23:19:05.369 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "testnewline", 2024-03-18T23:19:05.369 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:05.369 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.369 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:05.369 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:05.369 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.370 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.370 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.370 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.370 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.370 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.370 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.370 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:19:05.370 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:19:05.371 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.371 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-18T23:19:05.371 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "snap_schedule", 2024-03-18T23:19:05.371 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-18T23:19:05.371 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-18T23:19:05.371 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-18T23:19:05.371 INFO:teuthology.orchestra.run.smithi049.stdout: "allow_m_granularity": { 2024-03-18T23:19:05.371 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "allow_m_granularity", 2024-03-18T23:19:05.372 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:05.372 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.372 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.372 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:05.372 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.372 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.372 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.372 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "allow minute scheduled snapshots", 2024-03-18T23:19:05.372 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.373 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.373 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.373 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.373 INFO:teuthology.orchestra.run.smithi049.stdout: "dump_on_update": { 2024-03-18T23:19:05.373 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "dump_on_update", 2024-03-18T23:19:05.373 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:05.373 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.373 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.373 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:05.374 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.374 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.374 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.374 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "dump database to debug log on update", 2024-03-18T23:19:05.374 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.374 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.374 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.374 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.375 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-18T23:19:05.375 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-18T23:19:05.375 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:05.375 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.375 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.375 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:05.375 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.375 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.375 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-18T23:19:05.376 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-18T23:19:05.376 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-18T23:19:05.376 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-18T23:19:05.376 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-18T23:19:05.376 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-18T23:19:05.376 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-18T23:19:05.376 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-18T23:19:05.376 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.377 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.377 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.377 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.377 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.377 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-18T23:19:05.377 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-18T23:19:05.377 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:05.377 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.377 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.378 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:05.378 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.378 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.378 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.378 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.378 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.378 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.379 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.379 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.379 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-18T23:19:05.379 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-18T23:19:05.379 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:05.379 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.379 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.379 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-18T23:19:05.379 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.379 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.380 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-18T23:19:05.380 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-18T23:19:05.380 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-18T23:19:05.380 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-18T23:19:05.380 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-18T23:19:05.380 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-18T23:19:05.380 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-18T23:19:05.380 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-18T23:19:05.380 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.381 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.381 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.381 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.381 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.381 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-18T23:19:05.381 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-18T23:19:05.381 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:05.382 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.382 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.382 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:05.382 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.382 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.382 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.382 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.382 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.383 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.383 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.383 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:19:05.383 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:19:05.383 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.383 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-18T23:19:05.383 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "stats", 2024-03-18T23:19:05.383 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-18T23:19:05.383 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-18T23:19:05.384 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-18T23:19:05.384 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-18T23:19:05.384 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-18T23:19:05.384 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:05.384 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.384 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.384 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:05.384 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.384 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.385 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-18T23:19:05.385 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-18T23:19:05.385 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-18T23:19:05.385 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-18T23:19:05.385 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-18T23:19:05.385 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-18T23:19:05.385 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-18T23:19:05.385 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-18T23:19:05.386 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.386 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.386 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.386 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.386 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.386 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-18T23:19:05.386 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-18T23:19:05.386 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:05.387 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.387 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.387 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:05.387 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.387 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.387 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.387 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.387 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.387 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.388 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.388 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.388 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-18T23:19:05.388 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-18T23:19:05.388 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:05.388 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.388 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.388 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-18T23:19:05.388 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.389 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.389 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-18T23:19:05.389 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-18T23:19:05.389 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-18T23:19:05.389 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-18T23:19:05.389 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-18T23:19:05.389 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-18T23:19:05.389 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-18T23:19:05.389 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-18T23:19:05.390 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.390 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.390 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.390 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.390 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.390 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-18T23:19:05.390 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-18T23:19:05.390 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:05.391 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.391 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.391 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:05.391 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.391 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.391 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.391 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.391 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.392 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.392 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.392 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:19:05.392 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:19:05.392 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.392 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-18T23:19:05.392 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "status", 2024-03-18T23:19:05.392 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-18T23:19:05.392 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-18T23:19:05.393 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-18T23:19:05.393 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-18T23:19:05.393 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-18T23:19:05.393 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:05.393 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.393 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.393 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:05.393 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.394 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.394 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-18T23:19:05.394 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-18T23:19:05.394 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-18T23:19:05.394 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-18T23:19:05.394 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-18T23:19:05.394 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-18T23:19:05.394 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-18T23:19:05.395 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-18T23:19:05.395 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.395 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.395 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.395 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.395 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.395 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-18T23:19:05.395 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-18T23:19:05.396 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:05.396 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.396 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.396 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:05.396 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.396 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.396 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.396 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.397 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.397 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.397 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.397 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.397 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-18T23:19:05.397 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-18T23:19:05.397 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:05.397 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.397 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.398 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-18T23:19:05.398 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.398 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.398 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-18T23:19:05.398 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-18T23:19:05.398 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-18T23:19:05.398 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-18T23:19:05.398 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-18T23:19:05.399 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-18T23:19:05.399 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-18T23:19:05.399 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-18T23:19:05.399 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.399 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.399 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.399 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.399 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.400 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-18T23:19:05.400 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-18T23:19:05.400 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:05.400 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.400 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.400 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:05.400 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.400 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.400 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.401 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.401 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.401 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.401 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.401 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:19:05.401 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:19:05.401 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.401 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-18T23:19:05.401 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "telegraf", 2024-03-18T23:19:05.402 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-18T23:19:05.402 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-18T23:19:05.402 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-18T23:19:05.402 INFO:teuthology.orchestra.run.smithi049.stdout: "address": { 2024-03-18T23:19:05.402 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "address", 2024-03-18T23:19:05.402 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:05.402 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.402 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:05.403 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-03-18T23:19:05.403 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.403 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.403 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.403 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.403 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.403 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.403 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.403 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.404 INFO:teuthology.orchestra.run.smithi049.stdout: "interval": { 2024-03-18T23:19:05.404 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "interval", 2024-03-18T23:19:05.404 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-03-18T23:19:05.404 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.404 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:05.404 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "15", 2024-03-18T23:19:05.404 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.404 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.404 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.405 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.405 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.405 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.405 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.405 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.405 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-18T23:19:05.405 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-18T23:19:05.405 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:05.406 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.406 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.406 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:05.406 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.406 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.406 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-18T23:19:05.406 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-18T23:19:05.406 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-18T23:19:05.407 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-18T23:19:05.407 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-18T23:19:05.407 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-18T23:19:05.407 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-18T23:19:05.407 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-18T23:19:05.407 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.407 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.407 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.407 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.408 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.408 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-18T23:19:05.408 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-18T23:19:05.408 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:05.408 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.408 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.408 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:05.408 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.409 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.409 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.409 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.409 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.409 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.409 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.409 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.409 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-18T23:19:05.409 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-18T23:19:05.410 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:05.410 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.410 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.410 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-18T23:19:05.410 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.410 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.410 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-18T23:19:05.410 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-18T23:19:05.410 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-18T23:19:05.411 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-18T23:19:05.411 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-18T23:19:05.411 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-18T23:19:05.411 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-18T23:19:05.411 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-18T23:19:05.411 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.411 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.411 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.412 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.412 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.412 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-18T23:19:05.412 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-18T23:19:05.412 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:05.412 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.412 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.412 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:05.412 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.413 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.413 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.413 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.413 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.413 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.413 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.413 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:19:05.413 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:19:05.413 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.414 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-18T23:19:05.414 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "telemetry", 2024-03-18T23:19:05.414 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-18T23:19:05.414 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-18T23:19:05.414 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-18T23:19:05.414 INFO:teuthology.orchestra.run.smithi049.stdout: "channel_basic": { 2024-03-18T23:19:05.414 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "channel_basic", 2024-03-18T23:19:05.414 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:05.415 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.415 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:05.415 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-18T23:19:05.415 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.415 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.415 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.415 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Share basic cluster information (size, version)", 2024-03-18T23:19:05.415 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.415 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.416 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.416 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.416 INFO:teuthology.orchestra.run.smithi049.stdout: "channel_crash": { 2024-03-18T23:19:05.416 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "channel_crash", 2024-03-18T23:19:05.416 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:05.416 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.416 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:05.416 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-18T23:19:05.416 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.417 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.417 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.417 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-03-18T23:19:05.417 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.417 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.417 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.417 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.417 INFO:teuthology.orchestra.run.smithi049.stdout: "channel_device": { 2024-03-18T23:19:05.418 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "channel_device", 2024-03-18T23:19:05.418 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:05.418 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.418 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:05.418 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-18T23:19:05.418 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.418 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.418 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.418 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-03-18T23:19:05.419 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.419 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.419 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.419 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.419 INFO:teuthology.orchestra.run.smithi049.stdout: "channel_ident": { 2024-03-18T23:19:05.419 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "channel_ident", 2024-03-18T23:19:05.419 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:05.419 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.419 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:05.420 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:05.420 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.420 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.420 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.420 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-03-18T23:19:05.420 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.420 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.420 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.420 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.421 INFO:teuthology.orchestra.run.smithi049.stdout: "channel_perf": { 2024-03-18T23:19:05.421 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "channel_perf", 2024-03-18T23:19:05.421 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:05.421 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.421 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:05.421 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:05.421 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.421 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.421 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.422 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Share various performance metrics of a cluster", 2024-03-18T23:19:05.422 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.422 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.422 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.422 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.422 INFO:teuthology.orchestra.run.smithi049.stdout: "contact": { 2024-03-18T23:19:05.422 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "contact", 2024-03-18T23:19:05.422 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:05.423 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.423 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:05.423 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:05.423 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.423 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.423 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.423 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.423 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.423 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.424 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.424 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.424 INFO:teuthology.orchestra.run.smithi049.stdout: "description": { 2024-03-18T23:19:05.424 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "description", 2024-03-18T23:19:05.424 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:05.424 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.424 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:05.424 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:05.424 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.425 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.425 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.425 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.425 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.425 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.425 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.425 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.425 INFO:teuthology.orchestra.run.smithi049.stdout: "device_url": { 2024-03-18T23:19:05.425 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "device_url", 2024-03-18T23:19:05.426 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:05.426 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.426 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:05.426 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-03-18T23:19:05.426 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.426 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.426 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.426 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.427 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.427 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.427 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.427 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.427 INFO:teuthology.orchestra.run.smithi049.stdout: "enabled": { 2024-03-18T23:19:05.427 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "enabled", 2024-03-18T23:19:05.427 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:05.427 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.428 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:05.428 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:05.428 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.428 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.428 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.428 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.428 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.428 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.428 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.429 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.429 INFO:teuthology.orchestra.run.smithi049.stdout: "interval": { 2024-03-18T23:19:05.429 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "interval", 2024-03-18T23:19:05.429 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-18T23:19:05.429 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.429 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:05.429 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "24", 2024-03-18T23:19:05.429 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "8", 2024-03-18T23:19:05.429 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.429 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.430 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.430 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.430 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.430 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.430 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.430 INFO:teuthology.orchestra.run.smithi049.stdout: "last_opt_revision": { 2024-03-18T23:19:05.430 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "last_opt_revision", 2024-03-18T23:19:05.430 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-18T23:19:05.431 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.431 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:05.431 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "1", 2024-03-18T23:19:05.431 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.431 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.431 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.431 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.431 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.432 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.432 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.432 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.432 INFO:teuthology.orchestra.run.smithi049.stdout: "leaderboard": { 2024-03-18T23:19:05.432 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "leaderboard", 2024-03-18T23:19:05.432 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:05.432 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.432 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:05.432 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:05.433 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.433 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.433 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.433 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.433 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.433 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.433 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.433 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.433 INFO:teuthology.orchestra.run.smithi049.stdout: "leaderboard_description": { 2024-03-18T23:19:05.434 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "leaderboard_description", 2024-03-18T23:19:05.434 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:05.434 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.434 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:05.434 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:05.434 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.434 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.434 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.435 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.435 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.435 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.435 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.435 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.437 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-18T23:19:05.437 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-18T23:19:05.437 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:05.437 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.437 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.437 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:05.438 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.438 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.438 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-18T23:19:05.438 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-18T23:19:05.438 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-18T23:19:05.438 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-18T23:19:05.438 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-18T23:19:05.438 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-18T23:19:05.439 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-18T23:19:05.439 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-18T23:19:05.439 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.439 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.439 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.439 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.439 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.439 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-18T23:19:05.439 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-18T23:19:05.440 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:05.440 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.440 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.440 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:05.440 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.440 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.440 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.440 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.440 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.441 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.441 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.441 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.441 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-18T23:19:05.441 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-18T23:19:05.441 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:05.441 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.441 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.441 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-18T23:19:05.442 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.442 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.442 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-18T23:19:05.442 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-18T23:19:05.442 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-18T23:19:05.442 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-18T23:19:05.442 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-18T23:19:05.442 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-18T23:19:05.443 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-18T23:19:05.443 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-18T23:19:05.443 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.443 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.443 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.443 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.443 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.443 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-18T23:19:05.443 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-18T23:19:05.444 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:05.444 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.444 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.444 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:05.444 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.444 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.444 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.444 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.445 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.445 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.445 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.445 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.445 INFO:teuthology.orchestra.run.smithi049.stdout: "organization": { 2024-03-18T23:19:05.445 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "organization", 2024-03-18T23:19:05.445 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:05.445 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.445 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:05.446 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:05.446 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.446 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.446 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.446 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.446 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.446 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.446 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.447 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.447 INFO:teuthology.orchestra.run.smithi049.stdout: "proxy": { 2024-03-18T23:19:05.447 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "proxy", 2024-03-18T23:19:05.447 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:05.447 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.447 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:05.447 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:05.447 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.448 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.448 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.448 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.448 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.448 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.448 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.448 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.448 INFO:teuthology.orchestra.run.smithi049.stdout: "url": { 2024-03-18T23:19:05.448 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "url", 2024-03-18T23:19:05.449 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:05.449 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.449 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:05.449 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-03-18T23:19:05.449 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.449 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.449 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.449 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.449 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.450 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.450 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.450 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:19:05.450 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:19:05.450 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.450 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-18T23:19:05.450 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "test_orchestrator", 2024-03-18T23:19:05.450 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-18T23:19:05.450 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-18T23:19:05.451 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-18T23:19:05.451 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-18T23:19:05.451 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-18T23:19:05.451 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:05.451 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.451 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.451 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:05.451 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.452 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.452 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-18T23:19:05.452 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-18T23:19:05.452 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-18T23:19:05.452 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-18T23:19:05.452 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-18T23:19:05.452 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-18T23:19:05.452 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-18T23:19:05.452 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-18T23:19:05.452 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.453 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.453 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.453 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.453 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.453 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-18T23:19:05.453 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-18T23:19:05.453 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:05.453 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.454 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.454 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:05.454 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.454 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.454 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.454 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.454 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.454 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.455 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.455 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.455 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-18T23:19:05.455 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-18T23:19:05.455 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:05.455 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.455 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.455 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-18T23:19:05.455 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.456 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.456 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-18T23:19:05.456 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-18T23:19:05.456 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-18T23:19:05.456 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-18T23:19:05.456 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-18T23:19:05.456 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-18T23:19:05.456 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-18T23:19:05.457 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-18T23:19:05.457 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.457 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.457 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.457 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.457 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.457 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-18T23:19:05.457 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-18T23:19:05.458 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:05.458 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.458 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.458 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:05.458 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.458 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.458 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.458 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.458 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.459 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.459 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.459 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:19:05.459 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:19:05.459 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.459 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-18T23:19:05.459 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "volumes", 2024-03-18T23:19:05.459 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-18T23:19:05.460 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-18T23:19:05.460 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-18T23:19:05.460 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-18T23:19:05.460 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-18T23:19:05.460 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:05.460 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.460 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.460 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:05.460 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.461 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.461 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-18T23:19:05.461 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-18T23:19:05.461 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-18T23:19:05.461 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-18T23:19:05.461 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-18T23:19:05.461 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-18T23:19:05.461 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-18T23:19:05.461 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-18T23:19:05.462 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.462 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.462 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.462 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.462 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.462 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-18T23:19:05.462 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-18T23:19:05.462 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:05.463 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.463 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.463 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:05.463 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.463 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.463 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.463 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.463 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.464 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.464 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.464 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.464 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-18T23:19:05.464 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-18T23:19:05.464 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:05.464 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.464 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.464 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-18T23:19:05.465 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.465 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.465 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-18T23:19:05.465 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-18T23:19:05.465 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-18T23:19:05.465 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-18T23:19:05.465 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-18T23:19:05.465 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-18T23:19:05.465 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-18T23:19:05.466 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-18T23:19:05.466 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.466 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.466 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.466 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.466 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.466 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-18T23:19:05.467 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-18T23:19:05.467 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:05.467 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.467 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.467 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:05.467 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.467 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.467 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.467 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.468 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.468 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.468 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.468 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.468 INFO:teuthology.orchestra.run.smithi049.stdout: "max_concurrent_clones": { 2024-03-18T23:19:05.468 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "max_concurrent_clones", 2024-03-18T23:19:05.468 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-18T23:19:05.468 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.468 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:05.469 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "4", 2024-03-18T23:19:05.469 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.469 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.469 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.469 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Number of asynchronous cloner threads", 2024-03-18T23:19:05.469 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.469 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.469 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.469 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.470 INFO:teuthology.orchestra.run.smithi049.stdout: "periodic_async_work": { 2024-03-18T23:19:05.470 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "periodic_async_work", 2024-03-18T23:19:05.470 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:05.470 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.470 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:05.470 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:05.470 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.470 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.470 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.471 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Periodically check for async work", 2024-03-18T23:19:05.471 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.471 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.471 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.471 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.471 INFO:teuthology.orchestra.run.smithi049.stdout: "snapshot_clone_delay": { 2024-03-18T23:19:05.471 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "snapshot_clone_delay", 2024-03-18T23:19:05.471 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-18T23:19:05.471 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.471 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:05.472 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "0", 2024-03-18T23:19:05.472 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.472 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.472 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.472 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-03-18T23:19:05.472 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.472 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.472 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.472 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.473 INFO:teuthology.orchestra.run.smithi049.stdout: "snapshot_clone_no_wait": { 2024-03-18T23:19:05.473 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "snapshot_clone_no_wait", 2024-03-18T23:19:05.473 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:05.473 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.473 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:05.473 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-18T23:19:05.473 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.474 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.474 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.474 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-03-18T23:19:05.474 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.474 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.474 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.474 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:19:05.474 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:19:05.475 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.475 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-18T23:19:05.475 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "zabbix", 2024-03-18T23:19:05.475 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-18T23:19:05.475 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-18T23:19:05.475 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-18T23:19:05.475 INFO:teuthology.orchestra.run.smithi049.stdout: "discovery_interval": { 2024-03-18T23:19:05.475 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "discovery_interval", 2024-03-18T23:19:05.475 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "uint", 2024-03-18T23:19:05.476 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.476 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:05.476 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "100", 2024-03-18T23:19:05.476 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.476 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.476 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.476 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.476 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.477 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.477 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.477 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.477 INFO:teuthology.orchestra.run.smithi049.stdout: "identifier": { 2024-03-18T23:19:05.477 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "identifier", 2024-03-18T23:19:05.477 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:05.477 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.477 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:05.478 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:05.478 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.478 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.478 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.478 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.478 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.478 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.478 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.479 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.479 INFO:teuthology.orchestra.run.smithi049.stdout: "interval": { 2024-03-18T23:19:05.479 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "interval", 2024-03-18T23:19:05.479 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-03-18T23:19:05.479 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.479 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:05.479 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "60", 2024-03-18T23:19:05.479 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.479 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.480 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.480 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.480 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.480 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.480 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.480 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.480 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-18T23:19:05.480 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-18T23:19:05.480 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:05.481 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.481 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.481 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:05.481 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.481 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.481 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-18T23:19:05.481 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-18T23:19:05.481 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-18T23:19:05.481 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-18T23:19:05.482 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-18T23:19:05.482 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-18T23:19:05.482 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-18T23:19:05.482 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-18T23:19:05.482 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.482 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.482 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.482 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.482 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.483 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-18T23:19:05.483 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-18T23:19:05.483 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:05.483 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.483 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.483 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:05.483 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.483 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.484 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.484 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.484 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.484 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.484 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.484 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.484 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-18T23:19:05.484 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-18T23:19:05.484 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:05.485 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.485 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.485 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-18T23:19:05.485 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.485 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.485 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-18T23:19:05.485 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-18T23:19:05.485 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-18T23:19:05.485 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-18T23:19:05.486 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-18T23:19:05.486 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-18T23:19:05.486 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-18T23:19:05.486 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-18T23:19:05.486 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.486 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.486 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.486 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.487 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.487 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-18T23:19:05.487 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-18T23:19:05.487 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:05.487 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.487 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.487 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:05.487 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.487 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.488 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.488 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.488 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.488 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.488 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.488 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.488 INFO:teuthology.orchestra.run.smithi049.stdout: "zabbix_host": { 2024-03-18T23:19:05.488 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "zabbix_host", 2024-03-18T23:19:05.488 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:05.489 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.489 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:05.489 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:05.489 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.489 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.489 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.489 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.489 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.489 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.490 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.490 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.490 INFO:teuthology.orchestra.run.smithi049.stdout: "zabbix_port": { 2024-03-18T23:19:05.490 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "zabbix_port", 2024-03-18T23:19:05.490 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-18T23:19:05.490 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.490 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:05.490 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "10051", 2024-03-18T23:19:05.491 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.491 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.491 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.491 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.491 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.491 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.491 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.491 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.492 INFO:teuthology.orchestra.run.smithi049.stdout: "zabbix_sender": { 2024-03-18T23:19:05.492 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "zabbix_sender", 2024-03-18T23:19:05.492 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:05.492 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.492 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:05.492 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-03-18T23:19:05.492 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.492 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.492 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.493 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.493 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.493 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.493 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.493 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:19:05.493 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:19:05.493 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:19:05.493 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-03-18T23:19:05.494 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:19:05.494 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-18T23:19:05.494 INFO:teuthology.orchestra.run.smithi049.stdout: "modules": [ 2024-03-18T23:19:05.494 INFO:teuthology.orchestra.run.smithi049.stdout: "cephadm", 2024-03-18T23:19:05.494 INFO:teuthology.orchestra.run.smithi049.stdout: "dashboard", 2024-03-18T23:19:05.494 INFO:teuthology.orchestra.run.smithi049.stdout: "iostat", 2024-03-18T23:19:05.494 INFO:teuthology.orchestra.run.smithi049.stdout: "nfs", 2024-03-18T23:19:05.494 INFO:teuthology.orchestra.run.smithi049.stdout: "restful", 2024-03-18T23:19:05.495 INFO:teuthology.orchestra.run.smithi049.stdout: "snap_schedule" 2024-03-18T23:19:05.495 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-18T23:19:05.495 INFO:teuthology.orchestra.run.smithi049.stdout: "available_modules": [ 2024-03-18T23:19:05.495 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-18T23:19:05.495 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "alerts", 2024-03-18T23:19:05.495 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-18T23:19:05.495 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-18T23:19:05.495 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-18T23:19:05.495 INFO:teuthology.orchestra.run.smithi049.stdout: "interval": { 2024-03-18T23:19:05.496 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "interval", 2024-03-18T23:19:05.496 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-03-18T23:19:05.496 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.496 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.496 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "60", 2024-03-18T23:19:05.496 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.496 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.496 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.497 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "How frequently to reexamine health status", 2024-03-18T23:19:05.497 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.497 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.497 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.497 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.497 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-18T23:19:05.497 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-18T23:19:05.497 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:05.498 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.498 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.498 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:05.498 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.498 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.498 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-18T23:19:05.498 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-18T23:19:05.498 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-18T23:19:05.499 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-18T23:19:05.499 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-18T23:19:05.499 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-18T23:19:05.499 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-18T23:19:05.499 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-18T23:19:05.499 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.499 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.499 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.499 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.500 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.500 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-18T23:19:05.500 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-18T23:19:05.500 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:05.500 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.500 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.500 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:05.500 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.500 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.501 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.501 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.501 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.501 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.501 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.501 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.501 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-18T23:19:05.501 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-18T23:19:05.501 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:05.501 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.502 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.502 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-18T23:19:05.502 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.502 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.502 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-18T23:19:05.502 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-18T23:19:05.502 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-18T23:19:05.502 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-18T23:19:05.503 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-18T23:19:05.503 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-18T23:19:05.503 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-18T23:19:05.503 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-18T23:19:05.503 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.503 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.503 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.503 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.504 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.504 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-18T23:19:05.504 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-18T23:19:05.504 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:05.504 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.504 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.504 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:05.504 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.505 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.505 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.505 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.505 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.505 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.505 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.505 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.505 INFO:teuthology.orchestra.run.smithi049.stdout: "smtp_destination": { 2024-03-18T23:19:05.505 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "smtp_destination", 2024-03-18T23:19:05.506 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:05.506 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.506 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.506 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:05.506 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.506 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.506 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.506 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Email address to send alerts to", 2024-03-18T23:19:05.507 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.507 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.507 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.507 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.507 INFO:teuthology.orchestra.run.smithi049.stdout: "smtp_from_name": { 2024-03-18T23:19:05.507 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "smtp_from_name", 2024-03-18T23:19:05.507 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:05.507 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.507 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.508 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "Ceph", 2024-03-18T23:19:05.508 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.508 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.508 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.508 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Email From: name", 2024-03-18T23:19:05.508 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.508 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.508 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.508 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.509 INFO:teuthology.orchestra.run.smithi049.stdout: "smtp_host": { 2024-03-18T23:19:05.509 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "smtp_host", 2024-03-18T23:19:05.509 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:05.509 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.509 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.509 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:05.509 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.509 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.510 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.510 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "SMTP server", 2024-03-18T23:19:05.510 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.510 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.510 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.510 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.510 INFO:teuthology.orchestra.run.smithi049.stdout: "smtp_password": { 2024-03-18T23:19:05.510 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "smtp_password", 2024-03-18T23:19:05.510 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:05.511 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.511 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.511 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:05.511 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.511 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.511 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.511 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Password to authenticate with", 2024-03-18T23:19:05.511 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.511 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.512 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.512 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.512 INFO:teuthology.orchestra.run.smithi049.stdout: "smtp_port": { 2024-03-18T23:19:05.512 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "smtp_port", 2024-03-18T23:19:05.512 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-18T23:19:05.512 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.512 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.512 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "465", 2024-03-18T23:19:05.512 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.513 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.513 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.513 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "SMTP port", 2024-03-18T23:19:05.513 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.513 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.513 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.513 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.513 INFO:teuthology.orchestra.run.smithi049.stdout: "smtp_sender": { 2024-03-18T23:19:05.514 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "smtp_sender", 2024-03-18T23:19:05.514 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:05.514 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.514 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.514 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:05.514 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.514 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.514 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.514 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "SMTP envelope sender", 2024-03-18T23:19:05.515 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.515 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.515 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.515 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.515 INFO:teuthology.orchestra.run.smithi049.stdout: "smtp_ssl": { 2024-03-18T23:19:05.515 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "smtp_ssl", 2024-03-18T23:19:05.515 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:05.515 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.516 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.516 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-18T23:19:05.516 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.516 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.516 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.516 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Use SSL to connect to SMTP server", 2024-03-18T23:19:05.516 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.516 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.517 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.517 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.517 INFO:teuthology.orchestra.run.smithi049.stdout: "smtp_user": { 2024-03-18T23:19:05.517 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "smtp_user", 2024-03-18T23:19:05.517 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:05.517 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.517 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.517 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:05.518 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.518 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.518 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.518 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "User to authenticate as", 2024-03-18T23:19:05.518 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.518 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.518 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.518 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:19:05.518 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:19:05.519 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.519 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-18T23:19:05.519 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "balancer", 2024-03-18T23:19:05.519 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-18T23:19:05.519 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-18T23:19:05.519 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-18T23:19:05.519 INFO:teuthology.orchestra.run.smithi049.stdout: "active": { 2024-03-18T23:19:05.519 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "active", 2024-03-18T23:19:05.519 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:05.520 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.520 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.520 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-18T23:19:05.520 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.520 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.520 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.520 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "automatically balance PGs across cluster", 2024-03-18T23:19:05.520 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.521 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.521 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.521 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.521 INFO:teuthology.orchestra.run.smithi049.stdout: "begin_time": { 2024-03-18T23:19:05.521 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "begin_time", 2024-03-18T23:19:05.521 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:05.521 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.521 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.521 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "0000", 2024-03-18T23:19:05.522 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.522 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.522 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.522 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "beginning time of day to automatically balance", 2024-03-18T23:19:05.522 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-18T23:19:05.522 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.522 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.522 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.523 INFO:teuthology.orchestra.run.smithi049.stdout: "begin_weekday": { 2024-03-18T23:19:05.523 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "begin_weekday", 2024-03-18T23:19:05.523 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "uint", 2024-03-18T23:19:05.523 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.523 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.523 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "0", 2024-03-18T23:19:05.523 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "0", 2024-03-18T23:19:05.523 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "6", 2024-03-18T23:19:05.523 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.524 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-03-18T23:19:05.524 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-18T23:19:05.524 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.524 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.524 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.524 INFO:teuthology.orchestra.run.smithi049.stdout: "crush_compat_max_iterations": { 2024-03-18T23:19:05.524 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "crush_compat_max_iterations", 2024-03-18T23:19:05.524 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "uint", 2024-03-18T23:19:05.525 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.525 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.525 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "25", 2024-03-18T23:19:05.525 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "1", 2024-03-18T23:19:05.525 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "250", 2024-03-18T23:19:05.525 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.525 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-03-18T23:19:05.525 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.525 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.526 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.526 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.526 INFO:teuthology.orchestra.run.smithi049.stdout: "crush_compat_metrics": { 2024-03-18T23:19:05.526 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "crush_compat_metrics", 2024-03-18T23:19:05.526 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:05.526 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.526 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.526 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "pgs,objects,bytes", 2024-03-18T23:19:05.526 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.527 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.527 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.527 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-03-18T23:19:05.527 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-03-18T23:19:05.527 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.527 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.527 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.527 INFO:teuthology.orchestra.run.smithi049.stdout: "crush_compat_step": { 2024-03-18T23:19:05.528 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "crush_compat_step", 2024-03-18T23:19:05.528 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "float", 2024-03-18T23:19:05.528 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.528 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.528 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "0.5", 2024-03-18T23:19:05.528 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "0.001", 2024-03-18T23:19:05.528 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "0.999", 2024-03-18T23:19:05.528 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.528 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "aggressiveness of optimization", 2024-03-18T23:19:05.529 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-03-18T23:19:05.529 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.529 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.529 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.529 INFO:teuthology.orchestra.run.smithi049.stdout: "end_time": { 2024-03-18T23:19:05.529 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "end_time", 2024-03-18T23:19:05.529 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:05.529 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.529 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.530 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "2359", 2024-03-18T23:19:05.530 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.530 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.530 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.530 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "ending time of day to automatically balance", 2024-03-18T23:19:05.530 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-18T23:19:05.530 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.530 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.531 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.531 INFO:teuthology.orchestra.run.smithi049.stdout: "end_weekday": { 2024-03-18T23:19:05.531 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "end_weekday", 2024-03-18T23:19:05.531 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "uint", 2024-03-18T23:19:05.531 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.531 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.531 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "0", 2024-03-18T23:19:05.531 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "0", 2024-03-18T23:19:05.531 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "6", 2024-03-18T23:19:05.532 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.532 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-03-18T23:19:05.532 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-18T23:19:05.532 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.532 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.532 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.532 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-18T23:19:05.532 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-18T23:19:05.533 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:05.533 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.533 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.533 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:05.533 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.533 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.533 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-18T23:19:05.533 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-18T23:19:05.534 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-18T23:19:05.534 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-18T23:19:05.534 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-18T23:19:05.534 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-18T23:19:05.534 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-18T23:19:05.534 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-18T23:19:05.534 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.534 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.535 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.535 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.535 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.535 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-18T23:19:05.535 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-18T23:19:05.535 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:05.535 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.535 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.535 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:05.536 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.536 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.536 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.536 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.536 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.536 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.536 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.536 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.536 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-18T23:19:05.537 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-18T23:19:05.537 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:05.537 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.537 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.537 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-18T23:19:05.537 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.537 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.537 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-18T23:19:05.537 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-18T23:19:05.538 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-18T23:19:05.538 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-18T23:19:05.538 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-18T23:19:05.538 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-18T23:19:05.538 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-18T23:19:05.538 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-18T23:19:05.538 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.538 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.539 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.539 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.539 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.539 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-18T23:19:05.539 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-18T23:19:05.539 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:05.539 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.539 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.539 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:05.540 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.540 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.540 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.540 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.540 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.540 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.540 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.540 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.540 INFO:teuthology.orchestra.run.smithi049.stdout: "min_score": { 2024-03-18T23:19:05.541 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "min_score", 2024-03-18T23:19:05.541 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "float", 2024-03-18T23:19:05.541 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.541 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.541 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "0", 2024-03-18T23:19:05.541 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.541 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.541 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.541 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-03-18T23:19:05.542 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.542 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.542 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.542 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.542 INFO:teuthology.orchestra.run.smithi049.stdout: "mode": { 2024-03-18T23:19:05.542 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "mode", 2024-03-18T23:19:05.542 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:05.542 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.542 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.543 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "upmap", 2024-03-18T23:19:05.543 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.543 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.543 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-18T23:19:05.543 INFO:teuthology.orchestra.run.smithi049.stdout: "crush-compat", 2024-03-18T23:19:05.543 INFO:teuthology.orchestra.run.smithi049.stdout: "none", 2024-03-18T23:19:05.543 INFO:teuthology.orchestra.run.smithi049.stdout: "read", 2024-03-18T23:19:05.543 INFO:teuthology.orchestra.run.smithi049.stdout: "upmap", 2024-03-18T23:19:05.543 INFO:teuthology.orchestra.run.smithi049.stdout: "upmap-read" 2024-03-18T23:19:05.544 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-18T23:19:05.544 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Balancer mode", 2024-03-18T23:19:05.544 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.544 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.544 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.544 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.544 INFO:teuthology.orchestra.run.smithi049.stdout: "pool_ids": { 2024-03-18T23:19:05.544 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "pool_ids", 2024-03-18T23:19:05.544 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:05.545 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.545 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.545 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:05.545 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.545 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.545 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.545 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-03-18T23:19:05.545 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.545 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.546 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.546 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.546 INFO:teuthology.orchestra.run.smithi049.stdout: "sleep_interval": { 2024-03-18T23:19:05.546 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "sleep_interval", 2024-03-18T23:19:05.546 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-03-18T23:19:05.546 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.546 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.546 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "60", 2024-03-18T23:19:05.547 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.547 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.547 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.547 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-03-18T23:19:05.547 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.547 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.547 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.547 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.547 INFO:teuthology.orchestra.run.smithi049.stdout: "upmap_max_deviation": { 2024-03-18T23:19:05.548 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "upmap_max_deviation", 2024-03-18T23:19:05.548 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-18T23:19:05.548 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.548 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.548 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "5", 2024-03-18T23:19:05.548 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "1", 2024-03-18T23:19:05.548 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.548 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.548 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "deviation below which no optimization is attempted", 2024-03-18T23:19:05.549 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-03-18T23:19:05.549 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.549 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.549 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.549 INFO:teuthology.orchestra.run.smithi049.stdout: "upmap_max_optimizations": { 2024-03-18T23:19:05.549 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "upmap_max_optimizations", 2024-03-18T23:19:05.549 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "uint", 2024-03-18T23:19:05.549 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.549 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.550 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "10", 2024-03-18T23:19:05.550 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.550 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.550 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.550 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-03-18T23:19:05.550 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.550 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.550 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.550 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:19:05.551 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:19:05.551 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.551 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-18T23:19:05.551 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "cephadm", 2024-03-18T23:19:05.551 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-18T23:19:05.551 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-18T23:19:05.551 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-18T23:19:05.551 INFO:teuthology.orchestra.run.smithi049.stdout: "agent_down_multiplier": { 2024-03-18T23:19:05.551 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "agent_down_multiplier", 2024-03-18T23:19:05.552 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "float", 2024-03-18T23:19:05.552 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.552 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:05.552 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "3.0", 2024-03-18T23:19:05.552 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.552 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.552 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.552 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-03-18T23:19:05.552 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.553 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.553 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.553 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.553 INFO:teuthology.orchestra.run.smithi049.stdout: "agent_refresh_rate": { 2024-03-18T23:19:05.553 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "agent_refresh_rate", 2024-03-18T23:19:05.553 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-03-18T23:19:05.553 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.553 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:05.553 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "20", 2024-03-18T23:19:05.554 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.554 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.554 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.554 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-03-18T23:19:05.554 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.554 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.554 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.554 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.555 INFO:teuthology.orchestra.run.smithi049.stdout: "agent_starting_port": { 2024-03-18T23:19:05.555 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "agent_starting_port", 2024-03-18T23:19:05.555 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-18T23:19:05.555 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.555 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:05.555 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "4721", 2024-03-18T23:19:05.555 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.555 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.555 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.555 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-03-18T23:19:05.556 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.556 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.556 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.556 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.556 INFO:teuthology.orchestra.run.smithi049.stdout: "allow_ptrace": { 2024-03-18T23:19:05.556 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "allow_ptrace", 2024-03-18T23:19:05.556 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:05.556 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.557 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:05.557 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:05.557 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.557 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.557 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.557 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-03-18T23:19:05.557 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-03-18T23:19:05.557 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.558 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.558 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.558 INFO:teuthology.orchestra.run.smithi049.stdout: "autotune_interval": { 2024-03-18T23:19:05.558 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "autotune_interval", 2024-03-18T23:19:05.558 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-03-18T23:19:05.558 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.558 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:05.558 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "600", 2024-03-18T23:19:05.558 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.559 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.559 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.559 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "how frequently to autotune daemon memory", 2024-03-18T23:19:05.559 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.559 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.559 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.559 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.559 INFO:teuthology.orchestra.run.smithi049.stdout: "autotune_memory_target_ratio": { 2024-03-18T23:19:05.559 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "autotune_memory_target_ratio", 2024-03-18T23:19:05.560 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "float", 2024-03-18T23:19:05.560 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.560 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:05.560 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "0.7", 2024-03-18T23:19:05.560 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.560 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.560 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.560 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-03-18T23:19:05.560 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.561 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.561 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.561 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.561 INFO:teuthology.orchestra.run.smithi049.stdout: "cephadm_log_destination": { 2024-03-18T23:19:05.561 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "cephadm_log_destination", 2024-03-18T23:19:05.561 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:05.561 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.561 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:05.562 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:05.562 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.562 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.562 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-18T23:19:05.562 INFO:teuthology.orchestra.run.smithi049.stdout: "file", 2024-03-18T23:19:05.562 INFO:teuthology.orchestra.run.smithi049.stdout: "file,syslog", 2024-03-18T23:19:05.562 INFO:teuthology.orchestra.run.smithi049.stdout: "syslog" 2024-03-18T23:19:05.562 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-18T23:19:05.563 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-03-18T23:19:05.563 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.563 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.563 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.563 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.563 INFO:teuthology.orchestra.run.smithi049.stdout: "cgroups_split": { 2024-03-18T23:19:05.563 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "cgroups_split", 2024-03-18T23:19:05.563 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:05.563 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.564 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:05.564 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-18T23:19:05.564 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.564 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.564 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.564 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-03-18T23:19:05.564 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.564 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.565 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.565 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.565 INFO:teuthology.orchestra.run.smithi049.stdout: "config_checks_enabled": { 2024-03-18T23:19:05.565 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "config_checks_enabled", 2024-03-18T23:19:05.565 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:05.565 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.565 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:05.565 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:05.565 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.566 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.566 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.566 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-03-18T23:19:05.566 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.566 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.566 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.566 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.566 INFO:teuthology.orchestra.run.smithi049.stdout: "config_dashboard": { 2024-03-18T23:19:05.567 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "config_dashboard", 2024-03-18T23:19:05.567 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:05.567 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.567 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:05.567 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-18T23:19:05.567 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.567 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.567 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.567 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-03-18T23:19:05.568 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.568 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.568 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.568 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.568 INFO:teuthology.orchestra.run.smithi049.stdout: "container_image_alertmanager": { 2024-03-18T23:19:05.568 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_image_alertmanager", 2024-03-18T23:19:05.568 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:05.568 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.569 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:05.569 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-03-18T23:19:05.569 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.569 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.569 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.569 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Prometheus container image", 2024-03-18T23:19:05.569 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.569 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.569 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.570 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.570 INFO:teuthology.orchestra.run.smithi049.stdout: "container_image_base": { 2024-03-18T23:19:05.570 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_image_base", 2024-03-18T23:19:05.570 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:05.570 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.570 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.570 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "quay.io/ceph/ceph", 2024-03-18T23:19:05.570 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.570 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.571 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.571 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Container image name, without the tag", 2024-03-18T23:19:05.571 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.571 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.571 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.571 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.571 INFO:teuthology.orchestra.run.smithi049.stdout: "container_image_elasticsearch": { 2024-03-18T23:19:05.571 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_image_elasticsearch", 2024-03-18T23:19:05.572 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:05.572 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.572 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:05.572 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-03-18T23:19:05.572 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.572 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.572 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.572 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "elasticsearch container image", 2024-03-18T23:19:05.573 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.573 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.573 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.573 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.573 INFO:teuthology.orchestra.run.smithi049.stdout: "container_image_grafana": { 2024-03-18T23:19:05.573 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_image_grafana", 2024-03-18T23:19:05.573 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:05.573 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.573 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:05.574 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.12", 2024-03-18T23:19:05.574 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.574 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.574 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.574 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Prometheus container image", 2024-03-18T23:19:05.574 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.574 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.574 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.575 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.575 INFO:teuthology.orchestra.run.smithi049.stdout: "container_image_haproxy": { 2024-03-18T23:19:05.575 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_image_haproxy", 2024-03-18T23:19:05.575 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:05.575 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.575 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:05.575 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-03-18T23:19:05.575 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.575 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.576 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.576 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "HAproxy container image", 2024-03-18T23:19:05.576 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.576 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.576 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.576 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.576 INFO:teuthology.orchestra.run.smithi049.stdout: "container_image_jaeger_agent": { 2024-03-18T23:19:05.576 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_image_jaeger_agent", 2024-03-18T23:19:05.576 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:05.577 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.577 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:05.577 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-03-18T23:19:05.577 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.577 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.577 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.577 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Jaeger agent container image", 2024-03-18T23:19:05.577 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.577 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.578 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.578 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.578 INFO:teuthology.orchestra.run.smithi049.stdout: "container_image_jaeger_collector": { 2024-03-18T23:19:05.578 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_image_jaeger_collector", 2024-03-18T23:19:05.578 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:05.578 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.578 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:05.578 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-03-18T23:19:05.579 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.579 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.579 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.579 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Jaeger collector container image", 2024-03-18T23:19:05.579 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.579 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.579 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.579 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.579 INFO:teuthology.orchestra.run.smithi049.stdout: "container_image_jaeger_query": { 2024-03-18T23:19:05.580 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_image_jaeger_query", 2024-03-18T23:19:05.580 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:05.580 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.580 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:05.580 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-03-18T23:19:05.580 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.580 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.580 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.580 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Jaeger query container image", 2024-03-18T23:19:05.581 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.581 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.581 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.581 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.581 INFO:teuthology.orchestra.run.smithi049.stdout: "container_image_keepalived": { 2024-03-18T23:19:05.581 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_image_keepalived", 2024-03-18T23:19:05.581 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:05.581 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.582 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:05.582 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-03-18T23:19:05.582 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.582 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.582 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.582 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Keepalived container image", 2024-03-18T23:19:05.582 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.582 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.582 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.583 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.583 INFO:teuthology.orchestra.run.smithi049.stdout: "container_image_loki": { 2024-03-18T23:19:05.583 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_image_loki", 2024-03-18T23:19:05.583 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:05.583 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.583 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:05.583 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-03-18T23:19:05.583 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.583 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.584 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.584 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Loki container image", 2024-03-18T23:19:05.584 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.584 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.584 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.584 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.584 INFO:teuthology.orchestra.run.smithi049.stdout: "container_image_node_exporter": { 2024-03-18T23:19:05.584 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_image_node_exporter", 2024-03-18T23:19:05.585 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:05.585 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.585 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:05.585 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-03-18T23:19:05.585 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.585 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.585 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.585 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Prometheus container image", 2024-03-18T23:19:05.586 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.586 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.586 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.586 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.586 INFO:teuthology.orchestra.run.smithi049.stdout: "container_image_nvmeof": { 2024-03-18T23:19:05.586 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_image_nvmeof", 2024-03-18T23:19:05.586 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:05.586 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.586 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:05.587 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "quay.io/ceph/nvmeof:1.0.0", 2024-03-18T23:19:05.587 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.587 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.587 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.587 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Nvme-of container image", 2024-03-18T23:19:05.587 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.587 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.587 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.587 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.588 INFO:teuthology.orchestra.run.smithi049.stdout: "container_image_prometheus": { 2024-03-18T23:19:05.588 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_image_prometheus", 2024-03-18T23:19:05.588 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:05.588 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.588 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:05.588 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-03-18T23:19:05.588 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.588 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.589 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.589 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Prometheus container image", 2024-03-18T23:19:05.589 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.589 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.589 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.589 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.589 INFO:teuthology.orchestra.run.smithi049.stdout: "container_image_promtail": { 2024-03-18T23:19:05.589 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_image_promtail", 2024-03-18T23:19:05.589 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:05.590 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.590 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:05.590 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-03-18T23:19:05.590 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.590 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.590 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.590 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Promtail container image", 2024-03-18T23:19:05.590 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.590 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.591 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.591 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.591 INFO:teuthology.orchestra.run.smithi049.stdout: "container_image_snmp_gateway": { 2024-03-18T23:19:05.591 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_image_snmp_gateway", 2024-03-18T23:19:05.591 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:05.591 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.591 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:05.591 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-03-18T23:19:05.592 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.592 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.592 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.592 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "SNMP Gateway container image", 2024-03-18T23:19:05.592 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.592 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.592 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.592 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.592 INFO:teuthology.orchestra.run.smithi049.stdout: "container_init": { 2024-03-18T23:19:05.593 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_init", 2024-03-18T23:19:05.593 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:05.593 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.593 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:05.593 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-18T23:19:05.593 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.593 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.593 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.593 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Run podman/docker with `--init`", 2024-03-18T23:19:05.594 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.594 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.594 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.594 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.594 INFO:teuthology.orchestra.run.smithi049.stdout: "daemon_cache_timeout": { 2024-03-18T23:19:05.594 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "daemon_cache_timeout", 2024-03-18T23:19:05.594 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-03-18T23:19:05.594 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.595 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:05.595 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "600", 2024-03-18T23:19:05.595 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.595 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.595 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.595 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-03-18T23:19:05.595 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.595 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.596 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.596 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.596 INFO:teuthology.orchestra.run.smithi049.stdout: "default_cephadm_command_timeout": { 2024-03-18T23:19:05.596 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "default_cephadm_command_timeout", 2024-03-18T23:19:05.596 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-18T23:19:05.596 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.596 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:05.596 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "900", 2024-03-18T23:19:05.596 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.597 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.597 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.597 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-03-18T23:19:05.597 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.597 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.597 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.597 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.597 INFO:teuthology.orchestra.run.smithi049.stdout: "default_registry": { 2024-03-18T23:19:05.597 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "default_registry", 2024-03-18T23:19:05.598 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:05.598 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.598 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:05.598 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "docker.io", 2024-03-18T23:19:05.598 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.598 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.598 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.598 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-03-18T23:19:05.599 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.599 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.599 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.599 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.599 INFO:teuthology.orchestra.run.smithi049.stdout: "device_cache_timeout": { 2024-03-18T23:19:05.599 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "device_cache_timeout", 2024-03-18T23:19:05.599 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-03-18T23:19:05.599 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.599 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:05.600 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "1800", 2024-03-18T23:19:05.600 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.600 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.600 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.600 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "seconds to cache device inventory", 2024-03-18T23:19:05.600 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.600 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.600 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.601 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.601 INFO:teuthology.orchestra.run.smithi049.stdout: "device_enhanced_scan": { 2024-03-18T23:19:05.601 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "device_enhanced_scan", 2024-03-18T23:19:05.601 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:05.601 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.601 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:05.601 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:05.601 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.602 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.602 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.602 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Use libstoragemgmt during device scans", 2024-03-18T23:19:05.602 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.602 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.602 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.602 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.602 INFO:teuthology.orchestra.run.smithi049.stdout: "facts_cache_timeout": { 2024-03-18T23:19:05.603 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "facts_cache_timeout", 2024-03-18T23:19:05.603 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-03-18T23:19:05.603 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.603 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:05.603 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "60", 2024-03-18T23:19:05.603 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.603 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.603 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.604 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "seconds to cache host facts data", 2024-03-18T23:19:05.604 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.604 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.604 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.604 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.604 INFO:teuthology.orchestra.run.smithi049.stdout: "host_check_interval": { 2024-03-18T23:19:05.604 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "host_check_interval", 2024-03-18T23:19:05.604 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-03-18T23:19:05.604 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.605 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:05.605 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "600", 2024-03-18T23:19:05.605 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.605 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.605 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.605 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "how frequently to perform a host check", 2024-03-18T23:19:05.605 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.605 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.605 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.606 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.606 INFO:teuthology.orchestra.run.smithi049.stdout: "hw_monitoring": { 2024-03-18T23:19:05.606 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "hw_monitoring", 2024-03-18T23:19:05.606 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:05.606 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.606 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:05.606 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:05.606 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.607 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.607 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.607 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-03-18T23:19:05.607 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.607 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.607 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.607 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.607 INFO:teuthology.orchestra.run.smithi049.stdout: "inventory_list_all": { 2024-03-18T23:19:05.608 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "inventory_list_all", 2024-03-18T23:19:05.608 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:05.608 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.608 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:05.608 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:05.608 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.608 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.608 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.609 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-03-18T23:19:05.609 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.609 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.609 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.609 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.609 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-18T23:19:05.609 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-18T23:19:05.609 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:05.609 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.610 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.610 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:05.610 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.610 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.610 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-18T23:19:05.610 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-18T23:19:05.610 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-18T23:19:05.610 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-18T23:19:05.611 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-18T23:19:05.611 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-18T23:19:05.611 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-18T23:19:05.611 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-18T23:19:05.611 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.611 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.611 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.611 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.612 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.612 INFO:teuthology.orchestra.run.smithi049.stdout: "log_refresh_metadata": { 2024-03-18T23:19:05.612 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_refresh_metadata", 2024-03-18T23:19:05.612 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:05.612 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.612 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:05.612 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:05.612 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.612 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.613 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.613 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-03-18T23:19:05.613 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.613 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.613 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.613 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.613 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-18T23:19:05.613 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-18T23:19:05.613 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:05.614 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.614 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:05.614 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-18T23:19:05.614 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.614 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.614 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.614 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-03-18T23:19:05.614 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.615 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.615 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.615 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.615 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-18T23:19:05.615 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-18T23:19:05.615 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:05.615 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.615 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.615 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-18T23:19:05.616 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.616 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.616 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-18T23:19:05.616 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-18T23:19:05.616 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-18T23:19:05.616 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-18T23:19:05.616 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-18T23:19:05.616 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-18T23:19:05.616 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-18T23:19:05.617 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-18T23:19:05.617 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.617 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.617 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.617 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.617 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.617 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-18T23:19:05.617 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-18T23:19:05.618 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:05.618 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.618 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.618 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:05.618 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.618 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.618 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.618 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.619 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.619 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.619 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.619 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.619 INFO:teuthology.orchestra.run.smithi049.stdout: "manage_etc_ceph_ceph_conf": { 2024-03-18T23:19:05.619 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-03-18T23:19:05.619 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:05.619 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.619 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:05.620 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:05.620 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.620 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.620 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.620 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-03-18T23:19:05.620 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.620 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.620 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.620 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.621 INFO:teuthology.orchestra.run.smithi049.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-03-18T23:19:05.621 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-03-18T23:19:05.621 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:05.621 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.621 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:05.621 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "*", 2024-03-18T23:19:05.621 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.621 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.622 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.622 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-03-18T23:19:05.622 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.622 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.622 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.622 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.622 INFO:teuthology.orchestra.run.smithi049.stdout: "max_count_per_host": { 2024-03-18T23:19:05.622 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "max_count_per_host", 2024-03-18T23:19:05.623 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-18T23:19:05.623 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.623 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:05.623 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "10", 2024-03-18T23:19:05.623 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.623 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.623 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.623 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "max number of daemons per service per host", 2024-03-18T23:19:05.623 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.624 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.624 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.624 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.624 INFO:teuthology.orchestra.run.smithi049.stdout: "max_osd_draining_count": { 2024-03-18T23:19:05.624 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "max_osd_draining_count", 2024-03-18T23:19:05.624 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-18T23:19:05.624 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.624 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:05.624 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "10", 2024-03-18T23:19:05.625 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.625 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.625 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.625 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-03-18T23:19:05.625 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.625 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.625 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.625 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.625 INFO:teuthology.orchestra.run.smithi049.stdout: "migration_current": { 2024-03-18T23:19:05.626 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "migration_current", 2024-03-18T23:19:05.626 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-18T23:19:05.626 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.626 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:05.626 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:05.626 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.626 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.626 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.627 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "internal - do not modify", 2024-03-18T23:19:05.627 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.627 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.627 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.627 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.627 INFO:teuthology.orchestra.run.smithi049.stdout: "mode": { 2024-03-18T23:19:05.627 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "mode", 2024-03-18T23:19:05.627 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:05.628 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.628 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:05.628 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "root", 2024-03-18T23:19:05.628 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.628 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.628 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-18T23:19:05.628 INFO:teuthology.orchestra.run.smithi049.stdout: "cephadm-package", 2024-03-18T23:19:05.628 INFO:teuthology.orchestra.run.smithi049.stdout: "root" 2024-03-18T23:19:05.628 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-18T23:19:05.629 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "mode for remote execution of cephadm", 2024-03-18T23:19:05.629 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.629 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.629 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.629 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.629 INFO:teuthology.orchestra.run.smithi049.stdout: "oob_default_addr": { 2024-03-18T23:19:05.629 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "oob_default_addr", 2024-03-18T23:19:05.629 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:05.629 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.630 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:05.630 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "169.254.1.1", 2024-03-18T23:19:05.630 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.630 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.630 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.630 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Default address for RedFish API (oob management).", 2024-03-18T23:19:05.630 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.630 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.631 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.631 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.631 INFO:teuthology.orchestra.run.smithi049.stdout: "prometheus_alerts_path": { 2024-03-18T23:19:05.631 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "prometheus_alerts_path", 2024-03-18T23:19:05.631 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:05.631 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.631 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:05.631 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-03-18T23:19:05.632 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.632 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.632 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.632 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-03-18T23:19:05.632 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.632 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.632 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.632 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.632 INFO:teuthology.orchestra.run.smithi049.stdout: "registry_insecure": { 2024-03-18T23:19:05.633 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "registry_insecure", 2024-03-18T23:19:05.633 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:05.633 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.633 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:05.633 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:05.633 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.633 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.633 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.633 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-03-18T23:19:05.634 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.634 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.634 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.634 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.634 INFO:teuthology.orchestra.run.smithi049.stdout: "registry_password": { 2024-03-18T23:19:05.634 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "registry_password", 2024-03-18T23:19:05.634 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:05.634 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.634 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:05.635 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:05.635 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.635 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.635 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.635 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-03-18T23:19:05.635 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.635 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.635 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.635 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.636 INFO:teuthology.orchestra.run.smithi049.stdout: "registry_url": { 2024-03-18T23:19:05.636 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "registry_url", 2024-03-18T23:19:05.636 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:05.636 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.636 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:05.636 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:05.636 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.636 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.636 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.637 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-03-18T23:19:05.637 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.637 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.637 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.637 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.637 INFO:teuthology.orchestra.run.smithi049.stdout: "registry_username": { 2024-03-18T23:19:05.637 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "registry_username", 2024-03-18T23:19:05.637 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:05.637 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.638 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:05.638 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:05.638 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.638 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.638 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.638 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-03-18T23:19:05.638 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.638 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.638 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.639 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.639 INFO:teuthology.orchestra.run.smithi049.stdout: "secure_monitoring_stack": { 2024-03-18T23:19:05.639 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "secure_monitoring_stack", 2024-03-18T23:19:05.639 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:05.639 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.639 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:05.639 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:05.639 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.641 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.641 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.641 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-03-18T23:19:05.641 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.641 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.641 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.641 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.642 INFO:teuthology.orchestra.run.smithi049.stdout: "service_discovery_port": { 2024-03-18T23:19:05.642 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "service_discovery_port", 2024-03-18T23:19:05.642 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-18T23:19:05.642 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.642 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:05.642 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "8765", 2024-03-18T23:19:05.642 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.642 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.643 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.643 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "cephadm service discovery port", 2024-03-18T23:19:05.643 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.643 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.643 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.643 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.643 INFO:teuthology.orchestra.run.smithi049.stdout: "ssh_config_file": { 2024-03-18T23:19:05.643 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "ssh_config_file", 2024-03-18T23:19:05.643 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:05.644 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.644 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:05.644 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:05.644 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.644 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.644 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.644 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-03-18T23:19:05.644 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.645 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.645 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.645 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.645 INFO:teuthology.orchestra.run.smithi049.stdout: "use_agent": { 2024-03-18T23:19:05.645 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "use_agent", 2024-03-18T23:19:05.645 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:05.645 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.645 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:05.646 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:05.646 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.646 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.646 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.646 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-03-18T23:19:05.646 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.646 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.646 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.647 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.647 INFO:teuthology.orchestra.run.smithi049.stdout: "use_repo_digest": { 2024-03-18T23:19:05.647 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "use_repo_digest", 2024-03-18T23:19:05.647 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:05.647 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.647 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:05.647 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-18T23:19:05.647 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.647 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.648 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.648 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-03-18T23:19:05.648 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.648 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.648 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.648 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.648 INFO:teuthology.orchestra.run.smithi049.stdout: "warn_on_failed_host_check": { 2024-03-18T23:19:05.648 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "warn_on_failed_host_check", 2024-03-18T23:19:05.649 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:05.649 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.649 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:05.649 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-18T23:19:05.649 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.649 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.649 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.649 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "raise a health warning if the host check fails", 2024-03-18T23:19:05.649 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.650 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.650 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.650 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.650 INFO:teuthology.orchestra.run.smithi049.stdout: "warn_on_stray_daemons": { 2024-03-18T23:19:05.650 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "warn_on_stray_daemons", 2024-03-18T23:19:05.650 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:05.650 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.650 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:05.650 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-18T23:19:05.651 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.651 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.651 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.651 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-03-18T23:19:05.651 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.651 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.651 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.651 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.652 INFO:teuthology.orchestra.run.smithi049.stdout: "warn_on_stray_hosts": { 2024-03-18T23:19:05.652 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "warn_on_stray_hosts", 2024-03-18T23:19:05.652 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:05.652 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.652 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:05.652 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-18T23:19:05.652 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.652 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.652 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.653 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-03-18T23:19:05.653 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.653 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.653 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.653 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:19:05.653 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:19:05.653 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.653 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-18T23:19:05.653 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "crash", 2024-03-18T23:19:05.654 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-18T23:19:05.654 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-18T23:19:05.654 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-18T23:19:05.654 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-18T23:19:05.654 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-18T23:19:05.654 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:05.654 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.654 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.655 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:05.655 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.655 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.655 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-18T23:19:05.655 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-18T23:19:05.655 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-18T23:19:05.655 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-18T23:19:05.655 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-18T23:19:05.655 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-18T23:19:05.656 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-18T23:19:05.656 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-18T23:19:05.656 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.656 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.656 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.656 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.656 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.656 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-18T23:19:05.657 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-18T23:19:05.657 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:05.657 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.657 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.657 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:05.657 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.657 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.657 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.657 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.658 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.658 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.658 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.658 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.658 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-18T23:19:05.658 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-18T23:19:05.658 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:05.658 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.659 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.659 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-18T23:19:05.659 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.659 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.659 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-18T23:19:05.659 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-18T23:19:05.659 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-18T23:19:05.659 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-18T23:19:05.659 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-18T23:19:05.660 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-18T23:19:05.660 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-18T23:19:05.660 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-18T23:19:05.660 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.660 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.660 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.660 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.660 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.661 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-18T23:19:05.661 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-18T23:19:05.661 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:05.661 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.661 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.661 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:05.661 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.661 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.661 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.662 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.662 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.662 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.662 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.662 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.662 INFO:teuthology.orchestra.run.smithi049.stdout: "retain_interval": { 2024-03-18T23:19:05.662 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "retain_interval", 2024-03-18T23:19:05.662 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-03-18T23:19:05.663 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.663 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.663 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "31536000", 2024-03-18T23:19:05.663 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.663 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.663 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.663 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "how long to retain crashes before pruning them", 2024-03-18T23:19:05.663 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.663 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.664 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.664 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.664 INFO:teuthology.orchestra.run.smithi049.stdout: "warn_recent_interval": { 2024-03-18T23:19:05.664 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "warn_recent_interval", 2024-03-18T23:19:05.664 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-03-18T23:19:05.664 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.664 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.664 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "1209600", 2024-03-18T23:19:05.665 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.665 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.665 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.665 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "time interval in which to warn about recent crashes", 2024-03-18T23:19:05.665 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.665 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.665 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.665 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:19:05.665 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:19:05.666 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.666 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-18T23:19:05.666 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "dashboard", 2024-03-18T23:19:05.666 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-18T23:19:05.666 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-18T23:19:05.666 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-18T23:19:05.666 INFO:teuthology.orchestra.run.smithi049.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-03-18T23:19:05.666 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-03-18T23:19:05.667 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-18T23:19:05.667 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.667 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:05.667 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "10", 2024-03-18T23:19:05.667 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.667 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.667 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.667 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.668 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.668 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.668 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.668 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.668 INFO:teuthology.orchestra.run.smithi049.stdout: "ALERTMANAGER_API_HOST": { 2024-03-18T23:19:05.668 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "ALERTMANAGER_API_HOST", 2024-03-18T23:19:05.668 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:05.668 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.668 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:05.668 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:05.669 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.669 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.669 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.669 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.669 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.669 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.669 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.669 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.670 INFO:teuthology.orchestra.run.smithi049.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-03-18T23:19:05.670 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-03-18T23:19:05.670 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:05.670 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.670 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:05.670 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-18T23:19:05.670 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.670 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.671 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.671 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.671 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.671 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.671 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.671 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.671 INFO:teuthology.orchestra.run.smithi049.stdout: "AUDIT_API_ENABLED": { 2024-03-18T23:19:05.671 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "AUDIT_API_ENABLED", 2024-03-18T23:19:05.671 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:05.671 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.672 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:05.672 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:05.672 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.672 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.672 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.672 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.672 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.672 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.673 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.673 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.673 INFO:teuthology.orchestra.run.smithi049.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-03-18T23:19:05.673 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-03-18T23:19:05.673 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:05.673 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.673 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:05.673 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-18T23:19:05.673 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.674 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.674 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.674 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.674 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.674 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.674 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.674 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.674 INFO:teuthology.orchestra.run.smithi049.stdout: "ENABLE_BROWSABLE_API": { 2024-03-18T23:19:05.675 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "ENABLE_BROWSABLE_API", 2024-03-18T23:19:05.675 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:05.675 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.675 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:05.675 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-18T23:19:05.675 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.675 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.675 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.676 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.676 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.676 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.676 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.676 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.676 INFO:teuthology.orchestra.run.smithi049.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-03-18T23:19:05.676 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-03-18T23:19:05.676 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:05.676 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.677 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:05.677 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-18T23:19:05.677 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.677 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.677 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.677 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.677 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.677 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.677 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.678 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.678 INFO:teuthology.orchestra.run.smithi049.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-03-18T23:19:05.678 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-03-18T23:19:05.678 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:05.678 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.678 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:05.678 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-18T23:19:05.678 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.679 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.679 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.679 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.679 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.679 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.679 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.679 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.679 INFO:teuthology.orchestra.run.smithi049.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-03-18T23:19:05.679 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-03-18T23:19:05.680 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:05.680 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.680 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:05.680 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-18T23:19:05.680 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.680 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.680 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.680 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.681 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.681 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.681 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.681 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.681 INFO:teuthology.orchestra.run.smithi049.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-03-18T23:19:05.681 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-03-18T23:19:05.681 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:05.681 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.681 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:05.682 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-18T23:19:05.682 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.682 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.682 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.682 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.682 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.682 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.682 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.683 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.683 INFO:teuthology.orchestra.run.smithi049.stdout: "FEATURE_TOGGLE_NFS": { 2024-03-18T23:19:05.683 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-03-18T23:19:05.683 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:05.683 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.683 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:05.683 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-18T23:19:05.683 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.683 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.684 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.684 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.684 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.684 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.684 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.684 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.684 INFO:teuthology.orchestra.run.smithi049.stdout: "FEATURE_TOGGLE_RBD": { 2024-03-18T23:19:05.684 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-03-18T23:19:05.685 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:05.685 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.685 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:05.685 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-18T23:19:05.685 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.685 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.685 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.685 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.685 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.686 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.686 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.686 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.686 INFO:teuthology.orchestra.run.smithi049.stdout: "FEATURE_TOGGLE_RGW": { 2024-03-18T23:19:05.686 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-03-18T23:19:05.686 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:05.686 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.686 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:05.687 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-18T23:19:05.687 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.687 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.687 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.687 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.687 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.687 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.687 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.687 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.688 INFO:teuthology.orchestra.run.smithi049.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-03-18T23:19:05.688 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-03-18T23:19:05.688 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:05.688 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.688 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:05.688 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:05.688 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.688 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.688 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.689 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.689 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.689 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.689 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.689 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.689 INFO:teuthology.orchestra.run.smithi049.stdout: "GRAFANA_API_PASSWORD": { 2024-03-18T23:19:05.689 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "GRAFANA_API_PASSWORD", 2024-03-18T23:19:05.689 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:05.689 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.690 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:05.690 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "admin", 2024-03-18T23:19:05.690 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.690 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.690 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.690 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.690 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.690 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.691 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.691 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.691 INFO:teuthology.orchestra.run.smithi049.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-03-18T23:19:05.691 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-03-18T23:19:05.691 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:05.691 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.691 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:05.691 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-18T23:19:05.691 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.692 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.692 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.692 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.692 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.692 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.692 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.692 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.692 INFO:teuthology.orchestra.run.smithi049.stdout: "GRAFANA_API_URL": { 2024-03-18T23:19:05.692 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "GRAFANA_API_URL", 2024-03-18T23:19:05.693 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:05.693 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.693 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:05.693 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:05.693 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.693 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.693 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.693 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.694 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.694 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.694 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.694 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.694 INFO:teuthology.orchestra.run.smithi049.stdout: "GRAFANA_API_USERNAME": { 2024-03-18T23:19:05.694 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "GRAFANA_API_USERNAME", 2024-03-18T23:19:05.694 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:05.694 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.694 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:05.695 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "admin", 2024-03-18T23:19:05.695 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.695 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.695 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.695 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.695 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.695 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.695 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.695 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.696 INFO:teuthology.orchestra.run.smithi049.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-03-18T23:19:05.696 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-03-18T23:19:05.696 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:05.696 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.696 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:05.696 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:05.696 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.696 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.696 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.697 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.697 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.697 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.697 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.697 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.697 INFO:teuthology.orchestra.run.smithi049.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-03-18T23:19:05.697 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-03-18T23:19:05.697 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:05.698 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.698 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:05.698 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:05.698 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.698 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.698 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.698 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.698 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.699 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.699 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.699 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.699 INFO:teuthology.orchestra.run.smithi049.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-03-18T23:19:05.699 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-03-18T23:19:05.699 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:05.699 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.699 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:05.699 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-18T23:19:05.700 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.700 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.700 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.700 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.700 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.700 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.700 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.700 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.700 INFO:teuthology.orchestra.run.smithi049.stdout: "ISSUE_TRACKER_API_KEY": { 2024-03-18T23:19:05.701 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-03-18T23:19:05.701 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:05.701 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.701 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:05.701 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:05.701 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.701 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.701 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.702 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.702 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.702 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.702 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.702 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.702 INFO:teuthology.orchestra.run.smithi049.stdout: "MULTICLUSTER_CONFIG": { 2024-03-18T23:19:05.702 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "MULTICLUSTER_CONFIG", 2024-03-18T23:19:05.702 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:05.702 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.703 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:05.703 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "{}", 2024-03-18T23:19:05.703 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.703 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.703 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.703 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.703 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.703 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.703 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.704 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.704 INFO:teuthology.orchestra.run.smithi049.stdout: "PROMETHEUS_API_HOST": { 2024-03-18T23:19:05.704 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "PROMETHEUS_API_HOST", 2024-03-18T23:19:05.704 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:05.704 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.704 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:05.704 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:05.704 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.705 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.705 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.705 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.705 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.705 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.705 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.705 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.705 INFO:teuthology.orchestra.run.smithi049.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-03-18T23:19:05.705 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-03-18T23:19:05.706 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:05.706 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.706 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:05.706 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-18T23:19:05.706 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.706 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.706 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.706 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.707 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.707 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.707 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.707 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.707 INFO:teuthology.orchestra.run.smithi049.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-03-18T23:19:05.707 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-03-18T23:19:05.707 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:05.707 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.708 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:05.708 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:05.708 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.708 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.708 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.708 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.708 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.708 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.708 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.709 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.709 INFO:teuthology.orchestra.run.smithi049.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-03-18T23:19:05.709 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-03-18T23:19:05.709 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:05.709 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.709 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:05.709 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:05.709 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.709 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.710 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.710 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.710 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.710 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.710 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.710 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.710 INFO:teuthology.orchestra.run.smithi049.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-03-18T23:19:05.710 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-03-18T23:19:05.711 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:05.711 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.711 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:05.711 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-18T23:19:05.711 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.711 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.711 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.711 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.712 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.712 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.712 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.712 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.712 INFO:teuthology.orchestra.run.smithi049.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-03-18T23:19:05.712 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-03-18T23:19:05.712 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:05.712 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.712 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:05.713 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-18T23:19:05.713 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.713 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.713 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.713 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.713 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.713 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.713 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.714 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.714 INFO:teuthology.orchestra.run.smithi049.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-03-18T23:19:05.714 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-03-18T23:19:05.714 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:05.714 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.714 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:05.714 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:05.714 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.715 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.715 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.715 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.715 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.715 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.715 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.715 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.715 INFO:teuthology.orchestra.run.smithi049.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-03-18T23:19:05.715 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-03-18T23:19:05.716 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:05.716 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.716 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:05.716 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:05.716 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.716 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.716 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.716 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.717 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.717 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.717 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.717 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.717 INFO:teuthology.orchestra.run.smithi049.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-03-18T23:19:05.717 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-03-18T23:19:05.717 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:05.717 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.718 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:05.718 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:05.718 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.718 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.718 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.718 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.718 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.718 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.718 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.719 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.719 INFO:teuthology.orchestra.run.smithi049.stdout: "PWD_POLICY_ENABLED": { 2024-03-18T23:19:05.719 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "PWD_POLICY_ENABLED", 2024-03-18T23:19:05.719 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:05.719 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.719 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:05.719 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-18T23:19:05.719 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.719 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.720 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.720 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.720 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.720 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.720 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.720 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.720 INFO:teuthology.orchestra.run.smithi049.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-03-18T23:19:05.720 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-03-18T23:19:05.720 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:05.721 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.721 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:05.721 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-03-18T23:19:05.721 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.721 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.721 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.721 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.721 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.721 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.722 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.722 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.722 INFO:teuthology.orchestra.run.smithi049.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-03-18T23:19:05.722 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-03-18T23:19:05.722 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-18T23:19:05.722 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.722 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:05.722 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "10", 2024-03-18T23:19:05.723 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.723 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.723 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.723 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.723 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.723 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.723 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.723 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.723 INFO:teuthology.orchestra.run.smithi049.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-03-18T23:19:05.724 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-03-18T23:19:05.724 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-18T23:19:05.724 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.724 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:05.724 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "8", 2024-03-18T23:19:05.724 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.724 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.724 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.724 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.725 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.725 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.725 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.725 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.725 INFO:teuthology.orchestra.run.smithi049.stdout: "REST_REQUESTS_TIMEOUT": { 2024-03-18T23:19:05.725 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-03-18T23:19:05.725 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-18T23:19:05.725 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.725 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:05.726 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "45", 2024-03-18T23:19:05.726 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.726 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.726 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.726 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.726 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.726 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.726 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.727 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.727 INFO:teuthology.orchestra.run.smithi049.stdout: "RGW_API_ACCESS_KEY": { 2024-03-18T23:19:05.727 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "RGW_API_ACCESS_KEY", 2024-03-18T23:19:05.727 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:05.727 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.727 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:05.727 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:05.727 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.728 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.728 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.728 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.728 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.728 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.728 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.728 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.728 INFO:teuthology.orchestra.run.smithi049.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-03-18T23:19:05.728 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-03-18T23:19:05.729 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:05.729 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.729 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:05.729 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "admin", 2024-03-18T23:19:05.729 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.729 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.729 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.729 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.729 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.730 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.730 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.730 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.730 INFO:teuthology.orchestra.run.smithi049.stdout: "RGW_API_SECRET_KEY": { 2024-03-18T23:19:05.730 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "RGW_API_SECRET_KEY", 2024-03-18T23:19:05.730 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:05.730 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.730 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:05.730 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:05.731 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.731 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.731 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.731 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.731 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.731 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.731 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.731 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.732 INFO:teuthology.orchestra.run.smithi049.stdout: "RGW_API_SSL_VERIFY": { 2024-03-18T23:19:05.732 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "RGW_API_SSL_VERIFY", 2024-03-18T23:19:05.732 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:05.732 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.732 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:05.732 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-18T23:19:05.732 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.732 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.732 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.733 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.733 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.733 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.733 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.733 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.733 INFO:teuthology.orchestra.run.smithi049.stdout: "UNSAFE_TLS_v1_2": { 2024-03-18T23:19:05.733 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "UNSAFE_TLS_v1_2", 2024-03-18T23:19:05.733 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:05.734 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.734 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:05.734 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:05.734 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.734 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.734 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.734 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.734 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.734 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.735 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.735 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.735 INFO:teuthology.orchestra.run.smithi049.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-03-18T23:19:05.735 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-03-18T23:19:05.735 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-18T23:19:05.735 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.735 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:05.735 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "0", 2024-03-18T23:19:05.735 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.736 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.736 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.736 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.736 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.736 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.736 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.736 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.736 INFO:teuthology.orchestra.run.smithi049.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-03-18T23:19:05.737 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-03-18T23:19:05.737 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-18T23:19:05.737 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.737 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:05.737 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "10", 2024-03-18T23:19:05.737 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.737 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.737 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.737 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.738 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.738 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.738 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.738 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.738 INFO:teuthology.orchestra.run.smithi049.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-03-18T23:19:05.738 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-03-18T23:19:05.738 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-18T23:19:05.738 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.739 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:05.739 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "5", 2024-03-18T23:19:05.739 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.739 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.739 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.739 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.739 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.739 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.739 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.740 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.740 INFO:teuthology.orchestra.run.smithi049.stdout: "cross_origin_url": { 2024-03-18T23:19:05.740 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "cross_origin_url", 2024-03-18T23:19:05.740 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:05.740 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.740 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:05.740 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:05.740 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.741 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.741 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.741 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.741 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.741 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.741 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.741 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.741 INFO:teuthology.orchestra.run.smithi049.stdout: "crt_file": { 2024-03-18T23:19:05.741 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "crt_file", 2024-03-18T23:19:05.742 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:05.742 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.742 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:05.742 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:05.742 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.742 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.742 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.742 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.743 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.743 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.743 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.743 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.743 INFO:teuthology.orchestra.run.smithi049.stdout: "debug": { 2024-03-18T23:19:05.743 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "debug", 2024-03-18T23:19:05.743 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:05.743 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.744 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:05.744 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:05.744 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.744 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.744 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.744 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Enable/disable debug options", 2024-03-18T23:19:05.744 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.744 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.744 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.745 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.745 INFO:teuthology.orchestra.run.smithi049.stdout: "jwt_token_ttl": { 2024-03-18T23:19:05.745 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "jwt_token_ttl", 2024-03-18T23:19:05.745 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-18T23:19:05.745 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.745 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:05.745 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "28800", 2024-03-18T23:19:05.745 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.745 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.745 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.746 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.746 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.746 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.746 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.746 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.746 INFO:teuthology.orchestra.run.smithi049.stdout: "key_file": { 2024-03-18T23:19:05.746 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "key_file", 2024-03-18T23:19:05.746 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:05.746 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.747 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:05.747 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:05.747 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.747 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.747 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.747 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.747 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.747 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.747 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.748 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.748 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-18T23:19:05.748 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-18T23:19:05.748 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:05.748 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.748 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.748 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:05.748 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.748 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.749 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-18T23:19:05.749 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-18T23:19:05.749 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-18T23:19:05.749 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-18T23:19:05.749 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-18T23:19:05.749 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-18T23:19:05.749 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-18T23:19:05.749 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-18T23:19:05.749 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.750 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.750 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.750 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.750 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.750 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-18T23:19:05.750 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-18T23:19:05.750 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:05.750 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.750 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.751 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:05.751 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.751 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.751 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.751 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.751 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.751 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.751 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.751 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.752 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-18T23:19:05.752 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-18T23:19:05.752 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:05.752 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.752 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.752 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-18T23:19:05.752 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.752 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.752 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-18T23:19:05.753 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-18T23:19:05.753 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-18T23:19:05.753 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-18T23:19:05.753 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-18T23:19:05.753 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-18T23:19:05.753 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-18T23:19:05.753 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-18T23:19:05.753 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.753 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.754 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.754 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.754 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.754 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-18T23:19:05.754 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-18T23:19:05.754 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:05.754 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.754 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.755 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:05.755 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.755 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.755 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.755 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.755 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.755 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.755 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.755 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.755 INFO:teuthology.orchestra.run.smithi049.stdout: "motd": { 2024-03-18T23:19:05.756 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "motd", 2024-03-18T23:19:05.756 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:05.756 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.756 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:05.756 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:05.756 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.756 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.756 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.756 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "The message of the day", 2024-03-18T23:19:05.757 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.757 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.757 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.757 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.757 INFO:teuthology.orchestra.run.smithi049.stdout: "redirect_resolve_ip_addr": { 2024-03-18T23:19:05.757 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "redirect_resolve_ip_addr", 2024-03-18T23:19:05.757 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:05.757 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.758 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:05.758 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:05.758 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.758 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.758 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.758 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.758 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.758 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.759 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.759 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.759 INFO:teuthology.orchestra.run.smithi049.stdout: "server_addr": { 2024-03-18T23:19:05.759 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "server_addr", 2024-03-18T23:19:05.759 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:05.759 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.759 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:05.759 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "::", 2024-03-18T23:19:05.759 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.759 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.760 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.760 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.760 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.760 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.760 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.760 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.760 INFO:teuthology.orchestra.run.smithi049.stdout: "server_port": { 2024-03-18T23:19:05.760 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "server_port", 2024-03-18T23:19:05.760 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-18T23:19:05.761 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.761 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:05.761 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "8080", 2024-03-18T23:19:05.761 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.761 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.761 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.761 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.761 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.761 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.762 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.762 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.762 INFO:teuthology.orchestra.run.smithi049.stdout: "ssl": { 2024-03-18T23:19:05.762 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "ssl", 2024-03-18T23:19:05.762 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:05.762 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.762 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:05.762 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-18T23:19:05.763 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.763 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.763 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.763 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.763 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.763 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.763 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.763 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.763 INFO:teuthology.orchestra.run.smithi049.stdout: "ssl_server_port": { 2024-03-18T23:19:05.764 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "ssl_server_port", 2024-03-18T23:19:05.764 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-18T23:19:05.764 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.764 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:05.764 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "8443", 2024-03-18T23:19:05.764 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.764 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.764 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.765 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.765 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.765 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.765 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.765 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.765 INFO:teuthology.orchestra.run.smithi049.stdout: "standby_behaviour": { 2024-03-18T23:19:05.765 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "standby_behaviour", 2024-03-18T23:19:05.765 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:05.765 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.765 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:05.766 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "redirect", 2024-03-18T23:19:05.766 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.766 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.766 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-18T23:19:05.766 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-18T23:19:05.766 INFO:teuthology.orchestra.run.smithi049.stdout: "redirect" 2024-03-18T23:19:05.766 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-18T23:19:05.767 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.767 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.767 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.767 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.767 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.767 INFO:teuthology.orchestra.run.smithi049.stdout: "standby_error_status_code": { 2024-03-18T23:19:05.767 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "standby_error_status_code", 2024-03-18T23:19:05.767 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-18T23:19:05.767 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.768 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:05.768 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "500", 2024-03-18T23:19:05.768 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "400", 2024-03-18T23:19:05.768 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "599", 2024-03-18T23:19:05.768 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.768 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.768 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.768 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.769 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.769 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.769 INFO:teuthology.orchestra.run.smithi049.stdout: "url_prefix": { 2024-03-18T23:19:05.769 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "url_prefix", 2024-03-18T23:19:05.769 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:05.769 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.769 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:05.769 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:05.769 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.770 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.770 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.770 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.770 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.770 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.770 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.770 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:19:05.770 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:19:05.770 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.771 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-18T23:19:05.771 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "devicehealth", 2024-03-18T23:19:05.771 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-18T23:19:05.771 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-18T23:19:05.771 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-18T23:19:05.771 INFO:teuthology.orchestra.run.smithi049.stdout: "enable_monitoring": { 2024-03-18T23:19:05.771 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "enable_monitoring", 2024-03-18T23:19:05.771 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:05.772 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.772 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.772 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-18T23:19:05.772 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.772 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.772 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.772 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "monitor device health metrics", 2024-03-18T23:19:05.772 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.772 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.773 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.773 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.773 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-18T23:19:05.773 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-18T23:19:05.773 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:05.773 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.773 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.773 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:05.774 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.774 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.774 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-18T23:19:05.774 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-18T23:19:05.774 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-18T23:19:05.774 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-18T23:19:05.774 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-18T23:19:05.774 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-18T23:19:05.775 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-18T23:19:05.775 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-18T23:19:05.775 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.775 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.775 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.775 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.775 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.775 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-18T23:19:05.775 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-18T23:19:05.776 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:05.776 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.776 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.776 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:05.776 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.776 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.776 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.776 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.776 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.777 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.777 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.777 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.777 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-18T23:19:05.777 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-18T23:19:05.777 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:05.777 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.777 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.778 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-18T23:19:05.778 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.778 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.778 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-18T23:19:05.778 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-18T23:19:05.778 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-18T23:19:05.778 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-18T23:19:05.778 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-18T23:19:05.779 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-18T23:19:05.779 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-18T23:19:05.779 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-18T23:19:05.779 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.779 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.779 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.779 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.779 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.779 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-18T23:19:05.780 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-18T23:19:05.780 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:05.780 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.780 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.780 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:05.780 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.780 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.780 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.781 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.781 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.781 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.781 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.781 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.781 INFO:teuthology.orchestra.run.smithi049.stdout: "mark_out_threshold": { 2024-03-18T23:19:05.781 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "mark_out_threshold", 2024-03-18T23:19:05.781 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-03-18T23:19:05.781 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.782 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.782 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "2419200", 2024-03-18T23:19:05.782 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.782 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.782 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.782 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-03-18T23:19:05.782 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.782 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.782 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.783 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.783 INFO:teuthology.orchestra.run.smithi049.stdout: "pool_name": { 2024-03-18T23:19:05.783 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "pool_name", 2024-03-18T23:19:05.783 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:05.783 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.783 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.783 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "device_health_metrics", 2024-03-18T23:19:05.783 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.784 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.784 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.784 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "name of pool in which to store device health metrics", 2024-03-18T23:19:05.784 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.784 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.784 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.784 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.784 INFO:teuthology.orchestra.run.smithi049.stdout: "retention_period": { 2024-03-18T23:19:05.784 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "retention_period", 2024-03-18T23:19:05.785 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-03-18T23:19:05.785 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.785 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.785 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "15552000", 2024-03-18T23:19:05.785 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.785 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.785 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.785 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "how long to retain device health metrics", 2024-03-18T23:19:05.785 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.786 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.786 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.786 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.786 INFO:teuthology.orchestra.run.smithi049.stdout: "scrape_frequency": { 2024-03-18T23:19:05.786 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "scrape_frequency", 2024-03-18T23:19:05.786 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-03-18T23:19:05.786 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.786 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.787 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "86400", 2024-03-18T23:19:05.787 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.787 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.787 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.787 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "how frequently to scrape device health metrics", 2024-03-18T23:19:05.787 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.787 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.787 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.788 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.788 INFO:teuthology.orchestra.run.smithi049.stdout: "self_heal": { 2024-03-18T23:19:05.788 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "self_heal", 2024-03-18T23:19:05.788 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:05.788 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.788 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.788 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-18T23:19:05.788 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.788 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.788 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.789 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-03-18T23:19:05.789 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.789 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.789 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.789 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.789 INFO:teuthology.orchestra.run.smithi049.stdout: "sleep_interval": { 2024-03-18T23:19:05.789 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "sleep_interval", 2024-03-18T23:19:05.789 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-03-18T23:19:05.789 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.790 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.790 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "600", 2024-03-18T23:19:05.790 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.790 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.790 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.790 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "how frequently to wake up and check device health", 2024-03-18T23:19:05.790 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.790 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.791 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.791 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.791 INFO:teuthology.orchestra.run.smithi049.stdout: "warn_threshold": { 2024-03-18T23:19:05.791 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "warn_threshold", 2024-03-18T23:19:05.791 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-03-18T23:19:05.791 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.791 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.791 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "7257600", 2024-03-18T23:19:05.792 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.792 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.792 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.792 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-03-18T23:19:05.792 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.792 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.792 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.792 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:19:05.792 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:19:05.793 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.793 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-18T23:19:05.793 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "diskprediction_local", 2024-03-18T23:19:05.793 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-18T23:19:05.793 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-18T23:19:05.793 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-18T23:19:05.793 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-18T23:19:05.793 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-18T23:19:05.793 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:05.794 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.794 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.794 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:05.794 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.794 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.794 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-18T23:19:05.794 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-18T23:19:05.794 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-18T23:19:05.795 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-18T23:19:05.795 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-18T23:19:05.795 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-18T23:19:05.795 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-18T23:19:05.795 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-18T23:19:05.795 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.795 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.795 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.795 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.796 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.796 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-18T23:19:05.796 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-18T23:19:05.796 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:05.796 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.796 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.796 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:05.796 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.796 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.797 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.797 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.797 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.797 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.797 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.797 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.797 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-18T23:19:05.797 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-18T23:19:05.798 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:05.798 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.798 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.798 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-18T23:19:05.798 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.798 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.798 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-18T23:19:05.798 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-18T23:19:05.799 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-18T23:19:05.799 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-18T23:19:05.799 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-18T23:19:05.799 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-18T23:19:05.799 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-18T23:19:05.799 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-18T23:19:05.799 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.799 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.799 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.800 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.800 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.800 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-18T23:19:05.800 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-18T23:19:05.800 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:05.800 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.800 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.800 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:05.801 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.801 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.801 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.801 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.801 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.801 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.801 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.801 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.801 INFO:teuthology.orchestra.run.smithi049.stdout: "predict_interval": { 2024-03-18T23:19:05.802 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "predict_interval", 2024-03-18T23:19:05.802 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:05.802 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.802 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:05.802 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "86400", 2024-03-18T23:19:05.802 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.802 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.802 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.803 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.803 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.803 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.803 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.803 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.803 INFO:teuthology.orchestra.run.smithi049.stdout: "predictor_model": { 2024-03-18T23:19:05.803 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "predictor_model", 2024-03-18T23:19:05.803 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:05.804 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.804 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:05.804 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "prophetstor", 2024-03-18T23:19:05.804 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.804 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.804 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.804 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.805 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.805 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.805 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.805 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.805 INFO:teuthology.orchestra.run.smithi049.stdout: "sleep_interval": { 2024-03-18T23:19:05.805 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "sleep_interval", 2024-03-18T23:19:05.805 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:05.805 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.805 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:05.806 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "600", 2024-03-18T23:19:05.806 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.806 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.806 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.806 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.806 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.806 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.806 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.806 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:19:05.807 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:19:05.807 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.807 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-18T23:19:05.807 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "influx", 2024-03-18T23:19:05.807 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": false, 2024-03-18T23:19:05.807 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "influxdb python module not found", 2024-03-18T23:19:05.807 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-18T23:19:05.807 INFO:teuthology.orchestra.run.smithi049.stdout: "batch_size": { 2024-03-18T23:19:05.808 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "batch_size", 2024-03-18T23:19:05.808 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-18T23:19:05.808 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.808 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:05.808 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "5000", 2024-03-18T23:19:05.808 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.808 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.808 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.808 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-03-18T23:19:05.809 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.809 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.809 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.809 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.809 INFO:teuthology.orchestra.run.smithi049.stdout: "database": { 2024-03-18T23:19:05.809 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "database", 2024-03-18T23:19:05.809 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:05.809 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.809 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:05.810 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "ceph", 2024-03-18T23:19:05.810 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.810 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.810 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.810 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-03-18T23:19:05.810 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.810 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.810 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.811 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.811 INFO:teuthology.orchestra.run.smithi049.stdout: "hostname": { 2024-03-18T23:19:05.811 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "hostname", 2024-03-18T23:19:05.811 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:05.811 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.811 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:05.811 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:05.811 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.811 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.812 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.812 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "InfluxDB server hostname", 2024-03-18T23:19:05.812 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.812 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.812 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.812 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.812 INFO:teuthology.orchestra.run.smithi049.stdout: "interval": { 2024-03-18T23:19:05.812 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "interval", 2024-03-18T23:19:05.812 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-03-18T23:19:05.813 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.813 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:05.813 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "30", 2024-03-18T23:19:05.813 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "5", 2024-03-18T23:19:05.813 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.813 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.813 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-03-18T23:19:05.813 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.814 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.814 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.814 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.814 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-18T23:19:05.814 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-18T23:19:05.814 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:05.814 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.814 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.814 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:05.815 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.815 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.815 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-18T23:19:05.815 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-18T23:19:05.815 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-18T23:19:05.815 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-18T23:19:05.815 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-18T23:19:05.815 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-18T23:19:05.816 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-18T23:19:05.816 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-18T23:19:05.816 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.816 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.816 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.816 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.816 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.816 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-18T23:19:05.817 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-18T23:19:05.817 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:05.817 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.817 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.817 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:05.817 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.817 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.817 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.817 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.818 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.818 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.818 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.818 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.818 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-18T23:19:05.818 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-18T23:19:05.818 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:05.818 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.818 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.819 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-18T23:19:05.819 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.819 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.819 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-18T23:19:05.819 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-18T23:19:05.819 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-18T23:19:05.819 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-18T23:19:05.819 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-18T23:19:05.819 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-18T23:19:05.820 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-18T23:19:05.820 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-18T23:19:05.820 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.820 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.820 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.820 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.820 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.820 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-18T23:19:05.820 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-18T23:19:05.820 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:05.821 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.821 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.821 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:05.821 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.821 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.821 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.821 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.821 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.821 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.822 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.822 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.822 INFO:teuthology.orchestra.run.smithi049.stdout: "password": { 2024-03-18T23:19:05.822 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "password", 2024-03-18T23:19:05.822 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:05.822 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.822 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:05.822 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:05.822 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.823 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.823 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.823 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "password of InfluxDB server user", 2024-03-18T23:19:05.823 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.823 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.823 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.823 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.823 INFO:teuthology.orchestra.run.smithi049.stdout: "port": { 2024-03-18T23:19:05.824 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "port", 2024-03-18T23:19:05.824 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-18T23:19:05.824 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.824 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:05.824 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "8086", 2024-03-18T23:19:05.824 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.824 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.824 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.824 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "InfluxDB server port", 2024-03-18T23:19:05.824 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.825 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.825 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.825 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.825 INFO:teuthology.orchestra.run.smithi049.stdout: "ssl": { 2024-03-18T23:19:05.825 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "ssl", 2024-03-18T23:19:05.825 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:05.825 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.825 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:05.825 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "false", 2024-03-18T23:19:05.826 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.826 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.826 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.826 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-03-18T23:19:05.826 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.826 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.826 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.826 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.827 INFO:teuthology.orchestra.run.smithi049.stdout: "threads": { 2024-03-18T23:19:05.827 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "threads", 2024-03-18T23:19:05.827 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-18T23:19:05.827 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.827 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:05.827 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "5", 2024-03-18T23:19:05.827 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "1", 2024-03-18T23:19:05.827 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "32", 2024-03-18T23:19:05.827 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.828 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-03-18T23:19:05.828 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.828 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.828 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.828 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.828 INFO:teuthology.orchestra.run.smithi049.stdout: "username": { 2024-03-18T23:19:05.828 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "username", 2024-03-18T23:19:05.828 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:05.828 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.829 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:05.829 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:05.829 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.829 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.829 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.829 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "username of InfluxDB server user", 2024-03-18T23:19:05.829 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.829 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.829 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.830 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.830 INFO:teuthology.orchestra.run.smithi049.stdout: "verify_ssl": { 2024-03-18T23:19:05.830 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "verify_ssl", 2024-03-18T23:19:05.830 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:05.830 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.830 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:05.830 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "true", 2024-03-18T23:19:05.831 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.831 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.831 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.831 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-03-18T23:19:05.831 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.831 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.831 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.831 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:19:05.831 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:19:05.832 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.832 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-18T23:19:05.832 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "insights", 2024-03-18T23:19:05.832 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-18T23:19:05.832 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-18T23:19:05.832 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-18T23:19:05.832 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-18T23:19:05.832 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-18T23:19:05.833 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:05.833 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.833 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.833 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:05.833 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.833 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.833 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-18T23:19:05.833 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-18T23:19:05.833 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-18T23:19:05.834 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-18T23:19:05.834 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-18T23:19:05.834 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-18T23:19:05.834 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-18T23:19:05.834 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-18T23:19:05.834 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.834 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.834 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.835 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.835 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.835 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-18T23:19:05.835 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-18T23:19:05.835 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:05.835 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.835 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.835 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:05.835 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.836 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.836 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.836 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.836 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.836 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.836 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.836 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.836 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-18T23:19:05.836 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-18T23:19:05.837 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:05.837 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.837 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.837 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-18T23:19:05.837 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.837 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.837 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-18T23:19:05.837 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-18T23:19:05.837 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-18T23:19:05.838 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-18T23:19:05.838 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-18T23:19:05.838 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-18T23:19:05.838 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-18T23:19:05.838 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-18T23:19:05.838 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.838 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.838 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.838 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.839 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.839 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-18T23:19:05.839 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-18T23:19:05.839 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:05.839 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.839 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.839 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:05.839 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.839 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.840 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.840 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.840 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.840 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.840 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.840 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:19:05.840 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:19:05.840 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.840 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-18T23:19:05.841 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "iostat", 2024-03-18T23:19:05.841 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-18T23:19:05.841 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-18T23:19:05.841 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-18T23:19:05.841 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-18T23:19:05.841 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-18T23:19:05.841 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:05.841 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.841 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.842 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:05.842 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.842 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.842 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-18T23:19:05.842 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-18T23:19:05.842 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-18T23:19:05.842 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-18T23:19:05.842 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-18T23:19:05.842 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-18T23:19:05.843 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-18T23:19:05.843 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-18T23:19:05.843 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.843 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.843 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.843 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.843 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.843 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-18T23:19:05.843 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-18T23:19:05.844 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:05.844 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.844 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.844 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:05.844 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.844 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.844 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.844 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.844 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.845 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.845 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.845 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.845 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-18T23:19:05.845 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-18T23:19:05.845 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:05.845 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.845 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.845 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-18T23:19:05.846 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.846 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.846 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-18T23:19:05.846 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-18T23:19:05.846 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-18T23:19:05.846 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-18T23:19:05.846 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-18T23:19:05.846 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-18T23:19:05.846 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-18T23:19:05.847 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-18T23:19:05.847 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.847 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.847 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.847 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.847 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.847 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-18T23:19:05.847 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-18T23:19:05.848 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:05.848 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.848 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.848 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:05.848 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.848 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.848 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.848 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.848 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.849 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.849 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.849 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:19:05.849 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:19:05.849 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.849 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-18T23:19:05.849 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "k8sevents", 2024-03-18T23:19:05.849 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-18T23:19:05.850 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-18T23:19:05.850 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-18T23:19:05.850 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph_event_retention_days": { 2024-03-18T23:19:05.850 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "ceph_event_retention_days", 2024-03-18T23:19:05.850 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-18T23:19:05.850 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.850 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:05.850 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "7", 2024-03-18T23:19:05.850 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.851 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.851 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.851 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Days to hold ceph event information within local cache", 2024-03-18T23:19:05.851 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.851 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.851 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.851 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.851 INFO:teuthology.orchestra.run.smithi049.stdout: "config_check_secs": { 2024-03-18T23:19:05.851 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "config_check_secs", 2024-03-18T23:19:05.852 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-18T23:19:05.852 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.852 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:05.852 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "10", 2024-03-18T23:19:05.852 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "10", 2024-03-18T23:19:05.852 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.852 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.852 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-03-18T23:19:05.852 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.853 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.853 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.853 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.853 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-18T23:19:05.853 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-18T23:19:05.853 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:05.853 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.853 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.854 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:05.854 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.854 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.854 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-18T23:19:05.854 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-18T23:19:05.854 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-18T23:19:05.854 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-18T23:19:05.854 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-18T23:19:05.854 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-18T23:19:05.855 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-18T23:19:05.855 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-18T23:19:05.855 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.855 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.855 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.855 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.855 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.855 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-18T23:19:05.855 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-18T23:19:05.856 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:05.856 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.856 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.856 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:05.856 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.856 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.856 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.856 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.856 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.857 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.857 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.857 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.857 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-18T23:19:05.857 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-18T23:19:05.857 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:05.857 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.857 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.858 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-18T23:19:05.858 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.858 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.858 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-18T23:19:05.858 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-18T23:19:05.858 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-18T23:19:05.858 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-18T23:19:05.858 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-18T23:19:05.858 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-18T23:19:05.859 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-18T23:19:05.859 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-18T23:19:05.859 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.859 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.859 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.859 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.859 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.859 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-18T23:19:05.859 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-18T23:19:05.860 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:05.860 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.860 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.860 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:05.860 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.860 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.860 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.860 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.860 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.861 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.861 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.861 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:19:05.861 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:19:05.861 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.861 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-18T23:19:05.861 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "localpool", 2024-03-18T23:19:05.861 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-18T23:19:05.862 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-18T23:19:05.862 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-18T23:19:05.862 INFO:teuthology.orchestra.run.smithi049.stdout: "failure_domain": { 2024-03-18T23:19:05.862 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "failure_domain", 2024-03-18T23:19:05.862 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:05.862 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.862 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.862 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "host", 2024-03-18T23:19:05.863 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.863 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.863 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.863 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "failure domain for any created local pool", 2024-03-18T23:19:05.863 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-03-18T23:19:05.863 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.863 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.863 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.863 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-18T23:19:05.864 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-18T23:19:05.864 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:05.864 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.864 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.864 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:05.864 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.864 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.864 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-18T23:19:05.864 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-18T23:19:05.865 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-18T23:19:05.865 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-18T23:19:05.865 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-18T23:19:05.865 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-18T23:19:05.865 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-18T23:19:05.865 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-18T23:19:05.865 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.865 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.866 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.866 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.866 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.866 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-18T23:19:05.866 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-18T23:19:05.866 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:05.866 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.866 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.867 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:05.867 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.867 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.867 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.867 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.867 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.867 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.867 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.868 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.868 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-18T23:19:05.868 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-18T23:19:05.868 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:05.868 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.868 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.868 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-18T23:19:05.868 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.868 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.869 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-18T23:19:05.869 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-18T23:19:05.869 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-18T23:19:05.869 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-18T23:19:05.869 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-18T23:19:05.869 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-18T23:19:05.869 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-18T23:19:05.869 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-18T23:19:05.869 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.870 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.870 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.870 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.870 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.870 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-18T23:19:05.870 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-18T23:19:05.870 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:05.870 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.871 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.871 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:05.871 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.871 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.871 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.871 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.871 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.871 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.871 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.872 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.872 INFO:teuthology.orchestra.run.smithi049.stdout: "min_size": { 2024-03-18T23:19:05.872 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "min_size", 2024-03-18T23:19:05.872 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-18T23:19:05.872 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.872 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.872 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:05.872 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.872 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.873 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.873 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "default min_size for any created local pool", 2024-03-18T23:19:05.873 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-03-18T23:19:05.873 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.873 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.873 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.873 INFO:teuthology.orchestra.run.smithi049.stdout: "num_rep": { 2024-03-18T23:19:05.873 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "num_rep", 2024-03-18T23:19:05.873 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-18T23:19:05.874 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.874 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.874 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "3", 2024-03-18T23:19:05.874 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.874 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.874 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.874 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "default replica count for any created local pool", 2024-03-18T23:19:05.874 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.875 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.875 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.875 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.875 INFO:teuthology.orchestra.run.smithi049.stdout: "pg_num": { 2024-03-18T23:19:05.875 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "pg_num", 2024-03-18T23:19:05.875 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-18T23:19:05.875 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.875 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.875 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "128", 2024-03-18T23:19:05.876 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.876 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.876 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.876 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "default pg_num for any created local pool", 2024-03-18T23:19:05.876 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.876 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.876 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.876 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.876 INFO:teuthology.orchestra.run.smithi049.stdout: "prefix": { 2024-03-18T23:19:05.877 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "prefix", 2024-03-18T23:19:05.877 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:05.877 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.877 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.877 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:05.877 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.877 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.877 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.878 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "name prefix for any created local pool", 2024-03-18T23:19:05.878 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.878 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.878 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.878 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.878 INFO:teuthology.orchestra.run.smithi049.stdout: "subtree": { 2024-03-18T23:19:05.878 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "subtree", 2024-03-18T23:19:05.878 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:05.879 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.879 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.879 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "rack", 2024-03-18T23:19:05.879 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.879 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.879 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.879 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "CRUSH level for which to create a local pool", 2024-03-18T23:19:05.879 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-03-18T23:19:05.879 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.880 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.880 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:19:05.880 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:19:05.880 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.880 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-18T23:19:05.880 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "mds_autoscaler", 2024-03-18T23:19:05.880 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-18T23:19:05.880 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-18T23:19:05.880 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-18T23:19:05.880 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-18T23:19:05.881 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-18T23:19:05.881 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:05.881 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.881 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.881 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:05.881 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.881 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.881 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-18T23:19:05.881 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-18T23:19:05.882 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-18T23:19:05.882 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-18T23:19:05.882 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-18T23:19:05.882 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-18T23:19:05.882 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-18T23:19:05.882 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-18T23:19:05.882 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.882 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.882 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.883 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.883 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.883 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-18T23:19:05.883 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-18T23:19:05.883 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:05.883 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.883 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.883 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:05.884 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.884 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.884 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.884 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.884 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.884 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.884 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.884 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.884 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-18T23:19:05.885 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-18T23:19:05.885 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:05.885 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.885 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.885 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-18T23:19:05.885 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.885 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.885 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-18T23:19:05.885 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-18T23:19:05.886 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-18T23:19:05.886 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-18T23:19:05.886 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-18T23:19:05.886 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-18T23:19:05.886 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-18T23:19:05.886 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-18T23:19:05.886 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.886 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.887 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.887 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.887 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.887 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-18T23:19:05.887 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-18T23:19:05.887 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:05.887 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.887 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.887 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:05.888 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.888 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.888 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.888 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.888 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.888 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.888 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.888 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:19:05.888 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:19:05.889 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.889 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-18T23:19:05.889 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "mirroring", 2024-03-18T23:19:05.889 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-18T23:19:05.889 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-18T23:19:05.889 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-18T23:19:05.889 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-18T23:19:05.889 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-18T23:19:05.889 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:05.889 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.890 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.890 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:05.890 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.890 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.890 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-18T23:19:05.890 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-18T23:19:05.890 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-18T23:19:05.890 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-18T23:19:05.891 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-18T23:19:05.891 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-18T23:19:05.891 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-18T23:19:05.891 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-18T23:19:05.891 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.891 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.891 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.891 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.891 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.892 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-18T23:19:05.892 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-18T23:19:05.892 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:05.892 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.892 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.892 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:05.892 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.892 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.892 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.893 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.893 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.893 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.893 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.893 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.893 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-18T23:19:05.893 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-18T23:19:05.893 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:05.894 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.894 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.894 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-18T23:19:05.894 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.894 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.894 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-18T23:19:05.894 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-18T23:19:05.894 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-18T23:19:05.895 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-18T23:19:05.895 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-18T23:19:05.895 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-18T23:19:05.895 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-18T23:19:05.895 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-18T23:19:05.895 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.895 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.895 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.895 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.896 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.896 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-18T23:19:05.896 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-18T23:19:05.896 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:05.896 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.896 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.896 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:05.896 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.897 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.897 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.897 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.897 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.897 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.897 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.897 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:19:05.897 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:19:05.898 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.898 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-18T23:19:05.898 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "nfs", 2024-03-18T23:19:05.898 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-18T23:19:05.898 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-18T23:19:05.898 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-18T23:19:05.898 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-18T23:19:05.898 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-18T23:19:05.898 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:05.899 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.899 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.899 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:05.899 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.899 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.899 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-18T23:19:05.899 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-18T23:19:05.899 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-18T23:19:05.900 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-18T23:19:05.900 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-18T23:19:05.900 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-18T23:19:05.900 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-18T23:19:05.900 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-18T23:19:05.900 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.900 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.900 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.900 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.901 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.901 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-18T23:19:05.901 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-18T23:19:05.901 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:05.901 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.901 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.901 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:05.901 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.901 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.902 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.902 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.902 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.902 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.902 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.902 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.902 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-18T23:19:05.902 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-18T23:19:05.903 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:05.903 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.903 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.903 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-18T23:19:05.903 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.903 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.903 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-18T23:19:05.903 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-18T23:19:05.903 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-18T23:19:05.904 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-18T23:19:05.904 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-18T23:19:05.904 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-18T23:19:05.904 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-18T23:19:05.904 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-18T23:19:05.904 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.904 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.904 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.904 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.905 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.905 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-18T23:19:05.905 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-18T23:19:05.905 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:05.905 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.905 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.905 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:05.905 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.905 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.906 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.906 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.906 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.906 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.906 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.906 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:19:05.906 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:19:05.906 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.907 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-18T23:19:05.907 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "orchestrator", 2024-03-18T23:19:05.907 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-18T23:19:05.907 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-18T23:19:05.907 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-18T23:19:05.907 INFO:teuthology.orchestra.run.smithi049.stdout: "fail_fs": { 2024-03-18T23:19:05.907 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "fail_fs", 2024-03-18T23:19:05.907 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:05.907 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.908 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:05.908 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:05.908 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.908 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.908 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.908 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-03-18T23:19:05.908 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.908 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.908 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.909 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.909 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-18T23:19:05.909 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-18T23:19:05.909 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:05.909 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.909 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.909 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:05.909 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.909 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.910 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-18T23:19:05.910 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-18T23:19:05.910 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-18T23:19:05.910 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-18T23:19:05.910 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-18T23:19:05.910 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-18T23:19:05.910 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-18T23:19:05.910 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-18T23:19:05.911 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.911 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.911 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.911 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.911 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.911 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-18T23:19:05.911 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-18T23:19:05.911 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:05.911 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.912 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.912 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:05.912 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.912 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.912 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.912 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.912 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.912 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.912 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.913 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.913 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-18T23:19:05.913 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-18T23:19:05.913 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:05.913 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.913 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.913 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-18T23:19:05.913 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.914 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.914 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-18T23:19:05.914 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-18T23:19:05.914 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-18T23:19:05.914 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-18T23:19:05.914 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-18T23:19:05.914 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-18T23:19:05.914 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-18T23:19:05.914 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-18T23:19:05.915 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.915 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.915 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.915 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.915 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.915 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-18T23:19:05.915 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-18T23:19:05.915 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:05.916 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.916 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.916 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:05.916 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.916 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.916 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.916 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.916 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.916 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.917 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.917 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.917 INFO:teuthology.orchestra.run.smithi049.stdout: "orchestrator": { 2024-03-18T23:19:05.917 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "orchestrator", 2024-03-18T23:19:05.917 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:05.917 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.917 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.917 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:05.917 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.918 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.918 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-18T23:19:05.918 INFO:teuthology.orchestra.run.smithi049.stdout: "cephadm", 2024-03-18T23:19:05.918 INFO:teuthology.orchestra.run.smithi049.stdout: "rook", 2024-03-18T23:19:05.918 INFO:teuthology.orchestra.run.smithi049.stdout: "test_orchestrator" 2024-03-18T23:19:05.918 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-18T23:19:05.918 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Orchestrator backend", 2024-03-18T23:19:05.918 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.919 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.919 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.919 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:19:05.919 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:19:05.919 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.919 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-18T23:19:05.919 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "osd_perf_query", 2024-03-18T23:19:05.919 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-18T23:19:05.919 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-18T23:19:05.920 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-18T23:19:05.920 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-18T23:19:05.920 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-18T23:19:05.920 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:05.920 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.920 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.920 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:05.920 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.920 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.921 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-18T23:19:05.921 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-18T23:19:05.921 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-18T23:19:05.921 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-18T23:19:05.921 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-18T23:19:05.921 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-18T23:19:05.921 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-18T23:19:05.921 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-18T23:19:05.921 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.922 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.922 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.922 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.922 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.922 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-18T23:19:05.922 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-18T23:19:05.922 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:05.922 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.923 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.923 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:05.923 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.923 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.923 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.923 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.923 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.923 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.924 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.924 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.924 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-18T23:19:05.924 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-18T23:19:05.924 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:05.924 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.924 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.924 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-18T23:19:05.924 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.925 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.925 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-18T23:19:05.925 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-18T23:19:05.925 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-18T23:19:05.925 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-18T23:19:05.925 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-18T23:19:05.925 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-18T23:19:05.925 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-18T23:19:05.925 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-18T23:19:05.926 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.926 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.926 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.926 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.926 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.926 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-18T23:19:05.926 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-18T23:19:05.926 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:05.927 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.927 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.927 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:05.927 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.927 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.927 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.927 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.927 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.927 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.928 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.928 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:19:05.928 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:19:05.928 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.928 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-18T23:19:05.928 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "osd_support", 2024-03-18T23:19:05.928 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-18T23:19:05.928 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-18T23:19:05.928 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-18T23:19:05.929 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-18T23:19:05.929 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-18T23:19:05.929 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:05.929 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.929 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.929 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:05.929 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.929 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.930 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-18T23:19:05.930 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-18T23:19:05.930 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-18T23:19:05.930 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-18T23:19:05.930 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-18T23:19:05.930 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-18T23:19:05.930 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-18T23:19:05.930 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-18T23:19:05.931 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.931 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.931 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.931 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.931 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.931 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-18T23:19:05.931 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-18T23:19:05.931 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:05.931 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.932 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.932 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:05.932 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.932 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.932 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.932 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.932 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.932 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.932 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.933 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.933 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-18T23:19:05.933 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-18T23:19:05.933 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:05.933 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.933 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.933 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-18T23:19:05.933 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.933 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.934 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-18T23:19:05.934 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-18T23:19:05.934 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-18T23:19:05.934 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-18T23:19:05.934 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-18T23:19:05.934 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-18T23:19:05.934 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-18T23:19:05.934 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-18T23:19:05.935 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.935 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.935 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.935 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.935 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.935 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-18T23:19:05.935 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-18T23:19:05.935 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:05.935 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.936 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.936 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:05.936 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.936 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.936 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.936 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.936 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.936 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.937 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.937 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:19:05.937 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:19:05.937 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.937 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-18T23:19:05.937 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "pg_autoscaler", 2024-03-18T23:19:05.937 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-18T23:19:05.937 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-18T23:19:05.937 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-18T23:19:05.938 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-18T23:19:05.938 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-18T23:19:05.938 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:05.938 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.938 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.938 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:05.938 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.938 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.938 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-18T23:19:05.939 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-18T23:19:05.939 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-18T23:19:05.939 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-18T23:19:05.939 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-18T23:19:05.939 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-18T23:19:05.939 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-18T23:19:05.939 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-18T23:19:05.939 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.939 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.940 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.940 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.940 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.940 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-18T23:19:05.940 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-18T23:19:05.940 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:05.940 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.940 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.941 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:05.941 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.941 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.941 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.941 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.941 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.941 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.941 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.941 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.942 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-18T23:19:05.942 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-18T23:19:05.942 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:05.942 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.942 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.942 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-18T23:19:05.942 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.942 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.942 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-18T23:19:05.943 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-18T23:19:05.943 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-18T23:19:05.943 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-18T23:19:05.943 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-18T23:19:05.943 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-18T23:19:05.943 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-18T23:19:05.943 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-18T23:19:05.943 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.943 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.944 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.944 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.944 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.944 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-18T23:19:05.944 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-18T23:19:05.944 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:05.944 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.944 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.944 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:05.945 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.945 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.945 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.945 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.945 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.945 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.945 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.945 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.946 INFO:teuthology.orchestra.run.smithi049.stdout: "sleep_interval": { 2024-03-18T23:19:05.946 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "sleep_interval", 2024-03-18T23:19:05.946 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-03-18T23:19:05.946 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.946 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:05.946 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "60", 2024-03-18T23:19:05.946 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.946 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.946 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.947 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.947 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.947 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.947 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.947 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.947 INFO:teuthology.orchestra.run.smithi049.stdout: "threshold": { 2024-03-18T23:19:05.947 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "threshold", 2024-03-18T23:19:05.947 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "float", 2024-03-18T23:19:05.948 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.948 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:05.948 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "3.0", 2024-03-18T23:19:05.948 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "1.0", 2024-03-18T23:19:05.948 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.948 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.948 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "scaling threshold", 2024-03-18T23:19:05.948 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-03-18T23:19:05.949 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.949 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.949 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:19:05.949 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:19:05.949 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.949 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-18T23:19:05.949 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "progress", 2024-03-18T23:19:05.949 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-18T23:19:05.949 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-18T23:19:05.950 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-18T23:19:05.950 INFO:teuthology.orchestra.run.smithi049.stdout: "allow_pg_recovery_event": { 2024-03-18T23:19:05.950 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "allow_pg_recovery_event", 2024-03-18T23:19:05.950 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:05.950 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.950 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.950 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:05.950 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.951 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.951 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.951 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "allow the module to show pg recovery progress", 2024-03-18T23:19:05.951 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.951 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.951 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.951 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.951 INFO:teuthology.orchestra.run.smithi049.stdout: "enabled": { 2024-03-18T23:19:05.951 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "enabled", 2024-03-18T23:19:05.952 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:05.952 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.952 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:05.952 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-18T23:19:05.952 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.952 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.952 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.952 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.952 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.953 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.953 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.953 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.953 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-18T23:19:05.953 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-18T23:19:05.953 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:05.953 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.953 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.953 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:05.954 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.954 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.954 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-18T23:19:05.954 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-18T23:19:05.954 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-18T23:19:05.954 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-18T23:19:05.954 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-18T23:19:05.954 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-18T23:19:05.955 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-18T23:19:05.955 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-18T23:19:05.955 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.955 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.955 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.955 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.955 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.955 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-18T23:19:05.955 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-18T23:19:05.956 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:05.956 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.956 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.956 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:05.956 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.956 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.956 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.956 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.957 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.957 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.957 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.957 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.957 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-18T23:19:05.957 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-18T23:19:05.957 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:05.957 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.957 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.958 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-18T23:19:05.958 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.958 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.958 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-18T23:19:05.958 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-18T23:19:05.958 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-18T23:19:05.958 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-18T23:19:05.958 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-18T23:19:05.959 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-18T23:19:05.959 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-18T23:19:05.959 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-18T23:19:05.959 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.959 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.959 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.959 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.959 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.959 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-18T23:19:05.960 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-18T23:19:05.960 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:05.960 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.960 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.960 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:05.960 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.960 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.960 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.961 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.961 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.961 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.961 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.961 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.961 INFO:teuthology.orchestra.run.smithi049.stdout: "max_completed_events": { 2024-03-18T23:19:05.961 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "max_completed_events", 2024-03-18T23:19:05.961 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-18T23:19:05.961 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.962 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.962 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "50", 2024-03-18T23:19:05.962 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.962 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.962 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.962 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "number of past completed events to remember", 2024-03-18T23:19:05.962 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.962 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.962 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.963 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.963 INFO:teuthology.orchestra.run.smithi049.stdout: "sleep_interval": { 2024-03-18T23:19:05.963 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "sleep_interval", 2024-03-18T23:19:05.963 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-03-18T23:19:05.963 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.963 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.963 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "5", 2024-03-18T23:19:05.963 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.964 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.964 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.964 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "how long the module is going to sleep", 2024-03-18T23:19:05.964 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.964 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.964 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.964 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:19:05.964 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:19:05.964 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.965 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-18T23:19:05.965 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "prometheus", 2024-03-18T23:19:05.965 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-18T23:19:05.965 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-18T23:19:05.965 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-18T23:19:05.965 INFO:teuthology.orchestra.run.smithi049.stdout: "cache": { 2024-03-18T23:19:05.965 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "cache", 2024-03-18T23:19:05.965 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:05.965 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.966 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:05.966 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-18T23:19:05.966 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.966 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.966 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.966 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.966 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.966 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.966 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.967 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.967 INFO:teuthology.orchestra.run.smithi049.stdout: "exclude_perf_counters": { 2024-03-18T23:19:05.967 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "exclude_perf_counters", 2024-03-18T23:19:05.967 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:05.967 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.967 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.967 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-18T23:19:05.967 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.968 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.968 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.968 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-03-18T23:19:05.968 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-03-18T23:19:05.968 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.968 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.968 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.968 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-18T23:19:05.969 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-18T23:19:05.969 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:05.969 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.969 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.969 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:05.969 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.969 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.969 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-18T23:19:05.969 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-18T23:19:05.970 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-18T23:19:05.970 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-18T23:19:05.970 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-18T23:19:05.970 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-18T23:19:05.970 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-18T23:19:05.970 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-18T23:19:05.970 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.970 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.971 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.971 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.971 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.971 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-18T23:19:05.971 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-18T23:19:05.971 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:05.971 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.971 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.971 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:05.972 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.972 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.972 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.972 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.972 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.972 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.972 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.972 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.972 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-18T23:19:05.973 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-18T23:19:05.973 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:05.973 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.973 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.973 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-18T23:19:05.973 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.973 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.973 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-18T23:19:05.974 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-18T23:19:05.974 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-18T23:19:05.974 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-18T23:19:05.974 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-18T23:19:05.974 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-18T23:19:05.974 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-18T23:19:05.974 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-18T23:19:05.974 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.974 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.975 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.975 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.975 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.975 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-18T23:19:05.975 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-18T23:19:05.975 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:05.975 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.975 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.976 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:05.976 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.976 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.976 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.976 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.976 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.976 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.976 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.976 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.977 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_stats_pools": { 2024-03-18T23:19:05.977 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "rbd_stats_pools", 2024-03-18T23:19:05.977 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:05.977 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.977 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:05.977 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:05.977 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.977 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.977 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.978 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.978 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.978 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.978 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.978 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.978 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_stats_pools_refresh_interval": { 2024-03-18T23:19:05.978 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-03-18T23:19:05.978 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-18T23:19:05.978 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.979 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:05.979 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "300", 2024-03-18T23:19:05.979 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.979 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.979 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.979 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.979 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.979 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.980 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.980 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.980 INFO:teuthology.orchestra.run.smithi049.stdout: "scrape_interval": { 2024-03-18T23:19:05.980 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "scrape_interval", 2024-03-18T23:19:05.980 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "float", 2024-03-18T23:19:05.980 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.980 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:05.980 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "15.0", 2024-03-18T23:19:05.980 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.981 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.981 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.981 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.981 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.981 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.981 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.981 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.981 INFO:teuthology.orchestra.run.smithi049.stdout: "server_addr": { 2024-03-18T23:19:05.981 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "server_addr", 2024-03-18T23:19:05.982 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:05.982 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.982 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:05.982 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "::", 2024-03-18T23:19:05.982 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.982 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.982 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.982 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-03-18T23:19:05.982 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.983 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.983 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.983 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.983 INFO:teuthology.orchestra.run.smithi049.stdout: "server_port": { 2024-03-18T23:19:05.983 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "server_port", 2024-03-18T23:19:05.983 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-18T23:19:05.983 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.983 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.983 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "9283", 2024-03-18T23:19:05.984 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.984 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.984 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.984 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-03-18T23:19:05.984 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.984 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.984 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.984 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.984 INFO:teuthology.orchestra.run.smithi049.stdout: "stale_cache_strategy": { 2024-03-18T23:19:05.985 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "stale_cache_strategy", 2024-03-18T23:19:05.985 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:05.985 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.985 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:05.985 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "log", 2024-03-18T23:19:05.985 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.985 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.985 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.985 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.986 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.986 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.986 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.986 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.986 INFO:teuthology.orchestra.run.smithi049.stdout: "standby_behaviour": { 2024-03-18T23:19:05.986 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "standby_behaviour", 2024-03-18T23:19:05.986 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:05.986 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.987 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.987 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "default", 2024-03-18T23:19:05.987 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.987 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.987 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-18T23:19:05.987 INFO:teuthology.orchestra.run.smithi049.stdout: "default", 2024-03-18T23:19:05.987 INFO:teuthology.orchestra.run.smithi049.stdout: "error" 2024-03-18T23:19:05.987 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-18T23:19:05.988 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.988 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.988 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.988 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.988 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.988 INFO:teuthology.orchestra.run.smithi049.stdout: "standby_error_status_code": { 2024-03-18T23:19:05.988 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "standby_error_status_code", 2024-03-18T23:19:05.988 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-18T23:19:05.988 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.989 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.989 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "500", 2024-03-18T23:19:05.989 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "400", 2024-03-18T23:19:05.989 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "599", 2024-03-18T23:19:05.989 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.989 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.989 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.989 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.989 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.990 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:19:05.990 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:19:05.990 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.990 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-18T23:19:05.990 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "rbd_support", 2024-03-18T23:19:05.990 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-18T23:19:05.990 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-18T23:19:05.990 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-18T23:19:05.991 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-18T23:19:05.991 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-18T23:19:05.991 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:05.991 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.991 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.991 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:05.991 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.991 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.991 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-18T23:19:05.992 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-18T23:19:05.992 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-18T23:19:05.992 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-18T23:19:05.992 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-18T23:19:05.992 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-18T23:19:05.992 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-18T23:19:05.992 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-18T23:19:05.992 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.992 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.993 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.993 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.993 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.993 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-18T23:19:05.993 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-18T23:19:05.993 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:05.993 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.993 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.993 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:05.994 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.994 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.994 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.994 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.994 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.994 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.994 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.994 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.995 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-18T23:19:05.995 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-18T23:19:05.995 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:05.995 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.995 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.995 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-18T23:19:05.995 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.995 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.995 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-18T23:19:05.996 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-18T23:19:05.996 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-18T23:19:05.996 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-18T23:19:05.996 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-18T23:19:05.996 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-18T23:19:05.996 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-18T23:19:05.996 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-18T23:19:05.996 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.996 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.997 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.997 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.997 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.997 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-18T23:19:05.997 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-18T23:19:05.997 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:05.997 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.997 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:05.997 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:05.998 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.998 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.998 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:05.998 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:05.998 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:05.998 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:05.998 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:05.998 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:05.999 INFO:teuthology.orchestra.run.smithi049.stdout: "max_concurrent_snap_create": { 2024-03-18T23:19:05.999 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "max_concurrent_snap_create", 2024-03-18T23:19:05.999 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-18T23:19:05.999 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:05.999 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:05.999 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "10", 2024-03-18T23:19:05.999 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:05.999 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:05.999 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:06.000 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:06.000 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:06.000 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:06.000 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:06.000 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:06.000 INFO:teuthology.orchestra.run.smithi049.stdout: "mirror_snapshot_schedule": { 2024-03-18T23:19:06.000 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "mirror_snapshot_schedule", 2024-03-18T23:19:06.000 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:06.001 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:06.001 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:06.001 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:06.001 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:06.001 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:06.001 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:06.001 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:06.001 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:06.001 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:06.001 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:06.002 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:06.002 INFO:teuthology.orchestra.run.smithi049.stdout: "trash_purge_schedule": { 2024-03-18T23:19:06.002 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "trash_purge_schedule", 2024-03-18T23:19:06.002 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:06.002 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:06.002 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:06.002 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:06.002 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:06.003 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:06.003 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:06.003 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:06.003 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:06.003 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:06.003 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:06.003 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:19:06.003 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:19:06.004 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:06.004 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-18T23:19:06.004 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "restful", 2024-03-18T23:19:06.004 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-18T23:19:06.004 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-18T23:19:06.004 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-18T23:19:06.004 INFO:teuthology.orchestra.run.smithi049.stdout: "enable_auth": { 2024-03-18T23:19:06.004 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "enable_auth", 2024-03-18T23:19:06.004 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:06.005 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:06.005 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:06.005 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-18T23:19:06.005 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:06.005 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:06.005 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:06.005 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:06.005 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:06.006 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:06.006 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:06.006 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:06.006 INFO:teuthology.orchestra.run.smithi049.stdout: "key_file": { 2024-03-18T23:19:06.006 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "key_file", 2024-03-18T23:19:06.006 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:06.006 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:06.006 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:06.007 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:06.007 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:06.007 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:06.007 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:06.007 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:06.007 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:06.007 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:06.007 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:06.007 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:06.008 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-18T23:19:06.008 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-18T23:19:06.008 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:06.008 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:06.008 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:06.008 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:06.008 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:06.008 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:06.009 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-18T23:19:06.009 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-18T23:19:06.009 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-18T23:19:06.009 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-18T23:19:06.009 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-18T23:19:06.009 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-18T23:19:06.009 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-18T23:19:06.009 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-18T23:19:06.009 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:06.010 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:06.010 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:06.010 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:06.010 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:06.010 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-18T23:19:06.010 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-18T23:19:06.010 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:06.010 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:06.011 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:06.011 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:06.011 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:06.011 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:06.011 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:06.011 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:06.011 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:06.011 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:06.011 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:06.012 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:06.012 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-18T23:19:06.012 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-18T23:19:06.012 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:06.012 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:06.012 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:06.012 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-18T23:19:06.012 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:06.012 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:06.013 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-18T23:19:06.013 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-18T23:19:06.013 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-18T23:19:06.013 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-18T23:19:06.013 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-18T23:19:06.013 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-18T23:19:06.013 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-18T23:19:06.013 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-18T23:19:06.014 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:06.014 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:06.014 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:06.014 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:06.014 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:06.014 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-18T23:19:06.014 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-18T23:19:06.014 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:06.014 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:06.015 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:06.015 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:06.015 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:06.015 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:06.015 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:06.015 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:06.015 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:06.015 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:06.015 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:06.016 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:06.016 INFO:teuthology.orchestra.run.smithi049.stdout: "server_addr": { 2024-03-18T23:19:06.016 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "server_addr", 2024-03-18T23:19:06.016 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:06.016 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:06.016 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:06.016 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:06.016 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:06.017 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:06.017 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:06.017 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:06.017 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:06.017 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:06.017 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:06.017 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:06.017 INFO:teuthology.orchestra.run.smithi049.stdout: "server_port": { 2024-03-18T23:19:06.017 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "server_port", 2024-03-18T23:19:06.018 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:06.018 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:06.018 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:06.018 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:06.018 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:06.018 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:06.018 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:06.018 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:06.019 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:06.019 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:06.019 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:06.019 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:19:06.019 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:19:06.019 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:06.019 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-18T23:19:06.019 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "rgw", 2024-03-18T23:19:06.019 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-18T23:19:06.020 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-18T23:19:06.020 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-18T23:19:06.020 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-18T23:19:06.020 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-18T23:19:06.020 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:06.020 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:06.020 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:06.020 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:06.020 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:06.021 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:06.021 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-18T23:19:06.021 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-18T23:19:06.021 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-18T23:19:06.021 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-18T23:19:06.021 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-18T23:19:06.021 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-18T23:19:06.021 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-18T23:19:06.022 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-18T23:19:06.022 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:06.022 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:06.022 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:06.022 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:06.022 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:06.022 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-18T23:19:06.022 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-18T23:19:06.023 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:06.023 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:06.023 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:06.023 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:06.023 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:06.023 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:06.023 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:06.023 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:06.023 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:06.024 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:06.024 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:06.024 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:06.024 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-18T23:19:06.024 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-18T23:19:06.024 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:06.024 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:06.024 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:06.024 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-18T23:19:06.025 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:06.025 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:06.025 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-18T23:19:06.025 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-18T23:19:06.025 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-18T23:19:06.025 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-18T23:19:06.025 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-18T23:19:06.025 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-18T23:19:06.025 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-18T23:19:06.026 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-18T23:19:06.026 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:06.026 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:06.026 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:06.026 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:06.026 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:06.026 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-18T23:19:06.026 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-18T23:19:06.027 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:06.027 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:06.027 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:06.027 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:06.027 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:06.027 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:06.027 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:06.027 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:06.027 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:06.028 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:06.028 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:06.028 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:19:06.028 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:19:06.028 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:06.028 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-18T23:19:06.028 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "rook", 2024-03-18T23:19:06.028 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-18T23:19:06.028 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-18T23:19:06.029 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-18T23:19:06.029 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-18T23:19:06.029 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-18T23:19:06.029 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:06.029 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:06.029 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:06.029 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:06.029 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:06.029 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:06.030 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-18T23:19:06.030 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-18T23:19:06.030 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-18T23:19:06.030 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-18T23:19:06.030 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-18T23:19:06.030 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-18T23:19:06.030 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-18T23:19:06.030 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-18T23:19:06.031 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:06.031 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:06.031 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:06.031 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:06.031 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:06.031 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-18T23:19:06.031 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-18T23:19:06.031 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:06.031 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:06.032 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:06.032 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:06.032 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:06.032 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:06.032 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:06.032 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:06.032 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:06.032 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:06.032 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:06.033 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:06.033 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-18T23:19:06.033 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-18T23:19:06.033 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:06.033 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:06.033 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:06.033 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-18T23:19:06.033 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:06.033 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:06.034 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-18T23:19:06.034 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-18T23:19:06.034 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-18T23:19:06.034 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-18T23:19:06.034 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-18T23:19:06.034 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-18T23:19:06.034 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-18T23:19:06.034 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-18T23:19:06.035 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:06.035 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:06.035 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:06.035 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:06.035 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:06.035 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-18T23:19:06.035 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-18T23:19:06.035 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:06.035 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:06.036 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:06.036 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:06.036 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:06.036 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:06.036 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:06.036 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:06.036 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:06.036 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:06.036 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:06.037 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:06.037 INFO:teuthology.orchestra.run.smithi049.stdout: "storage_class": { 2024-03-18T23:19:06.037 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "storage_class", 2024-03-18T23:19:06.037 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:06.037 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:06.037 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:06.037 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "local", 2024-03-18T23:19:06.037 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:06.037 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:06.038 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:06.038 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-03-18T23:19:06.038 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:06.038 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:06.038 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:06.038 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:19:06.038 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:19:06.038 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:06.039 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-18T23:19:06.039 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "selftest", 2024-03-18T23:19:06.039 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-18T23:19:06.039 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-18T23:19:06.039 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-18T23:19:06.039 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-18T23:19:06.039 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-18T23:19:06.039 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:06.039 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:06.040 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:06.040 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:06.040 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:06.040 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:06.040 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-18T23:19:06.040 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-18T23:19:06.040 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-18T23:19:06.040 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-18T23:19:06.040 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-18T23:19:06.041 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-18T23:19:06.041 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-18T23:19:06.041 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-18T23:19:06.041 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:06.041 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:06.041 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:06.041 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:06.041 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:06.041 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-18T23:19:06.042 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-18T23:19:06.042 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:06.042 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:06.042 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:06.042 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:06.042 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:06.042 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:06.042 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:06.043 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:06.043 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:06.043 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:06.043 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:06.043 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:06.043 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-18T23:19:06.043 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-18T23:19:06.043 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:06.043 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:06.044 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:06.044 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-18T23:19:06.044 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:06.044 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:06.044 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-18T23:19:06.044 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-18T23:19:06.044 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-18T23:19:06.044 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-18T23:19:06.044 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-18T23:19:06.045 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-18T23:19:06.045 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-18T23:19:06.045 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-18T23:19:06.045 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:06.045 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:06.045 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:06.045 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:06.045 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:06.045 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-18T23:19:06.046 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-18T23:19:06.046 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:06.046 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:06.046 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:06.046 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:06.046 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:06.046 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:06.047 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:06.047 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:06.047 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:06.047 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:06.047 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:06.047 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:06.047 INFO:teuthology.orchestra.run.smithi049.stdout: "roption1": { 2024-03-18T23:19:06.047 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "roption1", 2024-03-18T23:19:06.047 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:06.048 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:06.048 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:06.048 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:06.048 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:06.048 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:06.048 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:06.048 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:06.048 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:06.049 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:06.049 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:06.049 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:06.049 INFO:teuthology.orchestra.run.smithi049.stdout: "roption2": { 2024-03-18T23:19:06.049 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "roption2", 2024-03-18T23:19:06.049 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:06.049 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:06.049 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:06.049 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "xyz", 2024-03-18T23:19:06.050 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:06.050 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:06.050 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:06.050 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:06.050 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:06.050 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:06.050 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:06.050 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:06.051 INFO:teuthology.orchestra.run.smithi049.stdout: "rwoption1": { 2024-03-18T23:19:06.051 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "rwoption1", 2024-03-18T23:19:06.051 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:06.051 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:06.051 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:06.051 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:06.051 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:06.051 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:06.051 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:06.052 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:06.052 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:06.052 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:06.052 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:06.052 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:06.052 INFO:teuthology.orchestra.run.smithi049.stdout: "rwoption2": { 2024-03-18T23:19:06.052 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "rwoption2", 2024-03-18T23:19:06.052 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-18T23:19:06.052 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:06.053 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:06.053 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:06.053 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:06.053 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:06.053 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:06.053 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:06.053 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:06.053 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:06.054 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:06.054 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:06.054 INFO:teuthology.orchestra.run.smithi049.stdout: "rwoption3": { 2024-03-18T23:19:06.054 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "rwoption3", 2024-03-18T23:19:06.054 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "float", 2024-03-18T23:19:06.054 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:06.054 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:06.054 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:06.055 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:06.055 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:06.055 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:06.055 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:06.055 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:06.055 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:06.055 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:06.055 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:06.056 INFO:teuthology.orchestra.run.smithi049.stdout: "rwoption4": { 2024-03-18T23:19:06.056 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "rwoption4", 2024-03-18T23:19:06.056 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:06.056 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:06.056 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:06.056 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:06.056 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:06.056 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:06.056 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:06.057 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:06.057 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:06.057 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:06.057 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:06.057 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:06.057 INFO:teuthology.orchestra.run.smithi049.stdout: "rwoption5": { 2024-03-18T23:19:06.057 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "rwoption5", 2024-03-18T23:19:06.057 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:06.058 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:06.058 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:06.058 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:06.058 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:06.058 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:06.058 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:06.058 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:06.058 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:06.059 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:06.059 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:06.059 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:06.059 INFO:teuthology.orchestra.run.smithi049.stdout: "rwoption6": { 2024-03-18T23:19:06.059 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "rwoption6", 2024-03-18T23:19:06.059 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:06.059 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:06.059 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:06.059 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-18T23:19:06.060 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:06.060 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:06.060 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:06.060 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:06.060 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:06.060 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:06.060 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:06.060 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:06.061 INFO:teuthology.orchestra.run.smithi049.stdout: "rwoption7": { 2024-03-18T23:19:06.061 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "rwoption7", 2024-03-18T23:19:06.061 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-18T23:19:06.061 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:06.061 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:06.061 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:06.061 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "1", 2024-03-18T23:19:06.061 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "42", 2024-03-18T23:19:06.061 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:06.062 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:06.062 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:06.062 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:06.062 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:06.062 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:06.062 INFO:teuthology.orchestra.run.smithi049.stdout: "testkey": { 2024-03-18T23:19:06.062 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "testkey", 2024-03-18T23:19:06.062 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:06.063 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:06.063 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:06.063 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:06.063 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:06.063 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:06.063 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:06.063 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:06.063 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:06.063 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:06.064 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:06.064 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:06.064 INFO:teuthology.orchestra.run.smithi049.stdout: "testlkey": { 2024-03-18T23:19:06.064 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "testlkey", 2024-03-18T23:19:06.064 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:06.064 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:06.064 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:06.064 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:06.065 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:06.065 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:06.065 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:06.065 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:06.065 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:06.065 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:06.065 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:06.065 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:06.065 INFO:teuthology.orchestra.run.smithi049.stdout: "testnewline": { 2024-03-18T23:19:06.066 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "testnewline", 2024-03-18T23:19:06.066 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:06.066 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:06.066 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:06.066 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:06.066 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:06.066 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:06.066 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:06.067 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:06.067 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:06.067 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:06.067 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:06.067 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:19:06.067 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:19:06.067 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:06.067 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-18T23:19:06.067 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "snap_schedule", 2024-03-18T23:19:06.068 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-18T23:19:06.068 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-18T23:19:06.068 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-18T23:19:06.068 INFO:teuthology.orchestra.run.smithi049.stdout: "allow_m_granularity": { 2024-03-18T23:19:06.068 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "allow_m_granularity", 2024-03-18T23:19:06.068 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:06.068 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:06.068 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:06.069 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:06.069 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:06.069 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:06.069 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:06.069 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "allow minute scheduled snapshots", 2024-03-18T23:19:06.069 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:06.069 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:06.069 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:06.069 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:06.070 INFO:teuthology.orchestra.run.smithi049.stdout: "dump_on_update": { 2024-03-18T23:19:06.070 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "dump_on_update", 2024-03-18T23:19:06.070 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:06.070 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:06.070 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:06.070 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:06.070 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:06.070 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:06.070 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:06.071 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "dump database to debug log on update", 2024-03-18T23:19:06.071 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:06.071 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:06.071 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:06.071 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:06.071 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-18T23:19:06.071 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-18T23:19:06.071 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:06.072 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:06.072 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:06.072 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:06.072 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:06.072 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:06.072 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-18T23:19:06.072 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-18T23:19:06.072 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-18T23:19:06.072 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-18T23:19:06.073 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-18T23:19:06.073 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-18T23:19:06.073 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-18T23:19:06.073 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-18T23:19:06.073 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:06.073 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:06.073 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:06.073 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:06.073 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:06.074 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-18T23:19:06.074 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-18T23:19:06.074 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:06.074 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:06.074 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:06.074 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:06.074 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:06.074 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:06.075 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:06.075 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:06.075 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:06.075 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:06.075 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:06.075 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:06.075 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-18T23:19:06.075 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-18T23:19:06.075 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:06.076 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:06.076 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:06.076 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-18T23:19:06.076 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:06.076 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:06.076 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-18T23:19:06.076 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-18T23:19:06.076 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-18T23:19:06.076 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-18T23:19:06.077 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-18T23:19:06.077 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-18T23:19:06.077 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-18T23:19:06.077 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-18T23:19:06.077 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:06.077 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:06.077 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:06.077 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:06.078 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:06.078 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-18T23:19:06.078 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-18T23:19:06.078 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:06.078 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:06.078 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:06.078 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:06.078 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:06.079 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:06.079 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:06.079 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:06.079 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:06.079 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:06.079 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:06.079 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:19:06.079 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:19:06.079 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:06.080 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-18T23:19:06.080 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "stats", 2024-03-18T23:19:06.080 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-18T23:19:06.080 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-18T23:19:06.080 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-18T23:19:06.080 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-18T23:19:06.080 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-18T23:19:06.080 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:06.080 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:06.081 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:06.081 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:06.081 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:06.081 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:06.081 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-18T23:19:06.081 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-18T23:19:06.081 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-18T23:19:06.081 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-18T23:19:06.082 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-18T23:19:06.082 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-18T23:19:06.082 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-18T23:19:06.082 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-18T23:19:06.082 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:06.082 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:06.082 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:06.082 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:06.082 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:06.083 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-18T23:19:06.083 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-18T23:19:06.083 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:06.083 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:06.083 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:06.083 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:06.083 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:06.083 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:06.083 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:06.084 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:06.084 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:06.084 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:06.084 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:06.084 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:06.084 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-18T23:19:06.084 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-18T23:19:06.084 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:06.085 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:06.085 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:06.085 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-18T23:19:06.085 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:06.085 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:06.085 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-18T23:19:06.085 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-18T23:19:06.085 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-18T23:19:06.085 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-18T23:19:06.085 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-18T23:19:06.086 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-18T23:19:06.086 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-18T23:19:06.086 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-18T23:19:06.086 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:06.086 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:06.086 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:06.086 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:06.086 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:06.087 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-18T23:19:06.087 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-18T23:19:06.087 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:06.087 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:06.087 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:06.087 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:06.087 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:06.087 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:06.088 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:06.088 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:06.088 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:06.088 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:06.088 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:06.088 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:19:06.088 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:19:06.088 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:06.088 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-18T23:19:06.088 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "status", 2024-03-18T23:19:06.089 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-18T23:19:06.089 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-18T23:19:06.089 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-18T23:19:06.089 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-18T23:19:06.089 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-18T23:19:06.089 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:06.089 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:06.089 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:06.090 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:06.090 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:06.090 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:06.090 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-18T23:19:06.090 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-18T23:19:06.090 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-18T23:19:06.090 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-18T23:19:06.090 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-18T23:19:06.091 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-18T23:19:06.091 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-18T23:19:06.091 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-18T23:19:06.091 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:06.091 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:06.091 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:06.091 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:06.091 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:06.091 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-18T23:19:06.092 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-18T23:19:06.092 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:06.092 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:06.092 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:06.092 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:06.092 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:06.092 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:06.092 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:06.092 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:06.093 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:06.093 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:06.093 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:06.093 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:06.093 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-18T23:19:06.093 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-18T23:19:06.093 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:06.093 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:06.093 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:06.094 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-18T23:19:06.094 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:06.094 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:06.094 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-18T23:19:06.094 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-18T23:19:06.094 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-18T23:19:06.094 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-18T23:19:06.094 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-18T23:19:06.095 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-18T23:19:06.095 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-18T23:19:06.095 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-18T23:19:06.095 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:06.095 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:06.095 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:06.095 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:06.095 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:06.095 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-18T23:19:06.096 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-18T23:19:06.096 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:06.096 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:06.096 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:06.096 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:06.096 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:06.096 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:06.096 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:06.097 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:06.097 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:06.097 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:06.097 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:06.097 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:19:06.097 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:19:06.097 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:06.097 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-18T23:19:06.097 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "telegraf", 2024-03-18T23:19:06.098 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-18T23:19:06.098 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-18T23:19:06.098 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-18T23:19:06.098 INFO:teuthology.orchestra.run.smithi049.stdout: "address": { 2024-03-18T23:19:06.098 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "address", 2024-03-18T23:19:06.098 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:06.098 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:06.098 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:06.099 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-03-18T23:19:06.099 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:06.099 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:06.099 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:06.099 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:06.099 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:06.099 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:06.099 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:06.099 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:06.100 INFO:teuthology.orchestra.run.smithi049.stdout: "interval": { 2024-03-18T23:19:06.100 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "interval", 2024-03-18T23:19:06.100 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-03-18T23:19:06.100 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:06.100 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:06.100 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "15", 2024-03-18T23:19:06.100 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:06.100 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:06.100 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:06.101 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:06.101 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:06.101 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:06.101 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:06.101 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:06.101 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-18T23:19:06.101 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-18T23:19:06.101 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:06.102 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:06.102 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:06.102 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:06.102 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:06.102 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:06.102 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-18T23:19:06.102 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-18T23:19:06.102 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-18T23:19:06.102 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-18T23:19:06.103 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-18T23:19:06.103 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-18T23:19:06.103 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-18T23:19:06.103 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-18T23:19:06.103 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:06.103 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:06.103 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:06.103 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:06.103 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:06.104 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-18T23:19:06.104 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-18T23:19:06.104 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:06.104 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:06.104 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:06.104 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:06.104 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:06.104 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:06.105 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:06.105 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:06.105 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:06.105 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:06.105 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:06.105 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:06.105 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-18T23:19:06.105 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-18T23:19:06.105 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:06.106 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:06.106 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:06.106 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-18T23:19:06.106 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:06.106 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:06.106 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-18T23:19:06.106 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-18T23:19:06.106 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-18T23:19:06.107 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-18T23:19:06.107 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-18T23:19:06.107 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-18T23:19:06.107 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-18T23:19:06.107 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-18T23:19:06.107 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:06.107 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:06.107 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:06.107 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:06.108 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:06.108 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-18T23:19:06.108 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-18T23:19:06.108 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:06.108 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:06.108 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:06.108 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:06.108 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:06.109 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:06.109 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:06.109 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:06.109 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:06.109 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:06.109 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:06.109 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:19:06.109 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:19:06.109 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:06.110 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-18T23:19:06.110 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "telemetry", 2024-03-18T23:19:06.110 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-18T23:19:06.110 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-18T23:19:06.110 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-18T23:19:06.110 INFO:teuthology.orchestra.run.smithi049.stdout: "channel_basic": { 2024-03-18T23:19:06.110 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "channel_basic", 2024-03-18T23:19:06.110 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:06.111 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:06.111 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:06.111 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-18T23:19:06.111 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:06.111 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:06.111 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:06.111 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Share basic cluster information (size, version)", 2024-03-18T23:19:06.111 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:06.111 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:06.112 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:06.112 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:06.112 INFO:teuthology.orchestra.run.smithi049.stdout: "channel_crash": { 2024-03-18T23:19:06.112 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "channel_crash", 2024-03-18T23:19:06.112 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:06.112 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:06.112 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:06.112 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-18T23:19:06.112 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:06.113 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:06.113 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:06.113 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-03-18T23:19:06.113 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:06.113 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:06.113 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:06.113 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:06.113 INFO:teuthology.orchestra.run.smithi049.stdout: "channel_device": { 2024-03-18T23:19:06.113 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "channel_device", 2024-03-18T23:19:06.114 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:06.114 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:06.114 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:06.114 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-18T23:19:06.114 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:06.114 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:06.114 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:06.114 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-03-18T23:19:06.115 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:06.115 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:06.115 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:06.115 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:06.115 INFO:teuthology.orchestra.run.smithi049.stdout: "channel_ident": { 2024-03-18T23:19:06.115 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "channel_ident", 2024-03-18T23:19:06.115 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:06.115 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:06.115 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:06.116 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:06.116 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:06.116 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:06.116 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:06.116 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-03-18T23:19:06.116 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:06.116 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:06.116 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:06.116 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:06.117 INFO:teuthology.orchestra.run.smithi049.stdout: "channel_perf": { 2024-03-18T23:19:06.117 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "channel_perf", 2024-03-18T23:19:06.117 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:06.117 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:06.117 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:06.117 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:06.117 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:06.117 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:06.117 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:06.118 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Share various performance metrics of a cluster", 2024-03-18T23:19:06.118 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:06.118 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:06.118 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:06.118 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:06.118 INFO:teuthology.orchestra.run.smithi049.stdout: "contact": { 2024-03-18T23:19:06.118 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "contact", 2024-03-18T23:19:06.118 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:06.118 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:06.119 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:06.119 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:06.119 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:06.119 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:06.119 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:06.119 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:06.119 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:06.119 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:06.119 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:06.120 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:06.120 INFO:teuthology.orchestra.run.smithi049.stdout: "description": { 2024-03-18T23:19:06.120 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "description", 2024-03-18T23:19:06.120 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:06.120 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:06.120 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:06.120 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:06.120 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:06.120 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:06.120 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:06.121 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:06.121 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:06.121 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:06.121 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:06.121 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:06.121 INFO:teuthology.orchestra.run.smithi049.stdout: "device_url": { 2024-03-18T23:19:06.121 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "device_url", 2024-03-18T23:19:06.121 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:06.121 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:06.122 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:06.122 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-03-18T23:19:06.122 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:06.122 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:06.122 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:06.122 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:06.122 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:06.122 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:06.122 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:06.123 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:06.123 INFO:teuthology.orchestra.run.smithi049.stdout: "enabled": { 2024-03-18T23:19:06.123 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "enabled", 2024-03-18T23:19:06.123 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:06.123 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:06.123 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:06.123 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:06.123 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:06.123 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:06.124 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:06.124 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:06.124 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:06.124 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:06.124 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:06.124 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:06.124 INFO:teuthology.orchestra.run.smithi049.stdout: "interval": { 2024-03-18T23:19:06.124 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "interval", 2024-03-18T23:19:06.124 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-18T23:19:06.125 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:06.125 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:06.125 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "24", 2024-03-18T23:19:06.125 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "8", 2024-03-18T23:19:06.125 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:06.125 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:06.125 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:06.125 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:06.125 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:06.126 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:06.126 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:06.126 INFO:teuthology.orchestra.run.smithi049.stdout: "last_opt_revision": { 2024-03-18T23:19:06.126 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "last_opt_revision", 2024-03-18T23:19:06.126 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-18T23:19:06.126 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:06.126 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:06.126 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "1", 2024-03-18T23:19:06.127 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:06.127 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:06.127 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:06.127 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:06.127 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:06.127 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:06.127 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:06.127 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:06.127 INFO:teuthology.orchestra.run.smithi049.stdout: "leaderboard": { 2024-03-18T23:19:06.128 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "leaderboard", 2024-03-18T23:19:06.128 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:06.128 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:06.128 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:06.128 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:06.128 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:06.128 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:06.128 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:06.129 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:06.129 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:06.129 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:06.129 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:06.129 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:06.129 INFO:teuthology.orchestra.run.smithi049.stdout: "leaderboard_description": { 2024-03-18T23:19:06.129 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "leaderboard_description", 2024-03-18T23:19:06.129 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:06.129 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:06.130 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:06.130 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:06.130 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:06.130 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:06.130 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:06.130 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:06.130 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:06.130 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:06.131 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:06.131 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:06.131 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-18T23:19:06.131 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-18T23:19:06.131 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:06.131 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:06.131 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:06.131 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:06.132 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:06.132 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:06.132 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-18T23:19:06.132 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-18T23:19:06.132 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-18T23:19:06.132 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-18T23:19:06.132 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-18T23:19:06.132 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-18T23:19:06.133 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-18T23:19:06.133 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-18T23:19:06.133 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:06.133 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:06.133 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:06.133 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:06.133 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:06.133 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-18T23:19:06.133 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-18T23:19:06.134 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:06.134 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:06.134 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:06.134 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:06.134 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:06.134 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:06.134 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:06.134 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:06.135 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:06.135 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:06.135 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:06.135 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:06.135 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-18T23:19:06.135 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-18T23:19:06.135 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:06.135 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:06.135 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:06.136 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-18T23:19:06.136 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:06.136 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:06.136 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-18T23:19:06.136 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-18T23:19:06.136 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-18T23:19:06.136 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-18T23:19:06.136 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-18T23:19:06.137 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-18T23:19:06.137 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-18T23:19:06.137 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-18T23:19:06.137 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:06.137 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:06.137 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:06.137 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:06.137 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:06.137 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-18T23:19:06.138 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-18T23:19:06.138 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:06.138 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:06.138 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:06.138 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:06.138 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:06.138 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:06.138 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:06.139 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:06.139 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:06.139 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:06.139 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:06.139 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:06.139 INFO:teuthology.orchestra.run.smithi049.stdout: "organization": { 2024-03-18T23:19:06.139 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "organization", 2024-03-18T23:19:06.139 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:06.139 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:06.140 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:06.140 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:06.140 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:06.140 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:06.140 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:06.140 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:06.140 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:06.140 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:06.140 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:06.141 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:06.141 INFO:teuthology.orchestra.run.smithi049.stdout: "proxy": { 2024-03-18T23:19:06.141 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "proxy", 2024-03-18T23:19:06.141 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:06.141 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:06.141 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:06.141 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:06.141 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:06.141 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:06.142 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:06.142 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:06.142 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:06.142 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:06.142 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:06.142 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:06.142 INFO:teuthology.orchestra.run.smithi049.stdout: "url": { 2024-03-18T23:19:06.142 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "url", 2024-03-18T23:19:06.143 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:06.143 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:06.143 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:06.143 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-03-18T23:19:06.143 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:06.143 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:06.143 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:06.143 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:06.143 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:06.144 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:06.144 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:06.144 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:19:06.144 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:19:06.144 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:06.144 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-18T23:19:06.144 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "test_orchestrator", 2024-03-18T23:19:06.144 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-18T23:19:06.144 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-18T23:19:06.145 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-18T23:19:06.145 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-18T23:19:06.145 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-18T23:19:06.145 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:06.145 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:06.145 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:06.145 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:06.145 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:06.145 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:06.146 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-18T23:19:06.146 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-18T23:19:06.146 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-18T23:19:06.146 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-18T23:19:06.146 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-18T23:19:06.146 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-18T23:19:06.146 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-18T23:19:06.146 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-18T23:19:06.146 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:06.147 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:06.147 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:06.147 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:06.147 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:06.147 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-18T23:19:06.147 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-18T23:19:06.147 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:06.147 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:06.148 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:06.148 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:06.148 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:06.148 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:06.148 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:06.148 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:06.148 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:06.148 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:06.148 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:06.149 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:06.149 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-18T23:19:06.149 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-18T23:19:06.149 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:06.149 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:06.149 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:06.149 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-18T23:19:06.149 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:06.149 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:06.150 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-18T23:19:06.150 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-18T23:19:06.150 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-18T23:19:06.150 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-18T23:19:06.150 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-18T23:19:06.150 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-18T23:19:06.150 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-18T23:19:06.150 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-18T23:19:06.151 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:06.151 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:06.151 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:06.151 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:06.151 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:06.151 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-18T23:19:06.151 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-18T23:19:06.151 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:06.151 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:06.152 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:06.152 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:06.152 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:06.152 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:06.152 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:06.152 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:06.152 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:06.152 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:06.152 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:06.153 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:19:06.153 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:19:06.153 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:06.153 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-18T23:19:06.153 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "volumes", 2024-03-18T23:19:06.153 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-18T23:19:06.153 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-18T23:19:06.153 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-18T23:19:06.153 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-18T23:19:06.154 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-18T23:19:06.154 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:06.154 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:06.154 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:06.154 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:06.154 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:06.154 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:06.154 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-18T23:19:06.155 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-18T23:19:06.155 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-18T23:19:06.155 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-18T23:19:06.155 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-18T23:19:06.155 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-18T23:19:06.155 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-18T23:19:06.155 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-18T23:19:06.155 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:06.155 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:06.156 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:06.156 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:06.156 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:06.156 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-18T23:19:06.156 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-18T23:19:06.156 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:06.156 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:06.156 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:06.156 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:06.156 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:06.157 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:06.157 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:06.157 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:06.157 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:06.157 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:06.157 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:06.157 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:06.157 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-18T23:19:06.158 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-18T23:19:06.158 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:06.158 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:06.158 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:06.158 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-18T23:19:06.158 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:06.158 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:06.158 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-18T23:19:06.159 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-18T23:19:06.159 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-18T23:19:06.159 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-18T23:19:06.159 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-18T23:19:06.159 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-18T23:19:06.159 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-18T23:19:06.159 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-18T23:19:06.159 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:06.159 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:06.160 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:06.160 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:06.160 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:06.160 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-18T23:19:06.160 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-18T23:19:06.160 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:06.160 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:06.160 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:06.160 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:06.161 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:06.161 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:06.161 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:06.161 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:06.161 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:06.161 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:06.161 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:06.161 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:06.161 INFO:teuthology.orchestra.run.smithi049.stdout: "max_concurrent_clones": { 2024-03-18T23:19:06.162 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "max_concurrent_clones", 2024-03-18T23:19:06.162 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-18T23:19:06.162 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:06.162 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:06.162 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "4", 2024-03-18T23:19:06.162 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:06.162 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:06.162 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:06.162 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Number of asynchronous cloner threads", 2024-03-18T23:19:06.163 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:06.163 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:06.163 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:06.163 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:06.163 INFO:teuthology.orchestra.run.smithi049.stdout: "periodic_async_work": { 2024-03-18T23:19:06.163 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "periodic_async_work", 2024-03-18T23:19:06.163 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:06.163 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:06.163 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:06.164 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:06.164 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:06.164 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:06.164 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:06.164 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Periodically check for async work", 2024-03-18T23:19:06.164 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:06.164 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:06.164 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:06.164 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:06.165 INFO:teuthology.orchestra.run.smithi049.stdout: "snapshot_clone_delay": { 2024-03-18T23:19:06.165 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "snapshot_clone_delay", 2024-03-18T23:19:06.165 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-18T23:19:06.165 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:06.165 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:06.165 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "0", 2024-03-18T23:19:06.165 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:06.165 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:06.165 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:06.166 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-03-18T23:19:06.166 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:06.166 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:06.166 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:06.166 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:06.166 INFO:teuthology.orchestra.run.smithi049.stdout: "snapshot_clone_no_wait": { 2024-03-18T23:19:06.166 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "snapshot_clone_no_wait", 2024-03-18T23:19:06.166 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:06.167 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:06.167 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:06.167 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-18T23:19:06.167 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:06.167 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:06.167 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:06.167 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-03-18T23:19:06.167 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:06.167 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:06.168 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:06.168 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:19:06.168 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:19:06.168 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:06.168 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-18T23:19:06.168 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "zabbix", 2024-03-18T23:19:06.168 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-18T23:19:06.168 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-18T23:19:06.168 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-18T23:19:06.168 INFO:teuthology.orchestra.run.smithi049.stdout: "discovery_interval": { 2024-03-18T23:19:06.169 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "discovery_interval", 2024-03-18T23:19:06.169 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "uint", 2024-03-18T23:19:06.169 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:06.169 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:06.169 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "100", 2024-03-18T23:19:06.169 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:06.169 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:06.169 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:06.169 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:06.170 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:06.170 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:06.170 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:06.170 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:06.170 INFO:teuthology.orchestra.run.smithi049.stdout: "identifier": { 2024-03-18T23:19:06.170 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "identifier", 2024-03-18T23:19:06.170 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:06.170 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:06.170 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:06.171 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:06.171 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:06.171 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:06.171 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:06.171 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:06.171 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:06.171 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:06.171 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:06.172 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:06.172 INFO:teuthology.orchestra.run.smithi049.stdout: "interval": { 2024-03-18T23:19:06.172 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "interval", 2024-03-18T23:19:06.172 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-03-18T23:19:06.172 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:06.172 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:06.172 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "60", 2024-03-18T23:19:06.172 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:06.173 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:06.173 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:06.173 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:06.173 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:06.173 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:06.173 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:06.173 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:06.173 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-18T23:19:06.173 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-18T23:19:06.174 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:06.174 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:06.174 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:06.174 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:06.174 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:06.174 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:06.174 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-18T23:19:06.174 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-18T23:19:06.174 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-18T23:19:06.175 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-18T23:19:06.175 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-18T23:19:06.175 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-18T23:19:06.175 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-18T23:19:06.175 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-18T23:19:06.175 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:06.175 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:06.175 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:06.175 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:06.176 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:06.176 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-18T23:19:06.176 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-18T23:19:06.176 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:06.176 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:06.176 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:06.176 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:06.176 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:06.176 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:06.177 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:06.177 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:06.177 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:06.177 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:06.177 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:06.177 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:06.177 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-18T23:19:06.177 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-18T23:19:06.178 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:06.178 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:06.178 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:06.178 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-18T23:19:06.178 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:06.178 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:06.178 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-18T23:19:06.178 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-18T23:19:06.178 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-18T23:19:06.179 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-18T23:19:06.179 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-18T23:19:06.179 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-18T23:19:06.179 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-18T23:19:06.179 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-18T23:19:06.179 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:06.179 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:06.179 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:06.180 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:06.180 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:06.180 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-18T23:19:06.180 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-18T23:19:06.180 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-18T23:19:06.180 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:06.180 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-18T23:19:06.180 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-18T23:19:06.180 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:06.181 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:06.181 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:06.181 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:06.181 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:06.181 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:06.181 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:06.181 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:06.181 INFO:teuthology.orchestra.run.smithi049.stdout: "zabbix_host": { 2024-03-18T23:19:06.181 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "zabbix_host", 2024-03-18T23:19:06.182 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:06.182 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:06.182 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:06.182 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-18T23:19:06.182 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:06.182 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:06.182 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:06.182 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:06.183 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:06.183 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:06.183 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:06.183 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:06.183 INFO:teuthology.orchestra.run.smithi049.stdout: "zabbix_port": { 2024-03-18T23:19:06.183 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "zabbix_port", 2024-03-18T23:19:06.183 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-18T23:19:06.183 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:06.183 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:06.184 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "10051", 2024-03-18T23:19:06.184 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:06.184 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:06.184 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:06.184 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:06.184 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:06.184 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:06.184 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:06.184 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:06.185 INFO:teuthology.orchestra.run.smithi049.stdout: "zabbix_sender": { 2024-03-18T23:19:06.185 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "zabbix_sender", 2024-03-18T23:19:06.185 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-18T23:19:06.185 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-18T23:19:06.185 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-18T23:19:06.185 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-03-18T23:19:06.185 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-18T23:19:06.185 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-18T23:19:06.186 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-18T23:19:06.186 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-18T23:19:06.186 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-18T23:19:06.186 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-18T23:19:06.186 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-18T23:19:06.186 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:19:06.186 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:19:06.186 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:19:06.186 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-18T23:19:06.187 INFO:teuthology.orchestra.run.smithi049.stdout: "services": { 2024-03-18T23:19:06.187 INFO:teuthology.orchestra.run.smithi049.stdout: "dashboard": "https://172.21.15.49:8443/" 2024-03-18T23:19:06.187 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:06.187 INFO:teuthology.orchestra.run.smithi049.stdout: "always_on_modules": { 2024-03-18T23:19:06.187 INFO:teuthology.orchestra.run.smithi049.stdout: "octopus": [ 2024-03-18T23:19:06.187 INFO:teuthology.orchestra.run.smithi049.stdout: "balancer", 2024-03-18T23:19:06.187 INFO:teuthology.orchestra.run.smithi049.stdout: "crash", 2024-03-18T23:19:06.187 INFO:teuthology.orchestra.run.smithi049.stdout: "devicehealth", 2024-03-18T23:19:06.188 INFO:teuthology.orchestra.run.smithi049.stdout: "orchestrator", 2024-03-18T23:19:06.188 INFO:teuthology.orchestra.run.smithi049.stdout: "pg_autoscaler", 2024-03-18T23:19:06.188 INFO:teuthology.orchestra.run.smithi049.stdout: "progress", 2024-03-18T23:19:06.188 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_support", 2024-03-18T23:19:06.188 INFO:teuthology.orchestra.run.smithi049.stdout: "status", 2024-03-18T23:19:06.188 INFO:teuthology.orchestra.run.smithi049.stdout: "telemetry", 2024-03-18T23:19:06.188 INFO:teuthology.orchestra.run.smithi049.stdout: "volumes" 2024-03-18T23:19:06.188 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-18T23:19:06.188 INFO:teuthology.orchestra.run.smithi049.stdout: "pacific": [ 2024-03-18T23:19:06.189 INFO:teuthology.orchestra.run.smithi049.stdout: "balancer", 2024-03-18T23:19:06.189 INFO:teuthology.orchestra.run.smithi049.stdout: "crash", 2024-03-18T23:19:06.189 INFO:teuthology.orchestra.run.smithi049.stdout: "devicehealth", 2024-03-18T23:19:06.189 INFO:teuthology.orchestra.run.smithi049.stdout: "orchestrator", 2024-03-18T23:19:06.189 INFO:teuthology.orchestra.run.smithi049.stdout: "pg_autoscaler", 2024-03-18T23:19:06.189 INFO:teuthology.orchestra.run.smithi049.stdout: "progress", 2024-03-18T23:19:06.189 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_support", 2024-03-18T23:19:06.189 INFO:teuthology.orchestra.run.smithi049.stdout: "status", 2024-03-18T23:19:06.189 INFO:teuthology.orchestra.run.smithi049.stdout: "telemetry", 2024-03-18T23:19:06.190 INFO:teuthology.orchestra.run.smithi049.stdout: "volumes" 2024-03-18T23:19:06.190 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-18T23:19:06.190 INFO:teuthology.orchestra.run.smithi049.stdout: "quincy": [ 2024-03-18T23:19:06.190 INFO:teuthology.orchestra.run.smithi049.stdout: "balancer", 2024-03-18T23:19:06.190 INFO:teuthology.orchestra.run.smithi049.stdout: "crash", 2024-03-18T23:19:06.190 INFO:teuthology.orchestra.run.smithi049.stdout: "devicehealth", 2024-03-18T23:19:06.190 INFO:teuthology.orchestra.run.smithi049.stdout: "orchestrator", 2024-03-18T23:19:06.190 INFO:teuthology.orchestra.run.smithi049.stdout: "pg_autoscaler", 2024-03-18T23:19:06.191 INFO:teuthology.orchestra.run.smithi049.stdout: "progress", 2024-03-18T23:19:06.191 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_support", 2024-03-18T23:19:06.191 INFO:teuthology.orchestra.run.smithi049.stdout: "status", 2024-03-18T23:19:06.191 INFO:teuthology.orchestra.run.smithi049.stdout: "telemetry", 2024-03-18T23:19:06.191 INFO:teuthology.orchestra.run.smithi049.stdout: "volumes" 2024-03-18T23:19:06.191 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-18T23:19:06.191 INFO:teuthology.orchestra.run.smithi049.stdout: "reef": [ 2024-03-18T23:19:06.191 INFO:teuthology.orchestra.run.smithi049.stdout: "balancer", 2024-03-18T23:19:06.191 INFO:teuthology.orchestra.run.smithi049.stdout: "crash", 2024-03-18T23:19:06.192 INFO:teuthology.orchestra.run.smithi049.stdout: "devicehealth", 2024-03-18T23:19:06.192 INFO:teuthology.orchestra.run.smithi049.stdout: "orchestrator", 2024-03-18T23:19:06.192 INFO:teuthology.orchestra.run.smithi049.stdout: "pg_autoscaler", 2024-03-18T23:19:06.192 INFO:teuthology.orchestra.run.smithi049.stdout: "progress", 2024-03-18T23:19:06.192 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_support", 2024-03-18T23:19:06.192 INFO:teuthology.orchestra.run.smithi049.stdout: "status", 2024-03-18T23:19:06.192 INFO:teuthology.orchestra.run.smithi049.stdout: "telemetry", 2024-03-18T23:19:06.192 INFO:teuthology.orchestra.run.smithi049.stdout: "volumes" 2024-03-18T23:19:06.193 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-18T23:19:06.193 INFO:teuthology.orchestra.run.smithi049.stdout: "squid": [ 2024-03-18T23:19:06.193 INFO:teuthology.orchestra.run.smithi049.stdout: "balancer", 2024-03-18T23:19:06.193 INFO:teuthology.orchestra.run.smithi049.stdout: "crash", 2024-03-18T23:19:06.193 INFO:teuthology.orchestra.run.smithi049.stdout: "devicehealth", 2024-03-18T23:19:06.193 INFO:teuthology.orchestra.run.smithi049.stdout: "orchestrator", 2024-03-18T23:19:06.193 INFO:teuthology.orchestra.run.smithi049.stdout: "pg_autoscaler", 2024-03-18T23:19:06.193 INFO:teuthology.orchestra.run.smithi049.stdout: "progress", 2024-03-18T23:19:06.193 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_support", 2024-03-18T23:19:06.194 INFO:teuthology.orchestra.run.smithi049.stdout: "status", 2024-03-18T23:19:06.194 INFO:teuthology.orchestra.run.smithi049.stdout: "telemetry", 2024-03-18T23:19:06.194 INFO:teuthology.orchestra.run.smithi049.stdout: "volumes" 2024-03-18T23:19:06.194 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-03-18T23:19:06.194 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:06.194 INFO:teuthology.orchestra.run.smithi049.stdout: "last_failure_osd_epoch": 89, 2024-03-18T23:19:06.194 INFO:teuthology.orchestra.run.smithi049.stdout: "active_clients": [ 2024-03-18T23:19:06.194 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-18T23:19:06.195 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "devicehealth", 2024-03-18T23:19:06.195 INFO:teuthology.orchestra.run.smithi049.stdout: "addrvec": [ 2024-03-18T23:19:06.195 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-18T23:19:06.195 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v2", 2024-03-18T23:19:06.195 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.49:0", 2024-03-18T23:19:06.195 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 2964904053 2024-03-18T23:19:06.195 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:19:06.195 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-03-18T23:19:06.196 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:06.196 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-18T23:19:06.196 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "libcephsqlite", 2024-03-18T23:19:06.196 INFO:teuthology.orchestra.run.smithi049.stdout: "addrvec": [ 2024-03-18T23:19:06.196 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-18T23:19:06.196 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v2", 2024-03-18T23:19:06.196 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.49:0", 2024-03-18T23:19:06.196 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 2800792270 2024-03-18T23:19:06.196 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:19:06.197 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-03-18T23:19:06.197 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:06.197 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-18T23:19:06.197 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "rbd_support", 2024-03-18T23:19:06.197 INFO:teuthology.orchestra.run.smithi049.stdout: "addrvec": [ 2024-03-18T23:19:06.197 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-18T23:19:06.197 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v2", 2024-03-18T23:19:06.197 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.49:0", 2024-03-18T23:19:06.197 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 2842437863 2024-03-18T23:19:06.198 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:19:06.198 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-03-18T23:19:06.198 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:06.198 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-18T23:19:06.198 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "snap_schedule", 2024-03-18T23:19:06.198 INFO:teuthology.orchestra.run.smithi049.stdout: "addrvec": [ 2024-03-18T23:19:06.198 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-18T23:19:06.198 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v2", 2024-03-18T23:19:06.199 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.49:0", 2024-03-18T23:19:06.199 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 1299672877 2024-03-18T23:19:06.199 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:19:06.199 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-03-18T23:19:06.199 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:06.199 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-18T23:19:06.199 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "snap_schedule", 2024-03-18T23:19:06.199 INFO:teuthology.orchestra.run.smithi049.stdout: "addrvec": [ 2024-03-18T23:19:06.199 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-18T23:19:06.199 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v2", 2024-03-18T23:19:06.200 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.49:0", 2024-03-18T23:19:06.200 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 4121114111 2024-03-18T23:19:06.200 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:19:06.200 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-03-18T23:19:06.200 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:06.200 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-18T23:19:06.200 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "volumes", 2024-03-18T23:19:06.200 INFO:teuthology.orchestra.run.smithi049.stdout: "addrvec": [ 2024-03-18T23:19:06.201 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-18T23:19:06.201 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v2", 2024-03-18T23:19:06.201 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.49:0", 2024-03-18T23:19:06.201 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 2912461262 2024-03-18T23:19:06.201 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:19:06.201 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-03-18T23:19:06.201 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:19:06.201 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-03-18T23:19:06.201 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-03-18T23:19:06.203 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:19:04 smithi139 ceph-mon[67282]: from='client.? 172.21.15.49:0/2217512547' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-18T23:19:06.203 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:19:04 smithi139 ceph-mon[67282]: from='client.? 172.21.15.49:0/928810796' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-03-18T23:19:06.203 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:19:05 smithi139 ceph-mon[67282]: pgmap v337: 97 pgs: 97 active+clean; 69 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.4 MiB/s wr, 249 op/s 2024-03-18T23:19:06.203 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:19:04 smithi175 ceph-mon[67352]: from='client.? 172.21.15.49:0/2217512547' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-18T23:19:06.204 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:19:04 smithi175 ceph-mon[67352]: from='client.? 172.21.15.49:0/928810796' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-03-18T23:19:06.204 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:19:05 smithi175 ceph-mon[67352]: pgmap v337: 97 pgs: 97 active+clean; 69 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.4 MiB/s wr, 249 op/s 2024-03-18T23:19:06.204 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:03.857+0000 7ff21ea7e640 1 -- 172.21.15.49:0/928810796 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7ff2000a2970 msgr2=0x7ff2000a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:19:06.205 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:03.857+0000 7ff21ea7e640 1 --2- 172.21.15.49:0/928810796 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7ff2000a2970 0x7ff2000a4e30 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7ff20c005ab0 tx=0x7ff20c005fd0 comp rx=0 tx=0).stop 2024-03-18T23:19:06.205 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:03.857+0000 7ff21ea7e640 1 -- 172.21.15.49:0/928810796 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff218100720 msgr2=0x7ff218198fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:19:06.205 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:03.857+0000 7ff21ea7e640 1 --2- 172.21.15.49:0/928810796 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff218100720 0x7ff218198fe0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7ff21000ebc0 tx=0x7ff21000c6a0 comp rx=0 tx=0).stop 2024-03-18T23:19:06.205 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:03.857+0000 7ff21ea7e640 1 -- 172.21.15.49:0/928810796 shutdown_connections 2024-03-18T23:19:06.205 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:03.857+0000 7ff21ea7e640 1 --2- 172.21.15.49:0/928810796 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7ff2000a2970 0x7ff2000a4e30 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:06.205 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:03.857+0000 7ff21ea7e640 1 --2- 172.21.15.49:0/928810796 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff218106730 0x7ff2181a4060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:06.205 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:03.857+0000 7ff21ea7e640 1 --2- 172.21.15.49:0/928810796 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff2181010e0 0x7ff218199520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:06.205 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:03.857+0000 7ff21ea7e640 1 --2- 172.21.15.49:0/928810796 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff218100720 0x7ff218198fe0 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:06.206 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:03.857+0000 7ff21ea7e640 1 -- 172.21.15.49:0/928810796 >> 172.21.15.49:0/928810796 conn(0x7ff218077820 msgr2=0x7ff218102f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:19:06.206 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:03.857+0000 7ff21ea7e640 1 -- 172.21.15.49:0/928810796 shutdown_connections 2024-03-18T23:19:06.206 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:03.857+0000 7ff21ea7e640 1 -- 172.21.15.49:0/928810796 wait complete. 2024-03-18T23:19:06.212 DEBUG:tasks.check_counter:Getting stats from a 2024-03-18T23:19:06.212 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell --fsid c63591ea-e57a-11ee-95c9-87774f69a715 -- ceph daemon mds.a perf dump 2024-03-18T23:19:06.213 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:19:04 smithi049 ceph-mon[62991]: from='client.? 172.21.15.49:0/2217512547' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-18T23:19:06.213 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:19:04 smithi049 ceph-mon[62991]: from='client.? 172.21.15.49:0/928810796' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-03-18T23:19:06.213 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:19:05 smithi049 ceph-mon[62991]: pgmap v337: 97 pgs: 97 active+clean; 69 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.4 MiB/s wr, 249 op/s 2024-03-18T23:19:06.660 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T23:19:07.460 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:19:07 smithi175 ceph-mon[67352]: pgmap v338: 97 pgs: 97 active+clean; 69 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.2 MiB/s wr, 241 op/s 2024-03-18T23:19:07.460 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:19:07 smithi175 ceph-mon[67352]: osdmap e100: 12 total, 12 up, 12 in 2024-03-18T23:19:07.564 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:19:07 smithi139 ceph-mon[67282]: pgmap v338: 97 pgs: 97 active+clean; 69 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.2 MiB/s wr, 241 op/s 2024-03-18T23:19:07.564 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:19:07 smithi139 ceph-mon[67282]: osdmap e100: 12 total, 12 up, 12 in 2024-03-18T23:19:07.595 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:19:07 smithi049 ceph-mon[62991]: pgmap v338: 97 pgs: 97 active+clean; 69 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.2 MiB/s wr, 241 op/s 2024-03-18T23:19:07.595 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:19:07 smithi049 ceph-mon[62991]: osdmap e100: 12 total, 12 up, 12 in 2024-03-18T23:19:09.103 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-03-18T23:19:09.104 INFO:teuthology.orchestra.run.smithi049.stdout: "AsyncMessenger::Worker-0": { 2024-03-18T23:19:09.104 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_messages": 827, 2024-03-18T23:19:09.104 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_messages": 829, 2024-03-18T23:19:09.104 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_bytes": 275080, 2024-03-18T23:19:09.104 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_bytes": 262987, 2024-03-18T23:19:09.104 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_created_connections": 6, 2024-03-18T23:19:09.105 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_active_connections": 2, 2024-03-18T23:19:09.105 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_total_time": 0.078351828, 2024-03-18T23:19:09.105 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_send_time": 0.026358763, 2024-03-18T23:19:09.105 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_recv_time": 0.019066564, 2024-03-18T23:19:09.105 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_fast_dispatch_time": 0.024495958, 2024-03-18T23:19:09.105 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_messages_queue_lat": { 2024-03-18T23:19:09.105 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 827, 2024-03-18T23:19:09.105 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.035787785, 2024-03-18T23:19:09.106 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000043274 2024-03-18T23:19:09.106 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:09.106 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_handle_ack_lat": { 2024-03-18T23:19:09.106 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-03-18T23:19:09.106 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-03-18T23:19:09.106 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-03-18T23:19:09.106 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:09.106 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-18T23:19:09.106 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_encrypted_bytes": 0 2024-03-18T23:19:09.107 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:09.107 INFO:teuthology.orchestra.run.smithi049.stdout: "AsyncMessenger::Worker-1": { 2024-03-18T23:19:09.107 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_messages": 707, 2024-03-18T23:19:09.107 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_messages": 865, 2024-03-18T23:19:09.107 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_bytes": 176442, 2024-03-18T23:19:09.107 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_bytes": 675034, 2024-03-18T23:19:09.107 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_created_connections": 7, 2024-03-18T23:19:09.107 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_active_connections": 5, 2024-03-18T23:19:09.107 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_total_time": 0.063709088, 2024-03-18T23:19:09.108 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_send_time": 0.022132980, 2024-03-18T23:19:09.108 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_recv_time": 0.013721886, 2024-03-18T23:19:09.108 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_fast_dispatch_time": 0.021353554, 2024-03-18T23:19:09.108 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_messages_queue_lat": { 2024-03-18T23:19:09.108 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 863, 2024-03-18T23:19:09.108 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.030524802, 2024-03-18T23:19:09.108 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000035370 2024-03-18T23:19:09.108 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:09.108 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_handle_ack_lat": { 2024-03-18T23:19:09.109 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-03-18T23:19:09.109 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-03-18T23:19:09.109 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-03-18T23:19:09.109 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:09.109 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_encrypted_bytes": 3344, 2024-03-18T23:19:09.109 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_encrypted_bytes": 452048 2024-03-18T23:19:09.109 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:09.109 INFO:teuthology.orchestra.run.smithi049.stdout: "AsyncMessenger::Worker-2": { 2024-03-18T23:19:09.109 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_messages": 980, 2024-03-18T23:19:09.110 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_messages": 964, 2024-03-18T23:19:09.110 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_bytes": 1688381, 2024-03-18T23:19:09.110 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_bytes": 366696, 2024-03-18T23:19:09.110 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_created_connections": 3, 2024-03-18T23:19:09.110 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_active_connections": 2, 2024-03-18T23:19:09.110 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_total_time": 0.089021898, 2024-03-18T23:19:09.110 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_send_time": 0.028478495, 2024-03-18T23:19:09.111 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_recv_time": 0.044780419, 2024-03-18T23:19:09.111 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_fast_dispatch_time": 0.019225336, 2024-03-18T23:19:09.111 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_messages_queue_lat": { 2024-03-18T23:19:09.111 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 963, 2024-03-18T23:19:09.111 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.030143499, 2024-03-18T23:19:09.111 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000031301 2024-03-18T23:19:09.111 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:09.111 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_handle_ack_lat": { 2024-03-18T23:19:09.111 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-03-18T23:19:09.111 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-03-18T23:19:09.112 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-03-18T23:19:09.112 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:09.112 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_encrypted_bytes": 1495248, 2024-03-18T23:19:09.112 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_encrypted_bytes": 133920 2024-03-18T23:19:09.112 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:09.112 INFO:teuthology.orchestra.run.smithi049.stdout: "cct": { 2024-03-18T23:19:09.112 INFO:teuthology.orchestra.run.smithi049.stdout: "total_workers": 1, 2024-03-18T23:19:09.112 INFO:teuthology.orchestra.run.smithi049.stdout: "unhealthy_workers": 0 2024-03-18T23:19:09.112 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:09.113 INFO:teuthology.orchestra.run.smithi049.stdout: "finisher-MDSRank": { 2024-03-18T23:19:09.113 INFO:teuthology.orchestra.run.smithi049.stdout: "queue_len": 0, 2024-03-18T23:19:09.113 INFO:teuthology.orchestra.run.smithi049.stdout: "complete_latency": { 2024-03-18T23:19:09.113 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 3728, 2024-03-18T23:19:09.113 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.094477011, 2024-03-18T23:19:09.113 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000025342 2024-03-18T23:19:09.113 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:19:09.113 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:09.113 INFO:teuthology.orchestra.run.smithi049.stdout: "finisher-PurgeQueue": { 2024-03-18T23:19:09.114 INFO:teuthology.orchestra.run.smithi049.stdout: "queue_len": 0, 2024-03-18T23:19:09.114 INFO:teuthology.orchestra.run.smithi049.stdout: "complete_latency": { 2024-03-18T23:19:09.114 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-03-18T23:19:09.114 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-03-18T23:19:09.114 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-03-18T23:19:09.114 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:19:09.114 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:09.114 INFO:teuthology.orchestra.run.smithi049.stdout: "mds": { 2024-03-18T23:19:09.115 INFO:teuthology.orchestra.run.smithi049.stdout: "request": 0, 2024-03-18T23:19:09.115 INFO:teuthology.orchestra.run.smithi049.stdout: "reply": 0, 2024-03-18T23:19:09.115 INFO:teuthology.orchestra.run.smithi049.stdout: "reply_latency": { 2024-03-18T23:19:09.115 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-03-18T23:19:09.115 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-03-18T23:19:09.115 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-03-18T23:19:09.115 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:09.115 INFO:teuthology.orchestra.run.smithi049.stdout: "slow_reply": 0, 2024-03-18T23:19:09.115 INFO:teuthology.orchestra.run.smithi049.stdout: "forward": 0, 2024-03-18T23:19:09.116 INFO:teuthology.orchestra.run.smithi049.stdout: "dir_fetch_complete": 0, 2024-03-18T23:19:09.116 INFO:teuthology.orchestra.run.smithi049.stdout: "dir_fetch_keys": 0, 2024-03-18T23:19:09.116 INFO:teuthology.orchestra.run.smithi049.stdout: "dir_commit": 0, 2024-03-18T23:19:09.116 INFO:teuthology.orchestra.run.smithi049.stdout: "dir_split": 0, 2024-03-18T23:19:09.116 INFO:teuthology.orchestra.run.smithi049.stdout: "dir_merge": 0, 2024-03-18T23:19:09.116 INFO:teuthology.orchestra.run.smithi049.stdout: "inodes": 0, 2024-03-18T23:19:09.116 INFO:teuthology.orchestra.run.smithi049.stdout: "inodes_top": 0, 2024-03-18T23:19:09.116 INFO:teuthology.orchestra.run.smithi049.stdout: "inodes_bottom": 0, 2024-03-18T23:19:09.116 INFO:teuthology.orchestra.run.smithi049.stdout: "inodes_pin_tail": 0, 2024-03-18T23:19:09.117 INFO:teuthology.orchestra.run.smithi049.stdout: "inodes_pinned": 0, 2024-03-18T23:19:09.117 INFO:teuthology.orchestra.run.smithi049.stdout: "inodes_expired": 0, 2024-03-18T23:19:09.117 INFO:teuthology.orchestra.run.smithi049.stdout: "inodes_with_caps": 0, 2024-03-18T23:19:09.117 INFO:teuthology.orchestra.run.smithi049.stdout: "caps": 0, 2024-03-18T23:19:09.117 INFO:teuthology.orchestra.run.smithi049.stdout: "subtrees": 1, 2024-03-18T23:19:09.117 INFO:teuthology.orchestra.run.smithi049.stdout: "traverse": 0, 2024-03-18T23:19:09.117 INFO:teuthology.orchestra.run.smithi049.stdout: "traverse_hit": 0, 2024-03-18T23:19:09.117 INFO:teuthology.orchestra.run.smithi049.stdout: "traverse_forward": 0, 2024-03-18T23:19:09.118 INFO:teuthology.orchestra.run.smithi049.stdout: "traverse_discover": 0, 2024-03-18T23:19:09.118 INFO:teuthology.orchestra.run.smithi049.stdout: "traverse_dir_fetch": 0, 2024-03-18T23:19:09.118 INFO:teuthology.orchestra.run.smithi049.stdout: "traverse_remote_ino": 0, 2024-03-18T23:19:09.118 INFO:teuthology.orchestra.run.smithi049.stdout: "traverse_lock": 0, 2024-03-18T23:19:09.118 INFO:teuthology.orchestra.run.smithi049.stdout: "load_cent": 0, 2024-03-18T23:19:09.119 INFO:teuthology.orchestra.run.smithi049.stdout: "q": 0, 2024-03-18T23:19:09.119 INFO:teuthology.orchestra.run.smithi049.stdout: "exported": 0, 2024-03-18T23:19:09.119 INFO:teuthology.orchestra.run.smithi049.stdout: "exported_inodes": 0, 2024-03-18T23:19:09.119 INFO:teuthology.orchestra.run.smithi049.stdout: "imported": 0, 2024-03-18T23:19:09.119 INFO:teuthology.orchestra.run.smithi049.stdout: "imported_inodes": 0, 2024-03-18T23:19:09.119 INFO:teuthology.orchestra.run.smithi049.stdout: "openino_dir_fetch": 0, 2024-03-18T23:19:09.119 INFO:teuthology.orchestra.run.smithi049.stdout: "openino_backtrace_fetch": 0, 2024-03-18T23:19:09.119 INFO:teuthology.orchestra.run.smithi049.stdout: "openino_peer_discover": 0, 2024-03-18T23:19:09.119 INFO:teuthology.orchestra.run.smithi049.stdout: "root_rfiles": 0, 2024-03-18T23:19:09.119 INFO:teuthology.orchestra.run.smithi049.stdout: "root_rbytes": 0, 2024-03-18T23:19:09.120 INFO:teuthology.orchestra.run.smithi049.stdout: "root_rsnaps": 0, 2024-03-18T23:19:09.120 INFO:teuthology.orchestra.run.smithi049.stdout: "scrub_backtrace_fetch": 0, 2024-03-18T23:19:09.120 INFO:teuthology.orchestra.run.smithi049.stdout: "scrub_set_tag": 0, 2024-03-18T23:19:09.120 INFO:teuthology.orchestra.run.smithi049.stdout: "scrub_backtrace_repaired": 0, 2024-03-18T23:19:09.120 INFO:teuthology.orchestra.run.smithi049.stdout: "scrub_inotable_repaired": 0, 2024-03-18T23:19:09.120 INFO:teuthology.orchestra.run.smithi049.stdout: "scrub_dir_inodes": 0, 2024-03-18T23:19:09.120 INFO:teuthology.orchestra.run.smithi049.stdout: "scrub_dir_base_inodes": 0, 2024-03-18T23:19:09.120 INFO:teuthology.orchestra.run.smithi049.stdout: "scrub_dirfrag_rstats": 0, 2024-03-18T23:19:09.120 INFO:teuthology.orchestra.run.smithi049.stdout: "scrub_file_inodes": 0, 2024-03-18T23:19:09.121 INFO:teuthology.orchestra.run.smithi049.stdout: "handle_inode_file_caps": 0, 2024-03-18T23:19:09.121 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph_cap_op_revoke": 0, 2024-03-18T23:19:09.121 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph_cap_op_grant": 0, 2024-03-18T23:19:09.121 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph_cap_op_trunc": 0, 2024-03-18T23:19:09.121 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-18T23:19:09.121 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-18T23:19:09.121 INFO:teuthology.orchestra.run.smithi049.stdout: "handle_client_caps": 0, 2024-03-18T23:19:09.121 INFO:teuthology.orchestra.run.smithi049.stdout: "handle_client_caps_dirty": 0, 2024-03-18T23:19:09.121 INFO:teuthology.orchestra.run.smithi049.stdout: "handle_client_cap_release": 0, 2024-03-18T23:19:09.122 INFO:teuthology.orchestra.run.smithi049.stdout: "process_request_cap_release": 0 2024-03-18T23:19:09.122 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:09.122 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_cache": { 2024-03-18T23:19:09.122 INFO:teuthology.orchestra.run.smithi049.stdout: "dir_update": 0, 2024-03-18T23:19:09.122 INFO:teuthology.orchestra.run.smithi049.stdout: "dir_update_receipt": 0, 2024-03-18T23:19:09.122 INFO:teuthology.orchestra.run.smithi049.stdout: "dir_try_discover": 0, 2024-03-18T23:19:09.122 INFO:teuthology.orchestra.run.smithi049.stdout: "dir_send_discover": 0, 2024-03-18T23:19:09.123 INFO:teuthology.orchestra.run.smithi049.stdout: "dir_handle_discover": 0, 2024-03-18T23:19:09.123 INFO:teuthology.orchestra.run.smithi049.stdout: "num_strays": 0, 2024-03-18T23:19:09.123 INFO:teuthology.orchestra.run.smithi049.stdout: "num_strays_delayed": 0, 2024-03-18T23:19:09.123 INFO:teuthology.orchestra.run.smithi049.stdout: "num_strays_enqueuing": 0, 2024-03-18T23:19:09.123 INFO:teuthology.orchestra.run.smithi049.stdout: "strays_created": 0, 2024-03-18T23:19:09.123 INFO:teuthology.orchestra.run.smithi049.stdout: "strays_enqueued": 0, 2024-03-18T23:19:09.123 INFO:teuthology.orchestra.run.smithi049.stdout: "strays_reintegrated": 0, 2024-03-18T23:19:09.123 INFO:teuthology.orchestra.run.smithi049.stdout: "strays_migrated": 0, 2024-03-18T23:19:09.124 INFO:teuthology.orchestra.run.smithi049.stdout: "num_recovering_processing": 0, 2024-03-18T23:19:09.124 INFO:teuthology.orchestra.run.smithi049.stdout: "num_recovering_enqueued": 0, 2024-03-18T23:19:09.124 INFO:teuthology.orchestra.run.smithi049.stdout: "num_recovering_prioritized": 0, 2024-03-18T23:19:09.124 INFO:teuthology.orchestra.run.smithi049.stdout: "recovery_started": 0, 2024-03-18T23:19:09.124 INFO:teuthology.orchestra.run.smithi049.stdout: "recovery_completed": 0, 2024-03-18T23:19:09.124 INFO:teuthology.orchestra.run.smithi049.stdout: "ireq_quiesce_path": 0, 2024-03-18T23:19:09.124 INFO:teuthology.orchestra.run.smithi049.stdout: "ireq_quiesce_inode": 0, 2024-03-18T23:19:09.124 INFO:teuthology.orchestra.run.smithi049.stdout: "ireq_enqueue_scrub": 0, 2024-03-18T23:19:09.125 INFO:teuthology.orchestra.run.smithi049.stdout: "ireq_exportdir": 0, 2024-03-18T23:19:09.125 INFO:teuthology.orchestra.run.smithi049.stdout: "ireq_flush": 0, 2024-03-18T23:19:09.125 INFO:teuthology.orchestra.run.smithi049.stdout: "ireq_fragmentdir": 0, 2024-03-18T23:19:09.125 INFO:teuthology.orchestra.run.smithi049.stdout: "ireq_fragstats": 0, 2024-03-18T23:19:09.125 INFO:teuthology.orchestra.run.smithi049.stdout: "ireq_inodestats": 0 2024-03-18T23:19:09.125 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:09.125 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_log": { 2024-03-18T23:19:09.125 INFO:teuthology.orchestra.run.smithi049.stdout: "evlrg": 0, 2024-03-18T23:19:09.126 INFO:teuthology.orchestra.run.smithi049.stdout: "evadd": 0, 2024-03-18T23:19:09.126 INFO:teuthology.orchestra.run.smithi049.stdout: "evex": 0, 2024-03-18T23:19:09.126 INFO:teuthology.orchestra.run.smithi049.stdout: "evtrm": 0, 2024-03-18T23:19:09.126 INFO:teuthology.orchestra.run.smithi049.stdout: "ev": 7, 2024-03-18T23:19:09.126 INFO:teuthology.orchestra.run.smithi049.stdout: "evexg": 0, 2024-03-18T23:19:09.126 INFO:teuthology.orchestra.run.smithi049.stdout: "evexd": 0, 2024-03-18T23:19:09.126 INFO:teuthology.orchestra.run.smithi049.stdout: "segadd": 0, 2024-03-18T23:19:09.126 INFO:teuthology.orchestra.run.smithi049.stdout: "segex": 0, 2024-03-18T23:19:09.126 INFO:teuthology.orchestra.run.smithi049.stdout: "segtrm": 0, 2024-03-18T23:19:09.127 INFO:teuthology.orchestra.run.smithi049.stdout: "seg": 2, 2024-03-18T23:19:09.127 INFO:teuthology.orchestra.run.smithi049.stdout: "segmjr": 2, 2024-03-18T23:19:09.127 INFO:teuthology.orchestra.run.smithi049.stdout: "segexg": 0, 2024-03-18T23:19:09.127 INFO:teuthology.orchestra.run.smithi049.stdout: "segexd": 0, 2024-03-18T23:19:09.127 INFO:teuthology.orchestra.run.smithi049.stdout: "expos": 4194304, 2024-03-18T23:19:09.127 INFO:teuthology.orchestra.run.smithi049.stdout: "wrpos": 4196264, 2024-03-18T23:19:09.127 INFO:teuthology.orchestra.run.smithi049.stdout: "rdpos": 4196123, 2024-03-18T23:19:09.128 INFO:teuthology.orchestra.run.smithi049.stdout: "jlat": { 2024-03-18T23:19:09.128 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-03-18T23:19:09.128 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-03-18T23:19:09.128 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-03-18T23:19:09.128 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:09.128 INFO:teuthology.orchestra.run.smithi049.stdout: "replayed": 7 2024-03-18T23:19:09.128 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:09.128 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_mem": { 2024-03-18T23:19:09.128 INFO:teuthology.orchestra.run.smithi049.stdout: "ino": 3, 2024-03-18T23:19:09.129 INFO:teuthology.orchestra.run.smithi049.stdout: "ino+": 3, 2024-03-18T23:19:09.129 INFO:teuthology.orchestra.run.smithi049.stdout: "ino-": 0, 2024-03-18T23:19:09.129 INFO:teuthology.orchestra.run.smithi049.stdout: "dir": 1, 2024-03-18T23:19:09.129 INFO:teuthology.orchestra.run.smithi049.stdout: "dir+": 1, 2024-03-18T23:19:09.129 INFO:teuthology.orchestra.run.smithi049.stdout: "dir-": 0, 2024-03-18T23:19:09.129 INFO:teuthology.orchestra.run.smithi049.stdout: "dn": 0, 2024-03-18T23:19:09.129 INFO:teuthology.orchestra.run.smithi049.stdout: "dn+": 0, 2024-03-18T23:19:09.129 INFO:teuthology.orchestra.run.smithi049.stdout: "dn-": 0, 2024-03-18T23:19:09.129 INFO:teuthology.orchestra.run.smithi049.stdout: "cap": 0, 2024-03-18T23:19:09.130 INFO:teuthology.orchestra.run.smithi049.stdout: "cap+": 0, 2024-03-18T23:19:09.130 INFO:teuthology.orchestra.run.smithi049.stdout: "cap-": 0, 2024-03-18T23:19:09.130 INFO:teuthology.orchestra.run.smithi049.stdout: "rss": 50760, 2024-03-18T23:19:09.130 INFO:teuthology.orchestra.run.smithi049.stdout: "heap": 215420 2024-03-18T23:19:09.130 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:09.130 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_server": { 2024-03-18T23:19:09.130 INFO:teuthology.orchestra.run.smithi049.stdout: "dispatch_client_request": 0, 2024-03-18T23:19:09.131 INFO:teuthology.orchestra.run.smithi049.stdout: "dispatch_server_request": 0, 2024-03-18T23:19:09.133 INFO:teuthology.orchestra.run.smithi049.stdout: "handle_client_request": 0, 2024-03-18T23:19:09.133 INFO:teuthology.orchestra.run.smithi049.stdout: "handle_client_session": 0, 2024-03-18T23:19:09.133 INFO:teuthology.orchestra.run.smithi049.stdout: "handle_peer_request": 0, 2024-03-18T23:19:09.133 INFO:teuthology.orchestra.run.smithi049.stdout: "req_create_latency": { 2024-03-18T23:19:09.134 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-03-18T23:19:09.134 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-03-18T23:19:09.134 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-03-18T23:19:09.134 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:09.134 INFO:teuthology.orchestra.run.smithi049.stdout: "req_getattr_latency": { 2024-03-18T23:19:09.134 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-03-18T23:19:09.134 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-03-18T23:19:09.134 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-03-18T23:19:09.135 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:09.135 INFO:teuthology.orchestra.run.smithi049.stdout: "req_getfilelock_latency": { 2024-03-18T23:19:09.135 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-03-18T23:19:09.135 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-03-18T23:19:09.135 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-03-18T23:19:09.135 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:09.135 INFO:teuthology.orchestra.run.smithi049.stdout: "req_link_latency": { 2024-03-18T23:19:09.135 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-03-18T23:19:09.135 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-03-18T23:19:09.136 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-03-18T23:19:09.136 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:09.136 INFO:teuthology.orchestra.run.smithi049.stdout: "req_lookup_latency": { 2024-03-18T23:19:09.136 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-03-18T23:19:09.136 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-03-18T23:19:09.136 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-03-18T23:19:09.136 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:09.136 INFO:teuthology.orchestra.run.smithi049.stdout: "req_lookuphash_latency": { 2024-03-18T23:19:09.137 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-03-18T23:19:09.137 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-03-18T23:19:09.137 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-03-18T23:19:09.137 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:09.137 INFO:teuthology.orchestra.run.smithi049.stdout: "req_lookupino_latency": { 2024-03-18T23:19:09.137 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-03-18T23:19:09.137 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-03-18T23:19:09.137 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-03-18T23:19:09.137 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:09.138 INFO:teuthology.orchestra.run.smithi049.stdout: "req_lookupname_latency": { 2024-03-18T23:19:09.138 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-03-18T23:19:09.138 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-03-18T23:19:09.138 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-03-18T23:19:09.138 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:09.138 INFO:teuthology.orchestra.run.smithi049.stdout: "req_lookupparent_latency": { 2024-03-18T23:19:09.138 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-03-18T23:19:09.138 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-03-18T23:19:09.138 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-03-18T23:19:09.139 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:09.139 INFO:teuthology.orchestra.run.smithi049.stdout: "req_lookupsnap_latency": { 2024-03-18T23:19:09.139 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-03-18T23:19:09.139 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-03-18T23:19:09.139 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-03-18T23:19:09.139 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:09.139 INFO:teuthology.orchestra.run.smithi049.stdout: "req_lssnap_latency": { 2024-03-18T23:19:09.139 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-03-18T23:19:09.140 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-03-18T23:19:09.140 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-03-18T23:19:09.140 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:09.140 INFO:teuthology.orchestra.run.smithi049.stdout: "req_mkdir_latency": { 2024-03-18T23:19:09.140 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-03-18T23:19:09.141 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-03-18T23:19:09.141 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-03-18T23:19:09.141 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:09.141 INFO:teuthology.orchestra.run.smithi049.stdout: "req_mknod_latency": { 2024-03-18T23:19:09.141 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-03-18T23:19:09.141 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-03-18T23:19:09.142 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-03-18T23:19:09.142 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:09.142 INFO:teuthology.orchestra.run.smithi049.stdout: "req_mksnap_latency": { 2024-03-18T23:19:09.142 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-03-18T23:19:09.142 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-03-18T23:19:09.142 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-03-18T23:19:09.142 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:09.142 INFO:teuthology.orchestra.run.smithi049.stdout: "req_open_latency": { 2024-03-18T23:19:09.143 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-03-18T23:19:09.143 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-03-18T23:19:09.143 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-03-18T23:19:09.143 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:09.143 INFO:teuthology.orchestra.run.smithi049.stdout: "req_readdir_latency": { 2024-03-18T23:19:09.143 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-03-18T23:19:09.143 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-03-18T23:19:09.143 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-03-18T23:19:09.143 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:09.144 INFO:teuthology.orchestra.run.smithi049.stdout: "req_rename_latency": { 2024-03-18T23:19:09.144 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-03-18T23:19:09.144 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-03-18T23:19:09.144 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-03-18T23:19:09.144 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:09.144 INFO:teuthology.orchestra.run.smithi049.stdout: "req_renamesnap_latency": { 2024-03-18T23:19:09.144 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-03-18T23:19:09.144 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-03-18T23:19:09.144 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-03-18T23:19:09.145 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:09.145 INFO:teuthology.orchestra.run.smithi049.stdout: "req_snapdiff_latency": { 2024-03-18T23:19:09.145 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-03-18T23:19:09.145 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-03-18T23:19:09.145 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-03-18T23:19:09.145 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:09.145 INFO:teuthology.orchestra.run.smithi049.stdout: "req_rmdir_latency": { 2024-03-18T23:19:09.145 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-03-18T23:19:09.146 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-03-18T23:19:09.146 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-03-18T23:19:09.146 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:09.146 INFO:teuthology.orchestra.run.smithi049.stdout: "req_rmsnap_latency": { 2024-03-18T23:19:09.146 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-03-18T23:19:09.146 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-03-18T23:19:09.146 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-03-18T23:19:09.146 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:09.146 INFO:teuthology.orchestra.run.smithi049.stdout: "req_rmxattr_latency": { 2024-03-18T23:19:09.147 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-03-18T23:19:09.147 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-03-18T23:19:09.147 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-03-18T23:19:09.147 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:09.147 INFO:teuthology.orchestra.run.smithi049.stdout: "req_setattr_latency": { 2024-03-18T23:19:09.147 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-03-18T23:19:09.147 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-03-18T23:19:09.147 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-03-18T23:19:09.148 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:09.148 INFO:teuthology.orchestra.run.smithi049.stdout: "req_setdirlayout_latency": { 2024-03-18T23:19:09.148 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-03-18T23:19:09.148 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-03-18T23:19:09.148 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-03-18T23:19:09.148 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:09.148 INFO:teuthology.orchestra.run.smithi049.stdout: "req_setfilelock_latency": { 2024-03-18T23:19:09.148 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-03-18T23:19:09.148 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-03-18T23:19:09.149 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-03-18T23:19:09.149 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:09.149 INFO:teuthology.orchestra.run.smithi049.stdout: "req_setlayout_latency": { 2024-03-18T23:19:09.149 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-03-18T23:19:09.149 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-03-18T23:19:09.149 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-03-18T23:19:09.149 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:09.149 INFO:teuthology.orchestra.run.smithi049.stdout: "req_setxattr_latency": { 2024-03-18T23:19:09.149 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-03-18T23:19:09.150 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-03-18T23:19:09.150 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-03-18T23:19:09.150 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:09.150 INFO:teuthology.orchestra.run.smithi049.stdout: "req_symlink_latency": { 2024-03-18T23:19:09.150 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-03-18T23:19:09.150 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-03-18T23:19:09.150 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-03-18T23:19:09.150 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:09.151 INFO:teuthology.orchestra.run.smithi049.stdout: "req_unlink_latency": { 2024-03-18T23:19:09.151 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-03-18T23:19:09.151 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-03-18T23:19:09.151 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-03-18T23:19:09.151 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:09.151 INFO:teuthology.orchestra.run.smithi049.stdout: "cap_revoke_eviction": 0, 2024-03-18T23:19:09.151 INFO:teuthology.orchestra.run.smithi049.stdout: "cap_acquisition_throttle": 0, 2024-03-18T23:19:09.151 INFO:teuthology.orchestra.run.smithi049.stdout: "req_getvxattr_latency": { 2024-03-18T23:19:09.151 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-03-18T23:19:09.152 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-03-18T23:19:09.152 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-03-18T23:19:09.152 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:19:09.152 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:09.152 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_sessions": { 2024-03-18T23:19:09.152 INFO:teuthology.orchestra.run.smithi049.stdout: "session_count": 1, 2024-03-18T23:19:09.152 INFO:teuthology.orchestra.run.smithi049.stdout: "session_add": 3, 2024-03-18T23:19:09.152 INFO:teuthology.orchestra.run.smithi049.stdout: "session_remove": 2, 2024-03-18T23:19:09.153 INFO:teuthology.orchestra.run.smithi049.stdout: "sessions_open": 2, 2024-03-18T23:19:09.153 INFO:teuthology.orchestra.run.smithi049.stdout: "sessions_stale": 0, 2024-03-18T23:19:09.153 INFO:teuthology.orchestra.run.smithi049.stdout: "total_load": 0, 2024-03-18T23:19:09.153 INFO:teuthology.orchestra.run.smithi049.stdout: "average_load": 0, 2024-03-18T23:19:09.153 INFO:teuthology.orchestra.run.smithi049.stdout: "avg_session_uptime": 9200, 2024-03-18T23:19:09.153 INFO:teuthology.orchestra.run.smithi049.stdout: "mdthresh_evicted": 0 2024-03-18T23:19:09.153 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:09.153 INFO:teuthology.orchestra.run.smithi049.stdout: "mempool": { 2024-03-18T23:19:09.153 INFO:teuthology.orchestra.run.smithi049.stdout: "bloom_filter_bytes": 0, 2024-03-18T23:19:09.154 INFO:teuthology.orchestra.run.smithi049.stdout: "bloom_filter_items": 0, 2024-03-18T23:19:09.154 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_alloc_bytes": 0, 2024-03-18T23:19:09.154 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_alloc_items": 0, 2024-03-18T23:19:09.154 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_data_bytes": 0, 2024-03-18T23:19:09.154 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_data_items": 0, 2024-03-18T23:19:09.154 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-18T23:19:09.154 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_onode_items": 0, 2024-03-18T23:19:09.154 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-18T23:19:09.155 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_meta_items": 0, 2024-03-18T23:19:09.155 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_other_bytes": 0, 2024-03-18T23:19:09.155 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_other_items": 0, 2024-03-18T23:19:09.155 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-18T23:19:09.155 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_buffer_items": 0, 2024-03-18T23:19:09.155 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_extent_bytes": 0, 2024-03-18T23:19:09.155 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_extent_items": 0, 2024-03-18T23:19:09.155 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_blob_bytes": 0, 2024-03-18T23:19:09.156 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_blob_items": 0, 2024-03-18T23:19:09.156 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-18T23:19:09.156 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_shared_blob_items": 0, 2024-03-18T23:19:09.156 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-18T23:19:09.156 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_inline_bl_items": 0, 2024-03-18T23:19:09.156 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_fsck_bytes": 0, 2024-03-18T23:19:09.156 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_fsck_items": 0, 2024-03-18T23:19:09.156 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_txc_bytes": 0, 2024-03-18T23:19:09.156 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_txc_items": 0, 2024-03-18T23:19:09.157 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-18T23:19:09.157 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_writing_deferred_items": 0, 2024-03-18T23:19:09.157 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_writing_bytes": 0, 2024-03-18T23:19:09.157 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_writing_items": 0, 2024-03-18T23:19:09.157 INFO:teuthology.orchestra.run.smithi049.stdout: "bluefs_bytes": 0, 2024-03-18T23:19:09.157 INFO:teuthology.orchestra.run.smithi049.stdout: "bluefs_items": 0, 2024-03-18T23:19:09.157 INFO:teuthology.orchestra.run.smithi049.stdout: "bluefs_file_reader_bytes": 0, 2024-03-18T23:19:09.157 INFO:teuthology.orchestra.run.smithi049.stdout: "bluefs_file_reader_items": 0, 2024-03-18T23:19:09.158 INFO:teuthology.orchestra.run.smithi049.stdout: "bluefs_file_writer_bytes": 0, 2024-03-18T23:19:09.158 INFO:teuthology.orchestra.run.smithi049.stdout: "bluefs_file_writer_items": 0, 2024-03-18T23:19:09.158 INFO:teuthology.orchestra.run.smithi049.stdout: "buffer_anon_bytes": 189393, 2024-03-18T23:19:09.158 INFO:teuthology.orchestra.run.smithi049.stdout: "buffer_anon_items": 51, 2024-03-18T23:19:09.158 INFO:teuthology.orchestra.run.smithi049.stdout: "buffer_meta_bytes": 0, 2024-03-18T23:19:09.158 INFO:teuthology.orchestra.run.smithi049.stdout: "buffer_meta_items": 0, 2024-03-18T23:19:09.158 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_bytes": 0, 2024-03-18T23:19:09.158 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_items": 0, 2024-03-18T23:19:09.159 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_mapbl_bytes": 0, 2024-03-18T23:19:09.159 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_mapbl_items": 0, 2024-03-18T23:19:09.159 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_pglog_bytes": 0, 2024-03-18T23:19:09.159 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_pglog_items": 0, 2024-03-18T23:19:09.159 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_bytes": 8376, 2024-03-18T23:19:09.159 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_items": 191, 2024-03-18T23:19:09.159 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_mapping_bytes": 0, 2024-03-18T23:19:09.159 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_mapping_items": 0, 2024-03-18T23:19:09.160 INFO:teuthology.orchestra.run.smithi049.stdout: "pgmap_bytes": 0, 2024-03-18T23:19:09.160 INFO:teuthology.orchestra.run.smithi049.stdout: "pgmap_items": 0, 2024-03-18T23:19:09.160 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_co_bytes": 8464, 2024-03-18T23:19:09.160 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_co_items": 13, 2024-03-18T23:19:09.160 INFO:teuthology.orchestra.run.smithi049.stdout: "unittest_1_bytes": 0, 2024-03-18T23:19:09.160 INFO:teuthology.orchestra.run.smithi049.stdout: "unittest_1_items": 0, 2024-03-18T23:19:09.160 INFO:teuthology.orchestra.run.smithi049.stdout: "unittest_2_bytes": 0, 2024-03-18T23:19:09.160 INFO:teuthology.orchestra.run.smithi049.stdout: "unittest_2_items": 0 2024-03-18T23:19:09.160 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:09.161 INFO:teuthology.orchestra.run.smithi049.stdout: "objecter": { 2024-03-18T23:19:09.161 INFO:teuthology.orchestra.run.smithi049.stdout: "op_active": 0, 2024-03-18T23:19:09.161 INFO:teuthology.orchestra.run.smithi049.stdout: "op_laggy": 0, 2024-03-18T23:19:09.161 INFO:teuthology.orchestra.run.smithi049.stdout: "op_send": 2260, 2024-03-18T23:19:09.161 INFO:teuthology.orchestra.run.smithi049.stdout: "op_send_bytes": 21, 2024-03-18T23:19:09.161 INFO:teuthology.orchestra.run.smithi049.stdout: "op_resend": 3, 2024-03-18T23:19:09.161 INFO:teuthology.orchestra.run.smithi049.stdout: "op_reply": 2260, 2024-03-18T23:19:09.161 INFO:teuthology.orchestra.run.smithi049.stdout: "op_latency": { 2024-03-18T23:19:09.161 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 2260, 2024-03-18T23:19:09.162 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 1.482958161, 2024-03-18T23:19:09.162 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000656176 2024-03-18T23:19:09.162 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:09.162 INFO:teuthology.orchestra.run.smithi049.stdout: "op_inflight": 0, 2024-03-18T23:19:09.162 INFO:teuthology.orchestra.run.smithi049.stdout: "oplen_avg": { 2024-03-18T23:19:09.163 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 2260, 2024-03-18T23:19:09.163 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 2261 2024-03-18T23:19:09.163 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:09.163 INFO:teuthology.orchestra.run.smithi049.stdout: "op": 2260, 2024-03-18T23:19:09.163 INFO:teuthology.orchestra.run.smithi049.stdout: "op_r": 2260, 2024-03-18T23:19:09.163 INFO:teuthology.orchestra.run.smithi049.stdout: "op_w": 0, 2024-03-18T23:19:09.163 INFO:teuthology.orchestra.run.smithi049.stdout: "op_rmw": 0, 2024-03-18T23:19:09.163 INFO:teuthology.orchestra.run.smithi049.stdout: "op_pg": 0, 2024-03-18T23:19:09.164 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_stat": 1474, 2024-03-18T23:19:09.164 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_create": 0, 2024-03-18T23:19:09.164 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_read": 784, 2024-03-18T23:19:09.164 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_write": 0, 2024-03-18T23:19:09.164 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_writefull": 0, 2024-03-18T23:19:09.164 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_writesame": 0, 2024-03-18T23:19:09.164 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_append": 0, 2024-03-18T23:19:09.164 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_zero": 0, 2024-03-18T23:19:09.164 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_truncate": 0, 2024-03-18T23:19:09.164 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_delete": 0, 2024-03-18T23:19:09.165 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_mapext": 0, 2024-03-18T23:19:09.165 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_sparse_read": 0, 2024-03-18T23:19:09.165 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_clonerange": 0, 2024-03-18T23:19:09.165 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_getxattr": 1, 2024-03-18T23:19:09.165 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_setxattr": 0, 2024-03-18T23:19:09.165 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_cmpxattr": 0, 2024-03-18T23:19:09.165 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_rmxattr": 0, 2024-03-18T23:19:09.165 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_resetxattrs": 0, 2024-03-18T23:19:09.165 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_call": 0, 2024-03-18T23:19:09.166 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_watch": 0, 2024-03-18T23:19:09.166 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_notify": 0, 2024-03-18T23:19:09.166 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_src_cmpxattr": 0, 2024-03-18T23:19:09.166 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_pgls": 0, 2024-03-18T23:19:09.166 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_pgls_filter": 0, 2024-03-18T23:19:09.166 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_other": 0, 2024-03-18T23:19:09.166 INFO:teuthology.orchestra.run.smithi049.stdout: "linger_active": 0, 2024-03-18T23:19:09.166 INFO:teuthology.orchestra.run.smithi049.stdout: "linger_send": 0, 2024-03-18T23:19:09.167 INFO:teuthology.orchestra.run.smithi049.stdout: "linger_resend": 0, 2024-03-18T23:19:09.167 INFO:teuthology.orchestra.run.smithi049.stdout: "linger_ping": 0, 2024-03-18T23:19:09.167 INFO:teuthology.orchestra.run.smithi049.stdout: "poolop_active": 0, 2024-03-18T23:19:09.167 INFO:teuthology.orchestra.run.smithi049.stdout: "poolop_send": 0, 2024-03-18T23:19:09.167 INFO:teuthology.orchestra.run.smithi049.stdout: "poolop_resend": 0, 2024-03-18T23:19:09.167 INFO:teuthology.orchestra.run.smithi049.stdout: "poolstat_active": 0, 2024-03-18T23:19:09.167 INFO:teuthology.orchestra.run.smithi049.stdout: "poolstat_send": 0, 2024-03-18T23:19:09.167 INFO:teuthology.orchestra.run.smithi049.stdout: "poolstat_resend": 0, 2024-03-18T23:19:09.168 INFO:teuthology.orchestra.run.smithi049.stdout: "statfs_active": 0, 2024-03-18T23:19:09.168 INFO:teuthology.orchestra.run.smithi049.stdout: "statfs_send": 0, 2024-03-18T23:19:09.168 INFO:teuthology.orchestra.run.smithi049.stdout: "statfs_resend": 0, 2024-03-18T23:19:09.168 INFO:teuthology.orchestra.run.smithi049.stdout: "command_active": 0, 2024-03-18T23:19:09.168 INFO:teuthology.orchestra.run.smithi049.stdout: "command_send": 0, 2024-03-18T23:19:09.168 INFO:teuthology.orchestra.run.smithi049.stdout: "command_resend": 0, 2024-03-18T23:19:09.168 INFO:teuthology.orchestra.run.smithi049.stdout: "map_epoch": 100, 2024-03-18T23:19:09.168 INFO:teuthology.orchestra.run.smithi049.stdout: "map_full": 0, 2024-03-18T23:19:09.168 INFO:teuthology.orchestra.run.smithi049.stdout: "map_inc": 14, 2024-03-18T23:19:09.169 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_sessions": 5, 2024-03-18T23:19:09.169 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_session_open": 5, 2024-03-18T23:19:09.169 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_session_close": 0, 2024-03-18T23:19:09.169 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_laggy": 0, 2024-03-18T23:19:09.169 INFO:teuthology.orchestra.run.smithi049.stdout: "omap_wr": 0, 2024-03-18T23:19:09.169 INFO:teuthology.orchestra.run.smithi049.stdout: "omap_rd": 2, 2024-03-18T23:19:09.169 INFO:teuthology.orchestra.run.smithi049.stdout: "omap_del": 0 2024-03-18T23:19:09.169 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:09.169 INFO:teuthology.orchestra.run.smithi049.stdout: "oft": { 2024-03-18T23:19:09.170 INFO:teuthology.orchestra.run.smithi049.stdout: "omap_total_objs": 0, 2024-03-18T23:19:09.170 INFO:teuthology.orchestra.run.smithi049.stdout: "omap_total_kv_pairs": 0, 2024-03-18T23:19:09.170 INFO:teuthology.orchestra.run.smithi049.stdout: "omap_total_updates": 0, 2024-03-18T23:19:09.170 INFO:teuthology.orchestra.run.smithi049.stdout: "omap_total_removes": 0 2024-03-18T23:19:09.170 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:09.170 INFO:teuthology.orchestra.run.smithi049.stdout: "osd-slow-ops": { 2024-03-18T23:19:09.170 INFO:teuthology.orchestra.run.smithi049.stdout: "slow_ops_count": 0 2024-03-18T23:19:09.170 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:09.171 INFO:teuthology.orchestra.run.smithi049.stdout: "purge_queue": { 2024-03-18T23:19:09.171 INFO:teuthology.orchestra.run.smithi049.stdout: "pq_executing_ops": 0, 2024-03-18T23:19:09.171 INFO:teuthology.orchestra.run.smithi049.stdout: "pq_executing_ops_high_water": 0, 2024-03-18T23:19:09.171 INFO:teuthology.orchestra.run.smithi049.stdout: "pq_executing": 0, 2024-03-18T23:19:09.171 INFO:teuthology.orchestra.run.smithi049.stdout: "pq_executing_high_water": 0, 2024-03-18T23:19:09.171 INFO:teuthology.orchestra.run.smithi049.stdout: "pq_executed_ops": 0, 2024-03-18T23:19:09.171 INFO:teuthology.orchestra.run.smithi049.stdout: "pq_executed": 0, 2024-03-18T23:19:09.171 INFO:teuthology.orchestra.run.smithi049.stdout: "pq_item_in_journal": 0 2024-03-18T23:19:09.171 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:09.172 INFO:teuthology.orchestra.run.smithi049.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-18T23:19:09.172 INFO:teuthology.orchestra.run.smithi049.stdout: "val": 0, 2024-03-18T23:19:09.172 INFO:teuthology.orchestra.run.smithi049.stdout: "max": 104857600, 2024-03-18T23:19:09.172 INFO:teuthology.orchestra.run.smithi049.stdout: "get_started": 0, 2024-03-18T23:19:09.172 INFO:teuthology.orchestra.run.smithi049.stdout: "get": 2510, 2024-03-18T23:19:09.172 INFO:teuthology.orchestra.run.smithi049.stdout: "get_sum": 1898388, 2024-03-18T23:19:09.172 INFO:teuthology.orchestra.run.smithi049.stdout: "get_or_fail_fail": 0, 2024-03-18T23:19:09.172 INFO:teuthology.orchestra.run.smithi049.stdout: "get_or_fail_success": 2510, 2024-03-18T23:19:09.173 INFO:teuthology.orchestra.run.smithi049.stdout: "take": 0, 2024-03-18T23:19:09.173 INFO:teuthology.orchestra.run.smithi049.stdout: "take_sum": 0, 2024-03-18T23:19:09.173 INFO:teuthology.orchestra.run.smithi049.stdout: "put": 2510, 2024-03-18T23:19:09.173 INFO:teuthology.orchestra.run.smithi049.stdout: "put_sum": 1898388, 2024-03-18T23:19:09.173 INFO:teuthology.orchestra.run.smithi049.stdout: "wait": { 2024-03-18T23:19:09.173 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-03-18T23:19:09.173 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-03-18T23:19:09.173 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-03-18T23:19:09.173 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:19:09.174 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:09.174 INFO:teuthology.orchestra.run.smithi049.stdout: "throttle-objecter_bytes": { 2024-03-18T23:19:09.174 INFO:teuthology.orchestra.run.smithi049.stdout: "val": 0, 2024-03-18T23:19:09.174 INFO:teuthology.orchestra.run.smithi049.stdout: "max": 104857600, 2024-03-18T23:19:09.174 INFO:teuthology.orchestra.run.smithi049.stdout: "get_started": 0, 2024-03-18T23:19:09.174 INFO:teuthology.orchestra.run.smithi049.stdout: "get": 0, 2024-03-18T23:19:09.174 INFO:teuthology.orchestra.run.smithi049.stdout: "get_sum": 0, 2024-03-18T23:19:09.174 INFO:teuthology.orchestra.run.smithi049.stdout: "get_or_fail_fail": 0, 2024-03-18T23:19:09.175 INFO:teuthology.orchestra.run.smithi049.stdout: "get_or_fail_success": 0, 2024-03-18T23:19:09.175 INFO:teuthology.orchestra.run.smithi049.stdout: "take": 2260, 2024-03-18T23:19:09.175 INFO:teuthology.orchestra.run.smithi049.stdout: "take_sum": 1965, 2024-03-18T23:19:09.175 INFO:teuthology.orchestra.run.smithi049.stdout: "put": 2260, 2024-03-18T23:19:09.175 INFO:teuthology.orchestra.run.smithi049.stdout: "put_sum": 1965, 2024-03-18T23:19:09.175 INFO:teuthology.orchestra.run.smithi049.stdout: "wait": { 2024-03-18T23:19:09.175 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-03-18T23:19:09.175 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-03-18T23:19:09.175 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-03-18T23:19:09.176 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:19:09.176 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:09.176 INFO:teuthology.orchestra.run.smithi049.stdout: "throttle-objecter_ops": { 2024-03-18T23:19:09.176 INFO:teuthology.orchestra.run.smithi049.stdout: "val": 0, 2024-03-18T23:19:09.176 INFO:teuthology.orchestra.run.smithi049.stdout: "max": 1024, 2024-03-18T23:19:09.176 INFO:teuthology.orchestra.run.smithi049.stdout: "get_started": 0, 2024-03-18T23:19:09.176 INFO:teuthology.orchestra.run.smithi049.stdout: "get": 0, 2024-03-18T23:19:09.176 INFO:teuthology.orchestra.run.smithi049.stdout: "get_sum": 0, 2024-03-18T23:19:09.177 INFO:teuthology.orchestra.run.smithi049.stdout: "get_or_fail_fail": 0, 2024-03-18T23:19:09.177 INFO:teuthology.orchestra.run.smithi049.stdout: "get_or_fail_success": 0, 2024-03-18T23:19:09.177 INFO:teuthology.orchestra.run.smithi049.stdout: "take": 2260, 2024-03-18T23:19:09.177 INFO:teuthology.orchestra.run.smithi049.stdout: "take_sum": 2260, 2024-03-18T23:19:09.177 INFO:teuthology.orchestra.run.smithi049.stdout: "put": 2260, 2024-03-18T23:19:09.177 INFO:teuthology.orchestra.run.smithi049.stdout: "put_sum": 2260, 2024-03-18T23:19:09.177 INFO:teuthology.orchestra.run.smithi049.stdout: "wait": { 2024-03-18T23:19:09.177 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-03-18T23:19:09.177 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-03-18T23:19:09.178 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-03-18T23:19:09.178 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:19:09.178 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:09.178 INFO:teuthology.orchestra.run.smithi049.stdout: "throttle-write_buf_throttle": { 2024-03-18T23:19:09.178 INFO:teuthology.orchestra.run.smithi049.stdout: "val": 0, 2024-03-18T23:19:09.178 INFO:teuthology.orchestra.run.smithi049.stdout: "max": 3758096384, 2024-03-18T23:19:09.178 INFO:teuthology.orchestra.run.smithi049.stdout: "get_started": 0, 2024-03-18T23:19:09.178 INFO:teuthology.orchestra.run.smithi049.stdout: "get": 0, 2024-03-18T23:19:09.179 INFO:teuthology.orchestra.run.smithi049.stdout: "get_sum": 0, 2024-03-18T23:19:09.179 INFO:teuthology.orchestra.run.smithi049.stdout: "get_or_fail_fail": 0, 2024-03-18T23:19:09.179 INFO:teuthology.orchestra.run.smithi049.stdout: "get_or_fail_success": 0, 2024-03-18T23:19:09.179 INFO:teuthology.orchestra.run.smithi049.stdout: "take": 0, 2024-03-18T23:19:09.179 INFO:teuthology.orchestra.run.smithi049.stdout: "take_sum": 0, 2024-03-18T23:19:09.179 INFO:teuthology.orchestra.run.smithi049.stdout: "put": 0, 2024-03-18T23:19:09.179 INFO:teuthology.orchestra.run.smithi049.stdout: "put_sum": 0, 2024-03-18T23:19:09.179 INFO:teuthology.orchestra.run.smithi049.stdout: "wait": { 2024-03-18T23:19:09.179 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-03-18T23:19:09.180 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-03-18T23:19:09.180 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-03-18T23:19:09.180 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:19:09.180 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:09.180 INFO:teuthology.orchestra.run.smithi049.stdout: "throttle-write_buf_throttle-0x5567c16a06e0": { 2024-03-18T23:19:09.180 INFO:teuthology.orchestra.run.smithi049.stdout: "val": 0, 2024-03-18T23:19:09.180 INFO:teuthology.orchestra.run.smithi049.stdout: "max": 3758096384, 2024-03-18T23:19:09.180 INFO:teuthology.orchestra.run.smithi049.stdout: "get_started": 0, 2024-03-18T23:19:09.180 INFO:teuthology.orchestra.run.smithi049.stdout: "get": 0, 2024-03-18T23:19:09.181 INFO:teuthology.orchestra.run.smithi049.stdout: "get_sum": 0, 2024-03-18T23:19:09.181 INFO:teuthology.orchestra.run.smithi049.stdout: "get_or_fail_fail": 0, 2024-03-18T23:19:09.181 INFO:teuthology.orchestra.run.smithi049.stdout: "get_or_fail_success": 0, 2024-03-18T23:19:09.181 INFO:teuthology.orchestra.run.smithi049.stdout: "take": 0, 2024-03-18T23:19:09.181 INFO:teuthology.orchestra.run.smithi049.stdout: "take_sum": 0, 2024-03-18T23:19:09.181 INFO:teuthology.orchestra.run.smithi049.stdout: "put": 0, 2024-03-18T23:19:09.181 INFO:teuthology.orchestra.run.smithi049.stdout: "put_sum": 0, 2024-03-18T23:19:09.181 INFO:teuthology.orchestra.run.smithi049.stdout: "wait": { 2024-03-18T23:19:09.181 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-03-18T23:19:09.182 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-03-18T23:19:09.182 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-03-18T23:19:09.182 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:19:09.182 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:19:09.182 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-03-18T23:19:09.379 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:19:09 smithi049 ceph-mon[62991]: pgmap v340: 97 pgs: 23 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 33 active+clean; 75 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.9 MiB/s wr, 177 op/s 2024-03-18T23:19:09.460 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:19:09 smithi175 ceph-mon[67352]: pgmap v340: 97 pgs: 23 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 33 active+clean; 75 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.9 MiB/s wr, 177 op/s 2024-03-18T23:19:09.564 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:19:09 smithi139 ceph-mon[67282]: pgmap v340: 97 pgs: 23 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 33 active+clean; 75 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.9 MiB/s wr, 177 op/s 2024-03-18T23:19:10.573 INFO:tasks.check_counter:Daemon mds.a mds.exported=0 2024-03-18T23:19:10.574 INFO:tasks.check_counter:Daemon mds.a mds.imported=0 2024-03-18T23:19:10.574 INFO:tasks.check_counter:Daemon mds.a mds.root_rsnaps=0 2024-03-18T23:19:10.574 INFO:tasks.check_counter:Daemon mds.a mds_server.req_mksnap_latency.avgcount=0 2024-03-18T23:19:10.574 INFO:tasks.check_counter:Daemon mds.a mds_server.req_rmsnap_latency.avgcount=0 2024-03-18T23:19:10.574 DEBUG:tasks.check_counter:Getting stats from d 2024-03-18T23:19:10.575 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell --fsid c63591ea-e57a-11ee-95c9-87774f69a715 -- ceph daemon mds.d perf dump 2024-03-18T23:19:10.830 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T23:19:11.460 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:19:11 smithi175 ceph-mon[67352]: pgmap v341: 97 pgs: 23 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 33 active+clean; 75 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.9 MiB/s wr, 177 op/s 2024-03-18T23:19:11.564 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:19:11 smithi139 ceph-mon[67282]: pgmap v341: 97 pgs: 23 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 33 active+clean; 75 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.9 MiB/s wr, 177 op/s 2024-03-18T23:19:11.629 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:19:11 smithi049 ceph-mon[62991]: pgmap v341: 97 pgs: 23 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 33 active+clean; 75 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.9 MiB/s wr, 177 op/s 2024-03-18T23:19:13.265 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-03-18T23:19:13.266 INFO:teuthology.orchestra.run.smithi049.stdout: "AsyncMessenger::Worker-0": { 2024-03-18T23:19:13.266 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_messages": 0, 2024-03-18T23:19:13.266 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_messages": 2, 2024-03-18T23:19:13.266 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_bytes": 0, 2024-03-18T23:19:13.266 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_bytes": 0, 2024-03-18T23:19:13.267 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_created_connections": 4, 2024-03-18T23:19:13.267 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_active_connections": 0, 2024-03-18T23:19:13.267 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_total_time": 0.001429853, 2024-03-18T23:19:13.267 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_send_time": 0.000000000, 2024-03-18T23:19:13.267 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_recv_time": 0.000248744, 2024-03-18T23:19:13.267 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-18T23:19:13.267 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_messages_queue_lat": { 2024-03-18T23:19:13.267 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-03-18T23:19:13.267 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-03-18T23:19:13.268 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-03-18T23:19:13.268 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:13.268 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_handle_ack_lat": { 2024-03-18T23:19:13.268 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-03-18T23:19:13.268 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-03-18T23:19:13.268 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-03-18T23:19:13.268 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:13.268 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-18T23:19:13.268 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_encrypted_bytes": 0 2024-03-18T23:19:13.269 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:13.269 INFO:teuthology.orchestra.run.smithi049.stdout: "AsyncMessenger::Worker-1": { 2024-03-18T23:19:13.269 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_messages": 229, 2024-03-18T23:19:13.269 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_messages": 219, 2024-03-18T23:19:13.269 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_bytes": 1439600, 2024-03-18T23:19:13.269 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_bytes": 134464, 2024-03-18T23:19:13.269 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_created_connections": 2, 2024-03-18T23:19:13.269 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_active_connections": 1, 2024-03-18T23:19:13.269 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_total_time": 0.038699909, 2024-03-18T23:19:13.270 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_send_time": 0.011548724, 2024-03-18T23:19:13.270 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_recv_time": 0.031914725, 2024-03-18T23:19:13.270 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_fast_dispatch_time": 0.003718354, 2024-03-18T23:19:13.270 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_messages_queue_lat": { 2024-03-18T23:19:13.271 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 218, 2024-03-18T23:19:13.271 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.008520380, 2024-03-18T23:19:13.271 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000039084 2024-03-18T23:19:13.271 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:13.271 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_handle_ack_lat": { 2024-03-18T23:19:13.271 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-03-18T23:19:13.271 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-03-18T23:19:13.271 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-03-18T23:19:13.272 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:13.272 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_encrypted_bytes": 1439600, 2024-03-18T23:19:13.272 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_encrypted_bytes": 134464 2024-03-18T23:19:13.272 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:13.272 INFO:teuthology.orchestra.run.smithi049.stdout: "AsyncMessenger::Worker-2": { 2024-03-18T23:19:13.272 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_messages": 6, 2024-03-18T23:19:13.272 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_messages": 169, 2024-03-18T23:19:13.272 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_bytes": 3344, 2024-03-18T23:19:13.272 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_bytes": 185504, 2024-03-18T23:19:13.273 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_created_connections": 5, 2024-03-18T23:19:13.273 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_active_connections": 3, 2024-03-18T23:19:13.273 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_total_time": 0.012558799, 2024-03-18T23:19:13.273 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_send_time": 0.008181737, 2024-03-18T23:19:13.273 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_recv_time": 0.002183519, 2024-03-18T23:19:13.273 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-18T23:19:13.273 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_messages_queue_lat": { 2024-03-18T23:19:13.273 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 167, 2024-03-18T23:19:13.273 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.008880885, 2024-03-18T23:19:13.274 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000053178 2024-03-18T23:19:13.274 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:13.274 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_handle_ack_lat": { 2024-03-18T23:19:13.274 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-03-18T23:19:13.274 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-03-18T23:19:13.274 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-03-18T23:19:13.274 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:13.274 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_encrypted_bytes": 3344, 2024-03-18T23:19:13.275 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_encrypted_bytes": 185504 2024-03-18T23:19:13.275 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:13.275 INFO:teuthology.orchestra.run.smithi049.stdout: "cct": { 2024-03-18T23:19:13.275 INFO:teuthology.orchestra.run.smithi049.stdout: "total_workers": 0, 2024-03-18T23:19:13.275 INFO:teuthology.orchestra.run.smithi049.stdout: "unhealthy_workers": 0 2024-03-18T23:19:13.275 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:13.275 INFO:teuthology.orchestra.run.smithi049.stdout: "mempool": { 2024-03-18T23:19:13.275 INFO:teuthology.orchestra.run.smithi049.stdout: "bloom_filter_bytes": 0, 2024-03-18T23:19:13.275 INFO:teuthology.orchestra.run.smithi049.stdout: "bloom_filter_items": 0, 2024-03-18T23:19:13.276 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_alloc_bytes": 0, 2024-03-18T23:19:13.276 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_alloc_items": 0, 2024-03-18T23:19:13.276 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_data_bytes": 0, 2024-03-18T23:19:13.276 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_data_items": 0, 2024-03-18T23:19:13.276 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-18T23:19:13.276 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_onode_items": 0, 2024-03-18T23:19:13.276 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-18T23:19:13.276 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_meta_items": 0, 2024-03-18T23:19:13.276 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_other_bytes": 0, 2024-03-18T23:19:13.277 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_other_items": 0, 2024-03-18T23:19:13.277 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-18T23:19:13.277 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_buffer_items": 0, 2024-03-18T23:19:13.277 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_extent_bytes": 0, 2024-03-18T23:19:13.277 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_extent_items": 0, 2024-03-18T23:19:13.277 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_blob_bytes": 0, 2024-03-18T23:19:13.277 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_blob_items": 0, 2024-03-18T23:19:13.277 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-18T23:19:13.277 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_shared_blob_items": 0, 2024-03-18T23:19:13.278 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-18T23:19:13.278 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_inline_bl_items": 0, 2024-03-18T23:19:13.278 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_fsck_bytes": 0, 2024-03-18T23:19:13.278 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_fsck_items": 0, 2024-03-18T23:19:13.278 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_txc_bytes": 0, 2024-03-18T23:19:13.278 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_txc_items": 0, 2024-03-18T23:19:13.278 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-18T23:19:13.278 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_writing_deferred_items": 0, 2024-03-18T23:19:13.279 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_writing_bytes": 0, 2024-03-18T23:19:13.279 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_writing_items": 0, 2024-03-18T23:19:13.279 INFO:teuthology.orchestra.run.smithi049.stdout: "bluefs_bytes": 0, 2024-03-18T23:19:13.279 INFO:teuthology.orchestra.run.smithi049.stdout: "bluefs_items": 0, 2024-03-18T23:19:13.279 INFO:teuthology.orchestra.run.smithi049.stdout: "bluefs_file_reader_bytes": 0, 2024-03-18T23:19:13.279 INFO:teuthology.orchestra.run.smithi049.stdout: "bluefs_file_reader_items": 0, 2024-03-18T23:19:13.279 INFO:teuthology.orchestra.run.smithi049.stdout: "bluefs_file_writer_bytes": 0, 2024-03-18T23:19:13.279 INFO:teuthology.orchestra.run.smithi049.stdout: "bluefs_file_writer_items": 0, 2024-03-18T23:19:13.279 INFO:teuthology.orchestra.run.smithi049.stdout: "buffer_anon_bytes": 148994, 2024-03-18T23:19:13.280 INFO:teuthology.orchestra.run.smithi049.stdout: "buffer_anon_items": 31, 2024-03-18T23:19:13.280 INFO:teuthology.orchestra.run.smithi049.stdout: "buffer_meta_bytes": 0, 2024-03-18T23:19:13.280 INFO:teuthology.orchestra.run.smithi049.stdout: "buffer_meta_items": 0, 2024-03-18T23:19:13.280 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_bytes": 0, 2024-03-18T23:19:13.280 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_items": 0, 2024-03-18T23:19:13.280 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_mapbl_bytes": 0, 2024-03-18T23:19:13.280 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_mapbl_items": 0, 2024-03-18T23:19:13.280 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_pglog_bytes": 0, 2024-03-18T23:19:13.280 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_pglog_items": 0, 2024-03-18T23:19:13.280 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_bytes": 0, 2024-03-18T23:19:13.281 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_items": 0, 2024-03-18T23:19:13.281 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_mapping_bytes": 0, 2024-03-18T23:19:13.281 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_mapping_items": 0, 2024-03-18T23:19:13.281 INFO:teuthology.orchestra.run.smithi049.stdout: "pgmap_bytes": 0, 2024-03-18T23:19:13.281 INFO:teuthology.orchestra.run.smithi049.stdout: "pgmap_items": 0, 2024-03-18T23:19:13.281 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_co_bytes": 928, 2024-03-18T23:19:13.281 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_co_items": 2, 2024-03-18T23:19:13.281 INFO:teuthology.orchestra.run.smithi049.stdout: "unittest_1_bytes": 0, 2024-03-18T23:19:13.282 INFO:teuthology.orchestra.run.smithi049.stdout: "unittest_1_items": 0, 2024-03-18T23:19:13.282 INFO:teuthology.orchestra.run.smithi049.stdout: "unittest_2_bytes": 0, 2024-03-18T23:19:13.282 INFO:teuthology.orchestra.run.smithi049.stdout: "unittest_2_items": 0 2024-03-18T23:19:13.282 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:13.282 INFO:teuthology.orchestra.run.smithi049.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-18T23:19:13.282 INFO:teuthology.orchestra.run.smithi049.stdout: "val": 0, 2024-03-18T23:19:13.282 INFO:teuthology.orchestra.run.smithi049.stdout: "max": 104857600, 2024-03-18T23:19:13.283 INFO:teuthology.orchestra.run.smithi049.stdout: "get_started": 0, 2024-03-18T23:19:13.283 INFO:teuthology.orchestra.run.smithi049.stdout: "get": 231, 2024-03-18T23:19:13.283 INFO:teuthology.orchestra.run.smithi049.stdout: "get_sum": 1407327, 2024-03-18T23:19:13.283 INFO:teuthology.orchestra.run.smithi049.stdout: "get_or_fail_fail": 0, 2024-03-18T23:19:13.283 INFO:teuthology.orchestra.run.smithi049.stdout: "get_or_fail_success": 231, 2024-03-18T23:19:13.283 INFO:teuthology.orchestra.run.smithi049.stdout: "take": 0, 2024-03-18T23:19:13.283 INFO:teuthology.orchestra.run.smithi049.stdout: "take_sum": 0, 2024-03-18T23:19:13.283 INFO:teuthology.orchestra.run.smithi049.stdout: "put": 231, 2024-03-18T23:19:13.283 INFO:teuthology.orchestra.run.smithi049.stdout: "put_sum": 1407327, 2024-03-18T23:19:13.284 INFO:teuthology.orchestra.run.smithi049.stdout: "wait": { 2024-03-18T23:19:13.284 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-03-18T23:19:13.284 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-03-18T23:19:13.284 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-03-18T23:19:13.284 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:19:13.284 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:19:13.284 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-03-18T23:19:13.460 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:19:13 smithi175 ceph-mon[67352]: pgmap v342: 97 pgs: 12 active+clean+snaptrim, 21 active+clean+snaptrim_wait, 64 active+clean; 72 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.4 MiB/s wr, 85 op/s 2024-03-18T23:19:13.564 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:19:13 smithi139 ceph-mon[67282]: pgmap v342: 97 pgs: 12 active+clean+snaptrim, 21 active+clean+snaptrim_wait, 64 active+clean; 72 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.4 MiB/s wr, 85 op/s 2024-03-18T23:19:13.629 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:19:13 smithi049 ceph-mon[62991]: pgmap v342: 97 pgs: 12 active+clean+snaptrim, 21 active+clean+snaptrim_wait, 64 active+clean; 72 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.4 MiB/s wr, 85 op/s 2024-03-18T23:19:13.960 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-03-18T23:19:13.960 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-03-18T23:19:13.960 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.d 2024-03-18T23:19:13.960 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.d 2024-03-18T23:19:13.960 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.d 2024-03-18T23:19:13.960 DEBUG:tasks.check_counter:Getting stats from g 2024-03-18T23:19:13.961 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell --fsid c63591ea-e57a-11ee-95c9-87774f69a715 -- ceph daemon mds.g perf dump 2024-03-18T23:19:14.216 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T23:19:15.379 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:19:15 smithi049 ceph-mon[62991]: pgmap v343: 97 pgs: 97 active+clean; 64 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 1.3 MiB/s wr, 61 op/s 2024-03-18T23:19:15.460 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:19:15 smithi175 ceph-mon[67352]: pgmap v343: 97 pgs: 97 active+clean; 64 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 1.3 MiB/s wr, 61 op/s 2024-03-18T23:19:15.564 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:19:15 smithi139 ceph-mon[67282]: pgmap v343: 97 pgs: 97 active+clean; 64 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 1.3 MiB/s wr, 61 op/s 2024-03-18T23:19:16.002 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-03-18T23:19:16.003 INFO:teuthology.orchestra.run.smithi049.stdout: "AsyncMessenger::Worker-0": { 2024-03-18T23:19:16.003 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_messages": 788, 2024-03-18T23:19:16.003 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_messages": 788, 2024-03-18T23:19:16.003 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_bytes": 266393, 2024-03-18T23:19:16.003 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_bytes": 249761, 2024-03-18T23:19:16.003 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_created_connections": 6, 2024-03-18T23:19:16.004 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_active_connections": 2, 2024-03-18T23:19:16.004 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_total_time": 0.075058155, 2024-03-18T23:19:16.004 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_send_time": 0.030314247, 2024-03-18T23:19:16.004 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_recv_time": 0.016703421, 2024-03-18T23:19:16.004 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_fast_dispatch_time": 0.019909719, 2024-03-18T23:19:16.004 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_messages_queue_lat": { 2024-03-18T23:19:16.004 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 787, 2024-03-18T23:19:16.004 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.032007031, 2024-03-18T23:19:16.005 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000040669 2024-03-18T23:19:16.005 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:16.005 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_handle_ack_lat": { 2024-03-18T23:19:16.005 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-03-18T23:19:16.005 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-03-18T23:19:16.005 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-03-18T23:19:16.005 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:16.005 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_encrypted_bytes": 3024, 2024-03-18T23:19:16.006 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_encrypted_bytes": 448 2024-03-18T23:19:16.006 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:16.006 INFO:teuthology.orchestra.run.smithi049.stdout: "AsyncMessenger::Worker-1": { 2024-03-18T23:19:16.006 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_messages": 1698, 2024-03-18T23:19:16.006 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_messages": 1683, 2024-03-18T23:19:16.006 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_bytes": 1866683, 2024-03-18T23:19:16.006 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_bytes": 596361, 2024-03-18T23:19:16.006 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_created_connections": 4, 2024-03-18T23:19:16.007 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_active_connections": 3, 2024-03-18T23:19:16.007 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_total_time": 0.114261932, 2024-03-18T23:19:16.007 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_send_time": 0.033222555, 2024-03-18T23:19:16.007 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_recv_time": 0.051121511, 2024-03-18T23:19:16.007 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_fast_dispatch_time": 0.033129001, 2024-03-18T23:19:16.007 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_messages_queue_lat": { 2024-03-18T23:19:16.007 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 1682, 2024-03-18T23:19:16.007 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.042293682, 2024-03-18T23:19:16.007 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000025144 2024-03-18T23:19:16.008 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:16.008 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_handle_ack_lat": { 2024-03-18T23:19:16.008 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-03-18T23:19:16.008 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-03-18T23:19:16.008 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-03-18T23:19:16.008 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:16.008 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_encrypted_bytes": 1496064, 2024-03-18T23:19:16.008 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_encrypted_bytes": 135840 2024-03-18T23:19:16.008 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:16.009 INFO:teuthology.orchestra.run.smithi049.stdout: "AsyncMessenger::Worker-2": { 2024-03-18T23:19:16.009 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_messages": 5, 2024-03-18T23:19:16.009 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_messages": 167, 2024-03-18T23:19:16.009 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_bytes": 1668, 2024-03-18T23:19:16.009 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_bytes": 454533, 2024-03-18T23:19:16.009 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_created_connections": 7, 2024-03-18T23:19:16.009 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_active_connections": 5, 2024-03-18T23:19:16.009 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_total_time": 0.011442943, 2024-03-18T23:19:16.009 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_send_time": 0.006887035, 2024-03-18T23:19:16.009 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_recv_time": 0.002546919, 2024-03-18T23:19:16.010 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_fast_dispatch_time": 0.000058059, 2024-03-18T23:19:16.010 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_messages_queue_lat": { 2024-03-18T23:19:16.010 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 164, 2024-03-18T23:19:16.010 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.009096391, 2024-03-18T23:19:16.010 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000055465 2024-03-18T23:19:16.010 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:16.010 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_handle_ack_lat": { 2024-03-18T23:19:16.010 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-03-18T23:19:16.011 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-03-18T23:19:16.011 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-03-18T23:19:16.011 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:16.011 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_encrypted_bytes": 320, 2024-03-18T23:19:16.011 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_encrypted_bytes": 453568 2024-03-18T23:19:16.011 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:16.011 INFO:teuthology.orchestra.run.smithi049.stdout: "cct": { 2024-03-18T23:19:16.011 INFO:teuthology.orchestra.run.smithi049.stdout: "total_workers": 1, 2024-03-18T23:19:16.011 INFO:teuthology.orchestra.run.smithi049.stdout: "unhealthy_workers": 0 2024-03-18T23:19:16.012 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:16.012 INFO:teuthology.orchestra.run.smithi049.stdout: "finisher-MDSRank": { 2024-03-18T23:19:16.012 INFO:teuthology.orchestra.run.smithi049.stdout: "queue_len": 0, 2024-03-18T23:19:16.012 INFO:teuthology.orchestra.run.smithi049.stdout: "complete_latency": { 2024-03-18T23:19:16.012 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 3793, 2024-03-18T23:19:16.012 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.080685555, 2024-03-18T23:19:16.012 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000021272 2024-03-18T23:19:16.012 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:19:16.013 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:16.013 INFO:teuthology.orchestra.run.smithi049.stdout: "finisher-PurgeQueue": { 2024-03-18T23:19:16.013 INFO:teuthology.orchestra.run.smithi049.stdout: "queue_len": 0, 2024-03-18T23:19:16.013 INFO:teuthology.orchestra.run.smithi049.stdout: "complete_latency": { 2024-03-18T23:19:16.013 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-03-18T23:19:16.013 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-03-18T23:19:16.013 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-03-18T23:19:16.013 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:19:16.014 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:16.014 INFO:teuthology.orchestra.run.smithi049.stdout: "mds": { 2024-03-18T23:19:16.014 INFO:teuthology.orchestra.run.smithi049.stdout: "request": 0, 2024-03-18T23:19:16.014 INFO:teuthology.orchestra.run.smithi049.stdout: "reply": 0, 2024-03-18T23:19:16.014 INFO:teuthology.orchestra.run.smithi049.stdout: "reply_latency": { 2024-03-18T23:19:16.014 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-03-18T23:19:16.014 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-03-18T23:19:16.014 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-03-18T23:19:16.014 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:16.015 INFO:teuthology.orchestra.run.smithi049.stdout: "slow_reply": 0, 2024-03-18T23:19:16.015 INFO:teuthology.orchestra.run.smithi049.stdout: "forward": 0, 2024-03-18T23:19:16.015 INFO:teuthology.orchestra.run.smithi049.stdout: "dir_fetch_complete": 0, 2024-03-18T23:19:16.015 INFO:teuthology.orchestra.run.smithi049.stdout: "dir_fetch_keys": 0, 2024-03-18T23:19:16.015 INFO:teuthology.orchestra.run.smithi049.stdout: "dir_commit": 0, 2024-03-18T23:19:16.015 INFO:teuthology.orchestra.run.smithi049.stdout: "dir_split": 0, 2024-03-18T23:19:16.015 INFO:teuthology.orchestra.run.smithi049.stdout: "dir_merge": 0, 2024-03-18T23:19:16.015 INFO:teuthology.orchestra.run.smithi049.stdout: "inodes": 0, 2024-03-18T23:19:16.016 INFO:teuthology.orchestra.run.smithi049.stdout: "inodes_top": 0, 2024-03-18T23:19:16.016 INFO:teuthology.orchestra.run.smithi049.stdout: "inodes_bottom": 0, 2024-03-18T23:19:16.016 INFO:teuthology.orchestra.run.smithi049.stdout: "inodes_pin_tail": 0, 2024-03-18T23:19:16.016 INFO:teuthology.orchestra.run.smithi049.stdout: "inodes_pinned": 0, 2024-03-18T23:19:16.016 INFO:teuthology.orchestra.run.smithi049.stdout: "inodes_expired": 0, 2024-03-18T23:19:16.016 INFO:teuthology.orchestra.run.smithi049.stdout: "inodes_with_caps": 0, 2024-03-18T23:19:16.016 INFO:teuthology.orchestra.run.smithi049.stdout: "caps": 0, 2024-03-18T23:19:16.016 INFO:teuthology.orchestra.run.smithi049.stdout: "subtrees": 1, 2024-03-18T23:19:16.017 INFO:teuthology.orchestra.run.smithi049.stdout: "traverse": 0, 2024-03-18T23:19:16.017 INFO:teuthology.orchestra.run.smithi049.stdout: "traverse_hit": 0, 2024-03-18T23:19:16.017 INFO:teuthology.orchestra.run.smithi049.stdout: "traverse_forward": 0, 2024-03-18T23:19:16.017 INFO:teuthology.orchestra.run.smithi049.stdout: "traverse_discover": 0, 2024-03-18T23:19:16.017 INFO:teuthology.orchestra.run.smithi049.stdout: "traverse_dir_fetch": 0, 2024-03-18T23:19:16.017 INFO:teuthology.orchestra.run.smithi049.stdout: "traverse_remote_ino": 0, 2024-03-18T23:19:16.017 INFO:teuthology.orchestra.run.smithi049.stdout: "traverse_lock": 0, 2024-03-18T23:19:16.017 INFO:teuthology.orchestra.run.smithi049.stdout: "load_cent": 0, 2024-03-18T23:19:16.017 INFO:teuthology.orchestra.run.smithi049.stdout: "q": 0, 2024-03-18T23:19:16.018 INFO:teuthology.orchestra.run.smithi049.stdout: "exported": 0, 2024-03-18T23:19:16.018 INFO:teuthology.orchestra.run.smithi049.stdout: "exported_inodes": 0, 2024-03-18T23:19:16.018 INFO:teuthology.orchestra.run.smithi049.stdout: "imported": 0, 2024-03-18T23:19:16.018 INFO:teuthology.orchestra.run.smithi049.stdout: "imported_inodes": 0, 2024-03-18T23:19:16.018 INFO:teuthology.orchestra.run.smithi049.stdout: "openino_dir_fetch": 0, 2024-03-18T23:19:16.018 INFO:teuthology.orchestra.run.smithi049.stdout: "openino_backtrace_fetch": 0, 2024-03-18T23:19:16.018 INFO:teuthology.orchestra.run.smithi049.stdout: "openino_peer_discover": 0, 2024-03-18T23:19:16.018 INFO:teuthology.orchestra.run.smithi049.stdout: "root_rfiles": 0, 2024-03-18T23:19:16.019 INFO:teuthology.orchestra.run.smithi049.stdout: "root_rbytes": 0, 2024-03-18T23:19:16.019 INFO:teuthology.orchestra.run.smithi049.stdout: "root_rsnaps": 0, 2024-03-18T23:19:16.019 INFO:teuthology.orchestra.run.smithi049.stdout: "scrub_backtrace_fetch": 0, 2024-03-18T23:19:16.019 INFO:teuthology.orchestra.run.smithi049.stdout: "scrub_set_tag": 0, 2024-03-18T23:19:16.019 INFO:teuthology.orchestra.run.smithi049.stdout: "scrub_backtrace_repaired": 0, 2024-03-18T23:19:16.019 INFO:teuthology.orchestra.run.smithi049.stdout: "scrub_inotable_repaired": 0, 2024-03-18T23:19:16.019 INFO:teuthology.orchestra.run.smithi049.stdout: "scrub_dir_inodes": 0, 2024-03-18T23:19:16.019 INFO:teuthology.orchestra.run.smithi049.stdout: "scrub_dir_base_inodes": 0, 2024-03-18T23:19:16.019 INFO:teuthology.orchestra.run.smithi049.stdout: "scrub_dirfrag_rstats": 0, 2024-03-18T23:19:16.019 INFO:teuthology.orchestra.run.smithi049.stdout: "scrub_file_inodes": 0, 2024-03-18T23:19:16.020 INFO:teuthology.orchestra.run.smithi049.stdout: "handle_inode_file_caps": 0, 2024-03-18T23:19:16.020 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph_cap_op_revoke": 0, 2024-03-18T23:19:16.020 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph_cap_op_grant": 0, 2024-03-18T23:19:16.020 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph_cap_op_trunc": 0, 2024-03-18T23:19:16.020 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-18T23:19:16.020 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-18T23:19:16.020 INFO:teuthology.orchestra.run.smithi049.stdout: "handle_client_caps": 0, 2024-03-18T23:19:16.020 INFO:teuthology.orchestra.run.smithi049.stdout: "handle_client_caps_dirty": 0, 2024-03-18T23:19:16.020 INFO:teuthology.orchestra.run.smithi049.stdout: "handle_client_cap_release": 0, 2024-03-18T23:19:16.021 INFO:teuthology.orchestra.run.smithi049.stdout: "process_request_cap_release": 0 2024-03-18T23:19:16.021 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:16.021 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_cache": { 2024-03-18T23:19:16.021 INFO:teuthology.orchestra.run.smithi049.stdout: "dir_update": 0, 2024-03-18T23:19:16.021 INFO:teuthology.orchestra.run.smithi049.stdout: "dir_update_receipt": 0, 2024-03-18T23:19:16.021 INFO:teuthology.orchestra.run.smithi049.stdout: "dir_try_discover": 0, 2024-03-18T23:19:16.021 INFO:teuthology.orchestra.run.smithi049.stdout: "dir_send_discover": 0, 2024-03-18T23:19:16.021 INFO:teuthology.orchestra.run.smithi049.stdout: "dir_handle_discover": 0, 2024-03-18T23:19:16.021 INFO:teuthology.orchestra.run.smithi049.stdout: "num_strays": 0, 2024-03-18T23:19:16.022 INFO:teuthology.orchestra.run.smithi049.stdout: "num_strays_delayed": 0, 2024-03-18T23:19:16.022 INFO:teuthology.orchestra.run.smithi049.stdout: "num_strays_enqueuing": 0, 2024-03-18T23:19:16.022 INFO:teuthology.orchestra.run.smithi049.stdout: "strays_created": 0, 2024-03-18T23:19:16.022 INFO:teuthology.orchestra.run.smithi049.stdout: "strays_enqueued": 0, 2024-03-18T23:19:16.022 INFO:teuthology.orchestra.run.smithi049.stdout: "strays_reintegrated": 0, 2024-03-18T23:19:16.022 INFO:teuthology.orchestra.run.smithi049.stdout: "strays_migrated": 0, 2024-03-18T23:19:16.022 INFO:teuthology.orchestra.run.smithi049.stdout: "num_recovering_processing": 0, 2024-03-18T23:19:16.022 INFO:teuthology.orchestra.run.smithi049.stdout: "num_recovering_enqueued": 0, 2024-03-18T23:19:16.022 INFO:teuthology.orchestra.run.smithi049.stdout: "num_recovering_prioritized": 0, 2024-03-18T23:19:16.023 INFO:teuthology.orchestra.run.smithi049.stdout: "recovery_started": 0, 2024-03-18T23:19:16.023 INFO:teuthology.orchestra.run.smithi049.stdout: "recovery_completed": 0, 2024-03-18T23:19:16.023 INFO:teuthology.orchestra.run.smithi049.stdout: "ireq_quiesce_path": 0, 2024-03-18T23:19:16.023 INFO:teuthology.orchestra.run.smithi049.stdout: "ireq_quiesce_inode": 0, 2024-03-18T23:19:16.023 INFO:teuthology.orchestra.run.smithi049.stdout: "ireq_enqueue_scrub": 0, 2024-03-18T23:19:16.023 INFO:teuthology.orchestra.run.smithi049.stdout: "ireq_exportdir": 0, 2024-03-18T23:19:16.023 INFO:teuthology.orchestra.run.smithi049.stdout: "ireq_flush": 0, 2024-03-18T23:19:16.023 INFO:teuthology.orchestra.run.smithi049.stdout: "ireq_fragmentdir": 0, 2024-03-18T23:19:16.023 INFO:teuthology.orchestra.run.smithi049.stdout: "ireq_fragstats": 0, 2024-03-18T23:19:16.024 INFO:teuthology.orchestra.run.smithi049.stdout: "ireq_inodestats": 0 2024-03-18T23:19:16.024 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:16.024 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_log": { 2024-03-18T23:19:16.024 INFO:teuthology.orchestra.run.smithi049.stdout: "evlrg": 0, 2024-03-18T23:19:16.024 INFO:teuthology.orchestra.run.smithi049.stdout: "evadd": 0, 2024-03-18T23:19:16.024 INFO:teuthology.orchestra.run.smithi049.stdout: "evex": 0, 2024-03-18T23:19:16.024 INFO:teuthology.orchestra.run.smithi049.stdout: "evtrm": 0, 2024-03-18T23:19:16.024 INFO:teuthology.orchestra.run.smithi049.stdout: "ev": 6, 2024-03-18T23:19:16.024 INFO:teuthology.orchestra.run.smithi049.stdout: "evexg": 0, 2024-03-18T23:19:16.024 INFO:teuthology.orchestra.run.smithi049.stdout: "evexd": 0, 2024-03-18T23:19:16.025 INFO:teuthology.orchestra.run.smithi049.stdout: "segadd": 0, 2024-03-18T23:19:16.025 INFO:teuthology.orchestra.run.smithi049.stdout: "segex": 0, 2024-03-18T23:19:16.025 INFO:teuthology.orchestra.run.smithi049.stdout: "segtrm": 0, 2024-03-18T23:19:16.025 INFO:teuthology.orchestra.run.smithi049.stdout: "seg": 2, 2024-03-18T23:19:16.025 INFO:teuthology.orchestra.run.smithi049.stdout: "segmjr": 2, 2024-03-18T23:19:16.025 INFO:teuthology.orchestra.run.smithi049.stdout: "segexg": 0, 2024-03-18T23:19:16.025 INFO:teuthology.orchestra.run.smithi049.stdout: "segexd": 0, 2024-03-18T23:19:16.025 INFO:teuthology.orchestra.run.smithi049.stdout: "expos": 4194304, 2024-03-18T23:19:16.025 INFO:teuthology.orchestra.run.smithi049.stdout: "wrpos": 4196123, 2024-03-18T23:19:16.026 INFO:teuthology.orchestra.run.smithi049.stdout: "rdpos": 4195982, 2024-03-18T23:19:16.026 INFO:teuthology.orchestra.run.smithi049.stdout: "jlat": { 2024-03-18T23:19:16.026 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-03-18T23:19:16.026 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-03-18T23:19:16.026 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-03-18T23:19:16.026 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:16.026 INFO:teuthology.orchestra.run.smithi049.stdout: "replayed": 6 2024-03-18T23:19:16.026 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:16.026 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_mem": { 2024-03-18T23:19:16.027 INFO:teuthology.orchestra.run.smithi049.stdout: "ino": 3, 2024-03-18T23:19:16.027 INFO:teuthology.orchestra.run.smithi049.stdout: "ino+": 3, 2024-03-18T23:19:16.027 INFO:teuthology.orchestra.run.smithi049.stdout: "ino-": 0, 2024-03-18T23:19:16.027 INFO:teuthology.orchestra.run.smithi049.stdout: "dir": 1, 2024-03-18T23:19:16.027 INFO:teuthology.orchestra.run.smithi049.stdout: "dir+": 1, 2024-03-18T23:19:16.027 INFO:teuthology.orchestra.run.smithi049.stdout: "dir-": 0, 2024-03-18T23:19:16.027 INFO:teuthology.orchestra.run.smithi049.stdout: "dn": 0, 2024-03-18T23:19:16.027 INFO:teuthology.orchestra.run.smithi049.stdout: "dn+": 0, 2024-03-18T23:19:16.027 INFO:teuthology.orchestra.run.smithi049.stdout: "dn-": 0, 2024-03-18T23:19:16.028 INFO:teuthology.orchestra.run.smithi049.stdout: "cap": 0, 2024-03-18T23:19:16.028 INFO:teuthology.orchestra.run.smithi049.stdout: "cap+": 0, 2024-03-18T23:19:16.028 INFO:teuthology.orchestra.run.smithi049.stdout: "cap-": 0, 2024-03-18T23:19:16.028 INFO:teuthology.orchestra.run.smithi049.stdout: "rss": 50500, 2024-03-18T23:19:16.028 INFO:teuthology.orchestra.run.smithi049.stdout: "heap": 215420 2024-03-18T23:19:16.028 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:16.028 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_server": { 2024-03-18T23:19:16.028 INFO:teuthology.orchestra.run.smithi049.stdout: "dispatch_client_request": 0, 2024-03-18T23:19:16.028 INFO:teuthology.orchestra.run.smithi049.stdout: "dispatch_server_request": 0, 2024-03-18T23:19:16.028 INFO:teuthology.orchestra.run.smithi049.stdout: "handle_client_request": 0, 2024-03-18T23:19:16.029 INFO:teuthology.orchestra.run.smithi049.stdout: "handle_client_session": 0, 2024-03-18T23:19:16.029 INFO:teuthology.orchestra.run.smithi049.stdout: "handle_peer_request": 0, 2024-03-18T23:19:16.029 INFO:teuthology.orchestra.run.smithi049.stdout: "req_create_latency": { 2024-03-18T23:19:16.029 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-03-18T23:19:16.029 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-03-18T23:19:16.029 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-03-18T23:19:16.029 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:16.029 INFO:teuthology.orchestra.run.smithi049.stdout: "req_getattr_latency": { 2024-03-18T23:19:16.029 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-03-18T23:19:16.029 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-03-18T23:19:16.030 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-03-18T23:19:16.030 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:16.030 INFO:teuthology.orchestra.run.smithi049.stdout: "req_getfilelock_latency": { 2024-03-18T23:19:16.030 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-03-18T23:19:16.030 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-03-18T23:19:16.030 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-03-18T23:19:16.030 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:16.030 INFO:teuthology.orchestra.run.smithi049.stdout: "req_link_latency": { 2024-03-18T23:19:16.031 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-03-18T23:19:16.031 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-03-18T23:19:16.031 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-03-18T23:19:16.031 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:16.031 INFO:teuthology.orchestra.run.smithi049.stdout: "req_lookup_latency": { 2024-03-18T23:19:16.031 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-03-18T23:19:16.031 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-03-18T23:19:16.031 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-03-18T23:19:16.031 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:16.032 INFO:teuthology.orchestra.run.smithi049.stdout: "req_lookuphash_latency": { 2024-03-18T23:19:16.032 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-03-18T23:19:16.032 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-03-18T23:19:16.032 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-03-18T23:19:16.032 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:16.032 INFO:teuthology.orchestra.run.smithi049.stdout: "req_lookupino_latency": { 2024-03-18T23:19:16.032 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-03-18T23:19:16.032 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-03-18T23:19:16.033 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-03-18T23:19:16.033 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:16.033 INFO:teuthology.orchestra.run.smithi049.stdout: "req_lookupname_latency": { 2024-03-18T23:19:16.033 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-03-18T23:19:16.033 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-03-18T23:19:16.033 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-03-18T23:19:16.033 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:16.033 INFO:teuthology.orchestra.run.smithi049.stdout: "req_lookupparent_latency": { 2024-03-18T23:19:16.033 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-03-18T23:19:16.034 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-03-18T23:19:16.034 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-03-18T23:19:16.034 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:16.034 INFO:teuthology.orchestra.run.smithi049.stdout: "req_lookupsnap_latency": { 2024-03-18T23:19:16.034 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-03-18T23:19:16.034 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-03-18T23:19:16.034 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-03-18T23:19:16.034 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:16.034 INFO:teuthology.orchestra.run.smithi049.stdout: "req_lssnap_latency": { 2024-03-18T23:19:16.035 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-03-18T23:19:16.035 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-03-18T23:19:16.035 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-03-18T23:19:16.035 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:16.035 INFO:teuthology.orchestra.run.smithi049.stdout: "req_mkdir_latency": { 2024-03-18T23:19:16.035 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-03-18T23:19:16.036 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-03-18T23:19:16.036 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-03-18T23:19:16.036 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:16.036 INFO:teuthology.orchestra.run.smithi049.stdout: "req_mknod_latency": { 2024-03-18T23:19:16.036 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-03-18T23:19:16.036 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-03-18T23:19:16.036 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-03-18T23:19:16.037 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:16.037 INFO:teuthology.orchestra.run.smithi049.stdout: "req_mksnap_latency": { 2024-03-18T23:19:16.037 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-03-18T23:19:16.037 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-03-18T23:19:16.037 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-03-18T23:19:16.037 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:16.037 INFO:teuthology.orchestra.run.smithi049.stdout: "req_open_latency": { 2024-03-18T23:19:16.037 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-03-18T23:19:16.037 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-03-18T23:19:16.038 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-03-18T23:19:16.038 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:16.038 INFO:teuthology.orchestra.run.smithi049.stdout: "req_readdir_latency": { 2024-03-18T23:19:16.038 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-03-18T23:19:16.038 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-03-18T23:19:16.038 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-03-18T23:19:16.038 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:16.038 INFO:teuthology.orchestra.run.smithi049.stdout: "req_rename_latency": { 2024-03-18T23:19:16.039 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-03-18T23:19:16.039 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-03-18T23:19:16.039 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-03-18T23:19:16.039 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:16.039 INFO:teuthology.orchestra.run.smithi049.stdout: "req_renamesnap_latency": { 2024-03-18T23:19:16.039 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-03-18T23:19:16.039 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-03-18T23:19:16.039 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-03-18T23:19:16.040 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:16.040 INFO:teuthology.orchestra.run.smithi049.stdout: "req_snapdiff_latency": { 2024-03-18T23:19:16.040 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-03-18T23:19:16.040 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-03-18T23:19:16.040 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-03-18T23:19:16.040 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:16.040 INFO:teuthology.orchestra.run.smithi049.stdout: "req_rmdir_latency": { 2024-03-18T23:19:16.040 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-03-18T23:19:16.040 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-03-18T23:19:16.041 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-03-18T23:19:16.041 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:16.041 INFO:teuthology.orchestra.run.smithi049.stdout: "req_rmsnap_latency": { 2024-03-18T23:19:16.041 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-03-18T23:19:16.041 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-03-18T23:19:16.041 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-03-18T23:19:16.041 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:16.041 INFO:teuthology.orchestra.run.smithi049.stdout: "req_rmxattr_latency": { 2024-03-18T23:19:16.041 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-03-18T23:19:16.042 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-03-18T23:19:16.042 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-03-18T23:19:16.042 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:16.042 INFO:teuthology.orchestra.run.smithi049.stdout: "req_setattr_latency": { 2024-03-18T23:19:16.042 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-03-18T23:19:16.042 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-03-18T23:19:16.042 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-03-18T23:19:16.042 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:16.043 INFO:teuthology.orchestra.run.smithi049.stdout: "req_setdirlayout_latency": { 2024-03-18T23:19:16.043 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-03-18T23:19:16.043 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-03-18T23:19:16.043 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-03-18T23:19:16.043 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:16.043 INFO:teuthology.orchestra.run.smithi049.stdout: "req_setfilelock_latency": { 2024-03-18T23:19:16.043 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-03-18T23:19:16.043 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-03-18T23:19:16.043 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-03-18T23:19:16.044 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:16.044 INFO:teuthology.orchestra.run.smithi049.stdout: "req_setlayout_latency": { 2024-03-18T23:19:16.044 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-03-18T23:19:16.044 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-03-18T23:19:16.044 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-03-18T23:19:16.044 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:16.044 INFO:teuthology.orchestra.run.smithi049.stdout: "req_setxattr_latency": { 2024-03-18T23:19:16.044 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-03-18T23:19:16.044 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-03-18T23:19:16.045 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-03-18T23:19:16.045 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:16.045 INFO:teuthology.orchestra.run.smithi049.stdout: "req_symlink_latency": { 2024-03-18T23:19:16.045 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-03-18T23:19:16.045 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-03-18T23:19:16.045 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-03-18T23:19:16.045 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:16.045 INFO:teuthology.orchestra.run.smithi049.stdout: "req_unlink_latency": { 2024-03-18T23:19:16.045 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-03-18T23:19:16.046 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-03-18T23:19:16.046 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-03-18T23:19:16.046 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:16.046 INFO:teuthology.orchestra.run.smithi049.stdout: "cap_revoke_eviction": 0, 2024-03-18T23:19:16.046 INFO:teuthology.orchestra.run.smithi049.stdout: "cap_acquisition_throttle": 0, 2024-03-18T23:19:16.046 INFO:teuthology.orchestra.run.smithi049.stdout: "req_getvxattr_latency": { 2024-03-18T23:19:16.047 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-03-18T23:19:16.047 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-03-18T23:19:16.047 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-03-18T23:19:16.047 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:19:16.047 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:16.047 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_sessions": { 2024-03-18T23:19:16.047 INFO:teuthology.orchestra.run.smithi049.stdout: "session_count": 2, 2024-03-18T23:19:16.047 INFO:teuthology.orchestra.run.smithi049.stdout: "session_add": 3, 2024-03-18T23:19:16.048 INFO:teuthology.orchestra.run.smithi049.stdout: "session_remove": 1, 2024-03-18T23:19:16.048 INFO:teuthology.orchestra.run.smithi049.stdout: "sessions_open": 3, 2024-03-18T23:19:16.048 INFO:teuthology.orchestra.run.smithi049.stdout: "sessions_stale": 0, 2024-03-18T23:19:16.048 INFO:teuthology.orchestra.run.smithi049.stdout: "total_load": 0, 2024-03-18T23:19:16.048 INFO:teuthology.orchestra.run.smithi049.stdout: "average_load": 0, 2024-03-18T23:19:16.048 INFO:teuthology.orchestra.run.smithi049.stdout: "avg_session_uptime": 7070, 2024-03-18T23:19:16.048 INFO:teuthology.orchestra.run.smithi049.stdout: "mdthresh_evicted": 0 2024-03-18T23:19:16.048 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:16.048 INFO:teuthology.orchestra.run.smithi049.stdout: "mempool": { 2024-03-18T23:19:16.049 INFO:teuthology.orchestra.run.smithi049.stdout: "bloom_filter_bytes": 0, 2024-03-18T23:19:16.049 INFO:teuthology.orchestra.run.smithi049.stdout: "bloom_filter_items": 0, 2024-03-18T23:19:16.049 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_alloc_bytes": 0, 2024-03-18T23:19:16.049 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_alloc_items": 0, 2024-03-18T23:19:16.049 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_data_bytes": 0, 2024-03-18T23:19:16.049 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_data_items": 0, 2024-03-18T23:19:16.049 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-18T23:19:16.049 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_onode_items": 0, 2024-03-18T23:19:16.049 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-18T23:19:16.049 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_meta_items": 0, 2024-03-18T23:19:16.050 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_other_bytes": 0, 2024-03-18T23:19:16.050 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_other_items": 0, 2024-03-18T23:19:16.050 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-18T23:19:16.050 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_buffer_items": 0, 2024-03-18T23:19:16.050 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_extent_bytes": 0, 2024-03-18T23:19:16.050 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_extent_items": 0, 2024-03-18T23:19:16.050 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_blob_bytes": 0, 2024-03-18T23:19:16.050 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_blob_items": 0, 2024-03-18T23:19:16.051 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-18T23:19:16.051 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_shared_blob_items": 0, 2024-03-18T23:19:16.051 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-18T23:19:16.051 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_inline_bl_items": 0, 2024-03-18T23:19:16.051 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_fsck_bytes": 0, 2024-03-18T23:19:16.051 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_fsck_items": 0, 2024-03-18T23:19:16.051 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_txc_bytes": 0, 2024-03-18T23:19:16.051 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_txc_items": 0, 2024-03-18T23:19:16.051 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-18T23:19:16.051 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_writing_deferred_items": 0, 2024-03-18T23:19:16.052 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_writing_bytes": 0, 2024-03-18T23:19:16.052 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_writing_items": 0, 2024-03-18T23:19:16.052 INFO:teuthology.orchestra.run.smithi049.stdout: "bluefs_bytes": 0, 2024-03-18T23:19:16.052 INFO:teuthology.orchestra.run.smithi049.stdout: "bluefs_items": 0, 2024-03-18T23:19:16.052 INFO:teuthology.orchestra.run.smithi049.stdout: "bluefs_file_reader_bytes": 0, 2024-03-18T23:19:16.052 INFO:teuthology.orchestra.run.smithi049.stdout: "bluefs_file_reader_items": 0, 2024-03-18T23:19:16.052 INFO:teuthology.orchestra.run.smithi049.stdout: "bluefs_file_writer_bytes": 0, 2024-03-18T23:19:16.052 INFO:teuthology.orchestra.run.smithi049.stdout: "bluefs_file_writer_items": 0, 2024-03-18T23:19:16.052 INFO:teuthology.orchestra.run.smithi049.stdout: "buffer_anon_bytes": 197553, 2024-03-18T23:19:16.053 INFO:teuthology.orchestra.run.smithi049.stdout: "buffer_anon_items": 57, 2024-03-18T23:19:16.053 INFO:teuthology.orchestra.run.smithi049.stdout: "buffer_meta_bytes": 0, 2024-03-18T23:19:16.053 INFO:teuthology.orchestra.run.smithi049.stdout: "buffer_meta_items": 0, 2024-03-18T23:19:16.053 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_bytes": 0, 2024-03-18T23:19:16.053 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_items": 0, 2024-03-18T23:19:16.053 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_mapbl_bytes": 0, 2024-03-18T23:19:16.053 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_mapbl_items": 0, 2024-03-18T23:19:16.053 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_pglog_bytes": 0, 2024-03-18T23:19:16.053 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_pglog_items": 0, 2024-03-18T23:19:16.054 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_bytes": 8376, 2024-03-18T23:19:16.054 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_items": 191, 2024-03-18T23:19:16.054 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_mapping_bytes": 0, 2024-03-18T23:19:16.054 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_mapping_items": 0, 2024-03-18T23:19:16.054 INFO:teuthology.orchestra.run.smithi049.stdout: "pgmap_bytes": 0, 2024-03-18T23:19:16.054 INFO:teuthology.orchestra.run.smithi049.stdout: "pgmap_items": 0, 2024-03-18T23:19:16.054 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_co_bytes": 8464, 2024-03-18T23:19:16.055 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_co_items": 13, 2024-03-18T23:19:16.055 INFO:teuthology.orchestra.run.smithi049.stdout: "unittest_1_bytes": 0, 2024-03-18T23:19:16.055 INFO:teuthology.orchestra.run.smithi049.stdout: "unittest_1_items": 0, 2024-03-18T23:19:16.055 INFO:teuthology.orchestra.run.smithi049.stdout: "unittest_2_bytes": 0, 2024-03-18T23:19:16.055 INFO:teuthology.orchestra.run.smithi049.stdout: "unittest_2_items": 0 2024-03-18T23:19:16.055 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:16.055 INFO:teuthology.orchestra.run.smithi049.stdout: "objecter": { 2024-03-18T23:19:16.055 INFO:teuthology.orchestra.run.smithi049.stdout: "op_active": 0, 2024-03-18T23:19:16.055 INFO:teuthology.orchestra.run.smithi049.stdout: "op_laggy": 0, 2024-03-18T23:19:16.056 INFO:teuthology.orchestra.run.smithi049.stdout: "op_send": 2235, 2024-03-18T23:19:16.056 INFO:teuthology.orchestra.run.smithi049.stdout: "op_send_bytes": 21, 2024-03-18T23:19:16.056 INFO:teuthology.orchestra.run.smithi049.stdout: "op_resend": 3, 2024-03-18T23:19:16.056 INFO:teuthology.orchestra.run.smithi049.stdout: "op_reply": 2235, 2024-03-18T23:19:16.056 INFO:teuthology.orchestra.run.smithi049.stdout: "op_latency": { 2024-03-18T23:19:16.056 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 2235, 2024-03-18T23:19:16.056 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 1.119968441, 2024-03-18T23:19:16.057 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000501104 2024-03-18T23:19:16.057 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:16.057 INFO:teuthology.orchestra.run.smithi049.stdout: "op_inflight": 0, 2024-03-18T23:19:16.057 INFO:teuthology.orchestra.run.smithi049.stdout: "oplen_avg": { 2024-03-18T23:19:16.057 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 2235, 2024-03-18T23:19:16.057 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 2236 2024-03-18T23:19:16.057 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:16.057 INFO:teuthology.orchestra.run.smithi049.stdout: "op": 2235, 2024-03-18T23:19:16.058 INFO:teuthology.orchestra.run.smithi049.stdout: "op_r": 2235, 2024-03-18T23:19:16.058 INFO:teuthology.orchestra.run.smithi049.stdout: "op_w": 0, 2024-03-18T23:19:16.058 INFO:teuthology.orchestra.run.smithi049.stdout: "op_rmw": 0, 2024-03-18T23:19:16.058 INFO:teuthology.orchestra.run.smithi049.stdout: "op_pg": 0, 2024-03-18T23:19:16.058 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_stat": 1442, 2024-03-18T23:19:16.058 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_create": 0, 2024-03-18T23:19:16.058 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_read": 791, 2024-03-18T23:19:16.058 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_write": 0, 2024-03-18T23:19:16.059 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_writefull": 0, 2024-03-18T23:19:16.059 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_writesame": 0, 2024-03-18T23:19:16.059 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_append": 0, 2024-03-18T23:19:16.059 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_zero": 0, 2024-03-18T23:19:16.059 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_truncate": 0, 2024-03-18T23:19:16.059 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_delete": 0, 2024-03-18T23:19:16.059 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_mapext": 0, 2024-03-18T23:19:16.059 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_sparse_read": 0, 2024-03-18T23:19:16.059 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_clonerange": 0, 2024-03-18T23:19:16.060 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_getxattr": 1, 2024-03-18T23:19:16.060 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_setxattr": 0, 2024-03-18T23:19:16.060 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_cmpxattr": 0, 2024-03-18T23:19:16.060 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_rmxattr": 0, 2024-03-18T23:19:16.060 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_resetxattrs": 0, 2024-03-18T23:19:16.060 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_call": 0, 2024-03-18T23:19:16.060 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_watch": 0, 2024-03-18T23:19:16.060 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_notify": 0, 2024-03-18T23:19:16.060 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_src_cmpxattr": 0, 2024-03-18T23:19:16.061 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_pgls": 0, 2024-03-18T23:19:16.061 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_pgls_filter": 0, 2024-03-18T23:19:16.061 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_other": 0, 2024-03-18T23:19:16.061 INFO:teuthology.orchestra.run.smithi049.stdout: "linger_active": 0, 2024-03-18T23:19:16.061 INFO:teuthology.orchestra.run.smithi049.stdout: "linger_send": 0, 2024-03-18T23:19:16.061 INFO:teuthology.orchestra.run.smithi049.stdout: "linger_resend": 0, 2024-03-18T23:19:16.061 INFO:teuthology.orchestra.run.smithi049.stdout: "linger_ping": 0, 2024-03-18T23:19:16.061 INFO:teuthology.orchestra.run.smithi049.stdout: "poolop_active": 0, 2024-03-18T23:19:16.062 INFO:teuthology.orchestra.run.smithi049.stdout: "poolop_send": 0, 2024-03-18T23:19:16.062 INFO:teuthology.orchestra.run.smithi049.stdout: "poolop_resend": 0, 2024-03-18T23:19:16.062 INFO:teuthology.orchestra.run.smithi049.stdout: "poolstat_active": 0, 2024-03-18T23:19:16.062 INFO:teuthology.orchestra.run.smithi049.stdout: "poolstat_send": 0, 2024-03-18T23:19:16.062 INFO:teuthology.orchestra.run.smithi049.stdout: "poolstat_resend": 0, 2024-03-18T23:19:16.062 INFO:teuthology.orchestra.run.smithi049.stdout: "statfs_active": 0, 2024-03-18T23:19:16.062 INFO:teuthology.orchestra.run.smithi049.stdout: "statfs_send": 0, 2024-03-18T23:19:16.062 INFO:teuthology.orchestra.run.smithi049.stdout: "statfs_resend": 0, 2024-03-18T23:19:16.063 INFO:teuthology.orchestra.run.smithi049.stdout: "command_active": 0, 2024-03-18T23:19:16.063 INFO:teuthology.orchestra.run.smithi049.stdout: "command_send": 0, 2024-03-18T23:19:16.063 INFO:teuthology.orchestra.run.smithi049.stdout: "command_resend": 0, 2024-03-18T23:19:16.063 INFO:teuthology.orchestra.run.smithi049.stdout: "map_epoch": 100, 2024-03-18T23:19:16.063 INFO:teuthology.orchestra.run.smithi049.stdout: "map_full": 0, 2024-03-18T23:19:16.063 INFO:teuthology.orchestra.run.smithi049.stdout: "map_inc": 14, 2024-03-18T23:19:16.063 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_sessions": 6, 2024-03-18T23:19:16.063 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_session_open": 6, 2024-03-18T23:19:16.063 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_session_close": 0, 2024-03-18T23:19:16.064 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_laggy": 0, 2024-03-18T23:19:16.064 INFO:teuthology.orchestra.run.smithi049.stdout: "omap_wr": 0, 2024-03-18T23:19:16.064 INFO:teuthology.orchestra.run.smithi049.stdout: "omap_rd": 2, 2024-03-18T23:19:16.064 INFO:teuthology.orchestra.run.smithi049.stdout: "omap_del": 0 2024-03-18T23:19:16.064 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:16.064 INFO:teuthology.orchestra.run.smithi049.stdout: "oft": { 2024-03-18T23:19:16.064 INFO:teuthology.orchestra.run.smithi049.stdout: "omap_total_objs": 0, 2024-03-18T23:19:16.064 INFO:teuthology.orchestra.run.smithi049.stdout: "omap_total_kv_pairs": 0, 2024-03-18T23:19:16.065 INFO:teuthology.orchestra.run.smithi049.stdout: "omap_total_updates": 0, 2024-03-18T23:19:16.065 INFO:teuthology.orchestra.run.smithi049.stdout: "omap_total_removes": 0 2024-03-18T23:19:16.065 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:16.065 INFO:teuthology.orchestra.run.smithi049.stdout: "osd-slow-ops": { 2024-03-18T23:19:16.065 INFO:teuthology.orchestra.run.smithi049.stdout: "slow_ops_count": 0 2024-03-18T23:19:16.065 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:16.065 INFO:teuthology.orchestra.run.smithi049.stdout: "purge_queue": { 2024-03-18T23:19:16.065 INFO:teuthology.orchestra.run.smithi049.stdout: "pq_executing_ops": 0, 2024-03-18T23:19:16.066 INFO:teuthology.orchestra.run.smithi049.stdout: "pq_executing_ops_high_water": 0, 2024-03-18T23:19:16.066 INFO:teuthology.orchestra.run.smithi049.stdout: "pq_executing": 0, 2024-03-18T23:19:16.066 INFO:teuthology.orchestra.run.smithi049.stdout: "pq_executing_high_water": 0, 2024-03-18T23:19:16.066 INFO:teuthology.orchestra.run.smithi049.stdout: "pq_executed_ops": 0, 2024-03-18T23:19:16.066 INFO:teuthology.orchestra.run.smithi049.stdout: "pq_executed": 0, 2024-03-18T23:19:16.066 INFO:teuthology.orchestra.run.smithi049.stdout: "pq_item_in_journal": 0 2024-03-18T23:19:16.066 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:16.066 INFO:teuthology.orchestra.run.smithi049.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-18T23:19:16.066 INFO:teuthology.orchestra.run.smithi049.stdout: "val": 0, 2024-03-18T23:19:16.067 INFO:teuthology.orchestra.run.smithi049.stdout: "max": 104857600, 2024-03-18T23:19:16.067 INFO:teuthology.orchestra.run.smithi049.stdout: "get_started": 0, 2024-03-18T23:19:16.067 INFO:teuthology.orchestra.run.smithi049.stdout: "get": 2487, 2024-03-18T23:19:16.067 INFO:teuthology.orchestra.run.smithi049.stdout: "get_sum": 1895143, 2024-03-18T23:19:16.067 INFO:teuthology.orchestra.run.smithi049.stdout: "get_or_fail_fail": 0, 2024-03-18T23:19:16.067 INFO:teuthology.orchestra.run.smithi049.stdout: "get_or_fail_success": 2487, 2024-03-18T23:19:16.067 INFO:teuthology.orchestra.run.smithi049.stdout: "take": 0, 2024-03-18T23:19:16.067 INFO:teuthology.orchestra.run.smithi049.stdout: "take_sum": 0, 2024-03-18T23:19:16.067 INFO:teuthology.orchestra.run.smithi049.stdout: "put": 2487, 2024-03-18T23:19:16.067 INFO:teuthology.orchestra.run.smithi049.stdout: "put_sum": 1895143, 2024-03-18T23:19:16.068 INFO:teuthology.orchestra.run.smithi049.stdout: "wait": { 2024-03-18T23:19:16.068 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-03-18T23:19:16.068 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-03-18T23:19:16.068 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-03-18T23:19:16.068 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:19:16.068 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:16.068 INFO:teuthology.orchestra.run.smithi049.stdout: "throttle-objecter_bytes": { 2024-03-18T23:19:16.068 INFO:teuthology.orchestra.run.smithi049.stdout: "val": 0, 2024-03-18T23:19:16.068 INFO:teuthology.orchestra.run.smithi049.stdout: "max": 104857600, 2024-03-18T23:19:16.068 INFO:teuthology.orchestra.run.smithi049.stdout: "get_started": 0, 2024-03-18T23:19:16.069 INFO:teuthology.orchestra.run.smithi049.stdout: "get": 0, 2024-03-18T23:19:16.069 INFO:teuthology.orchestra.run.smithi049.stdout: "get_sum": 0, 2024-03-18T23:19:16.069 INFO:teuthology.orchestra.run.smithi049.stdout: "get_or_fail_fail": 0, 2024-03-18T23:19:16.069 INFO:teuthology.orchestra.run.smithi049.stdout: "get_or_fail_success": 0, 2024-03-18T23:19:16.069 INFO:teuthology.orchestra.run.smithi049.stdout: "take": 2235, 2024-03-18T23:19:16.069 INFO:teuthology.orchestra.run.smithi049.stdout: "take_sum": 1824, 2024-03-18T23:19:16.069 INFO:teuthology.orchestra.run.smithi049.stdout: "put": 2235, 2024-03-18T23:19:16.069 INFO:teuthology.orchestra.run.smithi049.stdout: "put_sum": 1824, 2024-03-18T23:19:16.069 INFO:teuthology.orchestra.run.smithi049.stdout: "wait": { 2024-03-18T23:19:16.070 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-03-18T23:19:16.070 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-03-18T23:19:16.070 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-03-18T23:19:16.070 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:19:16.070 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:16.070 INFO:teuthology.orchestra.run.smithi049.stdout: "throttle-objecter_ops": { 2024-03-18T23:19:16.070 INFO:teuthology.orchestra.run.smithi049.stdout: "val": 0, 2024-03-18T23:19:16.070 INFO:teuthology.orchestra.run.smithi049.stdout: "max": 1024, 2024-03-18T23:19:16.071 INFO:teuthology.orchestra.run.smithi049.stdout: "get_started": 0, 2024-03-18T23:19:16.071 INFO:teuthology.orchestra.run.smithi049.stdout: "get": 0, 2024-03-18T23:19:16.071 INFO:teuthology.orchestra.run.smithi049.stdout: "get_sum": 0, 2024-03-18T23:19:16.071 INFO:teuthology.orchestra.run.smithi049.stdout: "get_or_fail_fail": 0, 2024-03-18T23:19:16.071 INFO:teuthology.orchestra.run.smithi049.stdout: "get_or_fail_success": 0, 2024-03-18T23:19:16.071 INFO:teuthology.orchestra.run.smithi049.stdout: "take": 2235, 2024-03-18T23:19:16.071 INFO:teuthology.orchestra.run.smithi049.stdout: "take_sum": 2235, 2024-03-18T23:19:16.071 INFO:teuthology.orchestra.run.smithi049.stdout: "put": 2235, 2024-03-18T23:19:16.071 INFO:teuthology.orchestra.run.smithi049.stdout: "put_sum": 2235, 2024-03-18T23:19:16.071 INFO:teuthology.orchestra.run.smithi049.stdout: "wait": { 2024-03-18T23:19:16.072 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-03-18T23:19:16.072 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-03-18T23:19:16.072 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-03-18T23:19:16.072 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:19:16.072 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:16.072 INFO:teuthology.orchestra.run.smithi049.stdout: "throttle-write_buf_throttle": { 2024-03-18T23:19:16.072 INFO:teuthology.orchestra.run.smithi049.stdout: "val": 0, 2024-03-18T23:19:16.072 INFO:teuthology.orchestra.run.smithi049.stdout: "max": 3758096384, 2024-03-18T23:19:16.072 INFO:teuthology.orchestra.run.smithi049.stdout: "get_started": 0, 2024-03-18T23:19:16.072 INFO:teuthology.orchestra.run.smithi049.stdout: "get": 0, 2024-03-18T23:19:16.073 INFO:teuthology.orchestra.run.smithi049.stdout: "get_sum": 0, 2024-03-18T23:19:16.073 INFO:teuthology.orchestra.run.smithi049.stdout: "get_or_fail_fail": 0, 2024-03-18T23:19:16.073 INFO:teuthology.orchestra.run.smithi049.stdout: "get_or_fail_success": 0, 2024-03-18T23:19:16.073 INFO:teuthology.orchestra.run.smithi049.stdout: "take": 0, 2024-03-18T23:19:16.073 INFO:teuthology.orchestra.run.smithi049.stdout: "take_sum": 0, 2024-03-18T23:19:16.073 INFO:teuthology.orchestra.run.smithi049.stdout: "put": 0, 2024-03-18T23:19:16.073 INFO:teuthology.orchestra.run.smithi049.stdout: "put_sum": 0, 2024-03-18T23:19:16.073 INFO:teuthology.orchestra.run.smithi049.stdout: "wait": { 2024-03-18T23:19:16.073 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-03-18T23:19:16.074 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-03-18T23:19:16.074 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-03-18T23:19:16.074 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:19:16.074 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:16.074 INFO:teuthology.orchestra.run.smithi049.stdout: "throttle-write_buf_throttle-0x55b6e85b66e0": { 2024-03-18T23:19:16.074 INFO:teuthology.orchestra.run.smithi049.stdout: "val": 0, 2024-03-18T23:19:16.074 INFO:teuthology.orchestra.run.smithi049.stdout: "max": 3758096384, 2024-03-18T23:19:16.074 INFO:teuthology.orchestra.run.smithi049.stdout: "get_started": 0, 2024-03-18T23:19:16.074 INFO:teuthology.orchestra.run.smithi049.stdout: "get": 0, 2024-03-18T23:19:16.075 INFO:teuthology.orchestra.run.smithi049.stdout: "get_sum": 0, 2024-03-18T23:19:16.075 INFO:teuthology.orchestra.run.smithi049.stdout: "get_or_fail_fail": 0, 2024-03-18T23:19:16.075 INFO:teuthology.orchestra.run.smithi049.stdout: "get_or_fail_success": 0, 2024-03-18T23:19:16.075 INFO:teuthology.orchestra.run.smithi049.stdout: "take": 0, 2024-03-18T23:19:16.075 INFO:teuthology.orchestra.run.smithi049.stdout: "take_sum": 0, 2024-03-18T23:19:16.075 INFO:teuthology.orchestra.run.smithi049.stdout: "put": 0, 2024-03-18T23:19:16.075 INFO:teuthology.orchestra.run.smithi049.stdout: "put_sum": 0, 2024-03-18T23:19:16.075 INFO:teuthology.orchestra.run.smithi049.stdout: "wait": { 2024-03-18T23:19:16.075 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-03-18T23:19:16.076 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-03-18T23:19:16.076 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-03-18T23:19:16.076 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:19:16.076 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:19:16.076 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-03-18T23:19:16.989 INFO:tasks.check_counter:Daemon mds.g mds.exported=0 2024-03-18T23:19:16.989 INFO:tasks.check_counter:Daemon mds.g mds.imported=0 2024-03-18T23:19:16.989 INFO:tasks.check_counter:Daemon mds.g mds.root_rsnaps=0 2024-03-18T23:19:16.989 INFO:tasks.check_counter:Daemon mds.g mds_server.req_mksnap_latency.avgcount=0 2024-03-18T23:19:16.989 INFO:tasks.check_counter:Daemon mds.g mds_server.req_rmsnap_latency.avgcount=0 2024-03-18T23:19:16.989 DEBUG:tasks.check_counter:Getting stats from j 2024-03-18T23:19:16.990 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell --fsid c63591ea-e57a-11ee-95c9-87774f69a715 -- ceph daemon mds.j perf dump 2024-03-18T23:19:17.243 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T23:19:17.510 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:19:17 smithi049 ceph-mon[62991]: pgmap v344: 97 pgs: 97 active+clean; 64 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 1.3 MiB/s wr, 61 op/s 2024-03-18T23:19:17.564 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:19:17 smithi139 ceph-mon[67282]: pgmap v344: 97 pgs: 97 active+clean; 64 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 1.3 MiB/s wr, 61 op/s 2024-03-18T23:19:17.710 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:19:17 smithi175 ceph-mon[67352]: pgmap v344: 97 pgs: 97 active+clean; 64 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 1.3 MiB/s wr, 61 op/s 2024-03-18T23:19:19.298 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-03-18T23:19:19.298 INFO:teuthology.orchestra.run.smithi049.stdout: "AsyncMessenger::Worker-0": { 2024-03-18T23:19:19.298 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_messages": 232, 2024-03-18T23:19:19.298 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_messages": 222, 2024-03-18T23:19:19.298 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_bytes": 1440416, 2024-03-18T23:19:19.299 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_bytes": 136384, 2024-03-18T23:19:19.299 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_created_connections": 4, 2024-03-18T23:19:19.299 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_active_connections": 0, 2024-03-18T23:19:19.299 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_total_time": 0.038656298, 2024-03-18T23:19:19.299 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_send_time": 0.011404015, 2024-03-18T23:19:19.299 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_recv_time": 0.032078204, 2024-03-18T23:19:19.299 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_fast_dispatch_time": 0.003853473, 2024-03-18T23:19:19.299 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_messages_queue_lat": { 2024-03-18T23:19:19.299 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 221, 2024-03-18T23:19:19.300 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.008357050, 2024-03-18T23:19:19.300 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000037814 2024-03-18T23:19:19.300 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:19.300 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_handle_ack_lat": { 2024-03-18T23:19:19.300 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-03-18T23:19:19.300 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-03-18T23:19:19.300 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-03-18T23:19:19.300 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:19.300 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_encrypted_bytes": 1440416, 2024-03-18T23:19:19.301 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_encrypted_bytes": 136384 2024-03-18T23:19:19.301 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:19.301 INFO:teuthology.orchestra.run.smithi049.stdout: "AsyncMessenger::Worker-1": { 2024-03-18T23:19:19.301 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_messages": 2, 2024-03-18T23:19:19.301 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_messages": 169, 2024-03-18T23:19:19.301 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_bytes": 320, 2024-03-18T23:19:19.301 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_bytes": 185408, 2024-03-18T23:19:19.301 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_created_connections": 5, 2024-03-18T23:19:19.302 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_active_connections": 3, 2024-03-18T23:19:19.302 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_total_time": 0.012232589, 2024-03-18T23:19:19.302 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_send_time": 0.007823393, 2024-03-18T23:19:19.302 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_recv_time": 0.001958673, 2024-03-18T23:19:19.302 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-18T23:19:19.302 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_messages_queue_lat": { 2024-03-18T23:19:19.302 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 166, 2024-03-18T23:19:19.302 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.012514307, 2024-03-18T23:19:19.302 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000075387 2024-03-18T23:19:19.303 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:19.303 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_handle_ack_lat": { 2024-03-18T23:19:19.303 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-03-18T23:19:19.303 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-03-18T23:19:19.303 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-03-18T23:19:19.303 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:19.303 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_encrypted_bytes": 320, 2024-03-18T23:19:19.303 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_encrypted_bytes": 185408 2024-03-18T23:19:19.304 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:19.304 INFO:teuthology.orchestra.run.smithi049.stdout: "AsyncMessenger::Worker-2": { 2024-03-18T23:19:19.304 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_messages": 4, 2024-03-18T23:19:19.304 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_messages": 4, 2024-03-18T23:19:19.304 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_bytes": 3024, 2024-03-18T23:19:19.304 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_bytes": 448, 2024-03-18T23:19:19.304 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_created_connections": 2, 2024-03-18T23:19:19.304 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_active_connections": 1, 2024-03-18T23:19:19.304 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_total_time": 0.000610668, 2024-03-18T23:19:19.305 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_send_time": 0.000041403, 2024-03-18T23:19:19.305 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_recv_time": 0.000557699, 2024-03-18T23:19:19.305 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-18T23:19:19.305 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_messages_queue_lat": { 2024-03-18T23:19:19.305 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 3, 2024-03-18T23:19:19.305 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000772691, 2024-03-18T23:19:19.305 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000257563 2024-03-18T23:19:19.305 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:19.306 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_handle_ack_lat": { 2024-03-18T23:19:19.306 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-03-18T23:19:19.306 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-03-18T23:19:19.306 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-03-18T23:19:19.306 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:19.306 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_encrypted_bytes": 3024, 2024-03-18T23:19:19.306 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_encrypted_bytes": 448 2024-03-18T23:19:19.306 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:19.307 INFO:teuthology.orchestra.run.smithi049.stdout: "cct": { 2024-03-18T23:19:19.307 INFO:teuthology.orchestra.run.smithi049.stdout: "total_workers": 0, 2024-03-18T23:19:19.307 INFO:teuthology.orchestra.run.smithi049.stdout: "unhealthy_workers": 0 2024-03-18T23:19:19.307 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:19.307 INFO:teuthology.orchestra.run.smithi049.stdout: "mempool": { 2024-03-18T23:19:19.307 INFO:teuthology.orchestra.run.smithi049.stdout: "bloom_filter_bytes": 0, 2024-03-18T23:19:19.307 INFO:teuthology.orchestra.run.smithi049.stdout: "bloom_filter_items": 0, 2024-03-18T23:19:19.307 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_alloc_bytes": 0, 2024-03-18T23:19:19.308 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_alloc_items": 0, 2024-03-18T23:19:19.308 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_data_bytes": 0, 2024-03-18T23:19:19.308 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_data_items": 0, 2024-03-18T23:19:19.308 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-18T23:19:19.308 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_onode_items": 0, 2024-03-18T23:19:19.308 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-18T23:19:19.308 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_meta_items": 0, 2024-03-18T23:19:19.308 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_other_bytes": 0, 2024-03-18T23:19:19.309 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_other_items": 0, 2024-03-18T23:19:19.309 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-18T23:19:19.309 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_buffer_items": 0, 2024-03-18T23:19:19.309 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_extent_bytes": 0, 2024-03-18T23:19:19.309 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_extent_items": 0, 2024-03-18T23:19:19.309 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_blob_bytes": 0, 2024-03-18T23:19:19.309 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_blob_items": 0, 2024-03-18T23:19:19.309 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-18T23:19:19.310 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_shared_blob_items": 0, 2024-03-18T23:19:19.310 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-18T23:19:19.310 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_inline_bl_items": 0, 2024-03-18T23:19:19.310 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_fsck_bytes": 0, 2024-03-18T23:19:19.310 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_fsck_items": 0, 2024-03-18T23:19:19.310 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_txc_bytes": 0, 2024-03-18T23:19:19.310 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_txc_items": 0, 2024-03-18T23:19:19.310 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-18T23:19:19.311 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_writing_deferred_items": 0, 2024-03-18T23:19:19.311 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_writing_bytes": 0, 2024-03-18T23:19:19.311 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_writing_items": 0, 2024-03-18T23:19:19.311 INFO:teuthology.orchestra.run.smithi049.stdout: "bluefs_bytes": 0, 2024-03-18T23:19:19.311 INFO:teuthology.orchestra.run.smithi049.stdout: "bluefs_items": 0, 2024-03-18T23:19:19.311 INFO:teuthology.orchestra.run.smithi049.stdout: "bluefs_file_reader_bytes": 0, 2024-03-18T23:19:19.311 INFO:teuthology.orchestra.run.smithi049.stdout: "bluefs_file_reader_items": 0, 2024-03-18T23:19:19.311 INFO:teuthology.orchestra.run.smithi049.stdout: "bluefs_file_writer_bytes": 0, 2024-03-18T23:19:19.312 INFO:teuthology.orchestra.run.smithi049.stdout: "bluefs_file_writer_items": 0, 2024-03-18T23:19:19.312 INFO:teuthology.orchestra.run.smithi049.stdout: "buffer_anon_bytes": 148977, 2024-03-18T23:19:19.312 INFO:teuthology.orchestra.run.smithi049.stdout: "buffer_anon_items": 30, 2024-03-18T23:19:19.312 INFO:teuthology.orchestra.run.smithi049.stdout: "buffer_meta_bytes": 0, 2024-03-18T23:19:19.312 INFO:teuthology.orchestra.run.smithi049.stdout: "buffer_meta_items": 0, 2024-03-18T23:19:19.312 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_bytes": 0, 2024-03-18T23:19:19.312 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_items": 0, 2024-03-18T23:19:19.312 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_mapbl_bytes": 0, 2024-03-18T23:19:19.313 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_mapbl_items": 0, 2024-03-18T23:19:19.313 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_pglog_bytes": 0, 2024-03-18T23:19:19.313 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_pglog_items": 0, 2024-03-18T23:19:19.313 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_bytes": 0, 2024-03-18T23:19:19.313 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_items": 0, 2024-03-18T23:19:19.313 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_mapping_bytes": 0, 2024-03-18T23:19:19.313 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_mapping_items": 0, 2024-03-18T23:19:19.313 INFO:teuthology.orchestra.run.smithi049.stdout: "pgmap_bytes": 0, 2024-03-18T23:19:19.314 INFO:teuthology.orchestra.run.smithi049.stdout: "pgmap_items": 0, 2024-03-18T23:19:19.314 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_co_bytes": 928, 2024-03-18T23:19:19.314 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_co_items": 2, 2024-03-18T23:19:19.314 INFO:teuthology.orchestra.run.smithi049.stdout: "unittest_1_bytes": 0, 2024-03-18T23:19:19.314 INFO:teuthology.orchestra.run.smithi049.stdout: "unittest_1_items": 0, 2024-03-18T23:19:19.314 INFO:teuthology.orchestra.run.smithi049.stdout: "unittest_2_bytes": 0, 2024-03-18T23:19:19.314 INFO:teuthology.orchestra.run.smithi049.stdout: "unittest_2_items": 0 2024-03-18T23:19:19.315 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-18T23:19:19.315 INFO:teuthology.orchestra.run.smithi049.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-18T23:19:19.315 INFO:teuthology.orchestra.run.smithi049.stdout: "val": 0, 2024-03-18T23:19:19.315 INFO:teuthology.orchestra.run.smithi049.stdout: "max": 104857600, 2024-03-18T23:19:19.315 INFO:teuthology.orchestra.run.smithi049.stdout: "get_started": 0, 2024-03-18T23:19:19.315 INFO:teuthology.orchestra.run.smithi049.stdout: "get": 234, 2024-03-18T23:19:19.315 INFO:teuthology.orchestra.run.smithi049.stdout: "get_sum": 1407717, 2024-03-18T23:19:19.315 INFO:teuthology.orchestra.run.smithi049.stdout: "get_or_fail_fail": 0, 2024-03-18T23:19:19.316 INFO:teuthology.orchestra.run.smithi049.stdout: "get_or_fail_success": 234, 2024-03-18T23:19:19.316 INFO:teuthology.orchestra.run.smithi049.stdout: "take": 0, 2024-03-18T23:19:19.316 INFO:teuthology.orchestra.run.smithi049.stdout: "take_sum": 0, 2024-03-18T23:19:19.316 INFO:teuthology.orchestra.run.smithi049.stdout: "put": 234, 2024-03-18T23:19:19.316 INFO:teuthology.orchestra.run.smithi049.stdout: "put_sum": 1407717, 2024-03-18T23:19:19.316 INFO:teuthology.orchestra.run.smithi049.stdout: "wait": { 2024-03-18T23:19:19.316 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-03-18T23:19:19.316 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-03-18T23:19:19.317 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-03-18T23:19:19.317 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:19:19.317 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-18T23:19:19.317 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-03-18T23:19:19.564 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:19:19 smithi139 ceph-mon[67282]: pgmap v345: 97 pgs: 97 active+clean; 69 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 1.7 MiB/s wr, 71 op/s 2024-03-18T23:19:19.564 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:19:19 smithi139 ceph-mon[67282]: osdmap e101: 12 total, 12 up, 12 in 2024-03-18T23:19:19.629 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:19:19 smithi049 ceph-mon[62991]: pgmap v345: 97 pgs: 97 active+clean; 69 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 1.7 MiB/s wr, 71 op/s 2024-03-18T23:19:19.629 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:19:19 smithi049 ceph-mon[62991]: osdmap e101: 12 total, 12 up, 12 in 2024-03-18T23:19:19.710 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:19:19 smithi175 ceph-mon[67352]: pgmap v345: 97 pgs: 97 active+clean; 69 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 1.7 MiB/s wr, 71 op/s 2024-03-18T23:19:19.710 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:19:19 smithi175 ceph-mon[67352]: osdmap e101: 12 total, 12 up, 12 in 2024-03-18T23:19:20.056 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-03-18T23:19:20.056 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-03-18T23:19:20.056 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.j 2024-03-18T23:19:20.056 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.j 2024-03-18T23:19:20.056 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.j 2024-03-18T23:19:20.056 DEBUG:tasks.check_counter:Getting stats from b 2024-03-18T23:19:20.057 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell --fsid c63591ea-e57a-11ee-95c9-87774f69a715 -- ceph daemon mds.b perf dump 2024-03-18T23:19:20.366 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/mon.b/config 2024-03-18T23:19:21.564 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:19:21 smithi139 ceph-mon[67282]: pgmap v347: 97 pgs: 97 active+clean; 69 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 1.3 MiB/s wr, 58 op/s 2024-03-18T23:19:21.628 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:19:21 smithi049 ceph-mon[62991]: pgmap v347: 97 pgs: 97 active+clean; 69 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 1.3 MiB/s wr, 58 op/s 2024-03-18T23:19:21.710 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:19:21 smithi175 ceph-mon[67352]: pgmap v347: 97 pgs: 97 active+clean; 69 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 1.3 MiB/s wr, 58 op/s 2024-03-18T23:19:22.280 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-03-18T23:19:22.280 INFO:teuthology.orchestra.run.smithi139.stdout: "AsyncMessenger::Worker-0": { 2024-03-18T23:19:22.280 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_recv_messages": 217352, 2024-03-18T23:19:22.280 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_messages": 391381, 2024-03-18T23:19:22.281 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_recv_bytes": 39736469, 2024-03-18T23:19:22.281 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_bytes": 455266534, 2024-03-18T23:19:22.281 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_created_connections": 11, 2024-03-18T23:19:22.281 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_active_connections": 4, 2024-03-18T23:19:22.281 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_total_time": 14.266127356, 2024-03-18T23:19:22.281 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_send_time": 6.071456316, 2024-03-18T23:19:22.281 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_recv_time": 7.087841688, 2024-03-18T23:19:22.281 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_fast_dispatch_time": 1.331335036, 2024-03-18T23:19:22.282 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_messages_queue_lat": { 2024-03-18T23:19:22.282 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 391385, 2024-03-18T23:19:22.282 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 29.733735074, 2024-03-18T23:19:22.282 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000075970 2024-03-18T23:19:22.282 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:22.282 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_handle_ack_lat": { 2024-03-18T23:19:22.282 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 520590, 2024-03-18T23:19:22.282 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.080829753, 2024-03-18T23:19:22.283 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000155 2024-03-18T23:19:22.283 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:22.283 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_recv_encrypted_bytes": 160, 2024-03-18T23:19:22.283 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_encrypted_bytes": 329840 2024-03-18T23:19:22.283 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:22.283 INFO:teuthology.orchestra.run.smithi139.stdout: "AsyncMessenger::Worker-1": { 2024-03-18T23:19:22.283 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_recv_messages": 430428, 2024-03-18T23:19:22.283 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_messages": 615834, 2024-03-18T23:19:22.283 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_recv_bytes": 151397982, 2024-03-18T23:19:22.284 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_bytes": 772489533, 2024-03-18T23:19:22.284 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_created_connections": 22, 2024-03-18T23:19:22.284 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_active_connections": 8, 2024-03-18T23:19:22.284 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_total_time": 27.334171912, 2024-03-18T23:19:22.284 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_send_time": 12.473895448, 2024-03-18T23:19:22.284 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_recv_time": 18.310522046, 2024-03-18T23:19:22.284 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_fast_dispatch_time": 1.771248015, 2024-03-18T23:19:22.284 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_messages_queue_lat": { 2024-03-18T23:19:22.285 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 615829, 2024-03-18T23:19:22.285 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 50.008613086, 2024-03-18T23:19:22.285 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000081205 2024-03-18T23:19:22.285 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:22.285 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_handle_ack_lat": { 2024-03-18T23:19:22.285 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 893758, 2024-03-18T23:19:22.285 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.112385936, 2024-03-18T23:19:22.285 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000125 2024-03-18T23:19:22.285 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:22.286 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_recv_encrypted_bytes": 133652912, 2024-03-18T23:19:22.286 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_encrypted_bytes": 514952864 2024-03-18T23:19:22.286 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:22.286 INFO:teuthology.orchestra.run.smithi139.stdout: "AsyncMessenger::Worker-2": { 2024-03-18T23:19:22.286 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_recv_messages": 204767, 2024-03-18T23:19:22.286 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_messages": 378640, 2024-03-18T23:19:22.286 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_recv_bytes": 36215517, 2024-03-18T23:19:22.287 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_bytes": 415482439, 2024-03-18T23:19:22.287 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_created_connections": 103, 2024-03-18T23:19:22.287 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_active_connections": 6, 2024-03-18T23:19:22.287 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_total_time": 15.149541641, 2024-03-18T23:19:22.287 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_send_time": 7.179887796, 2024-03-18T23:19:22.287 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_recv_time": 7.643713397, 2024-03-18T23:19:22.287 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_fast_dispatch_time": 0.996974816, 2024-03-18T23:19:22.287 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_messages_queue_lat": { 2024-03-18T23:19:22.287 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 378625, 2024-03-18T23:19:22.288 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 17.220446018, 2024-03-18T23:19:22.288 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000045481 2024-03-18T23:19:22.288 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:22.288 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_handle_ack_lat": { 2024-03-18T23:19:22.288 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 521202, 2024-03-18T23:19:22.288 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.080452675, 2024-03-18T23:19:22.288 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000154 2024-03-18T23:19:22.288 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:22.289 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_recv_encrypted_bytes": 48192, 2024-03-18T23:19:22.289 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_encrypted_bytes": 139696 2024-03-18T23:19:22.289 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:22.289 INFO:teuthology.orchestra.run.smithi139.stdout: "cct": { 2024-03-18T23:19:22.289 INFO:teuthology.orchestra.run.smithi139.stdout: "total_workers": 1, 2024-03-18T23:19:22.289 INFO:teuthology.orchestra.run.smithi139.stdout: "unhealthy_workers": 0 2024-03-18T23:19:22.289 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:22.289 INFO:teuthology.orchestra.run.smithi139.stdout: "finisher-MDSRank": { 2024-03-18T23:19:22.289 INFO:teuthology.orchestra.run.smithi139.stdout: "queue_len": 0, 2024-03-18T23:19:22.290 INFO:teuthology.orchestra.run.smithi139.stdout: "complete_latency": { 2024-03-18T23:19:22.290 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 122404, 2024-03-18T23:19:22.290 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 141.895627821, 2024-03-18T23:19:22.290 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.001159240 2024-03-18T23:19:22.290 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-03-18T23:19:22.290 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:22.290 INFO:teuthology.orchestra.run.smithi139.stdout: "finisher-PurgeQueue": { 2024-03-18T23:19:22.290 INFO:teuthology.orchestra.run.smithi139.stdout: "queue_len": 0, 2024-03-18T23:19:22.291 INFO:teuthology.orchestra.run.smithi139.stdout: "complete_latency": { 2024-03-18T23:19:22.291 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 68140, 2024-03-18T23:19:22.291 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 10.588750088, 2024-03-18T23:19:22.291 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000155396 2024-03-18T23:19:22.291 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-03-18T23:19:22.291 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:22.291 INFO:teuthology.orchestra.run.smithi139.stdout: "mds": { 2024-03-18T23:19:22.291 INFO:teuthology.orchestra.run.smithi139.stdout: "request": 292208, 2024-03-18T23:19:22.291 INFO:teuthology.orchestra.run.smithi139.stdout: "reply": 292206, 2024-03-18T23:19:22.292 INFO:teuthology.orchestra.run.smithi139.stdout: "reply_latency": { 2024-03-18T23:19:22.292 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 292206, 2024-03-18T23:19:22.292 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 457.210550112, 2024-03-18T23:19:22.292 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.001564685 2024-03-18T23:19:22.292 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:22.292 INFO:teuthology.orchestra.run.smithi139.stdout: "slow_reply": 0, 2024-03-18T23:19:22.292 INFO:teuthology.orchestra.run.smithi139.stdout: "forward": 2, 2024-03-18T23:19:22.292 INFO:teuthology.orchestra.run.smithi139.stdout: "dir_fetch_complete": 0, 2024-03-18T23:19:22.293 INFO:teuthology.orchestra.run.smithi139.stdout: "dir_fetch_keys": 0, 2024-03-18T23:19:22.293 INFO:teuthology.orchestra.run.smithi139.stdout: "dir_commit": 2962, 2024-03-18T23:19:22.293 INFO:teuthology.orchestra.run.smithi139.stdout: "dir_split": 17, 2024-03-18T23:19:22.293 INFO:teuthology.orchestra.run.smithi139.stdout: "dir_merge": 29, 2024-03-18T23:19:22.293 INFO:teuthology.orchestra.run.smithi139.stdout: "inodes": 2066, 2024-03-18T23:19:22.293 INFO:teuthology.orchestra.run.smithi139.stdout: "inodes_top": 733, 2024-03-18T23:19:22.293 INFO:teuthology.orchestra.run.smithi139.stdout: "inodes_bottom": 1333, 2024-03-18T23:19:22.293 INFO:teuthology.orchestra.run.smithi139.stdout: "inodes_pin_tail": 0, 2024-03-18T23:19:22.294 INFO:teuthology.orchestra.run.smithi139.stdout: "inodes_pinned": 1019, 2024-03-18T23:19:22.294 INFO:teuthology.orchestra.run.smithi139.stdout: "inodes_expired": 3, 2024-03-18T23:19:22.294 INFO:teuthology.orchestra.run.smithi139.stdout: "inodes_with_caps": 314, 2024-03-18T23:19:22.294 INFO:teuthology.orchestra.run.smithi139.stdout: "caps": 314, 2024-03-18T23:19:22.294 INFO:teuthology.orchestra.run.smithi139.stdout: "subtrees": 3, 2024-03-18T23:19:22.294 INFO:teuthology.orchestra.run.smithi139.stdout: "traverse": 405965, 2024-03-18T23:19:22.294 INFO:teuthology.orchestra.run.smithi139.stdout: "traverse_hit": 259217, 2024-03-18T23:19:22.294 INFO:teuthology.orchestra.run.smithi139.stdout: "traverse_forward": 2, 2024-03-18T23:19:22.295 INFO:teuthology.orchestra.run.smithi139.stdout: "traverse_discover": 0, 2024-03-18T23:19:22.295 INFO:teuthology.orchestra.run.smithi139.stdout: "traverse_dir_fetch": 0, 2024-03-18T23:19:22.295 INFO:teuthology.orchestra.run.smithi139.stdout: "traverse_remote_ino": 0, 2024-03-18T23:19:22.295 INFO:teuthology.orchestra.run.smithi139.stdout: "traverse_lock": 2965, 2024-03-18T23:19:22.295 INFO:teuthology.orchestra.run.smithi139.stdout: "load_cent": 0, 2024-03-18T23:19:22.295 INFO:teuthology.orchestra.run.smithi139.stdout: "q": 0, 2024-03-18T23:19:22.295 INFO:teuthology.orchestra.run.smithi139.stdout: "exported": 1, 2024-03-18T23:19:22.295 INFO:teuthology.orchestra.run.smithi139.stdout: "exported_inodes": 3, 2024-03-18T23:19:22.296 INFO:teuthology.orchestra.run.smithi139.stdout: "imported": 0, 2024-03-18T23:19:22.296 INFO:teuthology.orchestra.run.smithi139.stdout: "imported_inodes": 0, 2024-03-18T23:19:22.296 INFO:teuthology.orchestra.run.smithi139.stdout: "openino_dir_fetch": 0, 2024-03-18T23:19:22.296 INFO:teuthology.orchestra.run.smithi139.stdout: "openino_backtrace_fetch": 0, 2024-03-18T23:19:22.296 INFO:teuthology.orchestra.run.smithi139.stdout: "openino_peer_discover": 0, 2024-03-18T23:19:22.296 INFO:teuthology.orchestra.run.smithi139.stdout: "root_rfiles": 69, 2024-03-18T23:19:22.296 INFO:teuthology.orchestra.run.smithi139.stdout: "root_rbytes": 21153440, 2024-03-18T23:19:22.296 INFO:teuthology.orchestra.run.smithi139.stdout: "root_rsnaps": 6, 2024-03-18T23:19:22.296 INFO:teuthology.orchestra.run.smithi139.stdout: "scrub_backtrace_fetch": 0, 2024-03-18T23:19:22.297 INFO:teuthology.orchestra.run.smithi139.stdout: "scrub_set_tag": 0, 2024-03-18T23:19:22.297 INFO:teuthology.orchestra.run.smithi139.stdout: "scrub_backtrace_repaired": 0, 2024-03-18T23:19:22.297 INFO:teuthology.orchestra.run.smithi139.stdout: "scrub_inotable_repaired": 0, 2024-03-18T23:19:22.297 INFO:teuthology.orchestra.run.smithi139.stdout: "scrub_dir_inodes": 0, 2024-03-18T23:19:22.297 INFO:teuthology.orchestra.run.smithi139.stdout: "scrub_dir_base_inodes": 0, 2024-03-18T23:19:22.297 INFO:teuthology.orchestra.run.smithi139.stdout: "scrub_dirfrag_rstats": 0, 2024-03-18T23:19:22.297 INFO:teuthology.orchestra.run.smithi139.stdout: "scrub_file_inodes": 0, 2024-03-18T23:19:22.297 INFO:teuthology.orchestra.run.smithi139.stdout: "handle_inode_file_caps": 2, 2024-03-18T23:19:22.298 INFO:teuthology.orchestra.run.smithi139.stdout: "ceph_cap_op_revoke": 7642, 2024-03-18T23:19:22.298 INFO:teuthology.orchestra.run.smithi139.stdout: "ceph_cap_op_grant": 148256, 2024-03-18T23:19:22.298 INFO:teuthology.orchestra.run.smithi139.stdout: "ceph_cap_op_trunc": 0, 2024-03-18T23:19:22.298 INFO:teuthology.orchestra.run.smithi139.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-18T23:19:22.298 INFO:teuthology.orchestra.run.smithi139.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-18T23:19:22.298 INFO:teuthology.orchestra.run.smithi139.stdout: "handle_client_caps": 63956, 2024-03-18T23:19:22.298 INFO:teuthology.orchestra.run.smithi139.stdout: "handle_client_caps_dirty": 30226, 2024-03-18T23:19:22.298 INFO:teuthology.orchestra.run.smithi139.stdout: "handle_client_cap_release": 15093, 2024-03-18T23:19:22.299 INFO:teuthology.orchestra.run.smithi139.stdout: "process_request_cap_release": 203036 2024-03-18T23:19:22.299 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:22.299 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_cache": { 2024-03-18T23:19:22.299 INFO:teuthology.orchestra.run.smithi139.stdout: "dir_update": 12, 2024-03-18T23:19:22.299 INFO:teuthology.orchestra.run.smithi139.stdout: "dir_update_receipt": 0, 2024-03-18T23:19:22.299 INFO:teuthology.orchestra.run.smithi139.stdout: "dir_try_discover": 0, 2024-03-18T23:19:22.299 INFO:teuthology.orchestra.run.smithi139.stdout: "dir_send_discover": 0, 2024-03-18T23:19:22.299 INFO:teuthology.orchestra.run.smithi139.stdout: "dir_handle_discover": 32, 2024-03-18T23:19:22.299 INFO:teuthology.orchestra.run.smithi139.stdout: "num_strays": 244, 2024-03-18T23:19:22.300 INFO:teuthology.orchestra.run.smithi139.stdout: "num_strays_delayed": 0, 2024-03-18T23:19:22.300 INFO:teuthology.orchestra.run.smithi139.stdout: "num_strays_enqueuing": 0, 2024-03-18T23:19:22.300 INFO:teuthology.orchestra.run.smithi139.stdout: "strays_created": 38459, 2024-03-18T23:19:22.300 INFO:teuthology.orchestra.run.smithi139.stdout: "strays_enqueued": 38215, 2024-03-18T23:19:22.300 INFO:teuthology.orchestra.run.smithi139.stdout: "strays_reintegrated": 0, 2024-03-18T23:19:22.300 INFO:teuthology.orchestra.run.smithi139.stdout: "strays_migrated": 0, 2024-03-18T23:19:22.300 INFO:teuthology.orchestra.run.smithi139.stdout: "num_recovering_processing": 0, 2024-03-18T23:19:22.300 INFO:teuthology.orchestra.run.smithi139.stdout: "num_recovering_enqueued": 0, 2024-03-18T23:19:22.300 INFO:teuthology.orchestra.run.smithi139.stdout: "num_recovering_prioritized": 0, 2024-03-18T23:19:22.300 INFO:teuthology.orchestra.run.smithi139.stdout: "recovery_started": 0, 2024-03-18T23:19:22.301 INFO:teuthology.orchestra.run.smithi139.stdout: "recovery_completed": 0, 2024-03-18T23:19:22.301 INFO:teuthology.orchestra.run.smithi139.stdout: "ireq_quiesce_path": 0, 2024-03-18T23:19:22.301 INFO:teuthology.orchestra.run.smithi139.stdout: "ireq_quiesce_inode": 0, 2024-03-18T23:19:22.301 INFO:teuthology.orchestra.run.smithi139.stdout: "ireq_enqueue_scrub": 0, 2024-03-18T23:19:22.301 INFO:teuthology.orchestra.run.smithi139.stdout: "ireq_exportdir": 1, 2024-03-18T23:19:22.301 INFO:teuthology.orchestra.run.smithi139.stdout: "ireq_flush": 0, 2024-03-18T23:19:22.301 INFO:teuthology.orchestra.run.smithi139.stdout: "ireq_fragmentdir": 46, 2024-03-18T23:19:22.301 INFO:teuthology.orchestra.run.smithi139.stdout: "ireq_fragstats": 0, 2024-03-18T23:19:22.302 INFO:teuthology.orchestra.run.smithi139.stdout: "ireq_inodestats": 0 2024-03-18T23:19:22.302 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:22.302 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_log": { 2024-03-18T23:19:22.302 INFO:teuthology.orchestra.run.smithi139.stdout: "evlrg": 0, 2024-03-18T23:19:22.302 INFO:teuthology.orchestra.run.smithi139.stdout: "evadd": 179813, 2024-03-18T23:19:22.302 INFO:teuthology.orchestra.run.smithi139.stdout: "evex": 177751, 2024-03-18T23:19:22.302 INFO:teuthology.orchestra.run.smithi139.stdout: "evtrm": 177751, 2024-03-18T23:19:22.302 INFO:teuthology.orchestra.run.smithi139.stdout: "ev": 2062, 2024-03-18T23:19:22.303 INFO:teuthology.orchestra.run.smithi139.stdout: "evexg": 0, 2024-03-18T23:19:22.303 INFO:teuthology.orchestra.run.smithi139.stdout: "evexd": 3370, 2024-03-18T23:19:22.303 INFO:teuthology.orchestra.run.smithi139.stdout: "segadd": 279, 2024-03-18T23:19:22.303 INFO:teuthology.orchestra.run.smithi139.stdout: "segex": 276, 2024-03-18T23:19:22.303 INFO:teuthology.orchestra.run.smithi139.stdout: "segtrm": 276, 2024-03-18T23:19:22.303 INFO:teuthology.orchestra.run.smithi139.stdout: "seg": 3, 2024-03-18T23:19:22.303 INFO:teuthology.orchestra.run.smithi139.stdout: "segmjr": 1, 2024-03-18T23:19:22.303 INFO:teuthology.orchestra.run.smithi139.stdout: "segexg": 0, 2024-03-18T23:19:22.303 INFO:teuthology.orchestra.run.smithi139.stdout: "segexd": 5, 2024-03-18T23:19:22.304 INFO:teuthology.orchestra.run.smithi139.stdout: "expos": 671845785, 2024-03-18T23:19:22.304 INFO:teuthology.orchestra.run.smithi139.stdout: "wrpos": 678441461, 2024-03-18T23:19:22.304 INFO:teuthology.orchestra.run.smithi139.stdout: "rdpos": 0, 2024-03-18T23:19:22.304 INFO:teuthology.orchestra.run.smithi139.stdout: "jlat": { 2024-03-18T23:19:22.304 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 62017, 2024-03-18T23:19:22.304 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 158.359971276, 2024-03-18T23:19:22.304 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.002553492 2024-03-18T23:19:22.304 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:22.305 INFO:teuthology.orchestra.run.smithi139.stdout: "replayed": 0 2024-03-18T23:19:22.305 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:22.305 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_mem": { 2024-03-18T23:19:22.305 INFO:teuthology.orchestra.run.smithi139.stdout: "ino": 1084, 2024-03-18T23:19:22.305 INFO:teuthology.orchestra.run.smithi139.stdout: "ino+": 39183, 2024-03-18T23:19:22.305 INFO:teuthology.orchestra.run.smithi139.stdout: "ino-": 38099, 2024-03-18T23:19:22.305 INFO:teuthology.orchestra.run.smithi139.stdout: "dir": 58, 2024-03-18T23:19:22.305 INFO:teuthology.orchestra.run.smithi139.stdout: "dir+": 197, 2024-03-18T23:19:22.305 INFO:teuthology.orchestra.run.smithi139.stdout: "dir-": 139, 2024-03-18T23:19:22.306 INFO:teuthology.orchestra.run.smithi139.stdout: "dn": 2063, 2024-03-18T23:19:22.306 INFO:teuthology.orchestra.run.smithi139.stdout: "dn+": 84838, 2024-03-18T23:19:22.306 INFO:teuthology.orchestra.run.smithi139.stdout: "dn-": 82775, 2024-03-18T23:19:22.306 INFO:teuthology.orchestra.run.smithi139.stdout: "cap": 314, 2024-03-18T23:19:22.306 INFO:teuthology.orchestra.run.smithi139.stdout: "cap+": 42712, 2024-03-18T23:19:22.306 INFO:teuthology.orchestra.run.smithi139.stdout: "cap-": 42398, 2024-03-18T23:19:22.306 INFO:teuthology.orchestra.run.smithi139.stdout: "rss": 93316, 2024-03-18T23:19:22.306 INFO:teuthology.orchestra.run.smithi139.stdout: "heap": 231804 2024-03-18T23:19:22.307 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:22.307 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_server": { 2024-03-18T23:19:22.307 INFO:teuthology.orchestra.run.smithi139.stdout: "dispatch_client_request": 415941, 2024-03-18T23:19:22.307 INFO:teuthology.orchestra.run.smithi139.stdout: "dispatch_server_request": 12, 2024-03-18T23:19:22.307 INFO:teuthology.orchestra.run.smithi139.stdout: "handle_client_request": 292208, 2024-03-18T23:19:22.307 INFO:teuthology.orchestra.run.smithi139.stdout: "handle_client_session": 5431, 2024-03-18T23:19:22.307 INFO:teuthology.orchestra.run.smithi139.stdout: "handle_peer_request": 24, 2024-03-18T23:19:22.307 INFO:teuthology.orchestra.run.smithi139.stdout: "req_create_latency": { 2024-03-18T23:19:22.307 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 38528, 2024-03-18T23:19:22.308 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 60.536343725, 2024-03-18T23:19:22.308 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.001571229 2024-03-18T23:19:22.308 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:22.308 INFO:teuthology.orchestra.run.smithi139.stdout: "req_getattr_latency": { 2024-03-18T23:19:22.308 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 281, 2024-03-18T23:19:22.308 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.500721860, 2024-03-18T23:19:22.308 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.001781928 2024-03-18T23:19:22.308 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:22.309 INFO:teuthology.orchestra.run.smithi139.stdout: "req_getfilelock_latency": { 2024-03-18T23:19:22.309 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-03-18T23:19:22.309 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-03-18T23:19:22.309 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-03-18T23:19:22.309 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:22.309 INFO:teuthology.orchestra.run.smithi139.stdout: "req_link_latency": { 2024-03-18T23:19:22.309 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-03-18T23:19:22.309 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-03-18T23:19:22.309 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-03-18T23:19:22.310 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:22.310 INFO:teuthology.orchestra.run.smithi139.stdout: "req_lookup_latency": { 2024-03-18T23:19:22.310 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 87343, 2024-03-18T23:19:22.310 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 55.854851642, 2024-03-18T23:19:22.310 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000639488 2024-03-18T23:19:22.310 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:22.310 INFO:teuthology.orchestra.run.smithi139.stdout: "req_lookuphash_latency": { 2024-03-18T23:19:22.310 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-03-18T23:19:22.311 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-03-18T23:19:22.311 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-03-18T23:19:22.311 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:22.311 INFO:teuthology.orchestra.run.smithi139.stdout: "req_lookupino_latency": { 2024-03-18T23:19:22.311 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 1, 2024-03-18T23:19:22.311 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000186443, 2024-03-18T23:19:22.311 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000186443 2024-03-18T23:19:22.311 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:22.311 INFO:teuthology.orchestra.run.smithi139.stdout: "req_lookupname_latency": { 2024-03-18T23:19:22.312 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-03-18T23:19:22.312 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-03-18T23:19:22.312 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-03-18T23:19:22.312 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:22.312 INFO:teuthology.orchestra.run.smithi139.stdout: "req_lookupparent_latency": { 2024-03-18T23:19:22.312 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-03-18T23:19:22.312 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-03-18T23:19:22.312 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-03-18T23:19:22.313 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:22.313 INFO:teuthology.orchestra.run.smithi139.stdout: "req_lookupsnap_latency": { 2024-03-18T23:19:22.313 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-03-18T23:19:22.313 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-03-18T23:19:22.314 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-03-18T23:19:22.314 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:22.314 INFO:teuthology.orchestra.run.smithi139.stdout: "req_lssnap_latency": { 2024-03-18T23:19:22.314 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 11, 2024-03-18T23:19:22.315 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.008290960, 2024-03-18T23:19:22.315 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000753723 2024-03-18T23:19:22.315 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:22.315 INFO:teuthology.orchestra.run.smithi139.stdout: "req_mkdir_latency": { 2024-03-18T23:19:22.315 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 20, 2024-03-18T23:19:22.315 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.027022145, 2024-03-18T23:19:22.315 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.001351107 2024-03-18T23:19:22.316 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:22.316 INFO:teuthology.orchestra.run.smithi139.stdout: "req_mknod_latency": { 2024-03-18T23:19:22.316 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-03-18T23:19:22.316 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-03-18T23:19:22.316 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-03-18T23:19:22.316 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:22.316 INFO:teuthology.orchestra.run.smithi139.stdout: "req_mksnap_latency": { 2024-03-18T23:19:22.316 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 11, 2024-03-18T23:19:22.317 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.117199605, 2024-03-18T23:19:22.317 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.010654509 2024-03-18T23:19:22.317 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:22.317 INFO:teuthology.orchestra.run.smithi139.stdout: "req_open_latency": { 2024-03-18T23:19:22.317 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 1, 2024-03-18T23:19:22.317 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000145099, 2024-03-18T23:19:22.317 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000145099 2024-03-18T23:19:22.317 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:22.317 INFO:teuthology.orchestra.run.smithi139.stdout: "req_readdir_latency": { 2024-03-18T23:19:22.318 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 118251, 2024-03-18T23:19:22.318 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 256.684661184, 2024-03-18T23:19:22.318 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.002170676 2024-03-18T23:19:22.318 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:22.318 INFO:teuthology.orchestra.run.smithi139.stdout: "req_rename_latency": { 2024-03-18T23:19:22.318 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 8049, 2024-03-18T23:19:22.318 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 16.878187481, 2024-03-18T23:19:22.319 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.002096929 2024-03-18T23:19:22.319 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:22.319 INFO:teuthology.orchestra.run.smithi139.stdout: "req_renamesnap_latency": { 2024-03-18T23:19:22.319 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-03-18T23:19:22.319 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-03-18T23:19:22.319 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-03-18T23:19:22.319 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:22.319 INFO:teuthology.orchestra.run.smithi139.stdout: "req_snapdiff_latency": { 2024-03-18T23:19:22.320 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-03-18T23:19:22.320 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-03-18T23:19:22.320 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-03-18T23:19:22.320 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:22.320 INFO:teuthology.orchestra.run.smithi139.stdout: "req_rmdir_latency": { 2024-03-18T23:19:22.320 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-03-18T23:19:22.320 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-03-18T23:19:22.320 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-03-18T23:19:22.320 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:22.321 INFO:teuthology.orchestra.run.smithi139.stdout: "req_rmsnap_latency": { 2024-03-18T23:19:22.321 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 5, 2024-03-18T23:19:22.321 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.041058289, 2024-03-18T23:19:22.321 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.008211657 2024-03-18T23:19:22.321 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:22.321 INFO:teuthology.orchestra.run.smithi139.stdout: "req_rmxattr_latency": { 2024-03-18T23:19:22.321 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-03-18T23:19:22.321 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-03-18T23:19:22.321 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-03-18T23:19:22.322 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:22.322 INFO:teuthology.orchestra.run.smithi139.stdout: "req_setattr_latency": { 2024-03-18T23:19:22.322 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 2, 2024-03-18T23:19:22.322 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.005196758, 2024-03-18T23:19:22.322 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.002598379 2024-03-18T23:19:22.322 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:22.322 INFO:teuthology.orchestra.run.smithi139.stdout: "req_setdirlayout_latency": { 2024-03-18T23:19:22.322 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-03-18T23:19:22.323 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-03-18T23:19:22.323 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-03-18T23:19:22.323 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:22.323 INFO:teuthology.orchestra.run.smithi139.stdout: "req_setfilelock_latency": { 2024-03-18T23:19:22.323 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 1236, 2024-03-18T23:19:22.323 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 1.327591557, 2024-03-18T23:19:22.323 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.001074103 2024-03-18T23:19:22.323 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:22.324 INFO:teuthology.orchestra.run.smithi139.stdout: "req_setlayout_latency": { 2024-03-18T23:19:22.324 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-03-18T23:19:22.324 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-03-18T23:19:22.324 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-03-18T23:19:22.324 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:22.324 INFO:teuthology.orchestra.run.smithi139.stdout: "req_setxattr_latency": { 2024-03-18T23:19:22.324 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 8, 2024-03-18T23:19:22.324 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.019354002, 2024-03-18T23:19:22.324 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.002419250 2024-03-18T23:19:22.325 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:22.325 INFO:teuthology.orchestra.run.smithi139.stdout: "req_symlink_latency": { 2024-03-18T23:19:22.325 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-03-18T23:19:22.325 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-03-18T23:19:22.325 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-03-18T23:19:22.325 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:22.325 INFO:teuthology.orchestra.run.smithi139.stdout: "req_unlink_latency": { 2024-03-18T23:19:22.325 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 38459, 2024-03-18T23:19:22.326 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 65.209739362, 2024-03-18T23:19:22.326 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.001695565 2024-03-18T23:19:22.326 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:22.326 INFO:teuthology.orchestra.run.smithi139.stdout: "cap_revoke_eviction": 0, 2024-03-18T23:19:22.326 INFO:teuthology.orchestra.run.smithi139.stdout: "cap_acquisition_throttle": 0, 2024-03-18T23:19:22.326 INFO:teuthology.orchestra.run.smithi139.stdout: "req_getvxattr_latency": { 2024-03-18T23:19:22.326 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-03-18T23:19:22.326 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-03-18T23:19:22.326 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-03-18T23:19:22.327 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-03-18T23:19:22.327 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:22.327 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_sessions": { 2024-03-18T23:19:22.327 INFO:teuthology.orchestra.run.smithi139.stdout: "session_count": 2, 2024-03-18T23:19:22.327 INFO:teuthology.orchestra.run.smithi139.stdout: "session_add": 5, 2024-03-18T23:19:22.327 INFO:teuthology.orchestra.run.smithi139.stdout: "session_remove": 3, 2024-03-18T23:19:22.327 INFO:teuthology.orchestra.run.smithi139.stdout: "sessions_open": 2, 2024-03-18T23:19:22.327 INFO:teuthology.orchestra.run.smithi139.stdout: "sessions_stale": 0, 2024-03-18T23:19:22.328 INFO:teuthology.orchestra.run.smithi139.stdout: "total_load": 43545, 2024-03-18T23:19:22.328 INFO:teuthology.orchestra.run.smithi139.stdout: "average_load": 21772, 2024-03-18T23:19:22.328 INFO:teuthology.orchestra.run.smithi139.stdout: "avg_session_uptime": 647, 2024-03-18T23:19:22.328 INFO:teuthology.orchestra.run.smithi139.stdout: "mdthresh_evicted": 0 2024-03-18T23:19:22.328 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:22.328 INFO:teuthology.orchestra.run.smithi139.stdout: "mempool": { 2024-03-18T23:19:22.328 INFO:teuthology.orchestra.run.smithi139.stdout: "bloom_filter_bytes": 0, 2024-03-18T23:19:22.328 INFO:teuthology.orchestra.run.smithi139.stdout: "bloom_filter_items": 0, 2024-03-18T23:19:22.329 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_alloc_bytes": 0, 2024-03-18T23:19:22.329 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_alloc_items": 0, 2024-03-18T23:19:22.329 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_cache_data_bytes": 0, 2024-03-18T23:19:22.329 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_cache_data_items": 0, 2024-03-18T23:19:22.329 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-18T23:19:22.329 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_cache_onode_items": 0, 2024-03-18T23:19:22.329 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-18T23:19:22.329 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_cache_meta_items": 0, 2024-03-18T23:19:22.330 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_cache_other_bytes": 0, 2024-03-18T23:19:22.330 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_cache_other_items": 0, 2024-03-18T23:19:22.330 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-18T23:19:22.330 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_cache_buffer_items": 0, 2024-03-18T23:19:22.330 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_extent_bytes": 0, 2024-03-18T23:19:22.330 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_extent_items": 0, 2024-03-18T23:19:22.330 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_blob_bytes": 0, 2024-03-18T23:19:22.330 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_blob_items": 0, 2024-03-18T23:19:22.331 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-18T23:19:22.331 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_shared_blob_items": 0, 2024-03-18T23:19:22.331 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-18T23:19:22.331 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_inline_bl_items": 0, 2024-03-18T23:19:22.331 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_fsck_bytes": 0, 2024-03-18T23:19:22.331 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_fsck_items": 0, 2024-03-18T23:19:22.331 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_txc_bytes": 0, 2024-03-18T23:19:22.331 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_txc_items": 0, 2024-03-18T23:19:22.331 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-18T23:19:22.332 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_writing_deferred_items": 0, 2024-03-18T23:19:22.332 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_writing_bytes": 0, 2024-03-18T23:19:22.332 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_writing_items": 0, 2024-03-18T23:19:22.332 INFO:teuthology.orchestra.run.smithi139.stdout: "bluefs_bytes": 0, 2024-03-18T23:19:22.332 INFO:teuthology.orchestra.run.smithi139.stdout: "bluefs_items": 0, 2024-03-18T23:19:22.332 INFO:teuthology.orchestra.run.smithi139.stdout: "bluefs_file_reader_bytes": 0, 2024-03-18T23:19:22.332 INFO:teuthology.orchestra.run.smithi139.stdout: "bluefs_file_reader_items": 0, 2024-03-18T23:19:22.332 INFO:teuthology.orchestra.run.smithi139.stdout: "bluefs_file_writer_bytes": 0, 2024-03-18T23:19:22.333 INFO:teuthology.orchestra.run.smithi139.stdout: "bluefs_file_writer_items": 0, 2024-03-18T23:19:22.333 INFO:teuthology.orchestra.run.smithi139.stdout: "buffer_anon_bytes": 2619855, 2024-03-18T23:19:22.333 INFO:teuthology.orchestra.run.smithi139.stdout: "buffer_anon_items": 1461, 2024-03-18T23:19:22.333 INFO:teuthology.orchestra.run.smithi139.stdout: "buffer_meta_bytes": 440, 2024-03-18T23:19:22.333 INFO:teuthology.orchestra.run.smithi139.stdout: "buffer_meta_items": 5, 2024-03-18T23:19:22.333 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_bytes": 0, 2024-03-18T23:19:22.333 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_items": 0, 2024-03-18T23:19:22.333 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_mapbl_bytes": 0, 2024-03-18T23:19:22.334 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_mapbl_items": 0, 2024-03-18T23:19:22.334 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_pglog_bytes": 0, 2024-03-18T23:19:22.334 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_pglog_items": 0, 2024-03-18T23:19:22.334 INFO:teuthology.orchestra.run.smithi139.stdout: "osdmap_bytes": 8256, 2024-03-18T23:19:22.334 INFO:teuthology.orchestra.run.smithi139.stdout: "osdmap_items": 188, 2024-03-18T23:19:22.334 INFO:teuthology.orchestra.run.smithi139.stdout: "osdmap_mapping_bytes": 0, 2024-03-18T23:19:22.334 INFO:teuthology.orchestra.run.smithi139.stdout: "osdmap_mapping_items": 0, 2024-03-18T23:19:22.335 INFO:teuthology.orchestra.run.smithi139.stdout: "pgmap_bytes": 0, 2024-03-18T23:19:22.335 INFO:teuthology.orchestra.run.smithi139.stdout: "pgmap_items": 0, 2024-03-18T23:19:22.335 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_co_bytes": 4865020, 2024-03-18T23:19:22.335 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_co_items": 108534, 2024-03-18T23:19:22.335 INFO:teuthology.orchestra.run.smithi139.stdout: "unittest_1_bytes": 0, 2024-03-18T23:19:22.335 INFO:teuthology.orchestra.run.smithi139.stdout: "unittest_1_items": 0, 2024-03-18T23:19:22.335 INFO:teuthology.orchestra.run.smithi139.stdout: "unittest_2_bytes": 0, 2024-03-18T23:19:22.335 INFO:teuthology.orchestra.run.smithi139.stdout: "unittest_2_items": 0 2024-03-18T23:19:22.336 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:22.336 INFO:teuthology.orchestra.run.smithi139.stdout: "objecter": { 2024-03-18T23:19:22.336 INFO:teuthology.orchestra.run.smithi139.stdout: "op_active": 0, 2024-03-18T23:19:22.336 INFO:teuthology.orchestra.run.smithi139.stdout: "op_laggy": 0, 2024-03-18T23:19:22.336 INFO:teuthology.orchestra.run.smithi139.stdout: "op_send": 126095, 2024-03-18T23:19:22.336 INFO:teuthology.orchestra.run.smithi139.stdout: "op_send_bytes": 705290507, 2024-03-18T23:19:22.336 INFO:teuthology.orchestra.run.smithi139.stdout: "op_resend": 37, 2024-03-18T23:19:22.336 INFO:teuthology.orchestra.run.smithi139.stdout: "op_reply": 126095, 2024-03-18T23:19:22.336 INFO:teuthology.orchestra.run.smithi139.stdout: "op_latency": { 2024-03-18T23:19:22.337 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 126095, 2024-03-18T23:19:22.337 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 1523.675853298, 2024-03-18T23:19:22.337 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.012083554 2024-03-18T23:19:22.337 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:22.337 INFO:teuthology.orchestra.run.smithi139.stdout: "op_inflight": 0, 2024-03-18T23:19:22.337 INFO:teuthology.orchestra.run.smithi139.stdout: "oplen_avg": { 2024-03-18T23:19:22.337 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 126095, 2024-03-18T23:19:22.337 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 170801 2024-03-18T23:19:22.338 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:22.338 INFO:teuthology.orchestra.run.smithi139.stdout: "op": 126095, 2024-03-18T23:19:22.338 INFO:teuthology.orchestra.run.smithi139.stdout: "op_r": 1148, 2024-03-18T23:19:22.338 INFO:teuthology.orchestra.run.smithi139.stdout: "op_w": 124947, 2024-03-18T23:19:22.338 INFO:teuthology.orchestra.run.smithi139.stdout: "op_rmw": 0, 2024-03-18T23:19:22.338 INFO:teuthology.orchestra.run.smithi139.stdout: "op_pg": 0, 2024-03-18T23:19:22.338 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_stat": 2766, 2024-03-18T23:19:22.338 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_create": 18215, 2024-03-18T23:19:22.339 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_read": 1148, 2024-03-18T23:19:22.339 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_write": 63165, 2024-03-18T23:19:22.339 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_writefull": 705, 2024-03-18T23:19:22.339 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_writesame": 0, 2024-03-18T23:19:22.339 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_append": 0, 2024-03-18T23:19:22.339 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_zero": 0, 2024-03-18T23:19:22.339 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_truncate": 17, 2024-03-18T23:19:22.340 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_delete": 39168, 2024-03-18T23:19:22.340 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_mapext": 0, 2024-03-18T23:19:22.340 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_sparse_read": 0, 2024-03-18T23:19:22.340 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_clonerange": 0, 2024-03-18T23:19:22.340 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_getxattr": 0, 2024-03-18T23:19:22.340 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_setxattr": 36430, 2024-03-18T23:19:22.340 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_cmpxattr": 0, 2024-03-18T23:19:22.340 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_rmxattr": 0, 2024-03-18T23:19:22.341 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_resetxattrs": 0, 2024-03-18T23:19:22.341 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_call": 0, 2024-03-18T23:19:22.341 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_watch": 0, 2024-03-18T23:19:22.341 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_notify": 0, 2024-03-18T23:19:22.341 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_src_cmpxattr": 0, 2024-03-18T23:19:22.341 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_pgls": 0, 2024-03-18T23:19:22.341 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_pgls_filter": 0, 2024-03-18T23:19:22.341 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_other": 0, 2024-03-18T23:19:22.342 INFO:teuthology.orchestra.run.smithi139.stdout: "linger_active": 0, 2024-03-18T23:19:22.342 INFO:teuthology.orchestra.run.smithi139.stdout: "linger_send": 0, 2024-03-18T23:19:22.342 INFO:teuthology.orchestra.run.smithi139.stdout: "linger_resend": 0, 2024-03-18T23:19:22.342 INFO:teuthology.orchestra.run.smithi139.stdout: "linger_ping": 0, 2024-03-18T23:19:22.342 INFO:teuthology.orchestra.run.smithi139.stdout: "poolop_active": 0, 2024-03-18T23:19:22.342 INFO:teuthology.orchestra.run.smithi139.stdout: "poolop_send": 0, 2024-03-18T23:19:22.342 INFO:teuthology.orchestra.run.smithi139.stdout: "poolop_resend": 0, 2024-03-18T23:19:22.342 INFO:teuthology.orchestra.run.smithi139.stdout: "poolstat_active": 0, 2024-03-18T23:19:22.343 INFO:teuthology.orchestra.run.smithi139.stdout: "poolstat_send": 0, 2024-03-18T23:19:22.343 INFO:teuthology.orchestra.run.smithi139.stdout: "poolstat_resend": 0, 2024-03-18T23:19:22.343 INFO:teuthology.orchestra.run.smithi139.stdout: "statfs_active": 0, 2024-03-18T23:19:22.343 INFO:teuthology.orchestra.run.smithi139.stdout: "statfs_send": 0, 2024-03-18T23:19:22.343 INFO:teuthology.orchestra.run.smithi139.stdout: "statfs_resend": 0, 2024-03-18T23:19:22.343 INFO:teuthology.orchestra.run.smithi139.stdout: "command_active": 0, 2024-03-18T23:19:22.343 INFO:teuthology.orchestra.run.smithi139.stdout: "command_send": 0, 2024-03-18T23:19:22.343 INFO:teuthology.orchestra.run.smithi139.stdout: "command_resend": 0, 2024-03-18T23:19:22.343 INFO:teuthology.orchestra.run.smithi139.stdout: "map_epoch": 101, 2024-03-18T23:19:22.344 INFO:teuthology.orchestra.run.smithi139.stdout: "map_full": 0, 2024-03-18T23:19:22.344 INFO:teuthology.orchestra.run.smithi139.stdout: "map_inc": 16, 2024-03-18T23:19:22.344 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_sessions": 12, 2024-03-18T23:19:22.344 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_session_open": 12, 2024-03-18T23:19:22.344 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_session_close": 0, 2024-03-18T23:19:22.344 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_laggy": 0, 2024-03-18T23:19:22.344 INFO:teuthology.orchestra.run.smithi139.stdout: "omap_wr": 6731, 2024-03-18T23:19:22.344 INFO:teuthology.orchestra.run.smithi139.stdout: "omap_rd": 0, 2024-03-18T23:19:22.344 INFO:teuthology.orchestra.run.smithi139.stdout: "omap_del": 2456 2024-03-18T23:19:22.345 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:22.345 INFO:teuthology.orchestra.run.smithi139.stdout: "oft": { 2024-03-18T23:19:22.345 INFO:teuthology.orchestra.run.smithi139.stdout: "omap_total_objs": 1, 2024-03-18T23:19:22.345 INFO:teuthology.orchestra.run.smithi139.stdout: "omap_total_kv_pairs": 0, 2024-03-18T23:19:22.345 INFO:teuthology.orchestra.run.smithi139.stdout: "omap_total_updates": 0, 2024-03-18T23:19:22.345 INFO:teuthology.orchestra.run.smithi139.stdout: "omap_total_removes": 0 2024-03-18T23:19:22.345 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:22.345 INFO:teuthology.orchestra.run.smithi139.stdout: "osd-slow-ops": { 2024-03-18T23:19:22.345 INFO:teuthology.orchestra.run.smithi139.stdout: "slow_ops_count": 0 2024-03-18T23:19:22.346 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:22.346 INFO:teuthology.orchestra.run.smithi139.stdout: "purge_queue": { 2024-03-18T23:19:22.346 INFO:teuthology.orchestra.run.smithi139.stdout: "pq_executing_ops": 0, 2024-03-18T23:19:22.346 INFO:teuthology.orchestra.run.smithi139.stdout: "pq_executing_ops_high_water": 6, 2024-03-18T23:19:22.346 INFO:teuthology.orchestra.run.smithi139.stdout: "pq_executing": 0, 2024-03-18T23:19:22.346 INFO:teuthology.orchestra.run.smithi139.stdout: "pq_executing_high_water": 6, 2024-03-18T23:19:22.346 INFO:teuthology.orchestra.run.smithi139.stdout: "pq_executed_ops": 38215, 2024-03-18T23:19:22.346 INFO:teuthology.orchestra.run.smithi139.stdout: "pq_executed": 38215, 2024-03-18T23:19:22.346 INFO:teuthology.orchestra.run.smithi139.stdout: "pq_item_in_journal": 0 2024-03-18T23:19:22.347 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:22.347 INFO:teuthology.orchestra.run.smithi139.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-18T23:19:22.347 INFO:teuthology.orchestra.run.smithi139.stdout: "val": 0, 2024-03-18T23:19:22.347 INFO:teuthology.orchestra.run.smithi139.stdout: "max": 104857600, 2024-03-18T23:19:22.347 INFO:teuthology.orchestra.run.smithi139.stdout: "get_started": 0, 2024-03-18T23:19:22.347 INFO:teuthology.orchestra.run.smithi139.stdout: "get": 852543, 2024-03-18T23:19:22.347 INFO:teuthology.orchestra.run.smithi139.stdout: "get_sum": 133359243, 2024-03-18T23:19:22.347 INFO:teuthology.orchestra.run.smithi139.stdout: "get_or_fail_fail": 0, 2024-03-18T23:19:22.347 INFO:teuthology.orchestra.run.smithi139.stdout: "get_or_fail_success": 852543, 2024-03-18T23:19:22.348 INFO:teuthology.orchestra.run.smithi139.stdout: "take": 0, 2024-03-18T23:19:22.348 INFO:teuthology.orchestra.run.smithi139.stdout: "take_sum": 0, 2024-03-18T23:19:22.348 INFO:teuthology.orchestra.run.smithi139.stdout: "put": 852543, 2024-03-18T23:19:22.348 INFO:teuthology.orchestra.run.smithi139.stdout: "put_sum": 133359243, 2024-03-18T23:19:22.348 INFO:teuthology.orchestra.run.smithi139.stdout: "wait": { 2024-03-18T23:19:22.348 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-03-18T23:19:22.348 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-03-18T23:19:22.348 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-03-18T23:19:22.348 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-03-18T23:19:22.349 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:22.349 INFO:teuthology.orchestra.run.smithi139.stdout: "throttle-objecter_bytes": { 2024-03-18T23:19:22.349 INFO:teuthology.orchestra.run.smithi139.stdout: "val": 0, 2024-03-18T23:19:22.349 INFO:teuthology.orchestra.run.smithi139.stdout: "max": 104857600, 2024-03-18T23:19:22.349 INFO:teuthology.orchestra.run.smithi139.stdout: "get_started": 0, 2024-03-18T23:19:22.349 INFO:teuthology.orchestra.run.smithi139.stdout: "get": 0, 2024-03-18T23:19:22.349 INFO:teuthology.orchestra.run.smithi139.stdout: "get_sum": 0, 2024-03-18T23:19:22.349 INFO:teuthology.orchestra.run.smithi139.stdout: "get_or_fail_fail": 0, 2024-03-18T23:19:22.349 INFO:teuthology.orchestra.run.smithi139.stdout: "get_or_fail_success": 0, 2024-03-18T23:19:22.349 INFO:teuthology.orchestra.run.smithi139.stdout: "take": 126095, 2024-03-18T23:19:22.350 INFO:teuthology.orchestra.run.smithi139.stdout: "take_sum": 709154614, 2024-03-18T23:19:22.350 INFO:teuthology.orchestra.run.smithi139.stdout: "put": 126095, 2024-03-18T23:19:22.350 INFO:teuthology.orchestra.run.smithi139.stdout: "put_sum": 709154614, 2024-03-18T23:19:22.350 INFO:teuthology.orchestra.run.smithi139.stdout: "wait": { 2024-03-18T23:19:22.350 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-03-18T23:19:22.350 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-03-18T23:19:22.350 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-03-18T23:19:22.350 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-03-18T23:19:22.351 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:22.351 INFO:teuthology.orchestra.run.smithi139.stdout: "throttle-objecter_ops": { 2024-03-18T23:19:22.351 INFO:teuthology.orchestra.run.smithi139.stdout: "val": 0, 2024-03-18T23:19:22.351 INFO:teuthology.orchestra.run.smithi139.stdout: "max": 1024, 2024-03-18T23:19:22.351 INFO:teuthology.orchestra.run.smithi139.stdout: "get_started": 0, 2024-03-18T23:19:22.351 INFO:teuthology.orchestra.run.smithi139.stdout: "get": 0, 2024-03-18T23:19:22.351 INFO:teuthology.orchestra.run.smithi139.stdout: "get_sum": 0, 2024-03-18T23:19:22.351 INFO:teuthology.orchestra.run.smithi139.stdout: "get_or_fail_fail": 0, 2024-03-18T23:19:22.352 INFO:teuthology.orchestra.run.smithi139.stdout: "get_or_fail_success": 0, 2024-03-18T23:19:22.352 INFO:teuthology.orchestra.run.smithi139.stdout: "take": 126095, 2024-03-18T23:19:22.352 INFO:teuthology.orchestra.run.smithi139.stdout: "take_sum": 126095, 2024-03-18T23:19:22.352 INFO:teuthology.orchestra.run.smithi139.stdout: "put": 126095, 2024-03-18T23:19:22.352 INFO:teuthology.orchestra.run.smithi139.stdout: "put_sum": 126095, 2024-03-18T23:19:22.352 INFO:teuthology.orchestra.run.smithi139.stdout: "wait": { 2024-03-18T23:19:22.352 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-03-18T23:19:22.352 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-03-18T23:19:22.352 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-03-18T23:19:22.353 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-03-18T23:19:22.353 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:22.353 INFO:teuthology.orchestra.run.smithi139.stdout: "throttle-write_buf_throttle": { 2024-03-18T23:19:22.353 INFO:teuthology.orchestra.run.smithi139.stdout: "val": 0, 2024-03-18T23:19:22.353 INFO:teuthology.orchestra.run.smithi139.stdout: "max": 3758096384, 2024-03-18T23:19:22.353 INFO:teuthology.orchestra.run.smithi139.stdout: "get_started": 0, 2024-03-18T23:19:22.353 INFO:teuthology.orchestra.run.smithi139.stdout: "get": 38215, 2024-03-18T23:19:22.353 INFO:teuthology.orchestra.run.smithi139.stdout: "get_sum": 3864107, 2024-03-18T23:19:22.354 INFO:teuthology.orchestra.run.smithi139.stdout: "get_or_fail_fail": 0, 2024-03-18T23:19:22.354 INFO:teuthology.orchestra.run.smithi139.stdout: "get_or_fail_success": 38215, 2024-03-18T23:19:22.354 INFO:teuthology.orchestra.run.smithi139.stdout: "take": 0, 2024-03-18T23:19:22.354 INFO:teuthology.orchestra.run.smithi139.stdout: "take_sum": 0, 2024-03-18T23:19:22.354 INFO:teuthology.orchestra.run.smithi139.stdout: "put": 1148, 2024-03-18T23:19:22.354 INFO:teuthology.orchestra.run.smithi139.stdout: "put_sum": 3864107, 2024-03-18T23:19:22.354 INFO:teuthology.orchestra.run.smithi139.stdout: "wait": { 2024-03-18T23:19:22.354 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-03-18T23:19:22.355 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-03-18T23:19:22.355 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-03-18T23:19:22.355 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-03-18T23:19:22.355 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:22.355 INFO:teuthology.orchestra.run.smithi139.stdout: "throttle-write_buf_throttle-0x556fba1ae500": { 2024-03-18T23:19:22.355 INFO:teuthology.orchestra.run.smithi139.stdout: "val": 71281, 2024-03-18T23:19:22.355 INFO:teuthology.orchestra.run.smithi139.stdout: "max": 3758096384, 2024-03-18T23:19:22.355 INFO:teuthology.orchestra.run.smithi139.stdout: "get_started": 0, 2024-03-18T23:19:22.356 INFO:teuthology.orchestra.run.smithi139.stdout: "get": 179813, 2024-03-18T23:19:22.356 INFO:teuthology.orchestra.run.smithi139.stdout: "get_sum": 674247157, 2024-03-18T23:19:22.356 INFO:teuthology.orchestra.run.smithi139.stdout: "get_or_fail_fail": 0, 2024-03-18T23:19:22.356 INFO:teuthology.orchestra.run.smithi139.stdout: "get_or_fail_success": 179813, 2024-03-18T23:19:22.356 INFO:teuthology.orchestra.run.smithi139.stdout: "take": 0, 2024-03-18T23:19:22.356 INFO:teuthology.orchestra.run.smithi139.stdout: "take_sum": 0, 2024-03-18T23:19:22.356 INFO:teuthology.orchestra.run.smithi139.stdout: "put": 62017, 2024-03-18T23:19:22.356 INFO:teuthology.orchestra.run.smithi139.stdout: "put_sum": 674175876, 2024-03-18T23:19:22.357 INFO:teuthology.orchestra.run.smithi139.stdout: "wait": { 2024-03-18T23:19:22.357 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-03-18T23:19:22.357 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-03-18T23:19:22.357 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-03-18T23:19:22.357 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-03-18T23:19:22.357 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-03-18T23:19:22.357 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-03-18T23:19:23.271 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:19:23 smithi139 ceph-mon[67282]: pgmap v348: 97 pgs: 97 active+clean; 71 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 1.3 MiB/s wr, 52 op/s 2024-03-18T23:19:23.306 INFO:tasks.check_counter:Daemon mds.b mds.exported=1 2024-03-18T23:19:23.306 INFO:tasks.check_counter:Daemon mds.b mds.imported=0 2024-03-18T23:19:23.306 INFO:tasks.check_counter:Daemon mds.b mds.root_rsnaps=6 2024-03-18T23:19:23.306 INFO:tasks.check_counter:Daemon mds.b mds_server.req_mksnap_latency.avgcount=11 2024-03-18T23:19:23.306 INFO:tasks.check_counter:Daemon mds.b mds_server.req_rmsnap_latency.avgcount=5 2024-03-18T23:19:23.306 DEBUG:tasks.check_counter:Getting stats from e 2024-03-18T23:19:23.306 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell --fsid c63591ea-e57a-11ee-95c9-87774f69a715 -- ceph daemon mds.e perf dump 2024-03-18T23:19:23.573 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/mon.b/config 2024-03-18T23:19:23.629 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:19:23 smithi049 ceph-mon[62991]: pgmap v348: 97 pgs: 97 active+clean; 71 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 1.3 MiB/s wr, 52 op/s 2024-03-18T23:19:23.710 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:19:23 smithi175 ceph-mon[67352]: pgmap v348: 97 pgs: 97 active+clean; 71 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 1.3 MiB/s wr, 52 op/s 2024-03-18T23:19:25.313 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:19:25 smithi139 ceph-mon[67282]: pgmap v349: 97 pgs: 97 active+clean; 69 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 1.2 MiB/s wr, 47 op/s 2024-03-18T23:19:25.365 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-03-18T23:19:25.366 INFO:teuthology.orchestra.run.smithi139.stdout: "AsyncMessenger::Worker-0": { 2024-03-18T23:19:25.366 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_recv_messages": 22, 2024-03-18T23:19:25.366 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_messages": 158, 2024-03-18T23:19:25.366 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_recv_bytes": 5793, 2024-03-18T23:19:25.366 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_bytes": 338086, 2024-03-18T23:19:25.366 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_created_connections": 9, 2024-03-18T23:19:25.367 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_active_connections": 2, 2024-03-18T23:19:25.367 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_total_time": 0.008761042, 2024-03-18T23:19:25.367 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_send_time": 0.004328329, 2024-03-18T23:19:25.367 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_recv_time": 0.001995978, 2024-03-18T23:19:25.367 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_fast_dispatch_time": 0.000384279, 2024-03-18T23:19:25.367 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_messages_queue_lat": { 2024-03-18T23:19:25.367 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 156, 2024-03-18T23:19:25.367 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.021716722, 2024-03-18T23:19:25.368 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000139209 2024-03-18T23:19:25.368 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:25.368 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_handle_ack_lat": { 2024-03-18T23:19:25.368 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-03-18T23:19:25.368 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-03-18T23:19:25.368 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-03-18T23:19:25.368 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:25.368 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_recv_encrypted_bytes": 160, 2024-03-18T23:19:25.369 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_encrypted_bytes": 327680 2024-03-18T23:19:25.369 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:25.369 INFO:teuthology.orchestra.run.smithi139.stdout: "AsyncMessenger::Worker-1": { 2024-03-18T23:19:25.369 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_recv_messages": 180814, 2024-03-18T23:19:25.369 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_messages": 93916, 2024-03-18T23:19:25.369 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_recv_bytes": 97201019, 2024-03-18T23:19:25.369 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_bytes": 15187851, 2024-03-18T23:19:25.370 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_created_connections": 8, 2024-03-18T23:19:25.370 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_active_connections": 5, 2024-03-18T23:19:25.370 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_total_time": 8.254307287, 2024-03-18T23:19:25.370 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_send_time": 3.711093974, 2024-03-18T23:19:25.370 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_recv_time": 6.240182947, 2024-03-18T23:19:25.370 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_fast_dispatch_time": 0.003089144, 2024-03-18T23:19:25.370 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_messages_queue_lat": { 2024-03-18T23:19:25.370 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 93914, 2024-03-18T23:19:25.371 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 1.297217677, 2024-03-18T23:19:25.371 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000013812 2024-03-18T23:19:25.371 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:25.371 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_handle_ack_lat": { 2024-03-18T23:19:25.371 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 274474, 2024-03-18T23:19:25.371 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.036188678, 2024-03-18T23:19:25.371 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000131 2024-03-18T23:19:25.371 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:25.372 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_recv_encrypted_bytes": 2182272, 2024-03-18T23:19:25.372 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_encrypted_bytes": 1130272 2024-03-18T23:19:25.372 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:25.372 INFO:teuthology.orchestra.run.smithi139.stdout: "AsyncMessenger::Worker-2": { 2024-03-18T23:19:25.372 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_recv_messages": 313, 2024-03-18T23:19:25.372 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_messages": 294, 2024-03-18T23:19:25.373 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_recv_bytes": 1528740, 2024-03-18T23:19:25.373 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_bytes": 202343, 2024-03-18T23:19:25.373 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_created_connections": 11, 2024-03-18T23:19:25.373 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_active_connections": 3, 2024-03-18T23:19:25.373 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_total_time": 0.040915621, 2024-03-18T23:19:25.373 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_send_time": 0.009441958, 2024-03-18T23:19:25.373 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_recv_time": 0.039987713, 2024-03-18T23:19:25.373 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_fast_dispatch_time": 0.004287069, 2024-03-18T23:19:25.373 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_messages_queue_lat": { 2024-03-18T23:19:25.374 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 293, 2024-03-18T23:19:25.374 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.012436283, 2024-03-18T23:19:25.374 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000042444 2024-03-18T23:19:25.374 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:25.374 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_handle_ack_lat": { 2024-03-18T23:19:25.374 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 19, 2024-03-18T23:19:25.374 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000002251, 2024-03-18T23:19:25.375 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000118 2024-03-18T23:19:25.375 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:25.375 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_recv_encrypted_bytes": 1526560, 2024-03-18T23:19:25.375 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_encrypted_bytes": 146816 2024-03-18T23:19:25.375 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:25.375 INFO:teuthology.orchestra.run.smithi139.stdout: "cct": { 2024-03-18T23:19:25.376 INFO:teuthology.orchestra.run.smithi139.stdout: "total_workers": 1, 2024-03-18T23:19:25.376 INFO:teuthology.orchestra.run.smithi139.stdout: "unhealthy_workers": 0 2024-03-18T23:19:25.376 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:25.376 INFO:teuthology.orchestra.run.smithi139.stdout: "finisher-MDSRank": { 2024-03-18T23:19:25.376 INFO:teuthology.orchestra.run.smithi139.stdout: "queue_len": 0, 2024-03-18T23:19:25.376 INFO:teuthology.orchestra.run.smithi139.stdout: "complete_latency": { 2024-03-18T23:19:25.376 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 57, 2024-03-18T23:19:25.376 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.001278469, 2024-03-18T23:19:25.376 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000022429 2024-03-18T23:19:25.377 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-03-18T23:19:25.377 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:25.377 INFO:teuthology.orchestra.run.smithi139.stdout: "finisher-PurgeQueue": { 2024-03-18T23:19:25.377 INFO:teuthology.orchestra.run.smithi139.stdout: "queue_len": 0, 2024-03-18T23:19:25.377 INFO:teuthology.orchestra.run.smithi139.stdout: "complete_latency": { 2024-03-18T23:19:25.377 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 2, 2024-03-18T23:19:25.377 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000004551, 2024-03-18T23:19:25.377 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000002275 2024-03-18T23:19:25.378 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-03-18T23:19:25.378 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:25.378 INFO:teuthology.orchestra.run.smithi139.stdout: "mds": { 2024-03-18T23:19:25.378 INFO:teuthology.orchestra.run.smithi139.stdout: "request": 6897, 2024-03-18T23:19:25.379 INFO:teuthology.orchestra.run.smithi139.stdout: "reply": 0, 2024-03-18T23:19:25.379 INFO:teuthology.orchestra.run.smithi139.stdout: "reply_latency": { 2024-03-18T23:19:25.379 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-03-18T23:19:25.379 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-03-18T23:19:25.379 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-03-18T23:19:25.379 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:25.379 INFO:teuthology.orchestra.run.smithi139.stdout: "slow_reply": 0, 2024-03-18T23:19:25.379 INFO:teuthology.orchestra.run.smithi139.stdout: "forward": 6897, 2024-03-18T23:19:25.380 INFO:teuthology.orchestra.run.smithi139.stdout: "dir_fetch_complete": 0, 2024-03-18T23:19:25.380 INFO:teuthology.orchestra.run.smithi139.stdout: "dir_fetch_keys": 0, 2024-03-18T23:19:25.380 INFO:teuthology.orchestra.run.smithi139.stdout: "dir_commit": 11, 2024-03-18T23:19:25.380 INFO:teuthology.orchestra.run.smithi139.stdout: "dir_split": 0, 2024-03-18T23:19:25.380 INFO:teuthology.orchestra.run.smithi139.stdout: "dir_merge": 0, 2024-03-18T23:19:25.380 INFO:teuthology.orchestra.run.smithi139.stdout: "inodes": 22, 2024-03-18T23:19:25.380 INFO:teuthology.orchestra.run.smithi139.stdout: "inodes_top": 2, 2024-03-18T23:19:25.380 INFO:teuthology.orchestra.run.smithi139.stdout: "inodes_bottom": 20, 2024-03-18T23:19:25.380 INFO:teuthology.orchestra.run.smithi139.stdout: "inodes_pin_tail": 0, 2024-03-18T23:19:25.380 INFO:teuthology.orchestra.run.smithi139.stdout: "inodes_pinned": 19, 2024-03-18T23:19:25.381 INFO:teuthology.orchestra.run.smithi139.stdout: "inodes_expired": 0, 2024-03-18T23:19:25.381 INFO:teuthology.orchestra.run.smithi139.stdout: "inodes_with_caps": 0, 2024-03-18T23:19:25.381 INFO:teuthology.orchestra.run.smithi139.stdout: "caps": 0, 2024-03-18T23:19:25.381 INFO:teuthology.orchestra.run.smithi139.stdout: "subtrees": 2, 2024-03-18T23:19:25.381 INFO:teuthology.orchestra.run.smithi139.stdout: "traverse": 6902, 2024-03-18T23:19:25.381 INFO:teuthology.orchestra.run.smithi139.stdout: "traverse_hit": 3, 2024-03-18T23:19:25.381 INFO:teuthology.orchestra.run.smithi139.stdout: "traverse_forward": 6896, 2024-03-18T23:19:25.381 INFO:teuthology.orchestra.run.smithi139.stdout: "traverse_discover": 3, 2024-03-18T23:19:25.382 INFO:teuthology.orchestra.run.smithi139.stdout: "traverse_dir_fetch": 0, 2024-03-18T23:19:25.382 INFO:teuthology.orchestra.run.smithi139.stdout: "traverse_remote_ino": 0, 2024-03-18T23:19:25.382 INFO:teuthology.orchestra.run.smithi139.stdout: "traverse_lock": 0, 2024-03-18T23:19:25.382 INFO:teuthology.orchestra.run.smithi139.stdout: "load_cent": 0, 2024-03-18T23:19:25.382 INFO:teuthology.orchestra.run.smithi139.stdout: "q": 0, 2024-03-18T23:19:25.382 INFO:teuthology.orchestra.run.smithi139.stdout: "exported": 0, 2024-03-18T23:19:25.382 INFO:teuthology.orchestra.run.smithi139.stdout: "exported_inodes": 0, 2024-03-18T23:19:25.383 INFO:teuthology.orchestra.run.smithi139.stdout: "imported": 0, 2024-03-18T23:19:25.383 INFO:teuthology.orchestra.run.smithi139.stdout: "imported_inodes": 0, 2024-03-18T23:19:25.383 INFO:teuthology.orchestra.run.smithi139.stdout: "openino_dir_fetch": 0, 2024-03-18T23:19:25.383 INFO:teuthology.orchestra.run.smithi139.stdout: "openino_backtrace_fetch": 0, 2024-03-18T23:19:25.384 INFO:teuthology.orchestra.run.smithi139.stdout: "openino_peer_discover": 1, 2024-03-18T23:19:25.384 INFO:teuthology.orchestra.run.smithi139.stdout: "root_rfiles": 69, 2024-03-18T23:19:25.384 INFO:teuthology.orchestra.run.smithi139.stdout: "root_rbytes": 21153440, 2024-03-18T23:19:25.384 INFO:teuthology.orchestra.run.smithi139.stdout: "root_rsnaps": 6, 2024-03-18T23:19:25.384 INFO:teuthology.orchestra.run.smithi139.stdout: "scrub_backtrace_fetch": 0, 2024-03-18T23:19:25.384 INFO:teuthology.orchestra.run.smithi139.stdout: "scrub_set_tag": 0, 2024-03-18T23:19:25.384 INFO:teuthology.orchestra.run.smithi139.stdout: "scrub_backtrace_repaired": 0, 2024-03-18T23:19:25.384 INFO:teuthology.orchestra.run.smithi139.stdout: "scrub_inotable_repaired": 0, 2024-03-18T23:19:25.385 INFO:teuthology.orchestra.run.smithi139.stdout: "scrub_dir_inodes": 0, 2024-03-18T23:19:25.385 INFO:teuthology.orchestra.run.smithi139.stdout: "scrub_dir_base_inodes": 0, 2024-03-18T23:19:25.385 INFO:teuthology.orchestra.run.smithi139.stdout: "scrub_dirfrag_rstats": 0, 2024-03-18T23:19:25.385 INFO:teuthology.orchestra.run.smithi139.stdout: "scrub_file_inodes": 0, 2024-03-18T23:19:25.385 INFO:teuthology.orchestra.run.smithi139.stdout: "handle_inode_file_caps": 0, 2024-03-18T23:19:25.385 INFO:teuthology.orchestra.run.smithi139.stdout: "ceph_cap_op_revoke": 0, 2024-03-18T23:19:25.385 INFO:teuthology.orchestra.run.smithi139.stdout: "ceph_cap_op_grant": 0, 2024-03-18T23:19:25.386 INFO:teuthology.orchestra.run.smithi139.stdout: "ceph_cap_op_trunc": 0, 2024-03-18T23:19:25.386 INFO:teuthology.orchestra.run.smithi139.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-18T23:19:25.386 INFO:teuthology.orchestra.run.smithi139.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-18T23:19:25.386 INFO:teuthology.orchestra.run.smithi139.stdout: "handle_client_caps": 0, 2024-03-18T23:19:25.386 INFO:teuthology.orchestra.run.smithi139.stdout: "handle_client_caps_dirty": 0, 2024-03-18T23:19:25.386 INFO:teuthology.orchestra.run.smithi139.stdout: "handle_client_cap_release": 0, 2024-03-18T23:19:25.386 INFO:teuthology.orchestra.run.smithi139.stdout: "process_request_cap_release": 0 2024-03-18T23:19:25.386 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:25.387 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_cache": { 2024-03-18T23:19:25.387 INFO:teuthology.orchestra.run.smithi139.stdout: "dir_update": 0, 2024-03-18T23:19:25.387 INFO:teuthology.orchestra.run.smithi139.stdout: "dir_update_receipt": 9, 2024-03-18T23:19:25.387 INFO:teuthology.orchestra.run.smithi139.stdout: "dir_try_discover": 6, 2024-03-18T23:19:25.387 INFO:teuthology.orchestra.run.smithi139.stdout: "dir_send_discover": 7, 2024-03-18T23:19:25.387 INFO:teuthology.orchestra.run.smithi139.stdout: "dir_handle_discover": 0, 2024-03-18T23:19:25.387 INFO:teuthology.orchestra.run.smithi139.stdout: "num_strays": 0, 2024-03-18T23:19:25.387 INFO:teuthology.orchestra.run.smithi139.stdout: "num_strays_delayed": 0, 2024-03-18T23:19:25.388 INFO:teuthology.orchestra.run.smithi139.stdout: "num_strays_enqueuing": 0, 2024-03-18T23:19:25.388 INFO:teuthology.orchestra.run.smithi139.stdout: "strays_created": 0, 2024-03-18T23:19:25.388 INFO:teuthology.orchestra.run.smithi139.stdout: "strays_enqueued": 0, 2024-03-18T23:19:25.388 INFO:teuthology.orchestra.run.smithi139.stdout: "strays_reintegrated": 0, 2024-03-18T23:19:25.388 INFO:teuthology.orchestra.run.smithi139.stdout: "strays_migrated": 0, 2024-03-18T23:19:25.388 INFO:teuthology.orchestra.run.smithi139.stdout: "num_recovering_processing": 0, 2024-03-18T23:19:25.388 INFO:teuthology.orchestra.run.smithi139.stdout: "num_recovering_enqueued": 0, 2024-03-18T23:19:25.388 INFO:teuthology.orchestra.run.smithi139.stdout: "num_recovering_prioritized": 0, 2024-03-18T23:19:25.389 INFO:teuthology.orchestra.run.smithi139.stdout: "recovery_started": 0, 2024-03-18T23:19:25.389 INFO:teuthology.orchestra.run.smithi139.stdout: "recovery_completed": 0, 2024-03-18T23:19:25.389 INFO:teuthology.orchestra.run.smithi139.stdout: "ireq_quiesce_path": 0, 2024-03-18T23:19:25.389 INFO:teuthology.orchestra.run.smithi139.stdout: "ireq_quiesce_inode": 0, 2024-03-18T23:19:25.389 INFO:teuthology.orchestra.run.smithi139.stdout: "ireq_enqueue_scrub": 0, 2024-03-18T23:19:25.389 INFO:teuthology.orchestra.run.smithi139.stdout: "ireq_exportdir": 0, 2024-03-18T23:19:25.390 INFO:teuthology.orchestra.run.smithi139.stdout: "ireq_flush": 0, 2024-03-18T23:19:25.390 INFO:teuthology.orchestra.run.smithi139.stdout: "ireq_fragmentdir": 0, 2024-03-18T23:19:25.390 INFO:teuthology.orchestra.run.smithi139.stdout: "ireq_fragstats": 0, 2024-03-18T23:19:25.390 INFO:teuthology.orchestra.run.smithi139.stdout: "ireq_inodestats": 0 2024-03-18T23:19:25.390 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:25.390 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_log": { 2024-03-18T23:19:25.390 INFO:teuthology.orchestra.run.smithi139.stdout: "evlrg": 0, 2024-03-18T23:19:25.390 INFO:teuthology.orchestra.run.smithi139.stdout: "evadd": 7, 2024-03-18T23:19:25.391 INFO:teuthology.orchestra.run.smithi139.stdout: "evex": 0, 2024-03-18T23:19:25.391 INFO:teuthology.orchestra.run.smithi139.stdout: "evtrm": 0, 2024-03-18T23:19:25.391 INFO:teuthology.orchestra.run.smithi139.stdout: "ev": 7, 2024-03-18T23:19:25.391 INFO:teuthology.orchestra.run.smithi139.stdout: "evexg": 0, 2024-03-18T23:19:25.391 INFO:teuthology.orchestra.run.smithi139.stdout: "evexd": 0, 2024-03-18T23:19:25.391 INFO:teuthology.orchestra.run.smithi139.stdout: "segadd": 2, 2024-03-18T23:19:25.391 INFO:teuthology.orchestra.run.smithi139.stdout: "segex": 0, 2024-03-18T23:19:25.391 INFO:teuthology.orchestra.run.smithi139.stdout: "segtrm": 0, 2024-03-18T23:19:25.392 INFO:teuthology.orchestra.run.smithi139.stdout: "seg": 2, 2024-03-18T23:19:25.392 INFO:teuthology.orchestra.run.smithi139.stdout: "segmjr": 2, 2024-03-18T23:19:25.392 INFO:teuthology.orchestra.run.smithi139.stdout: "segexg": 0, 2024-03-18T23:19:25.392 INFO:teuthology.orchestra.run.smithi139.stdout: "segexd": 0, 2024-03-18T23:19:25.392 INFO:teuthology.orchestra.run.smithi139.stdout: "expos": 4194304, 2024-03-18T23:19:25.392 INFO:teuthology.orchestra.run.smithi139.stdout: "wrpos": 4196264, 2024-03-18T23:19:25.392 INFO:teuthology.orchestra.run.smithi139.stdout: "rdpos": 0, 2024-03-18T23:19:25.392 INFO:teuthology.orchestra.run.smithi139.stdout: "jlat": { 2024-03-18T23:19:25.392 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 6, 2024-03-18T23:19:25.393 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.010045197, 2024-03-18T23:19:25.393 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.001674199 2024-03-18T23:19:25.393 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:25.393 INFO:teuthology.orchestra.run.smithi139.stdout: "replayed": 0 2024-03-18T23:19:25.393 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:25.393 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_mem": { 2024-03-18T23:19:25.393 INFO:teuthology.orchestra.run.smithi139.stdout: "ino": 25, 2024-03-18T23:19:25.393 INFO:teuthology.orchestra.run.smithi139.stdout: "ino+": 25, 2024-03-18T23:19:25.394 INFO:teuthology.orchestra.run.smithi139.stdout: "ino-": 0, 2024-03-18T23:19:25.394 INFO:teuthology.orchestra.run.smithi139.stdout: "dir": 38, 2024-03-18T23:19:25.394 INFO:teuthology.orchestra.run.smithi139.stdout: "dir+": 41, 2024-03-18T23:19:25.394 INFO:teuthology.orchestra.run.smithi139.stdout: "dir-": 3, 2024-03-18T23:19:25.394 INFO:teuthology.orchestra.run.smithi139.stdout: "dn": 22, 2024-03-18T23:19:25.394 INFO:teuthology.orchestra.run.smithi139.stdout: "dn+": 22, 2024-03-18T23:19:25.395 INFO:teuthology.orchestra.run.smithi139.stdout: "dn-": 0, 2024-03-18T23:19:25.395 INFO:teuthology.orchestra.run.smithi139.stdout: "cap": 0, 2024-03-18T23:19:25.395 INFO:teuthology.orchestra.run.smithi139.stdout: "cap+": 0, 2024-03-18T23:19:25.395 INFO:teuthology.orchestra.run.smithi139.stdout: "cap-": 0, 2024-03-18T23:19:25.395 INFO:teuthology.orchestra.run.smithi139.stdout: "rss": 54388, 2024-03-18T23:19:25.395 INFO:teuthology.orchestra.run.smithi139.stdout: "heap": 223612 2024-03-18T23:19:25.395 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:25.395 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_server": { 2024-03-18T23:19:25.395 INFO:teuthology.orchestra.run.smithi139.stdout: "dispatch_client_request": 6897, 2024-03-18T23:19:25.395 INFO:teuthology.orchestra.run.smithi139.stdout: "dispatch_server_request": 0, 2024-03-18T23:19:25.396 INFO:teuthology.orchestra.run.smithi139.stdout: "handle_client_request": 6897, 2024-03-18T23:19:25.396 INFO:teuthology.orchestra.run.smithi139.stdout: "handle_client_session": 17, 2024-03-18T23:19:25.396 INFO:teuthology.orchestra.run.smithi139.stdout: "handle_peer_request": 0, 2024-03-18T23:19:25.396 INFO:teuthology.orchestra.run.smithi139.stdout: "req_create_latency": { 2024-03-18T23:19:25.396 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-03-18T23:19:25.396 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-03-18T23:19:25.397 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-03-18T23:19:25.397 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:25.397 INFO:teuthology.orchestra.run.smithi139.stdout: "req_getattr_latency": { 2024-03-18T23:19:25.397 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-03-18T23:19:25.397 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-03-18T23:19:25.397 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-03-18T23:19:25.397 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:25.397 INFO:teuthology.orchestra.run.smithi139.stdout: "req_getfilelock_latency": { 2024-03-18T23:19:25.398 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-03-18T23:19:25.398 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-03-18T23:19:25.398 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-03-18T23:19:25.398 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:25.398 INFO:teuthology.orchestra.run.smithi139.stdout: "req_link_latency": { 2024-03-18T23:19:25.398 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-03-18T23:19:25.398 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-03-18T23:19:25.398 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-03-18T23:19:25.399 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:25.399 INFO:teuthology.orchestra.run.smithi139.stdout: "req_lookup_latency": { 2024-03-18T23:19:25.399 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-03-18T23:19:25.399 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-03-18T23:19:25.399 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-03-18T23:19:25.399 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:25.400 INFO:teuthology.orchestra.run.smithi139.stdout: "req_lookuphash_latency": { 2024-03-18T23:19:25.400 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-03-18T23:19:25.400 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-03-18T23:19:25.400 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-03-18T23:19:25.400 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:25.400 INFO:teuthology.orchestra.run.smithi139.stdout: "req_lookupino_latency": { 2024-03-18T23:19:25.400 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-03-18T23:19:25.400 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-03-18T23:19:25.400 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-03-18T23:19:25.400 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:25.401 INFO:teuthology.orchestra.run.smithi139.stdout: "req_lookupname_latency": { 2024-03-18T23:19:25.401 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-03-18T23:19:25.401 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-03-18T23:19:25.401 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-03-18T23:19:25.401 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:25.401 INFO:teuthology.orchestra.run.smithi139.stdout: "req_lookupparent_latency": { 2024-03-18T23:19:25.401 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-03-18T23:19:25.402 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-03-18T23:19:25.402 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-03-18T23:19:25.402 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:25.402 INFO:teuthology.orchestra.run.smithi139.stdout: "req_lookupsnap_latency": { 2024-03-18T23:19:25.402 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-03-18T23:19:25.402 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-03-18T23:19:25.402 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-03-18T23:19:25.402 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:25.403 INFO:teuthology.orchestra.run.smithi139.stdout: "req_lssnap_latency": { 2024-03-18T23:19:25.403 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-03-18T23:19:25.403 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-03-18T23:19:25.403 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-03-18T23:19:25.403 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:25.403 INFO:teuthology.orchestra.run.smithi139.stdout: "req_mkdir_latency": { 2024-03-18T23:19:25.404 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-03-18T23:19:25.404 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-03-18T23:19:25.404 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-03-18T23:19:25.404 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:25.404 INFO:teuthology.orchestra.run.smithi139.stdout: "req_mknod_latency": { 2024-03-18T23:19:25.404 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-03-18T23:19:25.405 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-03-18T23:19:25.405 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-03-18T23:19:25.405 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:25.405 INFO:teuthology.orchestra.run.smithi139.stdout: "req_mksnap_latency": { 2024-03-18T23:19:25.405 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-03-18T23:19:25.405 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-03-18T23:19:25.405 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-03-18T23:19:25.405 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:25.406 INFO:teuthology.orchestra.run.smithi139.stdout: "req_open_latency": { 2024-03-18T23:19:25.406 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-03-18T23:19:25.406 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-03-18T23:19:25.406 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-03-18T23:19:25.406 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:25.406 INFO:teuthology.orchestra.run.smithi139.stdout: "req_readdir_latency": { 2024-03-18T23:19:25.407 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-03-18T23:19:25.407 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-03-18T23:19:25.407 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-03-18T23:19:25.407 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:25.407 INFO:teuthology.orchestra.run.smithi139.stdout: "req_rename_latency": { 2024-03-18T23:19:25.407 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-03-18T23:19:25.407 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-03-18T23:19:25.407 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-03-18T23:19:25.408 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:25.408 INFO:teuthology.orchestra.run.smithi139.stdout: "req_renamesnap_latency": { 2024-03-18T23:19:25.408 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-03-18T23:19:25.408 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-03-18T23:19:25.408 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-03-18T23:19:25.408 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:25.408 INFO:teuthology.orchestra.run.smithi139.stdout: "req_snapdiff_latency": { 2024-03-18T23:19:25.408 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-03-18T23:19:25.408 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-03-18T23:19:25.409 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-03-18T23:19:25.409 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:25.409 INFO:teuthology.orchestra.run.smithi139.stdout: "req_rmdir_latency": { 2024-03-18T23:19:25.409 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-03-18T23:19:25.409 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-03-18T23:19:25.409 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-03-18T23:19:25.409 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:25.410 INFO:teuthology.orchestra.run.smithi139.stdout: "req_rmsnap_latency": { 2024-03-18T23:19:25.410 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-03-18T23:19:25.410 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-03-18T23:19:25.410 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-03-18T23:19:25.411 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:25.411 INFO:teuthology.orchestra.run.smithi139.stdout: "req_rmxattr_latency": { 2024-03-18T23:19:25.411 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-03-18T23:19:25.411 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-03-18T23:19:25.411 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-03-18T23:19:25.411 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:25.411 INFO:teuthology.orchestra.run.smithi139.stdout: "req_setattr_latency": { 2024-03-18T23:19:25.411 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-03-18T23:19:25.411 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-03-18T23:19:25.412 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-03-18T23:19:25.412 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:25.412 INFO:teuthology.orchestra.run.smithi139.stdout: "req_setdirlayout_latency": { 2024-03-18T23:19:25.412 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-03-18T23:19:25.412 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-03-18T23:19:25.412 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-03-18T23:19:25.412 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:25.412 INFO:teuthology.orchestra.run.smithi139.stdout: "req_setfilelock_latency": { 2024-03-18T23:19:25.413 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-03-18T23:19:25.413 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-03-18T23:19:25.413 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-03-18T23:19:25.413 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:25.413 INFO:teuthology.orchestra.run.smithi139.stdout: "req_setlayout_latency": { 2024-03-18T23:19:25.413 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-03-18T23:19:25.413 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-03-18T23:19:25.413 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-03-18T23:19:25.413 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:25.414 INFO:teuthology.orchestra.run.smithi139.stdout: "req_setxattr_latency": { 2024-03-18T23:19:25.414 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-03-18T23:19:25.414 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-03-18T23:19:25.414 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-03-18T23:19:25.414 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:25.414 INFO:teuthology.orchestra.run.smithi139.stdout: "req_symlink_latency": { 2024-03-18T23:19:25.415 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-03-18T23:19:25.415 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-03-18T23:19:25.415 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-03-18T23:19:25.415 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:25.415 INFO:teuthology.orchestra.run.smithi139.stdout: "req_unlink_latency": { 2024-03-18T23:19:25.415 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-03-18T23:19:25.416 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-03-18T23:19:25.416 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-03-18T23:19:25.416 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:25.416 INFO:teuthology.orchestra.run.smithi139.stdout: "cap_revoke_eviction": 0, 2024-03-18T23:19:25.416 INFO:teuthology.orchestra.run.smithi139.stdout: "cap_acquisition_throttle": 0, 2024-03-18T23:19:25.416 INFO:teuthology.orchestra.run.smithi139.stdout: "req_getvxattr_latency": { 2024-03-18T23:19:25.416 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-03-18T23:19:25.416 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-03-18T23:19:25.417 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-03-18T23:19:25.417 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-03-18T23:19:25.417 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:25.417 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_sessions": { 2024-03-18T23:19:25.417 INFO:teuthology.orchestra.run.smithi139.stdout: "session_count": 1, 2024-03-18T23:19:25.417 INFO:teuthology.orchestra.run.smithi139.stdout: "session_add": 3, 2024-03-18T23:19:25.417 INFO:teuthology.orchestra.run.smithi139.stdout: "session_remove": 2, 2024-03-18T23:19:25.417 INFO:teuthology.orchestra.run.smithi139.stdout: "sessions_open": 1, 2024-03-18T23:19:25.417 INFO:teuthology.orchestra.run.smithi139.stdout: "sessions_stale": 0, 2024-03-18T23:19:25.418 INFO:teuthology.orchestra.run.smithi139.stdout: "total_load": 0, 2024-03-18T23:19:25.418 INFO:teuthology.orchestra.run.smithi139.stdout: "average_load": 0, 2024-03-18T23:19:25.418 INFO:teuthology.orchestra.run.smithi139.stdout: "avg_session_uptime": 663, 2024-03-18T23:19:25.418 INFO:teuthology.orchestra.run.smithi139.stdout: "mdthresh_evicted": 0 2024-03-18T23:19:25.418 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:25.418 INFO:teuthology.orchestra.run.smithi139.stdout: "mempool": { 2024-03-18T23:19:25.418 INFO:teuthology.orchestra.run.smithi139.stdout: "bloom_filter_bytes": 0, 2024-03-18T23:19:25.418 INFO:teuthology.orchestra.run.smithi139.stdout: "bloom_filter_items": 0, 2024-03-18T23:19:25.419 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_alloc_bytes": 0, 2024-03-18T23:19:25.419 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_alloc_items": 0, 2024-03-18T23:19:25.419 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_cache_data_bytes": 0, 2024-03-18T23:19:25.419 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_cache_data_items": 0, 2024-03-18T23:19:25.419 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-18T23:19:25.419 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_cache_onode_items": 0, 2024-03-18T23:19:25.419 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-18T23:19:25.419 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_cache_meta_items": 0, 2024-03-18T23:19:25.419 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_cache_other_bytes": 0, 2024-03-18T23:19:25.419 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_cache_other_items": 0, 2024-03-18T23:19:25.420 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-18T23:19:25.420 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_cache_buffer_items": 0, 2024-03-18T23:19:25.420 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_extent_bytes": 0, 2024-03-18T23:19:25.420 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_extent_items": 0, 2024-03-18T23:19:25.420 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_blob_bytes": 0, 2024-03-18T23:19:25.420 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_blob_items": 0, 2024-03-18T23:19:25.420 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-18T23:19:25.421 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_shared_blob_items": 0, 2024-03-18T23:19:25.421 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-18T23:19:25.421 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_inline_bl_items": 0, 2024-03-18T23:19:25.421 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_fsck_bytes": 0, 2024-03-18T23:19:25.421 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_fsck_items": 0, 2024-03-18T23:19:25.421 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_txc_bytes": 0, 2024-03-18T23:19:25.421 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_txc_items": 0, 2024-03-18T23:19:25.421 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-18T23:19:25.421 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_writing_deferred_items": 0, 2024-03-18T23:19:25.422 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_writing_bytes": 0, 2024-03-18T23:19:25.422 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_writing_items": 0, 2024-03-18T23:19:25.422 INFO:teuthology.orchestra.run.smithi139.stdout: "bluefs_bytes": 0, 2024-03-18T23:19:25.422 INFO:teuthology.orchestra.run.smithi139.stdout: "bluefs_items": 0, 2024-03-18T23:19:25.422 INFO:teuthology.orchestra.run.smithi139.stdout: "bluefs_file_reader_bytes": 0, 2024-03-18T23:19:25.423 INFO:teuthology.orchestra.run.smithi139.stdout: "bluefs_file_reader_items": 0, 2024-03-18T23:19:25.423 INFO:teuthology.orchestra.run.smithi139.stdout: "bluefs_file_writer_bytes": 0, 2024-03-18T23:19:25.423 INFO:teuthology.orchestra.run.smithi139.stdout: "bluefs_file_writer_items": 0, 2024-03-18T23:19:25.423 INFO:teuthology.orchestra.run.smithi139.stdout: "buffer_anon_bytes": 242258, 2024-03-18T23:19:25.423 INFO:teuthology.orchestra.run.smithi139.stdout: "buffer_anon_items": 107, 2024-03-18T23:19:25.423 INFO:teuthology.orchestra.run.smithi139.stdout: "buffer_meta_bytes": 0, 2024-03-18T23:19:25.423 INFO:teuthology.orchestra.run.smithi139.stdout: "buffer_meta_items": 0, 2024-03-18T23:19:25.423 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_bytes": 0, 2024-03-18T23:19:25.423 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_items": 0, 2024-03-18T23:19:25.424 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_mapbl_bytes": 0, 2024-03-18T23:19:25.424 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_mapbl_items": 0, 2024-03-18T23:19:25.424 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_pglog_bytes": 0, 2024-03-18T23:19:25.424 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_pglog_items": 0, 2024-03-18T23:19:25.424 INFO:teuthology.orchestra.run.smithi139.stdout: "osdmap_bytes": 8256, 2024-03-18T23:19:25.424 INFO:teuthology.orchestra.run.smithi139.stdout: "osdmap_items": 188, 2024-03-18T23:19:25.424 INFO:teuthology.orchestra.run.smithi139.stdout: "osdmap_mapping_bytes": 0, 2024-03-18T23:19:25.424 INFO:teuthology.orchestra.run.smithi139.stdout: "osdmap_mapping_items": 0, 2024-03-18T23:19:25.425 INFO:teuthology.orchestra.run.smithi139.stdout: "pgmap_bytes": 0, 2024-03-18T23:19:25.425 INFO:teuthology.orchestra.run.smithi139.stdout: "pgmap_items": 0, 2024-03-18T23:19:25.425 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_co_bytes": 133308, 2024-03-18T23:19:25.425 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_co_items": 710, 2024-03-18T23:19:25.425 INFO:teuthology.orchestra.run.smithi139.stdout: "unittest_1_bytes": 0, 2024-03-18T23:19:25.425 INFO:teuthology.orchestra.run.smithi139.stdout: "unittest_1_items": 0, 2024-03-18T23:19:25.426 INFO:teuthology.orchestra.run.smithi139.stdout: "unittest_2_bytes": 0, 2024-03-18T23:19:25.426 INFO:teuthology.orchestra.run.smithi139.stdout: "unittest_2_items": 0 2024-03-18T23:19:25.426 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:25.426 INFO:teuthology.orchestra.run.smithi139.stdout: "objecter": { 2024-03-18T23:19:25.426 INFO:teuthology.orchestra.run.smithi139.stdout: "op_active": 0, 2024-03-18T23:19:25.426 INFO:teuthology.orchestra.run.smithi139.stdout: "op_laggy": 0, 2024-03-18T23:19:25.426 INFO:teuthology.orchestra.run.smithi139.stdout: "op_send": 43, 2024-03-18T23:19:25.427 INFO:teuthology.orchestra.run.smithi139.stdout: "op_send_bytes": 12352, 2024-03-18T23:19:25.427 INFO:teuthology.orchestra.run.smithi139.stdout: "op_resend": 33, 2024-03-18T23:19:25.427 INFO:teuthology.orchestra.run.smithi139.stdout: "op_reply": 43, 2024-03-18T23:19:25.427 INFO:teuthology.orchestra.run.smithi139.stdout: "op_latency": { 2024-03-18T23:19:25.427 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 43, 2024-03-18T23:19:25.427 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.212995300, 2024-03-18T23:19:25.427 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.004953379 2024-03-18T23:19:25.428 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:25.428 INFO:teuthology.orchestra.run.smithi139.stdout: "op_inflight": 0, 2024-03-18T23:19:25.428 INFO:teuthology.orchestra.run.smithi139.stdout: "oplen_avg": { 2024-03-18T23:19:25.428 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 43, 2024-03-18T23:19:25.428 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 65 2024-03-18T23:19:25.428 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:25.428 INFO:teuthology.orchestra.run.smithi139.stdout: "op": 43, 2024-03-18T23:19:25.428 INFO:teuthology.orchestra.run.smithi139.stdout: "op_r": 0, 2024-03-18T23:19:25.428 INFO:teuthology.orchestra.run.smithi139.stdout: "op_w": 43, 2024-03-18T23:19:25.429 INFO:teuthology.orchestra.run.smithi139.stdout: "op_rmw": 0, 2024-03-18T23:19:25.429 INFO:teuthology.orchestra.run.smithi139.stdout: "op_pg": 0, 2024-03-18T23:19:25.429 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_stat": 0, 2024-03-18T23:19:25.429 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_create": 10, 2024-03-18T23:19:25.429 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_read": 0, 2024-03-18T23:19:25.429 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_write": 6, 2024-03-18T23:19:25.429 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_writefull": 8, 2024-03-18T23:19:25.430 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_writesame": 0, 2024-03-18T23:19:25.430 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_append": 0, 2024-03-18T23:19:25.430 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_zero": 0, 2024-03-18T23:19:25.430 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_truncate": 0, 2024-03-18T23:19:25.430 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_delete": 6, 2024-03-18T23:19:25.431 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_mapext": 0, 2024-03-18T23:19:25.431 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_sparse_read": 0, 2024-03-18T23:19:25.431 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_clonerange": 0, 2024-03-18T23:19:25.431 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_getxattr": 0, 2024-03-18T23:19:25.431 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_setxattr": 20, 2024-03-18T23:19:25.431 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_cmpxattr": 0, 2024-03-18T23:19:25.431 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_rmxattr": 0, 2024-03-18T23:19:25.431 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_resetxattrs": 0, 2024-03-18T23:19:25.432 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_call": 0, 2024-03-18T23:19:25.432 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_watch": 0, 2024-03-18T23:19:25.432 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_notify": 0, 2024-03-18T23:19:25.432 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_src_cmpxattr": 0, 2024-03-18T23:19:25.432 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_pgls": 0, 2024-03-18T23:19:25.432 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_pgls_filter": 0, 2024-03-18T23:19:25.432 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_other": 0, 2024-03-18T23:19:25.432 INFO:teuthology.orchestra.run.smithi139.stdout: "linger_active": 0, 2024-03-18T23:19:25.433 INFO:teuthology.orchestra.run.smithi139.stdout: "linger_send": 0, 2024-03-18T23:19:25.433 INFO:teuthology.orchestra.run.smithi139.stdout: "linger_resend": 0, 2024-03-18T23:19:25.433 INFO:teuthology.orchestra.run.smithi139.stdout: "linger_ping": 0, 2024-03-18T23:19:25.433 INFO:teuthology.orchestra.run.smithi139.stdout: "poolop_active": 0, 2024-03-18T23:19:25.433 INFO:teuthology.orchestra.run.smithi139.stdout: "poolop_send": 0, 2024-03-18T23:19:25.433 INFO:teuthology.orchestra.run.smithi139.stdout: "poolop_resend": 0, 2024-03-18T23:19:25.433 INFO:teuthology.orchestra.run.smithi139.stdout: "poolstat_active": 0, 2024-03-18T23:19:25.433 INFO:teuthology.orchestra.run.smithi139.stdout: "poolstat_send": 0, 2024-03-18T23:19:25.434 INFO:teuthology.orchestra.run.smithi139.stdout: "poolstat_resend": 0, 2024-03-18T23:19:25.434 INFO:teuthology.orchestra.run.smithi139.stdout: "statfs_active": 0, 2024-03-18T23:19:25.434 INFO:teuthology.orchestra.run.smithi139.stdout: "statfs_send": 0, 2024-03-18T23:19:25.434 INFO:teuthology.orchestra.run.smithi139.stdout: "statfs_resend": 0, 2024-03-18T23:19:25.434 INFO:teuthology.orchestra.run.smithi139.stdout: "command_active": 0, 2024-03-18T23:19:25.434 INFO:teuthology.orchestra.run.smithi139.stdout: "command_send": 0, 2024-03-18T23:19:25.434 INFO:teuthology.orchestra.run.smithi139.stdout: "command_resend": 0, 2024-03-18T23:19:25.434 INFO:teuthology.orchestra.run.smithi139.stdout: "map_epoch": 101, 2024-03-18T23:19:25.435 INFO:teuthology.orchestra.run.smithi139.stdout: "map_full": 0, 2024-03-18T23:19:25.435 INFO:teuthology.orchestra.run.smithi139.stdout: "map_inc": 15, 2024-03-18T23:19:25.435 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_sessions": 8, 2024-03-18T23:19:25.435 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_session_open": 8, 2024-03-18T23:19:25.435 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_session_close": 0, 2024-03-18T23:19:25.435 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_laggy": 0, 2024-03-18T23:19:25.435 INFO:teuthology.orchestra.run.smithi139.stdout: "omap_wr": 14, 2024-03-18T23:19:25.436 INFO:teuthology.orchestra.run.smithi139.stdout: "omap_rd": 0, 2024-03-18T23:19:25.436 INFO:teuthology.orchestra.run.smithi139.stdout: "omap_del": 1 2024-03-18T23:19:25.436 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:25.436 INFO:teuthology.orchestra.run.smithi139.stdout: "oft": { 2024-03-18T23:19:25.436 INFO:teuthology.orchestra.run.smithi139.stdout: "omap_total_objs": 1, 2024-03-18T23:19:25.437 INFO:teuthology.orchestra.run.smithi139.stdout: "omap_total_kv_pairs": 0, 2024-03-18T23:19:25.437 INFO:teuthology.orchestra.run.smithi139.stdout: "omap_total_updates": 0, 2024-03-18T23:19:25.437 INFO:teuthology.orchestra.run.smithi139.stdout: "omap_total_removes": 0 2024-03-18T23:19:25.437 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:25.438 INFO:teuthology.orchestra.run.smithi139.stdout: "osd-slow-ops": { 2024-03-18T23:19:25.438 INFO:teuthology.orchestra.run.smithi139.stdout: "slow_ops_count": 0 2024-03-18T23:19:25.438 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:25.438 INFO:teuthology.orchestra.run.smithi139.stdout: "purge_queue": { 2024-03-18T23:19:25.439 INFO:teuthology.orchestra.run.smithi139.stdout: "pq_executing_ops": 0, 2024-03-18T23:19:25.439 INFO:teuthology.orchestra.run.smithi139.stdout: "pq_executing_ops_high_water": 0, 2024-03-18T23:19:25.439 INFO:teuthology.orchestra.run.smithi139.stdout: "pq_executing": 0, 2024-03-18T23:19:25.439 INFO:teuthology.orchestra.run.smithi139.stdout: "pq_executing_high_water": 0, 2024-03-18T23:19:25.440 INFO:teuthology.orchestra.run.smithi139.stdout: "pq_executed_ops": 0, 2024-03-18T23:19:25.440 INFO:teuthology.orchestra.run.smithi139.stdout: "pq_executed": 0, 2024-03-18T23:19:25.440 INFO:teuthology.orchestra.run.smithi139.stdout: "pq_item_in_journal": 0 2024-03-18T23:19:25.441 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:25.441 INFO:teuthology.orchestra.run.smithi139.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-18T23:19:25.441 INFO:teuthology.orchestra.run.smithi139.stdout: "val": 0, 2024-03-18T23:19:25.441 INFO:teuthology.orchestra.run.smithi139.stdout: "max": 104857600, 2024-03-18T23:19:25.442 INFO:teuthology.orchestra.run.smithi139.stdout: "get_started": 0, 2024-03-18T23:19:25.442 INFO:teuthology.orchestra.run.smithi139.stdout: "get": 181145, 2024-03-18T23:19:25.442 INFO:teuthology.orchestra.run.smithi139.stdout: "get_sum": 82079608, 2024-03-18T23:19:25.442 INFO:teuthology.orchestra.run.smithi139.stdout: "get_or_fail_fail": 0, 2024-03-18T23:19:25.443 INFO:teuthology.orchestra.run.smithi139.stdout: "get_or_fail_success": 181145, 2024-03-18T23:19:25.443 INFO:teuthology.orchestra.run.smithi139.stdout: "take": 0, 2024-03-18T23:19:25.443 INFO:teuthology.orchestra.run.smithi139.stdout: "take_sum": 0, 2024-03-18T23:19:25.443 INFO:teuthology.orchestra.run.smithi139.stdout: "put": 181145, 2024-03-18T23:19:25.444 INFO:teuthology.orchestra.run.smithi139.stdout: "put_sum": 82079608, 2024-03-18T23:19:25.444 INFO:teuthology.orchestra.run.smithi139.stdout: "wait": { 2024-03-18T23:19:25.444 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-03-18T23:19:25.445 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-03-18T23:19:25.445 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-03-18T23:19:25.445 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-03-18T23:19:25.445 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:25.446 INFO:teuthology.orchestra.run.smithi139.stdout: "throttle-objecter_bytes": { 2024-03-18T23:19:25.446 INFO:teuthology.orchestra.run.smithi139.stdout: "val": 0, 2024-03-18T23:19:25.446 INFO:teuthology.orchestra.run.smithi139.stdout: "max": 104857600, 2024-03-18T23:19:25.446 INFO:teuthology.orchestra.run.smithi139.stdout: "get_started": 0, 2024-03-18T23:19:25.447 INFO:teuthology.orchestra.run.smithi139.stdout: "get": 0, 2024-03-18T23:19:25.447 INFO:teuthology.orchestra.run.smithi139.stdout: "get_sum": 0, 2024-03-18T23:19:25.447 INFO:teuthology.orchestra.run.smithi139.stdout: "get_or_fail_fail": 0, 2024-03-18T23:19:25.447 INFO:teuthology.orchestra.run.smithi139.stdout: "get_or_fail_success": 0, 2024-03-18T23:19:25.447 INFO:teuthology.orchestra.run.smithi139.stdout: "take": 43, 2024-03-18T23:19:25.447 INFO:teuthology.orchestra.run.smithi139.stdout: "take_sum": 12352, 2024-03-18T23:19:25.447 INFO:teuthology.orchestra.run.smithi139.stdout: "put": 43, 2024-03-18T23:19:25.447 INFO:teuthology.orchestra.run.smithi139.stdout: "put_sum": 12352, 2024-03-18T23:19:25.448 INFO:teuthology.orchestra.run.smithi139.stdout: "wait": { 2024-03-18T23:19:25.448 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-03-18T23:19:25.448 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-03-18T23:19:25.448 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-03-18T23:19:25.448 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-03-18T23:19:25.448 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:25.448 INFO:teuthology.orchestra.run.smithi139.stdout: "throttle-objecter_ops": { 2024-03-18T23:19:25.448 INFO:teuthology.orchestra.run.smithi139.stdout: "val": 0, 2024-03-18T23:19:25.449 INFO:teuthology.orchestra.run.smithi139.stdout: "max": 1024, 2024-03-18T23:19:25.449 INFO:teuthology.orchestra.run.smithi139.stdout: "get_started": 0, 2024-03-18T23:19:25.449 INFO:teuthology.orchestra.run.smithi139.stdout: "get": 0, 2024-03-18T23:19:25.449 INFO:teuthology.orchestra.run.smithi139.stdout: "get_sum": 0, 2024-03-18T23:19:25.449 INFO:teuthology.orchestra.run.smithi139.stdout: "get_or_fail_fail": 0, 2024-03-18T23:19:25.449 INFO:teuthology.orchestra.run.smithi139.stdout: "get_or_fail_success": 0, 2024-03-18T23:19:25.449 INFO:teuthology.orchestra.run.smithi139.stdout: "take": 43, 2024-03-18T23:19:25.449 INFO:teuthology.orchestra.run.smithi139.stdout: "take_sum": 43, 2024-03-18T23:19:25.450 INFO:teuthology.orchestra.run.smithi139.stdout: "put": 43, 2024-03-18T23:19:25.450 INFO:teuthology.orchestra.run.smithi139.stdout: "put_sum": 43, 2024-03-18T23:19:25.450 INFO:teuthology.orchestra.run.smithi139.stdout: "wait": { 2024-03-18T23:19:25.450 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-03-18T23:19:25.450 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-03-18T23:19:25.450 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-03-18T23:19:25.450 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-03-18T23:19:25.450 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:25.451 INFO:teuthology.orchestra.run.smithi139.stdout: "throttle-write_buf_throttle": { 2024-03-18T23:19:25.451 INFO:teuthology.orchestra.run.smithi139.stdout: "val": 0, 2024-03-18T23:19:25.451 INFO:teuthology.orchestra.run.smithi139.stdout: "max": 3758096384, 2024-03-18T23:19:25.451 INFO:teuthology.orchestra.run.smithi139.stdout: "get_started": 0, 2024-03-18T23:19:25.451 INFO:teuthology.orchestra.run.smithi139.stdout: "get": 0, 2024-03-18T23:19:25.451 INFO:teuthology.orchestra.run.smithi139.stdout: "get_sum": 0, 2024-03-18T23:19:25.451 INFO:teuthology.orchestra.run.smithi139.stdout: "get_or_fail_fail": 0, 2024-03-18T23:19:25.451 INFO:teuthology.orchestra.run.smithi139.stdout: "get_or_fail_success": 0, 2024-03-18T23:19:25.452 INFO:teuthology.orchestra.run.smithi139.stdout: "take": 0, 2024-03-18T23:19:25.452 INFO:teuthology.orchestra.run.smithi139.stdout: "take_sum": 0, 2024-03-18T23:19:25.452 INFO:teuthology.orchestra.run.smithi139.stdout: "put": 0, 2024-03-18T23:19:25.452 INFO:teuthology.orchestra.run.smithi139.stdout: "put_sum": 0, 2024-03-18T23:19:25.452 INFO:teuthology.orchestra.run.smithi139.stdout: "wait": { 2024-03-18T23:19:25.452 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-03-18T23:19:25.452 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-03-18T23:19:25.452 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-03-18T23:19:25.452 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-03-18T23:19:25.453 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:25.453 INFO:teuthology.orchestra.run.smithi139.stdout: "throttle-write_buf_throttle-0x55e3519ba500": { 2024-03-18T23:19:25.453 INFO:teuthology.orchestra.run.smithi139.stdout: "val": 0, 2024-03-18T23:19:25.453 INFO:teuthology.orchestra.run.smithi139.stdout: "max": 3758096384, 2024-03-18T23:19:25.453 INFO:teuthology.orchestra.run.smithi139.stdout: "get_started": 0, 2024-03-18T23:19:25.453 INFO:teuthology.orchestra.run.smithi139.stdout: "get": 7, 2024-03-18T23:19:25.453 INFO:teuthology.orchestra.run.smithi139.stdout: "get_sum": 1960, 2024-03-18T23:19:25.453 INFO:teuthology.orchestra.run.smithi139.stdout: "get_or_fail_fail": 0, 2024-03-18T23:19:25.454 INFO:teuthology.orchestra.run.smithi139.stdout: "get_or_fail_success": 7, 2024-03-18T23:19:25.454 INFO:teuthology.orchestra.run.smithi139.stdout: "take": 0, 2024-03-18T23:19:25.454 INFO:teuthology.orchestra.run.smithi139.stdout: "take_sum": 0, 2024-03-18T23:19:25.454 INFO:teuthology.orchestra.run.smithi139.stdout: "put": 6, 2024-03-18T23:19:25.454 INFO:teuthology.orchestra.run.smithi139.stdout: "put_sum": 1960, 2024-03-18T23:19:25.454 INFO:teuthology.orchestra.run.smithi139.stdout: "wait": { 2024-03-18T23:19:25.454 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-03-18T23:19:25.454 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-03-18T23:19:25.455 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-03-18T23:19:25.455 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-03-18T23:19:25.455 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-03-18T23:19:25.455 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-03-18T23:19:25.629 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:19:25 smithi049 ceph-mon[62991]: pgmap v349: 97 pgs: 97 active+clean; 69 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 1.2 MiB/s wr, 47 op/s 2024-03-18T23:19:25.710 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:19:25 smithi175 ceph-mon[67352]: pgmap v349: 97 pgs: 97 active+clean; 69 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 1.2 MiB/s wr, 47 op/s 2024-03-18T23:19:26.110 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2024-03-18T23:19:26.111 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2024-03-18T23:19:26.111 INFO:tasks.check_counter:Daemon mds.e mds.root_rsnaps=6 2024-03-18T23:19:26.111 INFO:tasks.check_counter:Daemon mds.e mds_server.req_mksnap_latency.avgcount=0 2024-03-18T23:19:26.111 INFO:tasks.check_counter:Daemon mds.e mds_server.req_rmsnap_latency.avgcount=0 2024-03-18T23:19:26.111 DEBUG:tasks.check_counter:Getting stats from h 2024-03-18T23:19:26.111 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell --fsid c63591ea-e57a-11ee-95c9-87774f69a715 -- ceph daemon mds.h perf dump 2024-03-18T23:19:26.364 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/mon.b/config 2024-03-18T23:19:27.476 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:19:27 smithi139 ceph-mon[67282]: pgmap v350: 97 pgs: 97 active+clean; 69 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 1.2 MiB/s wr, 47 op/s 2024-03-18T23:19:27.629 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:19:27 smithi049 ceph-mon[62991]: pgmap v350: 97 pgs: 97 active+clean; 69 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 1.2 MiB/s wr, 47 op/s 2024-03-18T23:19:27.710 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:19:27 smithi175 ceph-mon[67352]: pgmap v350: 97 pgs: 97 active+clean; 69 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 1.2 MiB/s wr, 47 op/s 2024-03-18T23:19:28.120 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-03-18T23:19:28.121 INFO:teuthology.orchestra.run.smithi139.stdout: "AsyncMessenger::Worker-0": { 2024-03-18T23:19:28.121 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_recv_messages": 688, 2024-03-18T23:19:28.121 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_messages": 690, 2024-03-18T23:19:28.121 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_recv_bytes": 184131747, 2024-03-18T23:19:28.121 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_bytes": 218790, 2024-03-18T23:19:28.121 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_created_connections": 8, 2024-03-18T23:19:28.121 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_active_connections": 4, 2024-03-18T23:19:28.122 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_total_time": 0.134117202, 2024-03-18T23:19:28.122 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_send_time": 0.012284102, 2024-03-18T23:19:28.122 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_recv_time": 0.102807239, 2024-03-18T23:19:28.122 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_fast_dispatch_time": 0.014146216, 2024-03-18T23:19:28.122 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_messages_queue_lat": { 2024-03-18T23:19:28.122 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 688, 2024-03-18T23:19:28.122 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.014775470, 2024-03-18T23:19:28.122 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000021475 2024-03-18T23:19:28.123 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:28.123 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_handle_ack_lat": { 2024-03-18T23:19:28.123 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-03-18T23:19:28.123 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-03-18T23:19:28.123 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-03-18T23:19:28.123 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:28.123 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-18T23:19:28.123 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_encrypted_bytes": 0 2024-03-18T23:19:28.124 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:28.124 INFO:teuthology.orchestra.run.smithi139.stdout: "AsyncMessenger::Worker-1": { 2024-03-18T23:19:28.124 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_recv_messages": 912, 2024-03-18T23:19:28.124 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_messages": 1006, 2024-03-18T23:19:28.124 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_recv_bytes": 130298614, 2024-03-18T23:19:28.124 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_bytes": 544935, 2024-03-18T23:19:28.124 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_created_connections": 7, 2024-03-18T23:19:28.124 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_active_connections": 6, 2024-03-18T23:19:28.124 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_total_time": 0.130352201, 2024-03-18T23:19:28.125 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_send_time": 0.023637430, 2024-03-18T23:19:28.125 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_recv_time": 0.080301309, 2024-03-18T23:19:28.125 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_fast_dispatch_time": 0.020765540, 2024-03-18T23:19:28.125 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_messages_queue_lat": { 2024-03-18T23:19:28.125 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 1005, 2024-03-18T23:19:28.125 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.035199809, 2024-03-18T23:19:28.125 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000035024 2024-03-18T23:19:28.125 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:28.126 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_handle_ack_lat": { 2024-03-18T23:19:28.126 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-03-18T23:19:28.126 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-03-18T23:19:28.126 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-03-18T23:19:28.126 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:28.126 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_recv_encrypted_bytes": 3184, 2024-03-18T23:19:28.126 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_encrypted_bytes": 256448 2024-03-18T23:19:28.126 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:28.126 INFO:teuthology.orchestra.run.smithi139.stdout: "AsyncMessenger::Worker-2": { 2024-03-18T23:19:28.127 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_recv_messages": 880, 2024-03-18T23:19:28.127 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_messages": 879, 2024-03-18T23:19:28.127 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_recv_bytes": 197110151, 2024-03-18T23:19:28.127 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_bytes": 314478, 2024-03-18T23:19:28.127 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_created_connections": 5, 2024-03-18T23:19:28.127 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_active_connections": 4, 2024-03-18T23:19:28.127 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_total_time": 0.160125219, 2024-03-18T23:19:28.127 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_send_time": 0.018692811, 2024-03-18T23:19:28.128 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_recv_time": 0.121303617, 2024-03-18T23:19:28.128 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_fast_dispatch_time": 0.017959573, 2024-03-18T23:19:28.128 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_messages_queue_lat": { 2024-03-18T23:19:28.128 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 878, 2024-03-18T23:19:28.128 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.018994027, 2024-03-18T23:19:28.128 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000021633 2024-03-18T23:19:28.128 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:28.128 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_handle_ack_lat": { 2024-03-18T23:19:28.128 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-03-18T23:19:28.129 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-03-18T23:19:28.129 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-03-18T23:19:28.129 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:28.129 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_recv_encrypted_bytes": 199552, 2024-03-18T23:19:28.129 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_encrypted_bytes": 81056 2024-03-18T23:19:28.129 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:28.129 INFO:teuthology.orchestra.run.smithi139.stdout: "cct": { 2024-03-18T23:19:28.129 INFO:teuthology.orchestra.run.smithi139.stdout: "total_workers": 1, 2024-03-18T23:19:28.130 INFO:teuthology.orchestra.run.smithi139.stdout: "unhealthy_workers": 0 2024-03-18T23:19:28.130 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:28.130 INFO:teuthology.orchestra.run.smithi139.stdout: "finisher-MDSRank": { 2024-03-18T23:19:28.130 INFO:teuthology.orchestra.run.smithi139.stdout: "queue_len": 0, 2024-03-18T23:19:28.130 INFO:teuthology.orchestra.run.smithi139.stdout: "complete_latency": { 2024-03-18T23:19:28.130 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 3637, 2024-03-18T23:19:28.130 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.305770236, 2024-03-18T23:19:28.130 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000084072 2024-03-18T23:19:28.130 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-03-18T23:19:28.131 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:28.131 INFO:teuthology.orchestra.run.smithi139.stdout: "finisher-PurgeQueue": { 2024-03-18T23:19:28.131 INFO:teuthology.orchestra.run.smithi139.stdout: "queue_len": 0, 2024-03-18T23:19:28.131 INFO:teuthology.orchestra.run.smithi139.stdout: "complete_latency": { 2024-03-18T23:19:28.131 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-03-18T23:19:28.131 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-03-18T23:19:28.131 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-03-18T23:19:28.131 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-03-18T23:19:28.131 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:28.132 INFO:teuthology.orchestra.run.smithi139.stdout: "mds": { 2024-03-18T23:19:28.132 INFO:teuthology.orchestra.run.smithi139.stdout: "request": 0, 2024-03-18T23:19:28.132 INFO:teuthology.orchestra.run.smithi139.stdout: "reply": 0, 2024-03-18T23:19:28.132 INFO:teuthology.orchestra.run.smithi139.stdout: "reply_latency": { 2024-03-18T23:19:28.132 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-03-18T23:19:28.132 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-03-18T23:19:28.132 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-03-18T23:19:28.132 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:28.132 INFO:teuthology.orchestra.run.smithi139.stdout: "slow_reply": 0, 2024-03-18T23:19:28.133 INFO:teuthology.orchestra.run.smithi139.stdout: "forward": 0, 2024-03-18T23:19:28.133 INFO:teuthology.orchestra.run.smithi139.stdout: "dir_fetch_complete": 0, 2024-03-18T23:19:28.133 INFO:teuthology.orchestra.run.smithi139.stdout: "dir_fetch_keys": 0, 2024-03-18T23:19:28.133 INFO:teuthology.orchestra.run.smithi139.stdout: "dir_commit": 0, 2024-03-18T23:19:28.133 INFO:teuthology.orchestra.run.smithi139.stdout: "dir_split": 0, 2024-03-18T23:19:28.133 INFO:teuthology.orchestra.run.smithi139.stdout: "dir_merge": 0, 2024-03-18T23:19:28.133 INFO:teuthology.orchestra.run.smithi139.stdout: "inodes": 27600, 2024-03-18T23:19:28.133 INFO:teuthology.orchestra.run.smithi139.stdout: "inodes_top": 18720, 2024-03-18T23:19:28.134 INFO:teuthology.orchestra.run.smithi139.stdout: "inodes_bottom": 8880, 2024-03-18T23:19:28.134 INFO:teuthology.orchestra.run.smithi139.stdout: "inodes_pin_tail": 0, 2024-03-18T23:19:28.134 INFO:teuthology.orchestra.run.smithi139.stdout: "inodes_pinned": 1101, 2024-03-18T23:19:28.134 INFO:teuthology.orchestra.run.smithi139.stdout: "inodes_expired": 0, 2024-03-18T23:19:28.134 INFO:teuthology.orchestra.run.smithi139.stdout: "inodes_with_caps": 0, 2024-03-18T23:19:28.134 INFO:teuthology.orchestra.run.smithi139.stdout: "caps": 0, 2024-03-18T23:19:28.134 INFO:teuthology.orchestra.run.smithi139.stdout: "subtrees": 3, 2024-03-18T23:19:28.134 INFO:teuthology.orchestra.run.smithi139.stdout: "traverse": 0, 2024-03-18T23:19:28.135 INFO:teuthology.orchestra.run.smithi139.stdout: "traverse_hit": 0, 2024-03-18T23:19:28.135 INFO:teuthology.orchestra.run.smithi139.stdout: "traverse_forward": 0, 2024-03-18T23:19:28.135 INFO:teuthology.orchestra.run.smithi139.stdout: "traverse_discover": 0, 2024-03-18T23:19:28.135 INFO:teuthology.orchestra.run.smithi139.stdout: "traverse_dir_fetch": 0, 2024-03-18T23:19:28.135 INFO:teuthology.orchestra.run.smithi139.stdout: "traverse_remote_ino": 0, 2024-03-18T23:19:28.135 INFO:teuthology.orchestra.run.smithi139.stdout: "traverse_lock": 0, 2024-03-18T23:19:28.135 INFO:teuthology.orchestra.run.smithi139.stdout: "load_cent": 0, 2024-03-18T23:19:28.135 INFO:teuthology.orchestra.run.smithi139.stdout: "q": 0, 2024-03-18T23:19:28.136 INFO:teuthology.orchestra.run.smithi139.stdout: "exported": 0, 2024-03-18T23:19:28.136 INFO:teuthology.orchestra.run.smithi139.stdout: "exported_inodes": 0, 2024-03-18T23:19:28.136 INFO:teuthology.orchestra.run.smithi139.stdout: "imported": 0, 2024-03-18T23:19:28.136 INFO:teuthology.orchestra.run.smithi139.stdout: "imported_inodes": 0, 2024-03-18T23:19:28.136 INFO:teuthology.orchestra.run.smithi139.stdout: "openino_dir_fetch": 0, 2024-03-18T23:19:28.136 INFO:teuthology.orchestra.run.smithi139.stdout: "openino_backtrace_fetch": 0, 2024-03-18T23:19:28.136 INFO:teuthology.orchestra.run.smithi139.stdout: "openino_peer_discover": 0, 2024-03-18T23:19:28.136 INFO:teuthology.orchestra.run.smithi139.stdout: "root_rfiles": 69, 2024-03-18T23:19:28.137 INFO:teuthology.orchestra.run.smithi139.stdout: "root_rbytes": 21153440, 2024-03-18T23:19:28.137 INFO:teuthology.orchestra.run.smithi139.stdout: "root_rsnaps": 6, 2024-03-18T23:19:28.137 INFO:teuthology.orchestra.run.smithi139.stdout: "scrub_backtrace_fetch": 0, 2024-03-18T23:19:28.137 INFO:teuthology.orchestra.run.smithi139.stdout: "scrub_set_tag": 0, 2024-03-18T23:19:28.137 INFO:teuthology.orchestra.run.smithi139.stdout: "scrub_backtrace_repaired": 0, 2024-03-18T23:19:28.137 INFO:teuthology.orchestra.run.smithi139.stdout: "scrub_inotable_repaired": 0, 2024-03-18T23:19:28.137 INFO:teuthology.orchestra.run.smithi139.stdout: "scrub_dir_inodes": 0, 2024-03-18T23:19:28.137 INFO:teuthology.orchestra.run.smithi139.stdout: "scrub_dir_base_inodes": 0, 2024-03-18T23:19:28.138 INFO:teuthology.orchestra.run.smithi139.stdout: "scrub_dirfrag_rstats": 0, 2024-03-18T23:19:28.138 INFO:teuthology.orchestra.run.smithi139.stdout: "scrub_file_inodes": 0, 2024-03-18T23:19:28.138 INFO:teuthology.orchestra.run.smithi139.stdout: "handle_inode_file_caps": 0, 2024-03-18T23:19:28.138 INFO:teuthology.orchestra.run.smithi139.stdout: "ceph_cap_op_revoke": 0, 2024-03-18T23:19:28.138 INFO:teuthology.orchestra.run.smithi139.stdout: "ceph_cap_op_grant": 0, 2024-03-18T23:19:28.138 INFO:teuthology.orchestra.run.smithi139.stdout: "ceph_cap_op_trunc": 0, 2024-03-18T23:19:28.138 INFO:teuthology.orchestra.run.smithi139.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-18T23:19:28.138 INFO:teuthology.orchestra.run.smithi139.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-18T23:19:28.139 INFO:teuthology.orchestra.run.smithi139.stdout: "handle_client_caps": 0, 2024-03-18T23:19:28.139 INFO:teuthology.orchestra.run.smithi139.stdout: "handle_client_caps_dirty": 0, 2024-03-18T23:19:28.139 INFO:teuthology.orchestra.run.smithi139.stdout: "handle_client_cap_release": 0, 2024-03-18T23:19:28.139 INFO:teuthology.orchestra.run.smithi139.stdout: "process_request_cap_release": 0 2024-03-18T23:19:28.139 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:28.139 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_cache": { 2024-03-18T23:19:28.139 INFO:teuthology.orchestra.run.smithi139.stdout: "dir_update": 0, 2024-03-18T23:19:28.139 INFO:teuthology.orchestra.run.smithi139.stdout: "dir_update_receipt": 0, 2024-03-18T23:19:28.139 INFO:teuthology.orchestra.run.smithi139.stdout: "dir_try_discover": 0, 2024-03-18T23:19:28.140 INFO:teuthology.orchestra.run.smithi139.stdout: "dir_send_discover": 0, 2024-03-18T23:19:28.140 INFO:teuthology.orchestra.run.smithi139.stdout: "dir_handle_discover": 0, 2024-03-18T23:19:28.140 INFO:teuthology.orchestra.run.smithi139.stdout: "num_strays": 0, 2024-03-18T23:19:28.140 INFO:teuthology.orchestra.run.smithi139.stdout: "num_strays_delayed": 0, 2024-03-18T23:19:28.140 INFO:teuthology.orchestra.run.smithi139.stdout: "num_strays_enqueuing": 0, 2024-03-18T23:19:28.140 INFO:teuthology.orchestra.run.smithi139.stdout: "strays_created": 0, 2024-03-18T23:19:28.140 INFO:teuthology.orchestra.run.smithi139.stdout: "strays_enqueued": 0, 2024-03-18T23:19:28.140 INFO:teuthology.orchestra.run.smithi139.stdout: "strays_reintegrated": 0, 2024-03-18T23:19:28.141 INFO:teuthology.orchestra.run.smithi139.stdout: "strays_migrated": 0, 2024-03-18T23:19:28.141 INFO:teuthology.orchestra.run.smithi139.stdout: "num_recovering_processing": 0, 2024-03-18T23:19:28.141 INFO:teuthology.orchestra.run.smithi139.stdout: "num_recovering_enqueued": 0, 2024-03-18T23:19:28.141 INFO:teuthology.orchestra.run.smithi139.stdout: "num_recovering_prioritized": 0, 2024-03-18T23:19:28.141 INFO:teuthology.orchestra.run.smithi139.stdout: "recovery_started": 0, 2024-03-18T23:19:28.141 INFO:teuthology.orchestra.run.smithi139.stdout: "recovery_completed": 0, 2024-03-18T23:19:28.141 INFO:teuthology.orchestra.run.smithi139.stdout: "ireq_quiesce_path": 0, 2024-03-18T23:19:28.141 INFO:teuthology.orchestra.run.smithi139.stdout: "ireq_quiesce_inode": 0, 2024-03-18T23:19:28.142 INFO:teuthology.orchestra.run.smithi139.stdout: "ireq_enqueue_scrub": 0, 2024-03-18T23:19:28.142 INFO:teuthology.orchestra.run.smithi139.stdout: "ireq_exportdir": 0, 2024-03-18T23:19:28.142 INFO:teuthology.orchestra.run.smithi139.stdout: "ireq_flush": 0, 2024-03-18T23:19:28.142 INFO:teuthology.orchestra.run.smithi139.stdout: "ireq_fragmentdir": 0, 2024-03-18T23:19:28.142 INFO:teuthology.orchestra.run.smithi139.stdout: "ireq_fragstats": 0, 2024-03-18T23:19:28.142 INFO:teuthology.orchestra.run.smithi139.stdout: "ireq_inodestats": 0 2024-03-18T23:19:28.142 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:28.142 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_log": { 2024-03-18T23:19:28.142 INFO:teuthology.orchestra.run.smithi139.stdout: "evlrg": 0, 2024-03-18T23:19:28.143 INFO:teuthology.orchestra.run.smithi139.stdout: "evadd": 0, 2024-03-18T23:19:28.143 INFO:teuthology.orchestra.run.smithi139.stdout: "evex": 0, 2024-03-18T23:19:28.143 INFO:teuthology.orchestra.run.smithi139.stdout: "evtrm": 0, 2024-03-18T23:19:28.143 INFO:teuthology.orchestra.run.smithi139.stdout: "ev": 123316, 2024-03-18T23:19:28.143 INFO:teuthology.orchestra.run.smithi139.stdout: "evexg": 0, 2024-03-18T23:19:28.143 INFO:teuthology.orchestra.run.smithi139.stdout: "evexd": 0, 2024-03-18T23:19:28.143 INFO:teuthology.orchestra.run.smithi139.stdout: "segadd": 0, 2024-03-18T23:19:28.143 INFO:teuthology.orchestra.run.smithi139.stdout: "segex": 0, 2024-03-18T23:19:28.143 INFO:teuthology.orchestra.run.smithi139.stdout: "segtrm": 0, 2024-03-18T23:19:28.144 INFO:teuthology.orchestra.run.smithi139.stdout: "seg": 3, 2024-03-18T23:19:28.144 INFO:teuthology.orchestra.run.smithi139.stdout: "segmjr": 34, 2024-03-18T23:19:28.144 INFO:teuthology.orchestra.run.smithi139.stdout: "segexg": 0, 2024-03-18T23:19:28.144 INFO:teuthology.orchestra.run.smithi139.stdout: "segexd": 0, 2024-03-18T23:19:28.144 INFO:teuthology.orchestra.run.smithi139.stdout: "expos": 671845785, 2024-03-18T23:19:28.144 INFO:teuthology.orchestra.run.smithi139.stdout: "wrpos": 678881804, 2024-03-18T23:19:28.144 INFO:teuthology.orchestra.run.smithi139.stdout: "rdpos": 678880189, 2024-03-18T23:19:28.144 INFO:teuthology.orchestra.run.smithi139.stdout: "jlat": { 2024-03-18T23:19:28.145 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-03-18T23:19:28.145 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-03-18T23:19:28.145 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-03-18T23:19:28.145 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:28.145 INFO:teuthology.orchestra.run.smithi139.stdout: "replayed": 123316 2024-03-18T23:19:28.145 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:28.145 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_mem": { 2024-03-18T23:19:28.145 INFO:teuthology.orchestra.run.smithi139.stdout: "ino": 731, 2024-03-18T23:19:28.145 INFO:teuthology.orchestra.run.smithi139.stdout: "ino+": 27409, 2024-03-18T23:19:28.146 INFO:teuthology.orchestra.run.smithi139.stdout: "ino-": 26678, 2024-03-18T23:19:28.146 INFO:teuthology.orchestra.run.smithi139.stdout: "dir": 56, 2024-03-18T23:19:28.146 INFO:teuthology.orchestra.run.smithi139.stdout: "dir+": 157, 2024-03-18T23:19:28.146 INFO:teuthology.orchestra.run.smithi139.stdout: "dir-": 101, 2024-03-18T23:19:28.146 INFO:teuthology.orchestra.run.smithi139.stdout: "dn": 27600, 2024-03-18T23:19:28.146 INFO:teuthology.orchestra.run.smithi139.stdout: "dn+": 27600, 2024-03-18T23:19:28.146 INFO:teuthology.orchestra.run.smithi139.stdout: "dn-": 0, 2024-03-18T23:19:28.146 INFO:teuthology.orchestra.run.smithi139.stdout: "cap": 0, 2024-03-18T23:19:28.146 INFO:teuthology.orchestra.run.smithi139.stdout: "cap+": 0, 2024-03-18T23:19:28.147 INFO:teuthology.orchestra.run.smithi139.stdout: "cap-": 0, 2024-03-18T23:19:28.147 INFO:teuthology.orchestra.run.smithi139.stdout: "rss": 76784, 2024-03-18T23:19:28.147 INFO:teuthology.orchestra.run.smithi139.stdout: "heap": 215396 2024-03-18T23:19:28.147 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:28.147 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_server": { 2024-03-18T23:19:28.147 INFO:teuthology.orchestra.run.smithi139.stdout: "dispatch_client_request": 0, 2024-03-18T23:19:28.147 INFO:teuthology.orchestra.run.smithi139.stdout: "dispatch_server_request": 0, 2024-03-18T23:19:28.147 INFO:teuthology.orchestra.run.smithi139.stdout: "handle_client_request": 0, 2024-03-18T23:19:28.148 INFO:teuthology.orchestra.run.smithi139.stdout: "handle_client_session": 0, 2024-03-18T23:19:28.148 INFO:teuthology.orchestra.run.smithi139.stdout: "handle_peer_request": 0, 2024-03-18T23:19:28.148 INFO:teuthology.orchestra.run.smithi139.stdout: "req_create_latency": { 2024-03-18T23:19:28.148 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-03-18T23:19:28.148 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-03-18T23:19:28.148 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-03-18T23:19:28.148 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:28.148 INFO:teuthology.orchestra.run.smithi139.stdout: "req_getattr_latency": { 2024-03-18T23:19:28.148 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-03-18T23:19:28.149 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-03-18T23:19:28.149 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-03-18T23:19:28.149 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:28.149 INFO:teuthology.orchestra.run.smithi139.stdout: "req_getfilelock_latency": { 2024-03-18T23:19:28.149 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-03-18T23:19:28.149 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-03-18T23:19:28.149 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-03-18T23:19:28.149 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:28.149 INFO:teuthology.orchestra.run.smithi139.stdout: "req_link_latency": { 2024-03-18T23:19:28.150 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-03-18T23:19:28.150 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-03-18T23:19:28.150 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-03-18T23:19:28.150 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:28.150 INFO:teuthology.orchestra.run.smithi139.stdout: "req_lookup_latency": { 2024-03-18T23:19:28.150 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-03-18T23:19:28.150 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-03-18T23:19:28.150 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-03-18T23:19:28.151 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:28.151 INFO:teuthology.orchestra.run.smithi139.stdout: "req_lookuphash_latency": { 2024-03-18T23:19:28.151 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-03-18T23:19:28.151 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-03-18T23:19:28.151 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-03-18T23:19:28.151 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:28.151 INFO:teuthology.orchestra.run.smithi139.stdout: "req_lookupino_latency": { 2024-03-18T23:19:28.151 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-03-18T23:19:28.151 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-03-18T23:19:28.152 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-03-18T23:19:28.152 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:28.152 INFO:teuthology.orchestra.run.smithi139.stdout: "req_lookupname_latency": { 2024-03-18T23:19:28.152 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-03-18T23:19:28.152 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-03-18T23:19:28.152 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-03-18T23:19:28.152 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:28.152 INFO:teuthology.orchestra.run.smithi139.stdout: "req_lookupparent_latency": { 2024-03-18T23:19:28.153 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-03-18T23:19:28.153 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-03-18T23:19:28.153 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-03-18T23:19:28.153 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:28.153 INFO:teuthology.orchestra.run.smithi139.stdout: "req_lookupsnap_latency": { 2024-03-18T23:19:28.153 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-03-18T23:19:28.153 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-03-18T23:19:28.153 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-03-18T23:19:28.153 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:28.154 INFO:teuthology.orchestra.run.smithi139.stdout: "req_lssnap_latency": { 2024-03-18T23:19:28.154 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-03-18T23:19:28.154 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-03-18T23:19:28.154 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-03-18T23:19:28.154 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:28.155 INFO:teuthology.orchestra.run.smithi139.stdout: "req_mkdir_latency": { 2024-03-18T23:19:28.155 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-03-18T23:19:28.155 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-03-18T23:19:28.155 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-03-18T23:19:28.155 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:28.155 INFO:teuthology.orchestra.run.smithi139.stdout: "req_mknod_latency": { 2024-03-18T23:19:28.156 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-03-18T23:19:28.156 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-03-18T23:19:28.156 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-03-18T23:19:28.156 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:28.156 INFO:teuthology.orchestra.run.smithi139.stdout: "req_mksnap_latency": { 2024-03-18T23:19:28.156 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-03-18T23:19:28.156 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-03-18T23:19:28.156 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-03-18T23:19:28.157 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:28.157 INFO:teuthology.orchestra.run.smithi139.stdout: "req_open_latency": { 2024-03-18T23:19:28.157 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-03-18T23:19:28.157 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-03-18T23:19:28.157 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-03-18T23:19:28.157 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:28.157 INFO:teuthology.orchestra.run.smithi139.stdout: "req_readdir_latency": { 2024-03-18T23:19:28.157 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-03-18T23:19:28.158 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-03-18T23:19:28.158 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-03-18T23:19:28.158 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:28.158 INFO:teuthology.orchestra.run.smithi139.stdout: "req_rename_latency": { 2024-03-18T23:19:28.158 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-03-18T23:19:28.158 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-03-18T23:19:28.158 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-03-18T23:19:28.158 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:28.159 INFO:teuthology.orchestra.run.smithi139.stdout: "req_renamesnap_latency": { 2024-03-18T23:19:28.159 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-03-18T23:19:28.159 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-03-18T23:19:28.159 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-03-18T23:19:28.159 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:28.159 INFO:teuthology.orchestra.run.smithi139.stdout: "req_snapdiff_latency": { 2024-03-18T23:19:28.159 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-03-18T23:19:28.159 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-03-18T23:19:28.159 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-03-18T23:19:28.160 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:28.160 INFO:teuthology.orchestra.run.smithi139.stdout: "req_rmdir_latency": { 2024-03-18T23:19:28.160 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-03-18T23:19:28.160 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-03-18T23:19:28.160 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-03-18T23:19:28.160 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:28.160 INFO:teuthology.orchestra.run.smithi139.stdout: "req_rmsnap_latency": { 2024-03-18T23:19:28.160 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-03-18T23:19:28.160 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-03-18T23:19:28.161 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-03-18T23:19:28.161 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:28.161 INFO:teuthology.orchestra.run.smithi139.stdout: "req_rmxattr_latency": { 2024-03-18T23:19:28.161 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-03-18T23:19:28.161 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-03-18T23:19:28.161 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-03-18T23:19:28.161 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:28.161 INFO:teuthology.orchestra.run.smithi139.stdout: "req_setattr_latency": { 2024-03-18T23:19:28.161 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-03-18T23:19:28.162 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-03-18T23:19:28.162 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-03-18T23:19:28.162 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:28.162 INFO:teuthology.orchestra.run.smithi139.stdout: "req_setdirlayout_latency": { 2024-03-18T23:19:28.162 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-03-18T23:19:28.162 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-03-18T23:19:28.162 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-03-18T23:19:28.162 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:28.162 INFO:teuthology.orchestra.run.smithi139.stdout: "req_setfilelock_latency": { 2024-03-18T23:19:28.163 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-03-18T23:19:28.163 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-03-18T23:19:28.163 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-03-18T23:19:28.163 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:28.163 INFO:teuthology.orchestra.run.smithi139.stdout: "req_setlayout_latency": { 2024-03-18T23:19:28.163 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-03-18T23:19:28.163 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-03-18T23:19:28.163 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-03-18T23:19:28.164 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:28.164 INFO:teuthology.orchestra.run.smithi139.stdout: "req_setxattr_latency": { 2024-03-18T23:19:28.164 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-03-18T23:19:28.164 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-03-18T23:19:28.164 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-03-18T23:19:28.164 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:28.164 INFO:teuthology.orchestra.run.smithi139.stdout: "req_symlink_latency": { 2024-03-18T23:19:28.164 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-03-18T23:19:28.164 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-03-18T23:19:28.165 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-03-18T23:19:28.165 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:28.165 INFO:teuthology.orchestra.run.smithi139.stdout: "req_unlink_latency": { 2024-03-18T23:19:28.165 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-03-18T23:19:28.165 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-03-18T23:19:28.165 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-03-18T23:19:28.165 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:28.165 INFO:teuthology.orchestra.run.smithi139.stdout: "cap_revoke_eviction": 0, 2024-03-18T23:19:28.165 INFO:teuthology.orchestra.run.smithi139.stdout: "cap_acquisition_throttle": 0, 2024-03-18T23:19:28.166 INFO:teuthology.orchestra.run.smithi139.stdout: "req_getvxattr_latency": { 2024-03-18T23:19:28.166 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-03-18T23:19:28.166 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-03-18T23:19:28.166 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-03-18T23:19:28.166 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-03-18T23:19:28.166 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:28.169 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_sessions": { 2024-03-18T23:19:28.169 INFO:teuthology.orchestra.run.smithi139.stdout: "session_count": 2, 2024-03-18T23:19:28.169 INFO:teuthology.orchestra.run.smithi139.stdout: "session_add": 3, 2024-03-18T23:19:28.170 INFO:teuthology.orchestra.run.smithi139.stdout: "session_remove": 1, 2024-03-18T23:19:28.170 INFO:teuthology.orchestra.run.smithi139.stdout: "sessions_open": 0, 2024-03-18T23:19:28.170 INFO:teuthology.orchestra.run.smithi139.stdout: "sessions_stale": 0, 2024-03-18T23:19:28.170 INFO:teuthology.orchestra.run.smithi139.stdout: "total_load": 0, 2024-03-18T23:19:28.170 INFO:teuthology.orchestra.run.smithi139.stdout: "average_load": 0, 2024-03-18T23:19:28.170 INFO:teuthology.orchestra.run.smithi139.stdout: "avg_session_uptime": 9138, 2024-03-18T23:19:28.170 INFO:teuthology.orchestra.run.smithi139.stdout: "mdthresh_evicted": 0 2024-03-18T23:19:28.170 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:28.171 INFO:teuthology.orchestra.run.smithi139.stdout: "mempool": { 2024-03-18T23:19:28.171 INFO:teuthology.orchestra.run.smithi139.stdout: "bloom_filter_bytes": 0, 2024-03-18T23:19:28.171 INFO:teuthology.orchestra.run.smithi139.stdout: "bloom_filter_items": 0, 2024-03-18T23:19:28.171 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_alloc_bytes": 0, 2024-03-18T23:19:28.171 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_alloc_items": 0, 2024-03-18T23:19:28.171 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_cache_data_bytes": 0, 2024-03-18T23:19:28.171 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_cache_data_items": 0, 2024-03-18T23:19:28.171 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-18T23:19:28.171 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_cache_onode_items": 0, 2024-03-18T23:19:28.172 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-18T23:19:28.172 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_cache_meta_items": 0, 2024-03-18T23:19:28.172 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_cache_other_bytes": 0, 2024-03-18T23:19:28.172 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_cache_other_items": 0, 2024-03-18T23:19:28.172 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-18T23:19:28.172 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_cache_buffer_items": 0, 2024-03-18T23:19:28.172 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_extent_bytes": 0, 2024-03-18T23:19:28.172 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_extent_items": 0, 2024-03-18T23:19:28.173 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_blob_bytes": 0, 2024-03-18T23:19:28.173 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_blob_items": 0, 2024-03-18T23:19:28.173 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-18T23:19:28.173 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_shared_blob_items": 0, 2024-03-18T23:19:28.173 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-18T23:19:28.173 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_inline_bl_items": 0, 2024-03-18T23:19:28.173 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_fsck_bytes": 0, 2024-03-18T23:19:28.173 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_fsck_items": 0, 2024-03-18T23:19:28.173 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_txc_bytes": 0, 2024-03-18T23:19:28.174 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_txc_items": 0, 2024-03-18T23:19:28.174 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-18T23:19:28.174 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_writing_deferred_items": 0, 2024-03-18T23:19:28.174 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_writing_bytes": 0, 2024-03-18T23:19:28.174 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_writing_items": 0, 2024-03-18T23:19:28.174 INFO:teuthology.orchestra.run.smithi139.stdout: "bluefs_bytes": 0, 2024-03-18T23:19:28.174 INFO:teuthology.orchestra.run.smithi139.stdout: "bluefs_items": 0, 2024-03-18T23:19:28.174 INFO:teuthology.orchestra.run.smithi139.stdout: "bluefs_file_reader_bytes": 0, 2024-03-18T23:19:28.175 INFO:teuthology.orchestra.run.smithi139.stdout: "bluefs_file_reader_items": 0, 2024-03-18T23:19:28.175 INFO:teuthology.orchestra.run.smithi139.stdout: "bluefs_file_writer_bytes": 0, 2024-03-18T23:19:28.175 INFO:teuthology.orchestra.run.smithi139.stdout: "bluefs_file_writer_items": 0, 2024-03-18T23:19:28.175 INFO:teuthology.orchestra.run.smithi139.stdout: "buffer_anon_bytes": 268510, 2024-03-18T23:19:28.175 INFO:teuthology.orchestra.run.smithi139.stdout: "buffer_anon_items": 885, 2024-03-18T23:19:28.175 INFO:teuthology.orchestra.run.smithi139.stdout: "buffer_meta_bytes": 0, 2024-03-18T23:19:28.175 INFO:teuthology.orchestra.run.smithi139.stdout: "buffer_meta_items": 0, 2024-03-18T23:19:28.175 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_bytes": 0, 2024-03-18T23:19:28.175 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_items": 0, 2024-03-18T23:19:28.175 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_mapbl_bytes": 0, 2024-03-18T23:19:28.176 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_mapbl_items": 0, 2024-03-18T23:19:28.176 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_pglog_bytes": 0, 2024-03-18T23:19:28.176 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_pglog_items": 0, 2024-03-18T23:19:28.176 INFO:teuthology.orchestra.run.smithi139.stdout: "osdmap_bytes": 8256, 2024-03-18T23:19:28.176 INFO:teuthology.orchestra.run.smithi139.stdout: "osdmap_items": 188, 2024-03-18T23:19:28.176 INFO:teuthology.orchestra.run.smithi139.stdout: "osdmap_mapping_bytes": 0, 2024-03-18T23:19:28.176 INFO:teuthology.orchestra.run.smithi139.stdout: "osdmap_mapping_items": 0, 2024-03-18T23:19:28.176 INFO:teuthology.orchestra.run.smithi139.stdout: "pgmap_bytes": 0, 2024-03-18T23:19:28.177 INFO:teuthology.orchestra.run.smithi139.stdout: "pgmap_items": 0, 2024-03-18T23:19:28.177 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_co_bytes": 21323511, 2024-03-18T23:19:28.177 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_co_items": 147974, 2024-03-18T23:19:28.177 INFO:teuthology.orchestra.run.smithi139.stdout: "unittest_1_bytes": 0, 2024-03-18T23:19:28.177 INFO:teuthology.orchestra.run.smithi139.stdout: "unittest_1_items": 0, 2024-03-18T23:19:28.177 INFO:teuthology.orchestra.run.smithi139.stdout: "unittest_2_bytes": 0, 2024-03-18T23:19:28.177 INFO:teuthology.orchestra.run.smithi139.stdout: "unittest_2_items": 0 2024-03-18T23:19:28.177 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:28.178 INFO:teuthology.orchestra.run.smithi139.stdout: "objecter": { 2024-03-18T23:19:28.178 INFO:teuthology.orchestra.run.smithi139.stdout: "op_active": 0, 2024-03-18T23:19:28.178 INFO:teuthology.orchestra.run.smithi139.stdout: "op_laggy": 0, 2024-03-18T23:19:28.178 INFO:teuthology.orchestra.run.smithi139.stdout: "op_send": 2329, 2024-03-18T23:19:28.178 INFO:teuthology.orchestra.run.smithi139.stdout: "op_send_bytes": 26, 2024-03-18T23:19:28.178 INFO:teuthology.orchestra.run.smithi139.stdout: "op_resend": 4, 2024-03-18T23:19:28.178 INFO:teuthology.orchestra.run.smithi139.stdout: "op_reply": 2329, 2024-03-18T23:19:28.178 INFO:teuthology.orchestra.run.smithi139.stdout: "op_latency": { 2024-03-18T23:19:28.179 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 2329, 2024-03-18T23:19:28.179 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 3.094935747, 2024-03-18T23:19:28.179 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.001328868 2024-03-18T23:19:28.179 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:28.179 INFO:teuthology.orchestra.run.smithi139.stdout: "op_inflight": 0, 2024-03-18T23:19:28.179 INFO:teuthology.orchestra.run.smithi139.stdout: "oplen_avg": { 2024-03-18T23:19:28.179 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 2329, 2024-03-18T23:19:28.179 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 2330 2024-03-18T23:19:28.180 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:28.180 INFO:teuthology.orchestra.run.smithi139.stdout: "op": 2329, 2024-03-18T23:19:28.180 INFO:teuthology.orchestra.run.smithi139.stdout: "op_r": 2329, 2024-03-18T23:19:28.180 INFO:teuthology.orchestra.run.smithi139.stdout: "op_w": 0, 2024-03-18T23:19:28.180 INFO:teuthology.orchestra.run.smithi139.stdout: "op_rmw": 0, 2024-03-18T23:19:28.180 INFO:teuthology.orchestra.run.smithi139.stdout: "op_pg": 0, 2024-03-18T23:19:28.180 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_stat": 1325, 2024-03-18T23:19:28.180 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_create": 0, 2024-03-18T23:19:28.180 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_read": 1001, 2024-03-18T23:19:28.180 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_write": 0, 2024-03-18T23:19:28.181 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_writefull": 0, 2024-03-18T23:19:28.181 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_writesame": 0, 2024-03-18T23:19:28.181 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_append": 0, 2024-03-18T23:19:28.181 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_zero": 0, 2024-03-18T23:19:28.181 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_truncate": 0, 2024-03-18T23:19:28.181 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_delete": 0, 2024-03-18T23:19:28.181 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_mapext": 0, 2024-03-18T23:19:28.181 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_sparse_read": 0, 2024-03-18T23:19:28.181 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_clonerange": 0, 2024-03-18T23:19:28.182 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_getxattr": 2, 2024-03-18T23:19:28.182 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_setxattr": 0, 2024-03-18T23:19:28.182 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_cmpxattr": 0, 2024-03-18T23:19:28.182 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_rmxattr": 0, 2024-03-18T23:19:28.182 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_resetxattrs": 0, 2024-03-18T23:19:28.182 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_call": 0, 2024-03-18T23:19:28.182 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_watch": 0, 2024-03-18T23:19:28.182 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_notify": 0, 2024-03-18T23:19:28.182 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_src_cmpxattr": 0, 2024-03-18T23:19:28.183 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_pgls": 0, 2024-03-18T23:19:28.183 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_pgls_filter": 0, 2024-03-18T23:19:28.183 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_other": 0, 2024-03-18T23:19:28.183 INFO:teuthology.orchestra.run.smithi139.stdout: "linger_active": 0, 2024-03-18T23:19:28.183 INFO:teuthology.orchestra.run.smithi139.stdout: "linger_send": 0, 2024-03-18T23:19:28.183 INFO:teuthology.orchestra.run.smithi139.stdout: "linger_resend": 0, 2024-03-18T23:19:28.183 INFO:teuthology.orchestra.run.smithi139.stdout: "linger_ping": 0, 2024-03-18T23:19:28.183 INFO:teuthology.orchestra.run.smithi139.stdout: "poolop_active": 0, 2024-03-18T23:19:28.184 INFO:teuthology.orchestra.run.smithi139.stdout: "poolop_send": 0, 2024-03-18T23:19:28.184 INFO:teuthology.orchestra.run.smithi139.stdout: "poolop_resend": 0, 2024-03-18T23:19:28.184 INFO:teuthology.orchestra.run.smithi139.stdout: "poolstat_active": 0, 2024-03-18T23:19:28.184 INFO:teuthology.orchestra.run.smithi139.stdout: "poolstat_send": 0, 2024-03-18T23:19:28.184 INFO:teuthology.orchestra.run.smithi139.stdout: "poolstat_resend": 0, 2024-03-18T23:19:28.184 INFO:teuthology.orchestra.run.smithi139.stdout: "statfs_active": 0, 2024-03-18T23:19:28.184 INFO:teuthology.orchestra.run.smithi139.stdout: "statfs_send": 0, 2024-03-18T23:19:28.184 INFO:teuthology.orchestra.run.smithi139.stdout: "statfs_resend": 0, 2024-03-18T23:19:28.184 INFO:teuthology.orchestra.run.smithi139.stdout: "command_active": 0, 2024-03-18T23:19:28.185 INFO:teuthology.orchestra.run.smithi139.stdout: "command_send": 0, 2024-03-18T23:19:28.185 INFO:teuthology.orchestra.run.smithi139.stdout: "command_resend": 0, 2024-03-18T23:19:28.185 INFO:teuthology.orchestra.run.smithi139.stdout: "map_epoch": 101, 2024-03-18T23:19:28.185 INFO:teuthology.orchestra.run.smithi139.stdout: "map_full": 0, 2024-03-18T23:19:28.185 INFO:teuthology.orchestra.run.smithi139.stdout: "map_inc": 11, 2024-03-18T23:19:28.185 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_sessions": 11, 2024-03-18T23:19:28.185 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_session_open": 11, 2024-03-18T23:19:28.185 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_session_close": 0, 2024-03-18T23:19:28.185 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_laggy": 0, 2024-03-18T23:19:28.186 INFO:teuthology.orchestra.run.smithi139.stdout: "omap_wr": 0, 2024-03-18T23:19:28.186 INFO:teuthology.orchestra.run.smithi139.stdout: "omap_rd": 2, 2024-03-18T23:19:28.186 INFO:teuthology.orchestra.run.smithi139.stdout: "omap_del": 0 2024-03-18T23:19:28.186 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:28.186 INFO:teuthology.orchestra.run.smithi139.stdout: "oft": { 2024-03-18T23:19:28.186 INFO:teuthology.orchestra.run.smithi139.stdout: "omap_total_objs": 0, 2024-03-18T23:19:28.186 INFO:teuthology.orchestra.run.smithi139.stdout: "omap_total_kv_pairs": 0, 2024-03-18T23:19:28.186 INFO:teuthology.orchestra.run.smithi139.stdout: "omap_total_updates": 0, 2024-03-18T23:19:28.187 INFO:teuthology.orchestra.run.smithi139.stdout: "omap_total_removes": 0 2024-03-18T23:19:28.187 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:28.187 INFO:teuthology.orchestra.run.smithi139.stdout: "osd-slow-ops": { 2024-03-18T23:19:28.187 INFO:teuthology.orchestra.run.smithi139.stdout: "slow_ops_count": 0 2024-03-18T23:19:28.187 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:28.187 INFO:teuthology.orchestra.run.smithi139.stdout: "purge_queue": { 2024-03-18T23:19:28.187 INFO:teuthology.orchestra.run.smithi139.stdout: "pq_executing_ops": 0, 2024-03-18T23:19:28.187 INFO:teuthology.orchestra.run.smithi139.stdout: "pq_executing_ops_high_water": 0, 2024-03-18T23:19:28.187 INFO:teuthology.orchestra.run.smithi139.stdout: "pq_executing": 0, 2024-03-18T23:19:28.188 INFO:teuthology.orchestra.run.smithi139.stdout: "pq_executing_high_water": 0, 2024-03-18T23:19:28.188 INFO:teuthology.orchestra.run.smithi139.stdout: "pq_executed_ops": 0, 2024-03-18T23:19:28.188 INFO:teuthology.orchestra.run.smithi139.stdout: "pq_executed": 0, 2024-03-18T23:19:28.188 INFO:teuthology.orchestra.run.smithi139.stdout: "pq_item_in_journal": 0 2024-03-18T23:19:28.188 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:28.188 INFO:teuthology.orchestra.run.smithi139.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-18T23:19:28.188 INFO:teuthology.orchestra.run.smithi139.stdout: "val": 0, 2024-03-18T23:19:28.188 INFO:teuthology.orchestra.run.smithi139.stdout: "max": 104857600, 2024-03-18T23:19:28.188 INFO:teuthology.orchestra.run.smithi139.stdout: "get_started": 0, 2024-03-18T23:19:28.189 INFO:teuthology.orchestra.run.smithi139.stdout: "get": 2476, 2024-03-18T23:19:28.189 INFO:teuthology.orchestra.run.smithi139.stdout: "get_sum": 511307195, 2024-03-18T23:19:28.189 INFO:teuthology.orchestra.run.smithi139.stdout: "get_or_fail_fail": 0, 2024-03-18T23:19:28.189 INFO:teuthology.orchestra.run.smithi139.stdout: "get_or_fail_success": 2476, 2024-03-18T23:19:28.189 INFO:teuthology.orchestra.run.smithi139.stdout: "take": 0, 2024-03-18T23:19:28.189 INFO:teuthology.orchestra.run.smithi139.stdout: "take_sum": 0, 2024-03-18T23:19:28.189 INFO:teuthology.orchestra.run.smithi139.stdout: "put": 2476, 2024-03-18T23:19:28.189 INFO:teuthology.orchestra.run.smithi139.stdout: "put_sum": 511307195, 2024-03-18T23:19:28.189 INFO:teuthology.orchestra.run.smithi139.stdout: "wait": { 2024-03-18T23:19:28.190 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-03-18T23:19:28.190 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-03-18T23:19:28.190 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-03-18T23:19:28.190 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-03-18T23:19:28.190 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:28.190 INFO:teuthology.orchestra.run.smithi139.stdout: "throttle-objecter_bytes": { 2024-03-18T23:19:28.190 INFO:teuthology.orchestra.run.smithi139.stdout: "val": 0, 2024-03-18T23:19:28.190 INFO:teuthology.orchestra.run.smithi139.stdout: "max": 104857600, 2024-03-18T23:19:28.190 INFO:teuthology.orchestra.run.smithi139.stdout: "get_started": 0, 2024-03-18T23:19:28.191 INFO:teuthology.orchestra.run.smithi139.stdout: "get": 0, 2024-03-18T23:19:28.191 INFO:teuthology.orchestra.run.smithi139.stdout: "get_sum": 0, 2024-03-18T23:19:28.191 INFO:teuthology.orchestra.run.smithi139.stdout: "get_or_fail_fail": 0, 2024-03-18T23:19:28.191 INFO:teuthology.orchestra.run.smithi139.stdout: "get_or_fail_success": 0, 2024-03-18T23:19:28.191 INFO:teuthology.orchestra.run.smithi139.stdout: "take": 2329, 2024-03-18T23:19:28.191 INFO:teuthology.orchestra.run.smithi139.stdout: "take_sum": 510699661, 2024-03-18T23:19:28.191 INFO:teuthology.orchestra.run.smithi139.stdout: "put": 2329, 2024-03-18T23:19:28.191 INFO:teuthology.orchestra.run.smithi139.stdout: "put_sum": 510699661, 2024-03-18T23:19:28.191 INFO:teuthology.orchestra.run.smithi139.stdout: "wait": { 2024-03-18T23:19:28.192 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-03-18T23:19:28.192 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-03-18T23:19:28.192 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-03-18T23:19:28.192 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-03-18T23:19:28.192 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:28.192 INFO:teuthology.orchestra.run.smithi139.stdout: "throttle-objecter_ops": { 2024-03-18T23:19:28.192 INFO:teuthology.orchestra.run.smithi139.stdout: "val": 0, 2024-03-18T23:19:28.192 INFO:teuthology.orchestra.run.smithi139.stdout: "max": 1024, 2024-03-18T23:19:28.192 INFO:teuthology.orchestra.run.smithi139.stdout: "get_started": 0, 2024-03-18T23:19:28.192 INFO:teuthology.orchestra.run.smithi139.stdout: "get": 0, 2024-03-18T23:19:28.193 INFO:teuthology.orchestra.run.smithi139.stdout: "get_sum": 0, 2024-03-18T23:19:28.193 INFO:teuthology.orchestra.run.smithi139.stdout: "get_or_fail_fail": 0, 2024-03-18T23:19:28.193 INFO:teuthology.orchestra.run.smithi139.stdout: "get_or_fail_success": 0, 2024-03-18T23:19:28.193 INFO:teuthology.orchestra.run.smithi139.stdout: "take": 2329, 2024-03-18T23:19:28.193 INFO:teuthology.orchestra.run.smithi139.stdout: "take_sum": 2329, 2024-03-18T23:19:28.193 INFO:teuthology.orchestra.run.smithi139.stdout: "put": 2329, 2024-03-18T23:19:28.193 INFO:teuthology.orchestra.run.smithi139.stdout: "put_sum": 2329, 2024-03-18T23:19:28.193 INFO:teuthology.orchestra.run.smithi139.stdout: "wait": { 2024-03-18T23:19:28.194 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-03-18T23:19:28.194 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-03-18T23:19:28.194 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-03-18T23:19:28.194 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-03-18T23:19:28.194 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:28.194 INFO:teuthology.orchestra.run.smithi139.stdout: "throttle-write_buf_throttle": { 2024-03-18T23:19:28.194 INFO:teuthology.orchestra.run.smithi139.stdout: "val": 0, 2024-03-18T23:19:28.194 INFO:teuthology.orchestra.run.smithi139.stdout: "max": 3758096384, 2024-03-18T23:19:28.194 INFO:teuthology.orchestra.run.smithi139.stdout: "get_started": 0, 2024-03-18T23:19:28.195 INFO:teuthology.orchestra.run.smithi139.stdout: "get": 0, 2024-03-18T23:19:28.195 INFO:teuthology.orchestra.run.smithi139.stdout: "get_sum": 0, 2024-03-18T23:19:28.195 INFO:teuthology.orchestra.run.smithi139.stdout: "get_or_fail_fail": 0, 2024-03-18T23:19:28.195 INFO:teuthology.orchestra.run.smithi139.stdout: "get_or_fail_success": 0, 2024-03-18T23:19:28.195 INFO:teuthology.orchestra.run.smithi139.stdout: "take": 0, 2024-03-18T23:19:28.195 INFO:teuthology.orchestra.run.smithi139.stdout: "take_sum": 0, 2024-03-18T23:19:28.195 INFO:teuthology.orchestra.run.smithi139.stdout: "put": 0, 2024-03-18T23:19:28.195 INFO:teuthology.orchestra.run.smithi139.stdout: "put_sum": 0, 2024-03-18T23:19:28.195 INFO:teuthology.orchestra.run.smithi139.stdout: "wait": { 2024-03-18T23:19:28.196 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-03-18T23:19:28.196 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-03-18T23:19:28.196 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-03-18T23:19:28.196 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-03-18T23:19:28.196 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:28.196 INFO:teuthology.orchestra.run.smithi139.stdout: "throttle-write_buf_throttle-0x55fce40206e0": { 2024-03-18T23:19:28.196 INFO:teuthology.orchestra.run.smithi139.stdout: "val": 0, 2024-03-18T23:19:28.196 INFO:teuthology.orchestra.run.smithi139.stdout: "max": 3758096384, 2024-03-18T23:19:28.196 INFO:teuthology.orchestra.run.smithi139.stdout: "get_started": 0, 2024-03-18T23:19:28.197 INFO:teuthology.orchestra.run.smithi139.stdout: "get": 0, 2024-03-18T23:19:28.197 INFO:teuthology.orchestra.run.smithi139.stdout: "get_sum": 0, 2024-03-18T23:19:28.197 INFO:teuthology.orchestra.run.smithi139.stdout: "get_or_fail_fail": 0, 2024-03-18T23:19:28.197 INFO:teuthology.orchestra.run.smithi139.stdout: "get_or_fail_success": 0, 2024-03-18T23:19:28.197 INFO:teuthology.orchestra.run.smithi139.stdout: "take": 0, 2024-03-18T23:19:28.197 INFO:teuthology.orchestra.run.smithi139.stdout: "take_sum": 0, 2024-03-18T23:19:28.197 INFO:teuthology.orchestra.run.smithi139.stdout: "put": 0, 2024-03-18T23:19:28.197 INFO:teuthology.orchestra.run.smithi139.stdout: "put_sum": 0, 2024-03-18T23:19:28.197 INFO:teuthology.orchestra.run.smithi139.stdout: "wait": { 2024-03-18T23:19:28.198 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-03-18T23:19:28.198 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-03-18T23:19:28.198 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-03-18T23:19:28.198 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-03-18T23:19:28.198 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-03-18T23:19:28.198 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-03-18T23:19:28.948 INFO:tasks.check_counter:Daemon mds.h mds.exported=0 2024-03-18T23:19:28.948 INFO:tasks.check_counter:Daemon mds.h mds.imported=0 2024-03-18T23:19:28.948 INFO:tasks.check_counter:Daemon mds.h mds.root_rsnaps=6 2024-03-18T23:19:28.948 INFO:tasks.check_counter:Daemon mds.h mds_server.req_mksnap_latency.avgcount=0 2024-03-18T23:19:28.949 INFO:tasks.check_counter:Daemon mds.h mds_server.req_rmsnap_latency.avgcount=0 2024-03-18T23:19:28.949 DEBUG:tasks.check_counter:Getting stats from k 2024-03-18T23:19:28.949 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell --fsid c63591ea-e57a-11ee-95c9-87774f69a715 -- ceph daemon mds.k perf dump 2024-03-18T23:19:29.201 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/mon.b/config 2024-03-18T23:19:29.477 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:19:29 smithi139 ceph-mon[67282]: pgmap v351: 97 pgs: 97 active+clean; 66 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 1.4 MiB/s wr, 65 op/s 2024-03-18T23:19:29.629 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:19:29 smithi049 ceph-mon[62991]: pgmap v351: 97 pgs: 97 active+clean; 66 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 1.4 MiB/s wr, 65 op/s 2024-03-18T23:19:29.710 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:19:29 smithi175 ceph-mon[67352]: pgmap v351: 97 pgs: 97 active+clean; 66 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 1.4 MiB/s wr, 65 op/s 2024-03-18T23:19:30.993 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-03-18T23:19:30.993 INFO:teuthology.orchestra.run.smithi139.stdout: "AsyncMessenger::Worker-0": { 2024-03-18T23:19:30.993 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_recv_messages": 871, 2024-03-18T23:19:30.993 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_messages": 873, 2024-03-18T23:19:30.993 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_recv_bytes": 478101, 2024-03-18T23:19:30.994 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_bytes": 277042, 2024-03-18T23:19:30.994 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_created_connections": 6, 2024-03-18T23:19:30.994 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_active_connections": 2, 2024-03-18T23:19:30.994 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_total_time": 0.057923469, 2024-03-18T23:19:30.994 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_send_time": 0.021026238, 2024-03-18T23:19:30.994 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_recv_time": 0.014165044, 2024-03-18T23:19:30.994 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_fast_dispatch_time": 0.017250122, 2024-03-18T23:19:30.995 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_messages_queue_lat": { 2024-03-18T23:19:30.995 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 871, 2024-03-18T23:19:30.995 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.019581026, 2024-03-18T23:19:30.995 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000022481 2024-03-18T23:19:30.995 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:30.995 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_handle_ack_lat": { 2024-03-18T23:19:30.996 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-03-18T23:19:30.996 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-03-18T23:19:30.996 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-03-18T23:19:30.996 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:30.996 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-18T23:19:30.996 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_encrypted_bytes": 0 2024-03-18T23:19:30.996 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:30.997 INFO:teuthology.orchestra.run.smithi139.stdout: "AsyncMessenger::Worker-1": { 2024-03-18T23:19:30.997 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_recv_messages": 263, 2024-03-18T23:19:30.997 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_messages": 383, 2024-03-18T23:19:30.997 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_recv_bytes": 1499650, 2024-03-18T23:19:30.997 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_bytes": 471889, 2024-03-18T23:19:30.997 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_created_connections": 4, 2024-03-18T23:19:30.997 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_active_connections": 3, 2024-03-18T23:19:30.998 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_total_time": 0.044814704, 2024-03-18T23:19:30.998 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_send_time": 0.013576572, 2024-03-18T23:19:30.998 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_recv_time": 0.037068274, 2024-03-18T23:19:30.998 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_fast_dispatch_time": 0.004278655, 2024-03-18T23:19:30.998 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_messages_queue_lat": { 2024-03-18T23:19:30.998 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 382, 2024-03-18T23:19:30.998 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.016210877, 2024-03-18T23:19:30.999 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000042436 2024-03-18T23:19:30.999 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:30.999 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_handle_ack_lat": { 2024-03-18T23:19:30.999 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-03-18T23:19:30.999 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-03-18T23:19:30.999 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-03-18T23:19:30.999 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:30.999 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_recv_encrypted_bytes": 1499136, 2024-03-18T23:19:31.000 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_encrypted_bytes": 471248 2024-03-18T23:19:31.000 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:31.000 INFO:teuthology.orchestra.run.smithi139.stdout: "AsyncMessenger::Worker-2": { 2024-03-18T23:19:31.000 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_recv_messages": 1477, 2024-03-18T23:19:31.000 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_messages": 1510, 2024-03-18T23:19:31.000 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_recv_bytes": 437116, 2024-03-18T23:19:31.000 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_bytes": 600512, 2024-03-18T23:19:31.000 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_created_connections": 6, 2024-03-18T23:19:31.001 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_active_connections": 4, 2024-03-18T23:19:31.001 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_total_time": 0.102774387, 2024-03-18T23:19:31.001 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_send_time": 0.028639840, 2024-03-18T23:19:31.001 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_recv_time": 0.025217042, 2024-03-18T23:19:31.001 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_fast_dispatch_time": 0.039896407, 2024-03-18T23:19:31.001 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_messages_queue_lat": { 2024-03-18T23:19:31.001 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 1508, 2024-03-18T23:19:31.002 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.035337600, 2024-03-18T23:19:31.002 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000023433 2024-03-18T23:19:31.002 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:31.002 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_handle_ack_lat": { 2024-03-18T23:19:31.002 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-03-18T23:19:31.002 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-03-18T23:19:31.002 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-03-18T23:19:31.002 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:31.003 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_recv_encrypted_bytes": 3184, 2024-03-18T23:19:31.003 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_encrypted_bytes": 132416 2024-03-18T23:19:31.003 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:31.003 INFO:teuthology.orchestra.run.smithi139.stdout: "cct": { 2024-03-18T23:19:31.003 INFO:teuthology.orchestra.run.smithi139.stdout: "total_workers": 1, 2024-03-18T23:19:31.003 INFO:teuthology.orchestra.run.smithi139.stdout: "unhealthy_workers": 0 2024-03-18T23:19:31.003 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:31.003 INFO:teuthology.orchestra.run.smithi139.stdout: "finisher-MDSRank": { 2024-03-18T23:19:31.003 INFO:teuthology.orchestra.run.smithi139.stdout: "queue_len": 0, 2024-03-18T23:19:31.004 INFO:teuthology.orchestra.run.smithi139.stdout: "complete_latency": { 2024-03-18T23:19:31.004 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 3975, 2024-03-18T23:19:31.004 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.086844644, 2024-03-18T23:19:31.004 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000021847 2024-03-18T23:19:31.004 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-03-18T23:19:31.004 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:31.004 INFO:teuthology.orchestra.run.smithi139.stdout: "finisher-PurgeQueue": { 2024-03-18T23:19:31.004 INFO:teuthology.orchestra.run.smithi139.stdout: "queue_len": 0, 2024-03-18T23:19:31.005 INFO:teuthology.orchestra.run.smithi139.stdout: "complete_latency": { 2024-03-18T23:19:31.005 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-03-18T23:19:31.005 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-03-18T23:19:31.005 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-03-18T23:19:31.005 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-03-18T23:19:31.005 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:31.005 INFO:teuthology.orchestra.run.smithi139.stdout: "mds": { 2024-03-18T23:19:31.005 INFO:teuthology.orchestra.run.smithi139.stdout: "request": 0, 2024-03-18T23:19:31.005 INFO:teuthology.orchestra.run.smithi139.stdout: "reply": 0, 2024-03-18T23:19:31.006 INFO:teuthology.orchestra.run.smithi139.stdout: "reply_latency": { 2024-03-18T23:19:31.006 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-03-18T23:19:31.006 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-03-18T23:19:31.006 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-03-18T23:19:31.006 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:31.006 INFO:teuthology.orchestra.run.smithi139.stdout: "slow_reply": 0, 2024-03-18T23:19:31.006 INFO:teuthology.orchestra.run.smithi139.stdout: "forward": 0, 2024-03-18T23:19:31.006 INFO:teuthology.orchestra.run.smithi139.stdout: "dir_fetch_complete": 0, 2024-03-18T23:19:31.007 INFO:teuthology.orchestra.run.smithi139.stdout: "dir_fetch_keys": 0, 2024-03-18T23:19:31.007 INFO:teuthology.orchestra.run.smithi139.stdout: "dir_commit": 0, 2024-03-18T23:19:31.007 INFO:teuthology.orchestra.run.smithi139.stdout: "dir_split": 0, 2024-03-18T23:19:31.007 INFO:teuthology.orchestra.run.smithi139.stdout: "dir_merge": 0, 2024-03-18T23:19:31.007 INFO:teuthology.orchestra.run.smithi139.stdout: "inodes": 53, 2024-03-18T23:19:31.007 INFO:teuthology.orchestra.run.smithi139.stdout: "inodes_top": 0, 2024-03-18T23:19:31.007 INFO:teuthology.orchestra.run.smithi139.stdout: "inodes_bottom": 53, 2024-03-18T23:19:31.007 INFO:teuthology.orchestra.run.smithi139.stdout: "inodes_pin_tail": 0, 2024-03-18T23:19:31.008 INFO:teuthology.orchestra.run.smithi139.stdout: "inodes_pinned": 53, 2024-03-18T23:19:31.008 INFO:teuthology.orchestra.run.smithi139.stdout: "inodes_expired": 0, 2024-03-18T23:19:31.009 INFO:teuthology.orchestra.run.smithi139.stdout: "inodes_with_caps": 0, 2024-03-18T23:19:31.009 INFO:teuthology.orchestra.run.smithi139.stdout: "caps": 0, 2024-03-18T23:19:31.009 INFO:teuthology.orchestra.run.smithi139.stdout: "subtrees": 3, 2024-03-18T23:19:31.009 INFO:teuthology.orchestra.run.smithi139.stdout: "traverse": 0, 2024-03-18T23:19:31.009 INFO:teuthology.orchestra.run.smithi139.stdout: "traverse_hit": 0, 2024-03-18T23:19:31.010 INFO:teuthology.orchestra.run.smithi139.stdout: "traverse_forward": 0, 2024-03-18T23:19:31.010 INFO:teuthology.orchestra.run.smithi139.stdout: "traverse_discover": 0, 2024-03-18T23:19:31.010 INFO:teuthology.orchestra.run.smithi139.stdout: "traverse_dir_fetch": 0, 2024-03-18T23:19:31.010 INFO:teuthology.orchestra.run.smithi139.stdout: "traverse_remote_ino": 0, 2024-03-18T23:19:31.010 INFO:teuthology.orchestra.run.smithi139.stdout: "traverse_lock": 0, 2024-03-18T23:19:31.010 INFO:teuthology.orchestra.run.smithi139.stdout: "load_cent": 0, 2024-03-18T23:19:31.010 INFO:teuthology.orchestra.run.smithi139.stdout: "q": 0, 2024-03-18T23:19:31.010 INFO:teuthology.orchestra.run.smithi139.stdout: "exported": 0, 2024-03-18T23:19:31.010 INFO:teuthology.orchestra.run.smithi139.stdout: "exported_inodes": 0, 2024-03-18T23:19:31.011 INFO:teuthology.orchestra.run.smithi139.stdout: "imported": 0, 2024-03-18T23:19:31.011 INFO:teuthology.orchestra.run.smithi139.stdout: "imported_inodes": 0, 2024-03-18T23:19:31.011 INFO:teuthology.orchestra.run.smithi139.stdout: "openino_dir_fetch": 0, 2024-03-18T23:19:31.011 INFO:teuthology.orchestra.run.smithi139.stdout: "openino_backtrace_fetch": 0, 2024-03-18T23:19:31.011 INFO:teuthology.orchestra.run.smithi139.stdout: "openino_peer_discover": 0, 2024-03-18T23:19:31.011 INFO:teuthology.orchestra.run.smithi139.stdout: "root_rfiles": 0, 2024-03-18T23:19:31.011 INFO:teuthology.orchestra.run.smithi139.stdout: "root_rbytes": 0, 2024-03-18T23:19:31.011 INFO:teuthology.orchestra.run.smithi139.stdout: "root_rsnaps": 0, 2024-03-18T23:19:31.012 INFO:teuthology.orchestra.run.smithi139.stdout: "scrub_backtrace_fetch": 0, 2024-03-18T23:19:31.012 INFO:teuthology.orchestra.run.smithi139.stdout: "scrub_set_tag": 0, 2024-03-18T23:19:31.012 INFO:teuthology.orchestra.run.smithi139.stdout: "scrub_backtrace_repaired": 0, 2024-03-18T23:19:31.012 INFO:teuthology.orchestra.run.smithi139.stdout: "scrub_inotable_repaired": 0, 2024-03-18T23:19:31.012 INFO:teuthology.orchestra.run.smithi139.stdout: "scrub_dir_inodes": 0, 2024-03-18T23:19:31.012 INFO:teuthology.orchestra.run.smithi139.stdout: "scrub_dir_base_inodes": 0, 2024-03-18T23:19:31.012 INFO:teuthology.orchestra.run.smithi139.stdout: "scrub_dirfrag_rstats": 0, 2024-03-18T23:19:31.012 INFO:teuthology.orchestra.run.smithi139.stdout: "scrub_file_inodes": 0, 2024-03-18T23:19:31.013 INFO:teuthology.orchestra.run.smithi139.stdout: "handle_inode_file_caps": 0, 2024-03-18T23:19:31.013 INFO:teuthology.orchestra.run.smithi139.stdout: "ceph_cap_op_revoke": 0, 2024-03-18T23:19:31.013 INFO:teuthology.orchestra.run.smithi139.stdout: "ceph_cap_op_grant": 0, 2024-03-18T23:19:31.013 INFO:teuthology.orchestra.run.smithi139.stdout: "ceph_cap_op_trunc": 0, 2024-03-18T23:19:31.013 INFO:teuthology.orchestra.run.smithi139.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-18T23:19:31.013 INFO:teuthology.orchestra.run.smithi139.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-18T23:19:31.013 INFO:teuthology.orchestra.run.smithi139.stdout: "handle_client_caps": 0, 2024-03-18T23:19:31.013 INFO:teuthology.orchestra.run.smithi139.stdout: "handle_client_caps_dirty": 0, 2024-03-18T23:19:31.013 INFO:teuthology.orchestra.run.smithi139.stdout: "handle_client_cap_release": 0, 2024-03-18T23:19:31.014 INFO:teuthology.orchestra.run.smithi139.stdout: "process_request_cap_release": 0 2024-03-18T23:19:31.014 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:31.014 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_cache": { 2024-03-18T23:19:31.014 INFO:teuthology.orchestra.run.smithi139.stdout: "dir_update": 0, 2024-03-18T23:19:31.014 INFO:teuthology.orchestra.run.smithi139.stdout: "dir_update_receipt": 0, 2024-03-18T23:19:31.014 INFO:teuthology.orchestra.run.smithi139.stdout: "dir_try_discover": 0, 2024-03-18T23:19:31.014 INFO:teuthology.orchestra.run.smithi139.stdout: "dir_send_discover": 0, 2024-03-18T23:19:31.015 INFO:teuthology.orchestra.run.smithi139.stdout: "dir_handle_discover": 0, 2024-03-18T23:19:31.015 INFO:teuthology.orchestra.run.smithi139.stdout: "num_strays": 0, 2024-03-18T23:19:31.015 INFO:teuthology.orchestra.run.smithi139.stdout: "num_strays_delayed": 0, 2024-03-18T23:19:31.015 INFO:teuthology.orchestra.run.smithi139.stdout: "num_strays_enqueuing": 0, 2024-03-18T23:19:31.015 INFO:teuthology.orchestra.run.smithi139.stdout: "strays_created": 0, 2024-03-18T23:19:31.015 INFO:teuthology.orchestra.run.smithi139.stdout: "strays_enqueued": 0, 2024-03-18T23:19:31.015 INFO:teuthology.orchestra.run.smithi139.stdout: "strays_reintegrated": 0, 2024-03-18T23:19:31.015 INFO:teuthology.orchestra.run.smithi139.stdout: "strays_migrated": 0, 2024-03-18T23:19:31.015 INFO:teuthology.orchestra.run.smithi139.stdout: "num_recovering_processing": 0, 2024-03-18T23:19:31.016 INFO:teuthology.orchestra.run.smithi139.stdout: "num_recovering_enqueued": 0, 2024-03-18T23:19:31.016 INFO:teuthology.orchestra.run.smithi139.stdout: "num_recovering_prioritized": 0, 2024-03-18T23:19:31.016 INFO:teuthology.orchestra.run.smithi139.stdout: "recovery_started": 0, 2024-03-18T23:19:31.016 INFO:teuthology.orchestra.run.smithi139.stdout: "recovery_completed": 0, 2024-03-18T23:19:31.016 INFO:teuthology.orchestra.run.smithi139.stdout: "ireq_quiesce_path": 0, 2024-03-18T23:19:31.016 INFO:teuthology.orchestra.run.smithi139.stdout: "ireq_quiesce_inode": 0, 2024-03-18T23:19:31.016 INFO:teuthology.orchestra.run.smithi139.stdout: "ireq_enqueue_scrub": 0, 2024-03-18T23:19:31.016 INFO:teuthology.orchestra.run.smithi139.stdout: "ireq_exportdir": 0, 2024-03-18T23:19:31.017 INFO:teuthology.orchestra.run.smithi139.stdout: "ireq_flush": 0, 2024-03-18T23:19:31.017 INFO:teuthology.orchestra.run.smithi139.stdout: "ireq_fragmentdir": 0, 2024-03-18T23:19:31.017 INFO:teuthology.orchestra.run.smithi139.stdout: "ireq_fragstats": 0, 2024-03-18T23:19:31.017 INFO:teuthology.orchestra.run.smithi139.stdout: "ireq_inodestats": 0 2024-03-18T23:19:31.017 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:31.017 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_log": { 2024-03-18T23:19:31.017 INFO:teuthology.orchestra.run.smithi139.stdout: "evlrg": 0, 2024-03-18T23:19:31.017 INFO:teuthology.orchestra.run.smithi139.stdout: "evadd": 0, 2024-03-18T23:19:31.017 INFO:teuthology.orchestra.run.smithi139.stdout: "evex": 0, 2024-03-18T23:19:31.018 INFO:teuthology.orchestra.run.smithi139.stdout: "evtrm": 0, 2024-03-18T23:19:31.018 INFO:teuthology.orchestra.run.smithi139.stdout: "ev": 184, 2024-03-18T23:19:31.018 INFO:teuthology.orchestra.run.smithi139.stdout: "evexg": 0, 2024-03-18T23:19:31.018 INFO:teuthology.orchestra.run.smithi139.stdout: "evexd": 0, 2024-03-18T23:19:31.018 INFO:teuthology.orchestra.run.smithi139.stdout: "segadd": 0, 2024-03-18T23:19:31.018 INFO:teuthology.orchestra.run.smithi139.stdout: "segex": 0, 2024-03-18T23:19:31.018 INFO:teuthology.orchestra.run.smithi139.stdout: "segtrm": 0, 2024-03-18T23:19:31.018 INFO:teuthology.orchestra.run.smithi139.stdout: "seg": 2, 2024-03-18T23:19:31.019 INFO:teuthology.orchestra.run.smithi139.stdout: "segmjr": 2, 2024-03-18T23:19:31.019 INFO:teuthology.orchestra.run.smithi139.stdout: "segexg": 0, 2024-03-18T23:19:31.019 INFO:teuthology.orchestra.run.smithi139.stdout: "segexd": 0, 2024-03-18T23:19:31.019 INFO:teuthology.orchestra.run.smithi139.stdout: "expos": 4194304, 2024-03-18T23:19:31.019 INFO:teuthology.orchestra.run.smithi139.stdout: "wrpos": 4444684, 2024-03-18T23:19:31.019 INFO:teuthology.orchestra.run.smithi139.stdout: "rdpos": 4444110, 2024-03-18T23:19:31.019 INFO:teuthology.orchestra.run.smithi139.stdout: "jlat": { 2024-03-18T23:19:31.019 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-03-18T23:19:31.019 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-03-18T23:19:31.020 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-03-18T23:19:31.020 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:31.020 INFO:teuthology.orchestra.run.smithi139.stdout: "replayed": 184 2024-03-18T23:19:31.020 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:31.020 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_mem": { 2024-03-18T23:19:31.020 INFO:teuthology.orchestra.run.smithi139.stdout: "ino": 50, 2024-03-18T23:19:31.020 INFO:teuthology.orchestra.run.smithi139.stdout: "ino+": 56, 2024-03-18T23:19:31.020 INFO:teuthology.orchestra.run.smithi139.stdout: "ino-": 6, 2024-03-18T23:19:31.020 INFO:teuthology.orchestra.run.smithi139.stdout: "dir": 13, 2024-03-18T23:19:31.021 INFO:teuthology.orchestra.run.smithi139.stdout: "dir+": 13, 2024-03-18T23:19:31.021 INFO:teuthology.orchestra.run.smithi139.stdout: "dir-": 0, 2024-03-18T23:19:31.021 INFO:teuthology.orchestra.run.smithi139.stdout: "dn": 53, 2024-03-18T23:19:31.021 INFO:teuthology.orchestra.run.smithi139.stdout: "dn+": 53, 2024-03-18T23:19:31.021 INFO:teuthology.orchestra.run.smithi139.stdout: "dn-": 0, 2024-03-18T23:19:31.021 INFO:teuthology.orchestra.run.smithi139.stdout: "cap": 0, 2024-03-18T23:19:31.021 INFO:teuthology.orchestra.run.smithi139.stdout: "cap+": 0, 2024-03-18T23:19:31.021 INFO:teuthology.orchestra.run.smithi139.stdout: "cap-": 0, 2024-03-18T23:19:31.021 INFO:teuthology.orchestra.run.smithi139.stdout: "rss": 50252, 2024-03-18T23:19:31.022 INFO:teuthology.orchestra.run.smithi139.stdout: "heap": 215420 2024-03-18T23:19:31.022 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:31.022 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_server": { 2024-03-18T23:19:31.022 INFO:teuthology.orchestra.run.smithi139.stdout: "dispatch_client_request": 0, 2024-03-18T23:19:31.022 INFO:teuthology.orchestra.run.smithi139.stdout: "dispatch_server_request": 0, 2024-03-18T23:19:31.022 INFO:teuthology.orchestra.run.smithi139.stdout: "handle_client_request": 0, 2024-03-18T23:19:31.022 INFO:teuthology.orchestra.run.smithi139.stdout: "handle_client_session": 0, 2024-03-18T23:19:31.022 INFO:teuthology.orchestra.run.smithi139.stdout: "handle_peer_request": 0, 2024-03-18T23:19:31.022 INFO:teuthology.orchestra.run.smithi139.stdout: "req_create_latency": { 2024-03-18T23:19:31.023 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-03-18T23:19:31.023 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-03-18T23:19:31.023 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-03-18T23:19:31.023 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:31.023 INFO:teuthology.orchestra.run.smithi139.stdout: "req_getattr_latency": { 2024-03-18T23:19:31.023 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-03-18T23:19:31.023 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-03-18T23:19:31.023 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-03-18T23:19:31.024 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:31.024 INFO:teuthology.orchestra.run.smithi139.stdout: "req_getfilelock_latency": { 2024-03-18T23:19:31.024 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-03-18T23:19:31.024 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-03-18T23:19:31.024 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-03-18T23:19:31.024 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:31.024 INFO:teuthology.orchestra.run.smithi139.stdout: "req_link_latency": { 2024-03-18T23:19:31.024 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-03-18T23:19:31.024 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-03-18T23:19:31.025 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-03-18T23:19:31.025 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:31.025 INFO:teuthology.orchestra.run.smithi139.stdout: "req_lookup_latency": { 2024-03-18T23:19:31.025 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-03-18T23:19:31.025 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-03-18T23:19:31.025 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-03-18T23:19:31.025 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:31.025 INFO:teuthology.orchestra.run.smithi139.stdout: "req_lookuphash_latency": { 2024-03-18T23:19:31.025 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-03-18T23:19:31.026 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-03-18T23:19:31.026 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-03-18T23:19:31.026 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:31.026 INFO:teuthology.orchestra.run.smithi139.stdout: "req_lookupino_latency": { 2024-03-18T23:19:31.026 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-03-18T23:19:31.026 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-03-18T23:19:31.026 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-03-18T23:19:31.026 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:31.027 INFO:teuthology.orchestra.run.smithi139.stdout: "req_lookupname_latency": { 2024-03-18T23:19:31.027 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-03-18T23:19:31.027 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-03-18T23:19:31.027 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-03-18T23:19:31.027 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:31.027 INFO:teuthology.orchestra.run.smithi139.stdout: "req_lookupparent_latency": { 2024-03-18T23:19:31.027 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-03-18T23:19:31.027 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-03-18T23:19:31.028 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-03-18T23:19:31.028 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:31.028 INFO:teuthology.orchestra.run.smithi139.stdout: "req_lookupsnap_latency": { 2024-03-18T23:19:31.028 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-03-18T23:19:31.028 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-03-18T23:19:31.028 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-03-18T23:19:31.028 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:31.028 INFO:teuthology.orchestra.run.smithi139.stdout: "req_lssnap_latency": { 2024-03-18T23:19:31.028 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-03-18T23:19:31.029 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-03-18T23:19:31.029 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-03-18T23:19:31.029 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:31.029 INFO:teuthology.orchestra.run.smithi139.stdout: "req_mkdir_latency": { 2024-03-18T23:19:31.029 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-03-18T23:19:31.030 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-03-18T23:19:31.030 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-03-18T23:19:31.030 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:31.030 INFO:teuthology.orchestra.run.smithi139.stdout: "req_mknod_latency": { 2024-03-18T23:19:31.030 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-03-18T23:19:31.031 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-03-18T23:19:31.031 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-03-18T23:19:31.031 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:31.031 INFO:teuthology.orchestra.run.smithi139.stdout: "req_mksnap_latency": { 2024-03-18T23:19:31.031 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-03-18T23:19:31.031 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-03-18T23:19:31.031 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-03-18T23:19:31.031 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:31.031 INFO:teuthology.orchestra.run.smithi139.stdout: "req_open_latency": { 2024-03-18T23:19:31.032 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-03-18T23:19:31.032 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-03-18T23:19:31.032 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-03-18T23:19:31.032 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:31.032 INFO:teuthology.orchestra.run.smithi139.stdout: "req_readdir_latency": { 2024-03-18T23:19:31.032 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-03-18T23:19:31.032 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-03-18T23:19:31.032 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-03-18T23:19:31.032 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:31.033 INFO:teuthology.orchestra.run.smithi139.stdout: "req_rename_latency": { 2024-03-18T23:19:31.033 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-03-18T23:19:31.033 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-03-18T23:19:31.033 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-03-18T23:19:31.033 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:31.033 INFO:teuthology.orchestra.run.smithi139.stdout: "req_renamesnap_latency": { 2024-03-18T23:19:31.033 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-03-18T23:19:31.033 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-03-18T23:19:31.034 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-03-18T23:19:31.034 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:31.034 INFO:teuthology.orchestra.run.smithi139.stdout: "req_snapdiff_latency": { 2024-03-18T23:19:31.034 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-03-18T23:19:31.034 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-03-18T23:19:31.034 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-03-18T23:19:31.034 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:31.034 INFO:teuthology.orchestra.run.smithi139.stdout: "req_rmdir_latency": { 2024-03-18T23:19:31.035 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-03-18T23:19:31.035 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-03-18T23:19:31.035 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-03-18T23:19:31.035 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:31.035 INFO:teuthology.orchestra.run.smithi139.stdout: "req_rmsnap_latency": { 2024-03-18T23:19:31.035 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-03-18T23:19:31.035 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-03-18T23:19:31.035 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-03-18T23:19:31.035 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:31.036 INFO:teuthology.orchestra.run.smithi139.stdout: "req_rmxattr_latency": { 2024-03-18T23:19:31.036 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-03-18T23:19:31.036 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-03-18T23:19:31.036 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-03-18T23:19:31.036 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:31.036 INFO:teuthology.orchestra.run.smithi139.stdout: "req_setattr_latency": { 2024-03-18T23:19:31.036 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-03-18T23:19:31.036 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-03-18T23:19:31.036 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-03-18T23:19:31.037 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:31.037 INFO:teuthology.orchestra.run.smithi139.stdout: "req_setdirlayout_latency": { 2024-03-18T23:19:31.037 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-03-18T23:19:31.037 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-03-18T23:19:31.037 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-03-18T23:19:31.037 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:31.037 INFO:teuthology.orchestra.run.smithi139.stdout: "req_setfilelock_latency": { 2024-03-18T23:19:31.037 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-03-18T23:19:31.037 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-03-18T23:19:31.038 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-03-18T23:19:31.038 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:31.038 INFO:teuthology.orchestra.run.smithi139.stdout: "req_setlayout_latency": { 2024-03-18T23:19:31.038 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-03-18T23:19:31.038 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-03-18T23:19:31.038 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-03-18T23:19:31.038 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:31.038 INFO:teuthology.orchestra.run.smithi139.stdout: "req_setxattr_latency": { 2024-03-18T23:19:31.039 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-03-18T23:19:31.039 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-03-18T23:19:31.039 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-03-18T23:19:31.039 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:31.039 INFO:teuthology.orchestra.run.smithi139.stdout: "req_symlink_latency": { 2024-03-18T23:19:31.039 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-03-18T23:19:31.039 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-03-18T23:19:31.039 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-03-18T23:19:31.040 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:31.040 INFO:teuthology.orchestra.run.smithi139.stdout: "req_unlink_latency": { 2024-03-18T23:19:31.040 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-03-18T23:19:31.040 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-03-18T23:19:31.040 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-03-18T23:19:31.040 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:31.040 INFO:teuthology.orchestra.run.smithi139.stdout: "cap_revoke_eviction": 0, 2024-03-18T23:19:31.040 INFO:teuthology.orchestra.run.smithi139.stdout: "cap_acquisition_throttle": 0, 2024-03-18T23:19:31.040 INFO:teuthology.orchestra.run.smithi139.stdout: "req_getvxattr_latency": { 2024-03-18T23:19:31.041 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-03-18T23:19:31.041 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-03-18T23:19:31.041 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-03-18T23:19:31.041 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-03-18T23:19:31.041 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:31.041 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_sessions": { 2024-03-18T23:19:31.041 INFO:teuthology.orchestra.run.smithi139.stdout: "session_count": 1, 2024-03-18T23:19:31.041 INFO:teuthology.orchestra.run.smithi139.stdout: "session_add": 2, 2024-03-18T23:19:31.041 INFO:teuthology.orchestra.run.smithi139.stdout: "session_remove": 1, 2024-03-18T23:19:31.042 INFO:teuthology.orchestra.run.smithi139.stdout: "sessions_open": 2, 2024-03-18T23:19:31.042 INFO:teuthology.orchestra.run.smithi139.stdout: "sessions_stale": 0, 2024-03-18T23:19:31.042 INFO:teuthology.orchestra.run.smithi139.stdout: "total_load": 0, 2024-03-18T23:19:31.042 INFO:teuthology.orchestra.run.smithi139.stdout: "average_load": 0, 2024-03-18T23:19:31.042 INFO:teuthology.orchestra.run.smithi139.stdout: "avg_session_uptime": 11826, 2024-03-18T23:19:31.042 INFO:teuthology.orchestra.run.smithi139.stdout: "mdthresh_evicted": 0 2024-03-18T23:19:31.042 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:31.042 INFO:teuthology.orchestra.run.smithi139.stdout: "mempool": { 2024-03-18T23:19:31.043 INFO:teuthology.orchestra.run.smithi139.stdout: "bloom_filter_bytes": 0, 2024-03-18T23:19:31.043 INFO:teuthology.orchestra.run.smithi139.stdout: "bloom_filter_items": 0, 2024-03-18T23:19:31.043 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_alloc_bytes": 0, 2024-03-18T23:19:31.043 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_alloc_items": 0, 2024-03-18T23:19:31.043 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_cache_data_bytes": 0, 2024-03-18T23:19:31.043 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_cache_data_items": 0, 2024-03-18T23:19:31.043 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-18T23:19:31.043 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_cache_onode_items": 0, 2024-03-18T23:19:31.043 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-18T23:19:31.044 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_cache_meta_items": 0, 2024-03-18T23:19:31.044 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_cache_other_bytes": 0, 2024-03-18T23:19:31.044 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_cache_other_items": 0, 2024-03-18T23:19:31.044 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-18T23:19:31.044 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_cache_buffer_items": 0, 2024-03-18T23:19:31.044 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_extent_bytes": 0, 2024-03-18T23:19:31.044 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_extent_items": 0, 2024-03-18T23:19:31.044 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_blob_bytes": 0, 2024-03-18T23:19:31.045 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_blob_items": 0, 2024-03-18T23:19:31.045 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-18T23:19:31.045 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_shared_blob_items": 0, 2024-03-18T23:19:31.045 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-18T23:19:31.045 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_inline_bl_items": 0, 2024-03-18T23:19:31.045 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_fsck_bytes": 0, 2024-03-18T23:19:31.045 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_fsck_items": 0, 2024-03-18T23:19:31.045 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_txc_bytes": 0, 2024-03-18T23:19:31.045 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_txc_items": 0, 2024-03-18T23:19:31.046 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-18T23:19:31.046 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_writing_deferred_items": 0, 2024-03-18T23:19:31.046 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_writing_bytes": 0, 2024-03-18T23:19:31.046 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_writing_items": 0, 2024-03-18T23:19:31.046 INFO:teuthology.orchestra.run.smithi139.stdout: "bluefs_bytes": 0, 2024-03-18T23:19:31.046 INFO:teuthology.orchestra.run.smithi139.stdout: "bluefs_items": 0, 2024-03-18T23:19:31.046 INFO:teuthology.orchestra.run.smithi139.stdout: "bluefs_file_reader_bytes": 0, 2024-03-18T23:19:31.046 INFO:teuthology.orchestra.run.smithi139.stdout: "bluefs_file_reader_items": 0, 2024-03-18T23:19:31.047 INFO:teuthology.orchestra.run.smithi139.stdout: "bluefs_file_writer_bytes": 0, 2024-03-18T23:19:31.047 INFO:teuthology.orchestra.run.smithi139.stdout: "bluefs_file_writer_items": 0, 2024-03-18T23:19:31.047 INFO:teuthology.orchestra.run.smithi139.stdout: "buffer_anon_bytes": 191149, 2024-03-18T23:19:31.047 INFO:teuthology.orchestra.run.smithi139.stdout: "buffer_anon_items": 99, 2024-03-18T23:19:31.047 INFO:teuthology.orchestra.run.smithi139.stdout: "buffer_meta_bytes": 0, 2024-03-18T23:19:31.047 INFO:teuthology.orchestra.run.smithi139.stdout: "buffer_meta_items": 0, 2024-03-18T23:19:31.047 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_bytes": 0, 2024-03-18T23:19:31.047 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_items": 0, 2024-03-18T23:19:31.048 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_mapbl_bytes": 0, 2024-03-18T23:19:31.048 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_mapbl_items": 0, 2024-03-18T23:19:31.048 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_pglog_bytes": 0, 2024-03-18T23:19:31.048 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_pglog_items": 0, 2024-03-18T23:19:31.048 INFO:teuthology.orchestra.run.smithi139.stdout: "osdmap_bytes": 8256, 2024-03-18T23:19:31.048 INFO:teuthology.orchestra.run.smithi139.stdout: "osdmap_items": 188, 2024-03-18T23:19:31.048 INFO:teuthology.orchestra.run.smithi139.stdout: "osdmap_mapping_bytes": 0, 2024-03-18T23:19:31.048 INFO:teuthology.orchestra.run.smithi139.stdout: "osdmap_mapping_items": 0, 2024-03-18T23:19:31.048 INFO:teuthology.orchestra.run.smithi139.stdout: "pgmap_bytes": 0, 2024-03-18T23:19:31.049 INFO:teuthology.orchestra.run.smithi139.stdout: "pgmap_items": 0, 2024-03-18T23:19:31.049 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_co_bytes": 181532, 2024-03-18T23:19:31.049 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_co_items": 1386, 2024-03-18T23:19:31.049 INFO:teuthology.orchestra.run.smithi139.stdout: "unittest_1_bytes": 0, 2024-03-18T23:19:31.049 INFO:teuthology.orchestra.run.smithi139.stdout: "unittest_1_items": 0, 2024-03-18T23:19:31.049 INFO:teuthology.orchestra.run.smithi139.stdout: "unittest_2_bytes": 0, 2024-03-18T23:19:31.049 INFO:teuthology.orchestra.run.smithi139.stdout: "unittest_2_items": 0 2024-03-18T23:19:31.049 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:31.049 INFO:teuthology.orchestra.run.smithi139.stdout: "objecter": { 2024-03-18T23:19:31.050 INFO:teuthology.orchestra.run.smithi139.stdout: "op_active": 0, 2024-03-18T23:19:31.050 INFO:teuthology.orchestra.run.smithi139.stdout: "op_laggy": 0, 2024-03-18T23:19:31.050 INFO:teuthology.orchestra.run.smithi139.stdout: "op_send": 2345, 2024-03-18T23:19:31.050 INFO:teuthology.orchestra.run.smithi139.stdout: "op_send_bytes": 21, 2024-03-18T23:19:31.050 INFO:teuthology.orchestra.run.smithi139.stdout: "op_resend": 3, 2024-03-18T23:19:31.050 INFO:teuthology.orchestra.run.smithi139.stdout: "op_reply": 2345, 2024-03-18T23:19:31.050 INFO:teuthology.orchestra.run.smithi139.stdout: "op_latency": { 2024-03-18T23:19:31.050 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 2345, 2024-03-18T23:19:31.051 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 1.213974373, 2024-03-18T23:19:31.051 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000517686 2024-03-18T23:19:31.051 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:31.051 INFO:teuthology.orchestra.run.smithi139.stdout: "op_inflight": 0, 2024-03-18T23:19:31.051 INFO:teuthology.orchestra.run.smithi139.stdout: "oplen_avg": { 2024-03-18T23:19:31.051 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 2345, 2024-03-18T23:19:31.051 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 2346 2024-03-18T23:19:31.051 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:31.051 INFO:teuthology.orchestra.run.smithi139.stdout: "op": 2345, 2024-03-18T23:19:31.052 INFO:teuthology.orchestra.run.smithi139.stdout: "op_r": 2345, 2024-03-18T23:19:31.052 INFO:teuthology.orchestra.run.smithi139.stdout: "op_w": 0, 2024-03-18T23:19:31.052 INFO:teuthology.orchestra.run.smithi139.stdout: "op_rmw": 0, 2024-03-18T23:19:31.052 INFO:teuthology.orchestra.run.smithi139.stdout: "op_pg": 0, 2024-03-18T23:19:31.052 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_stat": 1472, 2024-03-18T23:19:31.052 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_create": 0, 2024-03-18T23:19:31.052 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_read": 871, 2024-03-18T23:19:31.052 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_write": 0, 2024-03-18T23:19:31.053 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_writefull": 0, 2024-03-18T23:19:31.053 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_writesame": 0, 2024-03-18T23:19:31.053 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_append": 0, 2024-03-18T23:19:31.053 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_zero": 0, 2024-03-18T23:19:31.053 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_truncate": 0, 2024-03-18T23:19:31.053 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_delete": 0, 2024-03-18T23:19:31.053 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_mapext": 0, 2024-03-18T23:19:31.053 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_sparse_read": 0, 2024-03-18T23:19:31.054 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_clonerange": 0, 2024-03-18T23:19:31.054 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_getxattr": 1, 2024-03-18T23:19:31.054 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_setxattr": 0, 2024-03-18T23:19:31.054 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_cmpxattr": 0, 2024-03-18T23:19:31.054 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_rmxattr": 0, 2024-03-18T23:19:31.054 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_resetxattrs": 0, 2024-03-18T23:19:31.054 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_call": 0, 2024-03-18T23:19:31.054 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_watch": 0, 2024-03-18T23:19:31.055 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_notify": 0, 2024-03-18T23:19:31.055 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_src_cmpxattr": 0, 2024-03-18T23:19:31.055 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_pgls": 0, 2024-03-18T23:19:31.055 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_pgls_filter": 0, 2024-03-18T23:19:31.055 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_other": 0, 2024-03-18T23:19:31.055 INFO:teuthology.orchestra.run.smithi139.stdout: "linger_active": 0, 2024-03-18T23:19:31.055 INFO:teuthology.orchestra.run.smithi139.stdout: "linger_send": 0, 2024-03-18T23:19:31.055 INFO:teuthology.orchestra.run.smithi139.stdout: "linger_resend": 0, 2024-03-18T23:19:31.055 INFO:teuthology.orchestra.run.smithi139.stdout: "linger_ping": 0, 2024-03-18T23:19:31.056 INFO:teuthology.orchestra.run.smithi139.stdout: "poolop_active": 0, 2024-03-18T23:19:31.056 INFO:teuthology.orchestra.run.smithi139.stdout: "poolop_send": 0, 2024-03-18T23:19:31.056 INFO:teuthology.orchestra.run.smithi139.stdout: "poolop_resend": 0, 2024-03-18T23:19:31.056 INFO:teuthology.orchestra.run.smithi139.stdout: "poolstat_active": 0, 2024-03-18T23:19:31.056 INFO:teuthology.orchestra.run.smithi139.stdout: "poolstat_send": 0, 2024-03-18T23:19:31.056 INFO:teuthology.orchestra.run.smithi139.stdout: "poolstat_resend": 0, 2024-03-18T23:19:31.056 INFO:teuthology.orchestra.run.smithi139.stdout: "statfs_active": 0, 2024-03-18T23:19:31.057 INFO:teuthology.orchestra.run.smithi139.stdout: "statfs_send": 0, 2024-03-18T23:19:31.057 INFO:teuthology.orchestra.run.smithi139.stdout: "statfs_resend": 0, 2024-03-18T23:19:31.057 INFO:teuthology.orchestra.run.smithi139.stdout: "command_active": 0, 2024-03-18T23:19:31.057 INFO:teuthology.orchestra.run.smithi139.stdout: "command_send": 0, 2024-03-18T23:19:31.057 INFO:teuthology.orchestra.run.smithi139.stdout: "command_resend": 0, 2024-03-18T23:19:31.057 INFO:teuthology.orchestra.run.smithi139.stdout: "map_epoch": 101, 2024-03-18T23:19:31.057 INFO:teuthology.orchestra.run.smithi139.stdout: "map_full": 0, 2024-03-18T23:19:31.057 INFO:teuthology.orchestra.run.smithi139.stdout: "map_inc": 15, 2024-03-18T23:19:31.057 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_sessions": 5, 2024-03-18T23:19:31.058 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_session_open": 5, 2024-03-18T23:19:31.058 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_session_close": 0, 2024-03-18T23:19:31.058 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_laggy": 0, 2024-03-18T23:19:31.058 INFO:teuthology.orchestra.run.smithi139.stdout: "omap_wr": 0, 2024-03-18T23:19:31.058 INFO:teuthology.orchestra.run.smithi139.stdout: "omap_rd": 2, 2024-03-18T23:19:31.058 INFO:teuthology.orchestra.run.smithi139.stdout: "omap_del": 0 2024-03-18T23:19:31.058 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:31.058 INFO:teuthology.orchestra.run.smithi139.stdout: "oft": { 2024-03-18T23:19:31.059 INFO:teuthology.orchestra.run.smithi139.stdout: "omap_total_objs": 0, 2024-03-18T23:19:31.059 INFO:teuthology.orchestra.run.smithi139.stdout: "omap_total_kv_pairs": 0, 2024-03-18T23:19:31.059 INFO:teuthology.orchestra.run.smithi139.stdout: "omap_total_updates": 0, 2024-03-18T23:19:31.059 INFO:teuthology.orchestra.run.smithi139.stdout: "omap_total_removes": 0 2024-03-18T23:19:31.059 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:31.059 INFO:teuthology.orchestra.run.smithi139.stdout: "osd-slow-ops": { 2024-03-18T23:19:31.059 INFO:teuthology.orchestra.run.smithi139.stdout: "slow_ops_count": 0 2024-03-18T23:19:31.059 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:31.060 INFO:teuthology.orchestra.run.smithi139.stdout: "purge_queue": { 2024-03-18T23:19:31.060 INFO:teuthology.orchestra.run.smithi139.stdout: "pq_executing_ops": 0, 2024-03-18T23:19:31.060 INFO:teuthology.orchestra.run.smithi139.stdout: "pq_executing_ops_high_water": 0, 2024-03-18T23:19:31.060 INFO:teuthology.orchestra.run.smithi139.stdout: "pq_executing": 0, 2024-03-18T23:19:31.060 INFO:teuthology.orchestra.run.smithi139.stdout: "pq_executing_high_water": 0, 2024-03-18T23:19:31.060 INFO:teuthology.orchestra.run.smithi139.stdout: "pq_executed_ops": 0, 2024-03-18T23:19:31.060 INFO:teuthology.orchestra.run.smithi139.stdout: "pq_executed": 0, 2024-03-18T23:19:31.060 INFO:teuthology.orchestra.run.smithi139.stdout: "pq_item_in_journal": 0 2024-03-18T23:19:31.061 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:31.061 INFO:teuthology.orchestra.run.smithi139.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-18T23:19:31.061 INFO:teuthology.orchestra.run.smithi139.stdout: "val": 0, 2024-03-18T23:19:31.061 INFO:teuthology.orchestra.run.smithi139.stdout: "max": 104857600, 2024-03-18T23:19:31.061 INFO:teuthology.orchestra.run.smithi139.stdout: "get_started": 0, 2024-03-18T23:19:31.061 INFO:teuthology.orchestra.run.smithi139.stdout: "get": 2607, 2024-03-18T23:19:31.061 INFO:teuthology.orchestra.run.smithi139.stdout: "get_sum": 2163956, 2024-03-18T23:19:31.061 INFO:teuthology.orchestra.run.smithi139.stdout: "get_or_fail_fail": 0, 2024-03-18T23:19:31.061 INFO:teuthology.orchestra.run.smithi139.stdout: "get_or_fail_success": 2607, 2024-03-18T23:19:31.062 INFO:teuthology.orchestra.run.smithi139.stdout: "take": 0, 2024-03-18T23:19:31.062 INFO:teuthology.orchestra.run.smithi139.stdout: "take_sum": 0, 2024-03-18T23:19:31.062 INFO:teuthology.orchestra.run.smithi139.stdout: "put": 2607, 2024-03-18T23:19:31.062 INFO:teuthology.orchestra.run.smithi139.stdout: "put_sum": 2163956, 2024-03-18T23:19:31.062 INFO:teuthology.orchestra.run.smithi139.stdout: "wait": { 2024-03-18T23:19:31.062 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-03-18T23:19:31.062 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-03-18T23:19:31.062 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-03-18T23:19:31.063 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-03-18T23:19:31.063 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:31.063 INFO:teuthology.orchestra.run.smithi139.stdout: "throttle-objecter_bytes": { 2024-03-18T23:19:31.063 INFO:teuthology.orchestra.run.smithi139.stdout: "val": 0, 2024-03-18T23:19:31.063 INFO:teuthology.orchestra.run.smithi139.stdout: "max": 104857600, 2024-03-18T23:19:31.063 INFO:teuthology.orchestra.run.smithi139.stdout: "get_started": 0, 2024-03-18T23:19:31.063 INFO:teuthology.orchestra.run.smithi139.stdout: "get": 0, 2024-03-18T23:19:31.063 INFO:teuthology.orchestra.run.smithi139.stdout: "get_sum": 0, 2024-03-18T23:19:31.063 INFO:teuthology.orchestra.run.smithi139.stdout: "get_or_fail_fail": 0, 2024-03-18T23:19:31.064 INFO:teuthology.orchestra.run.smithi139.stdout: "get_or_fail_success": 0, 2024-03-18T23:19:31.064 INFO:teuthology.orchestra.run.smithi139.stdout: "take": 2345, 2024-03-18T23:19:31.064 INFO:teuthology.orchestra.run.smithi139.stdout: "take_sum": 250385, 2024-03-18T23:19:31.064 INFO:teuthology.orchestra.run.smithi139.stdout: "put": 2345, 2024-03-18T23:19:31.064 INFO:teuthology.orchestra.run.smithi139.stdout: "put_sum": 250385, 2024-03-18T23:19:31.064 INFO:teuthology.orchestra.run.smithi139.stdout: "wait": { 2024-03-18T23:19:31.064 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-03-18T23:19:31.064 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-03-18T23:19:31.065 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-03-18T23:19:31.065 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-03-18T23:19:31.065 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:31.065 INFO:teuthology.orchestra.run.smithi139.stdout: "throttle-objecter_ops": { 2024-03-18T23:19:31.065 INFO:teuthology.orchestra.run.smithi139.stdout: "val": 0, 2024-03-18T23:19:31.065 INFO:teuthology.orchestra.run.smithi139.stdout: "max": 1024, 2024-03-18T23:19:31.065 INFO:teuthology.orchestra.run.smithi139.stdout: "get_started": 0, 2024-03-18T23:19:31.065 INFO:teuthology.orchestra.run.smithi139.stdout: "get": 0, 2024-03-18T23:19:31.065 INFO:teuthology.orchestra.run.smithi139.stdout: "get_sum": 0, 2024-03-18T23:19:31.066 INFO:teuthology.orchestra.run.smithi139.stdout: "get_or_fail_fail": 0, 2024-03-18T23:19:31.066 INFO:teuthology.orchestra.run.smithi139.stdout: "get_or_fail_success": 0, 2024-03-18T23:19:31.066 INFO:teuthology.orchestra.run.smithi139.stdout: "take": 2345, 2024-03-18T23:19:31.066 INFO:teuthology.orchestra.run.smithi139.stdout: "take_sum": 2345, 2024-03-18T23:19:31.066 INFO:teuthology.orchestra.run.smithi139.stdout: "put": 2345, 2024-03-18T23:19:31.066 INFO:teuthology.orchestra.run.smithi139.stdout: "put_sum": 2345, 2024-03-18T23:19:31.066 INFO:teuthology.orchestra.run.smithi139.stdout: "wait": { 2024-03-18T23:19:31.066 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-03-18T23:19:31.067 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-03-18T23:19:31.067 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-03-18T23:19:31.067 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-03-18T23:19:31.067 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:31.067 INFO:teuthology.orchestra.run.smithi139.stdout: "throttle-write_buf_throttle": { 2024-03-18T23:19:31.067 INFO:teuthology.orchestra.run.smithi139.stdout: "val": 0, 2024-03-18T23:19:31.067 INFO:teuthology.orchestra.run.smithi139.stdout: "max": 3758096384, 2024-03-18T23:19:31.067 INFO:teuthology.orchestra.run.smithi139.stdout: "get_started": 0, 2024-03-18T23:19:31.068 INFO:teuthology.orchestra.run.smithi139.stdout: "get": 0, 2024-03-18T23:19:31.068 INFO:teuthology.orchestra.run.smithi139.stdout: "get_sum": 0, 2024-03-18T23:19:31.068 INFO:teuthology.orchestra.run.smithi139.stdout: "get_or_fail_fail": 0, 2024-03-18T23:19:31.068 INFO:teuthology.orchestra.run.smithi139.stdout: "get_or_fail_success": 0, 2024-03-18T23:19:31.068 INFO:teuthology.orchestra.run.smithi139.stdout: "take": 0, 2024-03-18T23:19:31.068 INFO:teuthology.orchestra.run.smithi139.stdout: "take_sum": 0, 2024-03-18T23:19:31.068 INFO:teuthology.orchestra.run.smithi139.stdout: "put": 0, 2024-03-18T23:19:31.068 INFO:teuthology.orchestra.run.smithi139.stdout: "put_sum": 0, 2024-03-18T23:19:31.068 INFO:teuthology.orchestra.run.smithi139.stdout: "wait": { 2024-03-18T23:19:31.069 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-03-18T23:19:31.069 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-03-18T23:19:31.069 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-03-18T23:19:31.069 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-03-18T23:19:31.069 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-03-18T23:19:31.069 INFO:teuthology.orchestra.run.smithi139.stdout: "throttle-write_buf_throttle-0x5652214d46e0": { 2024-03-18T23:19:31.069 INFO:teuthology.orchestra.run.smithi139.stdout: "val": 0, 2024-03-18T23:19:31.069 INFO:teuthology.orchestra.run.smithi139.stdout: "max": 3758096384, 2024-03-18T23:19:31.070 INFO:teuthology.orchestra.run.smithi139.stdout: "get_started": 0, 2024-03-18T23:19:31.070 INFO:teuthology.orchestra.run.smithi139.stdout: "get": 0, 2024-03-18T23:19:31.070 INFO:teuthology.orchestra.run.smithi139.stdout: "get_sum": 0, 2024-03-18T23:19:31.070 INFO:teuthology.orchestra.run.smithi139.stdout: "get_or_fail_fail": 0, 2024-03-18T23:19:31.070 INFO:teuthology.orchestra.run.smithi139.stdout: "get_or_fail_success": 0, 2024-03-18T23:19:31.070 INFO:teuthology.orchestra.run.smithi139.stdout: "take": 0, 2024-03-18T23:19:31.070 INFO:teuthology.orchestra.run.smithi139.stdout: "take_sum": 0, 2024-03-18T23:19:31.070 INFO:teuthology.orchestra.run.smithi139.stdout: "put": 0, 2024-03-18T23:19:31.070 INFO:teuthology.orchestra.run.smithi139.stdout: "put_sum": 0, 2024-03-18T23:19:31.071 INFO:teuthology.orchestra.run.smithi139.stdout: "wait": { 2024-03-18T23:19:31.071 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-03-18T23:19:31.071 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-03-18T23:19:31.071 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-03-18T23:19:31.071 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-03-18T23:19:31.071 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-03-18T23:19:31.071 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-03-18T23:19:31.564 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:19:31 smithi139 ceph-mon[67282]: pgmap v352: 97 pgs: 97 active+clean; 66 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 1.3 MiB/s wr, 63 op/s 2024-03-18T23:19:31.629 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:19:31 smithi049 ceph-mon[62991]: pgmap v352: 97 pgs: 97 active+clean; 66 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 1.3 MiB/s wr, 63 op/s 2024-03-18T23:19:31.710 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:19:31 smithi175 ceph-mon[67352]: pgmap v352: 97 pgs: 97 active+clean; 66 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 1.3 MiB/s wr, 63 op/s 2024-03-18T23:19:31.825 INFO:tasks.check_counter:Daemon mds.k mds.exported=0 2024-03-18T23:19:31.825 INFO:tasks.check_counter:Daemon mds.k mds.imported=0 2024-03-18T23:19:31.825 INFO:tasks.check_counter:Daemon mds.k mds.root_rsnaps=0 2024-03-18T23:19:31.825 INFO:tasks.check_counter:Daemon mds.k mds_server.req_mksnap_latency.avgcount=0 2024-03-18T23:19:31.825 INFO:tasks.check_counter:Daemon mds.k mds_server.req_rmsnap_latency.avgcount=0 2024-03-18T23:19:31.825 DEBUG:tasks.check_counter:Getting stats from c 2024-03-18T23:19:31.826 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell --fsid c63591ea-e57a-11ee-95c9-87774f69a715 -- ceph daemon mds.c perf dump 2024-03-18T23:19:32.118 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/mon.c/config 2024-03-18T23:19:33.500 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:19:33 smithi175 ceph-mon[67352]: pgmap v353: 97 pgs: 97 active+clean; 67 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 1.5 MiB/s wr, 64 op/s 2024-03-18T23:19:33.564 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:19:33 smithi139 ceph-mon[67282]: pgmap v353: 97 pgs: 97 active+clean; 67 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 1.5 MiB/s wr, 64 op/s 2024-03-18T23:19:33.629 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:19:33 smithi049 ceph-mon[62991]: pgmap v353: 97 pgs: 97 active+clean; 67 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 1.5 MiB/s wr, 64 op/s 2024-03-18T23:19:33.911 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-03-18T23:19:33.911 INFO:teuthology.orchestra.run.smithi175.stdout: "AsyncMessenger::Worker-0": { 2024-03-18T23:19:33.911 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_messages": 2305, 2024-03-18T23:19:33.911 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_messages": 2305, 2024-03-18T23:19:33.911 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_bytes": 657268, 2024-03-18T23:19:33.911 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_bytes": 732173, 2024-03-18T23:19:33.911 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_created_connections": 6, 2024-03-18T23:19:33.912 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_active_connections": 2, 2024-03-18T23:19:33.912 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_total_time": 0.141656813, 2024-03-18T23:19:33.912 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_send_time": 0.041894708, 2024-03-18T23:19:33.912 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_recv_time": 0.036042430, 2024-03-18T23:19:33.912 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_fast_dispatch_time": 0.054276885, 2024-03-18T23:19:33.912 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_messages_queue_lat": { 2024-03-18T23:19:33.912 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 2304, 2024-03-18T23:19:33.912 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.047446442, 2024-03-18T23:19:33.913 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000020593 2024-03-18T23:19:33.913 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:33.913 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_handle_ack_lat": { 2024-03-18T23:19:33.913 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-03-18T23:19:33.913 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-03-18T23:19:33.913 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-03-18T23:19:33.913 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:33.913 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_encrypted_bytes": 3024, 2024-03-18T23:19:33.913 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_encrypted_bytes": 448 2024-03-18T23:19:33.914 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:33.914 INFO:teuthology.orchestra.run.smithi175.stdout: "AsyncMessenger::Worker-1": { 2024-03-18T23:19:33.914 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_messages": 266, 2024-03-18T23:19:33.914 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_messages": 244, 2024-03-18T23:19:33.914 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_bytes": 1528108, 2024-03-18T23:19:33.914 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_bytes": 140554, 2024-03-18T23:19:33.914 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_created_connections": 3, 2024-03-18T23:19:33.914 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_active_connections": 2, 2024-03-18T23:19:33.915 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_total_time": 0.039185249, 2024-03-18T23:19:33.915 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_send_time": 0.009541708, 2024-03-18T23:19:33.915 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_recv_time": 0.036335392, 2024-03-18T23:19:33.915 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_fast_dispatch_time": 0.004298232, 2024-03-18T23:19:33.915 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_messages_queue_lat": { 2024-03-18T23:19:33.915 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 243, 2024-03-18T23:19:33.915 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.007838210, 2024-03-18T23:19:33.915 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000032256 2024-03-18T23:19:33.915 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:33.916 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_handle_ack_lat": { 2024-03-18T23:19:33.916 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-03-18T23:19:33.917 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-03-18T23:19:33.917 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-03-18T23:19:33.917 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:33.917 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_encrypted_bytes": 1527552, 2024-03-18T23:19:33.917 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_encrypted_bytes": 139856 2024-03-18T23:19:33.917 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:33.918 INFO:teuthology.orchestra.run.smithi175.stdout: "AsyncMessenger::Worker-2": { 2024-03-18T23:19:33.918 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_messages": 3, 2024-03-18T23:19:33.918 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_messages": 171, 2024-03-18T23:19:33.918 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_bytes": 588, 2024-03-18T23:19:33.918 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_bytes": 463022, 2024-03-18T23:19:33.918 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_created_connections": 6, 2024-03-18T23:19:33.918 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_active_connections": 4, 2024-03-18T23:19:33.918 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_total_time": 0.008822420, 2024-03-18T23:19:33.918 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_send_time": 0.004986347, 2024-03-18T23:19:33.919 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_recv_time": 0.002312445, 2024-03-18T23:19:33.919 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_fast_dispatch_time": 0.000023904, 2024-03-18T23:19:33.919 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_messages_queue_lat": { 2024-03-18T23:19:33.919 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 168, 2024-03-18T23:19:33.919 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.012366010, 2024-03-18T23:19:33.919 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000073607 2024-03-18T23:19:33.919 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:33.919 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_handle_ack_lat": { 2024-03-18T23:19:33.920 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-03-18T23:19:33.920 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-03-18T23:19:33.920 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-03-18T23:19:33.920 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:33.920 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_encrypted_bytes": 320, 2024-03-18T23:19:33.920 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_encrypted_bytes": 462704 2024-03-18T23:19:33.920 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:33.920 INFO:teuthology.orchestra.run.smithi175.stdout: "cct": { 2024-03-18T23:19:33.920 INFO:teuthology.orchestra.run.smithi175.stdout: "total_workers": 1, 2024-03-18T23:19:33.921 INFO:teuthology.orchestra.run.smithi175.stdout: "unhealthy_workers": 0 2024-03-18T23:19:33.921 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:33.921 INFO:teuthology.orchestra.run.smithi175.stdout: "finisher-MDSRank": { 2024-03-18T23:19:33.921 INFO:teuthology.orchestra.run.smithi175.stdout: "queue_len": 0, 2024-03-18T23:19:33.921 INFO:teuthology.orchestra.run.smithi175.stdout: "complete_latency": { 2024-03-18T23:19:33.921 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 3904, 2024-03-18T23:19:33.921 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.083452886, 2024-03-18T23:19:33.921 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000021376 2024-03-18T23:19:33.921 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-03-18T23:19:33.921 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:33.922 INFO:teuthology.orchestra.run.smithi175.stdout: "finisher-PurgeQueue": { 2024-03-18T23:19:33.922 INFO:teuthology.orchestra.run.smithi175.stdout: "queue_len": 0, 2024-03-18T23:19:33.922 INFO:teuthology.orchestra.run.smithi175.stdout: "complete_latency": { 2024-03-18T23:19:33.922 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-03-18T23:19:33.922 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-03-18T23:19:33.922 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-03-18T23:19:33.922 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-03-18T23:19:33.922 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:33.923 INFO:teuthology.orchestra.run.smithi175.stdout: "mds": { 2024-03-18T23:19:33.923 INFO:teuthology.orchestra.run.smithi175.stdout: "request": 0, 2024-03-18T23:19:33.923 INFO:teuthology.orchestra.run.smithi175.stdout: "reply": 0, 2024-03-18T23:19:33.923 INFO:teuthology.orchestra.run.smithi175.stdout: "reply_latency": { 2024-03-18T23:19:33.923 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-03-18T23:19:33.923 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-03-18T23:19:33.923 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-03-18T23:19:33.923 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:33.923 INFO:teuthology.orchestra.run.smithi175.stdout: "slow_reply": 0, 2024-03-18T23:19:33.924 INFO:teuthology.orchestra.run.smithi175.stdout: "forward": 0, 2024-03-18T23:19:33.924 INFO:teuthology.orchestra.run.smithi175.stdout: "dir_fetch_complete": 0, 2024-03-18T23:19:33.924 INFO:teuthology.orchestra.run.smithi175.stdout: "dir_fetch_keys": 0, 2024-03-18T23:19:33.924 INFO:teuthology.orchestra.run.smithi175.stdout: "dir_commit": 0, 2024-03-18T23:19:33.924 INFO:teuthology.orchestra.run.smithi175.stdout: "dir_split": 0, 2024-03-18T23:19:33.924 INFO:teuthology.orchestra.run.smithi175.stdout: "dir_merge": 0, 2024-03-18T23:19:33.925 INFO:teuthology.orchestra.run.smithi175.stdout: "inodes": 0, 2024-03-18T23:19:33.925 INFO:teuthology.orchestra.run.smithi175.stdout: "inodes_top": 0, 2024-03-18T23:19:33.925 INFO:teuthology.orchestra.run.smithi175.stdout: "inodes_bottom": 0, 2024-03-18T23:19:33.925 INFO:teuthology.orchestra.run.smithi175.stdout: "inodes_pin_tail": 0, 2024-03-18T23:19:33.925 INFO:teuthology.orchestra.run.smithi175.stdout: "inodes_pinned": 0, 2024-03-18T23:19:33.925 INFO:teuthology.orchestra.run.smithi175.stdout: "inodes_expired": 0, 2024-03-18T23:19:33.925 INFO:teuthology.orchestra.run.smithi175.stdout: "inodes_with_caps": 0, 2024-03-18T23:19:33.925 INFO:teuthology.orchestra.run.smithi175.stdout: "caps": 0, 2024-03-18T23:19:33.925 INFO:teuthology.orchestra.run.smithi175.stdout: "subtrees": 1, 2024-03-18T23:19:33.926 INFO:teuthology.orchestra.run.smithi175.stdout: "traverse": 0, 2024-03-18T23:19:33.926 INFO:teuthology.orchestra.run.smithi175.stdout: "traverse_hit": 0, 2024-03-18T23:19:33.926 INFO:teuthology.orchestra.run.smithi175.stdout: "traverse_forward": 0, 2024-03-18T23:19:33.926 INFO:teuthology.orchestra.run.smithi175.stdout: "traverse_discover": 0, 2024-03-18T23:19:33.926 INFO:teuthology.orchestra.run.smithi175.stdout: "traverse_dir_fetch": 0, 2024-03-18T23:19:33.926 INFO:teuthology.orchestra.run.smithi175.stdout: "traverse_remote_ino": 0, 2024-03-18T23:19:33.926 INFO:teuthology.orchestra.run.smithi175.stdout: "traverse_lock": 0, 2024-03-18T23:19:33.926 INFO:teuthology.orchestra.run.smithi175.stdout: "load_cent": 0, 2024-03-18T23:19:33.927 INFO:teuthology.orchestra.run.smithi175.stdout: "q": 0, 2024-03-18T23:19:33.927 INFO:teuthology.orchestra.run.smithi175.stdout: "exported": 0, 2024-03-18T23:19:33.927 INFO:teuthology.orchestra.run.smithi175.stdout: "exported_inodes": 0, 2024-03-18T23:19:33.927 INFO:teuthology.orchestra.run.smithi175.stdout: "imported": 0, 2024-03-18T23:19:33.927 INFO:teuthology.orchestra.run.smithi175.stdout: "imported_inodes": 0, 2024-03-18T23:19:33.927 INFO:teuthology.orchestra.run.smithi175.stdout: "openino_dir_fetch": 0, 2024-03-18T23:19:33.928 INFO:teuthology.orchestra.run.smithi175.stdout: "openino_backtrace_fetch": 0, 2024-03-18T23:19:33.928 INFO:teuthology.orchestra.run.smithi175.stdout: "openino_peer_discover": 0, 2024-03-18T23:19:33.928 INFO:teuthology.orchestra.run.smithi175.stdout: "root_rfiles": 0, 2024-03-18T23:19:33.928 INFO:teuthology.orchestra.run.smithi175.stdout: "root_rbytes": 0, 2024-03-18T23:19:33.928 INFO:teuthology.orchestra.run.smithi175.stdout: "root_rsnaps": 0, 2024-03-18T23:19:33.928 INFO:teuthology.orchestra.run.smithi175.stdout: "scrub_backtrace_fetch": 0, 2024-03-18T23:19:33.928 INFO:teuthology.orchestra.run.smithi175.stdout: "scrub_set_tag": 0, 2024-03-18T23:19:33.928 INFO:teuthology.orchestra.run.smithi175.stdout: "scrub_backtrace_repaired": 0, 2024-03-18T23:19:33.929 INFO:teuthology.orchestra.run.smithi175.stdout: "scrub_inotable_repaired": 0, 2024-03-18T23:19:33.929 INFO:teuthology.orchestra.run.smithi175.stdout: "scrub_dir_inodes": 0, 2024-03-18T23:19:33.929 INFO:teuthology.orchestra.run.smithi175.stdout: "scrub_dir_base_inodes": 0, 2024-03-18T23:19:33.929 INFO:teuthology.orchestra.run.smithi175.stdout: "scrub_dirfrag_rstats": 0, 2024-03-18T23:19:33.929 INFO:teuthology.orchestra.run.smithi175.stdout: "scrub_file_inodes": 0, 2024-03-18T23:19:33.929 INFO:teuthology.orchestra.run.smithi175.stdout: "handle_inode_file_caps": 0, 2024-03-18T23:19:33.929 INFO:teuthology.orchestra.run.smithi175.stdout: "ceph_cap_op_revoke": 0, 2024-03-18T23:19:33.929 INFO:teuthology.orchestra.run.smithi175.stdout: "ceph_cap_op_grant": 0, 2024-03-18T23:19:33.929 INFO:teuthology.orchestra.run.smithi175.stdout: "ceph_cap_op_trunc": 0, 2024-03-18T23:19:33.929 INFO:teuthology.orchestra.run.smithi175.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-18T23:19:33.930 INFO:teuthology.orchestra.run.smithi175.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-18T23:19:33.930 INFO:teuthology.orchestra.run.smithi175.stdout: "handle_client_caps": 0, 2024-03-18T23:19:33.930 INFO:teuthology.orchestra.run.smithi175.stdout: "handle_client_caps_dirty": 0, 2024-03-18T23:19:33.930 INFO:teuthology.orchestra.run.smithi175.stdout: "handle_client_cap_release": 0, 2024-03-18T23:19:33.930 INFO:teuthology.orchestra.run.smithi175.stdout: "process_request_cap_release": 0 2024-03-18T23:19:33.930 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:33.930 INFO:teuthology.orchestra.run.smithi175.stdout: "mds_cache": { 2024-03-18T23:19:33.930 INFO:teuthology.orchestra.run.smithi175.stdout: "dir_update": 0, 2024-03-18T23:19:33.931 INFO:teuthology.orchestra.run.smithi175.stdout: "dir_update_receipt": 0, 2024-03-18T23:19:33.931 INFO:teuthology.orchestra.run.smithi175.stdout: "dir_try_discover": 0, 2024-03-18T23:19:33.931 INFO:teuthology.orchestra.run.smithi175.stdout: "dir_send_discover": 0, 2024-03-18T23:19:33.931 INFO:teuthology.orchestra.run.smithi175.stdout: "dir_handle_discover": 0, 2024-03-18T23:19:33.931 INFO:teuthology.orchestra.run.smithi175.stdout: "num_strays": 0, 2024-03-18T23:19:33.931 INFO:teuthology.orchestra.run.smithi175.stdout: "num_strays_delayed": 0, 2024-03-18T23:19:33.932 INFO:teuthology.orchestra.run.smithi175.stdout: "num_strays_enqueuing": 0, 2024-03-18T23:19:33.932 INFO:teuthology.orchestra.run.smithi175.stdout: "strays_created": 0, 2024-03-18T23:19:33.932 INFO:teuthology.orchestra.run.smithi175.stdout: "strays_enqueued": 0, 2024-03-18T23:19:33.932 INFO:teuthology.orchestra.run.smithi175.stdout: "strays_reintegrated": 0, 2024-03-18T23:19:33.932 INFO:teuthology.orchestra.run.smithi175.stdout: "strays_migrated": 0, 2024-03-18T23:19:33.932 INFO:teuthology.orchestra.run.smithi175.stdout: "num_recovering_processing": 0, 2024-03-18T23:19:33.932 INFO:teuthology.orchestra.run.smithi175.stdout: "num_recovering_enqueued": 0, 2024-03-18T23:19:33.932 INFO:teuthology.orchestra.run.smithi175.stdout: "num_recovering_prioritized": 0, 2024-03-18T23:19:33.932 INFO:teuthology.orchestra.run.smithi175.stdout: "recovery_started": 0, 2024-03-18T23:19:33.933 INFO:teuthology.orchestra.run.smithi175.stdout: "recovery_completed": 0, 2024-03-18T23:19:33.933 INFO:teuthology.orchestra.run.smithi175.stdout: "ireq_quiesce_path": 0, 2024-03-18T23:19:33.933 INFO:teuthology.orchestra.run.smithi175.stdout: "ireq_quiesce_inode": 0, 2024-03-18T23:19:33.933 INFO:teuthology.orchestra.run.smithi175.stdout: "ireq_enqueue_scrub": 0, 2024-03-18T23:19:33.933 INFO:teuthology.orchestra.run.smithi175.stdout: "ireq_exportdir": 0, 2024-03-18T23:19:33.933 INFO:teuthology.orchestra.run.smithi175.stdout: "ireq_flush": 0, 2024-03-18T23:19:33.934 INFO:teuthology.orchestra.run.smithi175.stdout: "ireq_fragmentdir": 0, 2024-03-18T23:19:33.934 INFO:teuthology.orchestra.run.smithi175.stdout: "ireq_fragstats": 0, 2024-03-18T23:19:33.934 INFO:teuthology.orchestra.run.smithi175.stdout: "ireq_inodestats": 0 2024-03-18T23:19:33.934 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:33.934 INFO:teuthology.orchestra.run.smithi175.stdout: "mds_log": { 2024-03-18T23:19:33.934 INFO:teuthology.orchestra.run.smithi175.stdout: "evlrg": 0, 2024-03-18T23:19:33.934 INFO:teuthology.orchestra.run.smithi175.stdout: "evadd": 0, 2024-03-18T23:19:33.934 INFO:teuthology.orchestra.run.smithi175.stdout: "evex": 0, 2024-03-18T23:19:33.935 INFO:teuthology.orchestra.run.smithi175.stdout: "evtrm": 0, 2024-03-18T23:19:33.935 INFO:teuthology.orchestra.run.smithi175.stdout: "ev": 8, 2024-03-18T23:19:33.935 INFO:teuthology.orchestra.run.smithi175.stdout: "evexg": 0, 2024-03-18T23:19:33.935 INFO:teuthology.orchestra.run.smithi175.stdout: "evexd": 0, 2024-03-18T23:19:33.935 INFO:teuthology.orchestra.run.smithi175.stdout: "segadd": 0, 2024-03-18T23:19:33.935 INFO:teuthology.orchestra.run.smithi175.stdout: "segex": 0, 2024-03-18T23:19:33.935 INFO:teuthology.orchestra.run.smithi175.stdout: "segtrm": 0, 2024-03-18T23:19:33.935 INFO:teuthology.orchestra.run.smithi175.stdout: "seg": 2, 2024-03-18T23:19:33.936 INFO:teuthology.orchestra.run.smithi175.stdout: "segmjr": 2, 2024-03-18T23:19:33.936 INFO:teuthology.orchestra.run.smithi175.stdout: "segexg": 0, 2024-03-18T23:19:33.936 INFO:teuthology.orchestra.run.smithi175.stdout: "segexd": 0, 2024-03-18T23:19:33.936 INFO:teuthology.orchestra.run.smithi175.stdout: "expos": 4194304, 2024-03-18T23:19:33.936 INFO:teuthology.orchestra.run.smithi175.stdout: "wrpos": 4196655, 2024-03-18T23:19:33.936 INFO:teuthology.orchestra.run.smithi175.stdout: "rdpos": 4196514, 2024-03-18T23:19:33.936 INFO:teuthology.orchestra.run.smithi175.stdout: "jlat": { 2024-03-18T23:19:33.936 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-03-18T23:19:33.936 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-03-18T23:19:33.937 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-03-18T23:19:33.937 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:33.937 INFO:teuthology.orchestra.run.smithi175.stdout: "replayed": 8 2024-03-18T23:19:33.937 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:33.937 INFO:teuthology.orchestra.run.smithi175.stdout: "mds_mem": { 2024-03-18T23:19:33.937 INFO:teuthology.orchestra.run.smithi175.stdout: "ino": 3, 2024-03-18T23:19:33.938 INFO:teuthology.orchestra.run.smithi175.stdout: "ino+": 3, 2024-03-18T23:19:33.938 INFO:teuthology.orchestra.run.smithi175.stdout: "ino-": 0, 2024-03-18T23:19:33.938 INFO:teuthology.orchestra.run.smithi175.stdout: "dir": 1, 2024-03-18T23:19:33.938 INFO:teuthology.orchestra.run.smithi175.stdout: "dir+": 1, 2024-03-18T23:19:33.938 INFO:teuthology.orchestra.run.smithi175.stdout: "dir-": 0, 2024-03-18T23:19:33.938 INFO:teuthology.orchestra.run.smithi175.stdout: "dn": 0, 2024-03-18T23:19:33.938 INFO:teuthology.orchestra.run.smithi175.stdout: "dn+": 0, 2024-03-18T23:19:33.938 INFO:teuthology.orchestra.run.smithi175.stdout: "dn-": 0, 2024-03-18T23:19:33.939 INFO:teuthology.orchestra.run.smithi175.stdout: "cap": 0, 2024-03-18T23:19:33.939 INFO:teuthology.orchestra.run.smithi175.stdout: "cap+": 0, 2024-03-18T23:19:33.939 INFO:teuthology.orchestra.run.smithi175.stdout: "cap-": 0, 2024-03-18T23:19:33.939 INFO:teuthology.orchestra.run.smithi175.stdout: "rss": 50036, 2024-03-18T23:19:33.939 INFO:teuthology.orchestra.run.smithi175.stdout: "heap": 215420 2024-03-18T23:19:33.939 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:33.939 INFO:teuthology.orchestra.run.smithi175.stdout: "mds_server": { 2024-03-18T23:19:33.939 INFO:teuthology.orchestra.run.smithi175.stdout: "dispatch_client_request": 0, 2024-03-18T23:19:33.939 INFO:teuthology.orchestra.run.smithi175.stdout: "dispatch_server_request": 0, 2024-03-18T23:19:33.940 INFO:teuthology.orchestra.run.smithi175.stdout: "handle_client_request": 0, 2024-03-18T23:19:33.940 INFO:teuthology.orchestra.run.smithi175.stdout: "handle_client_session": 0, 2024-03-18T23:19:33.940 INFO:teuthology.orchestra.run.smithi175.stdout: "handle_peer_request": 0, 2024-03-18T23:19:33.940 INFO:teuthology.orchestra.run.smithi175.stdout: "req_create_latency": { 2024-03-18T23:19:33.940 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-03-18T23:19:33.940 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-03-18T23:19:33.940 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-03-18T23:19:33.940 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:33.940 INFO:teuthology.orchestra.run.smithi175.stdout: "req_getattr_latency": { 2024-03-18T23:19:33.941 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-03-18T23:19:33.941 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-03-18T23:19:33.941 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-03-18T23:19:33.941 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:33.941 INFO:teuthology.orchestra.run.smithi175.stdout: "req_getfilelock_latency": { 2024-03-18T23:19:33.941 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-03-18T23:19:33.942 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-03-18T23:19:33.942 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-03-18T23:19:33.942 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:33.942 INFO:teuthology.orchestra.run.smithi175.stdout: "req_link_latency": { 2024-03-18T23:19:33.942 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-03-18T23:19:33.942 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-03-18T23:19:33.942 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-03-18T23:19:33.942 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:33.942 INFO:teuthology.orchestra.run.smithi175.stdout: "req_lookup_latency": { 2024-03-18T23:19:33.943 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-03-18T23:19:33.943 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-03-18T23:19:33.943 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-03-18T23:19:33.943 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:33.943 INFO:teuthology.orchestra.run.smithi175.stdout: "req_lookuphash_latency": { 2024-03-18T23:19:33.943 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-03-18T23:19:33.943 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-03-18T23:19:33.943 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-03-18T23:19:33.943 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:33.944 INFO:teuthology.orchestra.run.smithi175.stdout: "req_lookupino_latency": { 2024-03-18T23:19:33.944 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-03-18T23:19:33.944 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-03-18T23:19:33.944 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-03-18T23:19:33.944 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:33.944 INFO:teuthology.orchestra.run.smithi175.stdout: "req_lookupname_latency": { 2024-03-18T23:19:33.944 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-03-18T23:19:33.944 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-03-18T23:19:33.945 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-03-18T23:19:33.945 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:33.945 INFO:teuthology.orchestra.run.smithi175.stdout: "req_lookupparent_latency": { 2024-03-18T23:19:33.945 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-03-18T23:19:33.945 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-03-18T23:19:33.945 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-03-18T23:19:33.945 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:33.945 INFO:teuthology.orchestra.run.smithi175.stdout: "req_lookupsnap_latency": { 2024-03-18T23:19:33.945 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-03-18T23:19:33.946 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-03-18T23:19:33.946 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-03-18T23:19:33.946 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:33.946 INFO:teuthology.orchestra.run.smithi175.stdout: "req_lssnap_latency": { 2024-03-18T23:19:33.946 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-03-18T23:19:33.946 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-03-18T23:19:33.946 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-03-18T23:19:33.946 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:33.947 INFO:teuthology.orchestra.run.smithi175.stdout: "req_mkdir_latency": { 2024-03-18T23:19:33.947 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-03-18T23:19:33.948 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-03-18T23:19:33.948 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-03-18T23:19:33.948 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:33.948 INFO:teuthology.orchestra.run.smithi175.stdout: "req_mknod_latency": { 2024-03-18T23:19:33.948 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-03-18T23:19:33.948 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-03-18T23:19:33.948 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-03-18T23:19:33.948 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:33.949 INFO:teuthology.orchestra.run.smithi175.stdout: "req_mksnap_latency": { 2024-03-18T23:19:33.949 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-03-18T23:19:33.949 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-03-18T23:19:33.949 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-03-18T23:19:33.949 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:33.949 INFO:teuthology.orchestra.run.smithi175.stdout: "req_open_latency": { 2024-03-18T23:19:33.949 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-03-18T23:19:33.949 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-03-18T23:19:33.950 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-03-18T23:19:33.950 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:33.950 INFO:teuthology.orchestra.run.smithi175.stdout: "req_readdir_latency": { 2024-03-18T23:19:33.950 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-03-18T23:19:33.950 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-03-18T23:19:33.950 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-03-18T23:19:33.950 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:33.950 INFO:teuthology.orchestra.run.smithi175.stdout: "req_rename_latency": { 2024-03-18T23:19:33.950 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-03-18T23:19:33.951 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-03-18T23:19:33.951 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-03-18T23:19:33.951 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:33.951 INFO:teuthology.orchestra.run.smithi175.stdout: "req_renamesnap_latency": { 2024-03-18T23:19:33.951 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-03-18T23:19:33.951 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-03-18T23:19:33.951 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-03-18T23:19:33.951 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:33.952 INFO:teuthology.orchestra.run.smithi175.stdout: "req_snapdiff_latency": { 2024-03-18T23:19:33.952 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-03-18T23:19:33.952 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-03-18T23:19:33.952 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-03-18T23:19:33.952 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:33.952 INFO:teuthology.orchestra.run.smithi175.stdout: "req_rmdir_latency": { 2024-03-18T23:19:33.952 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-03-18T23:19:33.952 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-03-18T23:19:33.952 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-03-18T23:19:33.953 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:33.953 INFO:teuthology.orchestra.run.smithi175.stdout: "req_rmsnap_latency": { 2024-03-18T23:19:33.953 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-03-18T23:19:33.953 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-03-18T23:19:33.953 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-03-18T23:19:33.953 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:33.953 INFO:teuthology.orchestra.run.smithi175.stdout: "req_rmxattr_latency": { 2024-03-18T23:19:33.953 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-03-18T23:19:33.954 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-03-18T23:19:33.954 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-03-18T23:19:33.954 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:33.954 INFO:teuthology.orchestra.run.smithi175.stdout: "req_setattr_latency": { 2024-03-18T23:19:33.954 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-03-18T23:19:33.954 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-03-18T23:19:33.954 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-03-18T23:19:33.954 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:33.954 INFO:teuthology.orchestra.run.smithi175.stdout: "req_setdirlayout_latency": { 2024-03-18T23:19:33.955 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-03-18T23:19:33.955 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-03-18T23:19:33.955 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-03-18T23:19:33.955 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:33.955 INFO:teuthology.orchestra.run.smithi175.stdout: "req_setfilelock_latency": { 2024-03-18T23:19:33.955 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-03-18T23:19:33.955 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-03-18T23:19:33.955 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-03-18T23:19:33.955 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:33.956 INFO:teuthology.orchestra.run.smithi175.stdout: "req_setlayout_latency": { 2024-03-18T23:19:33.956 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-03-18T23:19:33.956 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-03-18T23:19:33.956 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-03-18T23:19:33.956 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:33.956 INFO:teuthology.orchestra.run.smithi175.stdout: "req_setxattr_latency": { 2024-03-18T23:19:33.956 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-03-18T23:19:33.956 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-03-18T23:19:33.956 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-03-18T23:19:33.957 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:33.957 INFO:teuthology.orchestra.run.smithi175.stdout: "req_symlink_latency": { 2024-03-18T23:19:33.957 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-03-18T23:19:33.957 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-03-18T23:19:33.957 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-03-18T23:19:33.957 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:33.957 INFO:teuthology.orchestra.run.smithi175.stdout: "req_unlink_latency": { 2024-03-18T23:19:33.957 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-03-18T23:19:33.958 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-03-18T23:19:33.958 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-03-18T23:19:33.958 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:33.958 INFO:teuthology.orchestra.run.smithi175.stdout: "cap_revoke_eviction": 0, 2024-03-18T23:19:33.958 INFO:teuthology.orchestra.run.smithi175.stdout: "cap_acquisition_throttle": 0, 2024-03-18T23:19:33.958 INFO:teuthology.orchestra.run.smithi175.stdout: "req_getvxattr_latency": { 2024-03-18T23:19:33.958 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-03-18T23:19:33.958 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-03-18T23:19:33.958 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-03-18T23:19:33.959 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-03-18T23:19:33.959 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:33.959 INFO:teuthology.orchestra.run.smithi175.stdout: "mds_sessions": { 2024-03-18T23:19:33.959 INFO:teuthology.orchestra.run.smithi175.stdout: "session_count": 2, 2024-03-18T23:19:33.959 INFO:teuthology.orchestra.run.smithi175.stdout: "session_add": 4, 2024-03-18T23:19:33.959 INFO:teuthology.orchestra.run.smithi175.stdout: "session_remove": 2, 2024-03-18T23:19:33.959 INFO:teuthology.orchestra.run.smithi175.stdout: "sessions_open": 4, 2024-03-18T23:19:33.959 INFO:teuthology.orchestra.run.smithi175.stdout: "sessions_stale": 0, 2024-03-18T23:19:33.959 INFO:teuthology.orchestra.run.smithi175.stdout: "total_load": 0, 2024-03-18T23:19:33.960 INFO:teuthology.orchestra.run.smithi175.stdout: "average_load": 0, 2024-03-18T23:19:33.960 INFO:teuthology.orchestra.run.smithi175.stdout: "avg_session_uptime": 11834, 2024-03-18T23:19:33.960 INFO:teuthology.orchestra.run.smithi175.stdout: "mdthresh_evicted": 0 2024-03-18T23:19:33.960 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:33.960 INFO:teuthology.orchestra.run.smithi175.stdout: "mempool": { 2024-03-18T23:19:33.960 INFO:teuthology.orchestra.run.smithi175.stdout: "bloom_filter_bytes": 0, 2024-03-18T23:19:33.960 INFO:teuthology.orchestra.run.smithi175.stdout: "bloom_filter_items": 0, 2024-03-18T23:19:33.960 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_alloc_bytes": 0, 2024-03-18T23:19:33.961 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_alloc_items": 0, 2024-03-18T23:19:33.961 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_data_bytes": 0, 2024-03-18T23:19:33.961 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_data_items": 0, 2024-03-18T23:19:33.961 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-18T23:19:33.961 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_onode_items": 0, 2024-03-18T23:19:33.961 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-18T23:19:33.961 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_meta_items": 0, 2024-03-18T23:19:33.961 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_other_bytes": 0, 2024-03-18T23:19:33.961 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_other_items": 0, 2024-03-18T23:19:33.962 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-18T23:19:33.962 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_buffer_items": 0, 2024-03-18T23:19:33.962 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_extent_bytes": 0, 2024-03-18T23:19:33.962 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_extent_items": 0, 2024-03-18T23:19:33.962 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_blob_bytes": 0, 2024-03-18T23:19:33.962 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_blob_items": 0, 2024-03-18T23:19:33.962 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-18T23:19:33.962 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_shared_blob_items": 0, 2024-03-18T23:19:33.962 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-18T23:19:33.963 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_inline_bl_items": 0, 2024-03-18T23:19:33.963 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_fsck_bytes": 0, 2024-03-18T23:19:33.963 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_fsck_items": 0, 2024-03-18T23:19:33.963 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_txc_bytes": 0, 2024-03-18T23:19:33.963 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_txc_items": 0, 2024-03-18T23:19:33.963 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-18T23:19:33.963 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_writing_deferred_items": 0, 2024-03-18T23:19:33.963 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_writing_bytes": 0, 2024-03-18T23:19:33.964 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_writing_items": 0, 2024-03-18T23:19:33.964 INFO:teuthology.orchestra.run.smithi175.stdout: "bluefs_bytes": 0, 2024-03-18T23:19:33.964 INFO:teuthology.orchestra.run.smithi175.stdout: "bluefs_items": 0, 2024-03-18T23:19:33.964 INFO:teuthology.orchestra.run.smithi175.stdout: "bluefs_file_reader_bytes": 0, 2024-03-18T23:19:33.964 INFO:teuthology.orchestra.run.smithi175.stdout: "bluefs_file_reader_items": 0, 2024-03-18T23:19:33.964 INFO:teuthology.orchestra.run.smithi175.stdout: "bluefs_file_writer_bytes": 0, 2024-03-18T23:19:33.964 INFO:teuthology.orchestra.run.smithi175.stdout: "bluefs_file_writer_items": 0, 2024-03-18T23:19:33.964 INFO:teuthology.orchestra.run.smithi175.stdout: "buffer_anon_bytes": 181313, 2024-03-18T23:19:33.965 INFO:teuthology.orchestra.run.smithi175.stdout: "buffer_anon_items": 47, 2024-03-18T23:19:33.965 INFO:teuthology.orchestra.run.smithi175.stdout: "buffer_meta_bytes": 0, 2024-03-18T23:19:33.965 INFO:teuthology.orchestra.run.smithi175.stdout: "buffer_meta_items": 0, 2024-03-18T23:19:33.965 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_bytes": 0, 2024-03-18T23:19:33.965 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_items": 0, 2024-03-18T23:19:33.965 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_mapbl_bytes": 0, 2024-03-18T23:19:33.965 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_mapbl_items": 0, 2024-03-18T23:19:33.965 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_pglog_bytes": 0, 2024-03-18T23:19:33.965 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_pglog_items": 0, 2024-03-18T23:19:33.966 INFO:teuthology.orchestra.run.smithi175.stdout: "osdmap_bytes": 8256, 2024-03-18T23:19:33.966 INFO:teuthology.orchestra.run.smithi175.stdout: "osdmap_items": 188, 2024-03-18T23:19:33.966 INFO:teuthology.orchestra.run.smithi175.stdout: "osdmap_mapping_bytes": 0, 2024-03-18T23:19:33.966 INFO:teuthology.orchestra.run.smithi175.stdout: "osdmap_mapping_items": 0, 2024-03-18T23:19:33.966 INFO:teuthology.orchestra.run.smithi175.stdout: "pgmap_bytes": 0, 2024-03-18T23:19:33.966 INFO:teuthology.orchestra.run.smithi175.stdout: "pgmap_items": 0, 2024-03-18T23:19:33.966 INFO:teuthology.orchestra.run.smithi175.stdout: "mds_co_bytes": 8464, 2024-03-18T23:19:33.966 INFO:teuthology.orchestra.run.smithi175.stdout: "mds_co_items": 13, 2024-03-18T23:19:33.966 INFO:teuthology.orchestra.run.smithi175.stdout: "unittest_1_bytes": 0, 2024-03-18T23:19:33.967 INFO:teuthology.orchestra.run.smithi175.stdout: "unittest_1_items": 0, 2024-03-18T23:19:33.967 INFO:teuthology.orchestra.run.smithi175.stdout: "unittest_2_bytes": 0, 2024-03-18T23:19:33.967 INFO:teuthology.orchestra.run.smithi175.stdout: "unittest_2_items": 0 2024-03-18T23:19:33.967 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:33.967 INFO:teuthology.orchestra.run.smithi175.stdout: "objecter": { 2024-03-18T23:19:33.967 INFO:teuthology.orchestra.run.smithi175.stdout: "op_active": 0, 2024-03-18T23:19:33.967 INFO:teuthology.orchestra.run.smithi175.stdout: "op_laggy": 0, 2024-03-18T23:19:33.967 INFO:teuthology.orchestra.run.smithi175.stdout: "op_send": 2304, 2024-03-18T23:19:33.968 INFO:teuthology.orchestra.run.smithi175.stdout: "op_send_bytes": 21, 2024-03-18T23:19:33.968 INFO:teuthology.orchestra.run.smithi175.stdout: "op_resend": 3, 2024-03-18T23:19:33.968 INFO:teuthology.orchestra.run.smithi175.stdout: "op_reply": 2304, 2024-03-18T23:19:33.968 INFO:teuthology.orchestra.run.smithi175.stdout: "op_latency": { 2024-03-18T23:19:33.968 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 2304, 2024-03-18T23:19:33.968 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 1.185982629, 2024-03-18T23:19:33.968 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000514749 2024-03-18T23:19:33.968 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:33.969 INFO:teuthology.orchestra.run.smithi175.stdout: "op_inflight": 0, 2024-03-18T23:19:33.969 INFO:teuthology.orchestra.run.smithi175.stdout: "oplen_avg": { 2024-03-18T23:19:33.969 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 2304, 2024-03-18T23:19:33.969 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 2305 2024-03-18T23:19:33.969 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:33.969 INFO:teuthology.orchestra.run.smithi175.stdout: "op": 2304, 2024-03-18T23:19:33.969 INFO:teuthology.orchestra.run.smithi175.stdout: "op_r": 2304, 2024-03-18T23:19:33.969 INFO:teuthology.orchestra.run.smithi175.stdout: "op_w": 0, 2024-03-18T23:19:33.969 INFO:teuthology.orchestra.run.smithi175.stdout: "op_rmw": 0, 2024-03-18T23:19:33.970 INFO:teuthology.orchestra.run.smithi175.stdout: "op_pg": 0, 2024-03-18T23:19:33.970 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_stat": 1490, 2024-03-18T23:19:33.970 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_create": 0, 2024-03-18T23:19:33.970 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_read": 812, 2024-03-18T23:19:33.970 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_write": 0, 2024-03-18T23:19:33.970 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_writefull": 0, 2024-03-18T23:19:33.970 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_writesame": 0, 2024-03-18T23:19:33.970 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_append": 0, 2024-03-18T23:19:33.970 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_zero": 0, 2024-03-18T23:19:33.971 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_truncate": 0, 2024-03-18T23:19:33.971 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_delete": 0, 2024-03-18T23:19:33.971 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_mapext": 0, 2024-03-18T23:19:33.971 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_sparse_read": 0, 2024-03-18T23:19:33.971 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_clonerange": 0, 2024-03-18T23:19:33.971 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_getxattr": 1, 2024-03-18T23:19:33.971 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_setxattr": 0, 2024-03-18T23:19:33.971 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_cmpxattr": 0, 2024-03-18T23:19:33.971 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_rmxattr": 0, 2024-03-18T23:19:33.971 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_resetxattrs": 0, 2024-03-18T23:19:33.972 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_call": 0, 2024-03-18T23:19:33.972 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_watch": 0, 2024-03-18T23:19:33.972 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_notify": 0, 2024-03-18T23:19:33.972 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_src_cmpxattr": 0, 2024-03-18T23:19:33.972 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_pgls": 0, 2024-03-18T23:19:33.972 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_pgls_filter": 0, 2024-03-18T23:19:33.972 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_other": 0, 2024-03-18T23:19:33.972 INFO:teuthology.orchestra.run.smithi175.stdout: "linger_active": 0, 2024-03-18T23:19:33.973 INFO:teuthology.orchestra.run.smithi175.stdout: "linger_send": 0, 2024-03-18T23:19:33.973 INFO:teuthology.orchestra.run.smithi175.stdout: "linger_resend": 0, 2024-03-18T23:19:33.973 INFO:teuthology.orchestra.run.smithi175.stdout: "linger_ping": 0, 2024-03-18T23:19:33.973 INFO:teuthology.orchestra.run.smithi175.stdout: "poolop_active": 0, 2024-03-18T23:19:33.973 INFO:teuthology.orchestra.run.smithi175.stdout: "poolop_send": 0, 2024-03-18T23:19:33.973 INFO:teuthology.orchestra.run.smithi175.stdout: "poolop_resend": 0, 2024-03-18T23:19:33.973 INFO:teuthology.orchestra.run.smithi175.stdout: "poolstat_active": 0, 2024-03-18T23:19:33.973 INFO:teuthology.orchestra.run.smithi175.stdout: "poolstat_send": 0, 2024-03-18T23:19:33.973 INFO:teuthology.orchestra.run.smithi175.stdout: "poolstat_resend": 0, 2024-03-18T23:19:33.974 INFO:teuthology.orchestra.run.smithi175.stdout: "statfs_active": 0, 2024-03-18T23:19:33.974 INFO:teuthology.orchestra.run.smithi175.stdout: "statfs_send": 0, 2024-03-18T23:19:33.974 INFO:teuthology.orchestra.run.smithi175.stdout: "statfs_resend": 0, 2024-03-18T23:19:33.974 INFO:teuthology.orchestra.run.smithi175.stdout: "command_active": 0, 2024-03-18T23:19:33.974 INFO:teuthology.orchestra.run.smithi175.stdout: "command_send": 0, 2024-03-18T23:19:33.974 INFO:teuthology.orchestra.run.smithi175.stdout: "command_resend": 0, 2024-03-18T23:19:33.974 INFO:teuthology.orchestra.run.smithi175.stdout: "map_epoch": 101, 2024-03-18T23:19:33.974 INFO:teuthology.orchestra.run.smithi175.stdout: "map_full": 0, 2024-03-18T23:19:33.974 INFO:teuthology.orchestra.run.smithi175.stdout: "map_inc": 15, 2024-03-18T23:19:33.975 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_sessions": 4, 2024-03-18T23:19:33.975 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_session_open": 4, 2024-03-18T23:19:33.975 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_session_close": 0, 2024-03-18T23:19:33.975 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_laggy": 0, 2024-03-18T23:19:33.975 INFO:teuthology.orchestra.run.smithi175.stdout: "omap_wr": 0, 2024-03-18T23:19:33.975 INFO:teuthology.orchestra.run.smithi175.stdout: "omap_rd": 2, 2024-03-18T23:19:33.975 INFO:teuthology.orchestra.run.smithi175.stdout: "omap_del": 0 2024-03-18T23:19:33.975 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:33.975 INFO:teuthology.orchestra.run.smithi175.stdout: "oft": { 2024-03-18T23:19:33.976 INFO:teuthology.orchestra.run.smithi175.stdout: "omap_total_objs": 0, 2024-03-18T23:19:33.976 INFO:teuthology.orchestra.run.smithi175.stdout: "omap_total_kv_pairs": 0, 2024-03-18T23:19:33.976 INFO:teuthology.orchestra.run.smithi175.stdout: "omap_total_updates": 0, 2024-03-18T23:19:33.976 INFO:teuthology.orchestra.run.smithi175.stdout: "omap_total_removes": 0 2024-03-18T23:19:33.976 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:33.976 INFO:teuthology.orchestra.run.smithi175.stdout: "osd-slow-ops": { 2024-03-18T23:19:33.976 INFO:teuthology.orchestra.run.smithi175.stdout: "slow_ops_count": 0 2024-03-18T23:19:33.976 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:33.976 INFO:teuthology.orchestra.run.smithi175.stdout: "purge_queue": { 2024-03-18T23:19:33.977 INFO:teuthology.orchestra.run.smithi175.stdout: "pq_executing_ops": 0, 2024-03-18T23:19:33.977 INFO:teuthology.orchestra.run.smithi175.stdout: "pq_executing_ops_high_water": 0, 2024-03-18T23:19:33.977 INFO:teuthology.orchestra.run.smithi175.stdout: "pq_executing": 0, 2024-03-18T23:19:33.977 INFO:teuthology.orchestra.run.smithi175.stdout: "pq_executing_high_water": 0, 2024-03-18T23:19:33.977 INFO:teuthology.orchestra.run.smithi175.stdout: "pq_executed_ops": 0, 2024-03-18T23:19:33.977 INFO:teuthology.orchestra.run.smithi175.stdout: "pq_executed": 0, 2024-03-18T23:19:33.977 INFO:teuthology.orchestra.run.smithi175.stdout: "pq_item_in_journal": 0 2024-03-18T23:19:33.978 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:33.978 INFO:teuthology.orchestra.run.smithi175.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-18T23:19:33.978 INFO:teuthology.orchestra.run.smithi175.stdout: "val": 0, 2024-03-18T23:19:33.978 INFO:teuthology.orchestra.run.smithi175.stdout: "max": 104857600, 2024-03-18T23:19:33.978 INFO:teuthology.orchestra.run.smithi175.stdout: "get_started": 0, 2024-03-18T23:19:33.978 INFO:teuthology.orchestra.run.smithi175.stdout: "get": 2570, 2024-03-18T23:19:33.978 INFO:teuthology.orchestra.run.smithi175.stdout: "get_sum": 1938214, 2024-03-18T23:19:33.978 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_fail": 0, 2024-03-18T23:19:33.979 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_success": 2570, 2024-03-18T23:19:33.979 INFO:teuthology.orchestra.run.smithi175.stdout: "take": 0, 2024-03-18T23:19:33.979 INFO:teuthology.orchestra.run.smithi175.stdout: "take_sum": 0, 2024-03-18T23:19:33.979 INFO:teuthology.orchestra.run.smithi175.stdout: "put": 2570, 2024-03-18T23:19:33.979 INFO:teuthology.orchestra.run.smithi175.stdout: "put_sum": 1938214, 2024-03-18T23:19:33.979 INFO:teuthology.orchestra.run.smithi175.stdout: "wait": { 2024-03-18T23:19:33.980 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-03-18T23:19:33.980 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-03-18T23:19:33.980 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-03-18T23:19:33.980 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-03-18T23:19:33.980 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:33.980 INFO:teuthology.orchestra.run.smithi175.stdout: "throttle-objecter_bytes": { 2024-03-18T23:19:33.980 INFO:teuthology.orchestra.run.smithi175.stdout: "val": 0, 2024-03-18T23:19:33.981 INFO:teuthology.orchestra.run.smithi175.stdout: "max": 104857600, 2024-03-18T23:19:33.981 INFO:teuthology.orchestra.run.smithi175.stdout: "get_started": 0, 2024-03-18T23:19:33.981 INFO:teuthology.orchestra.run.smithi175.stdout: "get": 0, 2024-03-18T23:19:33.981 INFO:teuthology.orchestra.run.smithi175.stdout: "get_sum": 0, 2024-03-18T23:19:33.981 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_fail": 0, 2024-03-18T23:19:33.981 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_success": 0, 2024-03-18T23:19:33.982 INFO:teuthology.orchestra.run.smithi175.stdout: "take": 2304, 2024-03-18T23:19:33.982 INFO:teuthology.orchestra.run.smithi175.stdout: "take_sum": 2356, 2024-03-18T23:19:33.982 INFO:teuthology.orchestra.run.smithi175.stdout: "put": 2304, 2024-03-18T23:19:33.982 INFO:teuthology.orchestra.run.smithi175.stdout: "put_sum": 2356, 2024-03-18T23:19:33.982 INFO:teuthology.orchestra.run.smithi175.stdout: "wait": { 2024-03-18T23:19:33.982 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-03-18T23:19:33.982 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-03-18T23:19:33.982 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-03-18T23:19:33.983 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-03-18T23:19:33.983 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:33.983 INFO:teuthology.orchestra.run.smithi175.stdout: "throttle-objecter_ops": { 2024-03-18T23:19:33.983 INFO:teuthology.orchestra.run.smithi175.stdout: "val": 0, 2024-03-18T23:19:33.983 INFO:teuthology.orchestra.run.smithi175.stdout: "max": 1024, 2024-03-18T23:19:33.983 INFO:teuthology.orchestra.run.smithi175.stdout: "get_started": 0, 2024-03-18T23:19:33.983 INFO:teuthology.orchestra.run.smithi175.stdout: "get": 0, 2024-03-18T23:19:33.983 INFO:teuthology.orchestra.run.smithi175.stdout: "get_sum": 0, 2024-03-18T23:19:33.983 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_fail": 0, 2024-03-18T23:19:33.984 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_success": 0, 2024-03-18T23:19:33.984 INFO:teuthology.orchestra.run.smithi175.stdout: "take": 2304, 2024-03-18T23:19:33.984 INFO:teuthology.orchestra.run.smithi175.stdout: "take_sum": 2304, 2024-03-18T23:19:33.984 INFO:teuthology.orchestra.run.smithi175.stdout: "put": 2304, 2024-03-18T23:19:33.984 INFO:teuthology.orchestra.run.smithi175.stdout: "put_sum": 2304, 2024-03-18T23:19:33.984 INFO:teuthology.orchestra.run.smithi175.stdout: "wait": { 2024-03-18T23:19:33.984 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-03-18T23:19:33.984 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-03-18T23:19:33.984 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-03-18T23:19:33.985 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-03-18T23:19:33.985 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:33.985 INFO:teuthology.orchestra.run.smithi175.stdout: "throttle-write_buf_throttle": { 2024-03-18T23:19:33.985 INFO:teuthology.orchestra.run.smithi175.stdout: "val": 0, 2024-03-18T23:19:33.985 INFO:teuthology.orchestra.run.smithi175.stdout: "max": 3758096384, 2024-03-18T23:19:33.985 INFO:teuthology.orchestra.run.smithi175.stdout: "get_started": 0, 2024-03-18T23:19:33.985 INFO:teuthology.orchestra.run.smithi175.stdout: "get": 0, 2024-03-18T23:19:33.985 INFO:teuthology.orchestra.run.smithi175.stdout: "get_sum": 0, 2024-03-18T23:19:33.985 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_fail": 0, 2024-03-18T23:19:33.986 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_success": 0, 2024-03-18T23:19:33.986 INFO:teuthology.orchestra.run.smithi175.stdout: "take": 0, 2024-03-18T23:19:33.986 INFO:teuthology.orchestra.run.smithi175.stdout: "take_sum": 0, 2024-03-18T23:19:33.986 INFO:teuthology.orchestra.run.smithi175.stdout: "put": 0, 2024-03-18T23:19:33.986 INFO:teuthology.orchestra.run.smithi175.stdout: "put_sum": 0, 2024-03-18T23:19:33.986 INFO:teuthology.orchestra.run.smithi175.stdout: "wait": { 2024-03-18T23:19:33.986 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-03-18T23:19:33.986 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-03-18T23:19:33.987 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-03-18T23:19:33.987 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-03-18T23:19:33.987 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:33.987 INFO:teuthology.orchestra.run.smithi175.stdout: "throttle-write_buf_throttle-0x55c045f2e6e0": { 2024-03-18T23:19:33.987 INFO:teuthology.orchestra.run.smithi175.stdout: "val": 0, 2024-03-18T23:19:33.987 INFO:teuthology.orchestra.run.smithi175.stdout: "max": 3758096384, 2024-03-18T23:19:33.987 INFO:teuthology.orchestra.run.smithi175.stdout: "get_started": 0, 2024-03-18T23:19:33.987 INFO:teuthology.orchestra.run.smithi175.stdout: "get": 0, 2024-03-18T23:19:33.987 INFO:teuthology.orchestra.run.smithi175.stdout: "get_sum": 0, 2024-03-18T23:19:33.988 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_fail": 0, 2024-03-18T23:19:33.988 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_success": 0, 2024-03-18T23:19:33.988 INFO:teuthology.orchestra.run.smithi175.stdout: "take": 0, 2024-03-18T23:19:33.988 INFO:teuthology.orchestra.run.smithi175.stdout: "take_sum": 0, 2024-03-18T23:19:33.988 INFO:teuthology.orchestra.run.smithi175.stdout: "put": 0, 2024-03-18T23:19:33.988 INFO:teuthology.orchestra.run.smithi175.stdout: "put_sum": 0, 2024-03-18T23:19:33.988 INFO:teuthology.orchestra.run.smithi175.stdout: "wait": { 2024-03-18T23:19:33.988 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-03-18T23:19:33.988 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-03-18T23:19:33.989 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-03-18T23:19:33.989 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-03-18T23:19:33.989 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-03-18T23:19:33.989 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-03-18T23:19:34.564 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:19:34 smithi139 ceph-mon[67282]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:19:34.630 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:19:34 smithi049 ceph-mon[62991]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:19:34.638 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:19:34 smithi175 ceph-mon[67352]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:19:34.666 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2024-03-18T23:19:34.667 INFO:tasks.check_counter:Daemon mds.c mds.imported=0 2024-03-18T23:19:34.667 INFO:tasks.check_counter:Daemon mds.c mds.root_rsnaps=0 2024-03-18T23:19:34.667 INFO:tasks.check_counter:Daemon mds.c mds_server.req_mksnap_latency.avgcount=0 2024-03-18T23:19:34.667 INFO:tasks.check_counter:Daemon mds.c mds_server.req_rmsnap_latency.avgcount=0 2024-03-18T23:19:34.667 DEBUG:tasks.check_counter:Getting stats from f 2024-03-18T23:19:34.667 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell --fsid c63591ea-e57a-11ee-95c9-87774f69a715 -- ceph daemon mds.f perf dump 2024-03-18T23:19:34.994 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/mon.c/config 2024-03-18T23:19:35.324 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:19:35 smithi175 ceph-mon[67352]: pgmap v354: 97 pgs: 97 active+clean; 67 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 1.3 MiB/s wr, 62 op/s 2024-03-18T23:19:35.325 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:19:35 smithi175 ceph-mon[67352]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' 2024-03-18T23:19:35.325 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:19:35 smithi175 ceph-mon[67352]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' 2024-03-18T23:19:35.325 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:19:35 smithi175 ceph-mon[67352]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:19:35.325 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:19:35 smithi175 ceph-mon[67352]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:19:35.325 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:19:35 smithi175 ceph-mon[67352]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' 2024-03-18T23:19:35.564 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:19:35 smithi139 ceph-mon[67282]: pgmap v354: 97 pgs: 97 active+clean; 67 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 1.3 MiB/s wr, 62 op/s 2024-03-18T23:19:35.564 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:19:35 smithi139 ceph-mon[67282]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' 2024-03-18T23:19:35.564 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:19:35 smithi139 ceph-mon[67282]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' 2024-03-18T23:19:35.564 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:19:35 smithi139 ceph-mon[67282]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:19:35.564 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:19:35 smithi139 ceph-mon[67282]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:19:35.565 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:19:35 smithi139 ceph-mon[67282]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' 2024-03-18T23:19:35.629 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:19:35 smithi049 ceph-mon[62991]: pgmap v354: 97 pgs: 97 active+clean; 67 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 1.3 MiB/s wr, 62 op/s 2024-03-18T23:19:35.629 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:19:35 smithi049 ceph-mon[62991]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' 2024-03-18T23:19:35.629 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:19:35 smithi049 ceph-mon[62991]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' 2024-03-18T23:19:35.629 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:19:35 smithi049 ceph-mon[62991]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:19:35.629 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:19:35 smithi049 ceph-mon[62991]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:19:35.629 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:19:35 smithi049 ceph-mon[62991]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' 2024-03-18T23:19:36.747 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-03-18T23:19:36.748 INFO:teuthology.orchestra.run.smithi175.stdout: "AsyncMessenger::Worker-0": { 2024-03-18T23:19:36.748 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_messages": 174197, 2024-03-18T23:19:36.748 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_messages": 87206, 2024-03-18T23:19:36.748 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_bytes": 95100148, 2024-03-18T23:19:36.748 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_bytes": 14364205, 2024-03-18T23:19:36.748 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_created_connections": 8, 2024-03-18T23:19:36.748 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_active_connections": 2, 2024-03-18T23:19:36.748 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_total_time": 7.281462785, 2024-03-18T23:19:36.749 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_send_time": 2.806495692, 2024-03-18T23:19:36.749 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_recv_time": 5.895594841, 2024-03-18T23:19:36.749 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_fast_dispatch_time": 0.006493280, 2024-03-18T23:19:36.749 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_messages_queue_lat": { 2024-03-18T23:19:36.749 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 87209, 2024-03-18T23:19:36.749 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 1.788115300, 2024-03-18T23:19:36.749 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000020503 2024-03-18T23:19:36.749 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:36.750 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_handle_ack_lat": { 2024-03-18T23:19:36.750 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 260973, 2024-03-18T23:19:36.750 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.038729039, 2024-03-18T23:19:36.750 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000148 2024-03-18T23:19:36.750 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:36.750 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_encrypted_bytes": 3024, 2024-03-18T23:19:36.750 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_encrypted_bytes": 448 2024-03-18T23:19:36.750 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:36.751 INFO:teuthology.orchestra.run.smithi175.stdout: "AsyncMessenger::Worker-1": { 2024-03-18T23:19:36.751 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_messages": 7543, 2024-03-18T23:19:36.751 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_messages": 7589, 2024-03-18T23:19:36.751 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_bytes": 3843162, 2024-03-18T23:19:36.751 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_bytes": 1288681, 2024-03-18T23:19:36.751 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_created_connections": 6, 2024-03-18T23:19:36.751 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_active_connections": 5, 2024-03-18T23:19:36.751 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_total_time": 0.499644382, 2024-03-18T23:19:36.751 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_send_time": 0.190754389, 2024-03-18T23:19:36.751 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_recv_time": 0.416836556, 2024-03-18T23:19:36.752 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_fast_dispatch_time": 0.004988743, 2024-03-18T23:19:36.752 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_messages_queue_lat": { 2024-03-18T23:19:36.752 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 7588, 2024-03-18T23:19:36.752 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.143457630, 2024-03-18T23:19:36.752 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000018905 2024-03-18T23:19:36.752 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:36.752 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_handle_ack_lat": { 2024-03-18T23:19:36.752 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 14457, 2024-03-18T23:19:36.752 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.002028406, 2024-03-18T23:19:36.752 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000140 2024-03-18T23:19:36.753 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:36.753 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_encrypted_bytes": 3834960, 2024-03-18T23:19:36.753 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_encrypted_bytes": 1267872 2024-03-18T23:19:36.753 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:36.753 INFO:teuthology.orchestra.run.smithi175.stdout: "AsyncMessenger::Worker-2": { 2024-03-18T23:19:36.753 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_messages": 28, 2024-03-18T23:19:36.753 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_messages": 202, 2024-03-18T23:19:36.753 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_bytes": 7238, 2024-03-18T23:19:36.754 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_bytes": 519999, 2024-03-18T23:19:36.754 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_created_connections": 15, 2024-03-18T23:19:36.754 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_active_connections": 10, 2024-03-18T23:19:36.754 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_total_time": 0.012230696, 2024-03-18T23:19:36.754 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_send_time": 0.005211220, 2024-03-18T23:19:36.754 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_recv_time": 0.004928975, 2024-03-18T23:19:36.754 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_fast_dispatch_time": 0.000438456, 2024-03-18T23:19:36.754 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_messages_queue_lat": { 2024-03-18T23:19:36.754 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 195, 2024-03-18T23:19:36.755 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.017416187, 2024-03-18T23:19:36.755 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000089313 2024-03-18T23:19:36.755 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:36.755 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_handle_ack_lat": { 2024-03-18T23:19:36.755 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 12, 2024-03-18T23:19:36.755 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000002009, 2024-03-18T23:19:36.755 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000167 2024-03-18T23:19:36.755 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:36.756 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_encrypted_bytes": 1248, 2024-03-18T23:19:36.756 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_encrypted_bytes": 465472 2024-03-18T23:19:36.756 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:36.756 INFO:teuthology.orchestra.run.smithi175.stdout: "cct": { 2024-03-18T23:19:36.756 INFO:teuthology.orchestra.run.smithi175.stdout: "total_workers": 1, 2024-03-18T23:19:36.756 INFO:teuthology.orchestra.run.smithi175.stdout: "unhealthy_workers": 0 2024-03-18T23:19:36.756 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:36.756 INFO:teuthology.orchestra.run.smithi175.stdout: "finisher-MDSRank": { 2024-03-18T23:19:36.756 INFO:teuthology.orchestra.run.smithi175.stdout: "queue_len": 0, 2024-03-18T23:19:36.757 INFO:teuthology.orchestra.run.smithi175.stdout: "complete_latency": { 2024-03-18T23:19:36.757 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 243, 2024-03-18T23:19:36.757 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.043757925, 2024-03-18T23:19:36.757 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000180073 2024-03-18T23:19:36.757 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-03-18T23:19:36.757 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:36.757 INFO:teuthology.orchestra.run.smithi175.stdout: "finisher-PurgeQueue": { 2024-03-18T23:19:36.757 INFO:teuthology.orchestra.run.smithi175.stdout: "queue_len": 0, 2024-03-18T23:19:36.758 INFO:teuthology.orchestra.run.smithi175.stdout: "complete_latency": { 2024-03-18T23:19:36.758 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 31, 2024-03-18T23:19:36.758 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.001180817, 2024-03-18T23:19:36.758 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000038090 2024-03-18T23:19:36.758 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-03-18T23:19:36.758 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:36.758 INFO:teuthology.orchestra.run.smithi175.stdout: "mds": { 2024-03-18T23:19:36.758 INFO:teuthology.orchestra.run.smithi175.stdout: "request": 6926, 2024-03-18T23:19:36.759 INFO:teuthology.orchestra.run.smithi175.stdout: "reply": 27, 2024-03-18T23:19:36.759 INFO:teuthology.orchestra.run.smithi175.stdout: "reply_latency": { 2024-03-18T23:19:36.759 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 27, 2024-03-18T23:19:36.759 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.040522580, 2024-03-18T23:19:36.759 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.001500836 2024-03-18T23:19:36.759 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:36.759 INFO:teuthology.orchestra.run.smithi175.stdout: "slow_reply": 0, 2024-03-18T23:19:36.759 INFO:teuthology.orchestra.run.smithi175.stdout: "forward": 6899, 2024-03-18T23:19:36.760 INFO:teuthology.orchestra.run.smithi175.stdout: "dir_fetch_complete": 0, 2024-03-18T23:19:36.760 INFO:teuthology.orchestra.run.smithi175.stdout: "dir_fetch_keys": 0, 2024-03-18T23:19:36.760 INFO:teuthology.orchestra.run.smithi175.stdout: "dir_commit": 11, 2024-03-18T23:19:36.760 INFO:teuthology.orchestra.run.smithi175.stdout: "dir_split": 0, 2024-03-18T23:19:36.760 INFO:teuthology.orchestra.run.smithi175.stdout: "dir_merge": 0, 2024-03-18T23:19:36.760 INFO:teuthology.orchestra.run.smithi175.stdout: "inodes": 60, 2024-03-18T23:19:36.760 INFO:teuthology.orchestra.run.smithi175.stdout: "inodes_top": 2, 2024-03-18T23:19:36.760 INFO:teuthology.orchestra.run.smithi175.stdout: "inodes_bottom": 58, 2024-03-18T23:19:36.760 INFO:teuthology.orchestra.run.smithi175.stdout: "inodes_pin_tail": 0, 2024-03-18T23:19:36.761 INFO:teuthology.orchestra.run.smithi175.stdout: "inodes_pinned": 56, 2024-03-18T23:19:36.761 INFO:teuthology.orchestra.run.smithi175.stdout: "inodes_expired": 0, 2024-03-18T23:19:36.761 INFO:teuthology.orchestra.run.smithi175.stdout: "inodes_with_caps": 4, 2024-03-18T23:19:36.761 INFO:teuthology.orchestra.run.smithi175.stdout: "caps": 4, 2024-03-18T23:19:36.761 INFO:teuthology.orchestra.run.smithi175.stdout: "subtrees": 3, 2024-03-18T23:19:36.761 INFO:teuthology.orchestra.run.smithi175.stdout: "traverse": 6971, 2024-03-18T23:19:36.761 INFO:teuthology.orchestra.run.smithi175.stdout: "traverse_hit": 28, 2024-03-18T23:19:36.761 INFO:teuthology.orchestra.run.smithi175.stdout: "traverse_forward": 6899, 2024-03-18T23:19:36.761 INFO:teuthology.orchestra.run.smithi175.stdout: "traverse_discover": 4, 2024-03-18T23:19:36.762 INFO:teuthology.orchestra.run.smithi175.stdout: "traverse_dir_fetch": 0, 2024-03-18T23:19:36.762 INFO:teuthology.orchestra.run.smithi175.stdout: "traverse_remote_ino": 0, 2024-03-18T23:19:36.762 INFO:teuthology.orchestra.run.smithi175.stdout: "traverse_lock": 0, 2024-03-18T23:19:36.762 INFO:teuthology.orchestra.run.smithi175.stdout: "load_cent": 0, 2024-03-18T23:19:36.762 INFO:teuthology.orchestra.run.smithi175.stdout: "q": 0, 2024-03-18T23:19:36.762 INFO:teuthology.orchestra.run.smithi175.stdout: "exported": 0, 2024-03-18T23:19:36.762 INFO:teuthology.orchestra.run.smithi175.stdout: "exported_inodes": 0, 2024-03-18T23:19:36.762 INFO:teuthology.orchestra.run.smithi175.stdout: "imported": 1, 2024-03-18T23:19:36.763 INFO:teuthology.orchestra.run.smithi175.stdout: "imported_inodes": 3, 2024-03-18T23:19:36.763 INFO:teuthology.orchestra.run.smithi175.stdout: "openino_dir_fetch": 0, 2024-03-18T23:19:36.763 INFO:teuthology.orchestra.run.smithi175.stdout: "openino_backtrace_fetch": 0, 2024-03-18T23:19:36.763 INFO:teuthology.orchestra.run.smithi175.stdout: "openino_peer_discover": 0, 2024-03-18T23:19:36.763 INFO:teuthology.orchestra.run.smithi175.stdout: "root_rfiles": 69, 2024-03-18T23:19:36.763 INFO:teuthology.orchestra.run.smithi175.stdout: "root_rbytes": 21153440, 2024-03-18T23:19:36.763 INFO:teuthology.orchestra.run.smithi175.stdout: "root_rsnaps": 6, 2024-03-18T23:19:36.763 INFO:teuthology.orchestra.run.smithi175.stdout: "scrub_backtrace_fetch": 0, 2024-03-18T23:19:36.764 INFO:teuthology.orchestra.run.smithi175.stdout: "scrub_set_tag": 0, 2024-03-18T23:19:36.764 INFO:teuthology.orchestra.run.smithi175.stdout: "scrub_backtrace_repaired": 0, 2024-03-18T23:19:36.764 INFO:teuthology.orchestra.run.smithi175.stdout: "scrub_inotable_repaired": 0, 2024-03-18T23:19:36.764 INFO:teuthology.orchestra.run.smithi175.stdout: "scrub_dir_inodes": 0, 2024-03-18T23:19:36.764 INFO:teuthology.orchestra.run.smithi175.stdout: "scrub_dir_base_inodes": 0, 2024-03-18T23:19:36.764 INFO:teuthology.orchestra.run.smithi175.stdout: "scrub_dirfrag_rstats": 0, 2024-03-18T23:19:36.764 INFO:teuthology.orchestra.run.smithi175.stdout: "scrub_file_inodes": 0, 2024-03-18T23:19:36.764 INFO:teuthology.orchestra.run.smithi175.stdout: "handle_inode_file_caps": 0, 2024-03-18T23:19:36.765 INFO:teuthology.orchestra.run.smithi175.stdout: "ceph_cap_op_revoke": 12, 2024-03-18T23:19:36.765 INFO:teuthology.orchestra.run.smithi175.stdout: "ceph_cap_op_grant": 33, 2024-03-18T23:19:36.765 INFO:teuthology.orchestra.run.smithi175.stdout: "ceph_cap_op_trunc": 0, 2024-03-18T23:19:36.765 INFO:teuthology.orchestra.run.smithi175.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-18T23:19:36.765 INFO:teuthology.orchestra.run.smithi175.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-18T23:19:36.765 INFO:teuthology.orchestra.run.smithi175.stdout: "handle_client_caps": 126, 2024-03-18T23:19:36.765 INFO:teuthology.orchestra.run.smithi175.stdout: "handle_client_caps_dirty": 107, 2024-03-18T23:19:36.765 INFO:teuthology.orchestra.run.smithi175.stdout: "handle_client_cap_release": 4, 2024-03-18T23:19:36.765 INFO:teuthology.orchestra.run.smithi175.stdout: "process_request_cap_release": 25 2024-03-18T23:19:36.766 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:36.766 INFO:teuthology.orchestra.run.smithi175.stdout: "mds_cache": { 2024-03-18T23:19:36.766 INFO:teuthology.orchestra.run.smithi175.stdout: "dir_update": 0, 2024-03-18T23:19:36.766 INFO:teuthology.orchestra.run.smithi175.stdout: "dir_update_receipt": 9, 2024-03-18T23:19:36.766 INFO:teuthology.orchestra.run.smithi175.stdout: "dir_try_discover": 6, 2024-03-18T23:19:36.766 INFO:teuthology.orchestra.run.smithi175.stdout: "dir_send_discover": 8, 2024-03-18T23:19:36.766 INFO:teuthology.orchestra.run.smithi175.stdout: "dir_handle_discover": 0, 2024-03-18T23:19:36.766 INFO:teuthology.orchestra.run.smithi175.stdout: "num_strays": 0, 2024-03-18T23:19:36.767 INFO:teuthology.orchestra.run.smithi175.stdout: "num_strays_delayed": 0, 2024-03-18T23:19:36.767 INFO:teuthology.orchestra.run.smithi175.stdout: "num_strays_enqueuing": 0, 2024-03-18T23:19:36.767 INFO:teuthology.orchestra.run.smithi175.stdout: "strays_created": 6, 2024-03-18T23:19:36.767 INFO:teuthology.orchestra.run.smithi175.stdout: "strays_enqueued": 6, 2024-03-18T23:19:36.767 INFO:teuthology.orchestra.run.smithi175.stdout: "strays_reintegrated": 0, 2024-03-18T23:19:36.767 INFO:teuthology.orchestra.run.smithi175.stdout: "strays_migrated": 0, 2024-03-18T23:19:36.767 INFO:teuthology.orchestra.run.smithi175.stdout: "num_recovering_processing": 0, 2024-03-18T23:19:36.768 INFO:teuthology.orchestra.run.smithi175.stdout: "num_recovering_enqueued": 0, 2024-03-18T23:19:36.768 INFO:teuthology.orchestra.run.smithi175.stdout: "num_recovering_prioritized": 0, 2024-03-18T23:19:36.768 INFO:teuthology.orchestra.run.smithi175.stdout: "recovery_started": 0, 2024-03-18T23:19:36.768 INFO:teuthology.orchestra.run.smithi175.stdout: "recovery_completed": 0, 2024-03-18T23:19:36.768 INFO:teuthology.orchestra.run.smithi175.stdout: "ireq_quiesce_path": 0, 2024-03-18T23:19:36.768 INFO:teuthology.orchestra.run.smithi175.stdout: "ireq_quiesce_inode": 0, 2024-03-18T23:19:36.768 INFO:teuthology.orchestra.run.smithi175.stdout: "ireq_enqueue_scrub": 0, 2024-03-18T23:19:36.768 INFO:teuthology.orchestra.run.smithi175.stdout: "ireq_exportdir": 0, 2024-03-18T23:19:36.768 INFO:teuthology.orchestra.run.smithi175.stdout: "ireq_flush": 0, 2024-03-18T23:19:36.768 INFO:teuthology.orchestra.run.smithi175.stdout: "ireq_fragmentdir": 0, 2024-03-18T23:19:36.769 INFO:teuthology.orchestra.run.smithi175.stdout: "ireq_fragstats": 0, 2024-03-18T23:19:36.769 INFO:teuthology.orchestra.run.smithi175.stdout: "ireq_inodestats": 0 2024-03-18T23:19:36.769 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:36.769 INFO:teuthology.orchestra.run.smithi175.stdout: "mds_log": { 2024-03-18T23:19:36.769 INFO:teuthology.orchestra.run.smithi175.stdout: "evlrg": 0, 2024-03-18T23:19:36.769 INFO:teuthology.orchestra.run.smithi175.stdout: "evadd": 184, 2024-03-18T23:19:36.769 INFO:teuthology.orchestra.run.smithi175.stdout: "evex": 0, 2024-03-18T23:19:36.769 INFO:teuthology.orchestra.run.smithi175.stdout: "evtrm": 0, 2024-03-18T23:19:36.769 INFO:teuthology.orchestra.run.smithi175.stdout: "ev": 184, 2024-03-18T23:19:36.770 INFO:teuthology.orchestra.run.smithi175.stdout: "evexg": 0, 2024-03-18T23:19:36.770 INFO:teuthology.orchestra.run.smithi175.stdout: "evexd": 0, 2024-03-18T23:19:36.770 INFO:teuthology.orchestra.run.smithi175.stdout: "segadd": 2, 2024-03-18T23:19:36.770 INFO:teuthology.orchestra.run.smithi175.stdout: "segex": 0, 2024-03-18T23:19:36.770 INFO:teuthology.orchestra.run.smithi175.stdout: "segtrm": 0, 2024-03-18T23:19:36.770 INFO:teuthology.orchestra.run.smithi175.stdout: "seg": 2, 2024-03-18T23:19:36.770 INFO:teuthology.orchestra.run.smithi175.stdout: "segmjr": 2, 2024-03-18T23:19:36.770 INFO:teuthology.orchestra.run.smithi175.stdout: "segexg": 0, 2024-03-18T23:19:36.771 INFO:teuthology.orchestra.run.smithi175.stdout: "segexd": 0, 2024-03-18T23:19:36.771 INFO:teuthology.orchestra.run.smithi175.stdout: "expos": 4194304, 2024-03-18T23:19:36.771 INFO:teuthology.orchestra.run.smithi175.stdout: "wrpos": 4444684, 2024-03-18T23:19:36.771 INFO:teuthology.orchestra.run.smithi175.stdout: "rdpos": 0, 2024-03-18T23:19:36.771 INFO:teuthology.orchestra.run.smithi175.stdout: "jlat": { 2024-03-18T23:19:36.771 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 89, 2024-03-18T23:19:36.771 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.287158104, 2024-03-18T23:19:36.771 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.003226495 2024-03-18T23:19:36.771 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:36.772 INFO:teuthology.orchestra.run.smithi175.stdout: "replayed": 0 2024-03-18T23:19:36.772 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:36.772 INFO:teuthology.orchestra.run.smithi175.stdout: "mds_mem": { 2024-03-18T23:19:36.772 INFO:teuthology.orchestra.run.smithi175.stdout: "ino": 62, 2024-03-18T23:19:36.772 INFO:teuthology.orchestra.run.smithi175.stdout: "ino+": 68, 2024-03-18T23:19:36.772 INFO:teuthology.orchestra.run.smithi175.stdout: "ino-": 6, 2024-03-18T23:19:36.772 INFO:teuthology.orchestra.run.smithi175.stdout: "dir": 39, 2024-03-18T23:19:36.772 INFO:teuthology.orchestra.run.smithi175.stdout: "dir+": 42, 2024-03-18T23:19:36.773 INFO:teuthology.orchestra.run.smithi175.stdout: "dir-": 3, 2024-03-18T23:19:36.773 INFO:teuthology.orchestra.run.smithi175.stdout: "dn": 60, 2024-03-18T23:19:36.773 INFO:teuthology.orchestra.run.smithi175.stdout: "dn+": 66, 2024-03-18T23:19:36.773 INFO:teuthology.orchestra.run.smithi175.stdout: "dn-": 6, 2024-03-18T23:19:36.773 INFO:teuthology.orchestra.run.smithi175.stdout: "cap": 4, 2024-03-18T23:19:36.773 INFO:teuthology.orchestra.run.smithi175.stdout: "cap+": 10, 2024-03-18T23:19:36.773 INFO:teuthology.orchestra.run.smithi175.stdout: "cap-": 6, 2024-03-18T23:19:36.773 INFO:teuthology.orchestra.run.smithi175.stdout: "rss": 54072, 2024-03-18T23:19:36.773 INFO:teuthology.orchestra.run.smithi175.stdout: "heap": 223612 2024-03-18T23:19:36.774 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:36.774 INFO:teuthology.orchestra.run.smithi175.stdout: "mds_server": { 2024-03-18T23:19:36.774 INFO:teuthology.orchestra.run.smithi175.stdout: "dispatch_client_request": 6956, 2024-03-18T23:19:36.774 INFO:teuthology.orchestra.run.smithi175.stdout: "dispatch_server_request": 0, 2024-03-18T23:19:36.774 INFO:teuthology.orchestra.run.smithi175.stdout: "handle_client_request": 6926, 2024-03-18T23:19:36.774 INFO:teuthology.orchestra.run.smithi175.stdout: "handle_client_session": 13, 2024-03-18T23:19:36.774 INFO:teuthology.orchestra.run.smithi175.stdout: "handle_peer_request": 12, 2024-03-18T23:19:36.774 INFO:teuthology.orchestra.run.smithi175.stdout: "req_create_latency": { 2024-03-18T23:19:36.775 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 6, 2024-03-18T23:19:36.775 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.016235858, 2024-03-18T23:19:36.775 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.002705976 2024-03-18T23:19:36.775 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:36.775 INFO:teuthology.orchestra.run.smithi175.stdout: "req_getattr_latency": { 2024-03-18T23:19:36.775 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-03-18T23:19:36.775 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-03-18T23:19:36.775 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-03-18T23:19:36.776 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:36.776 INFO:teuthology.orchestra.run.smithi175.stdout: "req_getfilelock_latency": { 2024-03-18T23:19:36.776 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-03-18T23:19:36.776 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-03-18T23:19:36.776 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-03-18T23:19:36.776 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:36.776 INFO:teuthology.orchestra.run.smithi175.stdout: "req_link_latency": { 2024-03-18T23:19:36.776 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-03-18T23:19:36.776 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-03-18T23:19:36.777 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-03-18T23:19:36.777 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:36.777 INFO:teuthology.orchestra.run.smithi175.stdout: "req_lookup_latency": { 2024-03-18T23:19:36.777 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 8, 2024-03-18T23:19:36.777 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.002409349, 2024-03-18T23:19:36.777 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000301168 2024-03-18T23:19:36.777 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:36.777 INFO:teuthology.orchestra.run.smithi175.stdout: "req_lookuphash_latency": { 2024-03-18T23:19:36.777 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-03-18T23:19:36.778 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-03-18T23:19:36.778 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-03-18T23:19:36.778 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:36.778 INFO:teuthology.orchestra.run.smithi175.stdout: "req_lookupino_latency": { 2024-03-18T23:19:36.778 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-03-18T23:19:36.778 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-03-18T23:19:36.778 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-03-18T23:19:36.778 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:36.779 INFO:teuthology.orchestra.run.smithi175.stdout: "req_lookupname_latency": { 2024-03-18T23:19:36.779 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-03-18T23:19:36.781 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-03-18T23:19:36.781 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-03-18T23:19:36.782 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:36.782 INFO:teuthology.orchestra.run.smithi175.stdout: "req_lookupparent_latency": { 2024-03-18T23:19:36.782 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-03-18T23:19:36.782 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-03-18T23:19:36.782 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-03-18T23:19:36.782 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:36.782 INFO:teuthology.orchestra.run.smithi175.stdout: "req_lookupsnap_latency": { 2024-03-18T23:19:36.783 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-03-18T23:19:36.783 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-03-18T23:19:36.783 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-03-18T23:19:36.783 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:36.783 INFO:teuthology.orchestra.run.smithi175.stdout: "req_lssnap_latency": { 2024-03-18T23:19:36.783 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-03-18T23:19:36.783 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-03-18T23:19:36.783 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-03-18T23:19:36.783 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:36.784 INFO:teuthology.orchestra.run.smithi175.stdout: "req_mkdir_latency": { 2024-03-18T23:19:36.784 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-03-18T23:19:36.785 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-03-18T23:19:36.785 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-03-18T23:19:36.785 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:36.785 INFO:teuthology.orchestra.run.smithi175.stdout: "req_mknod_latency": { 2024-03-18T23:19:36.785 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-03-18T23:19:36.785 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-03-18T23:19:36.785 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-03-18T23:19:36.785 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:36.786 INFO:teuthology.orchestra.run.smithi175.stdout: "req_mksnap_latency": { 2024-03-18T23:19:36.786 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-03-18T23:19:36.786 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-03-18T23:19:36.786 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-03-18T23:19:36.786 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:36.786 INFO:teuthology.orchestra.run.smithi175.stdout: "req_open_latency": { 2024-03-18T23:19:36.786 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-03-18T23:19:36.786 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-03-18T23:19:36.787 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-03-18T23:19:36.787 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:36.787 INFO:teuthology.orchestra.run.smithi175.stdout: "req_readdir_latency": { 2024-03-18T23:19:36.787 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 7, 2024-03-18T23:19:36.787 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.003775433, 2024-03-18T23:19:36.787 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000539347 2024-03-18T23:19:36.787 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:36.787 INFO:teuthology.orchestra.run.smithi175.stdout: "req_rename_latency": { 2024-03-18T23:19:36.787 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-03-18T23:19:36.788 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-03-18T23:19:36.788 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-03-18T23:19:36.788 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:36.788 INFO:teuthology.orchestra.run.smithi175.stdout: "req_renamesnap_latency": { 2024-03-18T23:19:36.788 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-03-18T23:19:36.788 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-03-18T23:19:36.788 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-03-18T23:19:36.788 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:36.788 INFO:teuthology.orchestra.run.smithi175.stdout: "req_snapdiff_latency": { 2024-03-18T23:19:36.789 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-03-18T23:19:36.789 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-03-18T23:19:36.789 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-03-18T23:19:36.789 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:36.789 INFO:teuthology.orchestra.run.smithi175.stdout: "req_rmdir_latency": { 2024-03-18T23:19:36.789 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-03-18T23:19:36.789 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-03-18T23:19:36.789 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-03-18T23:19:36.789 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:36.790 INFO:teuthology.orchestra.run.smithi175.stdout: "req_rmsnap_latency": { 2024-03-18T23:19:36.790 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-03-18T23:19:36.790 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-03-18T23:19:36.790 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-03-18T23:19:36.790 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:36.790 INFO:teuthology.orchestra.run.smithi175.stdout: "req_rmxattr_latency": { 2024-03-18T23:19:36.790 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-03-18T23:19:36.790 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-03-18T23:19:36.791 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-03-18T23:19:36.791 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:36.791 INFO:teuthology.orchestra.run.smithi175.stdout: "req_setattr_latency": { 2024-03-18T23:19:36.791 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-03-18T23:19:36.791 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-03-18T23:19:36.791 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-03-18T23:19:36.791 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:36.791 INFO:teuthology.orchestra.run.smithi175.stdout: "req_setdirlayout_latency": { 2024-03-18T23:19:36.791 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-03-18T23:19:36.792 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-03-18T23:19:36.792 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-03-18T23:19:36.792 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:36.792 INFO:teuthology.orchestra.run.smithi175.stdout: "req_setfilelock_latency": { 2024-03-18T23:19:36.792 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-03-18T23:19:36.792 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-03-18T23:19:36.792 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-03-18T23:19:36.792 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:36.793 INFO:teuthology.orchestra.run.smithi175.stdout: "req_setlayout_latency": { 2024-03-18T23:19:36.793 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-03-18T23:19:36.793 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-03-18T23:19:36.793 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-03-18T23:19:36.793 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:36.793 INFO:teuthology.orchestra.run.smithi175.stdout: "req_setxattr_latency": { 2024-03-18T23:19:36.793 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-03-18T23:19:36.794 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-03-18T23:19:36.794 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-03-18T23:19:36.794 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:36.794 INFO:teuthology.orchestra.run.smithi175.stdout: "req_symlink_latency": { 2024-03-18T23:19:36.794 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-03-18T23:19:36.794 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-03-18T23:19:36.794 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-03-18T23:19:36.794 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:36.794 INFO:teuthology.orchestra.run.smithi175.stdout: "req_unlink_latency": { 2024-03-18T23:19:36.795 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 6, 2024-03-18T23:19:36.795 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.018101940, 2024-03-18T23:19:36.795 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.003016990 2024-03-18T23:19:36.795 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:36.795 INFO:teuthology.orchestra.run.smithi175.stdout: "cap_revoke_eviction": 0, 2024-03-18T23:19:36.795 INFO:teuthology.orchestra.run.smithi175.stdout: "cap_acquisition_throttle": 0, 2024-03-18T23:19:36.795 INFO:teuthology.orchestra.run.smithi175.stdout: "req_getvxattr_latency": { 2024-03-18T23:19:36.795 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-03-18T23:19:36.796 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-03-18T23:19:36.796 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-03-18T23:19:36.796 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-03-18T23:19:36.796 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:36.796 INFO:teuthology.orchestra.run.smithi175.stdout: "mds_sessions": { 2024-03-18T23:19:36.796 INFO:teuthology.orchestra.run.smithi175.stdout: "session_count": 1, 2024-03-18T23:19:36.796 INFO:teuthology.orchestra.run.smithi175.stdout: "session_add": 2, 2024-03-18T23:19:36.796 INFO:teuthology.orchestra.run.smithi175.stdout: "session_remove": 1, 2024-03-18T23:19:36.796 INFO:teuthology.orchestra.run.smithi175.stdout: "sessions_open": 1, 2024-03-18T23:19:36.797 INFO:teuthology.orchestra.run.smithi175.stdout: "sessions_stale": 0, 2024-03-18T23:19:36.797 INFO:teuthology.orchestra.run.smithi175.stdout: "total_load": 11, 2024-03-18T23:19:36.797 INFO:teuthology.orchestra.run.smithi175.stdout: "average_load": 11, 2024-03-18T23:19:36.797 INFO:teuthology.orchestra.run.smithi175.stdout: "avg_session_uptime": 678, 2024-03-18T23:19:36.797 INFO:teuthology.orchestra.run.smithi175.stdout: "mdthresh_evicted": 0 2024-03-18T23:19:36.797 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:36.797 INFO:teuthology.orchestra.run.smithi175.stdout: "mempool": { 2024-03-18T23:19:36.797 INFO:teuthology.orchestra.run.smithi175.stdout: "bloom_filter_bytes": 0, 2024-03-18T23:19:36.797 INFO:teuthology.orchestra.run.smithi175.stdout: "bloom_filter_items": 0, 2024-03-18T23:19:36.798 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_alloc_bytes": 0, 2024-03-18T23:19:36.798 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_alloc_items": 0, 2024-03-18T23:19:36.798 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_data_bytes": 0, 2024-03-18T23:19:36.798 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_data_items": 0, 2024-03-18T23:19:36.798 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-18T23:19:36.798 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_onode_items": 0, 2024-03-18T23:19:36.798 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-18T23:19:36.798 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_meta_items": 0, 2024-03-18T23:19:36.798 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_other_bytes": 0, 2024-03-18T23:19:36.799 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_other_items": 0, 2024-03-18T23:19:36.799 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-18T23:19:36.799 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_buffer_items": 0, 2024-03-18T23:19:36.799 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_extent_bytes": 0, 2024-03-18T23:19:36.799 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_extent_items": 0, 2024-03-18T23:19:36.799 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_blob_bytes": 0, 2024-03-18T23:19:36.799 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_blob_items": 0, 2024-03-18T23:19:36.799 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-18T23:19:36.800 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_shared_blob_items": 0, 2024-03-18T23:19:36.800 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-18T23:19:36.800 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_inline_bl_items": 0, 2024-03-18T23:19:36.800 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_fsck_bytes": 0, 2024-03-18T23:19:36.800 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_fsck_items": 0, 2024-03-18T23:19:36.800 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_txc_bytes": 0, 2024-03-18T23:19:36.800 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_txc_items": 0, 2024-03-18T23:19:36.800 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-18T23:19:36.800 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_writing_deferred_items": 0, 2024-03-18T23:19:36.800 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_writing_bytes": 0, 2024-03-18T23:19:36.801 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_writing_items": 0, 2024-03-18T23:19:36.801 INFO:teuthology.orchestra.run.smithi175.stdout: "bluefs_bytes": 0, 2024-03-18T23:19:36.801 INFO:teuthology.orchestra.run.smithi175.stdout: "bluefs_items": 0, 2024-03-18T23:19:36.801 INFO:teuthology.orchestra.run.smithi175.stdout: "bluefs_file_reader_bytes": 0, 2024-03-18T23:19:36.801 INFO:teuthology.orchestra.run.smithi175.stdout: "bluefs_file_reader_items": 0, 2024-03-18T23:19:36.801 INFO:teuthology.orchestra.run.smithi175.stdout: "bluefs_file_writer_bytes": 0, 2024-03-18T23:19:36.801 INFO:teuthology.orchestra.run.smithi175.stdout: "bluefs_file_writer_items": 0, 2024-03-18T23:19:36.801 INFO:teuthology.orchestra.run.smithi175.stdout: "buffer_anon_bytes": 269475, 2024-03-18T23:19:36.802 INFO:teuthology.orchestra.run.smithi175.stdout: "buffer_anon_items": 141, 2024-03-18T23:19:36.802 INFO:teuthology.orchestra.run.smithi175.stdout: "buffer_meta_bytes": 528, 2024-03-18T23:19:36.802 INFO:teuthology.orchestra.run.smithi175.stdout: "buffer_meta_items": 6, 2024-03-18T23:19:36.802 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_bytes": 0, 2024-03-18T23:19:36.802 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_items": 0, 2024-03-18T23:19:36.802 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_mapbl_bytes": 0, 2024-03-18T23:19:36.802 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_mapbl_items": 0, 2024-03-18T23:19:36.802 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_pglog_bytes": 0, 2024-03-18T23:19:36.803 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_pglog_items": 0, 2024-03-18T23:19:36.803 INFO:teuthology.orchestra.run.smithi175.stdout: "osdmap_bytes": 8256, 2024-03-18T23:19:36.803 INFO:teuthology.orchestra.run.smithi175.stdout: "osdmap_items": 188, 2024-03-18T23:19:36.803 INFO:teuthology.orchestra.run.smithi175.stdout: "osdmap_mapping_bytes": 0, 2024-03-18T23:19:36.803 INFO:teuthology.orchestra.run.smithi175.stdout: "osdmap_mapping_items": 0, 2024-03-18T23:19:36.803 INFO:teuthology.orchestra.run.smithi175.stdout: "pgmap_bytes": 0, 2024-03-18T23:19:36.803 INFO:teuthology.orchestra.run.smithi175.stdout: "pgmap_items": 0, 2024-03-18T23:19:36.803 INFO:teuthology.orchestra.run.smithi175.stdout: "mds_co_bytes": 249071, 2024-03-18T23:19:36.803 INFO:teuthology.orchestra.run.smithi175.stdout: "mds_co_items": 1414, 2024-03-18T23:19:36.804 INFO:teuthology.orchestra.run.smithi175.stdout: "unittest_1_bytes": 0, 2024-03-18T23:19:36.804 INFO:teuthology.orchestra.run.smithi175.stdout: "unittest_1_items": 0, 2024-03-18T23:19:36.804 INFO:teuthology.orchestra.run.smithi175.stdout: "unittest_2_bytes": 0, 2024-03-18T23:19:36.804 INFO:teuthology.orchestra.run.smithi175.stdout: "unittest_2_items": 0 2024-03-18T23:19:36.804 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:36.804 INFO:teuthology.orchestra.run.smithi175.stdout: "objecter": { 2024-03-18T23:19:36.804 INFO:teuthology.orchestra.run.smithi175.stdout: "op_active": 0, 2024-03-18T23:19:36.804 INFO:teuthology.orchestra.run.smithi175.stdout: "op_laggy": 0, 2024-03-18T23:19:36.804 INFO:teuthology.orchestra.run.smithi175.stdout: "op_send": 184, 2024-03-18T23:19:36.805 INFO:teuthology.orchestra.run.smithi175.stdout: "op_send_bytes": 267027, 2024-03-18T23:19:36.805 INFO:teuthology.orchestra.run.smithi175.stdout: "op_resend": 33, 2024-03-18T23:19:36.805 INFO:teuthology.orchestra.run.smithi175.stdout: "op_reply": 184, 2024-03-18T23:19:36.805 INFO:teuthology.orchestra.run.smithi175.stdout: "op_latency": { 2024-03-18T23:19:36.805 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 184, 2024-03-18T23:19:36.805 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.662990352, 2024-03-18T23:19:36.805 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.003603208 2024-03-18T23:19:36.805 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:36.805 INFO:teuthology.orchestra.run.smithi175.stdout: "op_inflight": 0, 2024-03-18T23:19:36.806 INFO:teuthology.orchestra.run.smithi175.stdout: "oplen_avg": { 2024-03-18T23:19:36.806 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 184, 2024-03-18T23:19:36.806 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 216 2024-03-18T23:19:36.806 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:36.806 INFO:teuthology.orchestra.run.smithi175.stdout: "op": 184, 2024-03-18T23:19:36.806 INFO:teuthology.orchestra.run.smithi175.stdout: "op_r": 3, 2024-03-18T23:19:36.806 INFO:teuthology.orchestra.run.smithi175.stdout: "op_w": 181, 2024-03-18T23:19:36.806 INFO:teuthology.orchestra.run.smithi175.stdout: "op_rmw": 0, 2024-03-18T23:19:36.806 INFO:teuthology.orchestra.run.smithi175.stdout: "op_pg": 0, 2024-03-18T23:19:36.807 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_stat": 0, 2024-03-18T23:19:36.807 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_create": 10, 2024-03-18T23:19:36.807 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_read": 3, 2024-03-18T23:19:36.807 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_write": 92, 2024-03-18T23:19:36.807 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_writefull": 38, 2024-03-18T23:19:36.807 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_writesame": 0, 2024-03-18T23:19:36.807 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_append": 0, 2024-03-18T23:19:36.807 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_zero": 0, 2024-03-18T23:19:36.808 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_truncate": 0, 2024-03-18T23:19:36.808 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_delete": 18, 2024-03-18T23:19:36.808 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_mapext": 0, 2024-03-18T23:19:36.808 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_sparse_read": 0, 2024-03-18T23:19:36.808 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_clonerange": 0, 2024-03-18T23:19:36.808 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_getxattr": 0, 2024-03-18T23:19:36.808 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_setxattr": 20, 2024-03-18T23:19:36.808 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_cmpxattr": 0, 2024-03-18T23:19:36.808 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_rmxattr": 0, 2024-03-18T23:19:36.808 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_resetxattrs": 0, 2024-03-18T23:19:36.809 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_call": 0, 2024-03-18T23:19:36.809 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_watch": 0, 2024-03-18T23:19:36.809 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_notify": 0, 2024-03-18T23:19:36.809 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_src_cmpxattr": 0, 2024-03-18T23:19:36.809 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_pgls": 0, 2024-03-18T23:19:36.809 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_pgls_filter": 0, 2024-03-18T23:19:36.809 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_other": 0, 2024-03-18T23:19:36.809 INFO:teuthology.orchestra.run.smithi175.stdout: "linger_active": 0, 2024-03-18T23:19:36.809 INFO:teuthology.orchestra.run.smithi175.stdout: "linger_send": 0, 2024-03-18T23:19:36.810 INFO:teuthology.orchestra.run.smithi175.stdout: "linger_resend": 0, 2024-03-18T23:19:36.810 INFO:teuthology.orchestra.run.smithi175.stdout: "linger_ping": 0, 2024-03-18T23:19:36.810 INFO:teuthology.orchestra.run.smithi175.stdout: "poolop_active": 0, 2024-03-18T23:19:36.810 INFO:teuthology.orchestra.run.smithi175.stdout: "poolop_send": 0, 2024-03-18T23:19:36.810 INFO:teuthology.orchestra.run.smithi175.stdout: "poolop_resend": 0, 2024-03-18T23:19:36.810 INFO:teuthology.orchestra.run.smithi175.stdout: "poolstat_active": 0, 2024-03-18T23:19:36.810 INFO:teuthology.orchestra.run.smithi175.stdout: "poolstat_send": 0, 2024-03-18T23:19:36.810 INFO:teuthology.orchestra.run.smithi175.stdout: "poolstat_resend": 0, 2024-03-18T23:19:36.811 INFO:teuthology.orchestra.run.smithi175.stdout: "statfs_active": 0, 2024-03-18T23:19:36.811 INFO:teuthology.orchestra.run.smithi175.stdout: "statfs_send": 0, 2024-03-18T23:19:36.811 INFO:teuthology.orchestra.run.smithi175.stdout: "statfs_resend": 0, 2024-03-18T23:19:36.811 INFO:teuthology.orchestra.run.smithi175.stdout: "command_active": 0, 2024-03-18T23:19:36.811 INFO:teuthology.orchestra.run.smithi175.stdout: "command_send": 0, 2024-03-18T23:19:36.811 INFO:teuthology.orchestra.run.smithi175.stdout: "command_resend": 0, 2024-03-18T23:19:36.811 INFO:teuthology.orchestra.run.smithi175.stdout: "map_epoch": 101, 2024-03-18T23:19:36.811 INFO:teuthology.orchestra.run.smithi175.stdout: "map_full": 0, 2024-03-18T23:19:36.811 INFO:teuthology.orchestra.run.smithi175.stdout: "map_inc": 15, 2024-03-18T23:19:36.812 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_sessions": 10, 2024-03-18T23:19:36.812 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_session_open": 10, 2024-03-18T23:19:36.812 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_session_close": 0, 2024-03-18T23:19:36.812 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_laggy": 0, 2024-03-18T23:19:36.812 INFO:teuthology.orchestra.run.smithi175.stdout: "omap_wr": 30, 2024-03-18T23:19:36.812 INFO:teuthology.orchestra.run.smithi175.stdout: "omap_rd": 0, 2024-03-18T23:19:36.812 INFO:teuthology.orchestra.run.smithi175.stdout: "omap_del": 5 2024-03-18T23:19:36.812 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:36.813 INFO:teuthology.orchestra.run.smithi175.stdout: "oft": { 2024-03-18T23:19:36.813 INFO:teuthology.orchestra.run.smithi175.stdout: "omap_total_objs": 1, 2024-03-18T23:19:36.813 INFO:teuthology.orchestra.run.smithi175.stdout: "omap_total_kv_pairs": 0, 2024-03-18T23:19:36.813 INFO:teuthology.orchestra.run.smithi175.stdout: "omap_total_updates": 0, 2024-03-18T23:19:36.813 INFO:teuthology.orchestra.run.smithi175.stdout: "omap_total_removes": 0 2024-03-18T23:19:36.813 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:36.813 INFO:teuthology.orchestra.run.smithi175.stdout: "osd-slow-ops": { 2024-03-18T23:19:36.813 INFO:teuthology.orchestra.run.smithi175.stdout: "slow_ops_count": 0 2024-03-18T23:19:36.813 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:36.814 INFO:teuthology.orchestra.run.smithi175.stdout: "purge_queue": { 2024-03-18T23:19:36.814 INFO:teuthology.orchestra.run.smithi175.stdout: "pq_executing_ops": 0, 2024-03-18T23:19:36.814 INFO:teuthology.orchestra.run.smithi175.stdout: "pq_executing_ops_high_water": 3, 2024-03-18T23:19:36.814 INFO:teuthology.orchestra.run.smithi175.stdout: "pq_executing": 0, 2024-03-18T23:19:36.814 INFO:teuthology.orchestra.run.smithi175.stdout: "pq_executing_high_water": 3, 2024-03-18T23:19:36.814 INFO:teuthology.orchestra.run.smithi175.stdout: "pq_executed_ops": 6, 2024-03-18T23:19:36.814 INFO:teuthology.orchestra.run.smithi175.stdout: "pq_executed": 6, 2024-03-18T23:19:36.814 INFO:teuthology.orchestra.run.smithi175.stdout: "pq_item_in_journal": 0 2024-03-18T23:19:36.815 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:36.815 INFO:teuthology.orchestra.run.smithi175.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-18T23:19:36.815 INFO:teuthology.orchestra.run.smithi175.stdout: "val": 0, 2024-03-18T23:19:36.815 INFO:teuthology.orchestra.run.smithi175.stdout: "max": 104857600, 2024-03-18T23:19:36.815 INFO:teuthology.orchestra.run.smithi175.stdout: "get_started": 0, 2024-03-18T23:19:36.815 INFO:teuthology.orchestra.run.smithi175.stdout: "get": 181764, 2024-03-18T23:19:36.815 INFO:teuthology.orchestra.run.smithi175.stdout: "get_sum": 82225125, 2024-03-18T23:19:36.815 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_fail": 0, 2024-03-18T23:19:36.815 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_success": 181764, 2024-03-18T23:19:36.815 INFO:teuthology.orchestra.run.smithi175.stdout: "take": 0, 2024-03-18T23:19:36.816 INFO:teuthology.orchestra.run.smithi175.stdout: "take_sum": 0, 2024-03-18T23:19:36.816 INFO:teuthology.orchestra.run.smithi175.stdout: "put": 181764, 2024-03-18T23:19:36.816 INFO:teuthology.orchestra.run.smithi175.stdout: "put_sum": 82225125, 2024-03-18T23:19:36.816 INFO:teuthology.orchestra.run.smithi175.stdout: "wait": { 2024-03-18T23:19:36.816 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-03-18T23:19:36.816 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-03-18T23:19:36.816 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-03-18T23:19:36.816 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-03-18T23:19:36.816 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:36.817 INFO:teuthology.orchestra.run.smithi175.stdout: "throttle-objecter_bytes": { 2024-03-18T23:19:36.817 INFO:teuthology.orchestra.run.smithi175.stdout: "val": 0, 2024-03-18T23:19:36.817 INFO:teuthology.orchestra.run.smithi175.stdout: "max": 104857600, 2024-03-18T23:19:36.817 INFO:teuthology.orchestra.run.smithi175.stdout: "get_started": 0, 2024-03-18T23:19:36.817 INFO:teuthology.orchestra.run.smithi175.stdout: "get": 0, 2024-03-18T23:19:36.817 INFO:teuthology.orchestra.run.smithi175.stdout: "get_sum": 0, 2024-03-18T23:19:36.817 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_fail": 0, 2024-03-18T23:19:36.817 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_success": 0, 2024-03-18T23:19:36.817 INFO:teuthology.orchestra.run.smithi175.stdout: "take": 184, 2024-03-18T23:19:36.818 INFO:teuthology.orchestra.run.smithi175.stdout: "take_sum": 267825, 2024-03-18T23:19:36.818 INFO:teuthology.orchestra.run.smithi175.stdout: "put": 184, 2024-03-18T23:19:36.818 INFO:teuthology.orchestra.run.smithi175.stdout: "put_sum": 267825, 2024-03-18T23:19:36.818 INFO:teuthology.orchestra.run.smithi175.stdout: "wait": { 2024-03-18T23:19:36.818 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-03-18T23:19:36.818 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-03-18T23:19:36.818 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-03-18T23:19:36.818 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-03-18T23:19:36.819 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:36.819 INFO:teuthology.orchestra.run.smithi175.stdout: "throttle-objecter_ops": { 2024-03-18T23:19:36.819 INFO:teuthology.orchestra.run.smithi175.stdout: "val": 0, 2024-03-18T23:19:36.819 INFO:teuthology.orchestra.run.smithi175.stdout: "max": 1024, 2024-03-18T23:19:36.819 INFO:teuthology.orchestra.run.smithi175.stdout: "get_started": 0, 2024-03-18T23:19:36.819 INFO:teuthology.orchestra.run.smithi175.stdout: "get": 0, 2024-03-18T23:19:36.819 INFO:teuthology.orchestra.run.smithi175.stdout: "get_sum": 0, 2024-03-18T23:19:36.819 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_fail": 0, 2024-03-18T23:19:36.819 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_success": 0, 2024-03-18T23:19:36.820 INFO:teuthology.orchestra.run.smithi175.stdout: "take": 184, 2024-03-18T23:19:36.820 INFO:teuthology.orchestra.run.smithi175.stdout: "take_sum": 184, 2024-03-18T23:19:36.820 INFO:teuthology.orchestra.run.smithi175.stdout: "put": 184, 2024-03-18T23:19:36.820 INFO:teuthology.orchestra.run.smithi175.stdout: "put_sum": 184, 2024-03-18T23:19:36.820 INFO:teuthology.orchestra.run.smithi175.stdout: "wait": { 2024-03-18T23:19:36.820 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-03-18T23:19:36.820 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-03-18T23:19:36.820 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-03-18T23:19:36.821 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-03-18T23:19:36.821 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:36.821 INFO:teuthology.orchestra.run.smithi175.stdout: "throttle-write_buf_throttle": { 2024-03-18T23:19:36.821 INFO:teuthology.orchestra.run.smithi175.stdout: "val": 0, 2024-03-18T23:19:36.821 INFO:teuthology.orchestra.run.smithi175.stdout: "max": 3758096384, 2024-03-18T23:19:36.821 INFO:teuthology.orchestra.run.smithi175.stdout: "get_started": 0, 2024-03-18T23:19:36.821 INFO:teuthology.orchestra.run.smithi175.stdout: "get": 6, 2024-03-18T23:19:36.821 INFO:teuthology.orchestra.run.smithi175.stdout: "get_sum": 798, 2024-03-18T23:19:36.821 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_fail": 0, 2024-03-18T23:19:36.822 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_success": 6, 2024-03-18T23:19:36.822 INFO:teuthology.orchestra.run.smithi175.stdout: "take": 0, 2024-03-18T23:19:36.822 INFO:teuthology.orchestra.run.smithi175.stdout: "take_sum": 0, 2024-03-18T23:19:36.822 INFO:teuthology.orchestra.run.smithi175.stdout: "put": 3, 2024-03-18T23:19:36.822 INFO:teuthology.orchestra.run.smithi175.stdout: "put_sum": 798, 2024-03-18T23:19:36.822 INFO:teuthology.orchestra.run.smithi175.stdout: "wait": { 2024-03-18T23:19:36.822 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-03-18T23:19:36.822 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-03-18T23:19:36.823 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-03-18T23:19:36.823 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-03-18T23:19:36.823 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:36.823 INFO:teuthology.orchestra.run.smithi175.stdout: "throttle-write_buf_throttle-0x555d17c04500": { 2024-03-18T23:19:36.823 INFO:teuthology.orchestra.run.smithi175.stdout: "val": 0, 2024-03-18T23:19:36.823 INFO:teuthology.orchestra.run.smithi175.stdout: "max": 3758096384, 2024-03-18T23:19:36.823 INFO:teuthology.orchestra.run.smithi175.stdout: "get_started": 0, 2024-03-18T23:19:36.823 INFO:teuthology.orchestra.run.smithi175.stdout: "get": 184, 2024-03-18T23:19:36.823 INFO:teuthology.orchestra.run.smithi175.stdout: "get_sum": 250380, 2024-03-18T23:19:36.824 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_fail": 0, 2024-03-18T23:19:36.824 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_success": 184, 2024-03-18T23:19:36.824 INFO:teuthology.orchestra.run.smithi175.stdout: "take": 0, 2024-03-18T23:19:36.824 INFO:teuthology.orchestra.run.smithi175.stdout: "take_sum": 0, 2024-03-18T23:19:36.824 INFO:teuthology.orchestra.run.smithi175.stdout: "put": 89, 2024-03-18T23:19:36.824 INFO:teuthology.orchestra.run.smithi175.stdout: "put_sum": 250380, 2024-03-18T23:19:36.824 INFO:teuthology.orchestra.run.smithi175.stdout: "wait": { 2024-03-18T23:19:36.824 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-03-18T23:19:36.824 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-03-18T23:19:36.825 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-03-18T23:19:36.825 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-03-18T23:19:36.825 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-03-18T23:19:36.825 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-03-18T23:19:37.564 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:19:37 smithi175 ceph-mon[67352]: pgmap v355: 97 pgs: 97 active+clean; 67 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 1.2 MiB/s wr, 49 op/s 2024-03-18T23:19:37.564 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:19:37 smithi139 ceph-mon[67282]: pgmap v355: 97 pgs: 97 active+clean; 67 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 1.2 MiB/s wr, 49 op/s 2024-03-18T23:19:37.591 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2024-03-18T23:19:37.592 INFO:tasks.check_counter:Daemon mds.f mds.imported=1 2024-03-18T23:19:37.592 INFO:tasks.check_counter:Daemon mds.f mds.root_rsnaps=6 2024-03-18T23:19:37.592 INFO:tasks.check_counter:Daemon mds.f mds_server.req_mksnap_latency.avgcount=0 2024-03-18T23:19:37.592 INFO:tasks.check_counter:Daemon mds.f mds_server.req_rmsnap_latency.avgcount=0 2024-03-18T23:19:37.592 DEBUG:tasks.check_counter:Getting stats from i 2024-03-18T23:19:37.592 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell --fsid c63591ea-e57a-11ee-95c9-87774f69a715 -- ceph daemon mds.i perf dump 2024-03-18T23:19:37.628 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:19:37 smithi049 ceph-mon[62991]: pgmap v355: 97 pgs: 97 active+clean; 67 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 1.2 MiB/s wr, 49 op/s 2024-03-18T23:19:37.878 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/mon.c/config 2024-03-18T23:19:39.349 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:19:39 smithi175 ceph-mon[67352]: pgmap v356: 97 pgs: 97 active+clean; 61 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 1.2 MiB/s wr, 66 op/s 2024-03-18T23:19:39.564 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:19:39 smithi139 ceph-mon[67282]: pgmap v356: 97 pgs: 97 active+clean; 61 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 1.2 MiB/s wr, 66 op/s 2024-03-18T23:19:39.629 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:19:39 smithi049 ceph-mon[62991]: pgmap v356: 97 pgs: 97 active+clean; 61 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 1.2 MiB/s wr, 66 op/s 2024-03-18T23:19:39.655 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-03-18T23:19:39.656 INFO:teuthology.orchestra.run.smithi175.stdout: "AsyncMessenger::Worker-0": { 2024-03-18T23:19:39.656 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_messages": 7225, 2024-03-18T23:19:39.656 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_messages": 7371, 2024-03-18T23:19:39.656 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_bytes": 3724807, 2024-03-18T23:19:39.656 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_bytes": 1538514, 2024-03-18T23:19:39.656 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_created_connections": 13, 2024-03-18T23:19:39.656 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_active_connections": 3, 2024-03-18T23:19:39.657 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_total_time": 0.481183943, 2024-03-18T23:19:39.657 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_send_time": 0.187053042, 2024-03-18T23:19:39.657 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_recv_time": 0.393259955, 2024-03-18T23:19:39.657 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_fast_dispatch_time": 0.004477852, 2024-03-18T23:19:39.657 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_messages_queue_lat": { 2024-03-18T23:19:39.657 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 7371, 2024-03-18T23:19:39.658 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.119628856, 2024-03-18T23:19:39.658 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000016229 2024-03-18T23:19:39.658 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:39.658 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_handle_ack_lat": { 2024-03-18T23:19:39.658 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 13837, 2024-03-18T23:19:39.658 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.001741015, 2024-03-18T23:19:39.658 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000125 2024-03-18T23:19:39.658 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:39.659 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_encrypted_bytes": 3719632, 2024-03-18T23:19:39.659 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_encrypted_bytes": 1481840 2024-03-18T23:19:39.659 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:39.659 INFO:teuthology.orchestra.run.smithi175.stdout: "AsyncMessenger::Worker-1": { 2024-03-18T23:19:39.659 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_messages": 173961, 2024-03-18T23:19:39.659 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_messages": 87050, 2024-03-18T23:19:39.659 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_bytes": 95032408, 2024-03-18T23:19:39.659 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_bytes": 14316931, 2024-03-18T23:19:39.660 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_created_connections": 10, 2024-03-18T23:19:39.660 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_active_connections": 5, 2024-03-18T23:19:39.660 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_total_time": 7.248531186, 2024-03-18T23:19:39.660 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_send_time": 2.768839128, 2024-03-18T23:19:39.660 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_recv_time": 5.912377747, 2024-03-18T23:19:39.660 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_fast_dispatch_time": 0.003424468, 2024-03-18T23:19:39.660 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_messages_queue_lat": { 2024-03-18T23:19:39.660 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 87047, 2024-03-18T23:19:39.660 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 1.908326733, 2024-03-18T23:19:39.661 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000021922 2024-03-18T23:19:39.661 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:39.661 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_handle_ack_lat": { 2024-03-18T23:19:39.661 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 260803, 2024-03-18T23:19:39.661 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.038342998, 2024-03-18T23:19:39.661 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000147 2024-03-18T23:19:39.661 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:39.661 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_encrypted_bytes": 2816, 2024-03-18T23:19:39.662 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_encrypted_bytes": 138112 2024-03-18T23:19:39.662 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:39.662 INFO:teuthology.orchestra.run.smithi175.stdout: "AsyncMessenger::Worker-2": { 2024-03-18T23:19:39.662 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_messages": 22, 2024-03-18T23:19:39.662 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_messages": 25, 2024-03-18T23:19:39.662 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_bytes": 5875, 2024-03-18T23:19:39.662 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_bytes": 14972, 2024-03-18T23:19:39.663 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_created_connections": 7, 2024-03-18T23:19:39.663 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_active_connections": 4, 2024-03-18T23:19:39.663 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_total_time": 0.003749944, 2024-03-18T23:19:39.663 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_send_time": 0.000482046, 2024-03-18T23:19:39.663 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_recv_time": 0.002624813, 2024-03-18T23:19:39.663 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_fast_dispatch_time": 0.000487032, 2024-03-18T23:19:39.663 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_messages_queue_lat": { 2024-03-18T23:19:39.664 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 23, 2024-03-18T23:19:39.664 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.015142871, 2024-03-18T23:19:39.664 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000658385 2024-03-18T23:19:39.664 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:39.664 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_handle_ack_lat": { 2024-03-18T23:19:39.664 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 8, 2024-03-18T23:19:39.664 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000001266, 2024-03-18T23:19:39.664 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000158 2024-03-18T23:19:39.665 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:39.665 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_encrypted_bytes": 928, 2024-03-18T23:19:39.665 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_encrypted_bytes": 976 2024-03-18T23:19:39.665 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:39.665 INFO:teuthology.orchestra.run.smithi175.stdout: "cct": { 2024-03-18T23:19:39.665 INFO:teuthology.orchestra.run.smithi175.stdout: "total_workers": 1, 2024-03-18T23:19:39.665 INFO:teuthology.orchestra.run.smithi175.stdout: "unhealthy_workers": 0 2024-03-18T23:19:39.665 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:39.666 INFO:teuthology.orchestra.run.smithi175.stdout: "finisher-MDSRank": { 2024-03-18T23:19:39.666 INFO:teuthology.orchestra.run.smithi175.stdout: "queue_len": 0, 2024-03-18T23:19:39.666 INFO:teuthology.orchestra.run.smithi175.stdout: "complete_latency": { 2024-03-18T23:19:39.666 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 48, 2024-03-18T23:19:39.666 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.001823671, 2024-03-18T23:19:39.666 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000037993 2024-03-18T23:19:39.666 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-03-18T23:19:39.667 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:39.667 INFO:teuthology.orchestra.run.smithi175.stdout: "finisher-PurgeQueue": { 2024-03-18T23:19:39.667 INFO:teuthology.orchestra.run.smithi175.stdout: "queue_len": 0, 2024-03-18T23:19:39.667 INFO:teuthology.orchestra.run.smithi175.stdout: "complete_latency": { 2024-03-18T23:19:39.667 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 2, 2024-03-18T23:19:39.667 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000007346, 2024-03-18T23:19:39.667 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000003673 2024-03-18T23:19:39.668 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-03-18T23:19:39.668 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:39.668 INFO:teuthology.orchestra.run.smithi175.stdout: "mds": { 2024-03-18T23:19:39.668 INFO:teuthology.orchestra.run.smithi175.stdout: "request": 6893, 2024-03-18T23:19:39.668 INFO:teuthology.orchestra.run.smithi175.stdout: "reply": 1, 2024-03-18T23:19:39.668 INFO:teuthology.orchestra.run.smithi175.stdout: "reply_latency": { 2024-03-18T23:19:39.668 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 1, 2024-03-18T23:19:39.668 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000786269, 2024-03-18T23:19:39.669 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000786269 2024-03-18T23:19:39.669 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:39.669 INFO:teuthology.orchestra.run.smithi175.stdout: "slow_reply": 0, 2024-03-18T23:19:39.669 INFO:teuthology.orchestra.run.smithi175.stdout: "forward": 6892, 2024-03-18T23:19:39.669 INFO:teuthology.orchestra.run.smithi175.stdout: "dir_fetch_complete": 0, 2024-03-18T23:19:39.669 INFO:teuthology.orchestra.run.smithi175.stdout: "dir_fetch_keys": 0, 2024-03-18T23:19:39.670 INFO:teuthology.orchestra.run.smithi175.stdout: "dir_commit": 11, 2024-03-18T23:19:39.670 INFO:teuthology.orchestra.run.smithi175.stdout: "dir_split": 0, 2024-03-18T23:19:39.670 INFO:teuthology.orchestra.run.smithi175.stdout: "dir_merge": 0, 2024-03-18T23:19:39.670 INFO:teuthology.orchestra.run.smithi175.stdout: "inodes": 22, 2024-03-18T23:19:39.670 INFO:teuthology.orchestra.run.smithi175.stdout: "inodes_top": 2, 2024-03-18T23:19:39.671 INFO:teuthology.orchestra.run.smithi175.stdout: "inodes_bottom": 20, 2024-03-18T23:19:39.671 INFO:teuthology.orchestra.run.smithi175.stdout: "inodes_pin_tail": 0, 2024-03-18T23:19:39.671 INFO:teuthology.orchestra.run.smithi175.stdout: "inodes_pinned": 19, 2024-03-18T23:19:39.671 INFO:teuthology.orchestra.run.smithi175.stdout: "inodes_expired": 0, 2024-03-18T23:19:39.671 INFO:teuthology.orchestra.run.smithi175.stdout: "inodes_with_caps": 1, 2024-03-18T23:19:39.672 INFO:teuthology.orchestra.run.smithi175.stdout: "caps": 1, 2024-03-18T23:19:39.672 INFO:teuthology.orchestra.run.smithi175.stdout: "subtrees": 2, 2024-03-18T23:19:39.672 INFO:teuthology.orchestra.run.smithi175.stdout: "traverse": 6900, 2024-03-18T23:19:39.672 INFO:teuthology.orchestra.run.smithi175.stdout: "traverse_hit": 4, 2024-03-18T23:19:39.672 INFO:teuthology.orchestra.run.smithi175.stdout: "traverse_forward": 6891, 2024-03-18T23:19:39.673 INFO:teuthology.orchestra.run.smithi175.stdout: "traverse_discover": 4, 2024-03-18T23:19:39.673 INFO:teuthology.orchestra.run.smithi175.stdout: "traverse_dir_fetch": 0, 2024-03-18T23:19:39.673 INFO:teuthology.orchestra.run.smithi175.stdout: "traverse_remote_ino": 0, 2024-03-18T23:19:39.673 INFO:teuthology.orchestra.run.smithi175.stdout: "traverse_lock": 0, 2024-03-18T23:19:39.673 INFO:teuthology.orchestra.run.smithi175.stdout: "load_cent": 0, 2024-03-18T23:19:39.674 INFO:teuthology.orchestra.run.smithi175.stdout: "q": 0, 2024-03-18T23:19:39.674 INFO:teuthology.orchestra.run.smithi175.stdout: "exported": 0, 2024-03-18T23:19:39.674 INFO:teuthology.orchestra.run.smithi175.stdout: "exported_inodes": 0, 2024-03-18T23:19:39.674 INFO:teuthology.orchestra.run.smithi175.stdout: "imported": 0, 2024-03-18T23:19:39.674 INFO:teuthology.orchestra.run.smithi175.stdout: "imported_inodes": 0, 2024-03-18T23:19:39.675 INFO:teuthology.orchestra.run.smithi175.stdout: "openino_dir_fetch": 0, 2024-03-18T23:19:39.675 INFO:teuthology.orchestra.run.smithi175.stdout: "openino_backtrace_fetch": 0, 2024-03-18T23:19:39.675 INFO:teuthology.orchestra.run.smithi175.stdout: "openino_peer_discover": 0, 2024-03-18T23:19:39.675 INFO:teuthology.orchestra.run.smithi175.stdout: "root_rfiles": 69, 2024-03-18T23:19:39.675 INFO:teuthology.orchestra.run.smithi175.stdout: "root_rbytes": 21157536, 2024-03-18T23:19:39.676 INFO:teuthology.orchestra.run.smithi175.stdout: "root_rsnaps": 6, 2024-03-18T23:19:39.676 INFO:teuthology.orchestra.run.smithi175.stdout: "scrub_backtrace_fetch": 0, 2024-03-18T23:19:39.676 INFO:teuthology.orchestra.run.smithi175.stdout: "scrub_set_tag": 0, 2024-03-18T23:19:39.676 INFO:teuthology.orchestra.run.smithi175.stdout: "scrub_backtrace_repaired": 0, 2024-03-18T23:19:39.676 INFO:teuthology.orchestra.run.smithi175.stdout: "scrub_inotable_repaired": 0, 2024-03-18T23:19:39.677 INFO:teuthology.orchestra.run.smithi175.stdout: "scrub_dir_inodes": 0, 2024-03-18T23:19:39.677 INFO:teuthology.orchestra.run.smithi175.stdout: "scrub_dir_base_inodes": 0, 2024-03-18T23:19:39.677 INFO:teuthology.orchestra.run.smithi175.stdout: "scrub_dirfrag_rstats": 0, 2024-03-18T23:19:39.677 INFO:teuthology.orchestra.run.smithi175.stdout: "scrub_file_inodes": 0, 2024-03-18T23:19:39.677 INFO:teuthology.orchestra.run.smithi175.stdout: "handle_inode_file_caps": 0, 2024-03-18T23:19:39.678 INFO:teuthology.orchestra.run.smithi175.stdout: "ceph_cap_op_revoke": 1, 2024-03-18T23:19:39.678 INFO:teuthology.orchestra.run.smithi175.stdout: "ceph_cap_op_grant": 2, 2024-03-18T23:19:39.678 INFO:teuthology.orchestra.run.smithi175.stdout: "ceph_cap_op_trunc": 0, 2024-03-18T23:19:39.678 INFO:teuthology.orchestra.run.smithi175.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-18T23:19:39.678 INFO:teuthology.orchestra.run.smithi175.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-18T23:19:39.679 INFO:teuthology.orchestra.run.smithi175.stdout: "handle_client_caps": 1, 2024-03-18T23:19:39.679 INFO:teuthology.orchestra.run.smithi175.stdout: "handle_client_caps_dirty": 0, 2024-03-18T23:19:39.679 INFO:teuthology.orchestra.run.smithi175.stdout: "handle_client_cap_release": 0, 2024-03-18T23:19:39.679 INFO:teuthology.orchestra.run.smithi175.stdout: "process_request_cap_release": 1 2024-03-18T23:19:39.679 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:39.679 INFO:teuthology.orchestra.run.smithi175.stdout: "mds_cache": { 2024-03-18T23:19:39.680 INFO:teuthology.orchestra.run.smithi175.stdout: "dir_update": 0, 2024-03-18T23:19:39.680 INFO:teuthology.orchestra.run.smithi175.stdout: "dir_update_receipt": 9, 2024-03-18T23:19:39.680 INFO:teuthology.orchestra.run.smithi175.stdout: "dir_try_discover": 6, 2024-03-18T23:19:39.680 INFO:teuthology.orchestra.run.smithi175.stdout: "dir_send_discover": 8, 2024-03-18T23:19:39.680 INFO:teuthology.orchestra.run.smithi175.stdout: "dir_handle_discover": 0, 2024-03-18T23:19:39.681 INFO:teuthology.orchestra.run.smithi175.stdout: "num_strays": 0, 2024-03-18T23:19:39.681 INFO:teuthology.orchestra.run.smithi175.stdout: "num_strays_delayed": 0, 2024-03-18T23:19:39.681 INFO:teuthology.orchestra.run.smithi175.stdout: "num_strays_enqueuing": 0, 2024-03-18T23:19:39.681 INFO:teuthology.orchestra.run.smithi175.stdout: "strays_created": 0, 2024-03-18T23:19:39.681 INFO:teuthology.orchestra.run.smithi175.stdout: "strays_enqueued": 0, 2024-03-18T23:19:39.681 INFO:teuthology.orchestra.run.smithi175.stdout: "strays_reintegrated": 0, 2024-03-18T23:19:39.681 INFO:teuthology.orchestra.run.smithi175.stdout: "strays_migrated": 0, 2024-03-18T23:19:39.681 INFO:teuthology.orchestra.run.smithi175.stdout: "num_recovering_processing": 0, 2024-03-18T23:19:39.681 INFO:teuthology.orchestra.run.smithi175.stdout: "num_recovering_enqueued": 0, 2024-03-18T23:19:39.682 INFO:teuthology.orchestra.run.smithi175.stdout: "num_recovering_prioritized": 0, 2024-03-18T23:19:39.682 INFO:teuthology.orchestra.run.smithi175.stdout: "recovery_started": 0, 2024-03-18T23:19:39.682 INFO:teuthology.orchestra.run.smithi175.stdout: "recovery_completed": 0, 2024-03-18T23:19:39.682 INFO:teuthology.orchestra.run.smithi175.stdout: "ireq_quiesce_path": 0, 2024-03-18T23:19:39.682 INFO:teuthology.orchestra.run.smithi175.stdout: "ireq_quiesce_inode": 0, 2024-03-18T23:19:39.682 INFO:teuthology.orchestra.run.smithi175.stdout: "ireq_enqueue_scrub": 0, 2024-03-18T23:19:39.682 INFO:teuthology.orchestra.run.smithi175.stdout: "ireq_exportdir": 0, 2024-03-18T23:19:39.682 INFO:teuthology.orchestra.run.smithi175.stdout: "ireq_flush": 0, 2024-03-18T23:19:39.683 INFO:teuthology.orchestra.run.smithi175.stdout: "ireq_fragmentdir": 0, 2024-03-18T23:19:39.683 INFO:teuthology.orchestra.run.smithi175.stdout: "ireq_fragstats": 0, 2024-03-18T23:19:39.683 INFO:teuthology.orchestra.run.smithi175.stdout: "ireq_inodestats": 0 2024-03-18T23:19:39.683 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:39.683 INFO:teuthology.orchestra.run.smithi175.stdout: "mds_log": { 2024-03-18T23:19:39.683 INFO:teuthology.orchestra.run.smithi175.stdout: "evlrg": 0, 2024-03-18T23:19:39.683 INFO:teuthology.orchestra.run.smithi175.stdout: "evadd": 8, 2024-03-18T23:19:39.683 INFO:teuthology.orchestra.run.smithi175.stdout: "evex": 0, 2024-03-18T23:19:39.684 INFO:teuthology.orchestra.run.smithi175.stdout: "evtrm": 0, 2024-03-18T23:19:39.684 INFO:teuthology.orchestra.run.smithi175.stdout: "ev": 8, 2024-03-18T23:19:39.684 INFO:teuthology.orchestra.run.smithi175.stdout: "evexg": 0, 2024-03-18T23:19:39.684 INFO:teuthology.orchestra.run.smithi175.stdout: "evexd": 0, 2024-03-18T23:19:39.684 INFO:teuthology.orchestra.run.smithi175.stdout: "segadd": 2, 2024-03-18T23:19:39.684 INFO:teuthology.orchestra.run.smithi175.stdout: "segex": 0, 2024-03-18T23:19:39.684 INFO:teuthology.orchestra.run.smithi175.stdout: "segtrm": 0, 2024-03-18T23:19:39.684 INFO:teuthology.orchestra.run.smithi175.stdout: "seg": 2, 2024-03-18T23:19:39.684 INFO:teuthology.orchestra.run.smithi175.stdout: "segmjr": 2, 2024-03-18T23:19:39.685 INFO:teuthology.orchestra.run.smithi175.stdout: "segexg": 0, 2024-03-18T23:19:39.685 INFO:teuthology.orchestra.run.smithi175.stdout: "segexd": 0, 2024-03-18T23:19:39.685 INFO:teuthology.orchestra.run.smithi175.stdout: "expos": 4194304, 2024-03-18T23:19:39.685 INFO:teuthology.orchestra.run.smithi175.stdout: "wrpos": 4196655, 2024-03-18T23:19:39.685 INFO:teuthology.orchestra.run.smithi175.stdout: "rdpos": 0, 2024-03-18T23:19:39.685 INFO:teuthology.orchestra.run.smithi175.stdout: "jlat": { 2024-03-18T23:19:39.685 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 7, 2024-03-18T23:19:39.685 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.011677243, 2024-03-18T23:19:39.685 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.001668177 2024-03-18T23:19:39.685 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:39.686 INFO:teuthology.orchestra.run.smithi175.stdout: "replayed": 0 2024-03-18T23:19:39.686 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:39.686 INFO:teuthology.orchestra.run.smithi175.stdout: "mds_mem": { 2024-03-18T23:19:39.686 INFO:teuthology.orchestra.run.smithi175.stdout: "ino": 25, 2024-03-18T23:19:39.686 INFO:teuthology.orchestra.run.smithi175.stdout: "ino+": 25, 2024-03-18T23:19:39.686 INFO:teuthology.orchestra.run.smithi175.stdout: "ino-": 0, 2024-03-18T23:19:39.686 INFO:teuthology.orchestra.run.smithi175.stdout: "dir": 38, 2024-03-18T23:19:39.686 INFO:teuthology.orchestra.run.smithi175.stdout: "dir+": 41, 2024-03-18T23:19:39.687 INFO:teuthology.orchestra.run.smithi175.stdout: "dir-": 3, 2024-03-18T23:19:39.687 INFO:teuthology.orchestra.run.smithi175.stdout: "dn": 22, 2024-03-18T23:19:39.687 INFO:teuthology.orchestra.run.smithi175.stdout: "dn+": 22, 2024-03-18T23:19:39.687 INFO:teuthology.orchestra.run.smithi175.stdout: "dn-": 0, 2024-03-18T23:19:39.687 INFO:teuthology.orchestra.run.smithi175.stdout: "cap": 1, 2024-03-18T23:19:39.687 INFO:teuthology.orchestra.run.smithi175.stdout: "cap+": 1, 2024-03-18T23:19:39.687 INFO:teuthology.orchestra.run.smithi175.stdout: "cap-": 0, 2024-03-18T23:19:39.687 INFO:teuthology.orchestra.run.smithi175.stdout: "rss": 54108, 2024-03-18T23:19:39.687 INFO:teuthology.orchestra.run.smithi175.stdout: "heap": 223612 2024-03-18T23:19:39.688 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:39.688 INFO:teuthology.orchestra.run.smithi175.stdout: "mds_server": { 2024-03-18T23:19:39.688 INFO:teuthology.orchestra.run.smithi175.stdout: "dispatch_client_request": 6894, 2024-03-18T23:19:39.688 INFO:teuthology.orchestra.run.smithi175.stdout: "dispatch_server_request": 0, 2024-03-18T23:19:39.688 INFO:teuthology.orchestra.run.smithi175.stdout: "handle_client_request": 6893, 2024-03-18T23:19:39.688 INFO:teuthology.orchestra.run.smithi175.stdout: "handle_client_session": 26, 2024-03-18T23:19:39.688 INFO:teuthology.orchestra.run.smithi175.stdout: "handle_peer_request": 0, 2024-03-18T23:19:39.688 INFO:teuthology.orchestra.run.smithi175.stdout: "req_create_latency": { 2024-03-18T23:19:39.688 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-03-18T23:19:39.689 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-03-18T23:19:39.689 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-03-18T23:19:39.689 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:39.689 INFO:teuthology.orchestra.run.smithi175.stdout: "req_getattr_latency": { 2024-03-18T23:19:39.689 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 1, 2024-03-18T23:19:39.689 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000786269, 2024-03-18T23:19:39.689 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000786269 2024-03-18T23:19:39.689 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:39.689 INFO:teuthology.orchestra.run.smithi175.stdout: "req_getfilelock_latency": { 2024-03-18T23:19:39.689 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-03-18T23:19:39.690 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-03-18T23:19:39.690 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-03-18T23:19:39.690 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:39.690 INFO:teuthology.orchestra.run.smithi175.stdout: "req_link_latency": { 2024-03-18T23:19:39.690 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-03-18T23:19:39.690 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-03-18T23:19:39.690 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-03-18T23:19:39.690 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:39.690 INFO:teuthology.orchestra.run.smithi175.stdout: "req_lookup_latency": { 2024-03-18T23:19:39.691 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-03-18T23:19:39.691 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-03-18T23:19:39.691 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-03-18T23:19:39.691 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:39.691 INFO:teuthology.orchestra.run.smithi175.stdout: "req_lookuphash_latency": { 2024-03-18T23:19:39.691 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-03-18T23:19:39.691 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-03-18T23:19:39.691 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-03-18T23:19:39.691 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:39.691 INFO:teuthology.orchestra.run.smithi175.stdout: "req_lookupino_latency": { 2024-03-18T23:19:39.692 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-03-18T23:19:39.692 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-03-18T23:19:39.692 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-03-18T23:19:39.692 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:39.692 INFO:teuthology.orchestra.run.smithi175.stdout: "req_lookupname_latency": { 2024-03-18T23:19:39.692 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-03-18T23:19:39.692 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-03-18T23:19:39.693 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-03-18T23:19:39.693 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:39.693 INFO:teuthology.orchestra.run.smithi175.stdout: "req_lookupparent_latency": { 2024-03-18T23:19:39.693 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-03-18T23:19:39.693 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-03-18T23:19:39.693 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-03-18T23:19:39.693 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:39.693 INFO:teuthology.orchestra.run.smithi175.stdout: "req_lookupsnap_latency": { 2024-03-18T23:19:39.693 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-03-18T23:19:39.693 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-03-18T23:19:39.694 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-03-18T23:19:39.694 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:39.694 INFO:teuthology.orchestra.run.smithi175.stdout: "req_lssnap_latency": { 2024-03-18T23:19:39.694 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-03-18T23:19:39.694 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-03-18T23:19:39.694 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-03-18T23:19:39.694 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:39.695 INFO:teuthology.orchestra.run.smithi175.stdout: "req_mkdir_latency": { 2024-03-18T23:19:39.695 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-03-18T23:19:39.695 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-03-18T23:19:39.695 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-03-18T23:19:39.695 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:39.696 INFO:teuthology.orchestra.run.smithi175.stdout: "req_mknod_latency": { 2024-03-18T23:19:39.696 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-03-18T23:19:39.696 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-03-18T23:19:39.696 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-03-18T23:19:39.696 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:39.696 INFO:teuthology.orchestra.run.smithi175.stdout: "req_mksnap_latency": { 2024-03-18T23:19:39.696 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-03-18T23:19:39.696 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-03-18T23:19:39.696 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-03-18T23:19:39.697 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:39.697 INFO:teuthology.orchestra.run.smithi175.stdout: "req_open_latency": { 2024-03-18T23:19:39.697 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-03-18T23:19:39.697 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-03-18T23:19:39.697 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-03-18T23:19:39.697 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:39.697 INFO:teuthology.orchestra.run.smithi175.stdout: "req_readdir_latency": { 2024-03-18T23:19:39.697 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-03-18T23:19:39.697 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-03-18T23:19:39.698 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-03-18T23:19:39.698 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:39.698 INFO:teuthology.orchestra.run.smithi175.stdout: "req_rename_latency": { 2024-03-18T23:19:39.698 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-03-18T23:19:39.698 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-03-18T23:19:39.698 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-03-18T23:19:39.698 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:39.698 INFO:teuthology.orchestra.run.smithi175.stdout: "req_renamesnap_latency": { 2024-03-18T23:19:39.699 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-03-18T23:19:39.699 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-03-18T23:19:39.699 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-03-18T23:19:39.699 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:39.699 INFO:teuthology.orchestra.run.smithi175.stdout: "req_snapdiff_latency": { 2024-03-18T23:19:39.699 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-03-18T23:19:39.699 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-03-18T23:19:39.699 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-03-18T23:19:39.699 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:39.699 INFO:teuthology.orchestra.run.smithi175.stdout: "req_rmdir_latency": { 2024-03-18T23:19:39.700 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-03-18T23:19:39.700 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-03-18T23:19:39.700 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-03-18T23:19:39.700 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:39.700 INFO:teuthology.orchestra.run.smithi175.stdout: "req_rmsnap_latency": { 2024-03-18T23:19:39.700 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-03-18T23:19:39.700 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-03-18T23:19:39.700 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-03-18T23:19:39.700 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:39.700 INFO:teuthology.orchestra.run.smithi175.stdout: "req_rmxattr_latency": { 2024-03-18T23:19:39.701 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-03-18T23:19:39.701 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-03-18T23:19:39.701 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-03-18T23:19:39.701 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:39.701 INFO:teuthology.orchestra.run.smithi175.stdout: "req_setattr_latency": { 2024-03-18T23:19:39.701 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-03-18T23:19:39.702 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-03-18T23:19:39.702 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-03-18T23:19:39.702 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:39.702 INFO:teuthology.orchestra.run.smithi175.stdout: "req_setdirlayout_latency": { 2024-03-18T23:19:39.702 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-03-18T23:19:39.702 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-03-18T23:19:39.702 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-03-18T23:19:39.702 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:39.703 INFO:teuthology.orchestra.run.smithi175.stdout: "req_setfilelock_latency": { 2024-03-18T23:19:39.703 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-03-18T23:19:39.703 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-03-18T23:19:39.703 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-03-18T23:19:39.703 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:39.703 INFO:teuthology.orchestra.run.smithi175.stdout: "req_setlayout_latency": { 2024-03-18T23:19:39.703 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-03-18T23:19:39.703 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-03-18T23:19:39.704 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-03-18T23:19:39.704 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:39.704 INFO:teuthology.orchestra.run.smithi175.stdout: "req_setxattr_latency": { 2024-03-18T23:19:39.704 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-03-18T23:19:39.704 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-03-18T23:19:39.704 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-03-18T23:19:39.704 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:39.704 INFO:teuthology.orchestra.run.smithi175.stdout: "req_symlink_latency": { 2024-03-18T23:19:39.704 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-03-18T23:19:39.705 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-03-18T23:19:39.705 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-03-18T23:19:39.705 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:39.705 INFO:teuthology.orchestra.run.smithi175.stdout: "req_unlink_latency": { 2024-03-18T23:19:39.705 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-03-18T23:19:39.705 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-03-18T23:19:39.705 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-03-18T23:19:39.705 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:39.705 INFO:teuthology.orchestra.run.smithi175.stdout: "cap_revoke_eviction": 0, 2024-03-18T23:19:39.706 INFO:teuthology.orchestra.run.smithi175.stdout: "cap_acquisition_throttle": 0, 2024-03-18T23:19:39.706 INFO:teuthology.orchestra.run.smithi175.stdout: "req_getvxattr_latency": { 2024-03-18T23:19:39.706 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-03-18T23:19:39.706 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-03-18T23:19:39.706 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-03-18T23:19:39.706 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-03-18T23:19:39.706 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:39.706 INFO:teuthology.orchestra.run.smithi175.stdout: "mds_sessions": { 2024-03-18T23:19:39.706 INFO:teuthology.orchestra.run.smithi175.stdout: "session_count": 2, 2024-03-18T23:19:39.707 INFO:teuthology.orchestra.run.smithi175.stdout: "session_add": 4, 2024-03-18T23:19:39.707 INFO:teuthology.orchestra.run.smithi175.stdout: "session_remove": 2, 2024-03-18T23:19:39.707 INFO:teuthology.orchestra.run.smithi175.stdout: "sessions_open": 2, 2024-03-18T23:19:39.707 INFO:teuthology.orchestra.run.smithi175.stdout: "sessions_stale": 0, 2024-03-18T23:19:39.707 INFO:teuthology.orchestra.run.smithi175.stdout: "total_load": 1, 2024-03-18T23:19:39.707 INFO:teuthology.orchestra.run.smithi175.stdout: "average_load": 1, 2024-03-18T23:19:39.707 INFO:teuthology.orchestra.run.smithi175.stdout: "avg_session_uptime": 663, 2024-03-18T23:19:39.707 INFO:teuthology.orchestra.run.smithi175.stdout: "mdthresh_evicted": 0 2024-03-18T23:19:39.707 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:39.708 INFO:teuthology.orchestra.run.smithi175.stdout: "mempool": { 2024-03-18T23:19:39.708 INFO:teuthology.orchestra.run.smithi175.stdout: "bloom_filter_bytes": 0, 2024-03-18T23:19:39.708 INFO:teuthology.orchestra.run.smithi175.stdout: "bloom_filter_items": 0, 2024-03-18T23:19:39.708 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_alloc_bytes": 0, 2024-03-18T23:19:39.708 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_alloc_items": 0, 2024-03-18T23:19:39.708 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_data_bytes": 0, 2024-03-18T23:19:39.708 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_data_items": 0, 2024-03-18T23:19:39.708 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-18T23:19:39.708 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_onode_items": 0, 2024-03-18T23:19:39.708 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-18T23:19:39.709 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_meta_items": 0, 2024-03-18T23:19:39.709 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_other_bytes": 0, 2024-03-18T23:19:39.709 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_other_items": 0, 2024-03-18T23:19:39.709 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-18T23:19:39.709 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_buffer_items": 0, 2024-03-18T23:19:39.709 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_extent_bytes": 0, 2024-03-18T23:19:39.709 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_extent_items": 0, 2024-03-18T23:19:39.709 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_blob_bytes": 0, 2024-03-18T23:19:39.710 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_blob_items": 0, 2024-03-18T23:19:39.710 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-18T23:19:39.710 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_shared_blob_items": 0, 2024-03-18T23:19:39.710 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-18T23:19:39.710 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_inline_bl_items": 0, 2024-03-18T23:19:39.710 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_fsck_bytes": 0, 2024-03-18T23:19:39.710 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_fsck_items": 0, 2024-03-18T23:19:39.710 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_txc_bytes": 0, 2024-03-18T23:19:39.710 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_txc_items": 0, 2024-03-18T23:19:39.710 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-18T23:19:39.711 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_writing_deferred_items": 0, 2024-03-18T23:19:39.711 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_writing_bytes": 0, 2024-03-18T23:19:39.711 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_writing_items": 0, 2024-03-18T23:19:39.711 INFO:teuthology.orchestra.run.smithi175.stdout: "bluefs_bytes": 0, 2024-03-18T23:19:39.711 INFO:teuthology.orchestra.run.smithi175.stdout: "bluefs_items": 0, 2024-03-18T23:19:39.711 INFO:teuthology.orchestra.run.smithi175.stdout: "bluefs_file_reader_bytes": 0, 2024-03-18T23:19:39.711 INFO:teuthology.orchestra.run.smithi175.stdout: "bluefs_file_reader_items": 0, 2024-03-18T23:19:39.711 INFO:teuthology.orchestra.run.smithi175.stdout: "bluefs_file_writer_bytes": 0, 2024-03-18T23:19:39.711 INFO:teuthology.orchestra.run.smithi175.stdout: "bluefs_file_writer_items": 0, 2024-03-18T23:19:39.712 INFO:teuthology.orchestra.run.smithi175.stdout: "buffer_anon_bytes": 274647, 2024-03-18T23:19:39.712 INFO:teuthology.orchestra.run.smithi175.stdout: "buffer_anon_items": 121, 2024-03-18T23:19:39.712 INFO:teuthology.orchestra.run.smithi175.stdout: "buffer_meta_bytes": 0, 2024-03-18T23:19:39.712 INFO:teuthology.orchestra.run.smithi175.stdout: "buffer_meta_items": 0, 2024-03-18T23:19:39.712 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_bytes": 0, 2024-03-18T23:19:39.712 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_items": 0, 2024-03-18T23:19:39.712 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_mapbl_bytes": 0, 2024-03-18T23:19:39.712 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_mapbl_items": 0, 2024-03-18T23:19:39.712 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_pglog_bytes": 0, 2024-03-18T23:19:39.712 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_pglog_items": 0, 2024-03-18T23:19:39.713 INFO:teuthology.orchestra.run.smithi175.stdout: "osdmap_bytes": 8256, 2024-03-18T23:19:39.713 INFO:teuthology.orchestra.run.smithi175.stdout: "osdmap_items": 188, 2024-03-18T23:19:39.713 INFO:teuthology.orchestra.run.smithi175.stdout: "osdmap_mapping_bytes": 0, 2024-03-18T23:19:39.713 INFO:teuthology.orchestra.run.smithi175.stdout: "osdmap_mapping_items": 0, 2024-03-18T23:19:39.713 INFO:teuthology.orchestra.run.smithi175.stdout: "pgmap_bytes": 0, 2024-03-18T23:19:39.713 INFO:teuthology.orchestra.run.smithi175.stdout: "pgmap_items": 0, 2024-03-18T23:19:39.713 INFO:teuthology.orchestra.run.smithi175.stdout: "mds_co_bytes": 131357, 2024-03-18T23:19:39.713 INFO:teuthology.orchestra.run.smithi175.stdout: "mds_co_items": 735, 2024-03-18T23:19:39.713 INFO:teuthology.orchestra.run.smithi175.stdout: "unittest_1_bytes": 0, 2024-03-18T23:19:39.714 INFO:teuthology.orchestra.run.smithi175.stdout: "unittest_1_items": 0, 2024-03-18T23:19:39.714 INFO:teuthology.orchestra.run.smithi175.stdout: "unittest_2_bytes": 0, 2024-03-18T23:19:39.714 INFO:teuthology.orchestra.run.smithi175.stdout: "unittest_2_items": 0 2024-03-18T23:19:39.714 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:39.714 INFO:teuthology.orchestra.run.smithi175.stdout: "objecter": { 2024-03-18T23:19:39.714 INFO:teuthology.orchestra.run.smithi175.stdout: "op_active": 0, 2024-03-18T23:19:39.714 INFO:teuthology.orchestra.run.smithi175.stdout: "op_laggy": 0, 2024-03-18T23:19:39.714 INFO:teuthology.orchestra.run.smithi175.stdout: "op_send": 46, 2024-03-18T23:19:39.715 INFO:teuthology.orchestra.run.smithi175.stdout: "op_send_bytes": 13148, 2024-03-18T23:19:39.715 INFO:teuthology.orchestra.run.smithi175.stdout: "op_resend": 33, 2024-03-18T23:19:39.715 INFO:teuthology.orchestra.run.smithi175.stdout: "op_reply": 46, 2024-03-18T23:19:39.715 INFO:teuthology.orchestra.run.smithi175.stdout: "op_latency": { 2024-03-18T23:19:39.715 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 46, 2024-03-18T23:19:39.715 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.215996821, 2024-03-18T23:19:39.715 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.004695583 2024-03-18T23:19:39.715 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:39.716 INFO:teuthology.orchestra.run.smithi175.stdout: "op_inflight": 0, 2024-03-18T23:19:39.716 INFO:teuthology.orchestra.run.smithi175.stdout: "oplen_avg": { 2024-03-18T23:19:39.716 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 46, 2024-03-18T23:19:39.716 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 70 2024-03-18T23:19:39.716 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:39.716 INFO:teuthology.orchestra.run.smithi175.stdout: "op": 46, 2024-03-18T23:19:39.716 INFO:teuthology.orchestra.run.smithi175.stdout: "op_r": 0, 2024-03-18T23:19:39.716 INFO:teuthology.orchestra.run.smithi175.stdout: "op_w": 46, 2024-03-18T23:19:39.716 INFO:teuthology.orchestra.run.smithi175.stdout: "op_rmw": 0, 2024-03-18T23:19:39.717 INFO:teuthology.orchestra.run.smithi175.stdout: "op_pg": 0, 2024-03-18T23:19:39.717 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_stat": 0, 2024-03-18T23:19:39.717 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_create": 10, 2024-03-18T23:19:39.717 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_read": 0, 2024-03-18T23:19:39.717 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_write": 7, 2024-03-18T23:19:39.717 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_writefull": 8, 2024-03-18T23:19:39.717 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_writesame": 0, 2024-03-18T23:19:39.717 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_append": 0, 2024-03-18T23:19:39.718 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_zero": 0, 2024-03-18T23:19:39.718 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_truncate": 0, 2024-03-18T23:19:39.718 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_delete": 6, 2024-03-18T23:19:39.718 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_mapext": 0, 2024-03-18T23:19:39.718 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_sparse_read": 0, 2024-03-18T23:19:39.718 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_clonerange": 0, 2024-03-18T23:19:39.718 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_getxattr": 0, 2024-03-18T23:19:39.719 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_setxattr": 20, 2024-03-18T23:19:39.719 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_cmpxattr": 0, 2024-03-18T23:19:39.719 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_rmxattr": 0, 2024-03-18T23:19:39.719 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_resetxattrs": 0, 2024-03-18T23:19:39.719 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_call": 0, 2024-03-18T23:19:39.719 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_watch": 0, 2024-03-18T23:19:39.719 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_notify": 0, 2024-03-18T23:19:39.719 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_src_cmpxattr": 0, 2024-03-18T23:19:39.719 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_pgls": 0, 2024-03-18T23:19:39.719 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_pgls_filter": 0, 2024-03-18T23:19:39.720 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_other": 0, 2024-03-18T23:19:39.720 INFO:teuthology.orchestra.run.smithi175.stdout: "linger_active": 0, 2024-03-18T23:19:39.720 INFO:teuthology.orchestra.run.smithi175.stdout: "linger_send": 0, 2024-03-18T23:19:39.720 INFO:teuthology.orchestra.run.smithi175.stdout: "linger_resend": 0, 2024-03-18T23:19:39.720 INFO:teuthology.orchestra.run.smithi175.stdout: "linger_ping": 0, 2024-03-18T23:19:39.720 INFO:teuthology.orchestra.run.smithi175.stdout: "poolop_active": 0, 2024-03-18T23:19:39.720 INFO:teuthology.orchestra.run.smithi175.stdout: "poolop_send": 0, 2024-03-18T23:19:39.720 INFO:teuthology.orchestra.run.smithi175.stdout: "poolop_resend": 0, 2024-03-18T23:19:39.720 INFO:teuthology.orchestra.run.smithi175.stdout: "poolstat_active": 0, 2024-03-18T23:19:39.721 INFO:teuthology.orchestra.run.smithi175.stdout: "poolstat_send": 0, 2024-03-18T23:19:39.721 INFO:teuthology.orchestra.run.smithi175.stdout: "poolstat_resend": 0, 2024-03-18T23:19:39.721 INFO:teuthology.orchestra.run.smithi175.stdout: "statfs_active": 0, 2024-03-18T23:19:39.721 INFO:teuthology.orchestra.run.smithi175.stdout: "statfs_send": 0, 2024-03-18T23:19:39.721 INFO:teuthology.orchestra.run.smithi175.stdout: "statfs_resend": 0, 2024-03-18T23:19:39.721 INFO:teuthology.orchestra.run.smithi175.stdout: "command_active": 0, 2024-03-18T23:19:39.721 INFO:teuthology.orchestra.run.smithi175.stdout: "command_send": 0, 2024-03-18T23:19:39.721 INFO:teuthology.orchestra.run.smithi175.stdout: "command_resend": 0, 2024-03-18T23:19:39.722 INFO:teuthology.orchestra.run.smithi175.stdout: "map_epoch": 101, 2024-03-18T23:19:39.722 INFO:teuthology.orchestra.run.smithi175.stdout: "map_full": 0, 2024-03-18T23:19:39.722 INFO:teuthology.orchestra.run.smithi175.stdout: "map_inc": 15, 2024-03-18T23:19:39.722 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_sessions": 9, 2024-03-18T23:19:39.722 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_session_open": 9, 2024-03-18T23:19:39.722 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_session_close": 0, 2024-03-18T23:19:39.722 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_laggy": 0, 2024-03-18T23:19:39.723 INFO:teuthology.orchestra.run.smithi175.stdout: "omap_wr": 18, 2024-03-18T23:19:39.723 INFO:teuthology.orchestra.run.smithi175.stdout: "omap_rd": 0, 2024-03-18T23:19:39.723 INFO:teuthology.orchestra.run.smithi175.stdout: "omap_del": 1 2024-03-18T23:19:39.723 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:39.723 INFO:teuthology.orchestra.run.smithi175.stdout: "oft": { 2024-03-18T23:19:39.723 INFO:teuthology.orchestra.run.smithi175.stdout: "omap_total_objs": 1, 2024-03-18T23:19:39.723 INFO:teuthology.orchestra.run.smithi175.stdout: "omap_total_kv_pairs": 0, 2024-03-18T23:19:39.724 INFO:teuthology.orchestra.run.smithi175.stdout: "omap_total_updates": 0, 2024-03-18T23:19:39.724 INFO:teuthology.orchestra.run.smithi175.stdout: "omap_total_removes": 0 2024-03-18T23:19:39.724 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:39.724 INFO:teuthology.orchestra.run.smithi175.stdout: "osd-slow-ops": { 2024-03-18T23:19:39.724 INFO:teuthology.orchestra.run.smithi175.stdout: "slow_ops_count": 0 2024-03-18T23:19:39.724 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:39.724 INFO:teuthology.orchestra.run.smithi175.stdout: "purge_queue": { 2024-03-18T23:19:39.724 INFO:teuthology.orchestra.run.smithi175.stdout: "pq_executing_ops": 0, 2024-03-18T23:19:39.725 INFO:teuthology.orchestra.run.smithi175.stdout: "pq_executing_ops_high_water": 0, 2024-03-18T23:19:39.725 INFO:teuthology.orchestra.run.smithi175.stdout: "pq_executing": 0, 2024-03-18T23:19:39.725 INFO:teuthology.orchestra.run.smithi175.stdout: "pq_executing_high_water": 0, 2024-03-18T23:19:39.725 INFO:teuthology.orchestra.run.smithi175.stdout: "pq_executed_ops": 0, 2024-03-18T23:19:39.725 INFO:teuthology.orchestra.run.smithi175.stdout: "pq_executed": 0, 2024-03-18T23:19:39.725 INFO:teuthology.orchestra.run.smithi175.stdout: "pq_item_in_journal": 0 2024-03-18T23:19:39.725 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:39.725 INFO:teuthology.orchestra.run.smithi175.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-18T23:19:39.726 INFO:teuthology.orchestra.run.smithi175.stdout: "val": 0, 2024-03-18T23:19:39.726 INFO:teuthology.orchestra.run.smithi175.stdout: "max": 104857600, 2024-03-18T23:19:39.726 INFO:teuthology.orchestra.run.smithi175.stdout: "get_started": 0, 2024-03-18T23:19:39.726 INFO:teuthology.orchestra.run.smithi175.stdout: "get": 181204, 2024-03-18T23:19:39.726 INFO:teuthology.orchestra.run.smithi175.stdout: "get_sum": 82100847, 2024-03-18T23:19:39.726 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_fail": 0, 2024-03-18T23:19:39.726 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_success": 181204, 2024-03-18T23:19:39.726 INFO:teuthology.orchestra.run.smithi175.stdout: "take": 0, 2024-03-18T23:19:39.727 INFO:teuthology.orchestra.run.smithi175.stdout: "take_sum": 0, 2024-03-18T23:19:39.727 INFO:teuthology.orchestra.run.smithi175.stdout: "put": 181204, 2024-03-18T23:19:39.727 INFO:teuthology.orchestra.run.smithi175.stdout: "put_sum": 82100847, 2024-03-18T23:19:39.727 INFO:teuthology.orchestra.run.smithi175.stdout: "wait": { 2024-03-18T23:19:39.727 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-03-18T23:19:39.727 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-03-18T23:19:39.727 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-03-18T23:19:39.727 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-03-18T23:19:39.728 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:39.728 INFO:teuthology.orchestra.run.smithi175.stdout: "throttle-objecter_bytes": { 2024-03-18T23:19:39.728 INFO:teuthology.orchestra.run.smithi175.stdout: "val": 0, 2024-03-18T23:19:39.728 INFO:teuthology.orchestra.run.smithi175.stdout: "max": 104857600, 2024-03-18T23:19:39.728 INFO:teuthology.orchestra.run.smithi175.stdout: "get_started": 0, 2024-03-18T23:19:39.728 INFO:teuthology.orchestra.run.smithi175.stdout: "get": 0, 2024-03-18T23:19:39.728 INFO:teuthology.orchestra.run.smithi175.stdout: "get_sum": 0, 2024-03-18T23:19:39.728 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_fail": 0, 2024-03-18T23:19:39.729 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_success": 0, 2024-03-18T23:19:39.729 INFO:teuthology.orchestra.run.smithi175.stdout: "take": 46, 2024-03-18T23:19:39.729 INFO:teuthology.orchestra.run.smithi175.stdout: "take_sum": 13148, 2024-03-18T23:19:39.729 INFO:teuthology.orchestra.run.smithi175.stdout: "put": 46, 2024-03-18T23:19:39.729 INFO:teuthology.orchestra.run.smithi175.stdout: "put_sum": 13148, 2024-03-18T23:19:39.729 INFO:teuthology.orchestra.run.smithi175.stdout: "wait": { 2024-03-18T23:19:39.729 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-03-18T23:19:39.730 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-03-18T23:19:39.730 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-03-18T23:19:39.730 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-03-18T23:19:39.730 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:39.730 INFO:teuthology.orchestra.run.smithi175.stdout: "throttle-objecter_ops": { 2024-03-18T23:19:39.730 INFO:teuthology.orchestra.run.smithi175.stdout: "val": 0, 2024-03-18T23:19:39.730 INFO:teuthology.orchestra.run.smithi175.stdout: "max": 1024, 2024-03-18T23:19:39.730 INFO:teuthology.orchestra.run.smithi175.stdout: "get_started": 0, 2024-03-18T23:19:39.730 INFO:teuthology.orchestra.run.smithi175.stdout: "get": 0, 2024-03-18T23:19:39.731 INFO:teuthology.orchestra.run.smithi175.stdout: "get_sum": 0, 2024-03-18T23:19:39.731 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_fail": 0, 2024-03-18T23:19:39.731 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_success": 0, 2024-03-18T23:19:39.731 INFO:teuthology.orchestra.run.smithi175.stdout: "take": 46, 2024-03-18T23:19:39.731 INFO:teuthology.orchestra.run.smithi175.stdout: "take_sum": 46, 2024-03-18T23:19:39.731 INFO:teuthology.orchestra.run.smithi175.stdout: "put": 46, 2024-03-18T23:19:39.731 INFO:teuthology.orchestra.run.smithi175.stdout: "put_sum": 46, 2024-03-18T23:19:39.731 INFO:teuthology.orchestra.run.smithi175.stdout: "wait": { 2024-03-18T23:19:39.731 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-03-18T23:19:39.732 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-03-18T23:19:39.732 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-03-18T23:19:39.732 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-03-18T23:19:39.732 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:39.732 INFO:teuthology.orchestra.run.smithi175.stdout: "throttle-write_buf_throttle": { 2024-03-18T23:19:39.732 INFO:teuthology.orchestra.run.smithi175.stdout: "val": 0, 2024-03-18T23:19:39.732 INFO:teuthology.orchestra.run.smithi175.stdout: "max": 3758096384, 2024-03-18T23:19:39.732 INFO:teuthology.orchestra.run.smithi175.stdout: "get_started": 0, 2024-03-18T23:19:39.732 INFO:teuthology.orchestra.run.smithi175.stdout: "get": 0, 2024-03-18T23:19:39.733 INFO:teuthology.orchestra.run.smithi175.stdout: "get_sum": 0, 2024-03-18T23:19:39.733 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_fail": 0, 2024-03-18T23:19:39.733 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_success": 0, 2024-03-18T23:19:39.733 INFO:teuthology.orchestra.run.smithi175.stdout: "take": 0, 2024-03-18T23:19:39.733 INFO:teuthology.orchestra.run.smithi175.stdout: "take_sum": 0, 2024-03-18T23:19:39.733 INFO:teuthology.orchestra.run.smithi175.stdout: "put": 0, 2024-03-18T23:19:39.733 INFO:teuthology.orchestra.run.smithi175.stdout: "put_sum": 0, 2024-03-18T23:19:39.733 INFO:teuthology.orchestra.run.smithi175.stdout: "wait": { 2024-03-18T23:19:39.733 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-03-18T23:19:39.734 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-03-18T23:19:39.734 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-03-18T23:19:39.734 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-03-18T23:19:39.734 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:39.734 INFO:teuthology.orchestra.run.smithi175.stdout: "throttle-write_buf_throttle-0x5608f82c4500": { 2024-03-18T23:19:39.734 INFO:teuthology.orchestra.run.smithi175.stdout: "val": 0, 2024-03-18T23:19:39.734 INFO:teuthology.orchestra.run.smithi175.stdout: "max": 3758096384, 2024-03-18T23:19:39.735 INFO:teuthology.orchestra.run.smithi175.stdout: "get_started": 0, 2024-03-18T23:19:39.735 INFO:teuthology.orchestra.run.smithi175.stdout: "get": 8, 2024-03-18T23:19:39.735 INFO:teuthology.orchestra.run.smithi175.stdout: "get_sum": 2351, 2024-03-18T23:19:39.735 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_fail": 0, 2024-03-18T23:19:39.735 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_success": 8, 2024-03-18T23:19:39.735 INFO:teuthology.orchestra.run.smithi175.stdout: "take": 0, 2024-03-18T23:19:39.735 INFO:teuthology.orchestra.run.smithi175.stdout: "take_sum": 0, 2024-03-18T23:19:39.735 INFO:teuthology.orchestra.run.smithi175.stdout: "put": 7, 2024-03-18T23:19:39.736 INFO:teuthology.orchestra.run.smithi175.stdout: "put_sum": 2351, 2024-03-18T23:19:39.736 INFO:teuthology.orchestra.run.smithi175.stdout: "wait": { 2024-03-18T23:19:39.736 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-03-18T23:19:39.736 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-03-18T23:19:39.736 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-03-18T23:19:39.736 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-03-18T23:19:39.736 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-03-18T23:19:39.736 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-03-18T23:19:40.420 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:19:40 smithi175 ceph-mon[67352]: pgmap v357: 97 pgs: 97 active+clean; 61 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 595 KiB/s wr, 36 op/s 2024-03-18T23:19:40.445 INFO:tasks.check_counter:Daemon mds.i mds.exported=0 2024-03-18T23:19:40.445 INFO:tasks.check_counter:Daemon mds.i mds.imported=0 2024-03-18T23:19:40.445 INFO:tasks.check_counter:Daemon mds.i mds.root_rsnaps=6 2024-03-18T23:19:40.445 INFO:tasks.check_counter:Daemon mds.i mds_server.req_mksnap_latency.avgcount=0 2024-03-18T23:19:40.445 INFO:tasks.check_counter:Daemon mds.i mds_server.req_rmsnap_latency.avgcount=0 2024-03-18T23:19:40.446 DEBUG:tasks.check_counter:Getting stats from l 2024-03-18T23:19:40.446 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell --fsid c63591ea-e57a-11ee-95c9-87774f69a715 -- ceph daemon mds.l perf dump 2024-03-18T23:19:40.564 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:19:40 smithi139 ceph-mon[67282]: pgmap v357: 97 pgs: 97 active+clean; 61 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 595 KiB/s wr, 36 op/s 2024-03-18T23:19:40.629 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:19:40 smithi049 ceph-mon[62991]: pgmap v357: 97 pgs: 97 active+clean; 61 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 595 KiB/s wr, 36 op/s 2024-03-18T23:19:40.715 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/mon.c/config 2024-03-18T23:19:42.380 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-03-18T23:19:42.381 INFO:teuthology.orchestra.run.smithi175.stdout: "AsyncMessenger::Worker-0": { 2024-03-18T23:19:42.381 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_messages": 173953, 2024-03-18T23:19:42.381 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_messages": 87021, 2024-03-18T23:19:42.381 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_bytes": 95026700, 2024-03-18T23:19:42.381 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_bytes": 14165368, 2024-03-18T23:19:42.382 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_created_connections": 9, 2024-03-18T23:19:42.382 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_active_connections": 4, 2024-03-18T23:19:42.382 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_total_time": 7.212817612, 2024-03-18T23:19:42.382 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_send_time": 2.752030429, 2024-03-18T23:19:42.382 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_recv_time": 5.890619261, 2024-03-18T23:19:42.382 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_fast_dispatch_time": 0.003225627, 2024-03-18T23:19:42.382 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_messages_queue_lat": { 2024-03-18T23:19:42.382 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 87020, 2024-03-18T23:19:42.382 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 1.541807605, 2024-03-18T23:19:42.383 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000017717 2024-03-18T23:19:42.383 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:42.383 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_handle_ack_lat": { 2024-03-18T23:19:42.383 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 260783, 2024-03-18T23:19:42.383 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.034547030, 2024-03-18T23:19:42.383 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000132 2024-03-18T23:19:42.383 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:42.383 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_encrypted_bytes": 6672, 2024-03-18T23:19:42.384 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_encrypted_bytes": 12784 2024-03-18T23:19:42.384 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:42.384 INFO:teuthology.orchestra.run.smithi175.stdout: "AsyncMessenger::Worker-1": { 2024-03-18T23:19:42.384 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_messages": 310, 2024-03-18T23:19:42.384 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_messages": 431, 2024-03-18T23:19:42.384 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_bytes": 1518073, 2024-03-18T23:19:42.384 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_bytes": 540749, 2024-03-18T23:19:42.384 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_created_connections": 11, 2024-03-18T23:19:42.384 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_active_connections": 6, 2024-03-18T23:19:42.385 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_total_time": 0.044427387, 2024-03-18T23:19:42.385 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_send_time": 0.013682618, 2024-03-18T23:19:42.385 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_recv_time": 0.037390698, 2024-03-18T23:19:42.385 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_fast_dispatch_time": 0.004325831, 2024-03-18T23:19:42.385 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_messages_queue_lat": { 2024-03-18T23:19:42.385 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 430, 2024-03-18T23:19:42.386 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.025066313, 2024-03-18T23:19:42.386 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000058293 2024-03-18T23:19:42.386 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:42.386 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_handle_ack_lat": { 2024-03-18T23:19:42.386 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 4, 2024-03-18T23:19:42.386 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000319, 2024-03-18T23:19:42.386 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000079 2024-03-18T23:19:42.386 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:42.386 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_encrypted_bytes": 1514992, 2024-03-18T23:19:42.387 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_encrypted_bytes": 483984 2024-03-18T23:19:42.387 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:42.387 INFO:teuthology.orchestra.run.smithi175.stdout: "AsyncMessenger::Worker-2": { 2024-03-18T23:19:42.387 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_messages": 36, 2024-03-18T23:19:42.387 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_messages": 105, 2024-03-18T23:19:42.387 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_bytes": 8889, 2024-03-18T23:19:42.387 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_bytes": 151590, 2024-03-18T23:19:42.387 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_created_connections": 9, 2024-03-18T23:19:42.389 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_active_connections": 6, 2024-03-18T23:19:42.389 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_total_time": 0.012576218, 2024-03-18T23:19:42.389 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_send_time": 0.003492646, 2024-03-18T23:19:42.390 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_recv_time": 0.007589866, 2024-03-18T23:19:42.390 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_fast_dispatch_time": 0.000308955, 2024-03-18T23:19:42.390 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_messages_queue_lat": { 2024-03-18T23:19:42.390 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 102, 2024-03-18T23:19:42.390 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.020651099, 2024-03-18T23:19:42.390 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000202461 2024-03-18T23:19:42.390 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:42.390 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_handle_ack_lat": { 2024-03-18T23:19:42.391 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 64, 2024-03-18T23:19:42.391 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000011874, 2024-03-18T23:19:42.391 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000185 2024-03-18T23:19:42.391 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:42.391 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_encrypted_bytes": 4224, 2024-03-18T23:19:42.391 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_encrypted_bytes": 143360 2024-03-18T23:19:42.391 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:42.391 INFO:teuthology.orchestra.run.smithi175.stdout: "cct": { 2024-03-18T23:19:42.391 INFO:teuthology.orchestra.run.smithi175.stdout: "total_workers": 1, 2024-03-18T23:19:42.392 INFO:teuthology.orchestra.run.smithi175.stdout: "unhealthy_workers": 0 2024-03-18T23:19:42.392 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:42.392 INFO:teuthology.orchestra.run.smithi175.stdout: "finisher-MDSRank": { 2024-03-18T23:19:42.392 INFO:teuthology.orchestra.run.smithi175.stdout: "queue_len": 0, 2024-03-18T23:19:42.392 INFO:teuthology.orchestra.run.smithi175.stdout: "complete_latency": { 2024-03-18T23:19:42.392 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 41, 2024-03-18T23:19:42.392 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.001220831, 2024-03-18T23:19:42.392 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000029776 2024-03-18T23:19:42.393 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-03-18T23:19:42.393 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:42.393 INFO:teuthology.orchestra.run.smithi175.stdout: "finisher-PurgeQueue": { 2024-03-18T23:19:42.393 INFO:teuthology.orchestra.run.smithi175.stdout: "queue_len": 0, 2024-03-18T23:19:42.393 INFO:teuthology.orchestra.run.smithi175.stdout: "complete_latency": { 2024-03-18T23:19:42.393 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 2, 2024-03-18T23:19:42.393 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000011860, 2024-03-18T23:19:42.393 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000005930 2024-03-18T23:19:42.394 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-03-18T23:19:42.394 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:42.394 INFO:teuthology.orchestra.run.smithi175.stdout: "mds": { 2024-03-18T23:19:42.394 INFO:teuthology.orchestra.run.smithi175.stdout: "request": 11, 2024-03-18T23:19:42.394 INFO:teuthology.orchestra.run.smithi175.stdout: "reply": 7, 2024-03-18T23:19:42.394 INFO:teuthology.orchestra.run.smithi175.stdout: "reply_latency": { 2024-03-18T23:19:42.394 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 7, 2024-03-18T23:19:42.394 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.002365289, 2024-03-18T23:19:42.394 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000337898 2024-03-18T23:19:42.395 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:42.395 INFO:teuthology.orchestra.run.smithi175.stdout: "slow_reply": 0, 2024-03-18T23:19:42.395 INFO:teuthology.orchestra.run.smithi175.stdout: "forward": 4, 2024-03-18T23:19:42.395 INFO:teuthology.orchestra.run.smithi175.stdout: "dir_fetch_complete": 0, 2024-03-18T23:19:42.395 INFO:teuthology.orchestra.run.smithi175.stdout: "dir_fetch_keys": 0, 2024-03-18T23:19:42.395 INFO:teuthology.orchestra.run.smithi175.stdout: "dir_commit": 11, 2024-03-18T23:19:42.395 INFO:teuthology.orchestra.run.smithi175.stdout: "dir_split": 0, 2024-03-18T23:19:42.395 INFO:teuthology.orchestra.run.smithi175.stdout: "dir_merge": 0, 2024-03-18T23:19:42.395 INFO:teuthology.orchestra.run.smithi175.stdout: "inodes": 22, 2024-03-18T23:19:42.396 INFO:teuthology.orchestra.run.smithi175.stdout: "inodes_top": 2, 2024-03-18T23:19:42.396 INFO:teuthology.orchestra.run.smithi175.stdout: "inodes_bottom": 20, 2024-03-18T23:19:42.396 INFO:teuthology.orchestra.run.smithi175.stdout: "inodes_pin_tail": 0, 2024-03-18T23:19:42.396 INFO:teuthology.orchestra.run.smithi175.stdout: "inodes_pinned": 19, 2024-03-18T23:19:42.396 INFO:teuthology.orchestra.run.smithi175.stdout: "inodes_expired": 1, 2024-03-18T23:19:42.396 INFO:teuthology.orchestra.run.smithi175.stdout: "inodes_with_caps": 1, 2024-03-18T23:19:42.396 INFO:teuthology.orchestra.run.smithi175.stdout: "caps": 2, 2024-03-18T23:19:42.396 INFO:teuthology.orchestra.run.smithi175.stdout: "subtrees": 2, 2024-03-18T23:19:42.397 INFO:teuthology.orchestra.run.smithi175.stdout: "traverse": 22, 2024-03-18T23:19:42.397 INFO:teuthology.orchestra.run.smithi175.stdout: "traverse_hit": 13, 2024-03-18T23:19:42.397 INFO:teuthology.orchestra.run.smithi175.stdout: "traverse_forward": 1, 2024-03-18T23:19:42.397 INFO:teuthology.orchestra.run.smithi175.stdout: "traverse_discover": 5, 2024-03-18T23:19:42.397 INFO:teuthology.orchestra.run.smithi175.stdout: "traverse_dir_fetch": 0, 2024-03-18T23:19:42.397 INFO:teuthology.orchestra.run.smithi175.stdout: "traverse_remote_ino": 0, 2024-03-18T23:19:42.397 INFO:teuthology.orchestra.run.smithi175.stdout: "traverse_lock": 0, 2024-03-18T23:19:42.397 INFO:teuthology.orchestra.run.smithi175.stdout: "load_cent": 0, 2024-03-18T23:19:42.397 INFO:teuthology.orchestra.run.smithi175.stdout: "q": 0, 2024-03-18T23:19:42.398 INFO:teuthology.orchestra.run.smithi175.stdout: "exported": 0, 2024-03-18T23:19:42.398 INFO:teuthology.orchestra.run.smithi175.stdout: "exported_inodes": 0, 2024-03-18T23:19:42.398 INFO:teuthology.orchestra.run.smithi175.stdout: "imported": 0, 2024-03-18T23:19:42.398 INFO:teuthology.orchestra.run.smithi175.stdout: "imported_inodes": 0, 2024-03-18T23:19:42.398 INFO:teuthology.orchestra.run.smithi175.stdout: "openino_dir_fetch": 0, 2024-03-18T23:19:42.398 INFO:teuthology.orchestra.run.smithi175.stdout: "openino_backtrace_fetch": 0, 2024-03-18T23:19:42.398 INFO:teuthology.orchestra.run.smithi175.stdout: "openino_peer_discover": 0, 2024-03-18T23:19:42.398 INFO:teuthology.orchestra.run.smithi175.stdout: "root_rfiles": 69, 2024-03-18T23:19:42.398 INFO:teuthology.orchestra.run.smithi175.stdout: "root_rbytes": 21157536, 2024-03-18T23:19:42.399 INFO:teuthology.orchestra.run.smithi175.stdout: "root_rsnaps": 6, 2024-03-18T23:19:42.399 INFO:teuthology.orchestra.run.smithi175.stdout: "scrub_backtrace_fetch": 0, 2024-03-18T23:19:42.399 INFO:teuthology.orchestra.run.smithi175.stdout: "scrub_set_tag": 0, 2024-03-18T23:19:42.399 INFO:teuthology.orchestra.run.smithi175.stdout: "scrub_backtrace_repaired": 0, 2024-03-18T23:19:42.399 INFO:teuthology.orchestra.run.smithi175.stdout: "scrub_inotable_repaired": 0, 2024-03-18T23:19:42.399 INFO:teuthology.orchestra.run.smithi175.stdout: "scrub_dir_inodes": 0, 2024-03-18T23:19:42.399 INFO:teuthology.orchestra.run.smithi175.stdout: "scrub_dir_base_inodes": 0, 2024-03-18T23:19:42.399 INFO:teuthology.orchestra.run.smithi175.stdout: "scrub_dirfrag_rstats": 0, 2024-03-18T23:19:42.400 INFO:teuthology.orchestra.run.smithi175.stdout: "scrub_file_inodes": 0, 2024-03-18T23:19:42.400 INFO:teuthology.orchestra.run.smithi175.stdout: "handle_inode_file_caps": 0, 2024-03-18T23:19:42.400 INFO:teuthology.orchestra.run.smithi175.stdout: "ceph_cap_op_revoke": 0, 2024-03-18T23:19:42.400 INFO:teuthology.orchestra.run.smithi175.stdout: "ceph_cap_op_grant": 0, 2024-03-18T23:19:42.400 INFO:teuthology.orchestra.run.smithi175.stdout: "ceph_cap_op_trunc": 0, 2024-03-18T23:19:42.400 INFO:teuthology.orchestra.run.smithi175.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-18T23:19:42.400 INFO:teuthology.orchestra.run.smithi175.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-18T23:19:42.400 INFO:teuthology.orchestra.run.smithi175.stdout: "handle_client_caps": 0, 2024-03-18T23:19:42.401 INFO:teuthology.orchestra.run.smithi175.stdout: "handle_client_caps_dirty": 0, 2024-03-18T23:19:42.401 INFO:teuthology.orchestra.run.smithi175.stdout: "handle_client_cap_release": 0, 2024-03-18T23:19:42.401 INFO:teuthology.orchestra.run.smithi175.stdout: "process_request_cap_release": 0 2024-03-18T23:19:42.401 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:42.401 INFO:teuthology.orchestra.run.smithi175.stdout: "mds_cache": { 2024-03-18T23:19:42.401 INFO:teuthology.orchestra.run.smithi175.stdout: "dir_update": 0, 2024-03-18T23:19:42.401 INFO:teuthology.orchestra.run.smithi175.stdout: "dir_update_receipt": 9, 2024-03-18T23:19:42.401 INFO:teuthology.orchestra.run.smithi175.stdout: "dir_try_discover": 6, 2024-03-18T23:19:42.401 INFO:teuthology.orchestra.run.smithi175.stdout: "dir_send_discover": 9, 2024-03-18T23:19:42.402 INFO:teuthology.orchestra.run.smithi175.stdout: "dir_handle_discover": 0, 2024-03-18T23:19:42.402 INFO:teuthology.orchestra.run.smithi175.stdout: "num_strays": 0, 2024-03-18T23:19:42.402 INFO:teuthology.orchestra.run.smithi175.stdout: "num_strays_delayed": 0, 2024-03-18T23:19:42.403 INFO:teuthology.orchestra.run.smithi175.stdout: "num_strays_enqueuing": 0, 2024-03-18T23:19:42.403 INFO:teuthology.orchestra.run.smithi175.stdout: "strays_created": 0, 2024-03-18T23:19:42.403 INFO:teuthology.orchestra.run.smithi175.stdout: "strays_enqueued": 0, 2024-03-18T23:19:42.404 INFO:teuthology.orchestra.run.smithi175.stdout: "strays_reintegrated": 0, 2024-03-18T23:19:42.404 INFO:teuthology.orchestra.run.smithi175.stdout: "strays_migrated": 0, 2024-03-18T23:19:42.404 INFO:teuthology.orchestra.run.smithi175.stdout: "num_recovering_processing": 0, 2024-03-18T23:19:42.404 INFO:teuthology.orchestra.run.smithi175.stdout: "num_recovering_enqueued": 0, 2024-03-18T23:19:42.405 INFO:teuthology.orchestra.run.smithi175.stdout: "num_recovering_prioritized": 0, 2024-03-18T23:19:42.405 INFO:teuthology.orchestra.run.smithi175.stdout: "recovery_started": 0, 2024-03-18T23:19:42.405 INFO:teuthology.orchestra.run.smithi175.stdout: "recovery_completed": 0, 2024-03-18T23:19:42.405 INFO:teuthology.orchestra.run.smithi175.stdout: "ireq_quiesce_path": 0, 2024-03-18T23:19:42.405 INFO:teuthology.orchestra.run.smithi175.stdout: "ireq_quiesce_inode": 0, 2024-03-18T23:19:42.405 INFO:teuthology.orchestra.run.smithi175.stdout: "ireq_enqueue_scrub": 0, 2024-03-18T23:19:42.406 INFO:teuthology.orchestra.run.smithi175.stdout: "ireq_exportdir": 0, 2024-03-18T23:19:42.406 INFO:teuthology.orchestra.run.smithi175.stdout: "ireq_flush": 0, 2024-03-18T23:19:42.406 INFO:teuthology.orchestra.run.smithi175.stdout: "ireq_fragmentdir": 0, 2024-03-18T23:19:42.406 INFO:teuthology.orchestra.run.smithi175.stdout: "ireq_fragstats": 0, 2024-03-18T23:19:42.406 INFO:teuthology.orchestra.run.smithi175.stdout: "ireq_inodestats": 0 2024-03-18T23:19:42.406 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:42.406 INFO:teuthology.orchestra.run.smithi175.stdout: "mds_log": { 2024-03-18T23:19:42.407 INFO:teuthology.orchestra.run.smithi175.stdout: "evlrg": 0, 2024-03-18T23:19:42.407 INFO:teuthology.orchestra.run.smithi175.stdout: "evadd": 6, 2024-03-18T23:19:42.407 INFO:teuthology.orchestra.run.smithi175.stdout: "evex": 0, 2024-03-18T23:19:42.407 INFO:teuthology.orchestra.run.smithi175.stdout: "evtrm": 0, 2024-03-18T23:19:42.407 INFO:teuthology.orchestra.run.smithi175.stdout: "ev": 6, 2024-03-18T23:19:42.407 INFO:teuthology.orchestra.run.smithi175.stdout: "evexg": 0, 2024-03-18T23:19:42.407 INFO:teuthology.orchestra.run.smithi175.stdout: "evexd": 0, 2024-03-18T23:19:42.407 INFO:teuthology.orchestra.run.smithi175.stdout: "segadd": 2, 2024-03-18T23:19:42.408 INFO:teuthology.orchestra.run.smithi175.stdout: "segex": 0, 2024-03-18T23:19:42.408 INFO:teuthology.orchestra.run.smithi175.stdout: "segtrm": 0, 2024-03-18T23:19:42.408 INFO:teuthology.orchestra.run.smithi175.stdout: "seg": 2, 2024-03-18T23:19:42.408 INFO:teuthology.orchestra.run.smithi175.stdout: "segmjr": 2, 2024-03-18T23:19:42.408 INFO:teuthology.orchestra.run.smithi175.stdout: "segexg": 0, 2024-03-18T23:19:42.408 INFO:teuthology.orchestra.run.smithi175.stdout: "segexd": 0, 2024-03-18T23:19:42.408 INFO:teuthology.orchestra.run.smithi175.stdout: "expos": 4194304, 2024-03-18T23:19:42.408 INFO:teuthology.orchestra.run.smithi175.stdout: "wrpos": 4196123, 2024-03-18T23:19:42.408 INFO:teuthology.orchestra.run.smithi175.stdout: "rdpos": 0, 2024-03-18T23:19:42.409 INFO:teuthology.orchestra.run.smithi175.stdout: "jlat": { 2024-03-18T23:19:42.409 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 5, 2024-03-18T23:19:42.409 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.009506014, 2024-03-18T23:19:42.409 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.001901202 2024-03-18T23:19:42.409 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:42.409 INFO:teuthology.orchestra.run.smithi175.stdout: "replayed": 0 2024-03-18T23:19:42.409 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:42.409 INFO:teuthology.orchestra.run.smithi175.stdout: "mds_mem": { 2024-03-18T23:19:42.410 INFO:teuthology.orchestra.run.smithi175.stdout: "ino": 25, 2024-03-18T23:19:42.410 INFO:teuthology.orchestra.run.smithi175.stdout: "ino+": 25, 2024-03-18T23:19:42.410 INFO:teuthology.orchestra.run.smithi175.stdout: "ino-": 0, 2024-03-18T23:19:42.410 INFO:teuthology.orchestra.run.smithi175.stdout: "dir": 38, 2024-03-18T23:19:42.410 INFO:teuthology.orchestra.run.smithi175.stdout: "dir+": 41, 2024-03-18T23:19:42.410 INFO:teuthology.orchestra.run.smithi175.stdout: "dir-": 3, 2024-03-18T23:19:42.411 INFO:teuthology.orchestra.run.smithi175.stdout: "dn": 22, 2024-03-18T23:19:42.411 INFO:teuthology.orchestra.run.smithi175.stdout: "dn+": 23, 2024-03-18T23:19:42.411 INFO:teuthology.orchestra.run.smithi175.stdout: "dn-": 1, 2024-03-18T23:19:42.411 INFO:teuthology.orchestra.run.smithi175.stdout: "cap": 2, 2024-03-18T23:19:42.411 INFO:teuthology.orchestra.run.smithi175.stdout: "cap+": 4, 2024-03-18T23:19:42.411 INFO:teuthology.orchestra.run.smithi175.stdout: "cap-": 2, 2024-03-18T23:19:42.411 INFO:teuthology.orchestra.run.smithi175.stdout: "rss": 52432, 2024-03-18T23:19:42.412 INFO:teuthology.orchestra.run.smithi175.stdout: "heap": 223612 2024-03-18T23:19:42.412 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:42.412 INFO:teuthology.orchestra.run.smithi175.stdout: "mds_server": { 2024-03-18T23:19:42.412 INFO:teuthology.orchestra.run.smithi175.stdout: "dispatch_client_request": 13, 2024-03-18T23:19:42.412 INFO:teuthology.orchestra.run.smithi175.stdout: "dispatch_server_request": 0, 2024-03-18T23:19:42.412 INFO:teuthology.orchestra.run.smithi175.stdout: "handle_client_request": 11, 2024-03-18T23:19:42.412 INFO:teuthology.orchestra.run.smithi175.stdout: "handle_client_session": 22, 2024-03-18T23:19:42.412 INFO:teuthology.orchestra.run.smithi175.stdout: "handle_peer_request": 0, 2024-03-18T23:19:42.412 INFO:teuthology.orchestra.run.smithi175.stdout: "req_create_latency": { 2024-03-18T23:19:42.413 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-03-18T23:19:42.413 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-03-18T23:19:42.413 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-03-18T23:19:42.413 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:42.413 INFO:teuthology.orchestra.run.smithi175.stdout: "req_getattr_latency": { 2024-03-18T23:19:42.413 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 4, 2024-03-18T23:19:42.413 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000993595, 2024-03-18T23:19:42.413 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000248398 2024-03-18T23:19:42.413 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:42.414 INFO:teuthology.orchestra.run.smithi175.stdout: "req_getfilelock_latency": { 2024-03-18T23:19:42.414 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-03-18T23:19:42.414 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-03-18T23:19:42.416 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-03-18T23:19:42.416 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:42.417 INFO:teuthology.orchestra.run.smithi175.stdout: "req_link_latency": { 2024-03-18T23:19:42.417 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-03-18T23:19:42.417 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-03-18T23:19:42.417 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-03-18T23:19:42.417 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:42.417 INFO:teuthology.orchestra.run.smithi175.stdout: "req_lookup_latency": { 2024-03-18T23:19:42.417 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 2, 2024-03-18T23:19:42.418 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.001236957, 2024-03-18T23:19:42.418 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000618478 2024-03-18T23:19:42.418 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:42.418 INFO:teuthology.orchestra.run.smithi175.stdout: "req_lookuphash_latency": { 2024-03-18T23:19:42.418 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-03-18T23:19:42.418 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-03-18T23:19:42.418 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-03-18T23:19:42.418 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:42.419 INFO:teuthology.orchestra.run.smithi175.stdout: "req_lookupino_latency": { 2024-03-18T23:19:42.419 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 1, 2024-03-18T23:19:42.419 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000134737, 2024-03-18T23:19:42.419 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000134737 2024-03-18T23:19:42.419 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:42.419 INFO:teuthology.orchestra.run.smithi175.stdout: "req_lookupname_latency": { 2024-03-18T23:19:42.419 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-03-18T23:19:42.419 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-03-18T23:19:42.419 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-03-18T23:19:42.420 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:42.420 INFO:teuthology.orchestra.run.smithi175.stdout: "req_lookupparent_latency": { 2024-03-18T23:19:42.420 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-03-18T23:19:42.420 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-03-18T23:19:42.420 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-03-18T23:19:42.420 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:42.420 INFO:teuthology.orchestra.run.smithi175.stdout: "req_lookupsnap_latency": { 2024-03-18T23:19:42.420 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-03-18T23:19:42.421 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-03-18T23:19:42.421 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-03-18T23:19:42.421 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:42.421 INFO:teuthology.orchestra.run.smithi175.stdout: "req_lssnap_latency": { 2024-03-18T23:19:42.421 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-03-18T23:19:42.421 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-03-18T23:19:42.422 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-03-18T23:19:42.422 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:42.422 INFO:teuthology.orchestra.run.smithi175.stdout: "req_mkdir_latency": { 2024-03-18T23:19:42.423 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-03-18T23:19:42.423 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-03-18T23:19:42.423 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-03-18T23:19:42.423 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:42.423 INFO:teuthology.orchestra.run.smithi175.stdout: "req_mknod_latency": { 2024-03-18T23:19:42.424 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-03-18T23:19:42.424 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-03-18T23:19:42.424 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-03-18T23:19:42.424 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:42.424 INFO:teuthology.orchestra.run.smithi175.stdout: "req_mksnap_latency": { 2024-03-18T23:19:42.425 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-03-18T23:19:42.425 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-03-18T23:19:42.425 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-03-18T23:19:42.425 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:42.425 INFO:teuthology.orchestra.run.smithi175.stdout: "req_open_latency": { 2024-03-18T23:19:42.425 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-03-18T23:19:42.425 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-03-18T23:19:42.425 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-03-18T23:19:42.425 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:42.426 INFO:teuthology.orchestra.run.smithi175.stdout: "req_readdir_latency": { 2024-03-18T23:19:42.426 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-03-18T23:19:42.426 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-03-18T23:19:42.426 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-03-18T23:19:42.426 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:42.426 INFO:teuthology.orchestra.run.smithi175.stdout: "req_rename_latency": { 2024-03-18T23:19:42.426 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-03-18T23:19:42.426 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-03-18T23:19:42.427 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-03-18T23:19:42.427 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:42.427 INFO:teuthology.orchestra.run.smithi175.stdout: "req_renamesnap_latency": { 2024-03-18T23:19:42.427 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-03-18T23:19:42.427 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-03-18T23:19:42.427 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-03-18T23:19:42.427 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:42.428 INFO:teuthology.orchestra.run.smithi175.stdout: "req_snapdiff_latency": { 2024-03-18T23:19:42.428 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-03-18T23:19:42.428 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-03-18T23:19:42.428 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-03-18T23:19:42.428 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:42.428 INFO:teuthology.orchestra.run.smithi175.stdout: "req_rmdir_latency": { 2024-03-18T23:19:42.428 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-03-18T23:19:42.428 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-03-18T23:19:42.429 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-03-18T23:19:42.429 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:42.429 INFO:teuthology.orchestra.run.smithi175.stdout: "req_rmsnap_latency": { 2024-03-18T23:19:42.429 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-03-18T23:19:42.429 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-03-18T23:19:42.429 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-03-18T23:19:42.429 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:42.430 INFO:teuthology.orchestra.run.smithi175.stdout: "req_rmxattr_latency": { 2024-03-18T23:19:42.430 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-03-18T23:19:42.430 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-03-18T23:19:42.430 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-03-18T23:19:42.430 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:42.430 INFO:teuthology.orchestra.run.smithi175.stdout: "req_setattr_latency": { 2024-03-18T23:19:42.430 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-03-18T23:19:42.430 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-03-18T23:19:42.431 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-03-18T23:19:42.431 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:42.431 INFO:teuthology.orchestra.run.smithi175.stdout: "req_setdirlayout_latency": { 2024-03-18T23:19:42.431 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-03-18T23:19:42.431 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-03-18T23:19:42.431 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-03-18T23:19:42.431 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:42.431 INFO:teuthology.orchestra.run.smithi175.stdout: "req_setfilelock_latency": { 2024-03-18T23:19:42.432 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-03-18T23:19:42.432 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-03-18T23:19:42.432 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-03-18T23:19:42.432 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:42.432 INFO:teuthology.orchestra.run.smithi175.stdout: "req_setlayout_latency": { 2024-03-18T23:19:42.433 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-03-18T23:19:42.433 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-03-18T23:19:42.433 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-03-18T23:19:42.433 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:42.433 INFO:teuthology.orchestra.run.smithi175.stdout: "req_setxattr_latency": { 2024-03-18T23:19:42.433 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-03-18T23:19:42.433 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-03-18T23:19:42.434 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-03-18T23:19:42.434 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:42.434 INFO:teuthology.orchestra.run.smithi175.stdout: "req_symlink_latency": { 2024-03-18T23:19:42.434 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-03-18T23:19:42.434 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-03-18T23:19:42.434 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-03-18T23:19:42.435 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:42.435 INFO:teuthology.orchestra.run.smithi175.stdout: "req_unlink_latency": { 2024-03-18T23:19:42.435 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-03-18T23:19:42.435 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-03-18T23:19:42.435 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-03-18T23:19:42.435 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:42.435 INFO:teuthology.orchestra.run.smithi175.stdout: "cap_revoke_eviction": 0, 2024-03-18T23:19:42.436 INFO:teuthology.orchestra.run.smithi175.stdout: "cap_acquisition_throttle": 0, 2024-03-18T23:19:42.436 INFO:teuthology.orchestra.run.smithi175.stdout: "req_getvxattr_latency": { 2024-03-18T23:19:42.436 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-03-18T23:19:42.436 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-03-18T23:19:42.436 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-03-18T23:19:42.436 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-03-18T23:19:42.436 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:42.436 INFO:teuthology.orchestra.run.smithi175.stdout: "mds_sessions": { 2024-03-18T23:19:42.437 INFO:teuthology.orchestra.run.smithi175.stdout: "session_count": 2, 2024-03-18T23:19:42.437 INFO:teuthology.orchestra.run.smithi175.stdout: "session_add": 3, 2024-03-18T23:19:42.437 INFO:teuthology.orchestra.run.smithi175.stdout: "session_remove": 1, 2024-03-18T23:19:42.437 INFO:teuthology.orchestra.run.smithi175.stdout: "sessions_open": 2, 2024-03-18T23:19:42.437 INFO:teuthology.orchestra.run.smithi175.stdout: "sessions_stale": 0, 2024-03-18T23:19:42.437 INFO:teuthology.orchestra.run.smithi175.stdout: "total_load": 5, 2024-03-18T23:19:42.437 INFO:teuthology.orchestra.run.smithi175.stdout: "average_load": 1, 2024-03-18T23:19:42.438 INFO:teuthology.orchestra.run.smithi175.stdout: "avg_session_uptime": 660, 2024-03-18T23:19:42.438 INFO:teuthology.orchestra.run.smithi175.stdout: "mdthresh_evicted": 0 2024-03-18T23:19:42.438 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:42.438 INFO:teuthology.orchestra.run.smithi175.stdout: "mempool": { 2024-03-18T23:19:42.438 INFO:teuthology.orchestra.run.smithi175.stdout: "bloom_filter_bytes": 0, 2024-03-18T23:19:42.438 INFO:teuthology.orchestra.run.smithi175.stdout: "bloom_filter_items": 0, 2024-03-18T23:19:42.439 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_alloc_bytes": 0, 2024-03-18T23:19:42.439 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_alloc_items": 0, 2024-03-18T23:19:42.439 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_data_bytes": 0, 2024-03-18T23:19:42.439 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_data_items": 0, 2024-03-18T23:19:42.439 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-18T23:19:42.439 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_onode_items": 0, 2024-03-18T23:19:42.439 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-18T23:19:42.439 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_meta_items": 0, 2024-03-18T23:19:42.440 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_other_bytes": 0, 2024-03-18T23:19:42.440 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_other_items": 0, 2024-03-18T23:19:42.440 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-18T23:19:42.440 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_buffer_items": 0, 2024-03-18T23:19:42.440 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_extent_bytes": 0, 2024-03-18T23:19:42.440 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_extent_items": 0, 2024-03-18T23:19:42.440 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_blob_bytes": 0, 2024-03-18T23:19:42.440 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_blob_items": 0, 2024-03-18T23:19:42.441 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-18T23:19:42.441 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_shared_blob_items": 0, 2024-03-18T23:19:42.441 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-18T23:19:42.441 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_inline_bl_items": 0, 2024-03-18T23:19:42.441 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_fsck_bytes": 0, 2024-03-18T23:19:42.441 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_fsck_items": 0, 2024-03-18T23:19:42.441 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_txc_bytes": 0, 2024-03-18T23:19:42.442 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_txc_items": 0, 2024-03-18T23:19:42.442 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-18T23:19:42.442 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_writing_deferred_items": 0, 2024-03-18T23:19:42.442 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_writing_bytes": 0, 2024-03-18T23:19:42.442 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_writing_items": 0, 2024-03-18T23:19:42.443 INFO:teuthology.orchestra.run.smithi175.stdout: "bluefs_bytes": 0, 2024-03-18T23:19:42.443 INFO:teuthology.orchestra.run.smithi175.stdout: "bluefs_items": 0, 2024-03-18T23:19:42.443 INFO:teuthology.orchestra.run.smithi175.stdout: "bluefs_file_reader_bytes": 0, 2024-03-18T23:19:42.443 INFO:teuthology.orchestra.run.smithi175.stdout: "bluefs_file_reader_items": 0, 2024-03-18T23:19:42.443 INFO:teuthology.orchestra.run.smithi175.stdout: "bluefs_file_writer_bytes": 0, 2024-03-18T23:19:42.443 INFO:teuthology.orchestra.run.smithi175.stdout: "bluefs_file_writer_items": 0, 2024-03-18T23:19:42.443 INFO:teuthology.orchestra.run.smithi175.stdout: "buffer_anon_bytes": 257308, 2024-03-18T23:19:42.443 INFO:teuthology.orchestra.run.smithi175.stdout: "buffer_anon_items": 116, 2024-03-18T23:19:42.443 INFO:teuthology.orchestra.run.smithi175.stdout: "buffer_meta_bytes": 176, 2024-03-18T23:19:42.444 INFO:teuthology.orchestra.run.smithi175.stdout: "buffer_meta_items": 2, 2024-03-18T23:19:42.444 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_bytes": 0, 2024-03-18T23:19:42.444 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_items": 0, 2024-03-18T23:19:42.444 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_mapbl_bytes": 0, 2024-03-18T23:19:42.444 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_mapbl_items": 0, 2024-03-18T23:19:42.444 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_pglog_bytes": 0, 2024-03-18T23:19:42.444 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_pglog_items": 0, 2024-03-18T23:19:42.444 INFO:teuthology.orchestra.run.smithi175.stdout: "osdmap_bytes": 8256, 2024-03-18T23:19:42.445 INFO:teuthology.orchestra.run.smithi175.stdout: "osdmap_items": 188, 2024-03-18T23:19:42.445 INFO:teuthology.orchestra.run.smithi175.stdout: "osdmap_mapping_bytes": 0, 2024-03-18T23:19:42.445 INFO:teuthology.orchestra.run.smithi175.stdout: "osdmap_mapping_items": 0, 2024-03-18T23:19:42.445 INFO:teuthology.orchestra.run.smithi175.stdout: "pgmap_bytes": 0, 2024-03-18T23:19:42.445 INFO:teuthology.orchestra.run.smithi175.stdout: "pgmap_items": 0, 2024-03-18T23:19:42.445 INFO:teuthology.orchestra.run.smithi175.stdout: "mds_co_bytes": 133268, 2024-03-18T23:19:42.445 INFO:teuthology.orchestra.run.smithi175.stdout: "mds_co_items": 720, 2024-03-18T23:19:42.445 INFO:teuthology.orchestra.run.smithi175.stdout: "unittest_1_bytes": 0, 2024-03-18T23:19:42.446 INFO:teuthology.orchestra.run.smithi175.stdout: "unittest_1_items": 0, 2024-03-18T23:19:42.446 INFO:teuthology.orchestra.run.smithi175.stdout: "unittest_2_bytes": 0, 2024-03-18T23:19:42.446 INFO:teuthology.orchestra.run.smithi175.stdout: "unittest_2_items": 0 2024-03-18T23:19:42.446 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:42.446 INFO:teuthology.orchestra.run.smithi175.stdout: "objecter": { 2024-03-18T23:19:42.446 INFO:teuthology.orchestra.run.smithi175.stdout: "op_active": 0, 2024-03-18T23:19:42.447 INFO:teuthology.orchestra.run.smithi175.stdout: "op_laggy": 0, 2024-03-18T23:19:42.447 INFO:teuthology.orchestra.run.smithi175.stdout: "op_send": 44, 2024-03-18T23:19:42.447 INFO:teuthology.orchestra.run.smithi175.stdout: "op_send_bytes": 12356, 2024-03-18T23:19:42.447 INFO:teuthology.orchestra.run.smithi175.stdout: "op_resend": 33, 2024-03-18T23:19:42.447 INFO:teuthology.orchestra.run.smithi175.stdout: "op_reply": 44, 2024-03-18T23:19:42.447 INFO:teuthology.orchestra.run.smithi175.stdout: "op_latency": { 2024-03-18T23:19:42.447 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 44, 2024-03-18T23:19:42.447 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.222996736, 2024-03-18T23:19:42.448 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.005068107 2024-03-18T23:19:42.448 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:42.448 INFO:teuthology.orchestra.run.smithi175.stdout: "op_inflight": 0, 2024-03-18T23:19:42.448 INFO:teuthology.orchestra.run.smithi175.stdout: "oplen_avg": { 2024-03-18T23:19:42.448 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 44, 2024-03-18T23:19:42.448 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 68 2024-03-18T23:19:42.448 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:42.449 INFO:teuthology.orchestra.run.smithi175.stdout: "op": 44, 2024-03-18T23:19:42.449 INFO:teuthology.orchestra.run.smithi175.stdout: "op_r": 0, 2024-03-18T23:19:42.449 INFO:teuthology.orchestra.run.smithi175.stdout: "op_w": 44, 2024-03-18T23:19:42.449 INFO:teuthology.orchestra.run.smithi175.stdout: "op_rmw": 0, 2024-03-18T23:19:42.449 INFO:teuthology.orchestra.run.smithi175.stdout: "op_pg": 0, 2024-03-18T23:19:42.449 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_stat": 0, 2024-03-18T23:19:42.449 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_create": 10, 2024-03-18T23:19:42.449 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_read": 0, 2024-03-18T23:19:42.449 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_write": 5, 2024-03-18T23:19:42.450 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_writefull": 8, 2024-03-18T23:19:42.450 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_writesame": 0, 2024-03-18T23:19:42.450 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_append": 0, 2024-03-18T23:19:42.450 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_zero": 0, 2024-03-18T23:19:42.450 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_truncate": 0, 2024-03-18T23:19:42.450 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_delete": 6, 2024-03-18T23:19:42.451 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_mapext": 0, 2024-03-18T23:19:42.451 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_sparse_read": 0, 2024-03-18T23:19:42.451 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_clonerange": 0, 2024-03-18T23:19:42.451 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_getxattr": 0, 2024-03-18T23:19:42.451 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_setxattr": 20, 2024-03-18T23:19:42.451 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_cmpxattr": 0, 2024-03-18T23:19:42.451 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_rmxattr": 0, 2024-03-18T23:19:42.451 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_resetxattrs": 0, 2024-03-18T23:19:42.452 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_call": 0, 2024-03-18T23:19:42.452 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_watch": 0, 2024-03-18T23:19:42.452 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_notify": 0, 2024-03-18T23:19:42.452 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_src_cmpxattr": 0, 2024-03-18T23:19:42.452 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_pgls": 0, 2024-03-18T23:19:42.452 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_pgls_filter": 0, 2024-03-18T23:19:42.452 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_other": 0, 2024-03-18T23:19:42.452 INFO:teuthology.orchestra.run.smithi175.stdout: "linger_active": 0, 2024-03-18T23:19:42.452 INFO:teuthology.orchestra.run.smithi175.stdout: "linger_send": 0, 2024-03-18T23:19:42.453 INFO:teuthology.orchestra.run.smithi175.stdout: "linger_resend": 0, 2024-03-18T23:19:42.453 INFO:teuthology.orchestra.run.smithi175.stdout: "linger_ping": 0, 2024-03-18T23:19:42.453 INFO:teuthology.orchestra.run.smithi175.stdout: "poolop_active": 0, 2024-03-18T23:19:42.453 INFO:teuthology.orchestra.run.smithi175.stdout: "poolop_send": 0, 2024-03-18T23:19:42.453 INFO:teuthology.orchestra.run.smithi175.stdout: "poolop_resend": 0, 2024-03-18T23:19:42.453 INFO:teuthology.orchestra.run.smithi175.stdout: "poolstat_active": 0, 2024-03-18T23:19:42.453 INFO:teuthology.orchestra.run.smithi175.stdout: "poolstat_send": 0, 2024-03-18T23:19:42.454 INFO:teuthology.orchestra.run.smithi175.stdout: "poolstat_resend": 0, 2024-03-18T23:19:42.454 INFO:teuthology.orchestra.run.smithi175.stdout: "statfs_active": 0, 2024-03-18T23:19:42.454 INFO:teuthology.orchestra.run.smithi175.stdout: "statfs_send": 0, 2024-03-18T23:19:42.454 INFO:teuthology.orchestra.run.smithi175.stdout: "statfs_resend": 0, 2024-03-18T23:19:42.454 INFO:teuthology.orchestra.run.smithi175.stdout: "command_active": 0, 2024-03-18T23:19:42.454 INFO:teuthology.orchestra.run.smithi175.stdout: "command_send": 0, 2024-03-18T23:19:42.454 INFO:teuthology.orchestra.run.smithi175.stdout: "command_resend": 0, 2024-03-18T23:19:42.454 INFO:teuthology.orchestra.run.smithi175.stdout: "map_epoch": 101, 2024-03-18T23:19:42.455 INFO:teuthology.orchestra.run.smithi175.stdout: "map_full": 0, 2024-03-18T23:19:42.455 INFO:teuthology.orchestra.run.smithi175.stdout: "map_inc": 15, 2024-03-18T23:19:42.455 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_sessions": 9, 2024-03-18T23:19:42.455 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_session_open": 9, 2024-03-18T23:19:42.455 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_session_close": 0, 2024-03-18T23:19:42.455 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_laggy": 0, 2024-03-18T23:19:42.455 INFO:teuthology.orchestra.run.smithi175.stdout: "omap_wr": 17, 2024-03-18T23:19:42.455 INFO:teuthology.orchestra.run.smithi175.stdout: "omap_rd": 0, 2024-03-18T23:19:42.455 INFO:teuthology.orchestra.run.smithi175.stdout: "omap_del": 2 2024-03-18T23:19:42.456 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:42.456 INFO:teuthology.orchestra.run.smithi175.stdout: "oft": { 2024-03-18T23:19:42.456 INFO:teuthology.orchestra.run.smithi175.stdout: "omap_total_objs": 1, 2024-03-18T23:19:42.456 INFO:teuthology.orchestra.run.smithi175.stdout: "omap_total_kv_pairs": 0, 2024-03-18T23:19:42.456 INFO:teuthology.orchestra.run.smithi175.stdout: "omap_total_updates": 0, 2024-03-18T23:19:42.456 INFO:teuthology.orchestra.run.smithi175.stdout: "omap_total_removes": 0 2024-03-18T23:19:42.456 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:42.456 INFO:teuthology.orchestra.run.smithi175.stdout: "osd-slow-ops": { 2024-03-18T23:19:42.457 INFO:teuthology.orchestra.run.smithi175.stdout: "slow_ops_count": 0 2024-03-18T23:19:42.457 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:42.457 INFO:teuthology.orchestra.run.smithi175.stdout: "purge_queue": { 2024-03-18T23:19:42.457 INFO:teuthology.orchestra.run.smithi175.stdout: "pq_executing_ops": 0, 2024-03-18T23:19:42.457 INFO:teuthology.orchestra.run.smithi175.stdout: "pq_executing_ops_high_water": 0, 2024-03-18T23:19:42.457 INFO:teuthology.orchestra.run.smithi175.stdout: "pq_executing": 0, 2024-03-18T23:19:42.457 INFO:teuthology.orchestra.run.smithi175.stdout: "pq_executing_high_water": 0, 2024-03-18T23:19:42.457 INFO:teuthology.orchestra.run.smithi175.stdout: "pq_executed_ops": 0, 2024-03-18T23:19:42.458 INFO:teuthology.orchestra.run.smithi175.stdout: "pq_executed": 0, 2024-03-18T23:19:42.458 INFO:teuthology.orchestra.run.smithi175.stdout: "pq_item_in_journal": 0 2024-03-18T23:19:42.458 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:42.458 INFO:teuthology.orchestra.run.smithi175.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-18T23:19:42.458 INFO:teuthology.orchestra.run.smithi175.stdout: "val": 0, 2024-03-18T23:19:42.458 INFO:teuthology.orchestra.run.smithi175.stdout: "max": 104857600, 2024-03-18T23:19:42.459 INFO:teuthology.orchestra.run.smithi175.stdout: "get_started": 0, 2024-03-18T23:19:42.459 INFO:teuthology.orchestra.run.smithi175.stdout: "get": 174295, 2024-03-18T23:19:42.459 INFO:teuthology.orchestra.run.smithi175.stdout: "get_sum": 80847718, 2024-03-18T23:19:42.459 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_fail": 0, 2024-03-18T23:19:42.459 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_success": 174295, 2024-03-18T23:19:42.459 INFO:teuthology.orchestra.run.smithi175.stdout: "take": 0, 2024-03-18T23:19:42.459 INFO:teuthology.orchestra.run.smithi175.stdout: "take_sum": 0, 2024-03-18T23:19:42.459 INFO:teuthology.orchestra.run.smithi175.stdout: "put": 174295, 2024-03-18T23:19:42.460 INFO:teuthology.orchestra.run.smithi175.stdout: "put_sum": 80847718, 2024-03-18T23:19:42.460 INFO:teuthology.orchestra.run.smithi175.stdout: "wait": { 2024-03-18T23:19:42.460 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-03-18T23:19:42.460 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-03-18T23:19:42.460 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-03-18T23:19:42.460 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-03-18T23:19:42.461 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:42.461 INFO:teuthology.orchestra.run.smithi175.stdout: "throttle-objecter_bytes": { 2024-03-18T23:19:42.461 INFO:teuthology.orchestra.run.smithi175.stdout: "val": 0, 2024-03-18T23:19:42.461 INFO:teuthology.orchestra.run.smithi175.stdout: "max": 104857600, 2024-03-18T23:19:42.461 INFO:teuthology.orchestra.run.smithi175.stdout: "get_started": 0, 2024-03-18T23:19:42.461 INFO:teuthology.orchestra.run.smithi175.stdout: "get": 0, 2024-03-18T23:19:42.461 INFO:teuthology.orchestra.run.smithi175.stdout: "get_sum": 0, 2024-03-18T23:19:42.461 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_fail": 0, 2024-03-18T23:19:42.462 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_success": 0, 2024-03-18T23:19:42.462 INFO:teuthology.orchestra.run.smithi175.stdout: "take": 44, 2024-03-18T23:19:42.462 INFO:teuthology.orchestra.run.smithi175.stdout: "take_sum": 12356, 2024-03-18T23:19:42.462 INFO:teuthology.orchestra.run.smithi175.stdout: "put": 44, 2024-03-18T23:19:42.462 INFO:teuthology.orchestra.run.smithi175.stdout: "put_sum": 12356, 2024-03-18T23:19:42.462 INFO:teuthology.orchestra.run.smithi175.stdout: "wait": { 2024-03-18T23:19:42.463 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-03-18T23:19:42.463 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-03-18T23:19:42.463 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-03-18T23:19:42.463 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-03-18T23:19:42.463 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:42.463 INFO:teuthology.orchestra.run.smithi175.stdout: "throttle-objecter_ops": { 2024-03-18T23:19:42.463 INFO:teuthology.orchestra.run.smithi175.stdout: "val": 0, 2024-03-18T23:19:42.463 INFO:teuthology.orchestra.run.smithi175.stdout: "max": 1024, 2024-03-18T23:19:42.464 INFO:teuthology.orchestra.run.smithi175.stdout: "get_started": 0, 2024-03-18T23:19:42.464 INFO:teuthology.orchestra.run.smithi175.stdout: "get": 0, 2024-03-18T23:19:42.464 INFO:teuthology.orchestra.run.smithi175.stdout: "get_sum": 0, 2024-03-18T23:19:42.464 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_fail": 0, 2024-03-18T23:19:42.464 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_success": 0, 2024-03-18T23:19:42.464 INFO:teuthology.orchestra.run.smithi175.stdout: "take": 44, 2024-03-18T23:19:42.465 INFO:teuthology.orchestra.run.smithi175.stdout: "take_sum": 44, 2024-03-18T23:19:42.465 INFO:teuthology.orchestra.run.smithi175.stdout: "put": 44, 2024-03-18T23:19:42.465 INFO:teuthology.orchestra.run.smithi175.stdout: "put_sum": 44, 2024-03-18T23:19:42.465 INFO:teuthology.orchestra.run.smithi175.stdout: "wait": { 2024-03-18T23:19:42.465 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-03-18T23:19:42.465 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-03-18T23:19:42.465 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-03-18T23:19:42.465 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-03-18T23:19:42.466 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:42.466 INFO:teuthology.orchestra.run.smithi175.stdout: "throttle-write_buf_throttle": { 2024-03-18T23:19:42.466 INFO:teuthology.orchestra.run.smithi175.stdout: "val": 0, 2024-03-18T23:19:42.466 INFO:teuthology.orchestra.run.smithi175.stdout: "max": 3758096384, 2024-03-18T23:19:42.466 INFO:teuthology.orchestra.run.smithi175.stdout: "get_started": 0, 2024-03-18T23:19:42.466 INFO:teuthology.orchestra.run.smithi175.stdout: "get": 0, 2024-03-18T23:19:42.466 INFO:teuthology.orchestra.run.smithi175.stdout: "get_sum": 0, 2024-03-18T23:19:42.466 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_fail": 0, 2024-03-18T23:19:42.466 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_success": 0, 2024-03-18T23:19:42.467 INFO:teuthology.orchestra.run.smithi175.stdout: "take": 0, 2024-03-18T23:19:42.467 INFO:teuthology.orchestra.run.smithi175.stdout: "take_sum": 0, 2024-03-18T23:19:42.467 INFO:teuthology.orchestra.run.smithi175.stdout: "put": 0, 2024-03-18T23:19:42.467 INFO:teuthology.orchestra.run.smithi175.stdout: "put_sum": 0, 2024-03-18T23:19:42.467 INFO:teuthology.orchestra.run.smithi175.stdout: "wait": { 2024-03-18T23:19:42.467 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-03-18T23:19:42.467 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-03-18T23:19:42.467 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-03-18T23:19:42.468 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-03-18T23:19:42.468 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-03-18T23:19:42.468 INFO:teuthology.orchestra.run.smithi175.stdout: "throttle-write_buf_throttle-0x557ba51ce500": { 2024-03-18T23:19:42.468 INFO:teuthology.orchestra.run.smithi175.stdout: "val": 0, 2024-03-18T23:19:42.468 INFO:teuthology.orchestra.run.smithi175.stdout: "max": 3758096384, 2024-03-18T23:19:42.468 INFO:teuthology.orchestra.run.smithi175.stdout: "get_started": 0, 2024-03-18T23:19:42.468 INFO:teuthology.orchestra.run.smithi175.stdout: "get": 6, 2024-03-18T23:19:42.468 INFO:teuthology.orchestra.run.smithi175.stdout: "get_sum": 1819, 2024-03-18T23:19:42.468 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_fail": 0, 2024-03-18T23:19:42.469 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_success": 6, 2024-03-18T23:19:42.469 INFO:teuthology.orchestra.run.smithi175.stdout: "take": 0, 2024-03-18T23:19:42.469 INFO:teuthology.orchestra.run.smithi175.stdout: "take_sum": 0, 2024-03-18T23:19:42.469 INFO:teuthology.orchestra.run.smithi175.stdout: "put": 5, 2024-03-18T23:19:42.469 INFO:teuthology.orchestra.run.smithi175.stdout: "put_sum": 1819, 2024-03-18T23:19:42.469 INFO:teuthology.orchestra.run.smithi175.stdout: "wait": { 2024-03-18T23:19:42.469 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-03-18T23:19:42.469 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-03-18T23:19:42.470 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-03-18T23:19:42.470 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-03-18T23:19:42.470 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-03-18T23:19:42.470 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-03-18T23:19:42.710 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:19:42 smithi175 ceph-mon[67352]: pgmap v358: 97 pgs: 97 active+clean; 61 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 600 KiB/s wr, 40 op/s 2024-03-18T23:19:42.814 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:19:42 smithi139 ceph-mon[67282]: pgmap v358: 97 pgs: 97 active+clean; 61 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 600 KiB/s wr, 40 op/s 2024-03-18T23:19:42.879 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:19:42 smithi049 ceph-mon[62991]: pgmap v358: 97 pgs: 97 active+clean; 61 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 600 KiB/s wr, 40 op/s 2024-03-18T23:19:43.158 INFO:tasks.check_counter:Daemon mds.l mds.exported=0 2024-03-18T23:19:43.158 INFO:tasks.check_counter:Daemon mds.l mds.imported=0 2024-03-18T23:19:43.158 INFO:tasks.check_counter:Daemon mds.l mds.root_rsnaps=6 2024-03-18T23:19:43.158 INFO:tasks.check_counter:Daemon mds.l mds_server.req_mksnap_latency.avgcount=0 2024-03-18T23:19:43.159 INFO:tasks.check_counter:Daemon mds.l mds_server.req_rmsnap_latency.avgcount=0 2024-03-18T23:19:43.159 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-03-18T23:19:43.171 INFO:tasks.kclient:Unmounting kernel clients... 2024-03-18T23:19:43.171 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-03-18T23:19:43.171 DEBUG:teuthology.orchestra.run.smithi049:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-18T23:19:43.192 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-03-18T23:19:43.192 DEBUG:teuthology.orchestra.run.smithi049:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-18T23:19:43.252 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-18T23:19:43.416 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:43.414+0000 7fbde7a0b640 1 -- 172.21.15.49:0/4070077899 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbde01021a0 msgr2=0x7fbde010e6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:19:43.416 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:43.414+0000 7fbde7a0b640 1 --2- 172.21.15.49:0/4070077899 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbde01021a0 0x7fbde010e6b0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fbdd8009cd0 tx=0x7fbdd802c0f0 comp rx=0 tx=0).stop 2024-03-18T23:19:43.416 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:43.414+0000 7fbde7a0b640 1 -- 172.21.15.49:0/4070077899 shutdown_connections 2024-03-18T23:19:43.417 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:43.414+0000 7fbde7a0b640 1 --2- 172.21.15.49:0/4070077899 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbde01021a0 0x7fbde010e6b0 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:43.417 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:43.414+0000 7fbde7a0b640 1 --2- 172.21.15.49:0/4070077899 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbde01017e0 0x7fbde0101c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:43.417 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:43.414+0000 7fbde7a0b640 1 --2- 172.21.15.49:0/4070077899 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbde01077f0 0x7fbde0107bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:43.417 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:43.414+0000 7fbde7a0b640 1 -- 172.21.15.49:0/4070077899 >> 172.21.15.49:0/4070077899 conn(0x7fbde00fd290 msgr2=0x7fbde00ff6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:19:43.417 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:43.414+0000 7fbde7a0b640 1 -- 172.21.15.49:0/4070077899 shutdown_connections 2024-03-18T23:19:43.417 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:43.414+0000 7fbde7a0b640 1 -- 172.21.15.49:0/4070077899 wait complete. 2024-03-18T23:19:43.417 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:43.414+0000 7fbde7a0b640 1 Processor -- start 2024-03-18T23:19:43.417 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:43.415+0000 7fbde7a0b640 1 -- start start 2024-03-18T23:19:43.418 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:43.415+0000 7fbde7a0b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbde01017e0 0x7fbde01980a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:19:43.418 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:43.415+0000 7fbde7a0b640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbde01021a0 0x7fbde01985e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:19:43.418 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:43.415+0000 7fbde7a0b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbde01077f0 0x7fbde019c960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:19:43.418 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:43.415+0000 7fbde7a0b640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fbde0111c00 con 0x7fbde01077f0 2024-03-18T23:19:43.418 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:43.415+0000 7fbde7a0b640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fbde0111a80 con 0x7fbde01017e0 2024-03-18T23:19:43.418 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:43.415+0000 7fbde7a0b640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fbde0111d80 con 0x7fbde01021a0 2024-03-18T23:19:43.418 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:43.415+0000 7fbde5fa8640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbde01077f0 0x7fbde019c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:19:43.418 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:43.415+0000 7fbde57a7640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbde01017e0 0x7fbde01980a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:19:43.419 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:43.415+0000 7fbde5fa8640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbde01077f0 0x7fbde019c960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:40294/0 (socket says 172.21.15.49:40294) 2024-03-18T23:19:43.419 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:43.415+0000 7fbde5fa8640 1 -- 172.21.15.49:0/949437498 learned_addr learned my addr 172.21.15.49:0/949437498 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:19:43.419 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:43.415+0000 7fbde4fa6640 1 --2- 172.21.15.49:0/949437498 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbde01021a0 0x7fbde01985e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:19:43.419 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:43.415+0000 7fbde5fa8640 1 -- 172.21.15.49:0/949437498 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbde01021a0 msgr2=0x7fbde01985e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:19:43.419 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:43.415+0000 7fbde5fa8640 1 --2- 172.21.15.49:0/949437498 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbde01021a0 0x7fbde01985e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:43.419 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:43.415+0000 7fbde5fa8640 1 -- 172.21.15.49:0/949437498 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbde01017e0 msgr2=0x7fbde01980a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:19:43.419 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:43.415+0000 7fbde5fa8640 1 --2- 172.21.15.49:0/949437498 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbde01017e0 0x7fbde01980a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:43.419 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:43.415+0000 7fbde5fa8640 1 -- 172.21.15.49:0/949437498 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbde019d090 con 0x7fbde01077f0 2024-03-18T23:19:43.419 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:43.415+0000 7fbde5fa8640 1 --2- 172.21.15.49:0/949437498 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbde01077f0 0x7fbde019c960 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fbdd80348f0 tx=0x7fbdd8032c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:19:43.420 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:43.416+0000 7fbdd67fc640 1 -- 172.21.15.49:0/949437498 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdd8041020 con 0x7fbde01077f0 2024-03-18T23:19:43.420 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:43.416+0000 7fbdd67fc640 1 -- 172.21.15.49:0/949437498 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbdd8032e80 con 0x7fbde01077f0 2024-03-18T23:19:43.420 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:43.416+0000 7fbde7a0b640 1 -- 172.21.15.49:0/949437498 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbde019d320 con 0x7fbde01077f0 2024-03-18T23:19:43.420 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:43.416+0000 7fbdd67fc640 1 -- 172.21.15.49:0/949437498 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdd8039040 con 0x7fbde01077f0 2024-03-18T23:19:43.420 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:43.416+0000 7fbde7a0b640 1 -- 172.21.15.49:0/949437498 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbde01a4c10 con 0x7fbde01077f0 2024-03-18T23:19:43.421 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:43.417+0000 7fbde7a0b640 1 -- 172.21.15.49:0/949437498 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbde0068a60 con 0x7fbde01077f0 2024-03-18T23:19:43.421 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:43.418+0000 7fbdd67fc640 1 -- 172.21.15.49:0/949437498 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 25) ==== 137421+0+0 (secure 0 0 0) 0x7fbdd8044020 con 0x7fbde01077f0 2024-03-18T23:19:43.421 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:43.418+0000 7fbdd67fc640 1 --2- 172.21.15.49:0/949437498 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7fbdcc0a2970 0x7fbdcc0a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:19:43.422 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:43.418+0000 7fbde57a7640 1 --2- 172.21.15.49:0/949437498 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7fbdcc0a2970 0x7fbdcc0a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:19:43.422 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:43.419+0000 7fbdd67fc640 1 -- 172.21.15.49:0/949437498 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8556+0+0 (secure 0 0 0) 0x7fbdd80f0960 con 0x7fbde01077f0 2024-03-18T23:19:43.422 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:43.419+0000 7fbde57a7640 1 --2- 172.21.15.49:0/949437498 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7fbdcc0a2970 0x7fbdcc0a4e30 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fbddc009b50 tx=0x7fbddc005f90 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:19:43.422 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:43.423+0000 7fbdd67fc640 1 -- 172.21.15.49:0/949437498 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fbdd804b020 con 0x7fbde01077f0 2024-03-18T23:19:43.608 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:43.608+0000 7fbde7a0b640 1 -- 172.21.15.49:0/949437498 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbde0109d80 con 0x7fbde01077f0 2024-03-18T23:19:43.609 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:43.610+0000 7fbdd67fc640 1 -- 172.21.15.49:0/949437498 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v101) ==== 74+0+23204 (secure 0 0 0) 0x7fbdd80bd7b0 con 0x7fbde01077f0 2024-03-18T23:19:43.610 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-18T23:19:43.610 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":101,"fsid":"c63591ea-e57a-11ee-95c9-87774f69a715","created":"2024-03-18T22:57:58.734467+0000","modified":"2024-03-18T23:19:19.125020+0000","last_up_change":"2024-03-18T23:05:16.576310+0000","last_in_change":"2024-03-18T23:04:57.024067+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-18T23:01:28.363432+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-18T23:06:03.172963+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-18T23:06:03.973303+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"100","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":17,"snap_epoch":100,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e7fc9daa-4098-4ba5-8019-9aaef1aab3e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":1651162501},{"type":"v1","addr":"172.21.15.49:6803","nonce":1651162501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":1651162501},{"type":"v1","addr":"172.21.15.49:6805","nonce":1651162501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":1651162501},{"type":"v1","addr":"172.21.15.49:6809","nonce":1651162501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":1651162501},{"type":"v1","addr":"172.21.15.49:6807","nonce":1651162501}]},"public_addr":"172.21.15.49:6803/1651162501","cluster_addr":"172.21.15.49:6805/1651162501","heartbeat_back_addr":"172.21.15.49:6809/1651162501","heartbeat_front_addr":"172.21.15.49:6807/1651162501","state":["exists","up"]},{"osd":1,"uuid":"40ac929c-e90f-434c-a796-9aabd686f1fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":1766700359},{"type":"v1","addr":"172.21.15.139:6801","nonce":1766700359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":1766700359},{"type":"v1","addr":"172.21.15.139:6803","nonce":1766700359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":1766700359},{"type":"v1","addr":"172.21.15.139:6807","nonce":1766700359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":1766700359},{"type":"v1","addr":"172.21.15.139:6805","nonce":1766700359}]},"public_addr":"172.21.15.139:6801/1766700359","cluster_addr":"172.21.15.139:6803/1766700359","heartbeat_back_addr":"172.21.15.139:6807/1766700359","heartbeat_front_addr":"172.21.15.139:6805/1766700359","state":["exists","up"]},{"osd":2,"uuid":"8ae3b666-00b4-45ed-a5ee-4d0771edabb6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":3752671234},{"type":"v1","addr":"172.21.15.175:6801","nonce":3752671234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":3752671234},{"type":"v1","addr":"172.21.15.175:6803","nonce":3752671234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":3752671234},{"type":"v1","addr":"172.21.15.175:6807","nonce":3752671234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":3752671234},{"type":"v1","addr":"172.21.15.175:6805","nonce":3752671234}]},"public_addr":"172.21.15.175:6801/3752671234","cluster_addr":"172.21.15.175:6803/3752671234","heartbeat_back_addr":"172.21.15.175:6807/3752671234","heartbeat_front_addr":"172.21.15.175:6805/3752671234","state":["exists","up"]},{"osd":3,"uuid":"1a62e89d-45d4-4237-b813-7f96a59aca7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":956160536},{"type":"v1","addr":"172.21.15.49:6811","nonce":956160536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":956160536},{"type":"v1","addr":"172.21.15.49:6813","nonce":956160536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":956160536},{"type":"v1","addr":"172.21.15.49:6817","nonce":956160536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":956160536},{"type":"v1","addr":"172.21.15.49:6815","nonce":956160536}]},"public_addr":"172.21.15.49:6811/956160536","cluster_addr":"172.21.15.49:6813/956160536","heartbeat_back_addr":"172.21.15.49:6817/956160536","heartbeat_front_addr":"172.21.15.49:6815/956160536","state":["exists","up"]},{"osd":4,"uuid":"c715101c-0c3f-4b98-b142-d64a35e187df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":3129712668},{"type":"v1","addr":"172.21.15.139:6809","nonce":3129712668}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":3129712668},{"type":"v1","addr":"172.21.15.139:6811","nonce":3129712668}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":3129712668},{"type":"v1","addr":"172.21.15.139:6815","nonce":3129712668}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":3129712668},{"type":"v1","addr":"172.21.15.139:6813","nonce":3129712668}]},"public_addr":"172.21.15.139:6809/3129712668","cluster_addr":"172.21.15.139:6811/3129712668","heartbeat_back_addr":"172.21.15.139:6815/3129712668","heartbeat_front_addr":"172.21.15.139:6813/3129712668","state":["exists","up"]},{"osd":5,"uuid":"4b641f81-77eb-45fd-8464-511e7b3062b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":2774567215},{"type":"v1","addr":"172.21.15.175:6809","nonce":2774567215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":2774567215},{"type":"v1","addr":"172.21.15.175:6811","nonce":2774567215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":2774567215},{"type":"v1","addr":"172.21.15.175:6815","nonce":2774567215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":2774567215},{"type":"v1","addr":"172.21.15.175:6813","nonce":2774567215}]},"public_addr":"172.21.15.175:6809/2774567215","cluster_addr":"172.21.15.175:6811/2774567215","heartbeat_back_addr":"172.21.15.175:6815/2774567215","heartbeat_front_addr":"172.21.15.175:6813/2774567215","state":["exists","up"]},{"osd":6,"uuid":"653da8e0-1197-4b48-abed-470f2a9c4dce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":1472215461},{"type":"v1","addr":"172.21.15.49:6819","nonce":1472215461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":1472215461},{"type":"v1","addr":"172.21.15.49:6821","nonce":1472215461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":1472215461},{"type":"v1","addr":"172.21.15.49:6825","nonce":1472215461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":1472215461},{"type":"v1","addr":"172.21.15.49:6823","nonce":1472215461}]},"public_addr":"172.21.15.49:6819/1472215461","cluster_addr":"172.21.15.49:6821/1472215461","heartbeat_back_addr":"172.21.15.49:6825/1472215461","heartbeat_front_addr":"172.21.15.49:6823/1472215461","state":["exists","up"]},{"osd":7,"uuid":"ad157c9b-4b75-451c-8740-630f0975b7fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":2163986787},{"type":"v1","addr":"172.21.15.139:6817","nonce":2163986787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":2163986787},{"type":"v1","addr":"172.21.15.139:6819","nonce":2163986787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":2163986787},{"type":"v1","addr":"172.21.15.139:6823","nonce":2163986787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":2163986787},{"type":"v1","addr":"172.21.15.139:6821","nonce":2163986787}]},"public_addr":"172.21.15.139:6817/2163986787","cluster_addr":"172.21.15.139:6819/2163986787","heartbeat_back_addr":"172.21.15.139:6823/2163986787","heartbeat_front_addr":"172.21.15.139:6821/2163986787","state":["exists","up"]},{"osd":8,"uuid":"c8e75f15-df75-4d5d-9b92-a064bffb6c01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":1194026992},{"type":"v1","addr":"172.21.15.175:6817","nonce":1194026992}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1194026992},{"type":"v1","addr":"172.21.15.175:6819","nonce":1194026992}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":1194026992},{"type":"v1","addr":"172.21.15.175:6823","nonce":1194026992}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1194026992},{"type":"v1","addr":"172.21.15.175:6821","nonce":1194026992}]},"public_addr":"172.21.15.175:6817/1194026992","cluster_addr":"172.21.15.175:6819/1194026992","heartbeat_back_addr":"172.21.15.175:6823/1194026992","heartbeat_front_addr":"172.21.15.175:6821/1194026992","state":["exists","up"]},{"osd":9,"uuid":"edb5ffef-b6a2-4941-b5ab-40f982ad30d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":2742084884},{"type":"v1","addr":"172.21.15.49:6827","nonce":2742084884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":2742084884},{"type":"v1","addr":"172.21.15.49:6829","nonce":2742084884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":2742084884},{"type":"v1","addr":"172.21.15.49:6833","nonce":2742084884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":2742084884},{"type":"v1","addr":"172.21.15.49:6831","nonce":2742084884}]},"public_addr":"172.21.15.49:6827/2742084884","cluster_addr":"172.21.15.49:6829/2742084884","heartbeat_back_addr":"172.21.15.49:6833/2742084884","heartbeat_front_addr":"172.21.15.49:6831/2742084884","state":["exists","up"]},{"osd":10,"uuid":"3945e46d-766a-4c4a-8821-d43648dd1578","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":2404740201},{"type":"v1","addr":"172.21.15.139:6825","nonce":2404740201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":2404740201},{"type":"v1","addr":"172.21.15.139:6827","nonce":2404740201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":2404740201},{"type":"v1","addr":"172.21.15.139:6831","nonce":2404740201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":2404740201},{"type":"v1","addr":"172.21.15.139:6829","nonce":2404740201}]},"public_addr":"172.21.15.139:6825/2404740201","cluster_addr":"172.21.15.139:6827/2404740201","heartbeat_back_addr":"172.21.15.139:6831/2404740201","heartbeat_front_addr":"172.21.15.139:6829/2404740201","state":["exists","up"]},{"osd":11,"uuid":"9954ba15-d877-46cc-b24a-893f9168deb8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":3519503868},{"type":"v1","addr":"172.21.15.175:6825","nonce":3519503868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":3519503868},{"type":"v1","addr":"172.21.15.175:6827","nonce":3519503868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":3519503868},{"type":"v1","addr":"172.21.15.175:6831","nonce":3519503868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":3519503868},{"type":"v1","addr":"172.21.15.175:6829","nonce":3519503868}]},"public_addr":"172.21.15.175:6825/3519503868","cluster_addr":"172.21.15.175:6827/3519503868","heartbeat_back_addr":"172.21.15.175:6831/3519503868","heartbeat_front_addr":"172.21.15.175:6829/3519503868","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:00:38.367865+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:01:02.237687+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:01:25.609751+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:01:51.268572+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:02:16.514605+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:02:41.586523+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:03:07.029964+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:03:32.956250+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:03:57.547984+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:04:23.948972+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:04:49.667210+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:05:15.284693+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.6","mappings":[{"from":4,"to":6}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.49:0/1463571219":"2024-03-19T00:13:11.015881+0000","172.21.15.49:6800/2609458951":"2024-03-19T22:58:59.806253+0000","172.21.15.49:6801/1985190312":"2024-03-19T22:58:31.972474+0000","172.21.15.49:0/2176210039":"2024-03-19T22:58:31.972474+0000","172.21.15.49:0/1912446700":"2024-03-19T22:58:59.806253+0000","172.21.15.49:0/21283102":"2024-03-19T22:58:31.972474+0000","172.21.15.49:6801/2609458951":"2024-03-19T22:58:59.806253+0000","172.21.15.49:0/1036383534":"2024-03-19T22:58:59.806253+0000","172.21.15.49:0/563527416":"2024-03-19T22:58:59.806253+0000","172.21.15.49:6801/2083805211":"2024-03-19T23:08:15.312250+0000","172.21.15.49:0/1976299520":"2024-03-19T23:08:15.312250+0000","172.21.15.49:0/861926960":"2024-03-19T23:08:15.312250+0000","172.21.15.49:0/1292058922":"2024-03-19T22:58:31.972474+0000","172.21.15.49:0/94393705":"2024-03-19T23:08:15.312250+0000","172.21.15.49:6800/1985190312":"2024-03-19T22:58:31.972474+0000","172.21.15.49:6800/2083805211":"2024-03-19T23:08:15.312250+0000","172.21.15.49:0/3268662009":"2024-03-19T23:08:15.312250+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":6,"length":1},{"begin":17,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-18T23:19:43.612 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:43.612+0000 7fbde7a0b640 1 -- 172.21.15.49:0/949437498 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7fbdcc0a2970 msgr2=0x7fbdcc0a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:19:43.612 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:43.612+0000 7fbde7a0b640 1 --2- 172.21.15.49:0/949437498 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7fbdcc0a2970 0x7fbdcc0a4e30 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fbddc009b50 tx=0x7fbddc005f90 comp rx=0 tx=0).stop 2024-03-18T23:19:43.612 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:43.612+0000 7fbde7a0b640 1 -- 172.21.15.49:0/949437498 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbde01077f0 msgr2=0x7fbde019c960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:19:43.612 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:43.612+0000 7fbde7a0b640 1 --2- 172.21.15.49:0/949437498 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbde01077f0 0x7fbde019c960 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fbdd80348f0 tx=0x7fbdd8032c90 comp rx=0 tx=0).stop 2024-03-18T23:19:43.612 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:43.612+0000 7fbde7a0b640 1 -- 172.21.15.49:0/949437498 shutdown_connections 2024-03-18T23:19:43.613 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:43.612+0000 7fbde7a0b640 1 --2- 172.21.15.49:0/949437498 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7fbdcc0a2970 0x7fbdcc0a4e30 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:43.613 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:43.612+0000 7fbde7a0b640 1 --2- 172.21.15.49:0/949437498 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbde01077f0 0x7fbde019c960 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:43.613 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:43.612+0000 7fbde7a0b640 1 --2- 172.21.15.49:0/949437498 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbde01021a0 0x7fbde01985e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:43.613 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:43.612+0000 7fbde7a0b640 1 --2- 172.21.15.49:0/949437498 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbde01017e0 0x7fbde01980a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:43.613 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:43.612+0000 7fbde7a0b640 1 -- 172.21.15.49:0/949437498 >> 172.21.15.49:0/949437498 conn(0x7fbde00fd290 msgr2=0x7fbde00fd9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:19:43.613 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:43.612+0000 7fbde7a0b640 1 -- 172.21.15.49:0/949437498 shutdown_connections 2024-03-18T23:19:43.613 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:43.612+0000 7fbde7a0b640 1 -- 172.21.15.49:0/949437498 wait complete. 2024-03-18T23:19:43.623 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-18T23:01:28.363432+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-18T23:06:03.172963+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-18T23:06:03.973303+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '100', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 17, 'snap_epoch': 100, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-18T23:19:43.624 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-18T23:19:43.737 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:43.735+0000 7f284b53b640 1 -- 172.21.15.49:0/2959921102 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2844103380 msgr2=0x7f284410b860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:19:43.738 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:43.735+0000 7f284b53b640 1 --2- 172.21.15.49:0/2959921102 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2844103380 0x7f284410b860 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f2834009900 tx=0x7f283402bcd0 comp rx=0 tx=0).stop 2024-03-18T23:19:43.738 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:43.735+0000 7f284b53b640 1 -- 172.21.15.49:0/2959921102 shutdown_connections 2024-03-18T23:19:43.738 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:43.735+0000 7f284b53b640 1 --2- 172.21.15.49:0/2959921102 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f284410be30 0x7f284410e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:43.738 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:43.735+0000 7f284b53b640 1 --2- 172.21.15.49:0/2959921102 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2844103380 0x7f284410b860 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:43.738 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:43.735+0000 7f284b53b640 1 --2- 172.21.15.49:0/2959921102 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2844102a60 0x7f2844102e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:43.738 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:43.735+0000 7f284b53b640 1 -- 172.21.15.49:0/2959921102 >> 172.21.15.49:0/2959921102 conn(0x7f2844076aa0 msgr2=0x7f2844076eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:19:43.738 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:43.736+0000 7f284b53b640 1 -- 172.21.15.49:0/2959921102 shutdown_connections 2024-03-18T23:19:43.738 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:43.736+0000 7f284b53b640 1 -- 172.21.15.49:0/2959921102 wait complete. 2024-03-18T23:19:43.739 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:43.736+0000 7f284b53b640 1 Processor -- start 2024-03-18T23:19:43.739 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:43.736+0000 7f284b53b640 1 -- start start 2024-03-18T23:19:43.739 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:43.736+0000 7f284b53b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2844102a60 0x7f2844198100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:19:43.739 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:43.736+0000 7f284b53b640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2844103380 0x7f2844198640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:19:43.739 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:43.736+0000 7f284b53b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f284410be30 0x7f284419c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:19:43.739 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:43.736+0000 7f284b53b640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f2844111bc0 con 0x7f284410be30 2024-03-18T23:19:43.739 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:43.736+0000 7f284b53b640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f2844111a40 con 0x7f2844102a60 2024-03-18T23:19:43.739 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:43.736+0000 7f284b53b640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f2844111d40 con 0x7f2844103380 2024-03-18T23:19:43.740 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:43.736+0000 7f28492d7640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2844102a60 0x7f2844198100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:19:43.740 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:43.737+0000 7f28492d7640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2844102a60 0x7f2844198100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.49:60000/0 (socket says 172.21.15.49:60000) 2024-03-18T23:19:43.740 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:43.737+0000 7f2848ad6640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2844103380 0x7f2844198640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:19:43.740 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:43.737+0000 7f28492d7640 1 -- 172.21.15.49:0/1367795195 learned_addr learned my addr 172.21.15.49:0/1367795195 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:19:43.740 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:43.737+0000 7f2849ad8640 1 --2- 172.21.15.49:0/1367795195 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f284410be30 0x7f284419c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:19:43.740 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:43.737+0000 7f28492d7640 1 -- 172.21.15.49:0/1367795195 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2844103380 msgr2=0x7f2844198640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:19:43.740 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:43.737+0000 7f28492d7640 1 --2- 172.21.15.49:0/1367795195 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2844103380 0x7f2844198640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:43.740 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:43.737+0000 7f28492d7640 1 -- 172.21.15.49:0/1367795195 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f284410be30 msgr2=0x7f284419c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:19:43.740 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:43.737+0000 7f28492d7640 1 --2- 172.21.15.49:0/1367795195 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f284410be30 0x7f284419c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:43.741 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:43.737+0000 7f28492d7640 1 -- 172.21.15.49:0/1367795195 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f284419d0f0 con 0x7f2844102a60 2024-03-18T23:19:43.741 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:43.737+0000 7f2849ad8640 1 --2- 172.21.15.49:0/1367795195 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f284410be30 0x7f284419c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:19:43.741 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:43.737+0000 7f28492d7640 1 --2- 172.21.15.49:0/1367795195 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2844102a60 0x7f2844198100 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f284000e9f0 tx=0x7f284000edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:19:43.741 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:43.737+0000 7f283a7fc640 1 -- 172.21.15.49:0/1367795195 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f284000ccc0 con 0x7f2844102a60 2024-03-18T23:19:43.741 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:43.737+0000 7f284b53b640 1 -- 172.21.15.49:0/1367795195 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f284419d3e0 con 0x7f2844102a60 2024-03-18T23:19:43.741 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:43.737+0000 7f283a7fc640 1 -- 172.21.15.49:0/1367795195 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2840010910 con 0x7f2844102a60 2024-03-18T23:19:43.741 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:43.737+0000 7f283a7fc640 1 -- 172.21.15.49:0/1367795195 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28400188a0 con 0x7f2844102a60 2024-03-18T23:19:43.741 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:43.737+0000 7f284b53b640 1 -- 172.21.15.49:0/1367795195 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28441a4c70 con 0x7f2844102a60 2024-03-18T23:19:43.742 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:43.738+0000 7f284b53b640 1 -- 172.21.15.49:0/1367795195 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f284406a610 con 0x7f2844102a60 2024-03-18T23:19:43.745 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:43.739+0000 7f283a7fc640 1 -- 172.21.15.49:0/1367795195 <== mon.2 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 25) ==== 137421+0+0 (secure 0 0 0) 0x7f2840010ac0 con 0x7f2844102a60 2024-03-18T23:19:43.745 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:43.740+0000 7f283a7fc640 1 --2- 172.21.15.49:0/1367795195 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f28300a2970 0x7f28300a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:19:43.745 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:43.740+0000 7f2848ad6640 1 --2- 172.21.15.49:0/1367795195 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f28300a2970 0x7f28300a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:19:43.746 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:43.740+0000 7f283a7fc640 1 -- 172.21.15.49:0/1367795195 <== mon.2 v2:172.21.15.139:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8556+0+0 (secure 0 0 0) 0x7f28400cf2d0 con 0x7f2844102a60 2024-03-18T23:19:43.746 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:43.740+0000 7f2848ad6640 1 --2- 172.21.15.49:0/1367795195 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f28300a2970 0x7f28300a4e30 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f283402efd0 tx=0x7f283402cf90 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:19:43.746 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:43.745+0000 7f283a7fc640 1 -- 172.21.15.49:0/1367795195 <== mon.2 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f284009c010 con 0x7f2844102a60 2024-03-18T23:19:43.926 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:43.926+0000 7f284b53b640 1 -- 172.21.15.49:0/1367795195 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f2844106f50 con 0x7f2844102a60 2024-03-18T23:19:43.926 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:43.927+0000 7f283a7fc640 1 -- 172.21.15.49:0/1367795195 <== mon.2 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v101) ==== 93+0+10 (secure 0 0 0) 0x7f284009b910 con 0x7f2844102a60 2024-03-18T23:19:43.927 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 1 2024-03-18T23:19:43.929 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:43.929+0000 7f284b53b640 1 -- 172.21.15.49:0/1367795195 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f28300a2970 msgr2=0x7f28300a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:19:43.929 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:43.929+0000 7f284b53b640 1 --2- 172.21.15.49:0/1367795195 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f28300a2970 0x7f28300a4e30 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f283402efd0 tx=0x7f283402cf90 comp rx=0 tx=0).stop 2024-03-18T23:19:43.929 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:43.929+0000 7f284b53b640 1 -- 172.21.15.49:0/1367795195 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2844102a60 msgr2=0x7f2844198100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:19:43.929 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:43.929+0000 7f284b53b640 1 --2- 172.21.15.49:0/1367795195 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2844102a60 0x7f2844198100 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f284000e9f0 tx=0x7f284000edf0 comp rx=0 tx=0).stop 2024-03-18T23:19:43.930 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:43.929+0000 7f284b53b640 1 -- 172.21.15.49:0/1367795195 shutdown_connections 2024-03-18T23:19:43.930 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:43.929+0000 7f284b53b640 1 --2- 172.21.15.49:0/1367795195 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f28300a2970 0x7f28300a4e30 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:43.930 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:43.929+0000 7f284b53b640 1 --2- 172.21.15.49:0/1367795195 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f284410be30 0x7f284419c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:43.930 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:43.929+0000 7f284b53b640 1 --2- 172.21.15.49:0/1367795195 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2844103380 0x7f2844198640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:43.930 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:43.929+0000 7f284b53b640 1 --2- 172.21.15.49:0/1367795195 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2844102a60 0x7f2844198100 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:43.930 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:43.929+0000 7f284b53b640 1 -- 172.21.15.49:0/1367795195 >> 172.21.15.49:0/1367795195 conn(0x7f2844076aa0 msgr2=0x7f28440fd620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:19:43.930 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:43.929+0000 7f284b53b640 1 -- 172.21.15.49:0/1367795195 shutdown_connections 2024-03-18T23:19:43.930 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:43.929+0000 7f284b53b640 1 -- 172.21.15.49:0/1367795195 wait complete. 2024-03-18T23:19:43.940 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-18T23:19:44.053 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:44.051+0000 7f60fb2dc640 1 -- 172.21.15.49:0/4250179538 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f60f410be60 msgr2=0x7f60f410e2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:19:44.053 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:44.051+0000 7f60fb2dc640 1 --2- 172.21.15.49:0/4250179538 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f60f410be60 0x7f60f410e2d0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f60ec009ca0 tx=0x7f60ec02c180 comp rx=0 tx=0).stop 2024-03-18T23:19:44.053 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:44.051+0000 7f60fb2dc640 1 -- 172.21.15.49:0/4250179538 shutdown_connections 2024-03-18T23:19:44.054 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:44.051+0000 7f60fb2dc640 1 --2- 172.21.15.49:0/4250179538 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f60f410be60 0x7f60f410e2d0 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:44.054 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:44.051+0000 7f60fb2dc640 1 --2- 172.21.15.49:0/4250179538 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f60f41033b0 0x7f60f410b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:44.054 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:44.051+0000 7f60fb2dc640 1 --2- 172.21.15.49:0/4250179538 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f60f4102a90 0x7f60f4102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:44.054 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:44.051+0000 7f60fb2dc640 1 -- 172.21.15.49:0/4250179538 >> 172.21.15.49:0/4250179538 conn(0x7f60f4077880 msgr2=0x7f60f4077c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:19:44.054 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:44.051+0000 7f60fb2dc640 1 -- 172.21.15.49:0/4250179538 shutdown_connections 2024-03-18T23:19:44.054 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:44.051+0000 7f60fb2dc640 1 -- 172.21.15.49:0/4250179538 wait complete. 2024-03-18T23:19:44.054 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:44.052+0000 7f60fb2dc640 1 Processor -- start 2024-03-18T23:19:44.054 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:44.052+0000 7f60fb2dc640 1 -- start start 2024-03-18T23:19:44.055 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:44.052+0000 7f60fb2dc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f60f4102a90 0x7f60f41006e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:19:44.055 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:44.052+0000 7f60fb2dc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f60f41033b0 0x7f60f4100c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:19:44.055 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:44.052+0000 7f60fb2dc640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f60f410be60 0x7f60f4101160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:19:44.055 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:44.052+0000 7f60fb2dc640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f60f4111c30 con 0x7f60f41033b0 2024-03-18T23:19:44.055 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:44.052+0000 7f60fb2dc640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f60f4111ab0 con 0x7f60f4102a90 2024-03-18T23:19:44.055 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:44.052+0000 7f60fb2dc640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f60f4111db0 con 0x7f60f410be60 2024-03-18T23:19:44.055 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:44.052+0000 7f60f8877640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f60f41033b0 0x7f60f4100c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:19:44.055 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:44.052+0000 7f60f8877640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f60f41033b0 0x7f60f4100c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:40324/0 (socket says 172.21.15.49:40324) 2024-03-18T23:19:44.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:44.052+0000 7f60f8877640 1 -- 172.21.15.49:0/4034176111 learned_addr learned my addr 172.21.15.49:0/4034176111 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:19:44.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:44.052+0000 7f60f9078640 1 --2- 172.21.15.49:0/4034176111 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f60f4102a90 0x7f60f41006e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:19:44.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:44.052+0000 7f60f9879640 1 --2- 172.21.15.49:0/4034176111 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f60f410be60 0x7f60f4101160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:19:44.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:44.053+0000 7f60f8877640 1 -- 172.21.15.49:0/4034176111 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f60f410be60 msgr2=0x7f60f4101160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:19:44.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:44.053+0000 7f60f8877640 1 --2- 172.21.15.49:0/4034176111 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f60f410be60 0x7f60f4101160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:44.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:44.053+0000 7f60f8877640 1 -- 172.21.15.49:0/4034176111 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f60f4102a90 msgr2=0x7f60f41006e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:19:44.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:44.053+0000 7f60f8877640 1 --2- 172.21.15.49:0/4034176111 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f60f4102a90 0x7f60f41006e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:44.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:44.053+0000 7f60f8877640 1 -- 172.21.15.49:0/4034176111 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60f41a65b0 con 0x7f60f41033b0 2024-03-18T23:19:44.057 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:44.053+0000 7f60f8877640 1 --2- 172.21.15.49:0/4034176111 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f60f41033b0 0x7f60f4100c20 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f60e400e9f0 tx=0x7f60e400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:19:44.057 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:44.053+0000 7f60ea7fc640 1 -- 172.21.15.49:0/4034176111 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60e400cd80 con 0x7f60f41033b0 2024-03-18T23:19:44.057 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:44.053+0000 7f60ea7fc640 1 -- 172.21.15.49:0/4034176111 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f60e4005500 con 0x7f60f41033b0 2024-03-18T23:19:44.057 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:44.053+0000 7f60fb2dc640 1 -- 172.21.15.49:0/4034176111 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60f41a68a0 con 0x7f60f41033b0 2024-03-18T23:19:44.057 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:44.053+0000 7f60ea7fc640 1 -- 172.21.15.49:0/4034176111 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60e4010690 con 0x7f60f41033b0 2024-03-18T23:19:44.057 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:44.053+0000 7f60fb2dc640 1 -- 172.21.15.49:0/4034176111 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60f41a6df0 con 0x7f60f41033b0 2024-03-18T23:19:44.058 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:44.054+0000 7f60fb2dc640 1 -- 172.21.15.49:0/4034176111 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60f4068a60 con 0x7f60f41033b0 2024-03-18T23:19:44.058 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:44.055+0000 7f60ea7fc640 1 -- 172.21.15.49:0/4034176111 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 25) ==== 137421+0+0 (secure 0 0 0) 0x7f60e40046c0 con 0x7f60f41033b0 2024-03-18T23:19:44.058 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:44.056+0000 7f60ea7fc640 1 --2- 172.21.15.49:0/4034176111 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f60e00a29f0 0x7f60e00a4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:19:44.059 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:44.056+0000 7f60ea7fc640 1 -- 172.21.15.49:0/4034176111 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8556+0+0 (secure 0 0 0) 0x7f60e40cf100 con 0x7f60f41033b0 2024-03-18T23:19:44.059 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:44.056+0000 7f60f9078640 1 --2- 172.21.15.49:0/4034176111 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f60e00a29f0 0x7f60e00a4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:19:44.059 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:44.056+0000 7f60f9078640 1 --2- 172.21.15.49:0/4034176111 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f60e00a29f0 0x7f60e00a4eb0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f60f00095f0 tx=0x7f60f0005f90 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:19:44.060 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:44.060+0000 7f60ea7fc640 1 -- 172.21.15.49:0/4034176111 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f60e409bee0 con 0x7f60f41033b0 2024-03-18T23:19:44.247 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:44.247+0000 7f60fb2dc640 1 -- 172.21.15.49:0/4034176111 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f60f41a7050 con 0x7f60f41033b0 2024-03-18T23:19:44.247 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:44.247+0000 7f60ea7fc640 1 -- 172.21.15.49:0/4034176111 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v101) ==== 104+0+11 (secure 0 0 0) 0x7f60e409b7e0 con 0x7f60f41033b0 2024-03-18T23:19:44.247 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 32 2024-03-18T23:19:44.249 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:44.249+0000 7f60fb2dc640 1 -- 172.21.15.49:0/4034176111 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f60e00a29f0 msgr2=0x7f60e00a4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:19:44.249 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:44.249+0000 7f60fb2dc640 1 --2- 172.21.15.49:0/4034176111 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f60e00a29f0 0x7f60e00a4eb0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f60f00095f0 tx=0x7f60f0005f90 comp rx=0 tx=0).stop 2024-03-18T23:19:44.249 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:44.249+0000 7f60fb2dc640 1 -- 172.21.15.49:0/4034176111 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f60f41033b0 msgr2=0x7f60f4100c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:19:44.249 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:44.249+0000 7f60fb2dc640 1 --2- 172.21.15.49:0/4034176111 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f60f41033b0 0x7f60f4100c20 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f60e400e9f0 tx=0x7f60e400edf0 comp rx=0 tx=0).stop 2024-03-18T23:19:44.250 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:44.249+0000 7f60fb2dc640 1 -- 172.21.15.49:0/4034176111 shutdown_connections 2024-03-18T23:19:44.250 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:44.249+0000 7f60fb2dc640 1 --2- 172.21.15.49:0/4034176111 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f60e00a29f0 0x7f60e00a4eb0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:44.250 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:44.249+0000 7f60fb2dc640 1 --2- 172.21.15.49:0/4034176111 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f60f410be60 0x7f60f4101160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:44.250 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:44.249+0000 7f60fb2dc640 1 --2- 172.21.15.49:0/4034176111 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f60f41033b0 0x7f60f4100c20 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:44.250 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:44.249+0000 7f60fb2dc640 1 --2- 172.21.15.49:0/4034176111 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f60f4102a90 0x7f60f41006e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:44.250 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:44.249+0000 7f60fb2dc640 1 -- 172.21.15.49:0/4034176111 >> 172.21.15.49:0/4034176111 conn(0x7f60f4077880 msgr2=0x7f60f40fd730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:19:44.250 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:44.249+0000 7f60fb2dc640 1 -- 172.21.15.49:0/4034176111 shutdown_connections 2024-03-18T23:19:44.251 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:44.249+0000 7f60fb2dc640 1 -- 172.21.15.49:0/4034176111 wait complete. 2024-03-18T23:19:44.260 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-18T23:19:44.374 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:44.372+0000 7f9ea8ee8640 1 -- 172.21.15.49:0/2884547442 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9ea41077d0 msgr2=0x7f9ea4107bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:19:44.374 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:44.372+0000 7f9ea8ee8640 1 --2- 172.21.15.49:0/2884547442 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9ea41077d0 0x7f9ea4107bb0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f9e98009890 tx=0x7f9e9802bb70 comp rx=0 tx=0).stop 2024-03-18T23:19:44.374 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:44.372+0000 7f9ea8ee8640 1 -- 172.21.15.49:0/2884547442 shutdown_connections 2024-03-18T23:19:44.374 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:44.372+0000 7f9ea8ee8640 1 --2- 172.21.15.49:0/2884547442 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9ea4102180 0x7f9ea410e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:44.375 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:44.372+0000 7f9ea8ee8640 1 --2- 172.21.15.49:0/2884547442 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9ea41017c0 0x7f9ea4101c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:44.375 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:44.372+0000 7f9ea8ee8640 1 --2- 172.21.15.49:0/2884547442 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9ea41077d0 0x7f9ea4107bb0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:44.375 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:44.372+0000 7f9ea8ee8640 1 -- 172.21.15.49:0/2884547442 >> 172.21.15.49:0/2884547442 conn(0x7f9ea40fd250 msgr2=0x7f9ea40ff670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:19:44.375 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:44.372+0000 7f9ea8ee8640 1 -- 172.21.15.49:0/2884547442 shutdown_connections 2024-03-18T23:19:44.375 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:44.372+0000 7f9ea8ee8640 1 -- 172.21.15.49:0/2884547442 wait complete. 2024-03-18T23:19:44.375 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:44.373+0000 7f9ea8ee8640 1 Processor -- start 2024-03-18T23:19:44.375 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:44.373+0000 7f9ea8ee8640 1 -- start start 2024-03-18T23:19:44.375 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:44.373+0000 7f9ea8ee8640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9ea41017c0 0x7f9ea4073b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:19:44.376 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:44.373+0000 7f9ea8ee8640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9ea4102180 0x7f9ea4074060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:19:44.376 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:44.373+0000 7f9ea8ee8640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9ea41077d0 0x7f9ea40745a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:19:44.376 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:44.373+0000 7f9ea8ee8640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f9ea4111be0 con 0x7f9ea41077d0 2024-03-18T23:19:44.376 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:44.373+0000 7f9ea8ee8640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f9ea4111a60 con 0x7f9ea41017c0 2024-03-18T23:19:44.376 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:44.373+0000 7f9ea8ee8640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f9ea4111d60 con 0x7f9ea4102180 2024-03-18T23:19:44.376 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:44.373+0000 7f9ea2d9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9ea41077d0 0x7f9ea40745a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:19:44.376 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:44.373+0000 7f9ea2d9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9ea41077d0 0x7f9ea40745a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:40342/0 (socket says 172.21.15.49:40342) 2024-03-18T23:19:44.377 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:44.373+0000 7f9ea2d9d640 1 -- 172.21.15.49:0/4009718898 learned_addr learned my addr 172.21.15.49:0/4009718898 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:19:44.377 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:44.373+0000 7f9ea1d9b640 1 --2- 172.21.15.49:0/4009718898 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9ea4102180 0x7f9ea4074060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:19:44.377 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:44.373+0000 7f9ea259c640 1 --2- 172.21.15.49:0/4009718898 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9ea41017c0 0x7f9ea4073b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:19:44.377 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:44.373+0000 7f9ea2d9d640 1 -- 172.21.15.49:0/4009718898 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9ea4102180 msgr2=0x7f9ea4074060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:19:44.377 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:44.373+0000 7f9ea2d9d640 1 --2- 172.21.15.49:0/4009718898 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9ea4102180 0x7f9ea4074060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:44.377 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:44.373+0000 7f9ea2d9d640 1 -- 172.21.15.49:0/4009718898 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9ea41017c0 msgr2=0x7f9ea4073b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:19:44.377 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:44.373+0000 7f9ea2d9d640 1 --2- 172.21.15.49:0/4009718898 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9ea41017c0 0x7f9ea4073b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:44.377 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:44.373+0000 7f9ea2d9d640 1 -- 172.21.15.49:0/4009718898 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ea41a6680 con 0x7f9ea41077d0 2024-03-18T23:19:44.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:44.374+0000 7f9ea2d9d640 1 --2- 172.21.15.49:0/4009718898 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9ea41077d0 0x7f9ea40745a0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f9e9c00eb10 tx=0x7f9e9c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:19:44.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:44.374+0000 7f9e937fe640 1 -- 172.21.15.49:0/4009718898 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e9c004b20 con 0x7f9ea41077d0 2024-03-18T23:19:44.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:44.374+0000 7f9e937fe640 1 -- 172.21.15.49:0/4009718898 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e9c004cc0 con 0x7f9ea41077d0 2024-03-18T23:19:44.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:44.374+0000 7f9ea8ee8640 1 -- 172.21.15.49:0/4009718898 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ea41a6970 con 0x7f9ea41077d0 2024-03-18T23:19:44.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:44.374+0000 7f9e937fe640 1 -- 172.21.15.49:0/4009718898 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e9c010760 con 0x7f9ea41077d0 2024-03-18T23:19:44.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:44.374+0000 7f9ea8ee8640 1 -- 172.21.15.49:0/4009718898 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ea41a6ec0 con 0x7f9ea41077d0 2024-03-18T23:19:44.379 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:44.375+0000 7f9ea8ee8640 1 -- 172.21.15.49:0/4009718898 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ea4068a60 con 0x7f9ea41077d0 2024-03-18T23:19:44.379 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:44.377+0000 7f9e937fe640 1 -- 172.21.15.49:0/4009718898 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 25) ==== 137421+0+0 (secure 0 0 0) 0x7f9e9c005000 con 0x7f9ea41077d0 2024-03-18T23:19:44.379 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:44.377+0000 7f9e937fe640 1 --2- 172.21.15.49:0/4009718898 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f9e8c0a2970 0x7f9e8c0a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:19:44.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:44.377+0000 7f9ea259c640 1 --2- 172.21.15.49:0/4009718898 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f9e8c0a2970 0x7f9e8c0a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:19:44.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:44.377+0000 7f9e937fe640 1 -- 172.21.15.49:0/4009718898 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8556+0+0 (secure 0 0 0) 0x7f9e9c0cf830 con 0x7f9ea41077d0 2024-03-18T23:19:44.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:44.378+0000 7f9ea259c640 1 --2- 172.21.15.49:0/4009718898 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f9e8c0a2970 0x7f9e8c0a4e30 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f9e980097b0 tx=0x7f9e980058d0 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:19:44.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:44.381+0000 7f9e937fe640 1 -- 172.21.15.49:0/4009718898 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f9e9c09c570 con 0x7f9ea41077d0 2024-03-18T23:19:44.568 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:44.567+0000 7f9ea8ee8640 1 -- 172.21.15.49:0/4009718898 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f9ea4109d30 con 0x7f9ea41077d0 2024-03-18T23:19:44.568 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:44.568+0000 7f9e937fe640 1 -- 172.21.15.49:0/4009718898 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v101) ==== 100+0+11 (secure 0 0 0) 0x7f9e9c09be70 con 0x7f9ea41077d0 2024-03-18T23:19:44.568 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 64 2024-03-18T23:19:44.570 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:44.570+0000 7f9ea8ee8640 1 -- 172.21.15.49:0/4009718898 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f9e8c0a2970 msgr2=0x7f9e8c0a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:19:44.571 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:44.570+0000 7f9ea8ee8640 1 --2- 172.21.15.49:0/4009718898 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f9e8c0a2970 0x7f9e8c0a4e30 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f9e980097b0 tx=0x7f9e980058d0 comp rx=0 tx=0).stop 2024-03-18T23:19:44.571 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:44.570+0000 7f9ea8ee8640 1 -- 172.21.15.49:0/4009718898 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9ea41077d0 msgr2=0x7f9ea40745a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:19:44.571 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:44.570+0000 7f9ea8ee8640 1 --2- 172.21.15.49:0/4009718898 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9ea41077d0 0x7f9ea40745a0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f9e9c00eb10 tx=0x7f9e9c00c2d0 comp rx=0 tx=0).stop 2024-03-18T23:19:44.571 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:44.570+0000 7f9ea8ee8640 1 -- 172.21.15.49:0/4009718898 shutdown_connections 2024-03-18T23:19:44.571 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:44.570+0000 7f9ea8ee8640 1 --2- 172.21.15.49:0/4009718898 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f9e8c0a2970 0x7f9e8c0a4e30 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:44.571 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:44.570+0000 7f9ea8ee8640 1 --2- 172.21.15.49:0/4009718898 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9ea41077d0 0x7f9ea40745a0 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:44.571 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:44.570+0000 7f9ea8ee8640 1 --2- 172.21.15.49:0/4009718898 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9ea4102180 0x7f9ea4074060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:44.571 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:44.570+0000 7f9ea8ee8640 1 --2- 172.21.15.49:0/4009718898 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9ea41017c0 0x7f9ea4073b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:44.571 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:44.570+0000 7f9ea8ee8640 1 -- 172.21.15.49:0/4009718898 >> 172.21.15.49:0/4009718898 conn(0x7f9ea40fd250 msgr2=0x7f9ea40fdaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:19:44.572 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:44.570+0000 7f9ea8ee8640 1 -- 172.21.15.49:0/4009718898 shutdown_connections 2024-03-18T23:19:44.572 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:44.570+0000 7f9ea8ee8640 1 -- 172.21.15.49:0/4009718898 wait complete. 2024-03-18T23:19:44.581 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-18T23:19:44.693 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:44.691+0000 7fecd1662640 1 -- 172.21.15.49:0/1884146045 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feccc102180 msgr2=0x7feccc10e690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:19:44.693 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:44.691+0000 7fecd1662640 1 --2- 172.21.15.49:0/1884146045 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feccc102180 0x7feccc10e690 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fecc4009cd0 tx=0x7fecc402c0f0 comp rx=0 tx=0).stop 2024-03-18T23:19:44.694 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:44.691+0000 7fecd1662640 1 -- 172.21.15.49:0/1884146045 shutdown_connections 2024-03-18T23:19:44.694 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:44.691+0000 7fecd1662640 1 --2- 172.21.15.49:0/1884146045 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feccc102180 0x7feccc10e690 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:44.694 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:44.691+0000 7fecd1662640 1 --2- 172.21.15.49:0/1884146045 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feccc1017c0 0x7feccc101c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:44.694 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:44.691+0000 7fecd1662640 1 --2- 172.21.15.49:0/1884146045 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feccc1077d0 0x7feccc107bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:44.694 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:44.691+0000 7fecd1662640 1 -- 172.21.15.49:0/1884146045 >> 172.21.15.49:0/1884146045 conn(0x7feccc0fd250 msgr2=0x7feccc0ff670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:19:44.694 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:44.691+0000 7fecd1662640 1 -- 172.21.15.49:0/1884146045 shutdown_connections 2024-03-18T23:19:44.694 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:44.691+0000 7fecd1662640 1 -- 172.21.15.49:0/1884146045 wait complete. 2024-03-18T23:19:44.694 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:44.692+0000 7fecd1662640 1 Processor -- start 2024-03-18T23:19:44.695 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:44.692+0000 7fecd1662640 1 -- start start 2024-03-18T23:19:44.695 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:44.692+0000 7fecd1662640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feccc1017c0 0x7feccc198140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:19:44.695 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:44.692+0000 7fecd1662640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feccc102180 0x7feccc198680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:19:44.695 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:44.692+0000 7fecd1662640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feccc1077d0 0x7feccc19c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:19:44.695 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:44.692+0000 7fecd1662640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7feccc111be0 con 0x7feccc102180 2024-03-18T23:19:44.695 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:44.692+0000 7fecd1662640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7feccc111a60 con 0x7feccc1077d0 2024-03-18T23:19:44.695 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:44.692+0000 7fecd1662640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7feccc111d60 con 0x7feccc1017c0 2024-03-18T23:19:44.695 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:44.692+0000 7fecca7fc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feccc102180 0x7feccc198680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:19:44.696 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:44.692+0000 7fecca7fc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feccc102180 0x7feccc198680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:40988/0 (socket says 172.21.15.49:40988) 2024-03-18T23:19:44.696 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:44.692+0000 7fecca7fc640 1 -- 172.21.15.49:0/1849178075 learned_addr learned my addr 172.21.15.49:0/1849178075 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:19:44.696 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:44.692+0000 7feccaffd640 1 --2- 172.21.15.49:0/1849178075 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feccc1017c0 0x7feccc198140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:19:44.696 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:44.692+0000 7feccb7fe640 1 --2- 172.21.15.49:0/1849178075 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feccc1077d0 0x7feccc19c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:19:44.696 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:44.693+0000 7fecca7fc640 1 -- 172.21.15.49:0/1849178075 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feccc1017c0 msgr2=0x7feccc198140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:19:44.696 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:44.693+0000 7fecca7fc640 1 --2- 172.21.15.49:0/1849178075 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feccc1017c0 0x7feccc198140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:44.696 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:44.693+0000 7fecca7fc640 1 -- 172.21.15.49:0/1849178075 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feccc1077d0 msgr2=0x7feccc19c9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:19:44.697 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:44.693+0000 7fecca7fc640 1 --2- 172.21.15.49:0/1849178075 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feccc1077d0 0x7feccc19c9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:44.697 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:44.693+0000 7fecca7fc640 1 -- 172.21.15.49:0/1849178075 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feccc19d0e0 con 0x7feccc102180 2024-03-18T23:19:44.697 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:44.693+0000 7fecca7fc640 1 --2- 172.21.15.49:0/1849178075 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feccc102180 0x7feccc198680 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fecc000e9f0 tx=0x7fecc000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:19:44.697 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:44.693+0000 7fecb3fff640 1 -- 172.21.15.49:0/1849178075 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecc000cd80 con 0x7feccc102180 2024-03-18T23:19:44.697 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:44.693+0000 7fecd1662640 1 -- 172.21.15.49:0/1849178075 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feccc19d3d0 con 0x7feccc102180 2024-03-18T23:19:44.697 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:44.693+0000 7fecb3fff640 1 -- 172.21.15.49:0/1849178075 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fecc0010910 con 0x7feccc102180 2024-03-18T23:19:44.697 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:44.693+0000 7fecb3fff640 1 -- 172.21.15.49:0/1849178075 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecc00188f0 con 0x7feccc102180 2024-03-18T23:19:44.697 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:44.693+0000 7fecd1662640 1 -- 172.21.15.49:0/1849178075 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feccc1a4c60 con 0x7feccc102180 2024-03-18T23:19:44.698 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:44.694+0000 7fecd1662640 1 -- 172.21.15.49:0/1849178075 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feccc068a60 con 0x7feccc102180 2024-03-18T23:19:44.699 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:44.695+0000 7fecb3fff640 1 -- 172.21.15.49:0/1849178075 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 25) ==== 137421+0+0 (secure 0 0 0) 0x7fecc0010ac0 con 0x7feccc102180 2024-03-18T23:19:44.699 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:44.696+0000 7fecb3fff640 1 --2- 172.21.15.49:0/1849178075 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7fecb40a2970 0x7fecb40a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:19:44.699 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:44.696+0000 7feccaffd640 1 --2- 172.21.15.49:0/1849178075 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7fecb40a2970 0x7fecb40a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:19:44.699 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:44.696+0000 7fecb3fff640 1 -- 172.21.15.49:0/1849178075 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8556+0+0 (secure 0 0 0) 0x7fecc00cf600 con 0x7feccc102180 2024-03-18T23:19:44.699 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:44.696+0000 7feccaffd640 1 --2- 172.21.15.49:0/1849178075 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7fecb40a2970 0x7fecb40a4e30 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fecbc009b50 tx=0x7fecbc005f90 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:19:44.699 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:44.699+0000 7fecb3fff640 1 -- 172.21.15.49:0/1849178075 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fecc009bcf0 con 0x7feccc102180 2024-03-18T23:19:44.814 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:19:44 smithi139 ceph-mon[67282]: pgmap v359: 97 pgs: 97 active+clean; 61 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 225 KiB/s wr, 32 op/s 2024-03-18T23:19:44.814 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:19:44 smithi139 ceph-mon[67282]: from='client.? 172.21.15.49:0/949437498' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T23:19:44.814 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:19:44 smithi139 ceph-mon[67282]: from='client.? 172.21.15.49:0/1367795195' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-18T23:19:44.814 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:19:44 smithi139 ceph-mon[67282]: from='client.? 172.21.15.49:0/4034176111' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-18T23:19:44.878 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:19:44 smithi049 ceph-mon[62991]: pgmap v359: 97 pgs: 97 active+clean; 61 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 225 KiB/s wr, 32 op/s 2024-03-18T23:19:44.879 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:19:44 smithi049 ceph-mon[62991]: from='client.? 172.21.15.49:0/949437498' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T23:19:44.879 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:19:44 smithi049 ceph-mon[62991]: from='client.? 172.21.15.49:0/1367795195' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-18T23:19:44.879 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:19:44 smithi049 ceph-mon[62991]: from='client.? 172.21.15.49:0/4034176111' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-18T23:19:44.894 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:44.894+0000 7fecd1662640 1 -- 172.21.15.49:0/1849178075 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7feccc199440 con 0x7feccc102180 2024-03-18T23:19:44.894 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:44.895+0000 7fecb3fff640 1 -- 172.21.15.49:0/1849178075 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 17 entries v101) ==== 81+0+998 (secure 0 0 0) 0x7feccc199440 con 0x7feccc102180 2024-03-18T23:19:44.895 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.15.49:0/1463571219 2024-03-19T00:13:11.015881+0000 2024-03-18T23:19:44.895 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.15.49:6800/2609458951 2024-03-19T22:58:59.806253+0000 2024-03-18T23:19:44.895 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.15.49:6801/1985190312 2024-03-19T22:58:31.972474+0000 2024-03-18T23:19:44.895 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.15.49:0/2176210039 2024-03-19T22:58:31.972474+0000 2024-03-18T23:19:44.895 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.15.49:0/1912446700 2024-03-19T22:58:59.806253+0000 2024-03-18T23:19:44.896 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.15.49:0/21283102 2024-03-19T22:58:31.972474+0000 2024-03-18T23:19:44.896 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.15.49:6801/2609458951 2024-03-19T22:58:59.806253+0000 2024-03-18T23:19:44.896 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.15.49:0/1036383534 2024-03-19T22:58:59.806253+0000 2024-03-18T23:19:44.896 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.15.49:0/563527416 2024-03-19T22:58:59.806253+0000 2024-03-18T23:19:44.896 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.15.49:6801/2083805211 2024-03-19T23:08:15.312250+0000 2024-03-18T23:19:44.896 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.15.49:0/1976299520 2024-03-19T23:08:15.312250+0000 2024-03-18T23:19:44.896 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.15.49:0/861926960 2024-03-19T23:08:15.312250+0000 2024-03-18T23:19:44.896 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.15.49:0/1292058922 2024-03-19T22:58:31.972474+0000 2024-03-18T23:19:44.897 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.15.49:0/94393705 2024-03-19T23:08:15.312250+0000 2024-03-18T23:19:44.897 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.15.49:6800/1985190312 2024-03-19T22:58:31.972474+0000 2024-03-18T23:19:44.897 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.15.49:6800/2083805211 2024-03-19T23:08:15.312250+0000 2024-03-18T23:19:44.897 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.15.49:0/3268662009 2024-03-19T23:08:15.312250+0000 2024-03-18T23:19:44.898 INFO:teuthology.orchestra.run.smithi049.stderr:listed 17 entries 2024-03-18T23:19:44.898 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:44.897+0000 7fecd1662640 1 -- 172.21.15.49:0/1849178075 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7fecb40a2970 msgr2=0x7fecb40a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:19:44.898 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:44.897+0000 7fecd1662640 1 --2- 172.21.15.49:0/1849178075 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7fecb40a2970 0x7fecb40a4e30 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fecbc009b50 tx=0x7fecbc005f90 comp rx=0 tx=0).stop 2024-03-18T23:19:44.898 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:44.897+0000 7fecd1662640 1 -- 172.21.15.49:0/1849178075 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feccc102180 msgr2=0x7feccc198680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:19:44.898 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:44.897+0000 7fecd1662640 1 --2- 172.21.15.49:0/1849178075 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feccc102180 0x7feccc198680 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fecc000e9f0 tx=0x7fecc000eeb0 comp rx=0 tx=0).stop 2024-03-18T23:19:44.898 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:44.897+0000 7fecd1662640 1 -- 172.21.15.49:0/1849178075 shutdown_connections 2024-03-18T23:19:44.899 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:44.897+0000 7fecd1662640 1 --2- 172.21.15.49:0/1849178075 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7fecb40a2970 0x7fecb40a4e30 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:44.899 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:44.897+0000 7fecd1662640 1 --2- 172.21.15.49:0/1849178075 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feccc1077d0 0x7feccc19c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:44.899 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:44.897+0000 7fecd1662640 1 --2- 172.21.15.49:0/1849178075 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feccc102180 0x7feccc198680 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:44.899 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:44.897+0000 7fecd1662640 1 --2- 172.21.15.49:0/1849178075 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feccc1017c0 0x7feccc198140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:44.899 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:44.897+0000 7fecd1662640 1 -- 172.21.15.49:0/1849178075 >> 172.21.15.49:0/1849178075 conn(0x7feccc0fd250 msgr2=0x7feccc0fd910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:19:44.899 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:44.897+0000 7fecd1662640 1 -- 172.21.15.49:0/1849178075 shutdown_connections 2024-03-18T23:19:44.899 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:44.897+0000 7fecd1662640 1 -- 172.21.15.49:0/1849178075 wait complete. 2024-03-18T23:19:44.908 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-03-18T23:19:44.908 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-18T23:19:44.908 DEBUG:teuthology.orchestra.run.smithi049:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-03-18T23:19:44.960 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:19:44 smithi175 ceph-mon[67352]: pgmap v359: 97 pgs: 97 active+clean; 61 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 225 KiB/s wr, 32 op/s 2024-03-18T23:19:44.960 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:19:44 smithi175 ceph-mon[67352]: from='client.? 172.21.15.49:0/949437498' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T23:19:44.960 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:19:44 smithi175 ceph-mon[67352]: from='client.? 172.21.15.49:0/1367795195' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-18T23:19:44.960 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:19:44 smithi175 ceph-mon[67352]: from='client.? 172.21.15.49:0/4034176111' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-18T23:19:45.027 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi049.front.sepia.ceph.com 2024-03-18T23:19:45.027 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-18T23:19:45.027 DEBUG:teuthology.orchestra.run.smithi049:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-03-18T23:19:45.079 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-18T23:19:45.080 DEBUG:teuthology.orchestra.run.smithi049:> ip netns list 2024-03-18T23:19:45.120 INFO:teuthology.orchestra.run.smithi049.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-03-18T23:19:45.121 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-18T23:19:45.121 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-18T23:19:45.196 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-18T23:19:45.196 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip link delete ceph-brx 2024-03-18T23:19:45.302 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-03-18T23:19:45.312 INFO:tasks.cephadm:Teardown begin 2024-03-18T23:19:45.312 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-03-18T23:19:45.313 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-18T23:19:45.424 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:45.422+0000 7f46bad22640 1 -- 172.21.15.49:0/3822423353 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f46b41010a0 msgr2=0x7f46b410d5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:19:45.425 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:45.422+0000 7f46bad22640 1 --2- 172.21.15.49:0/3822423353 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f46b41010a0 0x7f46b410d5b0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f46ac009cd0 tx=0x7f46ac02c0f0 comp rx=0 tx=0).stop 2024-03-18T23:19:45.425 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:45.422+0000 7f46bad22640 1 -- 172.21.15.49:0/3822423353 shutdown_connections 2024-03-18T23:19:45.425 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:45.422+0000 7f46bad22640 1 --2- 172.21.15.49:0/3822423353 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f46b41010a0 0x7f46b410d5b0 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:45.425 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:45.422+0000 7f46bad22640 1 --2- 172.21.15.49:0/3822423353 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f46b41006e0 0x7f46b4100b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:45.425 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:45.422+0000 7f46bad22640 1 --2- 172.21.15.49:0/3822423353 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f46b41066f0 0x7f46b4106ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:45.425 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:45.422+0000 7f46bad22640 1 -- 172.21.15.49:0/3822423353 >> 172.21.15.49:0/3822423353 conn(0x7f46b40777c0 msgr2=0x7f46b4077bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:19:45.425 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:45.422+0000 7f46bad22640 1 -- 172.21.15.49:0/3822423353 shutdown_connections 2024-03-18T23:19:45.425 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:45.422+0000 7f46bad22640 1 -- 172.21.15.49:0/3822423353 wait complete. 2024-03-18T23:19:45.426 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:45.423+0000 7f46bad22640 1 Processor -- start 2024-03-18T23:19:45.426 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:45.423+0000 7f46bad22640 1 -- start start 2024-03-18T23:19:45.426 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:45.423+0000 7f46bad22640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f46b41006e0 0x7f46b4198080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:19:45.426 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:45.423+0000 7f46bad22640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f46b41010a0 0x7f46b41985c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:19:45.426 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:45.423+0000 7f46bad22640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f46b41066f0 0x7f46b419c940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:19:45.426 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:45.423+0000 7f46bad22640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f46b4110b00 con 0x7f46b41006e0 2024-03-18T23:19:45.426 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:45.423+0000 7f46bad22640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f46b4110980 con 0x7f46b41066f0 2024-03-18T23:19:45.426 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:45.423+0000 7f46bad22640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f46b4110c80 con 0x7f46b41010a0 2024-03-18T23:19:45.426 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:45.423+0000 7f46b8abe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f46b41006e0 0x7f46b4198080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:19:45.427 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:45.423+0000 7f46b8abe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f46b41006e0 0x7f46b4198080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:41010/0 (socket says 172.21.15.49:41010) 2024-03-18T23:19:45.427 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:45.423+0000 7f46b8abe640 1 -- 172.21.15.49:0/2024149939 learned_addr learned my addr 172.21.15.49:0/2024149939 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:19:45.427 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:45.423+0000 7f46abfff640 1 --2- 172.21.15.49:0/2024149939 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f46b41010a0 0x7f46b41985c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:19:45.427 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:45.423+0000 7f46b92bf640 1 --2- 172.21.15.49:0/2024149939 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f46b41066f0 0x7f46b419c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:19:45.427 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:45.423+0000 7f46b8abe640 1 -- 172.21.15.49:0/2024149939 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f46b41010a0 msgr2=0x7f46b41985c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:19:45.427 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:45.423+0000 7f46b8abe640 1 --2- 172.21.15.49:0/2024149939 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f46b41010a0 0x7f46b41985c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:45.427 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:45.424+0000 7f46b8abe640 1 -- 172.21.15.49:0/2024149939 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f46b41066f0 msgr2=0x7f46b419c940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:19:45.427 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:45.424+0000 7f46b8abe640 1 --2- 172.21.15.49:0/2024149939 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f46b41066f0 0x7f46b419c940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:45.428 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:45.424+0000 7f46b8abe640 1 -- 172.21.15.49:0/2024149939 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46b419d070 con 0x7f46b41006e0 2024-03-18T23:19:45.428 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:45.424+0000 7f46b8abe640 1 --2- 172.21.15.49:0/2024149939 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f46b41006e0 0x7f46b4198080 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f46b000e9f0 tx=0x7f46b000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:19:45.428 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:45.424+0000 7f46a9ffb640 1 -- 172.21.15.49:0/2024149939 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46b000cd80 con 0x7f46b41006e0 2024-03-18T23:19:45.428 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:45.424+0000 7f46bad22640 1 -- 172.21.15.49:0/2024149939 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46b419d360 con 0x7f46b41006e0 2024-03-18T23:19:45.428 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:45.424+0000 7f46a9ffb640 1 -- 172.21.15.49:0/2024149939 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f46b0010910 con 0x7f46b41006e0 2024-03-18T23:19:45.428 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:45.424+0000 7f46a9ffb640 1 -- 172.21.15.49:0/2024149939 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46b00188f0 con 0x7f46b41006e0 2024-03-18T23:19:45.428 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:45.424+0000 7f46bad22640 1 -- 172.21.15.49:0/2024149939 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46b41a4bf0 con 0x7f46b41006e0 2024-03-18T23:19:45.429 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:45.425+0000 7f46bad22640 1 -- 172.21.15.49:0/2024149939 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46b4068a60 con 0x7f46b41006e0 2024-03-18T23:19:45.429 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:45.426+0000 7f46a9ffb640 1 -- 172.21.15.49:0/2024149939 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 25) ==== 137421+0+0 (secure 0 0 0) 0x7f46b0010ac0 con 0x7f46b41006e0 2024-03-18T23:19:45.429 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:45.427+0000 7f46a9ffb640 1 --2- 172.21.15.49:0/2024149939 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f469c0a2970 0x7f469c0a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:19:45.429 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:45.427+0000 7f46abfff640 1 --2- 172.21.15.49:0/2024149939 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f469c0a2970 0x7f469c0a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:19:45.430 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:45.427+0000 7f46a9ffb640 1 -- 172.21.15.49:0/2024149939 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8556+0+0 (secure 0 0 0) 0x7f46b00cf190 con 0x7f46b41006e0 2024-03-18T23:19:45.430 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:45.428+0000 7f46abfff640 1 --2- 172.21.15.49:0/2024149939 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f469c0a2970 0x7f469c0a4e30 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f46b4077e30 tx=0x7f46a4005f90 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:19:45.431 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:45.432+0000 7f46a9ffb640 1 -- 172.21.15.49:0/2024149939 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f46b009b880 con 0x7f46b41006e0 2024-03-18T23:19:45.656 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:45.656+0000 7f46bad22640 1 -- 172.21.15.49:0/2024149939 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f46b41993d0 con 0x7f46b41006e0 2024-03-18T23:19:45.656 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:45.657+0000 7f46a9ffb640 1 -- 172.21.15.49:0/2024149939 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11936 (secure 0 0 0) 0x7f46b009f920 con 0x7f46b41006e0 2024-03-18T23:19:45.657 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-18T23:19:45.657 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6835/3631301630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":3631301630},{"type":"v1","addr":"172.21.15.49:6835","nonce":3631301630}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6837/444305233","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":444305233},{"type":"v1","addr":"172.21.15.49:6837","nonce":444305233}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":32,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-18T23:06:04.575999+0000","modified":"2024-03-18T23:11:38.156159+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24488,"mds_1":24463,"mds_2":24430,"mds_3":24419,"mds_4":24391},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14622":{"gid":14622,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.49:6839/1934085093","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6838","nonce":1934085093},{"type":"v1","addr":"172.21.15.49:6839","nonce":1934085093}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14634":{"gid":14634,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.49:6841/2869298862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6840","nonce":2869298862},{"type":"v1","addr":"172.21.15.49:6841","nonce":2869298862}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24389":{"gid":24389,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6833/3766178603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3766178603},{"type":"v1","addr":"172.21.15.139:6833","nonce":3766178603}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24391":{"gid":24391,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.175:6833/1702997173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":1702997173},{"type":"v1","addr":"172.21.15.175:6833","nonce":1702997173}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":9,"addr":"172.21.15.139:6835/1663212937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":1663212937},{"type":"v1","addr":"172.21.15.139:6835","nonce":1663212937}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.175:6835/755474931","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":755474931},{"type":"v1","addr":"172.21.15.175:6835","nonce":755474931}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6837/379482756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":379482756},{"type":"v1","addr":"172.21.15.175:6837","nonce":379482756}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"172.21.15.175:6839/3640882084","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":3640882084},{"type":"v1","addr":"172.21.15.175:6839","nonce":3640882084}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.139:6839/4280128209","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6838","nonce":4280128209},{"type":"v1","addr":"172.21.15.139:6839","nonce":4280128209}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25553":{"gid":25553,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6837/2891095267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6836","nonce":2891095267},{"type":"v1","addr":"172.21.15.139:6837","nonce":2891095267}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24391,24419,24430,24463,24488]},"id":1}]} 2024-03-18T23:19:45.657 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 32 2024-03-18T23:19:45.659 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:45.659+0000 7f46bad22640 1 -- 172.21.15.49:0/2024149939 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f469c0a2970 msgr2=0x7f469c0a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:19:45.659 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:45.659+0000 7f46bad22640 1 --2- 172.21.15.49:0/2024149939 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f469c0a2970 0x7f469c0a4e30 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f46b4077e30 tx=0x7f46a4005f90 comp rx=0 tx=0).stop 2024-03-18T23:19:45.659 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:45.659+0000 7f46bad22640 1 -- 172.21.15.49:0/2024149939 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f46b41006e0 msgr2=0x7f46b4198080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:19:45.659 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:45.659+0000 7f46bad22640 1 --2- 172.21.15.49:0/2024149939 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f46b41006e0 0x7f46b4198080 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f46b000e9f0 tx=0x7f46b000eeb0 comp rx=0 tx=0).stop 2024-03-18T23:19:45.659 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:45.659+0000 7f46bad22640 1 -- 172.21.15.49:0/2024149939 shutdown_connections 2024-03-18T23:19:45.659 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:45.659+0000 7f46bad22640 1 --2- 172.21.15.49:0/2024149939 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f469c0a2970 0x7f469c0a4e30 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:45.660 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:45.659+0000 7f46bad22640 1 --2- 172.21.15.49:0/2024149939 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f46b41066f0 0x7f46b419c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:45.660 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:45.659+0000 7f46bad22640 1 --2- 172.21.15.49:0/2024149939 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f46b41010a0 0x7f46b41985c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:45.660 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:45.659+0000 7f46bad22640 1 --2- 172.21.15.49:0/2024149939 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f46b41006e0 0x7f46b4198080 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:45.660 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:45.659+0000 7f46bad22640 1 -- 172.21.15.49:0/2024149939 >> 172.21.15.49:0/2024149939 conn(0x7f46b40777c0 msgr2=0x7f46b406a0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:19:45.660 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:45.659+0000 7f46bad22640 1 -- 172.21.15.49:0/2024149939 shutdown_connections 2024-03-18T23:19:45.660 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:45.659+0000 7f46bad22640 1 -- 172.21.15.49:0/2024149939 wait complete. 2024-03-18T23:19:45.670 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-18T23:19:45.781 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:45.780+0000 7f9e4f43d640 1 -- 172.21.15.49:0/751353886 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9e4806a870 msgr2=0x7f9e4810e670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:19:45.782 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:45.780+0000 7f9e4f43d640 1 --2- 172.21.15.49:0/751353886 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9e4806a870 0x7f9e4810e670 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f9e40009ca0 tx=0x7f9e4002c180 comp rx=0 tx=0).stop 2024-03-18T23:19:45.782 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:45.780+0000 7f9e4f43d640 1 -- 172.21.15.49:0/751353886 shutdown_connections 2024-03-18T23:19:45.782 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:45.780+0000 7f9e4f43d640 1 --2- 172.21.15.49:0/751353886 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9e4806a870 0x7f9e4810e670 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:45.782 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:45.780+0000 7f9e4f43d640 1 --2- 172.21.15.49:0/751353886 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9e48069ed0 0x7f9e4806a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:45.782 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:45.780+0000 7f9e4f43d640 1 --2- 172.21.15.49:0/751353886 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9e48107820 0x7f9e48107c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:45.782 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:45.780+0000 7f9e4f43d640 1 -- 172.21.15.49:0/751353886 >> 172.21.15.49:0/751353886 conn(0x7f9e480777c0 msgr2=0x7f9e48077bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:19:45.782 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:45.780+0000 7f9e4f43d640 1 -- 172.21.15.49:0/751353886 shutdown_connections 2024-03-18T23:19:45.782 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:45.780+0000 7f9e4f43d640 1 -- 172.21.15.49:0/751353886 wait complete. 2024-03-18T23:19:45.783 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:45.781+0000 7f9e4f43d640 1 Processor -- start 2024-03-18T23:19:45.783 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:45.781+0000 7f9e4f43d640 1 -- start start 2024-03-18T23:19:45.783 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:45.781+0000 7f9e4f43d640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9e48069ed0 0x7f9e48198000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:19:45.783 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:45.781+0000 7f9e4f43d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9e4806a870 0x7f9e48198540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:19:45.783 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:45.781+0000 7f9e4f43d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9e48107820 0x7f9e4819c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:19:45.783 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:45.781+0000 7f9e4f43d640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f9e48111be0 con 0x7f9e4806a870 2024-03-18T23:19:45.783 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:45.781+0000 7f9e4f43d640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f9e48111a60 con 0x7f9e48107820 2024-03-18T23:19:45.783 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:45.781+0000 7f9e4f43d640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f9e48111d60 con 0x7f9e48069ed0 2024-03-18T23:19:45.783 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:45.781+0000 7f9e4c9d8640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9e4806a870 0x7f9e48198540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:19:45.783 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:45.781+0000 7f9e4d1d9640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9e48069ed0 0x7f9e48198000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:19:45.784 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:45.781+0000 7f9e4c9d8640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9e4806a870 0x7f9e48198540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:41014/0 (socket says 172.21.15.49:41014) 2024-03-18T23:19:45.784 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:45.781+0000 7f9e4d1d9640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9e48069ed0 0x7f9e48198000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.49:33780/0 (socket says 172.21.15.49:33780) 2024-03-18T23:19:45.784 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:45.781+0000 7f9e4c9d8640 1 -- 172.21.15.49:0/3821319232 learned_addr learned my addr 172.21.15.49:0/3821319232 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:19:45.784 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:45.781+0000 7f9e4d9da640 1 --2- 172.21.15.49:0/3821319232 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9e48107820 0x7f9e4819c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:19:45.784 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:45.781+0000 7f9e4c9d8640 1 -- 172.21.15.49:0/3821319232 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9e48069ed0 msgr2=0x7f9e48198000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:19:45.784 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:45.781+0000 7f9e4c9d8640 1 --2- 172.21.15.49:0/3821319232 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9e48069ed0 0x7f9e48198000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:45.784 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:45.781+0000 7f9e4c9d8640 1 -- 172.21.15.49:0/3821319232 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9e48107820 msgr2=0x7f9e4819c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:19:45.784 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:45.781+0000 7f9e4c9d8640 1 --2- 172.21.15.49:0/3821319232 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9e48107820 0x7f9e4819c8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:45.784 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:45.781+0000 7f9e4c9d8640 1 -- 172.21.15.49:0/3821319232 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e4819cff0 con 0x7f9e4806a870 2024-03-18T23:19:45.785 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:45.782+0000 7f9e4c9d8640 1 --2- 172.21.15.49:0/3821319232 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9e4806a870 0x7f9e48198540 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f9e3800e9f0 tx=0x7f9e3800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:19:45.785 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:45.782+0000 7f9e3e7fc640 1 -- 172.21.15.49:0/3821319232 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e3800cd80 con 0x7f9e4806a870 2024-03-18T23:19:45.785 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:45.782+0000 7f9e3e7fc640 1 -- 172.21.15.49:0/3821319232 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e38005500 con 0x7f9e4806a870 2024-03-18T23:19:45.785 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:45.782+0000 7f9e4f43d640 1 -- 172.21.15.49:0/3821319232 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e4819d2e0 con 0x7f9e4806a870 2024-03-18T23:19:45.785 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:45.782+0000 7f9e3e7fc640 1 -- 172.21.15.49:0/3821319232 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e38010690 con 0x7f9e4806a870 2024-03-18T23:19:45.785 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:45.782+0000 7f9e4f43d640 1 -- 172.21.15.49:0/3821319232 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e481a4b70 con 0x7f9e4806a870 2024-03-18T23:19:45.786 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:45.783+0000 7f9e4f43d640 1 -- 172.21.15.49:0/3821319232 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e48102f40 con 0x7f9e4806a870 2024-03-18T23:19:45.786 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:45.784+0000 7f9e3e7fc640 1 -- 172.21.15.49:0/3821319232 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 25) ==== 137421+0+0 (secure 0 0 0) 0x7f9e380046c0 con 0x7f9e4806a870 2024-03-18T23:19:45.786 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:45.785+0000 7f9e3e7fc640 1 --2- 172.21.15.49:0/3821319232 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f9e340a2970 0x7f9e340a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:19:45.786 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:45.785+0000 7f9e4d1d9640 1 --2- 172.21.15.49:0/3821319232 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f9e340a2970 0x7f9e340a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:19:45.786 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:45.785+0000 7f9e3e7fc640 1 -- 172.21.15.49:0/3821319232 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8556+0+0 (secure 0 0 0) 0x7f9e380cf270 con 0x7f9e4806a870 2024-03-18T23:19:45.787 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:45.785+0000 7f9e4d1d9640 1 --2- 172.21.15.49:0/3821319232 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f9e340a2970 0x7f9e340a4e30 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f9e44009b50 tx=0x7f9e44005f90 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:19:45.788 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:45.789+0000 7f9e3e7fc640 1 -- 172.21.15.49:0/3821319232 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f9e3809bfb0 con 0x7f9e4806a870 2024-03-18T23:19:45.814 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:19:45 smithi139 ceph-mon[67282]: from='client.? 172.21.15.49:0/4009718898' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-18T23:19:45.814 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:19:45 smithi139 ceph-mon[67282]: from='client.? 172.21.15.49:0/1849178075' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-18T23:19:45.878 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:19:45 smithi049 ceph-mon[62991]: from='client.? 172.21.15.49:0/4009718898' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-18T23:19:45.879 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:19:45 smithi049 ceph-mon[62991]: from='client.? 172.21.15.49:0/1849178075' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-18T23:19:45.960 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:19:45 smithi175 ceph-mon[67352]: from='client.? 172.21.15.49:0/4009718898' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-18T23:19:45.960 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:19:45 smithi175 ceph-mon[67352]: from='client.? 172.21.15.49:0/1849178075' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-18T23:19:46.016 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:46.018+0000 7f9e4f43d640 1 -- 172.21.15.49:0/3821319232 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9e48109cf0 con 0x7f9e4806a870 2024-03-18T23:19:46.017 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:46.018+0000 7f9e3e7fc640 1 -- 172.21.15.49:0/3821319232 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11936 (secure 0 0 0) 0x7f9e3809b8b0 con 0x7f9e4806a870 2024-03-18T23:19:46.017 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-18T23:19:46.018 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6835/3631301630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":3631301630},{"type":"v1","addr":"172.21.15.49:6835","nonce":3631301630}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6837/444305233","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":444305233},{"type":"v1","addr":"172.21.15.49:6837","nonce":444305233}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":32,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-18T23:06:04.575999+0000","modified":"2024-03-18T23:11:38.156159+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24488,"mds_1":24463,"mds_2":24430,"mds_3":24419,"mds_4":24391},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14622":{"gid":14622,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.49:6839/1934085093","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6838","nonce":1934085093},{"type":"v1","addr":"172.21.15.49:6839","nonce":1934085093}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14634":{"gid":14634,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.49:6841/2869298862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6840","nonce":2869298862},{"type":"v1","addr":"172.21.15.49:6841","nonce":2869298862}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24389":{"gid":24389,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6833/3766178603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3766178603},{"type":"v1","addr":"172.21.15.139:6833","nonce":3766178603}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24391":{"gid":24391,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.175:6833/1702997173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":1702997173},{"type":"v1","addr":"172.21.15.175:6833","nonce":1702997173}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":9,"addr":"172.21.15.139:6835/1663212937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":1663212937},{"type":"v1","addr":"172.21.15.139:6835","nonce":1663212937}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.175:6835/755474931","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":755474931},{"type":"v1","addr":"172.21.15.175:6835","nonce":755474931}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6837/379482756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":379482756},{"type":"v1","addr":"172.21.15.175:6837","nonce":379482756}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"172.21.15.175:6839/3640882084","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":3640882084},{"type":"v1","addr":"172.21.15.175:6839","nonce":3640882084}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.139:6839/4280128209","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6838","nonce":4280128209},{"type":"v1","addr":"172.21.15.139:6839","nonce":4280128209}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25553":{"gid":25553,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6837/2891095267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6836","nonce":2891095267},{"type":"v1","addr":"172.21.15.139:6837","nonce":2891095267}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24391,24419,24430,24463,24488]},"id":1}]} 2024-03-18T23:19:46.018 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 32 2024-03-18T23:19:46.019 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:46.020+0000 7f9e4f43d640 1 -- 172.21.15.49:0/3821319232 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f9e340a2970 msgr2=0x7f9e340a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:19:46.020 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:46.020+0000 7f9e4f43d640 1 --2- 172.21.15.49:0/3821319232 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f9e340a2970 0x7f9e340a4e30 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f9e44009b50 tx=0x7f9e44005f90 comp rx=0 tx=0).stop 2024-03-18T23:19:46.020 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:46.020+0000 7f9e4f43d640 1 -- 172.21.15.49:0/3821319232 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9e4806a870 msgr2=0x7f9e48198540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:19:46.020 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:46.020+0000 7f9e4f43d640 1 --2- 172.21.15.49:0/3821319232 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9e4806a870 0x7f9e48198540 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f9e3800e9f0 tx=0x7f9e3800edf0 comp rx=0 tx=0).stop 2024-03-18T23:19:46.020 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:46.021+0000 7f9e4f43d640 1 -- 172.21.15.49:0/3821319232 shutdown_connections 2024-03-18T23:19:46.020 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:46.021+0000 7f9e4f43d640 1 --2- 172.21.15.49:0/3821319232 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f9e340a2970 0x7f9e340a4e30 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:46.020 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:46.021+0000 7f9e4f43d640 1 --2- 172.21.15.49:0/3821319232 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9e48107820 0x7f9e4819c8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:46.020 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:46.021+0000 7f9e4f43d640 1 --2- 172.21.15.49:0/3821319232 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9e4806a870 0x7f9e48198540 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:46.020 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:46.021+0000 7f9e4f43d640 1 --2- 172.21.15.49:0/3821319232 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9e48069ed0 0x7f9e48198000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:46.021 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:46.021+0000 7f9e4f43d640 1 -- 172.21.15.49:0/3821319232 >> 172.21.15.49:0/3821319232 conn(0x7f9e480777c0 msgr2=0x7f9e480ff250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:19:46.021 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:46.021+0000 7f9e4f43d640 1 -- 172.21.15.49:0/3821319232 shutdown_connections 2024-03-18T23:19:46.021 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:46.021+0000 7f9e4f43d640 1 -- 172.21.15.49:0/3821319232 wait complete. 2024-03-18T23:19:46.030 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-18T23:19:46.143 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:46.142+0000 7f90b4e0b640 1 -- 172.21.15.49:0/894812945 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f90b010be60 msgr2=0x7f90b010e300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:19:46.143 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:46.142+0000 7f90b4e0b640 1 --2- 172.21.15.49:0/894812945 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f90b010be60 0x7f90b010e300 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f90a8009ca0 tx=0x7f90a802c0f0 comp rx=0 tx=0).stop 2024-03-18T23:19:46.143 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:46.142+0000 7f90b4e0b640 1 -- 172.21.15.49:0/894812945 shutdown_connections 2024-03-18T23:19:46.143 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:46.142+0000 7f90b4e0b640 1 --2- 172.21.15.49:0/894812945 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f90b010be60 0x7f90b010e300 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:46.143 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:46.142+0000 7f90b4e0b640 1 --2- 172.21.15.49:0/894812945 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f90b01033b0 0x7f90b010b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:46.144 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:46.142+0000 7f90b4e0b640 1 --2- 172.21.15.49:0/894812945 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f90b0102a90 0x7f90b0102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:46.144 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:46.142+0000 7f90b4e0b640 1 -- 172.21.15.49:0/894812945 >> 172.21.15.49:0/894812945 conn(0x7f90b0077820 msgr2=0x7f90b0077c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:19:46.144 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:46.142+0000 7f90b4e0b640 1 -- 172.21.15.49:0/894812945 shutdown_connections 2024-03-18T23:19:46.144 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:46.142+0000 7f90b4e0b640 1 -- 172.21.15.49:0/894812945 wait complete. 2024-03-18T23:19:46.144 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:46.142+0000 7f90b4e0b640 1 Processor -- start 2024-03-18T23:19:46.144 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:46.143+0000 7f90b4e0b640 1 -- start start 2024-03-18T23:19:46.144 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:46.143+0000 7f90b4e0b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f90b0102a90 0x7f90b0198100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:19:46.144 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:46.143+0000 7f90b4e0b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f90b01033b0 0x7f90b0198640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:19:46.145 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:46.143+0000 7f90b4e0b640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f90b010be60 0x7f90b019c970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:19:46.145 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:46.143+0000 7f90b4e0b640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f90b0111c60 con 0x7f90b01033b0 2024-03-18T23:19:46.145 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:46.143+0000 7f90b4e0b640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f90b0111ae0 con 0x7f90b0102a90 2024-03-18T23:19:46.145 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:46.143+0000 7f90b4e0b640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f90b0111de0 con 0x7f90b010be60 2024-03-18T23:19:46.145 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:46.143+0000 7f90add9b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f90b01033b0 0x7f90b0198640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:19:46.145 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:46.143+0000 7f90add9b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f90b01033b0 0x7f90b0198640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:41032/0 (socket says 172.21.15.49:41032) 2024-03-18T23:19:46.145 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:46.143+0000 7f90add9b640 1 -- 172.21.15.49:0/3936305178 learned_addr learned my addr 172.21.15.49:0/3936305178 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:19:46.145 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:46.143+0000 7f90ae59c640 1 --2- 172.21.15.49:0/3936305178 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f90b0102a90 0x7f90b0198100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:19:46.145 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:46.143+0000 7f90aed9d640 1 --2- 172.21.15.49:0/3936305178 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f90b010be60 0x7f90b019c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:19:46.146 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:46.143+0000 7f90add9b640 1 -- 172.21.15.49:0/3936305178 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f90b010be60 msgr2=0x7f90b019c970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:19:46.146 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:46.143+0000 7f90add9b640 1 --2- 172.21.15.49:0/3936305178 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f90b010be60 0x7f90b019c970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:46.146 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:46.143+0000 7f90add9b640 1 -- 172.21.15.49:0/3936305178 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f90b0102a90 msgr2=0x7f90b0198100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:19:46.146 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:46.143+0000 7f90add9b640 1 --2- 172.21.15.49:0/3936305178 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f90b0102a90 0x7f90b0198100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:46.146 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:46.143+0000 7f90add9b640 1 -- 172.21.15.49:0/3936305178 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90b019d0a0 con 0x7f90b01033b0 2024-03-18T23:19:46.146 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:46.143+0000 7f90ae59c640 1 --2- 172.21.15.49:0/3936305178 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f90b0102a90 0x7f90b0198100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:19:46.146 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:46.143+0000 7f90add9b640 1 --2- 172.21.15.49:0/3936305178 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f90b01033b0 0x7f90b0198640 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f90a400e9f0 tx=0x7f90a400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:19:46.146 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:46.143+0000 7f909f7fe640 1 -- 172.21.15.49:0/3936305178 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90a400cd80 con 0x7f90b01033b0 2024-03-18T23:19:46.147 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:46.144+0000 7f909f7fe640 1 -- 172.21.15.49:0/3936305178 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f90a4010910 con 0x7f90b01033b0 2024-03-18T23:19:46.147 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:46.144+0000 7f90b4e0b640 1 -- 172.21.15.49:0/3936305178 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90b019d390 con 0x7f90b01033b0 2024-03-18T23:19:46.147 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:46.144+0000 7f909f7fe640 1 -- 172.21.15.49:0/3936305178 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90a4018990 con 0x7f90b01033b0 2024-03-18T23:19:46.147 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:46.144+0000 7f90b4e0b640 1 -- 172.21.15.49:0/3936305178 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90b01a4c20 con 0x7f90b01033b0 2024-03-18T23:19:46.148 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:46.145+0000 7f90b4e0b640 1 -- 172.21.15.49:0/3936305178 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90b0068610 con 0x7f90b01033b0 2024-03-18T23:19:46.148 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:46.146+0000 7f909f7fe640 1 -- 172.21.15.49:0/3936305178 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 25) ==== 137421+0+0 (secure 0 0 0) 0x7f90a4010ac0 con 0x7f90b01033b0 2024-03-18T23:19:46.148 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:46.146+0000 7f909f7fe640 1 --2- 172.21.15.49:0/3936305178 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f90980a29f0 0x7f90980a4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:19:46.148 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:46.146+0000 7f90ae59c640 1 --2- 172.21.15.49:0/3936305178 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f90980a29f0 0x7f90980a4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:19:46.148 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:46.146+0000 7f909f7fe640 1 -- 172.21.15.49:0/3936305178 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8556+0+0 (secure 0 0 0) 0x7f90a40cf120 con 0x7f90b01033b0 2024-03-18T23:19:46.148 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:46.147+0000 7f90ae59c640 1 --2- 172.21.15.49:0/3936305178 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f90980a29f0 0x7f90980a4eb0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f90a0009fc0 tx=0x7f90a0005f90 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:19:46.149 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:46.151+0000 7f909f7fe640 1 -- 172.21.15.49:0/3936305178 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f90a409b810 con 0x7f90b01033b0 2024-03-18T23:19:46.335 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:46.336+0000 7f90b4e0b640 1 -- 172.21.15.49:0/3936305178 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f90b01a4e50 con 0x7f90b01033b0 2024-03-18T23:19:46.337 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:46.338+0000 7f909f7fe640 1 -- 172.21.15.49:0/3936305178 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v101) ==== 74+0+23204 (secure 0 0 0) 0x7f90a409f8b0 con 0x7f90b01033b0 2024-03-18T23:19:46.337 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-18T23:19:46.337 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":101,"fsid":"c63591ea-e57a-11ee-95c9-87774f69a715","created":"2024-03-18T22:57:58.734467+0000","modified":"2024-03-18T23:19:19.125020+0000","last_up_change":"2024-03-18T23:05:16.576310+0000","last_in_change":"2024-03-18T23:04:57.024067+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-18T23:01:28.363432+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-18T23:06:03.172963+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-18T23:06:03.973303+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"100","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":17,"snap_epoch":100,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e7fc9daa-4098-4ba5-8019-9aaef1aab3e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":1651162501},{"type":"v1","addr":"172.21.15.49:6803","nonce":1651162501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":1651162501},{"type":"v1","addr":"172.21.15.49:6805","nonce":1651162501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":1651162501},{"type":"v1","addr":"172.21.15.49:6809","nonce":1651162501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":1651162501},{"type":"v1","addr":"172.21.15.49:6807","nonce":1651162501}]},"public_addr":"172.21.15.49:6803/1651162501","cluster_addr":"172.21.15.49:6805/1651162501","heartbeat_back_addr":"172.21.15.49:6809/1651162501","heartbeat_front_addr":"172.21.15.49:6807/1651162501","state":["exists","up"]},{"osd":1,"uuid":"40ac929c-e90f-434c-a796-9aabd686f1fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":1766700359},{"type":"v1","addr":"172.21.15.139:6801","nonce":1766700359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":1766700359},{"type":"v1","addr":"172.21.15.139:6803","nonce":1766700359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":1766700359},{"type":"v1","addr":"172.21.15.139:6807","nonce":1766700359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":1766700359},{"type":"v1","addr":"172.21.15.139:6805","nonce":1766700359}]},"public_addr":"172.21.15.139:6801/1766700359","cluster_addr":"172.21.15.139:6803/1766700359","heartbeat_back_addr":"172.21.15.139:6807/1766700359","heartbeat_front_addr":"172.21.15.139:6805/1766700359","state":["exists","up"]},{"osd":2,"uuid":"8ae3b666-00b4-45ed-a5ee-4d0771edabb6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":3752671234},{"type":"v1","addr":"172.21.15.175:6801","nonce":3752671234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":3752671234},{"type":"v1","addr":"172.21.15.175:6803","nonce":3752671234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":3752671234},{"type":"v1","addr":"172.21.15.175:6807","nonce":3752671234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":3752671234},{"type":"v1","addr":"172.21.15.175:6805","nonce":3752671234}]},"public_addr":"172.21.15.175:6801/3752671234","cluster_addr":"172.21.15.175:6803/3752671234","heartbeat_back_addr":"172.21.15.175:6807/3752671234","heartbeat_front_addr":"172.21.15.175:6805/3752671234","state":["exists","up"]},{"osd":3,"uuid":"1a62e89d-45d4-4237-b813-7f96a59aca7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":956160536},{"type":"v1","addr":"172.21.15.49:6811","nonce":956160536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":956160536},{"type":"v1","addr":"172.21.15.49:6813","nonce":956160536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":956160536},{"type":"v1","addr":"172.21.15.49:6817","nonce":956160536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":956160536},{"type":"v1","addr":"172.21.15.49:6815","nonce":956160536}]},"public_addr":"172.21.15.49:6811/956160536","cluster_addr":"172.21.15.49:6813/956160536","heartbeat_back_addr":"172.21.15.49:6817/956160536","heartbeat_front_addr":"172.21.15.49:6815/956160536","state":["exists","up"]},{"osd":4,"uuid":"c715101c-0c3f-4b98-b142-d64a35e187df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":3129712668},{"type":"v1","addr":"172.21.15.139:6809","nonce":3129712668}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":3129712668},{"type":"v1","addr":"172.21.15.139:6811","nonce":3129712668}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":3129712668},{"type":"v1","addr":"172.21.15.139:6815","nonce":3129712668}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":3129712668},{"type":"v1","addr":"172.21.15.139:6813","nonce":3129712668}]},"public_addr":"172.21.15.139:6809/3129712668","cluster_addr":"172.21.15.139:6811/3129712668","heartbeat_back_addr":"172.21.15.139:6815/3129712668","heartbeat_front_addr":"172.21.15.139:6813/3129712668","state":["exists","up"]},{"osd":5,"uuid":"4b641f81-77eb-45fd-8464-511e7b3062b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":2774567215},{"type":"v1","addr":"172.21.15.175:6809","nonce":2774567215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":2774567215},{"type":"v1","addr":"172.21.15.175:6811","nonce":2774567215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":2774567215},{"type":"v1","addr":"172.21.15.175:6815","nonce":2774567215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":2774567215},{"type":"v1","addr":"172.21.15.175:6813","nonce":2774567215}]},"public_addr":"172.21.15.175:6809/2774567215","cluster_addr":"172.21.15.175:6811/2774567215","heartbeat_back_addr":"172.21.15.175:6815/2774567215","heartbeat_front_addr":"172.21.15.175:6813/2774567215","state":["exists","up"]},{"osd":6,"uuid":"653da8e0-1197-4b48-abed-470f2a9c4dce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":1472215461},{"type":"v1","addr":"172.21.15.49:6819","nonce":1472215461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":1472215461},{"type":"v1","addr":"172.21.15.49:6821","nonce":1472215461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":1472215461},{"type":"v1","addr":"172.21.15.49:6825","nonce":1472215461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":1472215461},{"type":"v1","addr":"172.21.15.49:6823","nonce":1472215461}]},"public_addr":"172.21.15.49:6819/1472215461","cluster_addr":"172.21.15.49:6821/1472215461","heartbeat_back_addr":"172.21.15.49:6825/1472215461","heartbeat_front_addr":"172.21.15.49:6823/1472215461","state":["exists","up"]},{"osd":7,"uuid":"ad157c9b-4b75-451c-8740-630f0975b7fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":2163986787},{"type":"v1","addr":"172.21.15.139:6817","nonce":2163986787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":2163986787},{"type":"v1","addr":"172.21.15.139:6819","nonce":2163986787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":2163986787},{"type":"v1","addr":"172.21.15.139:6823","nonce":2163986787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":2163986787},{"type":"v1","addr":"172.21.15.139:6821","nonce":2163986787}]},"public_addr":"172.21.15.139:6817/2163986787","cluster_addr":"172.21.15.139:6819/2163986787","heartbeat_back_addr":"172.21.15.139:6823/2163986787","heartbeat_front_addr":"172.21.15.139:6821/2163986787","state":["exists","up"]},{"osd":8,"uuid":"c8e75f15-df75-4d5d-9b92-a064bffb6c01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":1194026992},{"type":"v1","addr":"172.21.15.175:6817","nonce":1194026992}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1194026992},{"type":"v1","addr":"172.21.15.175:6819","nonce":1194026992}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":1194026992},{"type":"v1","addr":"172.21.15.175:6823","nonce":1194026992}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1194026992},{"type":"v1","addr":"172.21.15.175:6821","nonce":1194026992}]},"public_addr":"172.21.15.175:6817/1194026992","cluster_addr":"172.21.15.175:6819/1194026992","heartbeat_back_addr":"172.21.15.175:6823/1194026992","heartbeat_front_addr":"172.21.15.175:6821/1194026992","state":["exists","up"]},{"osd":9,"uuid":"edb5ffef-b6a2-4941-b5ab-40f982ad30d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":2742084884},{"type":"v1","addr":"172.21.15.49:6827","nonce":2742084884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":2742084884},{"type":"v1","addr":"172.21.15.49:6829","nonce":2742084884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":2742084884},{"type":"v1","addr":"172.21.15.49:6833","nonce":2742084884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":2742084884},{"type":"v1","addr":"172.21.15.49:6831","nonce":2742084884}]},"public_addr":"172.21.15.49:6827/2742084884","cluster_addr":"172.21.15.49:6829/2742084884","heartbeat_back_addr":"172.21.15.49:6833/2742084884","heartbeat_front_addr":"172.21.15.49:6831/2742084884","state":["exists","up"]},{"osd":10,"uuid":"3945e46d-766a-4c4a-8821-d43648dd1578","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":2404740201},{"type":"v1","addr":"172.21.15.139:6825","nonce":2404740201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":2404740201},{"type":"v1","addr":"172.21.15.139:6827","nonce":2404740201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":2404740201},{"type":"v1","addr":"172.21.15.139:6831","nonce":2404740201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":2404740201},{"type":"v1","addr":"172.21.15.139:6829","nonce":2404740201}]},"public_addr":"172.21.15.139:6825/2404740201","cluster_addr":"172.21.15.139:6827/2404740201","heartbeat_back_addr":"172.21.15.139:6831/2404740201","heartbeat_front_addr":"172.21.15.139:6829/2404740201","state":["exists","up"]},{"osd":11,"uuid":"9954ba15-d877-46cc-b24a-893f9168deb8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":3519503868},{"type":"v1","addr":"172.21.15.175:6825","nonce":3519503868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":3519503868},{"type":"v1","addr":"172.21.15.175:6827","nonce":3519503868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":3519503868},{"type":"v1","addr":"172.21.15.175:6831","nonce":3519503868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":3519503868},{"type":"v1","addr":"172.21.15.175:6829","nonce":3519503868}]},"public_addr":"172.21.15.175:6825/3519503868","cluster_addr":"172.21.15.175:6827/3519503868","heartbeat_back_addr":"172.21.15.175:6831/3519503868","heartbeat_front_addr":"172.21.15.175:6829/3519503868","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:00:38.367865+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:01:02.237687+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:01:25.609751+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:01:51.268572+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:02:16.514605+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:02:41.586523+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:03:07.029964+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:03:32.956250+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:03:57.547984+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:04:23.948972+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:04:49.667210+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:05:15.284693+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.6","mappings":[{"from":4,"to":6}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.49:0/1463571219":"2024-03-19T00:13:11.015881+0000","172.21.15.49:6800/2609458951":"2024-03-19T22:58:59.806253+0000","172.21.15.49:6801/1985190312":"2024-03-19T22:58:31.972474+0000","172.21.15.49:0/2176210039":"2024-03-19T22:58:31.972474+0000","172.21.15.49:0/1912446700":"2024-03-19T22:58:59.806253+0000","172.21.15.49:0/21283102":"2024-03-19T22:58:31.972474+0000","172.21.15.49:6801/2609458951":"2024-03-19T22:58:59.806253+0000","172.21.15.49:0/1036383534":"2024-03-19T22:58:59.806253+0000","172.21.15.49:0/563527416":"2024-03-19T22:58:59.806253+0000","172.21.15.49:6801/2083805211":"2024-03-19T23:08:15.312250+0000","172.21.15.49:0/1976299520":"2024-03-19T23:08:15.312250+0000","172.21.15.49:0/861926960":"2024-03-19T23:08:15.312250+0000","172.21.15.49:0/1292058922":"2024-03-19T22:58:31.972474+0000","172.21.15.49:0/94393705":"2024-03-19T23:08:15.312250+0000","172.21.15.49:6800/1985190312":"2024-03-19T22:58:31.972474+0000","172.21.15.49:6800/2083805211":"2024-03-19T23:08:15.312250+0000","172.21.15.49:0/3268662009":"2024-03-19T23:08:15.312250+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":6,"length":1},{"begin":17,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-18T23:19:46.339 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:46.340+0000 7f90b4e0b640 1 -- 172.21.15.49:0/3936305178 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f90980a29f0 msgr2=0x7f90980a4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:19:46.339 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:46.340+0000 7f90b4e0b640 1 --2- 172.21.15.49:0/3936305178 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f90980a29f0 0x7f90980a4eb0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f90a0009fc0 tx=0x7f90a0005f90 comp rx=0 tx=0).stop 2024-03-18T23:19:46.339 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:46.340+0000 7f90b4e0b640 1 -- 172.21.15.49:0/3936305178 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f90b01033b0 msgr2=0x7f90b0198640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:19:46.339 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:46.340+0000 7f90b4e0b640 1 --2- 172.21.15.49:0/3936305178 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f90b01033b0 0x7f90b0198640 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f90a400e9f0 tx=0x7f90a400eeb0 comp rx=0 tx=0).stop 2024-03-18T23:19:46.340 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:46.340+0000 7f90b4e0b640 1 -- 172.21.15.49:0/3936305178 shutdown_connections 2024-03-18T23:19:46.340 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:46.340+0000 7f90b4e0b640 1 --2- 172.21.15.49:0/3936305178 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f90980a29f0 0x7f90980a4eb0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:46.340 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:46.340+0000 7f90b4e0b640 1 --2- 172.21.15.49:0/3936305178 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f90b010be60 0x7f90b019c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:46.340 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:46.340+0000 7f90b4e0b640 1 --2- 172.21.15.49:0/3936305178 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f90b01033b0 0x7f90b0198640 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:46.340 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:46.340+0000 7f90b4e0b640 1 --2- 172.21.15.49:0/3936305178 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f90b0102a90 0x7f90b0198100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:46.340 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:46.340+0000 7f90b4e0b640 1 -- 172.21.15.49:0/3936305178 >> 172.21.15.49:0/3936305178 conn(0x7f90b0077820 msgr2=0x7f90b00fd730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:19:46.340 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:46.340+0000 7f90b4e0b640 1 -- 172.21.15.49:0/3936305178 shutdown_connections 2024-03-18T23:19:46.340 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:46.340+0000 7f90b4e0b640 1 -- 172.21.15.49:0/3936305178 wait complete. 2024-03-18T23:19:46.350 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-03-18T23:19:46.461 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:46.462+0000 7f5a2d6cc640 1 -- 172.21.15.49:0/1613328555 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5a28101050 msgr2=0x7f5a2810d560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:19:46.461 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:46.462+0000 7f5a2d6cc640 1 --2- 172.21.15.49:0/1613328555 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5a28101050 0x7f5a2810d560 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f5a18009cd0 tx=0x7f5a1802c0f0 comp rx=0 tx=0).stop 2024-03-18T23:19:46.461 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:46.462+0000 7f5a2d6cc640 1 -- 172.21.15.49:0/1613328555 shutdown_connections 2024-03-18T23:19:46.461 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:46.462+0000 7f5a2d6cc640 1 --2- 172.21.15.49:0/1613328555 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5a28101050 0x7f5a2810d560 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:46.461 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:46.462+0000 7f5a2d6cc640 1 --2- 172.21.15.49:0/1613328555 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5a28100690 0x7f5a28100b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:46.461 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:46.462+0000 7f5a2d6cc640 1 --2- 172.21.15.49:0/1613328555 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5a281066a0 0x7f5a28106a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:46.462 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:46.462+0000 7f5a2d6cc640 1 -- 172.21.15.49:0/1613328555 >> 172.21.15.49:0/1613328555 conn(0x7f5a28077820 msgr2=0x7f5a28077c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:19:46.462 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:46.462+0000 7f5a2d6cc640 1 -- 172.21.15.49:0/1613328555 shutdown_connections 2024-03-18T23:19:46.462 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:46.462+0000 7f5a2d6cc640 1 -- 172.21.15.49:0/1613328555 wait complete. 2024-03-18T23:19:46.462 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:46.462+0000 7f5a2d6cc640 1 Processor -- start 2024-03-18T23:19:46.464 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:46.462+0000 7f5a2d6cc640 1 -- start start 2024-03-18T23:19:46.464 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:46.463+0000 7f5a2d6cc640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5a28100690 0x7f5a28198160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:19:46.465 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:46.463+0000 7f5a2d6cc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5a28101050 0x7f5a281986a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:19:46.465 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:46.463+0000 7f5a2d6cc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5a281066a0 0x7f5a2819c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:19:46.465 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:46.463+0000 7f5a2d6cc640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f5a28110ab0 con 0x7f5a281066a0 2024-03-18T23:19:46.465 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:46.463+0000 7f5a2d6cc640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f5a28110930 con 0x7f5a28101050 2024-03-18T23:19:46.465 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:46.463+0000 7f5a2d6cc640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f5a28110c30 con 0x7f5a28100690 2024-03-18T23:19:46.465 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:46.463+0000 7f5a277fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5a281066a0 0x7f5a2819c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:19:46.465 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:46.463+0000 7f5a277fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5a281066a0 0x7f5a2819c9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:41054/0 (socket says 172.21.15.49:41054) 2024-03-18T23:19:46.465 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:46.463+0000 7f5a277fe640 1 -- 172.21.15.49:0/731017230 learned_addr learned my addr 172.21.15.49:0/731017230 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:19:46.465 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:46.463+0000 7f5a26ffd640 1 --2- 172.21.15.49:0/731017230 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5a28100690 0x7f5a28198160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:19:46.465 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:46.463+0000 7f5a267fc640 1 --2- 172.21.15.49:0/731017230 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5a28101050 0x7f5a281986a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:19:46.466 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:46.463+0000 7f5a277fe640 1 -- 172.21.15.49:0/731017230 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5a28100690 msgr2=0x7f5a28198160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:19:46.466 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:46.463+0000 7f5a277fe640 1 --2- 172.21.15.49:0/731017230 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5a28100690 0x7f5a28198160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:46.466 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:46.463+0000 7f5a277fe640 1 -- 172.21.15.49:0/731017230 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5a28101050 msgr2=0x7f5a281986a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:19:46.466 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:46.463+0000 7f5a277fe640 1 --2- 172.21.15.49:0/731017230 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5a28101050 0x7f5a281986a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:46.466 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:46.463+0000 7f5a277fe640 1 -- 172.21.15.49:0/731017230 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a2819d100 con 0x7f5a281066a0 2024-03-18T23:19:46.466 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:46.463+0000 7f5a277fe640 1 --2- 172.21.15.49:0/731017230 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5a281066a0 0x7f5a2819c9d0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f5a180348f0 tx=0x7f5a18032c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:19:46.466 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:46.463+0000 7f5a0ffff640 1 -- 172.21.15.49:0/731017230 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a18041020 con 0x7f5a281066a0 2024-03-18T23:19:46.466 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:46.463+0000 7f5a2d6cc640 1 -- 172.21.15.49:0/731017230 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a2819d390 con 0x7f5a281066a0 2024-03-18T23:19:46.467 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:46.463+0000 7f5a0ffff640 1 -- 172.21.15.49:0/731017230 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5a18032e80 con 0x7f5a281066a0 2024-03-18T23:19:46.467 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:46.464+0000 7f5a0ffff640 1 -- 172.21.15.49:0/731017230 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a18039040 con 0x7f5a281066a0 2024-03-18T23:19:46.467 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:46.464+0000 7f5a2d6cc640 1 -- 172.21.15.49:0/731017230 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a281a4c80 con 0x7f5a281066a0 2024-03-18T23:19:46.467 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:46.464+0000 7f5a2d6cc640 1 -- 172.21.15.49:0/731017230 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a28068610 con 0x7f5a281066a0 2024-03-18T23:19:46.467 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:46.466+0000 7f5a0ffff640 1 -- 172.21.15.49:0/731017230 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 25) ==== 137421+0+0 (secure 0 0 0) 0x7f5a18044020 con 0x7f5a281066a0 2024-03-18T23:19:46.468 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:46.466+0000 7f5a0ffff640 1 --2- 172.21.15.49:0/731017230 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f5a100a2970 0x7f5a100a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:19:46.468 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:46.466+0000 7f5a26ffd640 1 --2- 172.21.15.49:0/731017230 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f5a100a2970 0x7f5a100a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:19:46.468 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:46.466+0000 7f5a0ffff640 1 -- 172.21.15.49:0/731017230 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8556+0+0 (secure 0 0 0) 0x7f5a180f0d60 con 0x7f5a281066a0 2024-03-18T23:19:46.468 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:46.467+0000 7f5a26ffd640 1 --2- 172.21.15.49:0/731017230 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f5a100a2970 0x7f5a100a4e30 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f5a20009b50 tx=0x7f5a20005f90 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:19:46.469 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:46.470+0000 7f5a0ffff640 1 -- 172.21.15.49:0/731017230 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f5a1804b020 con 0x7f5a281066a0 2024-03-18T23:19:46.629 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:19:46 smithi049 ceph-mon[62991]: pgmap v360: 97 pgs: 97 active+clean; 61 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 75 KiB/s wr, 23 op/s 2024-03-18T23:19:46.629 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:19:46 smithi049 ceph-mon[62991]: from='client.? 172.21.15.49:0/2024149939' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T23:19:46.629 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:19:46 smithi049 ceph-mon[62991]: from='client.? 172.21.15.49:0/3821319232' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T23:19:46.629 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:19:46 smithi049 ceph-mon[62991]: from='client.? 172.21.15.49:0/3936305178' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T23:19:46.692 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:46.694+0000 7f5a2d6cc640 1 -- 172.21.15.49:0/731017230 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f5a28109c30 con 0x7f5a281066a0 2024-03-18T23:19:46.694 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:46.696+0000 7f5a0ffff640 1 -- 172.21.15.49:0/731017230 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v33) ==== 161+0+0 (secure 0 0 0) 0x7f5a180bdbb0 con 0x7f5a281066a0 2024-03-18T23:19:46.695 INFO:teuthology.orchestra.run.smithi049.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-03-18T23:19:46.698 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:46.699+0000 7f5a2d6cc640 1 -- 172.21.15.49:0/731017230 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f5a100a2970 msgr2=0x7f5a100a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:19:46.699 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:46.699+0000 7f5a2d6cc640 1 --2- 172.21.15.49:0/731017230 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f5a100a2970 0x7f5a100a4e30 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f5a20009b50 tx=0x7f5a20005f90 comp rx=0 tx=0).stop 2024-03-18T23:19:46.699 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:46.699+0000 7f5a2d6cc640 1 -- 172.21.15.49:0/731017230 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5a281066a0 msgr2=0x7f5a2819c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:19:46.699 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:46.699+0000 7f5a2d6cc640 1 --2- 172.21.15.49:0/731017230 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5a281066a0 0x7f5a2819c9d0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f5a180348f0 tx=0x7f5a18032c90 comp rx=0 tx=0).stop 2024-03-18T23:19:46.699 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:46.699+0000 7f5a2d6cc640 1 -- 172.21.15.49:0/731017230 shutdown_connections 2024-03-18T23:19:46.699 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:46.699+0000 7f5a2d6cc640 1 --2- 172.21.15.49:0/731017230 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f5a100a2970 0x7f5a100a4e30 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:46.699 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:46.699+0000 7f5a2d6cc640 1 --2- 172.21.15.49:0/731017230 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5a281066a0 0x7f5a2819c9d0 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:46.699 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:46.699+0000 7f5a2d6cc640 1 --2- 172.21.15.49:0/731017230 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5a28101050 0x7f5a281986a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:46.700 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:46.699+0000 7f5a2d6cc640 1 --2- 172.21.15.49:0/731017230 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5a28100690 0x7f5a28198160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:46.700 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:46.699+0000 7f5a2d6cc640 1 -- 172.21.15.49:0/731017230 >> 172.21.15.49:0/731017230 conn(0x7f5a28077820 msgr2=0x7f5a28068d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:19:46.700 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:46.700+0000 7f5a2d6cc640 1 -- 172.21.15.49:0/731017230 shutdown_connections 2024-03-18T23:19:46.700 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:46.700+0000 7f5a2d6cc640 1 -- 172.21.15.49:0/731017230 wait complete. 2024-03-18T23:19:46.710 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-03-18T23:19:46.817 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:19:46 smithi139 ceph-mon[67282]: pgmap v360: 97 pgs: 97 active+clean; 61 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 75 KiB/s wr, 23 op/s 2024-03-18T23:19:46.818 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:19:46 smithi139 ceph-mon[67282]: from='client.? 172.21.15.49:0/2024149939' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T23:19:46.818 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:19:46 smithi139 ceph-mon[67282]: from='client.? 172.21.15.49:0/3821319232' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T23:19:46.818 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:19:46 smithi139 ceph-mon[67282]: from='client.? 172.21.15.49:0/3936305178' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T23:19:46.818 INFO:journalctl@ceph.mds.e.smithi139.stdout:Mar 18 23:19:46 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mds-e[97494]: ignoring --setuser ceph since I am not root 2024-03-18T23:19:46.819 INFO:journalctl@ceph.mds.e.smithi139.stdout:Mar 18 23:19:46 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mds-e[97494]: ignoring --setgroup ceph since I am not root 2024-03-18T23:19:46.819 INFO:journalctl@ceph.mds.e.smithi139.stdout:Mar 18 23:19:46 smithi139 ceph-mds[97498]: -- 172.21.15.139:0/836857537 <== mon.0 v2:172.21.15.49:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56488b931a40 con 0x56488b904c00 2024-03-18T23:19:46.819 INFO:journalctl@ceph.mds.e.smithi139.stdout:Mar 18 23:19:46 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mds-e[97494]: starting mds.e at 2024-03-18T23:19:46.819 INFO:journalctl@ceph.mds.h.smithi139.stdout:Mar 18 23:19:46 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mds-h[97809]: ignoring --setuser ceph since I am not root 2024-03-18T23:19:46.819 INFO:journalctl@ceph.mds.h.smithi139.stdout:Mar 18 23:19:46 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mds-h[97809]: ignoring --setgroup ceph since I am not root 2024-03-18T23:19:46.819 INFO:journalctl@ceph.mds.h.smithi139.stdout:Mar 18 23:19:46 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mds-h[97809]: starting mds.h at 2024-03-18T23:19:46.820 INFO:journalctl@ceph.mds.k.smithi139.stdout:Mar 18 23:19:46 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mds-k[97173]: ignoring --setuser ceph since I am not root 2024-03-18T23:19:46.820 INFO:journalctl@ceph.mds.k.smithi139.stdout:Mar 18 23:19:46 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mds-k[97173]: ignoring --setgroup ceph since I am not root 2024-03-18T23:19:46.820 INFO:journalctl@ceph.mds.k.smithi139.stdout:Mar 18 23:19:46 smithi139 ceph-mds[97177]: -- 172.21.15.139:0/984646789 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x561bae9224e0 con 0x561bae8f8800 2024-03-18T23:19:46.820 INFO:journalctl@ceph.mds.k.smithi139.stdout:Mar 18 23:19:46 smithi139 ceph-mds[97177]: -- 172.21.15.139:0/984646789 <== mon.2 v2:172.21.15.139:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x561bae925a40 con 0x561bae8f8800 2024-03-18T23:19:46.821 INFO:journalctl@ceph.mds.k.smithi139.stdout:Mar 18 23:19:46 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mds-k[97173]: starting mds.k at 2024-03-18T23:19:46.821 INFO:journalctl@ceph.mds.b.smithi139.stdout:Mar 18 23:19:46 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mds-b[98118]: ignoring --setuser ceph since I am not root 2024-03-18T23:19:46.821 INFO:journalctl@ceph.mds.b.smithi139.stdout:Mar 18 23:19:46 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mds-b[98118]: ignoring --setgroup ceph since I am not root 2024-03-18T23:19:46.821 INFO:journalctl@ceph.mds.b.smithi139.stdout:Mar 18 23:19:46 smithi139 ceph-mds[98122]: -- 172.21.15.139:0/326199400 <== mon.1 v2:172.21.15.175:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d4d7435a40 con 0x55d4d7408c00 2024-03-18T23:19:46.821 INFO:journalctl@ceph.mds.b.smithi139.stdout:Mar 18 23:19:46 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mds-b[98118]: starting mds.b at 2024-03-18T23:19:46.830 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:46.828+0000 7f35f28a7640 1 -- 172.21.15.49:0/1935947922 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f35ec102a90 msgr2=0x7f35ec102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:19:46.830 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:46.828+0000 7f35f28a7640 1 --2- 172.21.15.49:0/1935947922 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f35ec102a90 0x7f35ec102e70 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f35dc009900 tx=0x7f35dc02bcd0 comp rx=0 tx=0).stop 2024-03-18T23:19:46.830 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:46.828+0000 7f35f28a7640 1 -- 172.21.15.49:0/1935947922 shutdown_connections 2024-03-18T23:19:46.830 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:46.828+0000 7f35f28a7640 1 --2- 172.21.15.49:0/1935947922 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f35ec10be60 0x7f35ec10e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:46.830 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:46.828+0000 7f35f28a7640 1 --2- 172.21.15.49:0/1935947922 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f35ec1033b0 0x7f35ec10b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:46.830 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:46.828+0000 7f35f28a7640 1 --2- 172.21.15.49:0/1935947922 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f35ec102a90 0x7f35ec102e70 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:46.831 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:46.828+0000 7f35f28a7640 1 -- 172.21.15.49:0/1935947922 >> 172.21.15.49:0/1935947922 conn(0x7f35ec077880 msgr2=0x7f35ec077c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:19:46.831 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:46.828+0000 7f35f28a7640 1 -- 172.21.15.49:0/1935947922 shutdown_connections 2024-03-18T23:19:46.831 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:46.828+0000 7f35f28a7640 1 -- 172.21.15.49:0/1935947922 wait complete. 2024-03-18T23:19:46.831 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:46.829+0000 7f35f28a7640 1 Processor -- start 2024-03-18T23:19:46.831 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:46.829+0000 7f35f28a7640 1 -- start start 2024-03-18T23:19:46.832 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:46.829+0000 7f35f28a7640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f35ec102a90 0x7f35ec1981b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:19:46.832 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:46.829+0000 7f35f28a7640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f35ec1033b0 0x7f35ec1986f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:19:46.832 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:46.829+0000 7f35f28a7640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f35ec10be60 0x7f35ec19ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:19:46.832 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:46.829+0000 7f35f28a7640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f35ec111c30 con 0x7f35ec10be60 2024-03-18T23:19:46.832 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:46.829+0000 7f35f28a7640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f35ec111ab0 con 0x7f35ec1033b0 2024-03-18T23:19:46.832 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:46.829+0000 7f35f28a7640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f35ec111db0 con 0x7f35ec102a90 2024-03-18T23:19:46.832 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:46.829+0000 7f35f0e44640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f35ec10be60 0x7f35ec19ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:19:46.832 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:46.829+0000 7f35f0e44640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f35ec10be60 0x7f35ec19ca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:41096/0 (socket says 172.21.15.49:41096) 2024-03-18T23:19:46.833 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:46.829+0000 7f35f0e44640 1 -- 172.21.15.49:0/1240361744 learned_addr learned my addr 172.21.15.49:0/1240361744 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:19:46.833 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:46.829+0000 7f35ebfff640 1 --2- 172.21.15.49:0/1240361744 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f35ec102a90 0x7f35ec1981b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:19:46.833 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:46.829+0000 7f35eb7fe640 1 --2- 172.21.15.49:0/1240361744 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f35ec1033b0 0x7f35ec1986f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:19:46.833 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:46.830+0000 7f35f0e44640 1 -- 172.21.15.49:0/1240361744 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f35ec102a90 msgr2=0x7f35ec1981b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:19:46.833 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:46.830+0000 7f35f0e44640 1 --2- 172.21.15.49:0/1240361744 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f35ec102a90 0x7f35ec1981b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:46.833 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:46.830+0000 7f35f0e44640 1 -- 172.21.15.49:0/1240361744 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f35ec1033b0 msgr2=0x7f35ec1986f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:19:46.833 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:46.830+0000 7f35f0e44640 1 --2- 172.21.15.49:0/1240361744 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f35ec1033b0 0x7f35ec1986f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:46.833 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:46.830+0000 7f35f0e44640 1 -- 172.21.15.49:0/1240361744 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35ec19d1a0 con 0x7f35ec10be60 2024-03-18T23:19:46.834 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:46.830+0000 7f35f0e44640 1 --2- 172.21.15.49:0/1240361744 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f35ec10be60 0x7f35ec19ca70 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f35e400eaa0 tx=0x7f35e400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:19:46.834 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:46.830+0000 7f35e97fa640 1 -- 172.21.15.49:0/1240361744 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35e400cdb0 con 0x7f35ec10be60 2024-03-18T23:19:46.834 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:46.830+0000 7f35f28a7640 1 -- 172.21.15.49:0/1240361744 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35ec19d490 con 0x7f35ec10be60 2024-03-18T23:19:46.834 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:46.830+0000 7f35e97fa640 1 -- 172.21.15.49:0/1240361744 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f35e4005020 con 0x7f35ec10be60 2024-03-18T23:19:46.834 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:46.830+0000 7f35e97fa640 1 -- 172.21.15.49:0/1240361744 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35e40116b0 con 0x7f35ec10be60 2024-03-18T23:19:46.834 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:46.830+0000 7f35f28a7640 1 -- 172.21.15.49:0/1240361744 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35ec1a4d20 con 0x7f35ec10be60 2024-03-18T23:19:46.834 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:46.831+0000 7f35f28a7640 1 -- 172.21.15.49:0/1240361744 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35ec068a60 con 0x7f35ec10be60 2024-03-18T23:19:46.835 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:46.832+0000 7f35e97fa640 1 -- 172.21.15.49:0/1240361744 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 25) ==== 137421+0+0 (secure 0 0 0) 0x7f35e4005560 con 0x7f35ec10be60 2024-03-18T23:19:46.835 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:46.833+0000 7f35e97fa640 1 --2- 172.21.15.49:0/1240361744 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f35d40a2970 0x7f35d40a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:19:46.836 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:46.833+0000 7f35ebfff640 1 --2- 172.21.15.49:0/1240361744 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f35d40a2970 0x7f35d40a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:19:46.836 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:46.833+0000 7f35e97fa640 1 -- 172.21.15.49:0/1240361744 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 8942+0+0 (secure 0 0 0) 0x7f35e40cf150 con 0x7f35ec10be60 2024-03-18T23:19:46.836 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:46.833+0000 7f35ebfff640 1 --2- 172.21.15.49:0/1240361744 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f35d40a2970 0x7f35d40a4e30 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f35dc02efd0 tx=0x7f35dc02cf90 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:19:46.836 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:46.837+0000 7f35e97fa640 1 -- 172.21.15.49:0/1240361744 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f35e4016030 con 0x7f35ec10be60 2024-03-18T23:19:46.960 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:19:46 smithi175 ceph-mon[67352]: pgmap v360: 97 pgs: 97 active+clean; 61 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 75 KiB/s wr, 23 op/s 2024-03-18T23:19:46.960 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:19:46 smithi175 ceph-mon[67352]: from='client.? 172.21.15.49:0/2024149939' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T23:19:46.960 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:19:46 smithi175 ceph-mon[67352]: from='client.? 172.21.15.49:0/3821319232' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T23:19:46.961 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:19:46 smithi175 ceph-mon[67352]: from='client.? 172.21.15.49:0/3936305178' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T23:19:46.961 INFO:journalctl@ceph.mds.c.smithi175.stdout:Mar 18 23:19:46 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mds-c[98512]: ignoring --setuser ceph since I am not root 2024-03-18T23:19:46.961 INFO:journalctl@ceph.mds.c.smithi175.stdout:Mar 18 23:19:46 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mds-c[98512]: ignoring --setgroup ceph since I am not root 2024-03-18T23:19:46.961 INFO:journalctl@ceph.mds.c.smithi175.stdout:Mar 18 23:19:46 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mds-c[98512]: starting mds.c at 2024-03-18T23:19:46.962 INFO:journalctl@ceph.mds.f.smithi175.stdout:Mar 18 23:19:46 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mds-f[98204]: ignoring --setuser ceph since I am not root 2024-03-18T23:19:46.962 INFO:journalctl@ceph.mds.f.smithi175.stdout:Mar 18 23:19:46 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mds-f[98204]: ignoring --setgroup ceph since I am not root 2024-03-18T23:19:46.962 INFO:journalctl@ceph.mds.f.smithi175.stdout:Mar 18 23:19:46 smithi175 ceph-mds[98208]: -- 172.21.15.175:0/3574986815 <== mon.1 v2:172.21.15.175:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x557468a2da40 con 0x557468a00c00 2024-03-18T23:19:46.962 INFO:journalctl@ceph.mds.f.smithi175.stdout:Mar 18 23:19:46 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mds-f[98204]: starting mds.f at 2024-03-18T23:19:46.962 INFO:journalctl@ceph.mds.i.smithi175.stdout:Mar 18 23:19:46 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mds-i[98824]: ignoring --setuser ceph since I am not root 2024-03-18T23:19:46.963 INFO:journalctl@ceph.mds.i.smithi175.stdout:Mar 18 23:19:46 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mds-i[98824]: ignoring --setgroup ceph since I am not root 2024-03-18T23:19:46.963 INFO:journalctl@ceph.mds.i.smithi175.stdout:Mar 18 23:19:46 smithi175 ceph-mds[98828]: -- 172.21.15.175:0/405299974 <== mon.1 v2:172.21.15.175:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x564a4b9a3a40 con 0x564a4b977000 2024-03-18T23:19:46.963 INFO:journalctl@ceph.mds.i.smithi175.stdout:Mar 18 23:19:46 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mds-i[98824]: starting mds.i at 2024-03-18T23:19:46.963 INFO:journalctl@ceph.mds.l.smithi175.stdout:Mar 18 23:19:46 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mds-l[97880]: ignoring --setuser ceph since I am not root 2024-03-18T23:19:46.963 INFO:journalctl@ceph.mds.l.smithi175.stdout:Mar 18 23:19:46 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mds-l[97880]: ignoring --setgroup ceph since I am not root 2024-03-18T23:19:46.964 INFO:journalctl@ceph.mds.l.smithi175.stdout:Mar 18 23:19:46 smithi175 ceph-mds[97884]: -- 172.21.15.175:0/1008860589 <== mon.1 v2:172.21.15.175:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55edcb861a40 con 0x55edcb834c00 2024-03-18T23:19:46.964 INFO:journalctl@ceph.mds.l.smithi175.stdout:Mar 18 23:19:46 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mds-l[97880]: starting mds.l at 2024-03-18T23:19:47.060 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:47.061+0000 7f35f28a7640 1 -- 172.21.15.49:0/1240361744 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f35ec106fb0 con 0x7f35ec10be60 2024-03-18T23:19:47.129 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:19:46 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mon-a[62987]: 2024-03-18T23:19:46.695+0000 7f280de3a700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-18T23:19:47.129 INFO:journalctl@ceph.mds.a.smithi049.stdout:Mar 18 23:19:46 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mds-a[105002]: ignoring --setuser ceph since I am not root 2024-03-18T23:19:47.129 INFO:journalctl@ceph.mds.a.smithi049.stdout:Mar 18 23:19:46 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mds-a[105002]: ignoring --setgroup ceph since I am not root 2024-03-18T23:19:47.129 INFO:journalctl@ceph.mds.a.smithi049.stdout:Mar 18 23:19:46 smithi049 ceph-mds[105006]: -- 172.21.15.49:0/2501524915 <== mon.0 v2:172.21.15.49:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55edef365a40 con 0x55edef339000 2024-03-18T23:19:47.130 INFO:journalctl@ceph.mds.a.smithi049.stdout:Mar 18 23:19:46 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mds-a[105002]: starting mds.a at 2024-03-18T23:19:47.130 INFO:journalctl@ceph.mds.g.smithi049.stdout:Mar 18 23:19:46 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mds-g[104653]: ignoring --setuser ceph since I am not root 2024-03-18T23:19:47.130 INFO:journalctl@ceph.mds.g.smithi049.stdout:Mar 18 23:19:46 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mds-g[104653]: ignoring --setgroup ceph since I am not root 2024-03-18T23:19:47.130 INFO:journalctl@ceph.mds.g.smithi049.stdout:Mar 18 23:19:46 smithi049 ceph-mds[104657]: -- 172.21.15.49:0/3039532543 <== mon.0 v2:172.21.15.49:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f6d9f99a40 con 0x55f6d9f6c800 2024-03-18T23:19:47.130 INFO:journalctl@ceph.mds.g.smithi049.stdout:Mar 18 23:19:46 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mds-g[104653]: starting mds.g at 2024-03-18T23:19:47.700 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:47.701+0000 7f35e97fa640 1 -- 172.21.15.49:0/1240361744 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v34) ==== 104+0+0 (secure 0 0 0) 0x7f35e409b6c0 con 0x7f35ec10be60 2024-03-18T23:19:47.704 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:47.705+0000 7f35f28a7640 1 -- 172.21.15.49:0/1240361744 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f35d40a2970 msgr2=0x7f35d40a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:19:47.704 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:47.705+0000 7f35f28a7640 1 --2- 172.21.15.49:0/1240361744 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f35d40a2970 0x7f35d40a4e30 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f35dc02efd0 tx=0x7f35dc02cf90 comp rx=0 tx=0).stop 2024-03-18T23:19:47.704 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:47.705+0000 7f35f28a7640 1 -- 172.21.15.49:0/1240361744 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f35ec10be60 msgr2=0x7f35ec19ca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:19:47.704 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:47.705+0000 7f35f28a7640 1 --2- 172.21.15.49:0/1240361744 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f35ec10be60 0x7f35ec19ca70 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f35e400eaa0 tx=0x7f35e400ef60 comp rx=0 tx=0).stop 2024-03-18T23:19:47.705 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:47.705+0000 7f35f28a7640 1 -- 172.21.15.49:0/1240361744 shutdown_connections 2024-03-18T23:19:47.705 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:47.705+0000 7f35f28a7640 1 --2- 172.21.15.49:0/1240361744 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f35d40a2970 0x7f35d40a4e30 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:47.705 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:47.705+0000 7f35f28a7640 1 --2- 172.21.15.49:0/1240361744 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f35ec10be60 0x7f35ec19ca70 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:47.705 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:47.705+0000 7f35f28a7640 1 --2- 172.21.15.49:0/1240361744 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f35ec1033b0 0x7f35ec1986f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:47.705 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:47.705+0000 7f35f28a7640 1 --2- 172.21.15.49:0/1240361744 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f35ec102a90 0x7f35ec1981b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:47.705 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:47.705+0000 7f35f28a7640 1 -- 172.21.15.49:0/1240361744 >> 172.21.15.49:0/1240361744 conn(0x7f35ec077880 msgr2=0x7f35ec0fd8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:19:47.705 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:47.705+0000 7f35f28a7640 1 -- 172.21.15.49:0/1240361744 shutdown_connections 2024-03-18T23:19:47.705 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:47.705+0000 7f35f28a7640 1 -- 172.21.15.49:0/1240361744 wait complete. 2024-03-18T23:19:47.721 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-03-18T23:19:47.814 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:19:47 smithi139 ceph-mon[67282]: from='client.? 172.21.15.49:0/731017230' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-03-18T23:19:47.814 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:19:47 smithi139 ceph-mon[67282]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-18T23:19:47.814 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:19:47 smithi139 ceph-mon[67282]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-18T23:19:47.814 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:19:47 smithi139 ceph-mon[67282]: from='client.? 172.21.15.49:0/731017230' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-03-18T23:19:47.814 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:19:47 smithi139 ceph-mon[67282]: osdmap e102: 12 total, 12 up, 12 in 2024-03-18T23:19:47.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:19:47 smithi139 ceph-mon[67282]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-03-18T23:19:47.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:19:47 smithi139 ceph-mon[67282]: from='client.? 172.21.15.49:0/1240361744' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-03-18T23:19:47.844 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:47.842+0000 7fa43d87e640 1 -- 172.21.15.49:0/3458953914 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa438100740 msgr2=0x7fa438100bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:19:47.844 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:47.842+0000 7fa43d87e640 1 --2- 172.21.15.49:0/3458953914 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa438100740 0x7fa438100bc0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fa42c009930 tx=0x7fa42c02bc30 comp rx=0 tx=0).stop 2024-03-18T23:19:47.845 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:47.842+0000 7fa43d87e640 1 -- 172.21.15.49:0/3458953914 shutdown_connections 2024-03-18T23:19:47.845 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:47.842+0000 7fa43d87e640 1 --2- 172.21.15.49:0/3458953914 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa438101100 0x7fa43810d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:47.845 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:47.842+0000 7fa43d87e640 1 --2- 172.21.15.49:0/3458953914 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa438100740 0x7fa438100bc0 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:47.845 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:47.842+0000 7fa43d87e640 1 --2- 172.21.15.49:0/3458953914 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa438106750 0x7fa438106b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:47.845 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:47.842+0000 7fa43d87e640 1 -- 172.21.15.49:0/3458953914 >> 172.21.15.49:0/3458953914 conn(0x7fa438077880 msgr2=0x7fa438077c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:19:47.845 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:47.842+0000 7fa43d87e640 1 -- 172.21.15.49:0/3458953914 shutdown_connections 2024-03-18T23:19:47.845 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:47.842+0000 7fa43d87e640 1 -- 172.21.15.49:0/3458953914 wait complete. 2024-03-18T23:19:47.845 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:47.843+0000 7fa43d87e640 1 Processor -- start 2024-03-18T23:19:47.846 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:47.843+0000 7fa43d87e640 1 -- start start 2024-03-18T23:19:47.846 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:47.843+0000 7fa43d87e640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa438100740 0x7fa438198180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:19:47.846 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:47.843+0000 7fa43d87e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa438101100 0x7fa4381986c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:19:47.846 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:47.843+0000 7fa43d87e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa438106750 0x7fa43819ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:19:47.846 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:47.843+0000 7fa43d87e640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fa438111c60 con 0x7fa438101100 2024-03-18T23:19:47.846 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:47.843+0000 7fa43d87e640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fa438111ae0 con 0x7fa438106750 2024-03-18T23:19:47.846 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:47.843+0000 7fa43d87e640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa438111de0 con 0x7fa438100740 2024-03-18T23:19:47.846 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:47.844+0000 7fa4367fc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa438101100 0x7fa4381986c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:19:47.847 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:47.844+0000 7fa4367fc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa438101100 0x7fa4381986c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:41116/0 (socket says 172.21.15.49:41116) 2024-03-18T23:19:47.847 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:47.844+0000 7fa4367fc640 1 -- 172.21.15.49:0/1290420087 learned_addr learned my addr 172.21.15.49:0/1290420087 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:19:47.847 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:47.844+0000 7fa4377fe640 1 --2- 172.21.15.49:0/1290420087 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa438106750 0x7fa43819ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:19:47.847 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:47.844+0000 7fa436ffd640 1 --2- 172.21.15.49:0/1290420087 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa438100740 0x7fa438198180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:19:47.847 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:47.844+0000 7fa4367fc640 1 -- 172.21.15.49:0/1290420087 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa438100740 msgr2=0x7fa438198180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:19:47.847 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:47.844+0000 7fa4367fc640 1 --2- 172.21.15.49:0/1290420087 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa438100740 0x7fa438198180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:47.847 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:47.844+0000 7fa4367fc640 1 -- 172.21.15.49:0/1290420087 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa438106750 msgr2=0x7fa43819ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:19:47.847 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:47.844+0000 7fa4367fc640 1 --2- 172.21.15.49:0/1290420087 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa438106750 0x7fa43819ca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:47.848 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:47.844+0000 7fa4367fc640 1 -- 172.21.15.49:0/1290420087 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa43819d170 con 0x7fa438101100 2024-03-18T23:19:47.848 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:47.844+0000 7fa436ffd640 1 --2- 172.21.15.49:0/1290420087 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa438100740 0x7fa438198180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T23:19:47.848 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:47.844+0000 7fa4367fc640 1 --2- 172.21.15.49:0/1290420087 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa438101100 0x7fa4381986c0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fa42c009230 tx=0x7fa42c02cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:19:47.848 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:47.844+0000 7fa43c87c640 1 -- 172.21.15.49:0/1290420087 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa42c041070 con 0x7fa438101100 2024-03-18T23:19:47.848 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:47.844+0000 7fa43d87e640 1 -- 172.21.15.49:0/1290420087 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa43819d400 con 0x7fa438101100 2024-03-18T23:19:47.848 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:47.844+0000 7fa43c87c640 1 -- 172.21.15.49:0/1290420087 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa42c004450 con 0x7fa438101100 2024-03-18T23:19:47.848 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:47.844+0000 7fa43c87c640 1 -- 172.21.15.49:0/1290420087 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa42c034440 con 0x7fa438101100 2024-03-18T23:19:47.848 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:47.844+0000 7fa43d87e640 1 -- 172.21.15.49:0/1290420087 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4381a4cf0 con 0x7fa438101100 2024-03-18T23:19:47.848 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:47.846+0000 7fa43d87e640 1 -- 172.21.15.49:0/1290420087 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa438068a60 con 0x7fa438101100 2024-03-18T23:19:47.849 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:47.847+0000 7fa43c87c640 1 -- 172.21.15.49:0/1290420087 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 25) ==== 137421+0+0 (secure 0 0 0) 0x7fa42c044070 con 0x7fa438101100 2024-03-18T23:19:47.849 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:47.848+0000 7fa43c87c640 1 --2- 172.21.15.49:0/1290420087 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7fa4200a2970 0x7fa4200a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:19:47.850 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:47.848+0000 7fa436ffd640 1 --2- 172.21.15.49:0/1290420087 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7fa4200a2970 0x7fa4200a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:19:47.850 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:47.848+0000 7fa43c87c640 1 -- 172.21.15.49:0/1290420087 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 8942+0+0 (secure 0 0 0) 0x7fa42c0f0d50 con 0x7fa438101100 2024-03-18T23:19:47.850 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:47.848+0000 7fa436ffd640 1 --2- 172.21.15.49:0/1290420087 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7fa4200a2970 0x7fa4200a4e30 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fa428005fd0 tx=0x7fa428005850 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:19:47.850 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:47.852+0000 7fa43c87c640 1 -- 172.21.15.49:0/1290420087 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fa42c0bda20 con 0x7fa438101100 2024-03-18T23:19:47.878 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:19:47 smithi049 ceph-mon[62991]: from='client.? 172.21.15.49:0/731017230' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-03-18T23:19:47.879 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:19:47 smithi049 ceph-mon[62991]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-18T23:19:47.879 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:19:47 smithi049 ceph-mon[62991]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-18T23:19:47.879 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:19:47 smithi049 ceph-mon[62991]: from='client.? 172.21.15.49:0/731017230' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-03-18T23:19:47.879 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:19:47 smithi049 ceph-mon[62991]: osdmap e102: 12 total, 12 up, 12 in 2024-03-18T23:19:47.879 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:19:47 smithi049 ceph-mon[62991]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-03-18T23:19:47.879 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:19:47 smithi049 ceph-mon[62991]: from='client.? 172.21.15.49:0/1240361744' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-03-18T23:19:47.960 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:19:47 smithi175 ceph-mon[67352]: from='client.? 172.21.15.49:0/731017230' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-03-18T23:19:47.961 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:19:47 smithi175 ceph-mon[67352]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-18T23:19:47.961 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:19:47 smithi175 ceph-mon[67352]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-18T23:19:47.961 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:19:47 smithi175 ceph-mon[67352]: from='client.? 172.21.15.49:0/731017230' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-03-18T23:19:47.961 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:19:47 smithi175 ceph-mon[67352]: osdmap e102: 12 total, 12 up, 12 in 2024-03-18T23:19:47.961 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:19:47 smithi175 ceph-mon[67352]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-03-18T23:19:47.961 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:19:47 smithi175 ceph-mon[67352]: from='client.? 172.21.15.49:0/1240361744' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-03-18T23:19:48.035 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:48.037+0000 7fa43d87e640 1 -- 172.21.15.49:0/1290420087 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fa438108cd0 con 0x7fa438101100 2024-03-18T23:19:48.701 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:48.702+0000 7fa43c87c640 1 -- 172.21.15.49:0/1290420087 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v103) ==== 181+0+0 (secure 0 0 0) 0x7fa42c0dbe20 con 0x7fa438101100 2024-03-18T23:19:48.701 INFO:teuthology.orchestra.run.smithi049.stderr:pool 'cephfs_metadata' removed 2024-03-18T23:19:48.704 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:48.705+0000 7fa43d87e640 1 -- 172.21.15.49:0/1290420087 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7fa4200a2970 msgr2=0x7fa4200a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:19:48.704 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:48.705+0000 7fa43d87e640 1 --2- 172.21.15.49:0/1290420087 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7fa4200a2970 0x7fa4200a4e30 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fa428005fd0 tx=0x7fa428005850 comp rx=0 tx=0).stop 2024-03-18T23:19:48.704 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:48.705+0000 7fa43d87e640 1 -- 172.21.15.49:0/1290420087 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa438101100 msgr2=0x7fa4381986c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:19:48.704 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:48.705+0000 7fa43d87e640 1 --2- 172.21.15.49:0/1290420087 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa438101100 0x7fa4381986c0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fa42c009230 tx=0x7fa42c02cc70 comp rx=0 tx=0).stop 2024-03-18T23:19:48.705 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:48.706+0000 7fa43d87e640 1 -- 172.21.15.49:0/1290420087 shutdown_connections 2024-03-18T23:19:48.705 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:48.706+0000 7fa43d87e640 1 --2- 172.21.15.49:0/1290420087 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7fa4200a2970 0x7fa4200a4e30 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:48.706 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:48.706+0000 7fa43d87e640 1 --2- 172.21.15.49:0/1290420087 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa438106750 0x7fa43819ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:48.706 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:48.706+0000 7fa43d87e640 1 --2- 172.21.15.49:0/1290420087 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa438101100 0x7fa4381986c0 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:48.706 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:48.706+0000 7fa43d87e640 1 --2- 172.21.15.49:0/1290420087 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa438100740 0x7fa438198180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:48.706 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:48.706+0000 7fa43d87e640 1 -- 172.21.15.49:0/1290420087 >> 172.21.15.49:0/1290420087 conn(0x7fa438077880 msgr2=0x7fa438103370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:19:48.706 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:48.706+0000 7fa43d87e640 1 -- 172.21.15.49:0/1290420087 shutdown_connections 2024-03-18T23:19:48.706 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:48.707+0000 7fa43d87e640 1 -- 172.21.15.49:0/1290420087 wait complete. 2024-03-18T23:19:48.721 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-03-18T23:19:48.814 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:19:48 smithi139 ceph-mon[67282]: pgmap v362: 97 pgs: 97 active+clean; 61 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 35 KiB/s wr, 16 op/s 2024-03-18T23:19:48.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:19:48 smithi139 ceph-mon[67282]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-18T23:19:48.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:19:48 smithi139 ceph-mon[67282]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-03-18T23:19:48.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:19:48 smithi139 ceph-mon[67282]: Cluster is now healthy 2024-03-18T23:19:48.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:19:48 smithi139 ceph-mon[67282]: from='client.? 172.21.15.49:0/1240361744' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-03-18T23:19:48.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:19:48 smithi139 ceph-mon[67282]: mds.? [v2:172.21.15.49:6839/2861813009,v1:172.21.15.49:6841/2861813009] up:boot 2024-03-18T23:19:48.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:19:48 smithi139 ceph-mon[67282]: mds.? [v2:172.21.15.175:6832/2422898529,v1:172.21.15.175:6833/2422898529] up:boot 2024-03-18T23:19:48.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:19:48 smithi139 ceph-mon[67282]: mds.? [v2:172.21.15.175:6834/1888523118,v1:172.21.15.175:6835/1888523118] up:boot 2024-03-18T23:19:48.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:19:48 smithi139 ceph-mon[67282]: mds.? [v2:172.21.15.49:6838/1975149090,v1:172.21.15.49:6840/1975149090] up:boot 2024-03-18T23:19:48.816 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:19:48 smithi139 ceph-mon[67282]: mds.? [v2:172.21.15.139:6838/2631990057,v1:172.21.15.139:6839/2631990057] up:boot 2024-03-18T23:19:48.816 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:19:48 smithi139 ceph-mon[67282]: mds.? [v2:172.21.15.175:6836/3062638179,v1:172.21.15.175:6837/3062638179] up:boot 2024-03-18T23:19:48.816 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:19:48 smithi139 ceph-mon[67282]: mds.? [v2:172.21.15.175:6838/3259356585,v1:172.21.15.175:6839/3259356585] up:boot 2024-03-18T23:19:48.816 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:19:48 smithi139 ceph-mon[67282]: mds.? [v2:172.21.15.139:6832/3590969313,v1:172.21.15.139:6833/3590969313] up:boot 2024-03-18T23:19:48.816 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:19:48 smithi139 ceph-mon[67282]: mds.? [v2:172.21.15.139:6834/2319476031,v1:172.21.15.139:6835/2319476031] up:boot 2024-03-18T23:19:48.816 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:19:48 smithi139 ceph-mon[67282]: mds.? [v2:172.21.15.139:6836/114584912,v1:172.21.15.139:6837/114584912] up:boot 2024-03-18T23:19:48.816 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:19:48 smithi139 ceph-mon[67282]: fsmap 12 up:standby 2024-03-18T23:19:48.816 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:19:48 smithi139 ceph-mon[67282]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-18T23:19:48.816 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:19:48 smithi139 ceph-mon[67282]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-18T23:19:48.817 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:19:48 smithi139 ceph-mon[67282]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-18T23:19:48.817 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:19:48 smithi139 ceph-mon[67282]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-18T23:19:48.817 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:19:48 smithi139 ceph-mon[67282]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-18T23:19:48.817 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:19:48 smithi139 ceph-mon[67282]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-18T23:19:48.817 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:19:48 smithi139 ceph-mon[67282]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-18T23:19:48.817 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:19:48 smithi139 ceph-mon[67282]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-18T23:19:48.817 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:19:48 smithi139 ceph-mon[67282]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-18T23:19:48.817 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:19:48 smithi139 ceph-mon[67282]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-18T23:19:48.818 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:19:48 smithi139 ceph-mon[67282]: from='client.? 172.21.15.49:0/1290420087' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-18T23:19:48.838 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:48.837+0000 7f3222bc4640 1 -- 172.21.15.49:0/2362040695 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f321c100690 msgr2=0x7f321c100b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:19:48.838 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:48.837+0000 7f3222bc4640 1 --2- 172.21.15.49:0/2362040695 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f321c100690 0x7f321c100b10 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f3208009930 tx=0x7f320802bc30 comp rx=0 tx=0).stop 2024-03-18T23:19:48.838 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:48.837+0000 7f3222bc4640 1 -- 172.21.15.49:0/2362040695 shutdown_connections 2024-03-18T23:19:48.838 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:48.837+0000 7f3222bc4640 1 --2- 172.21.15.49:0/2362040695 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f321c101050 0x7f321c10d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:48.838 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:48.837+0000 7f3222bc4640 1 --2- 172.21.15.49:0/2362040695 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f321c100690 0x7f321c100b10 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:48.839 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:48.837+0000 7f3222bc4640 1 --2- 172.21.15.49:0/2362040695 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f321c1066a0 0x7f321c106a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:48.839 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:48.837+0000 7f3222bc4640 1 -- 172.21.15.49:0/2362040695 >> 172.21.15.49:0/2362040695 conn(0x7f321c077820 msgr2=0x7f321c077c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:19:48.839 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:48.837+0000 7f3222bc4640 1 -- 172.21.15.49:0/2362040695 shutdown_connections 2024-03-18T23:19:48.839 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:48.837+0000 7f3222bc4640 1 -- 172.21.15.49:0/2362040695 wait complete. 2024-03-18T23:19:48.839 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:48.837+0000 7f3222bc4640 1 Processor -- start 2024-03-18T23:19:48.839 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:48.837+0000 7f3222bc4640 1 -- start start 2024-03-18T23:19:48.839 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:48.838+0000 7f3222bc4640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f321c100690 0x7f321c197000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:19:48.839 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:48.838+0000 7f3222bc4640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f321c101050 0x7f321c197540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:19:48.839 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:48.838+0000 7f3222bc4640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f321c1066a0 0x7f321c19b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:19:48.840 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:48.838+0000 7f3222bc4640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f321c110b00 con 0x7f321c100690 2024-03-18T23:19:48.840 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:48.838+0000 7f3222bc4640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f321c110980 con 0x7f321c101050 2024-03-18T23:19:48.840 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:48.838+0000 7f3222bc4640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f321c110c80 con 0x7f321c1066a0 2024-03-18T23:19:48.840 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:48.838+0000 7f3220960640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f321c100690 0x7f321c197000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:19:48.840 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:48.838+0000 7f3220960640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f321c100690 0x7f321c197000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:41140/0 (socket says 172.21.15.49:41140) 2024-03-18T23:19:48.840 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:48.838+0000 7f3220960640 1 -- 172.21.15.49:0/2397535019 learned_addr learned my addr 172.21.15.49:0/2397535019 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-18T23:19:48.840 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:48.838+0000 7f321bfff640 1 --2- 172.21.15.49:0/2397535019 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f321c101050 0x7f321c197540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:19:48.840 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:48.838+0000 7f3221161640 1 --2- 172.21.15.49:0/2397535019 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f321c1066a0 0x7f321c19b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:19:48.841 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:48.838+0000 7f3220960640 1 -- 172.21.15.49:0/2397535019 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f321c1066a0 msgr2=0x7f321c19b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:19:48.841 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:48.838+0000 7f3220960640 1 --2- 172.21.15.49:0/2397535019 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f321c1066a0 0x7f321c19b8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:48.841 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:48.838+0000 7f3220960640 1 -- 172.21.15.49:0/2397535019 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f321c101050 msgr2=0x7f321c197540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:19:48.841 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:48.838+0000 7f3220960640 1 --2- 172.21.15.49:0/2397535019 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f321c101050 0x7f321c197540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:48.841 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:48.838+0000 7f3220960640 1 -- 172.21.15.49:0/2397535019 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f321c19bff0 con 0x7f321c100690 2024-03-18T23:19:48.841 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:48.838+0000 7f3221161640 1 --2- 172.21.15.49:0/2397535019 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f321c1066a0 0x7f321c19b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T23:19:48.841 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:48.838+0000 7f3220960640 1 --2- 172.21.15.49:0/2397535019 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f321c100690 0x7f321c197000 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f321000d950 tx=0x7f321000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:19:48.841 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:48.838+0000 7f3219ffb640 1 -- 172.21.15.49:0/2397535019 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3210017070 con 0x7f321c100690 2024-03-18T23:19:48.842 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:48.838+0000 7f3222bc4640 1 -- 172.21.15.49:0/2397535019 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f321c19c2e0 con 0x7f321c100690 2024-03-18T23:19:48.842 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:48.838+0000 7f3219ffb640 1 -- 172.21.15.49:0/2397535019 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3210009c30 con 0x7f321c100690 2024-03-18T23:19:48.842 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:48.838+0000 7f3219ffb640 1 -- 172.21.15.49:0/2397535019 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f321000f6b0 con 0x7f321c100690 2024-03-18T23:19:48.842 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:48.839+0000 7f3222bc4640 1 -- 172.21.15.49:0/2397535019 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f321c1a3b70 con 0x7f321c100690 2024-03-18T23:19:48.843 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:48.840+0000 7f3222bc4640 1 -- 172.21.15.49:0/2397535019 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f321c068610 con 0x7f321c100690 2024-03-18T23:19:48.843 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:48.841+0000 7f3219ffb640 1 -- 172.21.15.49:0/2397535019 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 25) ==== 137421+0+0 (secure 0 0 0) 0x7f32100109d0 con 0x7f321c100690 2024-03-18T23:19:48.843 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:48.842+0000 7f3219ffb640 1 --2- 172.21.15.49:0/2397535019 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f320c0a2970 0x7f320c0a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:19:48.844 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:48.842+0000 7f321bfff640 1 --2- 172.21.15.49:0/2397535019 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f320c0a2970 0x7f320c0a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:19:48.844 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:48.842+0000 7f3219ffb640 1 -- 172.21.15.49:0/2397535019 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 8388+0+0 (secure 0 0 0) 0x7f32100cea50 con 0x7f321c100690 2024-03-18T23:19:48.844 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:48.843+0000 7f321bfff640 1 --2- 172.21.15.49:0/2397535019 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f320c0a2970 0x7f320c0a4e30 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f3208009900 tx=0x7f320802df90 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:19:48.846 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:48.847+0000 7f3219ffb640 1 -- 172.21.15.49:0/2397535019 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f3210009de0 con 0x7f321c100690 2024-03-18T23:19:48.879 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:19:48 smithi049 ceph-mon[62991]: pgmap v362: 97 pgs: 97 active+clean; 61 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 35 KiB/s wr, 16 op/s 2024-03-18T23:19:48.879 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:19:48 smithi049 ceph-mon[62991]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-18T23:19:48.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:19:48 smithi049 ceph-mon[62991]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-03-18T23:19:48.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:19:48 smithi049 ceph-mon[62991]: Cluster is now healthy 2024-03-18T23:19:48.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:19:48 smithi049 ceph-mon[62991]: from='client.? 172.21.15.49:0/1240361744' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-03-18T23:19:48.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:19:48 smithi049 ceph-mon[62991]: mds.? [v2:172.21.15.49:6839/2861813009,v1:172.21.15.49:6841/2861813009] up:boot 2024-03-18T23:19:48.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:19:48 smithi049 ceph-mon[62991]: mds.? [v2:172.21.15.175:6832/2422898529,v1:172.21.15.175:6833/2422898529] up:boot 2024-03-18T23:19:48.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:19:48 smithi049 ceph-mon[62991]: mds.? [v2:172.21.15.175:6834/1888523118,v1:172.21.15.175:6835/1888523118] up:boot 2024-03-18T23:19:48.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:19:48 smithi049 ceph-mon[62991]: mds.? [v2:172.21.15.49:6838/1975149090,v1:172.21.15.49:6840/1975149090] up:boot 2024-03-18T23:19:48.881 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:19:48 smithi049 ceph-mon[62991]: mds.? [v2:172.21.15.139:6838/2631990057,v1:172.21.15.139:6839/2631990057] up:boot 2024-03-18T23:19:48.881 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:19:48 smithi049 ceph-mon[62991]: mds.? [v2:172.21.15.175:6836/3062638179,v1:172.21.15.175:6837/3062638179] up:boot 2024-03-18T23:19:48.881 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:19:48 smithi049 ceph-mon[62991]: mds.? [v2:172.21.15.175:6838/3259356585,v1:172.21.15.175:6839/3259356585] up:boot 2024-03-18T23:19:48.881 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:19:48 smithi049 ceph-mon[62991]: mds.? [v2:172.21.15.139:6832/3590969313,v1:172.21.15.139:6833/3590969313] up:boot 2024-03-18T23:19:48.881 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:19:48 smithi049 ceph-mon[62991]: mds.? [v2:172.21.15.139:6834/2319476031,v1:172.21.15.139:6835/2319476031] up:boot 2024-03-18T23:19:48.881 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:19:48 smithi049 ceph-mon[62991]: mds.? [v2:172.21.15.139:6836/114584912,v1:172.21.15.139:6837/114584912] up:boot 2024-03-18T23:19:48.881 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:19:48 smithi049 ceph-mon[62991]: fsmap 12 up:standby 2024-03-18T23:19:48.881 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:19:48 smithi049 ceph-mon[62991]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-18T23:19:48.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:19:48 smithi049 ceph-mon[62991]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-18T23:19:48.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:19:48 smithi049 ceph-mon[62991]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-18T23:19:48.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:19:48 smithi049 ceph-mon[62991]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-18T23:19:48.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:19:48 smithi049 ceph-mon[62991]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-18T23:19:48.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:19:48 smithi049 ceph-mon[62991]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-18T23:19:48.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:19:48 smithi049 ceph-mon[62991]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-18T23:19:48.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:19:48 smithi049 ceph-mon[62991]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-18T23:19:48.883 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:19:48 smithi049 ceph-mon[62991]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-18T23:19:48.883 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:19:48 smithi049 ceph-mon[62991]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-18T23:19:48.883 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:19:48 smithi049 ceph-mon[62991]: from='client.? 172.21.15.49:0/1290420087' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-18T23:19:48.960 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:19:48 smithi175 ceph-mon[67352]: pgmap v362: 97 pgs: 97 active+clean; 61 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 35 KiB/s wr, 16 op/s 2024-03-18T23:19:48.961 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:19:48 smithi175 ceph-mon[67352]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-18T23:19:48.961 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:19:48 smithi175 ceph-mon[67352]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-03-18T23:19:48.961 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:19:48 smithi175 ceph-mon[67352]: Cluster is now healthy 2024-03-18T23:19:48.961 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:19:48 smithi175 ceph-mon[67352]: from='client.? 172.21.15.49:0/1240361744' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-03-18T23:19:48.961 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:19:48 smithi175 ceph-mon[67352]: mds.? [v2:172.21.15.49:6839/2861813009,v1:172.21.15.49:6841/2861813009] up:boot 2024-03-18T23:19:48.961 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:19:48 smithi175 ceph-mon[67352]: mds.? [v2:172.21.15.175:6832/2422898529,v1:172.21.15.175:6833/2422898529] up:boot 2024-03-18T23:19:48.961 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:19:48 smithi175 ceph-mon[67352]: mds.? [v2:172.21.15.175:6834/1888523118,v1:172.21.15.175:6835/1888523118] up:boot 2024-03-18T23:19:48.962 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:19:48 smithi175 ceph-mon[67352]: mds.? [v2:172.21.15.49:6838/1975149090,v1:172.21.15.49:6840/1975149090] up:boot 2024-03-18T23:19:48.962 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:19:48 smithi175 ceph-mon[67352]: mds.? [v2:172.21.15.139:6838/2631990057,v1:172.21.15.139:6839/2631990057] up:boot 2024-03-18T23:19:48.962 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:19:48 smithi175 ceph-mon[67352]: mds.? [v2:172.21.15.175:6836/3062638179,v1:172.21.15.175:6837/3062638179] up:boot 2024-03-18T23:19:48.962 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:19:48 smithi175 ceph-mon[67352]: mds.? [v2:172.21.15.175:6838/3259356585,v1:172.21.15.175:6839/3259356585] up:boot 2024-03-18T23:19:48.962 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:19:48 smithi175 ceph-mon[67352]: mds.? [v2:172.21.15.139:6832/3590969313,v1:172.21.15.139:6833/3590969313] up:boot 2024-03-18T23:19:48.962 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:19:48 smithi175 ceph-mon[67352]: mds.? [v2:172.21.15.139:6834/2319476031,v1:172.21.15.139:6835/2319476031] up:boot 2024-03-18T23:19:48.962 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:19:48 smithi175 ceph-mon[67352]: mds.? [v2:172.21.15.139:6836/114584912,v1:172.21.15.139:6837/114584912] up:boot 2024-03-18T23:19:48.962 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:19:48 smithi175 ceph-mon[67352]: fsmap 12 up:standby 2024-03-18T23:19:48.963 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:19:48 smithi175 ceph-mon[67352]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-18T23:19:48.963 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:19:48 smithi175 ceph-mon[67352]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-18T23:19:48.963 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:19:48 smithi175 ceph-mon[67352]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-18T23:19:48.963 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:19:48 smithi175 ceph-mon[67352]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-18T23:19:48.963 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:19:48 smithi175 ceph-mon[67352]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-18T23:19:48.963 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:19:48 smithi175 ceph-mon[67352]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-18T23:19:48.963 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:19:48 smithi175 ceph-mon[67352]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-18T23:19:48.963 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:19:48 smithi175 ceph-mon[67352]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-18T23:19:48.963 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:19:48 smithi175 ceph-mon[67352]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-18T23:19:48.964 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:19:48 smithi175 ceph-mon[67352]: from='mgr.15366 172.21.15.49:0/1719131904' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-18T23:19:48.964 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:19:48 smithi175 ceph-mon[67352]: from='client.? 172.21.15.49:0/1290420087' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-18T23:19:49.039 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:49.040+0000 7f3222bc4640 1 -- 172.21.15.49:0/2397535019 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f321c108c60 con 0x7f321c100690 2024-03-18T23:19:49.707 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:49.708+0000 7f3219ffb640 1 -- 172.21.15.49:0/2397535019 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v104) ==== 169+0+0 (secure 0 0 0) 0x7f321009b830 con 0x7f321c100690 2024-03-18T23:19:49.707 INFO:teuthology.orchestra.run.smithi049.stderr:pool 'cephfs_data' removed 2024-03-18T23:19:49.710 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:49.711+0000 7f3222bc4640 1 -- 172.21.15.49:0/2397535019 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f320c0a2970 msgr2=0x7f320c0a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:19:49.711 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:49.711+0000 7f3222bc4640 1 --2- 172.21.15.49:0/2397535019 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f320c0a2970 0x7f320c0a4e30 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f3208009900 tx=0x7f320802df90 comp rx=0 tx=0).stop 2024-03-18T23:19:49.711 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:49.711+0000 7f3222bc4640 1 -- 172.21.15.49:0/2397535019 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f321c100690 msgr2=0x7f321c197000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:19:49.711 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:49.711+0000 7f3222bc4640 1 --2- 172.21.15.49:0/2397535019 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f321c100690 0x7f321c197000 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f321000d950 tx=0x7f321000de10 comp rx=0 tx=0).stop 2024-03-18T23:19:49.711 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:49.712+0000 7f3222bc4640 1 -- 172.21.15.49:0/2397535019 shutdown_connections 2024-03-18T23:19:49.711 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:49.712+0000 7f3222bc4640 1 --2- 172.21.15.49:0/2397535019 >> [v2:172.21.15.49:6800/1112476276,v1:172.21.15.49:6801/1112476276] conn(0x7f320c0a2970 0x7f320c0a4e30 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:49.711 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:49.712+0000 7f3222bc4640 1 --2- 172.21.15.49:0/2397535019 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f321c1066a0 0x7f321c19b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:49.711 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:49.712+0000 7f3222bc4640 1 --2- 172.21.15.49:0/2397535019 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f321c101050 0x7f321c197540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:49.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:49.712+0000 7f3222bc4640 1 --2- 172.21.15.49:0/2397535019 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f321c100690 0x7f321c197000 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:49.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:49.712+0000 7f3222bc4640 1 -- 172.21.15.49:0/2397535019 >> 172.21.15.49:0/2397535019 conn(0x7f321c077820 msgr2=0x7f321c102ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:19:49.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:49.712+0000 7f3222bc4640 1 -- 172.21.15.49:0/2397535019 shutdown_connections 2024-03-18T23:19:49.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-18T23:19:49.712+0000 7f3222bc4640 1 -- 172.21.15.49:0/2397535019 wait complete. 2024-03-18T23:19:49.731 DEBUG:teuthology.orchestra.run.smithi049:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-18T23:19:49.775 DEBUG:teuthology.orchestra.run.smithi139:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-18T23:19:49.815 DEBUG:teuthology.orchestra.run.smithi175:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-18T23:19:49.845 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-03-18T23:19:49.845 DEBUG:teuthology.orchestra.run.smithi049:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-03-18T23:19:49.860 DEBUG:teuthology.orchestra.run.smithi139:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-03-18T23:19:49.879 DEBUG:teuthology.orchestra.run.smithi175:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-03-18T23:19:49.900 INFO:tasks.cephadm:Stopping all daemons... 2024-03-18T23:19:49.900 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-03-18T23:19:49.901 DEBUG:teuthology.orchestra.run.smithi049:> sudo systemctl stop ceph-c63591ea-e57a-11ee-95c9-87774f69a715@mon.a 2024-03-18T23:19:49.960 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:19:49 smithi175 ceph-mon[67352]: from='client.? 172.21.15.49:0/1290420087' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-03-18T23:19:49.960 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:19:49 smithi175 ceph-mon[67352]: osdmap e103: 12 total, 12 up, 12 in 2024-03-18T23:19:49.960 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:19:49 smithi175 ceph-mon[67352]: from='client.? 172.21.15.49:0/2397535019' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-18T23:19:50.060 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:19:49 smithi139 ceph-mon[67282]: from='client.? 172.21.15.49:0/1290420087' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-03-18T23:19:50.061 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:19:49 smithi139 ceph-mon[67282]: osdmap e103: 12 total, 12 up, 12 in 2024-03-18T23:19:50.061 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:19:49 smithi139 ceph-mon[67282]: from='client.? 172.21.15.49:0/2397535019' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-18T23:19:50.129 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:19:49 smithi049 ceph-mon[62991]: from='client.? 172.21.15.49:0/1290420087' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-03-18T23:19:50.129 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:19:49 smithi049 ceph-mon[62991]: osdmap e103: 12 total, 12 up, 12 in 2024-03-18T23:19:50.129 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:19:49 smithi049 ceph-mon[62991]: from='client.? 172.21.15.49:0/2397535019' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-18T23:19:50.129 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:19:49 smithi049 systemd[1]: Stopping Ceph mon.a for c63591ea-e57a-11ee-95c9-87774f69a715... 2024-03-18T23:19:50.629 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:19:50 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mon-a[62987]: 2024-03-18T23:19:50.243+0000 7f2813645700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-18T23:19:50.629 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:19:50 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mon-a[62987]: 2024-03-18T23:19:50.243+0000 7f2813645700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-03-18T23:19:50.629 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:19:50 smithi049 podman[123615]: 2024-03-18 23:19:50.356202798 +0000 UTC m=+0.245929582 container died 8521c70f9f18562cf8f0bcd4a0490e538a41d90c32424c25d0d78aa66254e35e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810, name=ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mon-a, org.label-schema.build-date=20240307, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-03-18T23:19:50.993 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:19:50 smithi049 podman[123615]: 2024-03-18 23:19:50.759957151 +0000 UTC m=+0.649683928 container cleanup 8521c70f9f18562cf8f0bcd4a0490e538a41d90c32424c25d0d78aa66254e35e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810, name=ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mon-a, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True) 2024-03-18T23:19:50.994 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 18 23:19:50 smithi049 bash[123615]: ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mon-a 2024-03-18T23:19:51.141 DEBUG:teuthology.orchestra.run.smithi049:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c63591ea-e57a-11ee-95c9-87774f69a715@mon.a.service' 2024-03-18T23:19:51.200 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-18T23:19:51.200 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-03-18T23:19:51.200 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-03-18T23:19:51.201 DEBUG:teuthology.orchestra.run.smithi139:> sudo systemctl stop ceph-c63591ea-e57a-11ee-95c9-87774f69a715@mon.b 2024-03-18T23:19:51.314 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:19:51 smithi139 systemd[1]: Stopping Ceph mon.b for c63591ea-e57a-11ee-95c9-87774f69a715... 2024-03-18T23:19:51.814 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:19:51 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mon-b[67278]: 2024-03-18T23:19:51.518+0000 7ff6d0150700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-18T23:19:51.814 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:19:51 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mon-b[67278]: 2024-03-18T23:19:51.518+0000 7ff6d0150700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-03-18T23:19:51.814 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:19:51 smithi139 podman[104247]: 2024-03-18 23:19:51.539526021 +0000 UTC m=+0.154074367 container died 127fbf6c9088fad00e275208d3b9999915d73439c03ce4c63e2469f97930981b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mon-b, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, ceph=True) 2024-03-18T23:19:52.305 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:19:52 smithi139 podman[104247]: 2024-03-18 23:19:52.005063099 +0000 UTC m=+0.619611444 container cleanup 127fbf6c9088fad00e275208d3b9999915d73439c03ce4c63e2469f97930981b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mon-b, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, GIT_CLEAN=True) 2024-03-18T23:19:52.306 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 18 23:19:52 smithi139 bash[104247]: ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mon-b 2024-03-18T23:19:52.428 DEBUG:teuthology.orchestra.run.smithi139:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c63591ea-e57a-11ee-95c9-87774f69a715@mon.b.service' 2024-03-18T23:19:52.503 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-18T23:19:52.503 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-03-18T23:19:52.503 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-03-18T23:19:52.504 DEBUG:teuthology.orchestra.run.smithi175:> sudo systemctl stop ceph-c63591ea-e57a-11ee-95c9-87774f69a715@mon.c 2024-03-18T23:19:52.828 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:19:52 smithi175 systemd[1]: Stopping Ceph mon.c for c63591ea-e57a-11ee-95c9-87774f69a715... 2024-03-18T23:19:53.200 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:19:52 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mon-c[67334]: 2024-03-18T23:19:52.828+0000 7f3528b5c700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-18T23:19:53.200 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:19:52 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mon-c[67334]: 2024-03-18T23:19:52.828+0000 7f3528b5c700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-03-18T23:19:53.201 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:19:53 smithi175 podman[104251]: 2024-03-18 23:19:53.023074624 +0000 UTC m=+0.323838338 container died 6f8dd756d02262224f9346bcaf6cb8b0f9253df9e461be56e3b37fc85425d92d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mon-c, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-03-18T23:19:53.790 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:19:53 smithi175 podman[104251]: 2024-03-18 23:19:53.538489109 +0000 UTC m=+0.839252804 container cleanup 6f8dd756d02262224f9346bcaf6cb8b0f9253df9e461be56e3b37fc85425d92d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mon-c, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240307) 2024-03-18T23:19:53.790 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:19:53 smithi175 bash[104251]: ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mon-c 2024-03-18T23:19:54.015 DEBUG:teuthology.orchestra.run.smithi175:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c63591ea-e57a-11ee-95c9-87774f69a715@mon.c.service' 2024-03-18T23:19:54.040 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:19:53 smithi175 podman[104265]: 2024-03-18 23:19:53.790464392 +0000 UTC m=+0.766790362 container remove 6f8dd756d02262224f9346bcaf6cb8b0f9253df9e461be56e3b37fc85425d92d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mon-c, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True) 2024-03-18T23:19:54.040 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:19:54 smithi175 systemd[1]: ceph-c63591ea-e57a-11ee-95c9-87774f69a715@mon.c.service: Deactivated successfully. 2024-03-18T23:19:54.040 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:19:54 smithi175 systemd[1]: Stopped Ceph mon.c for c63591ea-e57a-11ee-95c9-87774f69a715. 2024-03-18T23:19:54.040 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 18 23:19:54 smithi175 systemd[1]: ceph-c63591ea-e57a-11ee-95c9-87774f69a715@mon.c.service: Consumed 11.931s CPU time. 2024-03-18T23:19:54.061 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-18T23:19:54.061 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-03-18T23:19:54.061 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-03-18T23:19:54.062 DEBUG:teuthology.orchestra.run.smithi049:> sudo systemctl stop ceph-c63591ea-e57a-11ee-95c9-87774f69a715@mgr.x 2024-03-18T23:19:54.364 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 18 23:19:54 smithi049 systemd[1]: Stopping Ceph mgr.x for c63591ea-e57a-11ee-95c9-87774f69a715... 2024-03-18T23:19:54.629 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 18 23:19:54 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-x[63224]: 2024-03-18T23:19:54.364+0000 7f2e17ec9700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-18T23:19:54.629 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 18 23:19:54 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-x[63224]: 2024-03-18T23:19:54.364+0000 7f2e17ec9700 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2024-03-18T23:19:54.629 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 18 23:19:54 smithi049 podman[123736]: 2024-03-18 23:19:54.395838633 +0000 UTC m=+0.121240632 container died 75469e5b5506479dba1c6e29cc22757cfe9aa72093836210b62226b7d8c841a9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810, name=ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-x, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-18T23:19:55.066 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 18 23:19:54 smithi049 podman[123736]: 2024-03-18 23:19:54.824280191 +0000 UTC m=+0.549682182 container cleanup 75469e5b5506479dba1c6e29cc22757cfe9aa72093836210b62226b7d8c841a9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810, name=ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-x, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-18T23:19:55.066 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 18 23:19:54 smithi049 bash[123736]: ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-x 2024-03-18T23:19:55.307 DEBUG:teuthology.orchestra.run.smithi049:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c63591ea-e57a-11ee-95c9-87774f69a715@mgr.x.service' 2024-03-18T23:19:55.332 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 18 23:19:55 smithi049 podman[123750]: 2024-03-18 23:19:55.067969019 +0000 UTC m=+0.667785294 container remove 75469e5b5506479dba1c6e29cc22757cfe9aa72093836210b62226b7d8c841a9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810, name=ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-x, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-03-18T23:19:55.332 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 18 23:19:55 smithi049 systemd[1]: ceph-c63591ea-e57a-11ee-95c9-87774f69a715@mgr.x.service: Deactivated successfully. 2024-03-18T23:19:55.333 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 18 23:19:55 smithi049 systemd[1]: Stopped Ceph mgr.x for c63591ea-e57a-11ee-95c9-87774f69a715. 2024-03-18T23:19:55.333 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 18 23:19:55 smithi049 systemd[1]: ceph-c63591ea-e57a-11ee-95c9-87774f69a715@mgr.x.service: Consumed 1min 25.654s CPU time. 2024-03-18T23:19:55.350 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-18T23:19:55.350 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-03-18T23:19:55.350 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-03-18T23:19:55.350 DEBUG:teuthology.orchestra.run.smithi139:> sudo systemctl stop ceph-c63591ea-e57a-11ee-95c9-87774f69a715@mgr.y 2024-03-18T23:19:55.687 INFO:journalctl@ceph.mgr.y.smithi139.stdout:Mar 18 23:19:55 smithi139 systemd[1]: Stopping Ceph mgr.y for c63591ea-e57a-11ee-95c9-87774f69a715... 2024-03-18T23:19:55.977 INFO:journalctl@ceph.mgr.y.smithi139.stdout:Mar 18 23:19:55 smithi139 podman[104368]: 2024-03-18 23:19:55.688859546 +0000 UTC m=+0.145685901 container died 03a4539507343a2b75c8f0163a6e2b6e6440c56f827341e9f37ca854a769851d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-y, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-18T23:19:56.314 INFO:journalctl@ceph.mgr.y.smithi139.stdout:Mar 18 23:19:56 smithi139 podman[104368]: 2024-03-18 23:19:56.085365517 +0000 UTC m=+0.542191882 container cleanup 03a4539507343a2b75c8f0163a6e2b6e6440c56f827341e9f37ca854a769851d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-y, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-03-18T23:19:56.314 INFO:journalctl@ceph.mgr.y.smithi139.stdout:Mar 18 23:19:56 smithi139 bash[104368]: ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-y 2024-03-18T23:19:56.501 DEBUG:teuthology.orchestra.run.smithi139:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c63591ea-e57a-11ee-95c9-87774f69a715@mgr.y.service' 2024-03-18T23:19:56.539 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-18T23:19:56.539 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-03-18T23:19:56.539 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-03-18T23:19:56.539 DEBUG:teuthology.orchestra.run.smithi175:> sudo systemctl stop ceph-c63591ea-e57a-11ee-95c9-87774f69a715@mgr.z 2024-03-18T23:19:56.881 INFO:journalctl@ceph.mgr.z.smithi175.stdout:Mar 18 23:19:56 smithi175 systemd[1]: Stopping Ceph mgr.z for c63591ea-e57a-11ee-95c9-87774f69a715... 2024-03-18T23:19:57.177 INFO:journalctl@ceph.mgr.z.smithi175.stdout:Mar 18 23:19:56 smithi175 podman[104374]: 2024-03-18 23:19:56.881414874 +0000 UTC m=+0.151405016 container died 8a53c65637956d8964995800b50d22027ead645be69b2bfc84eaac7264e09068 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-z, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-03-18T23:19:57.460 INFO:journalctl@ceph.mgr.z.smithi175.stdout:Mar 18 23:19:57 smithi175 podman[104374]: 2024-03-18 23:19:57.277803753 +0000 UTC m=+0.547793878 container cleanup 8a53c65637956d8964995800b50d22027ead645be69b2bfc84eaac7264e09068 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-z, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-03-18T23:19:57.460 INFO:journalctl@ceph.mgr.z.smithi175.stdout:Mar 18 23:19:57 smithi175 bash[104374]: ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mgr-z 2024-03-18T23:19:57.699 DEBUG:teuthology.orchestra.run.smithi175:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c63591ea-e57a-11ee-95c9-87774f69a715@mgr.z.service' 2024-03-18T23:19:57.735 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-18T23:19:57.735 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-03-18T23:19:57.736 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-03-18T23:19:57.736 DEBUG:teuthology.orchestra.run.smithi049:> sudo systemctl stop ceph-c63591ea-e57a-11ee-95c9-87774f69a715@osd.0 2024-03-18T23:19:58.070 INFO:journalctl@ceph.osd.0.smithi049.stdout:Mar 18 23:19:57 smithi049 systemd[1]: Stopping Ceph osd.0 for c63591ea-e57a-11ee-95c9-87774f69a715... 2024-03-18T23:19:58.379 INFO:journalctl@ceph.osd.0.smithi049.stdout:Mar 18 23:19:58 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-0[77740]: 2024-03-18T23:19:58.071+0000 7fc4958e8700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-18T23:19:58.379 INFO:journalctl@ceph.osd.0.smithi049.stdout:Mar 18 23:19:58 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-0[77740]: 2024-03-18T23:19:58.071+0000 7fc4958e8700 -1 osd.0 104 *** Got signal Terminated *** 2024-03-18T23:19:58.379 INFO:journalctl@ceph.osd.0.smithi049.stdout:Mar 18 23:19:58 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-0[77740]: 2024-03-18T23:19:58.071+0000 7fc4958e8700 -1 osd.0 104 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-18T23:20:03.854 INFO:journalctl@ceph.osd.0.smithi049.stdout:Mar 18 23:20:03 smithi049 podman[123858]: 2024-03-18 23:20:03.583727516 +0000 UTC m=+5.638756009 container died 4f7da08d94bda4edbf54d2a0141b5b486fa6220cf5274888296dca7601d654c0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True) 2024-03-18T23:20:04.129 INFO:journalctl@ceph.osd.0.smithi049.stdout:Mar 18 23:20:03 smithi049 podman[123858]: 2024-03-18 23:20:03.946611816 +0000 UTC m=+6.001640317 container cleanup 4f7da08d94bda4edbf54d2a0141b5b486fa6220cf5274888296dca7601d654c0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-03-18T23:20:04.129 INFO:journalctl@ceph.osd.0.smithi049.stdout:Mar 18 23:20:03 smithi049 bash[123858]: ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-0 2024-03-18T23:20:04.622 INFO:journalctl@ceph.osd.0.smithi049.stdout:Mar 18 23:20:04 smithi049 podman[123876]: 2024-03-18 23:20:04.256641903 +0000 UTC m=+0.667535538 container remove 4f7da08d94bda4edbf54d2a0141b5b486fa6220cf5274888296dca7601d654c0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-0, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-18T23:20:04.879 INFO:journalctl@ceph.osd.0.smithi049.stdout:Mar 18 23:20:04 smithi049 podman[123966]: 2024-03-18 23:20:04.525894677 +0000 UTC m=+0.017530372 image pull 2312c8e42d0069a2dbb2364bddb00264db4b800e7f18a7e7e4eb8f3ecab2ff1c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639 2024-03-18T23:20:04.879 INFO:journalctl@ceph.osd.0.smithi049.stdout:Mar 18 23:20:04 smithi049 podman[123966]: 2024-03-18 23:20:04.65747381 +0000 UTC m=+0.149109498 container create 4a5b4c781969065b34c01e5eef449b46a943d5fe76736913e9ae2f73ab83f572 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-0-deactivate, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-03-18T23:20:05.234 INFO:journalctl@ceph.osd.0.smithi049.stdout:Mar 18 23:20:04 smithi049 podman[123966]: 2024-03-18 23:20:04.927531496 +0000 UTC m=+0.419167193 container init 4a5b4c781969065b34c01e5eef449b46a943d5fe76736913e9ae2f73ab83f572 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-0-deactivate, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2) 2024-03-18T23:20:05.235 INFO:journalctl@ceph.osd.0.smithi049.stdout:Mar 18 23:20:04 smithi049 podman[123966]: 2024-03-18 23:20:04.93103081 +0000 UTC m=+0.422666508 container start 4a5b4c781969065b34c01e5eef449b46a943d5fe76736913e9ae2f73ab83f572 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-0-deactivate, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-03-18T23:20:05.235 INFO:journalctl@ceph.osd.0.smithi049.stdout:Mar 18 23:20:04 smithi049 podman[123966]: 2024-03-18 23:20:04.994048068 +0000 UTC m=+0.485683775 container attach 4a5b4c781969065b34c01e5eef449b46a943d5fe76736913e9ae2f73ab83f572 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-0-deactivate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-18T23:20:05.547 INFO:journalctl@ceph.osd.0.smithi049.stdout:Mar 18 23:20:05 smithi049 podman[123966]: 2024-03-18 23:20:05.236056236 +0000 UTC m=+0.727691934 container died 4a5b4c781969065b34c01e5eef449b46a943d5fe76736913e9ae2f73ab83f572 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-0-deactivate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307) 2024-03-18T23:20:05.844 DEBUG:teuthology.orchestra.run.smithi049:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c63591ea-e57a-11ee-95c9-87774f69a715@osd.0.service' 2024-03-18T23:20:05.891 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-18T23:20:05.891 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-03-18T23:20:05.892 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-03-18T23:20:05.892 DEBUG:teuthology.orchestra.run.smithi139:> sudo systemctl stop ceph-c63591ea-e57a-11ee-95c9-87774f69a715@osd.1 2024-03-18T23:20:06.223 INFO:journalctl@ceph.osd.1.smithi139.stdout:Mar 18 23:20:05 smithi139 systemd[1]: Stopping Ceph osd.1 for c63591ea-e57a-11ee-95c9-87774f69a715... 2024-03-18T23:20:06.564 INFO:journalctl@ceph.osd.1.smithi139.stdout:Mar 18 23:20:06 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-1[71874]: 2024-03-18T23:20:06.223+0000 7fbfab8ce700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-18T23:20:06.564 INFO:journalctl@ceph.osd.1.smithi139.stdout:Mar 18 23:20:06 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-1[71874]: 2024-03-18T23:20:06.223+0000 7fbfab8ce700 -1 osd.1 104 *** Got signal Terminated *** 2024-03-18T23:20:06.564 INFO:journalctl@ceph.osd.1.smithi139.stdout:Mar 18 23:20:06 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-1[71874]: 2024-03-18T23:20:06.223+0000 7fbfab8ce700 -1 osd.1 104 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-18T23:20:11.898 INFO:journalctl@ceph.osd.1.smithi139.stdout:Mar 18 23:20:11 smithi139 podman[104492]: 2024-03-18 23:20:11.584970606 +0000 UTC m=+5.489711806 container died 3e2f5d678a0ecae4bb63c51a4fbeaea04f984657f3011454ffd27e9ce08513f3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-1, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-03-18T23:20:12.233 INFO:journalctl@ceph.osd.1.smithi139.stdout:Mar 18 23:20:12 smithi139 podman[104492]: 2024-03-18 23:20:12.007137694 +0000 UTC m=+5.911878895 container cleanup 3e2f5d678a0ecae4bb63c51a4fbeaea04f984657f3011454ffd27e9ce08513f3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-1, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, CEPH_POINT_RELEASE=) 2024-03-18T23:20:12.233 INFO:journalctl@ceph.osd.1.smithi139.stdout:Mar 18 23:20:12 smithi139 bash[104492]: ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-1 2024-03-18T23:20:12.564 INFO:journalctl@ceph.osd.1.smithi139.stdout:Mar 18 23:20:12 smithi139 podman[104509]: 2024-03-18 23:20:12.234632427 +0000 UTC m=+0.645614178 container remove 3e2f5d678a0ecae4bb63c51a4fbeaea04f984657f3011454ffd27e9ce08513f3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-1, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-03-18T23:20:12.937 INFO:journalctl@ceph.osd.1.smithi139.stdout:Mar 18 23:20:12 smithi139 podman[104600]: 2024-03-18 23:20:12.574031725 +0000 UTC m=+0.017526836 image pull 2312c8e42d0069a2dbb2364bddb00264db4b800e7f18a7e7e4eb8f3ecab2ff1c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639 2024-03-18T23:20:12.937 INFO:journalctl@ceph.osd.1.smithi139.stdout:Mar 18 23:20:12 smithi139 podman[104600]: 2024-03-18 23:20:12.693896472 +0000 UTC m=+0.137391578 container create 08e8825275d43f5f63588b5b1ea33b3755a1c017ce6857cfe9e262e49f4ae4f0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-1-deactivate, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD) 2024-03-18T23:20:13.261 INFO:journalctl@ceph.osd.1.smithi139.stdout:Mar 18 23:20:12 smithi139 podman[104600]: 2024-03-18 23:20:12.938834024 +0000 UTC m=+0.382329130 container init 08e8825275d43f5f63588b5b1ea33b3755a1c017ce6857cfe9e262e49f4ae4f0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-1-deactivate, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-03-18T23:20:13.261 INFO:journalctl@ceph.osd.1.smithi139.stdout:Mar 18 23:20:12 smithi139 podman[104600]: 2024-03-18 23:20:12.942237988 +0000 UTC m=+0.385733105 container start 08e8825275d43f5f63588b5b1ea33b3755a1c017ce6857cfe9e262e49f4ae4f0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-1-deactivate, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-03-18T23:20:13.261 INFO:journalctl@ceph.osd.1.smithi139.stdout:Mar 18 23:20:13 smithi139 podman[104600]: 2024-03-18 23:20:13.063775449 +0000 UTC m=+0.507270564 container attach 08e8825275d43f5f63588b5b1ea33b3755a1c017ce6857cfe9e262e49f4ae4f0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-1-deactivate, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-18T23:20:13.525 INFO:journalctl@ceph.osd.1.smithi139.stdout:Mar 18 23:20:13 smithi139 podman[104600]: 2024-03-18 23:20:13.263003932 +0000 UTC m=+0.706499043 container died 08e8825275d43f5f63588b5b1ea33b3755a1c017ce6857cfe9e262e49f4ae4f0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-1-deactivate, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240318.181317-b6ae249) 2024-03-18T23:20:13.941 DEBUG:teuthology.orchestra.run.smithi139:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c63591ea-e57a-11ee-95c9-87774f69a715@osd.1.service' 2024-03-18T23:20:13.981 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-18T23:20:13.982 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-03-18T23:20:13.982 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-03-18T23:20:13.982 DEBUG:teuthology.orchestra.run.smithi175:> sudo systemctl stop ceph-c63591ea-e57a-11ee-95c9-87774f69a715@osd.2 2024-03-18T23:20:14.297 INFO:journalctl@ceph.osd.2.smithi175.stdout:Mar 18 23:20:14 smithi175 systemd[1]: Stopping Ceph osd.2 for c63591ea-e57a-11ee-95c9-87774f69a715... 2024-03-18T23:20:14.559 INFO:journalctl@ceph.osd.2.smithi175.stdout:Mar 18 23:20:14 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-2[72488]: 2024-03-18T23:20:14.297+0000 7eff62f7b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-18T23:20:14.559 INFO:journalctl@ceph.osd.2.smithi175.stdout:Mar 18 23:20:14 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-2[72488]: 2024-03-18T23:20:14.297+0000 7eff62f7b700 -1 osd.2 104 *** Got signal Terminated *** 2024-03-18T23:20:14.560 INFO:journalctl@ceph.osd.2.smithi175.stdout:Mar 18 23:20:14 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-2[72488]: 2024-03-18T23:20:14.297+0000 7eff62f7b700 -1 osd.2 104 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-18T23:20:20.098 INFO:journalctl@ceph.osd.2.smithi175.stdout:Mar 18 23:20:19 smithi175 podman[104496]: 2024-03-18 23:20:19.821496104 +0000 UTC m=+5.646467448 container died 17b1cf604deb40118a64c7eb5eceec4db83c446a64210bc1192a4bda38e25c77 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-2, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.build-date=20240307) 2024-03-18T23:20:20.457 INFO:journalctl@ceph.osd.2.smithi175.stdout:Mar 18 23:20:20 smithi175 podman[104496]: 2024-03-18 23:20:20.182377127 +0000 UTC m=+6.007348472 container cleanup 17b1cf604deb40118a64c7eb5eceec4db83c446a64210bc1192a4bda38e25c77 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-2, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-03-18T23:20:20.458 INFO:journalctl@ceph.osd.2.smithi175.stdout:Mar 18 23:20:20 smithi175 bash[104496]: ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-2 2024-03-18T23:20:20.710 INFO:journalctl@ceph.osd.2.smithi175.stdout:Mar 18 23:20:20 smithi175 podman[104528]: 2024-03-18 23:20:20.458867069 +0000 UTC m=+0.634466634 container remove 17b1cf604deb40118a64c7eb5eceec4db83c446a64210bc1192a4bda38e25c77 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-03-18T23:20:21.061 INFO:journalctl@ceph.osd.2.smithi175.stdout:Mar 18 23:20:20 smithi175 podman[104623]: 2024-03-18 23:20:20.689794234 +0000 UTC m=+0.017777204 image pull 2312c8e42d0069a2dbb2364bddb00264db4b800e7f18a7e7e4eb8f3ecab2ff1c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639 2024-03-18T23:20:21.061 INFO:journalctl@ceph.osd.2.smithi175.stdout:Mar 18 23:20:20 smithi175 podman[104623]: 2024-03-18 23:20:20.818319594 +0000 UTC m=+0.146302557 container create 60e17f9d84518d4872dd5413dabc0578b7c47791ff0f9637d0d63d7bb4e3d48d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-2-deactivate, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-18T23:20:21.355 INFO:journalctl@ceph.osd.2.smithi175.stdout:Mar 18 23:20:21 smithi175 podman[104623]: 2024-03-18 23:20:21.062668419 +0000 UTC m=+0.390651389 container init 60e17f9d84518d4872dd5413dabc0578b7c47791ff0f9637d0d63d7bb4e3d48d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-2-deactivate, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-18T23:20:21.355 INFO:journalctl@ceph.osd.2.smithi175.stdout:Mar 18 23:20:21 smithi175 podman[104623]: 2024-03-18 23:20:21.066143893 +0000 UTC m=+0.394126861 container start 60e17f9d84518d4872dd5413dabc0578b7c47791ff0f9637d0d63d7bb4e3d48d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-2-deactivate, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-18T23:20:21.356 INFO:journalctl@ceph.osd.2.smithi175.stdout:Mar 18 23:20:21 smithi175 podman[104623]: 2024-03-18 23:20:21.171486925 +0000 UTC m=+0.499469924 container attach 60e17f9d84518d4872dd5413dabc0578b7c47791ff0f9637d0d63d7bb4e3d48d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-2-deactivate, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-18T23:20:21.656 INFO:journalctl@ceph.osd.2.smithi175.stdout:Mar 18 23:20:21 smithi175 podman[104623]: 2024-03-18 23:20:21.356884463 +0000 UTC m=+0.684867462 container died 60e17f9d84518d4872dd5413dabc0578b7c47791ff0f9637d0d63d7bb4e3d48d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-2-deactivate, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-03-18T23:20:21.906 DEBUG:teuthology.orchestra.run.smithi175:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c63591ea-e57a-11ee-95c9-87774f69a715@osd.2.service' 2024-03-18T23:20:21.931 INFO:journalctl@ceph.osd.2.smithi175.stdout:Mar 18 23:20:21 smithi175 podman[104649]: 2024-03-18 23:20:21.883846979 +0000 UTC m=+0.521532535 container remove 60e17f9d84518d4872dd5413dabc0578b7c47791ff0f9637d0d63d7bb4e3d48d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-2-deactivate, org.label-schema.build-date=20240307, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-03-18T23:20:21.931 INFO:journalctl@ceph.osd.2.smithi175.stdout:Mar 18 23:20:21 smithi175 systemd[1]: ceph-c63591ea-e57a-11ee-95c9-87774f69a715@osd.2.service: Deactivated successfully. 2024-03-18T23:20:21.931 INFO:journalctl@ceph.osd.2.smithi175.stdout:Mar 18 23:20:21 smithi175 systemd[1]: Stopped Ceph osd.2 for c63591ea-e57a-11ee-95c9-87774f69a715. 2024-03-18T23:20:21.932 INFO:journalctl@ceph.osd.2.smithi175.stdout:Mar 18 23:20:21 smithi175 systemd[1]: ceph-c63591ea-e57a-11ee-95c9-87774f69a715@osd.2.service: Consumed 1min 35.911s CPU time. 2024-03-18T23:20:21.946 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-18T23:20:21.947 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-03-18T23:20:21.947 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-03-18T23:20:21.947 DEBUG:teuthology.orchestra.run.smithi049:> sudo systemctl stop ceph-c63591ea-e57a-11ee-95c9-87774f69a715@osd.3 2024-03-18T23:20:22.258 INFO:journalctl@ceph.osd.3.smithi049.stdout:Mar 18 23:20:22 smithi049 systemd[1]: Stopping Ceph osd.3 for c63591ea-e57a-11ee-95c9-87774f69a715... 2024-03-18T23:20:22.628 INFO:journalctl@ceph.osd.3.smithi049.stdout:Mar 18 23:20:22 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-3[84547]: 2024-03-18T23:20:22.259+0000 7f9d53a3f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-18T23:20:22.629 INFO:journalctl@ceph.osd.3.smithi049.stdout:Mar 18 23:20:22 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-3[84547]: 2024-03-18T23:20:22.259+0000 7f9d53a3f700 -1 osd.3 104 *** Got signal Terminated *** 2024-03-18T23:20:22.629 INFO:journalctl@ceph.osd.3.smithi049.stdout:Mar 18 23:20:22 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-3[84547]: 2024-03-18T23:20:22.259+0000 7f9d53a3f700 -1 osd.3 104 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-18T23:20:24.314 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:20:24 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:20:24.187+0000 7f83cd240700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:19:57.904964+0000 front 2024-03-18T23:19:57.904988+0000 (oldest deadline 2024-03-18T23:20:23.805169+0000) 2024-03-18T23:20:24.710 INFO:journalctl@ceph.osd.8.smithi175.stdout:Mar 18 23:20:24 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-8[86041]: 2024-03-18T23:20:24.448+0000 7fa05c798700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:20:02.615291+0000 front 2024-03-18T23:20:02.615437+0000 (oldest deadline 2024-03-18T23:20:24.315195+0000) 2024-03-18T23:20:24.879 INFO:journalctl@ceph.osd.6.smithi049.stdout:Mar 18 23:20:24 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-6[91523]: 2024-03-18T23:20:24.542+0000 7f92dba96700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:20:01.671603+0000 front 2024-03-18T23:20:01.671661+0000 (oldest deadline 2024-03-18T23:20:23.971756+0000) 2024-03-18T23:20:25.564 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:20:25 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:20:25.192+0000 7f83cd240700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:19:57.904964+0000 front 2024-03-18T23:19:57.904988+0000 (oldest deadline 2024-03-18T23:20:23.805169+0000) 2024-03-18T23:20:25.710 INFO:journalctl@ceph.osd.8.smithi175.stdout:Mar 18 23:20:25 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-8[86041]: 2024-03-18T23:20:25.416+0000 7fa05c798700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:20:02.615291+0000 front 2024-03-18T23:20:02.615437+0000 (oldest deadline 2024-03-18T23:20:24.315195+0000) 2024-03-18T23:20:25.879 INFO:journalctl@ceph.osd.3.smithi049.stdout:Mar 18 23:20:25 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-3[84547]: 2024-03-18T23:20:25.719+0000 7f9d50033700 -1 osd.3 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:20:00.164417+0000 front 2024-03-18T23:20:00.164456+0000 (oldest deadline 2024-03-18T23:20:24.864555+0000) 2024-03-18T23:20:25.879 INFO:journalctl@ceph.osd.6.smithi049.stdout:Mar 18 23:20:25 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-6[91523]: 2024-03-18T23:20:25.558+0000 7f92dba96700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:20:01.671603+0000 front 2024-03-18T23:20:01.671661+0000 (oldest deadline 2024-03-18T23:20:23.971756+0000) 2024-03-18T23:20:26.210 INFO:journalctl@ceph.osd.5.smithi175.stdout:Mar 18 23:20:25 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-5[79436]: 2024-03-18T23:20:25.903+0000 7fa764b06700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:20:01.121579+0000 front 2024-03-18T23:20:01.121445+0000 (oldest deadline 2024-03-18T23:20:25.721983+0000) 2024-03-18T23:20:26.564 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:20:26 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:20:26.217+0000 7f83cd240700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:19:57.904964+0000 front 2024-03-18T23:19:57.904988+0000 (oldest deadline 2024-03-18T23:20:23.805169+0000) 2024-03-18T23:20:26.710 INFO:journalctl@ceph.osd.8.smithi175.stdout:Mar 18 23:20:26 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-8[86041]: 2024-03-18T23:20:26.461+0000 7fa05c798700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:20:02.615291+0000 front 2024-03-18T23:20:02.615437+0000 (oldest deadline 2024-03-18T23:20:24.315195+0000) 2024-03-18T23:20:26.879 INFO:journalctl@ceph.osd.3.smithi049.stdout:Mar 18 23:20:26 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-3[84547]: 2024-03-18T23:20:26.698+0000 7f9d50033700 -1 osd.3 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:20:00.164417+0000 front 2024-03-18T23:20:00.164456+0000 (oldest deadline 2024-03-18T23:20:24.864555+0000) 2024-03-18T23:20:26.879 INFO:journalctl@ceph.osd.6.smithi049.stdout:Mar 18 23:20:26 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-6[91523]: 2024-03-18T23:20:26.582+0000 7f92dba96700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:20:01.671603+0000 front 2024-03-18T23:20:01.671661+0000 (oldest deadline 2024-03-18T23:20:23.971756+0000) 2024-03-18T23:20:27.210 INFO:journalctl@ceph.osd.5.smithi175.stdout:Mar 18 23:20:26 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-5[79436]: 2024-03-18T23:20:26.910+0000 7fa764b06700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:20:01.121579+0000 front 2024-03-18T23:20:01.121445+0000 (oldest deadline 2024-03-18T23:20:25.721983+0000) 2024-03-18T23:20:27.564 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:20:27 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:20:27.230+0000 7f83cd240700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:19:57.904964+0000 front 2024-03-18T23:19:57.904988+0000 (oldest deadline 2024-03-18T23:20:23.805169+0000) 2024-03-18T23:20:27.622 INFO:journalctl@ceph.osd.6.smithi049.stdout:Mar 18 23:20:27 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-6[91523]: 2024-03-18T23:20:27.579+0000 7f92dba96700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:20:01.671603+0000 front 2024-03-18T23:20:01.671661+0000 (oldest deadline 2024-03-18T23:20:23.971756+0000) 2024-03-18T23:20:27.622 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 18 23:20:27 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-9[98163]: 2024-03-18T23:20:27.342+0000 7ff64b26e700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:20:01.074760+0000 front 2024-03-18T23:20:01.074900+0000 (oldest deadline 2024-03-18T23:20:26.974968+0000) 2024-03-18T23:20:27.878 INFO:journalctl@ceph.osd.3.smithi049.stdout:Mar 18 23:20:27 smithi049 podman[124073]: 2024-03-18 23:20:27.622767159 +0000 UTC m=+5.467120599 container died 4328db4d778b5585287fe37b5c7ea156c928aa374203177c402c0a1dcb37d581 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-3, org.label-schema.build-date=20240307, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True) 2024-03-18T23:20:27.948 INFO:journalctl@ceph.osd.8.smithi175.stdout:Mar 18 23:20:27 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-8[86041]: 2024-03-18T23:20:27.467+0000 7fa05c798700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:20:02.615291+0000 front 2024-03-18T23:20:02.615437+0000 (oldest deadline 2024-03-18T23:20:24.315195+0000) 2024-03-18T23:20:28.175 INFO:journalctl@ceph.osd.3.smithi049.stdout:Mar 18 23:20:27 smithi049 podman[124073]: 2024-03-18 23:20:27.950662751 +0000 UTC m=+5.795016191 container cleanup 4328db4d778b5585287fe37b5c7ea156c928aa374203177c402c0a1dcb37d581 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-3, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-03-18T23:20:28.175 INFO:journalctl@ceph.osd.3.smithi049.stdout:Mar 18 23:20:27 smithi049 bash[124073]: ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-3 2024-03-18T23:20:28.210 INFO:journalctl@ceph.osd.5.smithi175.stdout:Mar 18 23:20:27 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-5[79436]: 2024-03-18T23:20:27.950+0000 7fa764b06700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:20:01.121579+0000 front 2024-03-18T23:20:01.121445+0000 (oldest deadline 2024-03-18T23:20:25.721983+0000) 2024-03-18T23:20:28.564 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:20:28 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:20:28.210+0000 7f83cd240700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:19:57.904964+0000 front 2024-03-18T23:19:57.904988+0000 (oldest deadline 2024-03-18T23:20:23.805169+0000) 2024-03-18T23:20:28.567 INFO:journalctl@ceph.osd.3.smithi049.stdout:Mar 18 23:20:28 smithi049 podman[124088]: 2024-03-18 23:20:28.17704269 +0000 UTC m=+0.553521464 container remove 4328db4d778b5585287fe37b5c7ea156c928aa374203177c402c0a1dcb37d581 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-3, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, ceph=True) 2024-03-18T23:20:28.568 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 18 23:20:28 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-9[98163]: 2024-03-18T23:20:28.372+0000 7ff64b26e700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:20:01.074760+0000 front 2024-03-18T23:20:01.074900+0000 (oldest deadline 2024-03-18T23:20:26.974968+0000) 2024-03-18T23:20:28.710 INFO:journalctl@ceph.osd.8.smithi175.stdout:Mar 18 23:20:28 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-8[86041]: 2024-03-18T23:20:28.457+0000 7fa05c798700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:20:02.615291+0000 front 2024-03-18T23:20:02.615437+0000 (oldest deadline 2024-03-18T23:20:24.315195+0000) 2024-03-18T23:20:28.710 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:20:28 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:20:28.665+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:19:57.947723+0000 front 2024-03-18T23:19:57.947785+0000 (oldest deadline 2024-03-18T23:20:28.548565+0000) 2024-03-18T23:20:28.879 INFO:journalctl@ceph.osd.6.smithi049.stdout:Mar 18 23:20:28 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-6[91523]: 2024-03-18T23:20:28.569+0000 7f92dba96700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:20:01.671603+0000 front 2024-03-18T23:20:01.671661+0000 (oldest deadline 2024-03-18T23:20:23.971756+0000) 2024-03-18T23:20:28.879 INFO:journalctl@ceph.osd.3.smithi049.stdout:Mar 18 23:20:28 smithi049 podman[124180]: 2024-03-18 23:20:28.530379393 +0000 UTC m=+0.018201954 image pull 2312c8e42d0069a2dbb2364bddb00264db4b800e7f18a7e7e4eb8f3ecab2ff1c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639 2024-03-18T23:20:28.879 INFO:journalctl@ceph.osd.3.smithi049.stdout:Mar 18 23:20:28 smithi049 podman[124180]: 2024-03-18 23:20:28.653278642 +0000 UTC m=+0.141101242 container create 88d1512d9ac0272c004ed3d62be6f9d9c94b3e5d05d6fa66e7a82abb78ea14ab (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-3-deactivate, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-03-18T23:20:29.230 INFO:journalctl@ceph.osd.3.smithi049.stdout:Mar 18 23:20:28 smithi049 podman[124180]: 2024-03-18 23:20:28.92242643 +0000 UTC m=+0.410248991 container init 88d1512d9ac0272c004ed3d62be6f9d9c94b3e5d05d6fa66e7a82abb78ea14ab (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-3-deactivate, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240307, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5) 2024-03-18T23:20:29.230 INFO:journalctl@ceph.osd.3.smithi049.stdout:Mar 18 23:20:28 smithi049 podman[124180]: 2024-03-18 23:20:28.925742378 +0000 UTC m=+0.413564944 container start 88d1512d9ac0272c004ed3d62be6f9d9c94b3e5d05d6fa66e7a82abb78ea14ab (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-3-deactivate, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0) 2024-03-18T23:20:29.230 INFO:journalctl@ceph.osd.3.smithi049.stdout:Mar 18 23:20:29 smithi049 podman[124180]: 2024-03-18 23:20:29.044278093 +0000 UTC m=+0.532100678 container attach 88d1512d9ac0272c004ed3d62be6f9d9c94b3e5d05d6fa66e7a82abb78ea14ab (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-3-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=) 2024-03-18T23:20:29.440 INFO:journalctl@ceph.osd.5.smithi175.stdout:Mar 18 23:20:28 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-5[79436]: 2024-03-18T23:20:28.994+0000 7fa764b06700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:20:01.121579+0000 front 2024-03-18T23:20:01.121445+0000 (oldest deadline 2024-03-18T23:20:25.721983+0000) 2024-03-18T23:20:29.515 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 18 23:20:29 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-9[98163]: 2024-03-18T23:20:29.327+0000 7ff64b26e700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:20:01.074760+0000 front 2024-03-18T23:20:01.074900+0000 (oldest deadline 2024-03-18T23:20:26.974968+0000) 2024-03-18T23:20:29.515 INFO:journalctl@ceph.osd.3.smithi049.stdout:Mar 18 23:20:29 smithi049 podman[124180]: 2024-03-18 23:20:29.23177325 +0000 UTC m=+0.719595821 container died 88d1512d9ac0272c004ed3d62be6f9d9c94b3e5d05d6fa66e7a82abb78ea14ab (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-3-deactivate, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240307) 2024-03-18T23:20:29.564 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:20:29 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:20:29.213+0000 7f83cd240700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:19:57.904964+0000 front 2024-03-18T23:19:57.904988+0000 (oldest deadline 2024-03-18T23:20:23.805169+0000) 2024-03-18T23:20:29.710 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:20:29 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:20:29.646+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:19:57.947723+0000 front 2024-03-18T23:19:57.947785+0000 (oldest deadline 2024-03-18T23:20:28.548565+0000) 2024-03-18T23:20:29.710 INFO:journalctl@ceph.osd.8.smithi175.stdout:Mar 18 23:20:29 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-8[86041]: 2024-03-18T23:20:29.442+0000 7fa05c798700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:20:02.615291+0000 front 2024-03-18T23:20:02.615437+0000 (oldest deadline 2024-03-18T23:20:24.315195+0000) 2024-03-18T23:20:29.833 INFO:journalctl@ceph.osd.6.smithi049.stdout:Mar 18 23:20:29 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-6[91523]: 2024-03-18T23:20:29.522+0000 7f92dba96700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:20:01.671603+0000 front 2024-03-18T23:20:01.671661+0000 (oldest deadline 2024-03-18T23:20:23.971756+0000) 2024-03-18T23:20:29.855 DEBUG:teuthology.orchestra.run.smithi049:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c63591ea-e57a-11ee-95c9-87774f69a715@osd.3.service' 2024-03-18T23:20:29.900 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-18T23:20:29.901 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-03-18T23:20:29.901 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-03-18T23:20:29.901 DEBUG:teuthology.orchestra.run.smithi139:> sudo systemctl stop ceph-c63591ea-e57a-11ee-95c9-87774f69a715@osd.4 2024-03-18T23:20:30.180 INFO:journalctl@ceph.osd.4.smithi139.stdout:Mar 18 23:20:29 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-4[78727]: 2024-03-18T23:20:29.902+0000 7f0ffd364700 -1 osd.4 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:20:02.631336+0000 front 2024-03-18T23:20:02.631424+0000 (oldest deadline 2024-03-18T23:20:29.031856+0000) 2024-03-18T23:20:30.181 INFO:journalctl@ceph.osd.4.smithi139.stdout:Mar 18 23:20:29 smithi139 systemd[1]: Stopping Ceph osd.4 for c63591ea-e57a-11ee-95c9-87774f69a715... 2024-03-18T23:20:30.210 INFO:journalctl@ceph.osd.5.smithi175.stdout:Mar 18 23:20:29 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-5[79436]: 2024-03-18T23:20:29.951+0000 7fa764b06700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:20:01.121579+0000 front 2024-03-18T23:20:01.121445+0000 (oldest deadline 2024-03-18T23:20:25.721983+0000) 2024-03-18T23:20:30.564 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:20:30 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:20:30.181+0000 7f83cd240700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:19:57.904964+0000 front 2024-03-18T23:19:57.904988+0000 (oldest deadline 2024-03-18T23:20:23.805169+0000) 2024-03-18T23:20:30.564 INFO:journalctl@ceph.osd.4.smithi139.stdout:Mar 18 23:20:30 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-4[78727]: 2024-03-18T23:20:30.237+0000 7f1004d78700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-18T23:20:30.564 INFO:journalctl@ceph.osd.4.smithi139.stdout:Mar 18 23:20:30 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-4[78727]: 2024-03-18T23:20:30.237+0000 7f1004d78700 -1 osd.4 104 *** Got signal Terminated *** 2024-03-18T23:20:30.564 INFO:journalctl@ceph.osd.4.smithi139.stdout:Mar 18 23:20:30 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-4[78727]: 2024-03-18T23:20:30.237+0000 7f1004d78700 -1 osd.4 104 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-18T23:20:30.629 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 18 23:20:30 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-9[98163]: 2024-03-18T23:20:30.352+0000 7ff64b26e700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:20:01.074760+0000 front 2024-03-18T23:20:01.074900+0000 (oldest deadline 2024-03-18T23:20:26.974968+0000) 2024-03-18T23:20:30.629 INFO:journalctl@ceph.osd.6.smithi049.stdout:Mar 18 23:20:30 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-6[91523]: 2024-03-18T23:20:30.504+0000 7f92dba96700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:20:01.671603+0000 front 2024-03-18T23:20:01.671661+0000 (oldest deadline 2024-03-18T23:20:23.971756+0000) 2024-03-18T23:20:30.940 INFO:journalctl@ceph.osd.8.smithi175.stdout:Mar 18 23:20:30 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-8[86041]: 2024-03-18T23:20:30.485+0000 7fa05c798700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:20:02.615291+0000 front 2024-03-18T23:20:02.615437+0000 (oldest deadline 2024-03-18T23:20:24.315195+0000) 2024-03-18T23:20:30.940 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:20:30 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:20:30.617+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:19:57.947723+0000 front 2024-03-18T23:19:57.947785+0000 (oldest deadline 2024-03-18T23:20:28.548565+0000) 2024-03-18T23:20:31.141 INFO:journalctl@ceph.osd.4.smithi139.stdout:Mar 18 23:20:30 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-4[78727]: 2024-03-18T23:20:30.877+0000 7f0ffd364700 -1 osd.4 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:20:02.631336+0000 front 2024-03-18T23:20:02.631424+0000 (oldest deadline 2024-03-18T23:20:29.031856+0000) 2024-03-18T23:20:31.210 INFO:journalctl@ceph.osd.5.smithi175.stdout:Mar 18 23:20:30 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-5[79436]: 2024-03-18T23:20:30.941+0000 7fa764b06700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:20:01.121579+0000 front 2024-03-18T23:20:01.121445+0000 (oldest deadline 2024-03-18T23:20:25.721983+0000) 2024-03-18T23:20:31.564 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:20:31 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:20:31.141+0000 7f83cd240700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:19:57.904964+0000 front 2024-03-18T23:19:57.904988+0000 (oldest deadline 2024-03-18T23:20:23.805169+0000) 2024-03-18T23:20:31.629 INFO:journalctl@ceph.osd.6.smithi049.stdout:Mar 18 23:20:31 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-6[91523]: 2024-03-18T23:20:31.480+0000 7f92dba96700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:20:01.671603+0000 front 2024-03-18T23:20:01.671661+0000 (oldest deadline 2024-03-18T23:20:23.971756+0000) 2024-03-18T23:20:31.629 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 18 23:20:31 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-9[98163]: 2024-03-18T23:20:31.305+0000 7ff64b26e700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:20:01.074760+0000 front 2024-03-18T23:20:01.074900+0000 (oldest deadline 2024-03-18T23:20:26.974968+0000) 2024-03-18T23:20:31.711 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:20:31 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:20:31.613+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:19:57.947723+0000 front 2024-03-18T23:19:57.947785+0000 (oldest deadline 2024-03-18T23:20:28.548565+0000) 2024-03-18T23:20:31.711 INFO:journalctl@ceph.osd.8.smithi175.stdout:Mar 18 23:20:31 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-8[86041]: 2024-03-18T23:20:31.438+0000 7fa05c798700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:20:02.615291+0000 front 2024-03-18T23:20:02.615437+0000 (oldest deadline 2024-03-18T23:20:24.315195+0000) 2024-03-18T23:20:32.153 INFO:journalctl@ceph.osd.4.smithi139.stdout:Mar 18 23:20:31 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-4[78727]: 2024-03-18T23:20:31.851+0000 7f0ffd364700 -1 osd.4 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:20:02.631336+0000 front 2024-03-18T23:20:02.631424+0000 (oldest deadline 2024-03-18T23:20:29.031856+0000) 2024-03-18T23:20:32.211 INFO:journalctl@ceph.osd.5.smithi175.stdout:Mar 18 23:20:31 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-5[79436]: 2024-03-18T23:20:31.896+0000 7fa764b06700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:20:01.121579+0000 front 2024-03-18T23:20:01.121445+0000 (oldest deadline 2024-03-18T23:20:25.721983+0000) 2024-03-18T23:20:32.564 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:20:32 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:20:32.154+0000 7f83cd240700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:19:57.904964+0000 front 2024-03-18T23:19:57.904988+0000 (oldest deadline 2024-03-18T23:20:23.805169+0000) 2024-03-18T23:20:32.564 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:20:32 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:20:32.154+0000 7f83cd240700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-03-18T23:20:10.806661+0000 front 2024-03-18T23:20:10.806621+0000 (oldest deadline 2024-03-18T23:20:31.906747+0000) 2024-03-18T23:20:32.629 INFO:journalctl@ceph.osd.6.smithi049.stdout:Mar 18 23:20:32 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-6[91523]: 2024-03-18T23:20:32.504+0000 7f92dba96700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:20:01.671603+0000 front 2024-03-18T23:20:01.671661+0000 (oldest deadline 2024-03-18T23:20:23.971756+0000) 2024-03-18T23:20:32.629 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 18 23:20:32 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-9[98163]: 2024-03-18T23:20:32.344+0000 7ff64b26e700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:20:01.074760+0000 front 2024-03-18T23:20:01.074900+0000 (oldest deadline 2024-03-18T23:20:26.974968+0000) 2024-03-18T23:20:32.710 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:20:32 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:20:32.587+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:19:57.947723+0000 front 2024-03-18T23:19:57.947785+0000 (oldest deadline 2024-03-18T23:20:28.548565+0000) 2024-03-18T23:20:32.710 INFO:journalctl@ceph.osd.8.smithi175.stdout:Mar 18 23:20:32 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-8[86041]: 2024-03-18T23:20:32.432+0000 7fa05c798700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:20:02.615291+0000 front 2024-03-18T23:20:02.615437+0000 (oldest deadline 2024-03-18T23:20:24.315195+0000) 2024-03-18T23:20:33.159 INFO:journalctl@ceph.osd.4.smithi139.stdout:Mar 18 23:20:32 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-4[78727]: 2024-03-18T23:20:32.883+0000 7f0ffd364700 -1 osd.4 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:20:02.631336+0000 front 2024-03-18T23:20:02.631424+0000 (oldest deadline 2024-03-18T23:20:29.031856+0000) 2024-03-18T23:20:33.210 INFO:journalctl@ceph.osd.5.smithi175.stdout:Mar 18 23:20:32 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-5[79436]: 2024-03-18T23:20:32.876+0000 7fa764b06700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:20:01.121579+0000 front 2024-03-18T23:20:01.121445+0000 (oldest deadline 2024-03-18T23:20:25.721983+0000) 2024-03-18T23:20:33.210 INFO:journalctl@ceph.osd.5.smithi175.stdout:Mar 18 23:20:32 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-5[79436]: 2024-03-18T23:20:32.876+0000 7fa764b06700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-03-18T23:20:09.223527+0000 front 2024-03-18T23:20:09.223198+0000 (oldest deadline 2024-03-18T23:20:32.123184+0000) 2024-03-18T23:20:33.564 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:20:33 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:20:33.160+0000 7f83cd240700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:19:57.904964+0000 front 2024-03-18T23:19:57.904988+0000 (oldest deadline 2024-03-18T23:20:23.805169+0000) 2024-03-18T23:20:33.564 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:20:33 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:20:33.160+0000 7f83cd240700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-03-18T23:20:10.806661+0000 front 2024-03-18T23:20:10.806621+0000 (oldest deadline 2024-03-18T23:20:31.906747+0000) 2024-03-18T23:20:33.628 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 18 23:20:33 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-9[98163]: 2024-03-18T23:20:33.313+0000 7ff64b26e700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:20:01.074760+0000 front 2024-03-18T23:20:01.074900+0000 (oldest deadline 2024-03-18T23:20:26.974968+0000) 2024-03-18T23:20:33.629 INFO:journalctl@ceph.osd.6.smithi049.stdout:Mar 18 23:20:33 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-6[91523]: 2024-03-18T23:20:33.472+0000 7f92dba96700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:20:01.671603+0000 front 2024-03-18T23:20:01.671661+0000 (oldest deadline 2024-03-18T23:20:23.971756+0000) 2024-03-18T23:20:33.629 INFO:journalctl@ceph.osd.6.smithi049.stdout:Mar 18 23:20:33 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-6[91523]: 2024-03-18T23:20:33.472+0000 7f92dba96700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-03-18T23:20:08.573527+0000 front 2024-03-18T23:20:08.573679+0000 (oldest deadline 2024-03-18T23:20:32.673334+0000) 2024-03-18T23:20:33.710 INFO:journalctl@ceph.osd.8.smithi175.stdout:Mar 18 23:20:33 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-8[86041]: 2024-03-18T23:20:33.416+0000 7fa05c798700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:20:02.615291+0000 front 2024-03-18T23:20:02.615437+0000 (oldest deadline 2024-03-18T23:20:24.315195+0000) 2024-03-18T23:20:33.710 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:20:33 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:20:33.576+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:19:57.947723+0000 front 2024-03-18T23:19:57.947785+0000 (oldest deadline 2024-03-18T23:20:28.548565+0000) 2024-03-18T23:20:34.132 INFO:journalctl@ceph.osd.4.smithi139.stdout:Mar 18 23:20:33 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-4[78727]: 2024-03-18T23:20:33.870+0000 7f0ffd364700 -1 osd.4 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:20:02.631336+0000 front 2024-03-18T23:20:02.631424+0000 (oldest deadline 2024-03-18T23:20:29.031856+0000) 2024-03-18T23:20:34.132 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 18 23:20:34 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-7[85590]: 2024-03-18T23:20:34.056+0000 7f5dd9117700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-03-18T23:20:08.415464+0000 front 2024-03-18T23:20:08.415444+0000 (oldest deadline 2024-03-18T23:20:33.115572+0000) 2024-03-18T23:20:34.210 INFO:journalctl@ceph.osd.5.smithi175.stdout:Mar 18 23:20:33 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-5[79436]: 2024-03-18T23:20:33.839+0000 7fa764b06700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:20:01.121579+0000 front 2024-03-18T23:20:01.121445+0000 (oldest deadline 2024-03-18T23:20:25.721983+0000) 2024-03-18T23:20:34.210 INFO:journalctl@ceph.osd.5.smithi175.stdout:Mar 18 23:20:33 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-5[79436]: 2024-03-18T23:20:33.839+0000 7fa764b06700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-03-18T23:20:09.223527+0000 front 2024-03-18T23:20:09.223198+0000 (oldest deadline 2024-03-18T23:20:32.123184+0000) 2024-03-18T23:20:34.564 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:20:34 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:20:34.132+0000 7f83cd240700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:19:57.904964+0000 front 2024-03-18T23:19:57.904988+0000 (oldest deadline 2024-03-18T23:20:23.805169+0000) 2024-03-18T23:20:34.565 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:20:34 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:20:34.132+0000 7f83cd240700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-03-18T23:20:10.806661+0000 front 2024-03-18T23:20:10.806621+0000 (oldest deadline 2024-03-18T23:20:31.906747+0000) 2024-03-18T23:20:34.629 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 18 23:20:34 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-9[98163]: 2024-03-18T23:20:34.333+0000 7ff64b26e700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:20:01.074760+0000 front 2024-03-18T23:20:01.074900+0000 (oldest deadline 2024-03-18T23:20:26.974968+0000) 2024-03-18T23:20:34.630 INFO:journalctl@ceph.osd.6.smithi049.stdout:Mar 18 23:20:34 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-6[91523]: 2024-03-18T23:20:34.520+0000 7f92dba96700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:20:01.671603+0000 front 2024-03-18T23:20:01.671661+0000 (oldest deadline 2024-03-18T23:20:23.971756+0000) 2024-03-18T23:20:34.630 INFO:journalctl@ceph.osd.6.smithi049.stdout:Mar 18 23:20:34 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-6[91523]: 2024-03-18T23:20:34.520+0000 7f92dba96700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-03-18T23:20:08.573527+0000 front 2024-03-18T23:20:08.573679+0000 (oldest deadline 2024-03-18T23:20:32.673334+0000) 2024-03-18T23:20:34.710 INFO:journalctl@ceph.osd.8.smithi175.stdout:Mar 18 23:20:34 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-8[86041]: 2024-03-18T23:20:34.399+0000 7fa05c798700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:20:02.615291+0000 front 2024-03-18T23:20:02.615437+0000 (oldest deadline 2024-03-18T23:20:24.315195+0000) 2024-03-18T23:20:34.710 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:20:34 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:20:34.554+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:19:57.947723+0000 front 2024-03-18T23:19:57.947785+0000 (oldest deadline 2024-03-18T23:20:28.548565+0000) 2024-03-18T23:20:34.710 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:20:34 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:20:34.554+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-03-18T23:20:08.549207+0000 front 2024-03-18T23:20:08.549248+0000 (oldest deadline 2024-03-18T23:20:33.849233+0000) 2024-03-18T23:20:35.087 INFO:journalctl@ceph.osd.4.smithi139.stdout:Mar 18 23:20:34 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-4[78727]: 2024-03-18T23:20:34.829+0000 7f0ffd364700 -1 osd.4 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:20:02.631336+0000 front 2024-03-18T23:20:02.631424+0000 (oldest deadline 2024-03-18T23:20:29.031856+0000) 2024-03-18T23:20:35.087 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 18 23:20:35 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-7[85590]: 2024-03-18T23:20:35.032+0000 7f5dd9117700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-03-18T23:20:08.415464+0000 front 2024-03-18T23:20:08.415444+0000 (oldest deadline 2024-03-18T23:20:33.115572+0000) 2024-03-18T23:20:35.210 INFO:journalctl@ceph.osd.5.smithi175.stdout:Mar 18 23:20:34 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-5[79436]: 2024-03-18T23:20:34.884+0000 7fa764b06700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:20:01.121579+0000 front 2024-03-18T23:20:01.121445+0000 (oldest deadline 2024-03-18T23:20:25.721983+0000) 2024-03-18T23:20:35.210 INFO:journalctl@ceph.osd.5.smithi175.stdout:Mar 18 23:20:34 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-5[79436]: 2024-03-18T23:20:34.884+0000 7fa764b06700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-03-18T23:20:09.223527+0000 front 2024-03-18T23:20:09.223198+0000 (oldest deadline 2024-03-18T23:20:32.123184+0000) 2024-03-18T23:20:35.564 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:20:35 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:20:35.088+0000 7f83cd240700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:19:57.904964+0000 front 2024-03-18T23:19:57.904988+0000 (oldest deadline 2024-03-18T23:20:23.805169+0000) 2024-03-18T23:20:35.564 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:20:35 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:20:35.088+0000 7f83cd240700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-03-18T23:20:10.806661+0000 front 2024-03-18T23:20:10.806621+0000 (oldest deadline 2024-03-18T23:20:31.906747+0000) 2024-03-18T23:20:35.629 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 18 23:20:35 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-9[98163]: 2024-03-18T23:20:35.379+0000 7ff64b26e700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:20:01.074760+0000 front 2024-03-18T23:20:01.074900+0000 (oldest deadline 2024-03-18T23:20:26.974968+0000) 2024-03-18T23:20:35.629 INFO:journalctl@ceph.osd.6.smithi049.stdout:Mar 18 23:20:35 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-6[91523]: 2024-03-18T23:20:35.507+0000 7f92dba96700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:20:01.671603+0000 front 2024-03-18T23:20:01.671661+0000 (oldest deadline 2024-03-18T23:20:23.971756+0000) 2024-03-18T23:20:35.629 INFO:journalctl@ceph.osd.6.smithi049.stdout:Mar 18 23:20:35 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-6[91523]: 2024-03-18T23:20:35.507+0000 7f92dba96700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-03-18T23:20:08.573527+0000 front 2024-03-18T23:20:08.573679+0000 (oldest deadline 2024-03-18T23:20:32.673334+0000) 2024-03-18T23:20:35.710 INFO:journalctl@ceph.osd.8.smithi175.stdout:Mar 18 23:20:35 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-8[86041]: 2024-03-18T23:20:35.391+0000 7fa05c798700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:20:02.615291+0000 front 2024-03-18T23:20:02.615437+0000 (oldest deadline 2024-03-18T23:20:24.315195+0000) 2024-03-18T23:20:35.710 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:20:35 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:20:35.590+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:19:57.947723+0000 front 2024-03-18T23:19:57.947785+0000 (oldest deadline 2024-03-18T23:20:28.548565+0000) 2024-03-18T23:20:35.711 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:20:35 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:20:35.590+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-03-18T23:20:08.549207+0000 front 2024-03-18T23:20:08.549248+0000 (oldest deadline 2024-03-18T23:20:33.849233+0000) 2024-03-18T23:20:36.037 INFO:journalctl@ceph.osd.4.smithi139.stdout:Mar 18 23:20:35 smithi139 podman[104717]: 2024-03-18 23:20:35.762138647 +0000 UTC m=+5.667905423 container died 398fe5f95f4dcbdf40e570eede3665926f785016c56b60b99883c517c3fec344 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-4, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-03-18T23:20:36.210 INFO:journalctl@ceph.osd.5.smithi175.stdout:Mar 18 23:20:35 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-5[79436]: 2024-03-18T23:20:35.841+0000 7fa764b06700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:20:01.121579+0000 front 2024-03-18T23:20:01.121445+0000 (oldest deadline 2024-03-18T23:20:25.721983+0000) 2024-03-18T23:20:36.210 INFO:journalctl@ceph.osd.5.smithi175.stdout:Mar 18 23:20:35 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-5[79436]: 2024-03-18T23:20:35.841+0000 7fa764b06700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-03-18T23:20:09.223527+0000 front 2024-03-18T23:20:09.223198+0000 (oldest deadline 2024-03-18T23:20:32.123184+0000) 2024-03-18T23:20:36.314 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 18 23:20:36 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-7[85590]: 2024-03-18T23:20:36.037+0000 7f5dd9117700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-03-18T23:20:08.415464+0000 front 2024-03-18T23:20:08.415444+0000 (oldest deadline 2024-03-18T23:20:33.115572+0000) 2024-03-18T23:20:36.314 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:20:36 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:20:36.114+0000 7f83cd240700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:19:57.904964+0000 front 2024-03-18T23:19:57.904988+0000 (oldest deadline 2024-03-18T23:20:23.805169+0000) 2024-03-18T23:20:36.314 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:20:36 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:20:36.114+0000 7f83cd240700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-03-18T23:20:10.806661+0000 front 2024-03-18T23:20:10.806621+0000 (oldest deadline 2024-03-18T23:20:31.906747+0000) 2024-03-18T23:20:36.315 INFO:journalctl@ceph.osd.4.smithi139.stdout:Mar 18 23:20:36 smithi139 podman[104717]: 2024-03-18 23:20:36.154426993 +0000 UTC m=+6.060193762 container cleanup 398fe5f95f4dcbdf40e570eede3665926f785016c56b60b99883c517c3fec344 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-4, io.buildah.version=1.33.5, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307) 2024-03-18T23:20:36.315 INFO:journalctl@ceph.osd.4.smithi139.stdout:Mar 18 23:20:36 smithi139 bash[104717]: ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-4 2024-03-18T23:20:36.710 INFO:journalctl@ceph.osd.8.smithi175.stdout:Mar 18 23:20:36 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-8[86041]: 2024-03-18T23:20:36.412+0000 7fa05c798700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:20:02.615291+0000 front 2024-03-18T23:20:02.615437+0000 (oldest deadline 2024-03-18T23:20:24.315195+0000) 2024-03-18T23:20:36.710 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:20:36 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:20:36.573+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:19:57.947723+0000 front 2024-03-18T23:19:57.947785+0000 (oldest deadline 2024-03-18T23:20:28.548565+0000) 2024-03-18T23:20:36.711 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:20:36 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:20:36.573+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-03-18T23:20:08.549207+0000 front 2024-03-18T23:20:08.549248+0000 (oldest deadline 2024-03-18T23:20:33.849233+0000) 2024-03-18T23:20:36.760 INFO:journalctl@ceph.osd.4.smithi139.stdout:Mar 18 23:20:36 smithi139 podman[104735]: 2024-03-18 23:20:36.431041052 +0000 UTC m=+0.665099864 container remove 398fe5f95f4dcbdf40e570eede3665926f785016c56b60b99883c517c3fec344 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-4, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, ceph=True) 2024-03-18T23:20:36.879 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 18 23:20:36 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-9[98163]: 2024-03-18T23:20:36.400+0000 7ff64b26e700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:20:01.074760+0000 front 2024-03-18T23:20:01.074900+0000 (oldest deadline 2024-03-18T23:20:26.974968+0000) 2024-03-18T23:20:36.879 INFO:journalctl@ceph.osd.6.smithi049.stdout:Mar 18 23:20:36 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-6[91523]: 2024-03-18T23:20:36.458+0000 7f92dba96700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:20:01.671603+0000 front 2024-03-18T23:20:01.671661+0000 (oldest deadline 2024-03-18T23:20:23.971756+0000) 2024-03-18T23:20:36.879 INFO:journalctl@ceph.osd.6.smithi049.stdout:Mar 18 23:20:36 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-6[91523]: 2024-03-18T23:20:36.458+0000 7f92dba96700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-03-18T23:20:08.573527+0000 front 2024-03-18T23:20:08.573679+0000 (oldest deadline 2024-03-18T23:20:32.673334+0000) 2024-03-18T23:20:37.058 INFO:journalctl@ceph.osd.4.smithi139.stdout:Mar 18 23:20:36 smithi139 podman[104826]: 2024-03-18 23:20:36.664724268 +0000 UTC m=+0.017897682 image pull 2312c8e42d0069a2dbb2364bddb00264db4b800e7f18a7e7e4eb8f3ecab2ff1c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639 2024-03-18T23:20:37.058 INFO:journalctl@ceph.osd.4.smithi139.stdout:Mar 18 23:20:36 smithi139 podman[104826]: 2024-03-18 23:20:36.811929332 +0000 UTC m=+0.165102725 container create 75cb315802efe5b3f57ce2f75ecd4f7129ac4b71b6e2ca2fa13a6434e5408625 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-4-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-03-18T23:20:37.058 INFO:journalctl@ceph.osd.4.smithi139.stdout:Mar 18 23:20:37 smithi139 podman[104826]: 2024-03-18 23:20:37.059406429 +0000 UTC m=+0.412579830 container init 75cb315802efe5b3f57ce2f75ecd4f7129ac4b71b6e2ca2fa13a6434e5408625 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-4-deactivate, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5) 2024-03-18T23:20:37.210 INFO:journalctl@ceph.osd.5.smithi175.stdout:Mar 18 23:20:36 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-5[79436]: 2024-03-18T23:20:36.803+0000 7fa764b06700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:20:01.121579+0000 front 2024-03-18T23:20:01.121445+0000 (oldest deadline 2024-03-18T23:20:25.721983+0000) 2024-03-18T23:20:37.210 INFO:journalctl@ceph.osd.5.smithi175.stdout:Mar 18 23:20:36 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-5[79436]: 2024-03-18T23:20:36.803+0000 7fa764b06700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-03-18T23:20:09.223527+0000 front 2024-03-18T23:20:09.223198+0000 (oldest deadline 2024-03-18T23:20:32.123184+0000) 2024-03-18T23:20:37.314 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 18 23:20:37 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-7[85590]: 2024-03-18T23:20:37.083+0000 7f5dd9117700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-03-18T23:20:08.415464+0000 front 2024-03-18T23:20:08.415444+0000 (oldest deadline 2024-03-18T23:20:33.115572+0000) 2024-03-18T23:20:37.314 INFO:journalctl@ceph.osd.4.smithi139.stdout:Mar 18 23:20:37 smithi139 podman[104826]: 2024-03-18 23:20:37.062548151 +0000 UTC m=+0.415721551 container start 75cb315802efe5b3f57ce2f75ecd4f7129ac4b71b6e2ca2fa13a6434e5408625 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-4-deactivate, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, ceph=True, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD) 2024-03-18T23:20:37.314 INFO:journalctl@ceph.osd.4.smithi139.stdout:Mar 18 23:20:37 smithi139 podman[104826]: 2024-03-18 23:20:37.151177677 +0000 UTC m=+0.504351080 container attach 75cb315802efe5b3f57ce2f75ecd4f7129ac4b71b6e2ca2fa13a6434e5408625 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-4-deactivate, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, ceph=True) 2024-03-18T23:20:37.315 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:20:37 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:20:37.092+0000 7f83cd240700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:19:57.904964+0000 front 2024-03-18T23:19:57.904988+0000 (oldest deadline 2024-03-18T23:20:23.805169+0000) 2024-03-18T23:20:37.315 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:20:37 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:20:37.092+0000 7f83cd240700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-03-18T23:20:10.806661+0000 front 2024-03-18T23:20:10.806621+0000 (oldest deadline 2024-03-18T23:20:31.906747+0000) 2024-03-18T23:20:37.628 INFO:journalctl@ceph.osd.6.smithi049.stdout:Mar 18 23:20:37 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-6[91523]: 2024-03-18T23:20:37.484+0000 7f92dba96700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:20:01.671603+0000 front 2024-03-18T23:20:01.671661+0000 (oldest deadline 2024-03-18T23:20:23.971756+0000) 2024-03-18T23:20:37.629 INFO:journalctl@ceph.osd.6.smithi049.stdout:Mar 18 23:20:37 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-6[91523]: 2024-03-18T23:20:37.484+0000 7f92dba96700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-03-18T23:20:08.573527+0000 front 2024-03-18T23:20:08.573679+0000 (oldest deadline 2024-03-18T23:20:32.673334+0000) 2024-03-18T23:20:37.629 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 18 23:20:37 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-9[98163]: 2024-03-18T23:20:37.361+0000 7ff64b26e700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:20:01.074760+0000 front 2024-03-18T23:20:01.074900+0000 (oldest deadline 2024-03-18T23:20:26.974968+0000) 2024-03-18T23:20:37.629 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 18 23:20:37 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-9[98163]: 2024-03-18T23:20:37.361+0000 7ff64b26e700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-03-18T23:20:10.976967+0000 front 2024-03-18T23:20:10.976857+0000 (oldest deadline 2024-03-18T23:20:36.876783+0000) 2024-03-18T23:20:37.653 INFO:journalctl@ceph.osd.4.smithi139.stdout:Mar 18 23:20:37 smithi139 podman[104826]: 2024-03-18 23:20:37.350811168 +0000 UTC m=+0.703984570 container died 75cb315802efe5b3f57ce2f75ecd4f7129ac4b71b6e2ca2fa13a6434e5408625 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-4-deactivate, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , ceph=True) 2024-03-18T23:20:37.710 INFO:journalctl@ceph.osd.8.smithi175.stdout:Mar 18 23:20:37 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-8[86041]: 2024-03-18T23:20:37.407+0000 7fa05c798700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:20:02.615291+0000 front 2024-03-18T23:20:02.615437+0000 (oldest deadline 2024-03-18T23:20:24.315195+0000) 2024-03-18T23:20:37.710 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:20:37 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:20:37.563+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:19:57.947723+0000 front 2024-03-18T23:19:57.947785+0000 (oldest deadline 2024-03-18T23:20:28.548565+0000) 2024-03-18T23:20:37.711 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:20:37 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:20:37.563+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-03-18T23:20:08.549207+0000 front 2024-03-18T23:20:08.549248+0000 (oldest deadline 2024-03-18T23:20:33.849233+0000) 2024-03-18T23:20:37.967 DEBUG:teuthology.orchestra.run.smithi139:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c63591ea-e57a-11ee-95c9-87774f69a715@osd.4.service' 2024-03-18T23:20:38.008 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-18T23:20:38.009 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-03-18T23:20:38.009 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-03-18T23:20:38.009 DEBUG:teuthology.orchestra.run.smithi175:> sudo systemctl stop ceph-c63591ea-e57a-11ee-95c9-87774f69a715@osd.5 2024-03-18T23:20:38.210 INFO:journalctl@ceph.osd.5.smithi175.stdout:Mar 18 23:20:37 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-5[79436]: 2024-03-18T23:20:37.852+0000 7fa764b06700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:20:01.121579+0000 front 2024-03-18T23:20:01.121445+0000 (oldest deadline 2024-03-18T23:20:25.721983+0000) 2024-03-18T23:20:38.210 INFO:journalctl@ceph.osd.5.smithi175.stdout:Mar 18 23:20:37 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-5[79436]: 2024-03-18T23:20:37.852+0000 7fa764b06700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-03-18T23:20:09.223527+0000 front 2024-03-18T23:20:09.223198+0000 (oldest deadline 2024-03-18T23:20:32.123184+0000) 2024-03-18T23:20:38.210 INFO:journalctl@ceph.osd.5.smithi175.stdout:Mar 18 23:20:38 smithi175 systemd[1]: Stopping Ceph osd.5 for c63591ea-e57a-11ee-95c9-87774f69a715... 2024-03-18T23:20:38.314 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 18 23:20:38 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-7[85590]: 2024-03-18T23:20:38.058+0000 7f5dd9117700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-03-18T23:20:08.415464+0000 front 2024-03-18T23:20:08.415444+0000 (oldest deadline 2024-03-18T23:20:33.115572+0000) 2024-03-18T23:20:38.314 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:20:38 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:20:38.132+0000 7f83cd240700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:19:57.904964+0000 front 2024-03-18T23:19:57.904988+0000 (oldest deadline 2024-03-18T23:20:23.805169+0000) 2024-03-18T23:20:38.314 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:20:38 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:20:38.132+0000 7f83cd240700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-03-18T23:20:10.806661+0000 front 2024-03-18T23:20:10.806621+0000 (oldest deadline 2024-03-18T23:20:31.906747+0000) 2024-03-18T23:20:38.604 INFO:journalctl@ceph.osd.8.smithi175.stdout:Mar 18 23:20:38 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-8[86041]: 2024-03-18T23:20:38.397+0000 7fa05c798700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:20:02.615291+0000 front 2024-03-18T23:20:02.615437+0000 (oldest deadline 2024-03-18T23:20:24.315195+0000) 2024-03-18T23:20:38.605 INFO:journalctl@ceph.osd.5.smithi175.stdout:Mar 18 23:20:38 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-5[79436]: 2024-03-18T23:20:38.343+0000 7fa76c51a700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-18T23:20:38.605 INFO:journalctl@ceph.osd.5.smithi175.stdout:Mar 18 23:20:38 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-5[79436]: 2024-03-18T23:20:38.343+0000 7fa76c51a700 -1 osd.5 104 *** Got signal Terminated *** 2024-03-18T23:20:38.605 INFO:journalctl@ceph.osd.5.smithi175.stdout:Mar 18 23:20:38 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-5[79436]: 2024-03-18T23:20:38.343+0000 7fa76c51a700 -1 osd.5 104 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-18T23:20:38.879 INFO:journalctl@ceph.osd.6.smithi049.stdout:Mar 18 23:20:38 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-6[91523]: 2024-03-18T23:20:38.460+0000 7f92dba96700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:20:01.671603+0000 front 2024-03-18T23:20:01.671661+0000 (oldest deadline 2024-03-18T23:20:23.971756+0000) 2024-03-18T23:20:38.879 INFO:journalctl@ceph.osd.6.smithi049.stdout:Mar 18 23:20:38 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-6[91523]: 2024-03-18T23:20:38.460+0000 7f92dba96700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-03-18T23:20:08.573527+0000 front 2024-03-18T23:20:08.573679+0000 (oldest deadline 2024-03-18T23:20:32.673334+0000) 2024-03-18T23:20:38.879 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 18 23:20:38 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-9[98163]: 2024-03-18T23:20:38.407+0000 7ff64b26e700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:20:01.074760+0000 front 2024-03-18T23:20:01.074900+0000 (oldest deadline 2024-03-18T23:20:26.974968+0000) 2024-03-18T23:20:38.879 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 18 23:20:38 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-9[98163]: 2024-03-18T23:20:38.407+0000 7ff64b26e700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-03-18T23:20:10.976967+0000 front 2024-03-18T23:20:10.976857+0000 (oldest deadline 2024-03-18T23:20:36.876783+0000) 2024-03-18T23:20:38.960 INFO:journalctl@ceph.osd.5.smithi175.stdout:Mar 18 23:20:38 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-5[79436]: 2024-03-18T23:20:38.848+0000 7fa764b06700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:20:01.121579+0000 front 2024-03-18T23:20:01.121445+0000 (oldest deadline 2024-03-18T23:20:25.721983+0000) 2024-03-18T23:20:38.960 INFO:journalctl@ceph.osd.5.smithi175.stdout:Mar 18 23:20:38 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-5[79436]: 2024-03-18T23:20:38.848+0000 7fa764b06700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-03-18T23:20:09.223527+0000 front 2024-03-18T23:20:09.223198+0000 (oldest deadline 2024-03-18T23:20:32.123184+0000) 2024-03-18T23:20:38.961 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:20:38 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:20:38.605+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:19:57.947723+0000 front 2024-03-18T23:19:57.947785+0000 (oldest deadline 2024-03-18T23:20:28.548565+0000) 2024-03-18T23:20:38.961 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:20:38 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:20:38.605+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-03-18T23:20:08.549207+0000 front 2024-03-18T23:20:08.549248+0000 (oldest deadline 2024-03-18T23:20:33.849233+0000) 2024-03-18T23:20:39.564 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 18 23:20:39 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-7[85590]: 2024-03-18T23:20:39.079+0000 7f5dd9117700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-03-18T23:20:08.415464+0000 front 2024-03-18T23:20:08.415444+0000 (oldest deadline 2024-03-18T23:20:33.115572+0000) 2024-03-18T23:20:39.564 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:20:39 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:20:39.143+0000 7f83cd240700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:19:57.904964+0000 front 2024-03-18T23:19:57.904988+0000 (oldest deadline 2024-03-18T23:20:23.805169+0000) 2024-03-18T23:20:39.564 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:20:39 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:20:39.143+0000 7f83cd240700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-03-18T23:20:10.806661+0000 front 2024-03-18T23:20:10.806621+0000 (oldest deadline 2024-03-18T23:20:31.906747+0000) 2024-03-18T23:20:39.710 INFO:journalctl@ceph.osd.8.smithi175.stdout:Mar 18 23:20:39 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-8[86041]: 2024-03-18T23:20:39.359+0000 7fa05c798700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:20:02.615291+0000 front 2024-03-18T23:20:02.615437+0000 (oldest deadline 2024-03-18T23:20:24.315195+0000) 2024-03-18T23:20:39.711 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:20:39 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:20:39.570+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:19:57.947723+0000 front 2024-03-18T23:19:57.947785+0000 (oldest deadline 2024-03-18T23:20:28.548565+0000) 2024-03-18T23:20:39.711 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:20:39 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:20:39.570+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-03-18T23:20:08.549207+0000 front 2024-03-18T23:20:08.549248+0000 (oldest deadline 2024-03-18T23:20:33.849233+0000) 2024-03-18T23:20:39.879 INFO:journalctl@ceph.osd.6.smithi049.stdout:Mar 18 23:20:39 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-6[91523]: 2024-03-18T23:20:39.452+0000 7f92dba96700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:20:01.671603+0000 front 2024-03-18T23:20:01.671661+0000 (oldest deadline 2024-03-18T23:20:23.971756+0000) 2024-03-18T23:20:39.879 INFO:journalctl@ceph.osd.6.smithi049.stdout:Mar 18 23:20:39 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-6[91523]: 2024-03-18T23:20:39.452+0000 7f92dba96700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-03-18T23:20:08.573527+0000 front 2024-03-18T23:20:08.573679+0000 (oldest deadline 2024-03-18T23:20:32.673334+0000) 2024-03-18T23:20:39.879 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 18 23:20:39 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-9[98163]: 2024-03-18T23:20:39.389+0000 7ff64b26e700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:20:01.074760+0000 front 2024-03-18T23:20:01.074900+0000 (oldest deadline 2024-03-18T23:20:26.974968+0000) 2024-03-18T23:20:39.879 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 18 23:20:39 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-9[98163]: 2024-03-18T23:20:39.389+0000 7ff64b26e700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-03-18T23:20:10.976967+0000 front 2024-03-18T23:20:10.976857+0000 (oldest deadline 2024-03-18T23:20:36.876783+0000) 2024-03-18T23:20:40.210 INFO:journalctl@ceph.osd.5.smithi175.stdout:Mar 18 23:20:39 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-5[79436]: 2024-03-18T23:20:39.856+0000 7fa764b06700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:20:01.121579+0000 front 2024-03-18T23:20:01.121445+0000 (oldest deadline 2024-03-18T23:20:25.721983+0000) 2024-03-18T23:20:40.210 INFO:journalctl@ceph.osd.5.smithi175.stdout:Mar 18 23:20:39 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-5[79436]: 2024-03-18T23:20:39.856+0000 7fa764b06700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-03-18T23:20:09.223527+0000 front 2024-03-18T23:20:09.223198+0000 (oldest deadline 2024-03-18T23:20:32.123184+0000) 2024-03-18T23:20:40.564 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 18 23:20:40 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-7[85590]: 2024-03-18T23:20:40.090+0000 7f5dd9117700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-03-18T23:20:08.415464+0000 front 2024-03-18T23:20:08.415444+0000 (oldest deadline 2024-03-18T23:20:33.115572+0000) 2024-03-18T23:20:40.564 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:20:40 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:20:40.163+0000 7f83cd240700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:19:57.904964+0000 front 2024-03-18T23:19:57.904988+0000 (oldest deadline 2024-03-18T23:20:23.805169+0000) 2024-03-18T23:20:40.564 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:20:40 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:20:40.163+0000 7f83cd240700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-03-18T23:20:10.806661+0000 front 2024-03-18T23:20:10.806621+0000 (oldest deadline 2024-03-18T23:20:31.906747+0000) 2024-03-18T23:20:40.710 INFO:journalctl@ceph.osd.8.smithi175.stdout:Mar 18 23:20:40 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-8[86041]: 2024-03-18T23:20:40.405+0000 7fa05c798700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:20:02.615291+0000 front 2024-03-18T23:20:02.615437+0000 (oldest deadline 2024-03-18T23:20:24.315195+0000) 2024-03-18T23:20:40.710 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:20:40 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:20:40.565+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:19:57.947723+0000 front 2024-03-18T23:19:57.947785+0000 (oldest deadline 2024-03-18T23:20:28.548565+0000) 2024-03-18T23:20:40.710 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:20:40 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:20:40.565+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-03-18T23:20:08.549207+0000 front 2024-03-18T23:20:08.549248+0000 (oldest deadline 2024-03-18T23:20:33.849233+0000) 2024-03-18T23:20:40.878 INFO:journalctl@ceph.osd.6.smithi049.stdout:Mar 18 23:20:40 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-6[91523]: 2024-03-18T23:20:40.423+0000 7f92dba96700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:20:01.671603+0000 front 2024-03-18T23:20:01.671661+0000 (oldest deadline 2024-03-18T23:20:23.971756+0000) 2024-03-18T23:20:40.879 INFO:journalctl@ceph.osd.6.smithi049.stdout:Mar 18 23:20:40 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-6[91523]: 2024-03-18T23:20:40.423+0000 7f92dba96700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-03-18T23:20:08.573527+0000 front 2024-03-18T23:20:08.573679+0000 (oldest deadline 2024-03-18T23:20:32.673334+0000) 2024-03-18T23:20:40.879 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 18 23:20:40 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-9[98163]: 2024-03-18T23:20:40.435+0000 7ff64b26e700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:20:01.074760+0000 front 2024-03-18T23:20:01.074900+0000 (oldest deadline 2024-03-18T23:20:26.974968+0000) 2024-03-18T23:20:40.879 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 18 23:20:40 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-9[98163]: 2024-03-18T23:20:40.435+0000 7ff64b26e700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-03-18T23:20:10.976967+0000 front 2024-03-18T23:20:10.976857+0000 (oldest deadline 2024-03-18T23:20:36.876783+0000) 2024-03-18T23:20:41.210 INFO:journalctl@ceph.osd.5.smithi175.stdout:Mar 18 23:20:40 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-5[79436]: 2024-03-18T23:20:40.893+0000 7fa764b06700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:20:01.121579+0000 front 2024-03-18T23:20:01.121445+0000 (oldest deadline 2024-03-18T23:20:25.721983+0000) 2024-03-18T23:20:41.210 INFO:journalctl@ceph.osd.5.smithi175.stdout:Mar 18 23:20:40 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-5[79436]: 2024-03-18T23:20:40.893+0000 7fa764b06700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-03-18T23:20:09.223527+0000 front 2024-03-18T23:20:09.223198+0000 (oldest deadline 2024-03-18T23:20:32.123184+0000) 2024-03-18T23:20:41.564 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 18 23:20:41 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-7[85590]: 2024-03-18T23:20:41.067+0000 7f5dd9117700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-03-18T23:20:08.415464+0000 front 2024-03-18T23:20:08.415444+0000 (oldest deadline 2024-03-18T23:20:33.115572+0000) 2024-03-18T23:20:41.564 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:20:41 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:20:41.189+0000 7f83cd240700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:19:57.904964+0000 front 2024-03-18T23:19:57.904988+0000 (oldest deadline 2024-03-18T23:20:23.805169+0000) 2024-03-18T23:20:41.564 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:20:41 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:20:41.189+0000 7f83cd240700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-03-18T23:20:10.806661+0000 front 2024-03-18T23:20:10.806621+0000 (oldest deadline 2024-03-18T23:20:31.906747+0000) 2024-03-18T23:20:41.710 INFO:journalctl@ceph.osd.8.smithi175.stdout:Mar 18 23:20:41 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-8[86041]: 2024-03-18T23:20:41.369+0000 7fa05c798700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:20:02.615291+0000 front 2024-03-18T23:20:02.615437+0000 (oldest deadline 2024-03-18T23:20:24.315195+0000) 2024-03-18T23:20:41.711 INFO:journalctl@ceph.osd.8.smithi175.stdout:Mar 18 23:20:41 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-8[86041]: 2024-03-18T23:20:41.369+0000 7fa05c798700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-03-18T23:20:17.717904+0000 front 2024-03-18T23:20:17.718012+0000 (oldest deadline 2024-03-18T23:20:41.218013+0000) 2024-03-18T23:20:41.711 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:20:41 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:20:41.587+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:19:57.947723+0000 front 2024-03-18T23:19:57.947785+0000 (oldest deadline 2024-03-18T23:20:28.548565+0000) 2024-03-18T23:20:41.711 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:20:41 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:20:41.587+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-03-18T23:20:08.549207+0000 front 2024-03-18T23:20:08.549248+0000 (oldest deadline 2024-03-18T23:20:33.849233+0000) 2024-03-18T23:20:41.712 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:20:41 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:20:41.587+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-03-18T23:20:16.150579+0000 front 2024-03-18T23:20:16.150606+0000 (oldest deadline 2024-03-18T23:20:40.751401+0000) 2024-03-18T23:20:41.879 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 18 23:20:41 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-9[98163]: 2024-03-18T23:20:41.393+0000 7ff64b26e700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:20:01.074760+0000 front 2024-03-18T23:20:01.074900+0000 (oldest deadline 2024-03-18T23:20:26.974968+0000) 2024-03-18T23:20:41.879 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 18 23:20:41 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-9[98163]: 2024-03-18T23:20:41.393+0000 7ff64b26e700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-03-18T23:20:10.976967+0000 front 2024-03-18T23:20:10.976857+0000 (oldest deadline 2024-03-18T23:20:36.876783+0000) 2024-03-18T23:20:41.879 INFO:journalctl@ceph.osd.6.smithi049.stdout:Mar 18 23:20:41 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-6[91523]: 2024-03-18T23:20:41.458+0000 7f92dba96700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:20:01.671603+0000 front 2024-03-18T23:20:01.671661+0000 (oldest deadline 2024-03-18T23:20:23.971756+0000) 2024-03-18T23:20:41.879 INFO:journalctl@ceph.osd.6.smithi049.stdout:Mar 18 23:20:41 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-6[91523]: 2024-03-18T23:20:41.458+0000 7f92dba96700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-03-18T23:20:08.573527+0000 front 2024-03-18T23:20:08.573679+0000 (oldest deadline 2024-03-18T23:20:32.673334+0000) 2024-03-18T23:20:42.210 INFO:journalctl@ceph.osd.5.smithi175.stdout:Mar 18 23:20:41 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-5[79436]: 2024-03-18T23:20:41.943+0000 7fa764b06700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:20:01.121579+0000 front 2024-03-18T23:20:01.121445+0000 (oldest deadline 2024-03-18T23:20:25.721983+0000) 2024-03-18T23:20:42.210 INFO:journalctl@ceph.osd.5.smithi175.stdout:Mar 18 23:20:41 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-5[79436]: 2024-03-18T23:20:41.943+0000 7fa764b06700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-03-18T23:20:09.223527+0000 front 2024-03-18T23:20:09.223198+0000 (oldest deadline 2024-03-18T23:20:32.123184+0000) 2024-03-18T23:20:42.564 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 18 23:20:42 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-7[85590]: 2024-03-18T23:20:42.091+0000 7f5dd9117700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-03-18T23:20:08.415464+0000 front 2024-03-18T23:20:08.415444+0000 (oldest deadline 2024-03-18T23:20:33.115572+0000) 2024-03-18T23:20:42.564 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:20:42 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:20:42.232+0000 7f83cd240700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:19:57.904964+0000 front 2024-03-18T23:19:57.904988+0000 (oldest deadline 2024-03-18T23:20:23.805169+0000) 2024-03-18T23:20:42.564 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:20:42 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:20:42.232+0000 7f83cd240700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-03-18T23:20:10.806661+0000 front 2024-03-18T23:20:10.806621+0000 (oldest deadline 2024-03-18T23:20:31.906747+0000) 2024-03-18T23:20:42.564 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:20:42 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:20:42.232+0000 7f83cd240700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-03-18T23:20:16.508876+0000 front 2024-03-18T23:20:16.508656+0000 (oldest deadline 2024-03-18T23:20:41.808738+0000) 2024-03-18T23:20:42.618 INFO:journalctl@ceph.osd.8.smithi175.stdout:Mar 18 23:20:42 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-8[86041]: 2024-03-18T23:20:42.355+0000 7fa05c798700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:20:02.615291+0000 front 2024-03-18T23:20:02.615437+0000 (oldest deadline 2024-03-18T23:20:24.315195+0000) 2024-03-18T23:20:42.619 INFO:journalctl@ceph.osd.8.smithi175.stdout:Mar 18 23:20:42 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-8[86041]: 2024-03-18T23:20:42.355+0000 7fa05c798700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-03-18T23:20:17.717904+0000 front 2024-03-18T23:20:17.718012+0000 (oldest deadline 2024-03-18T23:20:41.218013+0000) 2024-03-18T23:20:42.878 INFO:journalctl@ceph.osd.6.smithi049.stdout:Mar 18 23:20:42 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-6[91523]: 2024-03-18T23:20:42.421+0000 7f92dba96700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:20:01.671603+0000 front 2024-03-18T23:20:01.671661+0000 (oldest deadline 2024-03-18T23:20:23.971756+0000) 2024-03-18T23:20:42.879 INFO:journalctl@ceph.osd.6.smithi049.stdout:Mar 18 23:20:42 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-6[91523]: 2024-03-18T23:20:42.421+0000 7f92dba96700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-03-18T23:20:08.573527+0000 front 2024-03-18T23:20:08.573679+0000 (oldest deadline 2024-03-18T23:20:32.673334+0000) 2024-03-18T23:20:42.879 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 18 23:20:42 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-9[98163]: 2024-03-18T23:20:42.382+0000 7ff64b26e700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:20:01.074760+0000 front 2024-03-18T23:20:01.074900+0000 (oldest deadline 2024-03-18T23:20:26.974968+0000) 2024-03-18T23:20:42.879 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 18 23:20:42 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-9[98163]: 2024-03-18T23:20:42.382+0000 7ff64b26e700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-03-18T23:20:10.976967+0000 front 2024-03-18T23:20:10.976857+0000 (oldest deadline 2024-03-18T23:20:36.876783+0000) 2024-03-18T23:20:42.879 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 18 23:20:42 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-9[98163]: 2024-03-18T23:20:42.382+0000 7ff64b26e700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-03-18T23:20:16.877280+0000 front 2024-03-18T23:20:16.877254+0000 (oldest deadline 2024-03-18T23:20:41.577384+0000) 2024-03-18T23:20:42.960 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:20:42 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:20:42.618+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:19:57.947723+0000 front 2024-03-18T23:19:57.947785+0000 (oldest deadline 2024-03-18T23:20:28.548565+0000) 2024-03-18T23:20:42.960 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:20:42 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:20:42.618+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-03-18T23:20:08.549207+0000 front 2024-03-18T23:20:08.549248+0000 (oldest deadline 2024-03-18T23:20:33.849233+0000) 2024-03-18T23:20:42.960 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:20:42 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:20:42.618+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-03-18T23:20:16.150579+0000 front 2024-03-18T23:20:16.150606+0000 (oldest deadline 2024-03-18T23:20:40.751401+0000) 2024-03-18T23:20:43.310 INFO:journalctl@ceph.osd.5.smithi175.stdout:Mar 18 23:20:42 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-5[79436]: 2024-03-18T23:20:42.971+0000 7fa764b06700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:20:01.121579+0000 front 2024-03-18T23:20:01.121445+0000 (oldest deadline 2024-03-18T23:20:25.721983+0000) 2024-03-18T23:20:43.310 INFO:journalctl@ceph.osd.5.smithi175.stdout:Mar 18 23:20:42 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-5[79436]: 2024-03-18T23:20:42.971+0000 7fa764b06700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-03-18T23:20:09.223527+0000 front 2024-03-18T23:20:09.223198+0000 (oldest deadline 2024-03-18T23:20:32.123184+0000) 2024-03-18T23:20:43.314 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 18 23:20:43 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-7[85590]: 2024-03-18T23:20:43.044+0000 7f5dd9117700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-03-18T23:20:08.415464+0000 front 2024-03-18T23:20:08.415444+0000 (oldest deadline 2024-03-18T23:20:33.115572+0000) 2024-03-18T23:20:43.314 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 18 23:20:43 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-7[85590]: 2024-03-18T23:20:43.044+0000 7f5dd9117700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-03-18T23:20:15.917098+0000 front 2024-03-18T23:20:15.917063+0000 (oldest deadline 2024-03-18T23:20:42.917807+0000) 2024-03-18T23:20:43.314 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:20:43 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:20:43.209+0000 7f83cd240700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:19:57.904964+0000 front 2024-03-18T23:19:57.904988+0000 (oldest deadline 2024-03-18T23:20:23.805169+0000) 2024-03-18T23:20:43.315 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:20:43 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:20:43.209+0000 7f83cd240700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-03-18T23:20:10.806661+0000 front 2024-03-18T23:20:10.806621+0000 (oldest deadline 2024-03-18T23:20:31.906747+0000) 2024-03-18T23:20:43.315 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:20:43 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:20:43.209+0000 7f83cd240700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-03-18T23:20:16.508876+0000 front 2024-03-18T23:20:16.508656+0000 (oldest deadline 2024-03-18T23:20:41.808738+0000) 2024-03-18T23:20:43.593 INFO:journalctl@ceph.osd.8.smithi175.stdout:Mar 18 23:20:43 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-8[86041]: 2024-03-18T23:20:43.310+0000 7fa05c798700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:20:02.615291+0000 front 2024-03-18T23:20:02.615437+0000 (oldest deadline 2024-03-18T23:20:24.315195+0000) 2024-03-18T23:20:43.593 INFO:journalctl@ceph.osd.8.smithi175.stdout:Mar 18 23:20:43 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-8[86041]: 2024-03-18T23:20:43.310+0000 7fa05c798700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-03-18T23:20:17.717904+0000 front 2024-03-18T23:20:17.718012+0000 (oldest deadline 2024-03-18T23:20:41.218013+0000) 2024-03-18T23:20:43.629 INFO:journalctl@ceph.osd.6.smithi049.stdout:Mar 18 23:20:43 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-6[91523]: 2024-03-18T23:20:43.462+0000 7f92dba96700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:20:01.671603+0000 front 2024-03-18T23:20:01.671661+0000 (oldest deadline 2024-03-18T23:20:23.971756+0000) 2024-03-18T23:20:43.629 INFO:journalctl@ceph.osd.6.smithi049.stdout:Mar 18 23:20:43 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-6[91523]: 2024-03-18T23:20:43.462+0000 7f92dba96700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-03-18T23:20:08.573527+0000 front 2024-03-18T23:20:08.573679+0000 (oldest deadline 2024-03-18T23:20:32.673334+0000) 2024-03-18T23:20:43.629 INFO:journalctl@ceph.osd.6.smithi049.stdout:Mar 18 23:20:43 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-6[91523]: 2024-03-18T23:20:43.462+0000 7f92dba96700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-03-18T23:20:17.974330+0000 front 2024-03-18T23:20:17.974308+0000 (oldest deadline 2024-03-18T23:20:42.674209+0000) 2024-03-18T23:20:43.629 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 18 23:20:43 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-9[98163]: 2024-03-18T23:20:43.370+0000 7ff64b26e700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:20:01.074760+0000 front 2024-03-18T23:20:01.074900+0000 (oldest deadline 2024-03-18T23:20:26.974968+0000) 2024-03-18T23:20:43.629 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 18 23:20:43 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-9[98163]: 2024-03-18T23:20:43.370+0000 7ff64b26e700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-03-18T23:20:10.976967+0000 front 2024-03-18T23:20:10.976857+0000 (oldest deadline 2024-03-18T23:20:36.876783+0000) 2024-03-18T23:20:43.630 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 18 23:20:43 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-9[98163]: 2024-03-18T23:20:43.370+0000 7ff64b26e700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-03-18T23:20:16.877280+0000 front 2024-03-18T23:20:16.877254+0000 (oldest deadline 2024-03-18T23:20:41.577384+0000) 2024-03-18T23:20:43.849 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:20:43 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:20:43.593+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:19:57.947723+0000 front 2024-03-18T23:19:57.947785+0000 (oldest deadline 2024-03-18T23:20:28.548565+0000) 2024-03-18T23:20:43.850 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:20:43 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:20:43.593+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-03-18T23:20:08.549207+0000 front 2024-03-18T23:20:08.549248+0000 (oldest deadline 2024-03-18T23:20:33.849233+0000) 2024-03-18T23:20:43.850 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:20:43 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:20:43.593+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-03-18T23:20:16.150579+0000 front 2024-03-18T23:20:16.150606+0000 (oldest deadline 2024-03-18T23:20:40.751401+0000) 2024-03-18T23:20:44.127 INFO:journalctl@ceph.osd.5.smithi175.stdout:Mar 18 23:20:43 smithi175 podman[104728]: 2024-03-18 23:20:43.850696529 +0000 UTC m=+5.650289912 container died 35bce1dd2dd6095273d91e0bbbbeb19f5aaeabef063b7808561ece39d99edca5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-5, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-18T23:20:44.314 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 18 23:20:44 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-7[85590]: 2024-03-18T23:20:44.019+0000 7f5dd9117700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-03-18T23:20:08.415464+0000 front 2024-03-18T23:20:08.415444+0000 (oldest deadline 2024-03-18T23:20:33.115572+0000) 2024-03-18T23:20:44.314 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 18 23:20:44 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-7[85590]: 2024-03-18T23:20:44.019+0000 7f5dd9117700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-03-18T23:20:15.917098+0000 front 2024-03-18T23:20:15.917063+0000 (oldest deadline 2024-03-18T23:20:42.917807+0000) 2024-03-18T23:20:44.314 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:20:44 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:20:44.172+0000 7f83cd240700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:19:57.904964+0000 front 2024-03-18T23:19:57.904988+0000 (oldest deadline 2024-03-18T23:20:23.805169+0000) 2024-03-18T23:20:44.314 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:20:44 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:20:44.172+0000 7f83cd240700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-03-18T23:20:10.806661+0000 front 2024-03-18T23:20:10.806621+0000 (oldest deadline 2024-03-18T23:20:31.906747+0000) 2024-03-18T23:20:44.315 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:20:44 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:20:44.172+0000 7f83cd240700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-03-18T23:20:16.508876+0000 front 2024-03-18T23:20:16.508656+0000 (oldest deadline 2024-03-18T23:20:41.808738+0000) 2024-03-18T23:20:44.460 INFO:journalctl@ceph.osd.8.smithi175.stdout:Mar 18 23:20:44 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-8[86041]: 2024-03-18T23:20:44.272+0000 7fa05c798700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:20:02.615291+0000 front 2024-03-18T23:20:02.615437+0000 (oldest deadline 2024-03-18T23:20:24.315195+0000) 2024-03-18T23:20:44.460 INFO:journalctl@ceph.osd.8.smithi175.stdout:Mar 18 23:20:44 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-8[86041]: 2024-03-18T23:20:44.273+0000 7fa05c798700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-03-18T23:20:17.717904+0000 front 2024-03-18T23:20:17.718012+0000 (oldest deadline 2024-03-18T23:20:41.218013+0000) 2024-03-18T23:20:44.460 INFO:journalctl@ceph.osd.5.smithi175.stdout:Mar 18 23:20:44 smithi175 podman[104728]: 2024-03-18 23:20:44.228316598 +0000 UTC m=+6.027909971 container cleanup 35bce1dd2dd6095273d91e0bbbbeb19f5aaeabef063b7808561ece39d99edca5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-5, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True) 2024-03-18T23:20:44.460 INFO:journalctl@ceph.osd.5.smithi175.stdout:Mar 18 23:20:44 smithi175 bash[104728]: ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-5 2024-03-18T23:20:44.837 INFO:journalctl@ceph.osd.5.smithi175.stdout:Mar 18 23:20:44 smithi175 podman[104748]: 2024-03-18 23:20:44.487775729 +0000 UTC m=+0.633167576 container remove 35bce1dd2dd6095273d91e0bbbbeb19f5aaeabef063b7808561ece39d99edca5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-5, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240307) 2024-03-18T23:20:44.838 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:20:44 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:20:44.544+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:19:57.947723+0000 front 2024-03-18T23:19:57.947785+0000 (oldest deadline 2024-03-18T23:20:28.548565+0000) 2024-03-18T23:20:44.838 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:20:44 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:20:44.544+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-03-18T23:20:08.549207+0000 front 2024-03-18T23:20:08.549248+0000 (oldest deadline 2024-03-18T23:20:33.849233+0000) 2024-03-18T23:20:44.838 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:20:44 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:20:44.544+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-03-18T23:20:16.150579+0000 front 2024-03-18T23:20:16.150606+0000 (oldest deadline 2024-03-18T23:20:40.751401+0000) 2024-03-18T23:20:44.879 INFO:journalctl@ceph.osd.6.smithi049.stdout:Mar 18 23:20:44 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-6[91523]: 2024-03-18T23:20:44.435+0000 7f92dba96700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:20:01.671603+0000 front 2024-03-18T23:20:01.671661+0000 (oldest deadline 2024-03-18T23:20:23.971756+0000) 2024-03-18T23:20:44.879 INFO:journalctl@ceph.osd.6.smithi049.stdout:Mar 18 23:20:44 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-6[91523]: 2024-03-18T23:20:44.435+0000 7f92dba96700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-03-18T23:20:08.573527+0000 front 2024-03-18T23:20:08.573679+0000 (oldest deadline 2024-03-18T23:20:32.673334+0000) 2024-03-18T23:20:44.879 INFO:journalctl@ceph.osd.6.smithi049.stdout:Mar 18 23:20:44 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-6[91523]: 2024-03-18T23:20:44.435+0000 7f92dba96700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-03-18T23:20:17.974330+0000 front 2024-03-18T23:20:17.974308+0000 (oldest deadline 2024-03-18T23:20:42.674209+0000) 2024-03-18T23:20:44.879 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 18 23:20:44 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-9[98163]: 2024-03-18T23:20:44.388+0000 7ff64b26e700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:20:01.074760+0000 front 2024-03-18T23:20:01.074900+0000 (oldest deadline 2024-03-18T23:20:26.974968+0000) 2024-03-18T23:20:44.879 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 18 23:20:44 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-9[98163]: 2024-03-18T23:20:44.388+0000 7ff64b26e700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-03-18T23:20:10.976967+0000 front 2024-03-18T23:20:10.976857+0000 (oldest deadline 2024-03-18T23:20:36.876783+0000) 2024-03-18T23:20:44.880 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 18 23:20:44 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-9[98163]: 2024-03-18T23:20:44.388+0000 7ff64b26e700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-03-18T23:20:16.877280+0000 front 2024-03-18T23:20:16.877254+0000 (oldest deadline 2024-03-18T23:20:41.577384+0000) 2024-03-18T23:20:45.106 INFO:journalctl@ceph.osd.5.smithi175.stdout:Mar 18 23:20:44 smithi175 podman[104836]: 2024-03-18 23:20:44.838770524 +0000 UTC m=+0.112083362 container create 72c677b61eb311fa839ea5c90a6488e953260093b4a6e2f21995ceebee420b4a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-5-deactivate, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.build-date=20240307, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-03-18T23:20:45.107 INFO:journalctl@ceph.osd.5.smithi175.stdout:Mar 18 23:20:44 smithi175 podman[104836]: 2024-03-18 23:20:44.744551771 +0000 UTC m=+0.017864624 image pull 2312c8e42d0069a2dbb2364bddb00264db4b800e7f18a7e7e4eb8f3ecab2ff1c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639 2024-03-18T23:20:45.107 INFO:journalctl@ceph.osd.5.smithi175.stdout:Mar 18 23:20:45 smithi175 podman[104836]: 2024-03-18 23:20:45.108275827 +0000 UTC m=+0.381588675 container init 72c677b61eb311fa839ea5c90a6488e953260093b4a6e2f21995ceebee420b4a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-5-deactivate, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-03-18T23:20:45.314 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 18 23:20:45 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-7[85590]: 2024-03-18T23:20:45.051+0000 7f5dd9117700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-03-18T23:20:08.415464+0000 front 2024-03-18T23:20:08.415444+0000 (oldest deadline 2024-03-18T23:20:33.115572+0000) 2024-03-18T23:20:45.314 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 18 23:20:45 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-7[85590]: 2024-03-18T23:20:45.051+0000 7f5dd9117700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-03-18T23:20:15.917098+0000 front 2024-03-18T23:20:15.917063+0000 (oldest deadline 2024-03-18T23:20:42.917807+0000) 2024-03-18T23:20:45.314 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:20:45 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:20:45.159+0000 7f83cd240700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:19:57.904964+0000 front 2024-03-18T23:19:57.904988+0000 (oldest deadline 2024-03-18T23:20:23.805169+0000) 2024-03-18T23:20:45.315 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:20:45 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:20:45.159+0000 7f83cd240700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-03-18T23:20:10.806661+0000 front 2024-03-18T23:20:10.806621+0000 (oldest deadline 2024-03-18T23:20:31.906747+0000) 2024-03-18T23:20:45.315 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:20:45 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:20:45.159+0000 7f83cd240700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-03-18T23:20:16.508876+0000 front 2024-03-18T23:20:16.508656+0000 (oldest deadline 2024-03-18T23:20:41.808738+0000) 2024-03-18T23:20:45.397 INFO:journalctl@ceph.osd.8.smithi175.stdout:Mar 18 23:20:45 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-8[86041]: 2024-03-18T23:20:45.292+0000 7fa05c798700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:20:02.615291+0000 front 2024-03-18T23:20:02.615437+0000 (oldest deadline 2024-03-18T23:20:24.315195+0000) 2024-03-18T23:20:45.398 INFO:journalctl@ceph.osd.8.smithi175.stdout:Mar 18 23:20:45 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-8[86041]: 2024-03-18T23:20:45.292+0000 7fa05c798700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-03-18T23:20:17.717904+0000 front 2024-03-18T23:20:17.718012+0000 (oldest deadline 2024-03-18T23:20:41.218013+0000) 2024-03-18T23:20:45.398 INFO:journalctl@ceph.osd.5.smithi175.stdout:Mar 18 23:20:45 smithi175 podman[104836]: 2024-03-18 23:20:45.111337855 +0000 UTC m=+0.384650693 container start 72c677b61eb311fa839ea5c90a6488e953260093b4a6e2f21995ceebee420b4a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-5-deactivate, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_CLEAN=True) 2024-03-18T23:20:45.398 INFO:journalctl@ceph.osd.5.smithi175.stdout:Mar 18 23:20:45 smithi175 podman[104836]: 2024-03-18 23:20:45.199835427 +0000 UTC m=+0.473148292 container attach 72c677b61eb311fa839ea5c90a6488e953260093b4a6e2f21995ceebee420b4a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-5-deactivate, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-03-18T23:20:45.710 INFO:journalctl@ceph.osd.5.smithi175.stdout:Mar 18 23:20:45 smithi175 podman[104836]: 2024-03-18 23:20:45.39904609 +0000 UTC m=+0.672358944 container died 72c677b61eb311fa839ea5c90a6488e953260093b4a6e2f21995ceebee420b4a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-5-deactivate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-batrick-testing-20240318.181317-b6ae249) 2024-03-18T23:20:45.710 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:20:45 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:20:45.512+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:19:57.947723+0000 front 2024-03-18T23:19:57.947785+0000 (oldest deadline 2024-03-18T23:20:28.548565+0000) 2024-03-18T23:20:45.711 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:20:45 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:20:45.512+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-03-18T23:20:08.549207+0000 front 2024-03-18T23:20:08.549248+0000 (oldest deadline 2024-03-18T23:20:33.849233+0000) 2024-03-18T23:20:45.711 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:20:45 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:20:45.512+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-03-18T23:20:16.150579+0000 front 2024-03-18T23:20:16.150606+0000 (oldest deadline 2024-03-18T23:20:40.751401+0000) 2024-03-18T23:20:45.879 INFO:journalctl@ceph.osd.6.smithi049.stdout:Mar 18 23:20:45 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-6[91523]: 2024-03-18T23:20:45.415+0000 7f92dba96700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:20:01.671603+0000 front 2024-03-18T23:20:01.671661+0000 (oldest deadline 2024-03-18T23:20:23.971756+0000) 2024-03-18T23:20:45.879 INFO:journalctl@ceph.osd.6.smithi049.stdout:Mar 18 23:20:45 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-6[91523]: 2024-03-18T23:20:45.415+0000 7f92dba96700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-03-18T23:20:08.573527+0000 front 2024-03-18T23:20:08.573679+0000 (oldest deadline 2024-03-18T23:20:32.673334+0000) 2024-03-18T23:20:45.879 INFO:journalctl@ceph.osd.6.smithi049.stdout:Mar 18 23:20:45 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-6[91523]: 2024-03-18T23:20:45.415+0000 7f92dba96700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-03-18T23:20:17.974330+0000 front 2024-03-18T23:20:17.974308+0000 (oldest deadline 2024-03-18T23:20:42.674209+0000) 2024-03-18T23:20:45.879 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 18 23:20:45 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-9[98163]: 2024-03-18T23:20:45.422+0000 7ff64b26e700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:20:01.074760+0000 front 2024-03-18T23:20:01.074900+0000 (oldest deadline 2024-03-18T23:20:26.974968+0000) 2024-03-18T23:20:45.880 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 18 23:20:45 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-9[98163]: 2024-03-18T23:20:45.422+0000 7ff64b26e700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-03-18T23:20:10.976967+0000 front 2024-03-18T23:20:10.976857+0000 (oldest deadline 2024-03-18T23:20:36.876783+0000) 2024-03-18T23:20:45.880 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 18 23:20:45 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-9[98163]: 2024-03-18T23:20:45.422+0000 7ff64b26e700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-03-18T23:20:16.877280+0000 front 2024-03-18T23:20:16.877254+0000 (oldest deadline 2024-03-18T23:20:41.577384+0000) 2024-03-18T23:20:46.056 DEBUG:teuthology.orchestra.run.smithi175:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c63591ea-e57a-11ee-95c9-87774f69a715@osd.5.service' 2024-03-18T23:20:46.100 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-18T23:20:46.100 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-03-18T23:20:46.100 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-03-18T23:20:46.101 DEBUG:teuthology.orchestra.run.smithi049:> sudo systemctl stop ceph-c63591ea-e57a-11ee-95c9-87774f69a715@osd.6 2024-03-18T23:20:46.314 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 18 23:20:46 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-7[85590]: 2024-03-18T23:20:46.060+0000 7f5dd9117700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-03-18T23:20:08.415464+0000 front 2024-03-18T23:20:08.415444+0000 (oldest deadline 2024-03-18T23:20:33.115572+0000) 2024-03-18T23:20:46.314 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 18 23:20:46 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-7[85590]: 2024-03-18T23:20:46.060+0000 7f5dd9117700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-03-18T23:20:15.917098+0000 front 2024-03-18T23:20:15.917063+0000 (oldest deadline 2024-03-18T23:20:42.917807+0000) 2024-03-18T23:20:46.314 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:20:46 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:20:46.156+0000 7f83cd240700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:19:57.904964+0000 front 2024-03-18T23:19:57.904988+0000 (oldest deadline 2024-03-18T23:20:23.805169+0000) 2024-03-18T23:20:46.314 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:20:46 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:20:46.156+0000 7f83cd240700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-03-18T23:20:10.806661+0000 front 2024-03-18T23:20:10.806621+0000 (oldest deadline 2024-03-18T23:20:31.906747+0000) 2024-03-18T23:20:46.315 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:20:46 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:20:46.156+0000 7f83cd240700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-03-18T23:20:16.508876+0000 front 2024-03-18T23:20:16.508656+0000 (oldest deadline 2024-03-18T23:20:41.808738+0000) 2024-03-18T23:20:46.426 INFO:journalctl@ceph.osd.6.smithi049.stdout:Mar 18 23:20:46 smithi049 systemd[1]: Stopping Ceph osd.6 for c63591ea-e57a-11ee-95c9-87774f69a715... 2024-03-18T23:20:46.710 INFO:journalctl@ceph.osd.8.smithi175.stdout:Mar 18 23:20:46 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-8[86041]: 2024-03-18T23:20:46.337+0000 7fa05c798700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:20:02.615291+0000 front 2024-03-18T23:20:02.615437+0000 (oldest deadline 2024-03-18T23:20:24.315195+0000) 2024-03-18T23:20:46.710 INFO:journalctl@ceph.osd.8.smithi175.stdout:Mar 18 23:20:46 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-8[86041]: 2024-03-18T23:20:46.337+0000 7fa05c798700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-03-18T23:20:17.717904+0000 front 2024-03-18T23:20:17.718012+0000 (oldest deadline 2024-03-18T23:20:41.218013+0000) 2024-03-18T23:20:46.710 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:20:46 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:20:46.531+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:19:57.947723+0000 front 2024-03-18T23:19:57.947785+0000 (oldest deadline 2024-03-18T23:20:28.548565+0000) 2024-03-18T23:20:46.711 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:20:46 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:20:46.531+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-03-18T23:20:08.549207+0000 front 2024-03-18T23:20:08.549248+0000 (oldest deadline 2024-03-18T23:20:33.849233+0000) 2024-03-18T23:20:46.711 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:20:46 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:20:46.531+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-03-18T23:20:16.150579+0000 front 2024-03-18T23:20:16.150606+0000 (oldest deadline 2024-03-18T23:20:40.751401+0000) 2024-03-18T23:20:46.879 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 18 23:20:46 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-9[98163]: 2024-03-18T23:20:46.424+0000 7ff64b26e700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:20:01.074760+0000 front 2024-03-18T23:20:01.074900+0000 (oldest deadline 2024-03-18T23:20:26.974968+0000) 2024-03-18T23:20:46.879 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 18 23:20:46 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-9[98163]: 2024-03-18T23:20:46.424+0000 7ff64b26e700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-03-18T23:20:10.976967+0000 front 2024-03-18T23:20:10.976857+0000 (oldest deadline 2024-03-18T23:20:36.876783+0000) 2024-03-18T23:20:46.879 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 18 23:20:46 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-9[98163]: 2024-03-18T23:20:46.424+0000 7ff64b26e700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-03-18T23:20:16.877280+0000 front 2024-03-18T23:20:16.877254+0000 (oldest deadline 2024-03-18T23:20:41.577384+0000) 2024-03-18T23:20:46.879 INFO:journalctl@ceph.osd.6.smithi049.stdout:Mar 18 23:20:46 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-6[91523]: 2024-03-18T23:20:46.444+0000 7f92e34aa700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-18T23:20:46.880 INFO:journalctl@ceph.osd.6.smithi049.stdout:Mar 18 23:20:46 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-6[91523]: 2024-03-18T23:20:46.444+0000 7f92e34aa700 -1 osd.6 104 *** Got signal Terminated *** 2024-03-18T23:20:46.880 INFO:journalctl@ceph.osd.6.smithi049.stdout:Mar 18 23:20:46 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-6[91523]: 2024-03-18T23:20:46.444+0000 7f92e34aa700 -1 osd.6 104 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-18T23:20:46.880 INFO:journalctl@ceph.osd.6.smithi049.stdout:Mar 18 23:20:46 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-6[91523]: 2024-03-18T23:20:46.456+0000 7f92dba96700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:20:01.671603+0000 front 2024-03-18T23:20:01.671661+0000 (oldest deadline 2024-03-18T23:20:23.971756+0000) 2024-03-18T23:20:46.880 INFO:journalctl@ceph.osd.6.smithi049.stdout:Mar 18 23:20:46 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-6[91523]: 2024-03-18T23:20:46.456+0000 7f92dba96700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-03-18T23:20:08.573527+0000 front 2024-03-18T23:20:08.573679+0000 (oldest deadline 2024-03-18T23:20:32.673334+0000) 2024-03-18T23:20:46.880 INFO:journalctl@ceph.osd.6.smithi049.stdout:Mar 18 23:20:46 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-6[91523]: 2024-03-18T23:20:46.456+0000 7f92dba96700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-03-18T23:20:17.974330+0000 front 2024-03-18T23:20:17.974308+0000 (oldest deadline 2024-03-18T23:20:42.674209+0000) 2024-03-18T23:20:47.314 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 18 23:20:47 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-7[85590]: 2024-03-18T23:20:47.022+0000 7f5dd9117700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-03-18T23:20:08.415464+0000 front 2024-03-18T23:20:08.415444+0000 (oldest deadline 2024-03-18T23:20:33.115572+0000) 2024-03-18T23:20:47.314 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 18 23:20:47 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-7[85590]: 2024-03-18T23:20:47.022+0000 7f5dd9117700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-03-18T23:20:15.917098+0000 front 2024-03-18T23:20:15.917063+0000 (oldest deadline 2024-03-18T23:20:42.917807+0000) 2024-03-18T23:20:47.314 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:20:47 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:20:47.118+0000 7f83cd240700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:19:57.904964+0000 front 2024-03-18T23:19:57.904988+0000 (oldest deadline 2024-03-18T23:20:23.805169+0000) 2024-03-18T23:20:47.314 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:20:47 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:20:47.118+0000 7f83cd240700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-03-18T23:20:10.806661+0000 front 2024-03-18T23:20:10.806621+0000 (oldest deadline 2024-03-18T23:20:31.906747+0000) 2024-03-18T23:20:47.315 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:20:47 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:20:47.118+0000 7f83cd240700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-03-18T23:20:16.508876+0000 front 2024-03-18T23:20:16.508656+0000 (oldest deadline 2024-03-18T23:20:41.808738+0000) 2024-03-18T23:20:47.490 INFO:journalctl@ceph.osd.8.smithi175.stdout:Mar 18 23:20:47 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-8[86041]: 2024-03-18T23:20:47.356+0000 7fa05c798700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:20:02.615291+0000 front 2024-03-18T23:20:02.615437+0000 (oldest deadline 2024-03-18T23:20:24.315195+0000) 2024-03-18T23:20:47.490 INFO:journalctl@ceph.osd.8.smithi175.stdout:Mar 18 23:20:47 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-8[86041]: 2024-03-18T23:20:47.356+0000 7fa05c798700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-03-18T23:20:17.717904+0000 front 2024-03-18T23:20:17.718012+0000 (oldest deadline 2024-03-18T23:20:41.218013+0000) 2024-03-18T23:20:47.490 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:20:47 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:20:47.490+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:19:57.947723+0000 front 2024-03-18T23:19:57.947785+0000 (oldest deadline 2024-03-18T23:20:28.548565+0000) 2024-03-18T23:20:47.491 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:20:47 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:20:47.490+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-03-18T23:20:08.549207+0000 front 2024-03-18T23:20:08.549248+0000 (oldest deadline 2024-03-18T23:20:33.849233+0000) 2024-03-18T23:20:47.491 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:20:47 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:20:47.490+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-03-18T23:20:16.150579+0000 front 2024-03-18T23:20:16.150606+0000 (oldest deadline 2024-03-18T23:20:40.751401+0000) 2024-03-18T23:20:47.878 INFO:journalctl@ceph.osd.6.smithi049.stdout:Mar 18 23:20:47 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-6[91523]: 2024-03-18T23:20:47.435+0000 7f92dba96700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:20:01.671603+0000 front 2024-03-18T23:20:01.671661+0000 (oldest deadline 2024-03-18T23:20:23.971756+0000) 2024-03-18T23:20:47.879 INFO:journalctl@ceph.osd.6.smithi049.stdout:Mar 18 23:20:47 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-6[91523]: 2024-03-18T23:20:47.435+0000 7f92dba96700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-03-18T23:20:08.573527+0000 front 2024-03-18T23:20:08.573679+0000 (oldest deadline 2024-03-18T23:20:32.673334+0000) 2024-03-18T23:20:47.879 INFO:journalctl@ceph.osd.6.smithi049.stdout:Mar 18 23:20:47 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-6[91523]: 2024-03-18T23:20:47.435+0000 7f92dba96700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-03-18T23:20:17.974330+0000 front 2024-03-18T23:20:17.974308+0000 (oldest deadline 2024-03-18T23:20:42.674209+0000) 2024-03-18T23:20:47.879 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 18 23:20:47 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-9[98163]: 2024-03-18T23:20:47.469+0000 7ff64b26e700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:20:01.074760+0000 front 2024-03-18T23:20:01.074900+0000 (oldest deadline 2024-03-18T23:20:26.974968+0000) 2024-03-18T23:20:47.879 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 18 23:20:47 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-9[98163]: 2024-03-18T23:20:47.469+0000 7ff64b26e700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-03-18T23:20:10.976967+0000 front 2024-03-18T23:20:10.976857+0000 (oldest deadline 2024-03-18T23:20:36.876783+0000) 2024-03-18T23:20:47.879 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 18 23:20:47 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-9[98163]: 2024-03-18T23:20:47.469+0000 7ff64b26e700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-03-18T23:20:16.877280+0000 front 2024-03-18T23:20:16.877254+0000 (oldest deadline 2024-03-18T23:20:41.577384+0000) 2024-03-18T23:20:48.314 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 18 23:20:48 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-7[85590]: 2024-03-18T23:20:48.019+0000 7f5dd9117700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-03-18T23:20:08.415464+0000 front 2024-03-18T23:20:08.415444+0000 (oldest deadline 2024-03-18T23:20:33.115572+0000) 2024-03-18T23:20:48.314 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 18 23:20:48 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-7[85590]: 2024-03-18T23:20:48.019+0000 7f5dd9117700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-03-18T23:20:15.917098+0000 front 2024-03-18T23:20:15.917063+0000 (oldest deadline 2024-03-18T23:20:42.917807+0000) 2024-03-18T23:20:48.314 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:20:48 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:20:48.091+0000 7f83cd240700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:19:57.904964+0000 front 2024-03-18T23:19:57.904988+0000 (oldest deadline 2024-03-18T23:20:23.805169+0000) 2024-03-18T23:20:48.314 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:20:48 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:20:48.091+0000 7f83cd240700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-03-18T23:20:10.806661+0000 front 2024-03-18T23:20:10.806621+0000 (oldest deadline 2024-03-18T23:20:31.906747+0000) 2024-03-18T23:20:48.315 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:20:48 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:20:48.091+0000 7f83cd240700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-03-18T23:20:16.508876+0000 front 2024-03-18T23:20:16.508656+0000 (oldest deadline 2024-03-18T23:20:41.808738+0000) 2024-03-18T23:20:48.710 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:20:48 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:20:48.487+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:19:57.947723+0000 front 2024-03-18T23:19:57.947785+0000 (oldest deadline 2024-03-18T23:20:28.548565+0000) 2024-03-18T23:20:48.710 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:20:48 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:20:48.487+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-03-18T23:20:08.549207+0000 front 2024-03-18T23:20:08.549248+0000 (oldest deadline 2024-03-18T23:20:33.849233+0000) 2024-03-18T23:20:48.710 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:20:48 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:20:48.487+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-03-18T23:20:16.150579+0000 front 2024-03-18T23:20:16.150606+0000 (oldest deadline 2024-03-18T23:20:40.751401+0000) 2024-03-18T23:20:48.710 INFO:journalctl@ceph.osd.8.smithi175.stdout:Mar 18 23:20:48 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-8[86041]: 2024-03-18T23:20:48.403+0000 7fa05c798700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:20:02.615291+0000 front 2024-03-18T23:20:02.615437+0000 (oldest deadline 2024-03-18T23:20:24.315195+0000) 2024-03-18T23:20:48.711 INFO:journalctl@ceph.osd.8.smithi175.stdout:Mar 18 23:20:48 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-8[86041]: 2024-03-18T23:20:48.403+0000 7fa05c798700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-03-18T23:20:17.717904+0000 front 2024-03-18T23:20:17.718012+0000 (oldest deadline 2024-03-18T23:20:41.218013+0000) 2024-03-18T23:20:48.878 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 18 23:20:48 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-9[98163]: 2024-03-18T23:20:48.437+0000 7ff64b26e700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:20:01.074760+0000 front 2024-03-18T23:20:01.074900+0000 (oldest deadline 2024-03-18T23:20:26.974968+0000) 2024-03-18T23:20:48.879 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 18 23:20:48 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-9[98163]: 2024-03-18T23:20:48.437+0000 7ff64b26e700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-03-18T23:20:10.976967+0000 front 2024-03-18T23:20:10.976857+0000 (oldest deadline 2024-03-18T23:20:36.876783+0000) 2024-03-18T23:20:48.879 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 18 23:20:48 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-9[98163]: 2024-03-18T23:20:48.437+0000 7ff64b26e700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-03-18T23:20:16.877280+0000 front 2024-03-18T23:20:16.877254+0000 (oldest deadline 2024-03-18T23:20:41.577384+0000) 2024-03-18T23:20:48.879 INFO:journalctl@ceph.osd.6.smithi049.stdout:Mar 18 23:20:48 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-6[91523]: 2024-03-18T23:20:48.408+0000 7f92dba96700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:20:01.671603+0000 front 2024-03-18T23:20:01.671661+0000 (oldest deadline 2024-03-18T23:20:23.971756+0000) 2024-03-18T23:20:48.879 INFO:journalctl@ceph.osd.6.smithi049.stdout:Mar 18 23:20:48 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-6[91523]: 2024-03-18T23:20:48.408+0000 7f92dba96700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-03-18T23:20:08.573527+0000 front 2024-03-18T23:20:08.573679+0000 (oldest deadline 2024-03-18T23:20:32.673334+0000) 2024-03-18T23:20:48.879 INFO:journalctl@ceph.osd.6.smithi049.stdout:Mar 18 23:20:48 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-6[91523]: 2024-03-18T23:20:48.408+0000 7f92dba96700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-03-18T23:20:17.974330+0000 front 2024-03-18T23:20:17.974308+0000 (oldest deadline 2024-03-18T23:20:42.674209+0000) 2024-03-18T23:20:49.314 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 18 23:20:49 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-7[85590]: 2024-03-18T23:20:49.034+0000 7f5dd9117700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-03-18T23:20:08.415464+0000 front 2024-03-18T23:20:08.415444+0000 (oldest deadline 2024-03-18T23:20:33.115572+0000) 2024-03-18T23:20:49.314 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 18 23:20:49 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-7[85590]: 2024-03-18T23:20:49.034+0000 7f5dd9117700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-03-18T23:20:15.917098+0000 front 2024-03-18T23:20:15.917063+0000 (oldest deadline 2024-03-18T23:20:42.917807+0000) 2024-03-18T23:20:49.314 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:20:49 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:20:49.079+0000 7f83cd240700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:19:57.904964+0000 front 2024-03-18T23:19:57.904988+0000 (oldest deadline 2024-03-18T23:20:23.805169+0000) 2024-03-18T23:20:49.315 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:20:49 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:20:49.079+0000 7f83cd240700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-03-18T23:20:10.806661+0000 front 2024-03-18T23:20:10.806621+0000 (oldest deadline 2024-03-18T23:20:31.906747+0000) 2024-03-18T23:20:49.315 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:20:49 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:20:49.079+0000 7f83cd240700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-03-18T23:20:16.508876+0000 front 2024-03-18T23:20:16.508656+0000 (oldest deadline 2024-03-18T23:20:41.808738+0000) 2024-03-18T23:20:49.710 INFO:journalctl@ceph.osd.8.smithi175.stdout:Mar 18 23:20:49 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-8[86041]: 2024-03-18T23:20:49.422+0000 7fa05c798700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:20:02.615291+0000 front 2024-03-18T23:20:02.615437+0000 (oldest deadline 2024-03-18T23:20:24.315195+0000) 2024-03-18T23:20:49.710 INFO:journalctl@ceph.osd.8.smithi175.stdout:Mar 18 23:20:49 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-8[86041]: 2024-03-18T23:20:49.422+0000 7fa05c798700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-03-18T23:20:17.717904+0000 front 2024-03-18T23:20:17.718012+0000 (oldest deadline 2024-03-18T23:20:41.218013+0000) 2024-03-18T23:20:49.710 INFO:journalctl@ceph.osd.8.smithi175.stdout:Mar 18 23:20:49 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-8[86041]: 2024-03-18T23:20:49.422+0000 7fa05c798700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-18T23:20:23.519110+0000 front 2024-03-18T23:20:23.519209+0000 (oldest deadline 2024-03-18T23:20:48.818956+0000) 2024-03-18T23:20:49.711 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:20:49 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:20:49.532+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:19:57.947723+0000 front 2024-03-18T23:19:57.947785+0000 (oldest deadline 2024-03-18T23:20:28.548565+0000) 2024-03-18T23:20:49.711 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:20:49 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:20:49.532+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-03-18T23:20:08.549207+0000 front 2024-03-18T23:20:08.549248+0000 (oldest deadline 2024-03-18T23:20:33.849233+0000) 2024-03-18T23:20:49.711 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:20:49 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:20:49.532+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-03-18T23:20:16.150579+0000 front 2024-03-18T23:20:16.150606+0000 (oldest deadline 2024-03-18T23:20:40.751401+0000) 2024-03-18T23:20:49.879 INFO:journalctl@ceph.osd.6.smithi049.stdout:Mar 18 23:20:49 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-6[91523]: 2024-03-18T23:20:49.391+0000 7f92dba96700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:20:01.671603+0000 front 2024-03-18T23:20:01.671661+0000 (oldest deadline 2024-03-18T23:20:23.971756+0000) 2024-03-18T23:20:49.879 INFO:journalctl@ceph.osd.6.smithi049.stdout:Mar 18 23:20:49 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-6[91523]: 2024-03-18T23:20:49.391+0000 7f92dba96700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-03-18T23:20:08.573527+0000 front 2024-03-18T23:20:08.573679+0000 (oldest deadline 2024-03-18T23:20:32.673334+0000) 2024-03-18T23:20:49.879 INFO:journalctl@ceph.osd.6.smithi049.stdout:Mar 18 23:20:49 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-6[91523]: 2024-03-18T23:20:49.391+0000 7f92dba96700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-03-18T23:20:17.974330+0000 front 2024-03-18T23:20:17.974308+0000 (oldest deadline 2024-03-18T23:20:42.674209+0000) 2024-03-18T23:20:49.879 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 18 23:20:49 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-9[98163]: 2024-03-18T23:20:49.442+0000 7ff64b26e700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:20:01.074760+0000 front 2024-03-18T23:20:01.074900+0000 (oldest deadline 2024-03-18T23:20:26.974968+0000) 2024-03-18T23:20:49.879 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 18 23:20:49 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-9[98163]: 2024-03-18T23:20:49.442+0000 7ff64b26e700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-03-18T23:20:10.976967+0000 front 2024-03-18T23:20:10.976857+0000 (oldest deadline 2024-03-18T23:20:36.876783+0000) 2024-03-18T23:20:49.880 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 18 23:20:49 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-9[98163]: 2024-03-18T23:20:49.442+0000 7ff64b26e700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-03-18T23:20:16.877280+0000 front 2024-03-18T23:20:16.877254+0000 (oldest deadline 2024-03-18T23:20:41.577384+0000) 2024-03-18T23:20:49.880 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 18 23:20:49 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-9[98163]: 2024-03-18T23:20:49.442+0000 7ff64b26e700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-18T23:20:23.278575+0000 front 2024-03-18T23:20:23.278614+0000 (oldest deadline 2024-03-18T23:20:49.178702+0000) 2024-03-18T23:20:50.314 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 18 23:20:50 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-7[85590]: 2024-03-18T23:20:50.042+0000 7f5dd9117700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-03-18T23:20:08.415464+0000 front 2024-03-18T23:20:08.415444+0000 (oldest deadline 2024-03-18T23:20:33.115572+0000) 2024-03-18T23:20:50.314 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 18 23:20:50 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-7[85590]: 2024-03-18T23:20:50.042+0000 7f5dd9117700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-03-18T23:20:15.917098+0000 front 2024-03-18T23:20:15.917063+0000 (oldest deadline 2024-03-18T23:20:42.917807+0000) 2024-03-18T23:20:50.314 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:20:50 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:20:50.121+0000 7f83cd240700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:19:57.904964+0000 front 2024-03-18T23:19:57.904988+0000 (oldest deadline 2024-03-18T23:20:23.805169+0000) 2024-03-18T23:20:50.314 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:20:50 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:20:50.121+0000 7f83cd240700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-03-18T23:20:10.806661+0000 front 2024-03-18T23:20:10.806621+0000 (oldest deadline 2024-03-18T23:20:31.906747+0000) 2024-03-18T23:20:50.315 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:20:50 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:20:50.121+0000 7f83cd240700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-03-18T23:20:16.508876+0000 front 2024-03-18T23:20:16.508656+0000 (oldest deadline 2024-03-18T23:20:41.808738+0000) 2024-03-18T23:20:50.629 INFO:journalctl@ceph.osd.6.smithi049.stdout:Mar 18 23:20:50 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-6[91523]: 2024-03-18T23:20:50.377+0000 7f92dba96700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:20:01.671603+0000 front 2024-03-18T23:20:01.671661+0000 (oldest deadline 2024-03-18T23:20:23.971756+0000) 2024-03-18T23:20:50.629 INFO:journalctl@ceph.osd.6.smithi049.stdout:Mar 18 23:20:50 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-6[91523]: 2024-03-18T23:20:50.377+0000 7f92dba96700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-03-18T23:20:08.573527+0000 front 2024-03-18T23:20:08.573679+0000 (oldest deadline 2024-03-18T23:20:32.673334+0000) 2024-03-18T23:20:50.629 INFO:journalctl@ceph.osd.6.smithi049.stdout:Mar 18 23:20:50 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-6[91523]: 2024-03-18T23:20:50.377+0000 7f92dba96700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-03-18T23:20:17.974330+0000 front 2024-03-18T23:20:17.974308+0000 (oldest deadline 2024-03-18T23:20:42.674209+0000) 2024-03-18T23:20:50.629 INFO:journalctl@ceph.osd.6.smithi049.stdout:Mar 18 23:20:50 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-6[91523]: 2024-03-18T23:20:50.377+0000 7f92dba96700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-18T23:20:24.975188+0000 front 2024-03-18T23:20:24.975251+0000 (oldest deadline 2024-03-18T23:20:49.675258+0000) 2024-03-18T23:20:50.630 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 18 23:20:50 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-9[98163]: 2024-03-18T23:20:50.428+0000 7ff64b26e700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:20:01.074760+0000 front 2024-03-18T23:20:01.074900+0000 (oldest deadline 2024-03-18T23:20:26.974968+0000) 2024-03-18T23:20:50.630 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 18 23:20:50 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-9[98163]: 2024-03-18T23:20:50.428+0000 7ff64b26e700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-03-18T23:20:10.976967+0000 front 2024-03-18T23:20:10.976857+0000 (oldest deadline 2024-03-18T23:20:36.876783+0000) 2024-03-18T23:20:50.630 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 18 23:20:50 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-9[98163]: 2024-03-18T23:20:50.428+0000 7ff64b26e700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-03-18T23:20:16.877280+0000 front 2024-03-18T23:20:16.877254+0000 (oldest deadline 2024-03-18T23:20:41.577384+0000) 2024-03-18T23:20:50.630 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 18 23:20:50 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-9[98163]: 2024-03-18T23:20:50.428+0000 7ff64b26e700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-18T23:20:23.278575+0000 front 2024-03-18T23:20:23.278614+0000 (oldest deadline 2024-03-18T23:20:49.178702+0000) 2024-03-18T23:20:50.710 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:20:50 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:20:50.571+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:19:57.947723+0000 front 2024-03-18T23:19:57.947785+0000 (oldest deadline 2024-03-18T23:20:28.548565+0000) 2024-03-18T23:20:50.710 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:20:50 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:20:50.571+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-03-18T23:20:08.549207+0000 front 2024-03-18T23:20:08.549248+0000 (oldest deadline 2024-03-18T23:20:33.849233+0000) 2024-03-18T23:20:50.710 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:20:50 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:20:50.571+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-03-18T23:20:16.150579+0000 front 2024-03-18T23:20:16.150606+0000 (oldest deadline 2024-03-18T23:20:40.751401+0000) 2024-03-18T23:20:50.711 INFO:journalctl@ceph.osd.8.smithi175.stdout:Mar 18 23:20:50 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-8[86041]: 2024-03-18T23:20:50.432+0000 7fa05c798700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:20:02.615291+0000 front 2024-03-18T23:20:02.615437+0000 (oldest deadline 2024-03-18T23:20:24.315195+0000) 2024-03-18T23:20:50.711 INFO:journalctl@ceph.osd.8.smithi175.stdout:Mar 18 23:20:50 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-8[86041]: 2024-03-18T23:20:50.432+0000 7fa05c798700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-03-18T23:20:17.717904+0000 front 2024-03-18T23:20:17.718012+0000 (oldest deadline 2024-03-18T23:20:41.218013+0000) 2024-03-18T23:20:50.711 INFO:journalctl@ceph.osd.8.smithi175.stdout:Mar 18 23:20:50 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-8[86041]: 2024-03-18T23:20:50.432+0000 7fa05c798700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-18T23:20:23.519110+0000 front 2024-03-18T23:20:23.519209+0000 (oldest deadline 2024-03-18T23:20:48.818956+0000) 2024-03-18T23:20:51.314 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 18 23:20:51 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-7[85590]: 2024-03-18T23:20:51.049+0000 7f5dd9117700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-03-18T23:20:08.415464+0000 front 2024-03-18T23:20:08.415444+0000 (oldest deadline 2024-03-18T23:20:33.115572+0000) 2024-03-18T23:20:51.314 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 18 23:20:51 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-7[85590]: 2024-03-18T23:20:51.049+0000 7f5dd9117700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-03-18T23:20:15.917098+0000 front 2024-03-18T23:20:15.917063+0000 (oldest deadline 2024-03-18T23:20:42.917807+0000) 2024-03-18T23:20:51.314 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 18 23:20:51 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-7[85590]: 2024-03-18T23:20:51.049+0000 7f5dd9117700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-18T23:20:26.820242+0000 front 2024-03-18T23:20:26.820256+0000 (oldest deadline 2024-03-18T23:20:50.320212+0000) 2024-03-18T23:20:51.314 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:20:51 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:20:51.154+0000 7f83cd240700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:19:57.904964+0000 front 2024-03-18T23:19:57.904988+0000 (oldest deadline 2024-03-18T23:20:23.805169+0000) 2024-03-18T23:20:51.315 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:20:51 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:20:51.154+0000 7f83cd240700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-03-18T23:20:10.806661+0000 front 2024-03-18T23:20:10.806621+0000 (oldest deadline 2024-03-18T23:20:31.906747+0000) 2024-03-18T23:20:51.315 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:20:51 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:20:51.154+0000 7f83cd240700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-03-18T23:20:16.508876+0000 front 2024-03-18T23:20:16.508656+0000 (oldest deadline 2024-03-18T23:20:41.808738+0000) 2024-03-18T23:20:51.315 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:20:51 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:20:51.154+0000 7f83cd240700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-18T23:20:24.710033+0000 front 2024-03-18T23:20:24.709803+0000 (oldest deadline 2024-03-18T23:20:50.510257+0000) 2024-03-18T23:20:51.629 INFO:journalctl@ceph.osd.6.smithi049.stdout:Mar 18 23:20:51 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-6[91523]: 2024-03-18T23:20:51.360+0000 7f92dba96700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:20:01.671603+0000 front 2024-03-18T23:20:01.671661+0000 (oldest deadline 2024-03-18T23:20:23.971756+0000) 2024-03-18T23:20:51.629 INFO:journalctl@ceph.osd.6.smithi049.stdout:Mar 18 23:20:51 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-6[91523]: 2024-03-18T23:20:51.360+0000 7f92dba96700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-03-18T23:20:08.573527+0000 front 2024-03-18T23:20:08.573679+0000 (oldest deadline 2024-03-18T23:20:32.673334+0000) 2024-03-18T23:20:51.629 INFO:journalctl@ceph.osd.6.smithi049.stdout:Mar 18 23:20:51 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-6[91523]: 2024-03-18T23:20:51.360+0000 7f92dba96700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-03-18T23:20:17.974330+0000 front 2024-03-18T23:20:17.974308+0000 (oldest deadline 2024-03-18T23:20:42.674209+0000) 2024-03-18T23:20:51.629 INFO:journalctl@ceph.osd.6.smithi049.stdout:Mar 18 23:20:51 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-6[91523]: 2024-03-18T23:20:51.360+0000 7f92dba96700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-18T23:20:24.975188+0000 front 2024-03-18T23:20:24.975251+0000 (oldest deadline 2024-03-18T23:20:49.675258+0000) 2024-03-18T23:20:51.630 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 18 23:20:51 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-9[98163]: 2024-03-18T23:20:51.433+0000 7ff64b26e700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:20:01.074760+0000 front 2024-03-18T23:20:01.074900+0000 (oldest deadline 2024-03-18T23:20:26.974968+0000) 2024-03-18T23:20:51.630 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 18 23:20:51 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-9[98163]: 2024-03-18T23:20:51.433+0000 7ff64b26e700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-03-18T23:20:10.976967+0000 front 2024-03-18T23:20:10.976857+0000 (oldest deadline 2024-03-18T23:20:36.876783+0000) 2024-03-18T23:20:51.630 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 18 23:20:51 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-9[98163]: 2024-03-18T23:20:51.433+0000 7ff64b26e700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-03-18T23:20:16.877280+0000 front 2024-03-18T23:20:16.877254+0000 (oldest deadline 2024-03-18T23:20:41.577384+0000) 2024-03-18T23:20:51.630 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 18 23:20:51 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-9[98163]: 2024-03-18T23:20:51.433+0000 7ff64b26e700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-18T23:20:23.278575+0000 front 2024-03-18T23:20:23.278614+0000 (oldest deadline 2024-03-18T23:20:49.178702+0000) 2024-03-18T23:20:51.710 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:20:51 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:20:51.538+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:19:57.947723+0000 front 2024-03-18T23:19:57.947785+0000 (oldest deadline 2024-03-18T23:20:28.548565+0000) 2024-03-18T23:20:51.710 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:20:51 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:20:51.538+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-03-18T23:20:08.549207+0000 front 2024-03-18T23:20:08.549248+0000 (oldest deadline 2024-03-18T23:20:33.849233+0000) 2024-03-18T23:20:51.710 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:20:51 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:20:51.538+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-03-18T23:20:16.150579+0000 front 2024-03-18T23:20:16.150606+0000 (oldest deadline 2024-03-18T23:20:40.751401+0000) 2024-03-18T23:20:51.710 INFO:journalctl@ceph.osd.8.smithi175.stdout:Mar 18 23:20:51 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-8[86041]: 2024-03-18T23:20:51.413+0000 7fa05c798700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:20:02.615291+0000 front 2024-03-18T23:20:02.615437+0000 (oldest deadline 2024-03-18T23:20:24.315195+0000) 2024-03-18T23:20:51.711 INFO:journalctl@ceph.osd.8.smithi175.stdout:Mar 18 23:20:51 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-8[86041]: 2024-03-18T23:20:51.413+0000 7fa05c798700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-03-18T23:20:17.717904+0000 front 2024-03-18T23:20:17.718012+0000 (oldest deadline 2024-03-18T23:20:41.218013+0000) 2024-03-18T23:20:51.711 INFO:journalctl@ceph.osd.8.smithi175.stdout:Mar 18 23:20:51 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-8[86041]: 2024-03-18T23:20:51.413+0000 7fa05c798700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-18T23:20:23.519110+0000 front 2024-03-18T23:20:23.519209+0000 (oldest deadline 2024-03-18T23:20:48.818956+0000) 2024-03-18T23:20:52.219 INFO:journalctl@ceph.osd.6.smithi049.stdout:Mar 18 23:20:51 smithi049 podman[124295]: 2024-03-18 23:20:51.920811911 +0000 UTC m=+5.616479258 container died 12795277ad141b4dc58ed8ff55fe29bb623fe4720997bc92bd860c5df5f5bcd1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-6, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-03-18T23:20:52.314 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 18 23:20:52 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-7[85590]: 2024-03-18T23:20:52.062+0000 7f5dd9117700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-03-18T23:20:08.415464+0000 front 2024-03-18T23:20:08.415444+0000 (oldest deadline 2024-03-18T23:20:33.115572+0000) 2024-03-18T23:20:52.314 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 18 23:20:52 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-7[85590]: 2024-03-18T23:20:52.062+0000 7f5dd9117700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-03-18T23:20:15.917098+0000 front 2024-03-18T23:20:15.917063+0000 (oldest deadline 2024-03-18T23:20:42.917807+0000) 2024-03-18T23:20:52.314 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 18 23:20:52 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-7[85590]: 2024-03-18T23:20:52.062+0000 7f5dd9117700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-18T23:20:26.820242+0000 front 2024-03-18T23:20:26.820256+0000 (oldest deadline 2024-03-18T23:20:50.320212+0000) 2024-03-18T23:20:52.315 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:20:52 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:20:52.106+0000 7f83cd240700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:19:57.904964+0000 front 2024-03-18T23:19:57.904988+0000 (oldest deadline 2024-03-18T23:20:23.805169+0000) 2024-03-18T23:20:52.315 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:20:52 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:20:52.106+0000 7f83cd240700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-03-18T23:20:10.806661+0000 front 2024-03-18T23:20:10.806621+0000 (oldest deadline 2024-03-18T23:20:31.906747+0000) 2024-03-18T23:20:52.315 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:20:52 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:20:52.106+0000 7f83cd240700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-03-18T23:20:16.508876+0000 front 2024-03-18T23:20:16.508656+0000 (oldest deadline 2024-03-18T23:20:41.808738+0000) 2024-03-18T23:20:52.315 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:20:52 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:20:52.107+0000 7f83cd240700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-18T23:20:24.710033+0000 front 2024-03-18T23:20:24.709803+0000 (oldest deadline 2024-03-18T23:20:50.510257+0000) 2024-03-18T23:20:52.477 INFO:journalctl@ceph.osd.6.smithi049.stdout:Mar 18 23:20:52 smithi049 podman[124295]: 2024-03-18 23:20:52.33310727 +0000 UTC m=+6.028774615 container cleanup 12795277ad141b4dc58ed8ff55fe29bb623fe4720997bc92bd860c5df5f5bcd1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-6, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-03-18T23:20:52.477 INFO:journalctl@ceph.osd.6.smithi049.stdout:Mar 18 23:20:52 smithi049 bash[124295]: ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-6 2024-03-18T23:20:52.710 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:20:52 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:20:52.543+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:19:57.947723+0000 front 2024-03-18T23:19:57.947785+0000 (oldest deadline 2024-03-18T23:20:28.548565+0000) 2024-03-18T23:20:52.710 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:20:52 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:20:52.543+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-03-18T23:20:08.549207+0000 front 2024-03-18T23:20:08.549248+0000 (oldest deadline 2024-03-18T23:20:33.849233+0000) 2024-03-18T23:20:52.710 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:20:52 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:20:52.544+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-03-18T23:20:16.150579+0000 front 2024-03-18T23:20:16.150606+0000 (oldest deadline 2024-03-18T23:20:40.751401+0000) 2024-03-18T23:20:52.711 INFO:journalctl@ceph.osd.8.smithi175.stdout:Mar 18 23:20:52 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-8[86041]: 2024-03-18T23:20:52.448+0000 7fa05c798700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:20:02.615291+0000 front 2024-03-18T23:20:02.615437+0000 (oldest deadline 2024-03-18T23:20:24.315195+0000) 2024-03-18T23:20:52.711 INFO:journalctl@ceph.osd.8.smithi175.stdout:Mar 18 23:20:52 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-8[86041]: 2024-03-18T23:20:52.448+0000 7fa05c798700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-03-18T23:20:17.717904+0000 front 2024-03-18T23:20:17.718012+0000 (oldest deadline 2024-03-18T23:20:41.218013+0000) 2024-03-18T23:20:52.711 INFO:journalctl@ceph.osd.8.smithi175.stdout:Mar 18 23:20:52 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-8[86041]: 2024-03-18T23:20:52.448+0000 7fa05c798700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-18T23:20:23.519110+0000 front 2024-03-18T23:20:23.519209+0000 (oldest deadline 2024-03-18T23:20:48.818956+0000) 2024-03-18T23:20:52.878 INFO:journalctl@ceph.osd.6.smithi049.stdout:Mar 18 23:20:52 smithi049 podman[124311]: 2024-03-18 23:20:52.555118578 +0000 UTC m=+0.633416960 container remove 12795277ad141b4dc58ed8ff55fe29bb623fe4720997bc92bd860c5df5f5bcd1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-6, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.33.5) 2024-03-18T23:20:52.879 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 18 23:20:52 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-9[98163]: 2024-03-18T23:20:52.476+0000 7ff64b26e700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:20:01.074760+0000 front 2024-03-18T23:20:01.074900+0000 (oldest deadline 2024-03-18T23:20:26.974968+0000) 2024-03-18T23:20:52.879 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 18 23:20:52 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-9[98163]: 2024-03-18T23:20:52.476+0000 7ff64b26e700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-03-18T23:20:10.976967+0000 front 2024-03-18T23:20:10.976857+0000 (oldest deadline 2024-03-18T23:20:36.876783+0000) 2024-03-18T23:20:52.879 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 18 23:20:52 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-9[98163]: 2024-03-18T23:20:52.476+0000 7ff64b26e700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-03-18T23:20:16.877280+0000 front 2024-03-18T23:20:16.877254+0000 (oldest deadline 2024-03-18T23:20:41.577384+0000) 2024-03-18T23:20:52.879 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 18 23:20:52 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-9[98163]: 2024-03-18T23:20:52.476+0000 7ff64b26e700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-18T23:20:23.278575+0000 front 2024-03-18T23:20:23.278614+0000 (oldest deadline 2024-03-18T23:20:49.178702+0000) 2024-03-18T23:20:53.282 INFO:journalctl@ceph.osd.6.smithi049.stdout:Mar 18 23:20:52 smithi049 podman[124403]: 2024-03-18 23:20:52.867504547 +0000 UTC m=+0.018964843 image pull 2312c8e42d0069a2dbb2364bddb00264db4b800e7f18a7e7e4eb8f3ecab2ff1c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639 2024-03-18T23:20:53.282 INFO:journalctl@ceph.osd.6.smithi049.stdout:Mar 18 23:20:52 smithi049 podman[124403]: 2024-03-18 23:20:52.981128553 +0000 UTC m=+0.132588814 container create b9c31e7cd55d70b974fc187f68f2a08c5b0a82fb4dad69f5c2aba7103672f799 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-6-deactivate, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.build-date=20240307, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-03-18T23:20:53.282 INFO:journalctl@ceph.osd.6.smithi049.stdout:Mar 18 23:20:53 smithi049 podman[124403]: 2024-03-18 23:20:53.209108939 +0000 UTC m=+0.360569217 container init b9c31e7cd55d70b974fc187f68f2a08c5b0a82fb4dad69f5c2aba7103672f799 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-6-deactivate, ceph=True, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-18T23:20:53.282 INFO:journalctl@ceph.osd.6.smithi049.stdout:Mar 18 23:20:53 smithi049 podman[124403]: 2024-03-18 23:20:53.212486749 +0000 UTC m=+0.363947018 container start b9c31e7cd55d70b974fc187f68f2a08c5b0a82fb4dad69f5c2aba7103672f799 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-6-deactivate, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, ceph=True, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307) 2024-03-18T23:20:53.283 INFO:journalctl@ceph.osd.6.smithi049.stdout:Mar 18 23:20:53 smithi049 podman[124403]: 2024-03-18 23:20:53.283512153 +0000 UTC m=+0.434972422 container attach b9c31e7cd55d70b974fc187f68f2a08c5b0a82fb4dad69f5c2aba7103672f799 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-6-deactivate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-18T23:20:53.565 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 18 23:20:53 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-7[85590]: 2024-03-18T23:20:53.065+0000 7f5dd9117700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-03-18T23:20:08.415464+0000 front 2024-03-18T23:20:08.415444+0000 (oldest deadline 2024-03-18T23:20:33.115572+0000) 2024-03-18T23:20:53.566 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 18 23:20:53 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-7[85590]: 2024-03-18T23:20:53.065+0000 7f5dd9117700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-03-18T23:20:15.917098+0000 front 2024-03-18T23:20:15.917063+0000 (oldest deadline 2024-03-18T23:20:42.917807+0000) 2024-03-18T23:20:53.566 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 18 23:20:53 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-7[85590]: 2024-03-18T23:20:53.065+0000 7f5dd9117700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-18T23:20:26.820242+0000 front 2024-03-18T23:20:26.820256+0000 (oldest deadline 2024-03-18T23:20:50.320212+0000) 2024-03-18T23:20:53.566 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:20:53 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:20:53.096+0000 7f83cd240700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:19:57.904964+0000 front 2024-03-18T23:19:57.904988+0000 (oldest deadline 2024-03-18T23:20:23.805169+0000) 2024-03-18T23:20:53.567 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:20:53 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:20:53.096+0000 7f83cd240700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-03-18T23:20:10.806661+0000 front 2024-03-18T23:20:10.806621+0000 (oldest deadline 2024-03-18T23:20:31.906747+0000) 2024-03-18T23:20:53.567 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:20:53 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:20:53.096+0000 7f83cd240700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-03-18T23:20:16.508876+0000 front 2024-03-18T23:20:16.508656+0000 (oldest deadline 2024-03-18T23:20:41.808738+0000) 2024-03-18T23:20:53.567 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:20:53 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:20:53.096+0000 7f83cd240700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-18T23:20:24.710033+0000 front 2024-03-18T23:20:24.709803+0000 (oldest deadline 2024-03-18T23:20:50.510257+0000) 2024-03-18T23:20:53.628 INFO:journalctl@ceph.osd.6.smithi049.stdout:Mar 18 23:20:53 smithi049 podman[124403]: 2024-03-18 23:20:53.514695832 +0000 UTC m=+0.666156093 container died b9c31e7cd55d70b974fc187f68f2a08c5b0a82fb4dad69f5c2aba7103672f799 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-6-deactivate, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-03-18T23:20:53.629 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 18 23:20:53 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-9[98163]: 2024-03-18T23:20:53.445+0000 7ff64b26e700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:20:01.074760+0000 front 2024-03-18T23:20:01.074900+0000 (oldest deadline 2024-03-18T23:20:26.974968+0000) 2024-03-18T23:20:53.629 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 18 23:20:53 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-9[98163]: 2024-03-18T23:20:53.445+0000 7ff64b26e700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-03-18T23:20:10.976967+0000 front 2024-03-18T23:20:10.976857+0000 (oldest deadline 2024-03-18T23:20:36.876783+0000) 2024-03-18T23:20:53.629 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 18 23:20:53 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-9[98163]: 2024-03-18T23:20:53.445+0000 7ff64b26e700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-03-18T23:20:16.877280+0000 front 2024-03-18T23:20:16.877254+0000 (oldest deadline 2024-03-18T23:20:41.577384+0000) 2024-03-18T23:20:53.629 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 18 23:20:53 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-9[98163]: 2024-03-18T23:20:53.445+0000 7ff64b26e700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-18T23:20:23.278575+0000 front 2024-03-18T23:20:23.278614+0000 (oldest deadline 2024-03-18T23:20:49.178702+0000) 2024-03-18T23:20:53.710 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:20:53 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:20:53.536+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:19:57.947723+0000 front 2024-03-18T23:19:57.947785+0000 (oldest deadline 2024-03-18T23:20:28.548565+0000) 2024-03-18T23:20:53.710 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:20:53 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:20:53.536+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-03-18T23:20:08.549207+0000 front 2024-03-18T23:20:08.549248+0000 (oldest deadline 2024-03-18T23:20:33.849233+0000) 2024-03-18T23:20:53.710 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:20:53 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:20:53.536+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-03-18T23:20:16.150579+0000 front 2024-03-18T23:20:16.150606+0000 (oldest deadline 2024-03-18T23:20:40.751401+0000) 2024-03-18T23:20:53.710 INFO:journalctl@ceph.osd.8.smithi175.stdout:Mar 18 23:20:53 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-8[86041]: 2024-03-18T23:20:53.431+0000 7fa05c798700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:20:02.615291+0000 front 2024-03-18T23:20:02.615437+0000 (oldest deadline 2024-03-18T23:20:24.315195+0000) 2024-03-18T23:20:53.711 INFO:journalctl@ceph.osd.8.smithi175.stdout:Mar 18 23:20:53 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-8[86041]: 2024-03-18T23:20:53.431+0000 7fa05c798700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-03-18T23:20:17.717904+0000 front 2024-03-18T23:20:17.718012+0000 (oldest deadline 2024-03-18T23:20:41.218013+0000) 2024-03-18T23:20:53.711 INFO:journalctl@ceph.osd.8.smithi175.stdout:Mar 18 23:20:53 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-8[86041]: 2024-03-18T23:20:53.431+0000 7fa05c798700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-18T23:20:23.519110+0000 front 2024-03-18T23:20:23.519209+0000 (oldest deadline 2024-03-18T23:20:48.818956+0000) 2024-03-18T23:20:54.170 INFO:journalctl@ceph.osd.6.smithi049.stdout:Mar 18 23:20:54 smithi049 podman[124430]: 2024-03-18 23:20:54.154383207 +0000 UTC m=+0.630561986 container remove b9c31e7cd55d70b974fc187f68f2a08c5b0a82fb4dad69f5c2aba7103672f799 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-6-deactivate, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.build-date=20240307, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-03-18T23:20:54.178 DEBUG:teuthology.orchestra.run.smithi049:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c63591ea-e57a-11ee-95c9-87774f69a715@osd.6.service' 2024-03-18T23:20:54.223 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-18T23:20:54.223 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-03-18T23:20:54.224 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-03-18T23:20:54.224 DEBUG:teuthology.orchestra.run.smithi139:> sudo systemctl stop ceph-c63591ea-e57a-11ee-95c9-87774f69a715@osd.7 2024-03-18T23:20:54.465 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 18 23:20:54 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-7[85590]: 2024-03-18T23:20:54.088+0000 7f5dd9117700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-03-18T23:20:08.415464+0000 front 2024-03-18T23:20:08.415444+0000 (oldest deadline 2024-03-18T23:20:33.115572+0000) 2024-03-18T23:20:54.465 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 18 23:20:54 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-7[85590]: 2024-03-18T23:20:54.088+0000 7f5dd9117700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-03-18T23:20:15.917098+0000 front 2024-03-18T23:20:15.917063+0000 (oldest deadline 2024-03-18T23:20:42.917807+0000) 2024-03-18T23:20:54.465 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 18 23:20:54 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-7[85590]: 2024-03-18T23:20:54.088+0000 7f5dd9117700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-18T23:20:26.820242+0000 front 2024-03-18T23:20:26.820256+0000 (oldest deadline 2024-03-18T23:20:50.320212+0000) 2024-03-18T23:20:54.465 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 18 23:20:54 smithi139 systemd[1]: Stopping Ceph osd.7 for c63591ea-e57a-11ee-95c9-87774f69a715... 2024-03-18T23:20:54.466 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:20:54 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:20:54.143+0000 7f83cd240700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:19:57.904964+0000 front 2024-03-18T23:19:57.904988+0000 (oldest deadline 2024-03-18T23:20:23.805169+0000) 2024-03-18T23:20:54.466 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:20:54 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:20:54.143+0000 7f83cd240700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-03-18T23:20:10.806661+0000 front 2024-03-18T23:20:10.806621+0000 (oldest deadline 2024-03-18T23:20:31.906747+0000) 2024-03-18T23:20:54.466 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:20:54 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:20:54.143+0000 7f83cd240700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-03-18T23:20:16.508876+0000 front 2024-03-18T23:20:16.508656+0000 (oldest deadline 2024-03-18T23:20:41.808738+0000) 2024-03-18T23:20:54.466 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:20:54 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:20:54.143+0000 7f83cd240700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-18T23:20:24.710033+0000 front 2024-03-18T23:20:24.709803+0000 (oldest deadline 2024-03-18T23:20:50.510257+0000) 2024-03-18T23:20:54.710 INFO:journalctl@ceph.osd.8.smithi175.stdout:Mar 18 23:20:54 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-8[86041]: 2024-03-18T23:20:54.457+0000 7fa05c798700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:20:02.615291+0000 front 2024-03-18T23:20:02.615437+0000 (oldest deadline 2024-03-18T23:20:24.315195+0000) 2024-03-18T23:20:54.710 INFO:journalctl@ceph.osd.8.smithi175.stdout:Mar 18 23:20:54 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-8[86041]: 2024-03-18T23:20:54.457+0000 7fa05c798700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-03-18T23:20:17.717904+0000 front 2024-03-18T23:20:17.718012+0000 (oldest deadline 2024-03-18T23:20:41.218013+0000) 2024-03-18T23:20:54.710 INFO:journalctl@ceph.osd.8.smithi175.stdout:Mar 18 23:20:54 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-8[86041]: 2024-03-18T23:20:54.457+0000 7fa05c798700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-18T23:20:23.519110+0000 front 2024-03-18T23:20:23.519209+0000 (oldest deadline 2024-03-18T23:20:48.818956+0000) 2024-03-18T23:20:54.711 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:20:54 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:20:54.506+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:19:57.947723+0000 front 2024-03-18T23:19:57.947785+0000 (oldest deadline 2024-03-18T23:20:28.548565+0000) 2024-03-18T23:20:54.711 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:20:54 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:20:54.506+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-03-18T23:20:08.549207+0000 front 2024-03-18T23:20:08.549248+0000 (oldest deadline 2024-03-18T23:20:33.849233+0000) 2024-03-18T23:20:54.711 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:20:54 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:20:54.506+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-03-18T23:20:16.150579+0000 front 2024-03-18T23:20:16.150606+0000 (oldest deadline 2024-03-18T23:20:40.751401+0000) 2024-03-18T23:20:54.813 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 18 23:20:54 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-7[85590]: 2024-03-18T23:20:54.541+0000 7f5ddcb23700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-18T23:20:54.814 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 18 23:20:54 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-7[85590]: 2024-03-18T23:20:54.541+0000 7f5ddcb23700 -1 osd.7 104 *** Got signal Terminated *** 2024-03-18T23:20:54.814 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 18 23:20:54 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-7[85590]: 2024-03-18T23:20:54.541+0000 7f5ddcb23700 -1 osd.7 104 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-18T23:20:54.879 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 18 23:20:54 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-9[98163]: 2024-03-18T23:20:54.423+0000 7ff64b26e700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:20:01.074760+0000 front 2024-03-18T23:20:01.074900+0000 (oldest deadline 2024-03-18T23:20:26.974968+0000) 2024-03-18T23:20:54.879 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 18 23:20:54 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-9[98163]: 2024-03-18T23:20:54.423+0000 7ff64b26e700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-03-18T23:20:10.976967+0000 front 2024-03-18T23:20:10.976857+0000 (oldest deadline 2024-03-18T23:20:36.876783+0000) 2024-03-18T23:20:54.879 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 18 23:20:54 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-9[98163]: 2024-03-18T23:20:54.423+0000 7ff64b26e700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-03-18T23:20:16.877280+0000 front 2024-03-18T23:20:16.877254+0000 (oldest deadline 2024-03-18T23:20:41.577384+0000) 2024-03-18T23:20:54.879 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 18 23:20:54 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-9[98163]: 2024-03-18T23:20:54.423+0000 7ff64b26e700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-18T23:20:23.278575+0000 front 2024-03-18T23:20:23.278614+0000 (oldest deadline 2024-03-18T23:20:49.178702+0000) 2024-03-18T23:20:55.478 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 18 23:20:55 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-7[85590]: 2024-03-18T23:20:55.108+0000 7f5dd9117700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-03-18T23:20:08.415464+0000 front 2024-03-18T23:20:08.415444+0000 (oldest deadline 2024-03-18T23:20:33.115572+0000) 2024-03-18T23:20:55.479 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 18 23:20:55 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-7[85590]: 2024-03-18T23:20:55.108+0000 7f5dd9117700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-03-18T23:20:15.917098+0000 front 2024-03-18T23:20:15.917063+0000 (oldest deadline 2024-03-18T23:20:42.917807+0000) 2024-03-18T23:20:55.479 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 18 23:20:55 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-7[85590]: 2024-03-18T23:20:55.108+0000 7f5dd9117700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-18T23:20:26.820242+0000 front 2024-03-18T23:20:26.820256+0000 (oldest deadline 2024-03-18T23:20:50.320212+0000) 2024-03-18T23:20:55.479 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:20:55 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:20:55.118+0000 7f83cd240700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:19:57.904964+0000 front 2024-03-18T23:19:57.904988+0000 (oldest deadline 2024-03-18T23:20:23.805169+0000) 2024-03-18T23:20:55.479 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:20:55 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:20:55.118+0000 7f83cd240700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-03-18T23:20:10.806661+0000 front 2024-03-18T23:20:10.806621+0000 (oldest deadline 2024-03-18T23:20:31.906747+0000) 2024-03-18T23:20:55.479 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:20:55 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:20:55.118+0000 7f83cd240700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-03-18T23:20:16.508876+0000 front 2024-03-18T23:20:16.508656+0000 (oldest deadline 2024-03-18T23:20:41.808738+0000) 2024-03-18T23:20:55.479 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:20:55 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:20:55.118+0000 7f83cd240700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-18T23:20:24.710033+0000 front 2024-03-18T23:20:24.709803+0000 (oldest deadline 2024-03-18T23:20:50.510257+0000) 2024-03-18T23:20:55.710 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:20:55 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:20:55.551+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:19:57.947723+0000 front 2024-03-18T23:19:57.947785+0000 (oldest deadline 2024-03-18T23:20:28.548565+0000) 2024-03-18T23:20:55.710 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:20:55 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:20:55.551+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-03-18T23:20:08.549207+0000 front 2024-03-18T23:20:08.549248+0000 (oldest deadline 2024-03-18T23:20:33.849233+0000) 2024-03-18T23:20:55.710 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:20:55 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:20:55.551+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-03-18T23:20:16.150579+0000 front 2024-03-18T23:20:16.150606+0000 (oldest deadline 2024-03-18T23:20:40.751401+0000) 2024-03-18T23:20:55.711 INFO:journalctl@ceph.osd.8.smithi175.stdout:Mar 18 23:20:55 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-8[86041]: 2024-03-18T23:20:55.435+0000 7fa05c798700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:20:02.615291+0000 front 2024-03-18T23:20:02.615437+0000 (oldest deadline 2024-03-18T23:20:24.315195+0000) 2024-03-18T23:20:55.711 INFO:journalctl@ceph.osd.8.smithi175.stdout:Mar 18 23:20:55 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-8[86041]: 2024-03-18T23:20:55.435+0000 7fa05c798700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-03-18T23:20:17.717904+0000 front 2024-03-18T23:20:17.718012+0000 (oldest deadline 2024-03-18T23:20:41.218013+0000) 2024-03-18T23:20:55.711 INFO:journalctl@ceph.osd.8.smithi175.stdout:Mar 18 23:20:55 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-8[86041]: 2024-03-18T23:20:55.435+0000 7fa05c798700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-18T23:20:23.519110+0000 front 2024-03-18T23:20:23.519209+0000 (oldest deadline 2024-03-18T23:20:48.818956+0000) 2024-03-18T23:20:55.879 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 18 23:20:55 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-9[98163]: 2024-03-18T23:20:55.404+0000 7ff64b26e700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:20:01.074760+0000 front 2024-03-18T23:20:01.074900+0000 (oldest deadline 2024-03-18T23:20:26.974968+0000) 2024-03-18T23:20:55.879 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 18 23:20:55 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-9[98163]: 2024-03-18T23:20:55.404+0000 7ff64b26e700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-03-18T23:20:10.976967+0000 front 2024-03-18T23:20:10.976857+0000 (oldest deadline 2024-03-18T23:20:36.876783+0000) 2024-03-18T23:20:55.879 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 18 23:20:55 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-9[98163]: 2024-03-18T23:20:55.404+0000 7ff64b26e700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-03-18T23:20:16.877280+0000 front 2024-03-18T23:20:16.877254+0000 (oldest deadline 2024-03-18T23:20:41.577384+0000) 2024-03-18T23:20:55.879 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 18 23:20:55 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-9[98163]: 2024-03-18T23:20:55.404+0000 7ff64b26e700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-18T23:20:23.278575+0000 front 2024-03-18T23:20:23.278614+0000 (oldest deadline 2024-03-18T23:20:49.178702+0000) 2024-03-18T23:20:56.564 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 18 23:20:56 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-7[85590]: 2024-03-18T23:20:56.119+0000 7f5dd9117700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-03-18T23:20:08.415464+0000 front 2024-03-18T23:20:08.415444+0000 (oldest deadline 2024-03-18T23:20:33.115572+0000) 2024-03-18T23:20:56.564 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 18 23:20:56 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-7[85590]: 2024-03-18T23:20:56.119+0000 7f5dd9117700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-03-18T23:20:15.917098+0000 front 2024-03-18T23:20:15.917063+0000 (oldest deadline 2024-03-18T23:20:42.917807+0000) 2024-03-18T23:20:56.564 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 18 23:20:56 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-7[85590]: 2024-03-18T23:20:56.119+0000 7f5dd9117700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-18T23:20:26.820242+0000 front 2024-03-18T23:20:26.820256+0000 (oldest deadline 2024-03-18T23:20:50.320212+0000) 2024-03-18T23:20:56.564 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:20:56 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:20:56.085+0000 7f83cd240700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:19:57.904964+0000 front 2024-03-18T23:19:57.904988+0000 (oldest deadline 2024-03-18T23:20:23.805169+0000) 2024-03-18T23:20:56.565 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:20:56 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:20:56.085+0000 7f83cd240700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-03-18T23:20:10.806661+0000 front 2024-03-18T23:20:10.806621+0000 (oldest deadline 2024-03-18T23:20:31.906747+0000) 2024-03-18T23:20:56.565 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:20:56 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:20:56.085+0000 7f83cd240700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-03-18T23:20:16.508876+0000 front 2024-03-18T23:20:16.508656+0000 (oldest deadline 2024-03-18T23:20:41.808738+0000) 2024-03-18T23:20:56.565 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:20:56 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:20:56.085+0000 7f83cd240700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-18T23:20:24.710033+0000 front 2024-03-18T23:20:24.709803+0000 (oldest deadline 2024-03-18T23:20:50.510257+0000) 2024-03-18T23:20:56.710 INFO:journalctl@ceph.osd.8.smithi175.stdout:Mar 18 23:20:56 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-8[86041]: 2024-03-18T23:20:56.457+0000 7fa05c798700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:20:02.615291+0000 front 2024-03-18T23:20:02.615437+0000 (oldest deadline 2024-03-18T23:20:24.315195+0000) 2024-03-18T23:20:56.710 INFO:journalctl@ceph.osd.8.smithi175.stdout:Mar 18 23:20:56 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-8[86041]: 2024-03-18T23:20:56.457+0000 7fa05c798700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-03-18T23:20:17.717904+0000 front 2024-03-18T23:20:17.718012+0000 (oldest deadline 2024-03-18T23:20:41.218013+0000) 2024-03-18T23:20:56.710 INFO:journalctl@ceph.osd.8.smithi175.stdout:Mar 18 23:20:56 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-8[86041]: 2024-03-18T23:20:56.457+0000 7fa05c798700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-18T23:20:23.519110+0000 front 2024-03-18T23:20:23.519209+0000 (oldest deadline 2024-03-18T23:20:48.818956+0000) 2024-03-18T23:20:56.711 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:20:56 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:20:56.583+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:19:57.947723+0000 front 2024-03-18T23:19:57.947785+0000 (oldest deadline 2024-03-18T23:20:28.548565+0000) 2024-03-18T23:20:56.711 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:20:56 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:20:56.583+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-03-18T23:20:08.549207+0000 front 2024-03-18T23:20:08.549248+0000 (oldest deadline 2024-03-18T23:20:33.849233+0000) 2024-03-18T23:20:56.711 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:20:56 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:20:56.583+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-03-18T23:20:16.150579+0000 front 2024-03-18T23:20:16.150606+0000 (oldest deadline 2024-03-18T23:20:40.751401+0000) 2024-03-18T23:20:56.879 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 18 23:20:56 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-9[98163]: 2024-03-18T23:20:56.449+0000 7ff64b26e700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:20:01.074760+0000 front 2024-03-18T23:20:01.074900+0000 (oldest deadline 2024-03-18T23:20:26.974968+0000) 2024-03-18T23:20:56.879 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 18 23:20:56 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-9[98163]: 2024-03-18T23:20:56.449+0000 7ff64b26e700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-03-18T23:20:10.976967+0000 front 2024-03-18T23:20:10.976857+0000 (oldest deadline 2024-03-18T23:20:36.876783+0000) 2024-03-18T23:20:56.879 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 18 23:20:56 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-9[98163]: 2024-03-18T23:20:56.449+0000 7ff64b26e700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-03-18T23:20:16.877280+0000 front 2024-03-18T23:20:16.877254+0000 (oldest deadline 2024-03-18T23:20:41.577384+0000) 2024-03-18T23:20:56.879 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 18 23:20:56 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-9[98163]: 2024-03-18T23:20:56.449+0000 7ff64b26e700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-18T23:20:23.278575+0000 front 2024-03-18T23:20:23.278614+0000 (oldest deadline 2024-03-18T23:20:49.178702+0000) 2024-03-18T23:20:57.564 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 18 23:20:57 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-7[85590]: 2024-03-18T23:20:57.083+0000 7f5dd9117700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-03-18T23:20:08.415464+0000 front 2024-03-18T23:20:08.415444+0000 (oldest deadline 2024-03-18T23:20:33.115572+0000) 2024-03-18T23:20:57.564 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 18 23:20:57 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-7[85590]: 2024-03-18T23:20:57.083+0000 7f5dd9117700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-03-18T23:20:15.917098+0000 front 2024-03-18T23:20:15.917063+0000 (oldest deadline 2024-03-18T23:20:42.917807+0000) 2024-03-18T23:20:57.564 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 18 23:20:57 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-7[85590]: 2024-03-18T23:20:57.083+0000 7f5dd9117700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-18T23:20:26.820242+0000 front 2024-03-18T23:20:26.820256+0000 (oldest deadline 2024-03-18T23:20:50.320212+0000) 2024-03-18T23:20:57.565 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:20:57 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:20:57.067+0000 7f83cd240700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:19:57.904964+0000 front 2024-03-18T23:19:57.904988+0000 (oldest deadline 2024-03-18T23:20:23.805169+0000) 2024-03-18T23:20:57.565 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:20:57 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:20:57.067+0000 7f83cd240700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-03-18T23:20:10.806661+0000 front 2024-03-18T23:20:10.806621+0000 (oldest deadline 2024-03-18T23:20:31.906747+0000) 2024-03-18T23:20:57.565 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:20:57 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:20:57.067+0000 7f83cd240700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-03-18T23:20:16.508876+0000 front 2024-03-18T23:20:16.508656+0000 (oldest deadline 2024-03-18T23:20:41.808738+0000) 2024-03-18T23:20:57.565 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:20:57 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:20:57.067+0000 7f83cd240700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-18T23:20:24.710033+0000 front 2024-03-18T23:20:24.709803+0000 (oldest deadline 2024-03-18T23:20:50.510257+0000) 2024-03-18T23:20:57.612 INFO:journalctl@ceph.osd.8.smithi175.stdout:Mar 18 23:20:57 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-8[86041]: 2024-03-18T23:20:57.503+0000 7fa05c798700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:20:02.615291+0000 front 2024-03-18T23:20:02.615437+0000 (oldest deadline 2024-03-18T23:20:24.315195+0000) 2024-03-18T23:20:57.613 INFO:journalctl@ceph.osd.8.smithi175.stdout:Mar 18 23:20:57 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-8[86041]: 2024-03-18T23:20:57.503+0000 7fa05c798700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-03-18T23:20:17.717904+0000 front 2024-03-18T23:20:17.718012+0000 (oldest deadline 2024-03-18T23:20:41.218013+0000) 2024-03-18T23:20:57.613 INFO:journalctl@ceph.osd.8.smithi175.stdout:Mar 18 23:20:57 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-8[86041]: 2024-03-18T23:20:57.503+0000 7fa05c798700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-18T23:20:23.519110+0000 front 2024-03-18T23:20:23.519209+0000 (oldest deadline 2024-03-18T23:20:48.818956+0000) 2024-03-18T23:20:57.878 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 18 23:20:57 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-9[98163]: 2024-03-18T23:20:57.449+0000 7ff64b26e700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:20:01.074760+0000 front 2024-03-18T23:20:01.074900+0000 (oldest deadline 2024-03-18T23:20:26.974968+0000) 2024-03-18T23:20:57.879 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 18 23:20:57 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-9[98163]: 2024-03-18T23:20:57.449+0000 7ff64b26e700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-03-18T23:20:10.976967+0000 front 2024-03-18T23:20:10.976857+0000 (oldest deadline 2024-03-18T23:20:36.876783+0000) 2024-03-18T23:20:57.879 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 18 23:20:57 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-9[98163]: 2024-03-18T23:20:57.449+0000 7ff64b26e700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-03-18T23:20:16.877280+0000 front 2024-03-18T23:20:16.877254+0000 (oldest deadline 2024-03-18T23:20:41.577384+0000) 2024-03-18T23:20:57.879 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 18 23:20:57 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-9[98163]: 2024-03-18T23:20:57.449+0000 7ff64b26e700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-18T23:20:23.278575+0000 front 2024-03-18T23:20:23.278614+0000 (oldest deadline 2024-03-18T23:20:49.178702+0000) 2024-03-18T23:20:57.879 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 18 23:20:57 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-9[98163]: 2024-03-18T23:20:57.449+0000 7ff64b26e700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-03-18T23:20:33.080980+0000 front 2024-03-18T23:20:33.081247+0000 (oldest deadline 2024-03-18T23:20:56.580916+0000) 2024-03-18T23:20:57.960 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:20:57 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:20:57.613+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:19:57.947723+0000 front 2024-03-18T23:19:57.947785+0000 (oldest deadline 2024-03-18T23:20:28.548565+0000) 2024-03-18T23:20:57.960 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:20:57 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:20:57.613+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-03-18T23:20:08.549207+0000 front 2024-03-18T23:20:08.549248+0000 (oldest deadline 2024-03-18T23:20:33.849233+0000) 2024-03-18T23:20:57.960 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:20:57 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:20:57.613+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-03-18T23:20:16.150579+0000 front 2024-03-18T23:20:16.150606+0000 (oldest deadline 2024-03-18T23:20:40.751401+0000) 2024-03-18T23:20:58.564 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 18 23:20:58 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-7[85590]: 2024-03-18T23:20:58.131+0000 7f5dd9117700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-03-18T23:20:08.415464+0000 front 2024-03-18T23:20:08.415444+0000 (oldest deadline 2024-03-18T23:20:33.115572+0000) 2024-03-18T23:20:58.564 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 18 23:20:58 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-7[85590]: 2024-03-18T23:20:58.131+0000 7f5dd9117700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-03-18T23:20:15.917098+0000 front 2024-03-18T23:20:15.917063+0000 (oldest deadline 2024-03-18T23:20:42.917807+0000) 2024-03-18T23:20:58.564 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 18 23:20:58 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-7[85590]: 2024-03-18T23:20:58.131+0000 7f5dd9117700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-18T23:20:26.820242+0000 front 2024-03-18T23:20:26.820256+0000 (oldest deadline 2024-03-18T23:20:50.320212+0000) 2024-03-18T23:20:58.565 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:20:58 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:20:58.097+0000 7f83cd240700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:19:57.904964+0000 front 2024-03-18T23:19:57.904988+0000 (oldest deadline 2024-03-18T23:20:23.805169+0000) 2024-03-18T23:20:58.565 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:20:58 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:20:58.097+0000 7f83cd240700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-03-18T23:20:10.806661+0000 front 2024-03-18T23:20:10.806621+0000 (oldest deadline 2024-03-18T23:20:31.906747+0000) 2024-03-18T23:20:58.565 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:20:58 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:20:58.097+0000 7f83cd240700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-03-18T23:20:16.508876+0000 front 2024-03-18T23:20:16.508656+0000 (oldest deadline 2024-03-18T23:20:41.808738+0000) 2024-03-18T23:20:58.565 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:20:58 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:20:58.097+0000 7f83cd240700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-18T23:20:24.710033+0000 front 2024-03-18T23:20:24.709803+0000 (oldest deadline 2024-03-18T23:20:50.510257+0000) 2024-03-18T23:20:58.780 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 18 23:20:58 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-9[98163]: 2024-03-18T23:20:58.409+0000 7ff64b26e700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:20:01.074760+0000 front 2024-03-18T23:20:01.074900+0000 (oldest deadline 2024-03-18T23:20:26.974968+0000) 2024-03-18T23:20:58.780 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 18 23:20:58 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-9[98163]: 2024-03-18T23:20:58.409+0000 7ff64b26e700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-03-18T23:20:10.976967+0000 front 2024-03-18T23:20:10.976857+0000 (oldest deadline 2024-03-18T23:20:36.876783+0000) 2024-03-18T23:20:58.780 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 18 23:20:58 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-9[98163]: 2024-03-18T23:20:58.409+0000 7ff64b26e700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-03-18T23:20:16.877280+0000 front 2024-03-18T23:20:16.877254+0000 (oldest deadline 2024-03-18T23:20:41.577384+0000) 2024-03-18T23:20:58.781 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 18 23:20:58 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-9[98163]: 2024-03-18T23:20:58.409+0000 7ff64b26e700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-18T23:20:23.278575+0000 front 2024-03-18T23:20:23.278614+0000 (oldest deadline 2024-03-18T23:20:49.178702+0000) 2024-03-18T23:20:58.781 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 18 23:20:58 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-9[98163]: 2024-03-18T23:20:58.409+0000 7ff64b26e700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-03-18T23:20:33.080980+0000 front 2024-03-18T23:20:33.081247+0000 (oldest deadline 2024-03-18T23:20:56.580916+0000) 2024-03-18T23:20:58.960 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:20:58 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:20:58.589+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:19:57.947723+0000 front 2024-03-18T23:19:57.947785+0000 (oldest deadline 2024-03-18T23:20:28.548565+0000) 2024-03-18T23:20:58.960 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:20:58 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:20:58.589+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-03-18T23:20:08.549207+0000 front 2024-03-18T23:20:08.549248+0000 (oldest deadline 2024-03-18T23:20:33.849233+0000) 2024-03-18T23:20:58.960 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:20:58 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:20:58.589+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-03-18T23:20:16.150579+0000 front 2024-03-18T23:20:16.150606+0000 (oldest deadline 2024-03-18T23:20:40.751401+0000) 2024-03-18T23:20:58.961 INFO:journalctl@ceph.osd.8.smithi175.stdout:Mar 18 23:20:58 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-8[86041]: 2024-03-18T23:20:58.497+0000 7fa05c798700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:20:02.615291+0000 front 2024-03-18T23:20:02.615437+0000 (oldest deadline 2024-03-18T23:20:24.315195+0000) 2024-03-18T23:20:58.961 INFO:journalctl@ceph.osd.8.smithi175.stdout:Mar 18 23:20:58 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-8[86041]: 2024-03-18T23:20:58.497+0000 7fa05c798700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-03-18T23:20:17.717904+0000 front 2024-03-18T23:20:17.718012+0000 (oldest deadline 2024-03-18T23:20:41.218013+0000) 2024-03-18T23:20:58.961 INFO:journalctl@ceph.osd.8.smithi175.stdout:Mar 18 23:20:58 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-8[86041]: 2024-03-18T23:20:58.497+0000 7fa05c798700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-18T23:20:23.519110+0000 front 2024-03-18T23:20:23.519209+0000 (oldest deadline 2024-03-18T23:20:48.818956+0000) 2024-03-18T23:20:58.961 INFO:journalctl@ceph.osd.8.smithi175.stdout:Mar 18 23:20:58 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-8[86041]: 2024-03-18T23:20:58.497+0000 7fa05c798700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-03-18T23:20:35.220337+0000 front 2024-03-18T23:20:35.220428+0000 (oldest deadline 2024-03-18T23:20:57.520416+0000) 2024-03-18T23:20:59.564 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 18 23:20:59 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-7[85590]: 2024-03-18T23:20:59.149+0000 7f5dd9117700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-03-18T23:20:08.415464+0000 front 2024-03-18T23:20:08.415444+0000 (oldest deadline 2024-03-18T23:20:33.115572+0000) 2024-03-18T23:20:59.565 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 18 23:20:59 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-7[85590]: 2024-03-18T23:20:59.149+0000 7f5dd9117700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-03-18T23:20:15.917098+0000 front 2024-03-18T23:20:15.917063+0000 (oldest deadline 2024-03-18T23:20:42.917807+0000) 2024-03-18T23:20:59.565 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 18 23:20:59 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-7[85590]: 2024-03-18T23:20:59.149+0000 7f5dd9117700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-18T23:20:26.820242+0000 front 2024-03-18T23:20:26.820256+0000 (oldest deadline 2024-03-18T23:20:50.320212+0000) 2024-03-18T23:20:59.565 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 18 23:20:59 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-7[85590]: 2024-03-18T23:20:59.149+0000 7f5dd9117700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-03-18T23:20:34.321798+0000 front 2024-03-18T23:20:34.321685+0000 (oldest deadline 2024-03-18T23:20:58.421909+0000) 2024-03-18T23:20:59.566 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:20:59 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:20:59.083+0000 7f83cd240700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:19:57.904964+0000 front 2024-03-18T23:19:57.904988+0000 (oldest deadline 2024-03-18T23:20:23.805169+0000) 2024-03-18T23:20:59.566 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:20:59 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:20:59.083+0000 7f83cd240700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-03-18T23:20:10.806661+0000 front 2024-03-18T23:20:10.806621+0000 (oldest deadline 2024-03-18T23:20:31.906747+0000) 2024-03-18T23:20:59.566 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:20:59 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:20:59.083+0000 7f83cd240700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-03-18T23:20:16.508876+0000 front 2024-03-18T23:20:16.508656+0000 (oldest deadline 2024-03-18T23:20:41.808738+0000) 2024-03-18T23:20:59.567 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:20:59 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:20:59.083+0000 7f83cd240700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-18T23:20:24.710033+0000 front 2024-03-18T23:20:24.709803+0000 (oldest deadline 2024-03-18T23:20:50.510257+0000) 2024-03-18T23:20:59.796 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 18 23:20:59 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-9[98163]: 2024-03-18T23:20:59.386+0000 7ff64b26e700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:20:01.074760+0000 front 2024-03-18T23:20:01.074900+0000 (oldest deadline 2024-03-18T23:20:26.974968+0000) 2024-03-18T23:20:59.796 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 18 23:20:59 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-9[98163]: 2024-03-18T23:20:59.386+0000 7ff64b26e700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-03-18T23:20:10.976967+0000 front 2024-03-18T23:20:10.976857+0000 (oldest deadline 2024-03-18T23:20:36.876783+0000) 2024-03-18T23:20:59.796 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 18 23:20:59 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-9[98163]: 2024-03-18T23:20:59.386+0000 7ff64b26e700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-03-18T23:20:16.877280+0000 front 2024-03-18T23:20:16.877254+0000 (oldest deadline 2024-03-18T23:20:41.577384+0000) 2024-03-18T23:20:59.796 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 18 23:20:59 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-9[98163]: 2024-03-18T23:20:59.386+0000 7ff64b26e700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-18T23:20:23.278575+0000 front 2024-03-18T23:20:23.278614+0000 (oldest deadline 2024-03-18T23:20:49.178702+0000) 2024-03-18T23:20:59.796 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 18 23:20:59 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-9[98163]: 2024-03-18T23:20:59.386+0000 7ff64b26e700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-03-18T23:20:33.080980+0000 front 2024-03-18T23:20:33.081247+0000 (oldest deadline 2024-03-18T23:20:56.580916+0000) 2024-03-18T23:20:59.960 INFO:journalctl@ceph.osd.8.smithi175.stdout:Mar 18 23:20:59 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-8[86041]: 2024-03-18T23:20:59.469+0000 7fa05c798700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:20:02.615291+0000 front 2024-03-18T23:20:02.615437+0000 (oldest deadline 2024-03-18T23:20:24.315195+0000) 2024-03-18T23:20:59.961 INFO:journalctl@ceph.osd.8.smithi175.stdout:Mar 18 23:20:59 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-8[86041]: 2024-03-18T23:20:59.469+0000 7fa05c798700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-03-18T23:20:17.717904+0000 front 2024-03-18T23:20:17.718012+0000 (oldest deadline 2024-03-18T23:20:41.218013+0000) 2024-03-18T23:20:59.961 INFO:journalctl@ceph.osd.8.smithi175.stdout:Mar 18 23:20:59 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-8[86041]: 2024-03-18T23:20:59.469+0000 7fa05c798700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-18T23:20:23.519110+0000 front 2024-03-18T23:20:23.519209+0000 (oldest deadline 2024-03-18T23:20:48.818956+0000) 2024-03-18T23:20:59.961 INFO:journalctl@ceph.osd.8.smithi175.stdout:Mar 18 23:20:59 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-8[86041]: 2024-03-18T23:20:59.469+0000 7fa05c798700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-03-18T23:20:35.220337+0000 front 2024-03-18T23:20:35.220428+0000 (oldest deadline 2024-03-18T23:20:57.520416+0000) 2024-03-18T23:20:59.962 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:20:59 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:20:59.634+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:19:57.947723+0000 front 2024-03-18T23:19:57.947785+0000 (oldest deadline 2024-03-18T23:20:28.548565+0000) 2024-03-18T23:20:59.962 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:20:59 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:20:59.634+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-03-18T23:20:08.549207+0000 front 2024-03-18T23:20:08.549248+0000 (oldest deadline 2024-03-18T23:20:33.849233+0000) 2024-03-18T23:20:59.962 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:20:59 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:20:59.634+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-03-18T23:20:16.150579+0000 front 2024-03-18T23:20:16.150606+0000 (oldest deadline 2024-03-18T23:20:40.751401+0000) 2024-03-18T23:20:59.963 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:20:59 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:20:59.634+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-03-18T23:20:33.653867+0000 front 2024-03-18T23:20:33.653710+0000 (oldest deadline 2024-03-18T23:20:59.553560+0000) 2024-03-18T23:21:00.274 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 18 23:20:59 smithi139 podman[104934]: 2024-03-18 23:20:59.995733108 +0000 UTC m=+5.584217284 container died e4866443a6ae3bed3442c84892e2a076f5964479b3ad4acfc5bb4c539e74c875 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-7, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_BRANCH=HEAD) 2024-03-18T23:21:00.275 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:21:00 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:21:00.060+0000 7f83cd240700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:19:57.904964+0000 front 2024-03-18T23:19:57.904988+0000 (oldest deadline 2024-03-18T23:20:23.805169+0000) 2024-03-18T23:21:00.275 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:21:00 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:21:00.060+0000 7f83cd240700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-03-18T23:20:10.806661+0000 front 2024-03-18T23:20:10.806621+0000 (oldest deadline 2024-03-18T23:20:31.906747+0000) 2024-03-18T23:21:00.275 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:21:00 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:21:00.060+0000 7f83cd240700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-03-18T23:20:16.508876+0000 front 2024-03-18T23:20:16.508656+0000 (oldest deadline 2024-03-18T23:20:41.808738+0000) 2024-03-18T23:21:00.275 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:21:00 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:21:00.060+0000 7f83cd240700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-18T23:20:24.710033+0000 front 2024-03-18T23:20:24.709803+0000 (oldest deadline 2024-03-18T23:20:50.510257+0000) 2024-03-18T23:21:00.564 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 18 23:21:00 smithi139 podman[104934]: 2024-03-18 23:21:00.365398079 +0000 UTC m=+5.953882256 container cleanup e4866443a6ae3bed3442c84892e2a076f5964479b3ad4acfc5bb4c539e74c875 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-7, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-03-18T23:21:00.564 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 18 23:21:00 smithi139 bash[104934]: ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-7 2024-03-18T23:21:00.879 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 18 23:21:00 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-9[98163]: 2024-03-18T23:21:00.413+0000 7ff64b26e700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:20:01.074760+0000 front 2024-03-18T23:20:01.074900+0000 (oldest deadline 2024-03-18T23:20:26.974968+0000) 2024-03-18T23:21:00.879 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 18 23:21:00 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-9[98163]: 2024-03-18T23:21:00.413+0000 7ff64b26e700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-03-18T23:20:10.976967+0000 front 2024-03-18T23:20:10.976857+0000 (oldest deadline 2024-03-18T23:20:36.876783+0000) 2024-03-18T23:21:00.879 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 18 23:21:00 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-9[98163]: 2024-03-18T23:21:00.413+0000 7ff64b26e700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-03-18T23:20:16.877280+0000 front 2024-03-18T23:20:16.877254+0000 (oldest deadline 2024-03-18T23:20:41.577384+0000) 2024-03-18T23:21:00.879 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 18 23:21:00 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-9[98163]: 2024-03-18T23:21:00.413+0000 7ff64b26e700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-18T23:20:23.278575+0000 front 2024-03-18T23:20:23.278614+0000 (oldest deadline 2024-03-18T23:20:49.178702+0000) 2024-03-18T23:21:00.879 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 18 23:21:00 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-9[98163]: 2024-03-18T23:21:00.413+0000 7ff64b26e700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-03-18T23:20:33.080980+0000 front 2024-03-18T23:20:33.081247+0000 (oldest deadline 2024-03-18T23:20:56.580916+0000) 2024-03-18T23:21:00.960 INFO:journalctl@ceph.osd.8.smithi175.stdout:Mar 18 23:21:00 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-8[86041]: 2024-03-18T23:21:00.508+0000 7fa05c798700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:20:02.615291+0000 front 2024-03-18T23:20:02.615437+0000 (oldest deadline 2024-03-18T23:20:24.315195+0000) 2024-03-18T23:21:00.960 INFO:journalctl@ceph.osd.8.smithi175.stdout:Mar 18 23:21:00 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-8[86041]: 2024-03-18T23:21:00.508+0000 7fa05c798700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-03-18T23:20:17.717904+0000 front 2024-03-18T23:20:17.718012+0000 (oldest deadline 2024-03-18T23:20:41.218013+0000) 2024-03-18T23:21:00.961 INFO:journalctl@ceph.osd.8.smithi175.stdout:Mar 18 23:21:00 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-8[86041]: 2024-03-18T23:21:00.508+0000 7fa05c798700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-18T23:20:23.519110+0000 front 2024-03-18T23:20:23.519209+0000 (oldest deadline 2024-03-18T23:20:48.818956+0000) 2024-03-18T23:21:00.961 INFO:journalctl@ceph.osd.8.smithi175.stdout:Mar 18 23:21:00 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-8[86041]: 2024-03-18T23:21:00.508+0000 7fa05c798700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-03-18T23:20:35.220337+0000 front 2024-03-18T23:20:35.220428+0000 (oldest deadline 2024-03-18T23:20:57.520416+0000) 2024-03-18T23:21:00.961 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:21:00 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:21:00.619+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:19:57.947723+0000 front 2024-03-18T23:19:57.947785+0000 (oldest deadline 2024-03-18T23:20:28.548565+0000) 2024-03-18T23:21:00.961 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:21:00 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:21:00.619+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-03-18T23:20:08.549207+0000 front 2024-03-18T23:20:08.549248+0000 (oldest deadline 2024-03-18T23:20:33.849233+0000) 2024-03-18T23:21:00.961 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:21:00 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:21:00.619+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-03-18T23:20:16.150579+0000 front 2024-03-18T23:20:16.150606+0000 (oldest deadline 2024-03-18T23:20:40.751401+0000) 2024-03-18T23:21:00.961 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:21:00 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:21:00.619+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-03-18T23:20:33.653867+0000 front 2024-03-18T23:20:33.653710+0000 (oldest deadline 2024-03-18T23:20:59.553560+0000) 2024-03-18T23:21:00.972 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 18 23:21:00 smithi139 podman[104963]: 2024-03-18 23:21:00.625021721 +0000 UTC m=+0.629290205 container remove e4866443a6ae3bed3442c84892e2a076f5964479b3ad4acfc5bb4c539e74c875 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-7, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_CLEAN=True) 2024-03-18T23:21:01.243 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 18 23:21:00 smithi139 podman[105056]: 2024-03-18 23:21:00.877479386 +0000 UTC m=+0.018363731 image pull 2312c8e42d0069a2dbb2364bddb00264db4b800e7f18a7e7e4eb8f3ecab2ff1c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639 2024-03-18T23:21:01.244 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 18 23:21:00 smithi139 podman[105056]: 2024-03-18 23:21:00.984439958 +0000 UTC m=+0.125324340 container create 22d87b13b3cd5ba645f3965a7c8f58cb2c07a4e116428bde0bbbba89e31cd17f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-7-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_CLEAN=True, org.label-schema.build-date=20240307, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-03-18T23:21:01.244 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:21:01 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:21:01.095+0000 7f83cd240700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:19:57.904964+0000 front 2024-03-18T23:19:57.904988+0000 (oldest deadline 2024-03-18T23:20:23.805169+0000) 2024-03-18T23:21:01.244 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:21:01 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:21:01.095+0000 7f83cd240700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-03-18T23:20:10.806661+0000 front 2024-03-18T23:20:10.806621+0000 (oldest deadline 2024-03-18T23:20:31.906747+0000) 2024-03-18T23:21:01.244 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:21:01 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:21:01.095+0000 7f83cd240700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-03-18T23:20:16.508876+0000 front 2024-03-18T23:20:16.508656+0000 (oldest deadline 2024-03-18T23:20:41.808738+0000) 2024-03-18T23:21:01.244 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:21:01 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:21:01.095+0000 7f83cd240700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-18T23:20:24.710033+0000 front 2024-03-18T23:20:24.709803+0000 (oldest deadline 2024-03-18T23:20:50.510257+0000) 2024-03-18T23:21:01.244 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:21:01 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:21:01.095+0000 7f83cd240700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-03-18T23:20:35.111771+0000 front 2024-03-18T23:20:35.111796+0000 (oldest deadline 2024-03-18T23:21:01.011950+0000) 2024-03-18T23:21:01.538 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 18 23:21:01 smithi139 podman[105056]: 2024-03-18 23:21:01.245109656 +0000 UTC m=+0.385993998 container init 22d87b13b3cd5ba645f3965a7c8f58cb2c07a4e116428bde0bbbba89e31cd17f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-7-deactivate, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD) 2024-03-18T23:21:01.538 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 18 23:21:01 smithi139 podman[105056]: 2024-03-18 23:21:01.248388714 +0000 UTC m=+0.389273039 container start 22d87b13b3cd5ba645f3965a7c8f58cb2c07a4e116428bde0bbbba89e31cd17f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-7-deactivate, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-18T23:21:01.538 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 18 23:21:01 smithi139 podman[105056]: 2024-03-18 23:21:01.36188455 +0000 UTC m=+0.502768881 container attach 22d87b13b3cd5ba645f3965a7c8f58cb2c07a4e116428bde0bbbba89e31cd17f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-7-deactivate, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, ceph=True, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True) 2024-03-18T23:21:01.814 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 18 23:21:01 smithi139 podman[105056]: 2024-03-18 23:21:01.539901201 +0000 UTC m=+0.680785531 container died 22d87b13b3cd5ba645f3965a7c8f58cb2c07a4e116428bde0bbbba89e31cd17f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-7-deactivate, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD) 2024-03-18T23:21:01.879 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 18 23:21:01 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-9[98163]: 2024-03-18T23:21:01.447+0000 7ff64b26e700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:20:01.074760+0000 front 2024-03-18T23:20:01.074900+0000 (oldest deadline 2024-03-18T23:20:26.974968+0000) 2024-03-18T23:21:01.879 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 18 23:21:01 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-9[98163]: 2024-03-18T23:21:01.447+0000 7ff64b26e700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-03-18T23:20:10.976967+0000 front 2024-03-18T23:20:10.976857+0000 (oldest deadline 2024-03-18T23:20:36.876783+0000) 2024-03-18T23:21:01.879 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 18 23:21:01 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-9[98163]: 2024-03-18T23:21:01.447+0000 7ff64b26e700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-03-18T23:20:16.877280+0000 front 2024-03-18T23:20:16.877254+0000 (oldest deadline 2024-03-18T23:20:41.577384+0000) 2024-03-18T23:21:01.879 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 18 23:21:01 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-9[98163]: 2024-03-18T23:21:01.447+0000 7ff64b26e700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-18T23:20:23.278575+0000 front 2024-03-18T23:20:23.278614+0000 (oldest deadline 2024-03-18T23:20:49.178702+0000) 2024-03-18T23:21:01.879 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 18 23:21:01 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-9[98163]: 2024-03-18T23:21:01.447+0000 7ff64b26e700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-03-18T23:20:33.080980+0000 front 2024-03-18T23:20:33.081247+0000 (oldest deadline 2024-03-18T23:20:56.580916+0000) 2024-03-18T23:21:01.961 INFO:journalctl@ceph.osd.8.smithi175.stdout:Mar 18 23:21:01 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-8[86041]: 2024-03-18T23:21:01.465+0000 7fa05c798700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:20:02.615291+0000 front 2024-03-18T23:20:02.615437+0000 (oldest deadline 2024-03-18T23:20:24.315195+0000) 2024-03-18T23:21:01.961 INFO:journalctl@ceph.osd.8.smithi175.stdout:Mar 18 23:21:01 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-8[86041]: 2024-03-18T23:21:01.465+0000 7fa05c798700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-03-18T23:20:17.717904+0000 front 2024-03-18T23:20:17.718012+0000 (oldest deadline 2024-03-18T23:20:41.218013+0000) 2024-03-18T23:21:01.962 INFO:journalctl@ceph.osd.8.smithi175.stdout:Mar 18 23:21:01 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-8[86041]: 2024-03-18T23:21:01.465+0000 7fa05c798700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-18T23:20:23.519110+0000 front 2024-03-18T23:20:23.519209+0000 (oldest deadline 2024-03-18T23:20:48.818956+0000) 2024-03-18T23:21:01.962 INFO:journalctl@ceph.osd.8.smithi175.stdout:Mar 18 23:21:01 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-8[86041]: 2024-03-18T23:21:01.465+0000 7fa05c798700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-03-18T23:20:35.220337+0000 front 2024-03-18T23:20:35.220428+0000 (oldest deadline 2024-03-18T23:20:57.520416+0000) 2024-03-18T23:21:01.962 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:21:01 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:21:01.589+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:19:57.947723+0000 front 2024-03-18T23:19:57.947785+0000 (oldest deadline 2024-03-18T23:20:28.548565+0000) 2024-03-18T23:21:01.962 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:21:01 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:21:01.589+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-03-18T23:20:08.549207+0000 front 2024-03-18T23:20:08.549248+0000 (oldest deadline 2024-03-18T23:20:33.849233+0000) 2024-03-18T23:21:01.962 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:21:01 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:21:01.589+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-03-18T23:20:16.150579+0000 front 2024-03-18T23:20:16.150606+0000 (oldest deadline 2024-03-18T23:20:40.751401+0000) 2024-03-18T23:21:01.962 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:21:01 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:21:01.589+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-03-18T23:20:33.653867+0000 front 2024-03-18T23:20:33.653710+0000 (oldest deadline 2024-03-18T23:20:59.553560+0000) 2024-03-18T23:21:02.147 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:21:02 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:21:02.056+0000 7f83cd240700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:19:57.904964+0000 front 2024-03-18T23:19:57.904988+0000 (oldest deadline 2024-03-18T23:20:23.805169+0000) 2024-03-18T23:21:02.147 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:21:02 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:21:02.056+0000 7f83cd240700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-03-18T23:20:10.806661+0000 front 2024-03-18T23:20:10.806621+0000 (oldest deadline 2024-03-18T23:20:31.906747+0000) 2024-03-18T23:21:02.147 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:21:02 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:21:02.056+0000 7f83cd240700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-03-18T23:20:16.508876+0000 front 2024-03-18T23:20:16.508656+0000 (oldest deadline 2024-03-18T23:20:41.808738+0000) 2024-03-18T23:21:02.147 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:21:02 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:21:02.056+0000 7f83cd240700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-18T23:20:24.710033+0000 front 2024-03-18T23:20:24.709803+0000 (oldest deadline 2024-03-18T23:20:50.510257+0000) 2024-03-18T23:21:02.148 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:21:02 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:21:02.056+0000 7f83cd240700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-03-18T23:20:35.111771+0000 front 2024-03-18T23:20:35.111796+0000 (oldest deadline 2024-03-18T23:21:01.011950+0000) 2024-03-18T23:21:02.163 DEBUG:teuthology.orchestra.run.smithi139:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c63591ea-e57a-11ee-95c9-87774f69a715@osd.7.service' 2024-03-18T23:21:02.205 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-18T23:21:02.206 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-03-18T23:21:02.206 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-03-18T23:21:02.206 DEBUG:teuthology.orchestra.run.smithi175:> sudo systemctl stop ceph-c63591ea-e57a-11ee-95c9-87774f69a715@osd.8 2024-03-18T23:21:02.521 INFO:journalctl@ceph.osd.8.smithi175.stdout:Mar 18 23:21:02 smithi175 systemd[1]: Stopping Ceph osd.8 for c63591ea-e57a-11ee-95c9-87774f69a715... 2024-03-18T23:21:02.521 INFO:journalctl@ceph.osd.8.smithi175.stdout:Mar 18 23:21:02 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-8[86041]: 2024-03-18T23:21:02.479+0000 7fa05c798700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:20:02.615291+0000 front 2024-03-18T23:20:02.615437+0000 (oldest deadline 2024-03-18T23:20:24.315195+0000) 2024-03-18T23:21:02.522 INFO:journalctl@ceph.osd.8.smithi175.stdout:Mar 18 23:21:02 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-8[86041]: 2024-03-18T23:21:02.479+0000 7fa05c798700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-03-18T23:20:17.717904+0000 front 2024-03-18T23:20:17.718012+0000 (oldest deadline 2024-03-18T23:20:41.218013+0000) 2024-03-18T23:21:02.522 INFO:journalctl@ceph.osd.8.smithi175.stdout:Mar 18 23:21:02 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-8[86041]: 2024-03-18T23:21:02.479+0000 7fa05c798700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-18T23:20:23.519110+0000 front 2024-03-18T23:20:23.519209+0000 (oldest deadline 2024-03-18T23:20:48.818956+0000) 2024-03-18T23:21:02.522 INFO:journalctl@ceph.osd.8.smithi175.stdout:Mar 18 23:21:02 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-8[86041]: 2024-03-18T23:21:02.479+0000 7fa05c798700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-03-18T23:20:35.220337+0000 front 2024-03-18T23:20:35.220428+0000 (oldest deadline 2024-03-18T23:20:57.520416+0000) 2024-03-18T23:21:02.522 INFO:journalctl@ceph.osd.8.smithi175.stdout:Mar 18 23:21:02 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-8[86041]: 2024-03-18T23:21:02.521+0000 7fa0641ac700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-18T23:21:02.522 INFO:journalctl@ceph.osd.8.smithi175.stdout:Mar 18 23:21:02 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-8[86041]: 2024-03-18T23:21:02.521+0000 7fa0641ac700 -1 osd.8 104 *** Got signal Terminated *** 2024-03-18T23:21:02.522 INFO:journalctl@ceph.osd.8.smithi175.stdout:Mar 18 23:21:02 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-8[86041]: 2024-03-18T23:21:02.521+0000 7fa0641ac700 -1 osd.8 104 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-18T23:21:02.878 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 18 23:21:02 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-9[98163]: 2024-03-18T23:21:02.489+0000 7ff64b26e700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:20:01.074760+0000 front 2024-03-18T23:20:01.074900+0000 (oldest deadline 2024-03-18T23:20:26.974968+0000) 2024-03-18T23:21:02.879 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 18 23:21:02 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-9[98163]: 2024-03-18T23:21:02.489+0000 7ff64b26e700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-03-18T23:20:10.976967+0000 front 2024-03-18T23:20:10.976857+0000 (oldest deadline 2024-03-18T23:20:36.876783+0000) 2024-03-18T23:21:02.879 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 18 23:21:02 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-9[98163]: 2024-03-18T23:21:02.489+0000 7ff64b26e700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-03-18T23:20:16.877280+0000 front 2024-03-18T23:20:16.877254+0000 (oldest deadline 2024-03-18T23:20:41.577384+0000) 2024-03-18T23:21:02.879 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 18 23:21:02 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-9[98163]: 2024-03-18T23:21:02.489+0000 7ff64b26e700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-18T23:20:23.278575+0000 front 2024-03-18T23:20:23.278614+0000 (oldest deadline 2024-03-18T23:20:49.178702+0000) 2024-03-18T23:21:02.879 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 18 23:21:02 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-9[98163]: 2024-03-18T23:21:02.489+0000 7ff64b26e700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-03-18T23:20:33.080980+0000 front 2024-03-18T23:20:33.081247+0000 (oldest deadline 2024-03-18T23:20:56.580916+0000) 2024-03-18T23:21:02.960 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:21:02 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:21:02.577+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:19:57.947723+0000 front 2024-03-18T23:19:57.947785+0000 (oldest deadline 2024-03-18T23:20:28.548565+0000) 2024-03-18T23:21:02.960 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:21:02 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:21:02.577+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-03-18T23:20:08.549207+0000 front 2024-03-18T23:20:08.549248+0000 (oldest deadline 2024-03-18T23:20:33.849233+0000) 2024-03-18T23:21:02.960 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:21:02 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:21:02.577+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-03-18T23:20:16.150579+0000 front 2024-03-18T23:20:16.150606+0000 (oldest deadline 2024-03-18T23:20:40.751401+0000) 2024-03-18T23:21:02.961 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:21:02 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:21:02.577+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-03-18T23:20:33.653867+0000 front 2024-03-18T23:20:33.653710+0000 (oldest deadline 2024-03-18T23:20:59.553560+0000) 2024-03-18T23:21:03.564 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:21:03 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:21:03.105+0000 7f83cd240700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:19:57.904964+0000 front 2024-03-18T23:19:57.904988+0000 (oldest deadline 2024-03-18T23:20:23.805169+0000) 2024-03-18T23:21:03.564 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:21:03 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:21:03.105+0000 7f83cd240700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-03-18T23:20:10.806661+0000 front 2024-03-18T23:20:10.806621+0000 (oldest deadline 2024-03-18T23:20:31.906747+0000) 2024-03-18T23:21:03.564 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:21:03 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:21:03.105+0000 7f83cd240700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-03-18T23:20:16.508876+0000 front 2024-03-18T23:20:16.508656+0000 (oldest deadline 2024-03-18T23:20:41.808738+0000) 2024-03-18T23:21:03.564 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:21:03 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:21:03.105+0000 7f83cd240700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-18T23:20:24.710033+0000 front 2024-03-18T23:20:24.709803+0000 (oldest deadline 2024-03-18T23:20:50.510257+0000) 2024-03-18T23:21:03.564 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:21:03 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:21:03.105+0000 7f83cd240700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-03-18T23:20:35.111771+0000 front 2024-03-18T23:20:35.111796+0000 (oldest deadline 2024-03-18T23:21:01.011950+0000) 2024-03-18T23:21:03.879 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 18 23:21:03 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-9[98163]: 2024-03-18T23:21:03.475+0000 7ff64b26e700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:20:01.074760+0000 front 2024-03-18T23:20:01.074900+0000 (oldest deadline 2024-03-18T23:20:26.974968+0000) 2024-03-18T23:21:03.879 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 18 23:21:03 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-9[98163]: 2024-03-18T23:21:03.475+0000 7ff64b26e700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-03-18T23:20:10.976967+0000 front 2024-03-18T23:20:10.976857+0000 (oldest deadline 2024-03-18T23:20:36.876783+0000) 2024-03-18T23:21:03.879 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 18 23:21:03 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-9[98163]: 2024-03-18T23:21:03.475+0000 7ff64b26e700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-03-18T23:20:16.877280+0000 front 2024-03-18T23:20:16.877254+0000 (oldest deadline 2024-03-18T23:20:41.577384+0000) 2024-03-18T23:21:03.879 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 18 23:21:03 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-9[98163]: 2024-03-18T23:21:03.475+0000 7ff64b26e700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-18T23:20:23.278575+0000 front 2024-03-18T23:20:23.278614+0000 (oldest deadline 2024-03-18T23:20:49.178702+0000) 2024-03-18T23:21:03.879 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 18 23:21:03 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-9[98163]: 2024-03-18T23:21:03.475+0000 7ff64b26e700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-03-18T23:20:33.080980+0000 front 2024-03-18T23:20:33.081247+0000 (oldest deadline 2024-03-18T23:20:56.580916+0000) 2024-03-18T23:21:03.960 INFO:journalctl@ceph.osd.8.smithi175.stdout:Mar 18 23:21:03 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-8[86041]: 2024-03-18T23:21:03.516+0000 7fa05c798700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:20:02.615291+0000 front 2024-03-18T23:20:02.615437+0000 (oldest deadline 2024-03-18T23:20:24.315195+0000) 2024-03-18T23:21:03.960 INFO:journalctl@ceph.osd.8.smithi175.stdout:Mar 18 23:21:03 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-8[86041]: 2024-03-18T23:21:03.516+0000 7fa05c798700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-03-18T23:20:17.717904+0000 front 2024-03-18T23:20:17.718012+0000 (oldest deadline 2024-03-18T23:20:41.218013+0000) 2024-03-18T23:21:03.960 INFO:journalctl@ceph.osd.8.smithi175.stdout:Mar 18 23:21:03 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-8[86041]: 2024-03-18T23:21:03.516+0000 7fa05c798700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-18T23:20:23.519110+0000 front 2024-03-18T23:20:23.519209+0000 (oldest deadline 2024-03-18T23:20:48.818956+0000) 2024-03-18T23:21:03.961 INFO:journalctl@ceph.osd.8.smithi175.stdout:Mar 18 23:21:03 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-8[86041]: 2024-03-18T23:21:03.516+0000 7fa05c798700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-03-18T23:20:35.220337+0000 front 2024-03-18T23:20:35.220428+0000 (oldest deadline 2024-03-18T23:20:57.520416+0000) 2024-03-18T23:21:03.961 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:21:03 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:21:03.565+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:19:57.947723+0000 front 2024-03-18T23:19:57.947785+0000 (oldest deadline 2024-03-18T23:20:28.548565+0000) 2024-03-18T23:21:03.961 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:21:03 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:21:03.565+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-03-18T23:20:08.549207+0000 front 2024-03-18T23:20:08.549248+0000 (oldest deadline 2024-03-18T23:20:33.849233+0000) 2024-03-18T23:21:03.961 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:21:03 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:21:03.565+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-03-18T23:20:16.150579+0000 front 2024-03-18T23:20:16.150606+0000 (oldest deadline 2024-03-18T23:20:40.751401+0000) 2024-03-18T23:21:03.961 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:21:03 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:21:03.565+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-03-18T23:20:33.653867+0000 front 2024-03-18T23:20:33.653710+0000 (oldest deadline 2024-03-18T23:20:59.553560+0000) 2024-03-18T23:21:04.564 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:21:04 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:21:04.070+0000 7f83cd240700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:19:57.904964+0000 front 2024-03-18T23:19:57.904988+0000 (oldest deadline 2024-03-18T23:20:23.805169+0000) 2024-03-18T23:21:04.564 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:21:04 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:21:04.070+0000 7f83cd240700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-03-18T23:20:10.806661+0000 front 2024-03-18T23:20:10.806621+0000 (oldest deadline 2024-03-18T23:20:31.906747+0000) 2024-03-18T23:21:04.564 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:21:04 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:21:04.070+0000 7f83cd240700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-03-18T23:20:16.508876+0000 front 2024-03-18T23:20:16.508656+0000 (oldest deadline 2024-03-18T23:20:41.808738+0000) 2024-03-18T23:21:04.564 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:21:04 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:21:04.070+0000 7f83cd240700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-18T23:20:24.710033+0000 front 2024-03-18T23:20:24.709803+0000 (oldest deadline 2024-03-18T23:20:50.510257+0000) 2024-03-18T23:21:04.564 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:21:04 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:21:04.070+0000 7f83cd240700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-03-18T23:20:35.111771+0000 front 2024-03-18T23:20:35.111796+0000 (oldest deadline 2024-03-18T23:21:01.011950+0000) 2024-03-18T23:21:04.878 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 18 23:21:04 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-9[98163]: 2024-03-18T23:21:04.525+0000 7ff64b26e700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:20:01.074760+0000 front 2024-03-18T23:20:01.074900+0000 (oldest deadline 2024-03-18T23:20:26.974968+0000) 2024-03-18T23:21:04.879 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 18 23:21:04 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-9[98163]: 2024-03-18T23:21:04.525+0000 7ff64b26e700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-03-18T23:20:10.976967+0000 front 2024-03-18T23:20:10.976857+0000 (oldest deadline 2024-03-18T23:20:36.876783+0000) 2024-03-18T23:21:04.879 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 18 23:21:04 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-9[98163]: 2024-03-18T23:21:04.525+0000 7ff64b26e700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-03-18T23:20:16.877280+0000 front 2024-03-18T23:20:16.877254+0000 (oldest deadline 2024-03-18T23:20:41.577384+0000) 2024-03-18T23:21:04.879 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 18 23:21:04 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-9[98163]: 2024-03-18T23:21:04.525+0000 7ff64b26e700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-18T23:20:23.278575+0000 front 2024-03-18T23:20:23.278614+0000 (oldest deadline 2024-03-18T23:20:49.178702+0000) 2024-03-18T23:21:04.879 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 18 23:21:04 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-9[98163]: 2024-03-18T23:21:04.525+0000 7ff64b26e700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-03-18T23:20:33.080980+0000 front 2024-03-18T23:20:33.081247+0000 (oldest deadline 2024-03-18T23:20:56.580916+0000) 2024-03-18T23:21:04.960 INFO:journalctl@ceph.osd.8.smithi175.stdout:Mar 18 23:21:04 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-8[86041]: 2024-03-18T23:21:04.557+0000 7fa05c798700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:20:02.615291+0000 front 2024-03-18T23:20:02.615437+0000 (oldest deadline 2024-03-18T23:20:24.315195+0000) 2024-03-18T23:21:04.960 INFO:journalctl@ceph.osd.8.smithi175.stdout:Mar 18 23:21:04 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-8[86041]: 2024-03-18T23:21:04.557+0000 7fa05c798700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-03-18T23:20:17.717904+0000 front 2024-03-18T23:20:17.718012+0000 (oldest deadline 2024-03-18T23:20:41.218013+0000) 2024-03-18T23:21:04.960 INFO:journalctl@ceph.osd.8.smithi175.stdout:Mar 18 23:21:04 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-8[86041]: 2024-03-18T23:21:04.557+0000 7fa05c798700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-18T23:20:23.519110+0000 front 2024-03-18T23:20:23.519209+0000 (oldest deadline 2024-03-18T23:20:48.818956+0000) 2024-03-18T23:21:04.961 INFO:journalctl@ceph.osd.8.smithi175.stdout:Mar 18 23:21:04 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-8[86041]: 2024-03-18T23:21:04.557+0000 7fa05c798700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-03-18T23:20:35.220337+0000 front 2024-03-18T23:20:35.220428+0000 (oldest deadline 2024-03-18T23:20:57.520416+0000) 2024-03-18T23:21:04.961 INFO:journalctl@ceph.osd.8.smithi175.stdout:Mar 18 23:21:04 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-8[86041]: 2024-03-18T23:21:04.557+0000 7fa05c798700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.175:6812 osd.5 since back 2024-03-18T23:20:42.622323+0000 front 2024-03-18T23:20:42.622266+0000 (oldest deadline 2024-03-18T23:21:04.322339+0000) 2024-03-18T23:21:04.961 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:21:04 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:21:04.612+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:19:57.947723+0000 front 2024-03-18T23:19:57.947785+0000 (oldest deadline 2024-03-18T23:20:28.548565+0000) 2024-03-18T23:21:04.961 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:21:04 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:21:04.612+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-03-18T23:20:08.549207+0000 front 2024-03-18T23:20:08.549248+0000 (oldest deadline 2024-03-18T23:20:33.849233+0000) 2024-03-18T23:21:04.961 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:21:04 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:21:04.612+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-03-18T23:20:16.150579+0000 front 2024-03-18T23:20:16.150606+0000 (oldest deadline 2024-03-18T23:20:40.751401+0000) 2024-03-18T23:21:04.961 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:21:04 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:21:04.612+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-03-18T23:20:33.653867+0000 front 2024-03-18T23:20:33.653710+0000 (oldest deadline 2024-03-18T23:20:59.553560+0000) 2024-03-18T23:21:04.961 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:21:04 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:21:04.612+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.175:6812 osd.5 since back 2024-03-18T23:20:39.553929+0000 front 2024-03-18T23:20:39.553999+0000 (oldest deadline 2024-03-18T23:21:04.254102+0000) 2024-03-18T23:21:05.314 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:21:05 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:21:05.043+0000 7f83cd240700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:19:57.904964+0000 front 2024-03-18T23:19:57.904988+0000 (oldest deadline 2024-03-18T23:20:23.805169+0000) 2024-03-18T23:21:05.314 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:21:05 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:21:05.043+0000 7f83cd240700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-03-18T23:20:10.806661+0000 front 2024-03-18T23:20:10.806621+0000 (oldest deadline 2024-03-18T23:20:31.906747+0000) 2024-03-18T23:21:05.314 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:21:05 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:21:05.043+0000 7f83cd240700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-03-18T23:20:16.508876+0000 front 2024-03-18T23:20:16.508656+0000 (oldest deadline 2024-03-18T23:20:41.808738+0000) 2024-03-18T23:21:05.314 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:21:05 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:21:05.043+0000 7f83cd240700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-18T23:20:24.710033+0000 front 2024-03-18T23:20:24.709803+0000 (oldest deadline 2024-03-18T23:20:50.510257+0000) 2024-03-18T23:21:05.315 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:21:05 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:21:05.043+0000 7f83cd240700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-03-18T23:20:35.111771+0000 front 2024-03-18T23:20:35.111796+0000 (oldest deadline 2024-03-18T23:21:01.011950+0000) 2024-03-18T23:21:05.879 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 18 23:21:05 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-9[98163]: 2024-03-18T23:21:05.556+0000 7ff64b26e700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:20:01.074760+0000 front 2024-03-18T23:20:01.074900+0000 (oldest deadline 2024-03-18T23:20:26.974968+0000) 2024-03-18T23:21:05.879 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 18 23:21:05 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-9[98163]: 2024-03-18T23:21:05.556+0000 7ff64b26e700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-03-18T23:20:10.976967+0000 front 2024-03-18T23:20:10.976857+0000 (oldest deadline 2024-03-18T23:20:36.876783+0000) 2024-03-18T23:21:05.879 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 18 23:21:05 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-9[98163]: 2024-03-18T23:21:05.556+0000 7ff64b26e700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-03-18T23:20:16.877280+0000 front 2024-03-18T23:20:16.877254+0000 (oldest deadline 2024-03-18T23:20:41.577384+0000) 2024-03-18T23:21:05.879 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 18 23:21:05 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-9[98163]: 2024-03-18T23:21:05.556+0000 7ff64b26e700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-18T23:20:23.278575+0000 front 2024-03-18T23:20:23.278614+0000 (oldest deadline 2024-03-18T23:20:49.178702+0000) 2024-03-18T23:21:05.879 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 18 23:21:05 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-9[98163]: 2024-03-18T23:21:05.556+0000 7ff64b26e700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-03-18T23:20:33.080980+0000 front 2024-03-18T23:20:33.081247+0000 (oldest deadline 2024-03-18T23:20:56.580916+0000) 2024-03-18T23:21:05.879 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 18 23:21:05 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-9[98163]: 2024-03-18T23:21:05.556+0000 7ff64b26e700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.175:6812 osd.5 since back 2024-03-18T23:20:39.382796+0000 front 2024-03-18T23:20:39.382823+0000 (oldest deadline 2024-03-18T23:21:05.282665+0000) 2024-03-18T23:21:05.960 INFO:journalctl@ceph.osd.8.smithi175.stdout:Mar 18 23:21:05 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-8[86041]: 2024-03-18T23:21:05.601+0000 7fa05c798700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:20:02.615291+0000 front 2024-03-18T23:20:02.615437+0000 (oldest deadline 2024-03-18T23:20:24.315195+0000) 2024-03-18T23:21:05.960 INFO:journalctl@ceph.osd.8.smithi175.stdout:Mar 18 23:21:05 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-8[86041]: 2024-03-18T23:21:05.601+0000 7fa05c798700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-03-18T23:20:17.717904+0000 front 2024-03-18T23:20:17.718012+0000 (oldest deadline 2024-03-18T23:20:41.218013+0000) 2024-03-18T23:21:05.960 INFO:journalctl@ceph.osd.8.smithi175.stdout:Mar 18 23:21:05 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-8[86041]: 2024-03-18T23:21:05.601+0000 7fa05c798700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-18T23:20:23.519110+0000 front 2024-03-18T23:20:23.519209+0000 (oldest deadline 2024-03-18T23:20:48.818956+0000) 2024-03-18T23:21:05.961 INFO:journalctl@ceph.osd.8.smithi175.stdout:Mar 18 23:21:05 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-8[86041]: 2024-03-18T23:21:05.601+0000 7fa05c798700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-03-18T23:20:35.220337+0000 front 2024-03-18T23:20:35.220428+0000 (oldest deadline 2024-03-18T23:20:57.520416+0000) 2024-03-18T23:21:05.961 INFO:journalctl@ceph.osd.8.smithi175.stdout:Mar 18 23:21:05 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-8[86041]: 2024-03-18T23:21:05.601+0000 7fa05c798700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.175:6812 osd.5 since back 2024-03-18T23:20:42.622323+0000 front 2024-03-18T23:20:42.622266+0000 (oldest deadline 2024-03-18T23:21:04.322339+0000) 2024-03-18T23:21:05.961 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:21:05 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:21:05.601+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:19:57.947723+0000 front 2024-03-18T23:19:57.947785+0000 (oldest deadline 2024-03-18T23:20:28.548565+0000) 2024-03-18T23:21:05.961 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:21:05 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:21:05.601+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-03-18T23:20:08.549207+0000 front 2024-03-18T23:20:08.549248+0000 (oldest deadline 2024-03-18T23:20:33.849233+0000) 2024-03-18T23:21:05.961 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:21:05 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:21:05.601+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-03-18T23:20:16.150579+0000 front 2024-03-18T23:20:16.150606+0000 (oldest deadline 2024-03-18T23:20:40.751401+0000) 2024-03-18T23:21:05.961 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:21:05 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:21:05.601+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-03-18T23:20:33.653867+0000 front 2024-03-18T23:20:33.653710+0000 (oldest deadline 2024-03-18T23:20:59.553560+0000) 2024-03-18T23:21:05.962 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:21:05 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:21:05.601+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.175:6812 osd.5 since back 2024-03-18T23:20:39.553929+0000 front 2024-03-18T23:20:39.553999+0000 (oldest deadline 2024-03-18T23:21:04.254102+0000) 2024-03-18T23:21:06.564 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:21:06 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:21:06.079+0000 7f83cd240700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:19:57.904964+0000 front 2024-03-18T23:19:57.904988+0000 (oldest deadline 2024-03-18T23:20:23.805169+0000) 2024-03-18T23:21:06.564 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:21:06 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:21:06.079+0000 7f83cd240700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-03-18T23:20:10.806661+0000 front 2024-03-18T23:20:10.806621+0000 (oldest deadline 2024-03-18T23:20:31.906747+0000) 2024-03-18T23:21:06.564 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:21:06 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:21:06.079+0000 7f83cd240700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-03-18T23:20:16.508876+0000 front 2024-03-18T23:20:16.508656+0000 (oldest deadline 2024-03-18T23:20:41.808738+0000) 2024-03-18T23:21:06.564 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:21:06 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:21:06.079+0000 7f83cd240700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-18T23:20:24.710033+0000 front 2024-03-18T23:20:24.709803+0000 (oldest deadline 2024-03-18T23:20:50.510257+0000) 2024-03-18T23:21:06.565 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:21:06 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:21:06.079+0000 7f83cd240700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-03-18T23:20:35.111771+0000 front 2024-03-18T23:20:35.111796+0000 (oldest deadline 2024-03-18T23:21:01.011950+0000) 2024-03-18T23:21:06.565 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:21:06 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:21:06.079+0000 7f83cd240700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.175:6812 osd.5 since back 2024-03-18T23:20:41.012537+0000 front 2024-03-18T23:20:41.012520+0000 (oldest deadline 2024-03-18T23:21:05.712608+0000) 2024-03-18T23:21:06.878 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 18 23:21:06 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-9[98163]: 2024-03-18T23:21:06.581+0000 7ff64b26e700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:20:01.074760+0000 front 2024-03-18T23:20:01.074900+0000 (oldest deadline 2024-03-18T23:20:26.974968+0000) 2024-03-18T23:21:06.879 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 18 23:21:06 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-9[98163]: 2024-03-18T23:21:06.581+0000 7ff64b26e700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-03-18T23:20:10.976967+0000 front 2024-03-18T23:20:10.976857+0000 (oldest deadline 2024-03-18T23:20:36.876783+0000) 2024-03-18T23:21:06.879 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 18 23:21:06 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-9[98163]: 2024-03-18T23:21:06.581+0000 7ff64b26e700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-03-18T23:20:16.877280+0000 front 2024-03-18T23:20:16.877254+0000 (oldest deadline 2024-03-18T23:20:41.577384+0000) 2024-03-18T23:21:06.879 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 18 23:21:06 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-9[98163]: 2024-03-18T23:21:06.581+0000 7ff64b26e700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-18T23:20:23.278575+0000 front 2024-03-18T23:20:23.278614+0000 (oldest deadline 2024-03-18T23:20:49.178702+0000) 2024-03-18T23:21:06.879 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 18 23:21:06 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-9[98163]: 2024-03-18T23:21:06.581+0000 7ff64b26e700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-03-18T23:20:33.080980+0000 front 2024-03-18T23:20:33.081247+0000 (oldest deadline 2024-03-18T23:20:56.580916+0000) 2024-03-18T23:21:06.879 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 18 23:21:06 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-9[98163]: 2024-03-18T23:21:06.581+0000 7ff64b26e700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.175:6812 osd.5 since back 2024-03-18T23:20:39.382796+0000 front 2024-03-18T23:20:39.382823+0000 (oldest deadline 2024-03-18T23:21:05.282665+0000) 2024-03-18T23:21:06.960 INFO:journalctl@ceph.osd.8.smithi175.stdout:Mar 18 23:21:06 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-8[86041]: 2024-03-18T23:21:06.577+0000 7fa05c798700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:20:02.615291+0000 front 2024-03-18T23:20:02.615437+0000 (oldest deadline 2024-03-18T23:20:24.315195+0000) 2024-03-18T23:21:06.960 INFO:journalctl@ceph.osd.8.smithi175.stdout:Mar 18 23:21:06 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-8[86041]: 2024-03-18T23:21:06.577+0000 7fa05c798700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-03-18T23:20:17.717904+0000 front 2024-03-18T23:20:17.718012+0000 (oldest deadline 2024-03-18T23:20:41.218013+0000) 2024-03-18T23:21:06.960 INFO:journalctl@ceph.osd.8.smithi175.stdout:Mar 18 23:21:06 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-8[86041]: 2024-03-18T23:21:06.577+0000 7fa05c798700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-18T23:20:23.519110+0000 front 2024-03-18T23:20:23.519209+0000 (oldest deadline 2024-03-18T23:20:48.818956+0000) 2024-03-18T23:21:06.961 INFO:journalctl@ceph.osd.8.smithi175.stdout:Mar 18 23:21:06 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-8[86041]: 2024-03-18T23:21:06.577+0000 7fa05c798700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-03-18T23:20:35.220337+0000 front 2024-03-18T23:20:35.220428+0000 (oldest deadline 2024-03-18T23:20:57.520416+0000) 2024-03-18T23:21:06.961 INFO:journalctl@ceph.osd.8.smithi175.stdout:Mar 18 23:21:06 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-8[86041]: 2024-03-18T23:21:06.577+0000 7fa05c798700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.175:6812 osd.5 since back 2024-03-18T23:20:42.622323+0000 front 2024-03-18T23:20:42.622266+0000 (oldest deadline 2024-03-18T23:21:04.322339+0000) 2024-03-18T23:21:06.961 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:21:06 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:21:06.641+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:19:57.947723+0000 front 2024-03-18T23:19:57.947785+0000 (oldest deadline 2024-03-18T23:20:28.548565+0000) 2024-03-18T23:21:06.961 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:21:06 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:21:06.641+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-03-18T23:20:08.549207+0000 front 2024-03-18T23:20:08.549248+0000 (oldest deadline 2024-03-18T23:20:33.849233+0000) 2024-03-18T23:21:06.961 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:21:06 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:21:06.641+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-03-18T23:20:16.150579+0000 front 2024-03-18T23:20:16.150606+0000 (oldest deadline 2024-03-18T23:20:40.751401+0000) 2024-03-18T23:21:06.961 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:21:06 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:21:06.641+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-03-18T23:20:33.653867+0000 front 2024-03-18T23:20:33.653710+0000 (oldest deadline 2024-03-18T23:20:59.553560+0000) 2024-03-18T23:21:06.961 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:21:06 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:21:06.641+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.175:6812 osd.5 since back 2024-03-18T23:20:39.553929+0000 front 2024-03-18T23:20:39.553999+0000 (oldest deadline 2024-03-18T23:21:04.254102+0000) 2024-03-18T23:21:07.564 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:21:07 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:21:07.110+0000 7f83cd240700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:19:57.904964+0000 front 2024-03-18T23:19:57.904988+0000 (oldest deadline 2024-03-18T23:20:23.805169+0000) 2024-03-18T23:21:07.564 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:21:07 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:21:07.110+0000 7f83cd240700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-03-18T23:20:10.806661+0000 front 2024-03-18T23:20:10.806621+0000 (oldest deadline 2024-03-18T23:20:31.906747+0000) 2024-03-18T23:21:07.564 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:21:07 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:21:07.110+0000 7f83cd240700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-03-18T23:20:16.508876+0000 front 2024-03-18T23:20:16.508656+0000 (oldest deadline 2024-03-18T23:20:41.808738+0000) 2024-03-18T23:21:07.564 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:21:07 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:21:07.110+0000 7f83cd240700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-18T23:20:24.710033+0000 front 2024-03-18T23:20:24.709803+0000 (oldest deadline 2024-03-18T23:20:50.510257+0000) 2024-03-18T23:21:07.564 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:21:07 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:21:07.110+0000 7f83cd240700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-03-18T23:20:35.111771+0000 front 2024-03-18T23:20:35.111796+0000 (oldest deadline 2024-03-18T23:21:01.011950+0000) 2024-03-18T23:21:07.565 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:21:07 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:21:07.110+0000 7f83cd240700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.175:6812 osd.5 since back 2024-03-18T23:20:41.012537+0000 front 2024-03-18T23:20:41.012520+0000 (oldest deadline 2024-03-18T23:21:05.712608+0000) 2024-03-18T23:21:07.879 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 18 23:21:07 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-9[98163]: 2024-03-18T23:21:07.539+0000 7ff64b26e700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:20:01.074760+0000 front 2024-03-18T23:20:01.074900+0000 (oldest deadline 2024-03-18T23:20:26.974968+0000) 2024-03-18T23:21:07.879 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 18 23:21:07 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-9[98163]: 2024-03-18T23:21:07.539+0000 7ff64b26e700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-03-18T23:20:10.976967+0000 front 2024-03-18T23:20:10.976857+0000 (oldest deadline 2024-03-18T23:20:36.876783+0000) 2024-03-18T23:21:07.879 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 18 23:21:07 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-9[98163]: 2024-03-18T23:21:07.539+0000 7ff64b26e700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-03-18T23:20:16.877280+0000 front 2024-03-18T23:20:16.877254+0000 (oldest deadline 2024-03-18T23:20:41.577384+0000) 2024-03-18T23:21:07.879 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 18 23:21:07 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-9[98163]: 2024-03-18T23:21:07.539+0000 7ff64b26e700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-18T23:20:23.278575+0000 front 2024-03-18T23:20:23.278614+0000 (oldest deadline 2024-03-18T23:20:49.178702+0000) 2024-03-18T23:21:07.879 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 18 23:21:07 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-9[98163]: 2024-03-18T23:21:07.539+0000 7ff64b26e700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-03-18T23:20:33.080980+0000 front 2024-03-18T23:20:33.081247+0000 (oldest deadline 2024-03-18T23:20:56.580916+0000) 2024-03-18T23:21:07.879 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 18 23:21:07 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-9[98163]: 2024-03-18T23:21:07.539+0000 7ff64b26e700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.175:6812 osd.5 since back 2024-03-18T23:20:39.382796+0000 front 2024-03-18T23:20:39.382823+0000 (oldest deadline 2024-03-18T23:21:05.282665+0000) 2024-03-18T23:21:07.960 INFO:journalctl@ceph.osd.8.smithi175.stdout:Mar 18 23:21:07 smithi175 podman[104954]: 2024-03-18 23:21:07.906761116 +0000 UTC m=+5.501605163 container died 61637f12f1c58b13b979bb7b2ab2948b6e7334bf5f2ca0d6b2677a4db5e23cf7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-8, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.build-date=20240307) 2024-03-18T23:21:07.961 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:21:07 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:21:07.667+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:19:57.947723+0000 front 2024-03-18T23:19:57.947785+0000 (oldest deadline 2024-03-18T23:20:28.548565+0000) 2024-03-18T23:21:07.961 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:21:07 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:21:07.667+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-03-18T23:20:08.549207+0000 front 2024-03-18T23:20:08.549248+0000 (oldest deadline 2024-03-18T23:20:33.849233+0000) 2024-03-18T23:21:07.961 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:21:07 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:21:07.667+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-03-18T23:20:16.150579+0000 front 2024-03-18T23:20:16.150606+0000 (oldest deadline 2024-03-18T23:20:40.751401+0000) 2024-03-18T23:21:07.961 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:21:07 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:21:07.667+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-03-18T23:20:33.653867+0000 front 2024-03-18T23:20:33.653710+0000 (oldest deadline 2024-03-18T23:20:59.553560+0000) 2024-03-18T23:21:07.961 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:21:07 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:21:07.667+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.175:6812 osd.5 since back 2024-03-18T23:20:39.553929+0000 front 2024-03-18T23:20:39.553999+0000 (oldest deadline 2024-03-18T23:21:04.254102+0000) 2024-03-18T23:21:08.314 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:21:08 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:21:08.061+0000 7f83cd240700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:19:57.904964+0000 front 2024-03-18T23:19:57.904988+0000 (oldest deadline 2024-03-18T23:20:23.805169+0000) 2024-03-18T23:21:08.314 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:21:08 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:21:08.061+0000 7f83cd240700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-03-18T23:20:10.806661+0000 front 2024-03-18T23:20:10.806621+0000 (oldest deadline 2024-03-18T23:20:31.906747+0000) 2024-03-18T23:21:08.314 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:21:08 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:21:08.061+0000 7f83cd240700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-03-18T23:20:16.508876+0000 front 2024-03-18T23:20:16.508656+0000 (oldest deadline 2024-03-18T23:20:41.808738+0000) 2024-03-18T23:21:08.314 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:21:08 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:21:08.061+0000 7f83cd240700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-18T23:20:24.710033+0000 front 2024-03-18T23:20:24.709803+0000 (oldest deadline 2024-03-18T23:20:50.510257+0000) 2024-03-18T23:21:08.315 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:21:08 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:21:08.061+0000 7f83cd240700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-03-18T23:20:35.111771+0000 front 2024-03-18T23:20:35.111796+0000 (oldest deadline 2024-03-18T23:21:01.011950+0000) 2024-03-18T23:21:08.315 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:21:08 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:21:08.061+0000 7f83cd240700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.175:6812 osd.5 since back 2024-03-18T23:20:41.012537+0000 front 2024-03-18T23:20:41.012520+0000 (oldest deadline 2024-03-18T23:21:05.712608+0000) 2024-03-18T23:21:08.460 INFO:journalctl@ceph.osd.8.smithi175.stdout:Mar 18 23:21:08 smithi175 podman[104954]: 2024-03-18 23:21:08.280940832 +0000 UTC m=+5.875784870 container cleanup 61637f12f1c58b13b979bb7b2ab2948b6e7334bf5f2ca0d6b2677a4db5e23cf7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-8, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, ceph=True, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-03-18T23:21:08.460 INFO:journalctl@ceph.osd.8.smithi175.stdout:Mar 18 23:21:08 smithi175 bash[104954]: ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-8 2024-03-18T23:21:08.879 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 18 23:21:08 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-9[98163]: 2024-03-18T23:21:08.538+0000 7ff64b26e700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:20:01.074760+0000 front 2024-03-18T23:20:01.074900+0000 (oldest deadline 2024-03-18T23:20:26.974968+0000) 2024-03-18T23:21:08.879 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 18 23:21:08 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-9[98163]: 2024-03-18T23:21:08.538+0000 7ff64b26e700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-03-18T23:20:10.976967+0000 front 2024-03-18T23:20:10.976857+0000 (oldest deadline 2024-03-18T23:20:36.876783+0000) 2024-03-18T23:21:08.879 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 18 23:21:08 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-9[98163]: 2024-03-18T23:21:08.538+0000 7ff64b26e700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-03-18T23:20:16.877280+0000 front 2024-03-18T23:20:16.877254+0000 (oldest deadline 2024-03-18T23:20:41.577384+0000) 2024-03-18T23:21:08.879 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 18 23:21:08 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-9[98163]: 2024-03-18T23:21:08.538+0000 7ff64b26e700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-18T23:20:23.278575+0000 front 2024-03-18T23:20:23.278614+0000 (oldest deadline 2024-03-18T23:20:49.178702+0000) 2024-03-18T23:21:08.879 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 18 23:21:08 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-9[98163]: 2024-03-18T23:21:08.538+0000 7ff64b26e700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-03-18T23:20:33.080980+0000 front 2024-03-18T23:20:33.081247+0000 (oldest deadline 2024-03-18T23:20:56.580916+0000) 2024-03-18T23:21:08.879 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 18 23:21:08 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-9[98163]: 2024-03-18T23:21:08.538+0000 7ff64b26e700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.175:6812 osd.5 since back 2024-03-18T23:20:39.382796+0000 front 2024-03-18T23:20:39.382823+0000 (oldest deadline 2024-03-18T23:21:05.282665+0000) 2024-03-18T23:21:08.881 INFO:journalctl@ceph.osd.8.smithi175.stdout:Mar 18 23:21:08 smithi175 podman[104971]: 2024-03-18 23:21:08.548752515 +0000 UTC m=+0.644347266 container remove 61637f12f1c58b13b979bb7b2ab2948b6e7334bf5f2ca0d6b2677a4db5e23cf7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-8, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240318.181317-b6ae249, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-18T23:21:08.881 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:21:08 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:21:08.687+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:19:57.947723+0000 front 2024-03-18T23:19:57.947785+0000 (oldest deadline 2024-03-18T23:20:28.548565+0000) 2024-03-18T23:21:08.881 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:21:08 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:21:08.687+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-03-18T23:20:08.549207+0000 front 2024-03-18T23:20:08.549248+0000 (oldest deadline 2024-03-18T23:20:33.849233+0000) 2024-03-18T23:21:08.881 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:21:08 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:21:08.687+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-03-18T23:20:16.150579+0000 front 2024-03-18T23:20:16.150606+0000 (oldest deadline 2024-03-18T23:20:40.751401+0000) 2024-03-18T23:21:08.881 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:21:08 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:21:08.687+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-03-18T23:20:33.653867+0000 front 2024-03-18T23:20:33.653710+0000 (oldest deadline 2024-03-18T23:20:59.553560+0000) 2024-03-18T23:21:08.882 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:21:08 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:21:08.687+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.175:6812 osd.5 since back 2024-03-18T23:20:39.553929+0000 front 2024-03-18T23:20:39.553999+0000 (oldest deadline 2024-03-18T23:21:04.254102+0000) 2024-03-18T23:21:09.184 INFO:journalctl@ceph.osd.8.smithi175.stdout:Mar 18 23:21:08 smithi175 podman[105060]: 2024-03-18 23:21:08.882885744 +0000 UTC m=+0.105938471 container create 8bc990b7636f3278b750ffaf6355a5099a192ed155bdea19db3525581a35f997 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-8-deactivate, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-18T23:21:09.184 INFO:journalctl@ceph.osd.8.smithi175.stdout:Mar 18 23:21:08 smithi175 podman[105060]: 2024-03-18 23:21:08.795249908 +0000 UTC m=+0.018302646 image pull 2312c8e42d0069a2dbb2364bddb00264db4b800e7f18a7e7e4eb8f3ecab2ff1c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639 2024-03-18T23:21:09.184 INFO:journalctl@ceph.osd.8.smithi175.stdout:Mar 18 23:21:09 smithi175 podman[105060]: 2024-03-18 23:21:09.118808444 +0000 UTC m=+0.341861173 container init 8bc990b7636f3278b750ffaf6355a5099a192ed155bdea19db3525581a35f997 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-8-deactivate, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.schema-version=1.0) 2024-03-18T23:21:09.184 INFO:journalctl@ceph.osd.8.smithi175.stdout:Mar 18 23:21:09 smithi175 podman[105060]: 2024-03-18 23:21:09.121869856 +0000 UTC m=+0.344922583 container start 8bc990b7636f3278b750ffaf6355a5099a192ed155bdea19db3525581a35f997 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-8-deactivate, org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-18T23:21:09.184 INFO:journalctl@ceph.osd.8.smithi175.stdout:Mar 18 23:21:09 smithi175 podman[105060]: 2024-03-18 23:21:09.185455547 +0000 UTC m=+0.408508293 container attach 8bc990b7636f3278b750ffaf6355a5099a192ed155bdea19db3525581a35f997 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-8-deactivate, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307) 2024-03-18T23:21:09.314 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:21:09 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:21:09.021+0000 7f83cd240700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:19:57.904964+0000 front 2024-03-18T23:19:57.904988+0000 (oldest deadline 2024-03-18T23:20:23.805169+0000) 2024-03-18T23:21:09.314 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:21:09 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:21:09.021+0000 7f83cd240700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-03-18T23:20:10.806661+0000 front 2024-03-18T23:20:10.806621+0000 (oldest deadline 2024-03-18T23:20:31.906747+0000) 2024-03-18T23:21:09.314 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:21:09 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:21:09.021+0000 7f83cd240700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-03-18T23:20:16.508876+0000 front 2024-03-18T23:20:16.508656+0000 (oldest deadline 2024-03-18T23:20:41.808738+0000) 2024-03-18T23:21:09.314 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:21:09 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:21:09.021+0000 7f83cd240700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-18T23:20:24.710033+0000 front 2024-03-18T23:20:24.709803+0000 (oldest deadline 2024-03-18T23:20:50.510257+0000) 2024-03-18T23:21:09.314 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:21:09 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:21:09.021+0000 7f83cd240700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-03-18T23:20:35.111771+0000 front 2024-03-18T23:20:35.111796+0000 (oldest deadline 2024-03-18T23:21:01.011950+0000) 2024-03-18T23:21:09.314 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:21:09 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:21:09.021+0000 7f83cd240700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.175:6812 osd.5 since back 2024-03-18T23:20:41.012537+0000 front 2024-03-18T23:20:41.012520+0000 (oldest deadline 2024-03-18T23:21:05.712608+0000) 2024-03-18T23:21:09.460 INFO:journalctl@ceph.osd.8.smithi175.stdout:Mar 18 23:21:09 smithi175 podman[105060]: 2024-03-18 23:21:09.408751347 +0000 UTC m=+0.631804085 container died 8bc990b7636f3278b750ffaf6355a5099a192ed155bdea19db3525581a35f997 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-8-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-03-18T23:21:09.879 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 18 23:21:09 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-9[98163]: 2024-03-18T23:21:09.547+0000 7ff64b26e700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:20:01.074760+0000 front 2024-03-18T23:20:01.074900+0000 (oldest deadline 2024-03-18T23:20:26.974968+0000) 2024-03-18T23:21:09.879 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 18 23:21:09 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-9[98163]: 2024-03-18T23:21:09.547+0000 7ff64b26e700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-03-18T23:20:10.976967+0000 front 2024-03-18T23:20:10.976857+0000 (oldest deadline 2024-03-18T23:20:36.876783+0000) 2024-03-18T23:21:09.879 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 18 23:21:09 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-9[98163]: 2024-03-18T23:21:09.547+0000 7ff64b26e700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-03-18T23:20:16.877280+0000 front 2024-03-18T23:20:16.877254+0000 (oldest deadline 2024-03-18T23:20:41.577384+0000) 2024-03-18T23:21:09.879 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 18 23:21:09 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-9[98163]: 2024-03-18T23:21:09.547+0000 7ff64b26e700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-18T23:20:23.278575+0000 front 2024-03-18T23:20:23.278614+0000 (oldest deadline 2024-03-18T23:20:49.178702+0000) 2024-03-18T23:21:09.879 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 18 23:21:09 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-9[98163]: 2024-03-18T23:21:09.547+0000 7ff64b26e700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-03-18T23:20:33.080980+0000 front 2024-03-18T23:20:33.081247+0000 (oldest deadline 2024-03-18T23:20:56.580916+0000) 2024-03-18T23:21:09.879 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 18 23:21:09 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-9[98163]: 2024-03-18T23:21:09.547+0000 7ff64b26e700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.175:6812 osd.5 since back 2024-03-18T23:20:39.382796+0000 front 2024-03-18T23:20:39.382823+0000 (oldest deadline 2024-03-18T23:21:05.282665+0000) 2024-03-18T23:21:09.962 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:21:09 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:21:09.686+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:19:57.947723+0000 front 2024-03-18T23:19:57.947785+0000 (oldest deadline 2024-03-18T23:20:28.548565+0000) 2024-03-18T23:21:09.962 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:21:09 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:21:09.686+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-03-18T23:20:08.549207+0000 front 2024-03-18T23:20:08.549248+0000 (oldest deadline 2024-03-18T23:20:33.849233+0000) 2024-03-18T23:21:09.962 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:21:09 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:21:09.686+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-03-18T23:20:16.150579+0000 front 2024-03-18T23:20:16.150606+0000 (oldest deadline 2024-03-18T23:20:40.751401+0000) 2024-03-18T23:21:09.962 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:21:09 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:21:09.686+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-03-18T23:20:33.653867+0000 front 2024-03-18T23:20:33.653710+0000 (oldest deadline 2024-03-18T23:20:59.553560+0000) 2024-03-18T23:21:09.962 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:21:09 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:21:09.686+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.175:6812 osd.5 since back 2024-03-18T23:20:39.553929+0000 front 2024-03-18T23:20:39.553999+0000 (oldest deadline 2024-03-18T23:21:04.254102+0000) 2024-03-18T23:21:09.991 DEBUG:teuthology.orchestra.run.smithi175:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c63591ea-e57a-11ee-95c9-87774f69a715@osd.8.service' 2024-03-18T23:21:10.047 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-18T23:21:10.047 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-03-18T23:21:10.048 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-03-18T23:21:10.048 DEBUG:teuthology.orchestra.run.smithi049:> sudo systemctl stop ceph-c63591ea-e57a-11ee-95c9-87774f69a715@osd.9 2024-03-18T23:21:10.314 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:21:09 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:21:09.978+0000 7f83cd240700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:19:57.904964+0000 front 2024-03-18T23:19:57.904988+0000 (oldest deadline 2024-03-18T23:20:23.805169+0000) 2024-03-18T23:21:10.314 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:21:09 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:21:09.978+0000 7f83cd240700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-03-18T23:20:10.806661+0000 front 2024-03-18T23:20:10.806621+0000 (oldest deadline 2024-03-18T23:20:31.906747+0000) 2024-03-18T23:21:10.314 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:21:09 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:21:09.978+0000 7f83cd240700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-03-18T23:20:16.508876+0000 front 2024-03-18T23:20:16.508656+0000 (oldest deadline 2024-03-18T23:20:41.808738+0000) 2024-03-18T23:21:10.314 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:21:09 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:21:09.978+0000 7f83cd240700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-18T23:20:24.710033+0000 front 2024-03-18T23:20:24.709803+0000 (oldest deadline 2024-03-18T23:20:50.510257+0000) 2024-03-18T23:21:10.314 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:21:09 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:21:09.978+0000 7f83cd240700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-03-18T23:20:35.111771+0000 front 2024-03-18T23:20:35.111796+0000 (oldest deadline 2024-03-18T23:21:01.011950+0000) 2024-03-18T23:21:10.314 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:21:09 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:21:09.978+0000 7f83cd240700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.175:6812 osd.5 since back 2024-03-18T23:20:41.012537+0000 front 2024-03-18T23:20:41.012520+0000 (oldest deadline 2024-03-18T23:21:05.712608+0000) 2024-03-18T23:21:10.379 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 18 23:21:10 smithi049 systemd[1]: Stopping Ceph osd.9 for c63591ea-e57a-11ee-95c9-87774f69a715... 2024-03-18T23:21:10.879 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 18 23:21:10 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-9[98163]: 2024-03-18T23:21:10.381+0000 7ff64ec7a700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-18T23:21:10.879 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 18 23:21:10 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-9[98163]: 2024-03-18T23:21:10.381+0000 7ff64ec7a700 -1 osd.9 104 *** Got signal Terminated *** 2024-03-18T23:21:10.879 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 18 23:21:10 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-9[98163]: 2024-03-18T23:21:10.381+0000 7ff64ec7a700 -1 osd.9 104 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-18T23:21:10.879 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 18 23:21:10 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-9[98163]: 2024-03-18T23:21:10.592+0000 7ff64b26e700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:20:01.074760+0000 front 2024-03-18T23:20:01.074900+0000 (oldest deadline 2024-03-18T23:20:26.974968+0000) 2024-03-18T23:21:10.879 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 18 23:21:10 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-9[98163]: 2024-03-18T23:21:10.592+0000 7ff64b26e700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-03-18T23:20:10.976967+0000 front 2024-03-18T23:20:10.976857+0000 (oldest deadline 2024-03-18T23:20:36.876783+0000) 2024-03-18T23:21:10.880 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 18 23:21:10 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-9[98163]: 2024-03-18T23:21:10.592+0000 7ff64b26e700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-03-18T23:20:16.877280+0000 front 2024-03-18T23:20:16.877254+0000 (oldest deadline 2024-03-18T23:20:41.577384+0000) 2024-03-18T23:21:10.880 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 18 23:21:10 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-9[98163]: 2024-03-18T23:21:10.592+0000 7ff64b26e700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-18T23:20:23.278575+0000 front 2024-03-18T23:20:23.278614+0000 (oldest deadline 2024-03-18T23:20:49.178702+0000) 2024-03-18T23:21:10.880 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 18 23:21:10 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-9[98163]: 2024-03-18T23:21:10.592+0000 7ff64b26e700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-03-18T23:20:33.080980+0000 front 2024-03-18T23:20:33.081247+0000 (oldest deadline 2024-03-18T23:20:56.580916+0000) 2024-03-18T23:21:10.880 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 18 23:21:10 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-9[98163]: 2024-03-18T23:21:10.592+0000 7ff64b26e700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.175:6812 osd.5 since back 2024-03-18T23:20:39.382796+0000 front 2024-03-18T23:20:39.382823+0000 (oldest deadline 2024-03-18T23:21:05.282665+0000) 2024-03-18T23:21:11.204 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:21:10 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:21:10.731+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:19:57.947723+0000 front 2024-03-18T23:19:57.947785+0000 (oldest deadline 2024-03-18T23:20:28.548565+0000) 2024-03-18T23:21:11.204 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:21:10 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:21:10.731+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-03-18T23:20:08.549207+0000 front 2024-03-18T23:20:08.549248+0000 (oldest deadline 2024-03-18T23:20:33.849233+0000) 2024-03-18T23:21:11.204 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:21:10 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:21:10.731+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-03-18T23:20:16.150579+0000 front 2024-03-18T23:20:16.150606+0000 (oldest deadline 2024-03-18T23:20:40.751401+0000) 2024-03-18T23:21:11.205 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:21:10 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:21:10.731+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-03-18T23:20:33.653867+0000 front 2024-03-18T23:20:33.653710+0000 (oldest deadline 2024-03-18T23:20:59.553560+0000) 2024-03-18T23:21:11.205 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:21:10 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:21:10.731+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.175:6812 osd.5 since back 2024-03-18T23:20:39.553929+0000 front 2024-03-18T23:20:39.553999+0000 (oldest deadline 2024-03-18T23:21:04.254102+0000) 2024-03-18T23:21:11.314 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:21:10 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:21:10.961+0000 7f83cd240700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:19:57.904964+0000 front 2024-03-18T23:19:57.904988+0000 (oldest deadline 2024-03-18T23:20:23.805169+0000) 2024-03-18T23:21:11.314 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:21:10 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:21:10.961+0000 7f83cd240700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-03-18T23:20:10.806661+0000 front 2024-03-18T23:20:10.806621+0000 (oldest deadline 2024-03-18T23:20:31.906747+0000) 2024-03-18T23:21:11.314 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:21:10 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:21:10.961+0000 7f83cd240700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-03-18T23:20:16.508876+0000 front 2024-03-18T23:20:16.508656+0000 (oldest deadline 2024-03-18T23:20:41.808738+0000) 2024-03-18T23:21:11.314 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:21:10 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:21:10.961+0000 7f83cd240700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-18T23:20:24.710033+0000 front 2024-03-18T23:20:24.709803+0000 (oldest deadline 2024-03-18T23:20:50.510257+0000) 2024-03-18T23:21:11.315 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:21:10 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:21:10.961+0000 7f83cd240700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-03-18T23:20:35.111771+0000 front 2024-03-18T23:20:35.111796+0000 (oldest deadline 2024-03-18T23:21:01.011950+0000) 2024-03-18T23:21:11.315 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:21:10 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:21:10.961+0000 7f83cd240700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.175:6812 osd.5 since back 2024-03-18T23:20:41.012537+0000 front 2024-03-18T23:20:41.012520+0000 (oldest deadline 2024-03-18T23:21:05.712608+0000) 2024-03-18T23:21:11.879 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 18 23:21:11 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-9[98163]: 2024-03-18T23:21:11.582+0000 7ff64b26e700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:20:01.074760+0000 front 2024-03-18T23:20:01.074900+0000 (oldest deadline 2024-03-18T23:20:26.974968+0000) 2024-03-18T23:21:11.879 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 18 23:21:11 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-9[98163]: 2024-03-18T23:21:11.582+0000 7ff64b26e700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-03-18T23:20:10.976967+0000 front 2024-03-18T23:20:10.976857+0000 (oldest deadline 2024-03-18T23:20:36.876783+0000) 2024-03-18T23:21:11.879 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 18 23:21:11 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-9[98163]: 2024-03-18T23:21:11.582+0000 7ff64b26e700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-03-18T23:20:16.877280+0000 front 2024-03-18T23:20:16.877254+0000 (oldest deadline 2024-03-18T23:20:41.577384+0000) 2024-03-18T23:21:11.879 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 18 23:21:11 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-9[98163]: 2024-03-18T23:21:11.582+0000 7ff64b26e700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-18T23:20:23.278575+0000 front 2024-03-18T23:20:23.278614+0000 (oldest deadline 2024-03-18T23:20:49.178702+0000) 2024-03-18T23:21:11.880 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 18 23:21:11 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-9[98163]: 2024-03-18T23:21:11.582+0000 7ff64b26e700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-03-18T23:20:33.080980+0000 front 2024-03-18T23:20:33.081247+0000 (oldest deadline 2024-03-18T23:20:56.580916+0000) 2024-03-18T23:21:11.880 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 18 23:21:11 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-9[98163]: 2024-03-18T23:21:11.582+0000 7ff64b26e700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.175:6812 osd.5 since back 2024-03-18T23:20:39.382796+0000 front 2024-03-18T23:20:39.382823+0000 (oldest deadline 2024-03-18T23:21:05.282665+0000) 2024-03-18T23:21:11.960 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:21:11 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:21:11.705+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:19:57.947723+0000 front 2024-03-18T23:19:57.947785+0000 (oldest deadline 2024-03-18T23:20:28.548565+0000) 2024-03-18T23:21:11.960 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:21:11 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:21:11.705+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-03-18T23:20:08.549207+0000 front 2024-03-18T23:20:08.549248+0000 (oldest deadline 2024-03-18T23:20:33.849233+0000) 2024-03-18T23:21:11.960 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:21:11 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:21:11.705+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-03-18T23:20:16.150579+0000 front 2024-03-18T23:20:16.150606+0000 (oldest deadline 2024-03-18T23:20:40.751401+0000) 2024-03-18T23:21:11.960 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:21:11 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:21:11.705+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-03-18T23:20:33.653867+0000 front 2024-03-18T23:20:33.653710+0000 (oldest deadline 2024-03-18T23:20:59.553560+0000) 2024-03-18T23:21:11.961 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:21:11 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:21:11.705+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.175:6812 osd.5 since back 2024-03-18T23:20:39.553929+0000 front 2024-03-18T23:20:39.553999+0000 (oldest deadline 2024-03-18T23:21:04.254102+0000) 2024-03-18T23:21:12.250 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:21:11 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:21:11.996+0000 7f83cd240700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:19:57.904964+0000 front 2024-03-18T23:19:57.904988+0000 (oldest deadline 2024-03-18T23:20:23.805169+0000) 2024-03-18T23:21:12.250 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:21:11 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:21:11.996+0000 7f83cd240700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-03-18T23:20:10.806661+0000 front 2024-03-18T23:20:10.806621+0000 (oldest deadline 2024-03-18T23:20:31.906747+0000) 2024-03-18T23:21:12.250 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:21:11 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:21:11.996+0000 7f83cd240700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-03-18T23:20:16.508876+0000 front 2024-03-18T23:20:16.508656+0000 (oldest deadline 2024-03-18T23:20:41.808738+0000) 2024-03-18T23:21:12.251 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:21:11 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:21:11.996+0000 7f83cd240700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-18T23:20:24.710033+0000 front 2024-03-18T23:20:24.709803+0000 (oldest deadline 2024-03-18T23:20:50.510257+0000) 2024-03-18T23:21:12.251 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:21:11 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:21:11.996+0000 7f83cd240700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-03-18T23:20:35.111771+0000 front 2024-03-18T23:20:35.111796+0000 (oldest deadline 2024-03-18T23:21:01.011950+0000) 2024-03-18T23:21:12.251 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:21:11 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:21:11.996+0000 7f83cd240700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.175:6812 osd.5 since back 2024-03-18T23:20:41.012537+0000 front 2024-03-18T23:20:41.012520+0000 (oldest deadline 2024-03-18T23:21:05.712608+0000) 2024-03-18T23:21:12.879 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 18 23:21:12 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-9[98163]: 2024-03-18T23:21:12.558+0000 7ff64b26e700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:20:01.074760+0000 front 2024-03-18T23:20:01.074900+0000 (oldest deadline 2024-03-18T23:20:26.974968+0000) 2024-03-18T23:21:12.879 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 18 23:21:12 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-9[98163]: 2024-03-18T23:21:12.558+0000 7ff64b26e700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-03-18T23:20:10.976967+0000 front 2024-03-18T23:20:10.976857+0000 (oldest deadline 2024-03-18T23:20:36.876783+0000) 2024-03-18T23:21:12.879 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 18 23:21:12 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-9[98163]: 2024-03-18T23:21:12.558+0000 7ff64b26e700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-03-18T23:20:16.877280+0000 front 2024-03-18T23:20:16.877254+0000 (oldest deadline 2024-03-18T23:20:41.577384+0000) 2024-03-18T23:21:12.879 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 18 23:21:12 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-9[98163]: 2024-03-18T23:21:12.558+0000 7ff64b26e700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-18T23:20:23.278575+0000 front 2024-03-18T23:20:23.278614+0000 (oldest deadline 2024-03-18T23:20:49.178702+0000) 2024-03-18T23:21:12.879 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 18 23:21:12 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-9[98163]: 2024-03-18T23:21:12.558+0000 7ff64b26e700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-03-18T23:20:33.080980+0000 front 2024-03-18T23:20:33.081247+0000 (oldest deadline 2024-03-18T23:20:56.580916+0000) 2024-03-18T23:21:12.879 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 18 23:21:12 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-9[98163]: 2024-03-18T23:21:12.558+0000 7ff64b26e700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.175:6812 osd.5 since back 2024-03-18T23:20:39.382796+0000 front 2024-03-18T23:20:39.382823+0000 (oldest deadline 2024-03-18T23:21:05.282665+0000) 2024-03-18T23:21:12.960 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:21:12 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:21:12.694+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:19:57.947723+0000 front 2024-03-18T23:19:57.947785+0000 (oldest deadline 2024-03-18T23:20:28.548565+0000) 2024-03-18T23:21:12.960 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:21:12 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:21:12.694+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-03-18T23:20:08.549207+0000 front 2024-03-18T23:20:08.549248+0000 (oldest deadline 2024-03-18T23:20:33.849233+0000) 2024-03-18T23:21:12.960 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:21:12 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:21:12.694+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-03-18T23:20:16.150579+0000 front 2024-03-18T23:20:16.150606+0000 (oldest deadline 2024-03-18T23:20:40.751401+0000) 2024-03-18T23:21:12.960 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:21:12 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:21:12.694+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-03-18T23:20:33.653867+0000 front 2024-03-18T23:20:33.653710+0000 (oldest deadline 2024-03-18T23:20:59.553560+0000) 2024-03-18T23:21:12.961 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:21:12 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:21:12.694+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.175:6812 osd.5 since back 2024-03-18T23:20:39.553929+0000 front 2024-03-18T23:20:39.553999+0000 (oldest deadline 2024-03-18T23:21:04.254102+0000) 2024-03-18T23:21:13.315 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:21:13 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:21:12.999+0000 7f83cd240700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:19:57.904964+0000 front 2024-03-18T23:19:57.904988+0000 (oldest deadline 2024-03-18T23:20:23.805169+0000) 2024-03-18T23:21:13.315 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:21:13 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:21:12.999+0000 7f83cd240700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-03-18T23:20:10.806661+0000 front 2024-03-18T23:20:10.806621+0000 (oldest deadline 2024-03-18T23:20:31.906747+0000) 2024-03-18T23:21:13.316 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:21:13 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:21:12.999+0000 7f83cd240700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-03-18T23:20:16.508876+0000 front 2024-03-18T23:20:16.508656+0000 (oldest deadline 2024-03-18T23:20:41.808738+0000) 2024-03-18T23:21:13.316 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:21:13 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:21:12.999+0000 7f83cd240700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-18T23:20:24.710033+0000 front 2024-03-18T23:20:24.709803+0000 (oldest deadline 2024-03-18T23:20:50.510257+0000) 2024-03-18T23:21:13.316 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:21:13 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:21:12.999+0000 7f83cd240700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-03-18T23:20:35.111771+0000 front 2024-03-18T23:20:35.111796+0000 (oldest deadline 2024-03-18T23:21:01.011950+0000) 2024-03-18T23:21:13.316 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:21:13 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:21:12.999+0000 7f83cd240700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.175:6812 osd.5 since back 2024-03-18T23:20:41.012537+0000 front 2024-03-18T23:20:41.012520+0000 (oldest deadline 2024-03-18T23:21:05.712608+0000) 2024-03-18T23:21:13.878 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 18 23:21:13 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-9[98163]: 2024-03-18T23:21:13.509+0000 7ff64b26e700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:20:01.074760+0000 front 2024-03-18T23:20:01.074900+0000 (oldest deadline 2024-03-18T23:20:26.974968+0000) 2024-03-18T23:21:13.879 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 18 23:21:13 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-9[98163]: 2024-03-18T23:21:13.509+0000 7ff64b26e700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-03-18T23:20:10.976967+0000 front 2024-03-18T23:20:10.976857+0000 (oldest deadline 2024-03-18T23:20:36.876783+0000) 2024-03-18T23:21:13.879 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 18 23:21:13 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-9[98163]: 2024-03-18T23:21:13.509+0000 7ff64b26e700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-03-18T23:20:16.877280+0000 front 2024-03-18T23:20:16.877254+0000 (oldest deadline 2024-03-18T23:20:41.577384+0000) 2024-03-18T23:21:13.879 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 18 23:21:13 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-9[98163]: 2024-03-18T23:21:13.509+0000 7ff64b26e700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-18T23:20:23.278575+0000 front 2024-03-18T23:20:23.278614+0000 (oldest deadline 2024-03-18T23:20:49.178702+0000) 2024-03-18T23:21:13.879 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 18 23:21:13 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-9[98163]: 2024-03-18T23:21:13.509+0000 7ff64b26e700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-03-18T23:20:33.080980+0000 front 2024-03-18T23:20:33.081247+0000 (oldest deadline 2024-03-18T23:20:56.580916+0000) 2024-03-18T23:21:13.879 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 18 23:21:13 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-9[98163]: 2024-03-18T23:21:13.509+0000 7ff64b26e700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.175:6812 osd.5 since back 2024-03-18T23:20:39.382796+0000 front 2024-03-18T23:20:39.382823+0000 (oldest deadline 2024-03-18T23:21:05.282665+0000) 2024-03-18T23:21:14.210 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:21:13 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:21:13.731+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:19:57.947723+0000 front 2024-03-18T23:19:57.947785+0000 (oldest deadline 2024-03-18T23:20:28.548565+0000) 2024-03-18T23:21:14.210 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:21:13 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:21:13.731+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-03-18T23:20:08.549207+0000 front 2024-03-18T23:20:08.549248+0000 (oldest deadline 2024-03-18T23:20:33.849233+0000) 2024-03-18T23:21:14.210 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:21:13 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:21:13.731+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-03-18T23:20:16.150579+0000 front 2024-03-18T23:20:16.150606+0000 (oldest deadline 2024-03-18T23:20:40.751401+0000) 2024-03-18T23:21:14.211 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:21:13 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:21:13.731+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-03-18T23:20:33.653867+0000 front 2024-03-18T23:20:33.653710+0000 (oldest deadline 2024-03-18T23:20:59.553560+0000) 2024-03-18T23:21:14.211 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:21:13 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:21:13.731+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.175:6812 osd.5 since back 2024-03-18T23:20:39.553929+0000 front 2024-03-18T23:20:39.553999+0000 (oldest deadline 2024-03-18T23:21:04.254102+0000) 2024-03-18T23:21:14.211 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:21:13 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:21:13.731+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.49:6822 osd.6 since back 2024-03-18T23:20:48.255879+0000 front 2024-03-18T23:20:48.255853+0000 (oldest deadline 2024-03-18T23:21:13.555739+0000) 2024-03-18T23:21:14.314 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:21:13 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:21:13.997+0000 7f83cd240700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:19:57.904964+0000 front 2024-03-18T23:19:57.904988+0000 (oldest deadline 2024-03-18T23:20:23.805169+0000) 2024-03-18T23:21:14.314 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:21:13 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:21:13.997+0000 7f83cd240700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-03-18T23:20:10.806661+0000 front 2024-03-18T23:20:10.806621+0000 (oldest deadline 2024-03-18T23:20:31.906747+0000) 2024-03-18T23:21:14.314 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:21:13 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:21:13.997+0000 7f83cd240700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-03-18T23:20:16.508876+0000 front 2024-03-18T23:20:16.508656+0000 (oldest deadline 2024-03-18T23:20:41.808738+0000) 2024-03-18T23:21:14.314 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:21:13 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:21:13.997+0000 7f83cd240700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-18T23:20:24.710033+0000 front 2024-03-18T23:20:24.709803+0000 (oldest deadline 2024-03-18T23:20:50.510257+0000) 2024-03-18T23:21:14.315 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:21:13 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:21:13.997+0000 7f83cd240700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-03-18T23:20:35.111771+0000 front 2024-03-18T23:20:35.111796+0000 (oldest deadline 2024-03-18T23:21:01.011950+0000) 2024-03-18T23:21:14.315 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:21:13 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:21:13.997+0000 7f83cd240700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.175:6812 osd.5 since back 2024-03-18T23:20:41.012537+0000 front 2024-03-18T23:20:41.012520+0000 (oldest deadline 2024-03-18T23:21:05.712608+0000) 2024-03-18T23:21:14.315 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:21:13 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:21:13.997+0000 7f83cd240700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.49:6822 osd.6 since back 2024-03-18T23:20:51.013879+0000 front 2024-03-18T23:20:51.013823+0000 (oldest deadline 2024-03-18T23:21:13.913724+0000) 2024-03-18T23:21:14.879 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 18 23:21:14 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-9[98163]: 2024-03-18T23:21:14.507+0000 7ff64b26e700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:20:01.074760+0000 front 2024-03-18T23:20:01.074900+0000 (oldest deadline 2024-03-18T23:20:26.974968+0000) 2024-03-18T23:21:14.879 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 18 23:21:14 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-9[98163]: 2024-03-18T23:21:14.507+0000 7ff64b26e700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-03-18T23:20:10.976967+0000 front 2024-03-18T23:20:10.976857+0000 (oldest deadline 2024-03-18T23:20:36.876783+0000) 2024-03-18T23:21:14.879 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 18 23:21:14 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-9[98163]: 2024-03-18T23:21:14.507+0000 7ff64b26e700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-03-18T23:20:16.877280+0000 front 2024-03-18T23:20:16.877254+0000 (oldest deadline 2024-03-18T23:20:41.577384+0000) 2024-03-18T23:21:14.879 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 18 23:21:14 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-9[98163]: 2024-03-18T23:21:14.507+0000 7ff64b26e700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-18T23:20:23.278575+0000 front 2024-03-18T23:20:23.278614+0000 (oldest deadline 2024-03-18T23:20:49.178702+0000) 2024-03-18T23:21:14.879 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 18 23:21:14 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-9[98163]: 2024-03-18T23:21:14.507+0000 7ff64b26e700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-03-18T23:20:33.080980+0000 front 2024-03-18T23:20:33.081247+0000 (oldest deadline 2024-03-18T23:20:56.580916+0000) 2024-03-18T23:21:14.879 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 18 23:21:14 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-9[98163]: 2024-03-18T23:21:14.507+0000 7ff64b26e700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.175:6812 osd.5 since back 2024-03-18T23:20:39.382796+0000 front 2024-03-18T23:20:39.382823+0000 (oldest deadline 2024-03-18T23:21:05.282665+0000) 2024-03-18T23:21:14.960 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:21:14 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:21:14.688+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:19:57.947723+0000 front 2024-03-18T23:19:57.947785+0000 (oldest deadline 2024-03-18T23:20:28.548565+0000) 2024-03-18T23:21:14.960 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:21:14 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:21:14.688+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-03-18T23:20:08.549207+0000 front 2024-03-18T23:20:08.549248+0000 (oldest deadline 2024-03-18T23:20:33.849233+0000) 2024-03-18T23:21:14.960 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:21:14 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:21:14.688+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-03-18T23:20:16.150579+0000 front 2024-03-18T23:20:16.150606+0000 (oldest deadline 2024-03-18T23:20:40.751401+0000) 2024-03-18T23:21:14.961 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:21:14 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:21:14.688+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-03-18T23:20:33.653867+0000 front 2024-03-18T23:20:33.653710+0000 (oldest deadline 2024-03-18T23:20:59.553560+0000) 2024-03-18T23:21:14.961 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:21:14 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:21:14.688+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.175:6812 osd.5 since back 2024-03-18T23:20:39.553929+0000 front 2024-03-18T23:20:39.553999+0000 (oldest deadline 2024-03-18T23:21:04.254102+0000) 2024-03-18T23:21:14.961 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:21:14 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:21:14.688+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.49:6822 osd.6 since back 2024-03-18T23:20:48.255879+0000 front 2024-03-18T23:20:48.255853+0000 (oldest deadline 2024-03-18T23:21:13.555739+0000) 2024-03-18T23:21:15.314 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:21:14 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:21:14.989+0000 7f83cd240700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:19:57.904964+0000 front 2024-03-18T23:19:57.904988+0000 (oldest deadline 2024-03-18T23:20:23.805169+0000) 2024-03-18T23:21:15.314 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:21:14 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:21:14.989+0000 7f83cd240700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-03-18T23:20:10.806661+0000 front 2024-03-18T23:20:10.806621+0000 (oldest deadline 2024-03-18T23:20:31.906747+0000) 2024-03-18T23:21:15.314 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:21:14 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:21:14.989+0000 7f83cd240700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-03-18T23:20:16.508876+0000 front 2024-03-18T23:20:16.508656+0000 (oldest deadline 2024-03-18T23:20:41.808738+0000) 2024-03-18T23:21:15.314 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:21:14 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:21:14.989+0000 7f83cd240700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-18T23:20:24.710033+0000 front 2024-03-18T23:20:24.709803+0000 (oldest deadline 2024-03-18T23:20:50.510257+0000) 2024-03-18T23:21:15.314 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:21:14 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:21:14.989+0000 7f83cd240700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-03-18T23:20:35.111771+0000 front 2024-03-18T23:20:35.111796+0000 (oldest deadline 2024-03-18T23:21:01.011950+0000) 2024-03-18T23:21:15.315 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:21:14 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:21:14.989+0000 7f83cd240700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.175:6812 osd.5 since back 2024-03-18T23:20:41.012537+0000 front 2024-03-18T23:20:41.012520+0000 (oldest deadline 2024-03-18T23:21:05.712608+0000) 2024-03-18T23:21:15.315 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:21:14 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:21:14.989+0000 7f83cd240700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.49:6822 osd.6 since back 2024-03-18T23:20:51.013879+0000 front 2024-03-18T23:20:51.013823+0000 (oldest deadline 2024-03-18T23:21:13.913724+0000) 2024-03-18T23:21:15.960 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:21:15 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:21:15.662+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:19:57.947723+0000 front 2024-03-18T23:19:57.947785+0000 (oldest deadline 2024-03-18T23:20:28.548565+0000) 2024-03-18T23:21:15.960 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:21:15 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:21:15.662+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-03-18T23:20:08.549207+0000 front 2024-03-18T23:20:08.549248+0000 (oldest deadline 2024-03-18T23:20:33.849233+0000) 2024-03-18T23:21:15.961 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:21:15 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:21:15.662+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-03-18T23:20:16.150579+0000 front 2024-03-18T23:20:16.150606+0000 (oldest deadline 2024-03-18T23:20:40.751401+0000) 2024-03-18T23:21:15.961 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:21:15 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:21:15.662+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-03-18T23:20:33.653867+0000 front 2024-03-18T23:20:33.653710+0000 (oldest deadline 2024-03-18T23:20:59.553560+0000) 2024-03-18T23:21:15.961 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:21:15 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:21:15.662+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.175:6812 osd.5 since back 2024-03-18T23:20:39.553929+0000 front 2024-03-18T23:20:39.553999+0000 (oldest deadline 2024-03-18T23:21:04.254102+0000) 2024-03-18T23:21:15.961 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:21:15 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:21:15.662+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.49:6822 osd.6 since back 2024-03-18T23:20:48.255879+0000 front 2024-03-18T23:20:48.255853+0000 (oldest deadline 2024-03-18T23:21:13.555739+0000) 2024-03-18T23:21:15.980 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 18 23:21:15 smithi049 podman[124522]: 2024-03-18 23:21:15.724132025 +0000 UTC m=+5.470115786 container died 0320517bd4300f53173ac26036a7f5e57e231c6e5708407a1da800d6f2bc15b8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-9, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-18T23:21:16.238 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 18 23:21:16 smithi049 podman[124522]: 2024-03-18 23:21:16.055569203 +0000 UTC m=+5.801552980 container cleanup 0320517bd4300f53173ac26036a7f5e57e231c6e5708407a1da800d6f2bc15b8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-9, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, ceph=True, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-03-18T23:21:16.239 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 18 23:21:16 smithi049 bash[124522]: ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-9 2024-03-18T23:21:16.314 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:21:15 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:21:15.977+0000 7f83cd240700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:19:57.904964+0000 front 2024-03-18T23:19:57.904988+0000 (oldest deadline 2024-03-18T23:20:23.805169+0000) 2024-03-18T23:21:16.314 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:21:15 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:21:15.977+0000 7f83cd240700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-03-18T23:20:10.806661+0000 front 2024-03-18T23:20:10.806621+0000 (oldest deadline 2024-03-18T23:20:31.906747+0000) 2024-03-18T23:21:16.314 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:21:15 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:21:15.977+0000 7f83cd240700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-03-18T23:20:16.508876+0000 front 2024-03-18T23:20:16.508656+0000 (oldest deadline 2024-03-18T23:20:41.808738+0000) 2024-03-18T23:21:16.314 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:21:15 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:21:15.977+0000 7f83cd240700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-18T23:20:24.710033+0000 front 2024-03-18T23:20:24.709803+0000 (oldest deadline 2024-03-18T23:20:50.510257+0000) 2024-03-18T23:21:16.314 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:21:15 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:21:15.977+0000 7f83cd240700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-03-18T23:20:35.111771+0000 front 2024-03-18T23:20:35.111796+0000 (oldest deadline 2024-03-18T23:21:01.011950+0000) 2024-03-18T23:21:16.315 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:21:15 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:21:15.977+0000 7f83cd240700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.175:6812 osd.5 since back 2024-03-18T23:20:41.012537+0000 front 2024-03-18T23:20:41.012520+0000 (oldest deadline 2024-03-18T23:21:05.712608+0000) 2024-03-18T23:21:16.315 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:21:15 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:21:15.977+0000 7f83cd240700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.49:6822 osd.6 since back 2024-03-18T23:20:51.013879+0000 front 2024-03-18T23:20:51.013823+0000 (oldest deadline 2024-03-18T23:21:13.913724+0000) 2024-03-18T23:21:16.628 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 18 23:21:16 smithi049 podman[124540]: 2024-03-18 23:21:16.240311406 +0000 UTC m=+0.514373059 container remove 0320517bd4300f53173ac26036a7f5e57e231c6e5708407a1da800d6f2bc15b8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-9, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240318.181317-b6ae249) 2024-03-18T23:21:16.960 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:21:16 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:21:16.683+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:19:57.947723+0000 front 2024-03-18T23:19:57.947785+0000 (oldest deadline 2024-03-18T23:20:28.548565+0000) 2024-03-18T23:21:16.960 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:21:16 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:21:16.683+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-03-18T23:20:08.549207+0000 front 2024-03-18T23:20:08.549248+0000 (oldest deadline 2024-03-18T23:20:33.849233+0000) 2024-03-18T23:21:16.961 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:21:16 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:21:16.683+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-03-18T23:20:16.150579+0000 front 2024-03-18T23:20:16.150606+0000 (oldest deadline 2024-03-18T23:20:40.751401+0000) 2024-03-18T23:21:16.961 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:21:16 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:21:16.683+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-03-18T23:20:33.653867+0000 front 2024-03-18T23:20:33.653710+0000 (oldest deadline 2024-03-18T23:20:59.553560+0000) 2024-03-18T23:21:16.961 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:21:16 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:21:16.683+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.175:6812 osd.5 since back 2024-03-18T23:20:39.553929+0000 front 2024-03-18T23:20:39.553999+0000 (oldest deadline 2024-03-18T23:21:04.254102+0000) 2024-03-18T23:21:16.961 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:21:16 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:21:16.683+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.49:6822 osd.6 since back 2024-03-18T23:20:48.255879+0000 front 2024-03-18T23:20:48.255853+0000 (oldest deadline 2024-03-18T23:21:13.555739+0000) 2024-03-18T23:21:16.992 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 18 23:21:16 smithi049 podman[124632]: 2024-03-18 23:21:16.579560052 +0000 UTC m=+0.018219236 image pull 2312c8e42d0069a2dbb2364bddb00264db4b800e7f18a7e7e4eb8f3ecab2ff1c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639 2024-03-18T23:21:16.992 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 18 23:21:16 smithi049 podman[124632]: 2024-03-18 23:21:16.683009111 +0000 UTC m=+0.121668341 container create 90fc515e3b3d0a51875b9f6dccc891ee3b91835c28ba20e3499190f801b2349c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-9-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-18T23:21:16.993 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 18 23:21:16 smithi049 podman[124632]: 2024-03-18 23:21:16.91855482 +0000 UTC m=+0.357214016 container init 90fc515e3b3d0a51875b9f6dccc891ee3b91835c28ba20e3499190f801b2349c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-9-deactivate, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240318.181317-b6ae249) 2024-03-18T23:21:16.993 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 18 23:21:16 smithi049 podman[124632]: 2024-03-18 23:21:16.921639515 +0000 UTC m=+0.360298705 container start 90fc515e3b3d0a51875b9f6dccc891ee3b91835c28ba20e3499190f801b2349c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-9-deactivate, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-18T23:21:16.993 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 18 23:21:16 smithi049 podman[124632]: 2024-03-18 23:21:16.993450916 +0000 UTC m=+0.432110118 container attach 90fc515e3b3d0a51875b9f6dccc891ee3b91835c28ba20e3499190f801b2349c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-9-deactivate, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-03-18T23:21:17.255 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 18 23:21:17 smithi049 podman[124632]: 2024-03-18 23:21:17.231634775 +0000 UTC m=+0.670293963 container died 90fc515e3b3d0a51875b9f6dccc891ee3b91835c28ba20e3499190f801b2349c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-9-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-03-18T23:21:17.314 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:21:16 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:21:16.953+0000 7f83cd240700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:19:57.904964+0000 front 2024-03-18T23:19:57.904988+0000 (oldest deadline 2024-03-18T23:20:23.805169+0000) 2024-03-18T23:21:17.314 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:21:16 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:21:16.953+0000 7f83cd240700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-03-18T23:20:10.806661+0000 front 2024-03-18T23:20:10.806621+0000 (oldest deadline 2024-03-18T23:20:31.906747+0000) 2024-03-18T23:21:17.314 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:21:16 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:21:16.953+0000 7f83cd240700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-03-18T23:20:16.508876+0000 front 2024-03-18T23:20:16.508656+0000 (oldest deadline 2024-03-18T23:20:41.808738+0000) 2024-03-18T23:21:17.314 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:21:16 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:21:16.953+0000 7f83cd240700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-18T23:20:24.710033+0000 front 2024-03-18T23:20:24.709803+0000 (oldest deadline 2024-03-18T23:20:50.510257+0000) 2024-03-18T23:21:17.314 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:21:16 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:21:16.953+0000 7f83cd240700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-03-18T23:20:35.111771+0000 front 2024-03-18T23:20:35.111796+0000 (oldest deadline 2024-03-18T23:21:01.011950+0000) 2024-03-18T23:21:17.315 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:21:16 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:21:16.953+0000 7f83cd240700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.175:6812 osd.5 since back 2024-03-18T23:20:41.012537+0000 front 2024-03-18T23:20:41.012520+0000 (oldest deadline 2024-03-18T23:21:05.712608+0000) 2024-03-18T23:21:17.315 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:21:16 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:21:16.953+0000 7f83cd240700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.49:6822 osd.6 since back 2024-03-18T23:20:51.013879+0000 front 2024-03-18T23:20:51.013823+0000 (oldest deadline 2024-03-18T23:21:13.913724+0000) 2024-03-18T23:21:17.810 DEBUG:teuthology.orchestra.run.smithi049:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c63591ea-e57a-11ee-95c9-87774f69a715@osd.9.service' 2024-03-18T23:21:17.858 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-18T23:21:17.858 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-03-18T23:21:17.858 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-03-18T23:21:17.859 DEBUG:teuthology.orchestra.run.smithi139:> sudo systemctl stop ceph-c63591ea-e57a-11ee-95c9-87774f69a715@osd.10 2024-03-18T23:21:18.191 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:21:17 smithi139 systemd[1]: Stopping Ceph osd.10 for c63591ea-e57a-11ee-95c9-87774f69a715... 2024-03-18T23:21:18.192 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:21:17 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:21:17.996+0000 7f83cd240700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:19:57.904964+0000 front 2024-03-18T23:19:57.904988+0000 (oldest deadline 2024-03-18T23:20:23.805169+0000) 2024-03-18T23:21:18.192 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:21:17 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:21:17.996+0000 7f83cd240700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-03-18T23:20:10.806661+0000 front 2024-03-18T23:20:10.806621+0000 (oldest deadline 2024-03-18T23:20:31.906747+0000) 2024-03-18T23:21:18.192 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:21:17 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:21:17.996+0000 7f83cd240700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-03-18T23:20:16.508876+0000 front 2024-03-18T23:20:16.508656+0000 (oldest deadline 2024-03-18T23:20:41.808738+0000) 2024-03-18T23:21:18.192 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:21:17 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:21:17.996+0000 7f83cd240700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-18T23:20:24.710033+0000 front 2024-03-18T23:20:24.709803+0000 (oldest deadline 2024-03-18T23:20:50.510257+0000) 2024-03-18T23:21:18.192 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:21:17 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:21:17.996+0000 7f83cd240700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-03-18T23:20:35.111771+0000 front 2024-03-18T23:20:35.111796+0000 (oldest deadline 2024-03-18T23:21:01.011950+0000) 2024-03-18T23:21:18.192 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:21:17 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:21:17.996+0000 7f83cd240700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.175:6812 osd.5 since back 2024-03-18T23:20:41.012537+0000 front 2024-03-18T23:20:41.012520+0000 (oldest deadline 2024-03-18T23:21:05.712608+0000) 2024-03-18T23:21:18.192 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:21:17 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:21:17.996+0000 7f83cd240700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.49:6822 osd.6 since back 2024-03-18T23:20:51.013879+0000 front 2024-03-18T23:20:51.013823+0000 (oldest deadline 2024-03-18T23:21:13.913724+0000) 2024-03-18T23:21:18.192 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:21:18 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:21:18.191+0000 7f83d0c4c700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-18T23:21:18.193 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:21:18 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:21:18.191+0000 7f83d0c4c700 -1 osd.10 104 *** Got signal Terminated *** 2024-03-18T23:21:18.193 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:21:18 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:21:18.191+0000 7f83d0c4c700 -1 osd.10 104 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-18T23:21:18.210 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:21:17 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:21:17.727+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:19:57.947723+0000 front 2024-03-18T23:19:57.947785+0000 (oldest deadline 2024-03-18T23:20:28.548565+0000) 2024-03-18T23:21:18.210 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:21:17 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:21:17.727+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-03-18T23:20:08.549207+0000 front 2024-03-18T23:20:08.549248+0000 (oldest deadline 2024-03-18T23:20:33.849233+0000) 2024-03-18T23:21:18.210 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:21:17 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:21:17.727+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-03-18T23:20:16.150579+0000 front 2024-03-18T23:20:16.150606+0000 (oldest deadline 2024-03-18T23:20:40.751401+0000) 2024-03-18T23:21:18.210 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:21:17 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:21:17.727+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-03-18T23:20:33.653867+0000 front 2024-03-18T23:20:33.653710+0000 (oldest deadline 2024-03-18T23:20:59.553560+0000) 2024-03-18T23:21:18.210 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:21:17 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:21:17.727+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.175:6812 osd.5 since back 2024-03-18T23:20:39.553929+0000 front 2024-03-18T23:20:39.553999+0000 (oldest deadline 2024-03-18T23:21:04.254102+0000) 2024-03-18T23:21:18.210 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:21:17 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:21:17.727+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.49:6822 osd.6 since back 2024-03-18T23:20:48.255879+0000 front 2024-03-18T23:20:48.255853+0000 (oldest deadline 2024-03-18T23:21:13.555739+0000) 2024-03-18T23:21:18.960 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:21:18 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:21:18.701+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:19:57.947723+0000 front 2024-03-18T23:19:57.947785+0000 (oldest deadline 2024-03-18T23:20:28.548565+0000) 2024-03-18T23:21:18.960 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:21:18 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:21:18.701+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-03-18T23:20:08.549207+0000 front 2024-03-18T23:20:08.549248+0000 (oldest deadline 2024-03-18T23:20:33.849233+0000) 2024-03-18T23:21:18.960 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:21:18 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:21:18.701+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-03-18T23:20:16.150579+0000 front 2024-03-18T23:20:16.150606+0000 (oldest deadline 2024-03-18T23:20:40.751401+0000) 2024-03-18T23:21:18.961 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:21:18 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:21:18.701+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-03-18T23:20:33.653867+0000 front 2024-03-18T23:20:33.653710+0000 (oldest deadline 2024-03-18T23:20:59.553560+0000) 2024-03-18T23:21:18.961 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:21:18 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:21:18.701+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.175:6812 osd.5 since back 2024-03-18T23:20:39.553929+0000 front 2024-03-18T23:20:39.553999+0000 (oldest deadline 2024-03-18T23:21:04.254102+0000) 2024-03-18T23:21:18.961 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:21:18 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:21:18.701+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.49:6822 osd.6 since back 2024-03-18T23:20:48.255879+0000 front 2024-03-18T23:20:48.255853+0000 (oldest deadline 2024-03-18T23:21:13.555739+0000) 2024-03-18T23:21:19.314 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:21:19 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:21:19.032+0000 7f83cd240700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:19:57.904964+0000 front 2024-03-18T23:19:57.904988+0000 (oldest deadline 2024-03-18T23:20:23.805169+0000) 2024-03-18T23:21:19.314 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:21:19 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:21:19.032+0000 7f83cd240700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-03-18T23:20:10.806661+0000 front 2024-03-18T23:20:10.806621+0000 (oldest deadline 2024-03-18T23:20:31.906747+0000) 2024-03-18T23:21:19.314 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:21:19 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:21:19.032+0000 7f83cd240700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-03-18T23:20:16.508876+0000 front 2024-03-18T23:20:16.508656+0000 (oldest deadline 2024-03-18T23:20:41.808738+0000) 2024-03-18T23:21:19.314 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:21:19 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:21:19.032+0000 7f83cd240700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-18T23:20:24.710033+0000 front 2024-03-18T23:20:24.709803+0000 (oldest deadline 2024-03-18T23:20:50.510257+0000) 2024-03-18T23:21:19.314 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:21:19 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:21:19.032+0000 7f83cd240700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-03-18T23:20:35.111771+0000 front 2024-03-18T23:20:35.111796+0000 (oldest deadline 2024-03-18T23:21:01.011950+0000) 2024-03-18T23:21:19.315 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:21:19 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:21:19.032+0000 7f83cd240700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.175:6812 osd.5 since back 2024-03-18T23:20:41.012537+0000 front 2024-03-18T23:20:41.012520+0000 (oldest deadline 2024-03-18T23:21:05.712608+0000) 2024-03-18T23:21:19.315 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:21:19 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:21:19.032+0000 7f83cd240700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.49:6822 osd.6 since back 2024-03-18T23:20:51.013879+0000 front 2024-03-18T23:20:51.013823+0000 (oldest deadline 2024-03-18T23:21:13.913724+0000) 2024-03-18T23:21:20.210 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:21:19 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:21:19.737+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:19:57.947723+0000 front 2024-03-18T23:19:57.947785+0000 (oldest deadline 2024-03-18T23:20:28.548565+0000) 2024-03-18T23:21:20.211 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:21:19 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:21:19.737+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-03-18T23:20:08.549207+0000 front 2024-03-18T23:20:08.549248+0000 (oldest deadline 2024-03-18T23:20:33.849233+0000) 2024-03-18T23:21:20.211 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:21:19 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:21:19.737+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-03-18T23:20:16.150579+0000 front 2024-03-18T23:20:16.150606+0000 (oldest deadline 2024-03-18T23:20:40.751401+0000) 2024-03-18T23:21:20.211 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:21:19 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:21:19.737+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-03-18T23:20:33.653867+0000 front 2024-03-18T23:20:33.653710+0000 (oldest deadline 2024-03-18T23:20:59.553560+0000) 2024-03-18T23:21:20.211 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:21:19 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:21:19.737+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.175:6812 osd.5 since back 2024-03-18T23:20:39.553929+0000 front 2024-03-18T23:20:39.553999+0000 (oldest deadline 2024-03-18T23:21:04.254102+0000) 2024-03-18T23:21:20.211 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:21:19 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:21:19.737+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.49:6822 osd.6 since back 2024-03-18T23:20:48.255879+0000 front 2024-03-18T23:20:48.255853+0000 (oldest deadline 2024-03-18T23:21:13.555739+0000) 2024-03-18T23:21:20.564 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:21:20 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:21:20.077+0000 7f83cd240700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:19:57.904964+0000 front 2024-03-18T23:19:57.904988+0000 (oldest deadline 2024-03-18T23:20:23.805169+0000) 2024-03-18T23:21:20.564 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:21:20 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:21:20.077+0000 7f83cd240700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-03-18T23:20:10.806661+0000 front 2024-03-18T23:20:10.806621+0000 (oldest deadline 2024-03-18T23:20:31.906747+0000) 2024-03-18T23:21:20.564 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:21:20 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:21:20.077+0000 7f83cd240700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-03-18T23:20:16.508876+0000 front 2024-03-18T23:20:16.508656+0000 (oldest deadline 2024-03-18T23:20:41.808738+0000) 2024-03-18T23:21:20.564 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:21:20 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:21:20.077+0000 7f83cd240700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-18T23:20:24.710033+0000 front 2024-03-18T23:20:24.709803+0000 (oldest deadline 2024-03-18T23:20:50.510257+0000) 2024-03-18T23:21:20.565 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:21:20 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:21:20.077+0000 7f83cd240700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-03-18T23:20:35.111771+0000 front 2024-03-18T23:20:35.111796+0000 (oldest deadline 2024-03-18T23:21:01.011950+0000) 2024-03-18T23:21:20.565 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:21:20 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:21:20.077+0000 7f83cd240700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.175:6812 osd.5 since back 2024-03-18T23:20:41.012537+0000 front 2024-03-18T23:20:41.012520+0000 (oldest deadline 2024-03-18T23:21:05.712608+0000) 2024-03-18T23:21:20.565 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:21:20 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:21:20.077+0000 7f83cd240700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.49:6822 osd.6 since back 2024-03-18T23:20:51.013879+0000 front 2024-03-18T23:20:51.013823+0000 (oldest deadline 2024-03-18T23:21:13.913724+0000) 2024-03-18T23:21:21.210 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:21:20 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:21:20.763+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:19:57.947723+0000 front 2024-03-18T23:19:57.947785+0000 (oldest deadline 2024-03-18T23:20:28.548565+0000) 2024-03-18T23:21:21.210 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:21:20 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:21:20.763+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-03-18T23:20:08.549207+0000 front 2024-03-18T23:20:08.549248+0000 (oldest deadline 2024-03-18T23:20:33.849233+0000) 2024-03-18T23:21:21.210 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:21:20 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:21:20.763+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-03-18T23:20:16.150579+0000 front 2024-03-18T23:20:16.150606+0000 (oldest deadline 2024-03-18T23:20:40.751401+0000) 2024-03-18T23:21:21.211 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:21:20 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:21:20.763+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-03-18T23:20:33.653867+0000 front 2024-03-18T23:20:33.653710+0000 (oldest deadline 2024-03-18T23:20:59.553560+0000) 2024-03-18T23:21:21.211 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:21:20 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:21:20.763+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.175:6812 osd.5 since back 2024-03-18T23:20:39.553929+0000 front 2024-03-18T23:20:39.553999+0000 (oldest deadline 2024-03-18T23:21:04.254102+0000) 2024-03-18T23:21:21.211 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:21:20 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:21:20.763+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.49:6822 osd.6 since back 2024-03-18T23:20:48.255879+0000 front 2024-03-18T23:20:48.255853+0000 (oldest deadline 2024-03-18T23:21:13.555739+0000) 2024-03-18T23:21:21.314 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:21:21 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:21:21.055+0000 7f83cd240700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:19:57.904964+0000 front 2024-03-18T23:19:57.904988+0000 (oldest deadline 2024-03-18T23:20:23.805169+0000) 2024-03-18T23:21:21.314 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:21:21 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:21:21.055+0000 7f83cd240700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-03-18T23:20:10.806661+0000 front 2024-03-18T23:20:10.806621+0000 (oldest deadline 2024-03-18T23:20:31.906747+0000) 2024-03-18T23:21:21.314 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:21:21 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:21:21.055+0000 7f83cd240700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-03-18T23:20:16.508876+0000 front 2024-03-18T23:20:16.508656+0000 (oldest deadline 2024-03-18T23:20:41.808738+0000) 2024-03-18T23:21:21.314 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:21:21 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:21:21.055+0000 7f83cd240700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-18T23:20:24.710033+0000 front 2024-03-18T23:20:24.709803+0000 (oldest deadline 2024-03-18T23:20:50.510257+0000) 2024-03-18T23:21:21.314 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:21:21 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:21:21.055+0000 7f83cd240700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-03-18T23:20:35.111771+0000 front 2024-03-18T23:20:35.111796+0000 (oldest deadline 2024-03-18T23:21:01.011950+0000) 2024-03-18T23:21:21.314 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:21:21 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:21:21.055+0000 7f83cd240700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.175:6812 osd.5 since back 2024-03-18T23:20:41.012537+0000 front 2024-03-18T23:20:41.012520+0000 (oldest deadline 2024-03-18T23:21:05.712608+0000) 2024-03-18T23:21:21.315 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:21:21 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:21:21.055+0000 7f83cd240700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.49:6822 osd.6 since back 2024-03-18T23:20:51.013879+0000 front 2024-03-18T23:20:51.013823+0000 (oldest deadline 2024-03-18T23:21:13.913724+0000) 2024-03-18T23:21:22.210 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:21:21 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:21:21.776+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:19:57.947723+0000 front 2024-03-18T23:19:57.947785+0000 (oldest deadline 2024-03-18T23:20:28.548565+0000) 2024-03-18T23:21:22.210 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:21:21 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:21:21.776+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-03-18T23:20:08.549207+0000 front 2024-03-18T23:20:08.549248+0000 (oldest deadline 2024-03-18T23:20:33.849233+0000) 2024-03-18T23:21:22.210 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:21:21 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:21:21.776+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-03-18T23:20:16.150579+0000 front 2024-03-18T23:20:16.150606+0000 (oldest deadline 2024-03-18T23:20:40.751401+0000) 2024-03-18T23:21:22.211 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:21:21 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:21:21.776+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-03-18T23:20:33.653867+0000 front 2024-03-18T23:20:33.653710+0000 (oldest deadline 2024-03-18T23:20:59.553560+0000) 2024-03-18T23:21:22.211 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:21:21 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:21:21.776+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.175:6812 osd.5 since back 2024-03-18T23:20:39.553929+0000 front 2024-03-18T23:20:39.553999+0000 (oldest deadline 2024-03-18T23:21:04.254102+0000) 2024-03-18T23:21:22.211 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:21:21 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:21:21.776+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.49:6822 osd.6 since back 2024-03-18T23:20:48.255879+0000 front 2024-03-18T23:20:48.255853+0000 (oldest deadline 2024-03-18T23:21:13.555739+0000) 2024-03-18T23:21:22.314 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:21:22 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:21:22.030+0000 7f83cd240700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:19:57.904964+0000 front 2024-03-18T23:19:57.904988+0000 (oldest deadline 2024-03-18T23:20:23.805169+0000) 2024-03-18T23:21:22.314 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:21:22 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:21:22.030+0000 7f83cd240700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-03-18T23:20:10.806661+0000 front 2024-03-18T23:20:10.806621+0000 (oldest deadline 2024-03-18T23:20:31.906747+0000) 2024-03-18T23:21:22.314 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:21:22 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:21:22.030+0000 7f83cd240700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-03-18T23:20:16.508876+0000 front 2024-03-18T23:20:16.508656+0000 (oldest deadline 2024-03-18T23:20:41.808738+0000) 2024-03-18T23:21:22.314 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:21:22 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:21:22.030+0000 7f83cd240700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-18T23:20:24.710033+0000 front 2024-03-18T23:20:24.709803+0000 (oldest deadline 2024-03-18T23:20:50.510257+0000) 2024-03-18T23:21:22.314 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:21:22 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:21:22.030+0000 7f83cd240700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-03-18T23:20:35.111771+0000 front 2024-03-18T23:20:35.111796+0000 (oldest deadline 2024-03-18T23:21:01.011950+0000) 2024-03-18T23:21:22.315 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:21:22 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:21:22.030+0000 7f83cd240700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.175:6812 osd.5 since back 2024-03-18T23:20:41.012537+0000 front 2024-03-18T23:20:41.012520+0000 (oldest deadline 2024-03-18T23:21:05.712608+0000) 2024-03-18T23:21:22.315 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:21:22 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:21:22.030+0000 7f83cd240700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.49:6822 osd.6 since back 2024-03-18T23:20:51.013879+0000 front 2024-03-18T23:20:51.013823+0000 (oldest deadline 2024-03-18T23:21:13.913724+0000) 2024-03-18T23:21:23.210 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:21:22 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:21:22.796+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:19:57.947723+0000 front 2024-03-18T23:19:57.947785+0000 (oldest deadline 2024-03-18T23:20:28.548565+0000) 2024-03-18T23:21:23.210 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:21:22 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:21:22.796+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-03-18T23:20:08.549207+0000 front 2024-03-18T23:20:08.549248+0000 (oldest deadline 2024-03-18T23:20:33.849233+0000) 2024-03-18T23:21:23.210 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:21:22 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:21:22.796+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-03-18T23:20:16.150579+0000 front 2024-03-18T23:20:16.150606+0000 (oldest deadline 2024-03-18T23:20:40.751401+0000) 2024-03-18T23:21:23.211 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:21:22 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:21:22.796+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-03-18T23:20:33.653867+0000 front 2024-03-18T23:20:33.653710+0000 (oldest deadline 2024-03-18T23:20:59.553560+0000) 2024-03-18T23:21:23.211 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:21:22 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:21:22.796+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.175:6812 osd.5 since back 2024-03-18T23:20:39.553929+0000 front 2024-03-18T23:20:39.553999+0000 (oldest deadline 2024-03-18T23:21:04.254102+0000) 2024-03-18T23:21:23.211 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:21:22 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:21:22.796+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.49:6822 osd.6 since back 2024-03-18T23:20:48.255879+0000 front 2024-03-18T23:20:48.255853+0000 (oldest deadline 2024-03-18T23:21:13.555739+0000) 2024-03-18T23:21:23.314 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:21:23 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:21:23.058+0000 7f83cd240700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:19:57.904964+0000 front 2024-03-18T23:19:57.904988+0000 (oldest deadline 2024-03-18T23:20:23.805169+0000) 2024-03-18T23:21:23.314 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:21:23 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:21:23.058+0000 7f83cd240700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-03-18T23:20:10.806661+0000 front 2024-03-18T23:20:10.806621+0000 (oldest deadline 2024-03-18T23:20:31.906747+0000) 2024-03-18T23:21:23.314 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:21:23 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:21:23.058+0000 7f83cd240700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-03-18T23:20:16.508876+0000 front 2024-03-18T23:20:16.508656+0000 (oldest deadline 2024-03-18T23:20:41.808738+0000) 2024-03-18T23:21:23.314 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:21:23 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:21:23.058+0000 7f83cd240700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-18T23:20:24.710033+0000 front 2024-03-18T23:20:24.709803+0000 (oldest deadline 2024-03-18T23:20:50.510257+0000) 2024-03-18T23:21:23.315 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:21:23 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:21:23.058+0000 7f83cd240700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-03-18T23:20:35.111771+0000 front 2024-03-18T23:20:35.111796+0000 (oldest deadline 2024-03-18T23:21:01.011950+0000) 2024-03-18T23:21:23.315 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:21:23 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:21:23.058+0000 7f83cd240700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.175:6812 osd.5 since back 2024-03-18T23:20:41.012537+0000 front 2024-03-18T23:20:41.012520+0000 (oldest deadline 2024-03-18T23:21:05.712608+0000) 2024-03-18T23:21:23.315 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:21:23 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10[92410]: 2024-03-18T23:21:23.058+0000 7f83cd240700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.49:6822 osd.6 since back 2024-03-18T23:20:51.013879+0000 front 2024-03-18T23:20:51.013823+0000 (oldest deadline 2024-03-18T23:21:13.913724+0000) 2024-03-18T23:21:24.015 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:21:23 smithi139 podman[105164]: 2024-03-18 23:21:23.738470663 +0000 UTC m=+5.684075233 container died 15d5cddd4307fb38191dcaa761f45fdab912c3afd4ce035e2c64737641b4d58a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-03-18T23:21:24.210 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:21:23 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:21:23.784+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:19:57.947723+0000 front 2024-03-18T23:19:57.947785+0000 (oldest deadline 2024-03-18T23:20:28.548565+0000) 2024-03-18T23:21:24.210 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:21:23 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:21:23.784+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-03-18T23:20:08.549207+0000 front 2024-03-18T23:20:08.549248+0000 (oldest deadline 2024-03-18T23:20:33.849233+0000) 2024-03-18T23:21:24.210 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:21:23 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:21:23.784+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-03-18T23:20:16.150579+0000 front 2024-03-18T23:20:16.150606+0000 (oldest deadline 2024-03-18T23:20:40.751401+0000) 2024-03-18T23:21:24.210 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:21:23 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:21:23.784+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-03-18T23:20:33.653867+0000 front 2024-03-18T23:20:33.653710+0000 (oldest deadline 2024-03-18T23:20:59.553560+0000) 2024-03-18T23:21:24.211 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:21:23 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:21:23.784+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.175:6812 osd.5 since back 2024-03-18T23:20:39.553929+0000 front 2024-03-18T23:20:39.553999+0000 (oldest deadline 2024-03-18T23:21:04.254102+0000) 2024-03-18T23:21:24.211 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:21:23 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:21:23.784+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.49:6822 osd.6 since back 2024-03-18T23:20:48.255879+0000 front 2024-03-18T23:20:48.255853+0000 (oldest deadline 2024-03-18T23:21:13.555739+0000) 2024-03-18T23:21:24.211 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:21:23 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:21:23.784+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.139:6820 osd.7 since back 2024-03-18T23:20:57.656895+0000 front 2024-03-18T23:20:57.656974+0000 (oldest deadline 2024-03-18T23:21:23.556813+0000) 2024-03-18T23:21:24.314 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:21:24 smithi139 podman[105164]: 2024-03-18 23:21:24.124901802 +0000 UTC m=+6.070506376 container cleanup 15d5cddd4307fb38191dcaa761f45fdab912c3afd4ce035e2c64737641b4d58a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, RELEASE=wip-batrick-testing-20240318.181317-b6ae249) 2024-03-18T23:21:24.314 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:21:24 smithi139 bash[105164]: ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10 2024-03-18T23:21:24.730 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:21:24 smithi139 podman[105181]: 2024-03-18 23:21:24.376058317 +0000 UTC m=+0.636508900 container remove 15d5cddd4307fb38191dcaa761f45fdab912c3afd4ce035e2c64737641b4d58a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True) 2024-03-18T23:21:24.986 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:21:24 smithi139 podman[105276]: 2024-03-18 23:21:24.634393385 +0000 UTC m=+0.017085403 image pull 2312c8e42d0069a2dbb2364bddb00264db4b800e7f18a7e7e4eb8f3ecab2ff1c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639 2024-03-18T23:21:24.986 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:21:24 smithi139 podman[105276]: 2024-03-18 23:21:24.735169564 +0000 UTC m=+0.117861606 container create 3ee7ba6c147f1eca8a8441fd37a84468947345fa31b5ecc62c3a5298ef961bdb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10-deactivate, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-03-18T23:21:24.986 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:21:24 smithi139 podman[105276]: 2024-03-18 23:21:24.987814191 +0000 UTC m=+0.370506207 container init 3ee7ba6c147f1eca8a8441fd37a84468947345fa31b5ecc62c3a5298ef961bdb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10-deactivate, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-03-18T23:21:25.048 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:21:24 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:21:24.791+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:19:57.947723+0000 front 2024-03-18T23:19:57.947785+0000 (oldest deadline 2024-03-18T23:20:28.548565+0000) 2024-03-18T23:21:25.048 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:21:24 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:21:24.791+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-03-18T23:20:08.549207+0000 front 2024-03-18T23:20:08.549248+0000 (oldest deadline 2024-03-18T23:20:33.849233+0000) 2024-03-18T23:21:25.048 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:21:24 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:21:24.791+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-03-18T23:20:16.150579+0000 front 2024-03-18T23:20:16.150606+0000 (oldest deadline 2024-03-18T23:20:40.751401+0000) 2024-03-18T23:21:25.048 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:21:24 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:21:24.791+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-03-18T23:20:33.653867+0000 front 2024-03-18T23:20:33.653710+0000 (oldest deadline 2024-03-18T23:20:59.553560+0000) 2024-03-18T23:21:25.048 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:21:24 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:21:24.791+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.175:6812 osd.5 since back 2024-03-18T23:20:39.553929+0000 front 2024-03-18T23:20:39.553999+0000 (oldest deadline 2024-03-18T23:21:04.254102+0000) 2024-03-18T23:21:25.048 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:21:24 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:21:24.791+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.49:6822 osd.6 since back 2024-03-18T23:20:48.255879+0000 front 2024-03-18T23:20:48.255853+0000 (oldest deadline 2024-03-18T23:21:13.555739+0000) 2024-03-18T23:21:25.048 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:21:24 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:21:24.791+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.139:6820 osd.7 since back 2024-03-18T23:20:57.656895+0000 front 2024-03-18T23:20:57.656974+0000 (oldest deadline 2024-03-18T23:21:23.556813+0000) 2024-03-18T23:21:25.286 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:21:24 smithi139 podman[105276]: 2024-03-18 23:21:24.990930935 +0000 UTC m=+0.373622933 container start 3ee7ba6c147f1eca8a8441fd37a84468947345fa31b5ecc62c3a5298ef961bdb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10-deactivate, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-03-18T23:21:25.286 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:21:25 smithi139 podman[105276]: 2024-03-18 23:21:25.054310967 +0000 UTC m=+0.437002982 container attach 3ee7ba6c147f1eca8a8441fd37a84468947345fa31b5ecc62c3a5298ef961bdb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10-deactivate, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240318.181317-b6ae249) 2024-03-18T23:21:25.551 INFO:journalctl@ceph.osd.10.smithi139.stdout:Mar 18 23:21:25 smithi139 podman[105276]: 2024-03-18 23:21:25.287738868 +0000 UTC m=+0.670430896 container died 3ee7ba6c147f1eca8a8441fd37a84468947345fa31b5ecc62c3a5298ef961bdb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-10-deactivate, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-03-18T23:21:25.869 DEBUG:teuthology.orchestra.run.smithi139:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c63591ea-e57a-11ee-95c9-87774f69a715@osd.10.service' 2024-03-18T23:21:25.910 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-18T23:21:25.910 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-03-18T23:21:25.910 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-03-18T23:21:25.911 DEBUG:teuthology.orchestra.run.smithi175:> sudo systemctl stop ceph-c63591ea-e57a-11ee-95c9-87774f69a715@osd.11 2024-03-18T23:21:26.210 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:21:25 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:21:25.797+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:19:57.947723+0000 front 2024-03-18T23:19:57.947785+0000 (oldest deadline 2024-03-18T23:20:28.548565+0000) 2024-03-18T23:21:26.211 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:21:25 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:21:25.797+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-03-18T23:20:08.549207+0000 front 2024-03-18T23:20:08.549248+0000 (oldest deadline 2024-03-18T23:20:33.849233+0000) 2024-03-18T23:21:26.211 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:21:25 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:21:25.797+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-03-18T23:20:16.150579+0000 front 2024-03-18T23:20:16.150606+0000 (oldest deadline 2024-03-18T23:20:40.751401+0000) 2024-03-18T23:21:26.211 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:21:25 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:21:25.797+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-03-18T23:20:33.653867+0000 front 2024-03-18T23:20:33.653710+0000 (oldest deadline 2024-03-18T23:20:59.553560+0000) 2024-03-18T23:21:26.211 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:21:25 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:21:25.797+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.175:6812 osd.5 since back 2024-03-18T23:20:39.553929+0000 front 2024-03-18T23:20:39.553999+0000 (oldest deadline 2024-03-18T23:21:04.254102+0000) 2024-03-18T23:21:26.211 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:21:25 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:21:25.797+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.49:6822 osd.6 since back 2024-03-18T23:20:48.255879+0000 front 2024-03-18T23:20:48.255853+0000 (oldest deadline 2024-03-18T23:21:13.555739+0000) 2024-03-18T23:21:26.211 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:21:25 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:21:25.797+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.139:6820 osd.7 since back 2024-03-18T23:20:57.656895+0000 front 2024-03-18T23:20:57.656974+0000 (oldest deadline 2024-03-18T23:21:23.556813+0000) 2024-03-18T23:21:26.211 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:21:25 smithi175 systemd[1]: Stopping Ceph osd.11 for c63591ea-e57a-11ee-95c9-87774f69a715... 2024-03-18T23:21:26.710 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:21:26 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:21:26.243+0000 7fd319dff700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-18T23:21:26.710 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:21:26 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:21:26.243+0000 7fd319dff700 -1 osd.11 104 *** Got signal Terminated *** 2024-03-18T23:21:26.710 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:21:26 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:21:26.243+0000 7fd319dff700 -1 osd.11 104 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-18T23:21:27.210 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:21:26 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:21:26.807+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:19:57.947723+0000 front 2024-03-18T23:19:57.947785+0000 (oldest deadline 2024-03-18T23:20:28.548565+0000) 2024-03-18T23:21:27.210 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:21:26 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:21:26.807+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-03-18T23:20:08.549207+0000 front 2024-03-18T23:20:08.549248+0000 (oldest deadline 2024-03-18T23:20:33.849233+0000) 2024-03-18T23:21:27.210 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:21:26 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:21:26.807+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-03-18T23:20:16.150579+0000 front 2024-03-18T23:20:16.150606+0000 (oldest deadline 2024-03-18T23:20:40.751401+0000) 2024-03-18T23:21:27.211 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:21:26 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:21:26.807+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-03-18T23:20:33.653867+0000 front 2024-03-18T23:20:33.653710+0000 (oldest deadline 2024-03-18T23:20:59.553560+0000) 2024-03-18T23:21:27.211 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:21:26 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:21:26.807+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.175:6812 osd.5 since back 2024-03-18T23:20:39.553929+0000 front 2024-03-18T23:20:39.553999+0000 (oldest deadline 2024-03-18T23:21:04.254102+0000) 2024-03-18T23:21:27.211 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:21:26 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:21:26.807+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.49:6822 osd.6 since back 2024-03-18T23:20:48.255879+0000 front 2024-03-18T23:20:48.255853+0000 (oldest deadline 2024-03-18T23:21:13.555739+0000) 2024-03-18T23:21:27.211 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:21:26 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:21:26.807+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.139:6820 osd.7 since back 2024-03-18T23:20:57.656895+0000 front 2024-03-18T23:20:57.656974+0000 (oldest deadline 2024-03-18T23:21:23.556813+0000) 2024-03-18T23:21:28.210 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:21:27 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:21:27.773+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:19:57.947723+0000 front 2024-03-18T23:19:57.947785+0000 (oldest deadline 2024-03-18T23:20:28.548565+0000) 2024-03-18T23:21:28.210 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:21:27 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:21:27.773+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-03-18T23:20:08.549207+0000 front 2024-03-18T23:20:08.549248+0000 (oldest deadline 2024-03-18T23:20:33.849233+0000) 2024-03-18T23:21:28.211 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:21:27 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:21:27.773+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-03-18T23:20:16.150579+0000 front 2024-03-18T23:20:16.150606+0000 (oldest deadline 2024-03-18T23:20:40.751401+0000) 2024-03-18T23:21:28.211 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:21:27 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:21:27.773+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-03-18T23:20:33.653867+0000 front 2024-03-18T23:20:33.653710+0000 (oldest deadline 2024-03-18T23:20:59.553560+0000) 2024-03-18T23:21:28.211 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:21:27 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:21:27.773+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.175:6812 osd.5 since back 2024-03-18T23:20:39.553929+0000 front 2024-03-18T23:20:39.553999+0000 (oldest deadline 2024-03-18T23:21:04.254102+0000) 2024-03-18T23:21:28.211 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:21:27 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:21:27.773+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.49:6822 osd.6 since back 2024-03-18T23:20:48.255879+0000 front 2024-03-18T23:20:48.255853+0000 (oldest deadline 2024-03-18T23:21:13.555739+0000) 2024-03-18T23:21:28.211 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:21:27 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:21:27.773+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.139:6820 osd.7 since back 2024-03-18T23:20:57.656895+0000 front 2024-03-18T23:20:57.656974+0000 (oldest deadline 2024-03-18T23:21:23.556813+0000) 2024-03-18T23:21:29.210 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:21:28 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:21:28.767+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:19:57.947723+0000 front 2024-03-18T23:19:57.947785+0000 (oldest deadline 2024-03-18T23:20:28.548565+0000) 2024-03-18T23:21:29.210 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:21:28 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:21:28.767+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-03-18T23:20:08.549207+0000 front 2024-03-18T23:20:08.549248+0000 (oldest deadline 2024-03-18T23:20:33.849233+0000) 2024-03-18T23:21:29.210 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:21:28 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:21:28.767+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-03-18T23:20:16.150579+0000 front 2024-03-18T23:20:16.150606+0000 (oldest deadline 2024-03-18T23:20:40.751401+0000) 2024-03-18T23:21:29.210 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:21:28 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:21:28.767+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-03-18T23:20:33.653867+0000 front 2024-03-18T23:20:33.653710+0000 (oldest deadline 2024-03-18T23:20:59.553560+0000) 2024-03-18T23:21:29.211 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:21:28 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:21:28.767+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.175:6812 osd.5 since back 2024-03-18T23:20:39.553929+0000 front 2024-03-18T23:20:39.553999+0000 (oldest deadline 2024-03-18T23:21:04.254102+0000) 2024-03-18T23:21:29.211 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:21:28 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:21:28.767+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.49:6822 osd.6 since back 2024-03-18T23:20:48.255879+0000 front 2024-03-18T23:20:48.255853+0000 (oldest deadline 2024-03-18T23:21:13.555739+0000) 2024-03-18T23:21:29.211 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:21:28 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:21:28.767+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.139:6820 osd.7 since back 2024-03-18T23:20:57.656895+0000 front 2024-03-18T23:20:57.656974+0000 (oldest deadline 2024-03-18T23:21:23.556813+0000) 2024-03-18T23:21:29.211 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:21:28 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:21:28.767+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.175:6820 osd.8 since back 2024-03-18T23:21:05.857650+0000 front 2024-03-18T23:21:05.857683+0000 (oldest deadline 2024-03-18T23:21:28.157667+0000) 2024-03-18T23:21:30.210 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:21:29 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:21:29.739+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:19:57.947723+0000 front 2024-03-18T23:19:57.947785+0000 (oldest deadline 2024-03-18T23:20:28.548565+0000) 2024-03-18T23:21:30.210 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:21:29 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:21:29.739+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-03-18T23:20:08.549207+0000 front 2024-03-18T23:20:08.549248+0000 (oldest deadline 2024-03-18T23:20:33.849233+0000) 2024-03-18T23:21:30.210 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:21:29 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:21:29.739+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-03-18T23:20:16.150579+0000 front 2024-03-18T23:20:16.150606+0000 (oldest deadline 2024-03-18T23:20:40.751401+0000) 2024-03-18T23:21:30.210 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:21:29 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:21:29.739+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-03-18T23:20:33.653867+0000 front 2024-03-18T23:20:33.653710+0000 (oldest deadline 2024-03-18T23:20:59.553560+0000) 2024-03-18T23:21:30.211 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:21:29 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:21:29.739+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.175:6812 osd.5 since back 2024-03-18T23:20:39.553929+0000 front 2024-03-18T23:20:39.553999+0000 (oldest deadline 2024-03-18T23:21:04.254102+0000) 2024-03-18T23:21:30.211 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:21:29 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:21:29.739+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.49:6822 osd.6 since back 2024-03-18T23:20:48.255879+0000 front 2024-03-18T23:20:48.255853+0000 (oldest deadline 2024-03-18T23:21:13.555739+0000) 2024-03-18T23:21:30.211 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:21:29 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:21:29.739+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.139:6820 osd.7 since back 2024-03-18T23:20:57.656895+0000 front 2024-03-18T23:20:57.656974+0000 (oldest deadline 2024-03-18T23:21:23.556813+0000) 2024-03-18T23:21:30.211 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:21:29 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:21:29.739+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.175:6820 osd.8 since back 2024-03-18T23:21:05.857650+0000 front 2024-03-18T23:21:05.857683+0000 (oldest deadline 2024-03-18T23:21:28.157667+0000) 2024-03-18T23:21:31.210 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:21:30 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:21:30.726+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-18T23:19:57.947723+0000 front 2024-03-18T23:19:57.947785+0000 (oldest deadline 2024-03-18T23:20:28.548565+0000) 2024-03-18T23:21:31.210 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:21:30 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:21:30.726+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-03-18T23:20:08.549207+0000 front 2024-03-18T23:20:08.549248+0000 (oldest deadline 2024-03-18T23:20:33.849233+0000) 2024-03-18T23:21:31.210 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:21:30 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:21:30.726+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-03-18T23:20:16.150579+0000 front 2024-03-18T23:20:16.150606+0000 (oldest deadline 2024-03-18T23:20:40.751401+0000) 2024-03-18T23:21:31.211 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:21:30 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:21:30.726+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-03-18T23:20:33.653867+0000 front 2024-03-18T23:20:33.653710+0000 (oldest deadline 2024-03-18T23:20:59.553560+0000) 2024-03-18T23:21:31.211 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:21:30 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:21:30.726+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.175:6812 osd.5 since back 2024-03-18T23:20:39.553929+0000 front 2024-03-18T23:20:39.553999+0000 (oldest deadline 2024-03-18T23:21:04.254102+0000) 2024-03-18T23:21:31.211 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:21:30 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:21:30.726+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.49:6822 osd.6 since back 2024-03-18T23:20:48.255879+0000 front 2024-03-18T23:20:48.255853+0000 (oldest deadline 2024-03-18T23:21:13.555739+0000) 2024-03-18T23:21:31.211 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:21:30 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:21:30.726+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.139:6820 osd.7 since back 2024-03-18T23:20:57.656895+0000 front 2024-03-18T23:20:57.656974+0000 (oldest deadline 2024-03-18T23:21:23.556813+0000) 2024-03-18T23:21:31.211 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:21:30 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11[93065]: 2024-03-18T23:21:30.726+0000 7fd3163f3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.175:6820 osd.8 since back 2024-03-18T23:21:05.857650+0000 front 2024-03-18T23:21:05.857683+0000 (oldest deadline 2024-03-18T23:21:28.157667+0000) 2024-03-18T23:21:31.930 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:21:31 smithi175 podman[105179]: 2024-03-18 23:21:31.615778571 +0000 UTC m=+5.508901930 container died 2b7c27ba092fbec4f2394f92328b6c503400b67affd027fff2e9053f8579bd00 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-18T23:21:32.210 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:21:31 smithi175 podman[105179]: 2024-03-18 23:21:31.931911802 +0000 UTC m=+5.825035145 container cleanup 2b7c27ba092fbec4f2394f92328b6c503400b67affd027fff2e9053f8579bd00 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-03-18T23:21:32.210 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:21:31 smithi175 bash[105179]: ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11 2024-03-18T23:21:32.210 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:21:32 smithi175 podman[105196]: 2024-03-18 23:21:32.133286626 +0000 UTC m=+0.519014904 container remove 2b7c27ba092fbec4f2394f92328b6c503400b67affd027fff2e9053f8579bd00 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-18T23:21:32.785 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:21:32 smithi175 podman[105285]: 2024-03-18 23:21:32.51787488 +0000 UTC m=+0.092015216 container create b1ce90db89bbcb974b5a7149e9668b575dceeb37716103fdd3c9c2ef254ea7e0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11-deactivate, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2) 2024-03-18T23:21:32.786 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:21:32 smithi175 podman[105285]: 2024-03-18 23:21:32.443595699 +0000 UTC m=+0.017736009 image pull 2312c8e42d0069a2dbb2364bddb00264db4b800e7f18a7e7e4eb8f3ecab2ff1c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639 2024-03-18T23:21:33.081 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:21:32 smithi175 podman[105285]: 2024-03-18 23:21:32.787237985 +0000 UTC m=+0.361378293 container init b1ce90db89bbcb974b5a7149e9668b575dceeb37716103fdd3c9c2ef254ea7e0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11-deactivate, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-03-18T23:21:33.081 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:21:32 smithi175 podman[105285]: 2024-03-18 23:21:32.79051161 +0000 UTC m=+0.364651908 container start b1ce90db89bbcb974b5a7149e9668b575dceeb37716103fdd3c9c2ef254ea7e0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11-deactivate, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.build-date=20240307) 2024-03-18T23:21:33.081 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:21:32 smithi175 podman[105285]: 2024-03-18 23:21:32.861718659 +0000 UTC m=+0.435858970 container attach b1ce90db89bbcb974b5a7149e9668b575dceeb37716103fdd3c9c2ef254ea7e0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11-deactivate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, CEPH_POINT_RELEASE=) 2024-03-18T23:21:33.460 INFO:journalctl@ceph.osd.11.smithi175.stdout:Mar 18 23:21:33 smithi175 podman[105285]: 2024-03-18 23:21:33.082547311 +0000 UTC m=+0.656687619 container died b1ce90db89bbcb974b5a7149e9668b575dceeb37716103fdd3c9c2ef254ea7e0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-c63591ea-e57a-11ee-95c9-87774f69a715-osd-11-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-03-18T23:21:33.586 DEBUG:teuthology.orchestra.run.smithi175:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c63591ea-e57a-11ee-95c9-87774f69a715@osd.11.service' 2024-03-18T23:21:33.627 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-18T23:21:33.628 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-03-18T23:21:33.629 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-03-18T23:21:33.629 DEBUG:teuthology.orchestra.run.smithi049:> sudo systemctl stop ceph-c63591ea-e57a-11ee-95c9-87774f69a715@mds.a 2024-03-18T23:21:33.957 INFO:journalctl@ceph.mds.a.smithi049.stdout:Mar 18 23:21:33 smithi049 systemd[1]: Stopping Ceph mds.a for c63591ea-e57a-11ee-95c9-87774f69a715... 2024-03-18T23:21:34.378 INFO:journalctl@ceph.mds.a.smithi049.stdout:Mar 18 23:21:33 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mds-a[105002]: 2024-03-18T23:21:33.958+0000 7f2594b79700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-18T23:21:34.379 INFO:journalctl@ceph.mds.a.smithi049.stdout:Mar 18 23:21:33 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mds-a[105002]: 2024-03-18T23:21:33.958+0000 7f2594b79700 -1 mds.a *** got signal Terminated *** 2024-03-18T23:21:38.299 INFO:journalctl@ceph.mds.a.smithi049.stdout:Mar 18 23:21:37 smithi049 podman[124753]: 2024-03-18 23:21:37.970295056 +0000 UTC m=+4.128127915 container died e54358a09cccd8f9a09ecafeb892957e681358e1d4493b0fc14fcf9a9e321982 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mds-a, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307) 2024-03-18T23:21:38.589 INFO:journalctl@ceph.mds.a.smithi049.stdout:Mar 18 23:21:38 smithi049 podman[124753]: 2024-03-18 23:21:38.381691119 +0000 UTC m=+4.539523985 container cleanup e54358a09cccd8f9a09ecafeb892957e681358e1d4493b0fc14fcf9a9e321982 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mds-a, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_CLEAN=True, org.label-schema.build-date=20240307) 2024-03-18T23:21:38.590 INFO:journalctl@ceph.mds.a.smithi049.stdout:Mar 18 23:21:38 smithi049 bash[124753]: ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mds-a 2024-03-18T23:21:38.827 DEBUG:teuthology.orchestra.run.smithi049:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c63591ea-e57a-11ee-95c9-87774f69a715@mds.a.service' 2024-03-18T23:21:38.855 INFO:journalctl@ceph.mds.a.smithi049.stdout:Mar 18 23:21:38 smithi049 podman[124766]: 2024-03-18 23:21:38.591423381 +0000 UTC m=+0.617181183 container remove e54358a09cccd8f9a09ecafeb892957e681358e1d4493b0fc14fcf9a9e321982 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mds-a, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, CEPH_POINT_RELEASE=, ceph=True) 2024-03-18T23:21:38.855 INFO:journalctl@ceph.mds.a.smithi049.stdout:Mar 18 23:21:38 smithi049 systemd[1]: ceph-c63591ea-e57a-11ee-95c9-87774f69a715@mds.a.service: Deactivated successfully. 2024-03-18T23:21:38.856 INFO:journalctl@ceph.mds.a.smithi049.stdout:Mar 18 23:21:38 smithi049 systemd[1]: Stopped Ceph mds.a for c63591ea-e57a-11ee-95c9-87774f69a715. 2024-03-18T23:21:38.856 INFO:journalctl@ceph.mds.a.smithi049.stdout:Mar 18 23:21:38 smithi049 systemd[1]: ceph-c63591ea-e57a-11ee-95c9-87774f69a715@mds.a.service: Consumed 2.060s CPU time. 2024-03-18T23:21:38.869 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-18T23:21:38.869 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-03-18T23:21:38.869 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-03-18T23:21:38.869 DEBUG:teuthology.orchestra.run.smithi049:> sudo systemctl stop ceph-c63591ea-e57a-11ee-95c9-87774f69a715@mds.d 2024-03-18T23:21:39.128 INFO:journalctl@ceph.mds.d.smithi049.stdout:Mar 18 23:21:38 smithi049 systemd[1]: Stopping Ceph mds.d for c63591ea-e57a-11ee-95c9-87774f69a715... 2024-03-18T23:21:39.629 INFO:journalctl@ceph.mds.d.smithi049.stdout:Mar 18 23:21:39 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mds-d[104042]: 2024-03-18T23:21:39.239+0000 7f65033b4700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-18T23:21:39.629 INFO:journalctl@ceph.mds.d.smithi049.stdout:Mar 18 23:21:39 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mds-d[104042]: 2024-03-18T23:21:39.239+0000 7f65033b4700 -1 mds.d *** got signal Terminated *** 2024-03-18T23:21:42.379 INFO:journalctl@ceph.mds.d.smithi049.stdout:Mar 18 23:21:42 smithi049 podman[124876]: 2024-03-18 23:21:42.115682115 +0000 UTC m=+3.019323399 container died 004eafb644c5cc5629f32dd13ce88e95cb2c0f4cbe2dc6d3d171b82e256cf002 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mds-d, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=) 2024-03-18T23:21:42.751 INFO:journalctl@ceph.mds.d.smithi049.stdout:Mar 18 23:21:42 smithi049 podman[124876]: 2024-03-18 23:21:42.485588817 +0000 UTC m=+3.389230103 container cleanup 004eafb644c5cc5629f32dd13ce88e95cb2c0f4cbe2dc6d3d171b82e256cf002 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mds-d, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-03-18T23:21:42.752 INFO:journalctl@ceph.mds.d.smithi049.stdout:Mar 18 23:21:42 smithi049 bash[124876]: ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mds-d 2024-03-18T23:21:42.904 DEBUG:teuthology.orchestra.run.smithi049:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c63591ea-e57a-11ee-95c9-87774f69a715@mds.d.service' 2024-03-18T23:21:42.945 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-18T23:21:42.945 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-03-18T23:21:42.945 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-03-18T23:21:42.945 DEBUG:teuthology.orchestra.run.smithi049:> sudo systemctl stop ceph-c63591ea-e57a-11ee-95c9-87774f69a715@mds.g 2024-03-18T23:21:43.288 INFO:journalctl@ceph.mds.g.smithi049.stdout:Mar 18 23:21:43 smithi049 systemd[1]: Stopping Ceph mds.g for c63591ea-e57a-11ee-95c9-87774f69a715... 2024-03-18T23:21:43.629 INFO:journalctl@ceph.mds.g.smithi049.stdout:Mar 18 23:21:43 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mds-g[104653]: 2024-03-18T23:21:43.289+0000 7fb365907700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-18T23:21:43.629 INFO:journalctl@ceph.mds.g.smithi049.stdout:Mar 18 23:21:43 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mds-g[104653]: 2024-03-18T23:21:43.289+0000 7fb365907700 -1 mds.g *** got signal Terminated *** 2024-03-18T23:21:47.054 INFO:journalctl@ceph.mds.g.smithi049.stdout:Mar 18 23:21:46 smithi049 podman[124996]: 2024-03-18 23:21:46.75592576 +0000 UTC m=+3.583973346 container died 6536069ee40744fa44cd76208b02e8b44cd89ef1f98d9c5da84fd868c70ee358 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mds-g, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-18T23:21:47.054 INFO:journalctl@ceph.mds.g.smithi049.stdout:Mar 18 23:21:47 smithi049 podman[124996]: 2024-03-18 23:21:47.055843685 +0000 UTC m=+3.883891267 container cleanup 6536069ee40744fa44cd76208b02e8b44cd89ef1f98d9c5da84fd868c70ee358 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mds-g, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307) 2024-03-18T23:21:47.370 INFO:journalctl@ceph.mds.g.smithi049.stdout:Mar 18 23:21:47 smithi049 bash[124996]: ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mds-g 2024-03-18T23:21:47.370 INFO:journalctl@ceph.mds.g.smithi049.stdout:Mar 18 23:21:47 smithi049 podman[125010]: 2024-03-18 23:21:47.255953452 +0000 UTC m=+0.495026847 container remove 6536069ee40744fa44cd76208b02e8b44cd89ef1f98d9c5da84fd868c70ee358 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mds-g, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux ) 2024-03-18T23:21:47.385 DEBUG:teuthology.orchestra.run.smithi049:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c63591ea-e57a-11ee-95c9-87774f69a715@mds.g.service' 2024-03-18T23:21:47.423 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-18T23:21:47.423 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-03-18T23:21:47.423 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-03-18T23:21:47.424 DEBUG:teuthology.orchestra.run.smithi049:> sudo systemctl stop ceph-c63591ea-e57a-11ee-95c9-87774f69a715@mds.j 2024-03-18T23:21:47.740 INFO:journalctl@ceph.mds.j.smithi049.stdout:Mar 18 23:21:47 smithi049 systemd[1]: Stopping Ceph mds.j for c63591ea-e57a-11ee-95c9-87774f69a715... 2024-03-18T23:21:48.128 INFO:journalctl@ceph.mds.j.smithi049.stdout:Mar 18 23:21:47 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mds-j[103694]: 2024-03-18T23:21:47.740+0000 7f025619d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-18T23:21:48.129 INFO:journalctl@ceph.mds.j.smithi049.stdout:Mar 18 23:21:47 smithi049 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mds-j[103694]: 2024-03-18T23:21:47.740+0000 7f025619d700 -1 mds.j *** got signal Terminated *** 2024-03-18T23:21:51.556 INFO:journalctl@ceph.mds.j.smithi049.stdout:Mar 18 23:21:51 smithi049 podman[125118]: 2024-03-18 23:21:51.294047532 +0000 UTC m=+3.644313313 container died 3a80eb2dbc25fb605480d6a449fcd8920f7e591b8a886b0574b6414e817131f4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mds-j, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240318.181317-b6ae249) 2024-03-18T23:21:51.556 INFO:journalctl@ceph.mds.j.smithi049.stdout:Mar 18 23:21:51 smithi049 podman[125118]: 2024-03-18 23:21:51.557929091 +0000 UTC m=+3.908194879 container cleanup 3a80eb2dbc25fb605480d6a449fcd8920f7e591b8a886b0574b6414e817131f4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mds-j, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-18T23:21:51.815 INFO:journalctl@ceph.mds.j.smithi049.stdout:Mar 18 23:21:51 smithi049 bash[125118]: ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mds-j 2024-03-18T23:21:51.989 DEBUG:teuthology.orchestra.run.smithi049:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c63591ea-e57a-11ee-95c9-87774f69a715@mds.j.service' 2024-03-18T23:21:52.026 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-18T23:21:52.027 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-03-18T23:21:52.027 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-03-18T23:21:52.027 DEBUG:teuthology.orchestra.run.smithi139:> sudo systemctl stop ceph-c63591ea-e57a-11ee-95c9-87774f69a715@mds.b 2024-03-18T23:21:52.313 INFO:journalctl@ceph.mds.b.smithi139.stdout:Mar 18 23:21:52 smithi139 systemd[1]: Stopping Ceph mds.b for c63591ea-e57a-11ee-95c9-87774f69a715... 2024-03-18T23:21:52.814 INFO:journalctl@ceph.mds.b.smithi139.stdout:Mar 18 23:21:52 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mds-b[98118]: 2024-03-18T23:21:52.332+0000 7f567fec2700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-18T23:21:52.814 INFO:journalctl@ceph.mds.b.smithi139.stdout:Mar 18 23:21:52 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mds-b[98118]: 2024-03-18T23:21:52.332+0000 7f567fec2700 -1 mds.b *** got signal Terminated *** 2024-03-18T23:21:55.040 INFO:journalctl@ceph.mds.b.smithi139.stdout:Mar 18 23:21:54 smithi139 podman[105394]: 2024-03-18 23:21:54.772399605 +0000 UTC m=+2.551466062 container died 4049f303e6e80d5528fa9e74ea442226870ef3024fd6b1e47a51f8eb50475726 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mds-b, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-18T23:21:55.313 INFO:journalctl@ceph.mds.b.smithi139.stdout:Mar 18 23:21:55 smithi139 podman[105394]: 2024-03-18 23:21:55.165729034 +0000 UTC m=+2.944795488 container cleanup 4049f303e6e80d5528fa9e74ea442226870ef3024fd6b1e47a51f8eb50475726 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mds-b, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240307, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240318.181317-b6ae249) 2024-03-18T23:21:55.314 INFO:journalctl@ceph.mds.b.smithi139.stdout:Mar 18 23:21:55 smithi139 bash[105394]: ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mds-b 2024-03-18T23:21:55.592 DEBUG:teuthology.orchestra.run.smithi139:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c63591ea-e57a-11ee-95c9-87774f69a715@mds.b.service' 2024-03-18T23:21:55.629 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-18T23:21:55.629 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-03-18T23:21:55.629 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-03-18T23:21:55.630 DEBUG:teuthology.orchestra.run.smithi139:> sudo systemctl stop ceph-c63591ea-e57a-11ee-95c9-87774f69a715@mds.e 2024-03-18T23:21:56.063 INFO:journalctl@ceph.mds.e.smithi139.stdout:Mar 18 23:21:55 smithi139 systemd[1]: Stopping Ceph mds.e for c63591ea-e57a-11ee-95c9-87774f69a715... 2024-03-18T23:21:56.064 INFO:journalctl@ceph.mds.e.smithi139.stdout:Mar 18 23:21:55 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mds-e[97494]: 2024-03-18T23:21:55.952+0000 7f5903f90700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-18T23:21:56.064 INFO:journalctl@ceph.mds.e.smithi139.stdout:Mar 18 23:21:55 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mds-e[97494]: 2024-03-18T23:21:55.952+0000 7f5903f90700 -1 mds.e *** got signal Terminated *** 2024-03-18T23:21:59.035 INFO:journalctl@ceph.mds.e.smithi139.stdout:Mar 18 23:21:58 smithi139 podman[105516]: 2024-03-18 23:21:58.76806082 +0000 UTC m=+2.921836319 container died 0431e396f28373930b194f794f07e2a5e665813485416e809818c2560e941b03 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mds-e, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-18T23:21:59.313 INFO:journalctl@ceph.mds.e.smithi139.stdout:Mar 18 23:21:59 smithi139 podman[105516]: 2024-03-18 23:21:59.136565382 +0000 UTC m=+3.290340871 container cleanup 0431e396f28373930b194f794f07e2a5e665813485416e809818c2560e941b03 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mds-e, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-18T23:21:59.314 INFO:journalctl@ceph.mds.e.smithi139.stdout:Mar 18 23:21:59 smithi139 bash[105516]: ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mds-e 2024-03-18T23:21:59.555 DEBUG:teuthology.orchestra.run.smithi139:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c63591ea-e57a-11ee-95c9-87774f69a715@mds.e.service' 2024-03-18T23:21:59.592 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-18T23:21:59.593 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-03-18T23:21:59.593 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-03-18T23:21:59.593 DEBUG:teuthology.orchestra.run.smithi139:> sudo systemctl stop ceph-c63591ea-e57a-11ee-95c9-87774f69a715@mds.h 2024-03-18T23:21:59.930 INFO:journalctl@ceph.mds.h.smithi139.stdout:Mar 18 23:21:59 smithi139 systemd[1]: Stopping Ceph mds.h for c63591ea-e57a-11ee-95c9-87774f69a715... 2024-03-18T23:22:00.314 INFO:journalctl@ceph.mds.h.smithi139.stdout:Mar 18 23:21:59 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mds-h[97809]: 2024-03-18T23:21:59.930+0000 7f50edbc0700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-18T23:22:00.314 INFO:journalctl@ceph.mds.h.smithi139.stdout:Mar 18 23:21:59 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mds-h[97809]: 2024-03-18T23:21:59.930+0000 7f50edbc0700 -1 mds.h *** got signal Terminated *** 2024-03-18T23:22:03.063 INFO:journalctl@ceph.mds.h.smithi139.stdout:Mar 18 23:22:02 smithi139 podman[105649]: 2024-03-18 23:22:02.770859376 +0000 UTC m=+2.955515849 container died d7e908d6f0324b5c47ae347bcd61f9fba75f2019756c6d99ae067ba7602df7f3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mds-h, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240318.181317-b6ae249) 2024-03-18T23:22:03.381 INFO:journalctl@ceph.mds.h.smithi139.stdout:Mar 18 23:22:03 smithi139 podman[105649]: 2024-03-18 23:22:03.164624331 +0000 UTC m=+3.349280807 container cleanup d7e908d6f0324b5c47ae347bcd61f9fba75f2019756c6d99ae067ba7602df7f3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mds-h, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-18T23:22:03.381 INFO:journalctl@ceph.mds.h.smithi139.stdout:Mar 18 23:22:03 smithi139 bash[105649]: ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mds-h 2024-03-18T23:22:03.519 DEBUG:teuthology.orchestra.run.smithi139:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c63591ea-e57a-11ee-95c9-87774f69a715@mds.h.service' 2024-03-18T23:22:03.554 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-18T23:22:03.555 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-03-18T23:22:03.555 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-03-18T23:22:03.555 DEBUG:teuthology.orchestra.run.smithi139:> sudo systemctl stop ceph-c63591ea-e57a-11ee-95c9-87774f69a715@mds.k 2024-03-18T23:22:04.064 INFO:journalctl@ceph.mds.k.smithi139.stdout:Mar 18 23:22:03 smithi139 systemd[1]: Stopping Ceph mds.k for c63591ea-e57a-11ee-95c9-87774f69a715... 2024-03-18T23:22:04.064 INFO:journalctl@ceph.mds.k.smithi139.stdout:Mar 18 23:22:03 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mds-k[97173]: 2024-03-18T23:22:03.876+0000 7f8452559700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-18T23:22:04.064 INFO:journalctl@ceph.mds.k.smithi139.stdout:Mar 18 23:22:03 smithi139 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mds-k[97173]: 2024-03-18T23:22:03.876+0000 7f8452559700 -1 mds.k *** got signal Terminated *** 2024-03-18T23:22:07.026 INFO:journalctl@ceph.mds.k.smithi139.stdout:Mar 18 23:22:06 smithi139 podman[105771]: 2024-03-18 23:22:06.767202309 +0000 UTC m=+2.993466668 container died c4db76d479ee9b57345187e99a3d1ddeae05157f9deb754bd0e0a1c56dfff4d7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mds-k, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-03-18T23:22:07.314 INFO:journalctl@ceph.mds.k.smithi139.stdout:Mar 18 23:22:07 smithi139 podman[105771]: 2024-03-18 23:22:07.093226501 +0000 UTC m=+3.319490868 container cleanup c4db76d479ee9b57345187e99a3d1ddeae05157f9deb754bd0e0a1c56dfff4d7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mds-k, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-03-18T23:22:07.314 INFO:journalctl@ceph.mds.k.smithi139.stdout:Mar 18 23:22:07 smithi139 bash[105771]: ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mds-k 2024-03-18T23:22:07.314 INFO:journalctl@ceph.mds.k.smithi139.stdout:Mar 18 23:22:07 smithi139 podman[105785]: 2024-03-18 23:22:07.244173965 +0000 UTC m=+0.473476695 container remove c4db76d479ee9b57345187e99a3d1ddeae05157f9deb754bd0e0a1c56dfff4d7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mds-k, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS) 2024-03-18T23:22:07.479 DEBUG:teuthology.orchestra.run.smithi139:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c63591ea-e57a-11ee-95c9-87774f69a715@mds.k.service' 2024-03-18T23:22:07.513 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-18T23:22:07.514 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-03-18T23:22:07.514 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-03-18T23:22:07.514 DEBUG:teuthology.orchestra.run.smithi175:> sudo systemctl stop ceph-c63591ea-e57a-11ee-95c9-87774f69a715@mds.c 2024-03-18T23:22:07.905 INFO:journalctl@ceph.mds.c.smithi175.stdout:Mar 18 23:22:07 smithi175 systemd[1]: Stopping Ceph mds.c for c63591ea-e57a-11ee-95c9-87774f69a715... 2024-03-18T23:22:08.170 INFO:journalctl@ceph.mds.c.smithi175.stdout:Mar 18 23:22:07 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mds-c[98512]: 2024-03-18T23:22:07.906+0000 7fb345917700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-18T23:22:08.170 INFO:journalctl@ceph.mds.c.smithi175.stdout:Mar 18 23:22:07 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mds-c[98512]: 2024-03-18T23:22:07.906+0000 7fb345917700 -1 mds.c *** got signal Terminated *** 2024-03-18T23:22:11.089 INFO:journalctl@ceph.mds.c.smithi175.stdout:Mar 18 23:22:10 smithi175 podman[105390]: 2024-03-18 23:22:10.776847316 +0000 UTC m=+3.069881834 container died 1bbba6b309808fbcfb1d3c2ab36f5c9cbe70bc6bcb7509b430858c581c14fd6c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mds-c, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-03-18T23:22:11.430 INFO:journalctl@ceph.mds.c.smithi175.stdout:Mar 18 23:22:11 smithi175 podman[105390]: 2024-03-18 23:22:11.181288207 +0000 UTC m=+3.474322722 container cleanup 1bbba6b309808fbcfb1d3c2ab36f5c9cbe70bc6bcb7509b430858c581c14fd6c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mds-c, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.build-date=20240307) 2024-03-18T23:22:11.430 INFO:journalctl@ceph.mds.c.smithi175.stdout:Mar 18 23:22:11 smithi175 bash[105390]: ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mds-c 2024-03-18T23:22:11.603 DEBUG:teuthology.orchestra.run.smithi175:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c63591ea-e57a-11ee-95c9-87774f69a715@mds.c.service' 2024-03-18T23:22:11.643 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-18T23:22:11.644 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-03-18T23:22:11.644 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-03-18T23:22:11.644 DEBUG:teuthology.orchestra.run.smithi175:> sudo systemctl stop ceph-c63591ea-e57a-11ee-95c9-87774f69a715@mds.f 2024-03-18T23:22:11.960 INFO:journalctl@ceph.mds.f.smithi175.stdout:Mar 18 23:22:11 smithi175 systemd[1]: Stopping Ceph mds.f for c63591ea-e57a-11ee-95c9-87774f69a715... 2024-03-18T23:22:11.960 INFO:journalctl@ceph.mds.f.smithi175.stdout:Mar 18 23:22:11 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mds-f[98204]: 2024-03-18T23:22:11.951+0000 7fcfb692b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-18T23:22:11.961 INFO:journalctl@ceph.mds.f.smithi175.stdout:Mar 18 23:22:11 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mds-f[98204]: 2024-03-18T23:22:11.951+0000 7fcfb692b700 -1 mds.f *** got signal Terminated *** 2024-03-18T23:22:15.058 INFO:journalctl@ceph.mds.f.smithi175.stdout:Mar 18 23:22:14 smithi175 podman[105522]: 2024-03-18 23:22:14.77035526 +0000 UTC m=+2.909815396 container died 66fdf9cee264975aed14cbe4703a2f040899e4bab87044a00970c629e19fa602 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mds-f, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, io.buildah.version=1.33.5) 2024-03-18T23:22:15.383 INFO:journalctl@ceph.mds.f.smithi175.stdout:Mar 18 23:22:15 smithi175 podman[105522]: 2024-03-18 23:22:15.151385707 +0000 UTC m=+3.290845837 container cleanup 66fdf9cee264975aed14cbe4703a2f040899e4bab87044a00970c629e19fa602 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mds-f, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-03-18T23:22:15.384 INFO:journalctl@ceph.mds.f.smithi175.stdout:Mar 18 23:22:15 smithi175 bash[105522]: ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mds-f 2024-03-18T23:22:15.521 DEBUG:teuthology.orchestra.run.smithi175:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c63591ea-e57a-11ee-95c9-87774f69a715@mds.f.service' 2024-03-18T23:22:15.556 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-18T23:22:15.556 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-03-18T23:22:15.557 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-03-18T23:22:15.557 DEBUG:teuthology.orchestra.run.smithi175:> sudo systemctl stop ceph-c63591ea-e57a-11ee-95c9-87774f69a715@mds.i 2024-03-18T23:22:15.960 INFO:journalctl@ceph.mds.i.smithi175.stdout:Mar 18 23:22:15 smithi175 systemd[1]: Stopping Ceph mds.i for c63591ea-e57a-11ee-95c9-87774f69a715... 2024-03-18T23:22:15.960 INFO:journalctl@ceph.mds.i.smithi175.stdout:Mar 18 23:22:15 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mds-i[98824]: 2024-03-18T23:22:15.861+0000 7f6f77565700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-18T23:22:15.960 INFO:journalctl@ceph.mds.i.smithi175.stdout:Mar 18 23:22:15 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mds-i[98824]: 2024-03-18T23:22:15.861+0000 7f6f77565700 -1 mds.i *** got signal Terminated *** 2024-03-18T23:22:18.960 INFO:journalctl@ceph.mds.i.smithi175.stdout:Mar 18 23:22:18 smithi175 podman[105644]: 2024-03-18 23:22:18.762035397 +0000 UTC m=+2.991549982 container died c9b962415920d774ae499323e32d191311f2ef10d7277ba5f771fd78b52a58ae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mds-i, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.vendor=CentOS) 2024-03-18T23:22:19.412 INFO:journalctl@ceph.mds.i.smithi175.stdout:Mar 18 23:22:19 smithi175 podman[105644]: 2024-03-18 23:22:19.139683375 +0000 UTC m=+3.369197963 container cleanup c9b962415920d774ae499323e32d191311f2ef10d7277ba5f771fd78b52a58ae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mds-i, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-18T23:22:19.412 INFO:journalctl@ceph.mds.i.smithi175.stdout:Mar 18 23:22:19 smithi175 bash[105644]: ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mds-i 2024-03-18T23:22:19.549 DEBUG:teuthology.orchestra.run.smithi175:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c63591ea-e57a-11ee-95c9-87774f69a715@mds.i.service' 2024-03-18T23:22:19.583 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-18T23:22:19.583 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-03-18T23:22:19.584 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-03-18T23:22:19.584 DEBUG:teuthology.orchestra.run.smithi175:> sudo systemctl stop ceph-c63591ea-e57a-11ee-95c9-87774f69a715@mds.l 2024-03-18T23:22:19.710 INFO:journalctl@ceph.mds.l.smithi175.stdout:Mar 18 23:22:19 smithi175 systemd[1]: Stopping Ceph mds.l for c63591ea-e57a-11ee-95c9-87774f69a715... 2024-03-18T23:22:20.210 INFO:journalctl@ceph.mds.l.smithi175.stdout:Mar 18 23:22:19 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mds-l[97880]: 2024-03-18T23:22:19.873+0000 7fd4aa44f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-18T23:22:20.210 INFO:journalctl@ceph.mds.l.smithi175.stdout:Mar 18 23:22:19 smithi175 ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mds-l[97880]: 2024-03-18T23:22:19.873+0000 7fd4aa44f700 -1 mds.l *** got signal Terminated *** 2024-03-18T23:22:23.048 INFO:journalctl@ceph.mds.l.smithi175.stdout:Mar 18 23:22:22 smithi175 podman[105766]: 2024-03-18 23:22:22.784219758 +0000 UTC m=+2.986770988 container died db3bb4b298f0fc3a888f37045f9117bd423f9f97f2451e8027f4aad3e39ef6c1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mds-l, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-18T23:22:23.299 INFO:journalctl@ceph.mds.l.smithi175.stdout:Mar 18 23:22:23 smithi175 podman[105766]: 2024-03-18 23:22:23.124306592 +0000 UTC m=+3.326857827 container cleanup db3bb4b298f0fc3a888f37045f9117bd423f9f97f2451e8027f4aad3e39ef6c1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mds-l, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-18T23:22:23.300 INFO:journalctl@ceph.mds.l.smithi175.stdout:Mar 18 23:22:23 smithi175 bash[105766]: ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mds-l 2024-03-18T23:22:23.300 INFO:journalctl@ceph.mds.l.smithi175.stdout:Mar 18 23:22:23 smithi175 podman[105781]: 2024-03-18 23:22:23.300211394 +0000 UTC m=+0.518947860 container remove db3bb4b298f0fc3a888f37045f9117bd423f9f97f2451e8027f4aad3e39ef6c1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-c63591ea-e57a-11ee-95c9-87774f69a715-mds-l, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.schema-version=1.0) 2024-03-18T23:22:23.541 DEBUG:teuthology.orchestra.run.smithi175:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c63591ea-e57a-11ee-95c9-87774f69a715@mds.l.service' 2024-03-18T23:22:23.564 INFO:journalctl@ceph.mds.l.smithi175.stdout:Mar 18 23:22:23 smithi175 systemd[1]: ceph-c63591ea-e57a-11ee-95c9-87774f69a715@mds.l.service: Deactivated successfully. 2024-03-18T23:22:23.565 INFO:journalctl@ceph.mds.l.smithi175.stdout:Mar 18 23:22:23 smithi175 systemd[1]: Stopped Ceph mds.l for c63591ea-e57a-11ee-95c9-87774f69a715. 2024-03-18T23:22:23.565 INFO:journalctl@ceph.mds.l.smithi175.stdout:Mar 18 23:22:23 smithi175 systemd[1]: ceph-c63591ea-e57a-11ee-95c9-87774f69a715@mds.l.service: Consumed 5min 2.550s CPU time. 2024-03-18T23:22:23.576 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-18T23:22:23.576 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-03-18T23:22:23.577 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid c63591ea-e57a-11ee-95c9-87774f69a715 --force --keep-logs 2024-03-18T23:22:23.779 INFO:teuthology.orchestra.run.smithi049.stdout:Deleting cluster with fsid: c63591ea-e57a-11ee-95c9-87774f69a715 2024-03-18T23:22:26.263 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid c63591ea-e57a-11ee-95c9-87774f69a715 --force --keep-logs 2024-03-18T23:22:26.461 INFO:teuthology.orchestra.run.smithi139.stdout:Deleting cluster with fsid: c63591ea-e57a-11ee-95c9-87774f69a715 2024-03-18T23:22:29.014 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid c63591ea-e57a-11ee-95c9-87774f69a715 --force --keep-logs 2024-03-18T23:22:29.203 INFO:teuthology.orchestra.run.smithi175.stdout:Deleting cluster with fsid: c63591ea-e57a-11ee-95c9-87774f69a715 2024-03-18T23:22:31.631 DEBUG:teuthology.orchestra.run.smithi049:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-18T23:22:31.671 DEBUG:teuthology.orchestra.run.smithi139:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-18T23:22:31.705 DEBUG:teuthology.orchestra.run.smithi175:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-18T23:22:31.733 INFO:tasks.cephadm:Archiving crash dumps... 2024-03-18T23:22:31.736 DEBUG:teuthology.misc:Transferring archived files from smithi049:/var/lib/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/crash to /home/teuthworker/archive/pdonnell-2024-03-18_21:05:49-fs-wip-batrick-testing-20240318.181317-distro-default-smithi/7609140/remote/smithi049/crash 2024-03-18T23:22:31.736 DEBUG:teuthology.orchestra.run.smithi049:> sudo tar c -f - -C /var/lib/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/crash -- . 2024-03-18T23:22:31.785 INFO:teuthology.orchestra.run.smithi049.stderr:tar: /var/lib/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/crash: Cannot open: No such file or directory 2024-03-18T23:22:31.785 INFO:teuthology.orchestra.run.smithi049.stderr:tar: Error is not recoverable: exiting now 2024-03-18T23:22:31.787 DEBUG:teuthology.misc:Transferring archived files from smithi139:/var/lib/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/crash to /home/teuthworker/archive/pdonnell-2024-03-18_21:05:49-fs-wip-batrick-testing-20240318.181317-distro-default-smithi/7609140/remote/smithi139/crash 2024-03-18T23:22:31.788 DEBUG:teuthology.orchestra.run.smithi139:> sudo tar c -f - -C /var/lib/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/crash -- . 2024-03-18T23:22:31.829 INFO:teuthology.orchestra.run.smithi139.stderr:tar: /var/lib/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/crash: Cannot open: No such file or directory 2024-03-18T23:22:31.830 INFO:teuthology.orchestra.run.smithi139.stderr:tar: Error is not recoverable: exiting now 2024-03-18T23:22:31.831 DEBUG:teuthology.misc:Transferring archived files from smithi175:/var/lib/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/crash to /home/teuthworker/archive/pdonnell-2024-03-18_21:05:49-fs-wip-batrick-testing-20240318.181317-distro-default-smithi/7609140/remote/smithi175/crash 2024-03-18T23:22:31.832 DEBUG:teuthology.orchestra.run.smithi175:> sudo tar c -f - -C /var/lib/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/crash -- . 2024-03-18T23:22:31.868 INFO:teuthology.orchestra.run.smithi175.stderr:tar: /var/lib/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/crash: Cannot open: No such file or directory 2024-03-18T23:22:31.868 INFO:teuthology.orchestra.run.smithi175.stderr:tar: Error is not recoverable: exiting now 2024-03-18T23:22:31.868 INFO:tasks.cephadm:Checking cluster log for badness... 2024-03-18T23:22:31.869 DEBUG:teuthology.orchestra.run.smithi049:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | head -n 1 2024-03-18T23:22:31.908 INFO:teuthology.orchestra.run.smithi049.stdout:2024-03-18T23:00:02.255078+0000 mon.a (mon.0) 210 : cluster [WRN] Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2024-03-18T23:22:31.909 WARNING:tasks.cephadm:Found errors (ERR|WRN|SEC) in cluster log 2024-03-18T23:22:31.909 INFO:tasks.cephadm:Compressing logs... 2024-03-18T23:22:31.909 DEBUG:teuthology.orchestra.run.smithi049:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-03-18T23:22:31.953 DEBUG:teuthology.orchestra.run.smithi139:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-03-18T23:22:31.955 DEBUG:teuthology.orchestra.run.smithi175:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-03-18T23:22:31.984 INFO:teuthology.orchestra.run.smithi175.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-03-18T23:22:31.984 INFO:teuthology.orchestra.run.smithi175.stderr:: No such file or directory 2024-03-18T23:22:31.984 INFO:teuthology.orchestra.run.smithi175.stderr:gzip -5 --verbose -- /var/log/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/ceph-mds.l.log 2024-03-18T23:22:31.985 INFO:teuthology.orchestra.run.smithi175.stderr:gzip -5 --verbose -- /var/log/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/ceph.audit.log 2024-03-18T23:22:31.985 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/ceph-mds.i.log 2024-03-18T23:22:31.985 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/ceph-mds.l.log: /var/log/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/ceph-osd.11.log 2024-03-18T23:22:31.985 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/ceph-mds.i.log: gzip -5 --verbose -- /var/log/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/ceph-mgr.z.log 2024-03-18T23:22:31.985 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/ceph-osd.8.log 2024-03-18T23:22:31.985 INFO:teuthology.orchestra.run.smithi175.stderr: 92.5% -- replaced with /var/log/ceph/cephadm.log.gz 2024-03-18T23:22:31.985 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/ceph-mgr.z.log: gzip -5 --verbose -- /var/log/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/ceph-osd.5.log 2024-03-18T23:22:31.986 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/ceph-mds.f.log 2024-03-18T23:22:31.986 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/ceph.cephadm.log 2024-03-18T23:22:31.994 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/ceph-mds.f.log: 90.5% -- replaced with /var/log/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/ceph.audit.log.gz 2024-03-18T23:22:31.994 INFO:teuthology.orchestra.run.smithi175.stderr:gzip -5 --verbose -- /var/log/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/ceph.log 2024-03-18T23:22:31.994 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/ceph.cephadm.log: 84.0% -- replaced with /var/log/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/ceph.cephadm.log.gz 2024-03-18T23:22:31.994 INFO:teuthology.orchestra.run.smithi175.stderr:gzip -5 --verbose -- /var/log/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/ceph-osd.2.log 2024-03-18T23:22:31.995 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/ceph.log: 86.6% -- replaced with /var/log/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/ceph.log.gz 2024-03-18T23:22:31.995 INFO:teuthology.orchestra.run.smithi175.stderr:gzip -5 --verbose -- /var/log/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/ceph-mds.c.log 2024-03-18T23:22:31.995 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/ceph-osd.2.log: 91.9% -- replaced with /var/log/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/ceph-mgr.z.log.gz 2024-03-18T23:22:31.995 INFO:teuthology.orchestra.run.smithi175.stderr:gzip -5 --verbose -- /var/log/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/ceph-volume.log 2024-03-18T23:22:31.995 INFO:teuthology.orchestra.run.smithi049.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log: No such file or directory 2024-03-18T23:22:31.995 INFO:teuthology.orchestra.run.smithi049.stderr: 2024-03-18T23:22:31.996 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/ceph-osd.6.log 2024-03-18T23:22:31.996 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/ceph-mon.a.log 2024-03-18T23:22:31.996 INFO:teuthology.orchestra.run.smithi049.stderr:gzip/var/log/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/ceph-osd.6.log: -5 --verbose -- /var/log/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/ceph.audit.log 2024-03-18T23:22:31.996 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/ceph-mds.g.log 2024-03-18T23:22:31.996 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/ceph-mds.j.log 2024-03-18T23:22:31.996 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/ceph.audit.log: /var/log/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/ceph-osd.3.log 2024-03-18T23:22:31.996 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/ceph-mds.j.log: gzip -5 --verbose -- /var/log/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/ceph-mgr.x.log 2024-03-18T23:22:31.997 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/ceph-osd.0.log 2024-03-18T23:22:31.997 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/ceph-mgr.x.log: 90.3% -- replaced with /var/log/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/ceph.audit.log.gz 2024-03-18T23:22:31.997 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/ceph.cephadm.log 2024-03-18T23:22:31.997 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/ceph-osd.0.log: 94.7% -- replaced with /var/log/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/ceph-mds.j.log.gz 2024-03-18T23:22:31.997 INFO:teuthology.orchestra.run.smithi049.stderr: 93.3% -- replaced with /var/log/ceph/cephadm.log.gz 2024-03-18T23:22:32.003 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/ceph.log 2024-03-18T23:22:32.003 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/ceph-volume.log 2024-03-18T23:22:32.003 INFO:teuthology.orchestra.run.smithi049.stderr: 84.3% -- replaced with /var/log/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/ceph.cephadm.log.gz/var/log/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/ceph.log: 2024-03-18T23:22:32.003 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/ceph-osd.9.log 2024-03-18T23:22:32.004 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/ceph-mon.c.log 2024-03-18T23:22:32.005 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/ceph-volume.log: 86.6% -- replaced with /var/log/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/ceph.log.gz 2024-03-18T23:22:32.006 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/ceph-mds.d.log 2024-03-18T23:22:32.019 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/ceph-osd.9.log: 91.0% -- replaced with /var/log/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/ceph-volume.log.gz 2024-03-18T23:22:32.019 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/ceph-mds.a.log 2024-03-18T23:22:32.024 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/ceph-mds.d.log: 94.5% -- replaced with /var/log/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/ceph-mds.d.log.gz 2024-03-18T23:22:32.042 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/ceph-volume.log: /var/log/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/ceph-mon.c.log: 90.9% -- replaced with /var/log/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/ceph-volume.log.gz 2024-03-18T23:22:32.047 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/ceph-mds.a.log: 94.5% -- replaced with /var/log/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/ceph-mds.g.log.gz 2024-03-18T23:22:32.059 INFO:teuthology.orchestra.run.smithi175.stderr: 94.6% -- replaced with /var/log/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/ceph-mds.c.log.gz 2024-03-18T23:22:32.069 INFO:teuthology.orchestra.run.smithi139.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-03-18T23:22:32.069 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-03-18T23:22:32.069 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/ceph-osd.10.log 2024-03-18T23:22:32.070 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/ceph.audit.log 2024-03-18T23:22:32.070 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/ceph-mds.b.log 2024-03-18T23:22:32.070 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/ceph-osd.4.log 2024-03-18T23:22:32.071 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/ceph.audit.log: /var/log/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/ceph-osd.1.log 2024-03-18T23:22:32.074 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/ceph-mds.k.log 2024-03-18T23:22:32.074 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/ceph-mds.h.log 2024-03-18T23:22:32.074 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/ceph-mds.k.log: gzip -5 --verbose -- /var/log/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/ceph-osd.7.log 2024-03-18T23:22:32.075 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/ceph-mds.h.log: gzip -5 --verbose -- /var/log/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/ceph-mds.e.log 2024-03-18T23:22:32.075 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/ceph-mgr.y.log 2024-03-18T23:22:32.075 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/ceph-mds.e.log: gzip -5 --verbose -- /var/log/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/ceph.cephadm.log 2024-03-18T23:22:32.075 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/ceph.log 2024-03-18T23:22:32.075 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/ceph-volume.log 2024-03-18T23:22:32.075 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/ceph.log: gzip -5 --verbose -- /var/log/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/ceph-mon.b.log 2024-03-18T23:22:32.078 INFO:teuthology.orchestra.run.smithi049.stderr: 94.5% -- replaced with /var/log/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/ceph-mds.a.log.gz 2024-03-18T23:22:32.079 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/ceph-volume.log: /var/log/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/ceph-mon.b.log: 91.8% -- replaced with /var/log/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/ceph-mgr.y.log.gz 2024-03-18T23:22:32.232 INFO:teuthology.orchestra.run.smithi139.stderr: 86.6% -- replaced with /var/log/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/ceph.log.gz 2024-03-18T23:22:32.241 INFO:teuthology.orchestra.run.smithi139.stderr: 92.1% -- replaced with /var/log/ceph/cephadm.log.gz 2024-03-18T23:22:32.242 INFO:teuthology.orchestra.run.smithi139.stderr: 84.1% -- replaced with /var/log/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/ceph.cephadm.log.gz 2024-03-18T23:22:32.248 INFO:teuthology.orchestra.run.smithi139.stderr: 90.4% -- replaced with /var/log/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/ceph.audit.log.gz 2024-03-18T23:22:32.273 INFO:teuthology.orchestra.run.smithi139.stderr: 93.6% -- replaced with /var/log/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/ceph-mds.k.log.gz 2024-03-18T23:22:32.323 INFO:teuthology.orchestra.run.smithi049.stderr: 89.7% -- replaced with /var/log/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/ceph-mgr.x.log.gz 2024-03-18T23:22:32.480 INFO:teuthology.orchestra.run.smithi175.stderr: 91.8% -- replaced with /var/log/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/ceph-mon.c.log.gz 2024-03-18T23:22:33.106 INFO:teuthology.orchestra.run.smithi139.stderr: 91.0% -- replaced with /var/log/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/ceph-volume.log.gz 2024-03-18T23:22:33.215 INFO:teuthology.orchestra.run.smithi049.stderr: 90.3% -- replaced with /var/log/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/ceph-mon.a.log.gz 2024-03-18T23:22:34.547 INFO:teuthology.orchestra.run.smithi139.stderr: 92.9% -- replaced with /var/log/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/ceph-mon.b.log.gz 2024-03-18T23:22:36.028 INFO:teuthology.orchestra.run.smithi175.stderr: 96.9% -- replaced with /var/log/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/ceph-mds.l.log.gz 2024-03-18T23:22:38.381 INFO:teuthology.orchestra.run.smithi175.stderr: 96.6% -- replaced with /var/log/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/ceph-mds.i.log.gz 2024-03-18T23:22:38.570 INFO:teuthology.orchestra.run.smithi175.stderr: 96.6% -- replaced with /var/log/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/ceph-mds.f.log.gz 2024-03-18T23:22:43.213 INFO:teuthology.orchestra.run.smithi049.stderr: 91.5% -- replaced with /var/log/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/ceph-osd.9.log.gz 2024-03-18T23:22:46.380 INFO:teuthology.orchestra.run.smithi049.stderr: 91.5% -- replaced with /var/log/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/ceph-osd.6.log.gz 2024-03-18T23:22:46.607 INFO:teuthology.orchestra.run.smithi175.stderr: 91.2% -- replaced with /var/log/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/ceph-osd.11.log.gz 2024-03-18T23:22:46.745 INFO:teuthology.orchestra.run.smithi049.stderr: 91.9% -- replaced with /var/log/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/ceph-osd.0.log.gz 2024-03-18T23:22:47.976 INFO:teuthology.orchestra.run.smithi175.stderr: 91.6% -- replaced with /var/log/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/ceph-osd.2.log.gz 2024-03-18T23:22:48.518 INFO:teuthology.orchestra.run.smithi175.stderr: 91.4% -- replaced with /var/log/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/ceph-osd.8.log.gz 2024-03-18T23:22:48.754 INFO:teuthology.orchestra.run.smithi049.stderr: 91.5% -- replaced with /var/log/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/ceph-osd.3.log.gz 2024-03-18T23:22:48.756 INFO:teuthology.orchestra.run.smithi049.stderr: 2024-03-18T23:22:48.757 INFO:teuthology.orchestra.run.smithi049.stderr:real 0m16.791s 2024-03-18T23:22:48.757 INFO:teuthology.orchestra.run.smithi049.stderr:user 0m57.004s 2024-03-18T23:22:48.757 INFO:teuthology.orchestra.run.smithi049.stderr:sys 0m1.686s 2024-03-18T23:22:52.079 INFO:teuthology.orchestra.run.smithi175.stderr: 91.2% -- replaced with /var/log/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/ceph-osd.5.log.gz 2024-03-18T23:22:52.081 INFO:teuthology.orchestra.run.smithi175.stderr: 2024-03-18T23:22:52.081 INFO:teuthology.orchestra.run.smithi175.stderr:real 0m20.112s 2024-03-18T23:22:52.081 INFO:teuthology.orchestra.run.smithi175.stderr:user 1m22.023s 2024-03-18T23:22:52.082 INFO:teuthology.orchestra.run.smithi175.stderr:sys 0m2.610s 2024-03-18T23:23:15.798 INFO:teuthology.orchestra.run.smithi139.stderr: 96.6% -- replaced with /var/log/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/ceph-mds.e.log.gz 2024-03-18T23:23:16.469 INFO:teuthology.orchestra.run.smithi139.stderr: 91.5% -- replaced with /var/log/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/ceph-osd.4.log.gz 2024-03-18T23:23:18.890 INFO:teuthology.orchestra.run.smithi139.stderr: 95.3% -- replaced with /var/log/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/ceph-mds.h.log.gz 2024-03-18T23:23:21.479 INFO:teuthology.orchestra.run.smithi139.stderr: 91.8% -- replaced with /var/log/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/ceph-osd.1.log.gz 2024-03-18T23:23:27.958 INFO:teuthology.orchestra.run.smithi139.stderr: 91.2% -- replaced with /var/log/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/ceph-osd.10.log.gz 2024-03-18T23:23:35.033 INFO:teuthology.orchestra.run.smithi139.stderr: 91.3% -- replaced with /var/log/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/ceph-osd.7.log.gz 2024-03-18T23:26:56.006 INFO:teuthology.orchestra.run.smithi139.stderr: 93.6% -- replaced with /var/log/ceph/c63591ea-e57a-11ee-95c9-87774f69a715/ceph-mds.b.log.gz 2024-03-18T23:26:56.033 INFO:teuthology.orchestra.run.smithi139.stderr: 2024-03-18T23:26:56.033 INFO:teuthology.orchestra.run.smithi139.stderr:real 4m24.064s 2024-03-18T23:26:56.033 INFO:teuthology.orchestra.run.smithi139.stderr:user 4m35.453s 2024-03-18T23:26:56.033 INFO:teuthology.orchestra.run.smithi139.stderr:sys 0m9.781s 2024-03-18T23:26:56.034 INFO:tasks.cephadm:Archiving logs... 2024-03-18T23:26:56.034 DEBUG:teuthology.misc:Transferring archived files from smithi049:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-03-18_21:05:49-fs-wip-batrick-testing-20240318.181317-distro-default-smithi/7609140/remote/smithi049/log 2024-03-18T23:26:56.035 DEBUG:teuthology.orchestra.run.smithi049:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-18T23:27:05.443 DEBUG:teuthology.misc:Transferring archived files from smithi139:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-03-18_21:05:49-fs-wip-batrick-testing-20240318.181317-distro-default-smithi/7609140/remote/smithi139/log 2024-03-18T23:27:05.444 DEBUG:teuthology.orchestra.run.smithi139:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-18T23:27:49.697 DEBUG:teuthology.misc:Transferring archived files from smithi175:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-03-18_21:05:49-fs-wip-batrick-testing-20240318.181317-distro-default-smithi/7609140/remote/smithi175/log 2024-03-18T23:27:49.699 DEBUG:teuthology.orchestra.run.smithi175:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-18T23:28:01.383 INFO:tasks.cephadm:Removing cluster... 2024-03-18T23:28:01.383 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid c63591ea-e57a-11ee-95c9-87774f69a715 --force 2024-03-18T23:28:01.590 INFO:teuthology.orchestra.run.smithi049.stdout:Deleting cluster with fsid: c63591ea-e57a-11ee-95c9-87774f69a715 2024-03-18T23:28:01.940 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid c63591ea-e57a-11ee-95c9-87774f69a715 --force 2024-03-18T23:28:02.286 INFO:teuthology.orchestra.run.smithi139.stdout:Deleting cluster with fsid: c63591ea-e57a-11ee-95c9-87774f69a715 2024-03-18T23:28:02.862 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid c63591ea-e57a-11ee-95c9-87774f69a715 --force 2024-03-18T23:28:03.053 INFO:teuthology.orchestra.run.smithi175.stdout:Deleting cluster with fsid: c63591ea-e57a-11ee-95c9-87774f69a715 2024-03-18T23:28:03.404 INFO:tasks.cephadm:Removing cephadm ... 2024-03-18T23:28:03.404 DEBUG:teuthology.orchestra.run.smithi049:> rm -rf /home/ubuntu/cephtest/cephadm 2024-03-18T23:28:03.423 DEBUG:teuthology.orchestra.run.smithi139:> rm -rf /home/ubuntu/cephtest/cephadm 2024-03-18T23:28:03.439 DEBUG:teuthology.orchestra.run.smithi175:> rm -rf /home/ubuntu/cephtest/cephadm 2024-03-18T23:28:03.455 INFO:tasks.cephadm:Teardown complete 2024-03-18T23:28:03.455 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-03-18T23:28:03.466 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-03-18T23:28:03.466 DEBUG:teuthology.orchestra.run.smithi049:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-18T23:28:03.469 DEBUG:teuthology.orchestra.run.smithi139:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-18T23:28:03.482 DEBUG:teuthology.orchestra.run.smithi175:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-18T23:28:03.598 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-03-18T23:28:03.608 INFO:teuthology.task.clock:Checking final clock skew... 2024-03-18T23:28:03.608 DEBUG:teuthology.orchestra.run.smithi049:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-18T23:28:03.611 DEBUG:teuthology.orchestra.run.smithi139:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-18T23:28:03.614 DEBUG:teuthology.orchestra.run.smithi175:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-18T23:28:03.625 INFO:teuthology.orchestra.run.smithi049.stderr:bash: line 1: ntpq: command not found 2024-03-18T23:28:03.626 INFO:teuthology.orchestra.run.smithi139.stderr:bash: line 1: ntpq: command not found 2024-03-18T23:28:03.627 INFO:teuthology.orchestra.run.smithi175.stderr:bash: line 1: ntpq: command not found 2024-03-18T23:28:03.630 INFO:teuthology.orchestra.run.smithi049.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-18T23:28:03.630 INFO:teuthology.orchestra.run.smithi049.stdout:=============================================================================== 2024-03-18T23:28:03.630 INFO:teuthology.orchestra.run.smithi049.stdout:^+ hv01.front.sepia.ceph.com 2 7 377 127 +1415us[+1415us] +/- 124ms 2024-03-18T23:28:03.630 INFO:teuthology.orchestra.run.smithi049.stdout:^* hv02.front.sepia.ceph.com 3 8 377 249 -164us[ -182us] +/- 55ms 2024-03-18T23:28:03.630 INFO:teuthology.orchestra.run.smithi049.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 61 -539us[ -539us] +/- 66ms 2024-03-18T23:28:03.630 INFO:teuthology.orchestra.run.smithi049.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2024-03-18T23:28:03.632 INFO:teuthology.orchestra.run.smithi175.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-18T23:28:03.632 INFO:teuthology.orchestra.run.smithi175.stdout:=============================================================================== 2024-03-18T23:28:03.632 INFO:teuthology.orchestra.run.smithi175.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 62 +1395us[+1395us] +/- 125ms 2024-03-18T23:28:03.632 INFO:teuthology.orchestra.run.smithi175.stdout:^* hv02.front.sepia.ceph.com 3 6 377 64 -108us[ -73us] +/- 58ms 2024-03-18T23:28:03.632 INFO:teuthology.orchestra.run.smithi175.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 63 -510us[ -510us] +/- 65ms 2024-03-18T23:28:03.632 INFO:teuthology.orchestra.run.smithi175.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2024-03-18T23:28:03.666 INFO:teuthology.orchestra.run.smithi139.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-18T23:28:03.666 INFO:teuthology.orchestra.run.smithi139.stdout:=============================================================================== 2024-03-18T23:28:03.666 INFO:teuthology.orchestra.run.smithi139.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 63 +1366us[+1394us] +/- 125ms 2024-03-18T23:28:03.666 INFO:teuthology.orchestra.run.smithi139.stdout:^* hv02.front.sepia.ceph.com 3 6 377 62 -130us[ -102us] +/- 58ms 2024-03-18T23:28:03.666 INFO:teuthology.orchestra.run.smithi139.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 60 -533us[ -533us] +/- 65ms 2024-03-18T23:28:03.666 INFO:teuthology.orchestra.run.smithi139.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2024-03-18T23:28:03.675 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-03-18T23:28:03.685 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-03-18T23:28:03.685 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-03-18T23:28:03.709 DEBUG:teuthology.orchestra.run.smithi049:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-03-18T23:28:03.803 DEBUG:teuthology.orchestra.run.smithi139:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-03-18T23:28:03.900 DEBUG:teuthology.orchestra.run.smithi175:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-03-18T23:28:03.971 DEBUG:teuthology.orchestra.run.smithi049:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-18T23:28:04.006 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-18T23:28:04.006 DEBUG:teuthology.orchestra.run.smithi139:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-18T23:28:04.036 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-18T23:28:04.037 DEBUG:teuthology.orchestra.run.smithi175:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-18T23:28:04.066 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-18T23:28:04.067 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-03-18T23:28:04.078 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-03-18T23:28:04.133 INFO:teuthology.task.internal:Duration was 5399.131583 seconds 2024-03-18T23:28:04.133 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-03-18T23:28:04.169 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-03-18T23:28:04.169 DEBUG:teuthology.orchestra.run.smithi049:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-18T23:28:04.172 DEBUG:teuthology.orchestra.run.smithi139:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-18T23:28:04.174 DEBUG:teuthology.orchestra.run.smithi175:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-18T23:28:04.217 INFO:teuthology.orchestra.run.smithi175.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-18T23:28:04.222 INFO:teuthology.orchestra.run.smithi049.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-18T23:28:04.236 INFO:teuthology.orchestra.run.smithi139.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-18T23:28:04.687 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-03-18T23:28:04.687 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi049.front.sepia.ceph.com 2024-03-18T23:28:04.687 DEBUG:teuthology.orchestra.run.smithi049:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-18T23:28:04.711 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi139.front.sepia.ceph.com 2024-03-18T23:28:04.711 DEBUG:teuthology.orchestra.run.smithi139:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-18T23:28:04.745 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi175.front.sepia.ceph.com 2024-03-18T23:28:04.746 DEBUG:teuthology.orchestra.run.smithi175:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-18T23:28:04.778 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-03-18T23:28:04.778 DEBUG:teuthology.orchestra.run.smithi049:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-18T23:28:04.780 DEBUG:teuthology.orchestra.run.smithi139:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-18T23:28:04.788 DEBUG:teuthology.orchestra.run.smithi175:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-18T23:28:04.847 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-03-18T23:28:04.847 DEBUG:teuthology.orchestra.run.smithi049:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-18T23:28:04.854 DEBUG:teuthology.orchestra.run.smithi139:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-18T23:28:04.857 DEBUG:teuthology.orchestra.run.smithi175:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-18T23:28:05.253 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-03-18T23:28:05.265 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-03-18T23:28:05.265 DEBUG:teuthology.orchestra.run.smithi049:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-18T23:28:05.296 DEBUG:teuthology.orchestra.run.smithi139:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-18T23:28:05.338 DEBUG:teuthology.orchestra.run.smithi175:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-18T23:28:05.366 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-03-18T23:28:05.378 DEBUG:teuthology.orchestra.run.smithi049:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-18T23:28:05.380 DEBUG:teuthology.orchestra.run.smithi139:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-18T23:28:05.382 DEBUG:teuthology.orchestra.run.smithi175:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-18T23:28:05.407 INFO:teuthology.orchestra.run.smithi049.stdout:kernel.core_pattern = core 2024-03-18T23:28:05.412 INFO:teuthology.orchestra.run.smithi139.stdout:kernel.core_pattern = core 2024-03-18T23:28:05.432 INFO:teuthology.orchestra.run.smithi175.stdout:kernel.core_pattern = core 2024-03-18T23:28:05.459 DEBUG:teuthology.orchestra.run.smithi049:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-18T23:28:05.481 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-18T23:28:05.482 DEBUG:teuthology.orchestra.run.smithi139:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-18T23:28:05.495 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-18T23:28:05.496 DEBUG:teuthology.orchestra.run.smithi175:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-18T23:28:05.513 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-18T23:28:05.513 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-03-18T23:28:05.525 INFO:teuthology.task.internal:Transferring archived files... 2024-03-18T23:28:05.526 DEBUG:teuthology.misc:Transferring archived files from smithi049:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-03-18_21:05:49-fs-wip-batrick-testing-20240318.181317-distro-default-smithi/7609140/remote/smithi049 2024-03-18T23:28:05.526 DEBUG:teuthology.orchestra.run.smithi049:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-18T23:28:05.577 DEBUG:teuthology.misc:Transferring archived files from smithi139:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-03-18_21:05:49-fs-wip-batrick-testing-20240318.181317-distro-default-smithi/7609140/remote/smithi139 2024-03-18T23:28:05.577 DEBUG:teuthology.orchestra.run.smithi139:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-18T23:28:05.615 DEBUG:teuthology.misc:Transferring archived files from smithi175:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-03-18_21:05:49-fs-wip-batrick-testing-20240318.181317-distro-default-smithi/7609140/remote/smithi175 2024-03-18T23:28:05.615 DEBUG:teuthology.orchestra.run.smithi175:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-18T23:28:05.653 INFO:teuthology.task.internal:Removing archive directory... 2024-03-18T23:28:05.654 DEBUG:teuthology.orchestra.run.smithi049:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-18T23:28:05.656 DEBUG:teuthology.orchestra.run.smithi139:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-18T23:28:05.659 DEBUG:teuthology.orchestra.run.smithi175:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-18T23:28:05.702 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-03-18T23:28:05.716 INFO:teuthology.task.internal:Not uploading archives. 2024-03-18T23:28:05.716 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-03-18T23:28:05.736 INFO:teuthology.task.internal:Tidying up after the test... 2024-03-18T23:28:05.736 DEBUG:teuthology.orchestra.run.smithi049:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-18T23:28:05.739 DEBUG:teuthology.orchestra.run.smithi139:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-18T23:28:05.741 DEBUG:teuthology.orchestra.run.smithi175:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-18T23:28:05.751 INFO:teuthology.orchestra.run.smithi049.stdout: 262170 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 18 23:28 /home/ubuntu/cephtest 2024-03-18T23:28:05.753 INFO:teuthology.orchestra.run.smithi139.stdout: 262162 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 18 23:28 /home/ubuntu/cephtest 2024-03-18T23:28:05.756 INFO:teuthology.orchestra.run.smithi175.stdout: 262162 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 18 23:28 /home/ubuntu/cephtest 2024-03-18T23:28:05.757 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-03-18T23:28:05.769 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-03-18T23:28:05.836 INFO:teuthology.nuke:Checking targets against current locks 2024-03-18T23:28:05.863 DEBUG:teuthology.nuke:shortname: smithi049 2024-03-18T23:28:05.864 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-18T23:28:05.888 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi049.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-03-18_21:05:49-fs-wip-batrick-testing-20240318.181317-distro-default-smithi/7609140', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-18 21:31:05.526149', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJrClwSujA3DxhjycCDFz+K4daUSvkXJBl95QePSunM+HntcgrIVUb8euaF3T0w9tjReVqIi006AQ3/nOy+gofA='} 2024-03-18T23:28:05.892 DEBUG:teuthology.nuke:shortname: smithi139 2024-03-18T23:28:05.892 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-18T23:28:05.915 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi139.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-03-18_21:05:49-fs-wip-batrick-testing-20240318.181317-distro-default-smithi/7609140', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-18 21:31:05.523470', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLwe1SvMi33uDIyC/VCOTEcd/IY0tArlGLqy6o9Bzy9NtvqPlBloCCaFq4bfWoTtxn8IppxcUnb0JBNfcQEiq8E='} 2024-03-18T23:28:05.920 DEBUG:teuthology.nuke:shortname: smithi175 2024-03-18T23:28:05.920 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-18T23:28:05.951 INFO:teuthology.orchestra.console.smithi049:Power off 2024-03-18T23:28:05.951 DEBUG:teuthology.orchestra.console.smithi049:pexpect command: ipmitool -H smithi049.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-18T23:28:05.965 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi175.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-03-18_21:05:49-fs-wip-batrick-testing-20240318.181317-distro-default-smithi/7609140', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-18 21:31:05.524876', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEHAkrTqf+Qms1LnY4oDNHCeEuoGn7ch+KOKfl2VjBe0g9M/00RWBB6IH67I7+E0oDx4dS8J2tNHdbJTkuQ7BYA='} 2024-03-18T23:28:05.977 DEBUG:teuthology.orchestra.console.smithi049:power off output: Chassis Power Control: Down/Off 2024-03-18T23:28:05.977 DEBUG:teuthology.orchestra.console.smithi049:pexpect command: ipmitool -H smithi049.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-18T23:28:05.994 INFO:teuthology.orchestra.console.smithi139:Power off 2024-03-18T23:28:05.995 DEBUG:teuthology.orchestra.console.smithi139:pexpect command: ipmitool -H smithi139.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-18T23:28:06.008 DEBUG:teuthology.orchestra.console.smithi049:check power output: Chassis Power is on 2024-03-18T23:28:06.017 DEBUG:teuthology.orchestra.console.smithi139:power off output: Chassis Power Control: Down/Off 2024-03-18T23:28:06.017 DEBUG:teuthology.orchestra.console.smithi139:pexpect command: ipmitool -H smithi139.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-18T23:28:06.037 INFO:teuthology.orchestra.console.smithi175:Power off 2024-03-18T23:28:06.037 DEBUG:teuthology.orchestra.console.smithi175:pexpect command: ipmitool -H smithi175.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-18T23:28:06.046 DEBUG:teuthology.orchestra.console.smithi139:check power output: Chassis Power is on 2024-03-18T23:28:06.059 DEBUG:teuthology.orchestra.console.smithi175:power off output: Chassis Power Control: Down/Off 2024-03-18T23:28:06.059 DEBUG:teuthology.orchestra.console.smithi175:pexpect command: ipmitool -H smithi175.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-18T23:28:06.080 DEBUG:teuthology.orchestra.console.smithi175:check power output: Chassis Power is on 2024-03-18T23:28:10.008 DEBUG:teuthology.orchestra.console.smithi049:pexpect command: ipmitool -H smithi049.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-18T23:28:10.047 DEBUG:teuthology.orchestra.console.smithi139:pexpect command: ipmitool -H smithi139.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-18T23:28:10.081 DEBUG:teuthology.orchestra.console.smithi175:pexpect command: ipmitool -H smithi175.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-18T23:28:10.120 DEBUG:teuthology.orchestra.console.smithi049:check power output: Chassis Power is on 2024-03-18T23:28:10.157 DEBUG:teuthology.orchestra.console.smithi139:check power output: Chassis Power is on 2024-03-18T23:28:10.192 DEBUG:teuthology.orchestra.console.smithi175:check power output: Chassis Power is on 2024-03-18T23:28:14.125 DEBUG:teuthology.orchestra.console.smithi049:pexpect command: ipmitool -H smithi049.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-18T23:28:14.158 DEBUG:teuthology.orchestra.console.smithi139:pexpect command: ipmitool -H smithi139.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-18T23:28:14.193 DEBUG:teuthology.orchestra.console.smithi175:pexpect command: ipmitool -H smithi175.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-18T23:28:14.236 DEBUG:teuthology.orchestra.console.smithi049:check power output: Chassis Power is off 2024-03-18T23:28:14.268 DEBUG:teuthology.orchestra.console.smithi139:check power output: Chassis Power is off 2024-03-18T23:28:14.304 DEBUG:teuthology.orchestra.console.smithi175:check power output: Chassis Power is off 2024-03-18T23:28:14.337 INFO:teuthology.orchestra.console.smithi049:Power off completed 2024-03-18T23:28:14.369 INFO:teuthology.orchestra.console.smithi139:Power off completed 2024-03-18T23:28:14.405 INFO:teuthology.orchestra.console.smithi175:Power off completed 2024-03-18T23:28:14.512 INFO:teuthology.run:Summary data: description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/secure wsync/no} objectstore-ec/bluestore-comp omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/random export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/yes 5-workunit/suites/dbench}} duration: 5399.131582736969 failure_reason: 'Command failed (workunit test suites/dbench.sh) on smithi049 with status 1: ''mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=e48b1f6c0608b2615e1890cc46286c3877d6e910 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh''' flavor: default owner: scheduled_pdonnell@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=e291eef62e39418ab6f34355786da3bc status: fail success: false 2024-03-18T23:28:14.512 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-18T23:28:14.611 INFO:teuthology.run:FAIL